Problem

GlobalGiving connects with organizations based in the US and nonprofits in other countries to create a comprehensive directory of nonprofits. However, the process of finding and applying to GlobalGiving remains significantly easier within the United States. There has been a slower onboarding and discovery of GlobalGiving from other countries, and it is inefficient for GlobalGiving to take the time to scout each nonprofit independently.

Solution

We worked with GlobalGiving to build a scalable, easy-to-use command-line interface that crawls the web for promising nonprofit websites. We then scrape through the information provided on these sites and collect the relevant data. This new information allows GlobalGiving to be more proactive in expanding its nonprofit database.

Tech Stack

Python, Flask, MongoDB

Features

Data Collection Through Scraping

Choose ten websites with directories of nonprofits and scrape each individual one to add to the database. Relevant data includes the organization’s name, country of registration, registration numbers, website URL, and email.

Command Line Tool

Allows the user to see all available scripts, run them, and submit entries to the GlobalGiving database using the organization parser.

Web Crawler

Crawls the web and Google searches to rank potential NGO directories.