Three figures standing on a laptop keyboard examining data in a pie chart and graph

(Illustration by Vanessa Lovegrove)

Technology has enabled nonprofits to access massive amounts of data to grow revenue, manage their operations, and pursue their mission. But access is not the same as understanding the full story that data can tell.

Next Generation Non-Profits

Cloud technology provides the innovations nonprofits need to more effectively serve their constituencies, increase donor and member engagement, and improve the stakeholder experience. This supplement shares insights into how your organization can use this technology to jumpstart success. Sponsored by AWS

It’s easy for organizations to make sense of their data when they have to manage only a few datasets. However, the process becomes more difficult when additional datasets are added, especially when those additional datasets come from tools purchased from different vendors and which have their own unique data structure. When that happens, traditional spreadsheets and databases aren’t enough. Simple databases can crash under the weight of too many users, and large, complex spreadsheets with thousands of rows of embedded calculations can crash or run maddeningly.

Fortunately, data lakes allow users to store, manage, cleanse, transform, and analyze diverse datasets in one place. To be clear, a data lake is not a data warehouse – the main difference is the order in which the data is loaded and transformed. Traditional data warehouses require data to be transformed first, and only data that adheres to a specific format and schema is accepted. This generally means that not all data can be archived, especially newly acquired or unstructured datasets, leading organizations to lose access to that data. Data warehouses are also typically just storage facilities. They don’t contain the tools that enable data cataloging, reporting, governance, advanced analytics, and machine learning.

Data lakes, on the other hand, allow you to store data immediately and create or change the schema over time. They also come with a comprehensive set of modular access control, cataloging, and analytics tools, which together allow organizations to easily customize their data lake for their specific needs.

At Share Our Strength, a US non-profit dedicated to ending childhood hunger, we addressed the challenges of managing our many streams of discrete fundraising data by initiating the process to implement a data lake in 2021. This took us well on its way to replacing the organization’s many independently managed spreadsheet data streams with a single accessible and shared system of record. It has reduced the load on our database administrators by providing automated reports in real time. It has further prepared us to use advanced AI and machine learning concepts to build better predictive models of donor behavior to improve our fundraising. Additionally, the data lake has made us more cyber secure by giving us the ability to grant or restrict access to sensitive datasets at a granular level.

The technical journey from identifying the need for a data lake to implementing one was relatively straightforward. Cloud-based tools for storing, transforming, cataloging and analyzing our data, as well as controlling access to it, were readily available from AWS. Far more difficult than data engineering, however, was the cultural engineering needed to bring new teams together, build trust in a new way of doing business, and agree on a new set of standards for handling our data.

Cultural reengineering

In the process we learned that the real work in building a data lake is reengineering how the organization thinks about its data. Our team estimates that they spent 80% of their time building consensus across teams and only 20% of their time designing and implementing the actual infrastructure.

One of the first areas of cultural reengineering was to make the IT department a better partner for the fundraising department. Before I joined Share Our Strength as CIO in 2017, the IT department was almost exclusively focused on server maintenance and desktop support, and while we had technical expertise to offer, we were never part of the conversation about how manage data better. Therefore, my first step was to expand the services offered by the IT department to include the creation and maintenance of the organization’s data lake. In doing so, my department has brought a new source of creativity, skill and perspective to our business units’ efforts to use data and technology wisely.

A second aspect of our cultural reengineering has been building trust between IT and fundraising departments. This effort required time and patience for people from different disciplines to learn each other’s language and trust each other’s experience. For example, when IT demonstrated the data lake’s capabilities, we were met with some skepticism because the analytics looked different from the Excel spreadsheets fundraisers were familiar with. At the same time, we in the IT department also needed to ensure that we were demonstrating capabilities that were relevant to current fundraising priorities, such as automating data cleansing tasks that took up a huge portion of their day, instead of demonstrating capabilities that would only be potentially useful in the future, how to find unique trends in donor behavior. It took several iterations and rounds of clarifying the questions before we began to understand each other’s perspectives and were then able to establish a priority list of problems to solve.

Finally, our organization has embarked on a third area of ​​cultural reengineering: developing a shared understanding of how our data is acquired and structured. Often the rules for storing and structuring data made sense for one fundraising team, but those same rules didn’t translate and applied easily to the work of other teams. This led to the storage of irrelevant data in some structured fields, the storage of a lot of important data in comment fields that did not lend itself easily to analysis, and the realization that our analyzes were sometimes incomplete because they did not fully account for the nuances of how or where our data was stored. Data cataloging, which is a central feature of data lakes, has given us the tools and opportunity to identify and fix these irregularities and ultimately improve our analytics.

Share Our Strength We’ve only been using a data lake to improve our fundraising for a year, but we’re already seeing huge gains. We are starting to measure returns on various investments with more specificity because we are including more and better data streams, which in turn allow us to make smarter decisions. We’re also working to automate complicated but repetitive data extractions and transformations, saving our donor operations team time while supporting mid-level and senior fundraising officials in more markets. Finally, our more consistent data standards are helping us better understand donor behaviors, which has enabled us to have more meaningful communications with them.

The data lake has contributed to each of these successes, but the lake itself was only possible because of the cultural transformation of our nonprofit. For organizations wishing to embark on a similar journey to enhance their fundraising capabilities, our advice is to build trust within the organization and then iterate, iterate, iterateas there are probably several unknown opportunities for innovation within your datasets.

Read more stories by Richard Kostro.

Leave a Reply

Your email address will not be published. Required fields are marked *