New Possibilities: The Role of Research in 4.0’s New Normal Fellowship

By: Hassan Hassan

For most of our history, when asked about our impact, 4.0 Schools has pointed to the people, communities, and stories that show that the changemakers we invest in are making a real, human difference. We haven’t wanted to reduce that impact to oversimplified numbers.

But we are coming to understand that our feelings of hope and confidence simply aren’t enough. To honor our commitments to change, 4.0 owes its entrepreneurs and communities—as well as our own investors—real data about whether what we’re doing is actually working. Or, in the words of one investor, “How do we know you’re not selling impossible dreams?” How can we demonstrate that our entrepreneurs are not only doing good, but ensure that they are not doing harm? 

We’ve resisted formal research and evaluation on the grounds that it’s expensive, time-consuming, and accessible primarily to the wealthy and the white. Developed in ivory towers far removed from schools and communities, research had seemed rarely useful or practical for our early-stage entrepreneurs.

We have also resisted the tug of philanthropy toward backing only educational ideas that meet the “gold standard” of randomized controlled trials (RCTs)—the most expensive and challenging of all research standards to meet. Funders who make RCTs or similar research a requirement, bias themselves towards established organizations, and this bias pulls resources away from those closest to the challenges they seek to address.

Our goal at 4.0 is to provide resources to help our fellows test their ideas so they can build their credibility from the ground up. As such, 4.0 is now pioneering new approaches to research and evaluation that combine rigor and accessibility. We believe it’s time to redefine the relationship between early-stage entrepreneurship and research to ensure that kids and communities are benefitting.

The Evolving Approach to Research at 4.0

4.0 is developing new approaches to research and evaluation that honor the wide variation in needs that our entrepreneurs are addressing and approaches they are taking to tackle those needs. Like other early-stage education investors such as LeanLab, LEAP Innovations, and NewSchools Venture Fund, 4.0 is working to “right-size” research. To us, “right-sizing” means collecting the most rigorous data possible to evaluate the demand and efficacy of ideas given the early stage of the small tests our fellows are running.

One key step we’ve taken is the establishment of a Measurement & Evaluation (M&E) Collaborative in which our fellows define success for themselves, rather than relying upon researchers alone to dictate approaches and metrics. The M&E Collaborative connects our fellows with a network of researchers who provide hands-on coaching to our fellows to help them craft logic models and metrics that fit their approach. These researchers have also run a peer review process to collaboratively vet entrepreneurs’ measurement approaches in ways that balance research rigor with practical value.

This strategy allows entrepreneurs to validate their own metrics through the best practices of the researchers and through direct experience with their own communities. This also helps shift the balance of power between community-led ventures and academic institutions. In this work, we draw inspiration from anti-racist approaches to evaluation, as well as frameworks for applying a DEI lens to evaluation. Certainly, research expertise has its place, but we believe our underserved communities need and deserve the ability and capacity to define and measure their own success.

Evaluation in Our New Normal Fellowship

The COVID-19 crisis has caused a range of economic and cultural changes, from joblessness to distance learning, that are placing additional strain and anxiety on our most struggling students and families. At 4.0, in the face of uncertainty, we tend toward optimism and action, so we quickly launched a New Normal Fellowship in the spring that gives 49 alumni teams stipends and coaching to help them run COVID-19 relief projects in their communities.

However, the urgent nature of the project led us to take a very different approach to research and evaluation. Rather than have fellows choose their own metrics in consultation with researchers through a peer review process which can take several months, we chose to partner with the Center for Research in Education and Social Policy (CRESP) at the University of Delaware to evaluate the impact of fellows’ pilot projects on participants’ social and emotional learning (SEL) skills. Although there are drawbacks to relying on a large research institution, this approach was the best way to allow fellows to launch pilots quickly and gather uniform data in the midst of an unprecedented moment in the lives and learning of children. This approach allowed us to deliver coaching and capital to pandemic-responsive entrepreneurs without sacrificing our research strategy completely.

Our partners at the University of Delaware have designed surveys (passed by the university’s Institutional Review Board) that gather data about SEL skills and will connect those outcomes to implementation through surveys of fellows and of site leaders/instructors, where relevant. The aggregated data that fellows receive about their own program and others will help them refine their programs over time.

The Goal: Right-Sizing Research

For many of our entrepreneurs and the marginalized students they serve, numbers have been used to judge, take away, and punish, but 4.0 is working with our fellows to change that by arming them with data about their strengths. “For black girls in Chicago, data is often a pathologizing force, but I want to use it as a positive force,” says Johnaé Strong, who piloted her Black Girls Restore(d) curriculum for teen changemakers as part of 4.0’s Tiny Fellowship back in April and will expand it this school year as a New Normal Fellow.

During the shelter-in-place, Strong was forced to adapt to an entirely online model while testing her curriculum with an initial cohort of 10 participants, which set her up to strengthen and expand her program to reach a larger group of 100 participants across the city this fall. She will administer the University of Delaware-developed pre- and post-program surveys that will help measure the impact of her program on participants’ SEL skills. She is also considering conducting focus groups to gather additional qualitative data about how programs like hers can influence black girls’ self-perception and confidence.

Like Strong, we are trying to flip deficit thinking on its head by embracing research that quantifies the strengths, assets, and growth in our communities. We hope to replace the impractical “gold standard” with what research consultant Kirsten Lee Hill calls an “aluminum standard”—flexible, inexpensive, and just strong enough to give entrepreneurs confidence in the data they are gathering about their work, but without the unnecessary shine of gold.

Ultimately, our charge at 4.0 is to support our fellows and their communities to quantify their impact on their own terms. That’s a dream we think just might be not only possible but necessary.

For more, see:

• 4.0 Schools Launches $15 Million Fund to Invest in the Future of Learning
• Getting Smart Podcast | Matt Candler on the #FutureofSchool
• Smart List: 40 Innovation Investors 

Hassan Hassan and his mother immigrated to the U.S. sixteen years ago through the Diversity Immigrant Visa Lottery. Today, he serves as the CEO of 4.0, a national incubator that connects, coaches, and invests in 100+ leaders every year to pilot their visions for promising learning models. Hassan is a Pahara and Education Pioneer alumnus. Follow him on Twitter at @hassantwice. 

Stay in-the-know with innovations in learning by signing up for the weekly Smart Update.