Since McKinsey released a report on how best to use prizes to incentivize innovation nearly a decade ago, an entire industry has grown around social innovation challenges. The formula for these “save the world” competitions has become standard. Drum up a lot of buzz around an award. Partner with big names to get funding and high-profile judges. Try and get as many submissions as possible from across the world. Whittle down the submissions and come up with a list of finalists that get to pitch at a glitzy event with a lot of media attention.

On the final stage, based on pitches that last mere minutes, pick a winner that can get upwards of millions in prize funding. Don’t have a software platform to run a challenge of this kind? No worries, numerous for-profit vendors have sprung up that can do all the work for you—for anywhere from ten to a few hundred thousand dollars. The growth has been so exponential that prizes awarded through competitions has grown from less than $20 million in 1970 to a whopping $375 million just four decades later.

But do these prizes get the sort of world-saving results they aim for? There’s little quantified evidence to back that, and some leaders in philanthropy are broadly skeptical.

For its part, the Massachusetts Institute of Technology is trying a different approach to innovation challenges with Solve, taking some of what’s worked in these challenges and fusing it with elements of tech accelerator programs, including post-award training that focuses on results.

Solve is entering an already crowded field of innovation challenges. Many of these prizes overlap, with each vying to be the “Nobel” of its field. More prizes means more noise—which has led to a race to offer more money to get attention.

But even private-sector riches do not guarantee that prize money for innovation gets good results. In 2004, Bigelow Enterprises sponsored a $50 million Space Prize but it failed to capture the imagination of space researchers and eventually folded. Back in 2009, Netflix invited outside teams to improve it movie recommendation algorithm by 10% for a $1 million reward. The Netflix Prize led to a race among programmers, only for Netflix to eventually kill the entire plan because it was getting better results in-house.

Overall, the social innovation competitions tend to reward presentation, glitz and charisma, and penalize speaking English as a second language, introversion and inability to make flashy slides.

Solve, which held its third annual finalists event on Sunday September 23 in New York, is taking a different approach.

Unlike other contests where questions are internally decided, Solve crowdsources the questions to begin with. Its team takes months to run hackathons and workshops around the world to decide on the four most pressing questions to become the focus of that year’s challenge. This year, the questions focused on teachers and educators, workforce of the future, frontlines of health and coastal communities.

The competition is then opened up to participants from around the world with relatively low barriers to entry, resulting in 1,150 submissions from 110 countries in the last competition round. (That’s at least one submission from nearly 60 percent of all countries in the world!)

The prize recipients of the GM Prize for Advanced Technology. Photo: Adam Schultz | MIT Solve

To qualify, though, participants need to have more than just an idea. They must have a prototype that works, be either in the growth, pilot or scale stage, and be tech-driven. Submissions are then evaluated by judges from across industry, intergovernmental organizations and academia to get to 15 finalists for each of the four challenge questions. These 60 finalists get a full day with judges to be asked in-depth questions and have their ideas evaluated.

The day after, with all the preparations completed, the finalists get three minutes apiece to present on stage. Crucially, instead of one winner, eight finalists are chosen for each of the challenge questions.

Each finalist receives an initial $10,000 prize, plus a pool of hundreds of thousands of dollars provided by partners including General Motors, the Patrick J. McGovern Foundation, Consensys, and RISE.

This year, for example, Ugandan health care startup Neopenda brought in an additional $30,000 in funding through Solve, from a UN program sponsored by Citi. An intelligent messaging app called TalkingPoints, meanwhile, received backing from General Motors and Save the Children to develop its personalized coaching technology for parents and educators. (You can see more details on this year’s winners and prizes here.)

As opposed to being a “one and done competition” where winning the prize money marks the end of the competition, managing director of community Hala Hanna tells me that the real work begins once the Solver teams are selected. Each qualifying Solver team gets 12 months of engagement and support from the organization. “Our value-add is providing a network, from MIT and beyond, and then brokering partnerships,” she explains.

Perhaps the biggest testament to the Solve method getting traction is its funders putting in even more cash in support. At the closing event on Sunday, an upbeat Matthew Minor, Solve’s director for international programs, took to the stage decked out in Solve-branded socks and a broad smile. He announced the winning finalists—and more funding opportunities. Two of Solve’s original backers, the Atlassian Foundation and the Australian government, are continuing to invest out of a standing $2.6 million budget for companies in the workforce track. RISE, a global impact investing fund, is putting an additional $1 million into companies focused on coastal communities.

The Australians have already put in funding to help past winners scale after the program. One of them is Ruangguru, a digital boot camp in Indonesia that gives youth dropouts resources they need to earn graduation certificates. The startup had reached nearly a million Indonesians prior to participating in Solve; through the program and the additional funding, it assisted more than 3 million Indonesian youth by the end of last year. Iman Usman, one of Ruangguru’s founders, tells me that Solve enabled them to enter into partnerships that helped them scale across Indonesia in a way they would have never been able to do on their own.

Solve has also been unequivocally good at ensuring diversity, both in its own staffing and—perhaps for related reasons—in those that are chosen as finalists. Of Solve’s 20 full-time staff, 14 are women, as are six out of the seven leadership team members and—by my count—at least seven nationalities from four continents are represented on staff.

The 33 Solver teams selected at the finals this year hail from 28 different countries, with 61 percent of them being women-led. At a time when the tech industry is struggling to increase diversity, Solve’s emphasis on diversity in challenge design and promotion has led to applicants and finalists that reflect the world Solve aims to help.

Hanna noted that increasing diversity is not as difficult as it’s made out to be. “Honestly, we’re not even trying that hard,” she explained. “So whoever says there are no women in tech, I say, crazy talk.”

The view from the Apella at Solve Challenge Finals on Sept. 23. Photo: Adam Schulz | MIT Solve

Still, Solve does have a few kinks to work out. By taking on extremely broad topics, the competition can sometimes lack focus. Lofty questions mean you can get very disparate answers, making it hard to compare them in a way that feels fair.

And while it’s great that the award monies are not all given to a single winner, it is not quite clear how funders pick the teams that do get funding. 15 qualifying finalists this year ended up winning money awards, some winning more than one, while the remaining 18 qualifying teams went home with the minimum amount. This is because Solve funders get to pick which of the teams that qualify at the finals get their respective monetary prizes. Of course, all 33 qualifying teams equally get to be a part of the Solve class with all the support and training that includes.

Another kink is the audience choice award—selected through open online voting prior to the finals—but not tied to any clear concrete benefit. Take the example of Science for Sharing (Sci4S), a Mexico-based startup that trains teachers to better engage students in STEM and has already reached nearly a million children across Latin America. It garnered 419 community votes in the Education Challenge, more votes than any other participant in the category, and handedly won the audience choice award. Ultimately, Sci4S was not selected as a Solver team. Another education startup, Kenya-based Moringa School, only got two votes but was selected. While Moringa and others were compelling and qualified in their own right,  it’s still hard not to think that Sci4S should have focused all of its time on its presentation and ignored the audience vote.

All in all, Solve does get a number of things right where other innovation challenges have failed. Instead of anointing one winner for the entire competition, it selects a class of dozens—reflecting the simple fact that the world’s most intractable problems are not going to be solved by any singular idea. Unlike many challenges put on by educational institutions and open only to their own students, Solve opens its doors wide. And winning at the finals doesn’t end your connection with MIT, it only starts it, with all qualifying finalists getting a year of individualized support, training and mentorship.

Done right, prizes can be effective at incentivizing startups to focus on pressing societal issues that can truly benefit from tech-driven solutions. But prizes for the sake of prizes can add to the noise and dissipate scarce public resources and entrepreneur attention. In the increasingly crowded world of innovation challenges promising to change the world, MIT’s Solve is a step away from the noise and towards effective prize granting.