The Proof is in the Playing: Efficacy in Game-Based Learning

Every learning game pitch eventually reaches the slide with the usage numbers on it, offering up the session counts and completion rates that show people are playing. Now, those numbers certainly matter and they belong in your deck, but the people signing the contract are working toward a second and arguably more important consideration, which is what actually changed for the learner. After more than 400 projects we've published a lot of different numbers about our games, and the useful thing about looking at them together is how clearly they sort into different kinds of claims. Below we've pulled four examples from our own portfolio to show what each kind of number can support.

Reach

Mission: Mars, our Roblox collaboration with the Museum of Science, reports 7.9 million visits, 28,600 bookmarks, and more than 8 million branded avatar items dispersed. These are distribution numbers, and for this project they're the key performance indicator. Mission: Mars teaches the Engineering Design Process to young people who chose to be there, outside of any classroom, on a platform where they already spend their time, so a museum partner whose mission is informal science learning at scale can demonstrate real impact and amplification through 7.9 million recorded visits. 

In-Game Performance

Salvage Safari, our Roblox collaboration with Niagara Cares, has logged 3.8 million digital waste items collected at an 83.9% sorting accuracy rate. Sorting accuracy is a super interesting number because it comes directly out of the core gameplay loop - sorting waste correctly is the thing the game exists to teach. In our design vocabulary, the central verb of this game is the learning objective, which means the telemetry produces evidence of competency as a byproduct of play. That alignment is a design decision that gets made during concepting while the core loop is still negotiable, and it's a big part of why we spend so much early effort mapping learning objectives onto identity, verbs, and systems.

Learning Outcomes

Do I Have a Right?, one of the games in our long partnership with iCivics, carries both kinds of numbers and carries them at remarkable scale. On the reach side there are more than 200 million total game plays, 145,000 teachers engaged annually, and 9 million students reached annually. On the outcome side there's a 26% increase in student civic knowledge and a 36% increase in civic dispositions, along with 95% of iCivics teachers reporting that their students are more engaged. That combination is the strongest position a learning product can occupy, because it pairs enormous distribution with measured gains in the exact competencies the game was designed to build. Results like those come from study work conducted alongside the game, and they're what makes a claim portable into a district procurement conversation.

Motivation vs. Competency

A 2024 meta-analysis in Educational Technology Research and Development puts a useful frame around why we care so much about motivation vs. actual measured competencies. Looking across a large body of studies, researchers found that gamification produced consistent gains in intrinsic motivation and in learners' sense of autonomy and relatedness, while its measured effect on competency was smaller. Motivation gains are worth designing for on their own terms, since a learner who wants to keep playing is a learner you can teach, and competency gains are built through a second and equally deliberate effort. Planning for both from the start is what puts a product in position to report the kind of paired results iCivics has demonstrated.

Evidence Tiers and Your Buyer

For K-12 buyers spending federal funds, the Every Student Succeeds Act sorts evidence into four tiers, running from a well-designed randomized controlled trial at the top down to a research-based logic model paired with a plan to study impact at the bottom. That bottom tier matters more to early-stage companies than most founders expect, because it's a legitimate and fundable starting point that you can reach in a matter of weeks, and a clear logic model explaining why your mechanics should produce your outcome will put you in a real conversation with a district while you build toward stronger evidence. Enterprise and corporate buyers have no equivalent framework, so you get to define the standard yourself, and borrowing this structure gives your claims a shape those buyers already recognize.

~

During preproduction, the best bet is to decide what you're claiming, which player decisions demonstrate it, how those events will be stored and queried, and who reviews the results. The numbers in our portfolio exist because somebody asked those questions while the games were still being designed. It’s worth noting that buyers in this market have become a lot more sophisticated about evidence over the last few years, and that's good for everyone building serious learning products, because it holds us all to a higher standard and ensures that Ed Tech maintains a strong reputation as a product category. Designing toward a measurable outcome forces clarity about what the game is teaching, which is where good learning design starts anyway. Ready to build measurement into your next project? Get in touch with our team and let's talk about what you need to prove.

‍