How to track progress in game-based learning?

Our online store uses games, puzzles, and other activities to teach. They’re the primary focus and, most likely, all you will ever need. A handful of other platforms/services exist that also provide a similar game-based curriculum, but there’s one major difference.

Our store does not provide a solution for testing students or tracking progress.

We regularly receive questions about this. Parents and teachers are used to standardized tests and measuring progress in education. Many believe this is a good thing or a necessity, but we don’t. That is why we wrote this article to clarify it and hopefully give some answers and advice.

Why do we take tests?

Tests are actually one of the major reasons why our systems of education are the way they are.

Rote memorization (and reproduction) is probably the worst way to teach and learn anything.

But it is one of the easiest ways to teach and test.

And so our schools use this extremely ineffective and damaging method to teach, purely because it makes for easy testing and progress tracking.

Rote memorization allows assigning a chapter in a book, then asking simple questions about the facts they memorized.

Other types of learning, such as through games, is harder to test.

The why behind measuring progress (through tests) is pretty self-evident. We only want to give degrees to students who have “proven” they actualyl have the skills. Parents want to ensure a bright future for their kids and want to keep them on track. Teachers want to test if their teaching actually led to something.

We completely understand the desire to test and track progress. It’s just that …

• Our current way of doing it isn’t the best way (at all).
• It’s not more important than other things like actual learning, happy kids, and privacy.

How to measure progress in game-based learning?

Let’s only focus on the first bullet point. Standardized tests and rote memorization are bad, so we switch to game-based learning. How would we measure progress now? What do “tests” look like?

It’s not impossible. In fact, it’s even easier!

Games naturally provide very easy equivalents to (standardized) tests.

• You can beat a game. That’s a big sign that you’ve mastered the relevant skills.
• Games have difficulty levels. You keep going up in difficulty until you hit a wall—that’s where you’re at right now.
• Games usually have lots of smaller progress trackers. What’s the level of your character? How many experience points do you have? Did you defeat that Level 10 monster so you’re able to go into the Forbidden Forest?

Games are nothing but tests and challenges. Every good game has a difficulty curve that trends upwards. As the player grows more skillful, the game also has to grow with them to stay interesting.

By putting games in order and saving highscores, you can track someone’s progress even more meticulously than before.

At the very least, you can track when they log in and start a game. You’re not just tracking if someone is succeeding, but also if they’re trying—if they’re showing up every day, so to speak.

At the same time, there are issues.

• This only works for solo video games. Board games, physical games, multiplayer games, it falls apart immediately.
• It requires a fixed order with narrowly defined games. Most games test a multitude of skills in different ways. The more narrow you make the game, the less fun and effective it is. You also increase the likelihood of boring students out of their minds because they already mastered a skill five games ago.
• If the digital activities of adults were tracked as meticulously, they would cry foul about privacy and what not. But for some reason we do this to kids/students and see no problem.

Additionally, it’s not exactly the same as a standardized test.

• There is randomness. Almost all games have some randomness. Either built in (random monster spawns, random quiz questions, dice rolls, etc) or simply because of the variance between computers and screens.
• There is “cheating”. It can be quite hard to lock down a game well enough that nobody can find shortcuts or workarounds or ways to cheat the system. And as opposed to real-life cheating, this is easier to do and impossible to catch/counteract.
• There is often a binary cutoff. You either beat the game or you don’t. You get 1, 2, or 3 stars. Though it’s possible to solve this, in practice there is way less granularity in your progress or grades. (Say you beat a game with only 1HP of health left. And another student beat the game with 100HP left. Do you get the same grade? Should it be different?)
• You can try over and over. Mind you, this is a good thing. Standardized tests should allow lots of repeated attempts too, if you ask us. But they don’t, so this is another way game-based learning differs. (And then we get into the same discussion about grades. Someone overcame the challenge on the first try, and another kept dying and retrying a hundred times until they won. Is that worth the same grade? Which of the two learned more?)

The problem with testing

In conclusion, testing (or tracking progress) in game-based learning is easier and more effective in many ways. But there are still glaring issues that make the results pretty meaningless, just like standardized tests.

If someone gets a bad grade, what does it mean?

• They might have felt sick or been distracted.
• They might have studied literally everything else except the few questions that appeared on the test.
• They might understand the concept deeply, but struggle to answer in the specific format this specific teacher/test happens to like.
• They might have had four other tests on the same day.

Conversely, if someone gets a good grade, what does it mean? Tests are usually pretty short and specific, so they might just get lucky and write down some of the right words in the moment.

The same things are still true for games. If you’ve played any game for a longer period of time, you know there are times when you’re much better at it and times when you’re much worse. The randomness of games can be in your favor and give you an easy win.

Scientific research demonstrates that testing and grades ruin intrinsic motivation, creativity, and actual (deep) learning. They make students only do the specific thing they need to do for an external reward (a passing grade). They destroy curiosity, engagement, actually wanting to learn something and therefore learning it more deeply.

Tests and grades do not exist for the student. They exist for the teacher/parent to regain some control over the nebulous ways of learning and personal growth.

But learning is not a straight line! There are hills and valleys. It happens in bursts. We can even quantify this now with neural networks training: these AIs model the human brain and how it learns, and their “intelligence” can be plotted, and it is definitely never a straight line.

But tracking progress expects a straight line. The parent expects to see the tracker move a few pixels every day, or every time their kid played. When it doesn’t move for a week, they think there are issues or the kid isn’t putting in effort. While in reality they’re just slowly working out something and will have their Eureka moment later next week.

In other words,

All that a grade/test means, is that at some point in time, someone did certain actions that this specific teacher/system/test marked as good.

That’s all you can say for certain. Most high school students only study for the test, then forget everything the day after. Knowledge and skills decay, rapidly, especially when taught in ineffective ways. Many students finish high school without actually being able to read at the appropriate level, or while still saying “oh I just suck at mathematics” and never doing any mental calculation again.

Grades, or progress tracking, they don’t actually mean much. It might even send you in completely the wrong direction. You think your kid is learning a lot, while they are actually frustrated because they keep getting unlucky in this one particular boring game. You think your students are great at math, but they simply figured out the types of questions you will ask and memorized that.

How we handle this problem

One of the biggest educational platforms in the world, for years, has been Duolingo. Yes, the famous free app/website for learning languages. And we see two reasons (among others) for this.

• They idea of decay is at the core of their platform. The longer you don’t train something, the more it will assume you lost the information. (And rightly so, most of the time.)
• The learning and the testing are the same thing. You’re just constantly creating sentences and getting feedback on whether they’re right or wrong. Unlike a standardized test, you don’t get one shot at something and a final grade. You just do things, get feedback, do them a little better. This means failure isn’t a reason to stop using the app. This means Duolingo rarely teaches you dry facts to memorize—you learn new words just because they appear more and more in the sentences.

Though their platform isn’t perfect, it points us in the right direction. It’s successful because many people like using it and actually find it effective.

If you want to test a skill, then the best way to do it is actually asking someone to do the thing. Want to know if someone can play the piano? Ask them to play. Want to know if someone can build a website? Ask them to build one. Want to know if someone can cook a dish? You get the idea.

If you want to test knowledge, then the best way is to use a quiz (with proper feedback). Turn it into a fun game of asking questions and seeing how many they can answer, which you can also do as a group. Once everyone has given their answers, go through all questions and discuss.

In both cases, take decay into account. Don’t hand out degrees saying someone has definitely mastered this one thing forever. Don’t take a single test with written answers and assign meaning where it isn’t.

Keep testing the knowledge and the skill in small ways, over time, repeatedly. By the law of decay, you can make these durations longer the more often you repeat. That is, the first repetition is after only a day or two. The second repetition after a week. The third one after two weeks. And so forth.

Not surprisingly, we designed our store to do just that.

• Every topic can be approached in different ways: quizzes, stories, games, etcetera.
• Every time you repeat the topic, you pick a different method. It stays interesting, students explore different angles, and you get repeated testing moments (without having to design a new written test each time). Nobody can cheat their way through that. You actually need to understand something deeply to overcome the challenge in all these different ways, at different moments.
• Games (and similar activities) are great at teaching multiple things at once, especially skills. Even though you might play a game to specifically explore counting, it will also sneakily strengthen and repeat a bunch of other skills in your students.
• We structure our resources into different levels (or step-by-step) whenever possible. This means that students who are already more skilled can just skip over the first levels, while students who fell behind can catch up. There isn’t one “perfect moment” in the progress curve where this resource should be used and that’s that. You don’t need a meticulously tracked individual profile for this.

There is no “certainty” beyond this. What someone knows today they might not know tomorrow. The fact someone beat a game does not mean they mastered some of the skills you likely require to beat it. It certainly doesn’t mean they mastered them forever.

So, instead, we like to reframe tracking progress (or testing) as signposts along a journey. Or check-ins, if you will.

It is valuable to have some global curriculum and some general order in which to do things. It helps choose sensible topics and pick what to do. Some skills objectively must be understood before others can be taught at all. That’s why our store does provide this curriculum and recommended order. To help you out and save the energy needed to figure this out yourself.

It just gets dangerous when that order must be followed and definitive numbers or grades are attached. It gets dangerous when the end goal of education is getting a degree or a checkmark, instead of learning the things.

Every time you explore a topic is an opportunity to prevent decay and strengthen the skill. If you tell someone they’ve mastered a topic and are now done, they will stop exploring and therefore guarantee losing whatever they gained. You’re never finished and tests can’t make accurate statements about this anyway.

If you want to know “where we’re standing right now” (a progress check-in), then you either do a quiz (for knowledge) or literally just ask someone to do something (for skills).

Yes, this is very loosey-goosey. Freeform. No hard numbers. No tests. Children can walk away from the screen and there are no privacy issues. It might sound scary.

This approach will sometimes lead to picking the wrong game or activity.

You pick a board game, for example, but within 10 minutes you notice it’s too hard for your students right now. But this isn’t bad. This is your test! Now you know where the students stand right now and that this game is too hard. You switch to an easier game and skies are blue again, no time wasted, no complicated privacy-invasive tracking needed.

Or, the other way around, you pick a game that tests some skills that all your Apprentices are already comfortable with. At worst, you play a fun game that doesn’t teach as effectively as you wanted, and merely strengthens what’s already there. At best, you still discover new things and find unexpected growth. Even so, our games/puzzles all have “expansions” that you can keep tacking on to raise the difficulty.

You need very little to quickly test where someone is standing right now with their skill and knowledge. (And, to some extent, their energy and motivation. We all know how impossible it is to do or learn anything when you’re tired!) Because that’s all that matters in practice. Pick something, see the outcome, scale the difficulty up/down after 5 minutes. Once you know the current level of your Apprentices, we (hopefully) make it easy to pick the next resource to try.

We do recommend taking simple notes about this process, but we would recommend that in any case, for anyone. Keep a simple diary or log book with the different activities you tried (games, puzzles, stories, lesson material, …) and if you noticed big struggles or big boredom. Also date those notes to take the natural decay of memory into account. No more tracking or testing is needed or beneficial.

Here’s to a future with education that is actually effective and preserves the basic human rights of children!