Katrina is the lead Instructional Designer for Andamio Games. She recently published an article on Medium.com, "How To Make A Learning Game That Works." I caught up with Katrina just before her presentation at the White House for a Department of Education meeting on learning games research.
When you were younger, what did you want to be when you grew up?
My first memory is that I wanted to be a mountain climber, which is a completely unfulfilled dream. I think I knew I wanted to go into a scientific field because my dad was an engineer, and he really encouraged me in science.
How did you become interested in neuroscience?
My goal in college was to have a double major in pre-med and dance, and then I actually flunked out of my first three pre-med classes of fall semester — physics, organic chemistry, and calculus. After a little soul searching I became interested in psychology for the reasons any 19-year-old would, because you want to learn more about yourself. It seemed fascinating. So the next semester I didn't sign up for any science classes — I took Intro Psych and just got hooked on it. What I liked about the idea of going to medical school was the idea of doing medical research. I wanted to understand the human body, not necessarily provide medical care, and psychology seemed like an interesting way to do that.
What was the moment when you realized psychology is science, too?
As it turned out, I became a double major in psychology and philosophy, and my big internal debate was, "Am I going to go on in psychology or philosophy?" because in both disciplines you get to study the mind, just from totally different perspectives. At some point I think I realized I can still do philosophy as a psychologist, but I'm going to have evidence and data to back it up. I won't be just sitting in an armchair and dreaming about stuff. So I could still do philosophy, but it would have more rigor.
I ended up taking a course called Cognition, Computation and Brain, taught by Chad Marsolek at the UMN who studies visual cognition and memory. He was a very charismatic teacher and I was totally hooked on the aspect of psychology that studies how the mind actually works. Of course this is a philosophical question, but the fact that you could investigate this as a scientist was really fascinating.
How does your experience studying with Dr. Marsolek inform your research today?
Well, as an undergraduate assistant in his lab I was assigned all the grunt work — I recruited subjects and collected data and that was pretty much it. It was a perfect introduction to research, because when you just take lecture classes you're getting "the greatest hits" — the most fascinating studies, the things that turn out great, the results that revolutionized the field. And then when you actually sit down to do the work, you realize that 80-90% of the experiments don't yield results that are interesting, that the day-to-day work of it is tedious and not at all glamorous.
You sort of need someone to walk in to your lab from an Oliver Sacks book, unable to recognize his hat or something.
Exactly! The stuff we were doing was studying basic visual perception — how people recognize objects was the main question that my advisor researched. And what I realized is how narrow the focus is of any individual researcher, even though the field as a whole is doing all kinds of interesting things.
So what got you interested in learning science?
After I graduated, realizing there was a time limit on my dream of being a professional dancer, I ended up dancing for the next six years. I didn't do anything related to science at all. Eventually I reconnected with Chad Marsolek, and through a quirk in timing and available funding, he recommended that I apply for a fellowship through the educational psychology department, an interdisciplinary training and research program in education sciences. To be honest, at first I didn't care that much about K-12 education; I had no background in it and, I hate to say it, no intrinsic interest. It was not the thing I thought I would end up doing. But I joined the program and met a lot of scientists who did educational research. I realized, I can do this research on memory and learning, but now I can think about what impact it might actually have on people outside of academia. In the lab I worked in we were asking really interesting questions like, "What is the structure and function of memory in the brain, how does it work, how does it connect this part of the brain to that part."
Your job as instructional designer depends on so many different disciplines: subject matter expert, lesson creator, art designer, part-time technologist, learning scientist - and for games - game theory, game mechanics, reward systems and such. What prepared you for all this?
A big part of my job is understanding the big picture - and maybe this is where the philosophy comes into play - but I gravitate towards, "What's the whole, big idea of this, and how can I organize this in a way that is helpful for learning." My background in teaching helps with this, obviously, because you take a curriculum and you have to figure out how to make it interesting, what to present, what to leave out. For iNeuron, the source material was neuroscience textbooks, which are written at the college-level not the high school level, so understanding what we could leave out was crucial. We collaborate with subject matter experts who tend to be content purists, but you have to temper that with, "OK, we have two class periods with these students, what can we realistically help them learn?" If they walk away with a new understanding of major processes and ideas, if there's a detail or two that's not entirely correct, that's ok — they'll have that misconception corrected later.
This reflects how people actually learn. We don't step through information incrementally learning everything perfectly, we get the big picture, but about half of it is wrong, then we get more information that revises our understanding, and now only a quarter of it is wrong, and then we revise it again if we keep learning. Just setting someone up with a theoretical structure for a topic is really valuable, even if part of it is wrong because you don't have time to get all the details right the first time around.
You just completed classroom research on iNeuron, first in pilot studies last fall, then in a randomized controlled study this past winter and spring. What did you learn about the teachers who took part in the study?
The teachers we recruited for the iNeuron study had all taken part in BrainU, a program conceived and led by Janet Dubinsky in the neuroscience department at the University of Minnesota since about 2000. It's a professional development workshop for middle and high school science teachers that teaches them how understanding neuroscience can inform the practice of teaching and learning. I helped coordinate and teach BrainU in the run-up to the study, and I was impressed by how motivated those teachers were to sacrifice two weeks of their summer to study neuroscience with everything else they have to do.
And after being in an academic environment where we think, "Here are the principles; go use them," I would hear the teachers come back and say, "What do you mean — I have to use this? This student didn't eat breakfast this morning, this student missed class the last two days, this student doesn't speak English very well," and there are just so many complex concerns that they have to negotiate every day. Asking teachers to suddenly apply this exciting new neuroscience finding in their lessons is sort of ridiculous. So I started to appreciate the complexity of trying to translate the research into practice.
Were their unexpected discoveries arising from the study?
I was pleasantly surprised by how many students really cared about solving the challenges and getting the points. Just the inherent motivation of a game, a game by the way that covered a complex topic, and that for most students wasn't part of their core curriculum. A large percentage were raising hands — "Hey, I'm stuck, can you help me with this?" — even though it didn't count towards their grade.
What has stood out for you as you have reviewed other scientists' learning games research?
It’s hard, there’s such a wide variety in the quality of published research on learning games that there's not a clear set of principles rising up from it. Each researcher's game is specific, and they are mostly trying to find out if their game works, not trying to extract general principles of games for learning. I just read a review and meta-analysis from 2015 by Douglas Clark and colleagues — he works with a really good group at Vanderbilt.
I was struck by their finding that visual realism and story depth didn't significantly predict efficacy. This is a little counterintuitive because the sense that I get from the field of learning game design is that we're all pushing toward games that look like the games kids play at home, and if we did that, they'd be better learning games. But they didn't find significant effects of, for instance, having a rich visual environment.
They also found an advantage overall for individual play versus group play. This is surprising because there's a whole body of research on peer learning, on why learning things in groups is advantageous, so it makes me wonder if we just haven't hit on the right way to do group games, or maybe we're limited by the technology or something else, because I would have expected the opposite. When we measured knowledge gains with the iNeuron study, we also didn't find a specific advantage to group play over individual play, although the students and teachers I worked with in the classroom seemed to really engage with it.
What games did you play as a kid?
I played a lot of board games growing up with my family. Trivial Pursuit and Taboo, and my mom loved Scrabble. I saved up my allowance money and bought the original NES [Nintendo Entertainment System] Console and I played hours and hours of Super Mario 3 and Tetris and Doctor Mario. I didn't keep up with it — until the Wii came out, I wasn't really interested in Call of Duty, RPG shooter games — those aren't really made for me.
I have read and heard a number of times now, "If kids sniff out this is an education game, they won't want to play it." Maybe the point is they won't want to play it outside of school?...
Right. And honestly, I think that is a really lofty goal. The idea that a kid is going to go running to an educational game instead of playing Minecraft or Halo or whatever. That's a tall order. Instead, would you rather play this educational game, or read this textbook chapter? The comparison should be other educational curriculum tools, not games out in the real world. I don't think we'll ever win that competition if our objective is teaching difficult-to-learn science concepts in the classroom.