I’ve been giving variations on the “Precision of Emotion” talk for some time*, and synchronously have become, somewhat to my horror, a writer who doesn’t write. A big part of this has been my focus on youth, who also, for good and ill, don’t much write; they communicate through dynamic and interactive media. So I’ve relied on things like the (dynamic) video recordings of my sophia talks to convey these ideas — along with, especially, the (interactive) discussions that follow — and have mostly hesitated to codify them in written form.
However, I’ve gotten increasing requests for ’something written’ for citation purposes, as well as further reading — and GDC16 bounced my expansion on this theory (shout to my frenemies on the committee and apologies to all of you who asked for a longer version of the talk — there has been a lot of iteration since last year, which you can see at the UXweek link), and it’s fresh in my mind from my trip to Copenhagen and the Play for Real summit, so this seems the right time to get it all down. We’ll see if I remember how.
In 2011 my cofounder (Michal Todorovic) and I were working for Zynga. Our company had been acquired by Zynga in part because of our early adoption of analytics applied to large player databases (GoPets had its own internal metrics system). Zynga applied these analytics to generate then-astronomical levels of revenue, a practice that later spread throughout the social and mobile-social (and finally midcore free-to-play) sub-markets. We wondered if the free-to-play model — which we had been experimenting with from the early 2000s, long before it was dominant in the US — could instead be used to achieve equity in education. This was the genesis concept of Sense of Wonder.
It led me, ultimately, to GlassLab, and now to Collective Shift — GlassLab being a massive partnership endeavor involving the ESA, MacArthur Foundation, Bill & Melinda Gates Foundation, ETS, Pearson, the Institute of Play, and Electronic Arts. GlassLab and Sense of Wonder have grown together, and the two of them for me have captured a deep passion to effect positive social change through the deeply humanistic (user-centered is or should be synonymous with humanistic) ethos of excellent game design.
GlassLab, which I think most of you are familiar with by now (www.glasslabgames.org) was incubated inside the Institute of Play and spun out into an independent nonprofit in 2014. We delivered four games with four different missions: SimCityEdu: Pollution Challenge modified 2013’s SimCity for the classroom, constraining it into 15 minute modules and aligning it with environmental science. (It remains our most popular title, though Ratio Rancher is catching up.) Mars Generation One: Argubot Academy was a somewhat accidentally titanic endeavor, a from-scratch Pokemon-style RPG created to teach argumentation, an extremely difficult-to-teach skill that was at the center of the new Common Core standards in English Language Arts. (For me, as a philosophy major, it was more personal — and as NASA fans, we were all thrilled to be working not just with the Mars education team at JPL but with the National Writing Project, an outstanding organization based in Berkeley.) Plants Vs. Zombies Edu was an experiment in collaboration with rockstar learning scientist Val Shute of the University of Florida to determine whether an unaltered video game contained the potential for measurable gains in learning (spoilers: it does; Plants Vs Zombies creates measurable effects on problem solving). Finally, Ratio Rancher applied what we learned on Mars Generation One, compressed into a much shorter development cycle (18 months -> 6 months), addressing an equally aggressive competency, but in a much more narrow and deeper slice (ratios as a subset of proportional reasoning).
The process of creating these learning games forced me to formalize a lot of my intuitive game design processes. One of these was a kind of mapping exercise I tend to do with core loops — adding the layer of emotion to the mechanical verb of the loop itself.
I didn’t really realize this was weird until other designers told me it was. I also create “verb maps”, but all of this is probably the work of another writeup. The gist of the core loop emotion exercise is that I map core mechanics to atomic emotions, then blend those emotions into an emergent, hybrid core emotion that characterizes a game. This can be pretty useful for focusing your efforts and for assessing the alignment of your mechanics to your artistic intent. And the pertinent thing here is that I came in with a predisposition to focus on emotion in game design, because to me it’s one of the most important lenses we have as designers.
I also believe that this lens on games suggests a new way of categorizing games, by their dominant emotions rather than their mechanics:
And I personally especially like games that have incredibly rich, uncommon emotions at their core:
Raph was Wrong…ish
We came to GlassLab’s ambitious mission (to create commercial-level engagement around the hardest learning concepts) boldly because of Raph Koster’s statement in A Theory of Fun that “…in the end, that’s what games are: teachers. Fun is just another word for learning.” Raph’s book is ten years old now and even has a second edition, so mostly his thinking has been absorbed into the collective game design consciousness as a fundamental truth, but when he first made this statement it echoed throughout the game design community because no one had crystallized this insightful, now-it’s-so-obvious thing. It opened up new doors in how we approached design, too, and maybe even in how we approached viewing our own careers. (I do believe that there are categories of designers and only some are more teacher-aligned, but again, another topic.)
This statement synthesized worlds of developmental psychology that had largely been unknown to the game design community. The seminal text remains Jim Gee’s What Video Games Have to Teach Us About Learning and Literacy, though I’ve recently advocated Greg Toppo’s The Game Believes in You as a more digestable introductory text (and wonderful love letter to video games deserving of celebration by our community), in part because it references Jim’s work and a lot of others. I describe this as the depth hierarchy of nerddom re: fun, and to me it’s fun to think about the depth of intellectualism that underlies this connection Raph made to game design.
But there was a problem.
Like I said, Raph’s tenet about games and learning, which rang so true to so many of us, led us to approach the challenge GlassLab was surmounting with confidence. It felt like something was in the air — that all we needed to do was turn our attention to education and all would fall before us like so many Tetris blocks.
But it turned out to be much more complicated — and much more interesting — than that.
We initially brought SimCity into classrooms and showed it to kids just to see what they’d think of it. In the first place, they found it intensely frustrating — which in itself says interesting things about the way SimCity has matured, and how very hardcore that game actually is (while seeming so beautiful and intuitive). When we sanded down some of its edges, simplified its UI, provided a focused goal — all things that were very necessary to have it effectively used by middle schoolers in a 30 minute window of time — they said something even more devastating:
“This is pretty fun, but we’re not learning anything.”
It was an intensely complex statement that I still mull over today. In the first place, they were wrong about not learning anything, and we were able to prove that — but they thought they weren’t learning anything, and isn’t that interesting in and of itself? How important is it to know that you’re learning, while you’re learning? Emotionally speaking, it turns out really important, for all that recent trends have lionized the concept of “stealth education” (which I have now come to believe is an idea that is deeply insulting to learners). Without this metacognition that one is learning, there’s a deep sense of dissatisfaction, especially in a school setting, where kids are expecting, demanding to be edified. It’s why they’re there, and it’s foolish to assume they just want candy instead. (I mean, sometimes they do — don’t we all? But mostly that would frustrate them.)
As we continued to develop our products, I started to notice that the affective state I was seeing kids settle into when we were hitting our mark wasn’t at all what I’d learned to identify as “fun” as a commercial designer:
They laugh sometimes, sure; especially with Mars Generation One we hit that magical “fight over the device” level of engagement. But what became clear over and over was that when they were genuinely learning they entered this zen-like, super-focused state that became immediately recognizable as a distinct emotion that wasn’t just “excitation” or “having fun” (the boy to the down-right is displaying more traditional “fun” — he just took down an opponent’s argubot, and it was hard to get him to go home with his parents). This emotion is deeper, richer — more profound.
Remember that I’m an emotion-focused designer. To me, the distinct and subtle differences between emotions are like wine (beer, actually; I don’t drink wine). I can stare into their minute variations endlessly — and this emotion was not only brand new to me but really fascinating for the deep sense of satisfaction it seemed to be activating in these kids.
So I started researching universal emotion systems, because I needed a name for it. It seemed to me that it must map to some concept in psychology. But — it didn’t. On the advice of one of our learning designers, I dug into the work of Paul Eckman. Eckman did the emotional categorization work (via facial expression) that became the foundation of the TV show Lie to Me, and Pixar’s more recent Inside Out. He took photographs of people expressing emotion all over the world and asked different cultures to label what they saw. What emerged was a core set of six emotions (he later added a seventh) that were universal, represented in all languages across all cultures.
However, looking at these universal emotions, I couldn’t find “learning” — or “fun”, for that matter. Happiness was about as close as it got, but happiness wasn’t it; surprise seemed to be really important and onto something. But the others were present also: fear, certainly; disgust is probably the core emotion for 14 year olds in League of Legends; sadness plays a part, and fiero is certainly grounded in anger.
As I looked for learning and fun in this collection, a pattern slowly started to emerge. “Fun”, and that learning core that is the seed of it, wasn’t one emotion, but a process between three of them — and that process was reliable.
I define fun as: the cognitive mechanical process by which we convert fear into happiness through surprise.
I believe that this psycho-emotional process is at the core of what Vygotsky, Piaget, Gee, and Raph are talking about: it is how our brains are structured to reward us for making additions to our world model. Adding to our world model is critical; the more effective our model, the greater our probability of survival is, and so it would make sense that our limbic system would directly reward us with this deep-seated satisfaction for successfully expanding it. The satisfaction that comes from learning I would argue is equal only to the satisfaction that comes from sex — our body’s way of rewarding that fundamental reproductive impulse.
And don’t listen to anyone who tells you some people don’t want to learn. Our problem across education and society has never been that people don’t want to learn. It’s that we can’t agree about what they should learn, and that they might want to learn things we don’t think are useful for society. Learning is fundamental to our humanity, which is what Gee and them all have been saying for centuries too. It rests at the center of our play drive, that thing that makes life worth living.
This isn’t to say that we don’t struggle with learning — far from it. The opposing force to that deep dopamine reward that comes with increasing the world model is the deep sense of threat we feel when we can’t accomplish that. So, using the gear metaphor above, if a person is grinding for too long on the left side of the wheel, they’re going to powerfully reject the whole structure, because the inability to add to the world model becomes an existential psychological threat. It’s why creating a great, broadly-applicable learning experience is so darn hard.
When I was initially developing sophia I got presentation advice from MJ and my friend Christina Wodke. I don’t remember which of them first told me this, but through their advice I realized I had to actually give the audience an experience of sophia for it to really resonate.
After I explain that the body contains 100 trillion microbes — a ratio of 10:1 microbes to human cells — I stop and ask everyone how they’re feeling. There is always an uneasy laugh. One of the important things about trying to measure emotion is method and timing (much like comedy itself). The immediate reaction to this new information has to be interrupted before the brain has time to sort away and sublimate those feelings, which it does very quickly. If you’re in an average state of not knowing this weird fact that our bodies contain so many microbes that we are arguably more microbe city than we are “human”, the reaction is pretty universal: do not want.
Fortunately, I don’t leave anyone there. I explain that not only does this give us new insight into ourselves — 100 trillion microbes is like saying we have a thousand Milky Way Galaxies inside us (the milky way contains 100 billion stars). These cycles and systems exist in a delicate balance, building us into these towers, these cities of living things, that make our bodies really more like coral reefs than we’ve previously realized. Not only that, this new awareness of what scientists are calling the second genome is leading to all kinds of insight about previously intractable systems: notably heart disease, just as one example.
A fascinating study in 2013 dug into the bizarre thing that happens if you feed a vegan a steak. In a person who eats meat regularly, consuming a steak releases a bloom of the chemical TMAO into the blood; this is the chemical that causes arteries to harden. In a vegan, eating a steak releases nowhere near the same amount of this chemical — they seem to be resistant to heart disease, even if they engage in the behavior that, in a meat-eating person, would cause it.
The answer lies in gut bacteria. Eating meat regularly puts a lot of L-carnitine into the gut, which causes certain colonies of bacteria to flourish that like to eat it. When they eat it, they release TMAO — hardening our arteries. If you don’t eat meat regularly, these colonies can’t survive and gradually die out. So it isn’t as simple as “red meat causes heart disease” — regular red meat, sufficient to sustain populations of TMAO-releasing gut bacteria, cause heart disease. (I eat about one steak a year, and it looks like I can stick to that.) Insights like these are opening avenues to new kinds of research on human health, and have led scientists to call the human microbiome “the second genome”.
Usually, by the end of this story, people are feeling a bit better about their microbiomes. Hopefully you, having gotten only the reading experience, feel similarly. That arc is sophia: from anxiety and disgust through a moment of realization and insight, and from that insight to the expansion of the world model, and the satisfaction and sense of wonder that that entails.
The Sophia Arc
After GDC, I continued to expand sophia in ways intended to make it more of an applicable tool, beyond just being a theory for what happens with effective learning. One of the critiques and points of confusion early on was my representation of the starting point being “fear”. I stand by this general opening when it comes to Eckman’s supercategories. I believe that, more properly, what happens when we first encounter something that might be outside our world model, our first response is curiosity — however, once we verify that it is indeed outside our model, we move quickly into anxiety. (Consider the microbes.) A smaller oscillation can happen here if we find that we initially think it might be and it turns out to be already contained by and confirmed within our world model — this might be a kind of rapid sophia oscillation that moves very quickly to satisfaction and is therefore both more pleasurable and less memorable.
Memorable learning happens when we continue past anxiety and arrive on threat. This is what really kicks up our limbic system. In this phase, we transition from this might be outside my world model (curiosity) to this is definitely outside my world model (anxiety) and then not only is this outside my world model, it could hurt me (threat/fear). I think a lot of this happens below a conscious level, but this descent into threat is what activates the centers of our brain that tend to retain memories longer and more formatively.
The interesting thing about this from a pure learning standpoint is that 1) we don’t typically think about threat being presented as a positive feature for learning, especially in school; 2) the actual conceptual analog to this threat-state is context. The threat kicks in because we recognize a connection to our own life experience wherein the absence of this system in our world model makes part of the world unknowable to us, and therefore potentially threatening. (Absence of context is what makes so much rote-based learning boring!) With the microbes, this sense of threat is pretty visceral: there are these little creatures swimming around, outnumbering me, and I didn’t even know they were there! What are they doing? What if they turn against me? What if they make me sick? How can I get rid of them? Maybe I can’t get rid of them!
We need to not stay in this threat state for too very long or we’ll burn out and quit — especially a possibility in the fundamentally voluntary world of video games. However, we do need to stay in it a little while, or those memory channels won’t be fully activated. We need to experience the stress of resisting the threat, and then actually fail a bit. This failure is a very important part of ingesting the new system into the world model. By experimenting with possible understandings and failing to match them to the reality of the system, we triangulate what the actual nature of the system is. If we succeed too quickly, we don’t necessarily know that our model is complete — we don’t have those points of failure as data to really home in on how the system functions. This is where my microbe example falls down; because I give you the answer right away, you don’t have an experiential map of the concept, you just have what I’ve told you. I might have assuaged your sense of threat, but I haven’t provided a deeply memorable experience because your incorporated system is small and simple, not comprised of the negative data points you would have gotten through exploration and failure before arriving upon the correct system structure.
Once we have failed a bit, we also have the opportunity to encounter that genuinely magical state called tenacity. Recent research into longitudinal success in life has begun to triangulate around this concept and its neighbors persistence and confidence, both of which rely on the friction of failure to really prove themselves. Tenacity is the experience of capability — feeling tenacity gives us the deep subconscious message that things that are threatening to us are surmountable. There is a deep and restful satisfaction in this — especially when it leads to the breakthrough into insight.
Insight is the close cousin of surprise, and is the most important moment in the curve. This is the point when the system clicks into place; when repeated failure has eliminated sufficient possibility that we land on the correct answer. It is a pure, thrilling feeling, possibly one of the greatest heights of pleasure that we know. This is the moment in which the new system locks into our world model. It should be noted that it’s fragile; further failure can disrupt it. Usually what we need from insight is a sequence of confirmation that follows — approaching the system from another direction using the same system model, confirming the truth of that model by running different data through it.
There is a side branch here that I don’t discuss in the talk: most “game fun” exists in this oscillation of insight and confirmation. It’s a subtle sub-area, but I would say that the difference between learning-learning and learning-fun and fun-fun is this balance between tenacity and insight-satisfaction. A “pure fun” entertainment game wants to keep that tenacity curve pretty shallow and provide insight quickly, then keep the player in this highly satisfying, drug-like state of insight-happiness-insight. The focus is on getting a player into mastery and then giving them tiny tiny variations on the model that allow them to re-experience insight and mastery again and again (the variations are very necessary because otherwise mastery without variation quickly becomes boring). Learning, being closer to the real world, requires more tenacity and involves more threat as true world systems are vastly more complex than game systems, and so we spend more time in this tenacity space and less in insight-happiness — but those rarer moments of insight-happiness are perhaps more deeply satisfying because they are connected to the larger nodes of our world model, rather than just game mastery. (This connection to major nodes rather than closed game systems is how I differentiate a meaningful game — a meaningful game changes our world model beyond the nodes of game mastery. In short it has transfer of some kind.)
This arc is also where charting this emotional progression maps over into storytelling, and I am probably accurately unmasked as something of a narratologist when it comes to my focus on sequences of emotion — but then, I have yet to meet a great game designer who wasn’t at least some kind of storyteller. (And by no means do I think games should be studied by narratologists in replacement of ludologists, so, you know, lay off.)
It all comes down to creating memorable sequences of experience, and storytellers have been doing that with no tools besides their own voices for a very long time. This arc, though, isn’t unique to storytelling so much as it reflects back the fundamental way that our limbic system relates to memory.
Aaaand this is already about 4,000 words, so I’m going to take a break here! Stay tuned next time for…Emotional Metrics: The New Frontier.
* Sophia began as an abstract I wrote for games+learning+society, which became a workshop at GLS2013, then a GDC talk in 2014, then culminated — I think — in my UX week talk (“Wind, Not Sand”) in 2015. Overall I’ve presented sophia more than a dozen times, at GDC, GLS, UXweek, ReDesign, USC, MSU, and most recently PlayReal in Copenhagen. … I consider all of these audiences my collaborators on this concept. If you were there, thank you!
Appendix: Further Reading:
- Jim Gee’s What Video Games Have to Teach Us About Learning and Literacy
- Raph Koster’s A Theory of Fun (and of course Raph’s blog, www.raphkoster.com)
- Katherine Isbister’s Game Usability: Advancing the Player Experience
- Greg Toppo’s The Game Believes in You
- Nico H. Frijda’s The Laws of Emotion
- Daniel Cook’s “Constructing Artificial Emotions”, and The Lost Garden (www.lostgarden.com)
- Will Meek’s “Processing Emotions”, particularly the body map of emotions and the Plutchik emotion circumplex