Thursday, January 14, 2010

Metrics

Nothing is perfect. That can be a kind of defeatist perspective if looked at the wrong way, but it's also a source of optimism. Everything provides an opportunity for improvement if you can just stop and learn the lessons of what went wrong.

This is often easier said than done. "What went wrong" is a terribly amorphous idea, especially when dealing with something that has lots and lots of moving parts but will still lurch forward instead. Some endeavors have outright failure conditions. If a car won't start or a house falls down, then there's clearly something wrong. But more often the car is making a strange noise or the house has intermittent electrical problems. These are much harder to address than a true failure, if only because failure demands action. Persistent, systemic problems that don't actually cause failure can be tolerated, often well past the point of reason.

This is why people in expensive suits talk a lot about "metrics". In the absence of a failure scenario, you need some way to improve upon things, and the only way to consistently do that is by measuring things, changing them, then measuring again to see if there's some improvement[1]. But what should you measure? This can be surprisingly hard to figure out, and it's valuable enough to explain why those guys can afford such expensive suits.

So, take all those problems and complicate them further with ideas of "fun" and "entertainment" and you have the problem of applying this thinking to RPGs, where I would argue it's desperately needed.

First and foremost, how can you even spot a failure scenario with an RPG? Is it the game that doesn't happen? That doesn't finish? That finishes badly? Who's to say those breakdowns came from the game and note from secondary failures (perhaps in transportation or coordination). As individuals, we may have some spectacular car crashes of games we can think of that we might be able to deconstruct, but those tend to be so intensely personal and specific that there's not much room for common language. That is to say, if you want to deconstruct one of your failures, then don't expect much help.[2]

And that's the big easy part. What about games that don't fail? What concrete things can you look at and say 'this didn't go so well' so that you can try something to fix it and see if it works? What kind of metrics can you even look for in an RPG?

I don't bring this up rhetorically. I genuinely don't know, and it frustrates me intensely. There are a few vague shapes - you can track resources in a resource management game and use that as a rough guideline, but that tends to be very rough indeed[3]. You can track concrete social elements like attendance and time played, but I'm not sure what those tell us.

Arguably, the best case is probably to just pick some things, assign some numbers and fake it. If I rate "Player enthusiasm" at the end of every session from 1-5 then there's lots of room for error, but over time I am likely to get some useful trending, even if the specific numbers themselves are only so reliable.[4]

But what to measure? What should the metrics for a successful (or failed) game be? I'm intensely curious to hear people's thoughts on this.



1 - This is contrasted with just going by gut. Some people can do this, and that is truth, but many many more people THINK they can do this than actually do. And, of course, determining whether or you can is probably best not judge by your gut.

2 - 'But I can discuss it on the Internet' says the hapless optimist. And technically, that's true, and there might even be some faint insight to be gained in that fashion, if you're willing to dig and sift through the noise, but at best you'll usually get the answer for how someone else (with perfect hindsight) *would* have done it. This is kind of poisonous because, like the TSA, we see clear trails into the past and think it should have been equally obvious in the other direction, and the problem is that clearly we just missed it. From that perspective, it is too easy for the guy giving advice to be taken as a guru. And we don't need any more gurus.

3 - For example in D&D, you an track damage taken, healing surges and powers used, and use that to gauge how well you've been balancing encounters. Unfortunately, there are so many other variables (such as changes form leveling up, different monster abilities and dumb luck) that it's only so useful.

4 - There's a dirty trick implicit in this, and that is this: you get what you measure for. As such, deciding what to measure for is also an implicit declaration of what you want to see more (or less) of in a system. There's some really fascinating stuff about this and the impact of the Apgar Score (a measure of the health of newborns) in Atul Gawande's Better.

Wednesday, January 13, 2010

Yes, I liked Casino Royale

You know what I love about the Daniel Craig Casino Royale? There's this great moment in is where everything goes pear shaped - Bond gets suckered in the big poker game, and it looks like the whole mission is crashing down around him. Daniel Craig totally sold me with what happened next, as he grabs a steak knife from a passing tray and hides it behind his forearm, stalking towards the villain with a pure murderous intent born from the loss of all other options.

I barely remember large swaths of the movie, but that scene sticks with me. I think it's because, in its intensity, it really gets across something that makes these sort of spy-ish stories resonate with me, and with a certain kind of audience.

This isn't intended as a deconstruction of espionage thriller - I'm hardly one to provide that - but rather just a spotlight on issues of knowledge and control in fiction and, by extension, in gaming. As I see it, there's a specific itch that some spy stories scratch, and itch for a sort of smartness. We delight in seeing the hero make the right decisions because h knows what's going on. Because he's the man with the plan.[1]

It's the little thrill when the bad guy pulls his gun and it clicks because the hero has already found it and removed the bullets. We love this idea because it showcases the idea of information as control, and that's an idea that appeals a lot to the information-centric (which includes most nerds). We embrace a hero who's in control. Even if the situation goes out of control, he's on top of the situation and in control of himself. He understands the odds, and only plays the game after he's stacked the deck as far as he can in his favor.

Now, if you stop there you end up with characters who are cool but ultimately shallow. The bit that pushes it over the top for me is that moment when it all goes to hell, but the hero must still act. What makes it so sublime for me is that at that point all of that understanding turns around and becomes a burden. Because the hero had such a profound understanding of the risks thathe has been carefully managing, he understands exactly how bad it is to go off plan.

This issue of understanding is where it really lights up for me. It underscores how important the issue and course of action may be. The emotional commitment that suggests is intensely powerful.

This, in turn, speaks to one of the things I love about crunchy games, like Rolemaster. There is a similar level of understanding (in the form of the mechanics of the system) of just how bad things can get, and a similar emphasis on stacking the deck in your favor before you want to enter into any conflict. And that knowledge sets up a great test of player engagement - at some point they will ask: are they going to be invested enough to go off plan? To face down a fight they have no reason to expect to walk away from?

One of the great conflicts in games is that the kind of capricious death I'm talking about risking here can be both intensely fun and intensely frustrating. Dying from a dumb turn of the dice can be dull, but so can any kind of plot immunity.

I sometimes suspect that the nature of this conflict is such that there is no real mechanical solution[2]. This requires a balancing of factors which falls cleanly within the domain of the skill and actions of the GM. It is this understanding that makes me very sympathetic on the topic of GM fudging as an implicit statement that an imperfect solution driven by a skilled hand is more powerful at any given moment than a more idealized solution.

Not to say that some games don't try, often admirably. But they do so by choosing certain priorities and running with them, or by leaving a large area open to GM interpretation. I don't really see a difference between "Fudging" and "Broad GM Interpretation" in any meaningful way, but I also know that others feel there is a very bright line between the two. The rub in this situation is that strong, explicit mechanical consequences (as opposed to interpretive ones) are what carry the player's ability to understand consequences before they happen, which is at the heart of them taking risks with both eyes open. That predictive element continue to exist with dice fudging (because the potential outcomes are still there to be seen, albeit with altered odds) while interpretive consequences remove that possibility entirely.[2]


1 - This is arguably a subset of competence porn.

2 - One other fix for this is to declare consequences in advance, through stake setting, but I always find that works better on paper than in practice, especially for nasty consequences. For whatever reason, I feel likt there's a lot more blood and fury as long as things happen with in the flow of play, and that's equally true for the purely mechanical (like Rolemaster) or if it strongly creative (as in Polaris with "But only if..."). Stopping to negotiate (or spend too much time looking things up) breaks that flow, and changes decisions that were colored by the rush of the emotion of the moment into a more bloodless sort of calculation.[3]

3 - I use bloodless as a condemnation, but that's unfair. Distance can often be used to create something that is more powerful and resonant as an
idea, especially one that can be expressed later. Distance can create better story than immediacy, and some players really jam on having gotten to the end and having something beautiful and fucked up to look back over. It's fun, and you can make for great play either way, but mixing them mostly just makes mud.

Tuesday, January 12, 2010

Brown M&Ms

Atul Gawande is well on his way to cementing his place as a man who I will read with no prompting at all. His previous book, "Better" was one of my favorites of the past few years, and his recent writings on health care have been fantastic. I recently started his most recent book, "The Checklist Manifesto" and it's fantastic so far. I'll probably talk more about it at some point, but right now I want to steal an anecdote from it.

So, back in the day, when Van Halen was touring, David Lee Roth put a line in the contract for every venue that there be a bowl of M&Ms in his dressing room with all the brown ones removed. If he found any brown ones, they could nix the show at no cost to themselves. On one occasion, the clause was actually used to cancel a show. Typical rock & roller nonsense, right?
Not so much. Turns out David Lee Roth is a canny fellow, and there was a deeper purpose to this request. See, this was early in the era of really big tours. Van Halen rolled in with a dozen trucks and busses where the norm had previously been three, and the sheer number of things that had to be done to make sure everything was set up functionally and safely was absolutely daunting. There were so many variables that there was no effective way to check everything, so a lot had to be taken on faith.

The M&Ms were something of a barometer. If Roth came in and there wasn't a brown M&M in sight, then he could feel confident that everything had been done by the numbers, with the right level of attention to detail. But if there were brown M&Ms in the bowl, that was a signal that perhaps this venue wasn't as on the ball as they needed to be, and that was the cue to trigger and intense review (and if that review came up short, the M&Ms provided a concrete, inarguable escape clause.

I admit I was absolutely delighted by this story. Some of the joy came from the idea of a cunning David Lee Roth, but it is also a great example of how we deal with complex systems that we're not necessarily in a position to totally review. Brown M&Ms work like personal red flags, things you look for to tell you where to look next.

We all have these brown M&Ms when we look at an RPG. Whether its an extensive weapon list, reference to Rule Zero, excess boobage or bad design, I think we all have certain things that we check for (consciously or unconsciously) when we pick up a book and flip through it. I know I do, but now I find myself thinking about them more explicitly in an attempt to pin them down. It's proving surprisingly slippery.

How about you? Any brown M&Ms to share with the world?

(My wife, when I mentioned this, already knew this story. Apparently it was on This American Life - one more reason I need to catch up on that.)

Monday, January 11, 2010

This Tumblr Thing

The topic of Tumblr came up in conversation the other day, and it lead to three types of responses.
  1. What's Tumblr?
  2. Tumblr's awesome!
  3. Sell me on Tumblr.

Twitter's not a great venue for real answers to 1 and 3, so I figured I'd take a swing at it here, starting with question 1.

Tumblr is a free blogging service, like blogger. You create an account, they host the blog, simple as that. It's very lightweight and easy to use, and it has some very nice bookmarklets available to make it easy to post rich content (embedded images, music, video and such). One of the guys behind it is also the guy[1] behind instapaper, one of my favorite services.

So, that's all well and good, but if that's all there is to it, who cares? And thus we get to response #3.

First off, Tumblr is probably the easiest of the blogging services to use. If you just want to capture your thoughts and the interesting things you've found on the Internet, then there's no easier platform. I had previously considered blogger to be the easiest one to use - the blog you'd set up for your mom - but tumblr is simpler still. So if you only want to maintain one blog, and you don't want to worry about things like versions and plugins, tumblr is probably the best choice.[2]

Second, that ease of use still floats on top of a layer of fairly robust system, allowing for an account to have numerous blogs. This offers a utility that might be familiar to Livejournal users - it's wonderfully set up for temporary, purposed accounts, such as a journal for a character in a game. As such, it's worth just having an account to be able to use it for specialized interests. Also, like Livejournal, it includes a view of all the other tumblr blogs you follow via your dashboard. it's not as sophisticated as LJ's friend management, but it does the job for basic friend & community stuff.

Third, it's an excellent clipping service. If you purely want to track images, articles and videos that catch your interest. Tumblr's a great bucket to dump those into.

The last advantage of Tumblr is one that's fairly specific to established bloggers. If you've got a blog that's committed to a particular topic (or even to a particular sort of tone) then you have a commitment to that mode. But sometimes, you just want to post a LOLcat, or a neat picture, or some snarky comment that particularly caught your interest. A second blog that's just for that sort for thing can be incredibly handy, if only for your sanity.

Anyway, Tumblr is far from an end-all-be-all, but it's pretty handy, and if you're curious it's probably worth checking out.

1 - His blog is also a great example of how good a blogging platform Tumblr can be. If you're wondering what a good tumblr can look like, check it out.
2 - I chose blogger for this blog for reasons of simplicity. In retrospect, Tumblr might have been an even better matc

Friday, January 8, 2010

Talent Borrows, Genius Steals

I didn't end up responding much to comments yesterday because one question has very much been sticking in my mind. Semiocity talked about licensed properties and pulling RPGs out of other mediums. The key question that got me thinking was this: [H]ow does looking at what unique experiences RPGs offer square with the media or properties they attempt to emulate? And why play Star Wars rather than watch one of the movies?

There are a couple of obvious answers to that last. There's a natural tendency to take stories we like and run with them: fanfiction provides eternal evidence of this. Even more, it's a simple assertion that the parts of the story you've liked are ones you think would be better told through play with the tools of RPGs (Reflection and collaboration[1]).

That's all well and good, and it's easy to say, but I don't think it answers the question as well as I'd like. I think this is because when you pull it up, there's a bigger question underneath it: Why Star Wars?

Not actually picking on Star Wars here so much as calling out an observation: some properties are more suited to RPGs than others, and the pattern among them is inobvious. The instinctive idea that a certain sort of fantastic/high action movie (like Star Wars) is the right mix ignores the simple fact that a lot of fantastic/high action material actually makes for fairly bad play.[2]

In most cases where the setting fails the RPG there's a common failing. However broad in scope the story itself might have been, the setting purely served telling that particular story. The issues in the setting have been resolved, and it is not obvious where a new team of heroes might fit in.

Sci Fi is chock full of good examples of this. Babylon 5 and the new BSG both were full of interesting material, but both also explored the hell out of it and left very little on the table. Heck, look at Avatar - it's got all sorts of great game elements, but now that the events in the movie have happened, what is still interesting to do?[3] Put another way: interesting things MUST happen someplace other than Arakis, but who cares?

There's much more to play when the heroes are only a small part of the larger picture (like early seasons of Supernatural) or if there is a sense that the setting is much larger than what we're seeing on screen. Lucas is really good at giving a sense of scope - of leaving enough loose ends lying around to give the sense that there's lots more to this setting, past and present, so Star Wars is pretty easy to play (even though it is rough to play at the same time as the movies).

And that dovetails back to reveal another answer to my original question. If your setting is particularly rich, then an RPG may be the best way to explore it. Not the only way, as I continue to qualify, but it plays to the particular strengths of the medium.

So there's one more. Thanks, Matt!

1 - Here's where I acknowledge several comments it this effect. Collaboration, and especially collaborative creativity, deserves mention too. While it's closer to a technique and not really a reason to tell a story in RPG form, it's an excellent reason to tell an *incomplete* story in RPG form, and that's good enough for me.

2 - I'm going to name some names here, but remember I'm not saying it's impossible to make a good game out of these settings - the right people could make a great game out of the setting material in a car ad - but rather that the setting as presented in its native medium is a poor contributor to the possibilities of play.

3 - The solution to this is to make stuff up, and since RPGs have a strong draw to creativity, that seems an obvious solution. You can introduce a new and mysterious race to B5 or an ancient threat from the Ascended Na'vi to Avatar, and that can be pretty cool, but it will call into question why you are using someone else's setting in the first place. By diverging, you are isolating yourself from the social element of using a familiar setting, and potentially invalidating further changes in the setting from the source material.[4] Perhaps more importantly, by creating, you are asking yourself why not just keep creating?

4 - This is the advantage of "dead" settings: Because the material won't me changing, it is more reasonable to take ownership of it. Once that happens, then the culture of the game shifts from a common setting to a common starting point. The best example of this is the Amber DRPG, but I'd be curious if there are any others that spring to mind.

Thursday, January 7, 2010

Considering the Medium

So, this is where I answer my own question. I'm echoing a few sentiments that came mup in my comments, but also differing from some of them

The question is what stories RPGs tell, or more specifically, what story elements RPGs excel at. I'm not saying that these are limits on RPGs - it is a nearly unbounded form - but as with every medium, there are things that it excels at and things that it's less strong at. With that in mind, I'm not looking to provide an encyclopedic list of things RPGs can do well. Rather, I'm focusing on the things that an RPG would be the first choice for.[1]

The first and most obvious element is agency - players in an RPG may make choice to impact play. This is almost unique to RPGs - there is some overlap with elements of improv (a reason many RPGers look to improv for inspiration) but it is a different sort of animal if only in terms of the framework it exists within. This is kind of awesome, because agency leads to investment, but in and of itself it a hollow thing, like saying painting lets you use colors. It's true, but it's kind of dwelling on a unique tool, not a unique result.


There may be some fertile ground around the idea of investment, but I admit I haven't really found it yet. Like agency, it seems to be a tool, but there are some unique manifestations of it. It can take the form of shared knowledge of a truly staggering scope but I'm not sure how much of that is unique to RPGs. The shared knowledge of the Forgotten Realms is staggering, but the yardstick to compare it to is Middle Earth, something born of novels.

Creation? Perhaps one of the strengths of RPGs is the profound blurring of the line between author and actor. But if so, what can we do with that?

Maybe it's something more obvious: what about Play? After all, if RPGs are art, they're art you can win.[2] That feels closer, if only because it's definitely a unique reason you'd want to play an RPG rather than write or create in another medium. Heck, compare it to writing contests: novels may make for better stories, but RPGs make for better competitions! But again, I'm not sure that suggests anything unique about what they say.

The thing I keep bumping my head against is that I can think of a number of fairly unique techniques to RPGs, but those are useful only after I have decided to use RPGs as my medium. They don't suggest reasons why I'd choose to tell a particular story with an RPG rather than in some other way.

But there's one exception. The element that I think might actually be the most important is reflection - the events of play reflect back on the player. In other media, you may become invested in the characters, but in RPGs, the characters may become invested in you (or at least your character). The creation of a reality that looks back on the actor is huge, and it's often dismissed as mere sleight of hand since these people and places are not real, but I would counter that the fact that they are fiction does not rob this of its power. Most creative media rests on the idea that we may be powerfully and truly moved by fiction, and I see no reason to carve out an exemption here.

And that suggests that the reason I'd want to use an RPG, rather than a book or a movie or a play or a painting is if I want it to be your story.

That's some powerful mojo, but it has a few implications. First, it underscores the fringey-ness of the hobby. A game doesn't say much to the people who are not playing it (as any number of recorded sessions will tell you) but that's because it's not supposed to. The more it speaks to the world, the less it speaks to the table. That's awesome for play, but it also means we're unlikely to end up with our games hung up in some equivalent of the louvre.

Second, and perhaps of more immediate consequence, it demolishes most any sense of ownership or authority in the creation of play. A GM may do everything he can to make play awesome, but if he creates his story, he's misusing the medium[3]. It is only by surrendering that power to the players that he will really succeed. This can be a really, really hard thing to grasp, and it can come a s slap to the face to a GM who busts his hump making play rock, but there it is.

There are some further implications of this, but they enter into the realm of conventional wisdom. The insights that our hobby isn't scalable or that the GM shouldn't treat players as an audience are far from new.[4] But I wonder what happens if we embrace them as weaknesses of the medium and try to focus on the strengths. I imagine it looks weird, sure, but at the same time I sometimes suspect that we're trying to write novels with paintbrushes, and that we'd be a lot better served deciding what we're actually doing, and pursuing that with passion.[5]



1 - The great pain is that "Telling a Story" does not make this list (or at least telling a specific story). Books, movies and plays all do a better job of this for reasons I hope are self evident. Yes, you can tell a story with an RPG, but you need to jump through more hoops than you would if you were just writing it.

2 - Though speaking of which, you know what I'd buy a book on to see how it applied to RPGs? Editing reality TV. Those guys are really good at creating a narrative out of a bunch of stuff that happens and putting it all together like that's what's actually going on. It's masterful fiction, and it resonates a lot with the idea of stories being created as a result (not an intention) of games.

3- We've all seen the GM who should just be writing their book.

4- Though I amuse myself at least in the route that lead to those points this time.

5 - I feel like this is also dovetailing into my growing certainty that setting is king, but I do not trust that conclusion quite yet.

Wednesday, January 6, 2010

James Cameron is my Master Now

So, Avatar has made a million zillion bucks (last I heard it's over a billion now) proving that James Cameron is welcome to go into a cave for years any damn time he likes. It's no flawless gem, but a lot of people are watching it, and that has lead to a lot of interesting theories about why this is so. Me, I've got a theory, and it's a pretty simple one.

It's a really, really good movie.

Now, before you jump to your feet and call out your favorite flaw, let me unpack that a little. I was listening to NPR last night as they were talking about the latest foreign film to catch the critical eye, a German film called The White Ribbon. Now, I don't mean to detract from this film - I'm sure it's great and I'll probably try to see it - so take this with a grain of salt. I was hearing about this movie, from the news and from the director, and as he described the things that seemed to grab them (the ending doesn't resolve the mystery, the narrator is unreliable, information is hidden from the viewer) I was struck by the thought "Why don't you just write a book?"

My thinking was pretty simple. However cleverly done in a film, these ideas are well used ones in literature, and when they're done in a book they're not messing with your audience. Using them in a movie is much more expensive, and is a bit of a slap to the face of the implicit contract.[1] And that got me thinking about what movies really are as compared to other media.

The big ones I zeroed in on are that movies are visual, auditory, and they are of limited scope. There's also a more subtle fourth in that movies anticipate having all of our attention, as they're designed for the theatre, and this makes for some subtle differences between made for TV Movies (which include commercial breaks in their pacing) and films.

And within those bounds, Avatar excels. Consider: If Avatar were a TV show, the flatness of the characters would wear things down over time. If it were a novel, the predictability of the story would kill it dead. If it were a comic book the blue chicks boobs would be WAY too small. If it were a play, the weakness of the dialog would be laid bare.

But its none of those things. It's a movie.

This is a hard thing for me to get my head around as a media saavy guy. I consume a lot of TV, books, comics and anything else I can find. One of the joy sof this is the ability to freely take lessons and ideas from one medium and transfer them over to another, so they're one great wash. But it makes me want the best of everything: I want the stories of literature and the characters of great TV combined with the visuals of film and the dialog of plays.

Usually, this is a good thing. It raises standards, challenges me and challenges the material I enjoy. Worse, this is complicated further by the fact that some of the bleed is legitimate. A movie should have a good story and good dialog. But are those things as important in a movie as they are in other media?

The money says otherwise. And I'm wondering if maybe the money knows what its talking about.

So even if you think Avatar's a shitburger served up to a populace of sheep, think about the question this raises: what are RPGs good at?


1 - Don't get me wrong, there is value in shaking up the contract, and it's been done well, but it steps outside of the area where film excels, and depends on the genius of the director or cast. To pull it off successfully requires enough brilliance that it guarantees the quality of the finished product, but pulling it off badly is really easy.