I think the idea of a smart character may be an essential dividing line between fiction and RPGs and in turn may be informative of a major split within RPGs along similar lines. So based on that let me throw out a few points that I'm hanging this off.
First, most RPG players (at least those over a certain age) were book readers in their youth. This is not a 100% map, especially as you broaden out into those who came in via LARP and later through video games. Still, if someone came into the hobby via D&D or something of its ilk, especially if they were a GM, odds are good they had left a trail of sci-fi and fantasy paperbacks in their wake during their formative years.
Second, the ideas of being a reader and being smart are hopelessly entangled in American culture. There are other elements to be found in that snarl about escape, outcasts, isolation and so forth, many fo which also have some resonable in RPGs, but for the moment I just want to focus on the idea of reading being something the smart kids did.
Second and a half, that idea is not just perpetuated by the culture, but also by the books themselves. A huge number of books (especially several schools of sci fi) are really about the struggles of the unappreciated smart people (a group the reader likely identifies himself with) against the masses of idiots or to save the masses of idiots who can't appreciate the real problem. No surprise - "You're smart and everyone else is stupid" is one of the most comforting narratives humanity has ever created.
Third, because of this, there is power in a smart protagonist. I don't mean in the Holmesian vein of smartness-as-superpower (more on that in a second) but rather a protagonist for whom being smart is an essential part of their nature. This is every fictional detective ever, sure, but it goes deeper than that. Go back to your fairy tales and consider how often they are resolved with cleverness - that's how far back this goes. It's no shock - storytelling is an action of thought and word, not muscle and power, and it has always been in the interest of storytellers to create a world where their virtues triumph.
All of which means that excepting when we read for Schaddenfreude, we look for a protagonist who is smart enough. Ideally, one who is just smart enough. If he's too smart, that's a problem because we don't want to feel stupid, so he'll need to be crazily, holmesian smart for us to be comfortable (because at that point comparison is just silly). Most cynically, you want a protagonist who values and presents smarts (as the reader does) but is perhaps fractionally less smart than the reader, but there are a lot of potential variables in that formula.
Now, like all statements about fiction, none of this applies universally. There are a lot of things that make a grippy protagonist. Some of them are unique to the protagonist, some are unique to the situations. Whatever the case, if you map it out, there's definitely a clustering on the line between "Everyman" and "Unique, special snowflake, chosen one" where you find the Smart everyman, and that cluster is full of Military fiction (Jack Ryan is the poster child for this in my mind), Sci Fi and detective stories.
Hopefully, none of this is terribly contentious yet. The books are out there, so it's pretty easy to check. The trick is where this ties in to gaming.
It seems reasonable that clusters of reading trends would be reflected in some way in RPG trends given the overlap between the groups. Even more, its possible that certain book trends will be reflected more strongly in gaming because the transition is easier. An obvious example would be Lord of the Rings vs Anna Karenina - tabletop RPGs have, historically offered many more opportunities for the former than the latter
My sense is that the Smart Everyman segment got pretty well represented, especially early on. The combination of sci fi interests and wargaming casts a very broad net over this audience.
Now, that's a lot of words to come to a point which is pretty much a "no duh" for anyone who has ever attended a convention. The presence of this segment is obvious to see, and make up a large part of the Sharks to the narrative Jets. It's a well known, well stablished divide.
But the reason I took this long route to get here is that common root of fiction. It seems to me that the Smart Everyman player is as much a product of his fiction as anyone else at the table, but everyone goes to great lengths to pretend otherwise, especially the Smart Everyman player himself. He wants the things he knows from his fiction - intelligence, challenge, high stakes and realism - and that's totally at odds with all this mamby pamby story stuff. And the serious dramatic player is ok with that as a point of division because the alternative would be accepting that Tom Clancy gets a seat at the table along with Calvino, Eco, Martin and the Coen brothers.
The problem, of course, is that this is kind of nonsense. Whether you like Tom Clancy or not, there is some serious craft that goes into what he does, and he is as much subject to rules of drama and fiction as any other writer you want to point at. But he absolutely has different priorities, goals and tools.
This intrigues me. I'm a shameless hippie narrative-leaning kind of player, and a lot of this is me struggling with my own blind spots, and the fact that I suspect we have often left a lot of excellent tools on the table because they weren't the right type. If we accept the premise that Tom Clancy style play is just as narrative as anything else, can we proceed forward from there in a way that is satisfying to those players? Can we make the game that gives them the experience they want, and will they welcome it? I have no idea, but it seems like an incredibly fun question.
You can find a lot of people who will tell you what fiction must have: conflicts, rising tension, shifting emotional charges, all that jazz. In fact, if you were to listen to most writing advice, you would think that the creation of great stories is a very nearly mechanical process. In fact, the more money the author stands to make based on you learning his lessons, the more likely it will seem that creating fiction TOTALLY MAKES SENSE.
Which is nonsense. The best advice is not couched in terms of what will work, but rather what might be worth a shot. Those who make compelling fiction can be just a surprised as anyone else at what people lock onto. The overlap between great books and books people read is always smaller than some would like.
But if you're creating or running a game, that's not your problem. You have a table to engage, and that is not bound by the rules of good fiction, that's bound by the rules of what people respond to. And while those are no more concrete than those of creating fiction, we can afford to get our hands muddy with "crap" fiction.
And, hell, maybe we owe it to ourselves to do so. In any case, I feel like this is the tip of an iceberg.
1 - I would normally say "him or herself" here, because the reader could just as easily be female, but i couldn't bring myself to for a simple reason. A lot of the fiction I'm talking about here is unapologetically manly. Women may be smart and capable, but only if they are supportive and sexually available, which in turn leads to weirdly screwed up ideas about "strong women". Anyway, not really intending to explore this topic except to say that some science fiction really screwed me up in some ways that it took me a long time to understand.
2 - So, no disrespect to the many talented writers at White Wolf, but the connection between the initial success of Vampire and tapping the crap fiction vein seems pretty obvious in retrospect. I doubt it was intended that way (because, man, that's a SINCERE book) but that doesn't mean it didn't benefit from it.