Its a rather innovative technique, and delivers great results for only a bit of computer power.

The Big Granddaddy Algorithms

First, a bit of background as to what has been done before. The Big Granddaddy of pathfinding algorithms is the A*(called the A-star) algorithm. It makes the complete plan, every step needed, to get from one point to another, then executes it. Unfortunately, the amount of time this takes scales exponentially with the size of the map. It, however, does provide an optimal path.

The other algorithm is the LRTA* algorithm, which looks ahead, and moves based only on that information. For example, it looks ahead, maybe 3 steps, and sees what sequence of 3 steps will bring it closer, in a straight-line measurement, to the goal. This has the benefit of always executing within a certain amount of time. Unfortunately, walls between the agent and the goal can and will prove difficult to surmount.

An addition to the LRTA* raises the “distance” weighting of a node every time the agent goes over it, thus eventually forcing the agent to go around the wall. But it still takes bumping into the wall three or four times to finally find the correct path.

Its all real(time that is).

There are a few constraints that pathfinding needs to work under in games, and even in robotics(Phoenix Mars Lander anyone?). You need to have something that can operate and find its way with only a bit of computation, and predictably work well. It would be a disaster for a $70 million dollar robot to keep bumping into a rock.

As well, with ever more powerful cpus and gpus, gamers demand more and more units, and bigger and better maps. The irony is, that with increasing hardware power… comes even less time to compute pathfinding and general AI. Computer power has only quadrupled a couple of times since Starcraft, yet, we demand 100 times more units, with smarter and more intuitive AI, coupled with four or five times bigger maps.

Dynamic Lookahead

So, the way that D LRTA* works is first, they have dynamic lookaheads. Some squares on a map require more searching to find a near optimal solution, particularly around corners. One way to determine the lookahead value for a square, is to naively compute the required minimum lookahead depth to any other possible square. Then, store the pairs of coordinates and that lookahead depth, and just look it up! Simple, eh?

Not really. On a 512×512 map, that results in ~68 billion pairs to store. On a normal desktop, that would take about 2 years to compute(probably a lot less on a compute farm… google could likely do it in one minute, because its an extremely parallelizable problem.) This is quite unacceptable, both due to time constraints and space constraints. This is where the research group’s second innovation comes into play.

A house divided…

They found that splitting up the map into spaces, and then computing the optimal lookahead depth from one space to another worked very well. In the worst case, it devolved into LRTA*. However, there must be a better way to divide things up…

Clique Abstraction

[Sturtevant, Buro 2005]

Clique abstraction was developed in the above cited paper. It is the concept of taking adjacent tiles, and lumping them into one larger tile. For example, four adjacent tiles that are all interconnected, could be replaced by one larger tile. And so on, with larger and larger abstraction at each level. (Take a look at page 38 from the PDF)

It becomes a lot easier to pre-calculate the optimal lookahead depths when you have a sufficient level of abstraction. Granted, you lose a bit of that oh so sweet optimalness, but its worth it for getting things to work within a certain amount of time. Of course, walls and pockets on maps still give the budding algorithm a bit of trouble. Which leads us to the next addition to D LRTA*:

Setting goals

The team found that by setting intermediate goals, they take advantage of a small property of lookahead algorithms: they’re more accurate closer to the goal. By setting goals in each of the subdivided spaces, its plainly obvious to the algorithm that going that-away is better than going this-away.

As well, by not placing goals in spaces that are obviously farther away from the ultimate goal than the starting point, it makes the algorithm ignore those spaces.

Right now, there are two ways to do dynamic lookahead: pre-calculated with clique abstractions, or calculated on the fly with sub-divided spaces and intermediate goals. They both come with their pros and cons, and if the game company can afford it, it appears that precomputed is much more reliable.

Optimal lookahead:

Optimal lookahead depths for a Baldur's gate map

Stored Lookahead

Stored lookahead depths for a Baldur's gate map

I do have a few questions about the research, that unfortunately I didn’t ask at the time, for example, the outliers on the graph on page 70. One of the results for the little dots was very efficient, but the other two dots are optimal, yet expanded lots and lots of moves. As well, I’d like to know exactly what the different algorithms that were used for the graphs, specifically. I will ask the presenter today.

What else?

What needs to be done to expand this research, is to look at how to handle dynamically changing maps, and moving targets. As well, they haven’t looked at memory storage techniques or compression of precomputed files.(According to Information Theory, the data files could likely be extremely compressed, on the order of 75% compression or so.)

I just had one idea to put out there, for the general improvement of mmorpgs as a whole. One problem with the addictiveness of say, WoW, is the unlimited play model currently in effect. The player pays ~$15 , and they could theoretically play 720 hours(One whole month, with no sleep, downtime, eating, etc).

The downsides of this model are readily apparent, in that being offered unlimited anything, is that we like to get the best bang for our buck. If you pay five bucks for a buffet, you’ll have several helpings, just to take advantage of the deal. It is a natural human tendency to do so, however, it has deleterious effects when applied to new players of an MMORPG.

It creates a pressure, both subconscious and cultural pressure, of playing as many hours as possible to take advantage of the theoretically unlimited play. This can, and does lead to addiction with the game, which in turn can destroy lives, just as any other addiction can do so.

So to combat this, the solution in retrospect is obvious: pay as you play. The idea is to offer new and casual players the opportunity to buy “hours” of game time. Instead of buying unlimited game time, the player instead buys 60 hours of game time, which can be used over several days, weeks, or months.

The benefits are simple and varied. The player can now relax, in that they have bought 60 hours of game time, and can use at any time, with no pressure to play every hour of every day. For the hard working 30-somethings that just want to relax and raid a dungeon with their friends, this is quite literally perfect. As well, it is an immense boon to not have to worry about regular credit card charges, since you’ve been charged once, and will not be charged again until you want more game time.

This is a good thing, for the players and the game designers. It produces a non-addictive game, and allows new players to literally try risk free, with no credit card numbers needed.

The more difficult question, however, is, Do you allow for players to purchase unlimited play alongside the pay as you play, or only do pay as you play?

A few tips: multiple stables or barracks are useless except to block movement.

sacrificing characters is nearly useless, as you can only make one per turn.

Consider all your moves carefully, and remember, you can move pieces diagonally!

First, here’s my game, based off the RTS genre, which I call Paper Knaves. (If you have a better name, let me know!)

Setup
You play on an 8×8 board, the color of the spaces do not matter.

Each player gets one castle, to be placed anywhere within the first two rows on their side.

A castle can make either a stable, or a barracks, and place them anywhere within two spaces of the castle.

At the beginning of your turn, you can either make a new building(Your only allowed one castle though), or a new unit.

Castle makes a pikeman, a stable makes a knight, and a barracks makes a peasant.  Units are generated right next to the building, on any available spot.

Buildings can be placed within at least two spaces of any other of your buildings.

Combat
Units must be next to each other to initiate combat.

Pikemen beat knights in one turn, knights beat peasants, and peasants beat pikemen.

Any other matchup(IE: Knight charging down a pikeman) results in self-destruction.

Buildings take two turns to be destroyed. However, castles take four turns.

When a building is being attacked, that building cannot make any units.

When your castle is destroyed, you lose.

Movement
Pieces can move in any direction. You can move one piece two spaces, or two pieces one space. Allied pieces can move through each other(like a Knight in Chess).

Buildings cannot move.

Turn order
Your turn is in three phases:
1)construction: You can either build a new building, or recruit a new unit.
2)movement of pieces
2.a)Then pieces can attack units beside them. Units do not occupy the space of vanquished units
3)clean up(you remove all destroyed pieces, buildings from the board, and play passes to your opponent.)
Design process

The design process was relatively simple. We looked at the very core of RTS gameplay, which is resource management and combat. This resource vs that resource, and one win. Placement of resources also factored in as well. So, I broke down the RTS genre into three types of resources: time, units, and buildings. If you make the game turn-based, it becomes easy to balance the time aspect, and the easiest and simplest way to balance unit superiority is to use a rock-paper-scissors style dominance method. And I chose to have the win condition as elimination of your castle, to save time on mop-up procedures. However, it does mean that pikemen are that much more valuable, since there is only one building to make it. I still haven’t playtested the game yet. If you have any thoughts, considerations, critiscisms, verbal diarrhea, let me know at zeroth[a t]oddco[dot]ca

EDIT: Added and updated the rules for clarity.

Ah, Assassin’s Creed. I wonder if anyone else remembers feeling excited for this game? It certainly looked awesome. Innovative per-npc AI, good graphics, intriguing story, interesting gameplay. And I do remember that simply awesome promo video with the Lonely Souls song in the background. I was hyped for the game.

Who wasn’t? Why, even Ctrl-Alt-Del featured AC, and expressed excitement. Penny-arcade, Kotaku, many others were all excited for AC. Why, for a week, the promo video was one of the most-watched on gametrailers.com.

There was a very large marketing push for this game, and it ended up… not exactly meeting expectations. Why, one could even go so far as to say it was a bad game. But, this whole fiasco does remind me of one key fact with games: the heavier the advertising, the worse the game will be.

Just think about every game you’ve seen, where there were banner ads, videos on tv and the internet(where applicable), even print ads! Now, tell me if that game was any good.

There are of course, a few exceptions, namely any game made by Nintendo, Blizzard, or Bioware, and games that have already proven popular across the ocean(either Europe or Japan).

But, it seems, that when a game is felt not to be… blockbuster calibre, the executives decide to ‘cover their ass’, and push the marketing. Of course, that just pushes up the expenditures related to said game. And they even put pressure onto the reviewers(as detailed here).

The end result are a lot of broken hopes and failed expectations, and yet another broken game, pushed out before it was ready, wasting people’s time, money and effort. Is it any wonder that gamers are becoming so jaded about advertised games?

Pretty simple. Thats the conclusion I’ve come to after talking with a few female gamers I know. I will paraphrase a bit of an interview I did with a close personal friend, whom goes by the pseudonym Ruby.

Ruby is an artist, in almost any medium. She wants to convey emotion, meaning, and depth in her work, and from what I’ve seen, she does a great job of it. She wants to date a J-Rocker, change the world, and look hot doing it.

She does work mainly in drawing, with paints, charcoals, sketches, on the computer. Soon, she’ll learn how to do 3d models as well, and use her photoshop skills to great effect. She is also the lead artist of the game we’re working on at the moment, and she’ll likely end up being the lead writer as well.

She also plays video games, and had some interesting things to say about them. First off, “Girl’s games suck.” They’re made easy, devoid of content and depth, and stereotype. Thats insulting to girls. They are just as smart, talented, and ambitious as men, and often, more so. So why would their game be simpler, easier, and stereotyped?

Yes, having little susie play Dress-up with a virtual barbie is very enjoyable. But as little susie gets not so little, we encounter a small issue: people don’t think girls should play games.

According to Ruby, “There are no positive role models, and no encouragement from society.” Theres nothing wrong with the games we have today, except for the culture surrounding them. Its as enjoyable for Ruby to snipe someone as it is for me to do so. So then why is it that we think girl’s games need to be different?

Maybe it could be due to the over-lying themes in games focusing around masculine concepts, such as women with big boobs. That really does not help the cause any, and it just makes the men playing the games appear misogynistic and perverted.

Another culprit is games marketing. Name three games that used sex to sell the game, that actually had nothing to do with the game. I can: Everquest, Grand Theft Auto, and some random shooting game that had women in Vegas on it.

I’m sure there are a lot more examples out there, but Ruby’s point has been made. She feels that the culture needs to change, and be more accepting. It needs to have a female role model, and Ruby is prepared to sacrifice to make that happen.

So, basically, what needs to be done is for game developers to realize how some of their choices are sexist and do not at all contribute to the self-esteem of women. And also, for their to be a push in the culture to be more welcoming. Look at all the articles about how women are treated with surprise and disdain in online games, because “women don’t exist on the internet. And they most certainly don’t play video games.”(source)

So, if you have any comments, flames, threats, leave me a comment.

A few relevant resources:

• Girl Game Zone
• WomenGamers.com

The reason why it has been largely ignored is mainly because it is hard. Faster processors, better algorithms will not ever improve stories in games. We are used to thinking primarily of one-way storytelling, of a storyteller, and an audience. In games, however, your audience is the story. Their actions are not controlled by a central storyteller. In a haunted house game, if you script the scene such that the background music swells ominously as you move closer to a locked door, what happens to the story if the player backs off?

In the old form of storytelling, whether the main character went through the door or not was up to the storyteller. Not so with games (unless you decide to force them through no matter what, but that itself is bad) .

It is easy to see why the old ways of telling stories do not work in games. However, there are some very good ways of establishing a story. Writers know that good storytelling needs three things: plot, characterization, and exposition. Lets look at each of these elements in the context of envisioning a new way of telling stories.

First up, is exposition. Some of the best ways are to show the player important facts, instead of telling them, as per the old adage, “It is better to show than to tell.” Rather than have someone say everyone is scared, show their responses to the haunted house. When your character mentions the haunted house, subtle clues like facial expressions(if possible), movement away from the player, and a change in the tone of their dialog works well to convey this fact. A sample dialog:

Random NPC A: Ho there traveller! How goes this fine day?
Hero A: Can you tell me about that haunted house over there?
Random NPC A: Uhmm, hmm, yes, indeedy this day is fine, but I think it’ll rain later. I think I shall go now.
Random NPC A walks off hurriedly.

If most of, or all of the dialog is at that level, of subtle personal clues, responses, it provides a better sense of what is being shown. In that dialog there, we saw friendliness, turn to avoidance of the issue, even running away.

Second, is characterization. If your game allows for dialog choices, then a good way to convey characterization is to colour the choices in the direction required. What that means is instead of offering a plain good or evil response, the responses offered should be more ambiguous. As an example:

Random NPC B(a girl): And here the heroes be! What brings you back here?
Hero A’s choices:

1. You, beautiful creature.
2. Your sweet charms.
3. Your bordello.

As you can see, the choices are all different, yet all convey a similar sense of who the character is. In this case, he’s a lady’s man. Maybe a good one. Or not. That would have to be shown by the responses. Or even, that would up to the player’s choices.

However, this is still guiding the player, the old way of storytelling. It all depends on if you want the character to have character traits. For other such characters, providing a range is great. Recommended even! So, then for everyone else involved with the character, the above points about exposition work great for character development.

The last point is plot. And that is, quite simply the hardest. Some random blogger (IE, me) will not have a solution unless they’re particularly brilliant. Which I am… except I don’t have one yet. Some previous suggestions are storytelling by AI/heuristics, branching plots, or sandbox mode. By sandbox mode, I mean, GTA style gameplay.

And thats what I have today.

That has been most of the focus for AI work in most rpg games. Consequently, the state of the art is much more advanced. The reasoning behind this, is to tell a story. To tell a good story. And people standing around repeating the same thing ruins the effect of the story. It takes the player out of the game, which is not a good thing. In fact, its exactly what the designer does not want. The player ends up noticing small things, tiny details, that did not bother them before. This serves to lower their enjoyment of the game. Again, a bad thing.

Good NPC AI is supposed to prevent this from happening, but I think there is something similar to the Uncanny Valley when it comes to NPC dialog. The closer to reality, the harder it is to believe, but if its sufficiently cartoon-like, its somehow easier to immerse yourself. (Of course, the opposite is also true, where the more ridiculously bad the dialog is, the harder it is to be immersed. And so you end up with kind of a bell curve, balanced between really unpredictable and random, to really really bad.)

One particularly effective example I have seen was used in the Golden Sun series. In that game, you could use psynergy, kind of like magic, and one of the moves was Telepathy. Using it next to an NPC, you could hear their thoughts. And by changing the thoughts based on any current events, it gave the npcs a hidden dimension, a personality.

The writing itself in those games was particularly brilliant, which also helps. However, in addition to good dialog writing there have been some interesting efforts to provide personality and unpredictability to npc AIs. The first is the “job system”, where npcs go around do jobs, live their lives, go shopping, have needs and wants, and react accordingly when you disrupt them.

I see often on indie rpg games their breathless promise of a job system like that, where all the npcs have “real lives, real personalities!”. What kind of benefit does this bring to the player? Well, for one, it does provide a kind of personality, as in, “Thats a blacksmith. He’s in the blacksmith building for eight hours, then goes home, eats, and sleeps.” Yep, thats a lot of personality.

Thats what happens often with job systems. You’ve just added a huge dimension of things to make and define for your designers, more than maps, items, doors, main plots, etc etc. And due to the sheer size, it becomes a copy+paste affair, where every blacksmith in the towns previous are identical to this town’s blacksmith. Real personality, oh my!

Then there are the dialog trees. This works a bit better to tell the story, provide some personality, but there are still a few major issues. The first is simple, and horrific. Dialog trees end up almost being like those phone menus. Thats right, imagine those dialog trees as being read our loud, with your options each as numbers.

If you’re like me, you shuddered at this image. Or rather sound recording or…okay, I’ve stretched the analogy too far. But back on point. The second issue is the representation. You get a bit of dialog, then have a bunch of dialog choices presented to you. Again, this commits the big story-teller no-no. You remove the player from the story, by showing them the different choices they could be, and maybe should be making. What if they think their character would do something different?

Also, lots of choices do not make people happy. When I’m faced with six or seven different dialog choices, I suddenly begin wondering, okay, okay, what if I pick the wrong one, and I don’t hear more about the story like I’m supposed to? And I begin worrying. Then if you can start the dialog tree over again, you can try the other choices.

Oh wow, more personality! I can do all the dialog choices to find out all the information! Yeah, thats totally realistic. But the designers did decide to make that choice because of the sheer number of choices they gave the player, and it made the test group happier.

Therefore, to fix dialog trees, I have a few ideas. First is cut down the number of choices you give the player! Fewer choices means more happiness, and less “min-maxing”. It also serves to tighten up your story-telling such that no npc should need more than three choices at any one time. If you find yourself needing more, then you need another npc.

By separating different parts of the plot to different npcs, you can focus more easily on developing them, or give it to someone else. It makes the writing more parallel. So, by making the user happier, and making it easier to make the game faster, this solution works pretty well.

The second idea is to not use real sentences, but a vague concept. After all, rpg stands for role-playing game. Make the player play a role. Make them pick a response that fits their character, and let the player know this! Have responses that fit that vague concept, like if the player expresses boredom, the npc becomes more insistent, maybe even angry based on the context.

And there we have it. More brilliance. Till next time.

Interior decorators do what I call appearance level design. And the benefits of appearance level design are not all that important. They make people feel better, more at home. Its a more impressive room/house/office.

That is not real design, in my view. Real design is figuring the best placement of bathrooms and appliances in the kitchen for maximum comfort and productivity. Real design is invisible, unseen, uncredited.

Here’s a little task for you, to see what design really is about. Walk into any room of your house/apartment/mud hut/palace, and choose a common household item. I’ll take a knife block as an example. Now, spend five or ten minutes thinking about how you could improve a knife block, to A) make it easier to use or B) add new functionality that is not just combining two opposite functions (like a clock on a VHF antenna).

If you can come up with an improvement that is not just glitz, that actually makes the item better than it was before, then the item does not have good design. If however, you can come up with nothing, and you realize changing anything would be for the worst, the item has good design.

My example the knife block, works really well for its purpose: hold knives, keep them sharp, and keep them organized. Can there be an improvement, that would make using it easier, faster, more efficient at its purpose? After ten minutes, all I could come up with was improving the knives themselves. The actual knife block works great. Its easy to pull out a knife, you know by the size of the hole how big the knife is, and its easy to put back in. Some knife blocks have built-in sharpeners, and thats about the only improvement that could be made.

The fact is, it takes experience and introspection to see design around us all the time. In our games, in TV shows, in our world all around us.

Another excellent example is in Final Fantasy III for the DS. The actual interface is a hodge-podge of button-clicking and touch-screen use, not very good. However, in the item list, for each category of items, there is an 8×8 icon representing it. This sits beside each item in your inventory, allowing you at a glance to see what it is. Axes for viking weapons, a glove icon for accessories, a spear icon for dragoons. Its simple, effective, nearly invisible. I caught myself using the icons, and thats when I actually noticed them. Someone consciously decided to make icons for each kind of item, and displayed them. Why? To make it easier to deal with a large inventory!

Good design is all around us. And good design takes conscious effort. Any time you find yourself making a snap decision on the placement of an item, or icon, or graphic interface, ask yourself why it goes there, and is there any way I can improve this?

As a personal example of good design, I was tasked to make it so that users of an application would know that the file has not been saved yet. My first response was a dialog on exit(or any other context change, more on that in future posts) if the file was not saved. Then I thought, okay, the user, she’s likely not going to want to have to click through a dialog all the time, so how else can I do this? I added an icon to the upper bar, that appears when the file is modified. Its red, shaped like a stop sign, and has the word stop on it. Its placed right by the exit button.

Nowhere else in the application is red used, so when you go to load a new file, exit, you see the icon if you did not already. It disappears when you save, and it appears when you modify a file. No dialogs, no clicks, simple information conveyed quickly and easily. That is good design. It makes the workflow faster, the user quickly and easily learns what the icon means, and it helps them!

Next time, ask yourself why you are making the decision this way. If you can improve it, do so!

However, I was thinking about different AI ideas that could be implemented in an RPG. For this post, I will focus only on monster/enemy AI.

Now, the first thing that can make rpg battles/dungeons a bit more interesting is a context outside of battle, where what happens in battle does affect the overall dynamic of the dungeon. Here is one possible idea, similar in aspect to a genetic algorithm. Each enemy has a fear and an intelligence rating, randomly assigned within a predetermined racial standard deviation. Say, slimes are smarter than kobolds for example, on average.

Now what this means is that if you cross an enemy’s fear/intelligence barrier, ie, “They’re really tough, maybe I should run.” So they’d try to escape. If they do escape, what happens? Normally, you’d never see that enemy again, or at the very least, never be able to tell. However, what if the enemies that survive or escape you, communicate with each other? And even are able to formulate plans and groups to take advantage of whatever they learn.

This can be done by giving escaping monsters a memory, a kind of statistical observation of your performance. Then in a background task, have a couple or more of such monsters communicating their knowledge to each other. Now, based on the length of time since the knowledge was obtained, the detail, and what is observed, different information will be valued more than others. By sharing the information between the monster memories, the background task is able to determine your weaknesses. Say it saw your white mage fall to two hits, but your knight took 10. That would be taken into account.

Then, the enemies would lead a charge against you, with slightly modified battle AI’s. As per our example, focus on the white mage!

And since this would be largely time and number of battles based, the longer you are in the dungeon, the tougher the enemies get.

To balance that, we make it so that when you leave the dungeon, the monsters all forget about you.

This behaves similarly to a genetic algorithm, where the fitness test is fighting you. And the sharing of information is similar to swapping genes, but, in this instance, its a method that can deal with unexpected shifts of the problem space. Whereas, in genetic algorithms, the genes have to fit the problem well, and take a while to adapt. By data-mining observed actions, the enemies can adapt faster than with a genetic algorithm.

Another idea I considered was enemy genes, where each enemy you manage to kill does not reproduce, and thus, the more you fight in an area, the tougher the monsters. However, without a very flexible genetic system(IE, one too powerful to adequately simulate) this would only lead to monsters with stronger and stronger genes. Also, the algorithm would have to be tweaked to show results quicker than most GA’s.

Next post: RPG AI Part II