Game Evaluation Form Legend
The game evaluation form is meant to provide a concise
description of a game and its behavior authoring and scenario generation
capabilities. The information in the
form was prepared by the BBN game evaluators and represents their personal
experiences playing the game, and reading through official and fan-created
material. The evaluation form consists
of twelve sections.
first section provides general information about the game, including what type
of game it is and when it was released.
- Name: The full title of the game
- Date Published: The
year that this game was released
- Platform/OS: Windows,
Linux, Unix, or Macintosh.
- Genre: While some games could rightfully be
places in multiple categories, we used the general “genre” and associated
“sub-genre” categories as they are generally used in the gaming
community: Strategy (player
controls multiple units, game play focuses on decision making more than
reflexes), Action (player controls one avatar, and game play focuses on
hand-eye coordination or reflexes), or Adventure (game play focuses on
- Subgenre: Role Playing Game, Real Time Strategy,
Turn Based Strategy, First Person
Shooter, God Game, Robot Building,
- Publisher: The company
responsible for publishing the game
- Developer: The company
responsible for game development
- Licensing: Commercial or Free
- Languages: When known, the language the game was
- Domain: Fantasy (alternate world, generally
Tolkien-esque), Futuristic (space travel, e.g. Star Wars), Historical (loosely based on our
Historical-Military (based on historical wars/battles) Modern
(loosely based in current times or the near future), Modern-Military (based on current military
units and capabilities)
- No: The game does not try to model
anything in the real world)
- Yes: The game attempts to realistically
model objects/events in the real world
- NA: Fantasty and Futuristic games have no
concept of realism
- Game Engine: Usually
“proprietary”, meaning the engine was developed in-house and not available
for licensing. If a game engine is
available for external use, or an external game engine was used, the name
of that engine will go here.
This section gives a brief
description of the game and a few sentences about the behavior and scenario
authoring capabilities. Most of the
comments here are the opinions and general impressions of the game evaluators.
Here we give a few opinions on the
state of the gaming community in relation to this game. A healthy, active, gaming community will
generally have numerous recently updated fan web sites devoted to the game, an
updated official web site, and activity on internet message boards related to
the game. We also give a general
impression on the state of the modding community, the gamers who are actually
using the tools described in the rest of the evaluation to author scenarios and
behaviors. Since it is difficult to
assess the size and activity of a loosely connected internet based community,
we opted to give the general impressions and opinions of the game evaluators
instead of attempting to construct a rating system.
1.4 Team Play
Most games can be played in either
single player mode (one human player competing against AI controlled
opponents), or in multiplayer mode, with some combination of human and computer
controlled players. This section gives
information on the multiplayer mode of play.
- Networked: Can the game be played over a computer
- No: Single player only games or games that
require all players to be at the same computer
- LAN/Internet: Game can be played over a LAN or over
the Internet, usually with one player’s computer functioning as the
- Free Server: Game can
be played over the internet on a persistant server
- Number of Players: The maximum number of human players
that can be playing in the same game world. Generally, some of those human players can be replaced by
computer controlled entities.
- Cooperative Play: Can the players cooperate with each
other? Generally, players can make
deals or form temporary alliances, however, a “no” here means that there
is only one winning player, whereas a “yes” means a multiple player team
- Teammate Types: For a cooperative game, can a players
team consist of other humans, other AI controlled players (bots), or
both? These are Yes/No slots.
Methods: How can players
communicate with each other.
Generally, all games have a “Chat” style interface, where players
communicate by typing text that gets sent to all members of the team, or
to everyone playing. “Third Party
Voice” is another method where players use third party software to enable
voice communications. While this
could be used in any game, it is listed here only if the evaluators found
examples of players actually using it.
“Pre-defined messages” is the last option, where players can select
from a limited list of messages to communicate.
- Player Role: None (All players are homogenous), Commander/Subordinate (the players are
part of a hierarchical group, where the commander has some measure of
control over his subordinates), Classes (The players can undertake
different roles with different capabilities, such as scout or pilot)
- W/L: win-loss record
- Composite “Comp”: Some arbitrary function of game
variables can reduce the player’s performance to a numerical score
- Ladder: Players are ranked in relation to each
other on some official scoreboard
1.5 Game AI
Here we describe some of the
characteristics of the computer controlled entities in the game. Many games do not provide detailed
information about their AI algorithms, since such information can be considered
a trade secret. The information
presented here is what the game evaluator could discern by interacting with the
AI entities in the game, reading official documentation, and reading fan
- Usage Roles: What role the AI entities play in the
game. Opponent AI (competes
against the player, and can “win” the game by beating all other
opponents), Henchman AI (AI
controlled teammate that is fairly autonomous but can be directed by a
human player), Unit AI (entity directly controlled by the player that may
have some basic autonomous routines)
- Techniques. When known, the general type of AI
algorithms used. A question mark
is appended if the evaluator is unsure and making a guess. The types we differentiate between
are: Finite State Machine, Rule
Based (a set of rules (action/reaction pairs) govern behavior),
Reinforcement Learning, Scripted (the entity follows a static set of
pre-coded commands), Priority Queues
(a variation of rule based, the AI has a queue of options whose
priorities can be modified by game events, and executes the highest
- Code Accessible: Is the source code or rule set or
priority queue data accessible?
- Level of Autonomy: Low (the AI is fully scripted or will
do very little without player interaction) or High (the AI can win the
game if the player plays poorly)
- Strength: A vague impression on how “good” the AI
is. A “good” AI can pose a
challenge to an experienced player without resorting to “cheating”. An AI is considered to be “cheating”
when it plays the game by a different set of rules so the player is
artificially handicapped. This is,
of course, the opinion of the evaluator.
1.6 Content and Scenario Authoring
One type of game modification done
by players is scenario authoring. A
scenario author will set up the initial situation and state of the game world
by drawing a game map and populating it with game objects and AI controlled
entities. Scenario authors sometimes
spend large effort drawing new graphics and sometimes even change the entire
domain of the game, from Fantasy to World War II, for example. Also some games allow a scenario author to
modify the rules of the game.
- Media File
Construction/Manipulation: Can a
scenario author edit the graphics of the game? Generally, this is possible with the use of third party
programs (such as 3D Studio) as long as the game uses a standard file
- Map/Level Construction
Methods: GUI (The game provides a
graphical user interface for map creation), File Editing (the author can
edit a text file that contains scenario information with a standard text
editor), or Bit Code Hacking (the author can use a hex editor to edit the
binary data associated with the game).
While, a scenario author could use bit code hacking for any game,
we only list it as a “yes” here if we could find examples of it actually
being done for that game.
- Rule Manipulation: Can the player change the rules of the
- Rule On/Off: “Some” means the author access to a
handful of game rules and can turn then on or off. “Yes” means the author has access to a
large set of game rules that can drastically affect game play. “No” means the author would have to
use bit code hacking to change any game rules.
Editing: “Yes” if some numerical
parameters governing the game engine can be set by the author
- Event Triggers: Can the author create action/reaction
triggers that can fire during game execution and dynamically modify the
state of the game world?
- Tools: The type of tools available to support
scenario authoring. These values
will either be Yes, No, or the name of the specific tool, if it is well
- First Party: Tools
released by the game publisher
- Third Party: Tools
created by the modding community
- Graphical: Is there a graphical user interface
available (first party or third), or is all editing done via a text
- Doc: Is there documentation about scenario
authoring: None, Little (very
little information beyond code comments is available) Some (text
documentation is available but incomplete), or Yes (full documentation
complete with tutorials and examples)
1.7 Behavior Authoring
This section deals with the
modification of the behaviors of AI entities.
- External API: Can an external program communicate
with the game and modify AI behaviors while the game is running? This is a “yes” if an API and/or a
working example exists.
- Core Behavior Source
Code Available: Can you view the
source code and/or data used for the core functions of the AI?
- Type of Authoring: This category is split into two
sections, First party and Third party.
The First party section lists the types of authoring made available
directly by the developers of the game, and the third party section lists
what additional types of authoring are available using tools created by
fans. The types of authoring we define
- Programming (Prog):
Authoring using a programming language, such as C,
- Visual Programming
(Vis Prog): Using a visual programming layer ontop of a programming
language or scripting language
- Scripting (Script):
Authoring using a scripting language specific to the game
- Parameter Editing
(Param): Changing an “aggressive” level for the AI entity, or adding a
- Rule Editing (Rule):
Adding/changing action/reaction rules
- Genetic Algorithm (GA)
- Game Play (Game):
Behavior authoring occurs as a fundamental component of game play, not
outside the game.
- GUI For Parameter
Editing: Is there a graphical
interface that lets a behavior author edit parameters or rules?
- Scripting: This section covers the tools available
for games that use Source Code or a Scripting Language. A “Yes” means that tool or concept is
available and directly supported
by the game, a “3rd Party” means such a tool is
available using third party software.
- Event Trigger: Direct support for event triggers (Action/Reaction)
pairs in the game engine.
- Event/Signal Passing:
Support for AI entities to communicate with each other
- Script Library: A library of behaviors exists where
authors can grab scripts and insert them into their game.
- Dialog Trees: Direct support for conversation trees,
where the player can interact with a AI entity by choosing a sentence to
speak from a list of choices.
- Doc: Is there documentation about behavior
authoring: None, Little (very
little information beyond code comments is available) Some (text
documentation/API function documentation is available but incomplete), or
Yes (full documentation complete with tutorials and examples)
- Dynamic Authoring: Can a participant edit AI behaviors
while a game is running? When this
feature is available, the “participant” is usually a game master, or
trainer, not a player.
- Yes: There is a
specific interface for an operator to modify behaviors online, usually
even allowing the operator to take direct control of an AI creature.
- Console: The game server has a text input based
console that an operator can use to input a set of commands. Some of these commands can change AI
behaviors at a high level, by making them hostile/friendly, making them
invincible, or perhaps setting a state variable.
- Console +
Plugins: The console commands are
extensible with the use of plugin code.
New commands can be created by a software developer that allow the
server operator to edit pretty much anything.
- Spectator View: Can a non-player connect to the game
and watch the action while the game is running?
- Replay: After the game is over, can a player
watch a replay of the action, possibly from a different point-of-view?
An expansion is an official addition
to the game, released by the game publisher, usually for a fee. A modification is a scenario or behavior
script released by a fan using the tools described in the above sections. In this section, we give a short sentence
description of the types of expansions and modifications available.
1.10 Current Military Applications
Is this game used by the US Military
for training purposes?
1.11 Information Sources
this section we provide a set of web links to sites that provide more
information about the game, including the official developer/publisher sites
and fan sites with modifications available for download.