News:

Welcome to Hero Mages!

Main Menu

Introducing the AI Thought Process

Started by Ross Przybylski, September 26, 2010, 02:59:54 AM MT

Ross Przybylski

Dear Hero Mages Fans,

What a productive week for Hero Mages development!  For those who weren't already aware, I took a week vacation specifically to dedicate myself to the development of Hero Mage's upcoming computer based player feature.  Up until now, the prototype AI was functionally on a purely random tactical basis- using the list of its available actions and seeking out legal targets in random order.  While this worked surprisingly effective due to the tactical intelligence abilities of individual unit actions, it resulted in odd behavior where the AI was conflicted in the use of its resources.  For example, a Warrior might move towards an enemy to attack, and then, on the next command, sprint back to base to protect another unit.

What my AI was lacking was a decision making tree for organizing and processing its available choices.  And so, I created one.  For those interested, check out the attached JPEG and follow along- I’ve kept the jargon in layman’s terms.  Each box on the flow chart represents an AI project with its own decision making process and tasks to carry out.  A summary of these projects follows below.  Questions and feedback are welcome!

Protect Mage
This project examines potential threats of the mage and weighs the risk of exposing it to the enemy to cast spells.  Depending on how the mage might be used during the turn, the AI take measures to protect it.  Such measures might include strong protective buffs (Shield), sprinting back to safety at the end of turn after casting, etc.

Scry for Cards
This project is assigned whenever the AI is looking for a particular kind of action but fails to find it in its current arsenal.  The AI will add any scrying actions (Transmute, Scribe) such to find the cards it needs for the task it’s trying to perform.

Heal Units
When the initial question “Do any of our units need healing?” is asked, the AI is also checking to see if any units are disrupted or defeated.  Healing actions are thus considering actual healing actions plus restorative actions and revive.  Available healing actions are sorted by priority based on the type of healing needed (group healing, single unit, etc.) and then assigned to the AI’s command cycle.  Since healing spells have built-in intelligence to know when it’s wise to actually use them (for example, don’t waste a group heal on a single unit with only 1 damage) we ensure that only the most efficient healing actions are used and the rest are conserved.

Summon Creatures
I have prioritized summoning creatures over direct engagement with the idea that it is best practice to bolster our forces with units over direct assault as this increases our threat in the long term and also reduces threats from the enemies (as creatures will often draw enemy fire away from heroes).  This project works by adding all available creature actions to the action command cycle, prioritizing by a combination of inherent creature value and greatest threat to enemy calculations.

Energize
The AI creates a command cycle to use any actions that would generate additional mana (Mana Surge, Meditate) so that we have the greatest supply of resources to work with in preparing our assault on the enemy.  I have purposively ordered this process after Summon Creatures under the notion that I believe it is best to use surplus mana for direct engagement actions vs. reinforcing the team with additional units which could potentially clog up the AI’s assault on the enemy.

Cleanse
The cleanse project is where the AI uses actions to remove enemy effects that are potentially giving them an advantage so we can accurately assess targeting priority in the next phase.  Currently, the only action that falls in this category is Dispel Magic, but I’ve labeled the project type as Cleanse to keep it open for future actions that may have similar purposes.

Engage Enemies
If the AI has been assigned this project it means that it has engage actions available to use this turn and it has deemed that the benefit of using these actions outweighs the potential risks of retaliation.

Engage actions are actions used directly against units to damage or disrupt them (attacks and disruptive actions) and offensive buff spells that contribute damage to attacks we can make in the current turn.  These actions are prioritized by the best threat to cost ratio and are added to the AI’s action command cycle one at a time until the AI determines it has no further resources left.  After all engage actions our resources allow have been assigned, the chosen actions are sorted by their proper use-order (ensuring such things as buffs are used before attacking and stuns are performed last) and carried out.  After this cycle is complete, the AI will attempt to engage again (as new actions may be available such as Warrior’s second attack or Barbarian’s frenzy attack).

Retreat and Recover
This project is triggered when the AI either lacks enough engage actions for an assault or has determined that an assault is not worth the risk given the health of its units.  In either case, there are two primary goals here- take defensive positions to avoid enemy retaliation and gather more cards.  If our hand is already full of spells, discard our lowest priority card for each surplus mana we have that would bring our hand back to full.

Manipulate Enemy
While manipulation has many potential uses in the game of Hero Mages (particularly in FFA games), the AI will be using it to expedite its ability to engage the enemy in the current turn.  A prime example would be the inability to attack a unit because it’s short of movement range.  We’d start by adding a single manipulation spell (prioritized by a fixed order) such as Mind Control and use this action only if it would bring the enemy in range to attack.

Rush the Enemy
The AI is assigned this project when it has determined its strength greatly overwhelms the enemy.  Rather than take its time to move cautiously through cover, the AI chooses to take the shortest path towards the enemy allowing sprint moves and opting for spaces adjacent and surrounding enemy units so as to trap and finish them off  before they can escape or recover.

Support and Advance
The goal of this project is to advance closer to the enemy in a cautious fashion, choosing spaces closer to the enemy with the least possible threat.  We can also use support actions here to prepare us for engagement next turn, such as Bloom, Armor of Aegis, and Archon’s Might â€" but the caveat is to not allow movement when calculating targeting for these actions.  We want to reserve movement and sprint for positioning (as this is more important here). So, first we run a cycle of support actions from current location.  Then, we initiate an action cycle allowing sprint based on each unit’s primary action (for most units, this is attack range, but for mages, this is line of sight).  This should put each unit in the safest, closest position.  Finally, we run another cycle of support actions (again, disabling movement) in the event that new positions opened up opportunities to use these actions.

Cast Bonus Spell
This project is used to catch any additional spells the AI could have used that slipped through the cracks of the initial process flow.

Use Bonus Action
This project is used to catch any bonus actions that the AI could have used that slipped through the cracks of the initial process flow.

Use Reserve Actions
This project is used to perform any actions the AI has reserved resources to perform at the end of its turn.  Reserve actions will typically be generated in the Protect Mage project (ensuring we save enough movement to sprint back into cover, cast imprison, etc.). 
Manager of D20Studios, LLC

the stoat

hey Ross. Launch this sucker! While there are so few players to make up teams, even a weak ai is way better than nothing as it can be used to make up the numbers.  Eg. even if the AI is doing crazy things, we can put it with an advanced player (dies) to make the game interesting.  It'll also give 2 people the option of playing a 4 player game to mix things up. Drendal firepit is getting a real workout lately :)

Ross Przybylski

I will aim to release an update to Hero Mages client in the next week or two with what I will call the "Prototype" or "Beta AI" (working but with high potential for bugs) once I've finished programming the AI projects listed here and developed a way to have dropped hosts correctly transfer AI hosting duties to the new game host.  This will give players a chance to begin experiencing how the AI will function, identify bugs, and offer suggestions to improve the script.  I'll consider the AI "technically complete" when it functions without bugs, though in truth, I intend to evolve it indefinately in future updates with better intelligence and customization features.
Manager of D20Studios, LLC

FrontaLobotomy

All sounds good. I look forward to seeing it in action.

Ross Przybylski

My brother and I had the first-ever team battles against the ever-more-intelligent AI tonight.  While I still have plenty of targeting prioritizing algorithms to complete as well as a number of the AI projects (Protect Mage being the most crucial one missing), the AI is already proving itself to be an exceptionally capable player.  Playing 1on1, an experienced player can defeat it fairly easy, but as a team of 2 humans vs 2 computers, it's an entirely different story.  The AI challenged us in ways that were totally unexpected and won about half of the matches we played.

For the record, I should also note that this new AI no longer uses the infinite mana cheat that I demoed at GenCon this year.  It won games fair and square.
Manager of D20Studios, LLC

FrontaLobotomy

Sick win %. I do like that the AI seems(from the little info we have) to play well "together".  Have you tried larger games, or  ffa (non-team situations)?

Kin


Ross Przybylski

Our latest test run of the AI went as followed:

In a 3 human vs 3 bots, the bots the first two games, we won the second two.

In our 4vs4 bots on Valley of War, what seemed like a sure win for the humans turned into a slaughter when the bots came back with a group heal and blizzard combo.

In our FFA 4 humans and 4 bot games, it came down to 1 human paladin vs 1 bot paladin.  The bots took the win for having the patience to wait out the dice slugging.  But the most notable AI move that took everyone by surprise was yellow bot's FM/Bard/Barbarian combo.  In a single turn, the Bard double buffed the barbarian and sent him on a rampage eliminating two mages and a soul reaver.  After we overcame the sheer terror, we used our remaining psionists to "heard" the Barbarian like an animal towards other enemy targets and had quite the laugh about it until he circled his way back around after eliminating said foes and proceeded to chop us to bits.

Oh, we also did a 2v2v2v2 where each player had a bot teammate which was also quite entertaining.  I can't wait to share the new update with everyone on Monday!
Manager of D20Studios, LLC

kefkam

congratulations. i think this is a very very very difficult to develop IA.

this is not like counter-strike, (move random, and shot with speed:x and accurate:x)

nor chess: each piece have a value, analice movements combinations and compare values.

great idea that of "rush the enemy". i didnt thought the enemy would put himself at risk for nothing in this turn, but taking closer for nexts turns so he can attack.


Gorstak

hey ross great jobe,i have one question:is discarding cards removd  i dont see u manchnd it somwere?

Ross Przybylski

Quote from: Gorstak on October 11, 2010, 10:44:18 AM MT
hey ross great jobe,i have one question:is discarding cards removd  i dont see u manchnd it somwere?

Thanks for letting me know, Gorstak.  This was a bug.  I have fixed it so discarding now works again.  Enjoy!
Manager of D20Studios, LLC