Artificial Intelligence Development

[Seravy]

Please clarify your position if you are interested in developing strong AI for this game or not.

I want AI that's as strong as reasonably possible.

Reasonable here meaning :

-Only changes that improve performance significantly for a low or medium time investment - There is only one developer and this is a hobby project, not a full time job. I was ok with working 12+ hours a day last year to make the game actually happen but I'm definitely not ok with spending that much time now that it's already good enough to be fun to play. I plan to keep improving it at least for the next year or two, maybe more, but only a few days per week like how I did with the DOS version.
-Only changes that are guaranteed to be an improvement within the timeframe between two updates. This is an already released, paid product so we can't do random experiments on it hoping it will get better in 10 more updates. That's something we could have done during alpha or while the DOS version was a free download but not anymore.
-Only changes that don't fundamentally change core code and systems that would require large scale, very long testing sessions and delaying updates by months. Again, this is no longer a beta, it's a released game already at version 1.2.0.
(Replacing the entire core combat pathfinder would violate this requirement. There is a huge chance for error and an already very high level of complexity with many special cases. It's guaranteed to add new, possibly serious, bugs. At best we can add minor changes like the one I proposed.)
-In general, I only start working on a new feature/change of existing one, if the person suggesting it can prove the benefits and that it actually works and is an improvement. I have been doing the work that way the past 6 years and it's the main reason why the game works. Safety first, zero risks taken.

For first draft AI can use a simple metric such as total cost of its army divided by total cost of the opponent’s army at that time point.

The AI already used this metric in the DOS version. It compared the two armies and depending on the ratio, it used Flee if there was a hero involved.
It was too inaccurate. I'd like to hear what kind of refinements would solve this that are simple enough to implement in a reasonable amount of time but good enough to make the AI perceived better by players than an AI that never flees.

I would like to add that I believe the AI not fleeing is much less likely to be considered a flaw by the human player than the AI fleeing at the wrong time once out of 50 cases. Failing to flee when we should is much more acceptable than fleeing when we shouldn't and this is a reasonable expectation - fleeing at a wrong time comes with a relevant loss (an unit that could have won) but not fleeing when you can only wastes something irrelevant (an army that's already very outnumbered and thus can't make a difference in the big picture most of the time, especially against an expert player).

This is why I have disabled AI fleeing in the windows version. There is very little benefit we can gain from the AI fleeing that comes at high risks.
If anything, an AI that is fleeing will be perceived as an annoyance for most casual players, as they have to fight armies they can easily kill multiple times. So we will likely need to also limit it to high difficulty, which makes mistakes even less acceptable.

Later we gradually improve on this by adding 1 refinement after another based on data. That is we play it and even better we let AI1 play vs AI2 many battles and collect data.

AI vs AI can be done for overland moves. It can't be done for combat. Two AI players can never fight each other in regular combat. The game is not coded to support that because it cannot happen normally. It is assumed that one player in the combat is always the human player.

This approach also only works for simple changes we can add, run the AI games, look at the data and then decide to keep or discard the feature. For anything that requires more than a few lines of code/minutes of work, we can't use that as I need to know whether I want to do that work or not in advance.

The typical run time of a fully AI played game is generally around 2-8 hours depending on map size and number of players. So we can't do more than a few of them to test a feature and even that might be pushing it a bit too far. I usually do only one of these on smaller updates, and 2-3 only on major updates such as 1.2.

[WhiteMage]

OK. I understand all this. Thanks for explaining. You have my high respect for this huge amount of quality work you have done on this project.
On the other hand, with all these constraints, strong AI development will not be possible.
Nevertheless, I will try to work within these boundaries you outlined. There are a few things we can still do to improve the AI.

“The AI already used this metric in the DOS version. It compared the two armies and depending on the ratio, it used Flee if there was a hero involved.
It was too inaccurate. I'd like to hear what kind of refinements would solve this that are simple enough to implement in a reasonable amount of time but good enough to make the AI perceived better by players than an AI that never flees.”

It was inaccurate because it was too simple. I think it needs a lot of work to get it to near optimal state.
Strong AI development has nothing to do with what players perceive. It is all about what the AI objectively does. Wins, loses, etc.

“I would like to add that I believe the AI not fleeing is much less likely to be considered a flaw by the human player than the AI fleeing at the wrong time once out of 50 cases. Failing to flee when we should is much more acceptable than fleeing when we shouldn't and this is a reasonable expectation - fleeing at a wrong time comes with a relevant loss (an unit that could have won) but not fleeing when you can only wastes something irrelevant (an army that's already very outnumbered and thus can't make a difference in the big picture most of the time, especially against an expert player).

This is why I have disabled AI fleeing in the windows version. There is very little benefit we can gain from the AI fleeing that comes at high risks.
If anything, an AI that is fleeing will be perceived as an annoyance for most casual players, as they have to fight armies they can easily kill multiple times. So we will likely need to also limit it to high difficulty, which makes mistakes even less acceptable.”

Again, strong AI development has nothing to do with what player perceive. It is about how the AI objectively performs.
I agree that AI fleeing is a low priority objective to work on with a lot of work involved.

“AI vs AI can be done for overland moves. It can't be done for combat. Two AI players can never fight each other in regular combat. The game is not coded to support that because it cannot happen normally. It is assumed that one player in the combat is always the human player.

This approach also only works for simple changes we can add, run the AI games, look at the data and then decide to keep or discard the feature. For anything that requires more than a few lines of code/minutes of work, we can't use that as I need to know whether I want to do that work or not in advance.

The typical run time of a fully AI played game is generally around 2-8 hours depending on map size and number of players. So we can't do more than a few of them to test a feature and even that might be pushing it a bit too far. I usually do only one of these on smaller updates, and 2-3 only on major updates such as 1.2.”

Unfortunate. This severely limits AI development.

Let’s work on the AI bugs first. Then work on the little work, guaranteed high outcome aspects. Nevertheless, as I stated earlier AI development is generally a lot of work and is a trial and error thing. So giving guarantees in AI development is difficult. Especially, since you already limited multiple areas of AI development. Essentially the approaches that have the highest probability and highest improvement to work are not allowed by your rules.

Would you be interested in setting up an easy to hook up external AI interface, so people can code their own AIs and hook those up to test without always asking you to spend time? Independent AI code from the rest of the code. This way a whole population of AIs can be independently developed by external programmers, similar to how C-Evo solved this problem. How much work and time would it take to set it up?

[Anskiy]

I personally would welcome exposing all the AI scripts for modding, that seems like a good way to let people experiment with the AI if you don't have the time to devote to it.

[Cruel100]

Would you be interested in setting up an easy to hook up external AI interface, so people can code their own AIs and hook those up to test without always asking you to spend time? Independent AI code from the rest of the code. This way a whole population of AIs can be independently developed by external programmers, similar to how C-Evo solved this problem. How much work and time would it take to set it up?

This sounds like a great idea.

[Seravy]

AI scripts are planned for most spellcasting related decisions, assuming we get far enough in spell modding for that.
City production scripts already exist in 1.2.

Movement related scripts are not planned, as that's a bottleneck that would make AI turn times way too slow.

[Cruel100]

I would like a spell blast to have more weight on the AI, especially on large maps versus spells like final wave or great unsommuning. That's because it can eliminate over a hundred units, strong mages would no doubt give it more priority.

[Seravy]

More? But it's literally the only spell the AI has like 99% chance to cast whenever they see a spell that's on their target list.
If they don't spell blast it, the spell either isn't on the list (unlikely) or other conditions are not met (no hostility or no contact with the target, or no detect magic to see the spell being cast), if neither of those apply, it might be a bug.

Also, the AI can only spell blast if they are ready to cast a spell. So if they cast a spell that takes multiple turns, there will be turns when they can't. They won't interrupt their ongoing spell for spell blasting.

[Seravy]

2. Couldn't reproduce, the AI never used Animate Ammo.
They never targeted a unit affected by Shatter though. That probably was caused by the bug already fixed during the work on 1.2.

I noticed the Halberdiers didn't shoot though, just walked forward. I see the magicians have resist elements cast on them, so maybe this is correct.
...no I think it's not. When the melee attack cannot hit the same turn, a much larger priority difference should be required to pick melee over ranged.
I tested and the Halberdiers were able to kill one and a half units worth of magicians by shooting, while if they walked forward, they killed nothing.

7. This is extremely unusual. I would need more information on how this game state came to be.
...I have a pretty good guess. I see some Shadow Demons on your plane. I think those were the fortress defenders. The plane shift function does not check if the unit shifting planes is garrisoning a city even though it should. Bug, will be fixed in the next update.

8. The AI already cannot target noncorporeal or flying units with Water Walking.
However, they can target water walking units with Flying or Wraith Form as they grant relevant additional combat abilities : being impossible to attack by melee, or having weapon immunity and the ability to bypass weapon immunity.
Less than ideal but this is not a bug if the water walking spell was cast first.
Wraith Form already has reduced priority to cast on an already flying or water walking unit. Maybe the reduction could be higher.
Flight has no such reduction and probably shouldn't have one, as it's a relevant combat advantage that should always be cast on the best unit(s) available.

11.

Attacking unit Warlocks considering target Warlocks
for ranged attack
Expected damage =61/10, Expected dead figures =3
Ranged target bonus =45%
Enemy ranged attack, priority +5%
Total priority multipler :150, base priority =61
Total priority =9150
Attacking unit Warlocks considering target Phantom Warrior
for ranged attack
Expected damage =132/10, Expected dead figures =7
This unit will die from this attack, bonus =40
Target is a combat summon, priority -30%
Enemy ranged attack, priority +20%
Total priority multipler :90, base priority =172
Total priority =15480

The obvious problem I see here is no handling of overkill damage. Damage dealt should likely be capped at the total health of the target+3 like in case of direct damage spells.
Even with that the Phantoms would receive the higher priority though, so maybe also raise the penalty for being a combat summon?

(btw "enemy ranged attack" is mislabeled and should read as "enemy melee attack". I will also adjust this amount to be calculated the same way as the ranged bonus but half the amount.)

14. I don't see anything that would make the AI prioritize the same target higher that was chosen the previous turn. I remember that used to be a feature in the DOS version but I suspect it was removed for some reason. We will need to consider if and how much priority change this should cause.

[WhiteMage]

“2. Couldn't reproduce, the AI never used Animate Ammo.”
I loaded that save again. This time into the latest Debug version you sent me. AI used Animate Ammo right on my first try. But the AI bug is not that it used Animate Ammo. It was that it kept targeting the already crippled unit. You understand all this, I guess. And yes, I started to use the log files to look into this stuff deeper.

“7. This is extremely unusual. I would need more information on how this game state came to be.”
It was immediately the next turn after I broke through the first tower. AI immediately swarmed my plane with Shadow Demons. And I promptly took its Fortress the next turn without losses. Your hypothesis seems correct why. I can dig up the Autosaves if needed, but I think you already understand. Somehow swarming my plane with Shadow Demons was more important for the AI than defending the capital that was in plain sight and within range to the AI that I could attack next turn.

“8. The AI already cannot target noncorporeal or flying units with Water Walking.
However, they can target water walking units with Flying or Wraith Form as they grant relevant additional combat abilities : being impossible to attack by melee, or having weapon immunity and the ability to bypass weapon immunity.
Less than ideal but this is not a bug if the water walking spell was cast first.
Wraith Form already has reduced priority to cast on an already flying or water walking unit. Maybe the reduction could be higher.
Flight has no such reduction and probably shouldn't have one, as it's a relevant combat advantage that should always be cast on the best unit(s) available.”
I understand all this. But IIRC the AI had many same units that had no spell on them. So maybe prioritize on those first? Rather than having soft duplicate spells?

AI11. was about overland map attacking decision. Not about battle AI. AI already has templates how to combine its own army, where to attack, with what to attack, etc. Attacking 9 warlocks is tough nut, but there are templates how to do it well.

AI12. In your AI script you quote, it shows Expected dead figures =7 for the Phantom Warriors. But in my example, the unit only had 1 figure. So it is even more extreme AI bug. Not to mention the phantom warriors were not in range to be attacked by.

“Even with that the Phantoms would receive the higher priority though, so maybe also raise the penalty for being a combat summon?”
You can do that too. But I think Warlock has low HP, but can do high damage, ranged attack, Doom Bolt, and expensive to build. 160 hammers. Phantom Warriors 14 mana and in battle only. You can use all these heuristics in choosing target. So it is easy and important to kill them ASAP. They need high value to attack them.

“14. I don't see anything that would make the AI prioritize the same target higher that was chosen the previous turn. I remember that used to be a feature in the DOS version but I suspect it was removed for some reason. We will need to consider if and how much priority change this should cause.”
I think the heuristic here is trying to corner a unit to the closest corner to the AI’s unit.
May I ask? Are you using the old AI codes from MoM and change them as needed? Or did you develop new ones from scratch?

[Seravy]

11. this was actually for 12. Typo.
The log is for my test, since you didn't post yours. My phantom warrior had all 7 figures.

14. Both.
For melee attacks specifically, the AI will walk towards the target on the shortest path found, until the target is reached for attacking or the unit runs out of movement. The path taken prefers to end on the horizontally or vertically adjacent tile instead of the diagonal one if the costs are equal, as that is usually more effective for cornering units.