Allegro.cc - Online Community

Allegro.cc Forums » Game Design & Concepts » Self-Learning AI for FPS tactics

This thread is locked; no one can reply to it. rss feed Print
Self-Learning AI for FPS tactics
Dario ff
Member #10,065
August 2008
avatar

I´m not wishing to do the system because I´m not actualy making a FPS or anything that may require it now.

But I just wanted to leave my thoughts about my project of True AI for a game.

//////
But before explaining my theory, I would like to know if there are actually some games out there with Learning AI on their NPCs. I would like to try them out BTW.
//////

Ok, my idea is pretty simple, it´s basically a trial and error with tactics ranking and some linear thinking. Let the NPC use a weapon and try different approaches with it. The NPC automatically rates the tactic with an integer value, depending on how much damage it actually did to an enemy, and if it got damaged itself. Also, he should watch out for allies. Then it should take some time to classify tactics(burst, Single, Area Effects).

Of course, he should add some conditions to using different tactics depending on its enemy condition. This might be something nice like for example, the NPC actually does not know if his enemy is immune to some type of damage, but actually trying it he might learn the consequences.

Basically, you let it play two hours and it might be the smartest bot ever ::) or maybe not :-X

I´m not going to do it for any game by now, but I though it would be pretty cool for a FPS game.

TranslatorHack 2010, a human translation chain in a.cc.
My games: [GiftCraft] - [Blocky Rhythm[SH2011]] - [Elven Revolution] - [Dune Smasher!]

Paul whoknows
Member #5,081
September 2004
avatar

I am not sure, but after years of playing Warcraft III TFT against the CPU in LAN mode makes me think that there must be some sort of learning AI there, because it is just too damn good.

____

"The unlimited potential has been replaced by the concrete reality of what I programmed today." - Jordan Mechner.

Kibiz0r
Member #6,203
September 2005
avatar

In my mind there are a few different things that go into making AI believable:

  • It has to have some inherent smarts

  • It has to retain information

  • It has to extrapolate that information

  • It has to have some "play" to it -- experiment every once in a while

  • It has to, occasionally, be stupid -- not very often, but it needs to make some mistakes

The first three are purely mechanical, the last two are a matter of tuning. I think these are easier to do in an RTS, but I think you can accomplish the same things in an FPS.

I think the things you need to figure out are:

  • Who is gathering the information

  • Who is receiving the information

  • What they already know about the enemy and environment

  • What information they're gathering

  • What kind of lifetime the information has

  • What actions depend on what kind of gathered information

For instance, you could have a group-based AI where every individual agent reports what it finds (direct encounters, ammo/health pickups, spent rounds, sounds, footprints...) to the group leader, who reacts by issuing orders and stores the information in an influence map for pathfinding later. That information has an exponential decay so that old information becomes irrelevant and eventually drops off.

Evert
Member #794
November 2000
avatar

Ok, my idea is pretty simple, it´s basically a trial and error with tactics ranking and some linear thinking. Let the NPC use a weapon and try different approaches with it. The NPC automatically rates the tactic with an integer value, depending on how much damage it actually did to an enemy, and if it got damaged itself. Also, he should watch out for allies. Then it should take some time to classify tactics(burst, Single, Area Effects).

Of course, he should add some conditions to using different tactics depending on its enemy condition. This might be something nice like for example, the NPC actually does not know if his enemy is immune to some type of damage, but actually trying it he might learn the consequences.

Basically, you let it play two hours and it might be the smartest bot ever ::) or maybe not :-X

Two guys I studied with did this as a project for their thesis a couple of years ago. Their research was about the question whether a learning rule for a neural net that had been shown to work well for a static situation would allow the neural net to learn effectively in a reactive environment. They ran a large number of simulations (using a modified version of Quake, I think) where the scoring system for the AI was based on how often it killed another bot versus how often it got killed itself. These simulations then ran for several days.
Several different strategies emerged, which was quite interesting to see, and it turned out that they had to adjust the learning rule for the neural net for it to perform optimally. Even their most well-trained bot would have been no match for even a beginning human player though. I don't know if their work ever got published, but I suspect not (they both quit science afterwards).

So bottom line, if you want to try this, by all means do, but be aware that it's by no means simple. Don't expect the AI to develop behavior where it can effectively challenge a human player (unless you pre-program certain behaviors, but then it's not training on its own) and don't expect training the AI to only take a few hours.

Thomas Fjellstrom
Member #476
June 2000
avatar

Evert said:

Don't expect the AI to develop behavior where it can effectively challenge a human player (unless you pre-program certain behaviors, but then it's not training on its own) and don't expect training the AI to only take a few hours.

Thats like saying "Don't expect a human to learn how to play a FPS a few hours after its born". Takes our neural net years to develop the capacity to do what it does. I can only imagine it'd take a simulated neural net as long or longer (using current technology at least).

--
Thomas Fjellstrom - [website] - [email] - [Allegro Wiki] - [Allegro TODO]
"If you can't think of a better solution, don't try to make a better solution." -- weapon_S
"The less evidence we have for what we believe is certain, the more violently we defend beliefs against those who don't agree" -- https://twitter.com/neiltyson/status/592870205409353730

Evert
Member #794
November 2000
avatar

Thats like saying "Don't expect a human to learn how to play a FPS a few hours after its born". Takes our neural net years to develop the capacity to do what it does.

While that analogy isn't necessarily accurate (some artificial neural nets can be trained for specific pattern recognition much more quickly than a human can) I won't go into that and just point out that that was a direct response to

Basically, you let it play two hours and it might be the smartest bot ever

Thomas Fjellstrom
Member #476
June 2000
avatar

Evert said:

While that analogy isn't necessarily accurate

Computers and the programs don't start out with incomplete core's either. If the computer the AI is running on and the AI itself had to grow and evolve like the human brain did, I think even in those special cases it'd take time ;)

Quote:

I won't go into that and just point out that that was a direct response to

Indeed. Theres no way you'll get a "learning computer" to learn what it needs in so short a time (imo). least not with current technology.

--
Thomas Fjellstrom - [website] - [email] - [Allegro Wiki] - [Allegro TODO]
"If you can't think of a better solution, don't try to make a better solution." -- weapon_S
"The less evidence we have for what we believe is certain, the more violently we defend beliefs against those who don't agree" -- https://twitter.com/neiltyson/status/592870205409353730

Kibiz0r
Member #6,203
September 2005
avatar

Human brains aren't a blank slate. We come with a default implementation and modify it through feedback. So there's no reason an AI has to be a blank slate either. And there's no reason you need to abstract the signals all the way down to a neural net. Figure out what's relevant to the game, what the basic building blocks of strategies are, and work from there.

This does seem like a fun problem, if you don't kill yourself trying to make it 100% "authentic." If I were to do it, I'd probably make it top-down 2D rather than FPS -- I find 2D a lot easier to demonstrate concepts in.

count
Member #5,401
January 2005

But before explaining my theory, I would like to know if there are actually some games out there with Learning AI on their NPCs. I would like to try them out BTW.

http://nerogame.org/

Dario ff
Member #10,065
August 2008
avatar

Thanks for the answers. They are all really helpful.

And I´m checking that game NERO by the way :)

Actually, this came out because y brother is starting in an AI course, and since I´m the programmer in my family, I started thinking about the interesting concept. His Project must be a game with the self-learning ability.

He actually wants to test first with a pokemon style game ;D Test which elements are effective aganist others, search for new training areas.

The FPS idea came out when I was playing Half-Life, and I was watching the soldiers and how they reacted.

Evert said:

don't expect training the AI to only take a few hours.

Making it learn faster might only be speeding up the game to ridicolous speeds and taking out any unnecesary processing out.

TranslatorHack 2010, a human translation chain in a.cc.
My games: [GiftCraft] - [Blocky Rhythm[SH2011]] - [Elven Revolution] - [Dune Smasher!]

Evert
Member #794
November 2000
avatar

Making it learn faster might only be speeding up the game to ridicolous speeds and taking out any unnecesary processing out.

What do you think those guys I talked about did?

Dario ff
Member #10,065
August 2008
avatar

Ok got it.
Anyways, I´m not the one doing the AI course, as I never got much of taste for self-managed computers :'( so much movies got me on my nerves :-X

Ok, thanks for the replies. They got me thinking. I´m going to play some NERO now ;) time to learn.

TranslatorHack 2010, a human translation chain in a.cc.
My games: [GiftCraft] - [Blocky Rhythm[SH2011]] - [Elven Revolution] - [Dune Smasher!]

weapon_S
Member #7,859
October 2006
avatar

CGamesPlay
Member #2,559
July 2002
avatar

Evert said:

(they both quit science afterwards).

;D

<segue />

I read an interesting article not long ago that brought up an interesting fact: the point of an AI isn't to perform the best at a game, it is to maximize the amount of fun the human players have. Just something to keep in mind.

--
Tomasu: Every time you read this: hugging!

Ryan Patterson - <http://cgamesplay.com/>

Ben Delacob
Member #6,141
August 2005
avatar

[quote]...[/quote]

Of course, he should add some conditions to using different tactics depending on its enemy condition. This might be something nice like for example, the NPC actually does not know if his enemy is immune to some type of damage, but actually trying it he might learn the consequences.

Usually with considerably smart AI, people don't understand what the AI is doing, and don't attribute actions to intelligence. If you have gaming applications for something you are looking into making, keep the player's perspective in mind when making it. Make the formations distinct, have them shout to each other, etc. Make sure the player can get what's going on. Your system sounds like a good system for breaking up monotony to a one-versus-many FPS if done right.

__________________________________
Allegro html mockup code thread -website-
"two to the fighting eighth power"

Trezker
Member #1,739
December 2001
avatar

Nero seems to be having severe problems with memory management on Linux. First time I ran it, after a while, it hogged 800+ meg, it shut down without my help. I think that out of memory feature that shuts down programs may have kicked in.

Thomas Fjellstrom
Member #476
June 2000
avatar

I thought they discontinued the Linux port :o k3b from kde3 is much better anyhow (last time I tried the kde4 version I couldn't get it to actually start burning a disk).

--
Thomas Fjellstrom - [website] - [email] - [Allegro Wiki] - [Allegro TODO]
"If you can't think of a better solution, don't try to make a better solution." -- weapon_S
"The less evidence we have for what we believe is certain, the more violently we defend beliefs against those who don't agree" -- https://twitter.com/neiltyson/status/592870205409353730

Trezker
Member #1,739
December 2001
avatar

I meant the nero game of course. http://nerogame.org/

Wilson Saunders
Member #5,872
May 2005
avatar

One way to make a learning FPS AI is to keep track of the locations the AI was Killed, or damaged and the locations the AI gave damage or got a kill. By gravitating towards the places where they got kills and avoiding places where they got killed, the agent will at least develop a sense of where to stand in a particular level. Also you can plug in data from human players to allow the AI to learn from other people's actions.

I know Steam is keeping track of this data for Team Fortress 2. Looking at their death density maps is pretty interesting.

________________________________________________
Play my games at http://monkeydev.com

Go to: