7 ms·

Supporting game design with evolutionary algorithms

75 points by kevthecoder 2y ago

jncfhnb 2y ago
Seems unlikely to work imo. Games need to account for the fact that 90% of the player base will be deeply unskilled and the remaining 10% will be tightly invested in a meta of dominant strategies. You’d need the bots to be able to consider this and also not just equalize every character.
- dartos 2y ago
  Yeah. A truly balanced game is boring. There’s no drama or tension. You need the power fantasy of being strongest characters and the underdog tale of winning against the strongest characters as a less meta one.
  - Log_out_ 2y ago
    Why not abandon the idea of balance entirely and tailor balance situational for maximum drama.make the meta shakespearean..
    - 1propionyl 2y ago
      Valve memorably did just this over a decade ago with the Director system in Left4Dead.
      - programd 2y ago
        See also the storytellers in Rimworld. Randy in particular is a master of the dramatic turn.
    - bookiest 2y ago
      [dead]
    - dartos 2y ago
      That’s exactly what “balancing” a game means.
- 10000truths 2y ago
  This can be accounted for by training bots using player inputs from historical games, no? If you then partition the training set by player ranking, you'd presumably get a set of bots that approximate a player from each ranking. With that, you could simulate the effect of a balance change on players of various skill levels.
  - jncfhnb 2y ago
    Not really. It would take immense effort to train bots to play “like humans” and not “as performantly” as humans which is very different. And if you’re going to be optimizing game parameters that means you’re assuming that either the AI doesn’t change its behaviors even though the game is different or you’re assuming that humans will adapts in the same way the bots do. Like if all the humans use the AK because it’s super over powered, and your optimization algorithm sets the AK damage to 0; what are your “human” bots going to do? Because all the training data says to use the AK. This approach only makes sense if you’re evaluating bot-optimal play outcomes. It also takes away a lot of the design thinking behind balance. You probably don’t want to nerf the AK. You probably want to buff counterplay options (guns are not a great example but still)
    - 10000truths 2y ago
      > Not really. It would take immense effort to train bots to play “like humans” and not “as performantly” as humans which is very different. There is precedent in Maia Chess, which does a good job of mimicking human chess players at various ELO ratings. Of course, it's a lot more difficult to extrapolate to games with significantly more state/movesets, but I imagine that this space will be further explored in the near future. > And if you’re going to be optimizing game parameters that means you’re assuming that either the AI doesn’t change its behaviors even though the game is different or you’re assuming that humans will adapts in the same way the bots do. This could be addressed by including the game parameters of interest (what map, what character, the weapon stats at time of gameplay, etc.) in the input to the training data. > It also takes away a lot of the design thinking behind balance. You probably don’t want to nerf the AK. You probably want to buff counterplay options (guns are not a great example but still) Tool-assisted QA is nothing new. Using AI is a newer iteration of the concept. You still have to interpret the results it gives and make decisions based on that. The design thinking isn't replaced, it's augmented with additional insights. Are those insights potentially inaccurate? Sure, but you can account for that with sanity checks/manual intervention/play testing.
- PicassoCTs 2y ago
  I found that to design games for the crowd seperatly and merge them, works wonders. One crowd gets a hyper-challenging rts, the others get to play heroe units making all the difference in a small dota game, playing off the the hyper-rts. The OPs can take over and micro, but they wont never micro as well as a human, for whom it is the whole game. One example.. of course, you goto keep it civil. No pro players dunking on the mom&pop gamers who just want to have 5mins of fun on a tablet.
- SpaceManNabs 2y ago
  I agree that a rudimentary evolutionary algorithm probably wouldn't work since they often don't take into account multimodality. There are global stochastic algorithms that take into account this sort of multi modality. Most particle swarm framworks could handle this sort of bipartite nature with a more complicated fitness function. Differential Evolution frameworks in R also allow you to do a post operation on each candidate, which include something like rejecting it because it does not seem to lie in one of the two modes.
- pfdietz 2y ago
  You can get around that by telling those 10% to f- off and play something else.
  - jncfhnb 2y ago
    The ones who are most interested in your game to actually learn it? That’s a strat but it’s certainly not the game dev meta
    - pfdietz 2y ago
      Amazingly, "better player" doesn't imply "better customer". That is, unless it's a P2W game, where better = willing to spend more money.
      - genocidicbunny 2y ago
        But so very often it implies "Loudest customer" -- that is, they are the most likely to loudly and publicly complain. Games live and die on the whims of their audiences; It's generally a very bad idea to piss off the most dedicated 10% of your playerbase.
        pfdietz 2y ago
        Unless that dedication is highly remunerative, pissing them off can be the smart thing to do, if that means pleasing the other 90%. The problem comes when the developers are having their egos stroked by this dedication. Imagining that those players are somehow valuable for PR is part of the rationalization for serving them, but experienced lesser players will learn that a game that caters to that elite won't be fun for them, and will treat that catering as the negative sign it is.
        genocidicbunny 2y ago
        You're not wrong, but the other end of the spectrum is that you don't have any players because the only ones talking about it are unhappy. I used to work on F2P games, and you have to balance things. As nice as it would be to ignore the loudmouths, unless you can guarantee that no one else is listening to them either, which you can't, you'll have to cater to them somewhat to pacify them.
- deleted 2y ago
  [deleted]
aschearer 2y ago
Programmers always want to automate content creation. I get it as I’m a coder and it’s a fun problem space. Plus creating content is super time and skill intensive. It’s really hard to get right in my experience. Kind of like trying to automate writing a novel… OK I read the article. I'm very skeptical of this approach. I doubt we can actually uncover fitness functions that reliably maps to "fun", and I believe it would require huge engineering effort to keep the game "simulable." Their examples aren't convincing. What would be convincing is a full, complex, and _fun_ game using these techniques. Also the article seems like an ad for their AI solution.
- cubefox 2y ago
  > Kind of like trying to automate writing a novel… So you are saying procedural level design could be solved with generative AI?
  - itishappy 2y ago
    In theory. It'll be more convincing once generative AI proves itself useful for writing novels.
    - DennisP 2y ago
      AI can already write short stories, just not very good ones. It'll be doing novels once the context windows are big enough.
      - itishappy 2y ago
        Agreed, the technology looks promising but isn't ready yet.
        cubefox 2y ago
        It's orders of magnitude better than procedural methods written by humans.
programd 2y ago
The whitepaper on the Grail framework is interesting [1]. It's an AI C++/C# framework you can use in game development. I would call it a very good implementation of "old school" AI, where the behavior of your actors is all about utility curves, Monte Carlo search, and genetic algorithms. Basically all math/algorithm based stuff, kind of like old expert system AI implementations. Of course "new school" AI is all about neural networks that can automatically learn those complex actor behaviors without the developer explicitly specifying all those mathematical algorithms. Like many people here I'm very interested in working on hooking up modern "new school" AIs to virtual worlds, so it's very interesting to see Grail as a good concrete example of the algorithmic approach to game AI. I suspect some hybrid fusion of both approaches may give us some interesting and fun AI behavior. [1] https://grail.com.pl/media/Grail_Whitepaper_June_2021.pdf https://grail.com.pl/media/Grail_Whitepaper_June_2021.pdf
- uoaei 2y ago
  The dichotomy of interest isn't really NNs vs GAs but about stochasticity vs determinism. The user/player doesn't care which it is until they learn to see the patterns and to reliably exploit the rules. Once you notice these patterns, the spell is partially broken and immersion is compromised. That's part of the reason stochastic methods have been such a hit throughout the history of game development for NPC behavior.
- bob1029 2y ago
  I think things get really interesting when you combine small, efficient ANNs with evolutionary/genetic algorithms. The biggest challenge with these is coming up with a function that clearly represents the fitness of a candidate. EAs are worthless if you can't provide good selection pressure over time. There are always multiple conflicting objectives in the most practical cases. I've recently discovered that you can skip all the nasty objective weighting business if you just select the Pareto front every generation. You never know when a trip down a less important path might result in the most optimal global solution.
shortrounddev2 2y ago
I think Quake 3 used a bit of genetic algorithms to optimize some of the variables of the bots
Rhapso 2y ago
Missing the OG: NERO https://nn.cs.utexas.edu/nero/about.php https://nn.cs.utexas.edu/nero/about.php NEAT is really well suited to modeling behaviors in a less structured way and gets used to play videogames.
NotGMan 2y ago
Optimal gameplay balance != fun gameplay. Many games were ruined post-release because developers tried to make the gameplay "more balanced". This usually leads to everything feeling the same. Helldivers 2 was a good recent example, which is suprising since it's not a PvP game at all but a co-op PvE only.
- Rhapso 2y ago
  well, extract a good metric for "fun" (like engagement) and make that the optimization criteria. It would be equally as easy.
  - adastra22 2y ago
    There is no good metric for fun.
    - mattigames 2y ago
      Call of duty occacionally asks you after a match if you had fun (5 options from "boring" to extremely fun), I bet thats a good metric for fun.
    - teamonkey 2y ago
      Heart rate, pupil dilation, breathing rate…
pfdietz 2y ago
Oh wow, this kind of idea goes way back, to Doug Lenat's Eurisko. Good old fashioned AI. He famously used that to craft winning strategies for Traveller Trillion Credit Squadron, a non-computer game. Today's game designers might use this sort of thing to find game-breaking exploits before those damned players do.

75 points by kevthecoder 2y ago

Seems unlikely to work imo. Games need to account for the fact that 90% of the player base will be deeply unskilled and the remaining 10% will be tightly invested in a meta of dominant strategies. You’d need the bots to be able to consider this and also not just equalize every character.

dartos 2y ago
Yeah. A truly balanced game is boring. There’s no drama or tension. You need the power fantasy of being strongest characters and the underdog tale of winning against the strongest characters as a less meta one.
- Log_out_ 2y ago
  Why not abandon the idea of balance entirely and tailor balance situational for maximum drama.make the meta shakespearean..
  - 1propionyl 2y ago
    Valve memorably did just this over a decade ago with the Director system in Left4Dead.
    - programd 2y ago
      See also the storytellers in Rimworld. Randy in particular is a master of the dramatic turn.
  - bookiest 2y ago
    [dead]
  - dartos 2y ago
    That’s exactly what “balancing” a game means.
10000truths 2y ago
This can be accounted for by training bots using player inputs from historical games, no? If you then partition the training set by player ranking, you'd presumably get a set of bots that approximate a player from each ranking. With that, you could simulate the effect of a balance change on players of various skill levels.
- jncfhnb 2y ago
  Not really. It would take immense effort to train bots to play “like humans” and not “as performantly” as humans which is very different. And if you’re going to be optimizing game parameters that means you’re assuming that either the AI doesn’t change its behaviors even though the game is different or you’re assuming that humans will adapts in the same way the bots do. Like if all the humans use the AK because it’s super over powered, and your optimization algorithm sets the AK damage to 0; what are your “human” bots going to do? Because all the training data says to use the AK. This approach only makes sense if you’re evaluating bot-optimal play outcomes. It also takes away a lot of the design thinking behind balance. You probably don’t want to nerf the AK. You probably want to buff counterplay options (guns are not a great example but still)
  - 10000truths 2y ago
    > Not really. It would take immense effort to train bots to play “like humans” and not “as performantly” as humans which is very different. There is precedent in Maia Chess, which does a good job of mimicking human chess players at various ELO ratings. Of course, it's a lot more difficult to extrapolate to games with significantly more state/movesets, but I imagine that this space will be further explored in the near future. > And if you’re going to be optimizing game parameters that means you’re assuming that either the AI doesn’t change its behaviors even though the game is different or you’re assuming that humans will adapts in the same way the bots do. This could be addressed by including the game parameters of interest (what map, what character, the weapon stats at time of gameplay, etc.) in the input to the training data. > It also takes away a lot of the design thinking behind balance. You probably don’t want to nerf the AK. You probably want to buff counterplay options (guns are not a great example but still) Tool-assisted QA is nothing new. Using AI is a newer iteration of the concept. You still have to interpret the results it gives and make decisions based on that. The design thinking isn't replaced, it's augmented with additional insights. Are those insights potentially inaccurate? Sure, but you can account for that with sanity checks/manual intervention/play testing.
PicassoCTs 2y ago
I found that to design games for the crowd seperatly and merge them, works wonders. One crowd gets a hyper-challenging rts, the others get to play heroe units making all the difference in a small dota game, playing off the the hyper-rts. The OPs can take over and micro, but they wont never micro as well as a human, for whom it is the whole game. One example.. of course, you goto keep it civil. No pro players dunking on the mom&pop gamers who just want to have 5mins of fun on a tablet.
SpaceManNabs 2y ago
I agree that a rudimentary evolutionary algorithm probably wouldn't work since they often don't take into account multimodality. There are global stochastic algorithms that take into account this sort of multi modality. Most particle swarm framworks could handle this sort of bipartite nature with a more complicated fitness function. Differential Evolution frameworks in R also allow you to do a post operation on each candidate, which include something like rejecting it because it does not seem to lie in one of the two modes.
pfdietz 2y ago
You can get around that by telling those 10% to f- off and play something else.
- jncfhnb 2y ago
  The ones who are most interested in your game to actually learn it? That’s a strat but it’s certainly not the game dev meta
  - pfdietz 2y ago
    Amazingly, "better player" doesn't imply "better customer". That is, unless it's a P2W game, where better = willing to spend more money.
    - genocidicbunny 2y ago
      But so very often it implies "Loudest customer" -- that is, they are the most likely to loudly and publicly complain. Games live and die on the whims of their audiences; It's generally a very bad idea to piss off the most dedicated 10% of your playerbase.
      - pfdietz 2y ago
        Unless that dedication is highly remunerative, pissing them off can be the smart thing to do, if that means pleasing the other 90%. The problem comes when the developers are having their egos stroked by this dedication. Imagining that those players are somehow valuable for PR is part of the rationalization for serving them, but experienced lesser players will learn that a game that caters to that elite won't be fun for them, and will treat that catering as the negative sign it is.
        genocidicbunny 2y ago
        You're not wrong, but the other end of the spectrum is that you don't have any players because the only ones talking about it are unhappy. I used to work on F2P games, and you have to balance things. As nice as it would be to ignore the loudmouths, unless you can guarantee that no one else is listening to them either, which you can't, you'll have to cater to them somewhat to pacify them.
deleted 2y ago
[deleted]

aschearer 2y ago

Programmers always want to automate content creation. I get it as I’m a coder and it’s a fun problem space. Plus creating content is super time and skill intensive. It’s really hard to get right in my experience. Kind of like trying to automate writing a novel… OK I read the article. I'm very skeptical of this approach. I doubt we can actually uncover fitness functions that reliably maps to "fun", and I believe it would require huge engineering effort to keep the game "simulable." Their examples aren't convincing. What would be convincing is a full, complex, and _fun_ game using these techniques. Also the article seems like an ad for their AI solution.

cubefox 2y ago
> Kind of like trying to automate writing a novel… So you are saying procedural level design could be solved with generative AI?
- itishappy 2y ago
  In theory. It'll be more convincing once generative AI proves itself useful for writing novels.
  - DennisP 2y ago
    AI can already write short stories, just not very good ones. It'll be doing novels once the context windows are big enough.
    - itishappy 2y ago
      Agreed, the technology looks promising but isn't ready yet.
      - cubefox 2y ago
        It's orders of magnitude better than procedural methods written by humans.

programd 2y ago

The whitepaper on the Grail framework is interesting [1]. It's an AI C++/C# framework you can use in game development. I would call it a very good implementation of "old school" AI, where the behavior of your actors is all about utility curves, Monte Carlo search, and genetic algorithms. Basically all math/algorithm based stuff, kind of like old expert system AI implementations. Of course "new school" AI is all about neural networks that can automatically learn those complex actor behaviors without the developer explicitly specifying all those mathematical algorithms. Like many people here I'm very interested in working on hooking up modern "new school" AIs to virtual worlds, so it's very interesting to see Grail as a good concrete example of the algorithmic approach to game AI. I suspect some hybrid fusion of both approaches may give us some interesting and fun AI behavior. [1] https://grail.com.pl/media/Grail_Whitepaper_June_2021.pdf https://grail.com.pl/media/Grail_Whitepaper_June_2021.pdf

uoaei 2y ago
The dichotomy of interest isn't really NNs vs GAs but about stochasticity vs determinism. The user/player doesn't care which it is until they learn to see the patterns and to reliably exploit the rules. Once you notice these patterns, the spell is partially broken and immersion is compromised. That's part of the reason stochastic methods have been such a hit throughout the history of game development for NPC behavior.
bob1029 2y ago
I think things get really interesting when you combine small, efficient ANNs with evolutionary/genetic algorithms. The biggest challenge with these is coming up with a function that clearly represents the fitness of a candidate. EAs are worthless if you can't provide good selection pressure over time. There are always multiple conflicting objectives in the most practical cases. I've recently discovered that you can skip all the nasty objective weighting business if you just select the Pareto front every generation. You never know when a trip down a less important path might result in the most optimal global solution.

shortrounddev2 2y ago

I think Quake 3 used a bit of genetic algorithms to optimize some of the variables of the bots

Rhapso 2y ago

Missing the OG: NERO https://nn.cs.utexas.edu/nero/about.php https://nn.cs.utexas.edu/nero/about.php NEAT is really well suited to modeling behaviors in a less structured way and gets used to play videogames.

NotGMan 2y ago

Optimal gameplay balance != fun gameplay. Many games were ruined post-release because developers tried to make the gameplay "more balanced". This usually leads to everything feeling the same. Helldivers 2 was a good recent example, which is suprising since it's not a PvP game at all but a co-op PvE only.

Rhapso 2y ago
well, extract a good metric for "fun" (like engagement) and make that the optimization criteria. It would be equally as easy.
- adastra22 2y ago
  There is no good metric for fun.
  - mattigames 2y ago
    Call of duty occacionally asks you after a match if you had fun (5 options from "boring" to extremely fun), I bet thats a good metric for fun.
  - teamonkey 2y ago
    Heart rate, pupil dilation, breathing rate…

pfdietz 2y ago

Oh wow, this kind of idea goes way back, to Doug Lenat's Eurisko. Good old fashioned AI. He famously used that to craft winning strategies for Traveller Trillion Credit Squadron, a non-computer game. Today's game designers might use this sort of thing to find game-breaking exploits before those damned players do.