7 ms·

This is really impressive, I didn't expect starcraft to be played this well by a machine learning based AI. I'm excited to read the paper when it comes out! Th

• cm2012 8y ago
  In the showmatched they made the computer have to look at a regular screen to control, the stalker micro was much less impressive - and mana won.
  • Jyaif 8y ago
    The AI lost because it completely messed up the response to the immortal drop, nothing to do with micro.
    • celeritascelery 8y ago
      Mana got himself in the same situation where he was surrounded by stalkers on multiple sides, but this time the micro wasn’t so crazy that he couldn’t manage it, and he was able to take on one group at a time. The immortal drop, while unanswered, was not really that effectual.
      • setr 8y ago
        That time, the ai didn’t really even try to engage. In fact, the ending of the match was marked by the entirely absent group of stalkers as the natural was engaged. It’s likely safer to say the AI was confused in general at that point, possibly related to the camera change, but we didn’t really get to see the quality of stalker micro that game
        • nickpsecurity 8y ago
          "possibly related to the camera change, but we didn’t really get to see the quality of stalker micro that game " In software, changes in assumptions can break what depended on them. There could be many assumptions in its neural net centered on full visibility. They should probably retrain all or just some from scratch with the camera change in from the beginning to see what happens. Then, it will be firmly encoded into the strategies over time.
          • maksimum 8y ago
            They mentioned that they retrained after the camera change and it was equivalent to the AIs that beat Mana 5-0 by their metrics.
      • cjbprime 8y ago
        But it was answered: AlphaStar pulled a huge stalker army that was about to hit MaNa's base all the way back home to (attempt to) answer the drop, repeatedly. If you have more complexity to your army but fewer army units, as MaNa did, a delay like that is how you win the game.
        • b_tterc_p 8y ago
          It’s funny because this works against the standard Ai too.
          • nickpsecurity 8y ago
            That's what I said on Lobsters. They were always good at builds, micro's, etc. The one thing they couldn't do was judge human intent, esp if they were being mislead (esp time wasting). I was waiting for one of the players to try to screw with its head to see what it did. Mana showed two gaps: the back and forth thing; that it ignored the observers giving up constant strategy information. Then, he got the first win. Now, the questions are how many more such glitches will show up and can they eliminate them with better algorithms?
    • Laremere 8y ago
      This was my read as well. It seems that Mana simply found a strategy that the AI had not found. Due to not having trained against it, the AI produced nonsense results. The commentators noted that the obvious response was to build a Pheonix and just completely shut down the harassment. The situation is similar to Alpha Go vs Lee Sedol match 4. One of the hardest parts about these kinds of human vs ai expositions is making sure the AI has explored the full possibility space, so that can handle all situations. The techniques at play lack the ability to perceive a completely new situation and formulate a good response. (Though anyone who's lost to cheese in games they later learned easy counters for know that humans, while better than state of the art AI, aren't perfect here either.)
  • CydeWeys 8y ago
    For now. Give them another month. This is like AlphaGo vs Fan Hui all over again -- people knocked that accomplishment at the time because he was just a master, not one of the top players in the world. Well, not much longer, AlphaGo beat Lee Sedol, the best player in the world. The ceiling here is going to be incredibly high, much higher than the level of play that people are capable of, even when restricted to a single window.
    • tsbinz 8y ago
      Lee Sedol was not the best player anymore at that time (not saying it wasn't an impressive/important achievement, but overstating it doesn't help either - the "beat best human players part" came later in 2017).
      • eurg 8y ago
        I don't understand who's downvoting you, this is accurate. While AlphaGo/Zero improved quickly to superhuman play, we are just in this thread comparing timelines, so that is relevant.
      • NhanH 8y ago
        Lee Sedol was still top 5, certainly no worse than top 10 at the time. By all mean he wasn't the best and most dominant, but the difference with the top was tiny.
    • gambler 8y ago
      It's not the accomplishment that people knock. It's the spin, the inaccurate article titles and the hype.
      • CydeWeys 8y ago
        It doesn't seem like hype to me -- it seems like a genuine, significant accomplishment. Sure, they might not be able to beat the best pro players consistently right now, but I suspect that is right around the corner. Would you rather they stay completely mum until they've reached that goal too? And why? I'd rather know now, and then be able to follow along as it gets better and beats higher and higher-ranked players.
• Thaxll 8y ago
  This is exactly what I think, I'd like to see how Alphastar react to "cannon rush" or other weird bo where you need to be "smart" to counter it and just not be based on insane / none human micro.
  • knicholes 8y ago
    This is how it responds to cannon rush. : ) https://www.youtube.com/watch?v=vYdWQjTWTFM https://www.youtube.com/watch?v=vYdWQjTWTFM
    • tjoff 8y ago
      Isn't the point of a cannon rush to build the first cannons where they can't be seen?
      • gpm 8y ago
        Surprisingly not. The trick is usually to build pylons (or other cannons) such that they protect the cannons from being attacked by probes. Building them out of sight is usually too slow as a rush. Still, he didn't do that either.
        • tjoff 8y ago
          In the beginning of SCII I only saw people trying to hide it. But I guess the strategy evolved, interesting.
          • arayh 8y ago
            Sometimes you may see a photon cannon used to deny an enemy's natural expansion to try to gain an economic advantage. Depending on the map and matchup, it may also complicate the enemy's early attempts at scouting and aggression. Typically, you don't see more than 1-2 photo cannons, because you don't usually want to "over-invest" and lose what advantage you gain.
    • Thaxll 8y ago
• gamegoblin 8y ago
  In the mass stalker battles, the AI APM exceeded 1000 a few times, and no doubt that most of that was precisely targeted. Whereas a human doing 500 APM micro is obviously going to be far more imprecise. I think a far more interesting limitation would be to cap APM at 150 or so, or to artificially limit action precision with some sort of virtual mouse that reduced accuracy as APM increased.
  • simmanian 8y ago
    > artificially limit action precision with some sort of virtual mouse that reduced accuracy as APM increased I like the idea of having action noise that's linearly related to APM
  • wnevets 8y ago
    >I think a far more interesting limitation would be to cap APM at 150 or so, or to artificially limit action precision with some sort of virtual mouse that reduced accuracy as APM increased. IIRC OpenAI limits the reaction time to ~200ms when playing DoTA2. AI employing better strategies than humans will always be more interesting than AI that can out click humans.
    • jgon 8y ago
      Even the 200ms reaction time seemed overly slanted towards the AI. I don't think that is the actual reaction time of top pros, in the matches the AI played the human player would teleport in from complete invisibility and try to use an instant cast spell and the AI would have already teleported out. Yes the theoretically may have been constrained to a 200ms reaction time, but in practice the AI was playing at a superhuman level. Even with that advantage in fights, the human team still demolished the AI. Oh well, lots of things to learn still.
      • lrem 8y ago
        I've recently watched a TED talk explaining how human perception has a lag of about a third of a second. Pro players might be better, but after noticing they also need to take an action.
• andreyk 8y ago
  Also worth noting that it starts by imitation learning from pros. I'd be curious to see if the macro can be learned without imitation; a much harder challenge. Also, playing with full visibility as was mostly the case in the demonstration is quite lame...
  • freeflight 8y ago
    If you learn, why not learn from the best, the pros? These people already have spent years figuring out what works and what doesn't. Why not draw from that pool of knowledge and instead spend extra time going through the same motions?
    • MereInterest 8y ago
      Because then you don't know whether the AI learned by experimentation or by mimicking. To draw an analogy, imagine the difference between somebody reading and following an algorithm to solve a Rubik's cube, as opposed to somebody being handed a Rubik's cube and experimenting. If expert-level strategies can be reproduced without being explicitly shown to the person/AI, then it means something is going right in your methodology.
    • kadoban 8y ago
      Two reasons I can think of: An AI trained from human strategy might end up more limited than one that could learn from scratch. It could be stuck in a local maximum of play and be unable to escape. An AI technique that requires a large dataset of pro play to learn will be much more limited in terms of applying it to other games.
  • kolinko 8y ago
    It wasn't a full visibility - Alpha had a fog of war. It just saw the whole map at the same time.
    • TulliusCicero 8y ago
      That's still a large advantage that humans don't have access to. Not just in the "pitiful humans can't take advantage of such a large viewing area" sense, but literally the game will not let human players zoom out that far.
• olliej 8y ago
  it seems like in some cases at least it didn't have to move the camera (it had direct interfaces) which for some of the stalker micro battles (especially in game 3 or 4?) the battles were larger than the screen space -- it would not have been possible to micro that well if your control interface limited what you can control or where you can place them.
  • methodover 8y ago
    This is a great point, and something that seems a bit lost in the discussion: In StarCraft 2, the game IS the interface. That is to say, the developers have constructed the game in such a way as to be difficult to control; and human mastery of the interface is a large percentage of the game. Strategy in the game is important, of course -- but this is not chess, where human beings are not limited by the interface of the game. In StarCraft, you are intentionally given a limited interface to monitor and control a gigantic game while under incredibly tight time controls. And I should also note that Blizzard is extremely reluctant to add features that make it easier to control the game. I have a friend who works on the StarCraft 2 team. We talked at length about this one feature that he designed and proposed for the team to make a specific aspect of the game friendlier towards players. It was turned down for exactly the reasoning above -- the game is the interface. By making the game easier to control, it disrupts the entire experience; an StarCraft 2 that is easier to control is no longer StarCraft 2.
    • olliej 8y ago
      That would actually be an interesting thing for someone from blizzard to do, get two similarly skilled high level players, and compare the win/loss rate by doing two 7 games matches with each player having a match with a 10% increased view size, and see what the impact is. Essentially try to quantify the advantage of increased view area.
    • foobiekr 8y ago
      Are there any bolt-on augmentation interfaces that utilize the same API the bots use to allow players to more effectively enter their intent?
    • 8y ago
• Cookingboy 8y ago
  The results are obviously impressive, but even then there is a lot of work to do as far as learning efficiency goes: "The AlphaStar league was run for 14 days, using 16 TPUs for each agent. During training, each agent experienced up to 200 years of real-time StarCraft play. " MaNa probably played less than 2-3 years of Starcraft in his whole life (by that I mean 24hr x 365d x 3), and was learning with a much less focused/rigorous methodology.
  • derefr 8y ago
    Another way to think about it is that a human brain is mostly doing transfer-learning, on top of a 99%-baked deep net that was wired up during foetal development from our DNA, where that DNA-persisted model has "seen" hundreds of millions of years of training data. Humans don't have to learn to process, recognize, and classify objects in visual sense-data, for example. We can do that from the moment we're born, because we already have hundreds of precisely-tuned "layers" laying around in our brains for doing just that. We just need to transfer-learn the relevant classes.
    • dcl 8y ago
      This is a widely underappreciated fact when it comes to comes to comparing the 'training experience' of humans versus bots. And it extends far beyond processing 'sense data' - A human likely has some level of understanding of how the game works based on experience from other games it has played and from 'real life' - we know almost instinctively that 'high ground' is likely to give a combat advantage without having test it in game.
      • derefr 8y ago
        Not only that, humans (and many other eusocial species) have an instinctual intuitional understanding of many aspects of game theory. For example, humans, even from infancy, prefer games where it is possible to punish cheating (i.e. take revenge upon cheaters) to games where it is not. This isn't just "we're animals that have evolved to enact tit-for-tat strategies [by e.g. injustice triggering rage] because they lead to cooperation which leads to egalitarian utility"; this is actual analysis—instantaneous, intuitive analysis—of a system of rules, to notice, in advance of ever being slighted, whether you'll be likely to end up in an "unjust" social situation if you agree to the given ruleset. There is an "accelerated co-processor" of high-level abstract game-theoretic information—and layers to extract that information from sense-data—that ship as part-and-parcel of the human brain model. We never need to learn how to judge unfairness, any more than we need to learn how to see.
• ehsankia 8y ago
  > While they have similar APM to SC2 pros Wasn't the APM closer to half that of the pros? https://storage.googleapis.com/deepmind-live-cms/images/SCII-BlogPost-Fig09.width-1500.png https://storage.googleapis.com/deepmind-live-cms/images/SCII...
  • mactrey 8y ago
    A huge part of a human's APM is meaningless spam, for example right-clicking the same unit multiple times to attack it, or setting the same waypoint thousands of times in the early game when there's nothing to do. The computer might be at double the human's effective APM, if only we had a credible way to measure that.
  • arcticfox 8y ago
    This is super deceiving and I'm kind of upset they posted this image, knowing it would mislead people not familiar with the game. The AI sits around during lulls at <30 APM - meanwhile MaNa and TLO were literally spamming keys to keep their fingers warm, not actually doing anything. During the fights, the critical moments in when MaNa would top out at ~600 humanly inaccurate APM (this is 10 inputs per second), the AI would jump up to over 1000 - we don't know exactly what it was doing, but it was presumably pixel-precise. Meanwhile the physical inertia of the mouse is a challenge for humans at that speed - imagine trying to click five totally different places with perfect precision in a single second.
    • backpropaganda 8y ago
      Do you know why TLO's APM is sometimes so large? Did he actually peak at 2000, or is he using a repeater or something like that?
      • Retric 8y ago
        APM gets inflated by counting several single actions as multiple separate actions. For example a Zerg player may want to turn larva into 30 Zerglings, they do this by pressing one button and holding it down as the UI repeats a separate action for each larva transformed. By comparison selecting a single stalker, and having it jump to a new location is much more effort, but counts as fewer actions.
• pesmhey 8y ago
  In a nutshell, AI micro was flawless, makes up for suboptimal macro?
  • cjbprime 8y ago
    The macro seemed fine -- AlphaStar usually had more workers than the human opponent, in every game, and was producing more army. The suboptimality seemed to be in army composition (blink stalkers) and strategic decision making (pulling all of a superior army back home to defend a single warp prism drop).
• sciyoshi 8y ago
  I would agree with that. If you take a look at the exhibition match replay, there's some cases where it makes objectively suboptimal decisions. We couldn't see this during the live stream, but the double immortal warp prism caused AlphaStar to bring back its entire army from across the map, when a few units at home would have been enough to defend. It even kept trying to blink its stalkers to a place where the warp prism couldn't be reached. Perhaps this version with the limited viewpoint hadn't been trained with enough games?
• throwawaymath 8y ago
  > For context, I've been ranked in the top 0.1% of players and beaten pros in Starcraft 2, and also work as a machine learning engineer. What's that hireability like?
• deleted 8y ago
  [deleted]
• sytelus 8y ago
  Also these wins are not using same inputs that human receive (ie on screen image) and outputs that humans are allowed. They instead use PySC APIs which has much more flexibility, perfect information and no constraints of limited screen real estate and pixels. There is a claim in that article that they have another version being trained that uses on screen only information but I still don’t know if AI is allowed to bypass the physical constraints of controller. So if AI has access to super human controller you will see AI performing super human actions like many commentators have described here.
  • kibibu 8y ago
    The final game in the video has this limitation. It does affect the performance of the agent.
  • ygra 8y ago
    Perfect information is a bit of a stretch. There was still fog of war. The AI just played as if the portion of the map visible and actionable at any point in time was the whole map. They retrained with a restriction to a given locus of attention that can change, akin to a screen the player is looking at and acting on.
• nopinsight 8y ago
  One macro technique used by AlphaStar agents that is not used by human pros is building extra workers beyond currently exploitable capacity. This gives them reserves when attacked and some workers killed. They can also ramp up mining at a new base quickly by moving the extra workers there. Apparently the benefits outweigh the costs for these workers for AlphaStar. It will be interesting to see if some pros decide to adopt the technique and if it improves human performance as well. Disclaimer: I do not have much Starcraft experience.
  • jammygit 8y ago
    Workers mine 40 minerals per minute and cost 50, taking... 15 seconds to build? I forget. Workers beyond 24 provide zero benefit (better to send them to the natural). Let's say you make 4 extra at a cost of 200 minerals and then lose 4 workers to harassment. You are out 200 minerals in both cases, but the prebuilt workers in the prebuilt case will mine an extra... 100 minerals? (40 + 30 + 20 + 10). This doesn't take chronoboost into account though. I don't know, the gain is marginal, and the opportunity cost is having a smaller army (2 zealots for example) Please correct my numbers if I've made a mistake, I forget build times and havent played since hots
    • keerthiko 8y ago
      The numbers you cite are close enough that your estimations are good to work with (12 seconds to build, closer to 60 minerals at full efficiency but down to 40 for probes #17-24, etc) The extra workers aspect was the most interesting decision-based adjustment AlphaStar made on conventional pro level wisdom of "standard" play. It has a couple of factors in play, that I trust the AI factored in and more and tested over several games for its long-term benefit to winning a game: - every 8 probes you build requires a pylon as well. total cost of 500 minerals - workers are safer in the main than in an unoccupied natural (long distance mining) to harassment and pressure - when your expansion completes, having 4 workers vs 8 workers vs 16 workers potentially has huge impact to the immediate spike in income - what you mention -- the prebuilt workers will dampen the impact of most worker harassment to purely the resource cost of the lost workers. My guess was that well executed harassment by an opponent in practice games put AlphaStar in very limited situations with a crippled economy that it couldn't fight its way out of, so this was a catch-all harassment "counter" -- it's ok if you kill a few probes, at least it won't throw off my economy completely and I can still continue my overall gameplan. After that I think the next most important aspect was planning ahead for a bigger income spike when their expansion was done without waiting to build out another 16 workers after the nexus was ready.
• rkagerer 8y ago
  Yeah that stalker micro really showcases a particular advantage leveraged by the AI. I'd love to watch the results of constraining the AI so instead of seeing the whole map at once it has to pan around the same way a human would to get updated information on each battle. Counting those "info-gathering" window pans against the actions tally might yield slightly fairer APM metrics. (EDIT: Turns out they built a new agent for game 11 to do just that) One of my biggest beefs with strategy games of this genre occurred around the time sprites went 3D and the player viewports got smaller (presumably to showcase all the cosmetic detail, and since it became harder to distinguish between visuals when zoomed out farther). I always feel too constrained on the modern games - like I can't see enough of the map at once. In my opinion that "full size viewport" gives a multi-tasking edge to the engine that the player doesn't share (beyond the human cognitive overhead from context switching you already pointed out). On the other hand I find it fascinating our AI's have become strong enough at our games that we're having to handicap them to avoid players crying foul that they're not fair.
  • fandango 8y ago
    I agree. Most RTS games feel constrained because of the limited viewport. Supreme Commander has a nice feature where you can zoom all the way out at any time.
    • yvdriess 8y ago
      And a very important part to SupCom's zoom feature is that at a certain zoom level it switches to a rich visual overlay of unit icons and pending/queued orders.
• javier2 8y ago
  I am really impressed it learned when to pull probes in that game against Mana where the AI was pressured into his natural. It was also extremely active with the stalkers, deciding to split them in three and not let Mana cross the map with his immortals.
• throwaway415415 8y ago
  Totally. What would be interesting to see is a low APM bot that still beats human players. A lot of that macro was unbeatable.
• stared 8y ago
  APM is one thing. I am curious what would happen if it could only see a limited view (as in the last game with MaNa, which it lost to him) and physical click dynamics (i.e. clicking + gaussian noise as an action, instead of giving direct commands). That way there will be misclicks, preventing this super-efficient Stalker micro.
• porky 8y ago
  And also, latency is lower
• ajuc 8y ago
  Mana tried to outblink an AI? Damn I really need to watch these games :)
• notSupplied 8y ago
  It's a question of whether "played with human level latency and precision" be a part of the rules of the game we are making the AI play. I would say yes, because StarCraft was very clearly balanced for human players. We already saw some indication that when played with super-human micro, mass blink stalkers is a stronger strategy than when humans are in control. Without the active intervention of game balancing, RTS metas tend to devolve into "mass one or two units" which was what happenes to every Command & Conquer game (and why SC is a respected eSport while C&C is not). I suspect this will happen when you have agents playing parameters that don't match what the game was balanced for. The strategic landscape will shrivel up and the game cease to captivate us.
• sidusknight 8y ago
  What was your SC2 alias? I played at a similar level as you.