Since the days of the first burst of electricity, people have feared the idea of robots taking over humanity, with entire movie franchises like Terminator based on the very notion. For most people, the idea of an android being able to do the washing or drive them to work sounds quite exciting, replacing our need to do menial tasks ourselves and have someone, or something, else to them for us. Yet how exciting is the idea of a computer being better at poker than us? Doesn’t that have all sorts of implications around cognition, thought processes and intuition?
We like to think of poker as being a game in which your ability to bluff is key, but there is also an element of statistical certainty involved. Is a computer’s ability to win consistently more likely to be down to mathematics than anything else? Possibly not, given that a machine that took on eleven professional poker players and beat ten of them used something that its creators referred to as an artificial intelligence akin to intuition. DeepStack, the name of the machine, is a step up from Deep Blue, that took on chess champions, because in chess all of the information you need is in front of you, which isn’t the case with Texas Hold’em poker.
What Is DeepStack?
A computer scientist at the University of Alberta named Michael Bowling and a group of fellow scientists created a computer program that took previous Artificial Intelligence to a whole new level, giving the computer the ability to learn and get better the more that it played games of poker. The idea of playing poker against a computer isn’t a new thing, with online poker networks having been around for years and games on the likes of iPads and smartphones allowing users to play against computers whenever they like.
The differences between those games and the work using DeepStack, however, were numerous. For starters, traditional AI is able to calculate the possible outcomes of a game before it even takes place, picking the most likely route to success. That wasn’t possible for DeepStack because it was used to play No Limit Texas Hold’em Heads Up, which is a variation of the game of poker in which just two people go up against each other and offers a number of possible decisions that many believe is higher than the number of atoms that there are in the universe.
Whilst it’s believes that there are 1 followed by 80 zeroes atoms in the universe, the number of options believed to be available in Heads Up No Limit Texas Hold’em are closer to 1 followed by 160 zeroes. That makes the version of poker played by DeepStack far too complicated for normal computer programs, so something different was needed in order to take on the eleven professional players representing the International Federation of Poker.
The Experiment: Man Versus Computer
In the experiment, the eleven players played three thousand hands each against DeepStack, all of which the artificial intelligence computer with the exception of one. The match with the eleventh player, Marc Sturc, was classed as too close to call. Sturc, who is the President of the Austrian Pokersport Association, said that he realised quickly that DeepStack was changing his gameplay and that he needed to be ‘one step ahead’. He described being the only player that DeepStack couldn’t beat as being ‘a huge compliment’.
Deep Learning
The more games DeepStack played, the better it was able to get thanks to the deep learning algorithm that scientists created for it. It was able to learn from each previous experience in order to handle new situations better. Bowling said, “Where we get (intuition) in DeepStack is very similar to how a human might get it … through experience”. The team of scientists behind it played millions of games against the AI and the results of the simulations were fed into a neural network that they referred to as the machine’s ‘digital brain’.
As the computer was unable to play out all variations to the end of the game, it instead looked a few moves ahead to get a sense of how the game might pan out. The move to cut the need to make multiple choices to just a few meant that the scientists were able to create an algorithm that may soon be available commercially on average laptops. In the end, Sturc said that the way he got the better of DeepStack was to ‘think like a computer’, which he was perhaps uniquely qualified to do after studying both poker and artificial intelligence at grad school.
Was DeepStack Better at Poker Than Humans?
Obviously one of the main questions that DeepStack’s existence and success asks is surrounding the idea of computers playing online poker on behalf of people, helping them to win money at the expense of less experienced players that don’t have access to such artificial intelligence. What we need to know, therefore, is whether DeepStack was actually better than its human opponents.
On a purely numbers basis, it’s difficult to argue otherwise, considering it beat ten of the eleven players it was up against by a wide margin. Yet there are still questions surrounding whether that was purely mathematical of whether there was more to it than that.
Huettel has used poker as a means of studying the human brain around the idea of decision making. Though he wasn’t involved in the creation of DeepStack, he did notice that excellent poker players are good not simply because they have good intuition, which DeepStack was aiming to recreate, but also because they understand the odds in any given moment.
When it comes to discussing whether the AI was ‘better’ at poker than a human, it depends entirely on what you mean. Yes, in terms of games won alone, DeepStack defeated all but one of the experts it was up against, yet that was in computerised versions of the game that are based entirely on the cards that users are presented with.
Poker in real life allows you to look into someone else’s eyes, notice the sweat on their brow and look out for their tells. DeepStack has no tells and can’t call someone’s bluff, so there’s still a long way to go before a robot will be sitting down at the final table of the World Poker Tour and taking him the grand prize.
What It Means for the Future
Though some people involved in the experiment are skeptical about what, exactly, it means for the future, there’s no doubt that it was an interesting and worthwhile experiment. At the time DeepStack was used, it was unable to learn from its opponent’s strategy as the game was taking place, but it’s something that Bowling and his team are hoping to look to introduce at some point in the future. Sturc was convinced that it is already a useful tool for poker players, who could use it to train for a competition and as a manner of comparing other players’ abilities in a neutral environment.
Computers in Medicine
Ultimately, of course, the desire is for computers to be used in more real-world scenarios than just poker tournaments. One such example is the world of medicine, which Bowling has begun creating a mathematical model for. At the moment he’s concentrating on diabetes, but he readily admits that they’re ‘miles away’ from being able to use it in trials on real people.
Scott Huettel is a professor of psychology and neuroscience at Duke University and believes that poker and decision making in medical care are different worlds, saying, “Poker is a zero-sum game. Medical decision making usually involves aligned interests”, with part of the problem being getting people to follow what they know is medically the right thing to do.