Assessing Game Balance with AlphaZero: Exploring Alternative Rule Sets in Chess (Paper Explained)
deep learning machine learning arxiv explained neural networks ai artificial intelligence paper deepmind chess kramnik fide rules alphago alpha go alphazero alpha zero mu zero muzero google reinforcement learning mcts rule change other rules alternate rules torpedo no castling pawn sideways self capture entropy opening theory rule based systems berlin defense opening stalemate deep rl deep reinforcement learning alphazero chess alphazero analysis
#ai #chess #alphazero

Chess is a very old game and both its rules and theory have evolved over thousands of years in the collective effort of millions of humans. Therefore, it is almost impossible to predict the effect of even minor changes to the game rules, because this collective process cannot be easily replicated. This paper proposes to use AlphaZero's ability to achieve superhuman performance in board games within one day of training to assess the effect of a series of small, but consequential rule changes. It analyzes the resulting strategies and sets the stage for broader applications of reinforcement learning to study rule-based systems.

OUTLINE:
0:00 - Intro & Overview
2:30 - Alternate Chess Rules
4:20 - Using AlphaZero to assess rule change outcomes
6:00 - How AlphaZero works
16:40 - Alternate Chess Rules continued
18:50 - Game outcome distributions
31:45 - e4 and Nf3 in classic vs no-castling chess
36:40 - Conclusions & comments

Paper: https://arxiv.org/abs/2009.04374

My Video on AI Economist: https://youtu.be/F5aaXrIMWyU

Abstract:
It is non-trivial to design engaging and balanced sets of game rules. Modern chess has evolved over centuries, but without a similar recourse to history, the consequences of rule changes to game dynamics are difficult to predict. AlphaZero provides an alternative in silico means of game balance assessment. It is a system that can learn near-optimal strategies for any rule set from scratch, without any human supervision, by continually learning from its own experience. In this study we use AlphaZero to creatively explore and design new chess variants. There is growing interest in chess variants like Fischer Random Chess, because of classical chess's voluminous opening theory, the high percentage of draws in professional play, and the non-negligible number of games that end while both players are still in their home preparation. We compare nine other variants that involve atomic changes to the rules of chess. The changes allow for novel strategic and tactical patterns to emerge, while keeping the games close to the original. By learning near-optimal strategies for each variant with AlphaZero, we determine what games between strong human players might look like if these variants were adopted. Qualitatively, several variants are very dynamic. An analytic comparison show that pieces are valued differently between variants, and that some variants are more decisive than classical chess. Our findings demonstrate the rich possibilities that lie beyond the rules of modern chess.

Authors: Nenad Tomašev, Ulrich Paquet, Demis Hassabis, Vladimir Kramnik

Links:
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
Parler: https://parler.com/profile/YannicKilcher
LinkedIn: https://www.linkedin.com/in/yannic-kilcher-488534136/

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
 Hi there! If you play chess, you'll probably recognize the following moves as illegal. In the top row, pawns move two squares at a time while they are not on their home row. In the bottom row you'll see a pawn moving backwards and another one moving sidewards even. So in classical chess these moves are illegal, but there are variants of chess where these moves aren't illegal, where they are actually explicitly part of the rules. These are alternate chess rules and this paper is about exploring those rules. What happens if you implement those rules? How does the gameplay change? And what can we learn for general games? So the paper here is called Assessing Game Balance with AlphaZero, Exploring Alternative Rulesets in Chess by Nenad Tomasev, Ulrich Paquet, Demis Hassabis and Vladimir Kramnik, the former three of DeepMind and the latter was the world chess champion for these eight years depicted. So the paper tries to bring together two different worlds. First it is the chess world. So a lot of this paper is explicitly about the game of chess. If you don't play chess, or if you occasionally play chess like myself, this might not be the most interesting paper, though it contains some really interesting kind of bits. The other world is the reinforcement learning world, which you'll see in the AlphaZero name right here. So the reasoning behind this is the following. Chess is a really, really old game and rules have evolved over time and have sort of consolidated on the rules we have today. But also strategy has evolved over time and lots and lots of thinking and theory has gone into the strategy of chess. And to change the rules around, you can change the rules of chess. However, you can't really assess how the game would be played by humans if the rules were changed, because you don't have a thousand years of the entire humanity studying these new rule sets. And therefore, you're kind of stuck with assessing the games from the perspective of someone who has learned the old rules. But reinforcement learning to the rescue. So consider the following rule changes. No castling. This is a really simple rule change. No castling. Castling is disallowed throughout the game. If you don't know what castling is, castling is like a special move where there is this rook and the king is right here. I don't know how to do the king. And if there's nothing in between, they can sort of swap positions. It's called castling. It's a special move that you can do. And it allows you to bring the king to the outside where the king is safe, and to bring the rook to the inside, where it can potentially cause a lot of damage. So it's a very, very favored move by a lot of players. And no castling, the rule change probably alters the game a lot. Because if you think of the chessboard, kings start about here, they can only move one square at a time. So to get them to safety will require like four or five steps for them, while you have to move everything else out of the way, including the rook that stands here. So players might elect to just leave their kings where they are, but then they can't really open up in the middle as much because that would leave their kings exposed. So it is fair to assume that just introducing this one rule might change the games around quite a bit, how the game is played. But as we said, we don't know. This is from someone who has learned classic chess, and all the grandmasters that we have have played and learned classic chess. So how do we assess this? This paper says that AlphaZero can be used to assess these new rules. So AlphaZero is a reinforcement learning algorithm that can learn these board games very, very quickly in within one day or so. And it can learn them so well, it can beat humans at the game easily. In fact, modern grandmasters and so on use these algorithms in order to learn and to better their play in order to expand their theory, their knowledge of the game, to play better against other humans. So AlphaZero, imagine AlphaZero can solve a game to perfection. What we could do is we could simply give this rule to AlphaZero together with the all the other chess rules, and then let AlphaZero solve the game, give it a day and 50 billion GPUs, solve the game to perfection, and then look at what AlphaZero came up with. Kind of look at the games, how they turn out, and whether or not they are more interesting, less interesting, longer, shorter, and so on. So that's, that's what this paper does. So there's the implicit assumption, which you need to believe in order to believe anything in this paper, is that AlphaZero actually has this ability. There is pretty good evidence that it does because AlphaZero can solve classical chess and Go and Shogi and a bunch of other board games, all with the same hyper parameters. It can solve them such that it is easily at superhuman power. So, but you need to recognize that this is an assumption. So what is AlphaZero? If you don't know what AlphaZero is, AlphaZero is a reinforcement learning algorithm, but not in the kind of base reinforcement learning sense. It is a reinforcement algorithm that has a planner included. What do I mean by this? So if you are in a let's consider the game tic tac toe, so AlphaZero for tic tac toe. In tic tac toe, you have this board, and you have a situation where let's say you play, your opponent plays this, and now you're tasked of playing something. You wonder, should I play maybe here or here or here? Where should I play? So what you can do is you can train a reinforcement learning algorithm. You can do Q learning, whatnot. Okay, that will maybe work. What's better to do is you can plan. So in planning, what you want to do is you want to build a tree of possibilities. So we're going to consider all your possibilities. And in this case, you have eight possibilities. So we want to consider all the eight possibilities. And I'm going to draw just some of them. So up here, you're going to consider the possibility that you place here. And here, you're going to consider the possibility that you place in a different spot right here. Okay. And you can see how this goes. So if you want to plan, and here you have your opponent has seven possibilities. And here your opponent also has seven possibilities and so on. So you get this entire tree of play. But if you could do that, and if you could do that to the end, then you could easily simply choose the path here where you win. Okay, where no matter what your opponent does, you win. You can find such a path if it is possible at all to win, which it is not in tic tac toe, right? If everyone plays optimally, it results in a draw. But let's say you could win, you could choose the path that gives you the best result. And that's it. There's no learning involved. Okay. So Alpha zero works with a planner, and planners usually construct a tree. So in an abstract way, you're in a situation, and you consider all your options. And with all your options, you consider again, all your options and so on. And you do a tree search. Now this tree in tic tac toe, it's already huge, as you can see, in something like chess, it is way, way huger. Okay. And therefore it's not possible to actually search the entire tree, because you need to consider every single possible future situation from the board position where you're in, right? This here is the board position where you're in. And this is the future, the entire future of the game. So every single possibility. So Alpha zero uses this thing called a Monte Carlo tree search. It has several components. So its first component, and they right here, they have a description, and it's very short. Alpha zero, this is Alpha zero. This is what it does. It's like this is almost comically short. So what you do is you put your state so s is your state, okay, s is it's the board as you have it right now. Okay, this here, that's this is s. Okay, you put this into a neural network, and the neural network gives you two things. First of all, it gives you P, and then you put this into a network, and, and V. So that's the second thing. So V will simply give you a number V will tell you that this thing right here is about a plus point five, maybe. So it says. So plus one is winning and minus one is losing. And it is this is called the value. So maybe it says, well, this position, I'm going to expect you to win roughly 75% of the time, right, which in expectation would be a value of positive 0.5 here, because 75% of the time you win and the rest you lose, let's say there is no draw in tic tac toe. So there's this value function. And the second thing is this P and the P is a policy function. So the P will and I've drawn this a little bit, maybe not super, super duper too large, but the P will tell you for every possible move you could make, which one should you consider even, okay, so it maybe it assigns this here, a point three, and this here, a point four. But this here is like a point 0001, and so on. So for every possible move that you could do, it will assign a number. And it's a distribution. So these numbers add up to one, but that's not important. It tells you which moves you should even consider going forward, right. So P, in this case is a distribution over the next moves. And with those two things together, we can reduce our tree search quite a bit. So now, instead of expanding all the tree, let's go back to the tree right here, you can ask your P, hey P, which one of these three should I even consider? And maybe P says you should only consider those two. Okay. And then you go down. And again, you ask your P, hey P, which one should you consider? And P maybe says, well, here, you should consider those two here, you should only consider that this one. And this tree over here, we've already discarded this from the beginning. Okay. So this P right here, it guides your search, it tells you at each point, which moves should you consider? And this, as you can see, reduces your tree dramatically. In fact, what AlphaZero does is it simply says you have one second of time. Now expand as much as you can in this tree, given this one second of time budget. And the second thing is the value. So what you would have to do expanding the tree is always to go to the end, right? So you always go to the end, where at the end, you have a fully filled board, I don't know here, x, so you consider every possible situation, okay, here, maybe this, this player wins, as you can see, you always have to go to the end. But in our case, we don't want to always go to the end, we'd rather explore more into like more branches than always go to the end. And this is where the value comes in. So at some point, you simply say now I'm deep enough. And now I'm going to ask my value V that there are slight differences with respect to AlphaGo and AlphaZero and so on. But they all have in common that they estimate the value of the intermediate nodes using this V model from over here. I have V as V was green. So they use this V model from over here to estimate at a certain depth. So V learns to look into the future. So everything that can happen from here, and it estimates and it says, well, from here, you maybe have a, you know, a point five value, or maybe a negative point seven, and so on. So V learns to assign these values to situations to states, which are these nodes right here, and P learns to suggest things to expand, right, that's AlphaZero. And then at the end, if you've expanded the tree enough and estimated, well, then you have a pretty good idea what's going to happen in each of the branches that you considered, right, in each of these branches, you look into the future from here, you look into the future here, look into the future by doing this PV play. And after one second after you've done, you know, a couple of hundred or 1000 or however many looks into the future, then you have a pretty good idea for each of the top level actions, what's going to happen in the future. And you can simply pick the one that has the best future for you, according to your own model. So that's what AlphaZero does. Note, so this is how you combine planning and neural networks, you want to do planning, but you can't because you can only go so deep. So you use neural networks to first of all, reduce the number of branches you consider, because the neural network will tell you which ones are worthy to even look at. And second of all, you don't always have to plan to the end because you can simply ask your neural network, how much an intermediate state is worth in expectation. And this turns out to be pretty good. Why don't we do this for every single problem? Well, we do for this, we do need a simulator. So you may recognize that right here, I said we consider all the possible actions that we have. And for each action, we know exactly what's going to happen. This is only possible like in a board game. It's not even possible in like a board game where you have a die to roll, or a card to draw, anything that is random. There is a way to include this right here. But in this simple formulation, we need to know exactly with 100% certainty, what is going to happen if we take a particular action. So this is only really applicable for the types of full information board games, where we can write simulators that are pretty fast, right. And even then, even though chess, you know, has lots of available actions and complications, it's nowhere near the complexity of like a, let's say a modern video game, or even or the real world is completely out of scope for now for these types of things. Alright, so that was AlphaGo, sorry, AlphaZero, which builds on AlphaGo, of course. And the rules of chess that we're going to consider using AlphaZero are the following. So there's no castling, no castling for 10 moves. Pawns can only move by one square. Forcing a stalemate is a win rather than a draw. So you may know this in chess, if you do not checkmate the opponent's king, but only put the king in a situation where it cannot move. That's called that's considered a draw. And I think even in the chess community, some people want to consider this a win. There's torpedo, where pawns can move by one or two squares anywhere on the board. And semi torpedo, where it's the same but only from the second and the third rank. Pawn back where pawns can move backwards and pawn sideways where pawns can move laterally by one squares, but captures are unchanged diagonally upwards. And there is self capture, where it's possible to capture one's own pieces. So there are, you know, slight, slight details here with respect to the 50 move rule and so on. But if you if you don't play chess, simply consider these are changes, minor in a lot of cases, minor changes to the chess rules that make the new rules either a superset or a subset of the original rules, but they are going to have quite some changes in for the play. And we're going to look at what happens. So the entire research setup, as you've seen, it's AlphaZero applied to these new rule sets, and under the assumption that AlphaZero will solve these will become master at these games, which we can't verify, we can verify in chess because right AlphaZero can beat people that have trained chess for all their life, we can't verify it here. So again, this is an assumption. So the rule set again, this is an assumption. So the first thing I want to look at here, and this is going to play a little bit into my criticism of this paper, is a pretty cool paper, but I do have some concerns right here is the following the following charts. So they do, we don't consider how you train AlphaZero, let's just say you can train it, you know, to whatever pretty good performance. Here is how they evaluate. So they evaluate for each variant, they do 10,000 games played at one second per move for each different chess variant. So if you remember, as we do our tree search, right, we expand the tree according to our P and we estimate the values according to our V. And we do this for one second in this first thing. So in one second, maybe this here is the tree. So we have some sort of an understanding of what's going to happen in the future. You can imagine, if we have more time, then we can expand this tree more and get a much more accurate picture of what happens in the future. Okay, so they do 10,000 games at one second per move. But they also in addition to 1000 games played at one minute per move. So there's 60 times more time and you can imagine that will add quite a number of nodes here. And you know, if if your P and V would be perfect, then it wouldn't matter as much how much time you have as long as you sort of have enough time. But since they're not going to be perfect, since they're only neural networks, they're not God or Schmidhuber. They cannot accurately, extremely accurately predict the future. So this planning, the more you plan, the more you actually look into the future, the bigger your tree becomes, the better moves you make. So on the left, you see the distributions of wins, losses, and draws for one second per move. And on the right for one minute per move. So both white and black pieces here are played by AlphaZero. So it's not AlphaZero against something else. This is playing against itself. And you can see in in classic chess, it's it's quite, it's quite saddening actually, that this game which is so famous, you can see that in of 10,000 plays, 8,820 end in a draw, which means that if both players are super duper good, and, and play, you know, play against each other, it most likely is going to be a draw. And this I think is the criticism, even in human chess is that it's not really a decisive game in that it ends a lot of times in a draw. So one of the motivations here would be, can we find a rule set that is maybe more decisive? So that's one of the investigations they do in the paper. But you can see that there are but you can see that there are actually so if you consider this torpedo chess right here, there it is more decisive, as you can see, in more times, either white or black wins right here. And there are others which are even less decisive, like pawn back. So when pawns can move back, then players may just camp, they like move a pawn forward and move it back again. And that will lead to a lot of closed plays and so on. Whereas torpedo makes you move much faster, you can advance your pawns much faster. And that will probably lead to the end much faster. So if you consider this on the right. So what changed the rules didn't change alpha zero didn't change, it simply changed that we now let alpha zero think for longer. And you can see that the decisiveness reduces dramatically. So whereas 88% resulted in a draw with one second per move, now 98% result in a draw with one minute per move. And this is a trend throughout these games. And that's also what they say in the text, it is to assume that if you let alpha zero plan for even longer, that this trend will continue. And ultimately, whatever rule set you make, the result is going to be a draw. If two, two, let's say perfect players play against each other, which is a bit, which is a bit saddening, right? Because yeah, that ultimately, ultimately means that all of these rules aren't decisive. It's only they're only decisive due to the fact that either one or the other players is way better or that in general that they are not they are not perfect. Which is an appeal of a game, but there are certainly games that are decisive, even though both players are pretty high level. I mean, think of every, every competitive video game. So yes, so that's a bit of my criticism, all of this, all of this needs to be analyzed in the background that what's actually happening here is that we're dealing with imperfect decision making due to a limit in resources. Okay. And this assumption now is already a little bit invalid, right? The assumption we made at the beginning, why I pointed this out, is that we're dealing with a game that is not really solid, right? The assumption we made at the beginning, why I pointed this out is that AlphaZero can solve these games, let's say to perfection. And here, when we analyze the decisiveness and so on, it seems to be purely or largely a factor of how much time AlphaZero has to spend on these two things. To me, they don't really go together, because we don't know if for a different rule set, you know, the training is harder, or might take longer and so on, or that this exact one second makes a difference or not. It's just, there are so many variables here. And when you're dealing with, let's say imperfect systems that are not trained to the end or potential, you're always dealing with the fact that you stopped each thing at some intermediate point. And that intermediate, where that intermediate point is can influence the results drastically. Now here, it seems at least the ordering isn't changed by much. But yeah, this is one, let's say one criticism. The other criticism here that I would have, again, is the fact that if you consider something like Torpedo, where you can move much, much faster, then yes, of course, let's say, I don't know, is it more interesting? That's the question right here. So they look at a lot of things like decisiveness, diversity, and so on. But the question is, is it more or less interesting to play? And I think that's what humans are really after. And they're sort of trying to find proxies to this. I would argue if you play something like Torpedo, the games may be much faster. And so you get to the end faster, but also maybe it might not be as interesting, even though it's faster, because the complexity is less. And with respect to the decisiveness here, so if you have a game that's faster, you also need to take this into account. Because here is another thing that is sort of an arbitrary choice. As moves are determined in a deterministic fashion, given the same condition, diversity was enforced by sampling the first 20 plays in each game proportional to their MCTS visit counts. So what does that mean? That means that if you run AlphaZero on the same situation, on the same tree, sorry, on the same board position, it will always come up with the same move, except for parallelism, inconsistencies, and so on. But it will in a lot of times, it will come up with the same move. So how do you play 10,000 games? Because you can just play one game, because each game will be the same, because you simply tell AlphaZero, give me your best move, right? So it will just play its optimal strategy. And all the games will be exactly the same. So there's no reason why these should come out different. So they enforce diversity by saying, okay, okay, in the first 20 moves of a game, we don't actually take the best move, right? Usually you have you have this distribution. At the end of the tree search, you have a distribution where you say, okay, this move right here is clearly the best move, I'm going to play this. However, if this is one of the first 20 moves of the game, they say no, we need a bit of diversity. So we're going to sample according to this distribution rather than just play the best one. Now this number 20, it's just sort of decided arbitrary, right? And if you consider something like Torpedo, it's a faster game. So you're faster in opening faster, making you faster to the end game, maybe, even though they say, well, the game length isn't affected this much, it could just be that you're faster in a situation where you're kind of forced to do certain moves. And maybe the difference in decisiveness here is simply a result of the combination of the faster moves in Torpedo together with this, the fact that they just keep the 20 plays for each game. Again, this is something that you need to consider when analyzing these results right here. And there are a number of these choices right here, like the one second or one minute per move, we sample for the first 20 plays before we play the max move that where I think the results of the study right here, they have rather limited interpretability, if you ask me, because of these choices. Now, of course, they're still the results are quite plausible, believable. And the idea is really cool to explore these rule sets. But this was this is just my criticism right here. So we'll go through the rest of the results pretty, pretty quickly. Because a lot of people aren't chess enthusiasts. And we'll just pick out kind of the core messages that the paper is trying to get across. So here the table again, with respect to decisiveness, and you can see even for so for classic chess, it's a white has a 50. This is the empirical score for white under different game conditions. So 50.8% means most of the time it's a draw. So white wins with a probability of 50.8. Most of the time, it's a draw. And you see even like the most decisive variant torpedo right here is a 54% only. So they they analyze different defenses and how the decisiveness is with respect to different defenses that are not really popular under classical chess. And the results are interesting if you play chess. But I would say they're rather, they're kind of aha, okay, if you do not play chess, because they consider individual moves and so on. What is an interesting part is this right here where they look at they look at one move that in classical chess, so E4 is a very, very popular opening, where you move your E pawn twice for white. And NF3 is not a super popular opening. And here they compare this in classic chess and in no castling chess. This thing right here is a histogram. And the histogram shows you the log probability of opening sequences when you play the individual moves. So what does this mean right here? If you play E4, then the distribution is something like this, which means that you have some sequences that have no entropy at all, which means that once you play E4, and maybe one move more, then it's almost it's almost determined what you have to do according to Alpha Zero, you have like no choice except play these few next moves. However, if you play NF3, then Alpha Zero says, look, this distribution is much more to the right, which means that you have a lot more options here. Now, again, this could be because the move is actually less decisive because the move leads to more balanced, more interesting situations where you can continue. However, you know, with many choices, it could also be because it's simply Alpha Zero simply doesn't know as well what to do because it leads to more complicated games, and you get to give each move one minute to evaluate Alpha Zero might just not be as good in those situations because it leads to more complicated situations. If it could search for longer, maybe this distribution would shift over here just as well. Again, we don't know because you only give this one second or one minute each time for both. And again, this goes under the assumption of Alpha Zero is this perfect player. However, back to what they want to say here, if you do this in no castling chess, you can see that this spike right here are all the these Berlin defense variants and castling this OO right here is a big part of that line. If you do this in no castling chess, you can see that these two moves, now the histograms overlap much more, which means that and in fact, you can see in the in this number of possible moves right here that they come closer together. So not only does the blue shift to the right, the orange actually shifts to the left. And it basically means that whether you open with E4 or Knight f3, you are going to have about the same complexity of game, the same number of moves available to you going from there, as you can see right here, these lines are the moves available for white and black under the different rule sets. So in E4, here, especially as black, you do not have many moves available as white a little bit more, but also not more. Whereas in no castling you do so, again, small rule change, big effect on the possible moves that you can consider. And this is the type of information that you would want to have when you design a game. And they allude to this also at the end here in their conclusions. So the last thing is they also compare the material values of the pieces here in the different rule sets, as you might imagine. So some pieces become much more or less valuable, I find it particularly interesting that if you do something like pawn sideways, or then where the pawns are much more powerful, of course, all the other pieces drop in value. Again, these results are pretty plausible. So I don't want to trash the paper right here. Because it seems like, it seems like the results are, as I say, plausible, and can give some cool insights. So the chess master also gives his opinions on these different strategies that AlphaZero comes up with for the different rules. And let's go through the conclusions quickly. So they say, assessing the consequence of rule changes in the game design process demonstrated on chess, where we've trained AlphaZero to evaluate nine different variants representing atomic changes to the rules of a game. Training AlphaZero model on these rules changes helps us effectively simulate decades of human play in a matter of hours and answer the what if question, what the play would potentially look like under developed theory in each chess variant. We believe that a similar approach could be used for auto balancing game mechanics in other types of games, including computer games, in cases when a sufficiently performant reinforcement learning system is available. And yes, this is, I mean, this the application here would be for something like this, if you design a new game, then you want to know what you have some choice with how you can make the rules. And you don't want to let humans become really good at each of the rules and then compare, you can simply give this to the algorithm, and the algorithm will tell you what kind of plays result from each rule set. And then you can choose the one that you find most interesting or most maybe commercially viable and whatnot. I actually see this much, I see this bigger than just games. And this alludes a bit to the Salesforce paper on this AI economist, I think we can let AI, you know, get tell us what happens if we change, for example, things like tax policy, or any any sort of policy, I know, humanity is very complex to model and so on. And you're never going to have a perfect simulator, which probably makes Alpha Zero not good. But in limited situations, like maybe also stock trading rules, and so on, you could definitely have situations where the rule set is too complicated to solve analytically. But you could give it to an RL algorithm and see what happens and whether or not you like the outcome and whether or not there are any obvious exploits that you did not see. So this, I find, you know, pretty, it's a pretty cool approach. And we should think of this in the future as we build systems that have rules in whatever capacity be this games or policy. So the they say, okay, yada, yada, yada, we showed that there are several chess variants among those considering the study that are even more decisive than classical chess, meaning torpedo chess, semi-tropical chess, no castling chess and stalemate equals win chess. We quantified arising diversity of opening play and the intersection of opening trees between chess variations, showing how different the opening theory is for each of the rule changes. Yeah, they again, this diversity of opening play, it really rests on this assumption that Alpha Zero is a good player and an sort of an equally good player in all of these variants, right? Because if it's worse in a variant, it might not be as sure about the moves and that would just look like, oh, you have many possibilities, but in fact, Alpha Zero is just worse at it. And it doesn't know. So they also look at the intersection of opening trees, like if you change a rule, how does this change the kind of how does this change the the initial game? So a lot of these grandmasters, they learn by heart all of these opening trees, the initial moves of a game, how much would they have to relearn? There is a negative correlation between the overall opening diversity and decisiveness, as decisive variants likely require more precise play with fewer plausible choices per move. Again, this is one view, right? The other view is that there are rule sets that are just make it into a harder game. And then Alpha Zero, given the same amount of compute is a worse player. And therefore, it can't play as well. Therefore, the games are less decisive. And also, the opening diversity is higher because it doesn't know. The game could be as decisive. It might just be an effect of Alpha Zero. For each of the chess variants, we estimated yada yada. Okay. No Castling Chess being the first variant that we analyzed has already been tried in experimental Blitz Grandmaster Tournament in Chennai, as well as a couple of longer Grandmaster games. Our assessment suggests that several of the assessed chess variants might be quite appealing to interested players. And we hope that this study will prove to be a valuable resource for the wider chess community. I don't know, is the chess community flourishing or going under recently? Because it seems to me like once a game is solved that hard by computers, I mean, it's still fun. But yeah, I guess Counter-Strike is also solved by bots real hard. It's still impressive when humans play or so. Yeah, I don't know. All of this is, again, if you're into chess, look into this paper, they have a lot of really interesting results that are not interesting to go into for the general community. But I believe this should give you a good impression of what you could do if you design a system that is built on rules. And I hope you enjoyed this. If you liked it, leave a comment, tell me what you think, and I'll see you next time. Bye bye.
 Hi there! If you play chess, you'll probably recognize the following moves as illegal.
 In the top row, pawns move two squares at a time while they are not on their home row. In the bottom
 row you'll see a pawn moving backwards and another one moving sidewards even. So in classical chess
 these moves are illegal, but there are variants of chess where these moves aren't illegal, where they
 are actually explicitly part of the rules. These are alternate chess rules and this paper is about
 exploring those rules. What happens if you implement those rules? How does the gameplay change?
 And what can we learn for general games? So the paper here is called
 Assessing Game Balance with AlphaZero, Exploring Alternative Rulesets in Chess by Nenad Tomasev,
 Ulrich Paquet, Demis Hassabis and Vladimir Kramnik, the former three of DeepMind and the latter
 was the world chess champion for these eight years depicted. So the paper tries to bring together
 two different worlds. First it is the chess world. So a lot of this paper is explicitly about the game
 of chess. If you don't play chess, or if you occasionally play chess like myself, this might
 not be the most interesting paper, though it contains some really interesting kind of bits.
 The other world is the reinforcement learning world, which you'll see in the AlphaZero name right here.
 So the reasoning behind this is the following. Chess is a really, really old game and rules have
 evolved over time and have sort of consolidated on the rules we have today. But also strategy has
 evolved over time and lots and lots of thinking and theory has gone into the strategy of chess.
 And to change the rules around, you can change the rules of chess. However, you can't really
 assess how the game would be played by humans if the rules were changed, because you don't have a
 thousand years of the entire humanity studying these new rule sets. And therefore, you're kind
 of stuck with assessing the games from the perspective of someone who has learned the old
 rules. But reinforcement learning to the rescue. So consider the following rule changes. No castling.
 This is a really simple rule change. No castling. Castling is disallowed throughout the game. If you
 don't know what castling is, castling is like a special move where there is this rook and the
 king is right here. I don't know how to do the king. And if there's nothing in between, they can
 sort of swap positions. It's called castling. It's a special move that you can do. And it allows you
 to bring the king to the outside where the king is safe, and to bring the rook to the inside,
 where it can potentially cause a lot of damage. So it's a very, very favored move by a lot of
 players. And no castling, the rule change probably alters the game a lot. Because if you think of the
 chessboard, kings start about here, they can only move one square at a time. So to get them to
 safety will require like four or five steps for them, while you have to move everything else out
 of the way, including the rook that stands here. So players might elect to just leave their kings
 where they are, but then they can't really open up in the middle as much because that would leave
 their kings exposed. So it is fair to assume that just introducing this one rule might change the
 games around quite a bit, how the game is played. But as we said, we don't know. This is from someone
 who has learned classic chess, and all the grandmasters that we have have played and learned
 classic chess. So how do we assess this? This paper says that AlphaZero can be used to assess
 these new rules. So AlphaZero is a reinforcement learning algorithm that can learn these board
 games very, very quickly in within one day or so. And it can learn them so well, it can beat humans
 at the game easily. In fact, modern grandmasters and so on use these algorithms in order to learn
 and to better their play in order to expand their theory, their knowledge of the game,
 to play better against other humans. So AlphaZero, imagine AlphaZero can solve a game to
 perfection. What we could do is we could simply give this rule to AlphaZero together with the all
 the other chess rules, and then let AlphaZero solve the game, give it a day and 50 billion GPUs,
 solve the game to perfection, and then look at what AlphaZero came up with. Kind of look at the
 games, how they turn out, and whether or not they are more interesting, less interesting, longer,
 shorter, and so on. So that's, that's what this paper does. So there's the implicit assumption,
 which you need to believe in order to believe anything in this paper, is that AlphaZero actually
 has this ability. There is pretty good evidence that it does because AlphaZero can solve classical
 chess and Go and Shogi and a bunch of other board games, all with the same hyper parameters.
 It can solve them such that it is easily at superhuman power. So, but you need to recognize
 that this is an assumption. So what is AlphaZero? If you don't know what AlphaZero is, AlphaZero
 is a reinforcement learning algorithm, but not in the kind of base reinforcement learning sense. It
 is a reinforcement algorithm that has a planner included. What do I mean by this? So if you are
 in a let's consider the game tic tac toe, so AlphaZero for tic tac toe. In tic tac toe,
 you have this board, and you have a situation where let's say you play, your opponent plays this,
 and now you're tasked of playing something. You wonder, should I play maybe here or here or here?
 Where should I play? So what you can do is you can train a reinforcement learning algorithm. You can
 do Q learning, whatnot. Okay, that will maybe work. What's better to do is you can plan. So in planning,
 what you want to do is you want to build a tree of possibilities. So we're going to consider all
 your possibilities. And in this case, you have eight possibilities. So we want to consider all
 the eight possibilities. And I'm going to draw just some of them. So up here, you're going to consider
 the possibility that you place here. And here, you're going to consider the possibility that you
 place in a different spot right here. Okay. And you can see how this goes. So if you want to plan,
 and here you have your opponent has seven possibilities. And here your opponent also
 has seven possibilities and so on. So you get this entire tree of play. But if you could do that,
 and if you could do that to the end, then you could easily simply choose the path here where
 you win. Okay, where no matter what your opponent does, you win. You can find such a path if it is
 possible at all to win, which it is not in tic tac toe, right? If everyone plays optimally,
 it results in a draw. But let's say you could win, you could choose the path that gives you the best
 result. And that's it. There's no learning involved. Okay. So Alpha zero works with a planner,
 and planners usually construct a tree. So in an abstract way, you're in a situation,
 and you consider all your options. And with all your options, you consider again, all your options
 and so on. And you do a tree search. Now this tree in tic tac toe, it's already huge, as you can see,
 in something like chess, it is way, way huger. Okay. And therefore it's not possible to actually
 search the entire tree, because you need to consider every single possible future situation
 from the board position where you're in, right? This here is the board position where you're in.
 And this is the future, the entire future of the game. So every single possibility.
 So Alpha zero uses this thing called a Monte Carlo tree search. It has several components.
 So its first component, and they right here, they have a description, and it's very short.
 Alpha zero, this is Alpha zero. This is what it does. It's like this is almost comically short.
 So what you do is you put your state so s is your state, okay, s is it's the board as you have it
 right now. Okay, this here, that's this is s. Okay, you put this into a neural network, and the
 neural network gives you two things. First of all, it gives you P, and then you put this into a
 network, and, and V. So that's the second thing. So V will simply give you a number V will tell you
 that this thing right here is about a plus point five, maybe. So it says. So plus one is winning
 and minus one is losing. And it is this is called the value. So maybe it says, well, this position,
 I'm going to expect you to win roughly 75% of the time, right, which in expectation would be a value
 of positive 0.5 here, because 75% of the time you win and the rest you lose, let's say there is no
 draw in tic tac toe. So there's this value function. And the second thing is this P and the P is a
 policy function. So the P will and I've drawn this a little bit, maybe not super, super duper too large,
 but the P will tell you for every possible move you could make, which one should you consider even,
 okay, so it maybe it assigns this here, a point three, and this here, a point four. But this here
 is like a point 0001, and so on. So for every possible move that you could do, it will assign
 a number. And it's a distribution. So these numbers add up to one, but that's not important. It
 tells you which moves you should even consider going forward, right. So P, in this case is a
 distribution over the next moves. And with those two things together, we can reduce our tree search
 quite a bit. So now, instead of expanding all the tree, let's go back to the tree right here,
 you can ask your P, hey P, which one of these three should I even consider? And maybe P says
 you should only consider those two. Okay. And then you go down. And again, you ask your P, hey P,
 which one should you consider? And P maybe says, well, here, you should consider those two here,
 you should only consider that this one. And this tree over here, we've already discarded this from
 the beginning. Okay. So this P right here, it guides your search, it tells you at each point,
 which moves should you consider? And this, as you can see, reduces your tree dramatically. In fact,
 what AlphaZero does is it simply says you have one second of time. Now expand as much as you can
 in this tree, given this one second of time budget. And the second thing is the value. So
 what you would have to do expanding the tree is always to go to the end, right? So you always go
 to the end, where at the end, you have a fully filled board, I don't know here, x, so you consider
 every possible situation, okay, here, maybe this, this player wins, as you can see,
 you always have to go to the end. But in our case, we don't want to always go to the end,
 we'd rather explore more into like more branches than always go to the end. And this is where the
 value comes in. So at some point, you simply say now I'm deep enough. And now I'm going to ask my
 value V that there are slight differences with respect to AlphaGo and AlphaZero and so on. But
 they all have in common that they estimate the value of the intermediate nodes using this V
 model from over here. I have V as V was green. So they use this V model from over here to estimate
 at a certain depth. So V learns to look into the future. So everything that can happen from here,
 and it estimates and it says, well, from here, you maybe have a, you know, a point five value,
 or maybe a negative point seven, and so on. So V learns to assign these values to situations to
 states, which are these nodes right here, and P learns to suggest things to expand, right, that's
 AlphaZero. And then at the end, if you've expanded the tree enough and estimated, well, then you have
 a pretty good idea what's going to happen in each of the branches that you considered, right, in each
 of these branches, you look into the future from here, you look into the future here, look into the
 future by doing this PV play. And after one second after you've done, you know, a couple of
 hundred or 1000 or however many looks into the future, then you have a pretty good idea for each
 of the top level actions, what's going to happen in the future. And you can simply pick the one
 that has the best future for you, according to your own model. So that's what AlphaZero does. Note,
 so this is how you combine planning and neural networks, you want to do planning, but you can't
 because you can only go so deep. So you use neural networks to first of all, reduce the number of
 branches you consider, because the neural network will tell you which ones are worthy to even look
 at. And second of all, you don't always have to plan to the end because you can simply ask your
 neural network, how much an intermediate state is worth in expectation. And this turns out to be
 pretty good. Why don't we do this for every single problem? Well, we do for this, we do need a
 simulator. So you may recognize that right here, I said we consider all the possible actions that we
 have. And for each action, we know exactly what's going to happen. This is only possible like in a
 board game. It's not even possible in like a board game where you have a die to roll, or a card to
 draw, anything that is random. There is a way to include this right here. But in this simple
 formulation, we need to know exactly with 100% certainty, what is going to happen if we take a
 particular action. So this is only really applicable for the types of full information board games,
 where we can write simulators that are pretty fast, right. And even then, even though chess,
 you know, has lots of available actions and complications, it's nowhere near the complexity
 of like a, let's say a modern video game, or even or the real world is completely out of scope
 for now for these types of things. Alright, so that was AlphaGo, sorry, AlphaZero, which builds on
 AlphaGo, of course. And the rules of chess that we're going to consider using AlphaZero are the
 following. So there's no castling, no castling for 10 moves. Pawns can only move by one square.
 Forcing a stalemate is a win rather than a draw. So you may know this in chess, if you do not
 checkmate the opponent's king, but only put the king in a situation where it cannot move.
 That's called that's considered a draw. And I think even in the chess community, some people
 want to consider this a win. There's torpedo, where pawns can move by one or two squares anywhere
 on the board. And semi torpedo, where it's the same but only from the second and the third rank.
 Pawn back where pawns can move backwards and pawn sideways where pawns can move
 laterally by one squares, but captures are unchanged diagonally upwards. And there is
 self capture, where it's possible to capture one's own pieces. So there are, you know, slight,
 slight details here with respect to the 50 move rule and so on. But if you if you don't play chess,
 simply consider these are changes, minor in a lot of cases, minor changes to the chess rules
 that make the new rules either a superset or a subset of the original rules, but they are going
 to have quite some changes in for the play. And we're going to look at what happens. So
 the entire research setup, as you've seen, it's AlphaZero applied to these new rule sets, and
 under the assumption that AlphaZero will solve these will become master at these games, which
 we can't verify, we can verify in chess because right AlphaZero can beat people that have trained
 chess for all their life, we can't verify it here. So again, this is an assumption. So the rule set
 again, this is an assumption. So the first thing I want to look at here, and this is going to
 play a little bit into my criticism of this paper, is a pretty cool paper, but I do have some
 concerns right here is the following the following charts. So they do, we don't consider how you train
 AlphaZero, let's just say you can train it, you know, to whatever pretty good performance. Here
 is how they evaluate. So they evaluate for each variant, they do 10,000 games played at one second
 per move for each different chess variant. So if you remember, as we do our tree search, right,
 we expand the tree according to our P and we estimate the values according to our V. And we
 do this for one second in this first thing. So in one second, maybe this here is the tree. So we have
 some sort of an understanding of what's going to happen in the future. You can imagine, if we have
 more time, then we can expand this tree more and get a much more accurate picture of what happens
 in the future. Okay, so they do 10,000 games at one second per move. But they also in addition to
 1000 games played at one minute per move. So there's 60 times more time and you can imagine
 that will add quite a number of nodes here. And you know, if if your P and V would be perfect,
 then it wouldn't matter as much how much time you have as long as you sort of have enough time.
 But since they're not going to be perfect, since they're only neural networks, they're not God or
 Schmidhuber. They cannot accurately, extremely accurately predict the future. So this planning,
 the more you plan, the more you actually look into the future, the bigger your tree becomes,
 the better moves you make. So on the left, you see the distributions of wins, losses, and draws
 for one second per move. And on the right for one minute per move. So both white and black pieces
 here are played by AlphaZero. So it's not AlphaZero against something else. This is playing against
 itself. And you can see in in classic chess, it's it's quite, it's quite saddening actually,
 that this game which is so famous, you can see that in of 10,000 plays, 8,820 end in a draw,
 which means that if both players are super duper good, and, and play, you know, play
 against each other, it most likely is going to be a draw. And this I think is the criticism,
 even in human chess is that it's not really a decisive game in that it ends a lot of times
 in a draw. So one of the motivations here would be, can we find a rule set that is maybe more
 decisive? So that's one of the investigations they do in the paper. But you can see that there are
 but you can see that there are actually so if you consider this torpedo chess right here,
 there it is more decisive, as you can see, in more times, either white or black wins right here.
 And there are others which are even less decisive, like pawn back. So when pawns can move back, then
 players may just camp, they like move a pawn forward and move it back again.
 And that will lead to a lot of closed plays and so on. Whereas torpedo makes you move much faster,
 you can advance your pawns much faster. And that will probably lead to the end much faster. So if
 you consider this on the right. So what changed the rules didn't change alpha zero didn't change,
 it simply changed that we now let alpha zero think for longer. And you can see that the decisiveness
 reduces dramatically. So whereas 88% resulted in a draw with one second per move, now 98%
 result in a draw with one minute per move. And this is a trend throughout these games. And that's
 also what they say in the text, it is to assume that if you let alpha zero plan for even longer,
 that this trend will continue. And ultimately, whatever rule set you make, the result is going
 to be a draw. If two, two, let's say perfect players play against each other, which is a bit,
 which is a bit saddening, right? Because yeah, that ultimately, ultimately means that
 all of these rules aren't decisive. It's only they're only decisive due to the fact that either
 one or the other players is way better or that in general that they are not they are not perfect.
 Which is an appeal of a game, but there are certainly games that are decisive, even though
 both players are pretty high level. I mean, think of every, every competitive video game.
 So yes, so that's a bit of my criticism, all of this, all of this needs to be analyzed in
 the background that what's actually happening here is that we're dealing with imperfect decision
 making due to a limit in resources. Okay. And this assumption now is already a little bit invalid,
 right? The assumption we made at the beginning, why I pointed this out, is that we're dealing
 with a game that is not really solid, right? The assumption we made at the beginning, why I pointed
 this out is that AlphaZero can solve these games, let's say to perfection. And here, when we analyze
 the decisiveness and so on, it seems to be purely or largely a factor of how much time AlphaZero has
 to spend on these two things. To me, they don't really go together, because we don't know if for
 a different rule set, you know, the training is harder, or might take longer and so on, or that
 this exact one second makes a difference or not. It's just, there are so many variables here. And
 when you're dealing with, let's say imperfect systems that are not trained to the end or
 potential, you're always dealing with the fact that you stopped each thing at some intermediate point.
 And that intermediate, where that intermediate point is can influence the results drastically.
 Now here, it seems at least the ordering isn't changed by much. But yeah, this is one, let's say
 one criticism. The other criticism here that I would have, again, is the fact that if you consider
 something like Torpedo, where you can move much, much faster, then yes, of course, let's say,
 I don't know, is it more interesting? That's the question right here. So they look at a lot of
 things like decisiveness, diversity, and so on. But the question is, is it more or less interesting
 to play? And I think that's what humans are really after. And they're sort of trying to
 find proxies to this. I would argue if you play something like Torpedo, the games may be much
 faster. And so you get to the end faster, but also maybe it might not be as interesting, even though
 it's faster, because the complexity is less. And with respect to the decisiveness here, so if you
 have a game that's faster, you also need to take this into account. Because here is another thing
 that is sort of an arbitrary choice. As moves are determined in a deterministic fashion, given the
 same condition, diversity was enforced by sampling the first 20 plays in each game proportional to
 their MCTS visit counts. So what does that mean? That means that if you run AlphaZero on the same
 situation, on the same tree, sorry, on the same board position, it will always come up with the
 same move, except for parallelism, inconsistencies, and so on. But it will in a lot of times, it will
 come up with the same move. So how do you play 10,000 games? Because you can just play one game,
 because each game will be the same, because you simply tell AlphaZero, give me your best move,
 right? So it will just play its optimal strategy. And all the games will be exactly the same. So
 there's no reason why these should come out different. So they enforce diversity by saying,
 okay, okay, in the first 20 moves of a game, we don't actually take the best move, right?
 Usually you have you have this distribution. At the end of the tree search, you have a distribution
 where you say, okay, this move right here is clearly the best move, I'm going to play this.
 However, if this is one of the first 20 moves of the game, they say no, we need a bit of diversity.
 So we're going to sample according to this distribution rather than just play the best one.
 Now this number 20, it's just sort of decided arbitrary, right? And if you consider something
 like Torpedo, it's a faster game. So you're faster in opening faster, making you faster to the end
 game, maybe, even though they say, well, the game length isn't affected this much, it could just be
 that you're faster in a situation where you're kind of forced to do certain moves. And maybe
 the difference in decisiveness here is simply a result of the combination of the faster moves
 in Torpedo together with this, the fact that they just keep the 20 plays for each game. Again,
 this is something that you need to consider when analyzing these results right here. And
 there are a number of these choices right here, like the one second or one minute per move,
 we sample for the first 20 plays before we play the max move that where I think the results of
 the study right here, they have rather limited interpretability, if you ask me, because of these
 choices. Now, of course, they're still the results are quite plausible, believable. And the idea is
 really cool to explore these rule sets. But this was this is just my criticism right here. So we'll
 go through the rest of the results pretty, pretty quickly. Because a lot of people aren't chess
 enthusiasts. And we'll just pick out kind of the core messages that the paper is trying to get
 across. So here the table again, with respect to decisiveness, and you can see even for so for
 classic chess, it's a white has a 50. This is the empirical score for white under different game
 conditions. So 50.8% means most of the time it's a draw. So white wins with a probability of 50.8.
 Most of the time, it's a draw. And you see even like the most decisive variant torpedo right here
 is a 54% only. So they they analyze different defenses and how the decisiveness is with respect
 to different defenses that are not really popular under classical chess. And the results are
 interesting if you play chess. But I would say they're rather, they're kind of aha, okay,
 if you do not play chess, because they consider individual moves and so on. What is an interesting
 part is this right here where they look at they look at one move that in classical chess, so E4
 is a very, very popular opening, where you move your E pawn twice for white. And NF3 is not
 a super popular opening. And here they compare this in classic chess and in no castling chess.
 This thing right here is a histogram. And the histogram shows you the log probability of
 opening sequences when you play the individual moves. So what does this mean right here?
 If you play E4, then the distribution is something like this, which means that you have some
 sequences that have no entropy at all, which means that once you play E4, and maybe one move more,
 then it's almost it's almost determined what you have to do according to Alpha Zero, you have like
 no choice except play these few next moves. However, if you play NF3, then Alpha Zero says,
 look, this distribution is much more to the right, which means that you have a lot more options here.
 Now, again, this could be because the move is actually less decisive because the move
 leads to more balanced, more interesting situations where you can continue. However,
 you know, with many choices, it could also be because it's simply Alpha Zero simply doesn't
 know as well what to do because it leads to more complicated games, and you get to give each move
 one minute to evaluate Alpha Zero might just not be as good in those situations because it leads to
 more complicated situations. If it could search for longer, maybe this distribution would shift
 over here just as well. Again, we don't know because you only give this one second or one minute each
 time for both. And again, this goes under the assumption of Alpha Zero is this perfect player.
 However, back to what they want to say here, if you do this in no castling chess, you can see that
 this spike right here are all the these Berlin defense variants and castling this OO right here
 is a big part of that line. If you do this in no castling chess, you can see that these two moves,
 now the histograms overlap much more, which means that and in fact, you can see in the
 in this number of possible moves right here that they come closer together. So not only does the
 blue shift to the right, the orange actually shifts to the left. And it basically means that
 whether you open with E4 or Knight f3, you are going to have about the same complexity
 of game, the same number of moves available to you going from there, as you can see right here,
 these lines are the moves available for white and black under the different rule sets. So in E4,
 here, especially as black, you do not have many moves available as white a little bit more,
 but also not more. Whereas in no castling you do so, again, small rule change, big effect on
 the possible moves that you can consider. And this is the type of information that you would want to
 have when you design a game. And they allude to this also at the end here in their conclusions.
 So the last thing is they also compare the material values of the pieces here in the different
 rule sets, as you might imagine. So some pieces become much more or less valuable, I find it
 particularly interesting that if you do something like pawn sideways, or then where the pawns are
 much more powerful, of course, all the other pieces drop in value. Again, these results are
 pretty plausible. So I don't want to trash the paper right here. Because it seems like, it seems
 like the results are, as I say, plausible, and can give some cool insights. So the chess master also
 gives his opinions on these different strategies that AlphaZero comes up with for the different
 rules. And let's go through the conclusions quickly. So they say, assessing the consequence
 of rule changes in the game design process demonstrated on chess, where we've trained AlphaZero
 to evaluate nine different variants representing atomic changes to the rules of a game. Training
 AlphaZero model on these rules changes helps us effectively simulate decades of human play in a
 matter of hours and answer the what if question, what the play would potentially look like under
 developed theory in each chess variant. We believe that a similar approach could be used for
 auto balancing game mechanics in other types of games, including computer games, in cases when a
 sufficiently performant reinforcement learning system is available. And yes, this is, I mean,
 this the application here would be for something like this, if you design a new game, then you want
 to know what you have some choice with how you can make the rules. And you don't want to let humans
 become really good at each of the rules and then compare, you can simply give this to the algorithm,
 and the algorithm will tell you what kind of plays result from each rule set. And then you can choose
 the one that you find most interesting or most maybe commercially viable and whatnot. I actually
 see this much, I see this bigger than just games. And this alludes a bit to the Salesforce paper on
 this AI economist, I think we can let AI, you know, get tell us what happens if we change,
 for example, things like tax policy, or any any sort of policy, I know, humanity is very complex
 to model and so on. And you're never going to have a perfect simulator, which probably makes Alpha
 Zero not good. But in limited situations, like maybe also stock trading rules, and so on, you
 could definitely have situations where the rule set is too complicated to solve analytically. But
 you could give it to an RL algorithm and see what happens and whether or not you like the outcome
 and whether or not there are any obvious exploits that you did not see. So this, I find, you know,
 pretty, it's a pretty cool approach. And we should think of this in the future as we build systems
 that have rules in whatever capacity be this games or policy. So the they say, okay, yada, yada, yada,
 we showed that there are several chess variants among those considering the study that are even
 more decisive than classical chess, meaning torpedo chess, semi-tropical chess, no castling
 chess and stalemate equals win chess. We quantified arising diversity of opening play and the
 intersection of opening trees between chess variations, showing how different the opening
 theory is for each of the rule changes. Yeah, they again, this diversity of opening play,
 it really rests on this assumption that Alpha Zero is a good player and an
 sort of an equally good player in all of these variants, right? Because if it's worse in a
 variant, it might not be as sure about the moves and that would just look like, oh, you have many
 possibilities, but in fact, Alpha Zero is just worse at it. And it doesn't know. So they also
 look at the intersection of opening trees, like if you change a rule, how does this change the
 kind of how does this change the the initial game? So a lot of these grandmasters, they learn by
 heart all of these opening trees, the initial moves of a game, how much would they have to relearn?
 There is a negative correlation between the overall opening diversity and decisiveness,
 as decisive variants likely require more precise play with fewer plausible choices per move.
 Again, this is one view, right? The other view is that there are rule sets that are just make it
 into a harder game. And then Alpha Zero, given the same amount of compute is a worse player.
 And therefore, it can't play as well. Therefore, the games are less decisive.
 And also, the opening diversity is higher because it doesn't know.
 The game could be as decisive. It might just be an effect of Alpha Zero. For each of the
 chess variants, we estimated yada yada. Okay. No Castling Chess being the first variant that we
 analyzed has already been tried in experimental Blitz Grandmaster Tournament in Chennai,
 as well as a couple of longer Grandmaster games. Our assessment suggests that several of the
 assessed chess variants might be quite appealing to interested players. And we hope that this study
 will prove to be a valuable resource for the wider chess community. I don't know, is the chess
 community flourishing or going under recently? Because it seems to me like once a game is solved
 that hard by computers, I mean, it's still fun. But yeah, I guess Counter-Strike is also solved by
 bots real hard. It's still impressive when humans play or so. Yeah, I don't know. All of this is,
 again, if you're into chess, look into this paper, they have a lot of really interesting results
 that are not interesting to go into for the general community. But I believe this should give you a
 good impression of what you could do if you design a system that is built on rules. And
 I hope you enjoyed this. If you liked it, leave a comment, tell me what you think,
 and I'll see you next time. Bye bye.