Predicting the rules behind - Deep Symbolic Regression for Recurrent Sequences (w/ author interview)
deep learning machine learning arxiv explained neural networks ai artificial intelligence paper research symbolic symbolic regression neuro symbolic computation integer sequences oeis number sequences ai number sequences machine learning sequences integer sequence rules embedding space transformers attention mechanism sequence generation learning number sequences predicting number sequences facebook ai meta ai beam search symbolic vs numeric
#deeplearning #symbolic #research

This video includes an interview with first author Stéphane d'Ascoli (https://sdascoli.github.io/).
Deep neural networks are typically excellent at numeric regression, but using them for symbolic computation has largely been ignored so far. This paper uses transformers to do symbolic regression on integer and floating point number sequences, which means that given the start of a sequence of numbers, the model has to not only predict the correct continuation, but also predict the data generating formula behind the sequence. Through clever encoding of the input space and a well constructed training data generation process, this paper's model can learn and represent many of the sequences in the OEIS, the online encyclopedia of integer sequences and it also features an interactive demo if you want to try it by yourself. 

OUTLINE:
0:00 - Introduction
2:20 - Summary of the Paper
16:10 - Start of Interview
17:15 - Why this research direction?
20:45 - Overview of the method
30:10 - Embedding space of input tokens
33:00 - Data generation process
42:40 - Why are transformers useful here?
46:40 - Beyond number sequences, where is this useful?
48:45 - Success cases and failure cases
58:10 - Experimental Results
1:06:30 - How did you overcome difficulties?
1:09:25 - Interactive demo

Paper: https://arxiv.org/abs/2201.04600
Interactive demo: https://symbolicregression.metademolab.com/

Abstract:
Symbolic regression, i.e. predicting a function from the observation of its values, is well-known to be a challenging task. In this paper, we train Transformers to infer the function or recurrence relation underlying sequences of integers or floats, a typical task in human IQ tests which has hardly been tackled in the machine learning literature. We evaluate our integer model on a subset of OEIS sequences, and show that it outperforms built-in Mathematica functions for recurrence prediction. We also demonstrate that our float model is able to yield informative approximations of out-of-vocabulary functions and constants, e.g. bessel0(x)≈sin(x)+cos(x)πx√ and 1.644934≈π2/6. An interactive demonstration of our models is provided at this https URL.

Authors: Stéphane d'Ascoli, Pierre-Alexandre Kamienny, Guillaume Lample, François Charton

Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
LinkedIn: https://www.linkedin.com/in/ykilcher
BiliBili: https://space.bilibili.com/2017636191

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
 Hello there! Today we'll look at Deep Symbolic Regression for Recurrent Sequences by Stefan Dascholi, Pierre-Alexandre Camienni, Guillaume Lomple and François Charton. This is another paper where the main part will be an interview with the first author Stefan and I'll just briefly introduce the paper right here for 10-ish minutes or so. So if you want to just skip to the interview, feel free. We'll go over the paper just so that you know what's going on and there is also an interactive demo online where you can try it out and it's a good place to start at what this paper is trying to do. So in this paper the authors care about symbolic regression to number sequences. They have a model for integer and float number sequences. In this case this is an example for an integer sequence. So you can enter any sequence right here. You can see that the sequence that is already entered is the Fibonacci sequence and you enter as many terms as you want. Obviously the more you enter the more success probability the model is going to have. What the model will do down here is it will predict an expression. You can see it correctly predicts the expression for the Fibonacci sequence saying that the current element is the last plus the last last element and it will predict the next terms for you and it will extrapolate the sequence that you've input. So you can do any that you want. I'm very bad at coming up with stuff on the spot. 2, 1, 3, 1, 4, 1, 5. Let's see if it can get that. So as soon as you exit from the model it will yeah look at that. So the quotient which is not even sure what that operation is but it divides the sum of the last element maybe by the last element. I figured it out somehow. It is not really good at if conditions and this is one thing we're going to talk about in the interview. But you can see it correctly predicts the next sequence right here. So give that a try. This pinpoint exactly what this paper does. It does symbolic regression for recurrent sequences. Recurrent sequences are sequences of numbers that can be somehow expressed as a logical rule as a function of the last elements of the sequence. Most sequences can be expressed like this. For example they give a bunch of examples right here 1, 2, 4, 7, 11, 16. So you can see that it's always sort of plus 1, plus 2, plus 3, plus 4, plus 5 and so on. Or this function right here these are simply the squares. So the recurrence relation actually isn't a recurrence relation at all but it is also a special case of a recurrence relation or this formula right here. It can get very complicated. They have a bunch of examples right here of recurrence relations. As you can see they can go pretty complicated to express something like the final digit of n times n plus 1 divided by 2 or the final two digits of 2 to the n or some maximum or anything like this. So the goal of the model is that you input a sequence like this and then the model will output this recurrence relation. It will not output the numbers directly of the sequence of the following numbers. That's what they would call a numeric model and they also train one as a baseline but the model would actually output exactly the formula itself. Then you can use the formula to produce the next elements. Now the good thing is we've all seen what happens if you train a numeric model on a bunch of data points. Let's say these are your input data points. You train a numeric model on that. It will perform pretty well on the data you give it but as soon as you go outside of that data, as soon as you extrapolate too much away from the support base of the training data without very strong inductive biases, it will sort of do whatever. You can't really predict it what it will do where there is no training data. That's why also deep learning relies on lots of training data in covering a lot of the input space. Whether that's called extra or interpolation or whatnot. We'll leave it at that. But if you have a symbolic regression and the symbolic regression actually predicts the correct formula to match this sequence right here like saying ah this is just a sine wave, then you can extrapolate indefinitely. Because you have the correct symbolic formula you'll be right in all places. So potentially this is a very strong method for certain types of problems. This paper considers this a sequence to sequence problem. So it considers transformer stacks and this is I guess along the classic transformer stack of you have an encoder and a decoder stack. The encoder stack gets fed with the input sequence as numbers. So here one, one, two, three, five and so on. That is the input sequence. It is fixed. And then the output sequence is the formula that you want to predict. And they predict the formula in reverse polish notation of the prefix tree of the formula. So they have an example down here. For example, the cosine of 3x can be expressed as this as cosine of multiplying three by x. So you would you would sort of load it onto the stack and then work your way down the stack in in this reverse reverse polish notation measure. So that would be cosine of mole of three of x, or whatever that formula is. And then you try to train your transformer to autoregressively predict first the first token without seeing those tokens. And then once you have the first token, you want to predict the second token given the input and the first token. There's like there's multi-head attention in here. Like there is cross attention over here. There's self-attention in here as well. So you can predict your regular transformer stack. So this is classic sequence to sequence problem. The only question is how do you obviously encode the input and the output. The output we've already discussed, and they have a very detailed description of how they produce the data. So what they do is they take a bunch of operators, you can see them in this table, and they make random formulas from those operators. They have a bunch of constraints on these formulas, but essentially they make random a data set out of just random formulas. So first of all, they sample the number of operators between one and a maximum number. In this case, that would be 10. 10 is the maximum number of operators. And then they build a unary binary tree with that many nodes. So they for example, they would sample two operators right here, like there are three, a relu, a sub and a mod. And then they would build a unary binary tree. So relu, then that is a unary thing, right? So it only has one input. So sub, that's a binary operation. So it needs two inputs. Here, let's say mod, that again needs two inputs. So the second step is to sample the nodes of the tree from the list of operators. Okay, that's what we've already done. We've combined steps one and two, sample the recurrence degree between one and D max, D max is six. So we're maximum allowed to look back six elements into the past. This is kind of a Markov condition. You can say your recurrence relation can only look back six items. That's kind of a limit. But most sequences that humans could come up with don't refer back to the seventh last element, right? There is usually a way to express it in forms of either the current index or the last few like three or four elements at max. Then they sample the leaves of the tree. So the leaves of the tree are either a constant with probability P constant, these all these probabilities are one third and they stress very much that hyper parameter settings are not very crucial in this way. They sample the leaves of the tree. So either it is a constant or the current index or one of the previous terms of the sequence. So let's do that. So we'll say here we sample the previous term, which is U n minus two, here we sample the index, which is n, and here we sample a constant, which is three. So that would result in the formula ReLU of U n minus two minus and then n mod three. That would be the formula for this. Then they need to sample initial terms of the sequence. So in with the formula, you also need to decide, you know, how the initial terms, the initial terms, since we go back two elements, we probably at least two elements at the beginning of the sequence. So let's call that one and two. That's we also need to sample that from a distribution. You can see here, that's just a uniform distribution from negative 10 to 10. And then what's the last sample the sequence length and compute the next L terms. So now we say, okay, how much leeway do we want to give the model to infer the sequence? Let's say we want to give it five elements. And now we use the formula to calculate the next three terms right here. All right, I tried it, it didn't work out, but it is a rather complicated sequence, I have to say. But now you see how this stuff is sampled. So you see how the formulas are made, they just define a maximum depth of maximum length and so on. And then it just sample random data from that, they create a data set, the data set would be this one right here, this would be the input, and the output to predict would be the formula in reverse Polish notation. It's a sequence to sequence task. That's it. Now during inference, they can do a beam search, they can input again, the sequence, they can output different formulas, different, they can start out different formulas, and then they can do a beam search and check which of the formulas actually match the input sequence that they have already. And they can discard or rank down formulas that don't match the input sequence on the first few terms. So that is an additional benefit they have from this symbolic regression. Ultimately, they will end up with a formula that probably fits the input terms, and hopefully is simple enough. And the simplicity comes from the data set, since shorter sequences are more likely to be sampled and longer sequences the model is implicitly biased towards easier formulas, which kind of plays into Occam's razor. So that's it, that's the method they create a data set, massive data set, they train on random formulas train train to predict them from the initial terms, and then they evaluate it. As I said, they also have float sequences, but I won't go into that too much. Notably, they do outperform this numeric model, the numeric model simply tries to learn the number to number sequence just directly without going to the symbolics. So as you can see, the symbolic method is better when evaluating on in distribution sequences, when evaluating on out of distribution sequences. And here's a question of how do you even do that. There is this database of integer sequences. And after a bunch of filtering, you end up with a validation set of 10,000 sequences. This validation set are human made number sequences like the Fibonacci sequence or anything essentially that where humans can come up with some sort of logic of how the sequence is generated. On this data set, they don't perform as well as the numeric model, as you can see right here. So the numeric model outperforms the symbolic model. But there are good reasons why that might be. And we also discussed this in the interview. Lastly, they also make do experiments with robustness to noise, which are also very interesting in that they can even suffer from a bit of noise if they train with the noise. And so the model is even a bit robust and can still do symbolic inference, which classically, if you have a symbolic system, these are usually not that robust to noise, because it's more like hit or miss. But if you train appropriately, you can handle that. Also interesting is that they encode the numbers not as continuous values in the transformer, but actually as tokens. So at least for the first 10,000 numbers, they are all their own tokens. So the number 19 and the number 20, they're just two tokens. But it turns out that if you train the model, then in the embedding space, the tokens will actually form a sort of continuous, not necessarily line, but a continuous manifold in the embedding space, which is really cool to see that the model, even though you give the numbers as different tokens, it learns to map them out according to their numerical values. They also have investigations into the similarities between embeddings and they uncover some interesting structures where similarities are also according to the numbers like common denominators and so on. And they give a bit of evidence that there seems to be kind of a natural base for mathematical operations of multiples of six and 12. And they say that six is a natural base for reasoning, reminiscent of much earlier explanation by other people. And you might know this cult of people, I don't even know what they're called, but this cult of people that says we should just switch to base 12 because it makes everything easier. So there might actually be, you know, stuff behind that, or it might just be a artifact of how we do math. Who knows? They experiment a bunch of stuff with expression simplification and so on, but the model seems to be quite robust to any of these modifications. I think this is a really interesting work in that symbolic inference, I believe, can lead us forward and tackle problems of extrapolation that we aren't necessarily going to be doing with these numeric models that we currently have. Obviously, this has its own limitations and its own biases built in. Most notably, how you construct the data set is very, very crucial to how the model is then going to perform. But it is interesting to see that you can train it like this. And essentially, it's a, you know, it's a it's a free free training data because you can just generate it by yourself. So without further ado, I want to jump directly into the interview because we go over the important aspects of the paper. Again, let me know if you like inter like interview content like this, I think it's super duper helpful. And the interview was very fun. I hope you find that as well. All right. See ya. Welcome, everyone. Today I have with me right here Stefan Daskoly, who is the first author of the paper Deep Symbolic Regression for recurrent sequences. Stefan, welcome. Thank you very much for being here. Yeah, pleasure. Bad timing to have COVID, but I'll try my best. Yeah, I hope this goes I hope this goes over relatively smoothly for you. But yeah, so this paper, I have to say it gathered quite some hype online, right. And because symbolic mathematics is something that is still still even though computers are very good at math per se at numerics, symbolics is something that has been maybe in the human domain a little bit more, especially these kind of sequence guessing, right, it seems to be a very, very human thing, something you would do maybe in high school to try to like figure out some sequence and figure out the rules behind it. What sort of what prompted you to go into this direction in the first place? Like why do you why do you think this is a fruitful direction? Or, you know, what made you come up with an idea? I know there's some previous work, but you know, why this? Yeah, so as you say, I mean, this kind of problem is very common, like IQ tests. So that was definitely one of the motivations. So originally, this project was born from Francois and Guillaume, who have been both working on papers first. So basically, deep learning for symbolic math for a couple of years. And what they've been exploring is several directions. The first one of them was a paper in 2019, called deep learning for symbolic regression, where they basically did symbolic to symbolic manipulations, basically just integrating functions, solving ODEs and stuff. And then more recently, Francois has been working on a numeric to numeric task involving math, which is basically doing linear algebra. So taking a matrix and then outputting its inverse or stuff like that. And so a natural continuation of this was to start from numeric data, and go to a symbolic formula. And that's basically symbolic regression, which means you take a function, you only see its values, and you have to try and infer the expression of the function. And indeed, it's kind of surprising that this has been studied quite a lot for quite a few decades, actually, this symbolic issue, the symbolic regression question, especially with genetic algorithms and stuff like that. But there hasn't yet been in the machine learning literature, a paper working on sequences. And as you said, it's a very common setup for us humans. And so this is originally the motivation. And so Francois came to discuss with me and Pierre Alexandre. Pierre Alexandre is more from the reinforcement learning background, which is also relevant to sequences because you have basically a sequence of states. And for me, it's because I came from the physics background. And this is also symbolic regression is useful also for physics for like inferring laws, etc. So yeah, that's kind of how we got together. Cool, excellent. And just so we're clear to anyone, the kind of sequences we talk about, we have a bunch of examples right here. So that would be, for example, here, the final, the final digit of n times n plus one divided by two, that's kind of the formula of all possible pairwise connections in a group of n points. Or is that n times n minus one? Times n minus one. Yeah, the sum of integers. Okay. And from that, we just want the final digit. So this the sequence here is 0136051865. That is, it is, it is, I would call it pretty complicated if you just gave me this as a human, but there is some kind of a rule behind it, right, that I can figure out. And that's the type of sequences you would, you would consider. This one is actually a good example. It's kind of hard to recognize for us. And if you look at the formula that the model gave us, you can actually figure out why it predicted that formula. It's un minus one plus n. And the reason for that is that nn plus one divided by two is the formula for the sum of integers. And so the way it built this formula is just to take Pries-Dulce turn, add n, and then take the modulus respect to 10, because that gives you the final digits. So it's kind of a clever thing that, you know, would be kind of hard to figure out for us. Yeah. So if you, if you could maybe give the pitch of your model itself, like the pitch of your paper itself, just before we get into more of the details, it's always super interesting to hear from the people themselves describing something like a brief pitch of what you did here. Yeah. So I think our starting point was less ambitious than what it came to. So we originally just started off from the, this sort of thing that, that is quite popular for math lovers, which is the OEIS database. So the online encyclopedia of integer sequences where you have all sorts of sequences, you can play around with them. You can you can try and guess the next term. It's quite fun to play around with. And the idea was to try and build a model which could complete the sequences. So sort of understand the logic behind the sequences. So originally we only started off with integer models. So we only wanted to predict integer sequences. And, and we actually realized that that was pretty easy. Pretty quickly, we managed to get a model working on integer sequences. And so we then started to think about, can we do the same thing for float sequences, which are a bit more challenging because you have more freedom in the expressions you can build. You have more operators, you have cosines and exponentials that come in. And, and so this is how we sort of, I'd say it was a lot of serendipity really in this work. We started off with this integer sequence problem, and then we figured out things as we were going on. So as you can see on the two tables you have there, the constant approximation thing, which we may discuss a bit later, was one of the fun side effects of trying to guess sequences. It's that you actually, the model actually learns to do stuff it wasn't trained for. And so yeah, I'd say the goal of the paper isn't to provide a, you know, a model which is useful for real world data. It's not going to be able to predict, you know, the stock market or weather forecast, et cetera. It's more of a like proof of concept of what you can do with transformers in terms of math. And you specifically restricted yourself to, to recurrent sequences. And it, I think it's important to point out sort of what, like what kind of inputs does your model take and what kind of outputs does your model give, right? Because a formula like, like these, they are, you know, written down in many ways. There's, there's ambiguities and I would guess the inputs are these numbers right here, right? So our model gets this as an input and then it's somehow has to predict the corresponding formula. So this is, the training data is also like this. How does it take the input and in what form does it output stuff? Okay. So those are like the two, two big questions. So maybe we can start with the, the inputs. So that's actually quite a tricky question. How do you feed in these, these inputs to the model? Because, you know, typically deep learning models don't, don't take like, if you think of a sequence, which is like an exponential, you're going to have very huge numbers. If the exponential has a positive sign and very small numbers, if the exponential has a negative sign. And so if you just feed these kinds of values into a deep learning model, it's not going to learn much, especially that here we're dealing with a transformer model. So you're going to have a transformer because essentially what we want to output is a mathematical formula, which is just like basically a language. And so this is why we use transformers. And so transformers need to take in embeddings. And so we need somehow to represent our input numbers as embeddings. And that's complicated because of course, integers, just like reals are an infinite set. So you have to sometime, somehow find them, find a way to encode them as a fixed vocabulary. And so this is where we really have to distinguish our two setups. We basically have two different transformers, one for integer sequences and one for float sequences. So the integer model, what it does is basically it writes numbers in a base B representation. So for example, for the number, like, yeah, exactly like here, 325, you could imagine writing it as three to five, in which case you only need 10 tokens, which is numbers between one to 10. Actually, it turns out that it's better to use a larger base because if you use a larger base, well, you're going to have a bigger vocabulary, but you're going to have shorter sequences. And typically, you know, transformers have quadratic complexity. They struggle a bit with very long sequences, which is why, yeah, we prefer to use a large base. Here we use 10,000 as our base. Yeah. So this will be base 30. And obviously in base 10,000, I think it's important to note that every single number from zero to 9999 is its own token, right? The model has no inherent knowledge of, you know, three comes after two and four comes after three and so on. All of this has to be learned. It seems so weird to say, you know, it is better to make the model learn essentially the entire ordering of 10,000 numbers rather than, you know, providing that as some sort of a, just to make the sequence a bit shorter, right? It's funny. Did you ever think of going with continuous values, right? Because the first, my first intuition would be that I feed the actual number, right? And then it's implicit, like it's in the number that two is larger than one and three is larger than two. Exactly. Yes. So that's what's really interesting is that that is one approach. And actually we had a couple of discussions on this, like how can we feed in our inductive bias on numbers directly into the model. And well, I mean, the problem with this is that here we're dealing with like just one dimensional vectors in some sense. Transformers need, you know, high dimensional vectors as inputs. And it's not obvious how you represent these numbers in a high dimension, you know, because the, as I was saying just before, the problem is that these numbers have very vastly different scales and, you know, deep learning models usually take normalized inputs. And so it's not obvious how you would, so what you want to do is basically map these numbers you have onto a sphere. And it's not obvious how you would encode, you would put these numbers on the sphere. And so one very simple way is just to put them randomly on the sphere and let the model decide all by itself how to put them in this sphere. And this is what we do. And what's interesting is that when you plot after training what the embeddings look like, you can see that it has learned in some sense our inductive bias of putting the numbers in order, et cetera. So these are, these are t-SNE plots right here. The left would be the integer embeddings. And it sort of forms this, this string. What do you make of the t-SNE plots here? Do you think these things are actually, you know, uniformly on a sphere or does the model just use like a tiny part of the sphere where it can make sort of a continuous path? Well, what's for sure is that the, it's definitely a low dimensional representation because you can see that the t-SNE is actually very, really shows a smooth pattern. Usually when you plot t-SNEs of like word embeddings in NLP, it's going to be a bit messy. Like you're going to get clusters, but it's not going to be as well organized as here. So clearly the embeddings are lying somehow in a low dimensional manifold. And so then you could think, okay, so why do we need like 512 dimensions if it's only using a small amount of them? But that's actually because, you know, the transformer is going to eventually use these extra dimensions to perform its calculations really. So it's not as if they're wasted. They're actually going to be used by the model. Yeah. And the float embeddings are very similar, right? In that you encode them as like a sign, a mantissa and an exponent. And again, the mantissa, if I understand correctly, same deal that you have a token per number between zero and 10,000 and the exponent, is that correct that you say you have exponent from negative 100 to 100? So one token would be E minus 100 and then another token would be E minus 99, E minus 98. So these are all different tokens. So now the transformer has to learn kind of two different embeddings. Both are somehow in sequence. Exactly. Yeah. So just to summarize, so for the integers, we encode the integer as the sign followed by tokens of the base B representation of the integer. And so for floats, we also have the sign token. Then indeed we have the mantissa token. So here the difference is that we only have one token for the mantissa. We don't have like a base B representation, which means that we do lose some information in the discretization process. And then indeed to represent the scale of the number, we use an exponent embedding. And that indeed goes between minus 100 and 100. And so here indeed we do plot the TSNE of the exponents because they really have a logic to them. For the mantissa, it's less obvious. If you plot a TSNE of the mantissas, it would look a bit anarchic. But here the exponents, you can, and actually just about this plot here, this plot is actually a tiny bit disappointing because we can't see some of the really interesting features we had with our first models. This is with the very big, big model, with embedding dimension 512. Actually, when we were using a smaller model with a smaller embedding dimension, we saw a really neat pattern, which was basically the fact that the model was learning the arithmetic properties of integers. So it was basically creating a line with two, four, six, eight, 10, etc., then three, six, nine, etc. And here it's a bit less obvious probably because the big model was learning something even more complex that we can't interpret as easily. If you go into the appendix, you do see actually a figure where we see that the model learns like a base six representation of the integers. The attention plots, you mean? Actually, not those ones. Yeah, those ones exactly. Like if you zoom in a lot on the left plot, you kind of see these diagonal lines which are spaced out to every six and every 12, showing that basically the model is recognizing numbers which have common devices and is specializing to the base six or 12 representation, which is often considered better than the base 10 representation. So these plots, just to make it clear, these are the cosine similarities between each of the tokens. So the tokens would be distributed on the axes here. These are tokens and these are tokens. And then we plot the cosine similarities between every two tokens. So naturally, obviously, every token is going to be very similar to itself, but also very similar to its immediate neighbors. So it seems to really learn the ordering of all the tokens. But then also, yeah, what I found special, there is this structure of the common factors, common divisors between the tokens. That's really cool. Yeah. One thing also that's hard to see in this big model, which was much clearer in a small model, is you could see, for example, the perfect squares would be complete outliers. You would get 9, 16, 25, 49, which would completely stand apart due to the special properties. I think that here, so here is 49, right? That kind of stands out, right? Yes. This gap. Yeah. That's something which we haven't really been able to understand. Some guy sent me an email actually saying, oh, maybe I have an idea that there's a gap between 46 and 48 because 45 has lots of factors of five and three, whereas 48 has lots of twos. There must be some explanation or maybe it's just something due to optimization. It's very hard to know. Okay. Yeah. I think at this point, it's a bit also important that we look at the data generation process. You give the model a bunch of options, right, to generate sequences. And these are, where do I have them? So here, we have the operators that it can use. On the left-hand side are the integer operators. And then the float operators would be in addition to the ones on, or sorry, they're repeated in part, but also there are more in the float formulas. And then you just generate in reverse polish notation. Is that correct? Exactly. So you generate reverse polish notation formulas given these things. And you can also have integer prefactors, right, for all the things. So either you sample integers or you sample the current element index, or you sample previous elements of the sequence. So the model could express, you know, if it's the fifth element, take that current number times the previous element plus two times the cosine of something either a constant or again, referring to some previous element or something like this. Is there a logic behind why you chose the, why you made these choices of how you generate these formulas? So actually, if you look at this table, indeed, there are much more operators for the real case, the floating point numbers, but you do notice that in terms of binary operators, there are two which you can see in the integer setup, but you don't see in the float setup, which are integer division and modulus. And this really illustrates that we're trying to learn rather different things in the two setups, really in the integer setup, we're focusing on sort of arithmetic and arithmetic properties of numbers, whereas in the float setup, we're really interested in a, let's say a more classic symbolic regression problem with complex operators. And yeah, as you said, our generation process is basically to build a mathematical tree. So a unary binary tree, this is like previous works by Francois and Guillaume. And then indeed, we fill in the nodes of these trees, either with operators. So the nodes are filled in with operators, either binary or unary. And then the leaves of the tree, indeed, as you said, can be either variables or constants. And as you said, the choice of generators actually basically the hardest part, let's say, of this problem, because one thing that's nice when you do these kind of symbolic math problems is that you basically have an infinite data set. Your data is just synthetically generated. And so you can train as long as you want. You don't have any sort of, you know, you don't have any overfitting issues. You don't have to regularize that much. You don't have to, even the hyperparameter choices aren't that important. What is really crucial here is like how you build your formulas. And that's what makes the problem, I think, really quite fun to play around with, because it's a bit like, you know, teaching a kid how to learn maths, like you really have to figure out what is the best thing to show the model at what time and what is going to you want the data set to be kind of hard, so they can deal with complex cases. But if it's too hard, it's going to learn more slowly. I mean, it's really an interesting problem how to generate the data. And you decided just by playing around because so you do have, as we said, you have these particular ingredients. And I mean, you can always say, why didn't you have more or less and so on. But you know, you have a table of a bunch of operations that you can do, you decided as well to make to allow the model to use these sort of recurrence relations, right to allow the model to say, not only I want five times n plus two, but I maybe I want five times n plus two times the previous or the time step, two steps back or something like this. Is there a reason behind, you know, including these recurrence relation? Is that just something you thought would be more interesting? Or did you look at the database and see that that's a lot of how these sequences are made? It's true that often people look at the problem they want to solve in order to choose the parameters of their generation. For example, sometimes people use different weights for how to sample which operators to sample, like they'll put more additions and multiplication or they'll here we have, for example, if you go right to the left here, we have these hyper parameters for our generator. For example, you can see here the probability of choosing a constant leaf or index leaf, so n or the previous term. Well, yeah, probably we could have like tuned these parameters somehow, but here we really wanted to have the simplest choice possible on the rationale that basically our data set is so huge that eventually we're going to see all possible formulas at some point. It doesn't matter that much, the specific values we choose, and we don't want to tune them to a specific problem. And so this is why we really chose like very standard and also for the operators, like we didn't use any particular probabilities with which to sample such and such operator. We just let everything as general as possible. And this would be, so this is built up as a tree because naturally you can parse these things as a tree, you can generate them as a tree to have the sort of correct grammar, but ultimately you end up with, as we said, this reverse polish notation, which is a sequence, right? So this would be one such formula, not you wouldn't have x, but you would maybe have n or something like this. So, but ultimately this results in a sequence of tokens, right? So the input, your model is these numbers encoded in tokens and the output is a sequence of these symbolic tokens. Yeah. Did you also investigate sort of the the embedding space of the output vocabulary? Yes, actually a good question. So we did look at that and actually it didn't have any particular structure. You could have expected maybe like cosine and sine are going to be close to in the embedding space. I think what's happening is that the output space is actually much smaller, right? Because in the input space, we have a lot of tokens, like we have for integers, we have one to 10,000, that's like 10,000 words. So it really tries to find a structure in the inputs. For the outputs, we only have a very small vocabulary compared to usual NLP tasks. We only have like about 30 operators. And so essentially if you look at the high dimensional space and you do it t-sne, you won't see much because it's just equally spreading these operators in the sphere or something like that. There isn't much logic to it here. And how, let's say, how universal are these sequences, right? How many sequences that I could come up with freely would be inside of the scope of your model? And like, are there, is there a significant class of sequences that your grammar could not express? So with this unary binary tree representation, you can pretty much represent any function. So of course, there are some sequences which don't have any logic to them, which aren't generated by a recurrence formula, in which case you can't represent these sequences. And that typically is the case with most of the sequences from the OEIS database. So we had to get rid of quite a lot of them and do some filtering. Now, I did say that you can represent any function, but there is a limitation. There is that some functions are very difficult to express with this tree approach. If you think, for example, of the collapse sequence, where basically for odd numbers, you multiply by three, add one, and for even numbers, you divide by two, that's a rule which is possible to express with a mathematical expression. Essentially, what you do is write it as n modulus two times what you do if it's even plus one minus that. But that's kind of an involved way to write it. And generally, the model is going to struggle to output that because it won't have seen it much during training. That's one important thing also, which we might discuss a bit more, is that our model is biased to the likelihood of the expression to be generated during training. Yeah, it's like a hack that we as programmers have for an if condition. It's just something we learned at some point. Oh, look, if you have an if condition, you can express it as if you, I don't know, people program NumPy or something like this. That's exactly what you do. You don't say if, you make your mask with one minus whatever condition and you multiply by this, and then you have that. And I think anyone who programs NumPy or TensorFlow or so on knows, okay, I can do it like this, and then my stuff is expressible and differentiable as one formula. But I think that's a hack we learn. And if we just generate data at random like you do, this is not something you come across as often as we come across when we program. Exactly. Yeah, it's very unlikely to see this formulation in our datasets. Yeah, absolutely. Okay, cool. But at the end of the day, you generate a giant dataset, right? You go through it with transformers and you emphasize transformers. Is there something special about transformers? Because couldn't I use any deep learning thing or why transformers? Well, first of all, like previous experience, I mean, Guillaume and Francois have been working on these transformers. They've basically always been good at the problems we've given them. Likely, one natural justification is that as we saw for the outputs, you can represent math as a language in a very easy way. It's actually, we can see here that it's much easier to use the inputs as tokens, but the formulas themselves are very easy to represent as a language with this Polish notation thing. And so it's very natural to use transformers because they are best models to deal with language. So yeah, I think that's the main reason. And yeah, I'm not sure what else we could particularly, I mean, we could use like RNNs, etc. But these days transformers are so powerful. I mean, these models we used, we didn't even, as I was saying before, we didn't have to tune them much. We just basically took the same architecture that was used in the paper two years ago. We didn't even have to change the learning rate. Like it's pretty amazing how easy it is to train these things. Okay. Yeah, so the transformers are a natural way to deal with sequences. And from text learning, we kind of know this, but we always learn sort of on human text, right? And that has a particular structure. And I want to think if I look at these sequences, there are almost like, there's so many symbolic formulas that could possibly explain these sequences. And yeah, I can, you say you make, you want maybe the simplest sequence or you know, you don't want your formulas to blow up. That's, you even generate only formulas that are, let's say, relatively simple. So there's clearly a bias towards simplicity, but still there are a lot of things that explain the same sequence. So I'm thinking more, is it like if when we as humans do these tasks, is it like a property of humanity and civilization that we kind of come up with the same sequences that the person you know, who made the riddle came up with? Is it because we kind of think alike, right? Because of whatever society or our environments that shaped us? Or is there like a property of math that says, well, if actually if you look for the simplest sequence, it is kind of defined even though there are infinite possibilities. Like, you know, a little bit what I mean, is it more like a property of humanity or of mathematics? I think it's probably two different things. So as far as humans is concerned, indeed, we we tend to prefer simplicity. That's like our OCam's Razor principle. We like going for the compressing information and going for the simplest representation. In terms of our algorithm here, we didn't put at all this simplicity inductive bias from our own understanding of the system. We didn't put the inductive bias from an explicit point of view. We didn't tell the model, give us the simplest formula. Actually, we could have done so because we could have, for example, given a penalty to like the decoder when it generates two long sequences, for example. But we didn't have to do this at all because the inductive bias comes from the fact that simple formulas are more likely to be generated by the generator. And that's basically the rationale behind our model is that it's always going to be biased towards the most likely formula corresponding to the sequence. And as we were saying before, sometimes that's not good because for the collapse sequence, it's going to struggle to output the one minus the mask thing. But in general, that's kind of what we want in IQ tests. We ask for the simplest formula to explain the observations. Mm-hmm. I'm thinking of are there more things rather than just number sequences where something like symbolic regression could be valuable? For example, I've always thought of maybe reinforcement learning would be much more powerful if we didn't only... Even if agents have a world model, what they call a world model, they usually have almost like a numeric world model. They just forward predict the values that are going to happen there. I always thought, well, if I had a symbolic representation of the world, I could do much more powerful planning. Are you thinking of applications like these when you develop this, right? Beyond number sequences? Or is there any interesting ones that come to your mind? So as I was saying, Pierre-Augurelien, my co-author, it comes from reinforcement learning. And there have already been a few papers inserting some symbolic parts into RL loops. And that's definitely going to help. Indeed, as you say, if you're a robot and you're trying to understand the world, then it's going to be much easier if you understand Newton's law. If you want to, for example, predict how objects are going to move, it's much easier once you understand Newton's law than using a specific vision model to try and predict. That's going to be much more complicated. So indeed, I think symbolic regression is going to be very useful for RL. From my point of view, I'm more from the physics background. And that's also a domain where symbolic regression would be very useful. Because typically, so we have these two approaches, right? We have numeric regression and we have symbolic regression. And I think they're very complimentary in the sense that numeric regression is very good on complex tasks where you don't necessarily have a simple explanation for the data. And symbolic regression is great for inferring data where you have a simple underlying rule, typically in physics, like inferring laws from observation. So yeah, I think RL and physics are definitely two huge domains of application for symbolic regression. And to make this a bit clearer, so what I've done is in the appendix, you actually have some success and failure cases of your model. And so I have made a little quiz out of them and hidden a bunch of them right here. And I just want to draw people's attention a little bit to some of the some of this. So on the left, the left three columns are success cases. And the right three columns are failure cases, both of the integer model, right? So these are integer valued sequences. And do I have this correctly, you do consider it only a success if the formula is equivalent? Or do you consider it already a success if just the predicted values are the same? You can have the two criteria and the criteria we choose in the papers, we want the the evaluations to be the same. So even if it comes up with like a different formula, it's it's fine as long as like that the ones you tested on match. Yeah, that's actually one tricky thing is that indeed, you can't really rely on the formula to check if it was correct or not due to the degeneracy. And so some papers have circumvented this by using like an RL loop. Because if you try to really supervise the formula, then you can't make some you have to evaluate the formula, which is non deterministic, and then you can't like back propagate this. And so some people have used sort of RL loops to provide reward signals from the evaluations. What we do is directly supervise the tokens of the formula. And, and that, okay, maybe we can discuss this a bit later. But that's also interesting, because, you know, you could think this is weird, because our model is supervised to a formula. And it's going to be penalized if it outputs at training an equivalent formula. Yeah, but that turns out to not be too bad. And we tried we tried we tried expression simplification, and it didn't help at all. It doesn't really matter. But yeah, this is very interesting what you're going to come to with the success and failure cases. Yeah, so the leftmost column here is is is pretty simple. These are okay, people already know it's success cases. So in nothing too unexpected right here, like it figures out that for example, the middle formula, this might be a bit small here, even for people to read. But this is n, n times the sine of gamma. And gamma is what exactly? Euler's constant, Euler's constant. Okay, so n times the the sine of gamma squared. So the entire thing on the right hand side is a sorry is a constant, right? So it's essentially n times a constant. Yeah. So the the model what it has to do is it has to somehow figure out the expression for the constant as a formula, right? Because it it can't it, it, it, it has to Yeah, it cannot just predict the number. And then it has to realize that I have to multiply this constant by n. And that's why it's a straight line. So and the other formulas are similar ish. The top one, for example, is n minus the cosine of n. And yeah, again, reminder, these are this is symbolic, symbolic regression. Now, the next ones are weird. So here, the top one, it starts off very, very weird, but then it continues in the same path. And you can still you can see sort of, okay, it's regular enough that the model could, you know, figure it out from the data points it has, by the way, that the green background, that's the input, right, the blue background, that's, that's the what it has to predict. So the next one I find particularly interesting, it is the formula is the tan of the tangent of n plus n times the last element. And this is what the output looks like. So, you know, how like, how can the model from the just the left part figure out that this is the correct formula? And then the the end date that just blows my mind, like, how does that work? Maybe the log scale would help a bit here, because there is probably quite a lot of variability in the in the first terms. And it's just squashed by the last term, which is huge. Okay, yeah, I should have made me put a log scale. That's a good question. Yeah, what is what I find really interesting with these plots. So here, you're showing the success plots. And on the right hand side, you have the failure plots, is that we really see how symbolic regression is different from numeric regression, like in numeric regression, you have this set of points. And basically, you're just trying to fit your function, you're trying to bend the function, so that it goes through the, through the input points. And so this is typically going to be very prone to overfitting, right? If you can't really understand the process, then you're just going to fit a function which goes through the points, whereas symbolic regression here isn't biased towards overfitting at all, it's just trying to find a formula. And so when it fails on the right hand side, it not only fails outside the input points, but also on the input points, it's not even able to fit the points you gave it. Yeah, this really shows a big difference. We can see this a little bit, I think. So on the bottom left, there's a there's a nice case, where it can it already fails. Yeah, on the inputs, like that's the best formula it can come up with, you do have a beam search in there, right? These ones, no, no, these ones, not even okay. Search does tend to pull a bit more towards overfitting because in search you so the way we rank our beam is that we evaluate how, how well the formula matches the input points. And so in that sense, you're coming a bit closer to like actually overfitting the input points. But if you use a beam size of one as using most of our experiments, then essentially, you're not at all biased towards overfitting. Okay. Yeah, I mean, this, it seems like here, it's just misjudged the formula on the top left is an interesting one, where it just it looks like it's done everything correctly, right? It looks like so the red ones are the the outputs that it's supposed to match. And the black one is the the line the function it produces. What's wrong here? Is it like off by a tiny bit? Yeah. So the screen is pixelated. So I can't see very well. But yeah, um, essentially, we get two kinds of mistakes, we get the mistakes where it's very close, for example, it confuses a like a four with a five. And so it's going to be very close. But then you have catastrophic failures, where basically, for example, to confuse a cosine with an exponential or something like that, you know, that's just one token error, but it's going to give completely wrong predictions. And that's something that you typically won't get for numerical regression, you'll always at least fit your inputs. Yeah. However, there is one thing where symbolic regression is better than numerical regression is that once it does find the correct formula, then it's going to get predict, you know, perfect precision on all all the, the subsequent numbers you're going to give it for if you think, for example, of extrapolating the sequence, with a numerical model, you're always at some point going to, you know, get wrong predictions, because you're not very good at generalizing outside, yes, typical thing that deep machine learning is good at interpolating, but bad at extrapolating. But with symbolic regression, once you've found the correct formula, you can basically extrapolate as far as you want, you've got the right formula. Yeah. And so just saying for people who probably even people in the video will not be able to read, I can confirm the formulas of these two things are completely different. Like the one is the sign of something simple. And the one that's predicted is a very, very complicated formula that just happens to almost fit or maybe even perfectly fit the input data points, right, but then it is just that tiny bit off. And that that gets worse and worse as the sort of the output progresses. Okay. So yeah, there are a bunch of about a bunch of other funny ones like this one, again, the scale here is absurd. It's like the exponent is 224. And there's just this one output that it's supposed to match. And I mean, that's just mean to the model, honestly. Yeah, we do have, I mean, horrible expressions, like our generator uses up to 10 operators. And so if you look at expressions here, we only chose expressions with three operators. So you can imagine how horrible the expressions are with 10 operators. Yeah. And of course, the accuracies are much lower. I mean, if you look at the ablation, like our performance at 10 operators is about 10% versus, you know, 100% when you have one operator. Yeah. So I will quickly uncover the rest of these, but people are encouraged people to actually go and look at the success and failure cases. Also for the floating models, I think it's really valuable. And you can directly see, as you say, you know, the differences between symbolic regression. And I mean, if you did numeric regression, even if it has like a pattern like this, like a zigzag pattern or something, it would quickly degrade. We've all seen sort of sort of numeric regression, although as in your experiments, so maybe we'll come to this last. So in your experiments, there are cases where the numeric regression is worse. And there are cases where the numeric regression is actually better than the symbolic regression. Would you want to maybe comment a little bit on the experiment, specifically like in distribution, out of distribution evaluation? So typically in in distribution, our symbolic model performs better than the numeric model because it's got the right inductive bias, right? Really, we feed in these sequences which are generated by a formula. And so it's much better than the numeric model at extrapolation because once it's got the correct formula, it's going to give perfectly precise predictions extrapolated as far as it wants, etc. However, it is slightly less good at out of domain generalization. So one thing you see here in is I can't remember where it is in the paper, but you see that, for example, numeric regression is better when you have complex pre factors, right? Because here the expressions we generate, the pre factors we have are built from like integers between one and 10, e and pi. Yeah. And so that's well fitted for the symbolic model. But what happens if you replace these pre factors by like pre factors which are sampled from a Gaussian distribution? So these these two columns right here, the difference between those. Yeah, exactly. And so what's interesting here is that in this case, of course, the numeric regression performs better than symbolic because numeric doesn't care at all about the fact that you're using these pre factors because it doesn't it doesn't really care. It isn't trying to approximate these complex pre factors. What's interesting though, is that the symbolic model still isn't that bad because it's actually able to approximate pre factors with its own vocabulary. And you've probably got a table with a few examples of this. And this was actually a purely something we discovered, we weren't expecting this at all. We suddenly like plotted the predictions of the model and we realized what it was doing. Yeah. So okay, for example, here, if you use the constants 0.3333, you feed it to our symbolic model. Well, of course, it can't directly output 0.3333 times n because it doesn't have 0.3333 in its vocabulary. And so it's going to have to build somehow this this constant with its own building blocks. And you can see that it does that pretty remarkably well. And this is very surprising. It's basically what happened is that during training, it has seen some expressions, because our expressions aren't simplified, right? So so we don't have something that is going to evaluate the expression. So sometimes it sees a formula, which has three plus exponential minus six, and it will notice what a numerical value that evaluates to in terms of the sequence. And so it kind of learns to build any constant with its own vocabulary. And it's important to say that you don't like other if I see this, I would first assume that you have some sort of gradient based regressor in there like that approximates these constants for you, but you don't write the model actually has learned that to output the symbolic expressions for particular constants. That's something I think which is a bit rather novel here is that we have an end to end transformer, usually in symbolic regression, you have a model which predicts a skeleton. So even expression without pre factors, and then you sort of fill in the pre factors with a separate solver. Here, our model does the finding the pre factors all by itself. So that's nice in a sense, because it's like mathematically satisfying. And it also gives us some quite nice approximations. For example, here you can see with 1.64493, it outputs pi squared over six. And you may know that that's the sum of the inverse of squares. And I think Euler in his time spent quite a lot, you know, he had to actually found he found this, you know, numerical value, and he spent some time figuring out that it was pi squared over six. So that can potentially be useful for mathematicians. Of course, the drawback of it is that this is a complex process. And if you have a very complex equation with lots of complex pre factors, then our model is going to spend a lot of its attention to building these pre factors. And it's going to make the task more complex. And this is why I think our model isn't directly applicable to like real world problems like, you know, forecasting where you have very complex pre factors in front of each term of the equation. Is there any any other surprising things that you learned in the in the experiments? I mean, maybe unsurprisingly, a model like this is better than Mathematica, which I would have expected because I'm not I'm not a big fan of Mathematica. Like, Stephen Wolfram is cool, but I'm not too too much into the way Mathematica does things except for very, very particular applications. Well, I mean, it isn't that bad. Actually, I was surprised at how how good it was. I mean, it has like these two built in functions, find sequence function and find the recurrence. And basically find sequence function is going to find like non recurrent formula, it verifies. So, for example, if you feed it to four, eight, sixteen is going to say two to the n. Whereas finally linear recurrence is really for when it depends on the previous terms in a linear fashion. And and these are actually pretty powerful because a lot of sequences are linear and Mathematica will always basically get these right. Because actually you can there's a there's a deterministic rule to find the linear recurrence. So that's that's fine. Find sequence function is very limited, of course, and you can see it gives worse results in OEIS. But still, I mean, these functions aren't miles away from our model. I think actually both our models and Mathematica models are struggling a bit with OEIS. They are outside of their comfort zone. Yeah, I think mainly because so one thing I should say is that here we're not evaluating on random sequences from OEIS. We selected those which have a label which says easy, which means that there is a logic behind them. There is a recurrence relation. However, or not necessarily a recurrence relation, but there is the other ones just just to clarify the other ones you gave some examples in the paper of the other ones would be like the number of bus stops and, you know, in successive streets in New York City or something where you can't possibly know unless you consult like some outside knowledge. Yeah, OEIS does have a lot of nerdy, nerdy sequences which are just for the fun of it basically. And but even in the ones which are labeled as easy, a lot of the sequences don't have a recurrence relation, for example, the sequence of primes, the sequence of divisors of n, the sequence of decimals of pi, all these things you can't really predict. And so these kind of hamper our model. So I don't think this is like the best way to show the power of our model. Our model is especially powerful on like the sequences which are built from the generator, which are very complex here in Mathematica. In OEIS, our models are just only a tiny bit better than Mathematica. I wouldn't say it's the most impressive result. And they are specifically also worse than numeric, right? You can see that the numeric models, they do outperform here, and that might also be because one of the distribution shift and two, if there are as well some, even though they're labeled easy, but actually you might still need some outside knowledge, a numeric model at least will sometimes come close to the solution, right? Close enough to count as correct. Yeah, exactly. Yeah, a numeric model is generally going to be better indeed when there isn't a simple formula, but you can still infer logic. It's here. Yeah. Yeah. Sometimes, I mean, you give very, I mean, if you've played a bit with the demo, you'll realize that sometimes you give a very simple sequence for us. And for some reason, the model won't be able to recognize it because it uses our kind of logic, which we can't really express simply as a formula. And the numeric model will be very good at that. So while, yeah, I'm going to quickly open the demo. I hope I have it ready somewhere. And maybe you can tell us, like, is there, like in the course of this research, was there a moment where it like didn't work at all? Or, I mean, you had some basis to go by, right? From the work of, let's say, let's say, Guillaume and Francois. But was there, like, what was the biggest problem that you encountered during this research? To be honest, the, this was the, this was, I was surprised at how quickly we were able to get models working in the first place, at least on the integer sequences. It was pretty quick to get some results from that point of view. As I was saying before, just plugged in our transformer. We just had to build the generator, basically, which isn't that hard. I think what we struggled with a bit was basically finding a baseline to compare with. This is why we built this numerical task, because this is such a a novel kind of path in symbolic regression to look at recurrent sequences that we didn't have, we didn't have benchmarks, we didn't have things to compare to. And, and, you know, it's a bit disappointing to show some results of in-distribution accuracy if you have nothing to compare to. So, yeah, we built this, this new rec model just for that purpose. And, and yeah, in terms of, yeah, challenges, I, I really, yeah, I was, I was surprised. It was much easier than I thought. Okay. It's interesting because I think we interviewed, we interviewed Guillaume and, and co-authors on a previous paper on the machine learning street talk. I asked them, like, pretty much, I think the same question and that they're all, they already said like, no, you know, kind of we plugged it in and it, you know, it worked out and it was cool. So I think this is like, maybe it's, it's forbidden knowledge, but this might be like a field of deep learning where there's, you know, things actually work. You, you, you, you can get, you can get like results. It kind of, it works maybe, or maybe let's say you get started with something that works pretty quickly. Whereas, whereas if you're in like reinforcement learning, you spend months until something actually starts working. Yeah. And the explanation is simple. It's basically just that you have this synthetic task and so you have infinite data. And the big problem of, of deep neural networks is when they don't have much data, then you really have to get clever about how you regularize, how you choose your hyperparameters, how you build your architecture. Here, you can just throw anything at it and it'll work. It'll learn as long as it's got enough parameters. And that's one thing that you have to have a lot of compute resource for this project. And I mean, here, the transformer is, is pretty big and it's trained on a huge, every epoch we train has 5 million equations and, and trained, you know, for like three weeks or something on 16 GPU. So it's, you know, pretty big scale thing. Nice. Lastly, I just want to present this demo you built so people can try this out for themselves. So if I input like one, two, four, eight, and that should probably already be enough. And then I have to like click away and then it will compute. It will tell me the next ones are 16, 32, 64. That's pretty impressive. I want to, I think I, I tried to challenge it a little bit. I like try to do, come up with some maybe, I thought of like a music sequence, like, that, that, that, that, that, that, that, that, that, that, that, that, that, that, that, that, and it's probably too regular. Right. Let's see. I think it'll get that one. Right. So yeah, it will, it will. Okay. That, that's, that is fairly regular if I look at the plot. But yeah, I invite people to go and challenge, challenge your model a little bit right here. You can also choose a sequences of this OEIS database and yeah, check out the model. This is really cool. All right. So I think this, this, is there anything you want to like special that we haven't come to you want to mention about the paper itself? That was, that was great for me. Thanks for your questions. I think that was great for me as well. I, I'm always happy if I can ask like all my, all my dumb questions to the people themselves. In this case, Stefan, thank you very much. Thank you and your coauthors for, for writing the paper and thank you so much for being here. This was really, really fun. Thanks a lot.
 Hello there! Today we'll look at Deep Symbolic Regression for Recurrent Sequences by Stefan
 Dascholi, Pierre-Alexandre Camienni, Guillaume Lomple and François Charton. This is another
 paper where the main part will be an interview with the first author Stefan and I'll just
 briefly introduce the paper right here for 10-ish minutes or so. So if you want to just skip to the
 interview, feel free. We'll go over the paper just so that you know what's going on and there is also
 an interactive demo online where you can try it out and it's a good place to start at what this
 paper is trying to do. So in this paper the authors care about symbolic regression to number sequences.
 They have a model for integer and float number sequences. In this case this is an example for
 an integer sequence. So you can enter any sequence right here. You can see that the sequence that is
 already entered is the Fibonacci sequence and you enter as many terms as you want. Obviously the more
 you enter the more success probability the model is going to have. What the model will do down
 here is it will predict an expression. You can see it correctly predicts the expression for
 the Fibonacci sequence saying that the current element is the last plus the last last element
 and it will predict the next terms for you and it will extrapolate the sequence that you've input.
 So you can do any that you want. I'm very bad at coming up with stuff on the spot.
 2, 1, 3, 1, 4, 1, 5. Let's see if it can get that. So as soon as you exit from the model it will
 yeah look at that. So the quotient which is not even sure what that operation is but
 it divides the sum of the last element maybe by the last element.
 I figured it out somehow. It is not really good at if conditions and this is one thing we're going
 to talk about in the interview. But you can see it correctly predicts the next sequence right here.
 So give that a try. This pinpoint exactly what this paper does. It does symbolic regression
 for recurrent sequences. Recurrent sequences are sequences of numbers that can be somehow
 expressed as a logical rule as a function of the last elements of the sequence. Most
 sequences can be expressed like this. For example they give a bunch of examples right here 1, 2, 4,
 7, 11, 16. So you can see that it's always sort of plus 1, plus 2, plus 3, plus 4, plus 5 and so on.
 Or this function right here these are simply the squares. So the recurrence relation actually isn't
 a recurrence relation at all but it is also a special case of a recurrence relation or this
 formula right here. It can get very complicated. They have a bunch of examples right here of
 recurrence relations. As you can see they can go pretty complicated to express something like the
 final digit of n times n plus 1 divided by 2 or the final two digits of 2 to the n or some maximum
 or anything like this. So the goal of the model is that you input a sequence like this and then the
 model will output this recurrence relation. It will not output the numbers directly of the sequence
 of the following numbers. That's what they would call a numeric model and they also train one as
 a baseline but the model would actually output exactly the formula itself. Then you can use the
 formula to produce the next elements. Now the good thing is we've all seen what happens if you train
 a numeric model on a bunch of data points. Let's say these are your input data points. You train
 a numeric model on that. It will perform pretty well on the data you give it but as soon as you
 go outside of that data, as soon as you extrapolate too much away from the support base of the training
 data without very strong inductive biases, it will sort of do whatever. You can't really predict it
 what it will do where there is no training data. That's why also deep learning relies on lots of
 training data in covering a lot of the input space. Whether that's called extra or interpolation or
 whatnot. We'll leave it at that. But if you have a symbolic regression and the symbolic regression
 actually predicts the correct formula to match this sequence right here like saying ah this is
 just a sine wave, then you can extrapolate indefinitely. Because you have the correct
 symbolic formula you'll be right in all places. So potentially this is a very strong method
 for certain types of problems. This paper considers this a sequence to sequence problem.
 So it considers transformer stacks and this is I guess along the classic transformer stack
 of you have an encoder and a decoder stack. The encoder stack gets fed with the input sequence
 as numbers. So here one, one, two, three, five and so on. That is the input sequence. It is fixed.
 And then the output sequence is the formula that you want to predict. And they predict the formula
 in reverse polish notation of the prefix tree of the formula. So they have an example down here.
 For example, the cosine of 3x can be expressed as this as cosine of multiplying three by x. So you
 would you would sort of load it onto the stack and then work your way down the stack in in this
 reverse reverse polish notation measure. So that would be cosine of mole of three of x, or whatever
 that formula is. And then you try to train your transformer to autoregressively predict first
 the first token without seeing those tokens. And then once you have the first token, you want to
 predict the second token given the input and the first token. There's like there's multi-head
 attention in here. Like there is cross attention over here. There's self-attention in here as well.
 So you can predict your regular transformer stack. So this is classic sequence to sequence problem.
 The only question is how do you obviously encode the input and the output. The output we've already
 discussed, and they have a very detailed description of how they produce the data. So what they do is
 they take a bunch of operators, you can see them in this table, and they make random formulas from
 those operators. They have a bunch of constraints on these formulas, but essentially they make random
 a data set out of just random formulas. So first of all, they sample the number of operators between
 one and a maximum number. In this case, that would be 10. 10 is the maximum number of operators. And
 then they build a unary binary tree with that many nodes. So they for example, they would sample
 two operators right here, like there are three, a relu, a sub and a mod. And then they would build
 a unary binary tree. So relu, then that is a unary thing, right? So it only has one input. So sub,
 that's a binary operation. So it needs two inputs. Here, let's say mod, that again needs two inputs.
 So the second step is to sample the nodes of the tree from the list of operators. Okay, that's what
 we've already done. We've combined steps one and two, sample the recurrence degree between one and
 D max, D max is six. So we're maximum allowed to look back six elements into the past. This is kind
 of a Markov condition. You can say your recurrence relation can only look back six items. That's
 kind of a limit. But most sequences that humans could come up with don't refer back to the seventh
 last element, right? There is usually a way to express it in forms of either the current index
 or the last few like three or four elements at max. Then they sample the leaves of the tree. So
 the leaves of the tree are either a constant with probability P constant, these all these probabilities
 are one third and they stress very much that hyper parameter settings are not very crucial in this
 way. They sample the leaves of the tree. So either it is a constant or the current index or one of
 the previous terms of the sequence. So let's do that. So we'll say here we sample the previous
 term, which is U n minus two, here we sample the index, which is n, and here we sample a constant,
 which is three. So that would result in the formula ReLU of U n minus two minus and then n mod three.
 That would be the formula for this. Then they need to sample initial terms of the sequence.
 So in with the formula, you also need to decide, you know, how the initial terms,
 the initial terms, since we go back two elements, we probably at least two elements at the beginning
 of the sequence. So let's call that one and two. That's we also need to sample that from a
 distribution. You can see here, that's just a uniform distribution from negative 10 to 10.
 And then what's the last sample the sequence length and compute the next L terms. So now we
 say, okay, how much leeway do we want to give the model to infer the sequence? Let's say we want to
 give it five elements. And now we use the formula to calculate the next three terms right here.
 All right, I tried it, it didn't work out, but it is a rather complicated sequence, I have to say.
 But now you see how this stuff is sampled. So you see how the formulas are made, they just define
 a maximum depth of maximum length and so on. And then it just sample random data from that,
 they create a data set, the data set would be this one right here, this would be the input,
 and the output to predict would be the formula in reverse Polish notation. It's a sequence to
 sequence task. That's it. Now during inference, they can do a beam search, they can input again,
 the sequence, they can output different formulas, different, they can start out different formulas,
 and then they can do a beam search and check which of the formulas actually match
 the input sequence that they have already. And they can discard or rank down formulas
 that don't match the input sequence on the first few terms. So that is an additional benefit they
 have from this symbolic regression. Ultimately, they will end up with a formula that probably
 fits the input terms, and hopefully is simple enough. And the simplicity comes from the data
 set, since shorter sequences are more likely to be sampled and longer sequences the model is
 implicitly biased towards easier formulas, which kind of plays into Occam's razor. So that's it,
 that's the method they create a data set, massive data set, they train on random formulas train
 train to predict them from the initial terms, and then they evaluate it. As I said, they also have
 float sequences, but I won't go into that too much. Notably, they do outperform this numeric
 model, the numeric model simply tries to learn the number to number sequence just directly without
 going to the symbolics. So as you can see, the symbolic method is better when evaluating on
 in distribution sequences, when evaluating on out of distribution sequences. And here's a question
 of how do you even do that. There is this database of integer sequences. And after a bunch of filtering,
 you end up with a validation set of 10,000 sequences. This validation set are human made
 number sequences like the Fibonacci sequence or anything essentially that where humans can come
 up with some sort of logic of how the sequence is generated. On this data set, they don't perform
 as well as the numeric model, as you can see right here. So the numeric model outperforms
 the symbolic model. But there are good reasons why that might be. And we also discussed this
 in the interview. Lastly, they also make do experiments with robustness to noise,
 which are also very interesting in that they can even suffer from a bit of noise if they train
 with the noise. And so the model is even a bit robust and can still do symbolic inference,
 which classically, if you have a symbolic system, these are usually not that robust to noise,
 because it's more like hit or miss. But if you train appropriately, you can handle that. Also
 interesting is that they encode the numbers not as continuous values in the transformer,
 but actually as tokens. So at least for the first 10,000 numbers, they are all their own tokens.
 So the number 19 and the number 20, they're just two tokens. But it turns out that if you train the
 model, then in the embedding space, the tokens will actually form a sort of continuous, not
 necessarily line, but a continuous manifold in the embedding space, which is really cool to see that
 the model, even though you give the numbers as different tokens, it learns to map them out
 according to their numerical values. They also have investigations into the similarities between
 embeddings and they uncover some interesting structures where similarities are also according
 to the numbers like common denominators and so on. And they give a bit of evidence that there seems
 to be kind of a natural base for mathematical operations of multiples of six and 12. And they
 say that six is a natural base for reasoning, reminiscent of much earlier explanation by other
 people. And you might know this cult of people, I don't even know what they're called, but this
 cult of people that says we should just switch to base 12 because it makes everything easier.
 So there might actually be, you know, stuff behind that, or it might just be a artifact of how
 we do math. Who knows? They experiment a bunch of stuff with expression simplification and so on,
 but the model seems to be quite robust to any of these modifications. I think this is a really
 interesting work in that symbolic inference, I believe, can lead us forward and tackle problems
 of extrapolation that we aren't necessarily going to be doing with these numeric models that we
 currently have. Obviously, this has its own limitations and its own biases built in.
 Most notably, how you construct the data set is very, very crucial to how the model is then
 going to perform. But it is interesting to see that you can train it like this. And essentially,
 it's a, you know, it's a it's a free free training data because you can just generate it by yourself.
 So without further ado, I want to jump directly into the interview because we go over the important
 aspects of the paper. Again, let me know if you like inter like interview content like this,
 I think it's super duper helpful. And the interview was very fun. I hope you find that as well.
 All right. See ya. Welcome, everyone. Today I have with me right here Stefan Daskoly, who is the
 first author of the paper Deep Symbolic Regression for recurrent sequences. Stefan, welcome. Thank
 you very much for being here. Yeah, pleasure. Bad timing to have COVID, but I'll try my best.
 Yeah, I hope this goes I hope this goes over relatively smoothly for you. But yeah, so this
 paper, I have to say it gathered quite some hype online, right. And because symbolic mathematics
 is something that is still still even though computers are very good at math per se at numerics,
 symbolics is something that has been maybe in the human domain a little bit more, especially these
 kind of sequence guessing, right, it seems to be a very, very human thing, something you would do
 maybe in high school to try to like figure out some sequence and figure out the rules behind it.
 What sort of what prompted you to go into this direction in the first place? Like why do you
 why do you think this is a fruitful direction? Or, you know, what made you come up with an idea?
 I know there's some previous work, but you know, why this? Yeah, so as you say, I mean, this kind
 of problem is very common, like IQ tests. So that was definitely one of the motivations. So
 originally, this project was born from Francois and Guillaume, who have been both working on
 papers first. So basically, deep learning for symbolic math for a couple of years. And what
 they've been exploring is several directions. The first one of them was a paper in 2019,
 called deep learning for symbolic regression, where they basically did symbolic to symbolic
 manipulations, basically just integrating functions, solving ODEs and stuff. And then
 more recently, Francois has been working on a numeric to numeric task involving math,
 which is basically doing linear algebra. So taking a matrix and then outputting its inverse or stuff
 like that. And so a natural continuation of this was to start from numeric data, and go to a
 symbolic formula. And that's basically symbolic regression, which means you take a function,
 you only see its values, and you have to try and infer the expression of the function.
 And indeed, it's kind of surprising that this has been studied quite a lot for quite a few decades,
 actually, this symbolic issue, the symbolic regression question, especially with genetic
 algorithms and stuff like that. But there hasn't yet been in the machine learning literature,
 a paper working on sequences. And as you said, it's a very common setup for us humans. And so
 this is originally the motivation. And so Francois came to discuss with me and Pierre Alexandre.
 Pierre Alexandre is more from the reinforcement learning background, which is also relevant to
 sequences because you have basically a sequence of states. And for me, it's because I came from
 the physics background. And this is also symbolic regression is useful also for physics for like
 inferring laws, etc. So yeah, that's kind of how we got together.
 Cool, excellent. And just so we're clear to anyone, the kind of sequences we talk about,
 we have a bunch of examples right here. So that would be, for example, here, the final,
 the final digit of n times n plus one divided by two, that's kind of the formula of all possible
 pairwise connections in a group of n points. Or is that n times n minus one?
 Times n minus one. Yeah, the sum of integers.
 Okay. And from that, we just want the final digit. So this the sequence here is 0136051865.
 That is, it is, it is, I would call it pretty complicated if you just gave me this as a human,
 but there is some kind of a rule behind it, right, that I can figure out. And that's the
 type of sequences you would, you would consider. This one is actually a good example. It's kind of
 hard to recognize for us. And if you look at the formula that the model gave us, you can actually
 figure out why it predicted that formula. It's un minus one plus n. And the reason for that is
 that nn plus one divided by two is the formula for the sum of integers. And so the way it built this
 formula is just to take Pries-Dulce turn, add n, and then take the modulus respect to 10, because
 that gives you the final digits. So it's kind of a clever thing that, you know, would be kind of
 hard to figure out for us. Yeah. So if you, if you could maybe give the pitch of your model itself,
 like the pitch of your paper itself, just before we get into more of the details, it's always
 super interesting to hear from the people themselves describing something like
 a brief pitch of what you did here. Yeah. So I think our starting point was less ambitious
 than what it came to. So we originally just started off from the, this sort of thing that,
 that is quite popular for math lovers, which is the OEIS database. So the online encyclopedia
 of integer sequences where you have all sorts of sequences, you can play around with them. You can
 you can try and guess the next term. It's quite fun to play around with. And the idea was to try
 and build a model which could complete the sequences. So sort of understand the logic
 behind the sequences. So originally we only started off with integer models. So we only
 wanted to predict integer sequences. And, and we actually realized that that was pretty easy.
 Pretty quickly, we managed to get a model working on integer sequences. And so we then started to
 think about, can we do the same thing for float sequences, which are a bit more challenging
 because you have more freedom in the expressions you can build. You have more operators, you have
 cosines and exponentials that come in. And, and so this is how we sort of, I'd say it was a lot of
 serendipity really in this work. We started off with this integer sequence problem, and then we
 figured out things as we were going on. So as you can see on the two tables you have there,
 the constant approximation thing, which we may discuss a bit later, was one of the fun side
 effects of trying to guess sequences. It's that you actually, the model actually learns to do stuff
 it wasn't trained for. And so yeah, I'd say the goal of the paper isn't to provide a, you know,
 a model which is useful for real world data. It's not going to be able to predict, you know,
 the stock market or weather forecast, et cetera. It's more of a like proof of concept of what you
 can do with transformers in terms of math. And you specifically restricted yourself to,
 to recurrent sequences. And it, I think it's important to point out sort of what,
 like what kind of inputs does your model take and what kind of outputs does your model give,
 right? Because a formula like, like these, they are, you know, written down in many ways. There's,
 there's ambiguities and I would guess the inputs are these numbers right here, right? So our model
 gets this as an input and then it's somehow has to predict the corresponding formula. So this is,
 the training data is also like this. How does it take the input and in what form does it output
 stuff? Okay. So those are like the two, two big questions. So maybe we can start with the,
 the inputs. So that's actually quite a tricky question. How do you feed in these, these inputs
 to the model? Because, you know, typically deep learning models don't, don't take like, if you
 think of a sequence, which is like an exponential, you're going to have very huge numbers. If the
 exponential has a positive sign and very small numbers, if the exponential has a negative sign.
 And so if you just feed these kinds of values into a deep learning model, it's not going to learn
 much, especially that here we're dealing with a transformer model. So you're going to have a
 transformer because essentially what we want to output is a mathematical formula, which is just
 like basically a language. And so this is why we use transformers. And so transformers need to take
 in embeddings. And so we need somehow to represent our input numbers as embeddings. And that's
 complicated because of course, integers, just like reals are an infinite set. So you have to sometime,
 somehow find them, find a way to encode them as a fixed vocabulary. And so this is where we really
 have to distinguish our two setups. We basically have two different transformers, one for integer
 sequences and one for float sequences. So the integer model, what it does is basically it writes
 numbers in a base B representation. So for example, for the number, like, yeah, exactly like here,
 325, you could imagine writing it as three to five, in which case you only need 10 tokens,
 which is numbers between one to 10. Actually, it turns out that it's better to use a larger base
 because if you use a larger base, well, you're going to have a bigger vocabulary, but you're
 going to have shorter sequences. And typically, you know, transformers have quadratic complexity.
 They struggle a bit with very long sequences, which is why, yeah, we prefer to use a large base.
 Here we use 10,000 as our base. Yeah. So this will be base 30. And obviously in base 10,000,
 I think it's important to note that every single number from zero to 9999 is its own token, right?
 The model has no inherent knowledge of, you know, three comes after two and four comes after three
 and so on. All of this has to be learned. It seems so weird to say, you know, it is better
 to make the model learn essentially the entire ordering of 10,000 numbers rather than, you know,
 providing that as some sort of a, just to make the sequence a bit shorter, right? It's funny.
 Did you ever think of going with continuous values, right? Because the first, my first intuition would
 be that I feed the actual number, right? And then it's implicit, like it's in the number that two is
 larger than one and three is larger than two. Exactly. Yes. So that's what's really interesting
 is that that is one approach. And actually we had a couple of discussions on this, like how can we
 feed in our inductive bias on numbers directly into the model. And well, I mean, the problem with
 this is that here we're dealing with like just one dimensional vectors in some sense. Transformers
 need, you know, high dimensional vectors as inputs. And it's not obvious how you represent these
 numbers in a high dimension, you know, because the, as I was saying just before, the problem is that
 these numbers have very vastly different scales and, you know, deep learning models usually take
 normalized inputs. And so it's not obvious how you would, so what you want to do is basically map
 these numbers you have onto a sphere. And it's not obvious how you would encode, you would put these
 numbers on the sphere. And so one very simple way is just to put them randomly on the sphere and let
 the model decide all by itself how to put them in this sphere. And this is what we do. And what's
 interesting is that when you plot after training what the embeddings look like, you can see that
 it has learned in some sense our inductive bias of putting the numbers in order, et cetera.
 So these are, these are t-SNE plots right here. The left would be the integer embeddings. And it
 sort of forms this, this string. What do you make of the t-SNE plots here? Do you think these things
 are actually, you know, uniformly on a sphere or does the model just use like a tiny part of the
 sphere where it can make sort of a continuous path? Well, what's for sure is that the, it's definitely
 a low dimensional representation because you can see that the t-SNE is actually very, really shows
 a smooth pattern. Usually when you plot t-SNEs of like word embeddings in NLP, it's going to be a
 bit messy. Like you're going to get clusters, but it's not going to be as well organized as here.
 So clearly the embeddings are lying somehow in a low dimensional manifold. And so then you could
 think, okay, so why do we need like 512 dimensions if it's only using a small amount of them? But
 that's actually because, you know, the transformer is going to eventually use these extra dimensions
 to perform its calculations really. So it's not as if they're wasted. They're actually going to be
 used by the model. Yeah. And the float embeddings are very similar, right? In that you encode them
 as like a sign, a mantissa and an exponent. And again, the mantissa, if I understand correctly,
 same deal that you have a token per number between zero and 10,000 and the exponent,
 is that correct that you say you have exponent from negative 100 to 100? So one token would be
 E minus 100 and then another token would be E minus 99, E minus 98. So these are all different
 tokens. So now the transformer has to learn kind of two different embeddings. Both are somehow in
 sequence. Exactly. Yeah. So just to summarize, so for the integers, we encode the integer as
 the sign followed by tokens of the base B representation of the integer. And so for
 floats, we also have the sign token. Then indeed we have the mantissa token. So here the difference
 is that we only have one token for the mantissa. We don't have like a base B representation,
 which means that we do lose some information in the discretization process. And then indeed to
 represent the scale of the number, we use an exponent embedding. And that indeed goes between
 minus 100 and 100. And so here indeed we do plot the TSNE of the exponents because they really have
 a logic to them. For the mantissa, it's less obvious. If you plot a TSNE of the mantissas,
 it would look a bit anarchic. But here the exponents, you can, and actually just about
 this plot here, this plot is actually a tiny bit disappointing because we can't see some of the
 really interesting features we had with our first models. This is with the very big, big model,
 with embedding dimension 512. Actually, when we were using a smaller model with a smaller
 embedding dimension, we saw a really neat pattern, which was basically the fact that the model was
 learning the arithmetic properties of integers. So it was basically creating a line with two,
 four, six, eight, 10, etc., then three, six, nine, etc. And here it's a bit less obvious probably
 because the big model was learning something even more complex that we can't interpret as easily.
 If you go into the appendix, you do see actually a figure where we see that the model learns like
 a base six representation of the integers. The attention plots, you mean?
 Actually, not those ones. Yeah, those ones exactly. Like if you zoom in a lot on the left plot,
 you kind of see these diagonal lines which are spaced out to every six and every 12,
 showing that basically the model is recognizing numbers which have common devices and is
 specializing to the base six or 12 representation, which is often considered better than the base 10
 representation. So these plots, just to make it clear, these are the cosine similarities between
 each of the tokens. So the tokens would be distributed on the axes here. These are tokens
 and these are tokens. And then we plot the cosine similarities between every two tokens. So naturally,
 obviously, every token is going to be very similar to itself, but also very similar to
 its immediate neighbors. So it seems to really learn the ordering of all the tokens. But then also,
 yeah, what I found special, there is this structure of the common factors, common divisors
 between the tokens. That's really cool. Yeah. One thing also that's hard to see in this big
 model, which was much clearer in a small model, is you could see, for example, the perfect squares
 would be complete outliers. You would get 9, 16, 25, 49, which would completely stand apart
 due to the special properties. I think that here, so here is 49, right? That kind of stands out,
 right? Yes. This gap. Yeah. That's something which we haven't really been able to understand.
 Some guy sent me an email actually saying, oh, maybe I have an idea that there's a gap between
 46 and 48 because 45 has lots of factors of five and three, whereas 48 has lots of twos.
 There must be some explanation or maybe it's just something due to optimization. It's very hard to
 know. Okay. Yeah. I think at this point, it's a bit also important that we look at the data generation
 process. You give the model a bunch of options, right, to generate sequences. And these are,
 where do I have them? So here, we have the operators that it can use. On the left-hand
 side are the integer operators. And then the float operators would be in addition to the ones on,
 or sorry, they're repeated in part, but also there are more in the float formulas. And then
 you just generate in reverse polish notation. Is that correct? Exactly. So you generate reverse
 polish notation formulas given these things. And you can also have integer prefactors, right,
 for all the things. So either you sample integers or you sample the current element index,
 or you sample previous elements of the sequence. So the model could express, you know, if it's the
 fifth element, take that current number times the previous element plus two times the cosine of
 something either a constant or again, referring to some previous element or something like this.
 Is there a logic behind why you chose the, why you made these choices of how you generate
 these formulas? So actually, if you look at this table, indeed, there are much more operators for
 the real case, the floating point numbers, but you do notice that in terms of binary operators,
 there are two which you can see in the integer setup, but you don't see in the float setup,
 which are integer division and modulus. And this really illustrates that we're trying to learn
 rather different things in the two setups, really in the integer setup, we're focusing on sort of
 arithmetic and arithmetic properties of numbers, whereas in the float setup, we're really interested
 in a, let's say a more classic symbolic regression problem with complex operators. And yeah, as you
 said, our generation process is basically to build a mathematical tree. So a unary binary tree,
 this is like previous works by Francois and Guillaume. And then indeed, we fill in the nodes
 of these trees, either with operators. So the nodes are filled in with operators, either binary or
 unary. And then the leaves of the tree, indeed, as you said, can be either variables or constants.
 And as you said, the choice of generators actually basically the hardest part, let's say,
 of this problem, because one thing that's nice when you do these kind of symbolic math problems
 is that you basically have an infinite data set. Your data is just synthetically generated. And so
 you can train as long as you want. You don't have any sort of, you know, you don't have any
 overfitting issues. You don't have to regularize that much. You don't have to, even the hyperparameter
 choices aren't that important. What is really crucial here is like how you build your formulas.
 And that's what makes the problem, I think, really quite fun to play around with, because it's a bit
 like, you know, teaching a kid how to learn maths, like you really have to figure out what is the
 best thing to show the model at what time and what is going to you want the data set to be kind of
 hard, so they can deal with complex cases. But if it's too hard, it's going to learn more slowly. I
 mean, it's really an interesting problem how to generate the data. And you decided just by playing
 around because so you do have, as we said, you have these particular ingredients. And I mean,
 you can always say, why didn't you have more or less and so on. But you know, you have a table of
 a bunch of operations that you can do, you decided as well to make to allow the model to use these
 sort of recurrence relations, right to allow the model to say, not only I want five times n plus
 two, but I maybe I want five times n plus two times the previous or the time step, two steps back or
 something like this. Is there a reason behind, you know, including these recurrence relation? Is that
 just something you thought would be more interesting? Or did you look at the database and see that
 that's a lot of how these sequences are made? It's true that often people look at the problem they
 want to solve in order to choose the parameters of their generation. For example, sometimes people
 use different weights for how to sample which operators to sample, like they'll put more
 additions and multiplication or they'll here we have, for example, if you go right to the left
 here, we have these hyper parameters for our generator. For example, you can see here the
 probability of choosing a constant leaf or index leaf, so n or the previous term. Well, yeah,
 probably we could have like tuned these parameters somehow, but here we really wanted to have the
 simplest choice possible on the rationale that basically our data set is so huge that
 eventually we're going to see all possible formulas at some point. It doesn't matter that much,
 the specific values we choose, and we don't want to tune them to a specific problem.
 And so this is why we really chose like very standard and also for the operators, like we
 didn't use any particular probabilities with which to sample such and such operator. We just let
 everything as general as possible. And this would be, so this is built up as a tree because
 naturally you can parse these things as a tree, you can generate them as a tree to have the sort
 of correct grammar, but ultimately you end up with, as we said, this reverse polish notation,
 which is a sequence, right? So this would be one such formula, not you wouldn't have x,
 but you would maybe have n or something like this. So, but ultimately this results in a sequence
 of tokens, right? So the input, your model is these numbers encoded in tokens and the output
 is a sequence of these symbolic tokens. Yeah. Did you also investigate sort of the
 the embedding space of the output vocabulary? Yes, actually a good question. So we did look at that
 and actually it didn't have any particular structure. You could have expected maybe like
 cosine and sine are going to be close to in the embedding space. I think what's happening is that
 the output space is actually much smaller, right? Because in the input space, we have a lot of
 tokens, like we have for integers, we have one to 10,000, that's like 10,000 words. So it really
 tries to find a structure in the inputs. For the outputs, we only have a very small vocabulary
 compared to usual NLP tasks. We only have like about 30 operators. And so essentially if you look
 at the high dimensional space and you do it t-sne, you won't see much because it's just
 equally spreading these operators in the sphere or something like that. There isn't
 much logic to it here. And how, let's say, how universal are these sequences, right? How many
 sequences that I could come up with freely would be inside of the scope of your model? And like,
 are there, is there a significant class of sequences that your grammar could not express?
 So with this unary binary tree representation, you can pretty much represent any function. So
 of course, there are some sequences which don't have any logic to them, which aren't generated by
 a recurrence formula, in which case you can't represent these sequences. And that typically
 is the case with most of the sequences from the OEIS database. So we had to get rid of quite a
 lot of them and do some filtering. Now, I did say that you can represent any function, but
 there is a limitation. There is that some functions are very difficult to express with this
 tree approach. If you think, for example, of the collapse sequence, where basically for
 odd numbers, you multiply by three, add one, and for even numbers, you divide by two,
 that's a rule which is possible to express with a mathematical expression. Essentially, what you do
 is write it as n modulus two times what you do if it's even plus one minus that. But that's kind of
 an involved way to write it. And generally, the model is going to struggle to output that because
 it won't have seen it much during training. That's one important thing also, which we might discuss
 a bit more, is that our model is biased to the likelihood of the expression to be generated
 during training. Yeah, it's like a hack that we as programmers have for an if condition. It's
 just something we learned at some point. Oh, look, if you have an if condition, you can express it
 as if you, I don't know, people program NumPy or something like this. That's exactly what you do.
 You don't say if, you make your mask with one minus whatever condition and you multiply by this,
 and then you have that. And I think anyone who programs NumPy or TensorFlow or so on knows,
 okay, I can do it like this, and then my stuff is expressible and differentiable as one formula.
 But I think that's a hack we learn. And if we just generate data at random like you do,
 this is not something you come across as often as we come across when we program.
 Exactly. Yeah, it's very unlikely to see this formulation in our datasets. Yeah, absolutely.
 Okay, cool. But at the end of the day, you generate a giant dataset, right? You go through it with
 transformers and you emphasize transformers. Is there something special about transformers?
 Because couldn't I use any deep learning thing or why transformers?
 Well, first of all, like previous experience, I mean, Guillaume and Francois have been working
 on these transformers. They've basically always been good at the problems we've given them.
 Likely, one natural justification is that as we saw for the outputs, you can represent math as a
 language in a very easy way. It's actually, we can see here that it's much easier to use
 the inputs as tokens, but the formulas themselves are very easy to represent as a language with
 this Polish notation thing. And so it's very natural to use transformers because they are
 best models to deal with language. So yeah, I think that's the main reason. And yeah,
 I'm not sure what else we could particularly, I mean, we could use like RNNs, etc. But these
 days transformers are so powerful. I mean, these models we used, we didn't even, as I was saying
 before, we didn't have to tune them much. We just basically took the same architecture that was used
 in the paper two years ago. We didn't even have to change the learning rate. Like it's pretty
 amazing how easy it is to train these things. Okay. Yeah, so the transformers are a natural way
 to deal with sequences. And from text learning, we kind of know this, but we always learn sort of
 on human text, right? And that has a particular structure. And I want to think if I look at these
 sequences, there are almost like, there's so many symbolic formulas that could possibly explain
 these sequences. And yeah, I can, you say you make, you want maybe the simplest sequence or you
 know, you don't want your formulas to blow up. That's, you even generate only formulas that are,
 let's say, relatively simple. So there's clearly a bias towards simplicity, but still there are a
 lot of things that explain the same sequence. So I'm thinking more, is it like if when we as humans
 do these tasks, is it like a property of humanity and civilization that we kind of come up with the
 same sequences that the person you know, who made the riddle came up with? Is it because we kind of
 think alike, right? Because of whatever society or our environments that shaped us? Or is there
 like a property of math that says, well, if actually if you look for the simplest sequence,
 it is kind of defined even though there are infinite possibilities. Like, you know,
 a little bit what I mean, is it more like a property of humanity or of mathematics?
 I think it's probably two different things. So as far as humans is concerned, indeed, we
 we tend to prefer simplicity. That's like our OCam's Razor principle. We like going for the
 compressing information and going for the simplest representation. In terms of our algorithm here,
 we didn't put at all this simplicity inductive bias from our own understanding of the system.
 We didn't put the inductive bias from an explicit point of view. We didn't tell the model, give us
 the simplest formula. Actually, we could have done so because we could have, for example, given a
 penalty to like the decoder when it generates two long sequences, for example. But we didn't have to
 do this at all because the inductive bias comes from the fact that simple formulas are more likely
 to be generated by the generator. And that's basically the rationale behind our model is that
 it's always going to be biased towards the most likely formula corresponding to the sequence.
 And as we were saying before, sometimes that's not good because for the collapse sequence,
 it's going to struggle to output the one minus the mask thing. But in general, that's kind of
 what we want in IQ tests. We ask for the simplest formula to explain the observations.
 Mm-hmm. I'm thinking of are there more things rather than just number sequences where something
 like symbolic regression could be valuable? For example, I've always thought of maybe
 reinforcement learning would be much more powerful if we didn't only... Even if agents have a world
 model, what they call a world model, they usually have almost like a numeric world model. They just
 forward predict the values that are going to happen there. I always thought, well, if I had
 a symbolic representation of the world, I could do much more powerful planning. Are you thinking of
 applications like these when you develop this, right? Beyond number sequences? Or is there any
 interesting ones that come to your mind? So as I was saying, Pierre-Augurelien,
 my co-author, it comes from reinforcement learning. And there have already been a few papers
 inserting some symbolic parts into RL loops. And that's definitely going to help. Indeed,
 as you say, if you're a robot and you're trying to understand the world, then it's going to be
 much easier if you understand Newton's law. If you want to, for example, predict how objects are
 going to move, it's much easier once you understand Newton's law than using a specific vision model
 to try and predict. That's going to be much more complicated. So indeed, I think symbolic
 regression is going to be very useful for RL. From my point of view, I'm more from the physics
 background. And that's also a domain where symbolic regression would be very useful.
 Because typically, so we have these two approaches, right? We have numeric regression and we have
 symbolic regression. And I think they're very complimentary in the sense that numeric regression
 is very good on complex tasks where you don't necessarily have a simple explanation for the
 data. And symbolic regression is great for inferring data where you have a simple underlying rule,
 typically in physics, like inferring laws from observation. So yeah, I think RL and physics are
 definitely two huge domains of application for symbolic regression. And to make this a bit
 clearer, so what I've done is in the appendix, you actually have some success and failure cases
 of your model. And so I have made a little quiz out of them and hidden a bunch of them right here.
 And I just want to draw people's attention a little bit to some of the some of this. So on the left,
 the left three columns are success cases. And the right three columns are failure cases, both of the
 integer model, right? So these are integer valued sequences. And do I have this correctly,
 you do consider it only a success if the formula is equivalent? Or do you consider it already a
 success if just the predicted values are the same? You can have the two criteria and the criteria we
 choose in the papers, we want the the evaluations to be the same. So even if it comes up with like
 a different formula, it's it's fine as long as like that the ones you tested on match. Yeah,
 that's actually one tricky thing is that indeed, you can't really rely on the formula to check
 if it was correct or not due to the degeneracy. And so some papers have circumvented this by
 using like an RL loop. Because if you try to really supervise the formula, then you can't make some
 you have to evaluate the formula, which is non deterministic, and then you can't like back
 propagate this. And so some people have used sort of RL loops to provide reward signals from the
 evaluations. What we do is directly supervise the tokens of the formula. And, and that, okay,
 maybe we can discuss this a bit later. But that's also interesting, because, you know, you could
 think this is weird, because our model is supervised to a formula. And it's going to be penalized if it
 outputs at training an equivalent formula. Yeah, but that turns out to not be too bad. And we tried
 we tried we tried expression simplification, and it didn't help at all. It doesn't really matter.
 But yeah, this is very interesting what you're going to come to with the success and failure cases.
 Yeah, so the leftmost column here is is is pretty simple. These are okay, people already know it's
 success cases. So in nothing too unexpected right here, like it figures out that for example, the
 middle formula, this might be a bit small here, even for people to read. But this is n, n times
 the sine of gamma. And gamma is what exactly? Euler's constant, Euler's constant. Okay, so n
 times the the sine of gamma squared. So the entire thing on the right hand side is a sorry is a
 constant, right? So it's essentially n times a constant. Yeah. So the the model what it has to
 do is it has to somehow figure out the expression for the constant as a formula, right? Because it
 it can't it, it, it, it has to Yeah, it cannot just predict the number. And then it has to realize
 that I have to multiply this constant by n. And that's why it's a straight line. So and the other
 formulas are similar ish. The top one, for example, is n minus the cosine of n. And yeah, again,
 reminder, these are this is symbolic, symbolic regression. Now, the next ones are weird. So here,
 the top one, it starts off very, very weird, but then it continues in the same path. And you can
 still you can see sort of, okay, it's regular enough that the model could, you know, figure it
 out from the data points it has, by the way, that the green background, that's the input, right,
 the blue background, that's, that's the what it has to predict. So the next one I find particularly
 interesting, it is the formula is the tan of the tangent of n plus n times the last element. And
 this is what the output looks like. So, you know, how like, how can the model from the just the left
 part figure out that this is the correct formula? And then the the end date that just blows my mind,
 like, how does that work? Maybe the log scale would help a bit here, because there is probably
 quite a lot of variability in the in the first terms. And it's just squashed by the last term,
 which is huge. Okay, yeah, I should have made me put a log scale. That's a good question. Yeah,
 what is what I find really interesting with these plots. So here, you're showing the success plots.
 And on the right hand side, you have the failure plots, is that we really see how symbolic
 regression is different from numeric regression, like in numeric regression, you have this set of
 points. And basically, you're just trying to fit your function, you're trying to bend the function,
 so that it goes through the, through the input points. And so this is typically going to be very
 prone to overfitting, right? If you can't really understand the process, then you're just going to
 fit a function which goes through the points, whereas symbolic regression here isn't biased
 towards overfitting at all, it's just trying to find a formula. And so when it fails on the
 right hand side, it not only fails outside the input points, but also on the input points,
 it's not even able to fit the points you gave it. Yeah, this really shows a big difference.
 We can see this a little bit, I think. So on the bottom left, there's a there's a nice case,
 where it can it already fails. Yeah, on the inputs, like that's the best formula it can come up with,
 you do have a beam search in there, right? These ones, no, no, these ones, not even okay.
 Search does tend to pull a bit more towards overfitting because in search you so the way
 we rank our beam is that we evaluate how, how well the formula matches the input points. And so in
 that sense, you're coming a bit closer to like actually overfitting the input points. But if you
 use a beam size of one as using most of our experiments, then essentially, you're not at all
 biased towards overfitting. Okay. Yeah, I mean, this, it seems like here, it's just misjudged
 the formula on the top left is an interesting one, where it just it looks like it's done
 everything correctly, right? It looks like so the red ones are the the outputs that it's supposed
 to match. And the black one is the the line the function it produces. What's wrong here? Is it
 like off by a tiny bit? Yeah. So the screen is pixelated. So I can't see very well. But yeah,
 um, essentially, we get two kinds of mistakes, we get the mistakes where it's very close, for example,
 it confuses a like a four with a five. And so it's going to be very close. But then you have
 catastrophic failures, where basically, for example, to confuse a cosine with an exponential
 or something like that, you know, that's just one token error, but it's going to give completely
 wrong predictions. And that's something that you typically won't get for numerical regression,
 you'll always at least fit your inputs. Yeah. However, there is one thing where symbolic
 regression is better than numerical regression is that once it does find the correct formula,
 then it's going to get predict, you know, perfect precision on all all the, the subsequent numbers
 you're going to give it for if you think, for example, of extrapolating the sequence, with a
 numerical model, you're always at some point going to, you know, get wrong predictions, because you're
 not very good at generalizing outside, yes, typical thing that deep machine learning is
 good at interpolating, but bad at extrapolating. But with symbolic regression, once you've found
 the correct formula, you can basically extrapolate as far as you want, you've got the right formula.
 Yeah. And so just saying for people who probably even people in the video will not be able to read,
 I can confirm the formulas of these two things are completely different. Like the one is
 the sign of something simple. And the one that's predicted is a very, very complicated formula
 that just happens to almost fit or maybe even perfectly fit the input data points, right, but
 then it is just that tiny bit off. And that that gets worse and worse as the sort of the output
 progresses. Okay. So yeah, there are a bunch of about a bunch of other funny ones like this one,
 again, the scale here is absurd. It's like the exponent is 224. And there's just this one output
 that it's supposed to match. And I mean, that's just mean to the model, honestly.
 Yeah, we do have, I mean, horrible expressions, like our generator uses up to 10 operators. And
 so if you look at expressions here, we only chose expressions with three operators. So you can
 imagine how horrible the expressions are with 10 operators. Yeah. And of course, the accuracies
 are much lower. I mean, if you look at the ablation, like our performance at 10 operators
 is about 10% versus, you know, 100% when you have one operator. Yeah. So I will quickly uncover
 the rest of these, but people are encouraged people to actually go and look at the success
 and failure cases. Also for the floating models, I think it's really valuable. And you can directly
 see, as you say, you know, the differences between symbolic regression. And I mean, if you did
 numeric regression, even if it has like a pattern like this, like a zigzag pattern or something,
 it would quickly degrade. We've all seen sort of sort of numeric regression, although as in your
 experiments, so maybe we'll come to this last. So in your experiments, there are cases where the
 numeric regression is worse. And there are cases where the numeric regression is actually better
 than the symbolic regression. Would you want to maybe comment a little bit on the experiment,
 specifically like in distribution, out of distribution evaluation? So typically in
 in distribution, our symbolic model performs better than the numeric model because it's
 got the right inductive bias, right? Really, we feed in these sequences which are generated by a
 formula. And so it's much better than the numeric model at extrapolation because once it's got the
 correct formula, it's going to give perfectly precise predictions extrapolated as far as it
 wants, etc. However, it is slightly less good at out of domain generalization. So one thing you see
 here in is I can't remember where it is in the paper, but you see that, for example, numeric
 regression is better when you have complex pre factors, right? Because here the expressions we
 generate, the pre factors we have are built from like integers between one and 10, e and pi.
 Yeah. And so that's well fitted for the symbolic model. But what happens if you replace these
 pre factors by like pre factors which are sampled from a Gaussian distribution? So these these two
 columns right here, the difference between those. Yeah, exactly. And so what's interesting here is
 that in this case, of course, the numeric regression performs better than symbolic because
 numeric doesn't care at all about the fact that you're using these pre factors because it doesn't
 it doesn't really care. It isn't trying to approximate these complex pre factors.
 What's interesting though, is that the symbolic model still isn't that bad because it's actually
 able to approximate pre factors with its own vocabulary. And you've probably got a table with
 a few examples of this. And this was actually a purely something we discovered, we weren't
 expecting this at all. We suddenly like plotted the predictions of the model and we realized what
 it was doing. Yeah. So okay, for example, here, if you use the constants 0.3333, you feed it to
 our symbolic model. Well, of course, it can't directly output 0.3333 times n because it doesn't
 have 0.3333 in its vocabulary. And so it's going to have to build somehow this this constant with
 its own building blocks. And you can see that it does that pretty remarkably well.
 And this is very surprising. It's basically what happened is that during training, it has seen some
 expressions, because our expressions aren't simplified, right? So so we don't have something
 that is going to evaluate the expression. So sometimes it sees a formula, which has three
 plus exponential minus six, and it will notice what a numerical value that evaluates to in terms
 of the sequence. And so it kind of learns to build any constant with its own vocabulary.
 And it's important to say that you don't like other if I see this, I would first assume that
 you have some sort of gradient based regressor in there like that approximates these constants for
 you, but you don't write the model actually has learned that to output the symbolic expressions
 for particular constants. That's something I think which is a bit rather novel here is that we have
 an end to end transformer, usually in symbolic regression, you have a model which predicts a
 skeleton. So even expression without pre factors, and then you sort of fill in the pre factors with
 a separate solver. Here, our model does the finding the pre factors all by itself. So that's nice in
 a sense, because it's like mathematically satisfying. And it also gives us some quite
 nice approximations. For example, here you can see with 1.64493, it outputs pi squared over six.
 And you may know that that's the sum of the inverse of squares. And I think Euler in his time spent
 quite a lot, you know, he had to actually found he found this, you know, numerical value, and he spent
 some time figuring out that it was pi squared over six. So that can potentially be useful for
 mathematicians. Of course, the drawback of it is that this is a complex process. And if you have a
 very complex equation with lots of complex pre factors, then our model is going to spend a lot
 of its attention to building these pre factors. And it's going to make the task more complex. And
 this is why I think our model isn't directly applicable to like real world problems like,
 you know, forecasting where you have very complex pre factors in front of each term of the equation.
 Is there any any other surprising things that you learned in the in the experiments?
 I mean, maybe unsurprisingly, a model like this is better than Mathematica, which I would have
 expected because I'm not I'm not a big fan of Mathematica. Like, Stephen Wolfram is cool, but
 I'm not too too much into the way Mathematica does things except for very, very particular
 applications. Well, I mean, it isn't that bad. Actually, I was surprised at how how good it was.
 I mean, it has like these two built in functions, find sequence function and find the recurrence.
 And basically find sequence function is going to find like non recurrent formula, it verifies.
 So, for example, if you feed it to four, eight, sixteen is going to say two to the n.
 Whereas finally linear recurrence is really for when it depends on the previous terms in a linear
 fashion. And and these are actually pretty powerful because a lot of sequences are linear and
 Mathematica will always basically get these right. Because actually you can there's a
 there's a deterministic rule to find the linear recurrence. So that's that's fine.
 Find sequence function is very limited, of course, and you can see it gives worse results in OEIS.
 But still, I mean, these functions aren't miles away from our model. I think actually both our
 models and Mathematica models are struggling a bit with OEIS. They are outside of their comfort zone.
 Yeah, I think mainly because so one thing I should say is that here we're not evaluating on random
 sequences from OEIS. We selected those which have a label which says easy, which means that there is
 a logic behind them. There is a recurrence relation. However, or not necessarily a recurrence
 relation, but there is the other ones just just to clarify the other ones you gave some examples in
 the paper of the other ones would be like the number of bus stops and, you know, in successive
 streets in New York City or something where you can't possibly know unless you consult like some
 outside knowledge. Yeah, OEIS does have a lot of nerdy, nerdy sequences which are just for the fun
 of it basically. And but even in the ones which are labeled as easy, a lot of the sequences don't
 have a recurrence relation, for example, the sequence of primes, the sequence of divisors of
 n, the sequence of decimals of pi, all these things you can't really predict. And so these kind of
 hamper our model. So I don't think this is like the best way to show the power of our model.
 Our model is especially powerful on like the sequences which are built from the generator,
 which are very complex here in Mathematica. In OEIS, our models are just only a tiny bit better
 than Mathematica. I wouldn't say it's the most impressive result. And they are specifically also
 worse than numeric, right? You can see that the numeric models, they do outperform here, and that
 might also be because one of the distribution shift and two, if there are as well some, even though
 they're labeled easy, but actually you might still need some outside knowledge, a numeric model at
 least will sometimes come close to the solution, right? Close enough to count as correct. Yeah,
 exactly. Yeah, a numeric model is generally going to be better indeed when there isn't a simple
 formula, but you can still infer logic. It's here. Yeah. Yeah. Sometimes, I mean, you give very,
 I mean, if you've played a bit with the demo, you'll realize that sometimes you give a very simple
 sequence for us. And for some reason, the model won't be able to recognize it because it uses our
 kind of logic, which we can't really express simply as a formula. And the numeric model will
 be very good at that. So while, yeah, I'm going to quickly open the demo. I hope I have it ready
 somewhere. And maybe you can tell us, like, is there, like in the course of this research,
 was there a moment where it like didn't work at all? Or, I mean, you had some basis to go by,
 right? From the work of, let's say, let's say, Guillaume and Francois. But was there, like,
 what was the biggest problem that you encountered during this research? To be honest, the, this was
 the, this was, I was surprised at how quickly we were able to get models working in the first place,
 at least on the integer sequences. It was pretty quick to get some results from that point of view.
 As I was saying before, just plugged in our transformer. We just had to build the generator,
 basically, which isn't that hard. I think what we struggled with a bit was basically finding a
 baseline to compare with. This is why we built this numerical task, because this is such a
 a novel kind of path in symbolic regression to look at recurrent sequences that we didn't have,
 we didn't have benchmarks, we didn't have things to compare to. And, and, you know, it's a bit
 disappointing to show some results of in-distribution accuracy if you have nothing to compare to. So,
 yeah, we built this, this new rec model just for that purpose. And, and yeah, in terms of,
 yeah, challenges, I, I really, yeah, I was, I was surprised. It was much easier than I thought.
 Okay. It's interesting because I think we interviewed, we interviewed Guillaume and,
 and co-authors on a previous paper on the machine learning street talk. I asked them,
 like, pretty much, I think the same question and that they're all, they already said like,
 no, you know, kind of we plugged it in and it, you know, it worked out and it was cool. So I think
 this is like, maybe it's, it's forbidden knowledge, but this might be like a field of deep learning
 where there's, you know, things actually work. You, you, you, you can get, you can get like results.
 It kind of, it works maybe, or maybe let's say you get started with something that works pretty
 quickly. Whereas, whereas if you're in like reinforcement learning, you spend months until
 something actually starts working. Yeah. And the explanation is simple. It's basically just that
 you have this synthetic task and so you have infinite data. And the big problem of, of deep
 neural networks is when they don't have much data, then you really have to get clever about how you
 regularize, how you choose your hyperparameters, how you build your architecture. Here, you can just
 throw anything at it and it'll work. It'll learn as long as it's got enough parameters.
 And that's one thing that you have to have a lot of compute resource for this project. And
 I mean, here, the transformer is, is pretty big and it's trained on a huge, every epoch we train
 has 5 million equations and, and trained, you know, for like three weeks or something on 16 GPU. So
 it's, you know, pretty big scale thing. Nice. Lastly, I just want to present this demo you built
 so people can try this out for themselves. So if I input like one, two, four, eight,
 and that should probably already be enough. And then I have to like click away and then it will
 compute. It will tell me the next ones are 16, 32, 64. That's pretty impressive. I want to,
 I think I, I tried to challenge it a little bit. I like try to do, come up with some maybe,
 I thought of like a music sequence, like,
 that, that, that, that, that, that, that, that, that, that, that, that, that, that, that, that,
 and it's probably too regular. Right. Let's see. I think it'll get that one. Right.
 So yeah, it will, it will. Okay. That, that's, that is fairly regular if I look at the plot.
 But yeah, I invite people to go and challenge, challenge your model a little bit right here.
 You can also choose a sequences of this OEIS database and yeah, check out the model. This is
 really cool. All right. So I think this, this, is there anything you want to like special that we
 haven't come to you want to mention about the paper itself? That was, that was great for me.
 Thanks for your questions. I think that was great for me as well. I, I'm always happy if I can ask
 like all my, all my dumb questions to the people themselves. In this case, Stefan, thank you very
 much. Thank you and your coauthors for, for writing the paper and thank you so much for being here.
 This was really, really fun. Thanks a lot.