On the Measure of Intelligence by François Chollet - Part 3: The Math (Paper Explained)
deep learning machine learning arxiv explained neural networks ai artificial intelligence paper chollet keras google francois intelligence iq iq test deep neural networks prior skill performance measurement measure test number intelligent smart learning generalization ability experience humans evolution nature nurture psychometrics range adaptability arc kaggle difficulty entropy core knowledge objectness navigation contact agent goal
In this part, we go over the formal definition of the measure of intelligence. In order to do this, we have to frame and quantify the notions of generalization difficulty, priors, and experience in terms of algorithmic complexity.

OUTLINE:
0:00 - Intro & Recap
2:50 - Concept Schema
10:00 - Algorithmic Complexity
13:00 - Definitions
15:25 - Generalization Difficulty
18:55 - Developer Aware Generalization Difficulty
22:40 - Priors
25:10 - Experience
30:50 - The Measure Of Intelligence
38:00 - An Ideal Intelligence Benchmark
42:30 - Conclusion

Paper: https://arxiv.org/abs/1911.01547

Part 1: https://youtu.be/3_qGrmD6iQY
Part 2: https://youtu.be/THcuTJbeD34

Abstract:
To make deliberate progress towards more intelligent and more human-like artificial systems, we need to be following an appropriate feedback signal: we need to be able to define and evaluate intelligence in a way that enables comparisons between two systems, as well as comparisons with humans. Over the past hundred years, there has been an abundance of attempts to define and measure intelligence, across both the fields of psychology and AI. We summarize and critically assess these definitions and evaluation approaches, while making apparent the two historical conceptions of intelligence that have implicitly guided them. We note that in practice, the contemporary AI community still gravitates towards benchmarking intelligence by comparing the skill exhibited by AIs and humans at specific tasks such as board games and video games. We argue that solely measuring skill at any given task falls short of measuring intelligence, because skill is heavily modulated by prior knowledge and experience: unlimited priors or unlimited training data allow experimenters to "buy" arbitrary levels of skills for a system, in a way that masks the system's own generalization power. We then articulate a new formal definition of intelligence based on Algorithmic Information Theory, describing intelligence as skill-acquisition efficiency and highlighting the concepts of scope, generalization difficulty, priors, and experience. Using this definition, we propose a set of guidelines for what a general AI benchmark should look like. Finally, we present a benchmark closely following these guidelines, the Abstraction and Reasoning Corpus (ARC), built upon an explicit set of priors designed to be as close as possible to innate human priors. We argue that ARC can be used to measure a human-like form of general fluid intelligence and that it enables fair general intelligence comparisons between AI systems and humans.

Authors: François Chollet

Links:
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
 Hello and welcome to the third part on On the Measure of Intelligence by François Chollet. Now this is a multi-part series. If you haven't seen the first two parts I recommend to watch at least one of them. They're somewhat overlapping but we've basically gone over the history of intelligence measurement and the foundations of what a measurement for intelligence for an AI system should look like. Today we're going to get into the formal definition of the intelligence that Chollet proposes right here. So this sentence here pretty much sums up what we're interested in. The intelligence of a system is a measure of its skill acquisition efficiency over a scope of tasks with respect to priors, experience and generalization difficulty. So these are the things that we've established so far basically. The intelligence of a system that's the thing we want to measure is a measure of its skill acquisition efficiency. So how fast does it acquire new skills? Important here is that we are measuring it over a scope of tasks. So it's not arbitrary skills it is a scope that we define and this is going to be mostly the human scope, the scope of tasks that humans can solve and are sort of different at. What we need to factor in are priors which is what is already built into a system because that doesn't count as intelligence that's already built in. If your ability to solve a problem is already built into you you don't have to use intelligence to solve the problem. Second, experience. If you have had lots and lots and lots of experience at the particular task you're asked to solve you don't have to use intelligence you can simply rely on your experience. And the third is generalization difficulty and that's a property of the task. So if the task is very difficult to generalize so if it's very difficult if the task itself is very difficult then achieving good score at it should count as having higher intelligence if all other things are equal. So this is going to be the basis and today we're going to watch Shirley define these things into a number that can give us the intelligence of any system with respect to these things. So that's the program for today. If you like content like this share it out and tell all your friends and leave a like so that YouTube knows that you do like it. So the conceptualization of the entire system is like this. There is a task and we're going to consider a series of tasks of course but if we just look at one task in our scope there is the task and the task outputs these situations. In a machine learning term these are like your training examples. And on the other side there is the intelligent system. Now the intelligent system in a pure machine learning side you would factor this as the task gives the intelligent system something like a training sample or in reinforcement learning it would be something like an observation and the intelligent system gives something back like a response. Here we have a kind of a in-between step. The intelligent system doesn't actually give back the response to the situation. The intelligent system generates a skill program. So the intelligent system will generate a program that can map the situation to a response and that skill program should be able to run on its own. So in the classic machine learning sense if we look at supervised learning for example the intelligent system would be like a ResNet plus SGD. That is an intelligent system and if it is output it is able to generate a skill program. So during training what happens during training? During training the intelligent system is able to intervene in the skill program at each step. So the situation comes in and then the skill program does something but the intelligent system can at any point it can kind of intervene and update the skill program and generate a new skill program for the next step. So there's a situation the skill program gives a response and the task gives feedback in form of a score. In machine learning terms this would be your training sample. Your training sample comes in, your neural network gives a response which are the logits of the classes, then the task gives a score to that which in the supervised learning case is the label or the loss function as a feedback to the intelligent system and the intelligent system using SGD would update the skill program for the next step. So at each step the intelligent system can update the skill program. That's why the intelligent system in this case is the architecture of the neural network and the procedure to update the weights. Not the weights themselves but the procedure to update the weights and the skill program here those would be the actual weights of the neural network or like the instantiation of the ResNet with these particular weights. Now at test time we sever this connection right here. So this is now severed at test time. At some point the training is done. The task says okay now training is done and then the intelligent system will produce one last skill program and then this connection is cut and the skill program must by itself answer to these situations. The intelligent system cannot intervene anymore and in this loop here it's situation response situation response this goes over for a number of steps and all the scores during that time are counted and tallied up and at the end you know the higher the score the better. So the intelligent system must at this end step produce a skill program that by itself can achieve a high score. So there's always this training phase first and then there is the test phase. Now the training phase these situations that we get in a training phase they are called a curriculum in this in this world. In our world this would be something like a training data set but this is curriculum it's slightly more intricate but just the notion here makes sense right the intelligent system produces the skill program. So there's a lot of formalisms right here like okay the task has a situation generator and that maps the task state to a situation so the task can have a state and the skill program can have a state and the intelligent system can have a state and I don't like this is all a bit too formal you don't really need to understand it except if François Cholet is watching this I think I have found I'm not sure if it's a mistake but you say the intelligent system here consists of three objects so it generates the skill program according to its internal state okay and it generates the skill program and when it learns when it learns it updates the internal state internal state according to let me if I can find it right here a self update function so this is how the intelligent system can update itself so its own state so it takes the internal state of the intelligent system and outputs another internal state and this is the you know where I said the internal the intelligent system at each training step it can observe what happens and basically react accordingly so it takes the situation the response and the feedback and its own internal state as an input now what do we have here it takes the situation which would be in our case the training sample the response or the the logits that the neural network has produced the feedback which is the loss and its internal state okay now what I argue is basically that it should also get the internal state of the skill program as an input right here because the skill program can have an internal state all of this like the response can be a stochastic procedure of the skill program and I guess it's not necessary because you can sort of infer it but I think the framework would be more complete if the internal state of the skill program at that time were part of the intelligent system update procedure just you don't okay this is not relevant this is just me bickering cool let's actually jump all of this this is boring this is very boring okay blah blah blah blah blah lots of definitions all right quantifying generalization difficulty experience and priors using algorithmic information theory so these things that at the beginning we said that we want to define intelligence with respect to we are now going to quantify using algorithmic information theory algorithmic information theory in this case right here that we're using it's not very complicated the main quantity is this H the algorithmic complexity it the H of s is the length of the shortest description of the string in a fixed universal language okay so it's the length of the shortest program that outputs the string when running on a fixed universal Turing machine so basically if if you have this string s right here as is a bit string the shortest program that can compute s or you know so so in the worst case that's the the string itself but if the string is like 0 1 0 1 0 1 0 1 all the way you can just say 0 1 times 50 and that's that would be like the shortest program to produce that it isn't it is an information theoretic concept here but in essence you can just think of it as a measure of how long is the program that I would need to write to output a given to to produce a given output okay so that's the algorithmic complexity and then the second quantity right here is the relative algorithmic complexity which is almost the same thing it's how long is the program that I have to write so the shortest we're always talking about how long is the shortest program that I have to write that produces s1 but is allowed to take s2 as an input okay so it can never it it can always ignore s2 that's always a possibility so if s1 is like a super easy string you can just output that but if s1 let's say s2 here is 0 1 0 0 1 okay and s1 is 0 1 0 0 1 0 1 0 0 1 okay so it's just twice that so you could you could sort of output that string here we could write a program that just outputs this or you could write a program just that just says two times s2 okay so that the the length of this is not part of the program the program is just two times s2 because it's allowed to take s2 as an input okay so this is the algorithmic the relative algorithmic complexity is how how much how long is the how complex is the program to get from s2 to s1 so you can almost already see how that will relate now to to generalization okay so a few quantities that we need to consider are a task called t here then solve t theta is the shortest of all possible solutions of t so task t has a solution of threshold theta which the threshold theta is just this is like the minimum score we need to achieve in a task we don't we consider tasks according to thresholds like you know you need to get a I don't know what a score of 9000 in Pong or so so the shortest of all programs that will will optimize that will solve the task to a threshold t sorry theta which is the shortest scale program that achieves at least theta during evaluation and the other quantity is this train soul opt TC this has a lot of quantity right here you can see the task we want to be optimal but with respect to a curriculum okay the curriculum is the training data so this quantity is the shortest optimal training time solution given a curriculum so it's the shortest scale program that achieves optimal training time performance over the situation in the curriculum so this this right here is if we could if we had an oracle that told us here is how to solve the task in general like the task of the task of of determining cats from dogs and images this this is this is this would be the program that does it okay you know overall over the entire the entirety of the task all cats and dog images there there are that's the solution now this quantity right here means sort of the the one neural network that is best at disturb determining cats from dogs in this particular training data set this curriculum see okay so this is the the one neural network that is hyper optimized in this particular training data set and now we assess the generalization difficulty so the generalization difficulty is going to be a measure of how hard is it to in a particular task to generalize to the whole task from the curriculum see and that's going to be the relative algorithmic complexity to go from this quantity to this quantity both quantities we've just explained so it basically means if if I had the perfect solution on the training data set how much how much more complex is it to get from that to the perfect solution on the entirety of data or you can also guess on the test data set right so if if this is really easy so if the training data set already perfectly captures all of the data there is that this quantity is zero like the out the program I don't need to write a program I already have the solution right and you can see here we divide by the age of salty but however if the training data has no information whatsoever about about the about the solution to the general task or if I just so horribly overfit on the training data such that it doesn't help me at all for the general task then this quantity is zero so this quantity is in zero one with with sorry the quantity is one of course yes because the shortest this thing up here will be equal to just h of salt e because this doesn't help me in that case and then this ratio will be one so generalization difficulty of one basically means that the training data solution doesn't help me at all this this particular training curriculum is useless because I'll just overfit so horribly that I will not learn anything about the task or I can't learn anything at all and generalization difficulty of zero oh that's yeah no yes generalization difficulty of zero basically means that all of the solution is already contained in the training solution and I require no work to get to the to the test set solution okay this is I mean I do this train test that this is all a bit more general as it is written here but I think it's a good a good way to think about it okay so the point here he makes is that is that so yeah he makes this example right here where he has these two data points where x minus point seven five has a label false and x point one five has a label true and the shortest possible solution will not help you to generalize to the to the other things so the nearest neighbor program would be better prepared for future uncertainty but would take significantly more space to write down so there's there's a trade-off there's direct trade-off to how much you optimize on the training data and how much generalization capability you have okay so the next quantity we want to assess is developer aware generalization difficulty because so far we've only considered generalization difficulty with respect to the task itself and to the to the curriculum but what you could do is you could simply you know you producing this intelligent system you could simply build in the solution to the entire task into your intelligent system that means it could completely ignore the training data and still perform pretty well on this thing even though even though the training data itself the algorithmic complexity it tells you nothing about about this so the generalization difficulty would be very high in the measure up here but so you would think wow this intelligent system solves this task really well but it's because you've baked the solution to the task into the system and it just ignores the training data so the developer aware generalization difficulty is going to capture that and basically punish you for building the solution the final solution directly into the task so this is the intelligent system right here at time 0 this is basically whatever you pre build into the intelligent system this is it hasn't interacted with the training data yet this is simply the state at the very beginning so this is all the priors you build in if you build a resin that you know it has certain you know it has convolutional filters and so on that's a certain prior on the translational invariance if you build a an alpha go system that certainly has the rules of go built in to the system and it has this Monte Carlo tree search which biases it towards a certain kind of learning and so on so all of this is captured in this quantity right here this basically means that how if I am given the optimal training solution as before and also the initial state of the learning system how much more work is it to get to the solution of the task and here you can clearly see if I have already built the solution into this system so if I'm building a tic-tac-toe learning system I call it the learning system but I like build in the optimal strategy from the beginning into my system and it just ignores the training data then this thing here would be low because it takes me it takes me a lot of work to own we finally have the training data to get to the solution but it takes me very little work if I also have the initial state of the system because the solution would be encoded into the initial state already right so any prior you put in there will be captured by this okay so otherwise it's the same the same metric zero means it's very easy to generalize to the entire solution one means it's even like it's given the training data solution and the system you give me that it is very hard for that system now consider here this quantity actually depends on the system you put in all right then we need two more things which are priors and experience so this was the difficulty this was how difficult is the task as such if for a given system and a curriculum now what we want we want to characterize priors and experience now priors are pretty easy what are what is a prior a prior we can capture by simply looking at the difference between how complex is the solution minus how complex is the solution if I'm given the initial state this is almost the same as before but it now only considers what you built into the system right there's no training data anymore it simply says if I have you know if you give me your the source code of your learning system can I if I can already read out the solution then this quantity right here will be zero there is zero complexity to get from your initial state of the learning system to the solution of the task and therefore this entire quantity would be one that means the prior all the information is in the prior however if your learning system is a very general learning system like it's a it's like a standard reinforcement learning algorithm with almost no assumption about the data then this quantity right here would be very low sorry it would be very high of course because the initial system doesn't tell you too much so it's still a lot of work if you if I gave you the source code it's that well this is very general this doesn't tell me anything about the task and it would require a lot of work to get the dissolution of the task and therefore the quantity up here would be very low and therefore this would be close to zero so that means there are no priors in this intelligent system for that given task okay and the quantity is always of how to reach the threshold the solution is always with respect to a threshold in skill so you must reach like this many points okay the the important thing that shall I notes here is that the priors captured not not the amount of information in the program in the intelligent system but the amount of relevant information for that task you can make a super duper complex intelligent system it the only thing that matters for this quantity is the amount of information that's irrelevant to get to the solution of the task T the last thing we need to capture is experience now experience basically means how how much during this learning phase from now we just talked about at the at the at the outset like the state at time zero for the priors now you remember we interact with the task for a number of time in this training phase right and the question is of course if we are given a longer training phase it is easier to generalize generally right in more training data makes it makes our life easier makes it easier to generalize and the intelligence is is inverse proportional to that so a system that had all else being equal that has less training data but is performing as well on a task as a system that had more training data that system that had less training data we consider to be a more intelligent system because it can generalize more efficiently so we need to quantify experience and experience now in the same kind of in the same train of thought is going to be the difference between two quantities so the first quantity is this so here we consider at each time step T okay so at each time step T we have the intelligent system and we have this thing here called data now data is everything that the intelligent system gets at that point in time so the intelligent system is here at time step T and then it outputs the skill program and that skill program gets a situation and gives a response and this gives a feedback and all of this data that's called data okay it's basically you can think of it as one additional training example right you're at time step T and you're given one additional training example the experience is going to quantify how much information is in that one additional example and that's and the and then we were going to sum this up over time down here which basically means over the entire course of training which is this curriculum see how much information did you get out of the training data at each step that's going to be your experience over the course of training which and this is the sum over the experience that you got at each step and the experience at each step is simply the following two things are going to assess is how difficult is it at time T so you've learned for T steps how difficult is it to go from that to the solution right so if you might have had some training data right and you you score a certain you score a certain you look score like 80% on the on the test set so that's basically how difficult it is it's it's like you makes you still make 20% of error that's your difficulty and then you get one more training sample this data here now you can ask again if I know everything I'm knowing my intelligence system but I also get one more training data point can I how easy is it now to arrive at the solution of the task and now you can say oh with this training data point I now can correct some of my mistakes and I only make like 18% of error okay so the difference here would be like 2% so that's going to be your experience is going to be worth of 2% of errors okay now the important thing here is that it is it is different if we we could have just written here minus H of you know Sol theta T given the intelligent system at time step T plus 1 right because the intelligent system at times that people's one has had that data point at time step T and incorporated it but that's not that's not the same thing here we in in this step right here when we say how difficult is it we assume that you know God or Vapnik himself tells us how like the optimal way to use that information okay whereas this thing here the it's not a given that the intelligent system will use that information in the most optimal way so this is basically the difference between how difficult is it to get from the intelligent system to the solution and how difficult is it to get from the intelligent system and the data point at time T if you could make optimal use of that data point to the solution all right so this this is going to be an assessment of how much experience you've had in the in the sense of had you been able to incorporate the experience properly at each time step because yeah because otherwise you know you you couldn't compare the experience if two systems had had the the same experience in the same task it should mean they had had the same you know data points in the same order or in in a simplistic sense all right so this is all we need intelligence boom this is it so there's a lot of stuff here okay intelligence of an intelligent system with respect to a scope and there are two definitions right here one is for optimal skill at each task and one is for threshold of skill now we're going to focus on the threshold as we said we at each task we require something like you must achieve 8,000 points and we're going to consider the shortest programs that will get to at least 8,000 points now there's a bit of confusion in the notation here as this I'm pretty sure this quantity right here you know should be called something different because it's you know it's the T is here and then there's this here this refers to this and this shouldn't be out of here this should be meaning something like Thresh I'm pretty sure this is just a name like here the name opt so yeah in any case the the intelligence is of an intelligent system with respect to a scope of tasks okay and the first thing we do is we're going to average over the tasks in the scope so we consider all the different tasks and each task has a weight associated with it this this is the threshold and skill that we want and this is sort of a mapping this is a conversion rate because this might be you know 9,000 points at Pong and another task might be you need to achieve point 2 and that's really good a point 2 is really good so this W for each task is simply going to map it to a like a uniform coordinate space of of of points of skill level of that particular task okay so but we're going to average over tasks now you can I guess disregard this this is not super this is just scaling we're going to average over tasks now in each task we're going to consider all curriculums that get you to this threshold so all curriculums that get you to the threshold T for theta T for task T which means sort of means all the possible permutations of training data sets for that task right it's more general than this but we yeah we want to assess all the all the different ones and as you can see here there's P of C so this is an expectation this is the probability of that particular curriculum this is this is the expectation over data right here this is the expectation over the training data distribution okay in the classical machine learning sense so we're going to take the average across all tasks over the expectation under the training data distribution so we're good so far and usually right here we would put something like the empirical risk right the minimum minimum loss min loss function min theta loss function over my term over my see over my training data set okay but not in this case because we now want to consider the priors and the experience and discount that from the difficulty and that's what's written here so this is the developer aware generalization difficulty this here is the amount of information that's already contained in the priors and this here is the amount of information that's contained in the experience in that curriculum as you can see here the experience is in that curriculum so basically a system is more intelligent if the task is harder for that given system and that given curriculum okay so that makes intelligence up a system is more intelligent if it gets to a certain threshold with lower priors okay if the priors are low this the whole quantity is high and the system is also more intelligent if it gets to the threshold with less experience okay so if the experience here is lower it is counts as more intelligent all right in this in this quantity and this is written all in the text here it has some properties in that it for example it it down values actors that in the same curriculum like in the same training data they if if an actor learns faster like it learns earlier to reach the threshold it would assign more intelligence to that actor and so on it's kind of sometimes it's hidden over the it's hidden in the definitions for example these curricula are not all the same at the curricula are specific the curricula that you need to reach this certain threshold so it's not always doesn't always sum up to one with this probability here that's why it's not exactly an expectation let's call it an expectation in quotation marks but in the general sense that's it so in sis the intelligence of a system is over a scope of tasks the expectation in quotation marks under the training distribution of the generalization difficulty account but accounted for discount we discount the prior knowledge of the system and the experience that the system has had okay and that's it he says p plus e prior suppose experience represents the total exposure of the system to information about the problem including the information it starts with at the beginning of training okay so if this is high then the system is not very intelligent or is not if a system that has more of this but generalizes to the same level as another system is considered less intelligent than the other system because it has had more exposure to information about the problem like it it makes a lot of sense right so schematically the contribution of each task is the expectation over skill times generalization divided by priors plus experience that's kind of in words what we looked at so it goes over a number of key observations and at last he goes over consequences or basically a recommendation for what a benchmark should look like if we regard it in this light now of course these complexities and so on they're not exactly computable right so it's like how much exactly the shortest the length of the shortest program is is not exactly computable but it can inform our notion of how we should test intelligence okay so what to expect of an ideal intelligence benchmark first of all it should describe its scope of application its own predictiveness with regard to this scope so that means the validity it should be wreck replicable it should be reproducible it should measure broad abilities and developer aware generalization sorry it should it should set out to measure broad abilities and developer aware generalization okay so that means it should not be solely measuring skill or potential it should not feature in its evaluation set any tasks that are known in advance either to the test taking system itself or to the developers of the system and that of course refers directly to the developer aware generalization and it should seek to quantify the generalization difficulty it measures or at least provide qualitative guidelines with regards to its generalization difficulty it should at least be made clear whether the benchmark seeks to measure local generalization broad generalization or extreme generalization so we've we've seen this in part one taking into account generalization difficulty minimizes the possibility that a given benchmark could be hacked by solvers that take undesired shortcuts that bypass broad abilities hey says it should control for the amount of experience leveraged by test taking systems during training it should not be possible to buy performance on the benchmark by sampling unlimited training data so this this already rules out sort of any let's say image recognition or NLP benchmarks because there we can always just feed in more data the more unlabeled data from the internet or even labeled data like if there's a benchmark that you know is on computer vision I can just pay more humans to label more data and then I will be better at that benchmark the benchmark should avoid tasks for which new data can be generated at will it should be in effect a game for which it is not possible to practice in advance of the evaluation session that's going to be hard right it should be it should explicitly and exhaustively decide describe the set of priors it assumes any task is going to involve priors but in many tasks used for a evaluation today priors stay implicit and the existence of implicit hidden priors may often give an unfair advantage to either humans or machines so this is for example if the test is like a speed test a lot of times machines are going to be way faster than humans because the hidden assumption in a speed test is that kind of your nerve conductivity is the same across all test takers and the last one it should work for both humans and machines fairly by only assessing the same priors as possessed by humans and it refers to core knowledge which we saw in the last part and only requiring a human sized amount of practice time or training data so this means if we want to compare humans and machines machines can often incorporate way more data than humans so the tasks in the benchmark should only like the amount of data should be such that a human could process that data now of course that that sort of also means that any task where basically you collect data during your life is all also sort of ruled out a bit so that means the AI benchmark task can't be like cook a pan of spaghetti or something like this yeah and in the end he says these recommendations for general AI evaluation wouldn't be complete without a concrete effort to implement them in part three we present our initial attempt which is going to be the ARC dataset and the ARC Kaggle challenge but that's a story for next time I hope you enjoyed this and at least got some bits of it it's very abstract this measure of intelligence of course it can never be computed exactly but the fact that someone is trying to formalize and it's not the first time this has been trying to formalize but this I feel it's quite understandable and it makes sort of sense and I'm I'm interested to see if people come up with exp like actual approximations to this quantity that you could actually compute sort of all right that was it thank you for watching and bye bye see you next time
 Hello and welcome to the third part on On the Measure of Intelligence by
 François Chollet. Now this is a multi-part series. If you haven't seen the
 first two parts I recommend to watch at least one of them. They're somewhat
 overlapping but we've basically gone over the history of intelligence
 measurement and the foundations of what a measurement for intelligence for an AI
 system should look like. Today we're going to get into the formal
 definition of the intelligence that Chollet proposes right here. So this
 sentence here pretty much sums up what we're interested in.
 The intelligence of a system is a measure of its skill acquisition
 efficiency over a scope of tasks with respect to priors, experience and
 generalization difficulty. So these are the things that we've established so far
 basically. The intelligence of a system that's the thing we want to measure
 is a measure of its skill acquisition efficiency. So how fast does it
 acquire new skills? Important here is that we are measuring it over a scope of
 tasks. So it's not arbitrary skills it is a scope that we define and this is going
 to be mostly the human scope, the scope of tasks that humans
 can solve and are sort of different at. What we need to factor in are priors
 which is what is already built into a system because that doesn't count as
 intelligence that's already built in. If your ability to solve a problem is
 already built into you you don't have to use intelligence to solve the problem.
 Second, experience. If you have had lots and lots and lots of experience at the
 particular task you're asked to solve you don't have to use intelligence you
 can simply rely on your experience. And the third is generalization difficulty
 and that's a property of the task. So if the task is very difficult to generalize
 so if it's very difficult if the task itself is very difficult then achieving
 good score at it should count as having higher intelligence if all other things
 are equal. So this is going to be the basis and today we're going to
 watch Shirley define these things into a number that can give us
 the intelligence of any system with respect to these things. So that's
 the program for today. If you like content like this share it out and tell
 all your friends and leave a like so that YouTube knows that you do like it.
 So the conceptualization of the entire system is like this. There is a
 task and we're going to consider a series of tasks of course but if we just
 look at one task in our scope there is the task and the task outputs these
 situations. In a machine learning term these are like your training
 examples. And on the other side there is the intelligent system. Now the
 intelligent system in a pure machine learning side you would factor this as
 the task gives the intelligent system something like a training sample or in
 reinforcement learning it would be something like an observation and the
 intelligent system gives something back like a response. Here we have a
 kind of a in-between step. The intelligent system doesn't actually give back the
 response to the situation. The intelligent system generates a skill
 program. So the intelligent system will generate a program that can map the
 situation to a response and that skill program should be able to run on its own.
 So in the classic machine learning sense if we look at supervised
 learning for example the intelligent system would be like a
 ResNet plus SGD. That is an intelligent system and if it is output
 it is able to generate a skill program. So during training what happens
 during training? During training the intelligent system is able to intervene
 in the skill program at each step. So the situation comes in and then the skill
 program does something but the intelligent system can at any point it
 can kind of intervene and update the skill program and generate a new skill
 program for the next step. So there's a situation the skill program
 gives a response and the task gives feedback in form of a score. In machine
 learning terms this would be your training sample. Your training sample
 comes in, your neural network gives a response which are the logits of the
 classes, then the task gives a score to that which in the supervised
 learning case is the label or the loss function as a feedback to the intelligent
 system and the intelligent system using SGD would update the skill program for
 the next step. So at each step the intelligent system can update the skill
 program. That's why the intelligent system in this case is the architecture
 of the neural network and the procedure to update the weights. Not the weights
 themselves but the procedure to update the weights and the skill program here
 those would be the actual weights of the neural network or like the
 instantiation of the ResNet with these particular weights. Now at test time
 we sever this connection right here. So this is now severed at test time. At
 some point the training is done. The task says okay now training is done and then
 the intelligent system will produce one last skill program and then this
 connection is cut and the skill program must by itself answer to these
 situations. The intelligent system cannot intervene anymore and in this
 loop here it's situation response situation response this goes over for a
 number of steps and all the scores during that time are counted and tallied
 up and at the end you know the higher the score the better. So the intelligent
 system must at this end step produce a skill program that by itself can achieve
 a high score. So there's always this training phase first and then there is
 the test phase. Now the training phase these situations that we get in a
 training phase they are called a curriculum in this in this world. In
 our world this would be something like a training data set but this is
 curriculum it's slightly more intricate but just the notion here makes sense
 right the intelligent system produces the skill program. So there's a lot
 of formalisms right here like okay the task has a situation generator
 and that maps the task state to a situation so the task can have a state
 and the skill program can have a state and the intelligent system can have a
 state and I don't like this is all a bit too formal you don't really need to
 understand it except if François Cholet is watching this I think I have
 found I'm not sure if it's a mistake but you say the intelligent system here
 consists of three objects so it generates the skill program according
 to its internal state okay and it generates the skill program and when it
 learns when it learns it updates the internal state internal state according
 to let me if I can find it right here a self update function so this is how the
 intelligent system can update itself so its own state so it takes the internal
 state of the intelligent system and outputs another internal state and this
 is the you know where I said the internal the intelligent system at each
 training step it can observe what happens and basically react accordingly
 so it takes the situation the response and the feedback and its own internal
 state as an input now what do we have here it takes the situation which would
 be in our case the training sample the response or the the logits that the
 neural network has produced the feedback which is the loss and its internal state
 okay now what I argue is basically that it should also get the internal state of
 the skill program as an input right here because the skill program can have an
 internal state all of this like the response can be a stochastic procedure
 of the skill program and I guess it's not necessary because you can sort of
 infer it but I think the framework would be more complete if the internal state
 of the skill program at that time were part of the intelligent system update
 procedure just you don't okay this is not relevant this is just me bickering
 cool let's actually jump all of this this is boring this is very boring okay
 blah blah blah blah blah lots of definitions all right quantifying
 generalization difficulty experience and priors using algorithmic information
 theory so these things that at the beginning we said that we want to define
 intelligence with respect to we are now going to quantify using algorithmic
 information theory algorithmic information theory in this case right
 here that we're using it's not very complicated the main quantity is this H
 the algorithmic complexity it the H of s is the length of the shortest
 description of the string in a fixed universal language okay so it's the
 length of the shortest program that outputs the string when running on a
 fixed universal Turing machine so basically if if you have this string s
 right here as is a bit string the shortest program that can compute s or
 you know so so in the worst case that's the the string itself but if the string
 is like 0 1 0 1 0 1 0 1 all the way you can just say 0 1 times 50 and that's
 that would be like the shortest program to produce that it isn't it is an
 information theoretic concept here but in essence you can just think of it as a
 measure of how long is the program that I would need to write to output a given
 to to produce a given output okay so that's the algorithmic complexity and
 then the second quantity right here is the relative algorithmic complexity
 which is almost the same thing it's how long is the program that I have to write
 so the shortest we're always talking about how long is the shortest program
 that I have to write that produces s1 but is allowed to take s2 as an input
 okay so it can never it it can always ignore s2 that's always a possibility so
 if s1 is like a super easy string you can just output that but if s1 let's say
 s2 here is 0 1 0 0 1 okay and s1 is 0 1 0 0 1 0 1 0 0 1 okay so it's just twice
 that so you could you could sort of output that string here we could write a
 program that just outputs this or you could write a program just that just
 says two times s2 okay so that the the length of this is not part of the
 program the program is just two times s2 because it's allowed to take s2 as an
 input okay so this is the algorithmic the relative algorithmic complexity is
 how how much how long is the how complex is the program to get from s2 to s1 so
 you can almost already see how that will relate now to to generalization okay so
 a few quantities that we need to consider are a task called t here then
 solve t theta is the shortest of all possible solutions of t so task t has a
 solution of threshold theta which the threshold theta is just this is like the
 minimum score we need to achieve in a task we don't we consider tasks according
 to thresholds like you know you need to get a I don't know what a score of 9000
 in Pong or so so the shortest of all programs that will will optimize that
 will solve the task to a threshold t sorry theta which is the shortest scale
 program that achieves at least theta during evaluation and the other quantity
 is this train soul opt TC this has a lot of quantity right here you can see the
 task we want to be optimal but with respect to a curriculum okay the
 curriculum is the training data so this quantity is the shortest optimal
 training time solution given a curriculum so it's the shortest scale
 program that achieves optimal training time performance over the situation in
 the curriculum so this this right here is if we could if we had an oracle that
 told us here is how to solve the task in general like the task of the task of of
 determining cats from dogs and images this this is this is this would be the
 program that does it okay you know overall over the entire the entirety of
 the task all cats and dog images there there are that's the solution now this
 quantity right here means sort of the the one neural network that is best at
 disturb determining cats from dogs in this particular training data set this
 curriculum see okay so this is the the one neural network that is hyper
 optimized in this particular training data set and now we assess the
 generalization difficulty so the generalization difficulty is going to be
 a measure of how hard is it to in a particular task to generalize to the
 whole task from the curriculum see and that's going to be the relative
 algorithmic complexity to go from this quantity to this quantity both quantities
 we've just explained so it basically means if if I had the perfect solution
 on the training data set how much how much more complex is it to get from that
 to the perfect solution on the entirety of data or you can also guess on the
 test data set right so if if this is really easy so if the training data set
 already perfectly captures all of the data there is that this quantity is zero
 like the out the program I don't need to write a program I already have the
 solution right and you can see here we divide by the age of salty but however
 if the training data has no information whatsoever about about the about the
 solution to the general task or if I just so horribly overfit on the training
 data such that it doesn't help me at all for the general task then this quantity
 is zero so this quantity is in zero one with with sorry the quantity is one of
 course yes because the shortest this thing up here will be equal to just h of
 salt e because this doesn't help me in that case and then this ratio will be
 one so generalization difficulty of one basically means that the training data
 solution doesn't help me at all this this particular training curriculum is
 useless because I'll just overfit so horribly that I will not learn anything
 about the task or I can't learn anything at all and generalization difficulty of
 zero oh that's yeah no yes generalization difficulty of zero basically
 means that all of the solution is already contained in the training
 solution and I require no work to get to the to the test set solution okay this
 is I mean I do this train test that this is all a bit more general as it is
 written here but I think it's a good a good way to think about it okay so the
 point here he makes is that is that so yeah he makes this example right here
 where he has these two data points where x minus point seven five has a label
 false and x point one five has a label true and the shortest possible solution
 will not help you to generalize to the to the other things so the nearest
 neighbor program would be better prepared for future uncertainty but would
 take significantly more space to write down so there's there's a trade-off
 there's direct trade-off to how much you optimize on the training data and how
 much generalization capability you have okay so the next quantity we want to
 assess is developer aware generalization difficulty because so far
 we've only considered generalization difficulty with respect to the task
 itself and to the to the curriculum but what you could do is you could simply
 you know you producing this intelligent system you could simply build in the
 solution to the entire task into your intelligent system that means it could
 completely ignore the training data and still perform pretty well on this thing
 even though even though the training data itself the algorithmic complexity
 it tells you nothing about about this so the generalization difficulty would be
 very high in the measure up here but so you would think wow this intelligent
 system solves this task really well but it's because you've baked the solution
 to the task into the system and it just ignores the training data so the
 developer aware generalization difficulty is going to capture that and
 basically punish you for building the solution the final solution directly
 into the task so this is the intelligent system right here at time 0 this is
 basically whatever you pre build into the intelligent system this is it hasn't
 interacted with the training data yet this is simply the state at the very
 beginning so this is all the priors you build in if you build a resin that you
 know it has certain you know it has convolutional filters and so on that's a
 certain prior on the translational invariance if you build a an alpha go
 system that certainly has the rules of go built in to the system and it has
 this Monte Carlo tree search which biases it towards a certain kind of
 learning and so on so all of this is captured in this quantity right here
 this basically means that how if I am given the optimal training solution as
 before and also the initial state of the learning system how much more work
 is it to get to the solution of the task and here you can clearly see if I have
 already built the solution into this system so if I'm building a tic-tac-toe
 learning system I call it the learning system but I like build in the optimal
 strategy from the beginning into my system and it just ignores the training
 data then this thing here would be low because it takes me it takes me a lot of
 work to own we finally have the training data to get to the solution but it takes
 me very little work if I also have the initial state of the system because the
 solution would be encoded into the initial state already right so any prior
 you put in there will be captured by this okay so otherwise it's the same the
 same metric zero means it's very easy to generalize to the entire solution one
 means it's even like it's given the training data solution and the system
 you give me that it is very hard for that system now consider here this
 quantity actually depends on the system you put in all right then we need two
 more things which are priors and experience so this was the difficulty
 this was how difficult is the task as such if for a given system and a
 curriculum now what we want we want to characterize priors and experience now
 priors are pretty easy what are what is a prior a prior we can capture by simply
 looking at the difference between how complex is the solution minus how
 complex is the solution if I'm given the initial state this is almost the same as
 before but it now only considers what you built into the system right there's
 no training data anymore it simply says if I have you know if you give me your
 the source code of your learning system can I if I can already read out the
 solution then this quantity right here will be zero there is zero complexity
 to get from your initial state of the learning system to the solution of the
 task and therefore this entire quantity would be one that means the prior all
 the information is in the prior however if your learning system is a very
 general learning system like it's a it's like a standard reinforcement learning
 algorithm with almost no assumption about the data then this quantity right
 here would be very low sorry it would be very high of course because the initial
 system doesn't tell you too much so it's still a lot of work if you if I gave you
 the source code it's that well this is very general this doesn't tell me
 anything about the task and it would require a lot of work to get the
 dissolution of the task and therefore the quantity up here would be very low
 and therefore this would be close to zero so that means there are no priors
 in this intelligent system for that given task okay and the quantity is
 always of how to reach the threshold the solution is always with respect to a
 threshold in skill so you must reach like this many points okay the the
 important thing that shall I notes here is that the priors captured not not the
 amount of information in the program in the intelligent system but the amount of
 relevant information for that task you can make a super duper complex
 intelligent system it the only thing that matters for this quantity is the
 amount of information that's irrelevant to get to the solution of the task T the
 last thing we need to capture is experience now experience basically
 means how how much during this learning phase from now we just talked about at
 the at the at the outset like the state at time zero for the priors now you
 remember we interact with the task for a number of time in this training phase
 right and the question is of course if we are given a longer training phase it
 is easier to generalize generally right in more training data makes it makes our
 life easier makes it easier to generalize and the intelligence is is
 inverse proportional to that so a system that had all else being equal that has
 less training data but is performing as well on a task as a system that had more
 training data that system that had less training data we consider to be a more
 intelligent system because it can generalize more efficiently so we need
 to quantify experience and experience now in the same kind of in the same train
 of thought is going to be the difference between two quantities so the first
 quantity is this so here we consider at each time step T okay so at each time
 step T we have the intelligent system and we have this thing here called data
 now data is everything that the intelligent system gets at that point
 in time so the intelligent system is here at time step T and then it outputs
 the skill program and that skill program gets a situation and gives a response and
 this gives a feedback and all of this data that's called data okay it's
 basically you can think of it as one additional training example right you're
 at time step T and you're given one additional training example the
 experience is going to quantify how much information is in that one additional
 example and that's and the and then we were going to sum this up over time down
 here which basically means over the entire course of training which is this
 curriculum see how much information did you get out of the training data at each
 step that's going to be your experience over the course of training which and
 this is the sum over the experience that you got at each step and the experience
 at each step is simply the following two things are going to assess is how
 difficult is it at time T so you've learned for T steps how difficult is it
 to go from that to the solution right so if you might have had some training
 data right and you you score a certain you score a certain you look score like
 80% on the on the test set so that's basically how difficult it is it's it's
 like you makes you still make 20% of error that's your difficulty and then
 you get one more training sample this data here now you can ask again if I
 know everything I'm knowing my intelligence system but I also get one
 more training data point can I how easy is it now to arrive at the solution of
 the task and now you can say oh with this training data point I now can
 correct some of my mistakes and I only make like 18% of error okay so the
 difference here would be like 2% so that's going to be your experience is
 going to be worth of 2% of errors okay now the important thing here is that it
 is it is different if we we could have just written here minus H of you know
 Sol theta T given the intelligent system at time step T plus 1 right because the
 intelligent system at times that people's one has had that data point at
 time step T and incorporated it but that's not that's not the same thing
 here we in in this step right here when we say how difficult is it we assume
 that you know God or Vapnik himself tells us how like the optimal way to use
 that information okay whereas this thing here the it's not a given that the
 intelligent system will use that information in the most optimal way so
 this is basically the difference between how difficult is it to get from the
 intelligent system to the solution and how difficult is it to get from the
 intelligent system and the data point at time T if you could make optimal use of
 that data point to the solution all right so this this is going to be an
 assessment of how much experience you've had in the in the sense of had you been
 able to incorporate the experience properly at each time step because yeah
 because otherwise you know you you couldn't compare the experience if two
 systems had had the the same experience in the same task it should mean they
 had had the same you know data points in the same order or in in a simplistic
 sense all right so this is all we need intelligence boom this is it so there's
 a lot of stuff here okay intelligence of an intelligent system with respect to a
 scope and there are two definitions right here one is for optimal skill at
 each task and one is for threshold of skill now we're going to focus on the
 threshold as we said we at each task we require something like you must achieve
 8,000 points and we're going to consider the shortest programs that will get to
 at least 8,000 points now there's a bit of confusion in the notation here as
 this I'm pretty sure this quantity right here you know should be called something
 different because it's you know it's the T is here and then there's this here this
 refers to this and this shouldn't be out of here this should be meaning
 something like Thresh I'm pretty sure this is just a name like here the name
 opt so yeah in any case the the intelligence is of an intelligent system
 with respect to a scope of tasks okay and the first thing we do is we're going
 to average over the tasks in the scope so we consider all the different tasks
 and each task has a weight associated with it this this is the threshold and
 skill that we want and this is sort of a mapping this is a conversion rate
 because this might be you know 9,000 points at Pong and another task might be
 you need to achieve point 2 and that's really good a point 2 is really good so
 this W for each task is simply going to map it to a like a uniform coordinate
 space of of of points of skill level of that particular task okay so but we're
 going to average over tasks now you can I guess disregard this this is not super
 this is just scaling we're going to average over tasks now in each task we're
 going to consider all curriculums that get you to this threshold so all
 curriculums that get you to the threshold T for theta T for task T which
 means sort of means all the possible permutations of training data sets for
 that task right it's more general than this but we yeah we want to assess all
 the all the different ones and as you can see here there's P of C so this is
 an expectation this is the probability of that particular curriculum this is
 this is the expectation over data right here this is the expectation over the
 training data distribution okay in the classical machine learning sense so
 we're going to take the average across all tasks over the expectation under the
 training data distribution so we're good so far and usually right here we would
 put something like the empirical risk right the minimum minimum loss min loss
 function min theta loss function over my term over my see over my training data
 set okay but not in this case because we now want to consider the priors and the
 experience and discount that from the difficulty and that's what's written
 here so this is the developer aware generalization difficulty this here is
 the amount of information that's already contained in the priors and this here is
 the amount of information that's contained in the experience in that
 curriculum as you can see here the experience is in that curriculum so
 basically a system is more intelligent if the task is harder for that given
 system and that given curriculum okay so that makes intelligence up a system is
 more intelligent if it gets to a certain threshold with lower priors okay if the
 priors are low this the whole quantity is high and the system is also more
 intelligent if it gets to the threshold with less experience okay so if the
 experience here is lower it is counts as more intelligent all right in this in
 this quantity and this is written all in the text here it has some properties in
 that it for example it it down values actors that in the same curriculum like
 in the same training data they if if an actor learns faster like it learns
 earlier to reach the threshold it would assign more intelligence to that actor
 and so on it's kind of sometimes it's hidden over the it's hidden in the
 definitions for example these curricula are not all the same at the curricula
 are specific the curricula that you need to reach this certain threshold so it's
 not always doesn't always sum up to one with this probability here that's why
 it's not exactly an expectation let's call it an expectation in quotation
 marks but in the general sense that's it so in sis the intelligence of a system
 is over a scope of tasks the expectation in quotation marks under the training
 distribution of the generalization difficulty account but accounted for
 discount we discount the prior knowledge of the system and the experience that
 the system has had okay and that's it he says p plus e prior suppose experience
 represents the total exposure of the system to information about the problem
 including the information it starts with at the beginning of training okay so if
 this is high then the system is not very intelligent or is not if a system that
 has more of this but generalizes to the same level as another system is
 considered less intelligent than the other system because it has had more
 exposure to information about the problem like it it makes a lot of sense
 right so schematically the contribution of each task is the expectation over
 skill times generalization divided by priors plus experience that's kind of
 in words what we looked at so it goes over a number of key observations and at
 last he goes over consequences or basically a recommendation for what a
 benchmark should look like if we regard it in this light now of course these
 complexities and so on they're not exactly computable right so it's like
 how much exactly the shortest the length of the shortest program is is not exactly
 computable but it can inform our notion of how we should test intelligence okay
 so what to expect of an ideal intelligence benchmark first of all it
 should describe its scope of application its own predictiveness with regard to
 this scope so that means the validity it should be wreck replicable it should be
 reproducible it should measure broad abilities and developer aware
 generalization sorry it should it should set out to measure broad abilities and
 developer aware generalization okay so that means it should not be solely
 measuring skill or potential it should not feature in its evaluation set any
 tasks that are known in advance either to the test taking system itself or to
 the developers of the system and that of course refers directly to the developer
 aware generalization and it should seek to quantify the generalization difficulty
 it measures or at least provide qualitative guidelines with regards to
 its generalization difficulty it should at least be made clear whether the
 benchmark seeks to measure local generalization broad generalization or
 extreme generalization so we've we've seen this in part one taking into
 account generalization difficulty minimizes the possibility that a given
 benchmark could be hacked by solvers that take undesired shortcuts that
 bypass broad abilities hey says it should control for the amount of
 experience leveraged by test taking systems during training it should not
 be possible to buy performance on the benchmark by sampling unlimited training
 data so this this already rules out sort of any let's say image recognition or
 NLP benchmarks because there we can always just feed in more data the more
 unlabeled data from the internet or even labeled data like if there's a
 benchmark that you know is on computer vision I can just pay more humans to
 label more data and then I will be better at that benchmark the benchmark
 should avoid tasks for which new data can be generated at will it should be in
 effect a game for which it is not possible to practice in advance of the
 evaluation session that's going to be hard right it should be it should
 explicitly and exhaustively decide describe the set of priors it assumes any
 task is going to involve priors but in many tasks used for a evaluation today
 priors stay implicit and the existence of implicit hidden priors may often give
 an unfair advantage to either humans or machines so this is for example if the
 test is like a speed test a lot of times machines are going to be way faster than
 humans because the hidden assumption in a speed test is that kind of your nerve
 conductivity is the same across all test takers and the last one it should work
 for both humans and machines fairly by only assessing the same priors as
 possessed by humans and it refers to core knowledge which we saw in the last
 part and only requiring a human sized amount of practice time or training
 data so this means if we want to compare humans and machines machines can often
 incorporate way more data than humans so the tasks in the benchmark should only
 like the amount of data should be such that a human could process that data now
 of course that that sort of also means that any task where basically you
 collect data during your life is all also sort of ruled out a bit so that
 means the AI benchmark task can't be like cook a pan of spaghetti or
 something like this yeah and in the end he says these recommendations for
 general AI evaluation wouldn't be complete without a concrete effort to
 implement them in part three we present our initial attempt which is going to be
 the ARC dataset and the ARC Kaggle challenge but that's a story for next
 time I hope you enjoyed this and at least got some bits of it it's very
 abstract this measure of intelligence of course it can never be computed exactly
 but the fact that someone is trying to formalize and it's not the first time
 this has been trying to formalize but this I feel it's quite understandable and
 it makes sort of sense and I'm I'm interested to see if people come up with
 exp like actual approximations to this quantity that you could actually compute
 sort of all right that was it thank you for watching and bye bye see you next time