 Hello community! I was asked to explain here a brand new paper about automated design of a genetic system in simple terms. So yes, I'm gonna accept this challenge. So you know in my last video here, PhD days isn't one day, with an open source AI, here the AI scientist, I showed you how you can build multi-AI agents that they write you a scientific paper. And now we go to the next step, if you want. Now we want to have an automated design of an agentic system, of a multi-agent system. And we have here University of British Columbia, Vector Institute in Canada, CIFAR AI Chair, and they have a new paper and they say, hey, we have a newly forming research area that is called the automated design of agentic systems, which aim to automatically create powerful agentic system designs that are specific to your job. So unfortunately, they still go here with a central intelligent they call this the meta agent i show you the code in a minute which is nothing to do with the facebook agent let's call it the super agent so we have here a central intelligent that builds and selects all the other little agent the other minions here ideas that you have somewhere a directory where you already have some interesting agents. And now this super agent comes, look at all the agents and chooses the right agent for the right task. And if an agent is not really 100% fit for the task, this super agent goes to GPT-4 and says, hey, can you optimize the code for my specific domain specific task that's all there is but let's have a deep dive so before we jump into the code and i explain the code let's just have a look what they give you and they say hey we have a simple three component design so we have a lot of little agents a lot of little minions where we have the code in our directory or in our github repo this is now our search space and then for our super agent we have a search algorithm the searches here those vector space of all available little minions little agents if we find one beautiful we take it or we optimize it a little bit. We evaluate this agent. And if it's not good, we optimize it. If it's perfect, we just use it. Those are the three elements. And they did a lot of experience, so let me show you. The first here is the meta agent search on the RSC challenge. For my little green glass hoppers, the RC price is here a multi-million dollar price and it is simply about here. RC AGI and you see here the prices are. What are the prices? Those are the sponsors. Where are the prices here? The ground price is 500. 500 000 the progress price is 100 000 pub verification price you get the idea and there's a beautiful challenge for you now let's look at the agents that you should have in your code repo so they compared here now to the agent that is a state-of-the-art design agent. They go here with an F1 score with accuracy, whatever you like here as your benchmark data. And those are the agents that they have. The chain of sort, the chain of sort here with let's go here. I show you this with a self consistency check. So it is not simply a greedy decode chain of sort prompting, but we want here that we have here a diverse set of reasoning tasks that are explored and then let's say we go with a majority voting. So self-consistency if you want to see this from a green cross, as this is the paper I would recommend here by Google. Or you go with a self-refinement, I have a video on this, or you go take a step back and look at the abstraction, or you go and assign here a specific role, a role assignment agent. So you can go there, you can take the original code from the authors of the paper or whatever, and they give you here for the RSC challenge here, the benchmark data. and as you can see we are here chain of thought 64 64.4 59 so beautiful for math the accuracy you see they're quite have a heavy spread here and then if you do here the automatic agent selection and the automatic agent update we have here now by the authors here the best agents for this specific rc challenge 79.4 so you see from 65 to 79 this is a significant jump so the system tries to automatically design here the perfect number of agents and the perfect function in the agents. And if an agent is missing some code block, it will ask GPT-4 to write this code block to get a better performance. Now, yeah, you can have this again here, dynamic role playing or structure, multimodal feedback loop or whatever. So this is an interesting topic we should have a look at. Now show you here the prime author. This is it gives up go there. He has there his automated design of a gigantic system repo. Really nice. You see here the complete code implementation and they explain how you can run this on your machine, how you can modify this for your domain. Nice course. Careful. there's a warning it says hey this code in this repo involves executing untrusted model generated code we strongly advise user to be aware of this safety concern could have some malicious action in your current setting so careful do this maybe in a sandbox environment but beautiful we have this now and yeah let's go here to the official publication before we do anything else and here we have an automated design of a genetic system 19 of august 2024 and here they give you here a beautiful explanation what is going on and we could theoretically look here at all the theory, but you know what? Since today I have to be simple, I just show you the real code and the real prompts. Let's start with this. So here we are now, prompts. We are at page 24. And here you have the system prompt for our super agent and it says hey you are a helpful assistant make sure to return here well-formed json observed and you said amazing so and then we have here the main prompt for the meta agent we tell it hey you are an expert machine learning researcher testing various agentic system your objective is to design a billing block such as prompts and control flows but we will focus at the first of the prompt and the second part here various agentic system. Your objective is to design a billing block such as prompts and control flows with this system to solve complex tasks. Your aim is to design an optimal agent performing well on a particular domain. Then we have to provide some framework code. I'll show you this in a minute. Some output instruction and some examples for in-context learning and then we have to give it some code base. So we have here some agent archive. This has of course the original code and then here are ideas that the super agent if it it finds here an agent, a little minion agent here in this directory that it can optimize, it stores here the optimized version of this minion. Beautiful. And then we say your task is, you are familiar with prompting techniques and the agent works from the literature. The goal is to maximize the performance matrix by proposing interesting new agents. So design new agents, take existing ones, optimize, permutate, combine them, alterate these agents, design some new conglomerate of agents. Just go. Of course, remember, you have to pay for everything. Ok, so be creative when thinking about this, use the knowledge from the archive, have some inspiration from academic literature, think outside of the box and you know all of these classical ways. Great. Now for the output instruction, we say hey, you have to respond in JSON to me and we have here a clear schema. So the first key should be sort and it should capture your sort process for designing the next function. So in the sort section your first reason, what is interesting, yes, the second has the name, the third has the code, yeah, forward. I will show you the forward function in about two minutes. We go through this step by step. And then here is an example i thought thoughts are abc the inside are yes yes the overall idea is you describe this implementation yeah beautiful the name you have your specific name and then the code you have here in this example you define a forward function and this is the code. And this is a really interesting part and we will focus here on this in part number two of this video. But let's go here with the classical process. So our super agent has now the instruction and now we have here a self-reflection round. So the system, the super agent, must now run one, two, three self-reflection rounds. And what it is doing? It is looking at the interestingness. Is this new idea interesting? Make sure that it has no implementation mistakes. That there is no error. Is an improvement given the classical agent that I just showed you, the chain of thought? What can you add? Maybe you can add a memory, maybe you can improve here the planning module. Whatever you find. Check that it is not redundant, try to avoid being too similar to the previous agent. Yes, yes, yes, so GPT-4 here has a main task to perform and then we tell this super agent running here on gpt4 omni for example in our example hey your response should be as follows you have a reflection so you provide your sorts of the interestingness of the new thing then you have revise your previous proposal and propose a new architecture if necessary, you provide a name and you provide a new codebase if this is necessary. And then you have a second run. Beautiful, beautiful. Now, you remember that I've showed you we have to provide the framework code to our super agent. With a simple framework to implement basic functions such as we have to query a foundation model gpd4 omni and this has to be done in a formatted prompt way so here we have it you have it here also in the github repo but let's just have a look at this so the auto information the format information the role that you have here, beautiful. And then we have here the foundation modules. Base class of the foundation model, we will go in detail on a concrete example, this is here just the abstraction. You have an output field, you have a name, you have a role, you have a model to be used, you have a temperature to be set set and you have a unique identifier. Then you generate here your first prompt for the foundation model. As an example, you are a helpful assistant, do exactly what we were told. I guess you have an idea. Then you define here the query for our super agent and then we define here an agent system with the forward function. Here is where our code will enter. And I will show you here the class of the agent system and the class of the base agent in a concrete example in one minute. So you see, this is all there is to our super agent. Beautiful. Now, next step is here, here, we do now this Minion agents, this little super trained, perfectly optimized, strategically aligned agents. Let's have a look what they give us here, a multi-step peer review agent. So you have a scientific publication and you just act as a human reviewer of the scientific information. So here we have the definition of our forward function with the task information. And now we have, for example, here the initial instruction. Please think step by step and solve the task. Wow. Then we have the critique instruction. Please review the answer above and provide feedback on where it might be wrong. If you're sure that it's correct, just say true correct. Refine instruction, give previous attempts and feedback, carefully consider where you should go wrong in your latest attempt, using insights from previous attempts from your memory try to solve the task better. The final decision, give them all of the above, thinking and answer, reason over all of them carefully and provide a final answer. So you see, simple task. Then we have our foundation model modules. Here a concrete example. You have here, for example, the role to be a physics expert, to be a chemistry expert, to be a biology expert, or a general science generalist, then you are not going to believe it if you go for the critic model, you are a physics critique, a chemistry critique, and you get the idea. Then we have here all the thinking and then we can implement this simply here with a for loop. Now you might say, hey wait a minute, in your videos you showed us here how to code multi-agent reinforcement learning, you showed 20 different code implementation plus recursive delegation and agent scope here for multi-agent. Yes, you could use this or you simply code it by hand here as a general case and this is it now you know me if i want to be simple i do have a problem therefore i ask here my chat gpt what is here this code and you can do the same if you want let me give you 35 yeah so i just copied here this code and i said hey explain this to me and gpt4 omni comes back let's make it a little bit bigger that you see this oops sorry where are we come Come on, here. So, this code, here, forward, this is it. Defines a method called forward, seems to be part of a larger class, yes, yes, yes, so let's break it down. Method definition and initial instruction. Initial instruction, critique instruction, refine instruction, final decision instruction. Ask the models to think step by step. Yes, critique, you know this. Then second step, we initialize the modules. A, B, C. Models, final decision models, then we initialize the storage for our sorts, for our answers and for our feedback. This is happening here. Explained here. These lists are stored, storing the thoughts and feedback for each of the foundation models through the process. And then we have the initial reasoning by each expert model. This is our for loop here. And whatever is the loop, you have here the purpose, the storage. And then we have here a cross critique by the critique modules each expert answer you know physics chemistry is critiqued by another critique model and this cross critique process generates the feedback of the initial answer you can store this and then you can refine this answer of course and then you have here with this code your final decision making. The final decision module combines all of the refined thoughts and answers from each expert module. It uses this input to make the final decision and the final output. The final answer is here presented and it is the conclusion reached by considering all the expertise and all the feedback that you have and how many runs you had for the self-reflection, for the optimization, for the evaluation and for the continuous improvement. So you see here, the forward method is designed to solve a complex task by simulating a collaborative process among multiple expert modules. Each module first tries to solve the task independently, then critiques the other module's solution, refines its own solution based on the feedback and finally, a decision is made by aggregating the refined answers. This process allows the system to combine diverse perspectives and iteratively improve the quality of the solution, making the final output more reliable and robust. So here you see why it is better to have those multiple expert modules instead just of a simple GPT-4 Omni that does all of this somehow internally. But here we have a clear defined process. Now, you notice that I thought for a moment, hey, I want to be consistent here in my terms, and I'm talking here about expert modules, but we should be talking about agent, because this is, you remember, this is here a multi-agent system. I was talking here about multi-agent and multimodal AI, if you're doing physics and so. So where is now the difference? Why do you say expert models instead of agents? I ask you my GPT-4. And it came back. And this was quite interesting. And this, you can have this discussion here with AI. It says, in the context of the code, the term model is often used to refer to the components of a system that perform specific functions. Here, the term foundation module likely stands for a foundation model module, which refers to instances of the model, of agents, or GPT-4, that have specific roles, where agents generally refer to autonomous entities that perform tasks or make decisions based on inputs and predefined rules or learning. So in this case, each module is functioning as an agent with a specific expertise. Ask with solving part of the problem. solving part of the problem. So if you want, each module is functioning as an agent with a specific expertise. Okay, so this is the big revelation. Why is it? I find it also nice here, the code defines these components as instances of a module which suggests here this modular approach. The models are likely configured with specific roles, like you are now a physics expert, and they operate as independent agents within the broader systems. And then it even explains, and this new GPT-4 Omni is really better than the old one, why both terms are interchangeable in this context. And I didn't ask it, it was an explanation given by GPT-4. And it says modules as agents. Each module is an agent with a specialized skill designed to solve tasks or critique solutions within its domain of expertise. So in essence these modules are agents, but they are referred to as modules in the code to reflect the role as components in a larger system and i say my goodness so i saw beautiful i was happy everybody was happy great so let's just go back here because i wanted to show you here the next step yeah okay so here we are and these are here the examples here of our little let's go back maybe where are we yeah and this is now the code of our minions yeah there's a very nice information about the cost of all these experiments and they say it costs about 500 us dollar here for a single run of search and evaluation and if we go here with the reasoning cost about 300 dollars so that you have an idea if you have one single run 500 bucks so you can imagine it gets more expensive real fast but of course this all depends on your financial background great so what else yeah well i wanted to show you here the official repo. Add us and if we go there and we have a look, I don't know what we're gonna choose, what we're gonna choose. Mathematics. Yeah, let's choose mathematics. And I told you here, if you don't know a code and I was asked, hey, I don't understand the code base I see in the github repos what can i do now again my simplest solution is here let me go there you see here is the code where we have here the class agent base we have here initialization generate the prompt do the query and then we have here the class agent system and we define here the search functionality, and if I'm not sure what this is, you know, very simple, you're not going to believe what I do, I just take the code, and as you can see here, class, llm, agent, base, I just copy the code, and I say, hey, explain this code to me, so gptT-4 comes back and I think beautifully explains this. Comes back and says, LLM agent-based class represents a basic template for an AI agent that interacts with the language model. The primary function of this class include generating prompts, querying the language model and handling the output. This is it. Yeah, I got another question. Just had an idea. I now was asked, hey, you showed us something here about, where is it, where is it, where is it here, about prompt optimization. This DSPy, our old model and now this new by Stanford here, this text-grad, this automatic differentiation via text is this not something similar or can we optimize now further with text grad here this code and the simple answer is yes of course you can either do it yourself or you can upload here the github repo of text grad from stanford to let's say gemini gemini 1.5 pro with 2 million context length and then you tell Gemini, hey look this is here the code base that I have and look over there I have here the repo from Stanford with an extended code version. Can you try to bring together and create here an optimized advanced code base integrating here the main modules here of both code implementation sometimes it works right out of the box sometimes you have to spend some hours to optimize it but it's really nice let you help you here with a code llm to put together here more complex code structures beautiful i was here with the llm agent base class yes we have the output name the agent named rule the model gpt4 for example would recommend the temperature the id you have an initialization then you generate the prompt you have of course a query and then you have the prompt, you have of course a query, and then you have here a string representation of the agent used for logging or debugging, and then you have here with the call a convenience method that allowed the agent to be called like a function, which internally calls the query method. So we are back here at the query, it simply sends a generated prompt to your GPT-4 and processes the response in gpt4 writes the codes optimize the codes tells you hey this is now a much better code than your other code example stores and you can then store now this new updated block or definition or code segment or whatever you have in your directory or in your github repo and you improve improve and improve continuously so to all my viewers who ask hey i don't understand this piece of code can you help yeah just take your 3.5 sonnet from cloud or whatever you like whatever code llm you like or deep sea code version 2 also real nice and you don't have to spend hours trying to understand it let it explain here the system to you then we have the agent system class then we have the search function central to the process of generating and refining the agent over multiple generation it uses a search process to evolve agents evaluates the agents and store here the optimized agents in the directory. Key concept, reflection, main functionality. And then you know me, I said hey, show me the code where this happens. This system uses a language model to generate new agent designs. And I was not sure what the authors meant with an agent design, because it could be theoretically everything. So, you know, when I'm simple, I'm asking simple questions, and therefore code to generate new agent designs tells me, okay, construct a message list for the model, query the language model to generate a new agent design. You have the reflection process, reflection one, yes, yes, yes. And then it just tells you here. Conducts the prompt. This list contains messages that are sent to the language model. The system prompt sets up the context and the user prompt specifies what the model should do in detail, like generate a new agent design. And then I said, okay, beautiful. Give me an example now of this agent design that we just generated with the code. The design includes the algorithm, the decision-making process itself, and any special features or module that the agent uses to achieve its goal. So, rather clear, even give me here a complete example, high level description using a chain of thought reasoning strategy, and we include here a memory module to recall our previous information, hey, is the system converging to a better solution or is it, has it a gradient methodology that we are leaving now, or even our local minimum, whatever there is, component of the agent, then chain of thought reasoning, the memory model, the decision making output, yes, yes, yes, and then they give you here simple Psyduck code so you understand what is part here in this publication, what they meant with the agent design. So, yes, yes, rest, so you just go there. How the agent design works. You have the input, you have the chain of thought reasoning, you have the agent compares the conclusion with a multiple choice option to select the best match, and then the agent outputs the final answer, choosing the option it determined to be the correct one. You can be here, standardize testing, consumer support chatbot educational tools whatever your complex question is to break it down in the manageable part give it an agent put an agent on each single now much simpler part let it come together let it critique all others let it bring together let's have a two self-reflection run before you finally then go here for the final answer. So you see, it is as simple as can be. Great! And if you made it, if you are one of the lucky ones to made it here till the end, I have a bonus for you. Because I also received a question that you said here. Hey, listen, there is a brand new literature now. August 13. Oh, this is not anymore brand new. Okay, sorry. It's called the Open Research Unleashing AI for Accelerating Scientific Research. last video was here on PhD thesis in Vendeea an open source AI who does here this computer simulation completely autonomous for us in an automatic version here synthetic science done by our AI models our multi-agent AI model and I was asked hey what about this open researcher is it better is it the next step what is it compared here to my last video so I had a look at it and let me formulate it here in a in a very diplomatic version I think they focus here that this is based on the retrieval augmented on our rack generation to integrate here different rack modules here with our llm to have to integrate here an up-to-date domain specific knowledge in our llm yes this is beautiful and they now argue that compared to a simple rack that their version is now a little bit more performant so and they have different tools so if we have a look at this so we go here let's say with the query tool you have an active query tool and a query rewriting tool so if your question if your input to the llm is too complex before it is sent off to rack the llm try or this tool here tries to rewrite it here in simpler terms and therefore you have this rewriting you have the query decomposition something similar then you have the retrieval tools the internet retrieval tools they use here the microsoft internet explorer then you have the retrieval tools, the internet retrieval tools. They use here the Microsoft Internet Explorer. Then you have here BM25 retrieval. Then you have a data routing strategy that is standard of current RAC applications. Maybe what you see especially is the re-ranking of RAC, but I think all the current RAC implementation already have a re-ranking of rag but i think all the current drag implementation already have a re-ranking of their results so i wouldn't call this here a novelty a brand new special idea filters also implemented here in the advanced rack system so yeah and then the generation tools as they call here the tool use they have here the generation they have a citation tool this is nice and then they have refinement tools but i think yeah this reflection we already have at the base level and then they have a polishing so yes it's an optimized formulation tool but if you ask me what is the unique currently unseen new component i would not be able to answer this because i think it is a very nice bring all concepts together whatever we know we built now one coherent system and you see their web application this is built here with streamlit and you can have a look at this and if you want here the complete system the complete case here they have this oops i have to make it a little bit bigger that we can see this. So here's the Open Researcher and you say, okay, what is PPO and sends here for the proximal policy optimization. Then you can say, help me to summarize its latest development and identify its common variants. And then you have a tool here in our Open Researcher that is now trying, interacting with you asking you for further details this is also something new as they specify here in their paper so the open researcher comes back and says could you specify the time range for the latest development that you are interested in so you're looking for the last month for the for the last three years so what is it and you say just look at the last year and then you get here this and you have sub queries retrieve information on recent developments in ppo within the past year and then identify common variants of ppo since your question here yes okay so you see and then you get the result so i would say now this is a beautiful brought together implementation of an advanced rack system that i think you and i already use there are tools like a refinement tool or a re-ranking tool or whatever tool you want but in basic it is all there but now in one complete system integration maybe if you look at the code you would like to see this now in one frame so if you look here and or if you ask me what is here the brand new thing with open researcher is it accelerating scientific research i think it is a very nice rag implementation with all the methodologies that we know from advanced rag and they have a very beautiful implementation so if we go there you see yeah there it is table of contents using api setting up the process introduction the performance here so this is it this is this paper therefore i hope i've answered all your question regarding here whatever it was we are continuing here in this if you want multi-agent multi-domain coding and now today we had a look can we automate here that a super agent is selecting its minion agents to have the perfect configuration for our specific task in our specific domain, like theoretical physics or medicine or whatever, UF chemistry. And then we have a self-learning, self-reflection process happening. Careful could be real expensive, real fast. So have you got Rails ready and code is open source? So why not give it a try? And if you have some feedback in your particular branch, in your particular domain, hey, I think the community would be happy to hear from you. I hope you enjoyed it. I hope you found some new ideas. And it would be great to see you in my new video.
NEW: AI Agents That Design Themselves w/ ADAS
AI Agents that create and design themselves: Explore a novel framework for the automated design, evaluation, and refinement of AI agents through iterative collaboration and self-improvement processes. 

The system leverages advanced language models to generate agent architectures that consist of multiple specialized modules, each embodying expertise in specific domains such as physics, chemistry, and biology.

The framework operates by initiating a task that each expert module approaches independently, generating initial solutions through step-by-step reasoning. These solutions are then subjected to a cross-examination phase where peer critic modules analyze and provide constructive feedback on each other's outputs, identifying potential errors and areas for improvement.

In response to the critiques, the expert modules refine their solutions, incorporating insights and corrections to enhance accuracy and reliability. A final decision module then aggregates the refined solutions, performs comprehensive reasoning, and produces a cohesive and optimized answer to the original task.

This iterative process fosters a collaborative environment where AI agents can self-improve by learning from peer feedback and refining their reasoning strategies. The automated nature of the design and evaluation pipeline accelerates the development of robust AI systems capable of complex problem-solving through collective intelligence and adaptive learning.

The results demonstrate the effectiveness of this approach in creating sophisticated AI agents that exhibit enhanced performance and adaptability across various domains, highlighting the potential for scalable and automated AI development methodologies in tackling increasingly complex challenges in all industrial and medical areas.

#ai 
#aiagents 
#aiagent 
#newtechnology 
#science 

All rights with authors:
Automated Design of Agentic Systems
https://arxiv.org/pdf/2408.08435

GitHub Code: 
https://github.com/ShengranHu/ADAS

00:00 Automated Design of Agentic Systems
03:19 SOTA Hand-design Agents vs Meta Agent Search
05:33 GitHub Code repo ADAS
06:33 Real Code and Prompt for Meta Agent 
09:35 Self-Reflection Prompt of Meta Agent 
10:35 Framework Code
12:21 Python code for Review Agent (Minion)
14:45 GPT-4o explains Python Code of Agent
21:14 Template for an AI Agent: class LLMAgentBase 
25:10 Search Function is central to refine Agents
26:23 Example of an Agent Design
28:12 Special Bonus: NEW Open Researcher for accelerating Research