gorilla-llm/Berkeley-Function-Calling-Leaderboard
Preview
•
Updated
•
744
•
55
Siddharth
sidbin
AI & ML interests
None yet
Recent Activity
liked
a dataset
13 days ago
gorilla-llm/Berkeley-Function-Calling-Leaderboard
liked
a model
about 1 month ago
meta-llama/Llama-3.2-11B-Vision-Instruct
upvoted
a
collection
about 2 months ago
Granite 3.0 Language Models
Organizations
sidbin's activity
liked
a
dataset
13 days ago
liked
a
model
about 1 month ago
upvoted
a
collection
about 2 months ago
liked
2
datasets
about 2 months ago
upvoted
a
paper
about 2 months ago
upvoted
a
collection
about 2 months ago
reacted to
m-ric's
post with 🚀🔥
about 2 months ago
Post
1285
🤖 𝗔𝗱𝗼𝗯𝗲'𝘀 𝗰𝗼𝗱𝗲-𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗻𝗴 𝗮𝗴𝗲𝗻𝘁 𝗿𝗲𝗮𝗰𝗵𝗲𝘀 𝘁𝗵𝗲 𝘁𝗼𝗽 𝗼𝗳 𝗚𝗔𝗜𝗔 𝗹𝗲𝗮𝗱𝗲𝗿𝗯𝗼𝗮𝗿𝗱 - and their paper cites my work!
💡 Reminder: In short, Agentic systems are a vehicle in which you put your LLM to allow it access to the outside world.
➡️ The team of researchers at Adobe started from the idea that current agentic systems lack the ability to define their own tools. So they decided to make an agent that writes actions as code, thus allowing it to write python functions that can be re-used later as tools!
Here's what the LLM generations can look like with the proper prompt:
Thought: I need to access the excel file using a different method.
Action:
Then your system executes this and appends the observation to the agent's memory.
Why is this code formulation better than classical tool use formulation as JSON? The paper explains:
"Most existing work uses text or JSON as the representation of actions, which significantly lacks the two criteria mentioned earlier: generality and composability. In contrast, DynaSaur can utilize available actions or create new ones if necessary, using code as a unified representation. In principle, acting with code enables agents to solve any Turing-complete problem."
The idea of using code is not new: in fact, we do it in transformers.agents (thus the citation that I got). They implementation adds further refinements, like using RAG to retrieve relevant functions before generating an action, which increases performance further.
And they observe that code agents perform much better, reaching the top of GAIA leaderboard! 🥇
Go take a look, it's really clear and informative!
Paper added to my agents collection 👉 m-ric/agents-65ba776fbd9e29f771c07d4e
💡 Reminder: In short, Agentic systems are a vehicle in which you put your LLM to allow it access to the outside world.
➡️ The team of researchers at Adobe started from the idea that current agentic systems lack the ability to define their own tools. So they decided to make an agent that writes actions as code, thus allowing it to write python functions that can be re-used later as tools!
Here's what the LLM generations can look like with the proper prompt:
Thought: I need to access the excel file using a different method.
Action:
def access_excel_file(file_path)
... # rest of the code (the agent does writes it, but I don't have room in this post)
return rowsThen your system executes this and appends the observation to the agent's memory.
Why is this code formulation better than classical tool use formulation as JSON? The paper explains:
"Most existing work uses text or JSON as the representation of actions, which significantly lacks the two criteria mentioned earlier: generality and composability. In contrast, DynaSaur can utilize available actions or create new ones if necessary, using code as a unified representation. In principle, acting with code enables agents to solve any Turing-complete problem."
The idea of using code is not new: in fact, we do it in transformers.agents (thus the citation that I got). They implementation adds further refinements, like using RAG to retrieve relevant functions before generating an action, which increases performance further.
And they observe that code agents perform much better, reaching the top of GAIA leaderboard! 🥇
Go take a look, it's really clear and informative!
Paper added to my agents collection 👉 m-ric/agents-65ba776fbd9e29f771c07d4e
liked
2
datasets
about 2 months ago
liked
a
Space
2 months ago
liked
a
dataset
2 months ago
updated
3
models
2 months ago