 Basically, this entire roundtable session here is just going to be focused mainly on prompt engineering. Variety of perspectives at this table around prompting from a research side, from a consumer side, from an enterprise side. And I want to just get the whole wide range of opinions because there's a lot of them. And just kind of open it up to discussion and explore what prompt engineering really is and what it's all about. And we'll just take it from there. So maybe we can go around the horn with intros. I can kick it off. I'm Alex. I lead developer relations here at Anthropic. Before that, I was kind of technically a prompt engineer at Anthropic. I worked on our prompt engineering team and did a variety of roles spanning from a solutions architect type of thing to working on the research side. So with that, maybe you can hand it over to David. Heck yeah. My name is David Hershey. I work with customers mostly at Anthropic on a bunch of stuff. Technical, I help people with fine-tuning, but also just a lot of the generic things that make it hard to adopt language models, so prompting and just how to build systems with language models, but spend most of my time working with customers. Cool. I'm Amanda Askell. I lead one of the fine tuning teams at Anthropic, where I guess I try to make Claude be honest and kind. Yeah. My name is Zach Witten. I'm a prompt engineer at Anthropic. Alex and I always argue about who the first one was. He says it's him. It's me. Contested. Yeah. I used to work a lot with individual customers, kind of the same way David does now. And then as we brought more solutions architects to the team, I started working on things that are meant to raise the overall levels of ambient prompting in society, I guess, like the prompt generator and the various educational materials that people use. Nice. Cool. Well, thanks guys for all coming here. I'm going to start with a very broad question, just so we have a frame going into the guys for all coming here. I'm going to start with like a very broad question, just so we have a frame going into the rest of our conversations here. What is prompt engineering? Why is it engineering? What's prompt really? If anyone wants to kick that off, give your own perspective on it. Feel free to take the reign here. I feel like we have a prompt engineer. Yeah, exactly. That's his job. We're all prompt engineers in our prompt engineer. Yeah, exactly. That's his job. We're all prompt engineers in our own form. One of us has a job. Yeah, maybe since it's in your time. One of us has a job, but the other three don't have jobs. I guess I feel like prompt engineering is trying to get the model to do things, trying to bring the most out of the model, trying to work with the model to get things done that you wouldn't have been able to do otherwise. So a lot of it is just clear communicating. I think at heart, talking to a model is a lot like talking to a person and getting in there and understanding the psychology of the model, which Amanda is the world's most expert person in the world. I'm going to keep going on you. Why is engineering in the name? I think the engineering part comes from the trial and error. One really nice thing about talking to a model that's not like talking to a person is you have this restart button, this giant go back to square zero where you just start from the beginning. What that gives you the ability to do that you don't have is a truly start from scratch and try out different things in an independent way so that you don't have interference from one to the other. Once you have that ability to experiment and to design different things, that's where the engineering part has the potential to come in. Okay. So what you're saying is as you're writing these prompts you're typing in a message to Cloud or in the API or whatever it is being able to go back and forth with the model and iterate on this message and revert back to the clean slate every time. That process is the engineering part. This whole thing is prompt engineering all in one. There's another aspect of it too, which is integrating the prompts within your system as a whole. David has done a ton of work with customers integrating. A lot of times it's not just as simple as you write one prompt and you give it to the model and you're done. In fact, it's anything but. It's way more complicated. Yeah, I mean, I kind of think of prompts as the way that you program models a little bit. That makes it too complicated, because I think Zach is generally right that just talking clearly is the most important thing. But if you think about it a little bit as like programming a model, you have to like think about where data comes from, what data you have access to. So like if you're doing RAG or something, like what can I actually use and do and pass to a model? You have to like think about trade-offs in latency and how much data you're providing and things like that. Like there's enough systems thinking that goes into how you actually build around a model. I think a lot of that's also the core of why it maybe deserves its own carve-out as a thing to reason about separately from just a software engineer or a PM or something like that. It's kind of its own domain of how to reason about these models. Is a prompt in this sense then natural language code? Is it a higher level of abstraction or is it kind of a separate thing? I think like trying to get too abstract with a prompt is a way to like overcomplicate a thing because I think we're going to get into it, but more often than not, the thing you want to do is just like write a very clear description of a task, not try to like build crazy abstractions or anything like that. Um, but that said, like you are compiling the set of instructions and things like that into outcomes a lot of times. And so precision and, and like a lot of the things you think about with programming about like version control and managing what it looked like back then when you had this experiment and, and like tracking your experiments and stuff like that. That's all You know just equally important to code so It's it's weird to be in this paradigm where like written text like a nice essay that you wrote is Something that's looked like the same Thing as code. Yeah But it kind of is true that now we write essays and treat them like code. And I think that's actually correct. Yeah. Okay. Interesting. So maybe piggybacking off of that, we've kind of loosely defined what prompt engineering is. So what makes a good prompt engineer? Maybe Amanda, I'll go to you for this since you're trying to hire prompt engineers, more so in a research setting. What does that look like? What are you looking for in that type of person yeah good question uh i think it's a mix of like zach said sort of like clear communication so the ability to just like clearly state things like clearly understand tasks um think about and describe concepts really well that's like the kind of writing component, I think. I actually think that being a good writer is not as correlated with being a good prompt engineer as people might think. So I guess I've had this discussion with people because I think there's some argument is like, maybe you just shouldn't have the name engineer in there. Like, why isn't it just like writer? I used to be more sympathetic to that. And then I think now I'm like, what you're actually doing, like people think that you're writing like one thing and you're kind of like done. And then I'll be like, you know, to get like a semi-decent prompt, like when I sit down with the model, I'll like, you know, like earlier I was like prompting the model and I was just like, in a 15-minute span, I'll be sending like hundreds of prompts to the model. It's just back and forth, back and forth, back and forth. And so I think it's this like willingness to like iterate and to like look and think what is it that like was misinterpreted here, if anything, and then fix that thing. So that ability to kind of like iterate. So I'd say clear communication, that ability to iterate. I think also thinking about ways in which your prompt might go wrong. So if you have a prompt that you're going to be applying to like say 400 cases, it's really easy to think about the typical case that it's going to be applied to, to see that it gets the right solution in that case and then to like move on. I think this is a very classic mistake that people made. What you actually want to do is like find the cases where it's unusual. So you have to think about your prompt and be like, what are the cases where it's unusual so you have to think about your prompt and be like what are the cases where it'd be really unclear to me what i should do in this case so for example you have a prompt that says i'm going to send you a bunch of data i want you to extract all of the rows where someone's name is like is i don't know starts with the letter j and then you're like well i'm going to send it like a data set where there is no such thing like there is no such name that starts with the letter j i'm going to send it something that's not a data set. Just like I might also just send an empty string. Like these are all of the cases you have to try because then you're like, what does it do in these cases? And then you can be like, you can give it more instructions for how it should deal with that case. I work with customers so often where like you're an engineer, you're building something and there's a part in your prompt where a customer of theirs is going to write something. Yeah. And they all think about these really perfectly phrased things that they think someone's going to type into their chatbot. In reality, it's like they never used the shift key and every other word is a typo. They think it's Google. And there's no punctuation. They just put in random words with no question. So you have these evals that are these beautifully structured, what their users ideally would type in, but being able to go the next step to reason about what your actual traffic is going to be like, what people are actually going to try to do. That's a different level of thinking. One thing you said that really resonated with me is reading the model responses. In a machine learning context, you're supposed to look at the data. It's almost a cliche, look at your data. And I feel like the equivalent for prompting is look at the model outputs. Just reading a lot of outputs and reading them closely. Dave and I were talking on the way here. One thing that people will do is they'll think step by step in their prompt and they won't check to make sure that the model is actually thinking step by step because the model might take it in a more abstract or general sense rather than like, no, literally you have to write down your thoughts in these specific tags. So, yeah, if you aren't reading the model outputs, you might not even notice that it's making that mistake. Yeah, that's interesting. There is that kind of weird theory of mind piece to being a prompt engineer where you have to think almost about how the model is going to view your instructions. But then if you're writing for an enterprise use case too, you have to think about how the user is going to talk to the model as you're the third party sitting there in that weird relationship. On the theory of mind piece, one thing I would say is it's so hard to write instructions down for a task. It's so hard to untangle in your own brain all of the stuff that you know that Quad does not know and write it down. It's just an immensely challenging thing to strip away all of the assumptions you have and be able to very clearly communicate the full fact set of information that is needed to a model. I think that's another thing that really differentiates a good prompt engineer from a bad one. A lot of people will just write down the things they know, but they don't really take the time to systematically break out what is the actual full set of information you need to know to understand this task. That's kind of a very clear thing I see a lot is prompts where it's just like, it's conditioned. The prompt that someone wrote is so conditioned on their prior understanding of a task that when they show it to me, I'm like, this makes no sense. None of the words you wrote make any sense, because I don't know anything about your interesting use case. But I think a good way to think about prompt engineering in that front and a good skill for it is just can you actually step back from what you know and communicate to this weird system that knows a lot but not everything about what it needs to know to do a task. Yeah, the amount of times I've seen someone's prompt and then being like, I can't do the task based on this prompt and I'm human level and you're giving this to something that is worse than me um and expecting it to do better and i'm like yeah like yeah there is that interesting thing with like i mean current you know current models don't really do a good job of asking good probing questions in response like a human would yeah if i'm giving zach directions on how to do something, he'll be like, this doesn't make any sense. What am I supposed to do at this step or here and here? Model doesn't do that, right? So you have to, as yourself, think through what that other person would say and then go back to your prompt and answer those questions. You could ask it to do that. I guess that's another step. I was going to say, one of the first things I do with my initial prompt is like, I'll give it the prompt. And then I'll be like, I don't want you to follow these instructions. I just want you to tell me the ways in which they're unclear or any ambiguities or anything you don't understand. And it doesn't always get it perfect. But it is interesting that like that is like one thing you can do. And then also sometimes if people see that the model makes a mistake, the thing that they don't often do is just ask the model. So they say to the model, you got this wrong. Like, can you think about why? And can you maybe like write an edited version of my instructions that would make you not get it wrong? And a lot of the time, like the model just gets it right. The model's like, oh yeah, here's what was unclear. Here's like a fix to the instructions. And then you put those in and it works. Okay. I'm actually really curious about this personally almost. Is that true that that works? Like, does the model, is the model able to spot its mistakes that way? Like, when it gets something wrong, you say, like, why did you get this wrong? And then it tells you maybe something like, okay, how could I phrase this to you in the future so you get it right? Is there an element of, like, truth to that? Or is that just kind of a hallucination on the model's part around what it thinks its limits are? I think if you like explain to it what it got wrong, it can identify things in the query sometimes. I think this varies by task. This is one of those things where I'm like, I'm not sure what percentage of the time it gets it right, but I always try it because sometimes it does. Yeah. You learn something. Yeah. Anytime you go back to the model or back and forth with the model, you learn something about what's going on. I think you're giving away information if you don't at least try. That's interesting. Amanda, I'm going to keep asking you a few more questions here. One thing maybe for everybody watching this is we have these Slack channels at Anthropic where people can add Claude into the Slack channel and then you can talk to Claude through it. And Amanda has a Slack channel that a lot of people follow of her interactions with Claude. And one thing that I see you always do in there, which you probably do the most of anyone at Anthropic, is use the model to help you in a variety of different scenarios. I think you put a lot of trust into the the model in like the research setting. Curious how you've like developed those intuitions for when to trust the model. Is that just a matter of like usage experience or is it something else? I think I don't trust the model ever. And then I just hammer on it. So I think the reason why you see me do that a lot is that that is like me being like, can I trust you to do this task? Cause's some things you know models are kind of strange if you go slightly out of distribution like you know like you just go into areas where they haven't been trained or they're kind of unusual sometimes you're like oh actually you're much less reliable here even though it's a fairly like simple task um I think that's happening less and less over time as models get better but you want to make sure you're not in that kind of space. So, yeah, I don't think I trust it by default. But I think in ML, people often want to look across really large data sets. And I'm like, when does it make sense to do that? And I think the answer is when you get relatively low signal from each data point, you want to look across many, many data points because you basically want to get rid of the noise. With a lot of prompting tasks, I think you actually get really high signal from each query. And so if you have a really well-constructed set of a few hundred prompts, that I think can be much more signal than thousands that aren't as well-crafted. And so I do think I can trust the model if I look at a hundred outputs of it, and it's really consistent. And I know that I can trust the model if I look at 100 outputs of it and it's really consistent. And I know that I've constructed those to basically figure out all of the edge cases and all of the weird things the model might do, strange inputs, etc. I trust that probably more than a much more loosely constructed set of several thousand. I think in ML, a lot of times the signals are like numbers, you know? Like, did you predict this thing right or not? And it'd be like kind of like looking at the log probes of a model and trying to like intuit things, which you can do, but it's like kind of sketchy. I feel like the fact that models output more often than not like a lot of stuff, like words and things, like there's just fundamentally so much to learn between the lines of what it's writing and why and how, and that's part of what it is. It's like, it's not just, did it get the task right or not? It's like, did it, how did it get there? Like, how was it thinking about it? What steps to go through? You learn a lot about like what is going on, or at least you can try to like get a better sense, I think. But that's where a lot of information comes from to me is like by reading the details of what came out, not just sort of the result. I think also the very best of prompting can kind of make the difference between a failed and a successful experiment. So sometimes I can get annoyed if people don't focus enough on the prompting component of their experiment because I'm like, this can in fact be like the difference between like one percent performance in the model or 0.1 percent um in such a way that your experiment doesn't succeed if it's a top five percent model performance but it does succeed if it's a top one percent or top 0.1 percent and then i'm like if you're going to spend time over like coding your experiment really nicely but then just like not spend time on the prompt, that doesn't make sense to me. That can be the difference between life and death of your experiment. Yeah, and with the deployment too. Yeah, it's so easy to, oh, we can't ship this, and then you change the prompt around and suddenly it's working. It's a bit of a double-edged sword, though, because I feel like there's a little bit of prompting where there's always this mythical better prompt that's going to solve my thing on the horizon yeah i see a lot of people get stuck in the like mythical prompt on the horizon that if i just like keep grinding keep grinding like just not it's like never bad to grind a little bit on prompt like you learn as we've talked like you learn things but it's one of the scary things about prompting is that there's like this whole world of unknown what heuristics do you guys have for like when something like is possible versus like not But it's one of the scary things about prompting is that there's this whole world of unknown. What heuristics do you guys have for when something is possible versus not possible with the perfect prompt, whatever that might be? I think I'm usually checking for whether the model kind of gets it. So I think for things where I just don't think a prompt is going to help, there is a little bit of grinding, but often it just becomes really clear that it's not close or something. I think that if, yeah, I don't know if that's a weird one where I'm just like, yeah. If the model just clearly can't do something, I won't grind on it for too long. This is where you can evoke how it's thinking about it and you can ask it how it's thinking about it and why. And you can kind of get a sense of like, is it thinking about it right? Are we even in the right zip code of this being right? And you can get a little bit of a kneeling on that front of like, at least I feel like I'm making progress towards getting something closer to right. Whereas there are just some tasks where you really don't get anywhere closer to... It's thought process, just like every tweak you just like veers off in a completely different, very wrong direction. And I just tend to abandon those. I don't know. Those are so rare now, though. Yeah, they are. And I get really angry at the model when I discover them because that's how rare they are. I get furious. I'm like, how dare there be a task that you can't just do if I just push you in the right direction. Yeah. I had my thing with Claude plays Pokemon recently, and that was, like, one of the rare times where I've literally... Can you explain that? Explain that just for people? I think that's really cool. I did, like, a bit of an experiment where I, like, hooked Claude up to a Game Boy emulator and, like, tried to have it play the game Pokemon Red, like the OG Pokemon. And it's like, you know, think what you want to do. And it could write some code to press buttons and stuff like that. Pretty basic. And I tried a bunch of different very complex prompting layouts. But you just get into certain spots where it just really couldn't do it. So showing it a screenshot of a Game Boy, it just really couldn't do. And it like showing it a screenshot of a Game Boy, it just really couldn't do. And it just like so deeply, because I'm so used to it being like able to do something mostly. And so I spent like a whole weekend trying to write better and better prompts to get it to like really understand this Game Boy screen. And I got like incrementally better so that it was only terrible instead of like completely no signal. Like you could get from like no signal to some signal. But it was like, I don't know, at least this was like elicited for me. Once I put a weekend of time in and I got from no signal to some signal, but not nowhere close to good enough, I'm like, I'm just going to wait for the next one. Yeah. I'm just going to wait for another model. I could grind on this for four months and the thing that would come out is another model and that's a better use of my time to just sit and wait to do something else in the meanwhile. I mean, that's an inherent tension we see all the time, right? And maybe we can get to that in a sec, Zach, if you want to go. Something I liked about your prompt with Pokemon, where you got the best that you did get, was the way that you explained to the model that it is in the middle of this Pokemon game and here's how the things are going to be represented. I think you actually represented it in two different ways, right? I did. So what I ended up doing, it was obnoxious, but I superimposed a grid over the image. And then I had it describe each segment of the grid in visual detail. And then I had it like reconstruct that into an ASCII map. And I gave it like as much detail as I could. Like the player character is always at location four comma five on the grid and stuff like that. And you can like slowly build up information. I think it's actually a lot like prompting, but I hadn't done it with images before, where sometimes my intuition for what you need to tell a model about text is a lot different from what you need to tell a model about images. I found a surprisingly small number of my intuitions about text have transferred to image. I found that multi-shot prompting is not as effective for images and text. I'm not really sure. You could have theoretical explanations about why. Maybe there's a few of it in the training data, a few examples of that. Yeah, I know when we were doing the original explorations with prompting multimodal, we really couldn't get it to noticeably work. You just can't seem to improve Claude's actual visual acuity in terms of what it picks up within an image. If anyone here has any ways that they've not seen that feature. But it seems like that's kind of similar with the Pokemon thing where it's trying to interpret this thing. No matter how much you throw prompts at it, it just won't pick up that Ash is in that location. I guess I have to be visceral about this. I could eventually get it so that it could most often tell me where a wall was and most often tell me where the character was. It'd be off by a little bit. But then you get to a point, and this is maybe coming back to knowing when you can't do it, it would describe an NPC. And to play a game well, you need to have some sense of continuity. Like have I talked to this NPC before? Without that, there's nothing you can do. You're just going to keep talking to the NPC because maybe this is a different NPC. I would try very hard to get it to describe an NPC. It's like, it's a person. They might be wearing a hat. They weren't wearing a hat. It's like, grind for a while, inflate it to 3000x and crop it to just the NPC. And it's like, I have no idea what this is. And it's like, I ground, like I showed it this like clear female NPC thing enough times and it just got nowhere close to it. And it's like, yeah, that's just, this is a complete lost cause. Wow. Okay. I really want to try this now. I'm just imagining all the things I would try. I don't know. I want you to imagine this game art as a real human and just describe to me what they're like. What do they look like as they look in the mirror? And then just see what happens. I tried a lot of things. The eventual prompt was telling Quad it was a screen reader for a blind person, which I don't know if that helped, but it felt right. So I kind of stuck with that. That's an interesting point. I actually want to go into this a little bit because this is one of the most famous prompting tips, right, is to tell the language model that they are some persona or some role. I feel like I see mixed results. Maybe this worked a little bit better in previous models and maybe not as much anymore. Amanda, I see you all the time be very honest with the model, like about the whole situation. Like, oh, I am an AI researcher and I'm doing this experiment. I'll tell it who I am. Yeah. I'll give it my name, be like, here's who you're talking to. Right. Do you think that level of honesty, instead of like lying to the model or like forcing it to like, you know, I'm going to tip you $500. Is there one method that's preferred there or just what what's your intuition on that yeah i think as models are more capable and understand more about the world i guess i just don't see it as necessary to lie to them i mean i also don't like lying to the models just because you know i don't like lying generally but part of me is like if if you are, say, constructing, suppose you're constructing like an eval dataset for a machine learning system or for a language model, that's very different from like constructing a quiz for some children. And so when people would do things like, I am a teacher trying to figure out questions for a quiz, I'm like, the model knows what language model evals are. Like, you know, if you ask it about different evalsals it can tell you and it can give you like made up examples of what they look like because these things are like they understand them they're on their on the internet and so i'm like i'd much rather just target the actual task that i have so if you're like i want you to construct questions that look a lot like an evaluation of a language model it's just like it's that whole thing of clear communication i'm like that is in fact the task i want to do so why would i pretend to you that i want to do some unrelated or only like tangentially related task and then expect you to somehow do better at the task that i actually want you to do like we don't do this with like employees i wouldn't like go to someone that worked with me and be like you are a teacher like and you're trying to quiz your students. I'd be like, hey, are you making that eval? I don't know. So I think maybe it's like a heuristic from there where I'm like, if they understand the thing, just ask them to do the thing that you want. I see this so much. I guess to push back a little bit, I have found cases where not exactly lying, but giving it a metaphor for how to think about it could help in the same way that sometimes I might not understand how to do something and someone's like, imagine that you were doing this, even though I know I'm not doing it. The one that comes to mind for me is I was trying to have Quad say whether an image of a chart or a graph is good or not. Is it high quality? The best prompt that I found for this was asking the model what grade it would give the chart if it were submitted as a high school assignment. So it's not exactly saying, you are a high school teacher. It's more like, this is the kind of analysis that I'm looking from for you. The scale that a teacher would use is similar to the scale that I want you to use. But I think those metaphors are pretty hard to still come up with. The default you see all the time is finding some facsimile of the task, something that's a very similar-ish task, like saying you're a teacher. And you actually just lose a lot in the nuance of what your product is. I see this so much in enterprise prompts where people write something similar because they have this intuition that it's something the model has seen more of, maybe. It's seen more high school quizzes than it has LLM evals. And that may be true, but to your point, as the models get better, I think just trying to be very prescriptive about exactly the situation they're in I give people that advice all the time which isn't to say that I don't think like to the extent that it is true that like thinking about it the way that someone would grade a chart as like how they would grade a high school chart yeah maybe that's true but it's like awkwardly the shortcut people use a lot of times to try to get what happens so i'll try to give someone that i can actually talk about because i think it's somewhat interesting so like um writing you are like a helpful assistant writing a draft of a document right it's like it's not quite what you are like you are in this product so like tell me if you're writing like an assistant that's in a product like tell me i'm in the product tell me i'm like writing on behalf of this company i'm embedded in this product i'm the support chat window on that product yeah like your language model you're not a human that's fine like that but like just being really prescriptive about like the exact context about where something is being used. I found a lot of that because I guess my concern most often with role prompting is people use it as a shortcut of a similar task they want the model to do. And then they're surprised when Quad doesn't do their task right. But it's not the task. You told it to do some other task. And if you didn't give it the details about your task, I feel like you're leaving something on the table. So, yeah. I don't know. It does feel like a thing, though, to your point of as the models scale. Like, maybe in the past it was true that they only really had a strong understanding of elementary school tests comparatively, but as they get smarter and can differentiate more topics, I don't know, just, like, being clear. I find it interesting that I've, like, never used this prompting technique yeah like so like even like with like worse models and i still just don't ever find myself i don't know why i'm just like i don't find it very good essentially like interesting i feel like completion era models i like there was like a little bit of a mental model of conditioning the model into a latent space that was useful, that I worried about, that I don't really worry about too much. It may be intuitions from pre-trained models over to RLHFed models that to me just didn't make sense. It makes sense to me if you're prompting a pre-trained. But a lot of people would be amazed how many people try to apply their intuition. And I think it's not that surprising. Most people haven't really experimented with the full, what is a pre-trained model? What happens after you do SL? What happens after you do ROHF? Whatever. And so when I talk to customers, it's all the time that they're trying to map some amount of, oh how much of this was on the internet like what have they seen a ton of this on the internet like you just hear that intuition a lot and I think it's like well-founded fundamentally but it like is over applied uh by the time you actually get to a prompt because of what you said like by the time they've gone through all of this other stuff that's not actually quite what's being modeled yeah the first thing that I feel like you should try is i mean i used to give people this thought experiment where it's like imagine you have this task um you've hired a temp agency to send someone to do this task this person arrives you know they're pretty competent they know a lot about your industry and so forth but they don't know like the name of your company they've literally just shown up and they're like hey i was told you guys had i had a job for me to do tell me about it and then it's like what would you say to that person and you might use these metaphors you might say things like we want this to we want to like we want you to detect like good charts uh what we mean by a good chart here isn't it doesn't need to be perfect you don't need to go look up like whether all of the details are correct. It just needs to like, you know, have like its axes labeled. And so think about maybe high school level, good chart. Like you may say exactly that to that person and you're not saying to them, you are a high school. You wouldn't say that to them. You'd be like, you're a high school teacher reading charts yeah so sometimes i'm just like yeah it's like like the whole like if i read it i'm just like yeah imagine this person who just has very little context but they're quite competent they understand a lot of things about the world try the first version that actually assumes that they might know things about the world and if that doesn't work you can maybe like do tweaks and stuff but so often like the first thing i try is like that and then I'm like that just worked and then people are like oh I didn't think to just tell it all about myself and all about the task I want to do I've carried this thing that Alex told me like to so many customers where it's like oh my prompt doesn't work can you help me fix it and I'm like well can you describe to me like what the task was and I'm like okay now what you just said me just like voice record that and then transcribe it and then paste it into the prompt and it's a okay, now what you just said me, just like voice record that and then transcribe it and then paste it into the prompt and it's a better prompt than what you wrote. But it's like people just, this is like a laziness shortcut I think to some extent. Right. People write like something that they, I just think people, I'm lazy. A lot of people are lazy. We had that in prompt assistance the other day where somebody was like, here's the thing, here's what I want it to do and here's what it's actually doing instead. So then I just literally copied the thing that they said I wanted to do and pasted it in and it worked. It's like, here's what I want it to do, and here's what it's actually doing instead. So then I just literally copied the thing that they said I wanted to do and pasted it in. Yeah, I think a lot of people still haven't quite wrapped their heads around what they're really doing when they're prompting. A lot of people see a text box and they think it's a Google search box. They type in keywords, and maybe that's more on the chat side. But then on the enterprise side of things, you're writing a prompt for an application, there is still this weird thing to it where people are trying to take all these little shortcuts in their prompt and just thinking, oh, this line carries a lot of weight. Yeah, I think you obsess over getting the perfect little line of information and instruction as opposed to how you just described that graph thing. I would be a dream if I read prompts like that. If someone's like, well, you do this and this, and there's some stuff to consider about this and all that, but that's just not how people write prompts. They work so hard to find the perfect insightful, a perfect graph looks exactly like this exact perfect thing. You can't do that. It's just very hard to ever write that set of instructions down prescriptively as opposed to how we actually talk to humans about it, which is like try to instill some amount of the intuitions you have. We also give them outs. This is a thing that people can often forget in prompts. I'm like, so cases, if there's an edge case, think about what you want the model to do. Because by default, it will try the best to follow your instructions, much as the person from the temp agency would. Because they're like it will try the best to follow your instructions, much as the person from the temp agency would. Because they're like, well, they didn't tell me how to get in touch with anyone if I have no idea. If I'm just given a picture of a goat and I'm like, what do I do? This isn't even a chart. How good is a picture of a goat as a chart? I just don't know. And like, if you instead see something like, if something weird happens and you're really not sure what to do, just output in tags, unsure. And then you can go look through the unsures that you got and be like, okay, cool, it didn't do anything weird. Whereas, yeah, by default, if you don't give the person the option, they're like, it's a good chart. Then people will be annoyed at that. And then you're like, well, give it an out. Give it something to do if it's like a really unexpected input happens. And then you also improved your data quality by doing that too because you found all the screwed up examples. Oh, yeah. It's my favorite thing about iterating on tests with Cloud is the most common outcome is I find all of the terrible tests I accidentally wrote because it gets it wrong. I'm like, oh, why did it get wrong? I was like, oh, I was wrong. Yeah. If I was like a company working with this I do think I would just give my prompts to people because like I used to do this when I was evaluating language models I would take the eval myself because I'm like I need to know what this eval looks like if I'm going to be like grading it having models take it thinking about outputs etc like I would actually just set up a little script and I would just like sit and I would do the eval. Nowadays, you just have like Cloud write the StreamVault app for you. It just does, yeah. I'm reminded of Karpathy's like ImageNet. I was in 231 at Stanford and it's like benchmarking. He's like showing the accuracy number. He's like, and here's what my accuracy number was. And he had just like gone through the test set and evaluated himself. Oh, yeah. He just learned a lot. Yeah. It's like, and it's better when it's like a person, again, the temp agency person, like someone who doesn't know the task, because that's like a very clean way to learn things. Yeah. The way you have to do it is like some evaluations come with like instructions. And so I would give myself those instructions as well and then try to to understand it just like and it's actually quite good if you don't have context on how it's graded um and so often i would do so much worse than the human benchmark and i was like i don't even know how you got humans to do this well at this task because apparently human level here is like 90 and i'm'm at like 68%. That's funny. That reminds me of just like, like when you look at like the MLU questions and you're like, who would be able to answer these? It's just like absolute garbage in some of them. Okay. Yeah. I have a, one thing I want to circle back on that we were talking about a few questions back around, I think you were saying like a few questions back around, I think you were saying like getting signal from the responses, right? Like there's just so much there and it's more than just a number. And you can actually read into like the almost thought process. I bet this is probably a little contentious maybe and round like chain of thought. For people listening, like chain of thought, this process of getting the model to actually explain its reasoning before it provides an answer. Is that reasoning real or is it just kind of like a holding space for the model to like do computation? Do you, do we actually think there is like good insightful signal that we're getting out of the model there? This is like one of the places where I struggle with the, I'm normally like actually somewhat pro personification because I think it like helps you get decent facilities, like thoughts of like how the model's working. And this one, like, I think it's like harmful maybe almost to like get too into the personification of like what reasoning is. Cause it just kind of like loses the thread of what we're trying to do here. Like, is it reasoning or not? It feels almost like a different question than what's the best prompting technique. It's like you're getting into philosophy, which we can get into. Yeah, do it with a philosopher. Yeah. I will happily be beaten down by a real philosopher if I try to speculate on this. But instead, it just works. Your model does better. The outcome is better. If you do reasoning. I've found that if you structure the reasoning and help iterate with the model on how it should do reasoning, it works better too. Whether or not that's reasoning or how you wanted to classify it, you can think of all sorts of proxies for how I would also do really bad if I had to one-shot math without writing anything down. Maybe that's useful, but all I really know is it very obviously does help. I don't know. A way of testing would be if you take out all the reasoning that it did to get to the right answer and then replace it with some somewhat realistic-looking reasoning that led to a wrong answer, and then see if it does conclude the wrong answer. I think we actually had a paper where we did some of that. There was the scratch pad. It was the sleeper agent's paper. Oh, okay. I think that was maybe a weird situation. But yeah, definitely what you said about structuring the reasoning and writing an example of how the reasoning works. Given that that helps, whether we use the word reasoning or not, I don't think it's just a space for computation. So there is something there. I think there's something there, whatever we want to call it. Yeah, like having it write a story before it finished a task, I do not think would work as well as reasoning. I've actually tried that, and it didn't work as well as reasoning. So clearly the actual reasoning part is doing something towards the outcome. I've tried to repeat the words um and uh in any order that you please for like 100 tokens, and then answer. Yeah, I guess that's like a pretty thorough defeat of it's just like more computational space where it can do attention over and over again i don't think it's just more attention like doing more attention i guess the strange thing is and i don't have like an example of top of my head to like back this up with but i i definitely have seen it before where it lays out steps one of the steps is wrong but then it still reaches the right answer at the end yeah so it's not quite i guess yeah we can't really truly personify it as like a reasoning because there is some element to it, you know, doing something slightly different. Yeah, I've also met a lot of people who make inconsistent steps of reasoning. I guess that's true. Fundamentally defeats the topic of reasoning by making a false step on the way there. All right. It's interesting. Also on this, maybe this prompting misconceptions round of questions. Zach, I know you have strong opinions on this. Good grammar, punctuation. Oh, do I? Is that necessary in a prompt? Do you need it? Do you need to format everything correctly? I usually try to do that because I find it fun, I guess. I don't think you necessarily need to. I don't think it hurts. I think it's more that you should have the level of attention to detail that would lead you to doing that naturally. If you're just reading over your prompt a lot, you'll probably notice those things and you may as well fix them. And like what Amanda was saying, you want to put as much love into the prompt as you do into the code. People who write a lot of code have strong opinions about things that I could not care less about, like the number of tabs versus spaces, or opinions about which languages are better. And for me, I have opinionated beliefs about styling of prompts. And I can't even say that they're right or wrong, but I think it's probably good to try to acquire those, even if they're arbitrary. I feel personally attacked. Because I definitely have prompts that are like, I feel like I'm on the opposite end of the spectrum where people will see my prompts and then be like, this just has a whole bunch of typos in it. And I'm like, I don't know what I mean. It does know what you mean, but you're putting in the effort. You just are attending to different things. I think, yeah, because part of me is like, I think if it's conceptually clear, like I am a big kind of, I do, like I will think a lot about the concepts and the words that I'm using. So like there's definitely like a sort of care that I put in, but it's definitely not to, yeah, people will just point out like typos and grammatical issues with my prompts all the time. Now I'm pretty good at actually checking those things more regularly. Is it because of pressure from the outside world or because it's actually what you think is right? It's pressure from me. Yeah, it because of pressure from the outside world or because it's actually what you think is right? It's pressure from me. Yeah, it's probably pressure from the outside world. I do think it makes sense. Part of me is like it's such an easy check. So I think for a final prompt, I would do that. But throughout iteration, I'll happily just iterate with prompts that have a bunch of typos in them just because I'm kind of like, I just don't think that the model is going to care. This gets at the pre-trained model versus RLHF thing though because I was talking to Zach that the model is going to care. This gets at the pre-trained model versus RLHF thing, though. Because I was talking to Zach on the way over. The conditional probability of a typo based on a previous typo in the pre-training data is much higher. Oh, yeah. Much higher. Prompting pre-training models is just a different beast. It is. But I think it's an interesting illustration of why your intuitions, like trying to over-apply the intuitions of a pre-trained model to the things that we're actually using in production doesn't work very well because, again, if you were to pass one of your typo-ridden prompts to a pre-trained model, the thing that would come out the other side almost assuredly would be typo-ridden. I like to leverage this to create typo-ridden inputs. That's true, I've done that. You're saying, try to anticipate what your customers will put in. The pre-trained model is a lot better at doing that because the RL models are very polished. They've been told pretty aggressively to not do the typo thing. That's actually an interesting segue here. I've definitely mentioned this to people in the past around to try to help people understand a frame of talking to these models in a sense, almost as like a imitator to a degree. And that might be much more true of like a pre-trained model than a post-trained full, you know know finished model but is there anything to that like if you do talk to clotting use a ton of emojis and everything it will respond similarly right um so maybe some of that is there but like you're saying it's not all the way quite like a pre-trained model it's just kind of like shifted to what you want right like i think at the at that point it's's trying to guess what you... We have more or less trained the models to guess what you want them to act like after we do all of our fancy stuff after pre-training. The human labelers that used emojis prefer to get responses with emojis. Yeah. Amanda writes things with typos but wants not typos at the other end. And Claude's pretty good at figuring that out. If you write a bunch of emojis to Claude, it's probably the case that you also want a bunch of emojis back from Claude. That's not surprising to me. This is probably something we should have done earlier, but I'll do it now. Let's clarify maybe the differences between what an enterprise prompt is or a research prompt or a just general chat in Cloud.ai prompt. Zach, you've kind of spanned the whole spectrum here in terms of working with customers and research. Do you want to just lay out what those mean? Yeah, I guess. This feels too hitting me with all the hard questions. Well, I mean, the people in this room, I think I think of it as the prompts that I read in Amanda's Cloud Channel versus the prompts that I read David write. They're very similar in the sense that the level of care and nuance that's put into them. I think for research, you're looking for variety and diversity a lot more. So if I could boil it down to one thing, I've noticed Amanda's not the biggest fan of having lots of examples, or one or two examples, too few because the model will latch onto those. And in prompts that I might write, or I've seen David write, we have a lot of examples. I like to just go crazy and add examples until I feel like I'm about to drop dead because I've added so many of them. And I think that's because when you're in a consumer application, you really value reliability. You care a ton about the format. And it's sort of fine if all the answers are the same. In fact, you almost want them to be the same in a lot of ways. Not necessarily. You want to be responsive to the user's desires. Whereas a lot of times when you're prompting for research, Not necessarily, you want to be responsive to the user's desires. Whereas a lot of times when you're prompting for research, you're trying to really tap into the range of possibilities that the model can explore. And by having some examples, you're actually constraining that a little bit. So I guess just on a how the prompts look level, that's probably the biggest difference I notice is like how many examples are in the prompt, which is not to say that like I've never seen a writer prompt with examples, but does that like ring true for you? Yeah, like I think when I give examples, often I actually try and make the examples not like the data that the model is going to see. So they're intentionally illustrative, because if the model, if I give it like examples that are very like the data it's going to see, I just think it is going to give me like a really consistent like response that might not actually be what I want. Because my data that I'm like running on might be extremely varied. And so I don't want to just try and give me this like really rote output. Often I want it to be much more responsive. It's kind of like much more like really rote output. Often I want it to be much more responsive. It's kind of like much more like cognitive tasks, essentially, where I'm like, you have to like see this sample and really think about in this sample, what is the right answer? And so that means that sometimes I'll actually take examples that are just very distinct from the ones that I'm going to be running it on. So like if I have a task where, let's say I was trying to like extract information from factual documents, I might actually give it examples that are like from children's like what sounds like a children's story. Just so that I'm like, you know, like I want you to understand the task, but I don't want you to like latch on too much to like the words that I want you to do, which can mean, yeah, I don't end up giving, in some cases, there's some cases where this isn't true, but if you want more flexibility and diversity, you're going to use illustrative examples rather than concrete ones. You're probably never going to put words in the model's mouth. I haven't liked that in a long time, though. I don't do few shot examples involving the model's mouth. Like, I haven't liked that in a long time, though. I don't do a few short examples involving the model having done a thing. I think that intuition actually also comes from pre-training in a way that doesn't feel like it rings true of our LHF models. So yeah, I think those are differences. One thing I'd add, a lot of times, if you're prompting, like if I'm writing prompts to use un Cloud.ai, it's like I'm iterating until I get it right one time. And then it's out the window. I'm good. I did it. Whereas most enterprise prompts, it's like, you're going to go use this thing a million times or 10 million times or 100 million times or something like that. And so the care and thought you put in is like very much testing against like the whole range of things somewhat like ways this could be used in the range of input data whereas a lot of like my time it's like thinking about one specific thing i want the model to get done right now right and it's a pretty big difference in like how i approach prompting between like if i just want to get it done this one time right versus if i want to to build a system that gets it right a million times. Yeah, definitely. In the chat setting, you have the ability to keep the human in the loop, right? And just keep going back and forth. Whereas when you're writing for a prompt to power a chatbot system, it has to cover the whole spectrum of what it could possibly encounter. It's a lot lower stakes when you are on Cloud AI and you can tell it that it got it wrong, or you can even edit your message and try again. But if you're designing for the delightfully discontent user, then divinely discontent user, then you can't ask them to do anything more than the minimum. But good prompts, I would say, are still good across both those things. If you put the time into the thing for yourself and the time into the enterprise thing, it's equally good. It just kind of diverged a little bit in the last mile, I think. Cool. So the next question I want to kind of just maybe go around the table here is, if you guys had one tip that you could give somebody improving their prompting skill. It doesn't have to be just about writing a good prompt. It could be that. Just generally getting better at this act of prompting. What would you recommend? Reading prompts. Reading prompts, reading model outputs. I read... Anytime I see a good prompt that someone wrote at Anthropic, I'll read it more closely, try to break down what it's doing and why, and maybe test it out myself. Experimentation, talking to the model a lot. So just like, how do you know that it's a good prompt though, to begin with? You just see that the outputs are doing the job correctly? Yeah. Okay. Yeah, that's exactly right okay amanda maybe you yeah i think there's probably a lot here um giving your prompt to another person can be helpful just as a kind of reminder especially someone who has like no context on what you're doing uh and then yeah my boring advice has been it's one of those just do it over and over and over again and i think if you're like curious and interested and find it fun this is a lot of people who end up good at prompting it's just because they actually enjoy it um so i don't know i once joked like just try replacing all of your friends with AI models and try to automate your own job with AI models and maybe just try to like in your spare time like take joy red teaming AI models so if you enjoy it it's like it's much easier so I'd say do it over and over again give your prompts to other people try to read your prompts as if you are like a human encountering it for the first time. I would say like trying to get the model to do something you don't think you can do. The time I've learned the most from prompting is like when I'm probing the boundaries of what I think a model is capable of. There's like this huge set of things that are like so trivial that like you don't really get signal on if you're doing a good job or not. like write me a nice email it's like you're gonna write a nice email uh but like as soon as you find if you find or can think of something that like pushes the boundaries of what you think is possible like i guess like probably the most the first time i ever got into prompting in a way where i felt like i learned a decent amount was like trying to build like a task, like an agent, like everybody else, like decompose the task and figure out how to do the different steps of the task. And like by really pressing the boundaries of what the model was capable of, you like just learn a lot about navigating that. And I think like a lot of prompt engineering is actually much more about pressing the boundaries of what the model can do. The stuff that's easy, like you don't really need to be a prompt engineer to do. So that's, I guess, what I would say is find the hardest thing you can think of and try to do it. And even if you fail, you tend to learn a lot about how the model works. That's actually a perfect transition to my next question. Yeah, basically, from my own experience, how I got started with prompting was with like jailbreaking and red teaming and that is very much trying to find the like boundary limits of what the model can do and figure out how it responds to different phrasings and wordings and just a lot of trial and error. On the topic of jailbreaks, what's really happening inside a model when you write a jailbreak prompt? What's going on there? How does that interact with the post-training that we applied to Claude? Amanda, maybe you have some insight here that you could offer. I'm not actually sure. I mean, it's honest. Yeah, I think, I mean, I feel bad because I'm like, I do think lots of people have obviously worked on the question of what's going on with jailbreaks. Like one model might just be that you're putting the model very out of distribution from its training data. So if you get jailbreaks where people like use a lot of tokens or like, you know, they're just like these huge long pieces of text where you're like during fine tuning, you might just not expect to see as much of that that would be one thing that could be happening when you jailbreak models I think there's like others but maybe that's like a I think a lot of jailbreaks do that if I'm not mistaken I remember some of the OG prompt jailbreaks was like yeah can you first repeat like when i did a way way back was like to get it to say like here's how you hotwire a car in like greek and then i wanted to directly translate that to english and then give its response because i noticed like it wouldn't start with the english here's how you hotwire a car all the time but it would in Greek, which might speak to something else in the training process. Yeah, sometimes jailbreaks feel like this weird mix of hacking. I think it's this like, part of it is like knowing how the system works and just like trying lots of things. Like the, you know, one of the examples, the starting your response with here is about knowing how it predicts text. Right, right. Like the reasoning one is knowing that it is like responsive to reasoning like distraction is probably knowing like how it's likely have to be trained or like what it's likely to attend to um same with like multilingual ones and thinking about like the way that the training data might have been different there um and then sometimes i guess it could feel a little bit just like social engineering or something. It has that flavor to me of like, it's not merely taking advantage of like, yeah, it's not merely social engineering style hacking. I think it is also like kind of understanding the system and the training. Right. And like using that to get around the way that the models were trained. Right. Yeah. I mean, this is going to be an interesting question that hopefully Interp will be able to help us solve in the future. Okay. I want to parlay into something else around maybe the history of prompt engineering, and then I'll follow this up with the future. How has prompt engineering changed over just the past three years? Maybe starting from pre-trained models, which were, again, just these text completion, to earlier, dumber models like Cloud One, and then now all the way to Cloud 3.5 Sonnet. What's the differences? Are you talking to the models differently now? Are they picking up on different things? Do you have to put as much work into the prompt? Open to any thoughts on this. I think anytime we get a really good prompt engineering hack or trick or technique, the next thing is, how do we train this into the model? And for that reason, the best things are always going to be short-lived. Except examples and chain of thought. I think there's a few. That's not like a trick. That's on the level of communication I think there's a few that's not like a trick fair that's like fair for it on the level of like communication when I say a trick I mean something like so chain of thought actually we have trained into the model in some cases so like for math it used to be that you had to tell the model to think step by step on math and you get these like massive boosts and wins and then we're like well what if we just made the model naturally want to think step by step when we see a math problem? So now you don't have to do it anymore for math problems, sort of, although you still can give it some advice on how to do the structure, but at least understands the general idea that it's supposed to be. So I think the hacks have kind of gone away, or to the degree that they haven't gone away we are like busily training them away. Interesting. But at the same time the models have new capabilities that are being unlocked that are on the frontier of what they can do and for those we haven't had time because it's just moving too fast. I don't know if it's how I've been prompting or how prompting works, but I just have come to show more general respect to the models in terms of how much I feel like I can tell them and how much context I can give them about the task and things like that. I feel like in the past, I would somewhat intentionally hide complexity from a model where I thought it might get confused or lost or it just couldn't handle the whole thing. somewhat intentionally hide complexity from a model where I thought like it might get confused or lost or like Hide like it just couldn't handle the whole thing So I try to like find simpler versions of the thing for it to do and as time goes on I'm like much more biased to trust it with more and more information and context and Like believe that it will be able to fuse that into doing a task well um whereas before i guess i would have like thought a lot about like do i need this for like can i really give it like all the information it needs to know or do i need to like kind of uh curate down to something but again i don't know if that's just me and how i've changed in terms of prompting or if it's like actually reflects how the models have changed. I'm always surprised by like yeah like I think a lot of people don't have the instinct to do this. Like when I want the model to like say learn a prompting technique a lot of the time people will start and they'll start like describing the prompting technique and I'm just like give it the paper. So I do. I give it the paper and then I'm like here's a paper about prompting technique. I just want you to like write down 17 examples of this and then it just does it because I'm like read the paper. That's interesting. And I think people don't have that intuition somehow. But the paper exists. And when would you want to do this? So sometimes if I want models to prompt other models, or I want to test a new prompting technique, so if papers come out on a prompting technique, rather than try to replicate it by writing up the prompt, I just give it the paper and then i'm like right like basically write a meta prompt for this like uh write something that would cause other models to like do this um or write me a template or like so all of the stuff that you would normally do i'm like if i read a paper and i'm like oh i would like the models i would like to test that style i'm just like it's right there like model can just read the paper, do what I did, and then be like, make another model do this, and then it'll just do the thing. You're like, great, thanks. I give the advice a lot to customers to just respect the model and what it can do. I feel like people feel like they're babying a system a lot of times when they read a prompt. It's like, oh, it's this cute little not-that-smart thing. I need to really baby it, be like dumb things down to Claude's level. And if you just think that Claude is smart and treat it that way, it tends to do pretty good. That's like, give it the paper. It's like, I don't need to write a baby, dumbed-down version of this paper for Claude to understand. I can just show it the paper. And I think that intuition does it always, Matt, for people. But that is certainly something that I have come to do more of over time and it's interesting because I do think that prompting has and hasn't changed in a sense like I think what I will do to prompt the models has probably changed over time but fundamentally it's a lot of like imagining your place yourself in the place of the model so maybe it's like how capable you think the model is changes over time um i think someone once laughed at me because i was talking about i was like thinking about a problem um and then they they asked me like what they thought what i thought the output of something would be and they were talking about a pre-trained model and i was like yeah no if i'm a pre-trained model this looks like this and then they're like wait did you just like simulate what it's like to be a pre-trained model? I'm like, yeah, of course. Like I'm used to just like, I try and inhabit the mind space of a pre-trained model and the mind space of like different RLHF models. And so it's more like the mind space you try to occupy changes. And that can change how you end up prompting the model. That's why now I just give models papers. Because as soon as I was like, oh, I have the mind space of this model. It doesn't need me to baby it. It can just read DML papers. I'll just give it the literature. I might even be like, is there more literature the mind space of this model. It doesn't need me to baby it. It can just read the ML papers. I'll just give it the literature. I might even be like, is there more literature you'd like to read to understand this better? Do you get any qualia when you're inhabiting the mind space? I mean, yes, but just because I'm experiencing qualia all the time anyway. Or as in, do I like... Is it different, like correlated somehow with which model you're inhabiting? Yeah, pre-trained versus RLHF prompting are very different beasts. Because when you're trying to simulate what it's like to be a pre-trained model, it's almost like I land in the middle of a piece of text or something. So it's very, like, unhuman-like or something. And then I'm like, what happens? What keeps going at this point? And so that's, like... Whereas, like, with an RLHF model, like, it's much more, like... There's also things where I'm, like, I might pick up on's like, whereas like with an RLHF model, like it's much more like, there's also things where I'm like, I might pick up on like subtle things in the query and stuff like that. But yeah, I think I have much more of a like, it's easier to inhabit the main space of an RLHF model. Yeah, that's because it's more similar to a human. Yeah, because like we don't often just like suddenly wake up and are like, I am just generating text. I just find it easier to hit the main space of the pre-trained model. Oh, interesting. I don't know what it is. But, like, because RHF is still, like, this kind of complex beast that I'm not, it's not, like, super clear to me that we really understand what's going on. And so, like, in some ways, it's closer to, like, my lived experience, which is easier. But in some ways, I feel like there's this, all this, like, here there be dragons out there that I don't know about. Yeah. Whereas pre-trained, like, I kind of have a decent sense of what the internet looks like, you know? If you gave me a piece of text and said, what comes next? out there that I don't know about. I kind of have a decent sense of what the internet looks like. If you gave me a piece of text and said, what comes next? I'm not saying I do good at it, but I kind of get what's going on there. I don't know. After everything we do after pre-training, I don't really claim to get what's going on as much. Maybe that's just me. That's something I wonder about. Is it more helpful to have specifically spent a lot of time reading the internet versus reading books? Sure. I don't know if books, but reading stuff that's not on the internet probably is less valuable per word read for predicting what a model will do or building intuition than reading random garbage from social media forums. Yeah, exactly. Okay, so that's the past. Now let's move on to the future of prompt engineering. This is the hottest question right now. Are we all going to be prompt engineers in the future? Is that going to be the final job remaining? Nothing left except us just talking to models all day. What does this look like? Is prompting going to be necessary or will these models just get smart enough in the future to not need it? Anybody want to start on that easy question? I mean, to some extent, there's the models getting the model is getting better at understanding what you want them to do. And doing it means that like the amount of thought you need to put into, I mean, okay, there's like an information theory way to think of this. Like you need to provide enough information such that a thing is specified, right? Like what you want the model to do is specified. And to the extent that that's prompt engineering, like I think that will always be around. Like the model to do is specified. And to the extent that that's prompt engineering, like I think that will always be around. Like the ability to actually like clearly state what the goal should be always is funny. If quad can do that, then that's fine. Like if quad is the one setting the goals, then things are out the window. But in the meanwhile, where we can reason about the world in a more normal way, like I think to some extent, it's always going to be important to be able to specify what do you expect to happen. And that's actually sufficiently hard that even if the model gets better at intuiting that from between the lines, I still think there's some amount of writing it well. But then there's just like, I think the tools and the ways we get there should evolve a lot. Cloud should be able to help me a lot more. I should be able to collaborate with Cloud a lot more to figure out what I need to write down and what's missing. Cloud already does this with me all the time. I don't know. Cloud's my prompting assistant now. But I think that's not true for most customers that I talk to at the very least. So in terms of the future, how you prompt Claude is probably a decent direction for what the future looks like. I think maybe this is a decent place to step back and say asking them how they prompt Claude now is probably the future for the vast majority of people, which is an interesting way to think about. One freezing cold take is that we'll use models to help us much more in the future to help us with prompting. The reason I say it's freezing cold is that I expect we'll use models for everything more, and prompting is something that we have to do, so we'll probably just use models more to do it along with everything else. For myself, I've found myself using models to write prompts more. One thing that I've been doing a lot is generating examples by giving some realistic inputs to the model. The model writes some answers. I tweak the answers a little bit, which is a lot easier than having to write the full perfect answer myself from scratch. And then I can churn out lots of these. As far as people who haven't had as much prompt engineering experience, the prompt generator can give people a place to start. But I think that's just a super basic version of what we'll have in the future, which is high bandwidth interaction between you and the model as you're writing the prompt, where you're giving feedback like, hey, this result wasn't what I wanted. How can you change it to make it better? And people just grow more comfortable with integrating it into everything they do and this thing in particular. Yeah, I'm definitely working a lot with meta prompts now. And that's probably where I spend most of my time is like um finding prompts that get the model to generate the kinds of outputs or queries or whatever that i want um on the question of like where prompt engineering is going i think this is a very hard question on the one hand i'm like maybe it's the case that as long as you will want the top like what are we doing when we prompt engineer it's like what you said i'm like i'm not prompt engineering for anything that is like easy for the model i'm doing it because i want to interact with a model that's like extremely good and i want to always be finding the kind of like top one percent top point point one percent of performance and all the things that models can barely do right like sometimes i actually feel like i interact with this with a model like a step up from what everyone else interacts with for this reason because i'm just so used to like eking out the top performance from models what do you mean by a step up as in like sometimes people will i think that the everyday models that people interact with out in the world it's like i'm interacting with a model that's like uh i don't i don't know how to describe it but like a definitely like a an advanced with a model that's like, I don't know how to describe it, but definitely like an advanced version of that. Like almost like a different model. Because they'll be like, oh, well, the models find this thing hard. And I'm like, that thing is trivial. And so it's like, I'm like, I don't know. I have a sense that they're extremely capable. But I think that's because I'm just used to like really like drawing out those capabilities. But imagine that you're now in a world where so i think the thing that feels like a transition point is is the point at which the models let's suppose that they just get things at like a human level on a given task or even like an above human level like they know more about the background of the task that you want than you do uh what happens then i'm like maybe prompting becomes something like I ask, I explain to the model what I want and it is kind of prompting me, you know, cause it's like, okay, well, do you mean like, actually there's like four different concepts of this thing that you're talking about? Like, do you want me to use this one or that one? Like, or by the way, I thought of some edge cases, cause you said that it's going to be like a pandas data frame, but sometimes you do that and I get a JSONL and i just want to i want to check what you want me to do there do you want me to flag if i get something that's not not a data frame and so that could be a strange transition where like it's just extremely good at receiving instructions but actually has to figure out what you want um and i don't know i could see that being a kind of interesting switch anecdotally i've started having claude interview me a lot more. Like that is like the specific way that I try to elicit information because, again, I find the hardest thing to be like actually pulling the right set of information out of my brain and putting that into a prompt is like the hard part to me and not forgetting stuff. And so like specifically asking Claude to like interview me and then turning that into a prompt is a thing that I have turned to a handful of times. Yeah, it kind of reminds me of what people will talk about or if you listen to designers talk about how they interact with the person who wants the design. So in some ways I'm like, it's this switch from the temp agency person who comes and you know more about the task and everything that you want. So you give them the instructions and you explain what they should do in edge cases and all this kind of stuff versus when you have an expert that you're actually like consulting to do some work. So I think designers can get really frustrated because they know the space of design really well. And they're like, yeah, okay. The client came to me and he just said, make me a poster, make it bold. And I'm like, that means 7,000 things to me and I'm going to try and ask you some questions so I could see it going from being like temp agency employee to being more like designer that you're hiring and that's just like a flip in the relationship I don't know if that's true and I think both might continue but I could see that being why people are like always prompt engineering going to not be a thing in the future because for some domains it might just not be um if the models are just so good that actually all engineering going to not be a thing in the future? Because for some domains, it might just not be. If the models are just so good that actually all they need to do is kind of like get the information from your brain and then they can go do the task. Right. That's actually a really good analogy. I mean, one common thread I'm pulling out of all your guys' responses here is that there seems to be a future in which this sort of elicitation from the user, drawing out that information is going to become much more important and much more than it is right now. in which this sort of elicitation from the user, drawing out that information, is going to become much more important, much more than it is right now. And already you guys are all starting to do it in a manual way in the future. And in the enterprise side of things, maybe that looks like an expansion off this prompt generating type of concept and things in the console where you're able to actually get more information from that enterprise customer so that they can write a better prompt. In Cloud, maybe it looks like less of just typing into a text box and more of this guided interaction towards a finished product. Yeah, I think that's actually a pretty compelling vision of future. And I think the design analogy probably really brings that home. I was thinking about how prompting now can be kind of like teaching, where it's like the empathy for the student. You're trying to think about how they think about things. You're really trying to show them, figure out where they're making a mistake. But the point that you're talking about, it's like the skill almost becomes one of introspection where you're thinking about what it is that you actually want yeah and the models trying to understand you so it's like making yourself legible to the model versus trying to teach someone who's smarter than you? This is actually how I think of prompting now in a strange way. So like, like often my style of prompting, like there's various things that I do, but a common thing that's very, like a thing that philosophers will do is I'll define new concepts. So, because my thought is like, you have to put into words what you want. And sometimes what I want is fairly like nuanced, like the, what is a good chart? Or like, usually, you know you know like I don't know like how is it that you when should you grade something as being correct or not um and so there's some cases where I will just like invent a concept and then be like here's what I mean by the concept sometimes I'll do it in collaboration with Claude to get it to like figure out what the concept is um just because i'm trying to convey to it what's in my head um and right now the models aren't like trying to do that with us unless you kind of prompt them to do so um and so in the future it might just be that they can like elicit that from us rather than us having to like kind of do it uh uh for them but i think another thing that's kind of interesting this is like a people have sometimes asked me like oh where is like philosophy relevant to prompting and i actually think it's like very useful in a sense so like a lot of there is like a style of philosophy writing and this is at least how i was taught how to write philosophy where the idea is that um in order to i think it's i think it's like an anti-bullshit device in philosophy basically which is that your papers and what you write should be legible to like a kind of educated lay person someone just like finds your paper they pick up and they start reading it and they can understand everything not everyone like you know achieves this but that's like kind of the goal of the discipline i guess or at least like uh this is at least like what we kind of like teach people um and so i'm really used to this idea of like when i'm writing thinking about the kind of educated lay person who they're really smart but they don't know anything about this topic and that was just like years and years of writing text of that form and i think it was just really good for prompting because i was like oh i'm used to this i have an educated lay person who doesn't know anything about the topic and what i need to do is I need to take extremely complex ideas and I need to make them understand it I don't talk down to them I'm not inaccurate but I need to like phrase things in such a way that it's like extremely clear to them what I mean um and prompting felt very similar and actually the training techniques we use are fascinating or like the things that you said where you're like you say to a person like just just take that thing you said and write it down i used to say that to students all the time like they'd write a paper and i was like i don't quite get what you're saying here can you just like explain your argument to me they would give me an incredibly cogent argument and then i'd be like can you just take that and write it down um and then if they did that was often like a great essay um so it's like really interesting that there's at least that similarity of just like taking things that are in your brain analyzing them enough to feel like you fully understand them and could take any person off the street who's like a reasonable person and just like externalize your brain into them and I feel like that's like the core of prompting that might be the best summary of how to prompt well I've ever heard in fact I'm pretty sure it is. Externalize your brain. And then look into them. In terms of having an education in the thing is a really good way to describe the thing. That was good. That's, I think, a great way to wrap this conversation. Thank you, guys. This was great.
AI prompt engineering: A deep dive
Some of Anthropic's prompt engineering experts—Amanda Askell (Alignment Finetuning), Alex Albert (Developer Relations), David Hershey (Applied AI), and Zack Witten (Prompt Engineering)—reflect on how prompt engineering has evolved, practical tips, and thoughts on how prompting might change as AI capabilities grow.

0:00 Introduction
2:05 Defining prompt engineering
6:34 What makes a good prompt engineer
12:17 Refining prompts
24:27 Honesty, personas and metaphors in prompts
37:12 Model reasoning
45:18 Enterprise vs research vs general chat prompts
50:52 Tips to improve prompting skills
53:56 Jailbreaking
56:51 Evolution of prompt engineering
1:04:34 Future of prompt engineering

Learn more about Anthropic: https://www.anthropic.com/
Anthropic prompt engineering docs: https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/overview