The Man behind Stable Diffusion
deep learning machine learning arxiv explained neural networks ai artificial intelligence paper stabilityai stabiliity ai stablediffusion stable diffusion eleuther ai laion laion 5b open source ai art diffusion models open source ai art
#stablediffusion #ai #stabilityai

An interview with Emad Mostaque, founder of Stability AI.

OUTLINE:
0:00 - Intro
1:30 - What is Stability AI?
3:45 - Where does the money come from?
5:20 - Is this the CERN of AI?
6:15 - Who gets access to the resources?
8:00 - What is Stable Diffusion?
11:40 - What if your model produces bad outputs?
14:20 - Do you employ people?
16:35 - Can you prevent the corruption of profit?
19:50 - How can people find you?
22:45 - Final thoughts, let's destroy PowerPoint

Links:
Homepage: https://ykilcher.com
Merch: https://ykilcher.com/merch
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://ykilcher.com/discord
LinkedIn: https://www.linkedin.com/in/ykilcher

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
 This is a mud. A mud is very rich, and he wants to put that money to good use. So just a few days ago, he presented something called stable diffusion through an initiative that he finances called stability AI stability AI is supposed to be a third pillar, there's industry, there's academia, and now there's something else. Remember when opening I started and they said they wanted to bring AI to the masses to democratize the technology and all that kind of stuff. Well, a month wants to do that, but for real. So this is an interview with a mud, he's going to tell us what he wants to achieve with stability AI, how he plans to go forward so that he's not the only one that's financing this admittedly very giant operation currently, and what you can do wherever you might be an academic person from industry, or just someone who's interested and wants to do something in the AI space and you need some compute, you need some help, you need some time, stability AI might be the place for you. If you haven't seen the outputs of stable diffusion yet, the first system coming out of this initiative, they are absolutely amazing. And not only that, the model is small and fast, it runs on a consumer GPU, and it creates pictures in about three seconds. And the model is released open source, fully up to you what to do with it. Very cool. So I don't want to stretch this intro too long, please listen to what a man has to say, I'm sure you'll be very interested. Hey, everyone, today I'm here with a mustac, who is, I have to say, I was contacted by a month through a mutual friend. And it was very intriguing. So all I know is that a month wants to tell us about exciting opportunities, essentially an alternative in research to big labs and big companies doing research, a essentially a third door, a third path of people having access to resources to do current deep learning research. And welcome, what brings you here? Hi, Yannick, I think that we're at a super exciting time in artificial intelligence, everything seems like it's about to take off. And I'm here to say, you know, let's all come together and make sure that it gets out to as many people as possible. And we'll unlock all the creativity that people have in front of them. So basically, I set up an organization called Stability AI, to remove many of the barriers for independent and academic researchers, to build some of these new models that we're seeing. Kind of in the early days of Eluthor AI and Lyon and others, we heard that compute and kind of funding were a key restriction. So everyone has basically three choices. You go into academia, you don't have compute access, and then you have to jump to big tech. And then you have 59 page MBAs, and you're working a corporate environment for product teams, or you have your own startup and running your own startup is terrible. And it's not something for most academics or researchers, although of course, some of them will hopefully be very successful doing legal AI and things like that. I thought there was going to be a better way, because this type of technology that we're seeing 80% of research dollars is going into next generation AI. And everybody has the potential to improve humanity. And so that's why with Stability AI, basically, we said, can we solve compute? Can we solve funding? And can we bring people together to build cool stuff? And we've actually achieved and managed that when we go live on the 8th of August. I don't know if this will be before or after, I think hopefully after. It all will be revealed, but I'm happy to discuss everything that we've done to date to address these and what's coming down the pipeline. So you say solve compute, solve funding essentially means money. So Stability AI, what's the source of funding or what's the money flow into this organization? And how is that money spent? So initially, it was primarily my funding. So I was lucky enough to have a good career as a hedge fund manager. Then in 2020, 2021, I led the Collective and Augmented Intelligence Against COVID-19 Initiative launch at Stanford to use the COVID-19 datasets and the backing of the WHO, UNESCO and World Bank to organize the world's COVID knowledge and make it understandable. So I've gotten lots of connections. So I pulled them together, primarily my own kind of funding. And basically, what we've done is we've built a 4,000 A100 plus stuff for open source artificial intelligence with the support of Amazon, but no control by them. So that ranks above Jool's Booster as potentially the 10th fastest public supercomputer. And Eluthor AI and Lyon have been basically building on top of that some of the most cool models that I've ever seen that are about to be released across modalities. I was about to say, kind of we've done, so we've done this as a community to date. The next stage is even more exciting. We're partnering up with countries and leading institutions to take this to the next level. Far more compute, far more funding, and most of all coordination, so that again, intelligence and creativity can be unlocked to build systems, both for countries communities and humanity that are open and not closed. Is there a comparison to maybe something that exists? Could it be compared to something like CERN or the International Space Station? What is it that you're aiming for when you say we're going for countries, we're going for collaboration? So we're already partnered with the United Nations. We're doing national level partnerships with for example, leading groups and institutions from India to Singapore to others, from universities to leading media conglomerates, telcos, the governments themselves to build national level models and data sets. So we have the plurality of kind of being around this. Kind of, this is kind of like we kicked it off as CERN, but from a discord group, probably through AI, and then it evolved into Lio and OpenBioML and a bunch of these others bring together really talented researchers. And then mine and my team's responsibility was to get them the resources they needed to unlock this. The next stage is a bit more institutional, but we really hope it keeps this kind of community vibe that we've got and this community structure that we've built. Community vibe I think is a good keyword. There are people who just come forward by themselves who want to build things who are quite clearly engaged, a lot of people in Neeluthor AI, also people from Lyon. Yet, when it I think gets more public that there is a lot of money, that there is, you know, funding, compute and so on, there is potentially going to be an influx of a lot of people with a lot of ideas and promises. How do you select who gets access to your resources and what can be done with it? So currently I am GPU Emperor. So kind of I decide which projects and things go forward. That's not sustainable. So instead, what we're doing is we're, again, without trying to kill the vibe of places like a Luther, Lyon, OpenBioML and other communities that we've got coming for audio and contrastive learning, robotics, etc. Set up processes by which grants can be given quickly for small research. And then we can really think about what the bigger runs and things like that are all about with a focus and a mission of, you know, what's cool and what's useful for humanity. Stability AI itself on the other side, you know, we are kind of commercializing these. We are a for-profit entity, but with a mission-based thing, so a benefit corporation. And that will inform some of it, but not all of it. So it's this balance of how do you have R&D and academic and independent, and then how do you productize that so it gets to a billion people. And we've got a very interesting case study that cracks next week around that. And I'll have to discuss with stable diffusion. What is stable diffusion? Stable diffusion is the last of this series of kind of diffusion models. It's the one that basically breaks through on quality, speed, and cost to enable anyone to create images. So Dali 2 was a fantastic experience. Stable diffusion is about 30 times more efficient and runs on a consumer graphics card for Dali 2 level image quality. So this was a combination of various groups such as Confiz from Heidelberg, who came up with VQGAN and latent diffusion. Our lead generative AI coder, Katherine Krausen, rivers have wings. Kind of a whole range of other kind of famous characters in the community to say, how can we build an efficient model that can scale to a billion people to enable them to be creative? And so that release is touch wood on the 8th or 9th of August. And we'll be releasing an open source along with instructions how to run it locally in the cloud and others. So what we've got is, you know, Dream, you see some Galgadaz there, right? Tesla Roadster on the streets of where are you, Yannick? Zurich, Switzerland. Streets of Zurich, right? You don't even need to dream that up. The streets here are filled with Teslas. They're filled with Teslas, right? Basically, kind of Dali 2 is, sorry my internet's a bit slow. Maybe we'll redo this demo and faster internet. Basically, this generates images in about three seconds on five gigabytes of VRAM. Whereas other image models require like 40 gigabytes or 20 gigabytes of VRAM and they're super slow. So now it's my internet that's actually slower than the actual box. So maybe we'll redo that demo in a bit. Oh, there we see it's coming. So I'm on dial-up right now, it seems. That gives me nostalgia feelings, I have to say. The line by line rendering of images. Exactly. It's pretty fun. If you're watching this and you're younger than 25, this is what the internet was like in the early days. That's an incident. So there you got your lovely Tesla in Zurich, right? But this is an image model that we built off Lyon 5B. The Lyon guys were obviously here a while ago, very close kind of working with us. Some of them are actually stability employees as well. Taking that 250 terabytes of data and we compress it down to two gigabytes kind of via this diffusion model type of thing. I mean, by the time this goes out, probably everyone will be able to play with it locally or kind of in the cloud, et cetera, because we really want to unlock this wave of innovation. Because I think that's how it happens. I don't know if Alutha's made the announcement yet, but GPT-Neo and GPT-NeoX and J have been downloaded 25 million times now by developers. That can really catalyze ecosystems for development against the more paternalistic instincts of some of the bigger AI players who refuse to release images, sorry, model the code or the weights. So like I said, stable diffusion is a very interesting one because we could have kept it closed source. It's a step forward. It's 30 times more efficient than Dali 2. You can have comparable image quality and you saw the raw output. But why would you if you can instead make it go from millions of people using this technology to billions of people using this technology? That's far more interesting. And again, I think that's the type of thing we need to do and make this technology really usable. So don't think 175 billion parameter language models or 540 billion parameter models are really usable for the vast majority of humanity. So you mentioned this open source, closed source paternalistic and so on. I agree there is a paternalistic element, but there's also a PR and a legal element, right? If Dali 2 was accessible to everyone and so on and people find, oh, I just need to enter this prompt to make it produce something that's that's really horrible. That may produce a backlash, right? Saying, well, these models are clearly not fit for release and so on. What is your sort of opinion if someone comes to you and says, your model produces horrible output here I can show you? What do you say to those people? I would say, of course, humanity is horrible and they use technology in horrible ways and good ways as well. But the reality is, for this particular output, the vast majority of people are creatively constipated. We have been conditioned to consume constantly by social media and big tech giants, and they want us to consume more according to their parameters. We see a model like this, like a three year we've had three year olds use it in refugee camps all the way to 90 year olds. You know, we're putting in mental health settings and other things. The benefits far outweigh any negativity. And the reality is that people need to get used to these models, because they're coming one way or another. And restricting them means that you become the arbiter. So as an example, we took some programmers out of Russia, because they spoke out against the government there, you know, and they came some came from the Ukraine as well. And we passed tracks their residency in the UK. You can't use the word Ukraine in Dali to, you know, because it's political. Then as well, if you type in sumo wrestler, they randomly added to the prompts, so they do pre prompt and post prompt processing, a diversity filter. So you get Asian female sumo wrestlers, because they randomly add ethnicities. There's nothing you can do about that, right? If you want to create a localized version, that you know, is more respective to your culture, for example, in India, you can't do that, because you can't access the model, right? And they don't have the capacity to let you fine tune it. So instead, what they're saying is, AI for us and our clients, because it's expensive to run these things, not for everyone else, you know, what they're really saying is we don't trust you as humanity, because we know better. I think that's wrong. You know, I actually trust people, I trust them to be weird, and nasty, in some cases, you know, 1% or 0.1% of people are weird. Many people on this call are weird, you know, I'm weird. But at the same time, like I said, I think that this is positive technology for humanity, and it should diffuse because then the pace of innovation, to make it beneficial, as well as to combat negative uses is far greater. You previously said stability AI employee. So not only do you give grants in terms of hardware and what to run, you do pay people to actually work part time or full time, can you specify a little bit of what just the what being an employee at stability AI means? Yeah, so you know, different people need different things. We come from all diverse backgrounds, some of them needed the equivalent to their jobs at Google or Microsoft when they left. So we pay competitive salaries, high bonuses. And in our contracts, no IP, all the work can be open sourced by any developer. Similarly, we have set it up. So as we run API's and our models, there's a revenue share for all developers, even if they don't work at stability who created the models. So 10% of revenue goes to this pool, half of which goes to the creators of the models and data sets and half of which goes to a communal pool, where everyone involved in stability as an employee or otherwise, which I'll come to in a second, basically awards it to the most interesting research, so that you can actually have a career from, you know, doing interesting research by open source, and it doesn't have to be commercial, you know, so the commercial is the running the API's, the non commercial is another 5% of revenue. We also do fellowships. So we're sponsoring a whole bunch of coders such as Lucid Rain, Skull Wang, through GitHub sponsors, and we ask what do you need to be comfortable? We're going to fund 100 PhDs in AI over the next year. And that comes with Compute for Academia, small and large as well. And we hope that will be a community within our communities and across communities that can coordinate global academic research. And we support as well. So for example, we have mental health support, we have grant writers, we have paper writers and other things, just to enable people to get on with what's interesting and be able to build in the open. We haven't been in the open until now because we've been building and also because it's quite fun to announce and release all this. But we hope that we can actually build in the open and change some of these incentive structures by unlocking people, be it grants, be it fellowships, be it PhD funding, be it part time jobs, full time jobs, or just being members of the community and getting prizes from this kind of pool that will hopefully become very large. We also have a charity as well, and that's where the PhD funding comes from. So charitable. What keeps you from becoming like going the same route as let's say open AI, any, all these companies from DeepMind, they have it, you know, we want to make AI for everyone. They've been for profit and very close from the beginning. Open AI actually started out with, we want to democratize, we want everyone to be accessible to give us money. And we know what's good for you, right? What keeps you like there, there's clearly a pull, right? There's clearly demands coming with any money that flows in. It's clearly attractive to sort of keep your, let's say, leading position to attract more researchers and so on. How do you prevent yourself from, let's say, succumbing to that pull of going close to or going profit? Well, I think it, you know, open AI, one of the founders is left. I won't mention on this call, maybe we can mention it privately said that kind of what we're creating is what he wanted to do when open AI was founded. It was just the wrong time. So obviously, you know, they had to scale up compute because you have this kind of stack more layers type thing. And there were all the issues that happened in 2019, the Elon Musk, etc. That basically led to a bailout and then a change in the entire corporate structure and then a change in focus to become more product ties, even though they're not actually product focused. DeepMind had a bit of a different kind of thing. But again, they were the wrong time because what you've seen is these models have lots of promise and they're powerful, but they haven't had that technological diffusion curve, right? What is the killer app? Natural language processing and kind of these large language models, they were tackling a problem I think was already 85% to 90% solved. And now we've gone to 95% solved. And they're large and bulky. Image I think is the killer app because when you look at this, it's a wonder for people that they can suddenly create rather than consume. And that's something that's across the board. You know, the comparators are Snapchat or TikTok, where you can create this Pokemon Go, you know, gacha games and these kinds of things. But it'll be integrated into so many different areas, it's got fast enough, cheap enough and good enough. And like I said, like this model file that we're releasing only a couple of gigabytes, you know, it can fit on eight gigabytes of VRAM. That's crazy. You know, like there'll be bigger models and better models like Imogen, but this inflection point is what makes our business sustainable. It allows us to do things like say, you can work just for open source to our employees, it allows us to do things like revenue share, where we'll be able to attract the best employees because if you believe this is going to a billion people, you'll have more than that. And then finally, the structure that we've employed is kind of one whereby we're partnering with various kinds of governments and leading institutions so that we build AI for each nation and communities in each nation. So we capture that cultural diversity. So again, it's very community focused, it's very oriented, there's a good business model. We've negotiated massive deals so we can be profitable at the door versus most money losing big corporations. There's a few extra things in there that I can't discuss right now. But we really kind of laid it out to be the right company at the right time to coordinate this all. And then hopefully, as this goes, this becomes an independent, more decentralized thing. Originally, we wanted to be web three with tokens and all that, but you don't need that. You know, you just need to have a good community that keeps you in check. And you need to build in the open and do things in the open, which I hope we'll manage to do over the next year. How can people find you? How can people find your models and work with your stuff? And how can people who are maybe interested in taking part in the community and contributing in some way, find you? So we have a website stability AI that will be updated when we launch publicly next week. You know, join our communities at Elutha AI or Lyon or others that we can accelerate and really, you know, put more structure around open bio mail, Harmoni for music, Carp for contrasted learning. You know, we've got education and many other things coming down the pipeline. Yeah, I think it's just community based. Be active in the community, you'll get rewarded with, you know, money and status and all sorts of other things if you do interesting stuff. You want to join stability, there are roles for exceptional programmers to come and help coordinate this. You want your PhD funded, we will announce the PhD funding program in a couple of months. You know, you want to tell us how to do this properly, open to advice, you know, like I don't think we have all the answers, but I hope we're kind of getting there and I think certainly we'll make a difference through this really flexible supercomputer cluster if nothing else. Again, it's a big, big cluster and it's available for the coolest research that can make an impact on humanity. And we'll get more, we have far bigger super compute lined up as well. So I think that's super exciting. What is the type of person that you're looking for in a contributor? And what is maybe a type of person that you're not looking for? So the type of person we're looking for a contributor are those that believe in open source AI and not open source entity, but open source innovation. You know, like we're bringing this technology to make humanity better. You can make profits, that's fine, right? But I think it should be secondary to just is this going to make a difference? You know, I don't mind if people are corporate, et cetera, but it needs to be people that integrate with the community, can work well with people from a whole bunch of different backgrounds and just are generally inquisitive that want to push the boundaries. I think some of the biggest breakthroughs we've had have been from non-traditional backgrounds. You know, I don't know if you've interviewed the Alutha AI founders, none of them have a computer science degree, you know? And yet they kind of managed to achieve such great things. Now obviously there's conjecture for alignment, and we're pushing some of the capabilities stuff there. So, you know, I think what we don't want to see is just people who are just highly corporatized, kind of stuck in one way of thinking, and want to see how to make a quick buck out of all of this. You can make money. But so what? We're at this pivotal point where this technology can maximize humanity's potential, or it can be corporatized and be used as a method of centralization and control. Which side do you want to be on? Yeah. Now you can make money on both sides. Is there anything else that you want to get out to people that you want to let people know that we haven't talked about yet? No, I mean, like I said, we've got an amazing pipeline and roadmap that we have to put out with them. So, you know, we're working everything from audio diffusion, video diffusion, 3D. I mean, I think in particular, if people want to try and create the metaverse, the Ready Player One one minus the micro transaction or holodeck, we're going to aim to do that. And I would say that probably our killer app, the one that I want to make most, and I'd invite anyone to contact me if they want to build this with me, is I want to destroy PowerPoint. I think the combination of language, image, kind of contrastive and other models means that if we work super hard in a few years, we'll never need to make a slide deck again. Tell the computer, tell it how you want to adjust it. It'll be beautiful each time. And think about how much happiness we'll bring to the world that way. No more stock images of little drawn people going like hmm. Very cool. Yeah, you know, dragging and dropping little bits on the slides and refining them. Tell the computer, it'll create the slide deck for you. Tell it how you want to adjust it, it'll adjust it. So much happiness brought to the world. I think that's another thing as well, like academia, companies, all these things. I think too many people in our community are unhappy. And obviously there's a lot of neurotypical people within our community, right? I'm neurotypical myself, you know? I want to see how we can have a happier community that supports each other, because otherwise there are these big highs and lows and things like that. And I think people focus enough on that. That's what I focus on with my engineers and what I'm trying to focus on with the community, because then people will be more productive, sure, but they'll also be more content. So it sounds a bit fuzzy, but I think it's really important and people don't pay enough attention to it. Wise words. So actually, maybe we should mention one of the projects we have, 7cups.com. It's something that we help kind of accelerate. You can go and you can chat to someone so you don't have the pressure of talking to someone online who's been trained in active listening. And we have studies showing it's as effective as taking Prozac, but then, and it's free, for $150 a month, you can talk to a qualified mental health therapist. So we've got 468,000 volunteers in 180 countries helping 80 million people each month. So I'd recommend people try that. And then if anyone wants to help me take that data set, you know, with full privacy and everything like that, to create systems that we can better listen and understand each other. Again, that's something that I'd be very interested in talking to people, because I really want to help people help people. Awesome. Imad, thank you very much for being here. Very exciting. I'm looking forward to the release next week. Maybe it's already out once this is out. Yeah, thanks a lot for being here. And good luck to the Endeavor. Thank you very much, Yannick. Pleasure. Awesome podcast you've had. I've enjoyed listening to it. Thanks for listening.
 This is a mud. A mud is very rich, and he wants to put that money to good use. So just
 a few days ago, he presented something called stable diffusion through an initiative that
 he finances called stability AI stability AI is supposed to be a third pillar, there's
 industry, there's academia, and now there's something else. Remember when opening I started
 and they said they wanted to bring AI to the masses to democratize the technology and all
 that kind of stuff. Well, a month wants to do that, but for real. So this is an interview
 with a mud, he's going to tell us what he wants to achieve with stability AI, how he
 plans to go forward so that he's not the only one that's financing this admittedly very
 giant operation currently, and what you can do wherever you might be an academic person
 from industry, or just someone who's interested and wants to do something in the AI space
 and you need some compute, you need some help, you need some time, stability AI might be
 the place for you. If you haven't seen the outputs of stable diffusion yet, the first
 system coming out of this initiative, they are absolutely amazing. And not only that,
 the model is small and fast, it runs on a consumer GPU, and it creates pictures in about
 three seconds. And the model is released open source, fully up to you what to do with it.
 Very cool. So I don't want to stretch this intro too long, please listen to what a man
 has to say, I'm sure you'll be very interested. Hey, everyone, today I'm here with a mustac,
 who is, I have to say, I was contacted by a month through a mutual friend. And it was
 very intriguing. So all I know is that a month wants to tell us about exciting opportunities,
 essentially an alternative in research to big labs and big companies doing research,
 a essentially a third door, a third path of people having access to resources to do current
 deep learning research. And welcome, what brings you here?
 Hi, Yannick, I think that we're at a super exciting time in artificial intelligence,
 everything seems like it's about to take off. And I'm here to say, you know, let's all come
 together and make sure that it gets out to as many people as possible. And we'll unlock
 all the creativity that people have in front of them. So basically, I set up an organization
 called Stability AI, to remove many of the barriers for independent and academic researchers,
 to build some of these new models that we're seeing. Kind of in the early days of Eluthor
 AI and Lyon and others, we heard that compute and kind of funding were a key restriction.
 So everyone has basically three choices. You go into academia, you don't have compute access,
 and then you have to jump to big tech. And then you have 59 page MBAs, and you're working
 a corporate environment for product teams, or you have your own startup and running your
 own startup is terrible. And it's not something for most academics or researchers, although
 of course, some of them will hopefully be very successful doing legal AI and things
 like that. I thought there was going to be a better way, because this type of technology
 that we're seeing 80% of research dollars is going into next generation AI. And everybody
 has the potential to improve humanity. And so that's why with Stability AI, basically,
 we said, can we solve compute? Can we solve funding? And can we bring people together
 to build cool stuff? And we've actually achieved and managed that when we go live on the 8th
 of August. I don't know if this will be before or after, I think hopefully after. It all
 will be revealed, but I'm happy to discuss everything that we've done to date to address
 these and what's coming down the pipeline. So you say solve compute, solve funding essentially
 means money. So Stability AI, what's the source of funding or what's the money flow into this
 organization? And how is that money spent? So initially, it was primarily my funding.
 So I was lucky enough to have a good career as a hedge fund manager. Then in 2020, 2021,
 I led the Collective and Augmented Intelligence Against COVID-19 Initiative launch at Stanford
 to use the COVID-19 datasets and the backing of the WHO, UNESCO and World Bank to organize
 the world's COVID knowledge and make it understandable. So I've gotten lots of connections. So I pulled
 them together, primarily my own kind of funding. And basically, what we've done is we've built
 a 4,000 A100 plus stuff for open source artificial intelligence with the support of Amazon, but
 no control by them. So that ranks above Jool's Booster as potentially the 10th fastest public
 supercomputer. And Eluthor AI and Lyon have been basically building on top of that some
 of the most cool models that I've ever seen that are about to be released across modalities.
 I was about to say, kind of we've done, so we've done this as a community to date. The
 next stage is even more exciting. We're partnering up with countries and leading institutions
 to take this to the next level. Far more compute, far more funding, and most of all coordination,
 so that again, intelligence and creativity can be unlocked to build systems, both for
 countries communities and humanity that are open and not closed.
 Is there a comparison to maybe something that exists? Could it be compared to something
 like CERN or the International Space Station? What is it that you're aiming for when you
 say we're going for countries, we're going for collaboration?
 So we're already partnered with the United Nations. We're doing national level partnerships
 with for example, leading groups and institutions from India to Singapore to others, from universities
 to leading media conglomerates, telcos, the governments themselves to build national level
 models and data sets. So we have the plurality of kind of being around this. Kind of, this
 is kind of like we kicked it off as CERN, but from a discord group, probably through
 AI, and then it evolved into Lio and OpenBioML and a bunch of these others bring together
 really talented researchers. And then mine and my team's responsibility was to get them
 the resources they needed to unlock this. The next stage is a bit more institutional,
 but we really hope it keeps this kind of community vibe that we've got and this community structure
 that we've built.
 Community vibe I think is a good keyword. There are people who just come forward by
 themselves who want to build things who are quite clearly engaged, a lot of people in
 Neeluthor AI, also people from Lyon. Yet, when it I think gets more public that there
 is a lot of money, that there is, you know, funding, compute and so on, there is potentially
 going to be an influx of a lot of people with a lot of ideas and promises. How do you select
 who gets access to your resources and what can be done with it?
 So currently I am GPU Emperor. So kind of I decide which projects and things go forward.
 That's not sustainable. So instead, what we're doing is we're, again, without trying to kill
 the vibe of places like a Luther, Lyon, OpenBioML and other communities that we've got coming
 for audio and contrastive learning, robotics, etc. Set up processes by which grants can
 be given quickly for small research. And then we can really think about what the bigger
 runs and things like that are all about with a focus and a mission of, you know, what's
 cool and what's useful for humanity. Stability AI itself on the other side, you know, we
 are kind of commercializing these. We are a for-profit entity, but with a mission-based
 thing, so a benefit corporation. And that will inform some of it, but not all of it.
 So it's this balance of how do you have R&D and academic and independent, and then how
 do you productize that so it gets to a billion people. And we've got a very interesting case
 study that cracks next week around that. And I'll have to discuss with stable diffusion.
 What is stable diffusion?
 Stable diffusion is the last of this series of kind of diffusion models. It's the one
 that basically breaks through on quality, speed, and cost to enable anyone to create
 images. So Dali 2 was a fantastic experience. Stable diffusion is about 30 times more efficient
 and runs on a consumer graphics card for Dali 2 level image quality. So this was a combination
 of various groups such as Confiz from Heidelberg, who came up with VQGAN and latent diffusion.
 Our lead generative AI coder, Katherine Krausen, rivers have wings. Kind of a whole range of
 other kind of famous characters in the community to say, how can we build an efficient model
 that can scale to a billion people to enable them to be creative? And so that release is
 touch wood on the 8th or 9th of August. And we'll be releasing an open source along with
 instructions how to run it locally in the cloud and others. So what we've got is, you
 know, Dream, you see some Galgadaz there, right? Tesla Roadster on the streets of where
 are you, Yannick?
 Zurich, Switzerland.
 Streets of Zurich, right?
 You don't even need to dream that up. The streets here are filled with Teslas.
 They're filled with Teslas, right? Basically, kind of Dali 2 is, sorry my internet's a bit
 slow. Maybe we'll redo this demo and faster internet. Basically, this generates images
 in about three seconds on five gigabytes of VRAM. Whereas other image models require like
 40 gigabytes or 20 gigabytes of VRAM and they're super slow. So now it's my internet that's
 actually slower than the actual box. So maybe we'll redo that demo in a bit.
 Oh, there we see it's coming. So I'm on dial-up right now, it seems.
 That gives me nostalgia feelings, I have to say. The line by line rendering of images.
 Exactly. It's pretty fun.
 If you're watching this and you're younger than 25, this is what the internet was like
 in the early days.
 That's an incident. So there you got your lovely Tesla in Zurich, right? But this is
 an image model that we built off Lyon 5B. The Lyon guys were obviously here a while
 ago, very close kind of working with us. Some of them are actually stability employees as
 well. Taking that 250 terabytes of data and we compress it down to two gigabytes kind
 of via this diffusion model type of thing. I mean, by the time this goes out, probably
 everyone will be able to play with it locally or kind of in the cloud, et cetera, because
 we really want to unlock this wave of innovation. Because I think that's how it happens. I don't
 know if Alutha's made the announcement yet, but GPT-Neo and GPT-NeoX and J have been downloaded
 25 million times now by developers. That can really catalyze ecosystems for development
 against the more paternalistic instincts of some of the bigger AI players who refuse to
 release images, sorry, model the code or the weights.
 So like I said, stable diffusion is a very interesting one because we could have kept
 it closed source. It's a step forward. It's 30 times more efficient than Dali 2. You can
 have comparable image quality and you saw the raw output. But why would you if you can
 instead make it go from millions of people using this technology to billions of people
 using this technology? That's far more interesting. And again, I think that's the type of thing
 we need to do and make this technology really usable. So don't think 175 billion parameter
 language models or 540 billion parameter models are really usable for the vast majority of
 humanity.
 So you mentioned this open source, closed source paternalistic and so on. I agree there
 is a paternalistic element, but there's also a PR and a legal element, right? If Dali 2
 was accessible to everyone and so on and people find, oh, I just need to enter this prompt
 to make it produce something that's that's really horrible. That may produce a backlash,
 right? Saying, well, these models are clearly not fit for release and so on. What is your
 sort of opinion if someone comes to you and says, your model produces horrible output
 here I can show you? What do you say to those people?
 I would say, of course, humanity is horrible and they use technology in horrible ways and
 good ways as well. But the reality is, for this particular output, the vast majority
 of people are creatively constipated. We have been conditioned to consume constantly by
 social media and big tech giants, and they want us to consume more according to their
 parameters. We see a model like this, like a three year we've had three year olds use
 it in refugee camps all the way to 90 year olds. You know, we're putting in mental health
 settings and other things. The benefits far outweigh any negativity. And the reality is
 that people need to get used to these models, because they're coming one way or another.
 And restricting them means that you become the arbiter. So as an example, we took some
 programmers out of Russia, because they spoke out against the government there, you know,
 and they came some came from the Ukraine as well. And we passed tracks their residency
 in the UK. You can't use the word Ukraine in Dali to, you know, because it's political.
 Then as well, if you type in sumo wrestler, they randomly added to the prompts, so they
 do pre prompt and post prompt processing, a diversity filter. So you get Asian female
 sumo wrestlers, because they randomly add ethnicities. There's nothing you can do about
 that, right? If you want to create a localized version, that you know, is more respective
 to your culture, for example, in India, you can't do that, because you can't access the
 model, right? And they don't have the capacity to let you fine tune it. So instead, what
 they're saying is, AI for us and our clients, because it's expensive to run these things,
 not for everyone else, you know, what they're really saying is we don't trust you as humanity,
 because we know better. I think that's wrong. You know, I actually trust people, I trust
 them to be weird, and nasty, in some cases, you know, 1% or 0.1% of people are weird.
 Many people on this call are weird, you know, I'm weird. But at the same time, like I said,
 I think that this is positive technology for humanity, and it should diffuse because then
 the pace of innovation, to make it beneficial, as well as to combat negative uses is far
 greater.
 You previously said stability AI employee. So not only do you give grants in terms of
 hardware and what to run, you do pay people to actually work part time or full time, can
 you specify a little bit of what just the what being an employee at stability AI means?
 Yeah, so you know, different people need different things. We come from all diverse backgrounds,
 some of them needed the equivalent to their jobs at Google or Microsoft when they left.
 So we pay competitive salaries, high bonuses. And in our contracts, no IP, all the work
 can be open sourced by any developer. Similarly, we have set it up. So as we run API's and
 our models, there's a revenue share for all developers, even if they don't work at stability
 who created the models. So 10% of revenue goes to this pool, half of which goes to the
 creators of the models and data sets and half of which goes to a communal pool, where everyone
 involved in stability as an employee or otherwise, which I'll come to in a second, basically
 awards it to the most interesting research, so that you can actually have a career from,
 you know, doing interesting research by open source, and it doesn't have to be commercial,
 you know, so the commercial is the running the API's, the non commercial is another 5%
 of revenue. We also do fellowships. So we're sponsoring a whole bunch of coders such as
 Lucid Rain, Skull Wang, through GitHub sponsors, and we ask what do you need to be comfortable?
 We're going to fund 100 PhDs in AI over the next year. And that comes with Compute for
 Academia, small and large as well. And we hope that will be a community within our communities
 and across communities that can coordinate global academic research. And we support as
 well. So for example, we have mental health support, we have grant writers, we have paper
 writers and other things, just to enable people to get on with what's interesting and be able
 to build in the open. We haven't been in the open until now because we've been building
 and also because it's quite fun to announce and release all this. But we hope that we
 can actually build in the open and change some of these incentive structures by unlocking
 people, be it grants, be it fellowships, be it PhD funding, be it part time jobs, full
 time jobs, or just being members of the community and getting prizes from this kind of pool
 that will hopefully become very large. We also have a charity as well, and that's where
 the PhD funding comes from. So charitable.
 What keeps you from becoming like going the same route as let's say open AI, any, all
 these companies from DeepMind, they have it, you know, we want to make AI for everyone.
 They've been for profit and very close from the beginning. Open AI actually started out
 with, we want to democratize, we want everyone to be accessible to give us money. And we
 know what's good for you, right? What keeps you like there, there's clearly a pull, right?
 There's clearly demands coming with any money that flows in. It's clearly attractive to
 sort of keep your, let's say, leading position to attract more researchers and so on. How
 do you prevent yourself from, let's say, succumbing to that pull of going close to or going profit?
 Well, I think it, you know, open AI, one of the founders is left. I won't mention on this
 call, maybe we can mention it privately said that kind of what we're creating is what he
 wanted to do when open AI was founded. It was just the wrong time. So obviously, you
 know, they had to scale up compute because you have this kind of stack more layers type
 thing. And there were all the issues that happened in 2019, the Elon Musk, etc. That
 basically led to a bailout and then a change in the entire corporate structure and then
 a change in focus to become more product ties, even though they're not actually product focused.
 DeepMind had a bit of a different kind of thing. But again, they were the wrong time
 because what you've seen is these models have lots of promise and they're powerful, but
 they haven't had that technological diffusion curve, right? What is the killer app? Natural
 language processing and kind of these large language models, they were tackling a problem
 I think was already 85% to 90% solved. And now we've gone to 95% solved. And they're
 large and bulky. Image I think is the killer app because when you look at this, it's a
 wonder for people that they can suddenly create rather than consume. And that's something
 that's across the board. You know, the comparators are Snapchat or TikTok, where you can create
 this Pokemon Go, you know, gacha games and these kinds of things. But it'll be integrated
 into so many different areas, it's got fast enough, cheap enough and good enough. And
 like I said, like this model file that we're releasing only a couple of gigabytes, you
 know, it can fit on eight gigabytes of VRAM. That's crazy. You know, like there'll be bigger
 models and better models like Imogen, but this inflection point is what makes our business
 sustainable. It allows us to do things like say, you can work just for open source to
 our employees, it allows us to do things like revenue share, where we'll be able to attract
 the best employees because if you believe this is going to a billion people, you'll
 have more than that. And then finally, the structure that we've employed is kind of one
 whereby we're partnering with various kinds of governments and leading institutions so
 that we build AI for each nation and communities in each nation. So we capture that cultural
 diversity. So again, it's very community focused, it's very oriented, there's a good business
 model. We've negotiated massive deals so we can be profitable at the door versus most
 money losing big corporations. There's a few extra things in there that I can't discuss
 right now. But we really kind of laid it out to be the right company at the right time
 to coordinate this all. And then hopefully, as this goes, this becomes an independent,
 more decentralized thing. Originally, we wanted to be web three with tokens and all that,
 but you don't need that. You know, you just need to have a good community that keeps you
 in check. And you need to build in the open and do things in the open, which I hope we'll
 manage to do over the next year.
 How can people find you? How can people find your models and work with your stuff? And
 how can people who are maybe interested in taking part in the community and contributing
 in some way, find you?
 So we have a website stability AI that will be updated when we launch publicly next week.
 You know, join our communities at Elutha AI or Lyon or others that we can accelerate
 and really, you know, put more structure around open bio mail, Harmoni for music, Carp for
 contrasted learning. You know, we've got education and many other things coming down the pipeline.
 Yeah, I think it's just community based. Be active in the community, you'll get rewarded
 with, you know, money and status and all sorts of other things if you do interesting stuff.
 You want to join stability, there are roles for exceptional programmers to come and help
 coordinate this. You want your PhD funded, we will announce the PhD funding program in
 a couple of months. You know, you want to tell us how to do this properly, open to advice,
 you know, like I don't think we have all the answers, but I hope we're kind of getting
 there and I think certainly we'll make a difference through this really flexible supercomputer
 cluster if nothing else. Again, it's a big, big cluster and it's available for the coolest
 research that can make an impact on humanity. And we'll get more, we have far bigger super
 compute lined up as well. So I think that's super exciting.
 What is the type of person that you're looking for in a contributor? And what is maybe a
 type of person that you're not looking for?
 So the type of person we're looking for a contributor are those that believe in open
 source AI and not open source entity, but open source innovation. You know, like we're
 bringing this technology to make humanity better. You can make profits, that's fine,
 right? But I think it should be secondary to just is this going to make a difference?
 You know, I don't mind if people are corporate, et cetera, but it needs to be people that
 integrate with the community, can work well with people from a whole bunch of different
 backgrounds and just are generally inquisitive that want to push the boundaries. I think
 some of the biggest breakthroughs we've had have been from non-traditional backgrounds.
 You know, I don't know if you've interviewed the Alutha AI founders, none of them have
 a computer science degree, you know? And yet they kind of managed to achieve such great
 things. Now obviously there's conjecture for alignment, and we're pushing some of the capabilities
 stuff there. So, you know, I think what we don't want to see is just people who are just
 highly corporatized, kind of stuck in one way of thinking, and want to see how to make
 a quick buck out of all of this. You can make money. But so what? We're at this pivotal
 point where this technology can maximize humanity's potential, or it can be corporatized and be
 used as a method of centralization and control. Which side do you want to be on? Yeah. Now
 you can make money on both sides.
 Is there anything else that you want to get out to people that you want to let people
 know that we haven't talked about yet?
 No, I mean, like I said, we've got an amazing pipeline and roadmap that we have to put out
 with them. So, you know, we're working everything from audio diffusion, video diffusion, 3D.
 I mean, I think in particular, if people want to try and create the metaverse, the Ready
 Player One one minus the micro transaction or holodeck, we're going to aim to do that.
 And I would say that probably our killer app, the one that I want to make most, and I'd
 invite anyone to contact me if they want to build this with me, is I want to destroy PowerPoint.
 I think the combination of language, image, kind of contrastive and other models means
 that if we work super hard in a few years, we'll never need to make a slide deck again.
 Tell the computer, tell it how you want to adjust it.
 It'll be beautiful each time.
 And think about how much happiness we'll bring to the world that way.
 No more stock images of little drawn people going like hmm.
 Very cool.
 Yeah, you know, dragging and dropping little bits on the slides and refining them.
 Tell the computer, it'll create the slide deck for you.
 Tell it how you want to adjust it, it'll adjust it.
 So much happiness brought to the world.
 I think that's another thing as well, like academia, companies, all these things.
 I think too many people in our community are unhappy.
 And obviously there's a lot of neurotypical people within our community, right?
 I'm neurotypical myself, you know?
 I want to see how we can have a happier community that supports each other, because otherwise
 there are these big highs and lows and things like that.
 And I think people focus enough on that.
 That's what I focus on with my engineers and what I'm trying to focus on with the community,
 because then people will be more productive, sure, but they'll also be more content.
 So it sounds a bit fuzzy, but I think it's really important and people don't pay enough
 attention to it.
 Wise words.
 So actually, maybe we should mention one of the projects we have, 7cups.com.
 It's something that we help kind of accelerate.
 You can go and you can chat to someone so you don't have the pressure of talking to
 someone online who's been trained in active listening.
 And we have studies showing it's as effective as taking Prozac, but then, and it's free,
 for $150 a month, you can talk to a qualified mental health therapist.
 So we've got 468,000 volunteers in 180 countries helping 80 million people each month.
 So I'd recommend people try that.
 And then if anyone wants to help me take that data set, you know, with full privacy and
 everything like that, to create systems that we can better listen and understand each other.
 Again, that's something that I'd be very interested in talking to people, because I really want
 to help people help people.
 Awesome.
 Imad, thank you very much for being here.
 Very exciting.
 I'm looking forward to the release next week.
 Maybe it's already out once this is out.
 Yeah, thanks a lot for being here.
 And good luck to the Endeavor.
 Thank you very much, Yannick.
 Pleasure.
 Awesome podcast you've had.
 I've enjoyed listening to it.
 Thanks for listening.