The Weird and Wonderful World of AI Art (w/ Author Jack Morris)
#aiart #deeplearning #clip

Since the release of CLIP, the world of AI art has seen an unprecedented level of acceleration in what's possible to do. Whereas image generation had previously been mostly in the domain of scientists, now a community of professional artists, researchers, and amateurs are sending around colab notebooks and sharing their creations via social media. How did this happen? What is going on? And where do we go from here? Jack Morris and I attempt to answer some of these questions, following his blog post "The Weird and Wonderful World of AI Art" (linked below).

OUTLINE:
0:00 - Intro
2:30 - How does one get into AI art?
5:00 - Deep Dream & Style Transfer: the early days of art in deep learning
10:50 - The advent of GANs, ArtBreeder and TikTok
19:50 - Lacking control: Pre-CLIP art
22:40 - CLIP & DALL-E
30:20 - The shift to shared colabs
34:20 - Guided diffusion models
37:20 - Prompt engineering for art models
43:30 - GLIDE
47:00 - Video production & Disco Diffusion
48:40 - Economics, money, and NFTs
54:15 - What does the future hold for AI art?

Blog post: https://jxmo.notion.site/The-Weird-and-Wonderful-World-of-AI-Art-b9615a2e7278435b98380ff81ae1cf09
Jack's Blog: https://jxmo.io/


Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://ykilcher.com/discord
BitChute: https://www.bitchute.com/channel/yannic-kilcher
LinkedIn: https://www.linkedin.com/in/ykilcher
BiliBili: https://space.bilibili.com/2017636191

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
 Hi, this is an interview with Jack Morris, who is a PhD student at Cornell in natural language processing. However, Jack has a really cool blog, and he's written a piece called The Weird and Wonderful World of AI Art, which we're going to discuss today. Now, as I said, Jack is a PhD student in NLP, but for this blog post, he dove into the world of AI art, which is sprawling currently. And we're going to talk about, you know, what happened so far, what are the origins of AI art, at least since the deep learning area, what's currently happening with all the diffusion models and clip combinations and VQ GANs and so on. And we'll also discuss a little bit where it's going in the future. This was a really cool conversation. I certainly learned a lot, and I invite you to check it out. Throughout the conversation, we have so many points to jump off of, and I'm sure you'll find something that's interesting to you. I'll leave a link to the blog post down in the description. So if you want to go and read that for yourself, I absolutely invite you to do so. As always, please leave a like if you do, let us know what you think in the comments. And thank you everyone who's sharing out these videos and helping others find my content. That's really nice. Thanks a lot. I hope you're having fun. Bye. Hi, everyone. Today, I'm here with Jack Morris, who is a PhD student at Cornell and works in a research group on NLP, but also writes about all kinds of things on his blog. Among other things, an article that I found really interesting called The Weird and Wonderful World of AI Art that is a description, a little bit of a history, a little bit of a summary and an overview, and a bit of an outlook as well over the current state of art in AI. Specifically, image generation models and beyond, which I found super fascinating. This is a topic that in recent years has picked up. There's almost an improvement every day now in this world, and it's crazy. And I thought it'd be a great opportunity to invite Jack here to talk to us about what's going on, how these different things work, and maybe also a bit why they work and what the sort of accelerators behind that is. So Jack, welcome very much to the channel. Yeah, thanks for having me. We were talking just a little bit before we started recording about this. How did you even get into this? You're a researcher in NLP, which has also seen its own revolution over the last few years. How does someone like you end up in the world of AI art, in the world of diffusion and clip and whatnot? Yeah. This is a really interesting research area because it's super new. So most of all the developments are happening online. And it's very distributed in the sense that I think a lot of the major participants aren't affiliated with big companies or universities. And so the way I kind of got involved was really just seeing the art online, specifically for me on Twitter, just seeing some of these images that are generated. This one on the screen is a pretty good example that just really challenged my beliefs of what neural networks could do. If you had shown me this a year or two ago, I probably wouldn't have believed that it was generated by a neural network. There is some really cool computer generated art, like procedural generated stuff. There are all sorts of techniques like that. But in terms of just abstract, open-ended image generation, these are just qualitatively, I think, a lot more interesting than the things that I'd seen before. And so anyways, I kind of went down this rabbit hole over this past winter of just looking at the art that a lot of artists were producing and trying to track down the techniques that they were using. It was actually pretty hard. Like there's this sort of like commodity in the form of Colab notebooks that people are sharing on Twitter. And there are a couple hubs. Like a few people are producing maybe like the most popular, the most interesting ones. And then the Colab notebooks get forked. And there's various versions of them. And they're all changing different things and using different versions of the techniques. But I think I was able to sort of identify what the most important things were and what most people were using. But it took a while. But anyways, to answer your question, I guess I just saw the art on Twitter and I thought it was really cool. Yeah, it's very interesting. And throughout the whole article, you make a point that you have maybe a hypothesis of what spurred these things. And that would be, if I represent this correctly, multimodal models, the advent of things like Dully and Clip combining different modalities together really gives an artist control over things. And this kind of brings us a step back into how things were first done initially. These pictures that you have on here, I remember fondly from my early days in deep learning, which was the sort of deep dream on the left or style transfer in the middle. This was the non plus deep dream was like that thing, right? It's like, oh, wow, like this is this is it's trippy. It's cool. And it kind of gave you an insight into what neural networks are doing. But things have come a long way, right? Can you I don't know, when you look at the history of all of these things, what what's the big arch? Well, do you want to just go through these three pictures real? Sure. Yeah. The deep dream is the thing on the left, which is I think based on the idea of finding the input that maximizes some certain like internal thing in the neural network. Like in this case, in that picture, I imagine it was something like like the dog class. And in this case, I'm really not sure what's going on. It's always the dog class, right? In ImageNet, it's like it's dog everywhere. Right. Yeah, you could you could excite like a class. You could excite some internal thing. Yeah, I remember people were very excited about this. Yeah, it's a cool idea. Like normally, at least a lot of the supervised learning people do. We we look at the gradients of the parameters with respect to the input. But deep dream is based on the gradient of the input, right? And actually, instead of changing the parameters of the model, changing changing the input to maximize something, which is which is a cool idea in and of itself. Yeah, it is. I mean, it is akin to an adversarial example in some way. Although I think this is heavily regularized because adversarial examples usually you don't necessarily see them or they give you some high frequency artifacts. And this this is very, very different. And people, you know, if we talk about art, with this already classify as art, like, you know, what's what what would an artist make of something like deep dream? Yeah, that's it. That's a philosophical question. I'm not sure I'm qualified to answer that one. But some of the some of the pieces produced with deep dream are really interesting. And they definitely fall under the realm of sort of like psychedelic, like trippy artwork. But some of them are really cool. The next thing the next iteration that you have right here are style transfer networks. Can you just briefly maybe someone hasn't heard of that? How does a how does style transfer do? How does it work on a very basic level? Yeah, yeah, it works by just exploiting the properties of convolutional neural networks to apply sort of like the texture from one image to the content of another. And so this case, the content of the image would be like the Mona Lisa. And in the middle one, that the style definitely comes from some Van Gogh starry night type of impressionist painting. Yeah. And those are really interesting, too. I think there are a bunch of apps that came out that are basically just like letting you do style transfer through an app on your phone, like input to images, and it'll copy the style from one onto the content of another. Yes. And and this was, I mean, it's still it's still it is definitely more controllable, let's say than the deep dream one. But it gives you much more predictable results. I think this is more akin to how I would describe like Photoshop or something, right? It's not really you're producing something, it's you're taking something and then you're kind of changing it, its properties a little bit, you can really imagine that in Photoshop, I'd have like a Van Gogh filter, and I just put it up and it produces something like this. Yeah, yeah. Um, well, first of all, I think that's a that's a useful distinction. This is more like an image editing technique, or at least it takes two images as an input and outputs one image. And a lot of the other things we're looking at take, take nothing as an input and output an image. Or in in the case of the stuff we'll get to take text as an input and output an image. So this is sort of like a stylistic combination of two images. And you can only do it with neural network. I think Photoshop specifically, you mentioned has this new, well, Adobe is doing all these cool things with this type of research. And the newest Photoshop's have these like neural filters, which are, which is a new feature that includes a bunch of different things you can apply to images that are based on neural networks. And I think one of the neural filters is, is using style transfer, like basically, it's built into Photoshop now, which is cool. Well, I mean, yeah, it's excellent. I would do the same if I were them, right? They, I think the Adobe suite is like insane powerhouse, like how much work went into that. So then the advent of GANs came. And I remember GANs fondly as well, because that's when I started going to conferences and every single track on every single room, and every single workshop was about GANs. Like, you could not, it is worse than Transformers today. It was just everywhere. And initially, it wasn't super duper hype, but then they got good. And here we see some, some, this person does not exist, which is a very famous website. And I think there's been everything from this shoe does not exist to this, I don't know, whatever does not exist. Well, however, again, these are these are now free form produced images, right? But they're very realistic. That is so we're at the other end of the spectrum, we are not modifying an existing image, but we producing something out of nothing. Yet, it they're very much along a data set. Yeah, so this this would be an example of one of the things that takes nothing as an input and just produces an image as the output. And that's probably like at least one of the reasons why GANs were so hyped is just because like these images are so realistic, it's it's somewhat terrifying. I've used this as an example to show my friends that aren't as like up to date in AI research and just just to scare them a little bit and show them like the kinds of things that could be done. And this is probably one of the most well known examples, I think of like, what neural networks can can actually do right now is produce these really realistic human looking images of people that I think they're sort of like, just interpolated versions of all the faces in the in the training data. But there's so many faces in the training data that it just forms like a totally new face. I don't think you could like map it back to any individual person. Yeah, and it's usually usually at the ears, you can recognize although here one is hidden, but usually kind of the ears would be would be kind of different, the left and right one enough for for you to recognize that if there's something wrong, but they are uncannily realistic, usually these GAN produced images. So this would be this would be a style GAN v2 probably. And maybe for someone who doesn't know at all how GANs work, there are two networks, one is trying to produce images, one is trying to distinguish whether or not a given image is real or fake. And these two they essentially play a game and they become better. They sort of level each other up until the the one that's generating images gets really good at confusing the other one. And in order to do that, it needs to produce realistic images. This is yeah, and GANs would make will make their appearance later on when we talk about things like VQ GAN and so on. But these were the first iterations of really realistic, realistic producing images. And you have this interesting thing here art breeder, which I was kind of aware, but there is a story behind this and tick tock. So what's that about? Oh, well, wait, can we can we stay on the GANs for a second? So it's not it's not immediately obvious, I think, why they work so well. Like there are other models that can generate random images and and some of them work well too. But GANs not only have that sort of cool explanation of being the result of two models competing with with each other. Well, we can be specific to this is if they're GAN generated, these are the outputs of the generator network of those two networks. And there are other networks that generate images, but GANs just tend to do it like really, really well. So the reason why I include them here is because they basically are the state of the art for generating realistic images. So yeah, so the on to art breeder. I think there's just a there's a famous tick tock that that showed generating faces using art breeder, which is another example of AI sort of like making its way into the mainstream with all this stuff. I included it because like you mentioned, I think the the main thesis of my article is that by training these multimodal models, we can generate art that's like specific to a level that we were never able to do before. And so starting with GANs, they start somewhere random, like they just start with this random initialization that's a vector of floating point numbers, and you have no idea what it means. So you have no idea how to like, position it in such a way that it's that it's useful. And so as an artist, you could probably do two things. One you could accept your fate, the fact that you have no control over the initialization and just sort of like, try to produce things that are that are cool, like either by brute force, just generating a lot of images or by like looking at the output of the GAN and maybe like editing it yourself, like maybe using it for inspiration or a starting point for some artwork, but actually like making changes to the artwork yourself. And the second thing you could do is maybe some kind of search like if you if you start with multiple initializations, you could examine them all and determine which one maybe has the most value to you or seems the most promising, and then do some kind of like recombination of the most interesting initializations, kind of like a binary search through the latent space of the GAN. And this is this is basically how art reader works. Instead of just generating one image and trying to edit it, or just generating a bunch of images and choosing the best one, art reader art reader has this iterative process where you generate like a few images, and you choose the one that you think is best and then generate more images based on that initial image. And you go through this process step by step in order to sort of like zero in on something that you find interesting. And this is probably better, but it's probably still not the best way to like coax interesting results out of GANs. There has been like a lot of research into making GANs more controllable. So people people trying to figure out, you know, how can you control the latent space, but we're still not there. I agree with you. It is quite hard to make these things actually to control these things and steer these things. I just want to so a few things to note right here. This is the original paper, just for people who are unaware how far we've come in this domain. The first outputs of these things, they looked they looked like, like this. So so these were faces that were totally aligned. So all the eyes are in the same place, all the noses are in the same place. And still, that was the output. Even worse, if you look at sort of the image data sets, it was it was very good at the time, but it was not, as you can see, it was there's there. These, the the progress is immense. The other thing for art breeder, I think, just also, you people may not know it's based on this idea called pick breeder. I don't actually know if this is the original site. The original site is by is by certainly Ken Stanley was part of it, where they had also these things creating pictures. And these were not neural networks. These were, I mean, they were they had a latent space, but the latent space was quite lower dimensional. And it's kind of a function, a function using trigonometric overlapping functions that produces these images, and then also pick people can sort of recombine images. So it's really cool to see that this comes to the world of neural networks, because pick breeder itself has been around for a long time. And yeah, there's, there's, you said there's a famous tick tock on on how these things are made. Yeah, there's, there's a link if you want to put up. Oh, is there? Let's check it out. There's a link to Reddit. And one tick. Once tick tock, once tick tock discovered it. Okay, so people, people making tick tock about how they art breed. I guess that's one way to go viral. So yeah, you had you had a you had you have this intermediate post here about the problem with pre clip art, and essentially, lacking control. That's the big deal, right? The artist can maybe influence stuff a little bit, but not too much, especially if they're not an expert in neural networks, they have no clue except to try it out. Yeah. And you mentioned that there's been a lot of efforts to make GANs like controllable and in some way or another. And I think that there's some success to that, like there, I know there are some interfaces where you can like generate faces and adjust, you know, the thickness of the eyebrows and the distance between the eyes and things like that. But if we just try and think about this from from first principles, I mean, if what kind of images are we trying to generate, I think the goal would be just some kind of like open ended thing where the model knows about the world and can generate pictures of whatever you want. And given that, what what would the UX look like, like in the case of faces, maybe they can design this this panel that has knobs and sliders and things where you can readjust how the face looks, but that doesn't apply to everything in the whole world. So at least one guess is just by typing stuff in, I think Texas is a really good user interface for this. You can basically be as specific as possible, but you can you can mention anything. And so we come to this idea where we have like a text box and you type in the text box, what you want to see, and the model like generates an image from that. And so everything we're going to talk about after here is some kind of like take on on that paradigm, essentially. There is Yeah, there is the paradigm of inputting text and the paradigm of actor critic, essentially an actor critic framework, where usually the way that these things work is that you'd have one model that produces stuff, which could be a GAN, but could also be other image producing models, and then a critic that judges whether it's good or not. Now, interestingly, that it's kind of the same setup as the GAN itself, right. But the critic right here is going to be clip or any sort of multimodal model where we can control what it does via text. And I find it interesting instead of instead of updating the parameters of the model like we would with the GAN. We're going back to the thing we discussed before, where we're updating the actual input itself. Yes, exactly. Yeah, it's kind of like it's sort of a deep dream GAN combination. And so I guess for that, we have to talk a little bit about clip. Now most people have probably heard of clip, but clip is essentially a model that takes a piece of text and an image and it tells you how well they go together, how well the piece of text describes the image, essentially. Now what we can do is we can simply keep the piece of text fixed and back propagate through the input in order to figure out the gradient of whatever the input currently is with respect to that text, which essentially means how do we need to change the image in order to make it more compatible to a piece of text. And we hope that if we walk that path many, many steps, then we'll arrive at an image that fits to the text very well. And the reason that we need sort of an artist in front of it, which is also interesting is because if we were to do this just starting from random pixels and then just optimize the pixels, the way neural networks work is we would probably get something quite, although I've seen some people do it directly, but we'd probably get a lot of high frequency noise and artifacts and so on. And having a GAN in front of it is almost a bit like a regularization or a constraint to make the outputs more, let's say, believable. Yeah, but I agree that's how it could work in principle. It's more an artifact of just the tools we have now is that Clip is trained to do this sort of like image caption appraisal, but it's not necessarily, it doesn't have the right parameters to generate images. And people try, but it's just not that good because of how it's trained. But we do have things that are really good at generating images, like all the various scans, and so the artist critic idea is to just sort of like couple them together. And because the whole thing is differentiable, you can use the critic to figure out how good is the art and then back propagate through the critic and through the artist back to the input itself and edit the input to maximize the output of the critic. I find it very interesting that, and obviously you go through a bit later through the initial successes of this model, Clip plus Clip plus BigGAN, for example, where we do exactly that here, for example, is a prompt that is, I don't even know, it's like a city. I don't know what the prompt was, but this picture was very famous because it kind of showed that, wow, you can actually do something. I find it interesting though, that the origin story simply came from the fact that OpenAI released this model, this blog post here about a model called Dali, which would actually do, it was trained to directly produce an image given a piece of text. There was no iterative process, no walking gradients, nothing. It was just input a piece of text and outcomes an image. It was insane. Like the blog post was insane, right? The avocado chair or here the teapot in the shape of an avocado. These are insane. Insane, yet OpenAI just didn't publish the model because I don't know, usually their go-to line is that it's too dangerous or something. Had OpenAI released this model, I think all of the things that we see in the rest of the blog post would have never happened. I'm pretty convinced. People were just stoked that we only have the clip model. We didn't have the Dali model. How can we get around this? Oh yeah, I absolutely agree. Although I feel it may have been somewhat inevitable. It's not that either Dali or clip was any major technical breakthrough, but there's a lot of engineering required and just a lot of monetary resources required to train the models. But I don't know how long it would have been before another multimodal model was released. That was equally good. But we can talk about Dali for a second. I know you said you made a video about it before. People do produce art with Dali and I think some people have a preference word. It's basically trained like a language model. Is that right? Just with text and then pixels? Yeah, essentially. So here you have a picture of Roo Dali, which is trained on the Russian language picture combinations. But yeah, people use this. I feel it is a bit more representative of maybe the data set that you put in, in that it gives a bit more realistic pictures. Yeah, and I think as an artifact of training it like a language model, Dali tends to produce like much more abstract pictures. Like it's sort of hedging between a bunch of different pictures that could satisfy the caption instead of what GANs do, which is just sort of like picking one thing and doing it as best as it can. And so it tends to be very different. I think in the glide paper, which we'll talk about later, they compare the output of this glide system to Dali and they just say like Dali tends to produce much more abstract images, I think maybe 80 or 90% of the time as rated by humans. I see. And also the shutter stock. The shutter stock watermarks are pretty cool. That's a data set thing. This is if anyone's listening to this and wants to try it out, the best open source model right now is this Roo Dali, I think, at least in best open source model that does the same thing as Dali. And they have a bit of a playground where you can try it out, right? Yeah, but it is it's trained on like Russian data. So the playground is like you import a translation model and then you type it if you're speaking English or whatever, you have to translate the prompt into Russian. So that probably makes it even more abstract. Yeah, pretty, pretty cool. There is also there are other really like true, let's say open source efforts to replicate this one is this Lyon 400 M data set, which is a data set of image text pairs, because none of these other models really release their data set. So I do believe it's not directly by a looter as you have right here. I don't know how much they are affiliated, but it is fully open source. And there's also there's there's also a project called I think Mini Dali that attempts to do Dali in less scale. And I think there are also people who are really trying to replicate this. That's pretty cool. Yeah, I linked to Mini Dali somewhere. I think they're they're scaling it up to so eventually it'll be a large Mini Dali. And here with with the advent of this with the advent of what was called the big sleep, which is this I don't even know if this isn't an illusion to to deep dream. This big come from big gan. I don't I don't know. But here we really start this advent of what you described of collab notebooks being passed around right and sort of this this art taking off really on Twitter and through Twitter and not anymore through because all the other things there they were kind of conceived in research papers and then people adapted it to things. And here we entered the realm of people doing just collabs and just kind of sharing them around right. Yeah, yeah. I think this month specifically was a really interesting time like Dali was an open source, but clip was and you can you can kind of track how the lineage of all of this through through the tweets like clip was released and there there were people that were already working on using deep learning to generate art. And some of those people did things like just the most basic thing the deep dream thing trying to optimize the picture that goes with a certain a certain caption and the results are like really like really bad looking like but they but they're they're promising like you would see sort of like outlines of things or like little words that were represented representative of the caption. And there were people like like day by day iterating on this concept. And the first thing that came out I think that was like pretty good was this notebook the big sleep and it got shared around like thousands and thousands of times on Twitter and forked a lot and stuff like that. And so I think it used big gan is that is that right again and clip began and clip. Yeah. And just that that method of like directly optimizing the input. And so now in 2022 we probably have we may would still use clip but probably would use something that works a little better than big gan. And one of these other methods for actually generating the image itself. But even just a few weeks after clip came out like you said it started this whole like craze on Twitter of people working on this. And this was like the first the first thing that really worked okay. And this so this is by people wonder this is by Ryan Murdoch who was one of one of certainly the defining people in the early days of of this clip plus X models. Also interesting here is the style clip. I didn't I didn't even know. Oh yeah I think I think I saw this somewhere but so people would try to use take a style gan and combine it with clip and off just off the nature big gan was sort of trained on image net and larger data sets to produce various different like a variety of images while the style gans would always be kind of constrained to single data sets. So it's natural to see that you cannot get the style gans to to do as crazy things but it's still pretty crazy what you can get them to do simply by mucking around essentially with their latent spaces. Yeah that's that's a really good point. That was something that I wanted to mention was some people have this theory that one of the reasons why we have this open ended generation tool that we didn't have before is because the new models were trained on just like all this data from the web that's just from all over like a much more rich diverse data set instead of just you know the 1000 classes from image net. Yeah I mean it it is reasonable. It's probably a combination of data set the models and technique but certainly the data place places and scale and scale obviously. Yeah so then a new after after the GANs a new contender let's say got released which people I remember were pretty fond of which was the guided diffusion clip guided diffusion and the pictures of that were also very impressive. So what was what is the difference between a GAN and a diffusion model as an artist? Well they both do kind of the same the same thing in the end which is that they they produce realistic images given a caption but it really was important because these this class of models called diffusion models just kind of upset GANs and the race for highest you know image generation fidelity and that that was just coincidentally by other people at Open AI during last year but these these became like the most powerful powerful models that we had for generating images but I I might have conflated two things in the in the caption for this section. Yeah these are just diffusion models no. Yeah these are just diffusion models and then the process of generating images from a caption one of the ways to do it with diffusion models is what people call like guided diffusion and you'll find all sorts of colab notebooks floating around that are helping you generate images using guided diffusion. And so just diffusion models they do work by they themselves are an iterative process of producing an image so they are usually trained by taking real images and applying noise over and over and over again so in a stepwise fashion you destroy the image and then you train a neural network to revert each one of those steps so to make a little less noisy image from a more noisy image and through some proper through some asymptotic properties you can essentially show that after after destroying an image with so much noise it is a defined distribution and from that you can calculate some bounds and then essentially you can revert the whole process using that trained neural network. And so we're layering iterative processes on top of iterative processes if we're doing clip guided diffusion but it's fun. And it makes for a very entertaining image generation. It's very satisfying kind of watching the thing emerge from a blur of noise over some time but also it's a problem because it makes the process take a very long time. And people yeah people I guess quickly figured out is that you can just wait for a long time and your quality will get better and better to the point where it could take hours to produce an image like this. Yeah and you get diminishing returns so it's hard to determine where to stop especially if it's the artistic process you know that we're talking about. So in GPT-3 it was pretty quickly clear that there is something like prompt engineering or even prompt hacking that by prompting the model in a certain way you could get certain very defined results and people have caught on to this thing in these models as well interestingly with something that's called the Unreal Engine trick. Do you want to elaborate what this was? Yeah yeah this is one of my favorite parts of the whole thing and relates back to what my research group works on and all the NLP stuff that people are talking about right now. I added this section mostly because of just this whole idea of prompt engineering like really applies to the art generation. In this case there was a buzz online where people were showing that if you type in in this case maybe the angel of air which I should have done for the blog post it might generate something like somewhat interesting but maybe not that specific or realistic but if you add if you append Unreal Engine to the prompt it'll like there's a lot of there's a lot of training data that's generated by this Unreal Engine thing and includes that in the caption so Clip is smart enough to know what Unreal Engine looks like and if you add that into the prompt it tends to generate images that that look way better and I don't know this is a specific style so maybe it's not for everyone but just the idea of like asking the model for what you want like if you if you type in a prompt and generate an image but you think it's too blurry like type not blurry or yeah or that was the most insane thing is like oh yeah just type not blurry it's like what yeah and it works or just people just type like beautiful yeah and it tends to just make the art look better and we've we've sort of stacked on this like people right now they they like write you know pipe and then they write I don't even I don't even know like these art sites VFX and scene on art station and things like this and you have the example here of you just append hashtag pixel art and it will give you pixel art yeah if I'm trying to generate anything realistic I usually put HD 4k at the end just just because and yeah so there you have a bunch of these things right here these go more back into the the style transfer type of thing like we give it a certain style but I think it's important to note that it really goes as far as just typing like not blurry and then you get something that's not blurry which is is crazy but also these right here the like German expressionism yeah this specific post is really cool this person just went through a few dozen artists and generated kind of like a bunch like the same images use the same prompts but appended the names of different artists to the prompt and they they look totally different I did something like this myself that I was tweeting about which was just typing in names of national parks and then generating them but images of them in an impressionist style and it also worked worked really well and it's a good way to kind of showcase what clip can do because it's yeah this is the same that we saw at the beginning right here right this is this is Kowloon City in the style of Wes Anderson mm-hmm yeah that's that's the thing that excites me the most about all of this is the integration of like world knowledge into the image generation process like to generate this image the model has to know what Kowloon City looks like and at least sort of the style of a Wes Anderson film and this is obviously like nothing that you can that you can find online there's another one that's oh yeah this this one on the right here can you click on that one it's just cookies made out of kimchi I don't know if you could ever actually cook them to look like this but this is probably the best one I have in terms of just showing off like the use of real world knowledge and the image generation process these are really awesome and the the prompt was can you imagine how cool it'd be to have some delicious kimchi cookies right now question mark it's also really interesting right that you prompt you really prompt by by using language now not it's not just keywords it's actual language yeah that's something I'm trying to improve upon as well like I if I were trying to do this I probably would have just typed in kimchi cookies and that doesn't always tend to give you the best outputs and yeah I mean it's it's interesting and I think this as I said this is the first time where probably research lags behind the the art production in this case I think it will be very interesting to pick all of this up and sort of explain all of these phenomena like why do certain things work better why does it work better if we you know have a whole story about can you imagine and stuff rather than keywords super interesting can we mention this one person that's up here Katherine Krausen yes her Twitter at rivers have wings she's if you had to pinpoint one person that's kind of the nexus of this whole movement it's it's probably her she's she's done so much the data set that I mentioned she helped lead people to collect that she trains all these different models that are that are useful she helped come up with this new metric that helps guide the art generation process to be better she's wrapped almost everything up in a colab notebook and released all these colab notebooks that are useful for people and I guess she she was the first person to combine like diffusion models with clip guidance which is why I referenced her here but she's done all sorts of really really awesome stuff yes this is definitely a known name in the in the community then you mentioned this glide model right here what what makes this different from what came before they directly trained a model to generate images instead of like using only clip and a and a model that was separately trained to generate images and they just scaled it up pretty pretty far and and generated some pretty cool stuff I think that the paper didn't do anything new necessarily they also did they used a lot of different techniques from Twitter but that but they cited them all they actually cited tweets in their paper which I've never seen before it's very cool it's a weird world yeah yeah and maybe a colab notebook or maybe they said it a tweet to a colab notebook can't remember which and these examples are are from the glide model so it's it's basically just trained to optimize the same thing that we're talking about already which is like the glide model does both the role of the artist and the critic at the same time and yeah you can you can given that it's a diffusion model you can do a lot of different things from it such as conditional generation only generate parts of the image and so on so that was that's also very very neat property of these diffusion models only changing yeah or only like changing the particular parts of the room all right so the top right one is is so so so the green mask is the area that's actually allowed to be optimized I think this this task is called like image inpainting it's kind of just like post text guided post hoc image editing and is it possible for you to like zoom in on the top right image so the the mask is is over the dog so the optimization process is only editing the pixels that are within that green mask and this is a famous painting that has like a king charles spaniel and then they just type the girl hugging a corgi on the pedestal and then optimized it until the glide model thought that the painting matched that caption as best as possible and it pretty much just like realistically substituted the the spaniel for the corgi which is so awesome and I guarantee you this will make its way into photoshop yes I just thought yeah I just thought of saying this like this is gonna be can you imagine just having this just painting a bit of a mask typing in a piece of text and then uh outcomes what you want this is going to I think yeah I think it's it's going to revolutionize uh maybe not art itself but certainly the way we interact with with pictures as such crazy at least clip art generation it would be nice every time you make a set of slides to just generate some unique little art pieces for your slides yes um so we've we've reached the conclusion of your article right here but the story is not over as we said uh things are coming out almost every day and one of the interesting things that has come out in the last I think weeks or months uh is this transition also into video content and specifically there is this um there is this technique called disco diffusion do you know that yeah what is that disco diffusion is is well it's actually the name of a of a colab notebook so maybe if you type disco diffusion colab oh I actually have a link to it at the bottom of my article I think okay okay but there there are different people trying to use these techniques to generate videos um I think the most common well probably the most common so disco isn't video itself disco but you can then make a video of it or yeah disco diffusion is is just the name of a of a colab notebook that generates images from prompts but it includes I in some versions tools for kind of like interpolating through the latent space from one prompt to another and so the the video is like taking I think a linear path from the image produced the latent space representation of the image for one prompt to the latent representation of an image for another prompt and it it tends to produce like these crazy videos but it's totally continuous because you're taking like a like a continuous path through the latent space so very very cool insane yeah this is a bit how I I don't know if you've seen this but I've made this music video and I did kind of the same thing and but obviously much more primitive these things are these things are crazy in how good they are there are a number of twitter accounts that people can follow and I think you link a lot of them in at the end of your article and you also link a lot of the of the notebooks of the colabs that do this now also in the recent times I've observed at the beginning I've observed I could find most of the colabs people would just kind of post them on twitter then there was some colabs where it was like you know you have to be like my my patreon in order to get the newest colab which I I thought it was what you know that's obviously cool because there's a lot of work going into them but recently I found is it people want to sell nfts of their stuff and that's why they don't give out the colabs anymore or what's happened like I've had a lot of trouble finding stuff recently yeah I'm not sure about the connection between that the nft generation and colab but that is a big source of the excitement for this kind of thing I kind of stayed away from that for my article I think I might have one example of an art piece that I thought was particularly compelling that was minted as an nft but there there are various collections that are kind of like this where it's like you just you click the mint button and a new piece of art is created and it's an nft and it uses these techniques behind the scenes and I think Katherine Krausen has her own line of nfts if I were someone who purchased nfts I would probably buy one of hers it's just it's just but it's just weird or is this a wrong impression of me that the colabs have become harder that people aren't sharing as much anymore oh definitely and everyone seems to have their own post-processing steps I haven't really talked about that but most of the stuff that I share is directly generated through the clip guided diffusion process or something like it but a lot of like the really good especially really high definition art has all sorts of steps besides just the art generation like they might up sample or upscale it using another GAN or use another GAN that takes art and produces new art that's supposed to be better than the first art that it saw and plus all sorts of regular you know photo post-processing like changing the saturation or editing all the different things you might edit so just a note to myself editing later that we were gonna have to censor this one just just saying there are body parts in that one that are not okay for YouTube good call I probably would have would have found you for that yeah sorry sorry I interrupt oh yeah so so people have their own kind of like personal stacks for art generation usually starting with some kind of art artist critic thing that outputs an image but then they do all sorts of stuff to adapt or and people can be pretty hesitant to share I think their personal art generation processes yeah it's it's interesting because at the beginning you could really feel it was more like a community together tries to figure out what's the best thing to produce art and now that it kind of is and it's almost an established field right it's more about it's more about you know I have my little secret thing and I can produce very cool things and I don't want anyone else to be able to do that and it's interesting do you do you also we talked about there being and I've pulled this up right here this was the first AI generated portrait ever sold at an auction it was sold by she's the giant amount of money is this a thing still like are these things you said there's like an NFT collection is this a big market AI generated art well our art is very subjective and I think a lot of the times a lot of the value comes from who created the art and I think in this case it was like a pretty well-known group of artists that generated art with computers and they made a piece that was generated with AI I'm not sure if maybe your concrete question was something like has anyone sold a physical painting like this that's been generated with clip and I haven't heard of that happening I think that part of that might be because it's just so accessible and easy to generate this type of art right now it kind of cheapens it in as a commodity and I don't know I'd be interested to see like what are the most valuable pieces of artwork that have been generated with clip we could probably look that up in terms of NFTs but it might not correlate that well with you know artistic value what where do you see this going in the in the future like right now I can type in yeah a bit of piece of text and so on are the future artists more gonna be computer scientists that figure out better post-processing and so on or how can this really help I feel it I feel that this is still not enough controllability for an artist to type in a piece of text and see what comes out I feel that the artists they still don't really actually think that they're in control of what's happening or that this is just a tool where do you see this going in the future especially in terms of in terms of you know how it interacts with art and artists yeah it's a really exciting time and you know it's impossible to predict the future I feel like we can definitely agree that something very important exists now that did not exist before it's hard to say like what kinds of innovations that will directly lead to I agree that the prompting process is pretty cumbersome I mean the images are are too slow to generate and you can you can type something in the prompt and you won't always see it in the output which is which is a big problem I think that the people that that share art on Twitter generally have some sort of process that resembles the art breeder thing we looked at where that would be something like you type in a prompt and then instead of just generating one output you generate four or sixty four and then you pick the one that's most interesting to you and work with that either like generating things that are similar to it or just upscaling it and and choosing like higher resolution versions that you like better I think I'm Katherine Kraus and has shared some like art exploration she does where she generates like this maybe 32 by 32 matrix of images that all that all fit a prompt and I think that's really really compelling to just to show how how cheap that this makes the art generation process like she'll type something in and and they'll all look you know pretty decent which is which is crazy so so I think people definitely not just be typing something in and producing a single piece of artwork I can probably guarantee that yeah but maybe the the mechanical aspect of producing art sort of the the going and and modifying the either pixels or or yeah brush strokes themselves or maybe a little bit more receding and maybe the sort of coming up interacting with these models in some way or selecting things that one likes or maybe a bit more in the foreground in the future yeah yeah absolutely and maybe it'll make art more more accessible to people like there there's kind of two skills maybe you could break art down into one being actually mechanically creating it and the other being like appraising it and deciding whether it's good or not that's kind of just like the the artist critic paradigm but maybe this would enable people to create art that have a good eye for things but didn't have you know the dexterity or whatever paintbrush skills they needed to create the art that they wanted to beforehand that's an exciting possibility cool anything else you oh wait here is Elon Musk experiencing pain we gotta look at this ah ah that's terrible anything else you you want to get you want to get anything else you'd like people to know about this stuff um well I think some of the examples that I shared were generated with the large glide model which is not open source yet and that is kind of a shame I think it'll I'm sure they have good reasons for not sharing it but hopefully within the year or so there will be an equally large equally capable model because glide is significant because it the I think that the generations from glide will be less abstract than the ones we see now um which will be good if you just want to type I don't know so if you want to visualize something that doesn't exist that the model could create for you like in these outputs that that's kind of like a separate thing that's closer to what I was saying about clipart generation but um that just the ones that are out right now just don't don't work particularly well and you could still get abstract stuff by typing abstract stuff like here like a dream like oil painting yeah that's a good um yeah but I think the rest of this stuff is open source so if anyone pulls up my blog post after watching this I encourage you to just scroll down to the colab part and open one of them up and try try running it it's free yeah and there's a there's a lot of there's a lot of references and links to all kinds of stuff here so I definitely invite people to check out the the blog post again it's called the weird and wonderful world of AI art and I'll certainly link to it in the description of this video all right Jack Morris thank you very much for being with us and explaining this to us yeah thanks for having me cool
 Hi, this is an interview with Jack Morris, who is a PhD student at Cornell in natural
 language processing.
 However, Jack has a really cool blog, and he's written a piece called The Weird and
 Wonderful World of AI Art, which we're going to discuss today.
 Now, as I said, Jack is a PhD student in NLP, but for this blog post, he dove into the world
 of AI art, which is sprawling currently.
 And we're going to talk about, you know, what happened so far, what are the origins of AI
 art, at least since the deep learning area, what's currently happening with all the diffusion
 models and clip combinations and VQ GANs and so on.
 And we'll also discuss a little bit where it's going in the future.
 This was a really cool conversation.
 I certainly learned a lot, and I invite you to check it out.
 Throughout the conversation, we have so many points to jump off of, and I'm sure you'll
 find something that's interesting to you.
 I'll leave a link to the blog post down in the description.
 So if you want to go and read that for yourself, I absolutely invite you to do so.
 As always, please leave a like if you do, let us know what you think in the comments.
 And thank you everyone who's sharing out these videos and helping others find my content.
 That's really nice.
 Thanks a lot.
 I hope you're having fun.
 Bye.
 Hi, everyone.
 Today, I'm here with Jack Morris, who is a PhD student at Cornell and works in a research
 group on NLP, but also writes about all kinds of things on his blog.
 Among other things, an article that I found really interesting called The Weird and Wonderful
 World of AI Art that is a description, a little bit of a history, a little bit of a summary
 and an overview, and a bit of an outlook as well over the current state of art in AI.
 Specifically, image generation models and beyond, which I found super fascinating.
 This is a topic that in recent years has picked up.
 There's almost an improvement every day now in this world, and it's crazy.
 And I thought it'd be a great opportunity to invite Jack here to talk to us about what's
 going on, how these different things work, and maybe also a bit why they work and what
 the sort of accelerators behind that is.
 So Jack, welcome very much to the channel.
 Yeah, thanks for having me.
 We were talking just a little bit before we started recording about this.
 How did you even get into this?
 You're a researcher in NLP, which has also seen its own revolution over the last few
 years.
 How does someone like you end up in the world of AI art, in the world of diffusion and clip
 and whatnot?
 Yeah.
 This is a really interesting research area because it's super new.
 So most of all the developments are happening online.
 And it's very distributed in the sense that I think a lot of the major participants aren't
 affiliated with big companies or universities.
 And so the way I kind of got involved was really just seeing the art online, specifically
 for me on Twitter, just seeing some of these images that are generated.
 This one on the screen is a pretty good example that just really challenged my beliefs of
 what neural networks could do.
 If you had shown me this a year or two ago, I probably wouldn't have believed that it
 was generated by a neural network.
 There is some really cool computer generated art, like procedural generated stuff.
 There are all sorts of techniques like that.
 But in terms of just abstract, open-ended image generation, these are just qualitatively,
 I think, a lot more interesting than the things that I'd seen before.
 And so anyways, I kind of went down this rabbit hole over this past winter of just looking
 at the art that a lot of artists were producing and trying to track down the techniques that
 they were using.
 It was actually pretty hard.
 Like there's this sort of like commodity in the form of Colab notebooks that people are
 sharing on Twitter.
 And there are a couple hubs.
 Like a few people are producing maybe like the most popular, the most interesting ones.
 And then the Colab notebooks get forked.
 And there's various versions of them.
 And they're all changing different things and using different versions of the techniques.
 But I think I was able to sort of identify what the most important things were and what
 most people were using.
 But it took a while.
 But anyways, to answer your question, I guess I just saw the art on Twitter and I thought
 it was really cool.
 Yeah, it's very interesting.
 And throughout the whole article, you make a point that you have maybe a hypothesis of
 what spurred these things.
 And that would be, if I represent this correctly, multimodal models, the advent of things like
 Dully and Clip combining different modalities together really gives an artist control over
 things.
 And this kind of brings us a step back into how things were first done initially.
 These pictures that you have on here, I remember fondly from my early days in deep learning,
 which was the sort of deep dream on the left or style transfer in the middle.
 This was the non plus deep dream was like that thing, right?
 It's like, oh, wow, like this is this is it's trippy.
 It's cool.
 And it kind of gave you an insight into what neural networks are doing.
 But things have come a long way, right?
 Can you I don't know, when you look at the history of all of these things, what what's
 the big arch?
 Well, do you want to just go through these three pictures real?
 Sure.
 Yeah.
 The deep dream is the thing on the left, which is I think based on the idea of finding the
 input that maximizes some certain like internal thing in the neural network.
 Like in this case, in that picture, I imagine it was something like like the dog class.
 And in this case, I'm really not sure what's going on.
 It's always the dog class, right?
 In ImageNet, it's like it's dog everywhere.
 Right.
 Yeah, you could you could excite like a class.
 You could excite some internal thing.
 Yeah, I remember people were very excited about this.
 Yeah, it's a cool idea.
 Like normally, at least a lot of the supervised learning people do.
 We we look at the gradients of the parameters with respect to the input.
 But deep dream is based on the gradient of the input, right?
 And actually, instead of changing the parameters of the model, changing changing the input
 to maximize something, which is which is a cool idea in and of itself.
 Yeah, it is.
 I mean, it is akin to an adversarial example in some way.
 Although I think this is heavily regularized because adversarial examples usually you don't
 necessarily see them or they give you some high frequency artifacts.
 And this this is very, very different.
 And people, you know, if we talk about art, with this already classify as art, like, you
 know, what's what what would an artist make of something like deep dream?
 Yeah, that's it.
 That's a philosophical question.
 I'm not sure I'm qualified to answer that one.
 But some of the some of the pieces produced with deep dream are really interesting.
 And they definitely fall under the realm of sort of like psychedelic, like trippy artwork.
 But some of them are really cool.
 The next thing the next iteration that you have right here are style transfer networks.
 Can you just briefly maybe someone hasn't heard of that?
 How does a how does style transfer do?
 How does it work on a very basic level?
 Yeah, yeah, it works by just exploiting the properties of convolutional neural networks
 to apply sort of like the texture from one image to the content of another.
 And so this case, the content of the image would be like the Mona Lisa.
 And in the middle one, that the style definitely comes from some Van Gogh starry night type
 of impressionist painting.
 Yeah.
 And those are really interesting, too.
 I think there are a bunch of apps that came out that are basically just like letting you
 do style transfer through an app on your phone, like input to images, and it'll copy the style
 from one onto the content of another.
 Yes.
 And and this was, I mean, it's still it's still it is definitely more controllable, let's
 say than the deep dream one.
 But it gives you much more predictable results.
 I think this is more akin to how I would describe like Photoshop or something, right?
 It's not really you're producing something, it's you're taking something and then you're
 kind of changing it, its properties a little bit, you can really imagine that in Photoshop,
 I'd have like a Van Gogh filter, and I just put it up and it produces something like this.
 Yeah, yeah.
 Um, well, first of all, I think that's a that's a useful distinction.
 This is more like an image editing technique, or at least it takes two images as an input
 and outputs one image.
 And a lot of the other things we're looking at take, take nothing as an input and output
 an image.
 Or in in the case of the stuff we'll get to take text as an input and output an image.
 So this is sort of like a stylistic combination of two images.
 And you can only do it with neural network.
 I think Photoshop specifically, you mentioned has this new, well, Adobe is doing all these
 cool things with this type of research.
 And the newest Photoshop's have these like neural filters, which are, which is a new
 feature that includes a bunch of different things you can apply to images that are based
 on neural networks.
 And I think one of the neural filters is, is using style transfer, like basically, it's
 built into Photoshop now, which is cool.
 Well, I mean, yeah, it's excellent.
 I would do the same if I were them, right?
 They, I think the Adobe suite is like insane powerhouse, like how much work went into that.
 So then the advent of GANs came.
 And I remember GANs fondly as well, because that's when I started going to conferences
 and every single track on every single room, and every single workshop was about GANs.
 Like, you could not, it is worse than Transformers today.
 It was just everywhere.
 And initially, it wasn't super duper hype, but then they got good.
 And here we see some, some, this person does not exist, which is a very famous website.
 And I think there's been everything from this shoe does not exist to this, I don't know,
 whatever does not exist.
 Well, however, again, these are these are now free form produced images, right?
 But they're very realistic.
 That is so we're at the other end of the spectrum, we are not modifying an existing image, but
 we producing something out of nothing.
 Yet, it they're very much along a data set.
 Yeah, so this this would be an example of one of the things that takes nothing as an
 input and just produces an image as the output.
 And that's probably like at least one of the reasons why GANs were so hyped is just because
 like these images are so realistic, it's it's somewhat terrifying.
 I've used this as an example to show my friends that aren't as like up to date in AI research
 and just just to scare them a little bit and show them like the kinds of things that could
 be done.
 And this is probably one of the most well known examples, I think of like, what neural
 networks can can actually do right now is produce these really realistic human looking
 images of people that I think they're sort of like, just interpolated versions of all
 the faces in the in the training data.
 But there's so many faces in the training data that it just forms like a totally new
 face.
 I don't think you could like map it back to any individual person.
 Yeah, and it's usually usually at the ears, you can recognize although here one is hidden,
 but usually kind of the ears would be would be kind of different, the left and right one
 enough for for you to recognize that if there's something wrong, but they are uncannily realistic,
 usually these GAN produced images.
 So this would be this would be a style GAN v2 probably.
 And maybe for someone who doesn't know at all how GANs work, there are two networks,
 one is trying to produce images, one is trying to distinguish whether or not a given image
 is real or fake.
 And these two they essentially play a game and they become better.
 They sort of level each other up until the the one that's generating images gets really
 good at confusing the other one.
 And in order to do that, it needs to produce realistic images.
 This is yeah, and GANs would make will make their appearance later on when we talk about
 things like VQ GAN and so on.
 But these were the first iterations of really realistic, realistic producing images.
 And you have this interesting thing here art breeder, which I was kind of aware, but there
 is a story behind this and tick tock.
 So what's that about?
 Oh, well, wait, can we can we stay on the GANs for a second?
 So it's not it's not immediately obvious, I think, why they work so well.
 Like there are other models that can generate random images and and some of them work well
 too.
 But GANs not only have that sort of cool explanation of being the result of two models competing
 with with each other.
 Well, we can be specific to this is if they're GAN generated, these are the outputs of the
 generator network of those two networks.
 And there are other networks that generate images, but GANs just tend to do it like really,
 really well.
 So the reason why I include them here is because they basically are the state of the art for
 generating realistic images.
 So yeah, so the on to art breeder.
 I think there's just a there's a famous tick tock that that showed generating faces using
 art breeder, which is another example of AI sort of like making its way into the mainstream
 with all this stuff.
 I included it because like you mentioned, I think the the main thesis of my article
 is that by training these multimodal models, we can generate art that's like specific to
 a level that we were never able to do before.
 And so starting with GANs, they start somewhere random, like they just start with this random
 initialization that's a vector of floating point numbers, and you have no idea what it
 means.
 So you have no idea how to like, position it in such a way that it's that it's useful.
 And so as an artist, you could probably do two things.
 One you could accept your fate, the fact that you have no control over the initialization
 and just sort of like, try to produce things that are that are cool, like either by brute
 force, just generating a lot of images or by like looking at the output of the GAN and
 maybe like editing it yourself, like maybe using it for inspiration or a starting point
 for some artwork, but actually like making changes to the artwork yourself.
 And the second thing you could do is maybe some kind of search like if you if you start
 with multiple initializations, you could examine them all and determine which one maybe has
 the most value to you or seems the most promising, and then do some kind of like recombination
 of the most interesting initializations, kind of like a binary search through the latent
 space of the GAN.
 And this is this is basically how art reader works.
 Instead of just generating one image and trying to edit it, or just generating a bunch of
 images and choosing the best one, art reader art reader has this iterative process where
 you generate like a few images, and you choose the one that you think is best and then generate
 more images based on that initial image.
 And you go through this process step by step in order to sort of like zero in on something
 that you find interesting.
 And this is probably better, but it's probably still not the best way to like coax interesting
 results out of GANs.
 There has been like a lot of research into making GANs more controllable.
 So people people trying to figure out, you know, how can you control the latent space,
 but we're still not there.
 I agree with you.
 It is quite hard to make these things actually to control these things and steer these things.
 I just want to so a few things to note right here.
 This is the original paper, just for people who are unaware how far we've come in this
 domain.
 The first outputs of these things, they looked they looked like, like this.
 So so these were faces that were totally aligned.
 So all the eyes are in the same place, all the noses are in the same place.
 And still, that was the output.
 Even worse, if you look at sort of the image data sets, it was it was very good at the
 time, but it was not, as you can see, it was there's there.
 These, the the progress is immense.
 The other thing for art breeder, I think, just also, you people may not know it's based
 on this idea called pick breeder.
 I don't actually know if this is the original site.
 The original site is by is by certainly Ken Stanley was part of it, where they had also
 these things creating pictures.
 And these were not neural networks.
 These were, I mean, they were they had a latent space, but the latent space was quite lower
 dimensional.
 And it's kind of a function, a function using trigonometric overlapping functions that produces
 these images, and then also pick people can sort of recombine images.
 So it's really cool to see that this comes to the world of neural networks, because pick
 breeder itself has been around for a long time.
 And yeah, there's, there's, you said there's a famous tick tock on on how these things
 are made.
 Yeah, there's, there's a link if you want to put up.
 Oh, is there?
 Let's check it out.
 There's a link to Reddit.
 And one tick.
 Once tick tock, once tick tock discovered it.
 Okay, so people, people making tick tock about how they art breed.
 I guess that's one way to go viral.
 So yeah, you had you had a you had you have this intermediate post here about the problem
 with pre clip art, and essentially, lacking control.
 That's the big deal, right?
 The artist can maybe influence stuff a little bit, but not too much, especially if they're
 not an expert in neural networks, they have no clue except to try it out.
 Yeah.
 And you mentioned that there's been a lot of efforts to make GANs like controllable
 and in some way or another.
 And I think that there's some success to that, like there, I know there are some interfaces
 where you can like generate faces and adjust, you know, the thickness of the eyebrows and
 the distance between the eyes and things like that.
 But if we just try and think about this from from first principles, I mean, if what kind
 of images are we trying to generate, I think the goal would be just some kind of like open
 ended thing where the model knows about the world and can generate pictures of whatever
 you want.
 And given that, what what would the UX look like, like in the case of faces, maybe they
 can design this this panel that has knobs and sliders and things where you can readjust
 how the face looks, but that doesn't apply to everything in the whole world.
 So at least one guess is just by typing stuff in, I think Texas is a really good user interface
 for this.
 You can basically be as specific as possible, but you can you can mention anything.
 And so we come to this idea where we have like a text box and you type in the text box,
 what you want to see, and the model like generates an image from that.
 And so everything we're going to talk about after here is some kind of like take on on
 that paradigm, essentially.
 There is Yeah, there is the paradigm of inputting text and the paradigm of actor critic, essentially
 an actor critic framework, where usually the way that these things work is that you'd have
 one model that produces stuff, which could be a GAN, but could also be other image producing
 models, and then a critic that judges whether it's good or not.
 Now, interestingly, that it's kind of the same setup as the GAN itself, right.
 But the critic right here is going to be clip or any sort of multimodal model where we can
 control what it does via text.
 And I find it interesting instead of instead of updating the parameters of the model like
 we would with the GAN.
 We're going back to the thing we discussed before, where we're updating the actual input
 itself.
 Yes, exactly.
 Yeah, it's kind of like it's sort of a deep dream GAN combination.
 And so I guess for that, we have to talk a little bit about clip.
 Now most people have probably heard of clip, but clip is essentially a model that takes
 a piece of text and an image and it tells you how well they go together, how well the
 piece of text describes the image, essentially.
 Now what we can do is we can simply keep the piece of text fixed and back propagate through
 the input in order to figure out the gradient of whatever the input currently is with respect
 to that text, which essentially means how do we need to change the image in order to
 make it more compatible to a piece of text.
 And we hope that if we walk that path many, many steps, then we'll arrive at an image
 that fits to the text very well.
 And the reason that we need sort of an artist in front of it, which is also interesting
 is because if we were to do this just starting from random pixels and then just optimize
 the pixels, the way neural networks work is we would probably get something quite, although
 I've seen some people do it directly, but we'd probably get a lot of high frequency
 noise and artifacts and so on.
 And having a GAN in front of it is almost a bit like a regularization or a constraint
 to make the outputs more, let's say, believable.
 Yeah, but I agree that's how it could work in principle.
 It's more an artifact of just the tools we have now is that Clip is trained to do this
 sort of like image caption appraisal, but it's not necessarily, it doesn't have the
 right parameters to generate images.
 And people try, but it's just not that good because of how it's trained.
 But we do have things that are really good at generating images, like all the various
 scans, and so the artist critic idea is to just sort of like couple them together.
 And because the whole thing is differentiable, you can use the critic to figure out how good
 is the art and then back propagate through the critic and through the artist back to
 the input itself and edit the input to maximize the output of the critic.
 I find it very interesting that, and obviously you go through a bit later through the initial
 successes of this model, Clip plus Clip plus BigGAN, for example, where we do exactly that
 here, for example, is a prompt that is, I don't even know, it's like a city.
 I don't know what the prompt was, but this picture was very famous because it kind of
 showed that, wow, you can actually do something.
 I find it interesting though, that the origin story simply came from the fact that OpenAI
 released this model, this blog post here about a model called Dali, which would actually
 do, it was trained to directly produce an image given a piece of text.
 There was no iterative process, no walking gradients, nothing.
 It was just input a piece of text and outcomes an image.
 It was insane.
 Like the blog post was insane, right?
 The avocado chair or here the teapot in the shape of an avocado.
 These are insane.
 Insane, yet OpenAI just didn't publish the model because I don't know, usually their
 go-to line is that it's too dangerous or something.
 Had OpenAI released this model, I think all of the things that we see in the rest of the
 blog post would have never happened.
 I'm pretty convinced.
 People were just stoked that we only have the clip model.
 We didn't have the Dali model.
 How can we get around this?
 Oh yeah, I absolutely agree.
 Although I feel it may have been somewhat inevitable.
 It's not that either Dali or clip was any major technical breakthrough, but there's
 a lot of engineering required and just a lot of monetary resources required to train the
 models.
 But I don't know how long it would have been before another multimodal model was released.
 That was equally good.
 But we can talk about Dali for a second.
 I know you said you made a video about it before.
 People do produce art with Dali and I think some people have a preference word.
 It's basically trained like a language model.
 Is that right?
 Just with text and then pixels?
 Yeah, essentially.
 So here you have a picture of Roo Dali, which is trained on the Russian language picture
 combinations.
 But yeah, people use this.
 I feel it is a bit more representative of maybe the data set that you put in, in that
 it gives a bit more realistic pictures.
 Yeah, and I think as an artifact of training it like a language model, Dali tends to produce
 like much more abstract pictures.
 Like it's sort of hedging between a bunch of different pictures that could satisfy the
 caption instead of what GANs do, which is just sort of like picking one thing and doing
 it as best as it can.
 And so it tends to be very different.
 I think in the glide paper, which we'll talk about later, they compare the output of this
 glide system to Dali and they just say like Dali tends to produce much more abstract images,
 I think maybe 80 or 90% of the time as rated by humans.
 I see.
 And also the shutter stock.
 The shutter stock watermarks are pretty cool.
 That's a data set thing.
 This is if anyone's listening to this and wants to try it out, the best open source
 model right now is this Roo Dali, I think, at least in best open source model that does
 the same thing as Dali.
 And they have a bit of a playground where you can try it out, right?
 Yeah, but it is it's trained on like Russian data.
 So the playground is like you import a translation model and then you type it if you're speaking
 English or whatever, you have to translate the prompt into Russian.
 So that probably makes it even more abstract.
 Yeah, pretty, pretty cool.
 There is also there are other really like true, let's say open source efforts to replicate
 this one is this Lyon 400 M data set, which is a data set of image text pairs, because
 none of these other models really release their data set.
 So I do believe it's not directly by a looter as you have right here.
 I don't know how much they are affiliated, but it is fully open source.
 And there's also there's there's also a project called I think Mini Dali that attempts to
 do Dali in less scale.
 And I think there are also people who are really trying to replicate this.
 That's pretty cool.
 Yeah, I linked to Mini Dali somewhere.
 I think they're they're scaling it up to so eventually it'll be a large Mini Dali.
 And here with with the advent of this with the advent of what was called the big sleep,
 which is this I don't even know if this isn't an illusion to to deep dream.
 This big come from big gan.
 I don't I don't know.
 But here we really start this advent of what you described of collab notebooks being passed
 around right and sort of this this art taking off really on Twitter and through Twitter
 and not anymore through because all the other things there they were kind of conceived in
 research papers and then people adapted it to things.
 And here we entered the realm of people doing just collabs and just kind of sharing them
 around right.
 Yeah, yeah.
 I think this month specifically was a really interesting time like Dali was an open source,
 but clip was and you can you can kind of track how the lineage of all of this through through
 the tweets like clip was released and there there were people that were already working
 on using deep learning to generate art.
 And some of those people did things like just the most basic thing the deep dream thing
 trying to optimize the picture that goes with a certain a certain caption and the results
 are like really like really bad looking like but they but they're they're promising like
 you would see sort of like outlines of things or like little words that were represented
 representative of the caption.
 And there were people like like day by day iterating on this concept.
 And the first thing that came out I think that was like pretty good was this notebook
 the big sleep and it got shared around like thousands and thousands of times on Twitter
 and forked a lot and stuff like that.
 And so I think it used big gan is that is that right again and clip began and clip.
 Yeah.
 And just that that method of like directly optimizing the input.
 And so now in 2022 we probably have we may would still use clip but probably would use
 something that works a little better than big gan.
 And one of these other methods for actually generating the image itself.
 But even just a few weeks after clip came out like you said it started this whole like
 craze on Twitter of people working on this.
 And this was like the first the first thing that really worked okay.
 And this so this is by people wonder this is by Ryan Murdoch who was one of one of certainly
 the defining people in the early days of of this clip plus X models.
 Also interesting here is the style clip.
 I didn't I didn't even know.
 Oh yeah I think I think I saw this somewhere but so people would try to use take a style
 gan and combine it with clip and off just off the nature big gan was sort of trained
 on image net and larger data sets to produce various different like a variety of images
 while the style gans would always be kind of constrained to single data sets.
 So it's natural to see that you cannot get the style gans to to do as crazy things but
 it's still pretty crazy what you can get them to do simply by mucking around essentially
 with their latent spaces.
 Yeah that's that's a really good point.
 That was something that I wanted to mention was some people have this theory that one
 of the reasons why we have this open ended generation tool that we didn't have before
 is because the new models were trained on just like all this data from the web that's
 just from all over like a much more rich diverse data set instead of just you know the 1000
 classes from image net.
 Yeah I mean it it is reasonable.
 It's probably a combination of data set the models and technique but certainly the data
 place places and scale and scale obviously.
 Yeah so then a new after after the GANs a new contender let's say got released which
 people I remember were pretty fond of which was the guided diffusion clip guided diffusion
 and the pictures of that were also very impressive.
 So what was what is the difference between a GAN and a diffusion model as an artist?
 Well they both do kind of the same the same thing in the end which is that they they produce
 realistic images given a caption but it really was important because these this class of
 models called diffusion models just kind of upset GANs and the race for highest you know
 image generation fidelity and that that was just coincidentally by other people at Open
 AI during last year but these these became like the most powerful powerful models that
 we had for generating images but I I might have conflated two things in the in the caption
 for this section.
 Yeah these are just diffusion models no.
 Yeah these are just diffusion models and then the process of generating images from a caption
 one of the ways to do it with diffusion models is what people call like guided diffusion
 and you'll find all sorts of colab notebooks floating around that are helping you generate
 images using guided diffusion.
 And so just diffusion models they do work by they themselves are an iterative process
 of producing an image so they are usually trained by taking real images and applying
 noise over and over and over again so in a stepwise fashion you destroy the image and
 then you train a neural network to revert each one of those steps so to make a little
 less noisy image from a more noisy image and through some proper through some asymptotic
 properties you can essentially show that after after destroying an image with so much noise
 it is a defined distribution and from that you can calculate some bounds and then essentially
 you can revert the whole process using that trained neural network.
 And so we're layering iterative processes on top of iterative processes if we're doing
 clip guided diffusion but it's fun.
 And it makes for a very entertaining image generation.
 It's very satisfying kind of watching the thing emerge from a blur of noise over some
 time but also it's a problem because it makes the process take a very long time.
 And people yeah people I guess quickly figured out is that you can just wait for a long time
 and your quality will get better and better to the point where it could take hours to
 produce an image like this.
 Yeah and you get diminishing returns so it's hard to determine where to stop especially
 if it's the artistic process you know that we're talking about.
 So in GPT-3 it was pretty quickly clear that there is something like prompt engineering
 or even prompt hacking that by prompting the model in a certain way you could get certain
 very defined results and people have caught on to this thing in these models as well interestingly
 with something that's called the Unreal Engine trick.
 Do you want to elaborate what this was?
 Yeah yeah this is one of my favorite parts of the whole thing and relates back to what
 my research group works on and all the NLP stuff that people are talking about right
 now.
 I added this section mostly because of just this whole idea of prompt engineering like
 really applies to the art generation.
 In this case there was a buzz online where people were showing that if you type in in
 this case maybe the angel of air which I should have done for the blog post it might generate
 something like somewhat interesting but maybe not that specific or realistic but if you
 add if you append Unreal Engine to the prompt it'll like there's a lot of there's a lot
 of training data that's generated by this Unreal Engine thing and includes that in the
 caption so Clip is smart enough to know what Unreal Engine looks like and if you add that
 into the prompt it tends to generate images that that look way better and I don't know
 this is a specific style so maybe it's not for everyone but just the idea of like asking
 the model for what you want like if you if you type in a prompt and generate an image
 but you think it's too blurry like type not blurry or yeah or that was the most insane
 thing is like oh yeah just type not blurry it's like what yeah and it works or just people
 just type like beautiful yeah and it tends to just make the art look better and we've
 we've sort of stacked on this like people right now they they like write you know pipe
 and then they write I don't even I don't even know like these art sites VFX and scene on
 art station and things like this and you have the example here of you just append hashtag
 pixel art and it will give you pixel art yeah if I'm trying to generate anything realistic
 I usually put HD 4k at the end just just because and yeah so there you have a bunch of these
 things right here these go more back into the the style transfer type of thing like
 we give it a certain style but I think it's important to note that it really goes as far
 as just typing like not blurry and then you get something that's not blurry which is is
 crazy but also these right here the like German expressionism yeah this specific post is really
 cool this person just went through a few dozen artists and generated kind of like a bunch
 like the same images use the same prompts but appended the names of different artists
 to the prompt and they they look totally different I did something like this myself that I was
 tweeting about which was just typing in names of national parks and then generating them
 but images of them in an impressionist style and it also worked worked really well and
 it's a good way to kind of showcase what clip can do because it's yeah this is the same
 that we saw at the beginning right here right this is this is Kowloon City in the style
 of Wes Anderson mm-hmm yeah that's that's the thing that excites me the most about all
 of this is the integration of like world knowledge into the image generation process like to
 generate this image the model has to know what Kowloon City looks like and at least
 sort of the style of a Wes Anderson film and this is obviously like nothing that you can
 that you can find online there's another one that's oh yeah this this one on the right
 here can you click on that one it's just cookies made out of kimchi I don't know if you could
 ever actually cook them to look like this but this is probably the best one I have in
 terms of just showing off like the use of real world knowledge and the image generation
 process these are really awesome and the the prompt was can you imagine how cool it'd be
 to have some delicious kimchi cookies right now question mark it's also really interesting
 right that you prompt you really prompt by by using language now not it's not just keywords
 it's actual language yeah that's something I'm trying to improve upon as well like I
 if I were trying to do this I probably would have just typed in kimchi cookies and that
 doesn't always tend to give you the best outputs and yeah I mean it's it's interesting and
 I think this as I said this is the first time where probably research lags behind the the
 art production in this case I think it will be very interesting to pick all of this up
 and sort of explain all of these phenomena like why do certain things work better why
 does it work better if we you know have a whole story about can you imagine and stuff
 rather than keywords super interesting can we mention this one person that's up here
 Katherine Krausen yes her Twitter at rivers have wings she's if you had to pinpoint one
 person that's kind of the nexus of this whole movement it's it's probably her she's she's
 done so much the data set that I mentioned she helped lead people to collect that she
 trains all these different models that are that are useful she helped come up with this
 new metric that helps guide the art generation process to be better she's wrapped almost
 everything up in a colab notebook and released all these colab notebooks that are useful
 for people and I guess she she was the first person to combine like diffusion models with
 clip guidance which is why I referenced her here but she's done all sorts of really really
 awesome stuff yes this is definitely a known name in the in the community then you mentioned
 this glide model right here what what makes this different from what came before they
 directly trained a model to generate images instead of like using only clip and a and
 a model that was separately trained to generate images and they just scaled it up pretty pretty
 far and and generated some pretty cool stuff I think that the paper didn't do anything
 new necessarily they also did they used a lot of different techniques from Twitter but
 that but they cited them all they actually cited tweets in their paper which I've never
 seen before it's very cool it's a weird world yeah yeah and maybe a colab notebook or maybe
 they said it a tweet to a colab notebook can't remember which and these examples are are
 from the glide model so it's it's basically just trained to optimize the same thing that
 we're talking about already which is like the glide model does both the role of the
 artist and the critic at the same time and yeah you can you can given that it's a diffusion
 model you can do a lot of different things from it such as conditional generation only
 generate parts of the image and so on so that was that's also very very neat property of
 these diffusion models only changing yeah or only like changing the particular parts
 of the room all right so the top right one is is so so so the green mask is the area
 that's actually allowed to be optimized I think this this task is called like image
 inpainting it's kind of just like post text guided post hoc image editing and is it possible
 for you to like zoom in on the top right image so the the mask is is over the dog so the
 optimization process is only editing the pixels that are within that green mask and this is
 a famous painting that has like a king charles spaniel and then they just type the girl hugging
 a corgi on the pedestal and then optimized it until the glide model thought that the
 painting matched that caption as best as possible and it pretty much just like realistically
 substituted the the spaniel for the corgi which is so awesome and I guarantee you this
 will make its way into photoshop yes I just thought yeah I just thought of saying this
 like this is gonna be can you imagine just having this just painting a bit of a mask
 typing in a piece of text and then uh outcomes what you want this is going to I think yeah
 I think it's it's going to revolutionize uh maybe not art itself but certainly the way
 we interact with with pictures as such crazy at least clip art generation it would be nice
 every time you make a set of slides to just generate some unique little art pieces for
 your slides yes um so we've we've reached the conclusion of your article right here
 but the story is not over as we said uh things are coming out almost every day and one of
 the interesting things that has come out in the last I think weeks or months uh is this
 transition also into video content and specifically there is this um there is this technique called
 disco diffusion do you know that yeah what is that disco diffusion is is well it's actually
 the name of a of a colab notebook so maybe if you type disco diffusion colab oh I actually
 have a link to it at the bottom of my article I think okay okay but there there are different
 people trying to use these techniques to generate videos um I think the most common well probably
 the most common so disco isn't video itself disco but you can then make a video of it
 or yeah disco diffusion is is just the name of a of a colab notebook that generates images
 from prompts but it includes I in some versions tools for kind of like interpolating through
 the latent space from one prompt to another and so the the video is like taking I think
 a linear path from the image produced the latent space representation of the image for
 one prompt to the latent representation of an image for another prompt and it it tends
 to produce like these crazy videos but it's totally continuous because you're taking like
 a like a continuous path through the latent space so very very cool insane yeah this is
 a bit how I I don't know if you've seen this but I've made this music video and I did kind
 of the same thing and but obviously much more primitive these things are these things are
 crazy in how good they are there are a number of twitter accounts that people can follow
 and I think you link a lot of them in at the end of your article and you also link a lot
 of the of the notebooks of the colabs that do this now also in the recent times I've
 observed at the beginning I've observed I could find most of the colabs people would
 just kind of post them on twitter then there was some colabs where it was like you know
 you have to be like my my patreon in order to get the newest colab which I I thought
 it was what you know that's obviously cool because there's a lot of work going into them
 but recently I found is it people want to sell nfts of their stuff and that's why they
 don't give out the colabs anymore or what's happened like I've had a lot of trouble finding
 stuff recently yeah I'm not sure about the connection between that the nft generation
 and colab but that is a big source of the excitement for this kind of thing I kind of
 stayed away from that for my article I think I might have one example of an art piece that
 I thought was particularly compelling that was minted as an nft but there there are various
 collections that are kind of like this where it's like you just you click the mint button
 and a new piece of art is created and it's an nft and it uses these techniques behind
 the scenes and I think Katherine Krausen has her own line of nfts if I were someone who
 purchased nfts I would probably buy one of hers it's just it's just but it's just weird
 or is this a wrong impression of me that the colabs have become harder that people aren't
 sharing as much anymore oh definitely and everyone seems to have their own post-processing
 steps I haven't really talked about that but most of the stuff that I share is directly
 generated through the clip guided diffusion process or something like it but a lot of
 like the really good especially really high definition art has all sorts of steps besides
 just the art generation like they might up sample or upscale it using another GAN or
 use another GAN that takes art and produces new art that's supposed to be better than
 the first art that it saw and plus all sorts of regular you know photo post-processing
 like changing the saturation or editing all the different things you might edit so just
 a note to myself editing later that we were gonna have to censor this one just just saying
 there are body parts in that one that are not okay for YouTube good call I probably
 would have would have found you for that yeah sorry sorry I interrupt oh yeah so so people
 have their own kind of like personal stacks for art generation usually starting with some
 kind of art artist critic thing that outputs an image but then they do all sorts of stuff
 to adapt or and people can be pretty hesitant to share I think their personal art generation
 processes yeah it's it's interesting because at the beginning you could really feel it
 was more like a community together tries to figure out what's the best thing to produce
 art and now that it kind of is and it's almost an established field right it's more about
 it's more about you know I have my little secret thing and I can produce very cool things
 and I don't want anyone else to be able to do that and it's interesting do you do you
 also we talked about there being and I've pulled this up right here this was the first
 AI generated portrait ever sold at an auction it was sold by she's the giant amount of money
 is this a thing still like are these things you said there's like an NFT collection is
 this a big market AI generated art well our art is very subjective and I think a lot of
 the times a lot of the value comes from who created the art and I think in this case it
 was like a pretty well-known group of artists that generated art with computers and they
 made a piece that was generated with AI I'm not sure if maybe your concrete question was
 something like has anyone sold a physical painting like this that's been generated with
 clip and I haven't heard of that happening I think that part of that might be because
 it's just so accessible and easy to generate this type of art right now it kind of cheapens
 it in as a commodity and I don't know I'd be interested to see like what are the most
 valuable pieces of artwork that have been generated with clip we could probably look
 that up in terms of NFTs but it might not correlate that well with you know artistic
 value what where do you see this going in the in
 the future like right now I can type in yeah a bit of piece of text and so on are the future
 artists more gonna be computer scientists that figure out better post-processing and
 so on or how can this really help I feel it I feel that this is still not enough controllability
 for an artist to type in a piece of text and see what comes out I feel that the artists
 they still don't really actually think that they're in control of what's happening or
 that this is just a tool where do you see this going in the future especially in terms
 of in terms of you know how it interacts with art and artists yeah it's a really exciting
 time and you know it's impossible to predict the future I feel like we can definitely agree
 that something very important exists now that did not exist before it's hard to say like
 what kinds of innovations that will directly lead to I agree that the prompting process
 is pretty cumbersome I mean the images are are too slow to generate and you can you can
 type something in the prompt and you won't always see it in the output which is which
 is a big problem I think that the people that that share art on Twitter generally have some
 sort of process that resembles the art breeder thing we looked at where that would be something
 like you type in a prompt and then instead of just generating one output you generate
 four or sixty four and then you pick the one that's most interesting to you and work with
 that either like generating things that are similar to it or just upscaling it and and
 choosing like higher resolution versions that you like better I think I'm Katherine Kraus
 and has shared some like art exploration she does where she generates like this maybe 32
 by 32 matrix of images that all that all fit a prompt and I think that's really really
 compelling to just to show how how cheap that this makes the art generation process like
 she'll type something in and and they'll all look you know pretty decent which is which
 is crazy so so I think people definitely not just be typing something in and producing
 a single piece of artwork I can probably guarantee that yeah but maybe the the mechanical aspect
 of producing art sort of the the going and and modifying the either pixels or or yeah
 brush strokes themselves or maybe a little bit more receding and maybe the sort of coming
 up interacting with these models in some way or selecting things that one likes or maybe
 a bit more in the foreground in the future yeah yeah absolutely and maybe it'll make
 art more more accessible to people like there there's kind of two skills maybe you could
 break art down into one being actually mechanically creating it and the other being like appraising
 it and deciding whether it's good or not that's kind of just like the the artist critic paradigm
 but maybe this would enable people to create art that have a good eye for things but didn't
 have you know the dexterity or whatever paintbrush skills they needed to create the art that
 they wanted to beforehand that's an exciting possibility cool anything else you oh wait
 here is Elon Musk experiencing pain we gotta look at this ah ah that's terrible anything
 else you you want to get you want to get anything else you'd like people to know about this
 stuff um well I think some of the examples that I shared were generated with the large
 glide model which is not open source yet and that is kind of a shame I think it'll I'm
 sure they have good reasons for not sharing it but hopefully within the year or so there
 will be an equally large equally capable model because glide is significant because it the
 I think that the generations from glide will be less abstract than the ones we see now
 um which will be good if you just want to type I don't know so if you want to visualize
 something that doesn't exist that the model could create for you like in these outputs
 that that's kind of like a separate thing that's closer to what I was saying about clipart
 generation but um that just the ones that are out right now just don't don't work particularly
 well and you could still get abstract stuff by typing abstract stuff like here like a
 dream like oil painting yeah that's a good um yeah but I think the rest of this stuff
 is open source so if anyone pulls up my blog post after watching this I encourage you to
 just scroll down to the colab part and open one of them up and try try running it it's
 free yeah and there's a there's a lot of there's a lot of references and links to all kinds
 of stuff here so I definitely invite people to check out the the blog post again it's
 called the weird and wonderful world of AI art and I'll certainly link to it in the
 description of this video all right Jack Morris thank you very much for being with us and
 explaining this to us yeah thanks for having me cool