 All right, so hi, my name is Mejdi. I'm a first year master's student at Mila doing part of the computer science program. My research focuses around machine learning, like more specifically generative modeling and geometric deep learning around like ligand protein binding, molecular dynamics, sampling, equilibrium states of molecules and all that kind of stuff. But prior to coming to Mila, I worked in industry in a company called Advi for four years, doing machine learning on chemistry, more specifically like property prediction and for Admi, and then a little bit of generative modeling. So I'm the point of this talk. I've been had the absolute honor to be invited by Nucleate to give this talk on an intro to AI or machine learning in drug discovery. And I'm excited to give it. And I would like to thank all the people who have helped me refine my slides, first and foremost. But let's get right into it. I'm going to be sharing my slides. Hopefully, everything's all good. All right. So this is an intro to AI and drug discovery. This is a talk I'm getting very clear. Hopefully everything's all good. All right. So this is an intro to AI and drug discovery. This is a talk I'm getting very clear. So let's jump right into it. So I'll be starting off. The frame of the talk will be, I'll give a brief background on drug discovery, like the entire pipeline and how ML might be able to help out in certain areas and then i'll dive into a background in two popular areas of ml for molecules and then briefly talk about open problems and challenges and i'll kind of conclude the entire talk. All right, so the chemical space, it is massive, it is huge. The number of molecules and number of compounds that can be created or possibly created is I believe 10 to the 60 compounds. And so I think I'd like to think about why it's called drug discovery is because there's this massive pool, this massive space of compounds that we can discover or sample from rather than, you know, I think some people call it drug design, but I think the more common term is like drug discovery right and so we're discovering these compounds that might become potential candidates to be effective drugs towards certain kind certain kinds of diseases and uh the compound to drug uh journey is usually follows this pipeline. So the drug pipeline starts off with this first component, which is drug discovery. And there's about four components into this. And so it starts off with target discovery, which and then identification, hit to lead, and lead optimization. We will dissect a little bit each of these categories in the drug discovery pipeline. But usually if we have a compound that has made it through these four stages and we have a drug candidate at the end of the lead optimization phase, this will then progress forward to another set of trials called the clinical trials, which is broken up to three phases. We won't be really talking about clinical trials in this talk. This is sort of out of scope for this talk. We're mostly focusing on drug discovery. But you just have to think of the clinical trials. We go from essentially wet labs and virtual labs and all this kind of stuff to testing on uh bigger and bigger batches of people trying to see if this drug uh will not have any harmful effects if it does the thing that it was designed to do and documenting all this very very rigorously and if it we have a drug that actually makes it all the way through all this, it gets submitted for FDA review and possibly approved to be sold in the market. And so, yeah, it's a long, arduous journey. I think it's quoted to be around 12 to 15 years for this, for the drug discovery pipeline, the drug discovery, sorry, like the whole drug pipeline. And really for the clinical trials and FDA review and approval, you can't really speed up that process. That process is like, it will take its time. It needs to take its time. There's like a set strict rules for it. But the area where we can save time and accelerate is this drug discovery process. If we can find drug candidates much faster thanks to computational tools and faster or higher throughput screens and we'll talk all about this in the talk we can we can hopefully find drugs much faster to to cure certain diseases and I believe so we're going to kind of go down and we're going to dissect each of these categories, but I think it's helpful to think of this whole pipeline as essentially being a funnel where in the early stages, you might have a lot of candidates. And as we go through each and every stage, you're filtering out ill-suited candidates and only keeping the ones that meet certain criteria. So just keep that image of a funnel in mind while we are talking about each of these categories. So as I said before, so drug discovery starts with a target discovery. Target discovery is essentially saying that what is the target that I need to be targeting, right? Whether it's a protein, let's say for simplicity, we'll refer to a protein, a protein that we need to target that would have some downstream effect on some disease, right? some downstream effect on some disease, right? So trying to find that protein that is like a long and painstakingly like heavy research process. But let's say if you found a protein that you know has some relation to a disease, is associated with a disease, we move on to the identification phase where here we are trying to essentially throw a bunch of molecules onto the target and seeing which one sticks. We're just trying to filter from that huge chemical space that we know of, trying to reduce the size of it to see just passable candidates. And once we have some passable candidates, we go further on to reduce that volume to find potential leads. So this goes from the hit to lead phase. And then from the hit to lead phase, let's say I think this goes from about 1,000 to 10,000 candidates down to like about a hundred candidates. And then the hit to lead phase will filter out those hundred candidates down to about like 10, like a dozen or so candidates, which then we go through a lead optimization phase where a program is started and that revolves around these molecules trying to optimize their properties. And we're gonna actually even go even deeper further to each of these categories. And then this is the funnel that I was talking about earlier. Just keep this in mind. We're going from thousands and thousands of compounds in the hit identification phase to hundreds of compounds in hit-to-lead to about a dozen in lead optimization. And out of the lead optimization, maybe one or three candidates will come out of it. They'll go through the clinical trials. Okay, so starting off with target discovery, I've mentioned how, so part of target discovery kind of boils down to two things two things first is identifying uh what is the relevant entity to disease of interest this doesn't necessarily have to be protein but for the sake of the rest of this talk we'll be referring to proteins but finding these targets and their associations with the disease uh requires a lot, a lot of research. There's a lot of data mining, like data mining literature and trying to study like omic databases, other reports from public research, experiments of various kinds. You're trying to put this all together in order to determine that if I target this protein, I will have some, hopefully, downstream effect, a likely downstream effect on this disease. That's what we're looking for. But the other criteria to the target is that it needs to be druggable. And what do I mean by druggable? So this is kind of like a sort of inaccurate analogy, but I think it kind of gets the point across. If you think of a lock and key mechanism, and if you think of the lock, what's important for a protein, for example, is that it has these binding pockets, as you see here on the the screen where this compound is sort of resting these pockets where compounds could nestle themselves in if you think of this pocket as like a lock and you think of the compound itself as a key you're trying to design or discover a key that best fits this lock in order to do the certain thing that you want to do, for example, open the door. But let's say that, but how can you design a key to open a door if that door doesn't even have a lock, right? So essentially what I'm saying is that if there is no suitable pockets for a compound to bind itself to the protein, then it is unable to drug the protein. It's unable to elicit a response or be bioactive against that protein. So these are the kind of two, I think, more important criterions essentially is that, is it associated with a disease of interest and can we drug it? And there's been a a lot of there's been a growing field of ml methods and even computational methods um around this two ones recently came out i think in the last year append and grasp um but for example pinned i think they just calculate a lot of uh different chemical physical chemical and uh features of the protein. They pass it in through some neural networks, and this neural network will output a score of how ligninable or how druggable this protein is. And GRASP does the same thing, but it uses things called graph attention networks. And I think there's a growing interest in this field. And you can also have ML methods for the other use case of disease target associations, where you apply ML on knowledge graphs, right? You have, you've accumulated all this literature, you form this graph of relations, and then you try to extract new relations, new disease target associations, using either graph methods or ML on graphs, right? So now we found our target. We are at a great like a initial start. We're at the start of the race, right? And so we're going to move on to hit identification. Now, what does it mean to be a hit? If you want to think of the target as being like a like a like a dartboard. And if you want to think of uh compounds as being the darts themselves and if i grab a handful of darts and i throw all of them on the dartboard uh the ones that stick to the dartboard it doesn't matter where it sticks it could be dead bullseye or it could be at the very edge of the dartboard. As long as it hits the board and stays there, these are hits. Essentially, initial compounds that have expressed some form of activity towards a target. And normally, the way these are done are, I think, mainly two methods. You can either do high-throughput screens, which are these large automated wet lab experiments and Essentially what they do is they they Essentially find they throw a bunch of compounds at a target and a solution and see which ones stick and then you can do this via virtual screens for use computational tools instead and you can either lightly dock these molecules against the protein see if it docks predict these properties using ml methods like predict certain bioactivities and or generate or use generative models to generate hits, right? So generate these molecules that might be potential candidates. So we've identified about a thousand compounds that might be possible hits to a protein of choice. And we move on to refining that candidate pool even further and like filtering it down. So we move on to a hit to lead phase where now we have a defined threshold of activity that we would like our compounds to be. Maybe like we'll take the top 100 compounds that have the best activity, but also we want to balance out other profiles. And so bioactivity is essentially how active the compound is towards the protein target. There's other things like ADME properties and we'll have a slide after this that will kind of explain what ADME properties are and selectivity which is essentially is my compound being selective towards this protein target? Is it I want to make sure you know I don't want this compound just binding to any other target because that might have some harmful downstream effects. Am I selectively targeting this protein? And so, and you have to think about like the, this whole process is like a spectrum essentially where some compounds are going to be active, but some compounds are going to be even more active and have better profile, like admin profiles than other compounds. So we're trying to get the best of the best out of this candidate pool. Think of it as if your favorite sports for tryouts. If there's a lot of people, they're going to have multiple rounds of tryouts in order to filter out, to filter for the best candidates or the best possible candidates for the team, right? And we're doing the very same thing for compounds. Example methods of where ML can be involved is like protein ligand docking. An example method is like DIPDOK. There are several other methods now since then that have also do protein ligand docking. Predicting bioactivity to a docked ligand and ADME property prediction. So this is kind of like a main line of work that I did during my work at ADME, where essentially we're trying to accurately predict these different ADME properties for molecules. And so what is ADME? I keep mentioning it, but I haven't defined it. And essentially, it governs four different things, but some people also refer to ADMET, which has a fifth component. But it's essentially is how well does this compound get absorbed? How well does it get distributed? Like, does it get distributed to where I want it to go? Does it get metabolized? How is it broken down? Is it broken down in the liver? Is it broken down in the gut? It depends on where you want your drug to be broken down, like where you're targeting. And then how is it extruded from the body? And then there's a fifth component called toxicity, which is essentially, is my compound toxic where it will cause some harmful side effects, right? And all compounds are in a sense toxic. It's just how toxic are they? If I can take a small dose and I'm okay, that's great. But if I can take a small dose and it leads a huge spike and a huge response in my body that causes ill effects then it's a pretty toxic molecule right um it's sort of like the analogy like all things are edible but how many times is it edible right like i can eat a rock but i can only drop once right so um these all these properties are very very very important for the drug discovery, for like a drug and how well it will behave in certain environments, right? So, we have now filtered out, we have now about a dozen or so compounds, and we're in the lead optimization phase. And usually, so what this has in this phase, you have had strong hits that have high potential to be effective drugs. We are now going to dedicate resources to these dozen candidates and we're going to develop a series of program around these candidates. program around these cannabis essentially that um we're going to more resources more focus is going to be addressed to these and you're going to design analogs of this lead where essentially you iteratively refine the molecule in order to increase its bioactivity um so essentially going from like 10 micromolars of activity down to one micromolars of activity, like closer to one micromolars, meaning like if you go, if you decrease that, the better it is, the more tightly it bounds to the target. I'm going to optimize towards my desired ADME profiles, like if I need it to be permeable, like a certain permeability, certain solubility to be cleared in a certain way. And you're going to try to increase the selectivity and reduce off target attractions. We don't want side effects. We want to minimize the side effects of this drug. And so you can see on the right, we have the starting compound. And if you take note, like you can see that there's minor changes happening to the molecule. Some of these are more major, but some of these are minor. And these changes, however small or however big they are, could either lead to the compound still being active or even being more active than its original starting state. Or even being inactive, like a small change to one of the substituents here. Like we added, I think, a carbon here and removed the chlorine, led the compound to being inactive, like a small change to one of the substituents here, like we added, I think, a carbon here and removed the chlorine, led the compound to being inactive. So you can see how painstaking it is to find, to optimize these compounds. Minor changes could really throw you off the other way, right? You could have had a really active molecule and a minor change nullified all that. So it is a painstaking process and having computational tools that can help you iterate faster or even suggest you ideas to like what to add or explain why this change that you did, that the medicinal chemist has done, made it inactive or made it more active are all like suitable areas of like research or like where ML might be able to help. Okay, so I think I've sort of implied a little bit of why the need for ML or how it might be able to influence the drug discovery pipeline. But I would like to summarize these points in the sense that we have this design-make-test-analyze cycle. And essentially where ML can help or even just computational methods in general is that it helps accelerate the design cycle of the drug. It's this whole idea of like in silico versus in vitro and in vivo. Now, what do these three terms mean? There's essentially the in silico means on the computer, in vitro means in the test tube, and in vivo means like animal testing. And so in vitro will maybe take a week or so, or a couple of days. In vivo might take anywhere between three weeks, essentially. And then in Silico, depending on the resources, the computation that you have and how expensive the method is, it can happen anywhere between seconds to maybe a couple hours. But essentially, it is much, much faster than the other two methods. And so this whole idea of can I design faster, can I iterate faster and get through these ideas much quicker, right? And then you can have things like generative models that will aid medicinal chemists in generating new ideas. Sometimes they just kind of need to be given like 10,000 analogs of molecules. And then you profile all these molecules with other computational tools and then filter down to another 20 analogs of that lead compound, right? Or 20 like minor change, minorly changed molecules for that same molecule. And, you know, test those out to see if they are even, they push in the right direction, right, for a better drug candidate. And then we get to reduce use of costly resources, such as materials and time, right? So a lot of times, like in vitro labs, for example, you have this well plate, it would be uh it might have like i don't know let's say like 96 tubes that you could fit on there um and let's say you're running an experiment for that week uh and you've filled up 48 of uh spots in that well plate well once you use the well plate it's done like you know you've lost out on essentially 48 potential other things right so um if let's say i have a model that i have strong confidence and that predicts really well and um and um uh predicts my property very, very well. Well, maybe for my high confidence stuff, I can just trust in the model's predictions. And then what I'll do is I'll fill the well plate with lower competence tubes. I think I'm gonna change my example real quick. Imagine if you had a well plate and then you filled up a little bit of the other well plate. Well, now you've kind of like wasted a whole well plate. So let me remove the compounds that I have strong competence in with my computational tools, and then put all my lower confidence predictions into that one well plate and then save on a resource. Right? So, and you get to save on time. So these are all like, I think there are many more benefits, but these are things that the three key ones that I can, that come to mind when I'm thinking about how ML can influence the drug discovery pipeline and then like i would like to remind uh like ml isn't magic and it's not going to be the end all be all it's not going to replace at least i don't well maybe not in a short time but maybe in like 100 years or 200 years but not in the near time future anytime any time that I'm alive. I don't think ML is going to completely replace the whole drug discovery pipeline. Right. And so I think it's a tool like any other computational tool and it's used in tandem with these tools. And here I'll point to an example. This is the CASH challenge, the Critical Assessment of Computational Head Finding Experiments. And recently there is a challenge um uh in this challenge um a lot of people they have like a list of all the methods or all the types of tools that people use for each of their hit finding experiments and uh a growing number of these tools, the ones I have highlighted in red, have machine learning involved in it. And in a lot of these cases, you'll see that ML is used in tandem with other computational tools. And it's fascinating. These things working together have helped potentially find a really good hit for a certain target of choice. Whatever, I forgot what the target was for this challenge. There was actually an ex-post, someone that kind of wrote out some of these pipelines for a lot of these methods, but I couldn't find it. So here I am just highlighting them from the webpage. But it's fascinating. It's like really cool. Like we should think of ML as being something that aids the scientists and not replaces them. So I've kind of gone over the whole drug discovery or parts of the drug discovery pipeline and covered a little bit of how ML might be useful in these pipelines. But I've mentioned things like generative models and graphs, neural networks, and all that kind of that sort. And so I kind of want to dissect these a little bit more and dive a little bit deeper into these popular ML methods on molecules. So the first topic will be governing something called geometric deep learning works so well. And it sort of boils down to like, we have different geometries that are represented. Like, for example, images can be represented as grids. Molecules can be represented as graphs. You know, language, like a sentence could be a sequence of nodes. And all these things have some symmetry, or some regularities in the data that we can exploit and take advantage of. And it helps the model generalize better or learn better for these cases. And so I'll be kind of diving a little bit deeper into that and so the first thing I want to like cover is graph node networks I think this is a very natural representation for at least the topology of the molecule so we see that we have here this is penicillin and you can see that they have your atoms that are connected via these bonds and it forms this topological graph structure of the molecule, right? So we can easily form a graph of nodes and edges where these nodes are characterized by certain properties of these atoms and these edges characterize like what kind of bond is it? Is it a single bond, double bond, triple bond? Is it in a ring? All that kind of stuff. And we can then, given this graph-like structure, use things called graph node networks, which help preserve certain things. Like, for example, in graph node networks, if I were to reorder the nodes, I want my output of the network to also match that reordering. And so this is called permutation equivariance in the graph known networks. But and we'll kind of dive a little bit deeper and subsequent slides about what is equivariance and invariance. But I'm just kind of just want to briefly expose you to the topic. And so you can pipe these into the graph neural networks, which leverage this innate and natural representation of the molecules in order to do certain things like property prediction or generation and stuff of that sort. And we'll go even further. We're not only going to be considering the topology of the molecule, but also its geometric representation, like the positions of the atoms, and be using these things together, right, for these neural networks. And so I just kind of want to point out that like before this before graph known networks became more popular recently in the i think since 2016 2017 uh before uh and still now to this day that's a very commonly used method is they used to pre-compute chemical like physical chemical properties of the molecules using computational like tools and methods and you would get essentially like this vector or this tabular data, essentially, of all your molecules as rows and all these different properties. And then you'd feed these into random forests or other methods or even a deep neural network. But really, random forests and XGBoost really did a... were actually really solid baselines for all of these. So just because deep learning is becoming the rage shouldn't mean you discount using simpler methods to at least baseline to see like, why do you need a golden hammer? If just a regular hammer works just as well or even better sometimes in some cases, right? Especially in low data regimes where non-net networks might struggle a bit in certain cases. But I've talked about graph neural networks and I kind of want to now widen it. So that was kind of like a narrow scope. I'm going to widen the view a little bit to talk about geometric deep learning and why is it so nice. So you can see here on the right in the picture the gnns is part of this group you also have other things that you might be more familiar with like cnn's and transformers and deep sets and rnms where it's this kind of like zoo of uh known networks that all kind of try to uh they're kind of framed in a way where uh it's framed it frames deep learning through the perspective of symmetry and variance and i think this is a popular uh perspective in physics and geometry so this is kind of where they take their inspiration from when when they apply this perspective to the deep learning But they have shown why certain design choices and models have performed so well. They're trying to inform future design choices to follow suit. Why should we follow these design choices? Because it generalizes better and reduces the complexity of the learning task. We'll dissect it a little bit more. But let's talk about let's look at an example. So let's try to discern what is invariance and what is equivariance. So let's say you have an image of a cat and let's say you apply some transformation S on it. This transformation essentially is translating this cat from the bottom right corner to the center top of the of that window right and so when let's say you have a classifier that's trying to identify what this thing is it's a cat right and and on the left hand side it's a cat and when i translate it it's still a cat it doesn't the properties of the cat don't change just because it's moved somewhere else in the image so this is something we call invariance essentially that your neural network is going to be invariant to certain transformations based on how you've designed it for like the output of the models give the invariant to certain transformations in the input of the model. And then we have, in the other case, equivariance, where your transformations that are the transformation on the input into the model is going to have a corresponding transformation on the output of the model. So here we translate the cat in a similar way, but now we're outlining this cat and essentially coloring it black, we're segmenting this cat. The segmentation is going to follow the cat when we translate it in the input, and we're going to have that similar translation in the output of the model when we're segmenting it. Now so this was kind of an example to kind of briefly introduce you to this but this talk is about molecules right so what does that look like like what does invariance and equivariance look like for molecules so let's say i have this molecule i believe this is caffeine maybe but um and let's say that we have a property for this molecule uh this is like the aqueous solubility of that of this molecule now if i were to rotate this molecule right does the property change is the were to rotate this molecule, right, does the property change? Does the global property of this molecule change? No, right? Like the solubility is still the same despite rotating the entire system, the entire molecular system, the molecule, right? So no matter how I rotate it or even in the case of when we translate it, this property is going to stay the same. we translate it, this property is going to say the same. Now, in the case of equivariance, let's say you're doing molecular dynamics, and you're trying to see where all the new positions of the molecules are going to be. So you're going to have these vectors that are going to push atoms in certain ways. Well, now, if I rotate the entire molecular system, right, all these vectors should rotate with it, like with the, you know, with respect to the rotation we applied onto the system. So that's kind of what it means to be like equivariant in the sense for molecules. So this is how it plays out when we rotate it all the way through back to its original position. All the vectors are just going to be rotated. And then we eventually end up to our original position when we rotate by 360 degrees. So I hope this kind of gave a clear understanding as to maybe, actually, let me say this. So why do we need this, right? So think about it in the case of property prediction, when we rotate a molecule, if we don't have this invariance kind of baked into the model, well, it's then what we're going to have to do is we're going to have to augment the data by rotating each molecule a set number of times. And then essentially we're going to do this for every molecule. So let's say you have 10,000 molecules and now you need like maybe 20 augmentations per molecule, like a little bit of rotation each. This expands your data set now to 200,000 molecules. Well, you kind of increase now your compute time because now you have to train for much longer. compute time because now you have to train for much longer and you've increased um uh the model has to try to learn this uh property of being invariant to rotations which kind of now puts more um more strain on the model to learn other things as well and um other things as well. And essentially, that had we baked this symmetry in before, we can now train out a 10k molecule, and it would still be able to generalize for any rotation of these molecules. So essentially, baking in these symmetry, preserving these symmetries in the molecule and baking them into the model um makes the training more efficient and will help the model generalize better because what if i rotate the molecule by rotations never seen before in the case where symmetry is not baked in well the model might uh mispredict uh the property of that molecule so just want to emphasize the importance of these things. And it's been pretty cool. Like there was this recent work, actually some folks from Mila and then I believe Cambridge worked on this essentially summary, a hitchhiker's guide to geometric GNMs for 3D atomic systems, where essentially they cover different molecular systems, like small molecules, biomolecules, materials. They have this timeline of all these geometric GNNs and then other applications like property prediction, dynamic simulation, general modeling, and structure prediction. And they've categorized these geometric GNNs into these four categories. It follows like invariant GNNs on the top in the green, Cartesian equivariant GNNs, spherical equivariant GNNs, and unconstrained GNNs. And so what's the difference between all these things, right? So invariant GNNs, you have these, what they do is they compute these properties of the molecule that would be invariant to these rotations and translations. So think of, for example, if I have two atoms, if I have two atoms that are a certain distance away from each other. So like this is maybe their bond length, right? And if I were to translate this molecule across by a certain translation vector, the bond lengths between these two atoms are going to stay the same. They're not going to, you know, all of a sudden grow wider or smaller because we've applied the same transformation on everything in the system. And you can say that you can easily extend this to bond angles and torsion angles, which is, and so all these properties, these invariant genomes take advantage of and incorporate them into their message message passing step which uh which uh essentially um uh preserves the the uh which bakes in the symmetry into the model therefore it becomes like rotationally invariant for the case of property prediction or rotationally equivariant in the case of the dynamic systems dynamical simulations so um and i think I've forgotten to mention this, but I've mentioned this topic of message passing, right? And you could think of in graphical networks where essentially if I have two nodes and some edge that connects them, they're gonna be communicating or sending messages to each other, passing message, passing information about their node state to the neighbors, right? And so it's going to involve both this, like, receiving information and sending information. So just think of message passing. Sorry, I should have clarified that earlier, but I don't know. Because we'll be talking about it a little bit as I dissect each of these different categories. So something to keep in mind. different categories. So something to keep in mind. So in varying GNNs, remember we've kind of pre-computed these bond lengths, bond angles, torsion angles. These are kind of like the differences between these models through time. But Cartesian arc-olivarian GNNs, essentially what they do is instead of computing these properties, what they do is they constrain the message passing in a way where it will preserve the symmetry. So instead of, so it's just kind of like they've designed the message passing in a certain way to preserve it. So it's kind of like a constrained message passing. But this helps because it avoids the need to calculate or pre-compute these like bond lengths, bond angles, and torsion angles, which could get very, very expensive as the molecules grow. Because you could think of like the minimum bond lengths you have to compute as n minus one atoms, if n is the number of atoms, and bond angles would be n minus two and torsion angles would be n minus three. And you can imagine if you have a protein of let's say, if you have anything like a protein, in the case, like if you have a thousand atoms in the system, calculating all these will be very expensive. So let's avoid that and instead just design the message passing in a way that we don't have to compute this, but we'll be able to preserve the symmetry. So that was pretty cool. We don't have to compute this, but we'll be able to preserve the symmetry. So that was pretty cool. And then what we have on top of that is a spherical equivariant genin. So before we were in Cartesian space, and now we're going to spherical space, which is essentially what they do is they work with spherical tensors that are irreducible representations of the rotation group SO3. And we're not going to really dive into deeply on it, but essentially is what they do is they, it allows for really cool things that they could do with essentially that, but they preserve the symmetry by essentially working with this rotation group with spherical tensors. And lastly, we have unconstrained GNNs, which has been pretty recent. It's been pretty interesting, actually. So in the above three methods, they've all kind of in a way tried to either work in a space like for example spherical genomes they work in a space that has rotation submit like uh equivariance and invariance uh or like a rotation symmetry and then cartesians have done this where in the message passing invariant genomes have done this in input features but unconstrained gs and all these prior ones, they are sort of like constraining the model. These are hard constraints on the models, like baking and that sort of thing. What unconstrained GNNs do is they kind of softly constrain it, where instead they go through these things like they do data augmentations on the input data, or they incorporate loss functions that would have the model learn to be rotationally equivariant or invariant and stuff of that sort. And what this does is now you're not constrained to a certain type of model architecture. Now you kind of have freedom to expand and it also allows for freedom in certain optimizations in the model. So just to take note is that the unconstrained genomes kind of enforce the symmetries in a soft way, while everything else does it in a hard way. So I hope you kind of understand like the importance of geometric deep learning and its kind of influence on the field. But now we're gonna move on to generative models and there's a whole slew generative models that we'll be focusing on ones that are kind of state of the art today and have been a very popular method over the last few years during diffusion models. And so we'll kind of start off with images and then I'll give a demonstration on molecules, but essentially, so let's say you have your data distribution X zero on the left and on the right, you have your X of T and in this case, and in diffusion models in general, this is a Gaussian distribution. And so why the need for this, like, you know, these two distributions, right? So the data distribution, we don't know how to, you know, what's the generating function for this distribution. We don't know how to sample from this distribution. If data is complex, there's a lot of things that make it really, really hard to sample from this data. But Gaussian distributions is something that is really, really easy to sample from. And we can do it really fast, essentially. We know the function. It is easy to sample from. And that's amazing, right? know the function, it is easy to sample from and that's amazing, right? So how do we get the benefits of being able to easily sample from a sampleable distribution but get something that is useful like something in the data distribution? So they do this concept called a diffusion which essentially you're slowly transforming the, or smoothly transforming the data from the data distribution to the Gaussian distribution. So you're essentially transforming the probability density so it matches that of a Gaussian. And what you're going to do is you're going to learn essentially the score function, which is the gradient of the log likelihood with respect to your input. So in this case, it would be the pixels of the image. But essentially, it's going to be learning the change in probability density at varying noise scales. And so here, noise scales are just like, you know, capital T, like we have T different, adding a little bits of Gaussian noise to it. So we're gonna essentially try to learn that gradient of the log likelihood and then that gradient of the log likelihood, and then try to reverse it. And what was really cool is that the diffusion, the reverse of a diffusion is just diffusion in reverse time. So this is kind of what they exploit here, where they will transform now sample from the Gaussian distribution, and slowly and iteratively remove noise or denoise the image so you get samples from your data distribution. So and they do this like the whole sampling scheme follows like follows like longivine dynamics which if any of you are familiar with molecular dynamics is a very um key thing in molecular dynamics so it's really interesting to see this like uh these diffusion models kind of uh take inspiration from physical um like how things work actually physically and so i think this is part of the reason why for molecules has been working pretty well. But and we'll see here, like this is an example for the molecules. So you can start on the left. This is like the Gaussian distribution on 3D coordinates. And then we iteratively, like slowly, we'll denoise the molecule till we get something that would be a sample from a distribution, from our learned distribution. we're in images, we do this with like the RGB values on the pixels. For molecules we would do this, like if you're doing conformal generation you just do the diffusion on the positions of the atom coordinates. And if you're trying to generate new molecules, you do the positions and you'll do like the atom features and um if you haven't already i recommend watching last month's talk uh with paul where he actually goes over the diffusion stuff for proteins by hand um and it's pretty informative and interesting. And what, you know, you can easily extend the things that he does in there and what he displays on there to molecules. So, you know, this is in the context of proteins, but it's easily transferable in the context of molecules. So I recommend watching that if you haven't already. And so these generative models have been very, very useful in things like So I recommend watching that if you haven't already. And so these generative models have been very, very useful in things like de novo generation, so generating novel compounds. And you probably don't want to just generate any compounds. So a lot of times that are like property guided generation. So using conditioning the generation process to generate molecules with certain ADME profiles. And so you can meet criterions, you can produce analogs of compounds to help you generate ideas. Or if you're given a protein pocket that you're trying to target things like I don't have a reference in here, but like a structure based drug design, this is work from Ijvom, I believe. I might be wrong, but essentially where they tried to generate a ligand within the protein pockets of a protein. So all fascinating works that involve diffusion. And I think there's a lot more to come. So keep an eye out for that. And then there are some other areas. I'm not gonna dive into them too deeply cause this is a little bit out of my expertise or I'm not even an expert, but just out of my domain of what I know. And so there are things like reaction synthesis planning. So like how to synthesize desired molecules. This is very important are things like reaction synthesis planning, so like how to synthesize desired molecules. This is very important for things like compounds. You can't just like make them. You have to design and follow like constructed from materials that you have and perform the right reactions in order to get your end result. So this whole reaction synthesis planning is very interesting. I think folks at Connor Cooley's lab in MIT have been doing that. Active learning is something like essentially it's this iterative cycle where you're trying to find the next best thing to generate data using labs that will fill in gaps in the model. So I'd say your model is predicting really badly on this chemical, this space of chemical, like molecules. Well, then let's just try to generate more data to fill in the gaps and then train the model to have it perform better. I think there's a recent trend in growing interest in machine learning and multiomics, and then learning on knowledge graphs, which is kind of leveraging all the data and relations accumulated to derive even new relations. And then finally, we're going to go over the open problems and challenges. Essentially, that's a few, I guess, these are mostly centered around generative modeling, but there's other stuff as well. But for generative models, for example, I think we need to rethink a lot of things about how we evaluate these molecules. For example, there's this metric on there called validity but it doesn't really measure if these are valid molecules on these from these generative models. Essentially a lot of times these generative models will produce molecules but these molecules are not synthesizable they're just like really weird or really reactionary. So kind of like these methods of measuring validity is not so accurate. And I think work and being able to better measure this validity and account for it and account for it in generation process. But other challenges are like data, right? We have unfortunately like image domain or natural language, where there's like, you know, lots and lots and lots of data out there for those domains, but for molecules, it's not so much, right? In the public domain, we have okay data sets. And even in an industry, things are things are you know they have a lot more data but um it's kind of scattered and broken at times but uh but i think um uh some collaboration between the two like a lot of pharma companies are benefiting from the research efforts in the public domain for drug discovery. So it'd be great to see more pharma companies or maybe forming like consortiums. There's like things like IQ Consortium where companies can come together and collaborate on something. But it'd be great to see them to, you know, being able to provide great quality data sets that the research community can use so that they may benefit from it, right? And so, because a lot of complaints from industry are like, oh, you know, these models don't transfer well into industry settings. Well, it's part of the reason is data. But even in industry, we don't have the volumes of data that we have for images in natural language then we have auto distribution generalization which is essentially how well can my model generalize uh when i'm really far out from my just my training distribution and biological complexity like uh you know biology is very very complex and you know we've made a lot of progress biology is very, very complex and we've made a lot of progress, but these are all still in isolated settings and biology is like this whole system of interactive things. So being able to scale up to that would be a really, really interesting challenge for the future. So I'm looking forward like developments in this area. And then just kind of like a, I think I gave a brief intro in the beginning, but like my research and kind of focused on like modeling, these things are called stochastic interpolates. This is from like a term point from Michael S. Embargo from NYU. And it's essentially trying to model it's these things of like diffusion and flow matching and all that kind of stuff. But I really like this paradigm where we're trying to have these generative processes that are very much like actual real physical processes. And I'm combining that with geometric deep learning for all these different areas, like protein-magnet interaction, sampling equilibrium states of molecules, and like incorporating physics in an efficient way to kind of reduce wonky molecules, kind of like those molecules I was talking about with the generative models earlier, trying to produce more realistic ones that obey physical laws, but we don't have to pay the price of physical laws. So I'm hoping to push that field a little bit further. And so yeah, we're going to work on some things like trying to incorporate these physics a little bit more in my research. So yeah. yeah, incorporate these physics a little bit more in my research. So yeah. So thank you for listening. And if you have any questions, you can always reach out to me. I probably should have put my contact on these slides, but always happy to discuss about this area of research and help people kind of get into it. I believe Nucleate will be releasing a resource page around this as well, which I think is a great first start to kind of getting into this. And I would like to thank Nucleate again for inviting me to give this talk. It's been an interesting and really fun challenge to kind of construct these PowerPoints in a way to introduce you guys to the school.
Introduction to AI in Drug Discovery
AI in Drug Discovery: Introductory Overview by Majdi Hassan 💊🧬💻

🎙️ Presenter:

Majdi is a Master's student in Machine Learning for Drug Discovery at the Mila Institute in Montréal. He has worked at AbbVie for several years, developing state-of-the-art deep learning, including knowledge graphs, geometric learning and molecular simulations, all within the context of early-stage drug discovery.

https://www.linkedin.com/in/mhassa22/

🔔 See more content of our initiative here - https://linktr.ee/aixbiotech_nucleate/

📧 Contact: AI.Bio@Nucleate.org 

#drugdiscovery #ArtificialIntelligence #BioTechAdvancements #ComputationalBiology #BiotechEducation #ScienceAndAI #InnovativeDesign #BiologicalSystems #TechInBiology #FutureOfBiotech #AIInHealthcare #ProteinStructures #TechAndScience #BioinformaticsExplained #SyntheticBiology #ScienceExplained #BiotechRevolution #AIInScience #MolecularInnovation #TechBreakthroughs #biomedicalresearch