 All right, cool. Welcome everybody. Usually I say good morning, good afternoon, good evening. In this case here, it's definitely good afternoon. Thank you for joining the Life Science Workshop 2024 in Munich and around the world globally. So thank you for joining. I'm looking forward to this extension of a very interesting morning session we already had here in Munich on-site with interesting presentations, interesting conversations throughout the morning and now we continue with more conversations more tech hands-on sessions and more you know interesting content throughout the rest of the day so we are doing this for a couple of times now we have I, done the last one in 2021. We did a couple of ones on site and online a couple of years before. But now again, we continue doing these life sciences workshops for the community, with the community, together with everybody involved. Again, here in Munich, in this nice venue here and online on the screens outside. So it's really cool to do this again and to continue to work with the life sciences and the healthcare community on graphs, technology, Neo4j. So a few bits and pieces. So we have a Slido link that also rotates here for the people in the room, but also online. I have shared the link in chat. Use that for your questions, for your Q&A. There will also be some polls in the break, so check it out and let us know if you have any questions. We have some minutes after each session so I can read out any questions you have and see if we can squeeze some questions in. So it's always cool to keep the interaction going and to get you outside basically involved here with us in Munich. And yeah, I think that's it. We will record the sessions, obviously. So if you're wanting to watch something afterwards, you can always head out to our YouTube channel, but we'll send you the link for that. And we share slides as we can with you as well. And yeah, let's have fun. Keep the conversation going. Enjoy the day. Enjoy the afternoon session with us. And I'm heading over to Alexander Jarosz, who will kick it off. Hi. Thanks for staying here. So we have half of the audience from the morning session. Hello everybody around the globe. My name is Alex. I'm a technical consultant for pharma and life sciences at Neo4j. I'm based in Munich. I consult pharma and life science customers. We have a great session in the afternoon. We will talk about Gen AI applications mostly. I'm basically setting a little bit the stage about vector indexes and what you can do with it. I think we have great stories that can the speakers vectors, vector indexes and what you can do with it. And I think we have great stories that the speakers will present them on their own, obviously, and present great use cases. So I'm really looking forward for the afternoon session. Let's get started. This is my pointer. For the ones that have been here in the audience in the morning session, we talked about vectors, right? And in the terms of Gen AI, we want to represent clunky data from audio or text or video or images, machine readable, but also to deduce insights from that and calculate similarities and so on. And this is where large language models or JNI in general come into place to represent such data, which is clunky and bulky, and we don't want to persist that in the database, so to say. We want to represent it in a lower dimension something like a hundred dimension a thousand in neo4j in order to use that for for quick lookups and the main purpose as i said is calculating similarity very efficiently while having lower computational cost in terms of storage but also in terms of the computation itself so just for the theory we have a graph here and two nodes and we want to know how similar node u to v is and this is one technique to traverse the context so the local neighborhood of both nodes here and then transforming that into a vector space. And if they are very close in this vector space, we say that node U and V are very similar to each other based on their context. And we can define the context on the node level down to properties as well as on the relationship level also down to properties. Just like here one example, which represents here some articles, can be PubMed, can be patents, can be any sorts of text, medical sheets, we will come to that in a second, that are here represented as vectors. I don't know, 1536 dimensions, 3000 dimensions that are coming from either PDFs or from unstructured data, Word, Excel, PowerPoint, whatever it is, that gets feed into the LLM and the LLM translates that or encodes that into a vector and we can store the vector in our vector index. And then we have these articles in this case here connected to different so-called entities, for example, that are mentioning a specific word and this is called named entity recognition neo4j and vector indexes um one limitation or many limitations that an llm has is that um llms tend to hallucinate so they make up things that we not necessarily know or sometimes we can recognize that. But it's really, really hard to trace that back and have this kind of explainability and transparency. And that's why hallucination is a hard problem with the LLMs. Sometimes they are unable to verify the answers or we are unable to verify the answers because in the end it's a black box, right? You get back a vector or an answer but you can ask is that true? I did it on my own and the LLM made something up and made it even worse. So it's really complicated to verify the answers. There are some ethical biases. Also, if we think about the approval by the FDA, so it has to be explainable, it has to be ethical. So this is where large language models are really not good. And this is where knowledge graphs comes into place. And in the morning session, we heard a lot about knowledge graphs. So what is that? Nodes and relationships in your company, in your organization, where you have facts and entities stored as nodes and relationships. So this is the left-hand side here. We have the facts that are explicit in the database, nodes and relationships. And if we take the time, we can explain it, right? We can traverse by eye and by human interpretation, we can traverse the graph. In general, that's possible. Can be tedious if we talk about large graphs, in general we can do that. And we use the advantages of an LLM that can easily deal with words and semantic search but this is implicit right so we can we do not have the facts from the LLM and we combine that as a better together story. There's also a slide that you hear in the real audience so early before. So we have the features of a vector database, so we have our own vector index and on top of that we can not only identify similar neighbors or decay nearest neighbors but we can also query the context of those neighbors which can enhance or enrich the answer or make the result more accurate. In addition to that, we also ship our graph data science library that has some algorithms for specific graph problems. Now, a little bit about the difference of vector indexes when compared to graphs. What you see here is a knowledge graph that represents your knowledge. Can be billions of nodes and relationships. This is what usually the customer builds on their own or downloads some public datasets and enhance that, enrich that, and represent the knowledge as it is, connected with heterogeneous data from the data sources. What we can do is we can calculate embedding vectors, so these ones here, so we can calculate vectors, these are represented here as these little sticky nodes, we can calculate them by using information from the node level, from the relationship level, and also from the property level. And they get projected in this very small vector that we can then use for the lookup. And what you see is kind of a dimensionality reduction, basically a projection of a multi-dimensional graph on, let's say in this case here, a 2D space. And then in the end what you do with the vector index, you look using natural language, you use, for example, you use kind of the, you're looking for the nearest neighbors concerning your question. And these get efficiently identified here in this low dimensional space. And in the end, what we do is we translate that back to the corresponding nodes in the relationship. And this is where your knowledge graphs comes into place, because we then traverse from these nodes to the context. That's basically a pretty simple basis for it. a pretty simple basis for it. And this brings me a little bit of how does that architectural look like. So we use LLMs in two ways. The one that I touched so far is the one on the left-hand side. So we try to integrate any sorts of data you can think of. RDF, as I said, unstructured data like text or images, videos, audio files, structured databases like SQL, Mongo, whatever it is, or even single files. We ingest that. We can do that automatically even using the LLM. So any sorts of LLM that you either host on your own or you use the one from the cloud providers that automatically do, for example, named entity recognition or feature engineering and store that in the graph database. We can visualize that with different visualization techniques. One of our in-house ones is called Bloom. We have NeoDash as a dashboarding functionality. And as I said, we can use the graph data science library on top of that to get more insights out of the data. On the front end side, we can use the LLM and kind of democratize our knowledge graph by having it kind of an interface between the user that doesn't speak any programming language or query language. language input that gets translated via the LLM into a cipher query language and with some traversal other queries so we will see that in one of the other presentations later and then deduce something from the graph and get it back as maybe a table maybe a natural language answer a little bit also more technical, how could that look like? So we can use here, as I said, the unstructured documents here. We can put that into a langchain. We can also use a reference ontology or many ontologies in order to say, well, while you're integrating, please use our standard terminology instead of using some random entities that you recognize and try to match them according to the ontology that at the same time gets stored into Neo4j. Talking about the hallucination effect and how we improve accuracy by using a knowledge graph. This is called graph-reg orval augmented generation. And this is a pretty simple architecture here on a high level. We have a user question here. I don't know, please give me all the patients older than 75 with a BMI index of 34. Something like that. This sentence from me, the query gets translated into a cipher query here, the cipher query gets performed on the knowledge graph, retrieves the information and as I said gives also back the context of the of the note here, of the notes. And that makes a big difference because usually this is the proprietary data or your secret data that you in your organization or company have, right? Nobody else has this and that data that is connected internally. So that's why you reduce the hallucination effect by feeding the answer with actual facts from the knowledge graph. Some of you may have seen this slide before. So I'm showing that in order to give you an understanding what that means. Let's assume we have this knowledge graph here on the left-hand side. Can be billions of nodes and relationships. And we query the graph and we get back the answer, which is AD. Now I ask you, what is AD? AD could be referring to this node here, if we got a little zoom in here, this node AD is connected to the context of a disease term. So that tells me AD here in this case is Alzheimer's disease. So the context gives me more insight and reduces the hallucination effect. Because it could be a different part of our knowledge graph where AD is returned with a different context being a protein. So in this case, AD could stand for alcohol dehydrogenase, which is a protein degrading alcohol. It's a completely different thing, right? And based on the context, we get a better answer and a more accurate answer. And that can be any other kind of result in your knowledge graph that has different context. And this is where Neo4j Enterprise comes into place because all of this is working out of the box with role-based access control. So see your enterprise knowledge graph having sensitive data from patients or HR data or R&D data, some of your users are meant to access the data or query the data but not displaying it. So we can use that with role-based access control, even value-based access control, so VBAC, in order to keep the graph safe but also be compliant. Just some words about the use cases. As I said, we have several other customers speaking. The use cases usually are there for getting similarities of, for example, text that we input and all these sentences here are basically saying the same. So the LLM understands that these sentences are basically all similar, so very close to each other, and we can look up vectors efficiently. Earlier in the morning, we talked about the applications, how can we use that, for example, as a recommendation engine based on the context and the vectors, we can recommend, I don't know, the best next hotel for your holiday. Or in terms of insurances and banking, we look for fraud using vectors. And the last one is the REC, so the retrieval augmented generation. Now, two real-world examples that we had at small demos, and since small demos never worked, I just used them here as screenshots. The first one is a data set of, I think it's 50 or 60 medical sheets that are publicly available. We just used them here in this little Jupiter notebook and it says here the patient was 34 years old blah blah blah s 100 protein CD 68 I don't know Typical AC age right you get the point. There are things in the text that a Normal user would read or an expert being a medical doctor would read and say, well, I recognize here the age of the patient and I recognize, I don't know, a mutation in S100 protein, whatever it is, right? But this is the bottleneck. The bottleneck is our users that understand the domain and we need them in order to transform this ugly text here and the kind of the speech and language and all the preferences, how they express different terms, and put that into a database. And this is what the large language model can do for us. And we are doing that by so-called prompt engineering. So we tell the LLM what to do in order to automatically populate the graph. And we tell the LLM from the case sheet for a patient below, extract the following entities and relationships. And step zero, we always say it should never give zero answers. So we ignore here step number zero. But step number one is I tell the LLM, because I have something in mind, I draw that usually on a whiteboard, I want to have something recognized being a case. I want to have recognized a person. And if you recognize something that could be like an ID from the patient, please put that as a property in my be like an ID from the patient, please put that as a property in my node person. And maybe another example, we have a disease. If you recognize something that is like a disease and has maybe an ontology term or an identity or a name or description, please automatically recognize that and put that into the node label with the property. And the second step, which is the third step, is once you recognize all these entities here, please connect the nodes in the following manner. So you connect the person with the case with a relationship called for. You connect the person node with, let's go back here in the end, you connect the person with the symptom via the relationship called hasSymptom. And we can give it some properties. So what we do is basically tell the LLM what to do, maybe even give it a role. You are an expert in pathology, blah, blah, blah, something like that. Or you are a patent lawyer, please recognize, I don't know, patent numbers from the UPC, something like that. And we can even feed here an ontology into it. So we can say, please, by using disease terms here, please use the standard terminology coming from Mondo or coming from something. Super cool. And it works astonishingly well. In this case, what I would like to have is a JSON format as an output and that's basically what the LLM returns me. And for this case here, I don't know where I have the time, I think it took non-optimized way and waiting a little bit in the query with chat GPT, it took like two minutes or so. Which is not scalable for a thousand things but as I said not optimized and I was in the queue. So usually you use your private LLM in order to transform a very ugly text or an image or whatever or kind of a mix of everything automatically into the database. Second example, what I hear from customers in the pharma industry, well we are a regulated industry so ontology terms are there every day. So you have to be compliant to the ontologies. It makes sense to speak standard terms, right, for all these global standard terms for all these global organizations. But the problem is that we have internal and external terminologies or ontologies, a mix and match. So, I don't know, for example, the AstraZeneca are more specialized on this topic. So the ontology would be extended by 25 terms, while bio on the other hand is more, I don't know, in the cardiovascular space or where it is, right, and they have more some kind of specialities or they treat things differently. So we can mix and match. And that is exactly the problem. So one example here, and we did this obviously with public data sets, we have mesh terms, which is not an ontology, but the mesh terms of scientific articles is a kind of a hierarchical structure here. So we treat that as an ontology or terminology. We have the disease ontology here and some sort of Wikidata export here for treating disease ontology terms. And the cool thing is public data sets is that we have these so-called cross references saying that we have an identifier here ABC, which tells me in this ontology, this corresponds to one, two, three. So the identifiers, what we can do in a knowledge graph is just create this relationship persisted in the database as an object to say, well, this term is the same as that term. And this term is the same as that term and this term is the same as that term and the end closing the loop here for the for the triangle and this is how you would mix and match internal and external terminologies it comes in every kind of flavor like xml oboe rdf turtle files and so on so we are easy with that we We can use that. We can import that and also export that using our Neo semantics plugin or if we talk about Python called RDF lib in our cloud environment. The problem is sometimes we do not have and maybe we go one step back. Sometimes we have partial information only or we don't know if this corresponds to that. Right? There are some gaps and I would like to know is this term the same as that term. So how do we do that? Well we can ask an expert, right? We can go into the disease ontology term, the best expert at Novo Nordisk ask him do you think that this subtype of diabetes is the same one as this term here in the other terminology? And they will say, yes or no. Well, obviously, it's not scaling, right? So how can we overcome that? Well, we can ask a large language model to help us here a little bit. And I hope you can read that. I have three examples here with me. I don't know, Alex, do we have a zoom here on this remote control? Double click. Here we go. Okay, first example. Human papillomavirus infection is the same as human papilloma virus infectious disease. Okay, makes sense. Human papilloma virus infectious disease is the same as papilloma virus infection. Okay, so we if we would read that, right, we would say, why not creating a new relationship in here. And this is exactly what an expert would do. But this is an easy example, right? We go to another example. Shigellosis is the same as Shigellosis. Okay, makes sense. Shigellosis is the same as dysentery bacillary. I don't have a clue. So I'm not the expert. So how do we deal with this one? Last example, Bejel. I hope I pronounced that correctly. Bejel is the same as Bejel. Okay. Bejel the same as trypanoma infectious. I don't have a clue. But what we can do is forward that to the large language model and ask, do you think that these are the same? And that's what we do. We have here the prompt. Given the two different disease names, tell me if they are the same. And we input here the first disease term and the second one. And we go through all the records here in our ontology. And if the LLM says yes, then we are parsing the information and the description why the LLM thinks this makes sense. Keep in mind the hallucination effect. And in the end, we are matching those two diseases here that we call, in this case, resources. And we create a new relationship. And I just gave it a name similar according to chat GPT. So pretty easy, pretty straightforward. And this is how it looks in the end. So the LLM tells me they are the same. And in this case, I just made one example here. It tells me even the description of it. Yes, they are the same. Shigellosis, also known as bacillary dysentery, is infection of the intestines caused by a group of bacteria called Shigella. Okay. Obviously we can ask an LLM back. So are there entities in my knowledge graph supporting that or not? So right, keep in mind the hallucination effect. But this is just like another example how you can use an LLM or generative AI methods in general in order to help you make sense of your data, enrich your data or give more context. Just like a small preview here that we have a so-called graph builder. I have that on the slides and there's a YouTube video tutorial. We have built a little demo tool for you that works with your either with a sandbox of Neo4j or your own Neo4j database you just connect to as the connector somewhere you connect here to the database so the screenshot was not connected. You can drag and drop any word file whatever PDF you can drag and drop it here the named entity recognition as in the recognition as in the first example is done automatically. You can also say I want to transcript and encode my YouTube video or Wikipedia article or I don't know, S3 bucket somewhere in my cloud, which has unstructured data and just automatically populate the graph. This is super easy for trying it out. So I was amazed how well that works. Obviously, you can improve and optimize things. But this is, I think, a cool start to get a little bit into the rhythm and to see what's possible and what's not possible. And with this, I'm already at the end of my intro to the JNAR part and afternoon part. Yeah, hope that was informative and easy to understand, easy to adjust and happy to take questions. And I'm really, really looking forward for the next presentations. Thank you. Cool. All right, Alex. Do we have questions? So far, we don't have questions. Do we have questions? So far we don't have questions. Maybe we have questions in the room. Somebody here on site. There's a question. Yes, Robert. Maybe I'll get your microphone. Yes. Sorry. Thank you very much for your nice presentation. So you showed what we can expect based on LLMs information to knowledge graph. Yes. And defining some subjects or some ontologies. So you have also functions that you include in entire ontology. Yes. Okay, you really can upload it. Yes. Perfect. Yeah. You can, it's a one-liner, say, use this ontology. Boom. You can also optimize it by giving it already a predefined data model. So not making up things. You can also say even like I'm totally unbiased, please come up with a data model. And then do a second round and a third round of optimizing it. Or you say, well, I want to have patient or patients or something, you know. Some people call it diagnosis, some people call it condition, or the ontology term. Totally flexible. Also by the way, if you're using the LLM as a user input, you can say, well, this is the prompt from your user, please use as far as you can get with the ontology term and the data model in order to give back the best answer yeah cool thank you any any other questions in the room sounds like maybe maybe i have a question then yes um in in your view i mean you talked a little bit about the llm about this this integration gen ai integration how much of an impact do you think does this make for for the, for the life science industry? I mean, you showcased a couple of cases, you showcased a little bit of what it could potentially be, but what's your thinking? I think, and I was a user and I was in the pharma industry before, I think our data in our domain is extremely connected, while at the same time extremely heterogeneous, which makes it really ugly to work with relational database systems sorry to say that here I'm very biased but at the same time we have problems of integrating that making sense out of it and in the end several people who are not trained for cypher Python is on to to get access to the data so I think that makes it a big difference yeah it's a huge a huge hurdle as well I mean to adopt and makes it easier and especially the visual aspect as well I think is is another doesn't have to be it's not a reconnection with LLMs of which any I but I think the visual aspect of the graph database in the deck is a huge huge plus yeah the only thing that is is very sad is that we for example in the pharma industry we will see in 10 to 15 years if this jenny i thingy and neo4j made a big difference and if there's the next drug which is probably based on a combination or on one of them so let's wait another 10 years we we have a couple of questions from from the internet so um let's uh let's see um from mark when llms create a graph database are there good tools for identifying structures that are likely to be hallucinations can this be easily used to improve locally trained llms i think we go maybe a little bit about this in the in the final session of today when thomas shows yeah hands-on like how how how to use uh neo4j and llms with in this specific use case so maybe we table that for later or do you have a quick that's a tough question i must say um so are there i think we are already seeing patterns of generated answers that are not specific enough or also tend to use the same words over and over again. But I don't have a clear answer on that. But maybe Thomas can comment on that because he's doing that all day long. So that would be a good question for him. Let's wait for that. That's a super good session. I think that fits um are there any use cases for predicting new nodes as a question from gregorius uh for sure so currently i don't work with any customer predicting nodes but well link prediction is one of the the big topics for creating links between, for example, drug targets and a new indication. New notes. Haven't seen something yet. Yeah, I mean, with the graph data science library and all the algorithms, we come I think the link prediction is probably the most likely one to make sense in terms of graph use case, because you then see, okay, you have a couple of links here, whereas a good link that is currently not established. Maybe one avenue towards that question, and we had a, what was it, a podcast with Jason from Developer Relations. He used an LLM for creating a mock-up data set. So for example, you don't have enough data from patients or whatever it is, you can tell the LLM, well, I have this and that data model, please generate a random number of records so that I can debug and kind of create my application. That's a good point. I think I can find out that video where we did a recording of that and share it in the chat. But yeah, that would be creating nodes, so to say. Yeah. Goes a little bit into the app you showed, and the Knowledge Graph Builder app. I think that takes a couple of good info points, data, PDFs, Wikipedia, YouTube videos even. And then from that, you create a Knowledge Graph. And that obviously adds nodes. So now I'm locked out of my system. Great. All right then there's a question from Shang. Could you share the repo of your demo and could you share some use cases of healthcare institutes also since you mentioned using GPT how do you deal with patient ethics issues I think that's a so the the Jupiter notebooks or github repositories will definitely share but there's also github repo for the graph builder we got a ton of others by request yeah we obviously we create more and more demos in the morning session I showed the the demo from Stuart about Campbell data integration. So usually we share that. What was the second part of the question? The second part was, since you mentioned using GPT, how do you deal with patient ethics issues or security of data, property, personal data, especially in patient? Wow, tough question. I would answer it in a way that you can, as Neo4j provides role-based access control to restrict access to certain sensitive data to certain users. And that can be a traversal, but can be that you allow the traversal, but not visualizing that or everything. So I would answer that on on role-based access control which is a database feature right yeah yeah and then I think you can also take it in in as far as that you should probably don't upload your patient journey data or your patient confidential patient data into an public chat GPT like system on the internet true there True. That's probably something you should not do. Obviously, yes, the data will be used in other ways. So that's something you should be aware of. But, I mean, you can... For the companies, that's a no-brainer, right? I don't have to tell anyone that AstraZeneca would never, ever use the public version of chat GPT to put in any sort of proprietary or confidential data or sensitive data right so that's why you have your own either private LLM that you that you trained on your own or you use kind of the from one of the cloud vendors which is in your VPC isolated ecosystem yeah and then you can obviously deploy a GPT likelike system on your own instances, on your own system, and then it's closed, fenced off. I think that's really important to be aware of this, that you can do this, you can do this locally or in your own virtual private cloud environment. Then connect it as well to a Neo4j instance to add the knowledge graph. Everything stays in your house, basically, in your private servers, in your private infrastructure. So you can do all that without going to the cloud if you don't want to. And especially in this kind of scenario where it's really confidential, it's very important to stay secure, keep it as secure as you can. All right. We have one more question maybe from Dag on the Knowledge Graph Builder app. They ask, when extracting data from the YouTube YouTube video is that only the description and title or does it include the transcript as well I think I can I can also answer that yeah it is I haven't used the YouTube part of that yeah it also is the transcript so it takes it takes a transcript and loads that in and creates out of the data that it finds in the transcript similar to a Wikipedia page takes it all in and creates the chunks and and adds that information to the knowledge graph does that refer to all the songs of Taylor Swift being transcribed and then put that into a knowledge graph I don't know do that build that and then chair link with us interesting good all right I think we have maybe one or two more minutes until the next session, but I don't know, do you want to say a few words before we dive in with Sebastian or should I take another question? Maybe one more question. Here from Alejandro. What is the challenge associated with the adoption of new ontologies that exist out there? It's a wide question, I would say. Yeah. Well, if you're very critical on everything what's new and not being already a standard, you can be very critical on that. But I don't see why that is an issue. Usually what we see with customers is that they integrate not only one ontology, maybe that is an avenue for the answer, they integrate more than one ontology, they versionize the ontology so we can even keep track of, yeah, an ontology is a living thing. And you can traverse back and forth, say, well, the new version or the new subpart or the new subgraph of the ontology, I'm not trusting that or I don't like that or it doesn't fit to my application. I'm using two versions before. And it's probably also a good kind of like almost planted question, but it isn't, to our next presentation right yeah we gonna talk about ontologies we're gonna talk about well not we but Sebastian will be talk about how you can load in different kind of anthologies make a knowledge graph out of out of external sources and enrich your knowledge so I think watch out for the the next session basically yeah so we hand over to Sebastian from University of Heidelberg, or University Clinic of Heidelberg. He should be connected remotely. He should be. Hello Sebastian. Oh wow. Oh here it is. All right. Hello. Good to have you here and looking forward for your presentation. Yeah. It'll be cool. So I hand over to you. Maybe you give a little introduction to yourself and yeah, take it on. Thank you. Do you hear me? Yes. I even get a little bit of feedback. Well, I guess that's fine. So my name is Sebastian. Many thanks for having me. I'm sorry that I can't be there in person today. I work, as Alex said, in Heidelberg with Julio Záenz-Rodríguez and more recently also connected to the EBI at Open Targets. And I want to talk a bit today about the kind of pressures that we had for finding solutions for dealing with knowledge representation and usage problems in this general area. So I want to maybe start a bit on a tangent So I want to maybe start a bit on a tangent. Because I want to motivate why we do what we do and the way that we do it. And I think it was a great introduction to the general principles of Gen AI in this previous talk. And many of the things that we do a little bit differently are motivated by the special requirements that we see in in biomed specifically and one of these is what we see here this seemingly trivial statement that correlation is not causation but while it's superficially pretty simple it gives us a hard time in gives us a hard time in applications to modern biomedical computation. As an example, you would, for example, be interested in the impact of nature versus nurture on a given trait or a disease. And this becomes complicated really quickly if you look at the modern omics that are supplied, because we very easily hit the curse of dimensionality looking at all these genes and variants. And this becomes even more complicated if you look at the complete picture, where most of the experiments that you run are informed by a multitude of different factors. And it is only natural now in the time of ChatGPT to be hopeful that generative AI can help us deal with these problems in the same way that it has helped me create this illustration in a matter of a couple of minutes which would have either taken me multiple hours to complete or could alternatively have looked like this. But if you go beyond artwork then it becomes more challenging to address the problems that we have in systems biology, systems pharmacology. For example if you compare the schematic that was drawn by a human with a very distinct purpose to inform the reader about the growth factor pathways in the human, particularly when it comes to pharmacological intervention, then if you want to try and get the same out of the current top of the line models, GPT-4 and DALI-3. You, and this will surprise no one, get something that looks pretty impressive, but has absolutely no meaning. But what is not trivial is how would we get GPT-4 and DALI-3 to actually adhere to the purpose that we want them to complete. Because GPT-4 does have all the information, it knows exactly which proteins are part of the EGF pathway and how they are connected. It just can't translate it into something useful for this case. And we looked at this in a recent perspective that we wrote together with Stefan Bauer from Munich, where we looked at the spectrum of models that we use in systems biology from the low parameter space until the newly minted foundation models with this unimaginable parameter space, and particularly looked at the impact of these knowledge representations that we use and the biases that they cause in the models and how that relates to this problem of correlation versus causation. And one of the interesting insights for me personally was that the more complicated the models get, the less intuitive it becomes how they use the knowledge that we give to them. And so we are starting a series of experiments, and I don't want to go too deep into this, but for example, we examine models with fairly high parameter values, such as the Giers perturbation model, which is used to predict the effects of knocking out two genes at the same time. Some of them may not have been seen during training of this architecture. It's pretty complex and it is about one million parameters. You can't say exactly because it depends on the size of these GNNs here. So two graph neural networks and then some MLP layers. I'm not going into details here, but essentially what is interesting about this model is that it has some discrete representation of prior knowledge in networks. For example, here they use a perturbation relationship graph that is based on gene ontology. So if two genes are represented in the same cellular function in gene ontology, they would be connected in this graph. ontology they would be connected in this graph and so we exchange this graph with a couple of alternatives and we retrain the entire architecture so from scratch and evaluate the performance against the original and then if we for example go to regulons so instead of gene ontology we use regular collections that just collect transcription factor target gene interactions two of them are called dorothea and collectory and the other colors are just unions regular collections that just collect transcription factor target gene interactions. Two of them are called Dorothea and Colectri and the other colors are just unions and intersections of these datasets. Not terribly important in this figure. Then we see that the training efficiency measured with the mean square error where lower is better in this figure really does not vary a lot between the gene ontology representation and the grn representation and then what was even more surprising to us was that if we replaced the prior knowledge with a random graph, in this case Erdos Renyi, simulated graph of different sizes and connectivities, then you would even see more similar performance in many cases of examining the retraining of the model. And now coming to the infrastructure part of the talk, this is just one example of the very complex tasks that we have to undertake in modern drug discovery and related fields and there are several other complicated tasks that we can imagine but there's also this other side to the job which is the more tedious menial tasks that you just need to do. And depending on who you ask, this may even be the majority of the time that you spend in your working day. And this, in general, motivates us to try to develop frameworks that get you to be able in a quicker way. We, of course, want to help with the complex tasks, but I think the trivial ones are the ones where we can more effectively make a difference, particularly with the software stack that we have now. So the first framework that I want to introduce is called BioCypher. It is used to organize and ground knowledge. The paper was in a somewhat grandiose manner called Democratizing Knowledge Representation. But what we essentially want to do is we want to maximize the accessibility of these knowledge representation solutions, not just to the big companies that can hire entire teams for their knowledge representation needs, but rather to anybody who has this problem. And in brief, this is an open framework to reduce redundancies in data curation and management by providing reusable components of these workflows, increase reproducibility by making this all containerizable and, for example, adding it to the Neo4j docker container. Reduce time spent bug hunting in your individual scripts because the framework is end-to-end unit tested. And also ground and harmonize the knowledge based on ontologies and symbolic abstraction, as we have just heard before the session. And this is one example, pseudo-configuration of this grounding which i will get to in a minute so this framework is modular in multiple ways the first modular component is the input side so any kind of resource that you have whether it's on the web or it's some flat file that is on your data lake etc you can pipe into the framework using what we call adapters which are essentially python etl modules and the second component are then the ontologies where you can select from the wide range of available ontologies that we have and we bring these together using this aforementioned configuration, where we say, for example, we have a knowledge graph that is supposed to represent some part of the open targets data set, where we have genes and diseases, and these are connected by edges that the gene is perturbed in this or that disease. And then the third modular component is the output side, which is procedurally very important because it allows you to maintain and analyze your data in the format that you would like, for example in a Neo4j database, but then also archive and share specific versions, for example if you want to publish a paper. And this redundancy that we want to alleviate in some fashion relates to the fact that many of these efforts are often in isolation. And even if they are open source, it's not really clear how to find these things. So what we do is we maintain on our GitHub this project board with all the components that we know of that use BioCypher in some way. So we have these pipelines and these pipelines use individual adapters for individual granular knowledge sources and they use it to produce some output and they use some ontologies. And this is just a way for users to find, in their area of research, has something already been done with my resource, or do I need to start from scratch? And we also build a BioCypher knowledge graph out of this project board, which can then be used to, for example, look at the different areas that are represented, like proteomics or metabolomics. And given that you believe me that BioCypher serves to increase productivity in this kind of knowledge management area, then we encountered the second problem, which is that knowledge usually isn't everything. You still need to use it effectively to draw some insights, to generate some ideas. And together with the recent success of large language models, this inspired us to create a second framework in extension of the first one, which we call BioChatter. And this revolves around this conversational circuit that we have come to get used to in the recent time, where you have a primary language model that responds to a researcher's question. And then we can enrich this conversational loop with all kinds of information. So for example, as Alex has already intimated, we can tune the prompts because we know what exactly we did to the data that we generated and we can inform the language model what we would like it to do more specifically. But I think more interestingly nowadays is having integrated software solutions for retrieval augmented generation that also thankfully was already introduced. So we, for example, obviously connect natively to BioCypher knowledge graphs, which you can use for long-term memory of some agent, or you can use it to inject prior knowledge into your conversation, but you can also use vector databases and then use unstructured knowledge in the same way. Now language models are pretty good at also parameterizing other softwares by just learning the API and then coming up with some sensible parameters. So if you want to use external software, you can also chain to these. And then there are many advanced use cases. For example, you could implement a fact-checking agent that looks to a second language model that does some stuff and then comes back to you with an estimation of whether the response of the first one was actually useful or not. And you could even do things like modality integrations or talking about images and these kinds of things. In the same way as with the first framework we want this to be as accessible as possible so on the extreme end we have a user with no programming knowledge and those would usually use one of the graphical user interfaces we have two prototypes or excuse me, to reference implementations that developers can use to create their own solutions, one of which is in pure Python that's called BioChatter Lite and the other of which is a relatively advanced use case in Next.js, which connects to a REST API server that can also be used in isolation. And if the user is proficient in Python, then of course they can also use the library directly. Now we have heard already that there often is some concern about whether these language models actually do what we want them to do, or whether they confabulate some things or they go in the wrong direction. And this is why in combination with the open source nature of the framework, we also have a living open benchmark in the framework that we continuously execute once new models come out or if we have a new function that we want to test. And this is essentially a PyTest matrix of all the models and all the tasks and the different quantizations, for example, of the open source models that then compares their individual performances and here we can see something not very surprising for example that open ai is still in the lead when it comes to comparing to open source models but some open source models for example this open hermes model are closing the gap noticeably and it does seem to depend a little bit on the quantization, how effective the model is, but there seems to be somewhat of a plateau around four or five bits, interestingly. And the size of the model is not the primary determining factor for the performance. So the bigger LAMA3 model, for example, still does perform worse in our specific biochatter benchmark than this Open open Hermes model, which is a mistral-based model that was fine-tuned on GPT-4 generated data. And then we have some very surprising things that we observe. For example, due to changes in behavior, maybe changes in the system prompt, we haven't had time to look into it, but these new GPT-4 versions that recently came out should be at the top, but they actually drop quite a bit in performance in our specific tasks, which probably can be addressed by prompt engineering. But it's nevertheless interesting that the newest GPT-3.5 Turbo is still among the top performers, but GPT-4 kind of drops down. Of course, we also benchmark the specific functions that we implement between BioChatter and BioCypher. And in this case, it's this text to cipher generation to query the knowledge graph for knowledge graph rack. And for this, we use this configuration that I've mentioned before, or actually a slightly modified version of this configuration that is created by BioCyCipher as it ingests the data and creates the knowledge graph. It also notes down some interesting things about the schema of the graph in this configuration, and this is then passed to BioChatter to parameterize the prompt to the LLM. And if we give these same LLMs, and these are all the LLMs that you saw on the previous slide, if we give these the same information, so the schema of the knowledge graph, the user question, maybe some context or a role description, and we compare the naive query, meaning we give it all this information, but we just put it into the prompt, query, meaning we give it all this information, but we just put it into the prompt, versus we use this implementation in biochatter, which gives it to the LLM in a specific way and order using this configuration, then we see that we have a dramatic increase in performance of the text-to-cypher performance as compared to the just put everything in the prompt process. just put everything in the prompt process. And then one very important component of all the work that we're doing is that we really need to push this open science direction of addressing this ecosystem of knowledge management and usage to go against these traditionally siloed efforts where some people may have questions that other people may have already answered and to kind of enforce this early sharing transparent interdisciplinary collaboration which I think if we want to make a difference we absolutely need to do and also to allow independent validation and then iterative improvement on all these processes and this means to some extent that we need to do and also to allow independent validation and then iterative improvement on all these processes. And this means to some extent that we need to chase additional metrics beyond what we usually do in science, which is looking at the publications, but rather go the way of probably more the corporate life of looking at real world adoption and reproducibility, the quality of the code and the documentation, etc. And taking all of this and wrapping it into a vision of what we could do with these GenAI and knowledge representation stacks, I like to think of it as somewhat of an integrated research environment, similar to the integrated development environments that we can already use programming where you have all of the tools that you require in a singular place that you can easily access and to illustrate I want to go just briefly through this process of a scientific discovery so if you have a new question, if you begin a new project, you often have to catch up on the background. You need to look into the literature and maybe into databases. What is already known about the phenomenon that you're looking into, you need to get caught up. And for this, the retrieval mechanisms with vector databases and knowledge graphs and then having a conversation with an LLM about catching up on the state of the art is very useful. Then once you have caught up, you probably want to design an experiment. And here you again refer to the data that you have, but also it can be very useful that the LLMs are very flexible in the personas that they can inhabit. So if you are, say, a pharmacologist, you maybe want to talk to a statistician. Whereas if you're a computer scientist, maybe you rather want to talk to a medical doctor in making your experimental design watertight. Then it comes to the application of the experiment. If it's in silico, then the LLMs can probably help in parameterizing some functions or just generally coding. And then you usually get a ton of results with these new modern methods that we have, which also means that the results in themselves become a high dimensional problem and here again it's it helps to go back to the previous knowledge that we have and then interrogate the language model for example with regard to the consistency of your result with previous results does it contradict something that we thought was true before etc then it comes to structuring and critiquing your dissemination where again the personas can help maybe you'd like an editor to look at the order of your findings and then at the very end of the scientific process, we are also thinking about ways to improve the peer review process in itself and then maybe help reviewer number two to not trash your paper in uppercase insults but rather give some constructive criticism. I'd like to mention one project that we're starting now due to funding that we've received from Open Targets which is called Open Targets 3088 or Automating Knowledge Management and this will be an open source project that concerns the entire knowledge engineering cycle that we know from the extraction of information identification identification of entities, relationships, etc. from text with a package that is supposed to complete our ecosystem for now. That doesn't exist yet, but is planned to be a result of this project. And here we have the fortune of working together with Europe, EMC and Campbell at the EBI with one position each. Then we want to create a flexible knowledge graph platform for the community where they can use open targets and other data and dynamically create individual knowledge graphs that also include, for example, private data in each case, and then also have an integrated conversational AI using the biochatter package and going to downstream applications. And then there we have this stretch goal of attempting some semi-autonomous human-in-the-loop applications, model chaining. So we would connect to the knowledge, maybe the knowledge that is extracted from papers. We would use the LLM to call simple models, but also more complex models. For example, the ones that Mo is developing in a parallel project for predicting perturbation effects and have a connection to a conversational AI that the user can do this in natural language. And then the stretch goal, as mentioned, to maybe do in situ, in silico experiments, if the questions or answers that we get in the conversational interface lead towards this direction, then we can ask what would happen, like what would be the prediction if we target this particular gene in this disease, what would be the transcriptional outcome or something like that. To conclude, these are the short-term plans for the ecosystem. So we want to work on strengthening the community by crowdsourcing features, but also feature requests. So we need to find out what does the community need? What are the use cases? What are the actual requirements that people have? And make the project more independent of the individuals, the lead developers that currently have. more independent of the individuals, the lead developers that currently have, and develop a unified API for this projected ecosystem, obviously create the BioGather package and make it consistent with the other two, and then having one sort of language to interact with the knowledge across these packages and connect more closely with the machine learning methods circling back to the what i said in the beginning for example to connect more intimately with these deep learning methods like if you want to create a graph neural network from a specific data set that you have about the clinical implications of a cohort that you had, it should be easier to get to a structure, to a particular neural net that you can then run on your data. And also to increase the robustness of the data management and machine learning, which traditionally has been rather on the side and more emphasis on the algorithms, I think it should be maybe equal effort in terms of robustness. And with this, I already need to thank all my colleagues, contributors and also the sponsors to Neo4j for inviting me, to you for listening. for jay for inviting me to you for listening and since you can't really hit me up in the break feel free to get in touch in any way that you feel is appropriate um my dms are always open and you can find everything that i talked about under these links here thank you links here. Thank you. Thank you very much, Sebastian. We have a couple of questions online. I wanted to make sure that everybody in the room also takes a chance to ask some questions. So if you have any questions, raise your hand or use the slide and it's fair for everybody to... The window closes. It's fair for everybody to... So I have a look up here. Sebastian, the first question comes from Georgios. He asks, what has been the main challenge during building the text to Cypher component? And what do you expect will the main challenge the users may face? challenge the users may face? So I think there are multiple dimensions to this. One of which is obviously which language model you use. So if you for some reason need to use an open source model, then obviously you're constrained by the performance of those. And some of them really don't really understand what the user wants from them. That's just a matter of fact in the early stages of open source LLM development. So the main hindrance is that we can't expect an LLM to respond the same way that we would expect a human to respond, like a person that knows cipher and we say like hey do me i want to identify this pattern in the graph and we kind of assume that the model thinks about it in the same way as the human i think we can't do that and we see that the prompts play a huge role and there are more we are integrating at the moment more modern way of generating prompts, for example, DSPY, where you need a metric. But then if you have that metric, you can basically do machine learning to generate the ideal prompt. But yeah, then you need to define a metric, which you would have to do in each individual case. And then this also you need to do again for each LLM. Like if you change LLMs, then you also need to update the the prompt generation but yeah practically the biggest problem is that the llm actually understands what you want and then also understands the graph yeah i think that's that's a good point sometimes you you give it some information, and it's just, as your image showed very nicely, some garbage comes back. And it's like, yeah, OK. But I think humans actually have the same problem, to be honest. Humans also don't understand what you want sometimes. That's a good point. Yeah, you think something is very clear in your mind, and you tell it to somebody else. And then they tell it to you back and then you realize oh they didn't understand at all what you were saying so yeah can happen I have another question from Magdalena she says what role do you think biochatter can play in addressing the issue of irreproducibility of published research I'm thinking in the direction of validating new publications against current knowledge hypothesis mentioned in other papers etc yeah that's one of the reasons why i'm very excited to be directly involved with european cena because this is one of the main concerns and they have been doing so with traditional methods before but now obviously we're looking into characterizing the publication landscape more dynamically or more extensively because previously if a human had to look at it there was no other way now we can use an llm so essentially projects will be created in the next months that look at embeddings of the entire corpus of the EuroPMC, looking at particular flags, looking at similarities in this embedding space. Are there particular criteria where we can say this is fishy paper, right? This looks like a paper mill paper or something like that. And then we can, in a second step, go towards these problematic areas in the embedding space and take a closer look at these papers, maybe also with the help of LLMs just to allow Europe PMC for example in in the submission process already to flag some papers that may be problematic from from one perspective or the other so definitely I think there can be much progress if the right people are involved. Again, invitation to everybody who wants to be involved. The biochatter community or biocypher community is open. And just like you said, Sebastian, ping you or just join the website. Maybe we squeeze in one more question before we continue with the next session. Right, Alex? Do we have more time? Can we do one more question? Sure. Yep. There's one from Rafsan and they ask how do you aim to optimize runtime of llms for biomedical entity detection and relation extraction we are working in a similar space and runtime is our biggest bottleneck at the moment yeah so i think it's a a very pivotal question for the near future of implementing these things at scale. I'm kind of in a convenient position when I say we don't really worry about this problem. We are not LLM engineers. We are just aware of this problem. And of course, we keep tabs on the current proposals, for example, parameter efficient training, whether it's post-training or whether it's parameter-aware, quantization-aware training. I think these look great so far, particularly in the open source space. But these really are inventions of this year, so I expect a lot to happen in our pipelines, for example, in the Open Targets project and with the Europe PMC pipelines that already exist. I think a multi-step procedure is the most useful thing. So if you have a dictionary-based approach that can already take care of 90% of your entities and that has a small economic footprint, then you would do 90% of your entities and that has a small economic footprint then you would do 90 of your data set using this dictionary based approach and then hopefully your approach is calibrated so it can tell you i'm not very confident in this particular entity maybe someone else should have a look at it and then the next stage is escalated to maybe an lLM, maybe a smaller, like not LLM, but still deep learning model and so on. Like doing this, I think people call it deferral architectures, where you defer only the complicated, only the problems that the simple system can't solve, you defer those to the next stage. And I guess it depends a lot on the problem how exactly this deferral architecture looks. Cool, super. Thank you very much, Sebastian. I think, Alexander, you will introduce our next speaker. But just wanted to share with everybody, because the question came up a couple of times now, we will record this session. We will share the slides with you afterwards if we are allowed to. we will definitely share the links to the recording so um don't worry about it you you will you will get that that content uh afterwards and yeah i think with that we we continue with with the program uh our next speaker uh will be introduced by by alex it's gonna be gusf. Can you hear me? Am I on? Okay, cool. Yeah, the next speaker is from Jon Snow Labs. Probably you have heard about Jon Snow Labs before. They are experts in natural language processing. Obviously now also digging deeper into the Gen-AI or LLM part of that. And they will show an exercise how to use LLMs in order to automatically integrate the data from unstructured text, for example, doing named entity recognition and then, as I said, ingested into Neo4j. And I hope you're already online. Yeah, I'm right here. Perfect. Good to see you. Handing over to you. And you have my screen, I guess. Yes. Hi, Alexander and everyone. So I'm working for Johnson of Labs as a data scientist. And I will discuss our application. We use the opioid research. It was it started as an internal project, but it is it's an epidemic right here in the United States, probably similar in many parts of the world. So this is the motivation for the opioid research. And we trained models, multiple models actually. I will give you an idea about these. And then we use the knowledge graphs to establish the relationships between different nodes. So I just remember from the previous presenter, so I'm working for Johnson Hall Labs before I worked for different data and data science, data characterization projects, the history in military, mostly, again again for materials research and characterization. So right now I'm working in healthcare. So again, I assume that you see my screen. So this is what we do actually at johnson labs and you can see that the entity recognition we call it uh ner named entity recognition and it's uh you you're gonna hear this word a lot we use this phrase a lot so what you see here on the screen for entity recognition is we start i mean again we are working nlp and llm so we start with I mean, again, we are working NLP and LLM. So we start with unstructured text. So just the clinical reports, mostly medical reports, radiology reports, but the initial starting document is PDF or just we use the OCR to get the text. So there are a lot of words in a sentence. We use our ADER models, deep learning models. Most often we have more than 2,000 for healthcare applications, 40,000 in total. And again, from the application or from the example that you see here, it's like you see that the dosage information is extracted for 40 units, and it's labeled as dosage. And then insulin is extracted and labeled as drug, and then the frequency. So we start with the text, and we get the entities which are of importance for this application. In this case, it may be like pathology, drug-related entities, plus the symptom or the disease. In the other case that I'm going to discuss today, it may be just the opioids and effects,, in the hospitals make the, just generate these reports and then we get the NERs, then we get the entities and then link to some standard codes. As you can see, again, the entity linking, you can see that we use SNOMED, RxNorm, ICD-10 codes mostly. The insurance companies need these, so it must be quite accurate. And then assertion status. Assertion is very important because the, like, let's say the patient is an old man like me, he may have a long history of illnesses, but it may be the, not a chronic, but acute illness. Like 20 years ago, he had some problem. And so we have to understand, okay, the NER model is going to get you this sickness, symptom, disorder, anything, label it. But it's not the case anymore. I mean, the person just get the drugs, use the drugs, and he doesn't have this illness. So we use negation or assertion for this one to understand if the symptom or disease is still valid. So it's going to be either present or absent. And in some cases in the clinical report, the patient is going to say that, OK, my father had the disease before, so it's going to be someone in the family. So that's important to eliminate some of the entities in the report. And then on the right, you will see relation extraction. We use it a lot. And we used it in this knowledge graph case. So OK, we have to find the relationships between different entities. Or NER model gets the starting point is mostly the NER model. The NER model is going to get you the entities of interest and then you try to establish the relationship like the drug, name of the drug, and then the dosage or frequency. And the identification is very important here because we have a lot of reports. We are mostly talking about hundreds of thousands, millions of reports. So the HIPAA rules are very strict here, same in Europe, maybe even stricter. So you have to get rid of the identifiers first, like the patient's name or the doctor's name, the hospital's name, date, social security number, and all these. So, in the medical record number, so we use the identifiers for these and then Q&A and summarization. These are LLMs. We use LLMs in this particular project. We use the deep learning model to get the opioid-related entities, but using LLM is also possible. And I guess the technology is moving in this direction, as everybody knows. So we are working on fine-tuning models because we don't want the LLM to hallucinate. So we fine-tune the models on the medical data. And on the left-hand side, OK, accuracy is crucial in health care. You don't want to make mistakes. And scalability is important. As I said, I mean, any NLP model, you can train and use it for 100 documents, but it's not going to work for millions of documents. So that's why we still use the deep learning models efficiently. And I can give you some benchmarks for these. How long does it take to run the pipeline on like a hundred thousand of documents, and that's why or why you still use the deep learning models. So again, I'm going to share this PowerPoint with everyone. As you can see, we provide these services, and I will advise everyone in the research to use this. I guess our company does not charge any fee for using the library for research, so I will advise you to take a shot, and there will be trainings. It will be in mid-July, so maybe one month away. It's going to be two plus two days total for like four hours a day about using this library. So why? I mean, why we use all these models and what we try to do? The problem is, as you can see on the left-hand side of the slide, there are a lot of data sources. So the technology, the smartphones, and all of these reports. So there is a constant flow or flood of data. So they are coming and they are meeting at the middle, as you can see, EHR or electronic health reports. So there will be a lot of EHR. I mean, as you can see, there is a rain or flood of information coming from the lab, let's say, going to the pharmacy. And then the lab makes the analysis, then sends it back to EHR again, which is the meeting point, and then go to the radiology, to imaging, and then comes the result. And then it goes all the way to clinical visit to clinical trial and then it will be constant change. I mean, the data never stops and we like it. I mean, we want data. We just want to analyze it. The basic problem with healthcare data is, I mean, I put some examples on the right and again, some triage notes. We know that I mean the there are a lot of abbreviations, typos, the healthcare personal doctors or registered nurses or other healthcare personnel they are in a hurry. They have to write a lot of things. They have to make a lot of notes. So there will be a lot of abbreviations. So OK, they understand. But we want the natural language. I mean, we have to understand this. I mean, the model has to learn from it. And the other one is there will be typos like yesterday. OK, you can't use a dictionary here, or like a text matcher which uses a dictionary. So that's why we use the NER models. And then Tylenol, basic painkiller. You can, again, make a typo. But the deep learning model is going to understand that, OK, it means probably the intention is here, again, this drug. So the model is going to label it as a drug and a lot i mean uh it's fuzzy so we use a lot of different embeddings here our basic embeddings model is 200 dimension and we use bird models also some some nr models are trained on birds some are some are trained on our embeddings model it's the the model the the model that we use is trained on birds, some are trained on our embeddings model. The model that we use is trained on healthcare data, so it's healthcare-aware, I mean, specific to healthcare. And again, at the end of the day, we try to understand the aim is to understand the symptoms, the diseases, get the, let's say, the standard codes, and find the relationships between these. And again, the data never ends. So that slide is pretty good in telling what I mean. Again, you can see the example sentence here. This patient, female patient, 28 years old. She has a history of type 2 diabetes. Eight years ago, diagnosis was eight years ago, and she is on metformin with the dosage and frequency information. So I'm going to show you the pipeline, a basic pipeline or NER pipeline. So normally, the NER model, you have to get the sentences first. There will be the clinical report maybe 10 pages long, but first you get the sentences, split the sentence. We have multiple models for these. There's a deep learning model again for getting the sentences, or again, it may be rule-based. It may say, okay, if there's a full slot, a deep learning model again for getting the sentences or again it may be rule-based it may say okay if there's a full slot it means that that's the end of the sentence but deep learning model performs much better then we get the tokens by using the tokenizer and then we are going to use the embedding so in any our models you're going to use the word embeddings in in the icd-10 or snomet or the standard codes you need the sentence embeddings. So you're going to use a different embeddings for getting these standard codes. And assertion, I mentioned it. I'm going to show an example about it. So it's important. Maybe this female does not have the diabetes. Maybe the diabetes was in her family. So we have to make sure that the disease or the symptoms belong to the patient. So we have to make sure that it's if the disease or the symptoms belong to the patient. So there will be a lot of information at the end of this pipeline. You will see that, okay, we have the, all these, I mean, I marked them in different colors. So we get the, and the, the you can see that the labels are right here. So you get the important tokens, merge them, make them a chunk, and then label them, age, gender. And here comes the accuracy. And then if needed, you're going to get the standard code. So this is like the JSON file. You may also have the option to get the CSV file. So you can see that you will have all the information that you need. So the rest is post-processing depending on your application. You can see that the highlighted verbs or chunks are important. You may say, okay, left kidney was not marked because the model did not work on this. There are models which work on the anatomy, so it's going to get the body part. So it is not extracted right here, but it is possible to extract it quite easily. And OK, medical QA, this is where we use LLMs. As I said, we use multiple models here. We trained on medical data. The good news with this one is it's constantly chat GPT-4. Excuse me, GPT-4. We are every day making analysis to understand if it's performing better or if it's making mistakes. We can say that, okay, the good news is we get the references or, I mean, we try to give the user the references so that it's going to be like a huge information for this. Looks like we lost Gursa for a second. We'll try if he comes back in a moment. It looks to be on his side. Well, we can dance. No, better not. I'm hoping he will be back any second. Hopefully he realizes that he lost connection so that we can reconnect with him in a second in the meantime maybe I used this this time to to highlight a few if you think exactly this slide a few a few things that are available for you to to look at once you are finished with today or generally interested in more Neo4j stuff. So we have an active community, we have a community forum, we have a Discord server, so whatever you prefer you can chat with the team from Neo4j but also chat with the wider community on the graph side of things, exchange ideas, exchange questions, get answers, and talk generally about what you are currently working on and find other people that maybe are doing the same or have similar experiences or can help out with questions you are stuck with something. A great way of learning is the Graph Academy. So no matter if you are more on the beginner side, just want to start with an intro to Neo4j, a fundamentals cipher course or some data modeling lessons, you can go there. But you can also do more technically advanced sessions. So we have an LLM course now, obviously, how to build a chatbot with Neo4j. We have a vector search has been added last year, if you're not aware. And our colleagues from the Graph Academy have added a specific vector course how to work with these in in the Graph Academy. So there are lots and lots of courses, also specific ones to data scientists. So if you're more into data science around, and I think that's that's a cool a cool way of you know learning things and in a mix between hands-on and and videos and and content to read so I think that's it's really nice and then if you want to try out new for J yourself I mean we have lots of options for you to do that you can download desktop which is a like a little suite that comes with you know you can deploy your Neo4j instance locally on your machine and play there with something with bloom with visualization with some add-ons or you just use Aura which is our graph database as a service offering where you just you can get a free instance it's limited with you know small small number of nodes and and relationships but the small number actually is not so small so i think it's 200 000 nodes and 400 000 relationships so you can actually get somewhere relatively decent in the free edition and just play with data play with it use data importer which is a pretty neat tool to get your data from an existing csv file into a graph model pretty easily. It's a very nice drag-and-drop module. So you pick your CSV file, you drag and drop onto an empty canvas how your nodes, how your relationship should look like, and you map that on from your CSV file into a graph, and then you load it back into, or actually load it into Neo4j. And it looks like we lost Gurshev for good. He is not coming back, so maybe his internet is completely out. So we can answer some question from the audience here or from remote. Otherwise, maybe we skip ahead one session and do the session with Katja and Marius next and see if Grisev joins back in and then we can either patch him up again towards the end or we'd have a little bit of a longer break today then. So I don't know, Alex, what do you think should we do? The audience here, because you are here. Should we, because the next one of Open Study Builder will be a recording because Katja and Marius are at NOVO today. Is that OK for you and then do the break or should we do a break now? Continue. All right. So in that case, then, do you want to say a few words? Yes. So NOVO Nordisk came up with the idea of basically doing more clinical trials with the same kind of stuff in the next couple of years by adding terminologies and talking standards from the very beginning, from the design of your clinical trial talking standards until basically you get the approval by the FDA or you hand in for the approval and that's an open source project called Open Study Builder that is based on Neo4j. That's basically it and can handle very complex data in the clinical space and and as i just briefly mentioned unfortunately katya and mario's um katya is the the um the project manager there and marius is from our professional services they are at novo um today so they recorded it and i think we can show the recording now We can show the recording now.... which is typically created in Word. And then we're having different users...... like to have, so the Open Study Builder Vision is really to replace the core information, all main information which are needed by different systems in one metadata repository. So that this metadata repository is used at the heart and then we can derive the protocol from and deliver different systems. Of course, this metadata needs also to be maintained and created, so here still human input is required, but we also are supporting standards. So it's a lot of things are driven by standards. We do have industry standards like CDISC, but also there could be compound standards, study standards, and also what's another important point is that we're not only having exports to systems, but if system information should be used also in a different place, there's also the root back. So systems can automatically also deploy information into the Open Study Builder so that it can be grabbed by another system. Open Study Builder so that it can be grabbed by another system. The Open Study Builder itself contains three core elements, which is the clinical metadata repository additionally to the study definition repository. So we are not only storing metadata, but also study definitions. Which studies do we have? What activities do we collect for them? What are the objectives, the endpoints and much more. Then we do have the application, which is the web interface. Here people are logging in, then they can see and browse the standards. They can define standards and they can use standards. They can create studies, they can select which studies and activities they would like to have, they define objectives, endpoints, titles, all information can be entered in the application. The third component is the API layer. So this is the application programming interface, meaning whatever is available as data or as tasks which could be performed are also available through this layer. This means that other systems can just access the data or create data. It's all automated possibilities. We also have a high need for interactive interactivity and interoperability. So a core component is the import of standards. We can import the CDISC like CDASH, also biomedical concepts from the CDISC Cosmos initiatives. We can also import dictionaries like SNOMED, MEDRA, LOINC, UCAMP. But yet there can also be importers to import standard sponsor libraries. For example, if you already have a metadata repository, you can create a script to just import also all your metadata into the OpenStudyBuilder. And on this other side, we do have consumers. So the idea is that every system we are using, whether it's a protocol add-in, whether it's the clinicaltrials.gov, whether it's an EDC system or also tools to create STTMs or TLF builders, they all can grab information through this Open Study Builder API. Another important point to mention here is that we are also a Transcelerate digital data flow project, meaning when you have a system which is compatible with the Transcelerate DDF standard, it will also be compatible with the Open Study Builder. So the idea is really to have exchangeable systems. For example, if you want to switch for one study from one EDC system to another, that should ideally be working by using the same APIs that would be possible and maintaining the standards in the Open Study Builder. So, as mentioned, the Open Study Builder is an open source tool. It's developed and maintained by novonordisk it's highly linked to standards and initiatives like cdisk standards we are a member of the cosa so the cdisk open source alliance we are an in we have the open study builder is an implementation of the transcelerate digital data flow project and we are looking forward for collaborators we are looking forward for collaborators hey so now um i would like to cover this question you see on the screen and that we actually um get quite often and it's also probably the reason why um you are sitting here today uh on this on this growth day farmer talk so why is there a graph inside of the open study builder and also where actually as you saw in the in the previous slide from from Katia she used the word really linked with standards so that's already a hint for what's coming so let's look at the data that we have inside of the metadata repository and the study definition repository. Let's start from a study. So that looks like a single object, but if you zoom in on exactly what you have inside of the study in terms of components, you will have many components. Let's take a first example, this one, the objective. So the study objective, as you know, will be measuring something about something. So you need to zoom in a bit more. So for example, the objective will have a category which can be the safety or the efficacy of a given molecule over a certain disease, for example. And what I just said, molecule and disease, we actually treat them as parameters to the objective. So we use a lot of templates instead of the OpenStudy Builder, so that you can pick from a lot of templates instead of the open study builder so that you can pick from a library of templates instead of always having to come up with your own content and wording so parameter here as i said can be a compound can be a disease can be units etc so these parameters can be units, et cetera. So these parameters, they also have allowed values inside of the data repository. And these allowed values, they can come from standards like CIDISC or it can be sponsored data. So for example, compounds owned by your company or it can be specific activities that you lead in the context of your inside of your medical therapeutic area so for example measuring the level of glucose in the blood so you see already we are heading toward something that looks a bit like a like a graph and then we could zoom in more and more and more and some objects are actually um composed of literally dozens of elements so if i look at this that we call the soa so the schedule of activities which is really at the core of the of the open study builder if you look quickly here you can see that it's the composition of many elements so you will have activities that happen in the context of a visit in the context of an epoch you can enable them disable them and each of those are actually made up of several objects themselves so if you if you go on and on like that you end up with the following which is our data model for the whole open study builder open study builder so as you can see there are a lot of of relationships everywhere so if I rephrase that congrats it's a graph so that's that's really the starting point for why do we even need a graph in the in the first place inside of the open study builder and as you saw here in the schedule of activity um the fact that we have a graph as the underlying data structure inside of the tool it doesn't prevent you for from displaying data in this tabular format that we have been used to for many years as also the data storage structure. But here we can really need a graph because everything we do inside of the open study builder is actually versioned and in top of being versioned you also need to preserve what we call an audit trail so history of exactly who did what when and why so let's take this example we have a root node which is a term coming from C disk and then it has value it has versions that can evolve over time which which is what we call the value objects so in 2014 that disease was called simplex virus but then in 2016 they decided that herpes simplex virus should be the right name but then in 2023 they said okay no it should be just simplex virus but in two words so you see it evolved over time and the way we do the versioning is that this part on the left never changes. That's really something which is if you change this, it's just a different object. But then if you change any of this, it's just different properties that evolve over time for this given object. But it's still the same object. So let's play a little game now. Which of the three versions earlier should you choose in the context of your study objective, for example? So you're working on a drug to heal this simplex virus. So let's say you start the study today or for some reason you don't care about the dates and values, so you just pick the latest, which is today. But what if you actually created a clinical study in 2015, then you were working update the clinical study to the current data available and we are actually in 2018 for example then you need to switch from this value to this new value here so you're writing the study objective which one should you pick? Well, it all depends on your context and it's all enabled by the versioning system that we have in the OpenStudyBuilder. But then if you add that to what I was showing earlier, then we can say that it's actually graphs, plural that you have. It's several graphs, all intermingled. And depending on the specific date that you choose, you will choose a subpart of your graph, which is the one available at that given date. And as I said, if you decide to change from one value to the other one, you need to explain who did that, when, and for what reason for auditing purposes. So I hope this really shows the power of having a graph as a data structure in the context of the OpenStudy Builder. And now I will pass it back to Katja to learn more about how you can move forward with the OpenStudyBuilder as a tool. So if you're interested in utilizing the Neo4j graph in the real application, like in the OpenStudyBuilder, there are various resources where you can go to. We do have a very extensive project homepage with a lot of information, so it's OpenStudyBuilder.com. For example, under info and resources, you have a very brief overview about the most important resources. We have linked videos, presentations, a poster, a flyer. We also have already created two papers. And on the homepage itself, you also will find a lot of information, like also our next events, various guides on different topics if you would like to get additional information. So please just check out our Open Study Builder homepage. The demonstration video is a real good resource to just get a brief overview about the capabilities and what the Open Study Builder is about. We also have a newsletter in LinkedIn. We do have regular releases. It's typically in about two, two to three months that we will always do a new Open Study Builder release with new features and enhancements. You can also reach out to us via Slack and via email and there's some complete source code, including the instructions on how to install everything is available in GitLab. What you will get with the OpenStudyBuilder is you can install it locally, even on your PC, whether it's Windows or Unix, a Linux machine, it even works on Macs, but you also can install it on a server. There's also a public sandbox environment available. You only have to print a send an email to Neo4j and then you will get access to a public sandbox which is just the playground area so everybody can see browse and edit with the tool. And then there's also the option for a dedicated hosted environment. For here, for example, you can ask Neo4j. The software itself consists of different elements. So you have the application, which is the web interface. The complete content is available in the graph database. And very important are these API endpoints. There is a Swagger online documentation live. So if you would like to browse and see what API endpoints are available, just use the Swagger documentation. In general, there's a lot of documentation. We do have our project homepage. The tool itself is also coming with a documentation portal. And there's a lot of documentation in gitlab available then we do have additional scripts available for example to load the latest cd standards into the open study builder but there are also general tools like a database browser and also neo4j dashboards where you can browse different contents. Yes. And in case you're interested by the OpenStudy Builder, but it lacks a few features that you need, or you want to explore more, or you want to customize anything inside of it, including the data, that's actually something where Neo4j can help you, especially the services part of Neo4j. So for example, if you want to edit the library data that you have inside, let's say you have some custom data on top of the CD standard, then that's something you can customize or you have some other library data available like the activities that you lead or or you use a different set of units that that open city builder uses that's also customizable then we we have also developed mappers so for example katya mentioned the d DDF Transcelerate. So the OpenStudyBuilder can actually export a study in the USDM v3 format. But if you use a different study format, then it's also possible to develop custom methods. And then you might also want to develop complete extensions to the OpenStudyBuilder, because some functionality is missing. And then here I invite you to come talk to us, so Neo4j and also Katia. And that's something where NoVanAllDisk is also open to talk about this, because it is an open source project. So thank you all for your attention. And here are our personal emails where you can reach out to us. And you can also write to openstudybuilder at neo4j.com and we will receive this email with questions or any requests so thank you and have a nice rest of the day cool thank you very much to to katya and marios even though they were not being able to join us live today but i think that was a super cool overview of the open study but it's a nice project so i think as as they mentioned already already, you can participate as well in the community if you like to, and if you just want to try it out, then it's an open source project which is always cool to play with and to try out yourself and to get a little bit into the weeds of things. And I think it's very nice that it is so inclusive and tries to get as many people on board as well as many companies. So it's like a true community effort, I would say. And it's really nice. And like Marius just said in the end, if anything is missing, if you think, oh, this would be a cool feature, this would be a cool addition, if you are like eager to add and to contribute to the project, then I think this would be a cool feature this would be a cool addition if you are like eager to add and to contribute to the project and I think they would be really happy for for you to join in and to to give them a feedback or to join at least for one of the community calls they're hosting regularly and and tell them about what's what's going on all right so that's that I think we have good stuff back. So he had an unfortunate internet problem. So we had to restart his machine. I think he's back online now. I hope that we can continue with his interesting session from Jon Snow Labs. And yeah, here he is. So I think without further ado, let's continue. Maybe rewind a little bit, Gurorsev, where we lost you, so that we can kind of like recap on what we had before, so that we don't lose the context. And yeah, with that, back to you, Gorsev. I assume that you can see my screen. So I'm terribly sorry for that. I was just, I mean, you can see my screen so i'm terribly sorry for that uh i was just i mean uh shopping around the new pc yesterday for like switch to macbook pro so i think it's a decision for me right now the the computer just crashed so i was gonna say uh normally we use the deep learning models mostly to get these entities, just to remind you what I was talking about. So you can see that these colored sections are, we don't use, I mean, not all of the texts are useful. So like a stop word, not a stop word, but we don't need this diagnose term here. So we get the chunk here, a couple of tokens, I mean, label it as age. The model will get you all of these. But as I was trying to say before the connection was lost, it's not always, not all of these entities are of interest for you or of value. the patient, it may be the symptom or the disease or the drug may even be belonging to the patient's mother, father, someone in the family, or it was just an illness 20 years ago. So we try to use the assertion models and then finally we are going to get these standard codes. And for these we use different entities. The end one, it's going to be CSV file or any structure format, but JSON is the most popular, as you can see from the text. At the end of the day, you use the pipeline, run the pipeline and the model, and you're going to get the resulting output is going to be a JSON file like this one. So the assertion said that, okay, it's absent because it says no evidence here. The diagnosis was this hematoma, the code, I guess it's a SNOMED code. And then we get all of these and we are going to use it for the post-processing or the next steps. And it's going to also give you the confidence level. Not all of these, because at the end of today, NER models, using the NER models or named entity recognition is token classification. You get the token in the previous stage, then the NER model is going to say, okay, this may be probably with this confidence level or with this probability, it may be labeled as age, date, like gender. So that's how we use. And I guess I was saying we are using LLM models because all the applications are switching to the LLM models every day. I mean, literally every day we compare our results with GPT-4 and other users. AWS has some medical users right now. So we try to understand if the model is working fine. The models, as I said, we have more than 2,000 medical models, healthcare models. So we try to understand what we are doing wrong. And the medical question answering for our chatbot is trained, does not hallucinate. Because if you ask a question like in Spanish and get him to get the chatbot to answer it, if it's unrelated to medical or healthcare data, it's not gonna answer it. It just says, I don't know. And the other one is we provide the references here, mostly from the PubMed. And so we use the knowledge graph. It's a, I'm sorry, it's a too quick switch. So how do we use it? So what you see here is, okay, there is some CT scan and it was, the date was March 12th, 2010. And all of these are, these entities are, or some of these nodes are extracted by the NER model. So this is the CPT code, which is the standard code for procedures. So the CT scan says that, okay, in the body, in the abdomen area, again, coming from the NER model, on the left-hand side, this is called the laterality coming from the NER model. The finding is, okay, there is a mess here. And the assertion model says that it's confirmed. It doesn't say, the report does not say there is no mess. There is definitely a mess. The radiology report says it. And the modifier says that, okay, stage two is mentioned in the text. Not something some machine learning model saying that is at stage two level, considering all the inputs. It's mentioned in the text not something some machine learning model saying that is at stage two level considering all the inputs it's mentioned in the text and then in your model just extracts it and we the imaging technique was if i don't know if you always need it but it's a content a contrast and for hematoma and the medical text says that it's absent, good news for the patient. But all the ICD-10 codes, SNOMED codes were extracted for hematoma, although it's absent, so we are not going to use them. The date is extracted, the date of the CT scan is, excuse me, hematoma. That's probably something in the patient's clinical file in history. And she used or he used metformin with this dosage and this frequency. So, again, you can see, yeah, we have the report here on the right. So, the reason for this radiology exam was suspected hematoma. And the patient is on Comedin, which is a blood thinner, so be careful with the exam. And the area is the CT scan was performed on the abdomen. So you see that we can go all over these, but these are extracted. You see that these are nodes here, and they were in indeed there are more details about as I'll start stunning starting point for or the starting node is right here and it's the hematoma retro so the CT scan is here they connected the abdomen area pelvis area bladder and you going to see all the, so these are the relationships, or we use the relation extraction models to get these, and the relation models are provided. So, you're going to see that, okay, the details of the diagnosis over here. So, I mean, sudden, so I'm going to discuss what we used. The example or the project that we used was opioids. We know that opioids are prescription, mainly used for very heavy painkillers, but I mean, they have some misuse recently, like last 10 years maybe. And they provide pain relief, sedation, and euphoria. There are natural semi-synthetic and synthetic opioids. Okay, we know that. The problem is the bold words are very crucial here. A high potential for misuse and addictive. I mean, huge addiction and overdose. So what I mean by overdoses you probably heard about this this is uh coming from nih uh so it's a government agency uh so you see that in 2022 the overdose due to opioid was more than 81 000000. I don't get the data yet for the last year, 2023, but it's around 100,000. And the total overdose, I mean, opioid plus all the other illicit drugs is around 100,000. So you can easily call the opioid addiction or opioid use as an epidemic. So it's a huge problem. That was a starting point for our project. So we trained this model, NER opioids. So what you see actually is a basic NER model pipeline. We get the stages. You see that there are six stages here. The first one is document assembler. We start with a text to just make a conversion to make the model understand it. So this is the sentence detection deep learning model. We get the sentences in the second stage. The output is right here. So the tokenizer is going to get the tokens from the third stage. And then we use the embeddings clinical this is the model we use for uh healthcare related text uh specific for healthcare applications 200 dimensional model and once you get the embeddings uh you're gonna then in the fifth stage we are gonna use the ner opioid model so what does model, this model give us? I'm sorry, I switched to another computer, so I'm not going to be able to show you the homepage of this model. But if you go to nlp.johnsonovlabs.com and then you switch to models in this page, you will be able to just search for NER opioid and see it. Every model has a homepage like this one. I just got the predicted entity. So if you run this model and if you do not filter, you're going to get all these labels. The model is going to try to use these entities, label all the tokens in the text with these entities. If it's not, then it's not going to label anything. But you can whitelist. I mean, if you are interested only in, let's say, alcohol use or like legal issue or violence, you can easily get only these entities. And at the end of the day, once you run this model, and the NER converter is going to convert these to chunks. So this is the chunk. You're going to get a, like, remember, we get the JSON file, or this is the Spark data frame. You can convert it to Pandas data frame, so that you can save it as a csv also so you see that uh unmarried was extracted from the text lay and begin and ended indices here and labeled as marital status and there is also a possibility to use a confidence level so that you can define a threshold if you want you may say that okay anything below 70 just ignore them a threshold if you want you may say that okay anything below 70 just ignore them and then all of these are right here there is uh okay this one is good so it is also possible to visualize it uh you don't use the visualizer this is uh ner visualizer that you use you don't use the visualizer in production notebooks but in the initial stages it, these visualizers are very useful. You're going to see that, okay, the model mislabeled any token or chunk, or the model did not extract anything. You will be able to see it. When I was preparing these slides, I ran the notebook and I noticed that, okay, opioids and heroin phenyl oxycodone, they are labeled as opioid drug. And right here, you can see that, okay, opioid drug, again, opioid drug. This one is labeled as test. So we have medical doctors in our team and I talked to one of these, okay, this model is not working fine. Oh, he said that, okay okay it's learning from the context in the first sentence the opioid was used in the opioid drug because it's illicit consumption of various opioids so it's really labeled correctly it's an opioid drug when in this one in this sentence you see that these are all tests the test was applied on the person. So, okay, opioids, same word, identical word, this time it's labeled test test. He said that, I mean, the medical doctor said that, okay, this model, it's a proof positive that the model is working very efficiently. And you can see that the test result is right here. So, as I said, you don't use the visualizer in a production notebook, but you will see that if the model is working fine, missing anything, mislabeling anything, and it will be very useful. The other one is we have three different assertion models for these. Remember, the assertion is very important here. And it's going to give you the default is present. You will try to understand if anything missing here. So present, present, okay. The patient has a history of substance abuse history, okay. It's hypothetical here with clinical signs. So that's not definite here. So you can use the assertion model to get if it's really present or absent or someone in the family. I don't see an example here for family. The other one is you also have the option to use the assertion models, not for negation, but other information. And I forgot to say, so there are two uh two different labels here again opioid it's labeled by the ner model so the ner model is working here but the assertion model is working on top of the ner model so the order is in your model gets the extracts the information from the text labels it and the and then the assertion model checks if it's really present or absent or whatever the predicted entities are for the assertion model. And in this one, we were interested in other issues. I mean, again, you see that this model is working with an NER model again, but the NER model is only trying to extract the general symptoms here. So we whitelisted for the same, you use the same model, same NER opioid model, but we whitelisted for the symptoms and we try to understand if these are withdrawal symptoms, overdose symptoms, or underlying pain. So again, I'm repeating myself, but it's important. So assertion model can be used for negation or other, let's say, labeling purposes. So, RME means relation extraction. We have this simple text here. So, you see that this patient started using lipidur. It's a statin drug for taking care of the LDL, some lipid. It's a lipoprotein. It's a sign of the over-cholesterol in your blood. And this one is the, check the second part of the sentence. So say that the patient didn't like the lipid oracles. There were a lot of adverse effects. So the doctor decided to prescribe Zocor instead of this. It's another statin drug, but coming from a different family. So what we use is, again, the NER model is going to get this one. This one is not opioid model. This NER model just extracts two entities. One of them is drug. The other one is adverse drug event effect. So you see that we try to understand or establish relation between drug and ADE. And we use a relation extraction or RE model for this. So it's better to get this visual here. So again, the same text, but this time we use the relation extraction visualizer. What you see is quite crucial here. So you see that normally you may expect some problems here, but the model is working fine. What I mean is, okay, there are many ADEs here, five different ADEs, and there are two drug names. And the model was very successful in getting that only Lipitor calls these ADEs. You see that it's a binary. The RE model is binary. It's one or zero. And the Zocor, the relationship between Zocor and all these ADEs was zero. So there is no problem with Zocor. So go ahead with Zocor, stop using Lipitor. That's what you get from this. And that's what we used. I mean, it's a long start, but that's what we used in the opioid project. So it's a very nice view. So I'm going to start with what we used. We have the conditions. They are mostly adverse effects of the opioid use differently search for different opioids and pmid means public uh excuse me pubmed identification number of this article so you see that we have uh a little short of 10 a thousand notes here and we have these relationships so the relationship is going to give you article of post by or mentioned in so these are these two are related to the pubmed article and this one is it's a global view it's going to show you okay it's going to get you understand some uh clustering maybe so which condition is around which and the yellow, excuse me, green parts, green nodes are very important. It's going to show you what the research is around. I mean most research is around this cluster. So you're gonna, you may take a look at the green nodes with the light blue or dark blue nodes to see which opioids are of more interest. If you take a look at this one, you see that this is a node about some opioid again. There are five PubMed articles about it and, five different conditions or adverse effects. So, this one is about the methadone. You see that I just focused on the same one. You can work with these, again, cipher codes. So, dolofine and methadone are here. You can see that methadone, you use these all of these relationships here for this opioid methadone uh there are articles a lot of articles by the way and also you can see that these methadone drug drug is uh oh excuse me the these uh ades a uh adverse effects are related to, or you may say caused by this drug. So you try to understand these relationships or they are very useful in understanding. And again, this one is the one that I mentioned before. So you can see that, okay, it's more focused. Alpha-entanil, maybe not too much research about it. You can see that these PubMed articles are right here. They are all articles of this opioid drug. And also, you can see that this one, recurrent respiratory problem probably, it's also mentioned in this article and it's also caused by this probably side effect or adverse effect of this opioid drug. So, they are quite useful in understanding these. And the other one is this one again heroin. We get these articles about heroin. Again, same Neo4j and you may define different property keys here. I just want to show you, okay, what I did. I mean, I just checked one of the PMID numbers and then I was supposed to put this license slide one before maybe. So you can see that, this uh pubmed article id and it's related to methadone so it's working fine so it's a very important reference for for for the users and this one uh this one again is a respiratory problem uh some condition or effect problem, some condition or effect. Again, Cypher, we just wrote the query for just limit by 25 to get more. So zoom on these or focus on these ones. So you can see that many of these side effects are related to these opioids. And the other one is, again, it I like to repeat, but this side effect or condition is mentioned in these PubMed articles. So I was going to say the, don't worry about the title, it's going to say, how we used it in our demo. So we collected data from PubMed. It's a, everyone knows it's just a definition free database. We scrapped them and then it's open to public. I mean, not all of the, not all of these articles are, you don't have access to all of these, but you may, you have access to all of these, but you may, you have access to all the abstracts. So we use this NER opioid model. This is in our library. We get the NERs, filter them by using assertions, and then establish the relations. And then we use the NERs to create this knowledge graph. So we linked by using RE models, getting the NERs first, and whitelisted for certain entities as you noticed. And then in the demo, you're going to see just right now that, okay, we try to get more details like this one. This is the demo that we prepared. You just write the question here. We have the text to SQL annotator in our library. So what it means is you just write the text here, you generate the query here, and once you execute the query, it's going to give you all these. So what I try to do here is, we know that opioid heroin has some side effects, and we try to understand which other opioids have similar side effects. So these are, you see that methadone has anxiety symptoms, dolophine also, phenanyl has other problems in the brain. It's also possible to write the number of these, I mean, how many symptoms you can add more symptoms. And the other one is, this one is more complex. So again, this time, which opioids have most similar conditions? Nothing to do with heroin this time. This time, we try to understand, we try to figure out which two opioids have these conditions. And don't just give me the number, just list all these conditions. So again, we generate the query here by text to SQL. And then once we execute, we see that, okay, morphine has more similar conditions with MS-contin, and these are the conditions. So, it was quite useful. So, I'm going to say the next. So, what in future? You notice that we use the deep learning model in your opioid. I'm going to call back. Yeah, right here. You're maybe you have the question in your mind saying that, okay, why don't you use LLMs? Yes, we did. LLMs, it's also possible to extract all of these LLMs, but the model here is quite specific. This is trained on data coming from opioids and annotated by, again, medical doctors, and then trained on some specific subjects. LLMs have huge potential, but this is very specific. Right now, I'm working on alcohol withdrawal, again, in an ER model, or smoking, menopause, hereditary diseases. These are all very specific subjects. So, LLMs are doing perfect, I mean almost perfect. I mean, saying that you don't work on 100,000 million documents. But if you have a very nice NER model, you will get better results. And as I said, we compare every day. And again, we are fine-tuning. We are using LAMA and Mistral open-source LLM models, 3, 7, and 70 billion. And we fine-tune them every day on healthcare data. So you get access to johnsonovlabs.com and for technical just ignore nlp.johnsonovlabs.com to get all the technical details and my info is right here. Thanks for listening. Again I'm sorry for this technical problem but it's gonna be good news for me. Thank you very much Gurze. Thank you very much, Gurzev. We have a couple of questions for you, so I'll read them out, and then we can see that we go through them. So if you have any questions, use the slide or link for Gurzev, and we can answer them. We have a bit of time left for them. So one question comes from Corey. They ask, do you use Neo4j Enterprise with Bloom Bloom or do you build the GUI on your own? And what, I guess I'll add on, NERs or entities, try to establish the relationships. And in this case, again, we get the side effects by using another NER model. I mean, we use more than one NER model. And then we get the knowledge graph. I guess I couldn't catch the question. That wasn't the question, I guess. Not really. The question was, what are you using to build the graphical user interface for your application you just showed? Are you using Neo4j Enterprise with Bloom or did you build it on your own? Okay, yeah. We use Model Park, Model Park enterprise with bloom or do you did you build it on your own um okay yeah we use model park model park app and it's quite safe and secure so we get the uh again the same stages we get the ners and then knowledge grab and then we put everything in this model park app model park app all right cool and then next question here for you it comes from georgios in your experience what steps could someone follow to predict ade using a knowledge graph so i'm assuming that we get the clinical report of a patient and then we try to predict the ADEs. Okay, ADE is tough, really. I mean, you just saw the example, Lipitor and Zocor. The other one is there will be a lot of symptoms. there will be a lot of symptoms. So the symptom may be totally unrelated to the specific drug use. I'm talking about prescription drugs. So ADE is difficult. We attack by multiple approaches. We have ADE in ER models, so we use more than one in the pipeline. And then we have ADE classification, text classification models. It will be in the previous stages of the pipeline. So what I mean is we get the text, start with the text. We use this text classification model to attack if the sentence includes any ADE. If it is okay, if it is in the sentence, then the next stage will be using the NER model to identify the ADE. And then, I mean, I'm being quite honest here, but ADE is very difficult. That's why we use multiple stages. We use more than one NER model to understand these. And if it's domain-specific, it will be much better. What I mean is, it may be difficult ADEs in oncology, different ones in, let's say diabetes so it's much better to use multiple uh nr models specific to this uh this domain let's say okay cool it's a good question i mean ad is like a problem sounds sounds like it um the other questions were for for alex's initial presentation so i think we we might skip that a little bit or or take it on um or actually maybe we take one now and uh anybody else in the room has a question for gussef before we we let you go looks like no yeah all right gussef thank you very much for for joining today thank you for for the nice presentations and for the um the cool um you know the graphic interface you showed for showcasing what you can do. And actually, the invitation goes out to everybody. Check out the website from Jon Snow Labs and play around with it a little bit yourself and see what there is. So it's a nice story. Thank you very much, Gurshev. Thanks for having me. Thank you. Thank you. nice story. Thank you very much, Gurcev. Thanks for having me. Thank you. And yeah, with that maybe we do this one, Alex, if you like. I think that one was left over from the beginning when you talked about the intro segment and Victor says, what are the main challenges that you have encountered when translating natural language questions coming from the user into Cypher? Am I? Okay. So, there are two aspects, if I understand the question correctly. One is if the LLM translates it correctly into Cypher. This is what we can augment if we input the data model as well. Makes it more precise. I think the challenge is to compete with, let's say, real developers who write Cypher code each and every day to make it more, you know, as an optimized query. I think it's straightforward for every programming language to spit out and output some query. But if it's the most performant one or if it's the most accurate one, that's kind of the challenge. And then to debug that and in the end you want to trust your LLM that it does the job that it should do. That is the main challenge that I see. But also one aspect is that not all the questions that are super sophisticated can be translated into Cypher or in an optimized way. So there's always like simple questions that can be translated into Cypher and performed on a database while others are not so much relevant for that. Yeah, I think that makes sense. I think it's more like a little bit like a sidekick or like a co-programming, co-pilot as they call it. You can ask some questions, you can ask something back and you still have to validate what the return is. Can you use that one-to-one? Sometimes yes, but like you said, if it's a very complicated story, a more complicated query, you might want to validate that with somebody who is more proficient in Cypher. And I think that is, Cypher obviously is a bit of a special case because probably not as much has been written about it. So that's also might be a problem problem so if you do the same with Java you might have probably an easier time you get probably more more more fat more more back that is that is probably you know had been the similar question might have been answered already before that that that answer had been fed into an NLM that LLM knows that and finds that answer for you and then gives you gives you something back if you ask something complicated around cipher that answer might not have been asked before or the the answer has been not hasn't been fed into an LLM so that the LLM doesn't know and then hallucination comes into play or they just say oh I'm sorry I have no idea So that also the LLM is not updated each and every day, every second. So there are new functionalities, new, yeah, new, new methods inside for APOC functionalities that the LLM doesn't know. So also one challenge. Exactly. Keep that in mind when when, as we always say, when working with LLMs. Cool. Cool. Cool. At this time, I think we are at the break slot for you. In Munich here, we have a little coffee and cakes outside. So you can get there, get refreshed, grab a coffee, grab a cake, and we'll take the 15 to 20 minutes. I think we're back at 4.45, so that means like in almost 20 minutes from from now on yeah then we have Dimitri and then we have Dimitri from Kyagin Kaizen I'm looking very much forward to that session on site here so that should be fun if you're watching remote I will play a couple of polls for you to join and we can before we break or or no, sorry, before we continue with second half of the day, we look into the results of these polls. And we can discuss that a little bit. But I'll play them for you. And until then, see you around. Type in chat if you have any other question, anything that keeps on your mind still. Use Slido. We will have some minutes left, as you see, between sessions or at the end to do a little bit more of a Q&A, Alex and I. So we can go through them. Exactly. See you later. See you and enjoy the coffee. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. An update from Australia. Yeah, I can totally recommend Nodes as a conference for technical people or community people, because it's all virtual, it will be recorded. But they are hands-on experiences and yeah people share their expertise and and their codes are super interesting to watch exactly so it'll be fun it'll be like you said online free so if you have any topic you are interested in and you're working with currently around graphs and around life senses or any anything else then please submit a session would be great to have you anybody else is obviously invited to join the event. It's going to take place on the 7th of November. So, yeah, with that, we had a little Slido run in the intermission, basically, and we asked what comes to mind when you think of life sciences and graph technology, and a couple of people submitted something. So here we see the little word cloud here, chatbot. I see explainability, nodes, obviously connections, discovery, clustering, knowledge graph. I think that all is something that comes to mind easily. And it's cool to see that you think of this as well. And we are continuing with this topic almost perfectly well that Dimitri from QIgen is bringing up next but I let you introduce Dimitri. It sounds like you submitted all the answers because I think you will cover basically 90% of that explainability and high quality data as well as the ability to connect it to a chatbot but I don't want to spoil too much. So Dimitri is coming from Kaigen's Digital Insights. We worked together for basically the last one and a half years with a very interesting data set from Kaigen that is now also available on Neo4j, where you can then do cool stuff. And you will also elaborate a little bit on your next part which is the AI part of BKB right so yeah great to have you here and over to you yeah thank you for the introduction Alex yeah so my name is Dmitry Kamayev and I'm a senior product owner in Kaiingen digital insights for data science knowledge base products. And just to give a broader context what QIAgen is, it's a big company that also has a mission sample to insights. So it produces everything from assay technologies, chemicals and extraction kits from wet lab to robots for processing and your samples. But also we are doing bioinformatics and we are doing software. And a big part of it is developed in Aarhus in Denmark. And it is under the part of the QIAGEN business called QIAGEN Digital Insights. And I have to show this legal disclaimer slide. Don't use our discovery products for treating people. It's only for discovery. So going back to QIAGEN Digital Insights, we are a leader in bioinformatics space and it's all here in numbers. So across all our software products, we have like 90,000 active users. Our curated knowledge base consists of 20 million curated findings and we integrate more than 40 scientific and critical databases in in our data products so just zooming in into database products we have several database and knowledge base products that we support and it's biomedical knowledge base HD and it is the same data that some of you may be familiar from using ingenuity pathway analysis it's a graphical user interface tool for exploring relationships biomedical kb hd is the same data that fuels ipa but with interfaces that allow easy access for bioinformaticians and data scientists. We also have biomedical KB AI and I will talk a bit about it later. It is an NLP curated, NLP processed extracted data. And other products is omics soft harmonized database with multi omics information and also we have variant databases such as Cosmic, HSMD and HGMD that we are also looking into integrating into a bigger knowledge graph in the future. So what are the main applications of all those data knowledge base products that we support and solutions? Mainly we are supporting pharma companies in their target identification and drug discovery and biomarker discovery. And we see that some pharma companies are also building their own knowledge bases and then they either use our knowledge graph as a foundation for integrating their knowledge and their internal data on top of our schema and structure, or they can already have an established structure and then the integration goes the other way around. And of course we can see with applications that some companies are building their own dashboards applications to support biologists in the company, and then they are using programmatic interfaces to interface with the data and build their own web applications and dashboards to access the data. So just diving deeper into biomedical knowledge bases. So we have two knowledge bases biomedical KBHD and that is human derived, so all the findings are manually created by scientists by reading through papers and following established curation process and filling in the database and we have over 24 million biomedical relationships that will be accumulated over the course of 25 years of aggregating this data and we are constantly updating it with the new curated applications but of course you cannot curate every publications coming out every day there are hundreds of them so for those for those purposes So for those purposes we use advanced AI processes for extracting the relationships automatically. And there we have over 600 million biomedical relationships. So this is a huge graph of data. And on the left it just shows kind of their visualization of how what is interacting and just basically they are we try to go for physical concepts biomedical concepts such as drugs, genes, chemicals and of course we also use pathways and diseases so maybe a bit more abstract but we try to keep it very concrete and also simple so we integrate a lot of data and we provide this information in many different ways to our users because our main users are data scientists and bioinformaticians in the pharma companies so the preferred way of accessing the data for data scientists is either Python or R so we we provide, of course, those interfaces that allow submission of SQL queries to query the data in a relational format. But also we have REST APIs, and on top of that, in Python and R, we also support extra functions, analytical functions, such as causal analysis to extract additional analytics from the data we expose. But of course this relationship data is more naturally presented as a graph, I will show it later. So we also have presented this information as a Neo4j import dump that users are able to import into their own knowledge graph and work with it. And it is all downloadable to our customers and we are also trying to support a smooth and to remove barriers in this data value chain. So it's both in accessing the data, we want to simplify access to the data, we want data to be also easy to understand and easy to analyze. So we try to productize this data and so we try to work with every step of this data value chain to make the journey as simple as possible. So going into tabular representation versus graph representation. We offer both because we know different data scientists different data scientists have very diverse preferences and some people maybe would prefer to use R and tabular data access. So we offer that and I will show you how it looks. So here is, for example, we have relationship tables, which I truncated. It has actually a lot of more context that's captured. But just for demonstration, you can have one column representing node 1 interacting, and node 2 interacting as a third column. And then the middle column is a type of relationship between gene DARP60 and the function quantity of nervous tissue, for example. But then you want to know what DARP60 really is. Well, for that we have extra entity metadata tables, which you can go from DARP 60 to metadata of what DARP 60 is oh well and you can see that it is a gene and it is a in this case evidence comes from a rat and actually the evidence comes from is the form level is a protein. So extra context is available there. But here's how it looks in labeled property graph format, much more condensed and easier to work with, in my opinion. So here's again, DARP 60 and all causal relationships with its functions and all causal relationships with its functions. And all those attributes that were previously had to join using SQL are now just sit as an attribute on the DARP 60 node directly. And it shows when you try to query the data and work with the data. The queries look much simpler. Just to show you what our Python interface looks like, we have our own Python package called BKB, where you import it, connect to a cloud version of our knowledge graph, or you can also connect to your local SQLite or Postgres, and you can just submit a SQL query here. For example, you select all molecule-molecule relationships for a gene NO1. And you get a data frame as a response. So this is how it looks. But interesting, actually, this is an example from our documentation that I hope we've corrected. Actually, this query is not exactly correct. It's a bit tricky to write the query if you want to extract all NO1 relationships. Well, here, for example, I specified that only node 1 equals NO1. Well, what about... Can NO1 be also as a node 2? Actually, yes, it can. So actually, the correct query would be where node 1 equals NO1 or node2 equals no1. So that would be a complete query. So you have to keep in mind those things when you work with SQL, but it's possible. So we created a new 4j version of our knowledge graph and this is a basic representation of the schema of how we try to organize and what the relationships are present in our graph. I show this slide because the real schema, we need to to prepare you looks like this uh so just uh entities are the same but we have a lot of different relationships between nodes that example like causation correlation this web of relationships. And this is an example of the neighborhood of Eugene. It's a really densely connected network. And when we were designing the Neo4j schema, we were trying to go for a compromise between completeness and simplicity. Because, of course, you can put all of the data, all the context in the graph. But then will it be easier to use for bioinformatician? Does it help them answer the questions? That's a tricky question. So yeah, if you go for completeness, definitely your graph will support all diverse user needs and you have like complete scientific thoroughness. But then you compromise on performance, certain queries would be slower and the adoption it just drops. Because if it takes two weeks to understand the schema and to understand the data, or even if you think you understand the data, you make mistakes, then there is no value in it. So that's why I want to talk right now about design choices that we made when designing knowledge graphs that might be also valuable for you and maybe you can learn from it. But I want to say it is a learning process all the time. I don't want to say there is one way of designing a knowledge graph. I think you have to offer different configuration options and design it to a specific use case sometimes. But let's start, for example, with gene model here. When people talk about gene, it can mean many things. What do you mean by a gene? Is a gene a species specific or not? Do you mean maybe actually a protein? Do you mean maybe a specific isoform of a gene? It becomes very complicated and especially when you're extracting information from a paper. So it's important to have an ontology of genes and at the top we have ortholog level and it shows for example here EGFR it's a across species concept of a gene and then on the next level of gene representation for EGFR you can have species specific versions of EGFR And then now we have a gene as a concept, of course, but then how this gene presents itself physically. Well, it can be either DNA, RNA, or protein, and that's what the next ontological level captures. It is form level and even going even deeper you can have isoforms so this is how we try to structure our relationships because in the paper when we have relationships between genes then it could be on either of those levels so it is important to represent this whole tree. But we realize that sometimes not everybody is interested in this specific isoform information. So that's why sometimes you would actually want to collapse this tree to the higher ortholog level. You just maybe in your analysis you don't need this resolution. And so then you would transform the graph and move the links to the higher level of the ontology. Another design choice that comes up when designing a knowledge graph is directionality. So Neo4j graph it doesn't have multiple directions or undirectional relationships they must have directions and it's tricky for example this protein-protein interaction interaction. There is no clear direction in those type of links. And we went with signifying those two choices, right? You can just say, like, well, let's just link them randomly. Or you can have relationships going in both ways. And we chose to have just relationship one way and signifying that relationship is non-directional as an attribute. And the reasoning for that was mainly that it saves efforts and duplicating relationships when you maybe want to extract data and then when you have actually evidence, one piece of evidence having two links that distorts statistics, or you would need to duplicate and it's much easier to have it represented in this form. Another design choice that we had to take was representation of clinical trial data from clinicaltrials.gov. And it's a very complex data set. And here is just a slide not to be understood. So this is an illustration from a publication called Constructing a Fine and Great Representation of Clinical Trial Results from ClinicalTrials.gov. And you can see it's very complex and again, for bioinformaticians who are trying to answer maybe some simple questions like what what drugs are in clinical trial for this target or for this indication. All this level of detail just becomes an extra hurdle to getting to the answer. So we had to come up with an easier representation for this type of data and we created this hub node called evidence that aggregates piece aggregates three interacting coccolates three interacting entities and that's a drug target and disease and whenever there is an evidence for relationships of drug targeting disease it will be aggregated on an evidence node that will also have as an attributes information of what type of evidence it is yeah and here's an example like here it says it's a phase 3 terminated drug trial. Yeah, so I talked about aggregating relationships, and here's an example of how it works. So, for example, you can have NO1 gene interacting with EGFR on form level and isoform level and using different types of relationships and you maybe have two pieces of evidence about of phosphorylation and P signifies phosphorylation. So, you want sometimes a more condensed view of this to simplify your analysis, and we offer aggregated version of the graph where all the different macromolecule levels just collapse to an ortholog level and all the relationships of the same type again collapse into the one relationship type H. So that's how the transformation can happen. And the same goes for disease ontology, because you can have a hierarchy of diseases and you would want in aggregation process we would then roll up the evidence for a gene participating in a certain disease to a higher level in ontology for disease. And yeah, so with all this, I just wanted to show how difficult and complex it is to design a knowledge graph and um yeah i i we believe that um you um often um it's valuable to try to build your own graph and for that uh you want your own certain customizations and for that we also have uh helper functions to to customize uh your Neo4j graph and create an export according to your specifications. So it's not very flexible right now, so the things that we support now in terms of flexibility on creating your own Neo4j graph export is custom names of nodes and relationships, so it's like labels and relationship types. Custom customization of what attributes on the nodes and relationships are included, again depending on how rich of a context for relationship you want, and whether you want edges aggregated or not, that's another dimension of what you can choose. And sometimes you may be not interested in the whole graph, so you would maybe want just subgraph centered around a certain node, and you can do that. And sometimes you want maybe negative conditions. You want to exclude irrelevant portions of the content. Maybe you're not interested in clinical trials. Again, you can do that. Yeah, so again, schema design is a is a tricky um subject but uh yeah i think very important for uh preparing data to be uh uh easily understood so i liked a a quote that sebastian in the previous talk cited about it's not enough to understand that tomato is a fruit. Sometimes you want also, it's more valuable to understand that you can put tomato in a fruit salad. So, yeah, actually, now with the data, like let's put it in a fruit salad. Let's make a salad. So what can you do now with data in your knowledge graph? Well, here's just one toy example. What about finding positive feedback loops in gene expression? Do those exist? Well, interesting question. And actually, yes, they do. So here is an example of positive feedback expression loop that this query identifies. It would be much more complex to do in a SQL. Imagine writing a recursive query to do that. Also what is simple is querying ontology. Again, you can do simple recursions and traversals. Here, for example, we just retrieve all chemicals that are of type corticosteroid. And just for demonstration, here is on the right side equivalent SQL to do that. I'd rather do something on the top for those purposes. So you can also visualize the networks in 3D. So here is HOPE1 and HOPE2 expression networks of gene N01. And yeah, you can see when we go to hop2, it just really explodes. It's a really dense network. So, but yeah, what value is it really seeing your data in 3D? Well, actually you can think certain clusters have become more apparent. So here's, for example, we can see all the pathways visualized in 3D, and it is just very easy to spot this clustering of core pathways. And if you're curious about how it was done, there is this open source library 3D force directed graph, and it just doesn't take much to do this. Here is two lines of cipher, and the rest is just very simple JavaScript, and you can have this visualization. Another very cool data exploration tool and a way for scientists and informaticians to explore the data is using Neo4j plugin called SemSpec. with biomedical data because it tends to blow up. There are a lot of relationships. And it allows to define strict constraints on relationships and attribute types. And it also collapses those multiple nodes of the same type into just one blob. So you avoid hairballs in your exploration. And as I said, you can also... So you can constrain... So it's similar to Bloom, right? Where you are able to have constraints on relationships. Here you also can have constraints on attributes of a node. So here, for example, we have 1,289 diseases and we can see them in the table and we can apply additional filters on that. And what is cool, we can also propagate those constraints backwards in this visualization. But we can do even more advanced things. So like you can already make some discoveries just by doing the simple exploration in the graph. But you can do more complex things. So example, let's look at asthma. You can try to retrieve all the genes that we know to be causally related in the correlation way to asthma. And well, we get 355 nodes, 8000 relationships, a bit hard to tackle this. So we would want an extra step to kind of try to make sense of it. You would bring pathways and functions into the picture. You connect those genes to pathways and functions and bring that into the picture. And with doing that, now you can apply graph data science methods like Louvain neighborhood detection to color to color this cloud of nodes in a functional way. And then you can also reduce the number of nodes by filtering by centrality. And now it becomes much more manageable. As an extra step, you can add also known drugs to the picture and then yeah here you already have then some hypothesis for maybe drug repurposing that you can start exploring maybe there is oh here is a for example immunosuppressant approved for atopic dermatitis and it appears here maybe connected through this chain to the asthma so this is already a hypothesis this is kind of an example of what users of our knowledge graph the type of analysis they might be doing another way of getting information from a knowledge graph that is not there because like this is like what we how do you get extra insights is using link prediction and here we I want to show you an example where we defined train and test data sets using Neo4j you have to really carefully dissect as a graph and split it so there is no leakage of information between those data sets. And we did a gene disease link prediction and we used complex embeddings, as I called complex, because they capture directionality in the data and also different relationship type. And we used a library called DGLKE for training and for benchmarking it, we compared it to to predictions based on node degree to eliminate the biases related to the fact that certain genes are just overly... there are a lot of connections just because there is a bias in certain types of genes to be over-researched. There are trends in science that certain types of links are multiple publications. So this is a way of eliminating this bias in our benchmarks and so the advantage of using complex embeddings and capturing topology of the relationships can be seen in this gap between the dashed line, which shows recall of using node degree and the solid line. So for certain types of diseases, the gap is really significant. the gap is really significant. For example, for colorectal cancer, there is a huge evidence for the model to be really powerful. And so now jumping to biomedical. So this is what is possible using biomedical knowledge base that is human curated. So there are extra things you can do with AI extracted data. And if you integrate the sources such as Google patents, bioarchive, meta-archive, you have a wider scope of information in your graph. And so it results in around 600 million relationships that we capture. It enables new use cases. So, of course, the old ones like discovery of targets, clinical developments, and business development strategy. So here's an example. I have to speed up. I can see. So here's an example of the structure of KBEI. And it's a capture of semantic relationships, which are basically when there is a concurrence of two terms in a meaningful way, there are causal relationships, have also adverse events, clinical trials, GVAS studies. And the new use case that I will just finish with, it enables is competitive intelligence in the preclinical stage so you can see how for certain targets what are the patent mentions across different competitors in different pharma and across the time you can also see the same slice of but across clinical trials again who is looking into the clinical trials for GLP-1 and yeah indications for GLP-1 by top pharma companies what are the indications that are being investigated this is all can be extracted from this from this vast amount of data that is available in KB-AI. And of course, rare disease use case is also because of the scope of information is enabled. And there is a hackathon ongoing right now exploring rare diseases with this data set. Yeah, I would like to thank Neo4j for invitation alex and alexander and of course all the kaijin folks who worked on creating those products and vince from samspec for the license thank you thank you dimitri for your nice presentations I think we have time for one quick question, then we have to hand over to AstraZeneca to Antonio. Is there a quick question here in the audience? Yes. You mentioned the important fact of being a scientist, biologist being biased towards certain pathways. You mentioned a very important point of biologists being biased towards investigating certain pathways because there's already something known so they can build hypotheses and then you know it's just like vicious circle so is your ai based database somehow democratizing it more or actually just propagating it because it finds even more of connections of the same type i i would say uh it would be subject to what it can it there is in the literature so whatever biases exist in the literature, they would be also seen in the KB AI data. Thank you. All right. Then we head over to Antonio. Is Antonio here? Thank you again, Dimitri. Antonio is here. Thanks for joining us from the UK. We'll see your slides in any second here. Thanks for joining us from the UK. We'll see your slides in any second here. Yeah, we look forward here. So we have kind of 10-ish, 15 people left in the live audience. Here we go, Antonio. And we're looking forward for your presentation about generative ai and yeah how you um place it as a platform at neo4j within astrazeneca so over to you thank you i can hear my own voice let me try again. Okay. Can you hear me now? Hello? Okay. Well, I will... Yeah, we can hear you. Fantastic. Well, thank you, Alex and Alexander, for inviting me and present the work that we are doing in AstraZeneca with Knowledge Graphs and Neo4j. As you said, I will be talking a bit about how we use graphs in AstraZeneca. We build a service, an internal service, and we are also applying generative AI in our different projects so let's start what what I will what I have here prepared is a bit on what graph knowledge graph and graph databases are but I guess that this won't be needed I will definitely stop for a little bit on how we go from data to for data and wisdom and then i will continue um with the rest of the of the presentation uh let me this one um one one thing that i i will just say is like i like this very much alex i i think i stole this one i think i know i stole this one from from your slide deck because it's like to explain what is a graph, let me tell you what is not a graph. Because sometimes I talk about graph to people and they imagine images like diagrams and this sort of things. This is not a graph. So graphs for us are this. But let's don't stop on the graph. I guess you have discussed today about the different approaches like the RDF or the LPG approach. the different approaches like the RDF or the LPG approach. In my team here in AstraZeneca, we are focused on the label property graphs. That doesn't mean that in AstraZeneca, we only use label property graph. We also have people who work with RDF. But in my case, what I will be showing you is all about Neo4j, so label property graphs. Let me jump this. Yeah, and I will then talk about this one. So I thought of titling this one, Getting Ready for the Future. Titling this one, getting ready for the future. And the reason why I put this is, um, nowadays the growth of the data that, the, you know, we are getting data from several devices, uh, producing data in our, um, supply chain or in many, many processes. So the growth of this data is huge. It is, it is, uh, is so much that we need a very performing way in which we can deal with all this data. And for that, we see that using knowledge graphs, it gives us a benefit. If we compare that with the traditional data management systems, also it depends on the use cases. But for most of them, when the relationships are really important, or as important as the data points themselves, or probably more, we just get into better performance, we get into better benchmarking. And also, if you just define or if you just store your data in the shape of a graph, because this is very good for machine learning, because it's contextualized, so you can now use your data in machine learning or artificial intelligence, because this sort of approaches, they take advantage of this way of representing the data. So now, why knowledge graph at labs and life sciences in AZ? Well, in AZ or any other place. I talk about AZ, but this is in general. So first, using graph databases helps you, as in other type of database, integrate data from several sources. But now there is a way in which we can keep the provenance and we can have all this data like contextualized based also on where the data came from and everything. Now biological data or data in biology or biopharm is very complex. It doesn't mean that in other areas it's not complex. It means that in our case, the level of interconnectivity with the different domains, not only within the domain, but across domains, is huge. domains not only within the domain but across domains is huge if we use graphs that is giving us an advantage and it also facilitates the advanced analytics generating hypotheses supporting the decision making for our professionals and many more so because having all our data contextualized and in the shape of a graph, that adds meaning to it. So it has semantics. These semantics, we can use them to perform advanced search. And with that, because it's also not only based on the data points, but how they are related one to each other, we take this to the next level. are related one to each other we take this to the next to the next level now with knowledge graphs and graph databases we have data-driven insights and support the decision making we can do that either you know programmatically speaking or we can make use of generative ai to to support us in this process so that's the why now if it allows me the challenges that we find ourselves is that we we need to decouple and specialize meaning that we need to have a very clear idea on the stages of how what how do we treat the data where do we do what, avoid duplication, and not only duplication of processes but duplication of data. So that means that there's the need for a modular design and always having the security in mind. As you know, security, if we are talking about trial data or patient' data, security is one of the most important things. We will be processing data that belongs to our patients and that has to be paid attention with very carefully. And for us, even though I talk about GAR graphs, we think of our data as data products. So we have to manage to put this data as a product to later on use it in a graph or use it in, you know, in a different flavor if that is required because of the asks. So now another challenge is integrating and standardizing i don't know if integrating is a is a challenge like you know but definitely standardizing it is so applying to the fair principles and making sure that you share vocabularies across the company is is a big thing because companies like astrazeneca are very big. So having every other, you know, department group or people working in different projects, they're using the same control vocabularies. That is a challenge, not impossible, but that's why this slide is called challenges. And finally abstract, abstract and automate is because the idea is, well, let's see what is common across the different domains and different projects and let's try to make that as a service. And let's leave what is different just to implement in the different cases. So these were the challenges. This is the plan. I won't go into a lot of detail into it. I just wanted to show you here because this is like, take this as a bird's eye view. So the idea is that you have the data sources, the ingestion and integration. Then in the data platform, as you can see, there are different ways in which we can process our data. Snowflake, Snowpark. We have the different tools that you can find in the cloud. We also have the Data Lake store or the Data Lake computer. So depending on the project, depending on the decisions made in the past as well, because you don't want to actually change absolutely everything that you do everywhere. So that's why we have this variety. And now the next step here would be the Knowledge Graph Service. And in the Knowledge Graph Service, what we have is the data store and the graph compute. We use the GDS, Data Science Library, and we use the libraries like APOC for cipher. This is all this part is in AZ. What we have done is that we have a service, an internal service that for our different projects, we can spin up Neo4j instances where it will, the data will be compartmentalized and we will apply the security, a role-based security in which we will define who can see what, to what level and later on it will also help us interconnecting the graph databases via the federated queries that I will be talking later on in this presentation. And finally, on top of the graph database, we have the software ecosystem on top of the database. And in there, we can have tools for exploration or graph analytics or in-house build custom tool depending on our BTGs or our project needs. That was regarding the plan. Now, this bit, I like it a lot because it's like how you go from raw data to having fair data and wisdom. it from the data sources i i we can see the data as dots on a plane where they might have they can have meaning or maybe they don't have any meaning by themselves but that it depends on the data and the source and the domain now if you apply some grouping in this data. And you can see these as different colors for the dots. What we get now is information. So at this point, we have a clearer view on how our data is classified. So if these data points, color data points are connected one to each other, at this point what we are doing is contextualizing. When we contextualize, we move from information to knowledge. And here you can already see where I'm going to. When you have your data contextualized which is knowledge then finding your insights becomes easier that doesn't mean easy but easier and once you have your insights getting into the wisdom on what you want to find out the relation in between this or how close or far or you know where in the in the landscape this is that is what what we call the wisdom and let me read the last sentence here wisdom is the ability to use insight to facilitate informed decision making and that is the key point so now fair data i guess everybody here knows what FAIR data is. It stands for Findable, Accessible, Interoperable and Reusable. I always say that in the past scientists they find, you know, they run their research, they demonstrated their hypothesis and then write a paper, send the paper away, it gets approved and someone else reads it and you don't need more than the paper, that's fine, you learn and you carry on the research on top of that. I mean, you don't reinvent the wheel. Now, nowadays it's not only reading a paper. The paper has to come with probably a lot of data for you to being able to replicate that. So if you don't have access to this data and you cannot reuse this data, it will be very difficult to continue the science the way it was that has always been done. And since we have entered in this digital age, the amount of data that is produced reaches astronomical sizes. So having a mechanism in which your data can be reused is very important. And that is what the FAIR principles are about. So now let's see how we combine these FAIR data principles and the pyramid from data to wisdom that I explained a moment ago. So how we do it in our team in AZ, what I'm going to tell you here is not that this is the only way or it is the way this is how we do it okay so i'm happy to have feedback and it's okay if people do it differently because there are many ways of doing this so how we did it how we do it is when we retrieve the data from source oh actually what you get at this point is data it's nothing more than what you retrieve now you have to go through an enrichment process in which you apply control vocabularies taxonomies ontologies you make this data interoperable by adhering to the same control vocabularies and then you jump from data to information now if you define a model for your domain for your use case for your project and then you contextualize into this data model then you move from having information to having knowledge and knowledge at this point is reusable because it it has the context it it has a model then out of this we instantiate this as data production data produce can be stored in many different formats JSON CSV XML packet you choose that is is depending on on your company requirements okay now if you put this data if data produce a catalog, you make it findable and then you also make it accessible. So at this point we have fair data or fair knowledge in this. And what we do in the projects we run is that we take these data products and we put them in the shape of a graph. And that is the graph database instantiation and with the graph traversal, the cipher queries, the graph data science is how we find the insights and get out the wisdom to answer the questions that we have at hand. Okay, so that would be the end to end. Now, a glimpse into the future of graph here in AZ. We have Johannes Zimmermann in the audience and he presented earlier today Ontoverse. Johannes, Ontoverse is not yet here because I'm using the slide deck that I had approved in the past. It will come in here but I'm not worried because today you guys have all heard about this nice Ontoverse project that johannes is leading so thank you now um potential applications well you can imagine supply chain planning activities quality control that is all in the make and supply then we also apply to drug discovery compound patient study this is all in the research and development, but you can also apply this to the sell and market for the CRM or competitive insights, real world evidence. I mean, everywhere where the relationship between the data points is as important as the data points themselves or more, is a good place to apply graph databases. And these are the potential applications. So now I will start explaining or talking a bit about the biological insights knowledge graph in which you know our scientists they we are providing with this graph in which which they encode drugs, diseases, indications, genes, interactions, everything. So this can be used in our, let's say, by our scientists to take advantage of this data. And of course they can use graph machine learning and that will help to have better drug discovery or you know to the to help them make better decisions on what is going to be studied or how things are going to be done on the other hand we have the competitive intelligence knowledge graph which integrates data from many different data sources where they gather information when the pharma companies will go into the trials public all this data needs to be shared and once this data is shared you can then retrieve it and we use this data in a graph because we what we want is to you know to have a view of of the world in in the pharma and and the different areas the different trials that are going on by other companies in different um um indications and so on and so forth just to help our leadership teams to to make better decisions or decisions based on data on what is next or where it's worth putting more effort. We also have supply chain knowledge graph. As you can imagine, this is all the digital twin in which you have all the information of what happens in your pipelines on how the materials from the different suppliers arrive to AstraZeneca, how you use them to to produce the the the products the tablets so all of this is put in here and then if we are information about the co2 or the water waste all this you know we can then leverage that in the supply in this in the sustainability initiative in which we can assess the the company to, hey, look, if we improve this process, we will make a huge change in the impact that the company has in the CO2 emissions and everything. And this one is a very exciting project, to be honest. We are very happy to be working for this as well. Now, in the area of compounds, we have three graphs, which are the compound synthesis and management, the high throughput screening, and the compounds and fragments. The one in the middle, this guy is huge. I mean, this has 1.3 terabytes of data as per today, and it has cost AstraZeneca millions of pounds to get into have all this data. of pounds to get into have all this data. And by having this in the shape of a graph, it is increasing the quality and efficiency of the future HTS screens that we will be performing. So this one is huge. On the compound synthesis and management, this is similar to the supply chain, but in how AstraZeneca produces the compounds or compounds that we buy from third parties and how these are distributed and it helps like managing the shipment and how we schedule the different, let's say experiments in the labs based on the availability of compounds. And then the compounds and fragments, this has all our database of compounds broken into fragments all the relationships and we use the neighborhood as one measure of similarity and we add many more algorithms in which we can then see other interesting things about the compounds. These three are in three different databases but we are using query federation to just run queries against the three of them at the same time. This is very exciting, it's impressive. Now in the farm side, we have formulation, the knowledge graph. I have here with me Sandra. She's working in the formulation knowledge graph and the Boston formulation knowledge graph. These are in the preclinical area. And these graphs are helping or leading us, leading our scientists to quicker and more effective scientific development. And of course, they enhance the collaboration by breaking down all these silos in which the data was before. So now we can connect all these disparate data sources into one place. It's a fair explanation, I would say. Then we also have the lipid nanoparticles. This one is run by Susi. And this is the same as in vivo. And it has also activity from in vitro. And well, they have put all this data in the shape of a graph. And later on, the idea is to apply the machine learning models and get more information or more wisdom out of this data. It's growing by the day, so it's very nice. Now let's talk about federated queries without showing you federated queries. And the reason why is because we say, or I like to say, that graphs are stronger together. So when we think of graph databases for different domains, we could imagine like islands in an archipelago in which one of these islands is a different graph database in different domains or different graph databases in the same domain. So now going from one island to the other is challenging. I mean, for sure there are many ways of going. You can use a boat or maybe if you don't mind getting wet, you just swim, you can go from one to the other. But the point being is, what if we build the right infrastructure? So in this case, you know, this road that nicely goes from one island to the next and so on. So if you think of query federation, this would be like these bridges that connect one island to the next, okay, and that will make it easier for us to just go through the different islands to reach our goal. By doing this, we achieve seamless integration of the data from the various sources, that's for sure, not only in one domain but across several domains. We overcome data silos, which was the idea, and by doing all of this, what we are doing is increasing the value of our company data assets, either internal or external, because you can now query them all as if it as as if they were in one place and that is very powerful if you think of it so um i i just put here a a an example but i i won't go through it because uh i don't know how am i doing in time, but I'm probably very close to the end of my of my slot. Just letting you know, we just run this POC using BIKG and CIKG as the internal databases, graph databases. These are company restricted. And just to prove that this can connect, we also use ReactOn, which is the biological pathways knowledge graph. This is publicly available. We just deployed it internally in Azure, and then we used query federation across this stream. And the idea was to have better data integration and analytics, more accurate information, gain novel insights, and then achieve better informed decision making. Of course, now we could have done this by just creating yet another ETL pipeline, taking data from all these three databases and create, you know, putting into a new one. But we didn't do it like that. We just ran the graph ran the federated queries. Plus, we can run on existing infrastructure. We don't need to create new. And then there's no need for upfront pipeline development. And yes, as the cons, you can say, Antonio, but the queries when are federated are a bit more verbose. Yes, but it's not that much and it's worth, you know, the benefits are worth. And of course, when you connect several elements in a system, the overall speed will be always tied to the speed of the lowest, of the slowest, sorry. Oh, you know, but let's assume that you configure all of this based on the requirements and the load that you are expecting to have. Okay. I will jump through the example. Maybe I will explain one of them only. Let's pick this one. Yeah, for example, here, let me go one earlier. So here the question was, oh, what is going on? Apologies. I have genes in BIKG and these genes, they have a side effect. Okay, they are um we know that the the there is some there are some genes you know in the bibliography that they are associated into a um side effect they call it side effect but you can say uh indication yeah um but also in that's in BIKG. But in CIKG, we know what are the drugs that are tested in trials. And then we know these drugs, which genes are targeting. So now if I just say, okay, I want to know genes that are related with pulmonary hypoplasia and these genes I want to know the trials that the drug is targeting these genes. So as you can see one bit of the information is in one place and the other is in another. So by running this federated query that you're seeing here the result is this and here we see the pulmonary hypoplasia is this red dot in the middle then if we go one level these are the the genes in bikg that we know that have some sort of relation with this now the purple ones are the counterparts in cikg and from this you know what are the drugs targeting this gene and from these drugs you know in which trials they are being tested so the answer to the question is the orange notes okay and of course if you now add react on data you can also add not only the the trials the genes but in the counterpart of the genes now you can know the areas of biology where these genes play a role, you know, the reactions, the pathways, so this would be the areas of biology, the reactions, and these are the identifiers of the trials. This was for PITEN, but in any case and to finish i will explain a bit on this talking to your knowledge graph sorry the the name is not final i just put the ragazza ragazza thing because i found it fun but this is me naming things this all of you for sure know when you use generative AI, you are at risk with this AI hallucination. The hallucination is when the generative AI just gives you an answer back that makes a lot of sense when you read it, but doesn't necessarily need to be true. If you just use generative AI to make decisions, that could lead you to a wrong decision, misinformation, or even dangerous situation. Imagine that you are just using that for a self-driving car. That could be a problem. So we were thinking, can we avoid hallucinations? That was our question. And then there are two things. One is like, well, you can keep training your models with exhaustive and updated information and then check very regularly for inconsistencies. But this is an ever-ending problem, at least if you do it with this naive approach, as I explained. But we could also, for example, use AI to write queries against one database and then run these queries against the database and see what the result is. So now hallucinations will still be there, but if the query is wrongly written, it won't probably give you any result. And if the query is written, it's right, then it will run and it will give you the results. So if you base your answer in those and the data in this result, you know, that is what we are looking for. So here, the point being is that we need to have a source of truth, a graph database in this case that we have built ourselves that is reliable or gold standard. We also need to make sure that this database is updated and accurately represents the domain for which it has been built and then by doing that, what we will do is generate these queries, run the queries and interpret the results. It's important to say this approach is not applicable to creative tasks. Okay. So what we end up having with the ragazza is we allow experts to ask questions using natural language. We have AI driven instant responses and avoiding hallucination or more than avoiding when hallucinations happen. What we have is that we don't give any answer. Then this is this approach will allow so we don't need to send this data or to train any model with our data. So this data is not shared with third parties. It provides a personalized experience per data domain because we can tailor this to the different domains. And by doing this, we have like an engineer that can help you writing your own queries 24-7, which is interesting. How this thing works, you probably have seen this during the day probably more than once. But there we go yet another time when our experts our professionals our scientists they type a question in the chat and this is sent what we do is that together with the model generic and domain specific rules some training set and of course the previous interactions because you want to keep having like a conversation with the chat. That is all sent to the OpenAI, Microsoft Azure OpenAI. What that gives us back is a cipher query that we run against the domain graph database that will produce a result which is in JSON. This JSON is now sent to the generative AI, to OpenAI, with some rules to say, just provide me an answer solely using the data that I'm providing you. So generate an answer to this question, make it look like a chatbot answer, but don't invent, don't add't don't put more but don't miss anything in this set of data that i'm giving you and this is what is now sent back to the to the user as the response yeah i hope i'm in time i think i'm a bit late apologies for that this is not a one one man band this is a group of people working on this actually a large group of people doing graph and knowledge graph in astrazeneca but not only employees in astrazeneca also people in neo4j that they have been helping us and you have them on the slide on the on the right hand side thank you very much and if there is time alex i'm happy to take any any questions thank you very much antonio all right is there one question from the audience here no question from the audience yeah i have one i have one from from the internet uh corey asks graphs get huge after adding all the info and you contextualize it how do you make it accessible graphs get huge after adding all the info and you contextualize it how do you make it accessible to to your clients to outside people do you use bloom or use custom web apis to get insights in a friendly use or do you train your clients to code cypher queries actually it's a very good question and the answer is it depends on the user. So there are more technical users that might be other developers that they use the graphs. In this case, we introduced them to the fantastic world of graphs and graph traversal using cipher queries, graph data science, and then they are in this interface of maybe the browser neo4j browser or if then what you have is that the user is not like computer scientists let's play like that but they still have a good technical knowledge and you can offer tools such as bloom in which you can explore your data, you know, and then take advantage of having it in the shape of a graph. We offer also this possibility. Let's not lose sight of other people that they don't have technical background and they need to have dashboards or they need to have custom UIs that are built just to answer their needs with some visualizations, some search, you know. So at the end of the day, we offer a variety of options. But we are always happy to teach Cypher to people and just to make this community bigger in AstraZeneca. So it's a bit of everything. It depends on the user. Unfortunately, sometimes it's often the answer we have to give. It depends. Even though we don't want to give that answer, but it is. It is. It is true. Cool. Thank you very much, Antonio, for the great presentation, for the great overview. And yeah, we continue with the program almost at the finish line. Second to last presentation is going to be from Matthew. You're going to say a few words, Alex, to introduce him and then we continue. Yeah, exactly. Thank you, Antonio, for your nice presentation and looking forward to see you live soon. So next presentation is about glycoproteins and sugar monomers. Super interesting presentation by a startup company from Australia. So that's why it's in the middle of the night. I think it's two o'clock at night. So that's why we have a recording from Matthew. Yeah, they look for glycopatterns and doing mass spectrometry and using cipher to query the incomplete dataset and then connecting it to literature data. So really looking forward for the update. Last presentation we had at Nodes 2023, which was already really interesting. So let's see what they have in their update. Hi and good afternoon. Thank you for that kind introduction and the opportunity to present some of our work at Interven to unlock glycoscience with knowledge graphs and the development of regenerative AI solutions. A majority of the work that I will present this afternoon is a continuation of the story that Karina presented at NOSE 2023. Last year, we introduced the Intervenz activity to build a glycocentric knowledge base that complemented other omics studies. Karina has been pivotal to the success of the work so far and has been instrumental in designing the models and schemas and the ingested data, both from internal and third-party resources. So protein oxidation plays a fundamental role in health and disease. It orchestrates steps of protein folding in NER. It fine-tunes protein interactions as well as protein-glycan interactions. Glycans shield proteins from antibody binding as well as pathogen interactions and also contributes to malignant transformation events. Glycans are highly dynamic in state. So if we look at the glycosate and protein model on the left hand side where glycans are shown in blue, over the time course of this simulation you can see that most of the protein surface is protected or covered by glycans at some stage. The aim of glycoproteomics is to understand how glycosylation shapes biological processes by understanding peptide sequences, glycan structures, and sites of modification in a system-wide context. In a simple example, we have three possible in-glycosylation sites. With 10 different possible glycans at each site, this would lead to 1000 glycoforms per protein. We also must be mindful that even though we have a possible in glycosylation site present on the peptide sequence, that site may or may not be occupied at any given time. It is therefore this rich diversity that contributes to protein function. It is therefore this rich diversity that contributes to protein function. At Intervene we have developed an integrated analytical and informatics solution for glycoscience. Blood samples are collected from patients. Liquid chromatography and mass spectrometry platform is then used to separate glycopeptides from these samples. The abundance of glycopeptides is then determined by an in-house platform called PIP. PIP has the capability of performing automatic peak selection from LC-MS data. In turn, ML models are built from the abundance data determined by PIP to diagnose disease or predict its pronostic state. In addition, the proteins and glycans associated with these glycopeptides, identified by the models, are further investigated in previous publications and over diseases. This is using data obtained from internal or public resources and contributes to the knowledge Graph that we are building. The aim of our GenAI and Knowledge Graph projects is to provide the community and industry with access to an AI glycoscience discovery platform. This will bring together clinical and translational data collections, complemented by analytical data collected through our MRN platform, enabling us to build a preparatory clinical, structure, and functional glycoproteomics graph knowledge base. Together, this will allow us to model changes in protein coagulation and disease. Ultimately, this will allow us to discover novel disease biomarkers, free multi-omic analysis, and identify functional pathways. We are also extending the look at generative biologics, building models of human disease, and exploring biology at scale, including the de-nerve optimization of glycosylated biologics. The GenAI and KnowledgeGraph platform that we are developing complements an existing product pipeline called GlycoVision. GlycoVision is generating insights from biomarker discovery to clinical actual testing. As previously mentioned, it relies upon PIP. PIP is a piece of software that translates complex LC-MS instrument output into easy interpretable digital data powered by AI. By using recurrent neural networks and trained on Manley-Anset and Gram-Douglas peaks, PIP automatically determines or extracts peaks from LC-MS data, therefore allowing high-throughput glycoproteomics analysis. This data, combined with information on the clinical samples, is then used in part to build components of our knowledge graph. A glycoproteomic data combined with public datasets to provide unique insights on disease and related pathways. In addition, we provide information from the glycovision package, which includes expression profiles, analysis, and reports of clinical samples run. Similar to other initiatives that we heard about earlier today, the Interven Knowledge Base provides information on pathways, including protein-protein interactions. It provides detailed information on disease and disease mechanisms. It also includes information on the binding and mode of action of chemicals, drugs and other chemical entities, and also protein function and gene regulation pi processes. Importantly, what sets the pi is the inclusion of proprietary biomarker data, and also the connection to glycomics and glycoprotiomics. We have widely provided information on site-specific annotations, and therefore the glycoprcol forms and the structures. But we also relate this back to the biosynthetic pathways and subsequently other protein-protein interactions and also the impact and role of how drugs or other chemical entities affect the regulation of the building blocks of glycans and subsequently glycoproteins. Here we provide a brief snapshot or an example of the main glycocentric subgraphs of the knowledge graph. On the left-hand side, we see the samples, i.e. the clinical data used to build the graph. We have the associated glycopeptides. We then link this to protein-protein interactions. These are linked to curated resources, i.e. the references. Here we have the glycan structures associated with the glycopeptides, and then how these are linked to protein-protein interactions or pathways. For each glycan, we then link to a glycan or structure topology subgraph. This subgraph shows the connections between the monosaccharide residues, therefore allowing both motif and epitope discovery of glycans, associated glycopeptides, and subsequently relationship back to clinical samples and therefore an understanding of how glycans change in disease context. Consequently, the Interven Knowledge Graph is really connecting the islands between multi-eumic data collections. Our Knowledge Graph offers access to comprehensive data collections from in-house clinical data to third-party resources. Ultimately, we are bridging the gap between other omics and gloca science. Therefore, this is enabling more stronger, broader biological discoveries to be made and ultimately more of biologically meaningful questions to be asked of the graph. The diagram on the right, similar to other diagrams we have seen throughout the day, really indicates the relationships that we are building in the graph, whereby for example a gene encodes RNA and subsequently a glycoprotein. This glycoprotein interacts with another glycoprotein. This glycoprotein interacts with another glycoprotein, which can be described by a publication, or is mentioned in some sort of clinical relevant patient sample data. We have information on drugs or other entities that can interact with a glycoprotein, and subsequently the protein-protein interaction, we have the information on the disease state and we can link this back to any clinical trial information that's available. So if we bring all this information together, our knowledge graph features exploration of complex molecular network interactions and pathways, drug binding actions, protein function and disease specific contexts. All of this information is also related back to glycoproteomic and glycomic associated knowledge. We also enable language search through custom phases, full text and pattern mining, building visualization and rendering engines for molecular pathways, providing complete information on data provenance and metadata, and building new and exciting tools in the GenAI space that will allow users to interact, interrogate this curated knowledge base with new AI tools. For the remainder of the presentation, we want to show a few use cases where we combine information from third-party resources or experimentally derived data and how it links with the Internet Knowledge Graph. As many of you would be aware, PD-1 engages with the PD-L1 and suppresses T cell activation. Many monoclonal antibodies target in PD-1 to block latex interaction with the L1 protein, enhance T cell activity. More interestingly is that several of these antibodies have an overlapping binding epitope that includes the N-calcisation site at position 58. And this is clearly shown on the models with the N-calcisation site circled on the top right corner. Work by ourselves and others has clearly shown that the core fucose, shown here in red, of PD-1 at position M58 is a key determinant in antibody binding. PD-1 fucosation determines PD-1L1's blocking efficiency of some given antibodies. It has been shown that fucosate is soluble PD-1 is higher in serum of late-stage lung cancer patients, and therefore variable glycosylation modulates the activity of monoclonal antibodies used in clinical practice. For the purposes of this exercise, we wanted to show if there was a connection between the PD-1 protein and CDL4, both immune checkpoint targets, and if it connects with a panel of glycopeptide biomarkers on our glycovision product. The clinically derived glycopeptide biomarkers of glycovision are stored in the subgraph of the intervent knowledge graph. As you can see here, by using three degrees of separation, we clearly show that both PD1 and CDL4 have direct connections to the regulation of biomarkers on the glycovision subgraph. As such this clearly demonstrates a role in regulation of these two proteins on the top and its connectivity with glycoprotein expression and regulation. For the purposes of this presentation, we omit the 10 proteins shown in the bottom panel, however we are quite happy to discuss these further offline. In addition to providing clinically relevant glycosylation data on PD-1, the IntervenGlycoKnowledge graph also includes data on other cancer types. This includes melanoma. As part of our GlycoVision products, we have identified biomarkers associated with immune checkpoint inhibition therapy and melanoma. The figure on the right-hand side clearly shows the network model, describing relationships of the clinical design studies shown in the center, and the glycopeptide markers identified has been off-regulated or down-regulated. For each of these N-glycopeptides, we showed a composition attached, in this case, by 412. In addition to displaying the glycopeptide, we then have a direct link to the protein node itself. And from this protein node, we can go away and extract data from Uniprot, Nexprot and other resources that describe the functionality of that protein. A summary of those 12 N-glycopeptide markers shown to be involved in melanoma immune checkpoint inhibitors displayed here on the left hand side. So we have the protein, the site of glycosylation, and the glycan composition. We also do have a non-glycosylated marker here as well. These two examples of clinical data collections graph really show the power of our design to incorporate metadata on clinically relevant samples, analytical data collected, and how we can connect that information to other functional pathways and interaction studies available in the public domain. This really allows the end user to ask very complex, biologically meaningful questions of the data available in the graph. So over the last few months, we have expanded the diversity of information in the graph. So over the last few months we have expanded the diversity of information in our graph by curating internal data collections and ingestion data from other facets of life science. As you can imagine this has increased the overall complexity but also the quality of the products. To help improve data discovery and end user access we are developing a suite of Genio tools. As an example we are developing a suite of GenIO tools. As an example, we are building upon the text-to-cypher shared by Obers, that we may hear about later today, to build agents that use the provider graph schema and LLMs to query the underlying database. So far this has proven relatively successful for more simple queries, and with the next releases of the products we'll be providing novel tools that unlock Glycoscience for the life science community and industry. During this presentation we have hopefully given a snapshot of how we can improve the connectivity of Glycoscience knowledge with other multi-economic data collections. The tools and databases we are building are truly novel, and we believe we can improve our understanding of the role of glycosylation in disease. Provision of an integrated analytical, informatic and gen-IA toolset will help simplify data discovery, but also accelerate the fine-tuning of models with glycocentric applications. So I'd like to take this opportunity to thank the organizers for this chance to present some of the work that we're doing at Interven. If you have any questions or would like to follow up, please feel free to contact me at these below details. Thank you. Super cool. Thank you very much, Matthew, in absence, but is excused at 2am in the morning. Exactly. It's clearly not a time you want to do a presentation. I hope you liked the presentation as well. I found that super interesting because most of the people talk about proteins, but always forget that these goddamn beasts are glycosylated and makes it even more complex to work with so interesting stuff so that brings us to the last presentation the last session of today uh i hope i hope too much can we you know we are a little bit early so um he's probably just joining about the session. The last session we are doing today is a bit of more hands-on session. Exactly. We're going to show how a graph rag actually works. If you want to implement it, if you want to do it really. We heard about it a lot. I mean, Antonio said that you probably heard it a couple of times when you showed the classical diagram of how it works. Actually, he was right. We heard it a couple of times, but it's good to hear it a couple of times more, I think. But today to finish the session, we are going to hear about it, but also see how it works and how you can experience it and take that on basically as a little inspiration if you want to continue doing the same at home. We have, or you have, prepared a data set for Tomas. Well, it's a publicly available clinical knowledge graph by Matthias Mann, so pretty much known in the field. That is kind of the basis for creating a chatbot, and I think it's one of the best person to have Thomas here, who is the champion of working with LLMs, optimizing them, trying things out. How do you say? Get the feedback, communicate the feedback, what works, what doesn't work. And I think that is really valuable for all of us, because in many ways, right, GenAI promises everything. But in the end, we have to be realistic. And I think that is really valuable for all of us, because in many ways, right, GenAI promises everything. But in the end, we have to be realistic. And that's why I think we should do things, break things, as I always say, and try to learn from that. So we shouldn't be shy of breaking databases. Yeah. Just don't do this with your production database. Yeah, just don't do this with your production data. Yeah, exactly. But yeah, Tomas is great. I mean, I don't know if you if you follow maybe the live streams of Neo4j, maybe you subscribe to the weekly newsletter we're sending out, or maybe you just follow us on a medium where we where we regularly post on the developer blog. But Tomas is like, like you said, a lighthouse instance in Gen AI blog since he took on the topic, I think, early last year already and before that already, but with the advent of Gen AI as a whole, he was like the leading voice, I would say, at Neo4j. He wrote so many blog articles, so many hands-on interesting pieces, how to work with Langchain, how to work with Lama Index. Just recently, he wrote an interesting integration there. We have a GenAI Labs page, so maybe that's an interesting shout-out before we dive in with Tomasz. If you go to neo4j.com slash lab slash gen ai ecosystems you find everything you can you can it's available in terms of content around neo4j and gen ai so some hands-on stuff as well should be interesting but i see Tomasz is ready so without further ado let's dive in to the final session of the life sciences workshop day of 2024 and yeah good good to have you too much uh thank you for doing this and uh yeah as a last point of the of the intro to thomas he is a real person as you see uh if you see the amount of posts and articles he does you could think like thomas is an llm itself but or maybe that's a new gen ai with with some animated image here. So, Thomas, over to you. Looking forward for your last presentation. So, okay, just let me know that you can hear me. We can hear you. We can hear you, yeah. And you can see my slides. Okay. So, I'm going to assume you can see my slides, but there's just some echo, unfortunately. Okay, it's fine now. Okay, so my name is Tomas Velatenic. I'm gonna talk about graph-electro-life science today. I'm not an MD, like a medical doctor, so don't expect in-depth medical explanations, although I worked on a medical project with a biomedical institution here in Slovenia, so I have some idea about biomedical knowledge graphs in practice. And that's enough about me. So let's dive right into it. Right. So today we're going to talk about LLM based chatbots using the retrieval augmented generation. So as we all know, LLMs are really great and they know how to answer a lot of questions. But one of the problems they have is the knowledge cutoff date. All the information that was part of the training data, you can kind of assume That was part of the training data. You can kind of assume that the LLM will have some idea about that information, although we don't exactly really know what happens under the hood. We just kind of take it for granted that LLMs can learn from training data. And like in medical field, like that's kind of a problem, right? Because I think GPT-4 is trained up to like October 2023. And then it's missing the last half year of research papers. And like in the biomedical space, you always want to know what's the latest and the greatest today and another problem is hallucinations so specifically with biomedical data for example if you're gonna ask uh uh llms to cite some research papers i am guessing you'll get quite bad results. So you don't want to do that. Like here we got examples where lawyers asked LLMs to cite legal cases, but like citations are notoriously bad with LLMs. And then another problem is the lack of information. So you don't really know the information the LLM was trained on. Because LLMs are trained on web data, And there's a lot of conspiracies on the web. So if your LLMs would start spewing conspiracies, that would also be a problem, right? And kind of the lack of domain specific information is also a problem. Although I don't really know how much of the biomedical information is fed to the general LLMs. So, the accepted solution to overcoming these problems is the so-called retrieval augmented generation, where you pass context at query time. So the flow is the following. A user asks a question, we have some sort of smart search tool. The tool has some ways of identifying or retrieving relevant information from your knowledge base. And when the element documents information is received, we feed that together with the question to the LLM. And then the LLM can generate hopefully an up-to-date and accurate answer. So in this case, we are kind of ignoring internal knowledge of an LLM. And we're just using LLMs as kind of like interpreter instead probably know, like chat with your PDFs is kind of the hello world of LLMs. So you take a bunch of PDFs, you chunk them into text, you embed them using a text embedding model embed them using a text embedding model into a vector database. It can also be just vector index because Neo4j has a vector index as well, and it's not a vector database. So that is kind of the ingestion part. Obviously there are nuances, how do you chunk the text, which embedding model do you use, all of that. But that's the gist of it. Then at the deliver time, you again use embedding models, and you embed the question that the user asks, and try to find the most similar documents that the user asked. And again, as the part of the retrieval augmented generation, you feed the documents to an RLM and then the RLM generates the final answer. So that's kind of the hello world of Rlan's and what you see in most examples. But since documents are basically unstructured text and then text embeddings is a way to find similar or relevant documents which are in unstructured text. That's all really nice. But there's a whole field and this is really relevant for life sciences or biomedicine. If there's a ton of structured data available. For example, a typical biomedical knowledge graph has a bunch of structured information available, right? So you can have genes, compounds, diseases. I just copied this from, I don't know, from a Neo4j page, but this is mostly what we kind of see as a typical biomedical knowledge graph. So a lot of structured information and coming from various sources. So and one thing how I see basically biomedical knowledge graphs are being constructed. So you have a bunch of existing biomedical ontologies that you can use to incorporate kind of existing expert curated data into your knowledge graphs. But then on the other hand, like biomedicine, before LLMs, I think, from my observations, is that biomedicine was kind of the driver behind NLP and constructing knowledge graphs from text even before LLMs. Because if you look at the history, historically, there are a bunch of named entity recognition models and the relationship extractions, and it's like academically backed projects. And then you have a bunch of here, I added to, I don't know if you're familiar, but it's basically a very sophisticated entity linking model, which can basically take the text and link entities in that text to existing ontologies. And then there's the Indra. I think it used to be from Harvard, but now they moved to a different college. And they do similar, but they also do relationship extraction. And so the field of data mining and extracting information from text is really strong in biomedicine. So all of that together allows you to create really sophisticated knowledge graphs which contain mostly structured information. And now that you have this structured information in the knowledge graph, the idea is to use it in a reg pipeline. So when I tuned in, I saw the presentation, they talked about text to cipher. So basically generating cipher statements using RLM, where you provide RLM with existing knowledge graph schema, and ask it to generate a corresponding cipher statement that returns the information that can answer the question. So here I have like one example that Alex actually gave me. So it's for finding clinically relevant variants of a gene. And as you can see, it's like not really advanced, but it's not the simplest retrieval query either. Right. So, and fortunately, my standing at the moment is that generating cipher statements with with llms is just not good enough as also the previous presentation stated it's good for like simple retrieval queries but when you get into more advanced ones or like even there's this problem of mapping like user there's this problem of mapping user questions to existing schema. Even if the question is simple, but the user is not aware of the information in the graph, the mapping from the question to the schema can be not straightforward. So there are a lot of problems in generating cipher statements with LLMs. But we still want to have REC using biomedical knowledge graphs to answer questions. So what's the solution? So the solution is, in my opinion, is to use basically LLM-Suite tools. Some of them call LLM-Suite tools agents, and I also like to use the word agents. So instead of providing an LLM with graph schema and optionally few short examples and asking it to generate the cipher statements, we provide LLM with a couple of tools it can use to interact with the knowledge graph. And then we also give it instructions, when to use those tools and how to use those tools. So we kind of, as you will see, we lower the complexity of the task given to an LLM and we make it more robust and deterministic. And the whole idea is also that instead of generating cipher statements with RLMs, we use predefined templates. And I will show you that as well. So here is an example agent I developed in collaboration with Alex so we have a user asking a question and then the LLM has three tools it can decide to use so it can if the user is asking about associations of food to diseases, it can use the associated food tool and that has a predefined cipher template, right? Or if there are protein expressions in tissues. And by the way, I might be saying something completely stupid from like a medical perspective because I'm not an MD but that's just my understanding of what Alex provided me and then the last one is the last tool is finding clinically relevant gene variants right relevant gene variants. And basically we provide those tools to an LLM as functions. I don't know if you're familiar with LLMs, but most of commercial LLMs nowadays provide native function calling and some open source models now also followed perspective where they support function calling natively. And so how do you define a tool? You can give it a pretty cipher template. So for example, this um a cipher template to find these like proteins that are associated with diseases and then those protein proteins uh basically which tissues are they associated with so alex and alex told me that that's basically proteins expressed in tissues. So basically, yeah, let's just put it that I don't want to explain too much because I don't really know. But basically the idea is we can provide a template with parameters, so that could be one or more. we just have one parameter so you can see that this is so basically when an s when a user will ask a question about proteins expressed in tissues for like cancer the lna will populate the disease parameter with cancer and then it will return the results. Now obviously this in practice this cipher would be much more advanced because there's like a whole lot of ranking and sorting and prioritizing of results that you should do because this simple query will return a lot of results so here I just put a limit 10 to not over the LLM. But in practice, the Cypher template could be much more involved. So this is basically defining the Cypher template, and then we have to inform the LLM when and how to use this tool. So, for example, here we have disease tissue input, and we say, okay, disease is basically just the disease mentioned in the question. And then we inform the LLM, you can use this tool when you need to find tissues where proteins with a specific disease are expressed. Now again, this is my interpretations of what Alex provided me, so it might be completely wrong, but that's not the most important thing here. And let's show you, oops, a demo. So hopefully if I can... Copies I created a simple GitHub repository that you can see basically here that you can see is basically just on my GitHub at clinical agent and all the information. But basically if I ask it a question, now hopefully it will work, But if I ask it what are associated foods with cancer, it says, OK, I am going to use the associated foods tool. And I'm going to populate the disease parameter with cancer. And then here we don't really see what we get out of the database, but we do see the interpretation by the LLM of the results, right? So for example, chocolate spread is associated with colorectal and lung cancer due to metabolite betaine. Because in our code, I can show you basically just a second. But for the associated food, let me zoom in a little bit. we return the food, but also the metabolite that's associated with the disease. So that's why we can see that the LLM knows the associations between diseases and foods, but also to do with metabolites. Then if I ask it a question that requires a different tool, you can see that now, okay, because we're asking for gene variants, I'm going to use a tool gene variant and I'm going to populate the parameters correctly. And as I mentioned, there could be one or multiple parameters that Dial-LM can populate so the Cypher templates can be much more involved and more advanced. But this is basically just a simple presentation or simple implementation I did in one day. And as you can see again again, the LLM then, I'm not even going to try to explain the results. But there is like the relevance that we get from the database, which disease, right? And then the variant ID. And with this approach, right, because we are not generating cipher statements, I am much more comfortable giving demos or using that in production, because LLM is not tasked with generating a query like this, which is not really complex, but still, like TextoCypher, I don't think it would get it all the time. Basically, we reduce the task complexity of an LLM, from generating such a Cypher statement statement to basically just deciding which tool to use and populating parameters. So the complexity goes way down and that in turn means that we get much more robust process, or the workflow, and more deterministic as well, right? Because when you're generating text-to-cypher, it's not a given that for the same question, you will always get the same text-to-cypher. But here, like, you will always, like, for the same parameter, you will always get the same results, right? So that's definitely an improvement. And then let's just see the protein tissue example just to show it all now. So there's the last one. And then here's just a bunch of tissues. Let's see. Yeah, it's expressed in various tissues. So this is how the LLM interpreted the results. So again, you can give more context to the LLM, how it should interpret results. And you can also like deterministically define what types of results that you want to give back to the LLM, right? So it makes the process much more robust and deterministic. And one thing I haven't tested, but the nice thing about this approach is also that it's scoped down so now if i ask it we do you have any drag think um i say candidates for aspirin right let. Let's see what happens. I haven't tested it out, but hopefully we say, okay, I don't know how to do this. Right. Ah, okay. But you could fine tune it. But the idea is that it says I don't have the capacity. Right. So it basically says, I don't know how to do this and this is really nice because we want the lm to have like some sort of scope uh so that it doesn't answer just any question how it feels like. And you also then know what information came from the knowledge graph. And here basically it's also really evident that this information came from like internal knowledge of an LLM. And in this case, we could also prompt engineer it a little bit to just say, I don't know, and leave it at that but you know because uh lms are trained for um to be i say what's the correct word to always kind of provide helpful answer it's still uh provided the answer so i see somebody writing something. Ah, he just left. Okay, fine. So that's basically it for my presentation. It was really short and condensed. But the main takeaway, if anything, the main takeaway is that instead of generating Cypher statements, what you want to do is to have here a call it semantic layer, but some people don't like that name. But basically, you just give an LLM a set of tools, like with predefined cipher templates so you narrow or lower the complexity like by a couple of levels and in turn you get much more robust and deterministic results which is basically what you want to have because if you ask the same question every day of the week and the data doesn't change in the knowledge if you want the same results so yeah and now we can do a couple of questions i see we have some time and yeah cool thank you very much too much great presentation as always especially the demo part is always interesting um like to open up to the room any any questions uh from from the audience here anything uh maybe you you had in mind i mean you you said a couple times you you don't know what you're talking about so i think you don't know what you're talking about so i i mean the biomedical domain right yeah yeah and i am they could say okay proteins are not a great issue or something with that there's one question here uh hi uh thank you for the presentation i was curious so how do you think this uh approach with semantic layer and defining tools scales uh because you showed like three very specific kind of tools uh and i imagine like if you would want to expand the vocabulary of the tools like to hundreds of course it will also increase the context size and I don't know maybe LM might become confused what do you think about it yes that's a good question because like you don't want to have hundreds of tools, right? You kind of want to have, I don't really know, but my gut feeling is like 10 to 20 tools at most for the LLM. But the thing with returning information from knowledge graphs is that it's not like basically with vector retrieval, let's say the most use cases return like 2000 to 3000 tokens of context and then ask the LLM to interpret that those results right so i think for the most part lms are quite good at handling contacts up to let's say five to ten thousand tokens and let's just go with the lower side which is five thousand tokens so imagine all the information you can uh information you can uh congest into 5 000 tokens if you were to build like these really general uh tools right because my idea or recommendations would be like you could build like I don't know what's the correct word but like model descriptions so like each gene you could like for each gene you could have like 2000 tokens of information and then you would have a tool that just identifies information for a specific gene so I don't really think there is a problem with over-development information because you have like a couple thousand tokens available to congest the information that you want. available to congest the information that you want to learn so hopefully i mean i i can't it's a convoluted answer but hopefully it answers your question cool thank you thank you too much one question from the internet can you dig in from the back by the way can you dig in a bit more on the semantic layer um no more context people the better i don't know if that helps um can you say glare maybe yeah i mean like it's all about instead of having one text-to-cypher tool that does everything, like here is a good example of an overwhelming LLM, right? You have this very complex biomedical knowledge graph and you just have like one entry point and it can be very overwhelming for the LLM because like even schema description could be a couple thousand tokens and then if you add a few short examples and additional instructions it could be like a lot of tokens and it can be quite overwhelming for an llm to generate quality uh ciphers so instead of that you how you say scope down and instead of having one entity point to your knowledge graph you can have like 10 20 uh entry points right like if you can uh i mean it's not like linearly but imagine if like instead of having one tool that does hundred percent of the world you have 20 tools that each does five percent of the work and then it's much easier to optimize each tool for those five percent right because it's a much lower scope of task and complexity cool thank you and then we have one more question so maybe maybe we take that on before we we close the the q a section here from david uh and he wants to know do you have the in of an llm plus the agent the graph data science are developed in we have in mind the integration of an agent with the graph data science with a graphic science algo i guess i don't know exactly what you're asking but if you're asking if an agent could be executing graph algorithms uh i think that's that's a question i think that's a question okay so that's generally advisable because graph algorithms like usually are graph global operations right so it can take it's not like on the scale of milliseconds and seconds that the execution takes. It's more like minutes or more, right? And you don't want the, for like this graph global operations, you don't want the agents to kind of decide this 30-minute operation and see what we get, right? So that's not a wise. Obviously, you could do that, but I would advise against it. It's more about running those algorithms like in a batch job nightly or something, and then just having the agent interpret the results. You run the graph algorithm overnight or something. So, the results go back to the database and then you have those for an agent to interpret. Now, obviously, if you have some algorithms that take a very Obviously, if you have some algorithms that take a very small amount of time, you could integrate them into the agent as a tool, but I don't have any examples. Again, you just provide a Cypher template, which is a graph algorithm. You have a couple of parameters that you want. You just have source nodes. You feed those source nodes as parameters to the Cypher template and there is your preset for the mouse part. That's not something you want to be doing so may i take a final question thomas do you see in in your experience with customers and and you know in all the work that you do ratio of notes to relationships where I don't know the graph is too tightly packed too much connected is that kind of limit with the LLM is that it is that okay yeah I mean, let's just see what happens. But like I said, the problem is biomedical knowledge gas, the problem is too much data. How do you return as much as little data, but still enough to get correct answers? And I mentioned you should be doing something like thing, ranking, prioritizing of results. So forever, probably, because there's like the graphs are very good. So it's more about graphs, like good medical domain. You see the numbers. It's too big to even for me. It's too big for me. So yeah, this is a problem. You don't want to overrun the error. You have to have some nice sorting priority systems in place so that you only let's say right and right. Some that's not mentioned, but you could also provide like a text answer, but also graph visualizations alongside as well for better understanding of the data. Thomas, thank you very much. Thank you very much, Thomas. Yeah, thank you for having me. With that, we're at the end of this year's Life Science Workshop. Thank you very much for tuning in today. Thank you for coming over on site here in Munich for the event. As I mentioned before before uh notes call for papers is is coming up so if you wanna submit something do that if you wanna know more check out the community pages check out neo4j.com come to the um come to the forum on discord obviously we'd like to know your feedback what you thought of the today event what you liked but you didn't like improvements comments you can send us emails you can write us you know messages in the forum you can comment on the youtube video obviously as well let us know what you what you thought of the day give us give us your thoughts give us your feedback we will be Give us your thoughts, give us your feedback. We will be doing another session surely again sometime. Absolutely. Yeah. Thanks. Continuing with the life science community. So that's crucial. We regularly host live streams on YouTube and on LinkedIn. So if you want to watch more content on that um that's always something to subscribe and to look out for but yeah in terms of life sciences we will do something and else again um and yeah i was i was glad and super delightful super delighted that you were all coming and that we had a nice event with super interesting presentations super content from from our friends and from the community and from customers and questions for your precious time for all the ones who watchers on the channels and thanks to all the presenters. I really appreciate the success of us. If you are successful, we are So, thanks a lot and yeah, hope to have you at the. Bye bye.
Neo4j LifeScience Workshop 2024
Welcome to the Neo4j Health Care & Life Sciences Workshop 2024!

Join us for an immersive exploration into the intersection of cutting-edge technologies and the life sciences domain. This year, we're thrilled to introduce GenAI and Knowledge Graphs as exciting additions to our event, promising innovative solutions to the challenges faced in data management and analysis.

Our workshop serves as a dynamic platform for professionals, researchers, and industry leaders to come together, share insights, and collaborate on pioneering approaches in leveraging graph databases and advanced technologies. From deciphering complex datasets to unlocking actionable insights, we'll delve into the latest trends, best practices, and practical solutions shaping the future of the life sciences industry.

Sessions:
Opening: Graphs and GenAI in Life Sciences - Dr. Alexander Jarasch; LifeScience Expert - Neo4j  

BioCypher/BioChatter: An Ecosystem for Connecting Knowledge Graphs and LLMs - Dr. Sebastian Lobentanzer; Heidelberg University Hospital  

Harnessing the Power of NLP and Knowledge Graphs for Opioid Research - Gursev Pirge, PhD; Senior Data Scientist - JohnSnowLabs  

Leveraging the Graph for Clinical Trials and Standards - Katja Glaß; OpenStudyBuilder Community Manager - Katja Glaß Consulting & Marius Conjeaud; Principal Consultant - Neo4j  

Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians - Dmitrii Kamaev, PhD; Senior Product Owner - QIAGEN  

How Knowledge Graphs and Generative AI Revolutionise Biopharma and Life Sciences - Antonio Fabregat; Knowledge Graph Lead - AstraZeneca  

Unlocking Glycoscience with Knowledge Graphs  - Matthew Campbell; Manager, Bioinformatics - InterVenn Biosciences
 
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph - Tomaz Bratanic; Graph ML and GenAI Expert - Neo4j

#neo4j #graphdatabase #lifescience #healthcare #pharma #graphrag #rag #genai #bioinformatics #knowledgegraph