 All right. Hello, everybody. Welcome to Going Meta. Thank you for joining. Good evening, good afternoon, good morning, depending on where you are. As you can see, I am in a funky hotel room with an interesting image at the back. It's like a piano and a trumpet, so I don't know what that means, but here we go. Are you going to play something for us today, Alex? I don't know, better not. I'm not so sure about that but I don't have my piano here so I couldn't even do that I'm very well thank you yeah and I was going to say that we have for a change and not that it's a good thing but I'm sure you're relieved about that and after the great success but we're not going to open by saying hey sign up for notes because notes relieved about that and after the great success. But we're not going to open by saying, hey, sign up for Notes, because Notes is behind us. And it was great. I was briefly thinking about it to say, hey, Notes is done and put up a little fly up. So now that you prompted me, I'm absolutely doing that. Because Notes was amazing, I think. It was a really, really good event um uh on the 26th so uh you know two weeks ago almost to the day um we had great sessions uh on basically 24 hour streams and three different tracks but i guess you uh you know you you uh you heard about this and you as you make fun fun of me constantly repeating about everything. I don't have to tell you that. But what I wanted to tell you now is that we have the videos available. So if you know, for some reason, because it was 3am in the morning for you, potentially couldn't watch all of the sessions uh you can now go to our youtube playlist for whatever reason you've only been able to consume like six hours well you still have 18 ahead of you right so there you go i mean there's plenty of everybody can be michael great content well exactly there might be one or two i mean we should that's great and i'm looking forward to catching up with some of the the great sessions because yeah i've watched a few of them but there's this loads there's many many more to watch um yeah so i have to i i myself have to have to catch up with a couple uh particularly in the ai track uh where i i wasn't able to uh to watch that many because i was kind of like on duty on the visualization part but that was that was also a very interesting interesting track to to follow I don't know if anybody who's watching us live now did attend notes and if they have anything to say maybe you have you know a favorite session already you can you can you know identify then let us know in chat uh if that is something um you uh you can recommend something uh to the viewers that will always be great uh it's it's interesting uh for others as well if you think something is worth worth a watch um so if there's anything let us know but um yeah today um we we talk about uh rag so retrieval argument generate uh knowledge graphs rag yeah rag yeah it's from rags to graphs not too riches i don't know. So we'll see. Well, yeah, no, I think it's impossible not to, you know, have a session on this, I would say. I mean, we kind of opened the gate somehow last week talking a little bit about explicit semantics, you know, vector-based semantics. And I thought that it would be a good idea to kind of build on that and talk about about rack which is something that you know i don't know maybe maybe there's some bias but i can't help but you know reading it everywhere i mean it's it's all over the place and and it's super interesting super exciting and i think graphs are are incredibly well placed to to be be a perfect, you know, grounding partner for large language models for RAC solutions. And I wanted to, of course, this is a large topic, but I wanted to spend a bit of time today talking about it because I think it's really, really interesting. Yeah, yeah, it definitely is. It's very interesting. I think it's a particularly RACs all the rage. Yeah, Tomas, exactly. And, you know, it's a it's a particularly racks all the rage yeah tomash exactly uh and and uh you know if i just you know it's this perfect fit it was not planned i promise you this but you know i was just saying ever since the the the a couple of the announcements uh we we we did over summers or one was the the vector search capabilities and all that stuff and open AI things. It was something that came up every time. And as you know, I'm writing the Twin4J This Week in Neo4j newsletter on an almost weekly basis. And Tomasz is one of the people that have almost added on a weekly basis another interesting blog post around around the topic so yeah he's gonna be on my slides today yeah all right good yeah he's amazing so yeah thank you very much thomas shout outs to him uh if you ever um you know look at uh at his blog look at his medium follow him there um him there and read some of his content. It's really amazing if you're interested in that topic. But yeah, what I wanted to say, all these words, what I wanted to say really are was that it is not only been in our minds but I think everybody else's mind as well. It was like, okay, this is interesting. We need to look at this. We need to absorb as much as possible so yeah we are raging exactly so it's it's for once i think it's not just neo4j that says this is this is amazing but kind of like everybody does it absolutely absolutely okay shall i go ahead and and uh share my my desk uh probably oops where did this go the usual fun when you have too many things open there so sharing my screen and hopefully you will be able to entire screen i only have one screen so it has to be this one and i'm going to move uh to this one can you see that yeah alex yeah it is yes perfect okay well as usual i mean maybe maybe today i'm going to spend a bit more time on slides than usual but i think it's it's important to to you know to get the basics and i'm sure you know people attending today or watching this offline are not new to this concept but i think it's um um yeah it's important to to um to get the the the basics right and then we'll we'll move as usual into a more hands-on session so uh a very brief recap of what we covered last session so you probably remember how uh we went over the difference between vector-based semantic search and graph-based semantic search right so essentially we we explain how uh you know semantics or meaning can be can be represented in two main ways one is in what we call an explicit fashion which is what we do in a graph when we state that an apple is a is a subcategory of a fruit or an orange is a type of fruit or in our example that you know Neo4j is a type of graph database and a graph database is a type of data about database management system so these are explicit statements that we can make uh um yeah explicit using the word but in the graph and that then we can we can explore and we can navigate to uh to come up with uh with uh intelligent answers to to questions now on the other side of the spectrum is uh the what's called sub-symbolic or implicit way of capturing these semantics and uh and that's by uh translating if you want uh words uh or structures or whatever into vector numeric representations that have very interesting characteristics right for example once you create a vector of of the word apple and another one of the word orange they happen to be topologically closed so when you can you can do things like semantic similarity based on the proximity between these two vectors and that's what we did last time so associated with both types of representations, we have two ways of computing how close or how distant two things are. In the vector space, it's purely mathematical, right? Well, not that the graph is not, but this is clearly a distance function, right? And mainly we use the Euclidean and the cosine distance in order to be able to tell this vector is close or far away from this other vector. In the graph graph we have different ways to to compute distance and and therefore proximity and similarity and we have a number of them right all of these are already built into Neo have been for quite a while you can find a node similarity based on an overlap or in jackhard based uh algorithms we covered and we'll probably go a little bit over today. Also, things like leveraging the taxonomies. When you build a classification, a taxonomy, an ontology on your graph to enrich your data, you can navigate that to do similarity computations as well. And for that, we have a number of metrics that we use. So, again, these are the two main ways of storing uh uh representing and storing semantics and and both are supported by neo4j and and that's what makes you know the platform great uh for what we're going to see today for rag because you know the combination of the two is really really powerful and and just to complete the picture you know the explicit uh the search uh in in a graph space is based on the exploration of the graph. Whereas in a vector space, what you're doing is a search based on algorithms around the approximate nearest neighbors, basically looking at which vectors are close to the one that you're looking. So this was the sort of the foundation and what we covered last week. And we use that example to show, you know, different characteristics because, you know, both are really powerful. But one of the things that differentiates them is that the explicit one is explainable. Because one of the things that, and that's why we call it sort of implicit, is because a vector captures the meaning, the semantics, that's what we're dealing with here. But we can't tell why an apple is close to an orange. I mean, we can tell because they show up in multiple documents that are part of the training of the model that generates these vectors. But the idea is that you cannot explain in an explicit way why you come up with the answers that you come up with. And that's where the combination of the two is really useful. I'll take a pause here before we go into RAC. All good? So remember from last session, that was all clear, Alex? Yeah, it was all clear. I think that's an important point to make again. So yes, good to repeat that. Perfect. Right. So today's topic is RAC. And what does RAC mean? And again, if you have not been living under a rock, you know what this is. I mean, basically, the idea is that we want to build applications that use large language models. And some of the great things that large language models can do is provide conversational interfaces, they can do translation, they can do summarisation, they can do loads of things, but they're only aware of, in terms of world knowledge, of the data or the facts that they were trained on. So, you know, that has a number of limitations in the sense that there's a training, a kind of a cutoff date. So you only have knowledge about facts that were available at the time of training. And that I think is less of an issue because we keep getting new and nothing yesterday we had a new release by OpenAI where they brought forward this cut-off date. But still, they will never have all the facts. They will never have all the information. And that's important. And the main reason for that is because these large language models are commonly trained on public data. And if you want to build a solution that leverages the capabilities of large language models, to build a solution that leverage the capabilities of large language models but you want to use your proprietary data, your enterprise data, that is never going to be part of these public large language models. So there has to be a way of combining these two, like leveraging these capabilities in the large language models but using the facts, the knowledge that you have accumulated and that you have on your enterprise side and that's where uh this pattern called retrieval augmented generation or rag comes into play and that diagram describes it very simply so effectively you will have a question that you want to ask your application is basically built on the idea that you will want to ask a question from your large language model and in order to do that we're going to see that this question will have probably to be vectorized somehow in order to interact with the large language model. But there's this intermediate step that you can see. I don't know if my pointer is visible, but in this step here, you see that we are accessing some external database, data store, knowledge base, whatever. In our case, it's going to be a knowledge graph. It's going to be Neo4j. But the idea is, okay, I'm going to go out and get all the potentially relevant information in order to answer this question, which that's what we're gonna call the context. And I'm going to pass it on along with my question, as part of the context for the LLM, in a way that the LLM can use all the capabilities that I was describing before, like all the reasoning, all the analogy, all these powerful things that large language models do. But I will explicitly ask the large language model to only use the information that I provide in context. Because another thing that happens, and I'm sure you've heard about hallucinations, is that large language models are generative AI and by definition they're built to be creative to be generative to create new new new data new knowledge new facts and uh what we want to do is kind of turn turn down or disable completely this creativity and say okay do what you can do but or you're at, but do it only based on the context that I'm providing you. And that's exactly the idea. And ultimately, you know, the large language model will produce a conversational experience. It will provide a summarization, all these super smart things, but exclusively based on the information that comes from my controlled knowledge base, which in our case is a knowledge graph. You will probably see that with other types of platforms, because, you know, why not? I mean, you can generate a query, a request to an external source that returns the contextual information, and that can be anything. But I will argue that knowledge graphs, and especially those built in a vector search-enabled platform like Neo4j, are superior to other alternatives. And that's what I'm going to spend a few minutes on. But is the concept of RAG clear now? I hope so, yeah. I think, at least to me, it is, yes. I think, at least to me, it is, yes. It's an important point to make, really, that sometimes, you know, you have your LLM system, you have something that almost seems a little bit like magic. It is a little bit like that, because it does something, and you don't quite know what it does and how it does that. There's obviously some kind of calculation of probability, how the answer is formed. And obviously, that makes sense to the system. But as a human, you cannot look into it and cannot understand how, becomes visible in the end. So it's tricky. It's tricky to formulate, to find out, okay, why do I find my baseline here? But I think this is why it is so important to explain this and how this can be more, yeah, the grounding is the proper term for it, to connect it with your data to make it more relevant and um yeah more explainable and more explainable controlled because you know think of it in in in scenarios i mean because ultimately what we're doing is building building applications that that do things for us and and think you're coming up with uh uh you know a, a medical diagnosis. You don't want the large language model to be creative in that context. You want to provide the right contextual information in order for the large language model to come with an explainable and valid answer. And the same, you're going to make an investment. In any case, in any critical type of system, you want to have control over the information that's available to the language model. So that's the idea with RAG. And that's what we are going to, yeah, to keep exploring today. So if it was just in chat here, Wenliang, absolutely. If you have questions, just type it in chat. We have a look for it. Either we slot them in as they fit or towards the end of the session. So anytime something is not clear, type it away because then we can directly elaborate on it or if you have a generic question towards what we're presenting then we we find we find uh the time for answering good and uh well before i go because i know there's three four aspects in which knowledge graphs improve retrieval augmented generation. But I want to provide a space line, probably the default, not the default, but the most simple approach to retrieval augmented generation, which is using a vector database as grounding part. I don't know how to refer to this term, but to the external knowledge base that provides the context to the RAG experience. So the idea is that this search is based on a vector exploration. So essentially what you have is a question or a request from your large language model. And this is typically vectorized. from your large language model. And this is typically vectorized. This vector is sent to a vector database or any kind of vector search enabled platform. And that will provide a number of approximate answers. We saw a minute ago how when you have a vector representation, you can have the nearest, the approximate nearest neighbors. And these will be the vectors that capture bits of information that is potentially close to the to the question that you're asking. Now, it's critical here for a number of reasons. I'm going to actually go over them is it's critical to get the right information, of course, and the best information and the most precise information we have to understand that the amount of information that can be passed to the large language model is limited. So there's a hard limit on the size of the context. And that keeps growing again. But an important element there is cost. The cost of the utilization of large language model is strongly related to the amount of data that is exchanged and provided as context. And also, of course, providing a large amount of context has implications also from the point of view of performance. I mean, it has to be parsed, it has to be analyzed, it has to be built into the answer. And so we have interest in providing the best possible contextual information and the most precise and minimize it in terms of amount and by removing superfluous or irrelevant information. So that's the point where I'm going to talk about ways in which knowledge graphs improve and actually, yeah, actually improve the the the construct retrieval augmented generation solutions compared to the pure and just basic uh um vector search so what we have in the fj is is a graph of course and and and the graph uh has uh nodes and these nodes have properties that can be uhed in a vector index, right? That's from 5.11 back in August, we have that available. Now the first thing that graphs provide is this thing that I think Joshua and some others and all these are Neo4j members that have been publishing, you know know relentlessly on this topic what he calls structure awareness so the idea is that when we have contextual information typically it goes through a process that's called chunking so again because we want to limit the amount of information that's passed to the large language model we normally break it down in smaller fragments so it would be great if i could pass all my internet so it wouldn't be ideal because you know this this the the confidentiality and and the data sharing uh component to it but you know it would be great if i could pass just all my information to the large language model and then just ask questions but uh that's for the reasons that i mentioned not ideal. So what we will want to do is break it down, and that's a common approach in what's called chunking, this unstructured data into fragments. Now, what we can do in the graph is build into these chunks also this document structure. These chunks are parts of documents, and documents have structure and organization. And I mentioned here two posts that have been uh published over the last few weeks months by members of our team the one to the left is a chatbot for the graph academy that adam cowley published a few weeks ago probably a couple of months already and what he did is is he analyzed the the pages in the Graph Academy, and he chunked it based on the different sections in the HTML blocks, and he kept a reference to the original page in that case, or some representation in the graph of the hierarchical structure of the documentation in the web pages. Something very similar is what Joshua in the one to the right, and have the URLs to to go and look at that in more detail and even the code he's looking at decomposing a document into sections chunks even tables there can be several elements and keep pointers to the document to the container why do we? Well, sometimes when we do a vector search, we might find a close match in one of the sections, but we have relevant information in all the components of the same document, of the same section, or the same whatever the container structure is there. So one thing that we can do with graphs is navigate this hierarchy, navigate this structure representation, and expand and augment the context by providing related information that might not be picked up by the pure vector search, but that we are aware of because we have this structural representation of the document. Does that make sense? I'm going to take a pause here. I'm going to take a pause here. Yeah, yeah. Good start. I think it does. Yeah. So that the first... I see no questions. Providing this structure awareness and that's the way of augmenting the context in an informed fashion because we have these additional information. The second one... Oops, there's an animation there. No, no it's animation you're just taking long too another one is is what we have not necessarily uh uh document structure so necessarily the information doesn't come from chunking documents and and trying to preserve the structure in the graph but it comes from a combination of data from different sources and uh and you have some textual information that's embedded in vectors connected to uh all the bits of information that are uh structured in the graph and in this case i mean one great example of that is the the the example that came came with the with the gen ai stack that also was released a couple of weeks ago and that um GenAI stack that also was released a couple of weeks ago and that I believe is Oscar Hane and some others published where it was based on on Stack Overflow data right so what we have and this is this is the simple model so Stack Overflow as you know is organizing questions and answers right so and both questions and answers are natural language. So fragments of text that can be embedded. We also have information about users and questions are also tagged. So we have some rich model there that can be explored again to provide an augmented context for the retrieval augmented generation solution, right? For the retrieval phase. So what are the kind of things that we can do here? So we can do, based on my question, I want to understand how, which are the parameters that algorithm X in graph data science takes, right? And give me some details about that. So maybe there's a question that matches that, and it's probably the closest to the one that I'm looking for, and I will find that with some vector search, but then which of the answers do i provide maybe systematically if there's if there is a an approved answer i will provide that answer so i will link i will expand because probably in order to provide an answer you know it's not so much the important bits to uh to map a question but to provide the fragments of text that will help the llm provide uh an answer to it so this navigation will augment the um the context again that we can provide to the to the llm and similarly i mean you can provide the approved answer if there's not an approved answer you can look at which of the answers is provided by the by your user with the highest credibility so you can really uh leverage the the structured information in the graph to navigate and expand and extract additional information that's that's uh relevant to the to the rank process that's very similar to the example that thomas created with um um it's a simple example where he combined unstructured information in the form of tasks right so tasks uh it was uh I mean he described it as a as a DevOps uh a rack application and and you find their information about teams uh involved in tasks uh the tax the tasks affect microservices they have dependencies between each other and individuals are members of teams so again it's a combination like in the other case of uh different entities different graph information with a a node that contains textual information so in this case what uh what he shows uh that's possible to do is to go and do a vector search on the text of the task and then expand it by saying, well, this task is actually currently active and it's associated to that particular team in which this person is. So again, this is a way of augmenting the information that we have in the graph where the vector index provides just the starting point and and the the approximate match based on the question there's additional patterns or templates i believe they're called that thomas has been working on and that's that's really really super interesting and he's contributed those to the langchen. Unfortunately, they don't have the URLs here, but I'll add them after the session, where these more sort of advanced rag patterns were explored, and he provides code on how to, and I think that the two or three that he covers is the parent pattern, which is similar to the one that we saw in the previous slide. So we have a document that has been chunked in a number of fragments. You can, when you find a match, navigate up to the document and find siblings to provide an additional context. So that's the parent template or pattern. I can't remember the right terminology there. There is another one where, in addition to each of the texts, each of the fragments, it provides hypothetical questions. So actually that's interesting because you have some fragment of natural language text and you can ask a large language model to generate some hypothetical questions that you can annotate this content in the graph. And then you can do your search when you ask questions on these hypothetical questions to enhance the probability of matches in this approximate search and then navigate to the actual content by doing this graph exploration to augment the context for RAG. I think that was the hypothetical question. That's the parent. There was a third one. I can't remember which one it was. But a couple of other advanced ones that leverage this idea of context augmentation by combining this initial vector search with graph exploration. And well, I could keep going. But the other one I can think of is fine-grained access control. So in a vector database, it's really hard. It's impossible to to be fine-grained about who can access what right because all you're doing is is exploring a vector uh that's the result of of embedding natural language texts in a graph i could be very uh very specific let's go back to thomas example i mean you can do a search on a task, but because you're a particular member or you're a member of a particular team, you will only get visibility over the tasks that are assigned to your team. And that's a great way in this case, not of augmenting, but of filtering and not providing information that should not be visible to you. So that's another great way in which graphs can help in that front. The one that we're gonna be looking at today, and we started looking last week, is this combination of vector search and graph search through explicit semantics and using ontologies and taxonomies. There's more, but I think I'm gonna pause here because I've been talking longer than usual, and it's probably time for some code. But maybe you have some questions there, Alex? There were a couple of questions. So we have Tomas, by the way. I think you posted the link to his GitHub repository. So I'll put that in chat. I'll add that to the video description on YouTube as well. So you can find it there if you're wondering. At least this first one here from Ben Young. He says, I can understand the rationale of using knowledge graph in context to vector databases as context. However, the main bottleneck for me is how to construct a reasonable knowledge graph that best serves the purpose. It seems to me that create vector database is much easier than construct a knowledge graph. Yeah, absolutely. Yeah. Yeah, totally true. And the answer is, well, this probably we will not have time to go in detail on that. But again, let's think of, I mean, a knowledge graph can be the combination of data from multiple different sources. But probably what he's thinking is, OK, we have unstructured data and I want it represented as a knowledge graph. And, you know, I can chunk the data and I can take that text and embed that fully. I can fully automate that. And that's simple. Now, the process of translating that into a graph is a bit trickier, but still large language models can go a long way there. And what I'll say is maybe I can provide a pointer, because again, that's a topic that some of our colleagues have covered, and Tomas in particular. I can send a pointer to the post, but that's something that I'm planning to cover in future sessions. So you can, again, take unstructured information, tell the large language model, kind of the target model that you want to populate. I mean, what kind of entities basically you want to extract. Somehow you provide the ontology of the knowledge graph that you want to extract somehow you provide the ontology of your of the knowledge graph that you want to create and the large language model will uh return the data in a format that match much as that uh that structure so that can be that can be done and then again the idea is that these will be combined with additional information like we saw in other examples so yes i totally agree the construction can be can be challenging but there's a lot that can be done with that with the use of large language models and and um yeah i'll i'll post the the the link or maybe thomas is online i'm sure he will put it there but uh if not we will do it and and again uh the plan is to have a a session specifically on on that particular on that particular topic. And we kind of did, I mean, we didn't use, you know, large language models, strictly speaking, but if you remember in the previous session, and I'm going to run it very quickly in one go today, what we did is we run an NLP step, right? So we took the, we were looking at articles, and what I did is a pass of natural language processing for entity extraction, right? So what I did is extract the entities that you can find in this document and then link them to my taxonomy. So in a way that was probably the most rudimentary step in the direction of constructing a knowledge graph, right? So it's entity extraction. And the next step would be also relationship extraction and then curation of all that. I mean it's not a simple process and again Thomas will show his scars from that and but it's definitely something that large language models can help with. Cool, yeah. And the other point was from Jaime. LLM's context window size growing constantly LLM's context window size growing constantly has, it's more like a comment, has a similar effect as RAGE, I'm not keeping saying it, RAGE is not right, RAGE rack, on the validity of the outputs. Do you think that could help? That would eventually keep up with it? Yes, of course, that will. that could help that that would eventually keep up with with it the fact I mean yes of course that that will I mean the the problem is not so much what's technically possible or what the large language models offer in terms of size of the of the context window it's also the cost associated with that and the precision of the answer I mean think of it I mean if you and the cost the financial cost because that's how they how they you know, the cost is, like I was saying before, directly related to that, but also the time that it takes to process that contextual information sometimes, you know, and it will depend on the on the application, the rag application that you're building. But if in order to answer a particular question about x or y, you have to process a 50 page document, which I think it's technically doable. I wouldn't recommend it, but it's something that can be done. I mean, that will probably break the interactive experience, right? So, and, but yes, I mean, this is something that we should expect to be constantly changing and growing, of course, but that doesn't remove the need for us to be to be precise to be um to be as granular as we can and and and to make sure that we only pass context that's as relevant as possible and as as as reduced as possible to the to the large language model i would say yeah yeah good okay well I'm conscious of time. And I wanted to, I probably will not be able to go all the way to the end. But at least I'm planning to go the first steps. And the idea, if you remember, and I'm going to run, I had my, you still can see my screen, right? Yeah, put it back on. So I have my uh um you know a freshly uh created um sandbox uh as always uh if you go to the repository you'll already have some some information and what i'm going to do is run in a single go this whole chunk of code that what it does is reconstruct the the graph uh that we created in the previous episode and while it runs hopefully there's no errors there i will quickly go over it so remember how we loaded a collection of articles that were represented in in notes articles that just have the title the body and the time uh the date time the time stamp of the publication then we uh enriched these articles with that with the taxonomy that we took from dvpedia or was it it was dvpd i think it was wikidata i might have got that it was wikidata i think yeah exactly so i've probably named it just randomly i don't know out of my head but anyway it was it was a rich taxonomy of of programming languages technologies of all sorts so we we loaded that using NeoSemantics. And then we did, I mean, we didn't really. I mean, because that's time consuming. I mean, and it's risky to run in a demo live. I didn't run the actual natural language processing to do the entity extraction. I had done it upfront and I've saved it in a file. So I had extracted entities. So for each of the files, I already had the result of these analyses done and what I did is I basically linked my articles to the topics in my taxonomy that were actually explicitly mentioned in those articles. And finally what I did is I created a vector index and populated it using again embeddings that I have initially pre-computed. All this becomes really, really simple, and that's something that I will touch from the next session, using things like langchain. I mean, the moment you start interacting with vector indexes, providing natural language questions, interacting with LLMs, I've been fascinated by the richness of this orchestration kind of framework, langchain, that really helps a lot here. But in this case, I'm going to use basic Cypher, and that's going to be it. So hopefully that's done exactly. So we have it here. And just as a reminder, we had the articles, a bunch of articles, and hopefully the articles should have, there you go, so we have articles that mention entities, and the entities, if we navigate, will be connected through subclass of, or some kind of taxonomical classification to other entities forming this kind of semantic layer, this taxonomy, this rich context in the graph. So that's the model that we have. And if you remember, what we were doing is we described how we could compute similarity using both vector similarity and graph-based similarity. So if I go first with the simple one, the way to do vector based similarity was copy that. And so what this query does is, okay, I'm reading an article. So in this case, what I'm going to expect my retrieval augmented generation application is okay, I'm reading an article, I'm going to, I wanted to do to write a summary, a recommendation of what should I read next with some context. So in this case, rather than sending a natural language question, I mean, it's not always a chat. Imagine that this is a, I want to build an agent that analyzes my, you know, the articles that I read and sends me an email at the end of the day or at the end of the week on what should I go and read next. And I wanted to write an email describing natural language, but generated by the large language model. So the kind of thing that I will do is, okay, I'm reading this article, that's one of them, and identify it with the URL, and given this article, which are the ones that are close to it? And that's something that we did last week. If I've made no mistake there, all I'm doing here, let's look at the query. It's as simple as, well, I find the article and then I do a vector search on the article embeddings index for all the articles that, and I set a limit of the number of them that i want i'm set to seven but it could be anything uh based on the on the embedding representation of the content of these articles so that's this kind of implicit semantic similarity which returns i mean the article that i was reading a number of similar articles and the score again the problem here is that if I want my llm to write this um email saying hey you should read that uh maybe my best bet and and I'm open to suggestion because that's something that will probably treat in the in in the next session would be I don't know to to pass the the the maybe a summary of each of the articles to the LLM and say, well, that's the summary. Tell me why these two are related. Because I know they are related because that's what my score is telling me. But can you articulate it? It would be a generated answer. So it's like a curated one, but it would be generated. Maybe that would be a way so if I pass you know these uh the article plus either the maybe the full the full text again depending on how big I want I want the context to be or maybe a summary maybe that would be the context that I want to pass to my my LLM and that's exactly what I do in this um in this um um notebook that's already committed in the repository as well. So again, this is purely a cipher base. So I connect to my graph. I, if you look at what I've done, I've just literally pasted here the query that I run in the browser. And all that this query does is takes the URI and it will return a summary uh let's run this in the form of you know here's the title here's the body i mean it could be maybe the like i said before a summary or some kind of uh simplified view but again a little bit into this uh second zoom yeah and you zoom in a little bit yeah cool thank you there you go so that will be the query i mean the the retrieval uh uh request on my graph so given this article which is my my input this is the output i mean this is determined by my vector index and this is what my llm should use for the recommendation so you should go with these these and these or and explain why and the why is the tricky part because you know my best effort is to give you my you know the the raw uh text of the article and maybe you know the llm can can cleverly uh generate something indicating how the two are similar now the other way to approach this is the vector-based one. And I think, well, and this is actually what I'm doing here. And that's my, again, I'm entirely new. So I'm open to feedback here. So I believe what I'm doing here is I'm, that's not working. I've done something wrong wrong but what's it so my idea here is i've created as a basic prompt saying i've read this article and tell me why i should read next uh in a brief description of why based on the provided context and the context is the output that i've just received now maybe uh what is it what's going on here so this oh yeah the the the window maybe one thing i can do to trick these at least to make it work for now and again that's what i i was saying um and this is a bit of a hack just for the for the live session but you know some some uh smart approach like i would say summarizing might be probably better here what i'm going to do is totally truncated so i'm going to say okay set a dot body to a dot body some string let's keep the first what uh three thousand characters there you go boom maybe this is gonna be small enough i don't know still too long yeah maybe i'm passing too much or or the oh another thing i could do here is instead of returning a file, I could return just three, right? Again, I'm playing here. This is totally experimental. Three, if we exclude the one that's two, but maybe that's, looks like it went through. But again- It's just for sharing exactly. So it doesn't, I mean, for this purpose. This is the part where I'm really being totally experimental and I'm planning to have a session with Tomasz on this to make sure that I'm approaching it in the right way. And again, the Langchain framework offers a very, very, very powerful and simple, intuitive way of automating the whole chain, like the whole retrieval and the passing to the to the to the large language model and producing the answer. So here here I'm intentionally decomposing it in two steps. So I'm querying first, I'm returning the context, I'm passing the context, but it can be all bundled in a single in a single uh and a much more compact and intuitive uh or way of of representing a good lang chain but you know let's let's keep that on the side for now and and well my model returns something i mean it it's probably not ideal and like i said maybe i could be more i can be smarter on what do i pass as context but i want to provide this as as one approach so that's that's the the information that I get using a pure vector similarity now there's this monster query that we worked on last last week which does let me look at the query in detail first so that's the that's the query and this is what it does so the difference here and let it run so it's that for the same database and the same article what it's doing is providing similar articles but remember i'm going to give it a nicer name here let's call this uh and like i said i'm going to explain a little bit of what's going on here so as you can see to the right it's it's giving me kind of an explanation of why this result is relevant because this is based on the path exploration on the graph based exploration of the of the of the similarities between the two so i'll go a little bit into the detail of the of the cipher but as you can see it's telling me you should read this one and the reason why is because the original article mentions explicitly neo4j and this the recommended article mentions also it's the neo4j that's a simple recommendation but it's you know it's it's explainable it can get a bit trickier for example like for example uh new sql uh an implementation blah blah blah well the original article mentions neo4j uh the second one it mentions new sql neo4j is a type of graph database a graph database is a database management system new sql is a database blah blah blah blah. So it gives us this collection of statements that provide, I mean, it's kind of a textual representation of what we saw last week. You remember how we could see the path that connected the two and we could determine these two are similar because there's a path that connects the two. What I've done here is I've translated it into text, into statements, because the way we communicate in a RAG environment, the context is passed as text, into statements, because the way we communicate in a rag environment, the context is passed as text, as natural language, or semi-structured text. So this is the. I would be passing this type of context to my LLM. And the way I work it out, and that's what I don't want to scare people, because I know this is a bit chunky, but here, I mean, it's not super complicated. It's just a bit verbose. But again, we start with an article, which is our starting point. The article mentions a number of entities, and that's what we did with the entity extraction. I can do a path similarity search where I i set the threshold and that will return a number of related topics and for each one of them i can get um well i can get the articles because the topics will be linked to articles and then it's all about how i format the output so that's where most of the code is used and uh and this is the the this is how I compute the similarity, which, like we discussed last time, because there can be multiple paths. Well, I take only the average of them. I think that's what we're doing, right? So I believe we return the average somewhere or the minimum. Collect. Yeah, it's the average here. So I compute the average of the similarities. But the interesting bit, or interesting and at the same time the verbose, and that makes me think that maybe that's something that we want the function to return out of the box. So this return is a path. And what I'm doing in this section here is translating a path into a collection of statements and without going into all all the details i'm iterating over the notes basically over the relationships in the path remember that the path is telling me why why this article is related to this other one and i iterate over the relationships and i say well the relationship is a verb and i can get the source and the target node and by doing that i i generate a a collection of maybe triple statements subject predicate object or node relationship node and um yeah it's a bit verbose and but i it probably it probably should be something in the same way that the path, the result returns a path. We could do a version of the path similarity that returns a textual representation and basically hides that cipher. Kind of a tricky part here in the question. But that's pretty much it. And back to our context, to our notebook, I can run exactly in the same way, the same query for the same article, and that would provide a similar but different context it will be a collection of articles but it will return hopefully the text that um just taking a bit too long let's leave it to run but um one important aspect of these and I think that's the whole reason why we do this. Oh, transaction failed. Couldn't connect. Maybe some, maybe my sandbox. Since there's still being that, right? I don't know. We'll give it another try. That's not a problem. That's the thing. Maybe it's my Wi-Fi here. You still can hear me, right, Alex? I'm not. I i can yeah yeah i can still hear you it's all good but yeah maybe the sandbox had a little hiccups so there it seems to be not connecting properly but anyway the important thing and the reason why we did that is because uh if you remember uh we have control over how these recommendations uh happen and and the way to prove that is that okay I can extend my my ontology my taxonomy that's new knowledge that I have now so my recommendation at the moment is this one so that's what you should read and this is the reason why but then I say hey but there's knowledge in my organization that basically but then I say hey but there's knowledge in my organization that basically states that there's this notion of technology stacks so something that's maybe not aware to my large language model is the fact that there's technology stacks that combine different technologies and let's look at what these if I do instead of input I do a preview before importing it. So this is a classification where I say, okay, there's something called the GRAND stack that connects GraphQL, React, Apollo, and Neo4j. There's also the MEAN stack that connects MongoDB, Express, Angular, and blah, blah, blah. Anyway, that's another way of connecting things and that will probably provide new paths for exploration. So I'm extending, again, my knowledge graph by importing these additional taxonomy elements. So if I do import, is that what I did? Yeah, import. So these will enrich my knowledge graph. It will add a bunch of triples. That's represented in the form of, again, of an ontology. And the result of doing that is that I will come back to my query. Let me delete it to make it more obvious here. So if I run exactly the same recommendation query now, hopefully I get new results. And see that effectively you know up to I'm going to have to reduce it a little bit so that we can see the two side by side but the first three are obvious because they both mentioned directly Neo4j so that's natural but then look at the third one so instead of being this new SQL now suddenly becomes a beginner's guide to GraphQL and the important thing is why that's what i get why this new thing has popped up well because the original article mentions neo4j and the recommended article mentions graphql neo4j is a grand stack component and so is graphql there you go you have the connection so this context can be passed in the form of sentences to the large language model to provide an explanation of why you should go and read this maybe because you're interested in the grand stack or maybe because you are at all these are that again I'll I'll pause here because I'm I'm I'm realizing that this uh we're hitting the the time but you know yeah it's timed out somehow. The notebook doesn't like to, doesn't want to talk to the sandbox, but they're there and you can, I hope you get the idea. So we've kind of covered how to get the context. Now, like I was saying, if I, and very rudimentary how to pass it to the LLM, but we're going to be able to bundle that in a very, very compact and concise fashion using using langchain and that's probably topic for the for the next session so i'll take a pose here get back to the stop sharing and hope that made sense and that made that made lots of sense i think that's great you know you you said it's a bit verbose. It's a bit very monster query-like. But I think it makes sense because we have the problem that otherwise we don't quite understand how does the recommendation come to be. I mean, it tells us you should read this article. And why? And it doesn't give you that explanation and through through through going step by step we we give we give these we can see the almost we can see the calculations we can see how how it came came to pass and that that is that is an important bit yeah which which is not to be covered the the yeah the the explainability the attribution because you know again remember that if we ask the large language model to provide an explanation it will provide it i mean it will generate it that's the thing but i want i want real attribution and i didn't complete it there but i could provide the uris and things like that like saying hey these uh you know you can say this is the the article that you should read and maybe provide a reference or if think of um adam's uh chatbot for the for the graph academy he was able to provide a link to the point in the documentation where he got the answer from so hey the the the parameters that the uh you know jacquard algorithm takes are these and that and i know because it comes from this document as opposed to i know because because i'm an llm that can generate an answer exactly my probability score tells me that this is the next value i have to print out yeah explainability and attribution are extremely important and also equally important is the you probably saw that we have control over the information that's used to deliver the answers i have no control i mean i could retrain a large language model i could you know fine tune and we could go it's a completely different and way more a more complex and expensive process what i can do is i i is have full control over my grounding database my grounding partner to the llm which is my knowledge graph. And I can add, remove, modify over time information. And that will have an impact on the behavior of my RAG application, like we saw in this example. I think this is exactly the comment from Peter here on traceability. I mean, this is what we want to achieve, right? Why are we building a rack tie-in? It is because we want to be contextualized. We want to be traceable. We want to have a grounding. We want to know why a query or a prompt to my LLM comes back with an answer and why is that answer and where is the the the base mind the trace that you know we're tracing back where does it come from in essence exactly yeah no I was I was saying because I really I'm reading in the in the Q&A that there's a question if I will share the notebook it's already in in link in the in the github repo i might need to i mean i don't know if it needs updating or there's some error there but the one that i've run today it's already there so feel free to again especially on the lang chance lan chain side is it's not perfect and that's something that i'm a bit of a novice on but i'm i'm working hard on it so i I'll and I have good teachers here. So exactly. So the the the the repository is in the in the video description. So if you if you click that, it's a GitHub link to to the going meta repository. If you don't find it, check check the description of the video, then you should be able to see it um yeah william agrees tracing back is the key yeah and then another question from will um does uh new semantics latest 5.11 work with the current new project 5.13 i guess you would be the perfect person to answer that question yeah we're working because know, still struggling with, you know, keeping the pace of releases with the platform together. But yes, so the 5.13, I mean, 5.11 works with 5.13. So that's, I mean, what I wanted to do is have uh somehow no uh you know the nifj desktop to default to the latest to the most recent version of neo semantics instead of saying there is not a version available for you but unfortunately that's the that's the way it behaves at the moment and and uh but yeah it's just one additional manual step which is go and and copy the the 511 jar um into your um plugins directory in your 513 and it should all work fine yeah cool all right um yeah i think we are at the end uh of the session um again uh please try the repository, try the playbook that Jesus provided. If you have any comments, any feedback, let us know, either in the YouTube comments or on the community forum or on Discord or anywhere else. We'd like to reach out and we can have a look at it. It's always nice. If you like that video, if you like GoinMeta as a series, please give it a thumbs up on YouTube and share it as much as you can. Obviously that helps out a lot so we can continue doing this in the future. I think with that, looking forward to next episode, December, final episode of the year coming up. um that's gonna be probably then a follow-up to this episode i assume um yeah somehow it feels like if you're not doing something lm related you're kind of uh outside now yeah probably i mean there's a couple of open questions that we that we left today and it might make sense to but you know maybe something else will come up you know we take suggestions don't we so if someone yes we do dying to to get us you know to to talk about something in particular please let us know and we'll we'll take it into account but yeah perfect that's good well with that have a great week everybody uh great rest of your day uh and um thank you very much, Jesus, for obviously the presentation, but also for live coding again, bearing the demo gods yet another time and succeeding with it. So that's really great. Thank you all. Thank you for watching. Thank you for your nice, friendly comments, as usual. And yeah, hope to see you for the next Going for your nice friendly comments as usual and yeah hope to see you um for the next going meta episode in a month in the meantime um there is a couple of sessions coming up so i'll give you a quick little little teaser um for what's coming in november in two weeks time on the 23rd. I'm talking to Elena Colway, famous for her being a ninja, about user-defined procedures. So that's Thursday 23rd of November live here on Neo4j live. And yeah, with that, have a good day everybody. Great to see you all and until next time. Thank you.
Going Meta -  Ep 22: RAG with Knowledge Graphs
Episode 22 of Going Meta – a series on graphs, semantics and knowledge Jesús Barrasa: https://twitter.com/BarrasaDV

Links from the Show:
Vector Search: https://bit.ly/4cuZoeS
Educational Chatbot: https://bit.ly/45RvnDz
Structure Aware Retrieval: https://medium.com/@yu-joshua/adding-structure-aware-retrieval-to-genai-stack-373976de14d6
GenAI Walkthrough: 
https://bit.ly/4cxfPXW
GenAI App Building: https://bit.ly/4cxfPXW
DevOps Rag Application: https://bratanic-tomaz.medium.com/using-a-knowledge-graph-to-implement-a-devops-rag-application-b6ba24831b16
LangChain: https://github.com/langchain-ai/langchain/tree/master/templates/neo4j-advanced-rag

0:00 Welcome
6:35 Recap on Data Semantics
11:28 RAG
20:40 Knowledge Graphs to improve RAG
31:11 Q&A
36:25 Code Example
55:50 More Q&A
1:00:55 WrapUp

Repository: https://github.com/jbarrasa/goingmeta
Knowledge Graph Book: https://bit.ly/3LaqE6b

Check out https://community.neo4j.com/ for questions and discussions around Neo4j

#neo4j #graphdatabase #knowledgegraphs #knowledgegraph #semantic #ontology #rag