 Hello everybody! Good morning, good evening, good afternoon and yeah, hello, welcome to the final episode of the year of Going Meta for 2024. It's season two episode number four. Can you believe it's December already, Jesus? It's crazy. So many angles, it's just time just flies, isn't it? Yeah, it does. It does. It feels like we just at the same time, it feels like we are just into April, but also in, you know, somewhat some completely different dimension. I have a question for you, Alex. Are we are we starting a new season in January? Or do we still always like an academic year till September? It's an academic year. Yeah, that's a good... No, I think we can continue with season two for a little bit more. I don't think we have to get up to 20 episodes for season two. We can make a break earlier. But yeah, as long as it feels like one story, I think we continue here. And then we maybe go about a season three eventually, maybe. Absolutely, yeah, no, we'll probably continue all the way to the summer, right? Might be a reasonable time to take a little break. Good, yeah, it's one of those questions that suddenly comes to you at the most unexpected time. Yeah, no, it's a good point. But I think we have a little bit more to tell about the topic we are covering since the beginning of season two. So I would say let's continue a little bit more in that direction, and then we can. We have plenty to tell. I think we're not going to run out of topics for the next four to five years. But yeah, probably not not no that that's true yeah so but yeah we we have uh uh an interesting topic today it's kind of like um you know we we progress here a little bit more but if you if you tune in for the first time if you if you watched a couple of episodes but haven't watched it as regular don't don't worry um we'll we'll keep keep you up to speed or or don't hopefully don't lose you uh this episode but you know you can always watch watch previous episodes um afterwards uh but today's topic is um and we try to make it i mean we always try to make it short and and interesting but this one yeah just again it was a bit difficult but i mean yeah we never tried to to pass it to to chat gpt maybe he could have given us a reduced version of it anyway but yeah that's that's uh that's our our challenge every every month every episode to try to come up with a concise title. But you know, it was hard. I mean, if anyone has any ideas of how could we have shortened these, you know, more than welcome. But yeah, we've spent the last three episodes focusing very, very much on the knowledge graph construction, right? The knowledge graph engineering part. So how do we build it out of specifically, mostly unstructured data or mixed data. And we looked at how to do it with the knowledge graph builder, you know, in a kind of no code fashion, where you just have to point and click and things happen for you that we look at the code base approach. And then we look at some kind of blueprints, as we called it, on how to approach more complex cases. And while I wanted to touch yet another angle, which is basically the use of the Graphrag Python package, I also wanted to look at it for the first time in a more kind of holistic way. So that's where the end-to-end comes, because we ultimately were building a knowledge graph, but we're doing it to then be able to do something with it. And to do RAG is where we're basically going. And I thought that that was a great opportunity for us to have a little tour of the GraphRag package. And I know you've been talking about it in other sessions with other people. Actually, that's part of the context. I'm going to bring up my slides, shouldn't I? Because I'm not sharing. And let me find my... I'm going to share my entire screen so let me share this one and bring this out of here and i'm probably sharing the wrong one right so as usual many okay not not what i was looking for but we'll find it here we go that's that's uh that's what it is so um yeah i was let's go on slideshow at least for this introductory part so yeah here's the the long title but i was saying that you know the the reason actually the context and there's always a reason why we bring something up is you know because i wanted to to take the time to work a little bit with the with the graph rag uh package and like i was saying uh i mentioned this was just a few weeks ago right the session that we had with alex and will on the yes that was uh exactly wrote to nodes workshop um ahead of ahead of nodes which happened um early early november so uh the latest one was end of October. So yeah, about one and a half or so months ago. Let me share the link in the channel as well to the workshop. Yeah, it's a very good one that also connects with the content in the Graph Academy, which also uses it. So I think it was a totally relevant topic. There's one of Zach's most recent posts. And then the, I actually forgot to put the URLs. I'll put them later. But that's the documentation, the manual pages for the GraphRack package, which has had a recent new release, and that's kind of a continuous thing. But yeah, so that's sort of the context and why we wanted to connect that with what we have been doing over the last few sessions, few episodes. So that's the plan for today. So this is the idea and that's kind of an extension of what we have seen. And we always, in this series, we always want to go meta, right? So we just not want to only use the package, but we also want to, in this case, to have its behavior driven by an ontology, which is something that we've seen in the knowledge graph engineering, knowledge graph construction part, but not so much. Yes, we have, because if you look at previous sort of season, we did it a few times, right, in the kind of advanced RAG sessions, episodes where we looked at how ontologies could do that. But I think that this was a good opportunity of connecting that with the Python package, because we're going to see how the ontology is, again, relevant in how we define what the structure of the graph is going to be, what kind of entities, what kind of relationships, and we're going to be able to inject that in the entity extraction step. We're going to see how it also, and that's something that we've only mentioned but not really got into, the entity resolution part. One of the problems that we saw is that, for example, Alex, how we imported data from unstructured sources, and we ended up finding nodes that were actually the same thing, but were not being merged properly. So we'll tackle that in at least a first iteration, a first pass of that, and we're going to see that the Python GraphRack package is going to help us, but it's going to be driven by whatever we define in the ontology. That's the objective, as always, is I don't want to write the code. I want the code to be generated for me or to run for me. And all I have to do is describe in the ontology what I expect to be the behavior of my whole system. And then it also, again, it's used in the RAG step, because there's many RAG patterns, RAG approaches, RAG strategies that the library offers. And we're going to see how the information in the ontology can also feed that. And I've kind of marked in yellow there all the elements, all the steps in the blocks that are actually implemented with the with the graph rack package so so these are the things if we have time that i'm planning to cover and um yeah of course at some points it interacts with llms as we've seen for the entity extraction for the rack part and uh ultimately there's also neo4j acting as the other knowledge acknowledge graph platform where we're going to be storing the graph that we built. So that's the idea. Make sense? Yeah, that's cool. It's good to see that this is pretty... It's called the GraphRack Python package, but you think maybe it's only usable for one specific use uh if you if you don't think about it or if you don't read about it what it can do but now this this this graphic here shows that it actually is much much much more versatile and can be used in uh into different um you know parts yeah yeah the entity recognition part and integration part i think is both are you know topics that came up ever since we we touched this for the first for the first time i think every every now and then we get the question hey but what about this uh how about a duplication how about uh you know recognizing my my entities basically what what do i how do i map them correctly and uh it's good that we you know it's it's it's it's difficult but we try to you know and and this this uh this package really really simplifies it you're going to see how the the code is there's going to be a fair amount of code but it's all pretty pretty straightforward to follow and and yes to your point i mean, I mean, this is the documentation. You can probably pick the link there and share, but you see how there's the three main elements. So there's the Knowledge Graph Builder part, which we're going to look at first. Then there's the Rack part. And then there's the notion of pipeline, the definition of a pipeline, which is a whole construction. Even remember from a couple of episodes how we wanted to build a graph that had the two parts, the domain graph and the lexical graph that chunk the unstructured data. So, so these will take care of all that. And it's very well described. So I encourage everyone who wants to play with it to go and look at the documentation because it will show all the steps and all the capabilities in the package, which is pretty comprehensive, as you said. And similarly, on the RAC side, the different approaches, the different types of strategy, the different retrievers, vector-based, vector and cipher, and custom retrievers. Lots of really, really high quality stuff in there. So that's what we're going to be doing today. So that's it. Yeah. And to quickly maybe cover, because there was a question in chat from Zaki Coder, and they ask, is ontology necessary or recommended so if you i guess if you if you work with the graph rack python package you don't need an ontology um you can just use it without without the ontology but because of these previous episodes where we had the ontology as a core part of um of us trying to to model data trying to get data in and trying to help with the ontology to make it a little bit more sensible. We are using ontology again in today's episode, but if you don't want to use an episode or you don't have an ontology available for you, then you can just not. Yeah, exactly. And if you go to the examples, I mean, in the documentation, you will see that in order to do entity, I mean, you can do blind entity extraction, as we've seen, but you will want to give some guidance to the entity extraction element on what you want the graph to look like. And you don't need to use a formal ontology like we're going to be using, you have to provide some kind of description, but then you're going to use probably the same description to feed to the rag step when you want Cypher to be generated on the fly out of some text. So you might as well have a standard, formal and controlled way of doing it, which is exactly what an ontology does so we find it a convenient way of doing it but strictly speaking I mean you may call it an ontology or not but you have to have this kind of definition and it might be more or less formal it will be managed in one way or another and and it's entirely up to you how to you how you want to do it yeah yeah that's a good point maybe too ontology always sounds like such a big word you you have to you want to do it yeah yeah that's a good point maybe too ontology always sounds like such a big word you you have to you have to set so many things up but all you do if you if you're working with a set set you know types or entities or labels or whatever you call it properties you always have to set up usually we recommend this set up some kind of semantic definition of what's what, basically. And this is an ontology, so you would create this anyway, but maybe it's just a very small one for your specific use case, for your specific application. Then it sounds like ontology has to have thousands of entries and listings of things. It can be very small, very tight, but it is, in essence, an ontology has to have thousands of entries and listings of things. It's just maybe it can be very small, very tight, but it is, in essence, an ontology still. Absolutely. And like we all say, never reinvent the wheel. Reuse what's out there. And we see how our ontologists pick things from schema.org, from maybe your organization has an enterprise ontology that already exists. And again, don't boil the ocean don't you know start small and uh and then grow from there i mean the powerful thing of this approach is that you can grow the ontology incrementally and and then repass this construction pipeline and keep adding elements to your graph so it doesn't have to be a daunting exercise it is it's daunting exercise it's kind of a natural thing that you're already doing and that there are tools that will help you kind of putting a bit more structure around and that's exactly what we're going to be showing today cool should we get started well just I'm going to again apply again, apply what I'm preaching, which is reuse what our datasets that we've used. So we had that from a couple of episodes where we looked at the Tate Modern dataset. So I'll reuse that. I've extended it a little bit, but basically it's going to be information about artists, artworks, and that's what we're going to be building the graph about so um let's uh let's start let's get things going and and i'm going to jump straight in the in the code and um let me uh well first of all let me know if that's readable i suppose that's more or less okay i think it should be fine. Yeah, from what I can. Perfect. No, this would be good. Yeah, if you think it's too small, please let us know in chat. But from my viewpoint, this looks good. Perfect. So I have a Neo4j database, an empty one here running locally, and I'm going to start, I've kind of broken it down in several parts, so I'll go through them in detail. The first thing that I'm going to do is just create the vector index. As you remember, we're going to load unstructured data, this description of artworks from the Tate Modern, and we're going to create a graph out of it according to our ontology. So this produces two graphs. Oh someone's saying that it's blurry. I don't know if that might be down to my broadband, my internet connection. You know it should be fine but you know maybe check that you actually receive uh in in uh 720p at least or i think linkedin only supports 720p um if you have the the highest receiving resolution if you still don't get a better picture you can always try to switch over to YouTube, which should support up to 1080p. But from my view, I don't think, no, I don't see your blurry, meaning you could try to zoom in once, but just to make it a little bit bigger, but then it's very big. So yeah. Yeah, I mean, the problem now is going to require a bit of a. So I was saying that we're going to produce this kind of double graph or two-layer graph where we have the lexical graph that's the one formed by the chunks that are created out of the unstructured documents and then the domain graph where what will have the entities that we are going to be extracting. This is purely initializing. As you can see, I'm pointing where's my local Neo4j instance is running locally, my credentials as you can see, high security, that's my user and password, and I'm just giving my index a name and defining this is important, the dimension of the vectors that's going to be storing. Other than that, it's just literally creating the vector index and giving it a name and specifying that it will be applied to entities, to nodes labeled as chunks. And in particular, we want to embed the property called embedding so that's that's all this does so if if we run it there will be no output only an index has been created on the neo4j side and we notice it because there is a um kind of a node label there and if I do schema, we will see that there is now a vector index here that's online and that's called chunk index just like I called it. So nothing more than that. That's just pure initialization. And with that, we're going to go straight to this simple, I've called it Python code. This is literally a copy paste with very, very minor changes of the example in the documentation. So I'm talking about this is the graph rack package. If you go to the Knowledge Graph Builder and you go to the, I think I took it from here, the pipeline structure, components, if it's described here, here we go. So I think there's a default pipeline, maybe it's in the actual GitHub. And yes, we will be sharing the code with you. It's in the episodes GitHub repository. There's a full repository. The link should be in the video description already. And each chapter or each episode has its own folder with all the code that is shown in the episode. So yeah, I was saying this is actually in the GitHub repository. So if you go to GitHub and we'll share the URI as well, you have a simple example here of an implementation of a simple pipeline, which is called simple knowledge graph pipeline, as the name shows. So I've literally copy pasted that and I've made small changes to it, but that's what we're going to be following. Other than the imports at the top, and I haven't been very careful or very clean on my... I still have the definition again, but this could be factored out, obviously, what's the pointer to my local instance and my credentials, but basically, I'm creating a driver, I'm going to be interacting with my Neo4j graph database. I create a driver instance, and here's where things start to diverge a little bit. We said we're going to have an ontology that's driving the behavior of our pipeline. Here's where I'm reading the ontology. Some of you must have seen us using the RDF lib library, which is probably the most popular one in Python to deal with RDF data, which we're going to use to read the ontology. The ontology is here. I'm not going to show this time. We created it in previous episodes. I think we use normally protg or web protg, some of the open source ontology editors there. But it's like you were saying, Alex, it's very, very, very simple. So this is what we call the art ontology. And there we define, I think it's in different order. So we define two classes. We define the notion of an artist and the notion of an artwork. Right. And all we did is say is that an artist is a class, a category, which happens to be a subclass of person. There's an artwork, which is a subclass of creative work. As I was saying before, we connect it to the schema of the orgontology. That's optional if you want to do it. But we're defining a category and we add a small comment indicating what that is. So it's a creative of artworks and an artistic creation. That's the definition of the two main categories that I'm caring about. Then we have relationships and attributes. If we go to the top, we see that there is a property called artwork known as, which is a pretty long name. I could have called it title, but why not? So that's just to show that. Actually, it's a subproperty of name, but it's a property of type string that applies to artworks. It's the name, the title, or a way of identifying an artwork. The same with professional name. Professional name is the name of the artist, but because artists sometimes don't use the real name, they might use a pseudonym or something, I decided to give it a... But, you know, again, to your point, Alex, this is a very simple enumeration of these are the categories, these are the properties. It's using the RDF standard. No one's meant to write this, so we create that with an application which is visual, we save it and it's stored in this RDF format. Same with the relationships. We say that an artwork represents a person, that an artwork was created by an artist, these sort of things. It's a very simple definition. The good thing is this is formal, we can parse it and we can make use of it. We're using it here to create the schema out of the ontology. Let me explain what I mean by that. Actually, let's put this on hold. So we read the ontology that's telling us what the graph is going to look like. Now, the thing is it has to look in a particular way, which is the schema that the graph rag package expects. And this method is actually applying the transformation. I'm going to show you in a minute what that looks like. But that's what we do. So we provide the schema. Then we need a splitter, which is basically a configuration on how we want to chunk our input data. So I'm going with some standards. Well, again, there's no standard, really. Whatever works for your case. I'm saying that the chunk size I've been playing around and these behave reasonably well is 2500. There's a minimal overlap so that there's no abrupt cut between the chunks. Then for each of the chunks, I need to create embeddings and I'm using a service, in this case, the OpenAI embedding service, and I'm using this service, in this case, the OpenAI embedding service, and I'm using this small model. I didn't want to go for a larger one. I think that's sufficient for this example, but not real code. I mean, this is just like I was saying, copy paste from the other case. And then there's going to be an LLM instantiation for the entity extraction. And again, standard, all that specifies which model do I want to use. I'm going to go with GPT-4.0. I may want to limit the tokens that I'm exchanging to keep the costs and things like that. So this is exactly the configuration that I've copy pasted from the previous example. And here's where I instantiate the pipeline itself. So I pass all the elements that I've created, the driver to interact with the graph, the LLM to do the entity extraction, the splitter that I've configured there, the embedder, the service for creating the embeddings, and ultimately I pass here the schema. This schema is in the form of a list of entities, a list of relations, and a list of potential schema as it's called, but basically it's triples of the form source, target, and relationship. Basically, I'm indicating the Knowledge Graph Builder how to construct the graph. I'm telling it to ignore if there are errors. I don't care too much for this experiment. And in this case, I'm not parsing PDF. I'm parsing text data directly from these files that I've downloaded. Makes sense? So this is what I'm going to run. And what I'm going to do basically is iterate over the files that I have. And this is just a local download. I wanted to avoid the usual problems reaching out to the internet and having issues. So this is just the text. It's the text that comes in the description of the artworks that you get in the modern catalog. So it's just a text out of there. All natural language text, as you can see, there's nothing structured there. And this is just a loop I'm saying, well, iterate over the files in this directory, which is the data directory, and pick the ones that end with txt. And for each one of them, all you have to do is read the context of the content, sorry, of the file. And then just, I indicate that I'm going to be processing that and just run the knowledge graph builder I run the pipeline and it will do the whole thing it will process the text chunk it embed it entity extraction and then populate the graph I don't want to run it on all of them so I'm going to to add an additional and file name starts with maybe just a bunch of the Hockneys. Hockney and image. So, it's just a way of telling it to load just one, two, three, the first four or five, right? Not all of them because i'm committing out this line where i do the the the resolution the entity resolution afterwards but i want to do it in in two steps just to see how things work so i'm going to save this and i'm going to run this so if if i if i run this step and hopefully there's no errors this is going to start processing each of the files one by one. Image of Kent, so probably it's going to run these three or four. And as it does, it's going to apply the pipeline that I was describing. So chunk the files, create the lexical graph representation in Neo4j, and we can go to Neo4j and start can go to Neo4j and start see the results. So the first thing that we're gonna notice is that we see a series of chunks, right? So this is just fragments of texts that have been embedded. So you see that's where the embedding step has taken place. So we have called the service, the OpenAI service that takes these texts and creates an embedding. And it all happens I would say transparently but quite easily quite compact way because I didn't I only have to configure the the elements in the pipeline and it does the whole the whole work for me so um right so now these uh these chunks contain the the kind of unstructured data vectorized but at the same time we have been extracting entities like artists, artwork, basically the elements that we described in the ontology, and persons, and then they are connected through the was created by, represent, works with, influences. So these are what we call the domain graph. Actually, we can maybe look at it with a query. Maybe let's check if it's finished. Yeah, it's done. So it's loaded five of the files. So we have the chunks that are going to make it possible for us to do the vector search, the vector rag. But more interesting is the domain graph. And for that, I can do something like, let's look at what's created by. So we see small fragments here. So we see the problem that we saw last time, right? So we, let's put it like that. There you go. So we see a number of artworks, right? And for each artwork, we're seeing the properties that we describe, which is artwork known as, I know it's a bit weird as a name, but it worked well, basically picked up my recommendation, which is, it describes what this is going to look like, and it's calling it artwork. And in the same way, David Hockney has a professional name instead of just a name. That's the way we expected things to be. Now, the problem is that I see one David Hockney has a professional name instead of just a name. That's the way we expected things to be. Now, the problem is that I see one David Hockney here, another one here, another one here, and these are meant to even Hockney without the first name. So I would expect these to be all connected, something like that. So this kind of entity resolution don't know if in the in the artworks there's some case but there might be also an image of cilia there you go so this is this there's duplication here as well so you see these two other things so now i'm i'm not being totally fair because by default this pipeline that i've just run in the GraphRack package will do a first pass of entity resolution. But without any configuration, all it will do is it will take things that have the same label and the same name. Because I'm a bit particular and I've not used the name property, I've used my own professional name and things like that, it didn't pick it up. But if you use the default name, you would have seen some of the duplication. So don't think that that's magic. It's the default behavior. Now, because I don't, because I'm not using that, I need to configure it myself and say, well, these are the properties that are meant to be interpreted as kind of primary keys. So if you find two David Hockneys of type person, type sorry artists well then these are the same pair of nodes so this is exactly what I'm going to do in the second pass so that's the better it's good to remember because if you do this you will have probably something that matches with predefined sessions like like like persons or title or, but you will have something completely on your own, which is obviously not predefined and you will have to then help the system because otherwise it'll make these errors. Absolutely. There you go. I can extend it and get all the other properties and we see that it's produced a pretty decent domain graph already. The yellow ones are persons. For persons, why do we not have the name? It's interesting. There might be some, sometimes the chunking makes, it knows that there's a person, but it doesn't get the name. But sometimes we get it right. So not a problem here. So this is already an interesting graph. So we see Hockney at the center because he's the author that we're looking at with some of the artworks. Sometimes the artworks represent individuals, right? So these three paintings, image of Celia, red Celia, and Mr. and Mrs. Clarke actually represent the same. Now the problem is, as we were seeing, that there's duplication. So we want to solve that problem and also the additional data. So let's go into the second step, which is the entity resolution or the duplication, if you want to call it like that. And again, that is going to be driven by the ontology. So that's the bit that I had commented down here. So there's the notion of a resolver in the GraphRap package. And quite conveniently, there's one that's called a single property exact match resolver, which has a long name, but it's exactly what I need. So basically, I want to say whenever you find an artist, there's a property that I want you to use as the primary key, which is the artist, what did we call it? The artist name. And again, my idea is I don't want to write additional code. I just want to describe it in the ontology. And the way you do it, this might be getting a little bit detail, is by using this keyword called inverse functional property. So that's, in simple terms, primary key. That means that when two triples have the same value for that property, it means that the subject is meant to be the same. Or in other words, if two artists have the same value for that property, it means that the subject is meant to be the same. Or in other words, if two artists have the same artwork known as, then they are meant to be the same. And the same applies to the professional name. Sorry, no, that's the artwork, not the artist. The professional name is the one that applies to the artist, and it's also inverse functional. And the artwork known as is like the title of the artwork. I mean, that's not the case for all the types of relationships, but it is for these two. So what I've done is, again, I've created a function that will go to my ontology and read, is there any definition of these sort of primary keys? Because if there is one, and this is all in the utils function, so it's a very simple parsing that gets the primary keys. It basically reads the graph and gets all the instances of inverse functional properties, and I return those. And that's exactly what I'm doing here. I'm going to go and read these inverse functional properties, and I'm going to create a resolver for each of them. I'm going to say, go and apply this entity resolution on the list of primary keys that detected from the ontology. Just to continue with the test, what I'm going to do now is I'm going to comment out this fragment. Just a bit dirty hack to rerun exactly the same code. I'm going to run this, but now it's not going to run the pipeline, it's going to go straight to the resolution. If I do that, that's very quick, but it's done it now. If we go to the graph and let's keep the old version here, and I rerun the same query, we should get a very different graph now. And we see things linking a little bit better. Now we still have the problem of Pablo Picasso and Picasso and Hockney and David Hockney, but all the David Hockneys have now been merged. So you see how we had one, two, three, and four David Hockneys. They have all been merged into one, and the same has happened to the artworks. So basically, it has... We also had like four little graphs, and this is now one. Correct. So that's this idea of having the entity resolution built into the pipeline and driven again by the ontology. Of course, this is still not perfect, and this is one basic way of doing it. So I'm saying, in this case, if the artists are called the same, then they should be the same node. We still have the problem of Hockney and David Hockney. Here, we're going to be able to do several things. If you think of it, string similarity might not work because it's a big distance. If you compare them using, I don't know, Jero W or um um which one is the the other one that we typically use them well there's there's a bunch of string similarities string distance uh functions in in in apoc standard ones uh but these typically count uh with different algorithms kind of the the number of of swapping, swaps in the characters, right? Or the number of changes required to go from one to the other. What's the name of the other one? I never remember it. It's Levenstein. Levenstein is the other one. There's three or four implementations, but I don't think that is a very good approach. So, I mean, one thing that we could do is, in these cases, is maybe have some reference data that could help us. So in this case, we have from yet another episode, the Tate Modern catalog is one that we love and we use several times. We have a structural representation of all the catalog, and we have all the artists there. So we could have that as a master and we could... Thank you. Yeah, people are reminding me of Levenstein. So here you have to be a bit more creative and more custom in this case. So you might, I mean, you can build that into the pipeline, but you might want to create your own custom resolution in this case. This is a topic that we will work a lot more because it's richer and it requires a lot more attention. But I guess I'll pause here on that idea. We do a first step, very basic one of entity resolution here, and we'll look at it more in future episodes. But that's the second step if we go back to our diagram here, right? So we, if you remember, we start from unstructured data, we do the entity extraction and also the chunk extraction, but that's kind of the easy part. And now we've done the entity resolution on the duplication, and both steps are driven by the ontology. Now, the data is in the knowledge graph, and the third sort of step is that we are going to interact with the graph, to ask questions from it, and we can do a number of different things here. So let's go back to the code. And well, actually, why don't we maybe clean the graph and now run it with the whole list of files, which will create a richer graph. And we chain basically the construction with the application. And now I can remove the additional condition. I don't want to run just the Hockneys. I want to run them all. But for that, I'm going to maybe, I can empty the graph. And I could even, well, if we want to completely wipe things out, I mean, this is not really neat, but I could drop the index that had been created. Oops, I didn't paste it. Oh, I thought I had copied it. There you go. So that drops this index. And we are also going to drop the vector index. OK. Just to show how we can get rid of everything. We can drop this index here. And like this, oh, there's a dash in it, which it doesn't like. So I'm going to escape it. OK, so now the database is back to its original state. So if I run schema, we have just a lookup on nodes and relationships, so entirely empty. And we can run it again. So we start by recreating the vector index, it's just the one-off thing, and we can go back to the simple data pipeline, knowledge graph creation pipeline, that will run the whole thing on all the files, and then run the entity resolution on the primary keys that are defined in the ontology. So if we run that, let's give it a few seconds, and it will process. They're not massive files, so that should go relatively quickly. And we'll see the graph show up on the other side. Should we take a minute to see if we have any questions? Yeah, I think that's a good point. There was one from Submendendo and uh he asks about the the entity resolution if fuzzy matching could be could be something um or pm25 which i don't actually know what bm25 is absolutely honest with you also um yeah we we could we could and we'll explore all those all those options. I mean, there's not only a single one that does the job for us, as we're going to see. We could even, yeah, we could do semantic search. We could embed these elements and find this sort of semantic similarity, right? So we could string similarity, semantic similarity, right? So we could string similarity, semantic similarity, structural similarity. I mean, we will definitely look at things like leverage the graph data science library. So there's this idea of node similarity based on the neighborhood, right? And probably once this is finished and the thing that duplicates, we'll find that the two, Hockney and David Hockney, will have a lot of neighbors in common because they probably are linked to the same artworks. That's giving us a clue as whether these two might be, even if the name is only partially similar, they have a lot of neighbors. These metrics that we can get from the Crap Data Science Library is something that we're going to build also into the duplication step. So yes, I mean, all techniques will be relevant and we'll explore those. K&N, absolutely. Totally. Good point. And then John paid attention last time and he says, if I recall correctly, you were previously considering using Pidentic models to define the ontology. Did you find an issue with this? Hence, you're using RDF. No, no, no. Here I'm using RDF, but in a totally different way. Because, you know, actually, in a way, I mean, I'm totally abstracting what's happening behind the scenes in terms of writing to the graph. I mean, that's a very good point, actually. So I don't know how the graph frag package is actually interacting with the LLM to do the entity extraction. So that's a guess, but I suppose if it understands that the LLM can use functions, maybe it can pass a pydantic definition of the target schema. That's a very good one. It's not my choice here. I'm using RDF just because the ontology is defined in RDF, but that doesn't mean that I'm injecting it in the process. I mean, actually, when I pass the configuration, the target model structure, I'm doing it in a kind of natural language or even JSON format. So that's a very interesting one. I'll actually ask the team and try to understand how these kind of, whether that's generating cipher, if it's generating any kind of intermediate format, or how this actual approach works. It looks like it's thrown an error. What's happening here? Some, oh, it's retrying some, oh. Hognit, trade, self-clients, and what did I do? What did happen? I'm not sure what's going on here. To make sense of this following exception event loop is closed. This file opening element. Event loop is closed. I don't know exactly what's happened. I don't know if it's died halfway, but... Wow, it's... How far did it get? Let's see. What do we have in our graph? That's a good point. Let's see if we have the previous query so it didn't get to the it probably died before the um did the duplication looks like another problem we can run it manually so i don't know if it's run uh don't need to have it complete doesn doesn't matter, but anyway, so we can go to this step and just run the validation. That was quick. And hopefully that will give us some more, a better structured graph. OK. There's probably too much in it, but command scroll to zoom in. So OK. So we have even influencers here. We have works by Picasso and others. Well, that looks decent to me, so it might not be perfect, but that's enough of what we need to go to the next step, which is where we wanted to interact with the graph, to do RAC actually, to ask questions and to have the retrievers do the job of extracting the answer out of the graph. And we can do different things. So the first one is... Let's show the code here. And again, that's a pretty simple approach. I mean, I'm repeating the whole configuration of the graph connection and credentials, but all I'm doing is I'm creating a connection to the database. I determine which embedding service I'm doing is I'm creating a connection to the database. I determine which embedding service I'm going to use. Same as before, it has to be the same embedding service that we use for the Knowledge Graph construction. Remember what's going to happen in the retrieval step, in the rag step. So we're going to have a question in natural language. We're going to embed that question. We're going to have a question in natural language we're going to embed that question we're going to do a vector search at least for this first version of the retriever we're going to do a vector search find the chunks this is going to use the um lexical graph find the chunks that might contain potentially relevant information and we're going to decide which ones to keep. I can decide top one, top two, top five, however many. And then we're going to generate an answer out of it. And I don't know if this is still valid. I don't know if we have this element here. We can check it because that was from my previous tests, but we can take one example for example here red cilia or something from picasso for example la femme qui pleure so who painted that we're going to ask that to the who painted. And if I run that, that will, like I say, vectorize. Let's remember what's happened here. So it's going to take that question. It's going to create a vector out of it. It's going to do a vector search. That's what the vector retriever does. It uses the chunks that we have populated in our previous step. And then it's going to return, in this case, the best match, the one that we think contains based on semantic similarity, a vector similarity, the one that contains the answer, and then we pass that to the LLM. In this case, to, I think it's GPT-4, right? That's the model name that will ultimately generate the answer. And what's produced as answer is that Pablo Picasso painted La Femme qui pleure, which is correct, right? So that's the answer that we were expecting. Good. But that's not very exciting, right? Because we know that vector you know, vector search works, but it can hit limits or hit problems when the answer to the question is spread across multiple vectors and we have to combine, we have to rank, we have to prioritize. And again, I'm not going to go into that because, you know, we don't have time in today's session, but we explain how the combination of vector plus graph exploration solves that problem. And I would invite everyone to go to, I can't remember, episodes, I think it was in the 26, 27, when we talked about graph rag in the previous season, and we explained in a lot more detail how that can be improved. So, but what i'm going to to test before we conclude today is the text to cipher because that's the you know the the other case where i wanted the ontology to drive the the the creation of the cipher so i'm going to do something similar so now i'm going to have a different retriever, and that's what this piece of code does. Usual connectivity to the graph database, LLM to generate the answer. In this case, we're not going to use vector search. We're going to try to generate the Cypher query out of a question in natural language. And for that, we need to tell our LLM what's in the graph. And I know what's in the graph because what's in the graph is what I described in my ontology. I'm going to pass the ontology again to the Cypher generator. The Cypher generator, this text to Cypher retriever, will take a Neo4j schema. The Neo4j schema is nothing but the content of my ontology. I could pass the ontology in its raw form like we saw last time, but we saw that it was nicer and sometimes it behaved even better when we passed it in natural language. I can put the print here if you remember, Alex, but basically it just reads the ontology and puts it in a much more nice and readable form where it says, the entities are these, properties are these, and the relationships are that. So that's the context that we're passing to the text to Cypher Retriever, and that plus a question, we are going to generate a Cypher query. Again, let's pick maybe another one. So let's, we're gonna get um a hot new one maybe the master printer of los angeles there you go, for example. So who painted that? Let's ask the same question. And while it's the same structure of question, it's not very complicated. This time, the approach to answer it is completely different. And let's see how it behaves. Because there's always an element of unpredictability here. There you go. So I mean, we'll have to make sense. Let me make it a bit bigger so that it's more readable. Here we go. So this is the output. Oh, damn it. Sorry. There you go. Well, there's two parts. This is the output, and I should have printed the result in a bit more detail. But anyway, I didn't look at the structure. There's two parts. There's a Cypher query that has been generated that I can copy and show in detail. Copy. And then there's the output, which is Hockney. So that's the answer, which is correct, is what we expected. So it came to the right answer, but following an entirely different way. And let's paste here the query that it's actually run, because what's happened here is that it's managed. Let me remove the line breaks. So this is the cipher that the text to cipher retriever has generated, which is a perfectly valid one, by the way. So it said, well, I'm looking for an artwork that's called the master printer of Los Angeles, which is exactly the question that I was asking. And I know that an artwork is connected to the artist through the was created by relationship. And from the artist, I want the property professional name. So all this was not made up. It was actually injected in this previous step. Let me go back to the normal size. But it was when I passed the Neo4j schema. And the Neo4j schema is nothing but the contented Neontology. Let me show you what that looks like. I'm going to use the usual trick of commenting these out and showing you what I'm passing to the LLM. There you go. So this is the kind of natural language description of my ontology. So I'm saying the node labels are artists, which is the creator of an artwork. Artwork is a person, the properties are boom, boom, boom. So all these things that then it used in a very correct way were passed as context so that it could work out, it knows Cypher, it can speak Cypher. What it doesn't know, of course, is what are the terms that I'm using in my graph. And that's what I'm passing by sending the ontology in context. So I don't know, maybe a lot of code, a lot of jumping backwards and forwards, but I hope the idea was clear because ultimately we saw how the ontology was injected at the three steps. So the code that I'm going to share in GitHub is entirely reusable. There's not a line of code that was specific about the domain model that I was creating. So what you have to do is, not you Alex, but anyone attending, you can do that too as well, of course. But basically, you pick your data set. Now, you're not going to be loading artworks. You might be loading a product catalog from whatever you're interested in. You want to load the Apple catalog, the products, or you want to load literature or whatever you're interested in, you want to load the Apple catalog, the products, or you want to load literature, or whatever you're interested in. All you have to do is define an ontology with the relevant elements, like schema elements, what categories, what relationships, what properties you want to extract out of it, and exactly the same code that I've shared with you is going to work because we didn't write a single line of code. It's going to pick the ontology, that's going to determine the type of graph that we do. It's going to determine how do we duplicate, at least in this first pass of the duplication that we use, and it's going to drive the text to cipher creation. That's how powerful to the question that was asked at the beginning. It's really useful to have a centralized way of doing that, because that will be reused across so many different steps in our pipeline and that's that's where i wanted to to go today i hope that that was not too confusing i know when we show code sometimes it's it's hard to follow but it will definitely make more sense when people get their hands on it on github but um but yeah that was that was it yeah definitely yeah no that's very good and i think with the ontology again uh there was an interesting a comment and i'm not sure if you if you were aware of of this existence but there is a from getty um there is a the getty research institute they have an own ontology the a art and architecture the sorrows or a aat and basically is a library or an ontology on its own for um that's you know i would say artists as a whole pictures probably images um maybe videos i don't know yeah um so that's pretty if you are in the space i guess it's a good good good thing to look at and to to have like we said in the beginning don't don't try to reinvent the wheel if you if there's something already existing that you can use and this is a good example there are others for life sciences there are others for manufacturing form for you know material anything. If there's anything that is in your field, then you can use that and you can rely on that in addition to what you have for your, let's say, company or use case specific application. The standardization part has probably been done by somebody already, and you you can use that as a as a um you know as a back end which you don't have to do yourself again from from scratch very interesting yeah totally and i totally missed it and one of the thing one of my you know my complaints is why are these things so hidden sometimes why there's not like a kind of universal repository of all ontologies and standard vocabularies no i mean the answer is yes we didn't use it i mean we thought we were kind of being kind of applying good practice in the sense that we were not reinventing the wheel we were extending the schema.org but i was not aware of the existence of uh which makes a lot of sense a domain specific ontology so maybe it's a it's a good exercise to replace the ontology that we created, the simple one with the actual terms from the Getty ontology. That would be an interesting way of kind of refining this example that we run today. Definitely, definitely. Another comment here came in from John. And he says, I also wonder if a modified version of Anthropics contextual rag can help with deduplicating nodes, the idea of adding a summary of entities from the chunk source document to each child chunk. Oh, look. I didn't follow the modified version. We can definitely use Anthropocontextual.rag. One of the things that I was considering, I don't know if that's what you mean, John, is building into the entity extraction part, some kind of basically fine-tune the model to introduce things like reference elements, so reference data. So if I have a list of authors, in this case, artists in my catalog, maybe I can, instead of passing it in the context, which would be too much and super high cost, why don't we fine tune the model so that whenever it comes up with Hockney, it doesn't use Hockney, it uses David Hockney or even a unique identifier for David Hockney. So if that's part of the fine tune, I don't know if that approach is even viable, but I wanted to explore that and maybe, you know, enriching the entity extraction in that way could be one way. So, i'm i'm totally open to to exploring all these all these ideas absolutely yeah another comment from cameron here would love to see how it performs if we can automate passing schema right from the graph itself this is yeah i mean in in what step but at the at the retrieval because yeah that that's one of the ways in which we can do it, too. I mean, in this step here for RAG, if you don't have an ontology, you can always, you know, introspect the graph and get the labels. So you can get somehow the ontology from there, too. And I would expect the performance to be comparable because by injecting the ontology and the entity extraction step, we guarantee somehow that the graph is going to be aligned with it. So whether it comes from the ontology or from introspecting the graph, I don't expect it to be very, very different. So if that's what you mean, I totally agree. I think that's what he means, but yeah. Yeah, and I don't know why these are so super hidden always. It's hard. I think there's no... It would be great to have one page or like a repository of ontologies. Maybe there is one, but we don't know. If there is one and if you know one, then let us know, and we can we can talk about it maybe at some point um about all these different ontologies and um and yeah how to customize them as well because this could this is another question is how some of these ontologies are very very big and it's almost overwhelming especially i could imagine the getty one is like you know imagine you're into let's say art and the Getty one covers videos and pictures and all of the other things. How do you shrink it down? You don't want to ingest a whole super ontology into your data if you don't need all of this, if you just need a specific part of it. Exactly. Yeah, yeah a that's a fair point yeah totally because um yeah in this case we're injecting the whole target schema but maybe yeah maybe there's there's a previous step of of because we knew what kind of data was uh in the input files and what kind of data we wanted to extract but yeah i suppose if you're in an entirely blind space where you probably need to do some form of sampling of the source data and then identify the parts of the ontology, which ontologies might be relevant for your case, I mean, that's taking it even more meta, right? So figuring out what should be the ontology to be fed. But yeah, I suppose that can be, that's another area to explore, right? How to determine which is the portion of the ontology in an automated fashion that's relevant to specific source data. Yeah, and then if you, like what Benjamin just said, I would like to answer questions using two knowledge graphs to answer questions of chemical compatibility for products used in art restoration. So you have arts, the whole ontology, and then you have chemical and they're, they're all, so you, you combine these two and then you, you have all of that together. So it's a, yeah, it's a cool project, yeah, definitely. Super, yeah. Yeah, Sumendu is saying that he's never written an ontology. I don't think you should write an ontology. I would say go to Web Protégé, which is something that we've used in the past, Web Protégé, and use any of these visual environments. I mean, that's going to be a lot easier. So, I mean, I don't know what I have here. I know that we've created some in the past, but going meta 19, I don't know what was in there. Oh, the clothing. So, there you go. So, you can, this is a very visual environment where you can define, that's a class, you give it a label, a description, you can even do, this was the one about multilingual, multiple, so you can have multiple languages and annotate it in RDF and use it to have different representations. But yeah, you can create additional classes here, then you would create the properties. In this case, you would create, you know, so use any of these visual environments. And all you have to do at the end is just download the project, and you will get exactly the format. Because these are W3C standards. So these are serializations. If you go back to the, where would it be? If you go to home, I suppose you can go and download. Here we go. And you can, there you go. You can specify which serialization format you want. So RDF XML, Turtle. So all these are, the one that I was using is Turtle. So if I do that, you just need to download, and it will give you the file that you need. Yeah, that's the kind of process that I would recommend. So go to WebProteger and do it there, rather than, if you're keen to write the RDF yourself, totally go for it. But it's a much simpler yeah exactly great yeah I think with that we we are at the end of of the episode today thank you everybody for for joining and for your questions and thank you Jesus for presenting today and going through the three three use cases for for the the graphic Python library but I think there are more but these three in a in a architectural overview so I think that was that was really interesting that it is more versatile than you think on the first when you look at it on the on the front i think it gives you more options than you actually realize um the next session and this is going to be the last live stream of this year of 2024 will be on the 17th of december where uh our dear colleague zit is going to join me to talk about the demystifying the mahabaratha as a chatbot with graph rack so this is a a huge tome of of knowledge and like a story i would say like um and he will show us how how the data is is is put into a graph database to use it as a as a chatbot to talk to. So that should be very interesting in two weeks' time. Our graph, Neo4j Graph Developer Survey is still up. So if you'd like to give us your thoughts on how you learn, how you discover new things, how you find Neo4j pages that we make available to you, and if you can improve something then use um use the link to to let us know and then finally if you are now just to do some more stuff the graph academy courses that be depending to the graph um rack package they are now available on graph academy other courses as well so if you are interested in doing that, there's also a Knowledge Graph one. So if you like to extend that knowledge from a little bit further, a little bit more deeper, then check out the Graph Academy. Check out Aura-free if you want to get a free Aura instance to play around with Neo4j and check out our community and developer pages for questions around anything really. There's lots of documentation, a lot of great content available, the developer blog with lots of interesting in-depth reading material, but also the forum on our Discord server where you can ask other Graph fans and developers for help or if you want to comment something. And yeah, with that, thank you so much for watching. Thank you for joining today. Thank you, Jesus, again for being part of this great episode today. And I guess I'll see you all in the next year with Going Meta, Episode 5, Season 2.
Going Meta S02E04 – Ontology-driven end-to-end GraphRAG with the GraphRAG Python Package
Season 02 Episode 04 of Going Meta – a Series on Semantics, Knowledge Graphs and All Things AI
Topic: Ontology driven end-to-end GraphRAG with the GraphRAG Python Package

Jesús Barrasa: https://twitter.com/BarrasaDV
Repository: https://github.com/jbarrasa/goingmeta
Knowledge Graph Book: https://bit.ly/3LaqE6b
Previous Episodes: https://neo4j.com/video/going-meta-a-series-on-graphs-semantics-and-knowledge/

GraphRAG Python Package: https://neo4j.com/labs/genai-ecosystem/graphrag-python/
GraphRAG Pythong Package Docs: https://neo4j.com/docs/neo4j-graphrag-python/current/
Road to NODES GraphRAG Workshop: https://youtube.com/live/OALrsghrP_I 
GraphRAG Github: https://github.com/neo4j/neo4j-graphrag-python
WebProtege: https://webprotege.stanford.edu/


#graphdatabase #neo4j #graphrag #knowledgegraphs #ontology #data #python