 here we go hello everybody good good morning good evening good afternoon and thank you for joining the going meta episode two of season two with uh jesus how is it how's it going it's going very well but the question is how's it going for you because you're the one that's away from home today how is i'm away from home yes uh it's uh it's devox in marocco in marrakesh should be precise happening this week so we have a little presence there we have three sessions in the program we have a little stand um so i don't know it's probably very late but if you are happen to listen to this now and are in and around marrakesh come by the uh the event and then say hi um i'll be there uh obviously uh from from tomorrow on but yeah i arrived here it's very warm not to say hot uh it's very nice so it's good very different very different from what we're experiencing here in the in the uk i don't know if you can hear the rain hitting the hitting. No, I cannot hear the wind. It's a wet afternoon and pretty cold. So yeah, different experience, I suppose. But yeah, definitely. Exciting over there. Well, we hope that your Wi-Fi connection, so far, you know, you come great, loud, clear, and great video. So looking good. That's good. That's good. Yeah, I hope it holds so far it's been stable uh since uh since i'm arriving here so i think it's it's it's going going well but should i disappear all of a sudden you know why i'll either continue or go with some dad jokes like we've done in the past right yeah exactly Yeah, exactly. Yeah. So, yeah, I'm looking forward to the second episode. So we kicked it off last month with, you know, using, it's a mouthful. Our title is getting longer and longer. I know. So I thought maybe just to make it a little bit more readable and probably the season two, episode two can go at some point. But still, it is a long title. And we kicked it off last time with the trip into how to use ontologies, why they make sense and why it is a good idea to add ontology to guide knowledge graph creation from unstructured data. And we had an early peak, I'd say, into the Knowledge Graph Builder app that has been released a couple months ago. Yeah. And today, the idea is to take this in the next step because it's a lot and we couldn't really cover everything in the last episode. So we look at it some more, but this time with more code. So this is more or less a part two. If you haven't watched episode one last time, it's fine. I think you get up to speed quickly, and you can always recap on episode one later on. So as usual, obviously these are stringed together as to some greater idea, but it's not really necessary to have watched uh the previous episode i'd say that's right yeah and we'll do a quick refresh here on what we did last last month and and connect it with that because this is um you know kind of a different approach to the same same type of problem so yeah that's the plan good any announcements should we get started any thing we should know anything coming up anything coming so i can i can do this now yes um we can we can talk about this now um so the um the next session after this one is october 8th so that's uh next week 8th of october we talk about data-driven career guidance with GraphRack so I'll have a new guest on the show next week to talk about how to you know find the right people for the right job and how to develop yourself in your skills and your knowledge basically further on then Road to Notes is happening so that's another one happening also this week on Thursday so in two days time we do app building on Aura. So that's another one happening also this week on Thursday. So in two days time, we do app building on Aura. So if that's something that you're interested in, check out the Road to Nodes website. I'll share it in the chat as well, where you can register and can subscribe to this upcoming session. We have the others upcoming are Mastering Graph Rack and Learn Neo4j Bloom, so graph data visualization course. So these are coming up. But obviously, this is all leading towards the big one, which is nodes in 7th of November. So in a couple of weeks, and obviously today, this is as early as a going meta episode can be on the first of the month. But on the seventh, so it's a couple more weeks. So we have NOLES 2024. The agenda is now live as well. So if you check out the website, if you go to the NOLES website, you can see the agenda, see all the interesting sessions on the day. So yeah. I think that's what I want to get out to you ahead of or the episode depending on time I can always repeat that but I put the links in the chat for now perfect so I'm probably gonna share my screen and get started let me know if you can see it well first of course i'm gonna have to share it otherwise if i share this window will it be happy with him yeah okay perfect so here's the super long title extended even more but yeah the idea like you said is to continue what we left last month. So we were looking at building a knowledge graph. And we tried before in towards the end of season one, doing that from structured data. When data comes from tables, from databases, we looked at how to get the help of LLMs to do that, and even without LLMs. So we've done that in many different ways over the episodes. But since last month, what we've tried to do is unstructured data. So we're looking at data that comes from web pages, from documents, from whatever, but it's kind of a more heuristic approach. We're reading text and we're trying to structure it in a way that we can ingest it into our Knowledge Graph. And in episode one last month, we looked at the Knowledge Graph Builder. That was the kind of no code approach. It's a great app that you can go on a browser and feel free to review the episode, the recording and the assets. But all you have to do is point at your data sources. It can be, like I said, PDF documents, YouTube videos that you want to consume the transcript of, or like we did, web pages. So you can point at web pages, and the app will read them, will generate the knowledge graph, and will give you a... And thinking about it, I don't know if we even tested, but once you build a knowledge graph, not only you can query it in Neo4j, of course, but you have a conversational kind of a chatbot. I don't know if we tested, did we, Alex? So... No, I think we didn't go that far as far as I remember. Yeah, but the great thing about the app is in a single kind of environment, without having to write any code, you point at the sources, the knowledge graph is populated, and you have a chatbot that you can interact with, and that consumes the graph that you created. Now, we saw that there were some, and that was the core of last episode, right? So we actually, let me move to the next slide, because this is kind of the refresher of what we did, right? So we saw that we can do that without any additional information, but sometimes we want to take a little bit of control over the knowledge graph that was built. And the way to do that, it was through the app to provide, it's not so much an ontology, but a list of the labels, the types of nodes that we want to create, the types of relationships. And in our example, we saw how to do that by picking them manually from schema.org, from a standard ontology like that one. So we saw how we could pick concepts, classes, and relationships, like they call it properties in the RDF world, and we could just feed them to the app, and it would create a graph that's a bit more predictable in terms of the schema. But we saw that that was a limited approach, because we, for example, could not set properties or have control over the source and target of relationships. We could say there is a relationship of this nature, but we couldn't be more specific about what these relationships should be connecting. And you probably remember this funny situation where it was connecting Mozart to a location through a composed, right? So there was a bit of uncontrolled generations, which is to be expected when you throw an LLM into the game. But that was the idea, right? So we kind of show how we could, to some extent through the app, use an ontology to kind of put some guardrails on the graph that's generated, And ultimately, we did some validations using Shackle. But the idea, as we see in this diagram, is we have some unstructured data in different sources. We use that through some programmatic API, some form that involves LLMs, to populate our graph in Neo4j. And once it's in Neo4j, we can interact with it and do loads of things. So last episode, using the Knowledge Graph Builder. Today's episode, we're going to look at that using code, using Python. We're going to do exactly the same. So, that's the idea. We're going to get some unstructured data and build a Knowledge Graph, and we're going to get the help of LLMs. But this time, I'm going to do it in Python, and I'm going to show how we can use an ontology in a bit more detail so how we can be more precise and more strict about the the structure of the graph does that make sense yeah sounds good and we kind of can answer the question that came in chat just now from create the imaginable who missed part one um and if we were using an open source AI ontology. So I don't know if you're going to do this at some point, but in the past episode, we did kind of like crudely created our own ontology just to make the point. I mean, I can probably click back, and we still have the slides from last time. But we, I don't know if it's mentioned here. No, it's not in the slides. But we use schema.org, and we, I don't know if it's mentioned here. No, it's not in the slides, but we use schema.org and we extend it manually. So schema.org is a standard, is a public one, and we added some elements that were convenient for the, and we're going to do something similar today. And yes, but in any case, whether they are public standards or something that we create, it's going to end up in our GitHub repository. So you're going to be able to take it and use it and modify it. And yeah, yeah, absolutely. So yeah, I wanted to kind of do a more detailed description of the elements involving this. But that's, you know, I run out of time, so I'll fix that and when we upload it to GitHub, you'll have a better described diagram of the elements. But basically, the data that we're going to be using, it's one that we've played with in the past. I mean, we love the Tate Modern catalog, so... Yeah, that's great. And it's a great website, and, you know, it contains great textual description of some of the artworks so I thought why not you know doing a search for David Hockney for example and we picked a couple of two or three of the of his works and that's what we're going to be using so you can you have the url there and it looks something like this so if you click on some of the artworks, you will get the text. And what I've done is I've extracted the text and put it on text files. But you can do that in many ways, depending on what stack you use. But if you use Langchain, you know that it can go in pages and extract the text. But I'm going to focus on the ontology part and the guidance to the creation of the knowledge graph, rather than on the extraction of the text. That's why I literally collected that in a file, as we're going to see, and we're going to consume the text from it. And we did that a couple of times also. Looking better in season one, the Tate Modern data, I think it was also the New York Met Museum at some um museum at some point so if you if you want to know how this works check out the the previous episodes and uh you'll see it a little bit more i say long form yeah so yeah the the the approach that i've taken and we're going to go in details into the the text uh and the code is to um remember that the the and actually we mentioned that we might bring morgan or neil we still have to do that to to discuss you know maybe on the knowledge graph builder right because you know it's open course and some of the prompts that i'm going to use today have been inspired from the ones that exist there. So you feel free to explore the coding GitHub of the Knowledge Graph Builder we shared last week. And that's basically what we're going to be using today in my own kind of custom version, simplified version for these exercise. So I had asked, but unfortunately they are both out this week so it didn't work out but i'll ask them again for um for a couple of minutes uh next episode maybe we can do that next time yeah so uh and and you probably remember that uh the one of the things interesting things about the knowledge graph builder that we saw last week was that it created two different graphs. One that was a document graph that was based on chunking the content, the text, and creating basically a linked list of chunks. And then linked to the chunks, we had the actual domain graph that talked about the entities that were mentioned in the text. So in this case, whoops, there's an animation. In this case, I'm going to focus exclusively on the domain graph. So again, the chunking is something that we've looked at before that you can do with Lank, Chain, and many of these libraries. It's just about defining the size of the chunks and build. But I thought it was more interesting, the domain graph, which is basically the entity extraction part which is the one that's more driven by the ontology so we won't generate the document structure graph we will focus on them what we call the domain graph basically the entities that are mentioned in these in these articles okay so that's uh that's the introduction and with that you know let's get our hands on to the experiment so good yeah so i don't know if this dark background or is better than the other one hope that's readable i think dark is fine i think you need to zoom in a little bit yeah the problem is uh maybe it's more if uh yeah if uh i can try but it might get off the screen but anyway that's well that should be should be yeah right let's try this and uh yeah if if it's really not working let us know in uh in chat please perfect so uh you know if if we um let me hide this the idea as we said is we're going to ask an llm to do the work for us and and extract a knowledge graph out of some unstructured data. And to do that, we're going to create, well, we're going to use the system setting, and that's kind of a slightly modified version of the one I mentioned you can find in the knowledge graph builder, right? So I'm telling the LLM that you're an expert in extracting structural information out of natural language text. And you can extract entities with their attributes and relationships, et cetera, et cetera. And you can produce the output as RDF triples or as Cypher. In this case, we're going to use Cypher, but we could generate RDF triples as well, depending on the request. And basically, the prompt looks something like this. It's pretty, I'll go into the details in a minute, but you see that, right? So given the ontology below, and I'm going to show how to pass the ontology, run your best entity extraction over the content, and the extracted entities and relationships should be described using exclusively, let's say must be described, to be more, must be described using exclusively the terms in the ontology. So I'm being very explicit about the fact that I want the LLM to just describe the extracted entities using this vocabulary, the ontology that I'm going to provide, and in the way they're defined. And by that I mean, explicit about it, this means that for attributes and relationships, you have to respect the domain and range constraints and domain and range if you remember is for our relationships where does it start and where does it end right that's the domain in the range for example the author relationship connects a book with the author right right? And that's the direction. So the domain would be book and the range would be the author. So that's what we specify in an ontology. We're gonna see that in a minute. We've seen it in many episodes before, but that's what we specify. And in this case, I'm asking it to return the output of a cipher using merge. And the merge, of course, uh for linkage of nodes of course if i do multiple passes i expect entities to be uh to be mentioned in multiple documents and multiple text sources and i would like that to build over time as i run the process uh to build a a single graph and merge merge is the perfect way to do that. I understand that this is not very rigorous, and we'll leave that for iteration three or part three of this because that's. So but this is kind of the entity, the duplication, right? So we're going to see that David Hockney is going to be mentioned in multiple sources. We, for this case, we're going to merge them on the name, which might not be perfect. Probably there are not many artists called David Hockney, but you can think of things that are on many locations that are called Boston, right? There's one in the UK, one in the US. So you have to be more precise and we'll go into the space of unique identifiers and that's that's a separate topic that I would like to leave out for today let's assume that we simplify the problem in that way we're going to focus on the structure of the graph right and I say absolutely no comments on the output sometimes the LLMsMs, you know, they add something like, this is the model that I've come up with, boom, and the output. I want it to produce something that I can use to populate my graph. Yeah, that's important. And I think here, another comment here from Saha makes a lot of sense just to remember, be as precise as possible with these prompts and be as narrow as you possibly can be. So here, Saha says, must is really important, otherwise it sounds it does not listen. And that's probably true. If you give it a range or some vague description, it'll vary a lot and you don't really know what you get. But if you say, okay, this is what you have to do this is what i want and then this is you know still probably in some way but uh you know on things like the temperature right and how yeah exactly for creativity you leave and maybe you run the same the same code on the same source and you get slightly different things. But with these, at least in the test so far, and I get pretty consistent results. And that's the reason why the ontology. So what I pass is with this prompt, I concatenate the ontology. I'm going to show in a minute how do we get it. And the actual content, which is the natural language text that we get from the Tate Modern catalog pages. So that's pretty much it. Then I pass the system information in one single call. In this case, I'm using directly the Python library for OpenAI. I didn't use a Langchain. If you want to see Langchain, we'll use it in other episodes. But in this case, I go with this simple approach. So just the system information that I show at the top and then the prompt and I'm using GPT4O in this case, right? So that's what I'm doing. Now let's go through the different parts. So we are passing the ontology, right? The ontology that I want my extraction to be aligned with. So if we look at the code, we see that... let me do one thing at a time. I'm going to comment this. So the ontology is read from this file. So it's called... I think it's called art.notfinal. Let me fix that. That's the name of the file, as you can see. I'm going to show it in a minute and for that i'm reading it using uh rdf lib which is the more popular python uh um parser well it's a it's a python package that deals with rdf and a number of things so we um we parse the the file that i'm going to show in a minute and we just serialize it as as ttl i mean literally we read it we could have just read it and pass it through, but I wanted to show this library because we're going to use it in a minute for some more sophisticated things. So that's how I get the ontology. It's simple. And then the content, I read it from the files, and you will see that we have some files here, and they're pretty straightforward. So what I've done is I've copied the text text and we have a text uh description and just the text information that comes with each of the of the uh artworks right so that's basically copied from the the website uh about the description yeah to simplify the code i mean i'm just reading from this file but you could go access to http paint and extract it blah blah but you know the interesting part is in the ontology. So we're using this art ontology. For those of you who are familiar with Turtle language with RDF, you will see that there is a bunch of property definitions. But this has been edited. And instead of showing the code, I'm going to show. I must have it somewhere here. I've edited it with... Of course, I logged out, so let me log back in. So I used Web Protégé. Web Protégé is an ontology editor, one of the free one that you can access online. And in it, you will find a number of things. What I've done is define a very simple ontology. And the reason why you see so many items here is because I've linked it to, let me see where it is. Here we go. To the schema.orgontology, for example. And I've done it intentionally because I want to pass myorg ontology, for example. And I've done it intentionally because I want to pass my own ontology. It's true that to some large language models, they might be, because they're trained on public data, they might be aware of schema.org, right? So if you tell chatGPT, well, GPT-4 in this case, produce the output based on schema.org, maybe you don't even need to pass it. I mean, I wouldn't trust that because I've seen it generate some, you know, something accurate, but something generated. But in this case, I wanted to show how we can pass our own ontology. And in this case, just good practice is to, rather than creating a random one, what I've done is I've created it extending schema.org. And there's a bunch of of uh of classes that i've created not many but um let me find it here on the thing there was uh person and under person i've defined artist right so this is uh it's a bit small we can be there can i make this bigger yeah i can oh yeah this is better yes this is good yeah so it shows it in in the whole hierarchy so person as you can see is a schema.org one so that's not me that's i i'm importing that ontology and what i've defined is artist by the notion of an artist and you see that it's my own because it's it my own namespace. I created neo4j.com slash vocabulary slash art which is my art ontology. So I've defined the notion of an artist and I say that an artist is a subcategory of person. So I define artist in the same way I've defined, I believe I've defined artwork under creative work. So there's plenty of them, but I've created artwork which was not there as a subcategory of creative work. And you can see that I've added a description which also helps to the LLM. That's an artistic creation. In the same way with the person, the artist, I said is the creator of an artwork. What else was there? I think there's relationships that I've created. For example, there's object properties that we call relationships in Neo4j, like was created by. So was created by, as you can see, is a relationship that connects an artwork to an artist, right? So he's kind of the the creator but specifically it's kind of a refined relationship that's specific for artworks and artists i've also created a represents a represents is kind of a relationship as you can see that connects an artwork with something that's in it so this is a portrait of alex so i would create a note representing alex uh an artwork and I would create a note representing Alex, an artwork, and I would say, Alex is depicted in this, and the name is represent. So that's how I've named it. And importantly here, that's what I was talking about, domain and range. So the domain is where the relationship starts. It's an artwork that represents a person. In this particular case, I focus on persons, not something in general, but persons. What else? There's the notion of an influence. So an artist influences another artist. So that connects an artist to another artist, indicating that the former is an influence to the latter. And this is, of course, a reflexive, not reflexive, but it connects an artist to another artist, indicating that this has been influential to these other ones. So anyway, there's a bunch of relationships there. There's represents, there's created by, and there's some data properties, so like the attributes, and I've created my specific one. For example, instead of using the generic name in schema.org, I've refined it to create artwork known as, right? So it represents the title or an identifying description of an artwork. Sometimes the things don't have a title. I mean, paintings tend to have titles, but maybe displays or other types of artworks maybe don't have such a precise description. So it's kind of a more general term. So I call it artwork known as. And its domain is it applies to artworks and the range is a string so it's a value that indicates that it's not a relationship but a property of an artwork and the same for artists. I've defined professional name as an extension of the name so it's the name used to refer to an artist so it may not necessarily match the real name but maybe it can be a pseudonym or whatever. But it's a name for the artist and the intention was to show the case where we potentially are extending existing ontologies like I've done here but I'm using my own which is specific of the art domain. Well this has has been defined in schema.org. In Prodge, it can be exported, and that's what I was showing. But just quickly, Jesus, it makes a lot of sense because, I mean, obviously, if you're doing this in a business environment, you probably are not importing artists, or maybe you are, but if you're not, if you don't, then think about it. I mean, you could talk about product names and product could be the product name. Once it's final and released and ready for people to buy, it might have or probably very likely has a different name than when it was still in development. So, but still it would refer to that. So internal documents during development would refer to the code name and then afterwards the code name refers to the final release product. And maybe there's a version 1, or version 2, or version 3. And they would reference it. This is in no way available in any ontology, depending on your own business. Absolutely. Absolutely. And that's the idea. I know some people might say, hey, there's some, and there are some museum ontologies out there. I could have literally just picked one, how we refer to things. And I still may map it to existing ontologies, but I want to use my own vocabulary, my own ontology. So that's what I'm showing in these examples. So that's the idea. And the great thing about ontologies is they have formal ways of have them serialized. And I'm using Turtle here. So this is just one possible serialization of what I've just shown you in the application. So you see here the artwork known as, where you have data property, the comment that I showed, the domain, the range, the fact that it's a property of name, et cetera, et cetera. So all the things that I've shown you through the application, you have it here in a machine-readable format in a kind of a well-structured form. So that's the ontology that I'm going to pass my LLM as a way of saying align whatever you extract to this ontology. And that's exactly what we saw briefly in the prompt. So we're passing the ontology as a lump of text and actually i'm going to print it so that it's more more readable not more readable but more more more visible what's going on and then the rest is just the the content itself the text so that's what's happening and the text like i said is just reading this this text file that I showed. And ultimately, what we do, and let me, I'm going to, maybe I'm going to print the prompt just to make it more, print the prompt. Print the prompt. There you go. And I'm going to run it because, and well, this is going to generate some cipher. And the cipher, I could just copy it and manipulate it. Or I can just have it run directly on my database. And that's what I've done. So I have my local database running here on my laptop. There's an empty database. So you have my user and my password, triple Neo there. And super high security. So this is just a connection. I have a helper function. But this is kind of boilerplate code. So you have it here. But just does the connection. It instantiates the driver, creates a session, and then runs the Cypher that I pass it. So there's nothing special there. That's what I'm calling here when I say run Cypher. So there's nothing more than that. I'll share all that, like I said. And this is hopefully going to run on my local instance, which I have running here, and at the moment is an empty database. Right? So that's what we have. As you can see, this is my local Neo4j instance running. So I'm going to run this code, right? And I'm going to do it directly here. Boom. And you should see, let me run. And first of all, we see, oh, there's a bunch of, let me see, the last relationships. No, this is here. Oh, no such file. It's, I must have, Hockney, Mr. and Mrs. Did I not write the name correctly? So it's not finding the, it's reading the file from, oh name correctly so it's not finding them it's reading the file from oh of course it needs the prefix because it's in the content directory right so content save and rerun and here we go that looks better so let's there's a lot of uh output let me show that to you because I remember that I printed the whole prompt, right? So you see that this is the text that I show you, given the ontology below, blah, blah, blah. I said that that's hard-coded and this is the ontology. So I pass the ontology and this is literally the text in the file that I showed you a minute ago, the art ontology. All this to say that I'm passing a minute ago, the art ontology. All this to say that I'm passing the ontology serialized as RDF. So I hope, I expect, and that is the case, that the LLM is going to be able to decipher that and make sense of this total serialization of my ontology. And then I pass the content, and the content is just the text in this file that I showed you a minute ago. Okay, now that's the output so this is the cipher that it's generated so it's extracted an artist an artwork and and the first thing that we realize and i'm going to show that to you in the graph because there you go we have the instances now created here so i can uh show it and we're going to explore it visually in a minute. But if we look at the cipher first, before I get into this nice representation, what we see is that, first of all, it seems to be following the terminology that I described, right? So it's calling things artist, it's using the professional name, remember? So it's not using name or any other term. So it's calling things artist. It's using the professional name, remember? So it's not using name or any other term. So it's being pretty strict at that. It's calling this artwork, artwork known as. So that's pretty good. Then the relationships are also pretty strict. There's was created by that connects. We're going to see in a minute, but it's connecting artist to art, artwork. So that's looking good. Then there's a representation, represents relationship that connects. So it looks pretty correct, right, I would say, because all the terms are the ones defined in my ontology, and if I explore the graph visually, we see that it looks exactly like it should, right? So there's David Hockney at the center and we see that he's... Oh, that's very interesting. So that should go the other way around. That's not being that strict, is it? The direction is... Yeah, I see. The direction is not being treated properly. So that's a negative point. That doesn't mean it's going to happen like that, but let's continue the experiment so but the represents looks good so mr clark works with david hofney that's good he's represented in the picture uh represented and they work together well anyway that that looks pretty good with the exception of the um there's the influence. It would be... so it looks good from the terminology, from the vocabulary point of view. I don't like this was created by because the relationship should be the other one. I mean we saw in the ontology that was created by... let me show it to you here... was created by connects an artwork to the artist and not the other way around. So, the domain is artwork and the range is artist. So, that's inverted. Anyway, that's an error. But basically, what we would do is now maybe we can... So, that shows that we have achieved what we wanted to do. We have controlled the generation of the LLM by passing an ontology with some imprecisions. But we can now take, for example, the other file, the image of Gregory, which is another of the files. I'm going to just replace. I'm doing this a bit manually but you see that we could iterate over the files in the in the directory and that would work as well but if I have rerun this it's going to and actually I can remove the print of the prompt because that's not that useful anymore but uh the next iteration is has generated something similar. I mean, the first thing, and here we see the problem, right? Before, remember how the previous generation created an artist with a professional name David Hockney. Now, this second iteration has created an artist with a professional name Hockney. So they're, of course, not going to link. So that's a manifestation of the problem that we were mentioning before. We don't have global unique identifiers. And that's what we're going to address next episode. But, you know, that's an unsolved problem, let's say here. But the rest, it's all looking pretty good, so good structure so good um uh labels good relationship we could have so sorry we could have maybe solved this by using by not using merge yeah we i mean we could use create for the but but uh but then we would need to do a post process indicating hey hokney and david hokney are the same so we need to merge them together or or we could do a post process indicating hey Hockney and David Hockney are the same so we need to merge them together or we could do a merge on some really unique identifier like some global URI like let's say for example the wikidata URI or whatever we want to call it and again let's translate that to our our our enterprise environment maybe we have a global unique identifier for things maybe we don't so in some cases we will not be able to do it in this first pass and we have to inevitably do some post-processing but first thing i notice look at that now the was created by is now used correctly right so there's a bit a bit of unpredictability yeah you're right but here it goes from p um no it's still wrong it's still wrong. It's still wrong. So it's artwork. It goes from P3's artwork. Artwork to artist, right? So that's good. Oh, yeah, you're right. Yes, yeah. So if we look at the graph now and we extend it, so normally this would have, if we can, so we have two subgroups of these not merging properly on David Hockney and Hockney but uh the second was created by is in the good direction so it goes from the artwork to the artist whereas here uh it went in the wrong direction so still we see that it's a bit of a rebel right so we're telling it to strictly follow our instructions and still. So this is the kind of thing that we would detect using the shapes that we introduced in the last episode. So you can still add the constraints in the form of shapes and have a validation that would tell you, hey, this has been generated in the wrong direction, right? So the domain and the range, the direction of the arrow should be a different one, right? So there you go. So that is, I mean, one thing that this, well, we've not identified the problem of the unique identifier, right? That's a known thing. But in terms of the consistency of the graph, that's very, very good, I would say. So if we look at that, we see that we have a very consistent set of labels. So we have artist, artwork, person, and we have influencers represent exactly the elements that I define in my ontology. So that's perfect, I would say. So the only unsolved problem is the identifier, the merging of the nodes that represent the same entities. Now, there's another thing that some people might be asking themselves, OK, that looks good. But what if I cannot or don't know how? Or is there a way to pass this information that's captured by the ontology in another way because maybe i've not been uh trained on on on on on owl the ontology languages etc etc or to use a tool and the answer is yes and we get we're gonna let me empty the graph and we're going to show what that looks like so now we have another empty graph and we go back to the code and and um and this is going to show what that looks like. So now we have another empty graph, and we go back to the code. And this is going to show us the following. So you might have noticed that I commented this line, where I said, OK, I can get a natural language representation of the ontology. I mean, here I'm going to generate a natural language representation of the elementsology or you know i mean i'm here i'm going to generate a a natural language representation of the elements in my ontology but this is something that you could type yourself let me show what i mean by that so if i if i do that and i reprint the prompt uh let's save that and run it and see what this i mean this is calling get natural representation of the ontology is in the utils package and it's just I'll briefly comment on it but it uses the rdf lived to iterate over the content in the ontology in order to present it in the following way let me run it and you'll see what I mean so if I run again on the same file with the same ontology but now I'm passing the LLM and ontology in a slightly different way. And let me show that. Here we go. So that's the ontology. So now it's not a total representation of the ontology, but it's a description. It's something that I could type manually. I'm not an expert, but I can tell, well, in my ontology, there's a bunch of categories. There's an artist, which is the creator of an artwork. There's an artwork that describes that. There are some attributes, like artwork known as. This is an attribute that applies to entities of this type, and it represents the da da da da da. So it's an entirely textual representation of exactly the same thing. Yeah. So in this case, I've generated from the ontology. So that's programmatically generated. But let's say you're not comfortable taking Web Protégé or an ontology modeling tool, but you could describe that because anyone can do that, right? So you're kind of using natural language to describe the structure of your graph. And the same with relationships. Influences. Influences is a relationship that connects entities of type artists to entities of type artists. It connects to artists, blah, blah, blah. So entirely text. And what's interesting is that the cipher generated is perfectly valid and perfectly well aligned with the ontology that we've passed. So it can be equally strict. And if we look at the graph, now we have something, if not identical, very, very similar. Very similar, yes. And in this case, look at that. The direction was correct. I'm not saying that this is a better approach. There seems to be an element of randomness there. Yeah, yeah. But we see that the result is identical, right? So first thing is, yes, large language models seem to understand decently RDF languages like OWL that we're using, but don't let that be a blocker. I mean, you can define your ontology in many ways. I mean, it's true that there's advantages to the definition. It's because you can do a number of things. They're machine readable and everything. But if you can't, I mean, don't true that there's advantages to the definition. It's because you can do a number of things. They're machine readable and everything. But if you can't, I mean, don't let that stop you. I mean, you can pass this definition in a perfectly, as long as you are not contradictory and pretty clear, the LLM will do a good job of making sense of this ontology definition. And I could, just like we did before, maybe we can try our luck and and say, well, was this file that we were looking at? So, maybe this time in the second pass, he will name Hockney in the same way, maybe not. So, we cannot rely on that, right? So, that's the kind of thing that we need to build a more solid solution for. And let's see, I mean, now we can look at that. No, still not making things. Same thing. So we see that we still have the problem, Hogni, Hogni. So if these two are the same entity, we have to introduce some way of either teaching, training the LLM to detect that, which might be a relatively complicated thing. We will try, I mean, don't get me wrong. You can also do some post-processing, right? But yeah and and what i want to explore and and to what extent that can be that's not the kind of thing that you can easily pass in the context and and and that's before i ask this question let me take a pause because i see there's some conversation in the chat anything that we should address before i kind of get into the conclusion or where we're going to go next? So I see one question that hints to a little bit of what we talked about in the problem with the duplication here in the entry is if entity resolution could not solve that problem. That's a suggestion from Darwin. I think that's something you already mentioned that we might look at in another episode. So yeah. And then another question more generally from Eshwar. If we merged a graph of any kind, really, I guess, with the direction being wrong of relationships, is there a way to change that? Or what would you recommend if you'd say i imported a lot of data and then i realized all my direction of of of this relationship is is going in the wrong direction yeah well that that i think we in that case i would recommend because that part we covered in the in the previous episode so i would not try to detect it visually because let's i mean i'm running this in a simple example with two or three files but the idea is that you're going to run these at scale and there's no chance that you're going to be able to pick up all the violations for that you have ways of encoding what you know the expected shape of your graph and and there's a way of formalizing I expect that relationship to go from an artwork to an artist and not the other way around. And we show that using this idea of shapes, shackle, and you can just formalize it and have the graph, have the code that we shared, detecting any violation of that. And like we did in the previous episode, you can either remove them, you can invert them, you can flag them, you can basically take control of what you want to do with them because maybe there's different people involved. All you want to do is flag them you can basically you take control of what you want to do with them because maybe there's different people involved all you want to do is flag them mark them as incorrect and then someone else takes care of it maybe some human curate it maybe you want to automate it maybe you want to remove them so keep in mind that the moment that's in neo4j you have full control over what you want to do is data in the database and it's up to you what you want to do to rectify that issue so we have mechanisms to detect it and to ultimately automate the correction so yeah that's I see someone asking what are shapes yeah so exactly a formal way of defining you know the expected structure of the graph so there's a language that's used to, I mean, you can do it in many ways. You can use it, you can do it with the cipher, but we've shown how to use Shackle, which is another order, sorry, W3C standard, and that's supported by Neo Semantics. And yeah, I invite you to look at a bunch of episodes, because that's something that we've shown at least two or three times. You showed it a couple of times, yeah, and it's a really good way to, like you just said, you define how a pattern or a certain, yeah, think of it as a pattern. You define that. You think this is what you want to shape. It's the same as a pattern, I guess. And that's why we said here, if we had defined this shape or this pattern, that artist was created by, sorry, artifact was created by artist, and this is the direction it follows, so from artifact to artist and not the other way around, then this would have been corrected or flagged. Exactly. So, yeah, two open questions that we're going to address in the future episodes. So the first one has been clear, right? The unique identifier. So that's something that... and connected to that is what happens when the ontology grows. I mean, we've shown a relatively small ontology with a very small terminology around art, and we're passing the ontology in every single request. So basically we're telling the LLM, take this text, take this ontology and extract structured information from that text according to this ontology. Now, what if my ontology is schema or has a couple of thousand types of entities, but there are bigger ones. So it gets to a certain size where it's too expensive, too heavy to pass a whole ontology in every single request. So we have to look at that problem as well, because normally, most of the times, you will be working on reduced domains, and this approach will work. But in some cases, you won't. So you will have to train, I mean, to fine tune your model. And I want to explore, and that's one of the things I want to test, how some fine tuning can build up all that knowledge in VLLM so that then each request is already pre-trained and is going to behave as if it was already familiar with my my ontology and the same applies to the unique identifiers because if i go now and i tell the llm by the way try to attach to every entity that you extract something generic like the wiki data id you will go ahead and do it but it will hallucinate and we've seen that many times so as much as it's strained on all the data it's not capable of or it does in a hallucinating way and i've tested that and you might get huckney with the with the uri that points at the the county of of of of half a share right so if so if the llm could consistently attach a unique identifier to things, then we would have solved the problem of the duplicating. So that's another thing that we're going to be exploring, because ontologies sometimes are extended with reference data in what's called named entities, named individuals. So it's something like saying, I define the notion of a country. And because there's a well-defined list of countries, don't refer to Spain as Spain. Use this code, which is my unique identifier of Spain. It's like I'm giving you the reference data, and that helps consistency and then integration. And the same applies to basically any domain. So we could have a unique identifier for every painter, but this is too much data to pass in a single context. So we have to pass it to the LLM through some form of fine tuning training, and that's what we're going to explore in future episodes. But I hope today at least we have covered how we can be very strict, specific about the vocabulary that we want to use down to the to the property to the relationship which is something that we could not achieve uh last uh last time through the through the um the knowledge graph builder application so so that's what i'll probably stop and uh and i don't know if there's any questions we can take them and if not we'll um yeah super yeah cool uh thank you very much jesus uh i don't see any any new questions coming in so but that sounds like a very very interesting uh setup for the the upcoming episodes um and we have you know a variety of things to look at and I think that's topics that people are interested in, you know, entity resolutions, different ways of fine tuning, maybe even, I think we touched this a little bit, but even agents, AI agents, I could imagine that could be something. We have like an ontology agent sort of that's like working beside your prompt agent, and then you have a data modeling agent and then they work together. I think we covered this before. So you have a variety of options, a variety of ways to do it, to get to the goal of having that database in the end that makes sense and it connects your data correctly. So yeah, thank you very much, Jesus, again. I think that concludes episode number two. We will be back in November for episode number three. As always, the code will be in the GitHub repository. Let us know if you liked the episodes by giving it a thumbs up. Start the code on GitHub in the github repository let us know if you if you like the episodes by giving it a thumbs up uh start the code on github so that also helps so we know that you are doing it if you're using it if you are using it if you are doing anything cool if you have any ideas for future episodes for content let us know ideally in the youtube comments of you know this video or any other video of going meta and we can uh we can have a look at that and think about if that makes sense and uh yeah share it with your friends your family your colleagues and uh and let everybody know uh that this is uh is happening and taking place excellent well thank you everyone and thanks alex i hope you have a nice day in marrakesh i don't know how long you're going to stay there but enjoy and uh and see you next time from wherever you are i don't know what are the plans for for next i don't know either no uh yeah i'll see you uh i'll be i'll be back at home in november for sure no i'll stay here until Friday. So it's a short trip. All right. Take care, everybody. Thank you for watching. Thank you again, Jesus, for the presentation and the demo. And see you around either next month or this Thursday for the Road to Nodes app development workshop or on the 8th of October for career development with Gen AI. So a couple of options. Until then, take care, everybody. See you soon and have a good rest of your day, rest of your week. And, you know.
Going Meta - S02 Ep02: Using Ontologies to Guide Knowledge Graph Creation Part 2
Season 02 Episode 02 of Going Meta – a Series on Semantics, Knowledge Graphs and All Things AI
Topic: Using Ontologies to Guide Knowledge Graph Creation from Unstructured Data Part 2

Jesús Barrasa: https://twitter.com/BarrasaDV
Repository: https://github.com/jbarrasa/goingmeta
Knowledge Graph Book: https://neo4j.com/knowledge-graphs-practitioners-guide/
Previous Episodes: https://neo4j.com/video/going-meta-a-series-on-graphs-semantics-and-knowledge/

LLM Knowledge Graph Builder: https://neo4j.com/labs/genai-ecosystem/llm-graph-builder/
Tate Modern: https://www.tate.org.uk/

#graphdatabase #neo4j #graphrag #knowledgegraphs #ontology #data