 Hello, good morning, good evening, good afternoon, and welcome to Going Meta episode 25 today. Out of the darkness, I'm talking to you. I was going to say, Alex, you're an unusually dark background. Yeah, it's a little bit weird. I'm in this hotel room and they have like this corner space for the desk that is here. And there is no lamp except for this one here at the desk. And underneath me, or rather it's the rest of the room, and it seems like the lights are on. It's a bit weird. But I'm here in Stockholm this week for JFocus, which happens tomorrow and on Wednesday. So that's why a different location but yeah i'm i just realized earlier but i couldn't do anything about it so there is no i know don't worry we can see you very well and we can hear you very well so it's it's all good that's great it's great to have you all uh join i i saw a lot of people already before we actually uh you know actually went live, talked to a few people from Shared where they're from and what they watch from Amsterdam, from New Jersey, from North Carolina, from other locations, New Hampshire here. So it's great just to have you all here. Even for some of you, it must be a little bit early in the day. For some of you, a little bit later here in Europe, for sure, or even in Asia, it's very late. So if you're watching this live, as always, it's great to have you. It's great to have your comments, your feedback, your insights. And yes, Walter, it looks like Harry Potter Room Under the Stairs. I get the feeling. Now I know how Harry Potter Monster felt. It is almost as narrow. My bed is a little bit bigger, so at least I don't have to sleep here. It is exactly that room. Yeah, today's topic is LLMs for automated knowledge graph construction. So this continues a little bit our journey of LLMs, RAC, knowledge graph, a bundle of topics that we discussed since I think episode 21 or so. Not necessarily you have to watch them all in order or have to have watched everything before when you when you watch now but they kind of i think uh connect well with each other um and build up on each other a little bit i actually actually think that they kind of make make sense i mean it kind of uh they sort of build somehow on top of each other and and i mean of course if you're familiar with some of these topics of course you can jump straight on any of them but i think you know if you're on a learning journey and and how you know what the the value of the combination of knowledge graphs with lms is i think it makes sense to to to watch them in order and and you're right alex i mean today we are kind of stepping i mean staying in the in the you know topic of GenAI, but stepping a little bit out of RAG. I mean, we've been doing RAG for the last two or three sessions. So today, we're still going to be using LLMs, but this time to help in the construction of the knowledge graph. And in particular, we're going to be looking at how to do that from this time from structured data. So we've been seeing you know a lot of interest in in okay having structured data i can do entity extraction and populate knowledge graph what we're going to be looking at today is okay i have some structured data and that can come from a relational database from a collection of csv files so what do i do with these and and how do i build the graph out of that so that's that's going to be the topic of today. And we're going to see how LLMs do actually help. Yeah, sounds good. And I guess, correct me if I'm wrong, but I think that was just today's topic was a little bit influenced by a comment we received on YouTube. You're going to see that in the first two slides, because I'm going to tell you in a couple of minutes, and you're going to immediately recognize. That's right. I mean, there's a couple of reasons that brought us to run this session. And one of them is exactly people's comments and people's suggestions, which actually it's a great segue to say, please keep coming to us and keep telling us the things that are not clear, that you want us to go deeper in, and that will help us coming up with new episodes and new experiments to run. Exactly. Yeah, that's what I wanted to say as well. If something isn't clear, obviously, that's great. But if something else is on your mind, if you think, hey, wouldn't it be nice if Jesus were to discover this or were to to dive into this or that topic then let us know and we can we can see if that makes sense in one of upcoming episodes but if you don't have anything to suggest now then you obviously you can hopefully you interact with us during the the episode so you know you lots of people chatting already away so that's great to see uh keep that going so we can make it a conversation rather than a monologue even though we are two people exactly oh that's very interesting unable to share my screen. And that will be because I have obviously changed laptops. So Alex, you're going to have to cover me for a couple of minutes. Why? Because I'm going to have to change the settings and leave the meeting and rejoin. So you have to come up with a few jokes. I'll be back. rejoin. So you have to come up with a few jokes. I'll be back. All right. Good. See you in a minute. See you in a minute. All right. So good. I'm not really good at jokes. So I hope you helped me out here. Dear, dear watchers. Maybe somebody has a joke they can, they can type in and I can read off on your, maybe I should ask a chat GPT if he's got a graph joke, but usually they are super bad. But I can open that up just to keep going. Tell me a graph joke. Let's see what he says. Why did the graph go to therapy? Because it had too many unresolved complexes and couldn't find its own limits. Oh, well. I'm not sure. Dead jokes. Yeah, okay. Maybe I should ask, make it more like a dead joke. Okay, another one. Why don't graphs ever go to parties? Because they always have too many points and can't connect the lines. I'm not sure. I mean, no. I think we have to cut this out. Well, I mean, until Jesus is back, I can only reiterate that the previous episodes are all in our repository with the code, but also all the videos you can find in our landing page. Whoopsie, where have I gone? Which I'll share in chat. So you can look there and find all these previous episodes we did since episode one, basically, and this is updated constantly. So following through to today's episode. So if you are ever wondering why, where to look at and how to find that. Emil wants to hear another test. Okay, well, I mean, I guess another one. If you can bear the pain. Another one here. What do you call a graph that's been lying on the beach too long? A tangent? I don't think chat GPT understands jokes very well. Oh man, you're gonna kill me. I'm back but it's not showing the same... How do I help Santa? Damn it. How did we... help Santa damn it how did we maybe do yeah help me Windows user max user well I'm willing to take tips here because I'm what did you do did you do? Did you let the app... Presentation, presentation to... Yeah, share. If I share entire screen and I select the screen and I click share, it's still showing update system permissions. But which permission is that? Because I thought I had... permission is that because I thought I had. I think on the Mac you need to Yeah, you need to switch it on. And privacy, maybe. Here we go. So camera. But maybe... Look at the comments, oh my goodness, we're going to get fired. We are getting... Can we have our CEO helping us changing the settings so that we can share our bloody screen? Oh my gosh, that's embarrassing. Can you share a window instead? Does that work? Did you try? I can try, but I believe... yeah, that's a good point. Good. Let me. Last joke at least. Yeah. It's good. Yeah, that works. Did that work? Wow. There you go. Perfect. Well, good. Okay. Well, that's great because that's all going to work from a single browser window. So, well, you can see it, that's great. What if I go on full screen on slideshow? Is it still showing nicely? Yes. Yes. Okay. Good. Brilliant. Like we're saying, well, that's unexpected. Anyway, we're going to be focusing today, staying within GenAI and using LLMs. But this time, instead of RAG is to help us building a knowledge graph. Right. So and in particular from structured data. So I said that it's kind of a, you know, we're kind of revisiting what we did in in in going meta episode five. That's many, many months ago. But what we did back then is we had some structured data in the form of CSV files, and we have a kind of a target model defined by an ontology, right? So we know the kind of graph that we want to build, and we define some mappings, and then some kind of general purpose code, some Python code basically would read that target model, would read the mappings, and generate the data ingestion pipeline to populate the graph. So that's what we did back then. Right. There were some questions, though, and that's probably where you were going, Alex. But at the origin of this was, of course, Episode five, but also something that Dan posted, you know, a few a few weeks now ago. And, and then if you if you're not following him, you should be, because he's great and he's doing amazing stuff also in the Gen AI space. And one of the things that he talked about, and this was, like I say, two, three weeks ago, was that he was trying to use LLMs to kind of help in the construction of a knowledge graph, right? The creation of the graph model, he focused on that. And I'm going to show in a minute. I mean, you have the link there to his LinkedIn profile because he, like I say, he posts super interesting stuff, but also his GitHub account where you can see his experiments. And actually, I probably have it here. And what he did, basically, you have the notebook, you can you can play yourselves. And, and he went and looked at the structure of a CSV file, he's using some thing insurance, public data, government data around insurance. And he analyzes like the headers, he even does some profiling of the minimum, maximum values. And using that, he kind of interacts with an LLM to come up with a graph model. So that's one approach. What I'm going to do today is slightly different, but it's along the same lines. So it's also answering to these questions that you were mentioning before, Alex, where some people actually at the end of this episode five, where we did what I described a minute ago, it was something like, hey, give me best practices around modeling, because you start with a target model. What if I don't know how to build that model? How do I create this kind of declarative schema definition of the model of the graph that I want to build? So these two things are the kind of the reasons that made me explore these, these idea that we're that we're working on today. So and I think we got an last or previous episode, we also got a comment on, hey, how can I can I use LLMs to to, to date to model my my knowledge graph. So it's, it's not exactly the same topic we talked today, but it goes into a, I would say it goes in a similar direction. Kind of, you know, it's kind of, you know, similar. And these diagrams shows the process. I mean, if you follow the numbers, and I try to pick, you know, a common or popular source of structured data, CSV data mostly, which is Kaggle. And you will see that most of the pages, and we're going to see that in a minute, offer access, of course, to the data, some very rich description of the structure of the data, some metadata describing which are the key features in the data set, some description of the data set itself, etc. And that can be extracted in a structured fashion. That was a nice surprise. And I'm going to talk more about that. So you can extract both the metadata and the data itself, right in the form of CSV. So I'm focusing on CSV data here. So what we're going to be looking at is, okay, I'm going to use LLM, and in this case, we're going to use again, OpenAI to generate a data ingestion pipeline, which is basically a collection of CSV scripts that will consume the data, but this process is going to be fully automated. So I'm going to, at the high level, although I'll go into the details in the session today, I'm going to go and read the metadata description, right? The formal description of what the structure of my of my data set. I'm going to pass that in a consumable format to the to the LLM with some some prompt engineering there. And I'm going to ask the LLM to generate a model, a target model. So I'm not going to pass the model, I'm going to ask the LLM, hey there's some latent entities because the thing is in these CSV files sometimes there's, you might find them very highly normalized, where you have a lot of help already, let's say you have it broken down into files in the form of a kind of a relational database where you would have a file for customers or file for products or file for orders. You kind of have a kind of the model there, right, if you read the files. But sometimes you have a large CSV file completely denormalized and all you have is header descriptions. And you have to go there and say, hey, actually, these three columns are one entity and these three columns are another entity and this is another entity and so on and so forth. So this is something that you can do if you have some experience, but this is something that LLMs can help us with. And that's exactly what we're going to be experimenting. We're going to pass this metadata description. The LLM is going to come up with the model and the mapping, and then we're going to fully automate the process of, okay, now I'm going to read the CSV and I'm going to populate the graph. So that's what we're going to do. So step one is extraction of the metadata, generation of the model and the mapping. And third, importing the actual data. Make sense? Yep. So I'm not giving any kind of guidance. The guidance will be in the metadata about the data set and the LLM will do the rest. So the first surprise that I was mentioning, I was not aware of this. It's called a Croissant format, right? That's something I haven't seen before, but if you go to any dataset in Kaggle, you will see that when you download, you can download the data itself, but there's also the option of exporting the metadata as Croissant. I mean, a croissant is a JSON format. You see an excerpt there on the right that actually it's a very rich and formal description of what the dataset contains. And the dataset will have a collection of record sets, and the records will have fields. And the fields will have a name, they will have some description, et cetera,, etc. It's a very rich description. And interestingly, it's JSON, actually JSON-LD, which as we know, and those of you who have been following, JSON-LD is a graph format. So we can easily import it. And that's what we're going to do. Because once we have that description, we're going to be able to use that to create a prompt that the LLM will be able to consume. So first, a nice surprise is this cross-format. I put the GitHub repo there because it's being developed by the ML Commons body. And that's what it looks like. We'll see more of it in a minute. Now, once we have that, we're going to pass that, I mean, not exactly that, but in a nicer form of that to the LLM, so that it can generate a description of the model that ultimately is going to be our graph model, right? But we're going to have it in a form that we can consume in a machine-readable format. And it will be basically a collection of entities, which is in Neo4j terms, the labels, the types of nodes that we're going to have. And for each of the entities, the entity types and the list of attributes or properties. And then we're going to map them to the elements in the file. So this is all going to be worked out by the LLM. So the LLM is going to go over all the features in the dataset and will determine which latent or kind of hidden entities are there. And that's what we're going to use. When we have that, we're going to write some Python like we saw that's going to do the whole work of generating the graph model. Very simple. So in terms of interaction with the OpenAI API, it's going to be, we're going to be generating a prompt that looks pretty much like what you see there. So I'm passing the description of the dataset, the name, and the features in a readable format, and I'm going to use the completions API for it to generate a collection of entities and relationships, which is our graph model. So we'll get to see these in detail. Does that make sense, Alex? Any questions? Do we have anything there that we need to make clear before we actually get to see it in action? No, I think that that is pretty clear. I'm excited to see it in action. So I think we can go there. Perfect. So I assume you still can see all the tabs in my notebook. Can see all the tabs. Browser. I'm going to connect. I'm going to work in, as usual, in a notebook, in a Python notebook. And while we talk, I'm going to be- I think we have to zoom in a little bit in this view. Yeah, let's do that. Is it better? A little more, maybe. Okay, let's do that. I can do that. Yeah, I think that's good. So the first dataset that we're going to use is this New York housing market. This is a pretty simple dataset, but as you can see, we have the human readable version of it. This is in Kaggle, and it's a single CSV file. I'm working with this type of denormalized structures. You could do with others. But for this example, I'm going to go with this. And we have a description of all the columns there. So you see that there is a description about the property, there's a description about the address, there's a description about the property. There's a description about the address. There's a description about, it looks like there's a broker involved in this housing market. So these are, it looks like there could be several entities in these dataset, but we'll let the LLM figure that out. So this is like I say, the human readable version of the dataset. But if you go up here, you can export the metadata as a croissant and and this is this uh rich format that i've shown a minute ago and i'm going to show again so that you can uh see it probably by now we've already finished doing the um install i'm going to set up my connections but basically we have the the the source here so the cross and i've copied it to my uh just for this example and i'm going to show it to you here so this is what it looks like so like i said before there's some header uh some information about language and some high level information but then a... This is also a little bit small. Also smaller. Let me make it bigger. Apologies about that. So there you go. We have a data set that contains a reg set, which is a collection of fields. So we have for each one of the fields that we saw in the more human readable, we see the broker title, which is the title of the broker, the type of the house, the price of the house, the beds, the number of of bedrooms so we see that they are all listed there and uh and uh they come with the description of the name of the column and some additional information so this like i say is uh is json ld format and as you know json ld can be imported and um and i'm going to use today um something that we have not talked about in quite a while. I'm working with Aura, right? So I have my Aura instance up and running. It's an empty database at the moment. So if I, let me make sure, just to make sure that I'm on the right instance. It's an empty database. Perfect. If you remember in order, we cannot use extensions at the moment, and that prevents us from using Neo semantics, which would be our default option for importing RDF data in JSON-LD is one form of that. Now what we're going to use is um uh rdf lib neo4j which is kind of uh it's a project that we talked about in one of the episodes quite a while ago but that has a new a new version something that our professional services team has been working on and that makes it really really easy to work with uh rdf data writing to Aura because it works entirely on the client side. I mean, it just works using the Python driver and interacts with the database without having to install any extension. So the way it works, without going into too much detail, I'm going to click for it to run. I mean, I think I've set my database. I'm going to set my files. I have a couple of extra examples. If we have time, we'll go through a couple of them. And the first thing I'm going to do is, okay, I'm going to have a look at this croissant description of my dataset. So if I run this code, what this is going to do, what I'm doing essentially is I'm importing the RDF Lit Neo4j, and I'm creating a store configuration. I have to go quickly over that, otherwise we're going to not have time to see the whole thing. But we'll get back to it. There's an episode and maybe Alex, you can find it, but it's one of the early ones where we talked about how to import RDF data into Aura. But basically we define the the some of the the kind of the the the way in which one to the rdf to be imported and we want to basically ignore the namespaces we want to batch the input and we pass the configuration the the authentication configuration which is my my credentials that i have locally. Once I have it, what I'm doing, basically, it's I parse my Croissant file. And by parsing it, it automatically gets persisted in my Aura instance. So we've imported 179 triples, which is roughly what we had in that data set. You will notice that there is a kind of a temporary hack. I've commented it here. And that's because at the moment, the current version of RDF Lit Neo4j does not support JSON-LD. So what I have to do is I have to import it and then serialize it as Turtle to be able to import it. This will not be the case in the new release. You will check, if you check in the issues, it's already been committed to the source, to the code base. And the moment the team decides to do a new release it will be out there and we can just get rid of these intermediate steps but anyway we have now the the i put the link in the in the comments and in the video description that's it was episode number 12 where we talked about importing rdf data into aura and it's it's one of the labs projects so so now we've imported the the this description the the cross and description of our data set so if i if i um copy this query it's going to make it possible for me to explore that in my aura instance so there you go let me um that makes it a bit more readable so what do we have here so we have a data set which is this orange node i'm going to have to probably make it bigger uh yeah thank you actually take all the state there you go so we have the the i'm gonna make it uh maybe a bit bigger so that's um damn it the the problem with the browser when damn it, the problem with the browser when, is that when we have multiple labels, it gets a bit confused. But anyway, so we have a node representing the dataset, right? If I click on it, you see the, oops, what's it not? There you go. So you see all the attributes to the right. Let me make that a bit bigger. And so we have the description, the name, all the information that that we had in human readable form, we have it already imported in the graph. And the direct the data set has in this case, a single again, I'm going to make it bigger, a single record set, I'm going to reorganize that to make it a bit more understandable. But what we have is that um this uh record set is a single one again it's described in this collection of attributes and then we have a number of fields and for the fields i'm going to show the damn it that's that's the problem i mean it just uh because it has labels it just goes a bit crazy the browser but let me try it again and now it's ignoring me anyway well we'll have to deal with that so the visualization is not going to look great but basically these are these are the the the the fields you see that there's a field connection and the fields fields, the name, and the data types. So they're connected to the data types. These are two fields that happen to be integers. You can see here, two additional fields that happen to be floats and then a bunch of them that are strings texts. You see that for each of the fields you have the name and so on and so forth. So we have it loaded in the graph. So we have a rich description of the data that we're going to be importing. Good. Now we have that and we can use that to generate a textual description to the LLM. And what we're going to do is we're going to pass a more readable version of it. And for that, we're going to be running this query. Let me show you what it does. It basically does the same, but it returns it in a tabular form so it tells me the data set name that's the data set description and then for each of the features each of the columns in the file I give uh the name and the data type that's all and it is with all the columns in my data set so I'm going to use that uh and I'm going to run this query now so that you can see what it does. So it basically runs this query and we've stored in this variable the name of the dataset, the description of the dataset, and the features. All this to create a prompt. And this is the one that I showed before. So I'm telling you're a data modeling expert, that you're capable to create an entity relationship model, which in this case is exactly what we need, collection of entities and collection of relationships. And I'm passing the three elements that I've just extracted from the graph. If I run this, you're going to see what it looks like. So there you go. So it just passes the name of the data set, a description, not very useful in this case, but sometimes they're richer, and then a collection of features, which is a list, a collection of pairs, feature name, data type, feature name, data type. That's what I'm going to pass. This is the prompt that I'm going to be passing to the completions API. If I run this, it's going to take a few seconds. But basically what we're asking the LLM is, yes, go explore this collection of features that are flattened in a single file and determine if there are recognizable entities. Basically, can you come up with a graph model for me? I don't wanna do it. I want you to give me some guidance and give me some assistance in doing that. So it's done. And now we can print out the results. So we can see that it's come up with a plausible collection of entities. For example, there's a notion of property. A property has a type, a price, a number of bathrooms, square feet, and so on and so forth. There's also the address. So it's decided to split into the entities, which kind of makes sense right because they consider uh two different entities and there's the notion of a broker there could be different properties at the same address exactly so so we have three different types of entities and then it's identified three relationships that connect these uh entities so there's a connection has address between the property and the address and the has broker between the property and the broker. So pretty nice, nice job. Right. So I think it's made a good interpretation of the of the of the data set of the description. And we're going to go with that. Right. All we need to do is. OK, well, that's all the information I need because I have the name of my of the fields or the features in my data set, which is this map two. And this is the name in my in the property in my mode. And that's all. Like I say, that's that's all I need. So the next next part, I'm going to run these. I might go in detail, but basically, this is where I reuse the code from Episode five. This basically is generic code that is going to explore structure and generate cipher. So it's going to parse that and it's going to create both the constraints, but also the the data input the cipher input, which cipher input, which basically are load CSV instructions that can out of this description. I'm going to show the result and maybe it's just a bunch of lines. So it's not super complex. And we're going to see what it looks like. So this is the definition of the function. I'm going to actually call it on the output of the LLM. I have to make sure, and here there's a caveat, because there are mechanisms to make sure that the output of the LLM matches a particular schema, right? So sometimes it's generative AI, so it can be creative and it can come up, it can name properties in different ways. For example, sometimes it's name, it will call it entity name or it will call it relationship name, or instead of mapped to, it would call it map to feature. So there's ways of controlling that and there's function calling, there's Pythonic, there's a number of ways of doing it, i didn't have time to prepare it and i would it would have made a super long session so what i've done is i've made my code flexible enough to so that i can tune it depending on on but it looks like the names are pretty much the same that i was expecting so there's a from there's a two there's a type mapped two his the same and this name for both names. So it looks like it's the, you know, the expected type of structure. But in case there was some variation that would have helped me, you know, react to it. OK. Essentially, what I was saying, what this does, if I run it on the output of the LLM, it has produced the following output. Look at that. so it's generated because there are three entities we're going to be creating three constraints property address and broker so this is valid cipher this is the creation of the constraints and at the same time it's created uh um a loader basically it was going to load from this file which is the one that we specified at the top and we're going to be uh unwinding because we're going to read the file pass it stream it to the to this code and read record by record which is what the unwind does and create entities i mean create nodes of type property with the properties as defined and you see that it's going to be reading them from the corresponding column in our dataset because we had it in the mapping that the LLM has generated. So very similar, a bit simplified from what we did in episode five, but that's pretty much all we need, right? So I have all the cipher that I need to ingest the model, to ingest the data and create the model that the LLM has come up with. So, and the same for the relationships and so on and so forth. So all I have to do is actually run this code. And that's what we do with this, in this function. Again, you can have a look at it, but basically all it does is read the, I'm gonna read the CSV file and I'm going to batch the file and send it in chunks to the graph so that it's uh persisted in neo4j so and the the code is available after that so we put it again in the right now but it's going to be no but you know soon soon yeah absolutely i'm going to go in you know in a little bit of detail but i wanted to kick it off just to see if it throw any any errors or anything unexpected. There you go. So it's already, like I said, it's basically using pandas to read the CSV file. It's calling the insert data function, which is this one that we created here, that chunks the file in batches and sends it to Neo4j by running, you know, execute the cipher that we've just created. It takes the cipher and runs it and imports the data. There you go. It's imported the properties, the addresses, the brokers, it's loading the relationships and it looks like it's done. We can now click here and explore. We have as expected, let's see. For example, we have the has address, know, let's see, for example, we have the has address. Let's make this bigger. But, you know, no surprises. So we have properties that we can change slightly. Maybe we can refer to a property by, I don't know, maybe we can put the type. And the properties are linked to addresses. And we might even find things like properties. There are multiple properties in the same address. There can be a has address. Let's look at that. There you go. It's done a good job. It's transformed our original flat file CSV file that contained what you can explore it here, you can transform this flat file into a graph model and load it. And we had also if I expand, we had also the broker remember. So I mean, it's worth saying that this is a fully automated process. So I've, you know, I kind of eyeballed the output and this might not be a perfect model, but it's a great starting point to say, hey, I want to see my data as a graph and I want to come up with a plausible model, a reasonably okay model. So that's what we've done. Does that make sense? Because you might say, I suppose the question might be, well, this is a pretty simple model. And yeah, I agree. I mean, this is, as usual, we try to kind of come up with kind of didactic examples. But we're going to, just to show that it's a generalizable process, I thought, well, let's give it a try with another data set. Because it's exactly the same code, and it's entirely data driven so what i'm going to do is i'm going to empty the database so if i empty that shouldn't take too long it's just a few thousand nodes there you go so we're starting again and we can um actually because we created a bunch of constraints, I need to, I mean, not strictly needed, but it would be cleaner to delete them. So I have this magic piece of APOC that clears the schema. There you go. And by doing that, I get rid of all the all the schema elements and now we just stay with another relationship that was a nice trick so that i learned also this week but like i was saying why don't we go to another one and i have to you have to show that again oh yes sure sorry that was that was a quick one but uh it's um and uh let me show that apoc schema assert so you can actually that that helps you essentially i can um if i'll link to a link to the to the docs yeah it helps you defining it kind of in in one go a collection of constraints without having to call them one by one. And in the same way, you can remove it. So it's pretty convenient. So yeah, sorry. That is convenient. But I was saying that, okay, let's look at another data set. And I found this supply chain that I thought it might be a bit richer. I mean, in this case, it contains 33 columns. So maybe there's potential for additional entities there and we're going to do exactly the same process so once we've cleared the database i'm going to start again from my notebook and you know i'm running it in blocks here so that we can we can explore what's going on but this can be can be run in a single go so what I'm going to do here is okay I'm going to comment they said which is the housing market and we're going to go with the third one the second one was about interesting touristic destinations in India I'll skip that one in the interest of time and And we'll go with the supply chain example. So there you go. Another pair of metadata file plus data set, which are... And the supply chain data set is that also what was also... Okay, here you go. Okay. Do you have it or shall I paste it on the chat? I think I'll find it. But if you have it, or shall I paste it on the chat? I think I'll find it. But if you have it quickly, yeah. OK, perfect. Yeah, thank you. Then I'll put it in there. So yeah, like I said, the usual process. So we start by, OK, we're going to import the croissant file. So the croissant file, similar to the one we saw before. I mean, you can download it from here and we have it in GitHub so you can import it directly. Once we have it, we are immediately ready to explore it. Same structure that we had before. So the queries that we run should work again. So that was it. So we can return the sub graph. we'll have the usual problem with the visualization let's give it a try let's try it yeah let's oh yeah it did the coloring but it didn't do the the the naming but anyway so we have the data set at least the colors that's helpful let's make it bigger so we have the the data set the data set contains uh the record set and then there's a bunch of properties grouped by data types so and again you know it's not super readable but these should be the 33 columns that were listed there these are the floats these are the dates as you can see these are the booleans these are the integers and for each one of them we have the the name of the price pack price we have the line item insurance usd so this unit price so there's lots of columns completely flattened and like we did before we're going to present that in a way that is going to make it easy for us to, I think that was it, to pass it to the LLM. So this is the dataset name, the dataset description, and the collection of features with their data types. That's what we're doing, and that's what we're going to be doing again in our code. So here we go. So we extract these elements, and we generate the prompt. So the prompt has exactly the same structure, except that now it's about, it's not the house market, it's supply chain. There's sometimes this is pretty poor. I mean, it's kind of okay, but sometimes it's a much richer description depending on the data set. And I let you play with Kaggle and your preferred ones. But what's important is we have the collection of features with the data types, right? So we have the prompt. We send it to the LLM. That's where it's going to take a few seconds. That's where it's going to take a few seconds. And here is where it's exploring the 33 features, the 33 columns in our CSV, and coming up with the model by uncovering or surfacing the latent kind of entities and relationships that can be extracted from it. So the moment this is done, we're gonna be able to... I see we're having loads of questions about the sharing the notebook. We will, of course, as soon as the session is over, you will find it in the GitHub repo for going meta. This is the phase that takes takes a bit longer still should only be a few a few seconds and um let's any any questions that we can answer while this thing is going it takes a bit longer today um there was a question on LinkedIn. Yeah. I'm reading, is this RAG still? This is not RAG. I mean, RAG is some. Let me actually, well, oh, it's done now. But it's a very, very, very important question. And let me make sure that I get back to this one. Alex, I'm going to complete the example for.. And then we'll get back to that. This is not RAG. This is previous to RAG. This is the construction of the knowledge graph. So we run the generation of the model. If we explore it, there you go. So it's a bit more, it's a bit longer, more entities, but it looks at the structure of the same entities with the name. It says that there's a notion of a country with a country name. There's a shipment with a scheduled delivery date and so on and so forth. So a collection of entities. There's a notion of a product. There's a notion of an order. There's a vendor. And then there will be, there's a brand. And, well, I can keep, this is a much richer one. That's very, very interesting. I'm happy that we chose this one. And then, oh, it didn't find any relationships. That's annoying. No, that's not good. Bad. Why, why, why? Oh, why? Let's run it one more time. Because this is, I mean, if it doesn't work, that's the unpredictable part. I mean, it doesn't make a lot of sense that he hasn't found it. But let's give it another try. Let's try it again. And I'll take a minute to answer what rag is. And I'm going to go to the episode 24. This is rag. So rag is once we have, I mean RAG is, if you want an architecture, a pattern, yeah, kind of a integration pattern somehow, where we're building an application that interacts with an LLM, right? So we are asking questions from an LLM, and that can be to provide a kind of a conversational interface, or it can be to do any kind of customization or generation by the LLM. Now, if we build an application that just interacts with the LLM, we rely entirely on the LLM's capabilities to generate, to reason, but also on the knowledge about the world that the LLM has, which is sometimes incomplete, as we've mentioned several times, because LLMs are trained on public data and they will not be aware of your enterprise business data or your personal data. So we have to somehow inject that knowledge, these facts, these grounding facts into the process. And that's what happens in this intermediate step. And that's where we augment the generation. That's why it's called retrieval augmented generation. I like to call it augmented generation, but it's based on retrieving data from an external source, a source that's curated, that's controlled. And that in our case is a knowledge graph. So what we do is based on the task that we want to solve, that we want the LLM to work on, we're gonna go out to an external source. And there's a number of ways to do that and we've touched that on episodes 22, 23, 24. We retrieve all the potentially relevant information from these external source, our knowledge graph, and then we pass it in the context to the LLM and then the LLM will return an answer but grounded and based on these facts that I've passed in what's called the context. And therefore, I'm injecting this external knowledge, controlled, curated, trustable knowledge in the process. That is RAC. What we're doing now is we're constructing a knowledge graph out of, in this case, structured data. And this is kind of a different process. Let's see if this has finished. I hope that clarifies things. Let's see if that's returned now some relationships. If not, we'll have to unfortunately give up, but... Oh, there you go. We have, I know this is, oh, relationships we have. Yeah, perfect. Now it came up with some. So that's good. So I don't know. It's, you know, like you see, it's a little bit. Anyway, so we have some entities that have been extracted. And now we're going like we did before. I don't need to recreate this function because it's it's exactly the same and it's using the same schema elements so all I need to do is rerun call this function but this time oh damn it something has happened here of course that's when type, key arrow type, why there's not a type. Let's try to add the term. So we're not having, our name is not type. Oh, data type, there we go. No, is that what we're looking at? Yeah, I think that's it. I think it's data type. Data type, so that's, there you go. That's the reason why i did that um is this uh no that's not what i'm looking at map two relationship name is the tag no that's um relationship name is name so that's the thing the relationship name is name. So that's the thing. The relationship name is called name. I believe that's the thing. So maybe we can give it another try. Perfect. Now it looks better. And if we, so again, what I've done there is I've had to deal manually with the fact that I'm not enforcing the structure returned by the LLM. Like I said before, there's mechanisms to make sure that this returns the right JSON structure, but I didn't have time. I didn't have something that would fit in this session. But there's mechanisms to guarantee that because I did not enforce it. I have to kind of read what the output of this looks like. And it's using relationships name to from to make sure that my function uh consumes that output so that's that's the only thing that we've done so there you go so we have a a new set of uh of cipher instructions that will populate the graph and um if i uh if we run this again If we run this again, this will hopefully populate the graph and now it's importing countries, it's importing shipments. These are the entities that have been extracted from this supply chain data set. So this, as we saw before, is a flat data set with 33 columns and it contains loads of information. So what we've asked the LLM to do is to extract a model out of it, a graph model. And that's what it's done. And it should be done shortly. It's finishing now, populating the relationships. And the moment it's done, we should be able to have a look at the, let's wait till it's done. So it's item order. Good. Finished. So we can now, okay, we can remove this because this is the, this is the, because we still have in the same graph, this is the metadata description. Remember that we have the data set that contains record set with fields, et cetera. But the actual model from the import looks something like this. So we have a shipment that contains items that have,'s used in orders and and shipments you know are associated to countries so that's the model that it came up with and we can now explore it in the graph right so we can see that we have orders and if we expand orders will be linked to items an item will be part of multiple orders will also be part of multiple shipments the shipments will be associated with countries i can do the usual to make it look a bit nicer but there you go so that's uh that's where uh i probably stop and look at some questions but we've come up with a decent and reasonable graph model out of entirely unstructured and flattened CSV data and all through a fully automated process. So I hope that kind of made sense. I know in some parts I went a bit too quick, or maybe I didn't go into a lot of detail in particular the the function that generates the cipher out of the mapping but you can I mean like I say it's just a bunch of lines it's pretty straightforward it iterates over the the strong return by the llm and generates cipher out of it so you can you can explore it once I share these and that should be uh like I said relatively relatively straightforward. But yeah, let's take a pause here, Alex, and look at some questions. Alex Sarros- I think it is remarkable in a sense also that we used two datasets that we haven't really looked at a lot in detail before and we kind of like still can work out what they mean and how it works and how it fits together. So I think that's very interesting and very nice to do and to work with. Probably just one comment before you go ahead with the questions. I mean, in this case, and again, it was my personal curiosity. I found that there's this kind of structured description of the metadata of the file, right? This kind of JSON definition in cross and forward. But, you know, again, replace that with whatever is your preference. I mean, sometimes you may not have that description. And what you have to do is what Dan did, right? You have to maybe parse the CSV file, extract the headers, maybe do a bit of profiling. You have to try to extract some of the metadata. But the moment you have a description that you can pass to the LLM, I mean, the LLM, you know, the more clues you give it, the better model it will come up with. I mean, if the, you know, if the columns come with a description, then it will probably come up with a better model. So that is available in the case of Kaggle, and I invite you all to go and check your favorite data sets, and it will generate a model out of them. But if you don't have that description, you can generate it in other way. Yeah. And with that, you already answered a question from Stefan who wanted to know if this only works with structured data, but, you know, it does work with unstructured data as well, but you probably have to do a little bit more prep work and a little bit more, I guess, train LLM so that the LLM knows what's going on. Yeah. I mean, the thing is with unstructured data, the process is slightly different because there, I guess train LLM so the JLM knows what's going on. Yeah. I mean, the thing is with unstructured data, the process is slightly different because there what you're doing here, what we're doing is, you know, I know what the kind of have understandable or well-defined descriptions for my columns. And I'm going to come up with a model. Now, if you have unstructured data as a paragraph or a document, the process is called entity extraction. And I think we've talked about that in other episodes, right? And there it's more of a I'm going to extract maybe entities of different types, but it's a substantially different approach, I would say. Yeah, good point. Could you summarize what is the advantage of doing this with an LLM? Well, it did it for us, basically. That's not doing it means I have to be an expert. I mean, I'm not saying that what we did is rocket science, right? With some training and with just looking at our great documentation and materials, you will get best practices and guidance on how to do graph modeling. But sometimes for people, it can be a bit of a blocker. I mean, I'm used to work in Excel, but I cannot determine which things can be entities and how should I come up with a plausible graph model. Well, let's ask the LLM to do it for us. And that's the advantage, that it's done it for us without us having to know anything about graph modeling. Right? So that's again, take this with a pinch of salt. This is a fully generated model and this will probably require iterations, right? It might not come up with the perfect model, but it will be definitely a starting point to see your data, your tabular data in the graph without having to basically to write a single line of code. Yeah, exactly. I think that's a pretty good advantage. And especially, like we said in the beginning, this is not, I mean, you can, this is an idea and a concept and a proof of concept almost that it can be done and that you that you get quickly to a first result if you then decide on exploring this data set further and if you want to then incorporate this and work with it as a graph then you probably should step back still and and have a little think about is this actually right is is maybe other mistakes in there is it is it making complete logical sense because you know LLM is not 100% right. So they might hallucinate some stuff. They might come up with some connections that are not really the right ones and stuff like that. But I think as a general starting point to something where you might have a little bit, you are at a loss of what's actually in my data and you just have a huge table and you don't really know what to do with that. I think this gives you a great view of the, hey, how do I get from there to a graph in relatively no time and relatively no writing code and then exploring that data. Absolutely. Yeah. of questions will go into this same vein of of of why why using neo4j or by using a native graph database for a knowledge graph so a couple of people asking here is how is this better than onto text rd fox cambridge semantics label property graph are still not semantic knowledge graphs or another comment here i'm a long time Neo4j user. There are other options for data store like Vector Store, Azure, Cosmos DB and so on that work as a backend for a RAC application. What are the key justifications for having a native graph database or a pure knowledge graph as the backend data store? Wow. Many of them. Long, long question with. Long, long question with potentially long, long answer. I mean, I'll take the one from Joe on how it is better than Ontotex, RDF, Focus Cambridge, Semantics, and blah, blah, blah. I mean, we've talked about this many, many times, so I would refer you to earlier episodes when we talked about the difference between RDF and property graphs. I mean, it's, I mean, with different types of stores. And this, I would say the quick answer is we work at different levels of abstractions. Triple stores break everything down into statements, into triples. And from a storage point of view, they are a table, a table with three columns. That's what a triple store is. I'm simplifying. Of course, it will have indexes etc etc but basically you work at the triple level at the statement level in the property graph we work at a different level of abstraction which we believe it's kind of nicer it's more developer friendly more more data person friendly because we work at the nodes and relationship level so for our node is a is a rich structure that uh that that describes for example example, Alex. We can say that Alex and Jesus are friends. So there's two nodes connected with a friendship relationship, right? If you want to represent that as RDF, you have to decompose it. Basically, every node will be a sequence of triples that you can put on a triple store, and the relationship will be more triples, et cetera. So two different ways of looking at data. I mean, different levels of abstractions. There's lots of advantages in the property graphs. We've talked about that. We put them side by side in many episodes. So I totally refer you to look at that. And I think that answers the question. The other one, I think you mentioned, I don't know if it's there, the vector databases. I don't think, you know, vectors play a key role in this episode, but Neo4j is a graph database with, you know, vector search enabled graph database. So we combine the two ways of exploring data storing and doing semantic search, graph semantic search and vector search. Again, we cover that in episodes, I believe, 22, 23, 24. So please go back to those because there's a high level of detail. And like I say, it was not part of today's episode. Okay, cool. Maybe to finish today, one more question here, also all coming in via LinkedIn. Maybe going a little bit more into what you said before, is if you don't have a cross-honed metadata text file, if you have a, here's somebody asking, hey, can we do it without tabular data, but for example, PDF text data or something? I think we showed it in a way that makes makes sense it was easy to showcase but you you know um you can use more or less any other uh data set you have or data data format or however the data is structured it will take like you said it will take a little bit more time to to prep I guess but you know technically this should work yeah just more like remember that ultimately what we do I mean we have this cross on former which is super rich but ultimately what we're doing remember is that I mean we're passing I mean the key thing is is this this portion here I hope it's readable so we're going to call the completions API. We're going to call the LLM and ask it, hey, build me a graph model out of this flat data. And what I'm passing is this prompt. So how you come up with this is entirely your choice. So I'm passing, this is my data set. This is what it contains. And this is the collection of features or columns, right? This is what it contains. If you is the collection of features or columns, right? This is what it contains. If you can come up with these in a different way without using the croissant, and probably you can just, I mean, in pandas, if you use Python, you know, there's loads of functions to, you know, to read a CSV file and return the headers. You can even manually add annotations to add additional descriptions, but ultimately you have to, excuse me, headers. You can even manually add annotations to add additional descriptions. But ultimately, you have to pass these features in your dataset to the LLM. And I choose this format. How do you come up with this format is entirely your choice. If you have the cross-hand, great. If you don't, there's many other ways. Yeah. I mean, for starters, you can potentially ask an LLM to create these for you out of your file you want to send to the LLM. It might not be really, you know, but as a starting point it probably could be good enough. Yeah, and it depends in a number of factors, right? If your data is well described and you have headers that are readable and meaningful, then perfect, because the LLM is going to make sense of those. Maybe sometimes data comes, look at this, this PQ. What the heck does PQ mean? P-O-P-L-S-O. Maybe by the context, it will make sense of it, A-S-N-D-N. But sometimes this is not great, and what you have to do is do a bit of profiling. Maybe you want to read a little bit of the data and pass that to the LLM and say, can you make sense of that? And maybe these are names, these are dates, these are, and so, and there, I would again, refer you to Dan's notebook. And I think you've put it on, I think it was one of his likes It's in the video description on YouTube. It was in one of the slides, but he kind of goes into a more conversational iterative approach. So he, you know, parses the CSV and asks the LLM, what could I give you so that you can produce a model? So there's loads of things that can be done. I mean, we went with one approach that kind of fully automates the process, but it's not the only one. I wanted to kind of show how LLMs can help. We've talked many times about how they can deal with unstructured data to do the entity extraction. Today, I wanted to explore how they can also help in dealing with structured data for which we don't have necessarily a rich model description. Yeah, absolutely. Cool. I think that brings us perfectly to the end of today's session. We thank you all for watching today. Thank you for presenting and for showcasing the demo. That was really insightful. I think that should encourage lots of people to try it out yourselves. I mean, easiest way is wait for the GitHub repo to be updated and then pick your favorite Kaggle dataset and give it a spin yourself and see where you go. And obviously, you can obviously add on it, expand on it. Let us know when you do that. It's always interesting to see how you work with the stuff we're providing here. As we said in the beginning, if you have any questions, any feedback, any comment, anything you'd like us to cover in one of the future episodes, then don't be shy. Best place, I think, is probably the YouTube comment section, but you can reach us in other channels as well. And if you are interested in what's coming up, then I can give you a short glimpse of what's happening next week. I'm having Michal Stefaniak as a guest, and he will talk about the Cypher GUI, user-friendly administration for Neo4j. That should be fun and interesting. That's happening on the 13th of February, so next Tuesday. And yeah, we'll be back in March, beginning of March, and with another topic, another session of Going Meta, episode 26. Until then, I'm wishing you all the best of days. Enjoy time, enjoy the rest of your week, and yeah, see you soon. Thank you very much. Thank you. Bye, everyone. Bye.
Going Meta - Ep 25: LLMs for Automated KG Construction
Episode 25 of Going Meta – a series on graphs, semantics and knowledge

Jesús Barrasa: https://twitter.com/BarrasaDV
Repository: https://github.com/jbarrasa/goingmeta
Knowledge Graph Book: https://bit.ly/3LaqE6b
Previous Episodes: https://neo4j.com/video/going-meta-a-series-on-graphs-semantics-and-knowledge/

Daniel Bukowski on Github: https://github.com/danb-neo4j/llm
Croissant File: https://github.com/mlcommons/croissant
New York Housing Market Kaggle: https://www.kaggle.com/datasets/nelgiriyewithana/new-york-housing-market
Neo4j AuraDB: https://dev.neo4j.com/discover-aura
rdflib: https://bit.ly/4eSRPQE
Ep 5 Ontology-driven Knowledge Graph construction: https://youtube.com/live/05Wkg1p34ek
Ep 12: Importing RDF in Aura: https://youtube.com/live/DWINSvRxIbw
APOC Schema Assert: https://neo4j.com/docs/apoc/current/overview/apoc.schema/apoc.schema.assert/
Supply Chain Pricing Kaggle: https://www.kaggle.com/datasets/apoorvwatsky/supply-chain-shipment-pricing-data

Check out https://community.neo4j.com/ for questions and discussions around Neo4j.

0:00 Intro
5:23 Graph Jokes
9:55 LLMs for Automated KG Construction
19:25 Demo #1 Housing Market
37:10 Demo #2 Supply Chain
52.50 Q&A
1:04:33 Wrap-Up

#neo4j #graphdatabase #semantic #knowledgegraph #rag #llm