 Hello and welcome everybody. Good morning, good evening and afternoon. Hi Jesus, how's it going? Hi Alex, happy new year. Happy new year. I'm very well, thank you. Happy to be here one more time. A bit later than usual, but lots of things going on. Yeah, so we had to shift the week one week and we had to change the date, the day from our regular Tuesday to a Monday because otherwise we would have to shift it even more and so we are sorry that this is not the usual time and well, it's usual time but not usual day of the week in what was going meta for the previous 11 episodes but so episode 12, the one year anniversary episode, I guess, is a little bit different. But we try to be back on maybe a more, our usual slot for the next month, if possible. But I hope that's okay and you tune in nevertheless. If you tune in, it would be nice to hear from you now that we've completed the 12-year episode. Any thoughts, any feedback, anything is always welcome. And maybe this 12th episode sparks you to give us some of your thoughts. Either if you watch this afterwards in the comments on the YouTube video, or if you watch it live, obviously always open to hear you in chat. So please don't hesitate to type in anything really you'd like to get out to us. So I'm having an eye on two eyes uh on the chat um that's that's cool i think we we had a we had a great great 12 episodes and i think you you posted recently i need to find this this this article you you recently posted on linkedin this was maybe i'll i can find this quickly and link it and i'll say a couple of words about it because you know I guess that's the first of all credit to Michael Hunger who's been insisting me to do a session on this one because yeah I mean most of us in this series are familiar with new semantics but that we have been working on a kind of alternative. Well, not really an alternative, but it's kind of a different implementation of the same principle. And we thought it would be interesting to look at it, to share it, first of all, but also to look at how it's been implemented as a way to explain another way to interact with Neo4j, and also to you know to welcome uh any contributions to to the code base so that's that's the idea and now we'll uh yeah give a bit more context on where that comes from and and what what is it for exactly yeah i found the link in the meantime, so I shared it. So perfect. Yeah. Good. So I mean, if I don't know if you need to give any additional updates or anything coming up, Alex, or maybe at the end, or if you want me to get started, I'm happy to share my screen and we can kick it off. I think we can. We can get started. I can mention the other upcoming things this week or next week when we're finished. So yeah. Perfect. So we can get going. So today's topic, while you get your screen ready, is importing RDF data into Neo4j Aura with Python and RDF lib. So sounds to me and as usual, I got this topic like, well, 10 minutes ago. But it sounds more like a more like a hands on. Well, all of these episodes have been very hands-on and we explored some technical background and some theoretical background rather and some explanation and then went into the technicalities and into practice and hands-on. But reporting RAF data sounds very, we don't explore, I mean, maybe you, I mean, I guess you can tell. To me it sounds like we have some data, we're gonna get it in a graph database. Exactly, exactly. And you know, can you see my screen first thing? Yes. Just make sure it looks so good. Perfect. So as usual I'm going to give a little bit of an introduction on slides and then like you say we'll jump into the code and we'll get our hands dirty as usual but uh yeah the idea is you know the title i mean maybe some of you who are in this space i mean work with rdf and python are familiar with this rdf lib package we'll work with it and and we'll show how we have extended to be able to interact with with aura and let me tell you why where this all comes from so i was saying that what we know what we're all familiar with, and I assume, you know, it's not crazy to assume after 11 episodes that most of you are familiar with NeoSemantics, you know, N2S. So this is a library that has been, you know, a Neo4j extension that has been around for years now. And that helps us precisely with this problem. I mean, it's importing RDF into Neo4j. And in the process of importing Neo4j, sorry, RDF into Neo4j, solving the problem of translating this triple-based representation in RDF into a property graph representation. So that's what we all know know and love of course don't we Alex so but uh it um I would say that it has uh a limitation and let me explain what I mean by that so Neo semantics is implemented as a uh an ifj extension is a set of store procedures so in other words it needs to be uh deployed on the server side so that we can then, and I've literally copied a couple of statements from the manual, where you can see that we can, I mean, basically we use Neo semantics by calling store procedures. And the syntax in Cypher is called by the name of the store procedure. And we do things like, you know, configuring the graph and telling it how to behave when it imports RDF. And then the actual import works in two ways. We can either pass some RDF payload, or we can pass a reference to an external document or RDF generating an endpoint. It can be a Sparkle endpoint, whatever. But the idea is that all these calls require this NeoSemantics module installed on the server side, which is what this diagram kind of indicates. So the client, the Cypher client, expects some elements to be available on the server side in the form of stored procedures. Now, this is an assumption that it's not always valid. And maybe some of you who have requested this kind of alternative are joining us today, because you know that there are deployments of Neo4j where maybe you don't have control over what kind of extensions are available. And one of them, one prime example of that is is aura right aura in its current version does not allow the the deployment of uh custom uh extensions right so you can do that in your self-hosted uh install but you cannot do it in aura but people still want to be able to input rdf data in aura so the question is what do we do so does that give the the right context? It does. Yeah, that's a question we got a lot, I guess, since the beginning of this episode or this series. And I guess in combination with there being Aura Free, an easy and a very accessible way of getting into Neo4j with a graph database as a service, the start is made super easy. But then the limitation, considering RDF data, is that it's not compatible at the moment or it doesn't used to be compatible. And therefore, people were getting ready, getting excited and realizing, oh, I can do this. Exactly. We've been doing it. I mean, in most of the series that we've, in the episodes that we've run here, we've used Sandbox. Sandbox is a completely different environment and that happens to be, you know, it's a temporary instance that you don't really, I mean, you just own for a number of days. It's for playing around a little bit. But it's very different from Aura, right? And Sandbox does come with these extensions installed, but that's not a general assumption. And that's the problem that we wanted to address today. So, yeah, just to complete the title that we came up with, the solution is based around RDF lib. And in particular, an extension to RDFlib and I'll double click on that in a minute. But it's a Python-based solution. So RDFlib, for those of you who are not familiar with it, is a Python package that deals with RDF in general. So we'll make it possible for you to parse RDF, to generate RDF. It's an RDF management library in Python. So one of the possibilities that RDF lib opens is the extension in a number of ways. And one of them is the type of backend storage. We'll get back to that in a minute because in the second half of this episode, I want to have just a quick look at the code and the implementation of how we build that. But first, I guess we're going to look at how to use it, which is the most immediate, I would say, need. But you can find it in the package index, as usual. So if you do a search by RDF lib, you will find it as the first match. It's still in an early version, but it already supports the ingestion of RDF in any of this validation formats. And it's as simple as import, I mean, doing a install of the package itself, RDF lib Neo4j, which has a dependency on RDF lib. So it will import RDF lib with it. And then you're off to the races. I mean, you just need to interact on RDF lib, so it will import RDF lib with it. And then you're off to the races. I mean, you just need to interact with RDF lib as you would if there was not Neo4j behind, but in any RDF lib type of example, and you just have to specify that you want your graph stored in Neo4j. And we'll see that in a minute in a bit more detail, but the idea is that it's entirely seamless if you are used to work with RDF. So you will be working with your library without necessarily knowing that behind the scenes, you know, the data is being stored in Neo4j. That's what this library takes care of. So that's the idea. That's how we've done it. And we want to see it in action. And for that, we built a notebook. So let me stop the presentation and go into my notebook. But before that, we said we're going to be using Aura. So I should have actually kicked it off earlier. So I'm going to start an empty instance just to show that we're working alive i'll copy the password and continue i'll save the password here on my desktop just to be able to use it but yeah i mean now you skipped it over but it's always there's now with with aura uh this option to download the the the file the the you know login file so you get the the url you get the password uh as well and you can download it there uh from from the screen you just saw before and with that you have it on your on your local machine and you you you don't have to worry about forgetting it uh or losing it yeah i'll i'll you know i'll that's that's the kind of the right way the way people should be should be doing it i'll i'll probably do it in a in a quick and dirty way just copy the password use it for now and at the end of course i'll destroy this instance exactly this is this is just for demo purposes not not the problem but if you are planning to use this uh more often than uh you know it's better better not to do to save it in some text file you you might lose again exactly exactly so well let's let's give it a few a few seconds because that's the the time it takes to to get started but um while it does let's let's have a look at them of this of the notebook let me make it a bit a bit bigger to hope it's readable you tell me alex if it's uh oops that's not you yeah am i doing it right yeah here you go yeah i think it's great yeah right so well like i was saying before all we have to do to begin is just do a pip install. So I'll kick this off. And because there's dependencies defined, it will bring RDF lib and Neo4j. Both drivers will come as a consequence of these installs. So no need to worry about anything more than RDF lib Neo4j. And once this package is installed installed all I have to do like I was saying before is say well I'm going to and maybe it's worth just for let me do a quick well not even a search I can even copy the this manual on RDF lib so this is the documentation on on RDF lib if you want want to, you can Google that, of course. But basically, I'm doing exactly the same. So I'm doing the installation, and I'm going to take you through some examples. So like I say, if you're familiar with RDF lib, you know, the way you work with RDF lib is you instantiate a graph, and then you parse data into it. So that's exactly the way in which we're going to be working, right? So the only difference, like I was saying before, is that when I instantiate my graph, I'm going to be saying, hey, I'm setting the store parameter to Neo4j cipher. What I'm saying by that is that I want the back-end of my store to be Neo4j and to use the specific implementation that I mentioned that we're going to be seeing later on. But it's the one that we built with RDF lib Neo4j and to use the specific implementation that I mentioned that we're going to be seeing later on. But it's the one that we built with RDF lib Neo4j. So if I do that, I'm creating a new graph, an empty graph, that's going to be backed by Neo4j. Now, I need to provide some configuration, and in particular, where is that graph located? And in this case i need uh the detail from aura which hopefully should be running by now exactly so we have aura here and we can uh we can copy the uh the connection uri again this is probably not the most elegant way to do it but uh for this simple example i'm going to just paste it and i'll i can say i will uh destroy the the instance hang on is this that's instance and i need to paste uh the password that i copied earlier so listen to alex not the right way to do it but for now let's do that you know quick and dirty here i'll paste it and I'm going to install, I mean, this is my configuration with basically the URI, the database I want to connect to, my user and my password. And I'm setting, I'm calling the open method, passing the configuration. And if everything looks good, basically it's testing that there's connectivity to that database and that we are ready to go. So we now have a graph, right? And the way we interact with the graph, I'm going to use you as an example, Alex. But I'm following, especially for those of you who are familiar with RDF lib already, I'm going to use very, very similar examples to the ones that you had in the documentation. So RDF lib deals with RDF data, right? So because it deals with RDF data, it expects interactions in the forms of triples, right? So that's what I'm doing in this next section. So I won't go into all those details because we've talked too much about that. But, you know, triples, all the elements in the triple subject, predicate, object are uniquely identified by URIs. URI has a namespace component plus a local name. What I'm doing here basically is I'm defining two namespaces in my vocabulary. I'm going to have individuals, Alex and me afterwards, and I'm going to have a vocabulary as a schema, an ontology, if you want, that adds structural elements to it. So I define an individual's namespace by giving it a base URI and then a schema namespace, a vocabulary namespace or ontology namespace. And once I have them, I can create resources, which is the term used to identify URI in RDF. For example, I call one Alex, which is an individual called Alex, identified by this string Alex Erdo. So it's basically a concatenation of the individual's namespace, followed by this name. And the same with me, JB, is another individual followed by this name, and the same with me. JV is another individual identified by Jesus Barraza. So once I have these individuals, I can generate triples out of them, and that's the fun part. So I'm going to say, okay, Alex is of type person. So Alex is, of course, I'm referring to this individual that has been created here, and RDF type is a standard rdf is a vocabulary type is kind of a in our property graph world is a way of giving a resource a type which would be the equivalent of a of a label in a property graph right we're tagging it we're saying that alex is a person right we could add more of them actually we could say hey alex is a person but he's also a neo4j employee and so on and so forth so as we know things can have multiple types or in the property graph we would say that nodes can have multiple labels that's perfectly fine let's add more triples so i'm i'm saying also that alex has a name which is a Erdl, which is basically the same as the URI, but I'm giving it a nice spaced human readable version of the name. And I don't know, just an additional property. I said Alex has a Twitter handle that I have taken from actually your real one. So these are one, two, three, four statements, four triples about you that I want to write to Neo4j. I mean, that I want to write to my RDF graph, which is what RDF lib does. But I expect the magic to happen behind the scenes. And these writes, which I am effectively doing when I say add a triple to my graph, G is the graph that I created up here. add a triple to my graph g is the graph that i created up here right so when i run that and i hope everything goes well here so uh and i go i haven't opened it yet let's go back to aura and uh go to my uh let's go to query let's open a browser on my aura instance i'm gonna find already some information in it i need to copy once more my password. That's why it was not a good idea. But hey, that's me being a bit... So if we come here, let's give it a minute to refresh. Hey, we have already some notes. And interestingly, one is a new employee and a person. So let's see what it is. Hey, it's Alex. Not surprising. So we have a property graph. I mean, we have the RDF data that we've just created, that we've just written to our RDF lib graph, being persisted on the Neo4j side. And we see the translation that we're familiar with. So we see how all literal properties are translated into attributes of the node. So we have the name, Alex, the figure handle. And we have an additional one, which is the URI, which is the unique identifier. And we have seen also how the types have been translated into labels, which is the way Neo4j, in the property graph, to set the types of individuals, right? So, well, so far, so good. Let's add a bit more of information because, you know, the interesting thing, of course, in a graph is when we create connections. So, I'm going to just quickly copy it and adapt it to this other individual that I've called JB and let's do the same with me so I'm going to have another person I'm also an info J employee I have a different name and I have a different Twitter handle and and finally why not let's add another connection. Let's, I'm very lazy and like the copy paste, but I'm going to say that, that Jesus is friend of Alex, right? So these, as you can see, I mean, it's easy to interpret that. This is going to create another node with a set of properties, pretty much like what happened with Alex, but now with Jesus. And ultimately what we're going to do is create a connection between the two, a triple that connects two objects, JV and Alex, using the predicate, the vocabulary element friend. So if I rerun this block, it will hopefully not recreate Alex because it's already there. And this will not happen because the translator is aware of the fact that subjects URIs are unique. So if I'm using the same URI and that already existed in the database, it's not going to recreate. It will only reuse it it so if we go this has successfully finished so if we go back to our instance we now have uh see two notes and a friend of relationship so if i click on that we see that we have the two nodes uh you know with very similar structure the uri the twitter handle and the name and a connection between the two. So great, right? So pretty cool. What's interesting and probably, you know, this is more visible to the RDF-Lit practitioners is that, you know, if you look at this code, that is, there's nothing RDF related here, sorry, nothing Neo4j related here. And that's why I was bringing the manual. So you have an example, and that's exactly what they provide in the manual. So they create an individual that's done in this case, we're saying there's a person, he has a name, blah, blah, blah, nickname, a mailbox, blah, blah, blah. So exactly the same way in which you would interact with RDF lib, I mean, that doesn't change at all, is what you do, because effectively you're working with RD outlet, except that, you know, transparently and by setting these initial configuration parameters, saying that the store is an Neo4j cipher, every change to the graph, every interaction to the graph will be ported and persisted in Neo4j. Two important things here. Well, first one, the interaction in this example has happened at the triple level. What does that mean? It means that every time I run, and we're going to see the code in a minute, but every time I run a graph add, every time I add a triple to the graph, behind the scenes, I'm starting a transaction to the graph, and I'm writing that triple. So that is okay if we're in a kind of transactional situation, we're writing individual triples one after the other. But it probably is not the best idea if we want to import a few thousand or a few hundred thousand triples. Because I don't want to create a transaction for every triple. I probably want to write a bunch of triples in a single transaction and then commit it at the end. So that's what we're going to see in the next example. So here, instead of me manually adding triples one by one and manually creating them, what I'm going to do is using another feature in RDF lib, which is the parse function, where I only need to set the location of the RDF data that I want to import in the serialization format, and it will go fetch it from there and import it. And this in particular is this one here. So this is a dataset that we've used in other episodes in the past, but basically it's a SCoS-based. SCoS is a vocabulary, an RDF-based vocabulary, with a large taxonomy about software systems that we extracted from Wikidata. You probably remember, I think we used it in the, I think it was in the semantic search episode in the two or three. So I persisted that as RDF, turtle serialization, and we have loads of elements in here, right? For example, I'm sure this Neo4j here. Also there's an instance representing Neo4j and probably it has a unique identifier. Remember how Wikidata used these numeric identifiers? We're saying that Neo4j is broader than, I mean, has a broader element in this concept, which probably will be something like, let's, oops, I clicked too many times. So it's a label property graph. So you see how it's a taxonomy, right? So it's a bit hard to follow because this is in theory machine readable data. But the idea is that this large RDF set contains a taxonomy of concepts, as we call them here in SCOS, but basically represents software systems or software technologies. So yeah, this is a large file that I can access through this URL. And what I wanted to show is what happens if in my notebook, I want to import a large data set like this one. So I don't want this interaction to happen at the triple level. So I have this extra element, and this is Neo4j, RDF Neo4j specific. I can say, hey, and this is a command or a method on the back-end store. I can tell the store, hey, back-end store, go into batched write mode. And like this, basically what I'm saying is open a transaction, right? All the writes that I'm going to send you now, and these will be thousands of triples coming from this file, keep executing them in the context of the same transaction. And only when I end the batched write, we will commit the transaction. That will also create not only a single transaction, but also it will chunk the triples in batches and it will make the exchange with the server way more efficient. So if I do that and we can run that and that might take a few seconds, I'll touch in the indexes in a minute, but this is running the input of this large, I believe it contains, well, you can see it here, large data set. I believe it contains, well, you can see it here, 15,000, maybe a few more triples, something like that, right? 15,000 triples. So still a tiny data set, but these, instead of running it one by one, it does it in a kind of batched mode. So again, when this is done and should be done anytime, we can go to our browser. We see that the data is already here. So we have now a concept I have not deleted, you and me, Alex, doesn't matter. That's all good. We now see that we have we have a concept related through the broader relationship which is the one that we were using in scos let me zoom in and we see things like well there's uh programming languages and we have uh i don't know i have never heard of drama but it seems to be a a programming language uh george i don't know i have never heard of drama but seems to be a a programming language uh george there's plenty of of super i don't even know how to name that but there's there's loads of of of uh this is a very rich uh taxonomy that comes from wikidata yeah this is only a snapshot of the data this is by no means exactly yeah it's uh um yeah uh yeah it's it's not necessarily a tree because it's a taxonomy with, it's effectively a graph. But we can do something like, we can explore it and say, okay, let's find a concept. Because we're labeling things as concepts in Scots. That is, we see that all the elements have a preferred label. So we can say where C preferred label equals, oh, we think we saw it was Neo4j. I don't know if I'm going to be using the right capitalization, but we can do that. Return the concept if I've done it right, good. So that's this node that we saw before. And if I expand it, we see that it links to, we saw before that it was a subclass of graph database or property label, property graph, but it's also free software and proprietary software, interesting. So it's again, this is, I mean, the data comes from Wikidata, so it doesn't really matter. The focus here is on the fact that we've managed to import this large taxonomy of RDF into Aura directly. So, I mean, just because, you know, I suppose one of the interesting things is that, you know, we have offered the possibility of interacting with the graph on the RDF lib side as triples, as RDF, but at the same time, we have this property graph interface, and we can do cool things like, let me do that. So if I take the concept, I want to find the whole taxonomy for Neo4j until we get to a root. And we know that we've got to a root because the root does not have any parent. So it won't have, I mean, that's the way of indicating that, so the idea is the same. So I'm still looking for a concept that's called Neo4j. I will navigate the broader relationship, basically follow the taxonomy, until I get to a root. And I know that it's a root because it does not have any more broader than. So we go to the top. And we can store that in a, we can call it subgraph. And return this subgraph. So we run that. Little detail, of course. I'm not wanting to go just one, but as many levels as possible. So that's what the asterisk does. Otherwise, it was just going one hop. So with this, there you go. So again, the browser doesn't do the hierarchical layout, but we see pretty much here that Neo4j, as we saw is an instance of graph database, which is a database management system, which is a software system, which is a some software and so on. So that's kind of the tree for Neo4j. And again, we can interact with that on Cypher and as a property graph on this side, as opposed to in the form of triples as we were doing before. Right. So now what's happened here? And if you remember, let's connect with the introduction of the problem. So there's no extensions installed on the Aura side, of course. If I do a call db, I mean, that's of course apoc, which comes with it, but if I do vbms procedures, that's not what I meant. So that was procedures. What we can do here. Oh, the syntax has changed. It's show procedures. Ah, I can't remember. Anyway, there must be a way of... without this, I think. There you go. Show procedures. So, we'll see that there's some APOC elements and that comes bundled with Aura, but there's no Neo semantics right here as you would expect when you have it on your self-hosted. And so how did that happen? I mean, how did we manage to import this RDF data? Well, let's get back briefly to the slides. So what was happening before in NeoSemantics, so in this type of requests, it was up to NeoSemantics to go fetch the data. And the translation took place within the NeoSemantics and within the server-side extension in Neo4j. within the the server side extension in neo4j what's happening here what's happening when we use uh rdf lib is that the translation takes place on the client side so it's rdf lib and in particular the extension that we're going to see in a minute when i do add that it's transforming this triple into a cipher statement because because we cannot rely on Neo semantics being installed on the server side. What we need to do is create a cipher statement that actually stores the RDF data in the form of a property graph. And that's what this Neo4j cipher indicates. So it's a Neo4j backed store, but it's also cipher based. So it does not assume that Neo semantics is installed. it's a Neo4j backed store, but it's also cipher base. So it does not assume that Neo semantics is installed. I see. So you can run this in anywhere basically. So that's- Exactly. You can run it in a self hosted as well. If you're running it locally, but you don't have Neo semantics and you want to use this, all that's happening is that the interaction between my notebook here and my Neo4j instance or in this case is purely based on on cypher over bolt so it makes no assumptions on what type of you know components may or may not exist on the other side and I say that because there's there's two RDF lib implementations, and that's what we're going to get in a minute now. So the next slide was what's under the hood. So we've seen it in action briefly. So that's essentially, there's not too much to it. I mean, you can just interact with RDF lib as normal, but it's back to Neo4j. But then what's happening? So what we've done is in the RDF.Lit Neo4j module, and we're going to jump into that code in a minute, we've extended. Let's make that a bit bigger, although it's not super useful here. But one of the things that RDF.Lit offers is an API, store API, that makes it possible. And actually, why not? Let's look at this, because that will show us. So if I go to my ADFLib. So we see that this description of the store API makes it possible for developers to create multiple backends. There's some available backends for RDF Live already, and some of them are relational. If you look at Berkeley DB, that's a relational database. You can create a backend of different types, even based on SQL Alchemy, which is another SQL general relational database module. So you can create the backend that you prefer. So it gives you that possibility. And that's exactly what we've done with RDF lib Neo4j. And that's what I can show you here here any any questions so far i don't know if there's if there's any anything coming up on the but we can i don't see uh any questions but um i'm happy to take uh any questions that are that are coming so yeah if you if you're wondering i i posted the links to RDF lib and all the other information in the chat, so you should be able to see them. I will post the API link that you just showed in a moment. Perfect. So, right. So this is the, I was saying the GitHub repository, again Neo4j Labs, that we've built these implementations. So I'll go straight to the one based on Cypher, but there's this code basically creates two different implementations of the store backend for RDF. Right, so the first option is implemented relying on having NeoSemantics in the server, and then that makes the interaction slightly simpler, and it can leverage all the capabilities already built in NeoSemantics. But because there are cases where we don't have NeoSemantics, and that's exactly the Aura case, we have an alternative version that's down here. Is it here? Hang on. I'm not following. Initialize the database. Check the following section. It's further down. Here we go. So this implementation is the cipher-based one and assumes no new semantics on the server side. So what's happening here? So let's look at the code very briefly. And I don't want to... It's rather simple code, I have to say but uh it might be interesting to have a quick look at it so uh there's i mean the the store api which is what the cypher neo4j store implementation kind of uh implements uh has a number of of uh of required methods so the open one uh that you remember tested the connection basically takes care of instantiating the driver and creating a session. So basically it checks that there is a connectivity and that we can write to the database. But what's more important is the implementation of the add method. So remember how we essentially kept calling the add method and passing a triple as parameters. So the logic here is pretty straightforward. So if you remember, there's two different types of triples. And we can go back to our example. There's the triples that have a literal in the object. So remember, a triple is subject, predicate, and object. So if the object is a literal in the object. So remember, triple is subject, predicate an object. So if the object is a literal value, so what we're doing is we're giving an attribute, a property to that node. So if it's a literal, it means that we are giving the node a property. And that happens here as well. But if the third part of the object is not a literal, but it's another object, then it means that it's a relationship. And that's the case when we were connecting Alex and Jesus, right? So the first check that we need to do to understand what we're doing, if we're creating a property or if we're creating a relationship, is to understand if the object, which is the third element in the triple, is of type literal or not. And that's exactly what happens here. So if the object in the triple, and the triple is, like I was saying, a subject, predicate, and object. So if the object is a literal, is an instance of literal, then what we're doing is we are creating a property value. And we deal with the data types and so on and so forth. But remember that ultimately what we're doing is we are creating a cipher that can be sent to the database. So before I look at the specifics of that one, what we're doing is we, in the case, let's say literal. So, the URI is the subject, right? So remember that we store the subject, the URI, the identifier of the triple in the uri property in uh in um in in the node that's being created and then the rest of the parameters are stored in a in in a resource i mean everything in neo4j when we store it we create a uh a node of type resource we can go to the, so all these things are in addition to being a concept or whatever types we give it, it's also a resource and that's a way of having a way of merging on a unique identifier, which is the URI. We make sure that if there's already a resource with the same URI, we don't want to recreate it. We want to reuse it. And in order to do that, we leverage the merge keyword. So that's what's being generated here. So we are merging on a resource node with a particular URI. So that's the URI that we get here. And then because we're in the context of a literal property, we're setting a property and with the value. So that's what we're doing. And we're using the format to pass the actual value for the name of the property and the parameter to the right. But you kind of, without going into too much detail, we see that what we're doing is we're doing a merge and setting a property, right? If the predicate is type, is RDF type, what we're doing, remember here, is we're setting a label because types in Neo4j become labels. So the second check that we do here is if it's type, then we're setting a label, right? And that's what we're doing here. So we're again merging on the same URI, but then we're setting a label and the type will be, I mean, the label will be the actual type that we get in the object. Does that make sense? And finally, if it's not a type or a property or a literal, then we will be creating a relationship. So basically we're generating the cipher that merges on the from and the to, because that's what we get in a triple. So we get the from, we get the to, and we get the type of relationship in the middle. So that's the from is the subject. We get the to from the object, and we merge the relationship with the property name that we get from the predicate, right? So, you know, again, there's some additional details in terms of how we, instead of running that directly, we can put that in a batch and we execute it if we are in this kind of batch execution mode. But essentially what we're doing is we're taking the triple that we get and we're generating a Cypher statement that basically stores that triple into Neo4j purely based on Cypher without relying on any additional element. So yeah, I think that's probably enough in terms of looking at the code. I don't expect people to be super interested. Probably worth mentioning that there's the other one that we said that relies on NeoSemantics. So there's a very similar implementation. But if we look at the add implementation of it, interestingly, all it does, and look at the difference in the code. I mean, I don't have to deal with all the exceptions. All I have to do is, hey, I'm going to call ntns import because this method knows how to deal with it. So all I have to do is take my triple and then pass it to that method. But then, of course, that assumes that this method is going to exist in my database, which is not the case in the other. So, right. So there's the possibility of working with RDF lib in Neo4j, both in the case that you have Neo semantics or where you don't, but because it was about Aura where that's not present, we've shown how it's possible to do that in a purely cipher-based approach. So this RDF-lib implementation just takes care of the translation and sends the cipher over bolt to Aura, and the interaction takes place just like that. Good. I'm going to take a pause here. I mean, I think that's the core of what I wanted to show today. And we've had a look at the source, so we kind of covered the content for today. So I think I'm going to take a pause here. It's slightly unusual that we're slightly short. We're a little ahead of time. Not a bad thing, I suppose. But I think it was particularly probably dense in terms of But I think it was particularly dense in terms of the code and all that. But any questions from your side, Alex? Does it make sense? It makes a lot of sense to me. I think it's very interesting so that you can use this with, like I just said, with NeoSemantics installed or no. or a particular user of RDFlib, I guess, makes it very easy to just continue using it and you have connected to Neo4j Graph Database no matter what setup you have been given. Let's say you have an IT department that sets up your Neo4j instance for you and they don't want to install NeoSemantics for you, you can do it. Or if they have installed it for you can you can still continue using it and I guess it makes it very easy to to continue working with with in in your in your environment and your current stack I guess where you don't have to learn how to work with Cypher and Neo4j and all of that. So you can just go with it. And on that point, I'm going to interrupt you. Sorry, Alex, but it's very important because, you know, we've decided to start with this Python implementation using RDF lib. But in the same way, there's this package of RDF lib in Python. There are packages in Java. There are packages in.NET. There are packages, you know, to deal with RDF in Java, there are packages in.NET, there are packages to deal with RDF in JavaScript, right? And we're talking to different teams that we're into, because basically what we've done is we've taken the input part of Neo semantics and we've kind of extracted it from the server side and put it on the client and make it a kind of a client-side feature. We've done it with Python, but if you're working with another programming language, the same thing can be done or even can be created as a, I don't know, let's call it a service like a Lambda or something where we just pass the RDF and it does the translation. So there's multiple implementations, but the idea is exactly what you say. So you can stick to your stack, to what you're comfortable with, whatever your RDF library, RDF package to interact with RDF, and let that take care of the translation into Neo. So that's, in a way, kind of a proposal. Is one possible. Of course, you're free to use it, but it can be used as inspiration to implement something similar in other programming languages. So very, very, very good point there. Yeah, I think that's very important to make it as easy as possible to combine the two worlds, graph and RDF. And that's, I guess, the goal of this work. to combine the two worlds, graph and RDF. And then just, I guess the goal of this work. So it's cool. I think you are looking forward to feedback. So if you try this out, if you have any feedback for Jesus, for the team that's working on this, do it via GitHub, I I guess or what would be the best ways to I mean the usual I mean if it's something specific about the code GitHub is perfect because all these projects are in GitHub so you can open of course an issue or even better you can you can submit a pull request and and any contributions are more than welcome but as an alternative there's always the the community where you can ask more general questions and and any contributions are more than welcome but as an alternative there's always the the community where you can ask more general questions and and we can point you in the right direction so any any channel is is it's great probably much better than the comments on on youtube sometimes you know we get a youtube no yeah yeah that's probably not the best way super welcome but for this kind of more code specific, you know, this type of, I mean, it's a lot easier. So, yeah, that's, yeah, absolutely. So feedback is more than welcome and suggestions and comments and even shared experience of what's it looking like for you. What are we not covering yet or what have you extended to cover? yet or what have you extended to cover and um and yeah so i hope that the you know the the ingest i mean the fact that that aura uh is your preferred uh deployment it's not an obstacle anymore for importing rdf data into your into your graph and and working with it pretty much the way you were used to with uh with new semantics although you know let's let's make it clear at the moment this is you know in in early stages and and we've covered the import part but there's a number of features that still not supported here for example the export part i mean it sounds like a kind of a symmetric one it should be relatively straightforward but this is work in progress so keep an eye on this and and we you should be seeing soon uh the the equivalent part on on soon the equivalent part on querying the Neo4j graph and streaming out RDF from it. So what I don't know is if we will get feature parity eventually, because there's a number of things like mappings and other types of things that we can do in Neo semantics. We'll see. Basically, again, this all comes down to people's interests. And if we see enough, enough interest, we'll, we'll, we'll consider implementing that or, or, or maybe we'll just keep it more, more simple and more basic for these, for this type of access. When you're using this, I mean, RDF lib, I guess it's used to, to work with RDF data with, with not, not necessarily Neo4j, I guess, but with other, other tools. Yeah, exactly. So can you, and this is maybe a stupid question, but could you use this to work, to do the same, to do, to parallel work, basically say, okay i have my my rdf data i want to one one one way goes into neo4j database because i want it there um to to know and then my in parallel it also goes to this rdf tool i'm i used to i used to work with in the past so just to see for transitional areas. Absolutely. So that's perfectly valid. I mean, you could have, you know, that would be, I'm going to just sketch it here. But you can say, hey, I have a, let's call it, that's Neo Graph. That's backed by Neo4j. But then I have, let's say, triple, triple graph that's backed by, and I'm not very familiar with all the implementations that exist there, but we can look at some of them. Where was that? So, here we look at the documentation, oh, that's the API. That's not what I wanted, but we wanted... Hang on. So, where was it? So installation, more examples. I have to say that this is you know not the best the best extra i mean i want to be critic but it's a bit tricky to to uh loading and saving so let's go here further help source code for developers i think it's here uh plug-in stores here we go so for example here you go so there's a a So, for example, here you go. So there's a Sparkle store, for example, or different types of the Berkeley one. I mean, when I use a relational, but you can, you know, Oxigraph. I'm not familiar with all of them, but these seem to be HCT. This is another triple-based store. So whichever is your preferred store, you can have, you can have your choice here. And you can work interactively with the two of them. So you can, like you say, maybe want to have a replica. And everything that goes to my triple store goes to my Neo4j graph. And again, the good thing is that the syntax would be identical. So I would be just writing ads to one and the other. One would be writing to Neo and the other one would be writing to the triple store. Or you can decide which triples go to one and which triples go to the other. I mean, whichever logic is the one that makes sense for you. But yes, absolutely. You can have as many as as you like um and work with them in yeah the same way you would i think that could be interesting uh and particularly for for users that have been working with with rdf previously but not with graft of india for j and want to see uh how how it looks like uh but continue working with the data and then see the advantages and then maybe get into the world. Yeah. And I mean, we haven't touched on it today, but there's the, I mean, the whole, I mean, if you, what you're looking at is that a migration, of course you can, you can suck data from one and move it into the other because, you know, in the end, RDF is a, is a data exchange model. So that's the whole purpose of it. So whether that's to move data out of a triple store into Neo, or maybe query something we've done in the past here, right? Query a triple store like Wikidata, DBpedia, any of those with Sparkle, stream triples out and use that to populate Neo4j. So that's absolutely fine. I mean, conceptually, it's exactly the same thing that we've done here, except that instead of reading from a static file like this one that we were showing here, right, instead of reading from a static file, you would be reading the output of some execution, whether that's reading from a triple store, right? Totally. So it's, again, the important thing is that you are working with RDF-Lit pretty much in the same way you've been working with all the time, except that, you know, by using it as a side effect somehow, your triples are being persisted in Neo4j. So that's the benefit of it. Yeah. Very cool. We have comments from Peter Dimitrios. Performance looks pretty good for basic RDF Python notebook with Neo4j Cypher for doing analysis using Cypher queries. Well, if that's, and I've just realized that I didn't do it. I mean, if that performance looked good, I mean, that's nothing compared to what you would get really because I should have created an index. I mean, that's nothing compared to what you would get really, because I should have created an index. I mean, that's something that I should have done and I didn't. So it's because, you know, keep in mind that we're constantly merging in the database because it's all about, you know, when you have triples that refer to each other, same subject, you have to look it up and reuse it if it exists. So you're doing constant merges, which is lookups and writes. So if you back that by an index it's going to be a lot faster and i completely forgot to do that but essentially all you have to do and i have my my notes here but yeah i mean kind of interrupted you there alex but uh yes it's quite quite decent so what you would do normally is in a in a first uh interaction you would probably just create uh an index that's what i normally do you create a uh you know or if a constraint would be even better but an index is fine you would say i want the uh uri property in a resource to be unique and once you do that, essentially, we can do it automatically creates an index right for the for the constraint. And something like what we just done, I hope it hasn't broken things. But now, that should be significantly faster, right? Cause it's, I don't know if it's still keeping the session, but anyway, there you go. So you see how it was, you know, it took, you know, it still is a small dataset, but the performance is decent, right? So it's, keep in mind that we're translating into Cypher and it's not as efficient as with new semantics, which is a much more kind of lower level uh type of access but but it's it's quite decent so we're looking at you know taking just uh you know getting to to a few thousand thousand triples per second right so here that's uh that's the kind of yeah i think that's always a good way to do it. I mean, indexes help a lot. Another question from Mark. If you add extra properties with Cypher to a new node generated by RDF lib, does the RDF view still work? I guess that's the expert view, which is not implemented. Absolutely. I mean, the thing is at the moment, I mean, it won't be a problem at all. So we can just, I mean, whatever's persisted on the Neo4j side can be modified pretty much in the normal way. So you can have a, I don't know, take a concept and you can delete it. You can add a property. You can do whatever you want. That's absolutely fine. At the moment, the RDF implementation is one way only. So it only writes to Neo4j from RDF. So it's absolutely fine. It's a very valid question when we talk about regenerating the RDF or reading RDF out of Neo4j. So maybe in that case, and we'll figure out how the implementation goes, maybe there are some requirements for the information in Neo4j to be serialized correctly, or at least consistently with the one imported from RDF. But in principle, you can hit the graph from two sides. So you can write to your Neo4j graph from RDF with RDF lib, but at the same time have some cipher scripts from this side or have another element. That's the whole idea of Neo4j is built to take multiple, multiple write write throughputs and write workloads with through any channel. So shouldn't be any problem at all. And it kind of makes sense, right? Because sometimes what you do with importing RDF is enriching a graph that already exists. I mean, in this case, we started from scratch with an empty or instance, but sometimes very often, I i don't know you have a graph already and you want to enrich with geographical information with organization information with you know stakeholders so it's very common that the graph is is populated not just uh via rdf but through other other other channels and other other ways yeah that makes a lot of sense. Very good question. Peter, different Peter, Peter asks, are you, Jesus, are you working on a book on semantics? I'm doing work here and there. I mean, it's not specifically on semantics. I mean, on the new semantics. And I know that that's the problem is there's so many hours in the day, but I, you know, I feel sometimes guilty about the fact that there are some features in new semantics and in the different components. For example, this one, that's not sufficiently documented, but I think it was, you know, more interesting to run this session, to put out some examples, rather than write a proper documentation. So the answer is that I'd love to write a book. And even before a book, I'd love to write some proper documentation or get some help on that. But I'm struggling with that. I mean, this is Alex. Are you going to help me on that? ALEX KOLBARNIKOV me on that? Writing the book? Of course. We can call for that. Yeah. Let's see. Yeah, it's always difficult. I'm doing the GraphQL Book Club with Will, and I think Will has very enjoyed the process, but it's a lot of work. It is indeed, yeah. How many hours you need to pull into this book. It's a lot. It's a lot. Yeah. Nights and weekends. And yeah, it's a lot. Okay. Peter, different Peter again is interesting to have some guidance on use of constraints or just indexes on things don't want to go overboard but eventual generic tuning techniques for Neo4j could apply. I guess that's true, right? I mean if it works generally for Neo4j, if it's a good best practice it will work in this case as well yeah absolutely absolutely yeah in this case the only i would say strong requirement is uh it's not even a requirement because you've seen that we've run the session without it but the thing that will really make a difference is to have this constraint this index on the uri because that's the fundamental element when when working with rdf the uri is the unique identifier and it's, you know, the whole RDF graph is based on these notions. So you have to have a fast way to access and to retrieve and to modify them quickly. So that's the only thing. All the rest is, you know, it will not have an impact on the RDF input, but it can be beneficial in the usual way, depending on the type of of queries or the type of analysis that you run on your data. Indexing can definitely be a useful tool. I would recommend maybe have a look at the manual or the developer guides that your team, Alex, produces. It's probably the best reference that I would recommend. In the docs on Neo4j there is a lot lots of uh you know important guiding uh and and help and there is something on indexing which i don't why find right from top of my mind but i'll work it uh into the uh the description of of the youtube video so you find it there in a second i'm sure the graph Academy also touches on that. So I know. It does. Yeah. Yeah. I mean, this is, yeah. So it's everywhere. Cool. Yeah. I think that's all the question. Thank you for your questions. Thank you for the participation again, if you try it out and give some feedback afterwards. So it's always, it would be interesting to hear your thoughts from now you've seen it when you use it. And then we can see what else there is we can implement or what's a, you know, are you RDF lib hardcore user? And this is the fun feature you'd like to see, let us know. So we can see, okay, this is maybe more important than something else. And we can prioritize, I guess, development. And I would say also from the point of view, you know, because the whole driver to this, I mean, RDF lib is probably the most popular package to deal with RDF in Python. But the whole driver for this is being able to support, I mean, RDF lib is probably the most popular package to deal with RDF in Python, but the whole driver for these is being able to support, you know, the integration with RDF from Aura. And, you know, if there's something specific that you want on your Aura deployment or your Neo4j Aura graph that's currently not supported or that is only partially, I mean, this is the kind of thing that we also want to want to hear and as usual uh the code will be on will be on github uh you know probably show immediately after after this this session so all the code that we've run in the absence of a good documentation at least we'll have the we'll have the examples the example exactly in the video i mean mean, this helps obviously a lot with follow through, at least as a demo kind of step-by-step guide and then do it yourself and give it a spin and that usually helps a lot as well. Yeah. All right. I think that's the end of episode number 12. So one year of going meta concluded. So it was really- High five, Oleg. High five, exactly. Really good, good, good, good fun. Good, good series. Looking forward to season two, as they say. So I guess we continue with this looking forward to next next month hopefully back on the usual slot the Tuesday Tuesday of the month right first Tuesday of the month exactly so that's what we try try to to to be back And yeah, if you are interested in more Neo4j live streams, there's a couple of going on this week and over the next couple of weeks still. So tomorrow I have Tom Azel join me to talk about Wardley maps and how they work in Neo4j. So that's an interesting session, I think tomorrow afternoon, European time. And on Wednesday, we go into the next chapter of the GraphQL book club. I just mentioned earlier, Will Lyon, our colleague has written a book on GraphQL, a full stack application development. And we are going over each chapter in each session. And this is chapter seven. So if you're interested in GraphQL, join this. If you're interested in seeing how, what is a Wadley map, how does a Wadley map work and how to combine the two with Neo4j, tune in tomorrow. And with that, yeah, thank you, Jesus. Thank you for the work on all the episodes, specifically on the, now on the, on the i guess neo4j aura import functionality of rdf that's really cool and thank you all for watching yes thank you thank you bye
Going Meta - Ep 12: Importing RDF data into Aura with Python + RDFLib
Episode 12 of Going Meta - a series on graphs, semantics and knowledge

Jesús Barrasa: https://twitter.com/BarrasaDV
Repository: https://github.com/jbarrasa/goingmeta

Going Meta 12 Month Recap: https://www.linkedin.com/feed/update/urn:li:activity:7016121635761971200/
neosemantics: https://neo4j.com/labs/neosemantics/
RDFLib: https://rdflib.dev/
RDFLib Neo4j: https://github.com/neo4j-labs/rdflib-neo4j
RDFLib Persitence: https://rdflib.readthedocs.io/en/stable/persistence.html

0:00 Intro
3:38 Topic Overview with RDFLib
11:11 RDFLib with Neo4j
16:10 RDF Data Handling
24:10 Parsing RDF Data
30:10 Working with the Data as Graph in Neo4j
35:30 Code of RDFLib with Neo4j extension
46:10 Summary
51:25 Q&A
1:05:45 Wrap-Up


Check out https://community.neo4j.com/ for questions and discussions around Neo4j

#neo4j #graphdatabase #semantic #knowledgegraph #rdf #semantics