 Hello, Jesus. Good afternoon, everybody. Thank you for joining. How's it going, Jesus? Where are you? It's going great. Well, I'm in a different room, absolutely. And one of my concerns was actually whether I was going to have a good reception. So please do tell me if things come, you know, all right. So any any any uh interruption or whatever so just to answer your question i'm i'm in spain so i'm i'm in an unusual uh location so unusual for me anyway spending just a couple days with a family but uh yeah it's a very short visit so i'll be i'll be back in the uk tomorrow so it was just uh an accident an accident that we're running this session from here but you know why not i mean we have we run one uh one of the early ones when i was in new york at the at the knowledge graph conference so yeah i remember just to make the the webcast more more international right so yeah anyway same quality of content doesn't matter where it comes from. Exactly. That's the most important thing that we make sure that this definitely hits the mark. And, you know, I change location, you change location. I think that's the beauty that we can do this and make it as available as possible to as many people as possible. So I think that's the core of the target. Before we jump in, I think I will do this until October, but I'd like to mention notes briefly. Call for Paper ended last Friday. So if you missed that, there might be a way, if you say pretty please until end of this week, I might be able to get you in if you want to submit something. So, you know, think about it quickly. Otherwise, you can obviously register for notes already. So it's now seems October a long way away, but if you want to, the registration page is up and running, so you can already grab your ticket, basically your virtual seat, and be ready for it. So you can do that, and all you need to do is go on the webpage. I'll post that in the chat as well. Let me find the link, which I don't have ready right now. Sorry for that. Here it is. So it's the same I shared generally over the last couple of weeks. But yeah, it's going to be a fun event, very interesting. We're currently setting up the agenda. So in a few weeks, end of next week, maybe already early, early week after that, you should see early, early agenda coming together. And I'm looking very much forward to it. So we had lots, lots of submissions. So we have the, you know, difficult task now of sifting through all of them and finding the right ones. So that will be that will be done um by michael hunger and a couple of other for the team and um yeah i'm looking forward forward to the to the day in october so that's that's what i wanted to bring quickly get out um yeah looking forward to that should be a should be a great one which edition are we at i mean it's it's already the fifth edition yeah yeah yeah yeah crazy yeah that's good but now let's let's dive in uh to to what everybody has come to to see and want what what to hear about and want to learn about is um is going meta so in this episode uh the title is easy full graph migrations from turtle stores to neo4j in general the the aim of this so if you maybe are new to the episodes or new to the series and didn't didn't watch all of the episodes so you don't have to this is not a it's a continuation somewhat but it's not like necessary that you have to you didn't have to you know watch all the all the 17 now previous episodes you're welcome to of course and by the way by the way alex i know we're in big numbers now and you know it's harder and harder to keep track but there is a typo there it's not 17 he's 18 in the in the banner so it's true i know i know we'll be in 125. It will be hard to know, but it's one of those things. But yeah, no, you're right. I mean, maybe I'll make some references to some of the past episodes, but the great thing is that they're all up there online, all the recordings, all the resources, everything if you want to go and reproduce them at home, it's up there. So yeah, that's always a possibility. Exactly. So everything is there. So if you watch it outside of YouTube right now live, then go to our YouTube page, neoforj.com, youtube.com slash at Neo4j. And then you find our page and then on the live you see the all the previous episodes and then there's a going meta playlist there's a going meta website which i could also link i think i have it in the video description so there is there is there is that and yeah we we talk about knowledge graphs, we talk about semantics, ontologies, taxonomies. That kind of is the couple of the keywords, I guess, that usually fall when doing this. RDF is a big, big part of it. And I guess today, again, RDF will be something we're going to look at or talk about when, I guess, also using new semantics to work to work with with that kind of data and get it in a it's it's a graph pattern already but getting into it in Neo4j and having making the making the data a little bit more usable in in in certain, I would say. It makes sense to migrate from RDF data to a graph, to a property graph like Neo4j is one, but I don't wanna speak ahead of time. No, no, of course, and you're right. I mean, I suppose the content that we cover here is for, you know, I think it combines some elements of the traditional sort of RDF or semantics stack, Neo4j. One element, of course, is moving data across from one to the other. And, but yeah, we just try to experiment and we've tried all sorts of things. Yeah, some people will find that we're trying things that normally are know are associated with the semantic stack but uh but yeah that's what we like it i mean we we try new things and and and i believe that that's the interesting thing of these series and and today you're totally right so um i'm actually going to if you don't mind start uh probably sharing my screen. Is it, should I start presenting? Yeah, forgotten it's been. So. I don't see it yet. So once it's. Yep, let me share my screen. It's just the usual couple of slides. So I don't want to take a lot of time, but just go straight into coding. But the idea, like i was saying yeah you know it it's just one slide because it's it says it in the in the box right so the idea is to to to do simple uh full graph migrations from from triple stores to to neo4j and and um and the idea again is is quite simple we keep reiterating that. So we really value RDF as a standard that makes it really, really straightforward to exchange graph data. And that's been a theme during this series. And I'm happy to be aligned with the W3C. If you go to the page, it is actually defined as a standard for data interchange in the web so it's like i say a great way of streaming out your graph or you know ingesting graph data in a in a format that's neutral dates as we're going to see it's entirely independent of the way you you store your data you persist your data and um you know it's important also to understand that uh you know my position personal but i think you know that's the shared approach at neo4j is that um you know while we find rdf a great format for exchanging data we have the impression that developers data people tend to feel more comfortable with the level of abstraction that property graphs offer and and again i i you know when i think of alex and me in a graph i see two nodes right and there's connections between us because we work together because we're friends because we run these webcast together because we have common friends, right? So, and it's easy for me to think in those terms rather than kind of decomposing Alex in statements as, you know, Alex's date of the start date at Neo4j is XYZ or Alex's name is Alex Erdl, Alex's employee number, blah, blah. So which is exactly the idea of RDF. I mean, in terms of working with data, manipulating data, I'm very comfortable with the level of abstraction, again, that the property graph offers. And Cypher is a great environment to manipulate the data. But I appreciate, and that's exactly what we're gonna see here today, that if I want to exchange data, RDF is a great mechanism. And that's the topic for today. And the idea is pretty, pretty simple, right? So I'm going to play with a couple of popular triple stores out there. In particular today, I'll use Stardog and OntoText. I was working on Neptune, but I didn't get it to complete. So I'll add it to the repository, but we probably won't cover it in this session. But the principle is identical. I mean, these three platforms store data as RDF. And because RDF is just, like I said, easy to stream out of a platform and easy for us to ingest, thanks to new semantics, the migration is going to be really, really straightforward. That was where I introduced the easy, a full graph migration in the title, but that's the idea. Does it make sense, Alex? Yeah, yeah, that sounds good. Maybe before we dive in, there was one comment from episode 17, the repository folder is missing. That is right. Yeah, yeah. And that's entirely my fault. It's true that it was a bit probably the most experimental of all of them. And it's the one where we tried the open AI interface. It was kind of I had mixed mixed feelings about the results. But anyway, yeah, I'll put it all up there. Sorry. I'll make sure that happens after today with both 17 and 18. yeah thanks thanks for thank you thank you jesus thank you for the comment uh yeah um so we we don't uh forget about it yeah that sounds good um um i think that makes sense that's a that's probably a common a common path i would say or lots of people might have the same, not necessarily challenge, but maybe think about doing it or want to do it or have a data store similar to this and think, OK, what do I need to do if I were to be migrating this over? And I think that's great to show it how it works and how it could be done. Yeah, and that can be, you know, it can be, of course a migration because you want to to work on a property graph because you want to run some analysis in neo-vij let's say you want to run some sort of graph algorithms and and then enrich graph and then bring it back to the to your triple stores whatever the logic in this case we're going to be just focusing on the on the pure migration the moving data out and what we're going to see is that depending on which environment you work on and today's environment you know just last slide is a python notebook but uh depending on what environment you use i mean you will have additional tools in the form of drivers for example you know startup will see that they offer a very nice Python driver that makes it really, really easy to work with a Python notebook environment. Onto text, I haven't been able to find one, it might be my fault, but what I'm going to do is just purely using HTTP requests, which is something that we can do in a notebook environment. But, you know, again, depending on what's uh the uh you know the environment you want to run your migration on you will have better or more or less support but um yeah these two approaches that we're gonna see today uh cover kind of uh at least two options for these and like i say neptune i'll add it later on to the repository, but the ideas should be entirely valid. So let's get started. Good. So what I've done is I'm going to work entirely in cloud instances. So for Neo4j, as usual, while we were starting, I created a sandbox, which should be now up and running. Perfect. So we have it here which should be now up and running. Perfect. So we have it here and the connection details and everything. I've done the same with Stardog. I mean, I looked at the free cloud offering and I've created a simple database that I've called Migration Test. And I'm going to interact with it through the Stardog Studio, which is the equivalent of our Neo4j browser to interact, to write some queries and get the results. Of course, because this is RDF, we'll be writing Sparkle, but the idea is pretty similar. So I will not spend too much time because the focus today is on the migration itself, right? So, but just so, you know, what I did is I created a database that I call test database and it contains music data. Again, it's a data set that startup provide. And let me, I think I have a couple of URIs here that are going to be useful so I can show you what that can look like. But if I want to extract information about Nirvana, the band, we will be able to write something like this. This is the URI that uniquely identifies Nirvana, and I want to say, well, what? Tell me all the triples that have Nirvana as the subject. I'm going to call this predicate and object, and I'm going to return precisely that, predicate and object. So if I run something like that, this should return, well, everything we know about Nirvana. Well, not everything, but because it's just the outgoing triples, not the incoming one, because, you know, I have to be able to speak about that we'll look at that in a minute but we see things like yeah nirvana is a is a band is a producer is a songwriter uh we have a a description of it here which is a longer text description of it it has a name nirvana the band and it's connected to a number of of uh of individuals through the member relationships so we can uh i don't know we could extend that query and say okay uh if i follow the member predicate i would get to the actual members and maybe i would like to get for these members uh some uh predicate object information about them right so and I could actually return the members. So, well, I said that I wasn't gonna do it and I've done it, I forgot the dot. So I don't want to spend too much time basically running queries here, but you know, the idea is, you know, it's a simple dataset that, again, it's a public one and I'll share the link. But here we get the information about the members of the band, right? Kurt Cobain, Dave Corle and Chris Nubosovic. So yeah, that's, like I say, an Adia-based database that we can query with Sparkle. But today, we might just check that things have been migrated correctly with the query. But the idea is not so much exploring it, but looking at how to migrate. And like I said before, I have created this notebook, what I'm going to basically write the logic. And I'm gonna take you through it. I'm probably gonna make it a bit bigger. Alex, tell me if it's readable. Oh, this is readable. Yeah, maybe I put our faces a little bit more further. Let me get that done, But yeah, you can continue. Perfect. So as I mentioned before, Stardock in particular offer a nice Python driver, which is the one I'm importing here. So I've run the install for the PyStardock, but also for Neo4j. And after the imports, what I do is I pass the connection the connection details of course you can copy those but i'll disable them so don't so that you don't use my uh but anyway that that's basically where you know the the instance the endpoint which you get from your uh stardoc studio so you get easily from here so you get the endpoint url so that's pretty much what i'm what i'm using there and um and this is the connection details user and password so you start a connection right pretty straightforward so you use the credentials and you in it indicate the database that you want to connect to in my case is test database which is the one that i was showing here there's i think there's another one that comes by default, the catalog, and you can create additional ones, but I'm using the test database. So that's what I'm doing here. So I have a connection now. And again, this optional part is how I loaded the data because by default it was empty. So this is, I'm not going to run it now because the database is already loaded. But basically I imported the music data set from this URL, which is again another one provided by a start. What's interesting and what we're going to focus today is this section here and that's all. That's why I said it was easy. So this block here is all we need. So basically what we need to do is, we have to create a connection. Maybe we can zoom in one more step, Jesus. Sure, absolutely. Even, does that look better? Yeah, that's perfect. Yeah, thank you. Good. So, I mean, that's a previous instance. So let me make sure that I'm pointing at the right sandbox. So let's open it here. And while I do that, I'm going to look at the connection details. So the URL is this one. So I'm going to look at the connection details. So the URL is this one. So I'm going to copy to the clipboard and get back here and replace it with the right one. So that's my database. That's my authentication. Probably the password has changed as well. Let me copy it here. So it's NeoPJ and this is the password. There we go and i'm going to use i mean i can decide to use uh the database that i prefer but i'll use the default one which is which is neo pj right which is uh created by default and which should be um empty Let me connect to it through the browser like this. I can see what's in there. And while that connects, I'll keep moving here. So what are we going to do? So we same way created a connection to the startup database. I'm going to create a connection to the Neo4j database. And these are the credentials and the URL. And once I have the connection created, the driver created, which I've called NeoDB, I'm going to initialize it. You know that before we can import RDF, in order to use Neo semantics, we have to both create constraint. You might have seen me create every time I use it. But again, you know, won't spend too much time on the details. That's a requirement. That's what this statement does. And then we need to initialize the way we want the RDF to be imported out. I'm going with the defaults. So I'm using, I'm calling the init in NeoSemantics, the GraphConfig init. And then I actually go ahead with the actual migration. And the migration is as simple as, well, if we start from the end, we're calling on the connection that we have to Stardog. Remember this con here is the connection that we created to the Stardog database, to our start of cloud. And they offer a quite convenient function that's called export. Right? So that's, that's exactly what I'm using. So the export returns bytes. So I have to decode to, so that it transforms it into a string, but that's, you know, the RDF as it's streamed by the export function is the thing that I'm directly ingesting on the Neo4j side. And to do that, again, you might have seen many times as using the RDF import method in Neo Semantics that takes RDF in any serialization, by default, the export produces turtle, which is what I'm specifying here on this side. And as you can see, all I'm doing is, you know, import whatever comes as a payload. And the payload is a variable, a parameter, that I've set as the output of these export on the startup side. Does that make sense? Yeah. So that's it. So let's write, and let's see what happens. This indeed sounds very easy. Yeah, exactly. So, you know, it's pretty much, I mean, I think I've run most of these, but let's rewrite again in case some credentials have changed. So I set my connection details. I create the start of connection using the driver. I'm not going to populate the database because it's already populated and what i'm going to do here is uh like i say initialize my neo4j database which by now should be already up and running so we can see it here i'm going to do something because you know i don't like when it connects things for me uh but yeah it's uh it's uh you know the neo4j database is absolutely empty, no labels, nothing in there. And let's run that step here. So all we need to do is, now it's happening actually, it's exporting the database and at the same time is importing it here. Then we close the transaction and hopefully that should leave us in a state where we have the graph loaded on the Neo4j side. So it'll take a few seconds, but what we're going to try next, which is obviously the proof of the pudding, all that's done, it's not a very large dataset, but we should see now on on our neo4j database that we have a bunch of albums producers bands artists again those of you who have seen neo4j uh new semantics before you know that everything is prefixed uh if we didn't specify and we didn't uh you will assign numeric prefixes and as 0, 1, 2, 3, because in RDF, everything is uniquely identified with URI. So that's, I know it's not the nicest thing, the NS0, we could have assigned a nicer prefix, we didn't do it, but it's not the point of this session. There's plenty of previous ones where you can see how to how to do that but what's important is that now like i said we have um in the graph in if you like so if i click and and want to look at some albums and here's what you see how my uh wife is a bit flaky but anyway let's uh take one of them anyway so let the music play uh that's by barry white but you can see how a node like we said before right the node alex aggregates all the statements all the information that we know about alex so the same here so we have a node representing this album and it collects all the triples that we have ingested so for example the release date the description the name the uri the name, the URI. So all this happens transparently and that's what NeoSemantics takes care of. But now we can start navigating and see that this album is produced and actually performed by Barry White. It has a track that we can keep expanding. I don't know in this case, but there you go. It's also connected redundantly to Barry White, there's another writer and so on and so forth. So we can keep expanding here. So that's it. So migration couldn't be really easier because literally, again, thanks to the entirely generic nature of RDF, we don't really care about the types of entities. There's no notion of tables. I mean, there is. And we could be more precise and we could say, hey, just load that portion of the graph. But if we say, whatever you find in this triple store, just dump it into Neo4j. RDF does the job. I mean, it's already a graph. So it's trimmed as a graph as triples, and then we ingest it on the other side and we have exactly the same information and i think that's that's very convenient i mean now you know i get some comments when we uh import this is a small data set and and when you import larger ones it might not be the most efficient uh format and and and it's absolutely normal, right? That's, I mean, you have to think of RDF. And again, we can look at that. Let's go back to the studio just to look at it for just a second. So this is a triple, right? So for example, Kurt Cobain is of type a songwriter. We know it has a description. So everything is a statement. And each statement is self-describing., is an atom of information on its own. I mean, if you think of it as these are the three members of the band, if we were thinking in tabular terms, you probably would have a table of members or persons or individuals or artists where you would have maybe a column with a name, a column with a description, a column with a label, I mean with the description, a column with the label, I mean, whatever you have in this database. And for each line, you would have specific fields. So a triple is like each of those fields on its own. Basically, you provide the coordinates, right? You provide the X coordinate, which is what's the primary key, the unique identifier that something applies to. The Y would be the column. In this case, I'm providing the name and then actually the value. So each atom of information, each field, if you think of it in a data set, is a triple and you have to provide the X and the Y. Now, while that isn't super generic and it's great for data exchange and it's great for aggregating data i think we i wanted to combine the data that comes from startup with data from other parts i mean i can bring it all and it will link nicely assuming the schemas link nicely and everything but what i was going is uh this is super generic but it's not the most efficient i mean if you stream data like that think of the huge amount of data redundancy that you see so the subject is repeated at every triple the predicate will be repeated every time so and especially these days when we're getting used to formats like arrow that are super efficient i mean all those formats that exchange large volumes of data are based on the idea of of defining uh uh repeatable structures so if you define a data frame with columns with fields then you can compact it a lot more and do a lot more compression rdf is super generic but it will not be the most efficient way of changing data so be patient if you're loading a large a large data set and that happens you know across the board i mean moving data into neo out of neo into triple store because of the nature of the of the format itself so that's that's it's it's what it is basically but i think it it kind of did the job in terms of hey that's that that was really really genuinely easy right so yeah um and. And I would say, you know, let's check that what we have done was what we intended. And basically, the intention of a migration is that at the end of the process, you have the same information on the new store as you have in the original one, right? And we have the tools in NeoJ to prove that. Let's take, for example, and I have, I think, another URI here. Shake it off. I have one of my daughter's favorite songs from Taylor Swift. Yeah, well, everybody gets the earworm for the rest of the day. So let's find that song. And again, we can look it up on the i mean we could continue with nirvana but you know taking this one so we have this uri that uniquely identifies a resource so if i return that that would be the song um identified by um these um uri and as we can see it's connected to well to a number of individuals taylor swift as the right i mean it seems to be co-written by a number of individuals and it's part of an album that where the track is found so um right so i let me again go back to my my cheat sheet here so i have this um well i'm actually gonna write it so we can we can go to to startup and say i saved it here now that i remember so this is a general general triple that would get me everything we know about shake it off right and and again okay because um let's try to dissect this but basically i'm saying give me every triple subject predicate object that where either the subject uh is shake it off or or the object basically what i'm saying is give me all the triples that are uh where shake it off the subject so like but also the ones that for for incoming incoming relationships where shake it off is the is the object so if I if I run this query uh that should return me in this case well just eight triples which is a pretty uh small data set easy to to test and then I go to the Neo4j side it would be something similar to that but in order to get the relationships I would need to do something like that i mean incoming and outgoing let's call this rel for example and i'm going to return r and rel so what i'm returning is uh the all the triples about the the shake it off song itself but also any relationships that come in or out of it so if i do that and i put it on my um rdf endpoint which i get by running post um rdf neo2j which is the num the name of the database uh and cypher so i'm using the cypher endpoint so my query is um this one oh and i have to of course avoid the double quote inside double quote let's make this one single quotes but basically what i've done is i pasted that cipher and i'm going to um run that and that returns the rdf but I like it more in, let's make it format and triples because it's gonna be more similar to the format and triples. Now it looked like subject-predicate object and it's easier to compare, but we see that one, two, three, four, five, six, seven eight the eight triples that we that we're getting on the other side right so if i look at my studio i get exactly the same collection of statements right so for the same query so uh important to see that there's no data loss and that the information that has been imported has been totally preserved and can be regenerated which is another important aspect if you want to like i was saying before right maybe you want to um you've loaded your data into neo4j you want to do your analysis you could even actually let's why not let's run it as a as a background thing here let's try to open this in bloom and uh and yeah let's say you know you have it in neo4j because you want to run some uh some algorithms and and then enrich your graph and eventually bring it back to to um to your triple store or do whatever with it so um so yes that's that's pretty uh i would say you know mission accomplished so we have migrated the data in a very very straightforward manner and we have seen that that the data has been has been preserved so that's test number one and well this if this one's up here let's hang on let's run it neo4j and the password. I don't know if I'm doing this right. Am I supposed to enter it twice? You're right. No, I think you didn't enter it the first time. Now, yeah, should be good now. Okay, so that should be... Yeah, it's good. Don't wanna save that, so that should... Let's do a quick test, because one of the cool things in Bloom is that we can run the algorithms without having to write code, right? And that's something that... Anyway, well, I'll leave that because it might take a few seconds to initialize, and I don't want to diverge, you know, distract the attention from the topic of the day, but we'll get back to that. Good. So that covers the first, you know, the first example, right? So, RDF data in Stardock, we moved out with these export statements, and again, you can be way more granular. I mean, we're it out with these export statements. And again, you can be a way more granular. I mean, we're going with a full graph migration, which is what export does. But of course you can be more selective, more precise and say, hey, I'm going to write a query and specify the portion of the graph that I want to import. Ultimately it's exactly the same. So you can be more granular, but what you would be doing is, like we described in the graph here, you're streaming out RDF, and NeoSemantics is persisting at a Neo4j. So same idea, still valid. Now, if we move to the other database, to the triple store that we're playing with today, which is Ont ontotext that means this has gone to sleep but again they they offer it's not like um like a sandbox environment but it's more kind of a public data set and in particular this one is uh is a demo uh data set on the um i I4.0 knowledge graph. So it's like, it's an ontology. In this case, it's a schema. So kind of a model for industry 4.0. So if we... Ah, okay. I think there was a link here to... Yeah, exactly. I think that's... I can open here a tab for additional information but it's yeah it's an anthology where we define a number of concepts and well that's kind of the visuals of it and it's pre-loaded in in onto text in in graph ap in the form of of triples again and we can run some example query. That's a very, you know, the one that comes by default, getting all the triples here. But again, subject predicate object is the format. Now, if we go back to our notebook, the problem is, like I was mentioning before, is I couldn't find a driver for Python driver for onto text. But again, the idea is pretty simple. It offers a Sparkle endpoint and that's the URL of it. That's the one as you can see that I get from here. And all I have to do is write a full generic query that basically streams out every single triple of the graph. That's what it looks like. OK. So it's basically take every triple, and we can test it here. It's very similar to this one. So it's a construct query as opposed to a select query in Sparkle. It's one that returns triples instead of tabular tabular results so i can set a limit here basically i'm saying take any any triple subject predicate object and stream it out and i can say put a limit of 10 and i can run that and that will i could put the raw response and it shows you know subject predicate object but you can download it i mean you can export it in any any of the rdf serialization formats so that's that's my kind of work around for that type of scenario so i don't have that nice driver but i can say okay run this query and that's an http endpoint so i'm using the request uh a module in python and i'm saying okay uh get which is basically a gate http request on this endpoint which is the the where the database is located um and these are the parameters it's there's a query which is my full generic query that is take anything and bring it on. And I'm specifying the serialization format. Like before I'm using turtle, but could use any other. So basically that returns an export result in the form of the result of a get HTTP request. And what we're doing, and this is literally copy-paste from the previous one, so I'm going to connect to my Neo4j instance, and I have to update that with the new details, because that's in the previous examples of connection details, that's the new URI. I'm going to replace the old one. And I'm connecting with the credentials that appear here. So replacing that. And I'm going to create a database so that we don't overwrite the other one, one that I'm going to call I40 Knowledge Graph. So let's go to the database and i can do create database i40 knowledge graph it will take a few seconds but like this i kind of work in two separate uh environments you see that the database is already available so i can now from my notebook I can it's a like I say it's a copy paste so I'm starting a connection but this time on a different database I force it kg I'm initializing it exactly in the same way I'm creating the constraint and initializing or the default options and then what I do is okay if my request before return at 200 which in http terms means okay success and then what i do is i do exactly the same thing as we did before we call the rdf input inline and the payload that i pass is the text returned by the export that we run before, these export up here. Does it make sense? Yeah. And if you were to do this without having previously loaded anything, or you can just keep the database as Neo4j, but in this case, it makes sense to do a different one, but yeah, it's very similar it's almost uh like you said it's almost the same the same literally uh if we go back here is copy paste except you know even the last line except that the payload instead of getting it from calling the the the export in the driver what i'm doing is uh getting the result of this HTTP request. But the concept is identical. So both operations, one more Pythonic, more driver rather than HTTP request is producing the RDF and I'm consuming it. So if I have made no mistakes, let's give it a go. So this, oh, did I, i don't know if i run this maybe it will still exist for my previous execution but but that's uh that's pretty again it's a small data set so um we should have it um see there you go so that's it yeah so there's something there yeah so there's all the because it's an ontology there's lots of of types lots of labels but again we can run a simple test and i have my cheat sheet here with some uh uris and um if i go to my sparkle end point uh i'm going to look at, for example, one of the concepts, which is this one. Instead of a control account, I'm going to use a select star from here. So basically, I'm returning all the statements, I mean, the predicate and the object about a standard organization. Okay. If I run that and I want to put it in the form of the table, we see that standard organization is a class. Actually, yeah, it's class according to RDFS, class according to OWL. It's a thing. It's the same as a standard organization. Well, you see there's 10 triples here. I'm limiting it to 10. So maybe there's more. Let me remove that limit and see what that returns. But no, it's actually 10 anyway. So yeah, and there's a comment label and some other equivalent classes. So if we try exactly the same query on the NeoPJ side, we should check that the input has been successful. So that would look something like match. I mean, I'm looking up by uh uri which is the quickest way of of doing this but and actually we would get um in this case i believe we were specific about the the outgoing ones because i only put the ones that had these as subjects so if i return r and rel that will return the visuals but again what we're interested in is in the post request on RDF. This time, instead of Neo4j, it's I40KG database. And I'm going to use the Cypher endpoint. And just like before, my Cypher query is this one. And as always, I have to be careful in the browser of the quotes, the way we're using them. And that should be it but just like before because it returns if not total by default I'm going to say the format I want is and that's again entirely because it makes it easier to compare. And we can see one, two, three, four, five, six, seven, eight, nine, ten triples, just like the other case. So we have the thing, we have the class according to OWL and RDF schema, the equivalent classes which were standard organization and same as. So pretty much the same information, well, so pretty much the same same information not pretty much exactly the same information that we get on the other side right so these these 10 10 triples so no data loss again and uh like we saw before entirely possible to completely regenerate the um the rdf as if as itF as it was imported. So in case we wanted to bring it back for whatever reason. So yeah, I think I should probably put a comment in the, because this is the onto text example. Let's make it clear. onto text example. Let's make it clear. But yeah, that's pretty much it. I hope that that was clear and simple enough, because that's, again, one of the requests that we often get from our Neo4j users. How can we do a simple migration? And yeah, it doesn't get much simpler than these in terms of of migrating data from one platform to another and again this this is something that we've shown in the past that doesn't work only a triple store to neo4j we also use it to move data between neo4j and neo4j so sometimes it's a convenient way of moving data from one instance to another a whole graph or a portion of the graph so again keep the idea that as we said at the beginning the rdf is a great model for data exchange very flexible very easy to use but also with the caveat that you know don't expect the best performance because we are literally streaming uh atom by atom of information so take a pause here and and see if we have any questions does that make sense alex did that kind of cover the the the topic clearly what happened today i can't hear you i don't know if it's yeah because i'm i was wondering um that was very clear uh thank you jesus um the question i had was is is stardog and onto text as the two examples you showed, did you pick them because they are the most famous ones? Or is this a particularly easy example? Or is this the procedure you would go for no matter what triple store you use in general? Yeah, yeah. So I think that they're some of the most popular. I mean, I mentioned in the slides, these two plus Neptune. I work locally with RDF4J, which is another, you know, it's more an API for Java API to work with RDF, but also offers kind of a mini triple store. But the, you know, the three are some of the most popular, some of the ones that our users mention more frequently, but the ADA is exactly the same. And it's the great thing is that, you know, they all share the fact that they offer an out of the box RDF because they natively and store the data as RDF they expose it as RDF and that's a format that we can easily consume so yeah probably the most popular but the principle is exactly the same with any RDF store okay good and then there was a question from why what's the advantage and drawback of migrating a triple store into a neo4j i think you mentioned some of those like you want to say like you say streaming atom by atom can be a little bit slow and it's not a sort of high performance i mean if you have i guess would it if you have the same data available as csv would you prefer to go by a via via csv or import or would you still do the rdf route right okay so if i had uh csv data and i i mean i would need to choose i mean because i suppose the question here is what type of grab store you want to use and and that's kind of the the the the the i would say the the question that's been going on forever. And, you know, whatever I say, it's going to be biased. But I think at least I try to be objective and articulate my answer. So I really value RDF as a format for data exchange. And we prove how it's extremely useful to very simply have a graph and reproduce it on a different platform. But it's not. And again, that's my impression. But I think it's backed by the fact that, I don't know, go to Stack Overflow. And you will see how you know there's uh surprisingly or unsurprisingly more questions tagged with cypher than they are with sparkle maybe that is because you know developers ultimately the people who interact with uh with stack overflow are the people who are building software asking questions that's because they they're they're they're facing these type of issues so i believe that uh I believe that when working with data as a graph, I think it's a more natural, closer to developers, closer to data people, the level of abstraction that Neo4j offers, that the property graph offers. That's my intuition and that's what I feel. But again, if you talk to maybe some startup guys and they will say that no way that this is. So, yeah. And then that's looking purely at the model, at what the data looks like and what the experience of interacting with the data from the point of view of your query language and the model. Then beyond that, of course, there's a whole host of questions that you have to ask yourself in terms of what's the underlying data store. Because things like what type of workloads, what type of high availability you want, what type of, do you want an open source platform? I mean, there's so many questions that you need to ask and that are not model dependent but more platform dependent then you know again i i would have plenty of opinions that i would be able to to justify but i think you know whatever i say my people might take it with uh you know with a bit of a pinch of salt because they might say hey you work for neo that's that's true but we're doing these're doing these webcasts where we try to kind of be objective about the two things. So I hope what I say gives you some hints and some clues and some ideas of the kind of things that people look at when working with graph data. Yeah. Yeah. I think that makes a lot of sense. I mean, in general, we try always to say um you know use the data use the tool that fits your data and no matter what you do think about you know what what kind of data model do you want to go with but what also what kind of you know challenges do you have with your data what what what's the ultimate outcome you you want to achieve by by using data and what what in terms of data combined with a platform and if if you you um you realize that maybe you want like you said right you you want to do a little bit more with your with your rdf type data you want to do some algorithms some some uh some graph algorithms in particular you want to find out a little bit more about the the the connections beyond the first degree which is i think super easy visible in in rdf but if you go a little bit further two three four maybe more steps it can become more complicated and then it can become a little bit more overloaded you know might make sense to at least look at it in in a in a property graph way and uh and see if that helps you um with with uh with that approach absolutely yeah integrations i mean what kind of ecosystem you work with where are you going to get your data from what kind of which which are the options i mean we've seen today that you know if you want to get data out of one of these platforms, you have different experiences. If you're a Python writer, maybe you prefer, you know, startup because they offer you these, these driver then, then onto things like that are important. You know, how you consuming your data, what kind of visualization, what kind of integration, like you say, with, with analytics, with machine learning you know all these elements are important and you know it's it's it's it's a big decision and you have to do your due diligence when you're starting a project and and try to understand which you know uh benefits one has uh compared to the other i mean today we're just focusing on the mechanics of moving data across and showing how you know the rdf gives us you know a really really straightforward way of doing it. But all those questions are relevant and you have to, of course, answer them before you choose the type of platform that you want to use to manipulate the data. Yeah, that's always true. Always think about that and then do some testing, do some evaluation, obviously, and see if that works. And we showed the free tools, at least from our side, you can give it a go and see if that is something you see is maybe worth further exploration. Sorry. There's another question from why, and I don't know if I missed something here, but is there a competitive modeling tool for graph db such as neo4j of protege i think we talked about protege before but i don't know if i mean modeling tool does is it is it like something like arrow or yeah it could be it could be i mean that's yeah we we we often use Arrow as a modeling tool. But again, we've shown, like you said, Alex, I think only a couple of, I don't remember how many, but we've shown how you can perfectly use Protégé as your modeling environment and create your ontologies. And then the model that you create with Protégé, I think in one of the episodes, we use it as the target model to ingest data in the graph and to automatically build the pipeline. I mean, we've also used it to, I think the last time was to create a streamlit application that would read the ontology. So again, these tools, Protégé, BullParty is another one, they produce models in RDF and we've shown how easy, the moment it's RDF, we can bring it into, you can create programmatic, how would I say, actions on it, whether that's to ingest data, whether that's to run inferences, whether that's to run validations. So yes, there's arrows as an option. There's the data modeler as well, which is kind of a variant of arrows as well, right? That's the data modeling tool that's part of the workbench. Exactly, yeah, that's a version of it. But I would say that, you know, use the one that fits your your need and and uh and yeah and you will find possibilities for for interoperability again in some of the episodes in this series yeah and then there is a question i think two two questions um more or less the same um uh same way um how how is it can we can you explain how how the neosemantics knows what to do with with what basically so why how does it know that something becomes a node and something becomes a relationship sure that's uh that's pretty straightforward. So it's describing the documentation, but I can put a pointer. It's basically three rules. I mean, triples come in two shapes, right? So a triple can have a, and maybe I can start sharing my screen again, just to show an example. So let me share it again and maybe bring the Stardog. So this example we're showing. Can we see my screen? Am I sharing it? Yep. Yes. Cool. So this is a collection of triples about the song Shake It Off that we mentioned before, right? So as you can see, there's two types of triples. I mean, let's call it types. But these three at the bottom is what we call literal triples, right? And they're literal because the object, the third component of a subject predicate object is an actual, we call it literal value in ARIA, which is a string, is a number, is actually a property value in a way. So these literal triples become a part of the node structure. So all the ones that contain that, that share the same subject, for example, shake it off, will become a node. And all these will become attributes of the node. So literal triples become part of the internal structure of the node. And then object triples, which are the ones that have in the object another resource. These are relationships. This is a connection to something else. So, for example, we're saying that ShakeItOff has a writer, which is Max Martin or Taylor Swift or this other guy. And so this is another thing, another resource that will have itself. We can actually check it. So we can take this Max Martin and say, what do we know about Max Martin? And we can keep navigating the graph. And so if we do that and run the query, we will get all the triples about Max Martin, right? So there's a bunch of them. And again, we'll see that some of them are incoming, some of them are outgoing. But in this case, we can see here at the bottom, let me hide that, that Maxin uh has a description has a name so some some literal properties as before which would become part of the internal structure of the node and some relationships that will connect max martin to to other things so these are the two rules and then there's a third one where uh we have a type nodes and type nodes in rdf it's a keyword rdf type which basically gives you the the class the category that that this resource belongs to and this in this case this is a songwriter and a producer so these types can be represented in neo4j in two different ways so you can use uh what we call in neo4j a label which is a tag that you associate to the to the node which is the the option that we use by default, or you can create a separate node and link them together through a type relationship. But these are just the three rules that we apply. And that's what semantics does, under the hood, I mean, not under the hood, but internally as you ingest, which is the kind of standard natural way of of of uh of importing uh um data let me uh stop sharing yeah so that that's it i believe it's it's described i can i can send you a uh a link to a uh post where i describe that uh i'll maybe put it in the in the in the chat but that's that's it the logic is is pretty simple yeah yeah that's good data but i'll i'll shall i put it on the i'll put it on the chat for you alex and then you can yeah yeah put it in the chat and then i'll put it and i get perfect thank you so that's um that's basically a longer description of what i just um described yeah yeah i'll put it also in the video description so um so people felt they'll be able to find it yeah and and and i i i appreciate that you know we we've just focused on the migration part and we kind of assumed that people were you know familiar with this but uh yeah again there's just plenty of previous episodes that we can yeah look at for more details absolutely um yeah um i think that's that's it for today. Thank you for watching. Thank you, Jesus, for showing this. I think this is very good. I mean, if any episode seems to me like the best one to try out at home, basically, is going to be this one, because it just seems to be three lines of code and then you're good to go. So it's very friendly and makes it very easy to try and see. So if you are on the fence of giving it a spin, see what Neo4j can do with RDF type data. And if that works for you, then hopefully we showed you how in this episode. Yeah, in terms of upcoming sessions, tomorrow at the same time, so 5 p.m. Central European time, I have Maria De Maro as a guest, and we talk about linguistics and graphs. So she wrote a paper on that. So how to analyze conflict dialogue, argumentation-based dialogue in NLP and graph. So that should be a very interesting session tomorrow. And yeah, I'm looking forward to seeing you then. Hopefully you can make the time. And if not, obviously all the episodes will be available as recording on YouTube. So just go to our YouTube channel, and you find all the playlists of Going Meta, also the other channels, the stuff we're doing. So just check it out if you want to. And if you like it, then please give it a thumbs up or a better yet, or both, actually. Subscribe to the channel or follow the channel that would be nice um and yeah um thank you jesus again for presenting for for giving the demo and for um for showing how easy it is and obviously yeah for taking the time today i'll see you soon perfect no my my pleasure see you soon and i'll make sure that all the content for 18 and 17 is up with the rest in GitHub. So thank you all. Perfect. Thank you very much. And yeah, have a good rest of your day. Bye-bye.
Going Meta - Ep 18: Easy Full-Graph Migrations from Triple Stores to Neo4j
Episode 18 of Going Meta - a series on graphs, semantics and knowledge

Jesús Barrasa: https://twitter.com/BarrasaDV
Repository: https://github.com/jbarrasa/goingmeta
Previous episodes: https://neo4j.com/video/going-meta-a-series-on-graphs-semantics-and-knowledge/
Neo4j Sandbox: https://dev.neo4j.com/try
neosemantics: https://neo4j.com/labs/neosemantics/
Arrows: https://arrows.app
Blog on Neosemantics: https://jbarrasa.com/2016/06/07/importing-rdf-data-into-neo4j/

Check out https://community.neo4j.com/ for questions and discussions around Neo4j

#neo4j #graphdatabase #semantic #knowledgegraph #rdf #semantics #migration