 hello good morning good evening good afternoon why is this view so weird hi alex how are you yeah it's a bit different now it looks more as we're used to yeah this is the other one was a little bit more zoomed in than i wanted to be good stuff how are you how are you alex thanks uh accommodating I know I don't make your life easy that of of attendees because you know we were supposed to be the the monthly webcast on the first Tuesday of the month and we've tried probably all days in the week already right yeah we we kept I think we kept it relatively consistent but there is another there's a one or two outliers uh where we just couldn't couldn't manage this is one of those those weeks um so um obviously first used every month uh the third was um a public holiday in Germany was me being on the road in London, you not being there as well. So we couldn't really make it. So we tried to... Yeah, I'm sure people understand. I mean, the great thing is that we've kept delivering month after month and that's the important thing to me. And I'm really, really happy that we're on on 21st already so me too and uh and and speaking of of of that i think uh if if you haven't uh looked uh at the the recap blog um that that we published and please uh please have a look at that it's uh out on our um the neo4j developer blog let me you should have the link here let me put that in why is this so weird sometimes medium makes completely ridiculous links oh here it is. This is much better. Yeah, I think that was a great one in the end. So we had a substantial amount of content and material over the months and over the years. And it kind of made sense to kind of put a bit of structure in. And you've done a great job at putting it in written, because I'm sure that will help people find their way in the super rich catalog of content that we've created. Exactly. And we had defined by, admittedly, by coincidence, they came to be basically these four themes out of the background of content we had during these 20 episodes of Going Meta and a couple of them identified sort of themselves, which made sense to include them and to spread it out a little bit. So if any topic is not quite interesting to you but generally the topic of semantics ontologies taxonomies knowledge graphs all that if that is something you're generally interested in but you think okay that episode right now maybe not so much or a different one then go go back to that blog post and and and see that there is a couple of themes that were that were identified a couple of episodes and you can find the one that probably makes most sense for you then or which you think is more interesting, could be more interesting for your case. Good. And I'm sure you want to talk for at least a minute about something that's coming up, right? Oh, yes. Yeah. It's notes, obviously. So happening in, well, yesterday, I think, to the day, or, you know, it depends on what times you're in. But on the 26th, so in three weeks, more or less, is when Nodes is happening this year. Let me put a link in chat as well. So it's really going to be a fun and interesting event and very exciting on all kinds of fronts. So if you haven't registered yet, go to the link I posted here or go to neo4j.com slash notes and you find everything there. The agenda is now live since a couple of days. So you can check that out. If the trendy hot topic of vector, gen AI and everything, chatbot, chat GPT, all this stuff, open AI, you name it. If that is interesting to you, there is like, I think I wrote an email with a couple of highlights, five or six sessions on LLM's OpenAI, Gen AI Vector last week. But I had to skip, like, I think probably another easily five to ten. So if that's your game, then definitely notes will be interesting to you. If you think, oh, my God, leave me alone with all of that. We have other things as well. So we talk about all kinds of things on, visualization is a big part, you know, everything, app development, smarter, better app development with graphs is a big one. So I think you'll find something interesting there and it's happening across all the time zones. So depending or independent rather, independent on where you're based, you have sessions live in your time zone and then the rest you can obviously watch on demand. So that's the beauty of it all. So that's going to be available afterwards on our youtube channel that's great yeah everyone join notes and because it will be it's going to be it's going to be great for sure yeah so um yeah so i guess we're on to today's topic, which is- Today's topic, yeah. Fitting to what I said earlier. Surprise, semantics in the title. Yeah, and as usual, I'm going to try and share my screen, put the video to the side and do a short introduction as we always do. Let me know, Alexlex when you can uh see my screen i should be sharing my whole desktop but i'll go into slideshow mode so that is you know just just you know the previous title and like i say when i say that i don't make alex's life easy i was saying hey what about these the difference between what it's like we ended up agreeing yeah and and that's that's exactly the reason why why we went with this topic today right so we we hear loads of of talk about semantic search and i think there's very two very distinguishable and very clear types of semantic search and that's how we uh agree with you alex suggested a much more reasonable title that was much clearer. And the two approaches are the one based on vectors, based on embeddings, and the one based on graphs and path exploration, which is something that we've talked about in previous episodes. But that's the subject of today. And of course, in the backdrop, of course, is the fact that a few weeks ago, a couple of months now, right, we released the vector index and enabled vector search capabilities in Neo4j. And now we can combine the two and explore the two together. And that's exactly what we're going to do today. So that's the history and the rationale for today's session. And well, like I mentioned, I'll spend three minutes giving the context and we'll go ahead and get our hands dirty with Neo4j on the sandbox. So where do we start? If we want to understand semantic search we need to understand semantic similarity first and if we want to understand semantic similarity the two types of semantic similarity that exist we need to understand how data is uh represented in general in in in ai right and in ai this this is kind of two uh big spaces right The one that's called the area that's called knowledge representation that uses what's called a symbolic representation of information, the one you see to the left where graphs fit. And then there's the more machine learning focused where information tends to be represented in the form of vectors, right? So something, I mean, knowledge like the one you see in that sentence, like apples and oranges are both fruits, can be captured in many different ways. So you have seen how we have represented that in a taxonomy, in an ontology, in a graph, basically. So you say that an apple is a type of fruit and an orange is also a type of fruit. And in the context there, you see that there's a relatively short distance between these two concepts. So there's some kind of proximity between them, some similarities, some... And that's what we mean by graph-based. I mean, that's of course the biggest simplification, but essentially the graph provides the context that explains the semantic similarity between two concepts, like apple and orange. And again, semantic is a general term to refer to the fact that we don't do a string similarity. So you can think of dog is close syntactically or from a string point of view to doc as the O-C-K, but the meanings are completely unrelated. These would be one type of similarity, and there's search that enables these type of inaccuracies, which is fine when you want to find words that are not explicitly, or not explicitly, but not exactly spelled in a particular way. But we're moving from that type of search and that type of similarity to one that's more based on the meaning. And the meaning can be captured in a graph, and the graph provides the context. So that's one type of search. That's one type of similarity. One type of, one approach to AI, which is knowledge representation. Then on the other side, we're not as explicit. I mean, the bit to the left, someone has to go and say, I know that an apple and an orange are both subcategories or types of fruits. On the right-hand side, what we have is the result of probably exploring and learning over a large amount of examples. So the sentence that you have there might be a fragment of a chapter in a book, or an article, or a blog post. And that's the approach that we follow in the machine learning space. So we learn from loads of examples, and we find that consistently, apple tends to appear in text close to orange and close to fruit. So we're capturing that in some way. So it's much less explicit. So what you see to the right is the way we tend to capture this knowledge. And it's in the form of a numeric sequence, right? Well, it's a vector, right? So we represent, and we won't go into how these embeddings are calculated, but there's a numeric representation of an apple, an orange, and a fruit. And this numeric representation has very particular characteristics, and is that it reflects the fact that there is a short distance between apple and orange. And that's completely independent on how you type it. Because if you do, let's say, if you had the embedding for, let's say, a plum or pear, I mean, it will probably close because these are topics that tend to appear close together or related to each other in the corpus, in the data that's used for training. So both, as you can see, capture some meaning, some semantics, but in very, very different ways. Let me take a pause here. Am I making sense, Alex? Is that clear enough? It's clear. It's always a little bit difficult, I think, if you're not into the topic of vectors. I think it's a very very uh pointed pointed topic no pun intended here but um but it is um it if you if you look at it like this it it can be very complicated and can be very confusing because you you see okay i see 0.2435 comma 3.7 and so on. And I don't know what that means because obviously I'm a human and not a machine and I cannot read that in that way. I think when it first explained to me, somebody said, okay, think about the RGB color spectrum. You have a three-dimensional space and you think you want to point out a point in that three-dimensional space, and you think you want to point out a point in that three-dimensional space, and that point is a color. And if you have 115, 50, and 64, that is a specific color. And that color is in that three-dimensional space somewhere. And then you can look at that. And that makes it a little bit more understandable. That makes it a little bit more clearer. Also with depicting it as arrows through a space in a field, that helps a lot, but it is a complicated thing to understand. But once you think, okay, you don't necessarily, you have to don't worry about as much, maybe about the individual values, but more about, okay what what is it what is this about what is the point of this this uh this vector then i think it makes it makes more sense and you understand it a little bit better absolutely and and the whole point is that the the representation to the right we're not i mean it doesn't even make sense to try and make sense of it because you know what's uh the powerful thing is that this distance and there are some functions to compute that distance you know can reflect the fact that these are related related concepts and again why you know there's obviously a very good reason to go with the one in the right and and uh you probably have figured that out the one to the left is human curated we have to go i mean it can be discovered from the data but someone has to go and say hey an apple is a type of fruit and that that requires well some domain understanding in that simple example but think of your enterprise right so it's human curated it's controlled is is and that's these are very important characteristics when we compare them but the one to the right it can be completely automated so you can just give a huge amount of data and it will be it will learn these vectors for you so automation is a huge factor there because well if you think of of the the a session with chat gpt you can ask about anything in the world. You cannot even think of capturing everything in a graph. So you will pick fragments of it, the ones that you care about, the ones that your enterprise cares about. But, of course, with huge amount of resources, you can fully automate the process of learning these type of representations. And they're very powerful. And the great thing is that we can have the two and combine them together. But I think it's important to understand the two and that's what you know the example that we're going to be using today. Good so well so this is the starting point right so we have two different ways of representing capturing knowledge, and depending on which one we use, that will open different possibilities for comparing similarity and applying search. So this is material that we've shared in the past. For the first approach, we will start with, let's say, some unstructured documents. Again, we're going to use a collection of articles that we've used in the past. There's a process that will extract entities out of it and make them explicit, and then we'll enrich that with an ontology, with a taxonomy, with a classification, something that provides additional context, something that puts in context the apple with the orange, or a topic with another because they have a common ancestor, or because they can can be classified in ontology. Because when we have these in a graph, then we can find the similarity between two, semantic similarity between two documents based on the fact that there is a path that connects the two across the graph. Advantages of this, I mean it takes longer to build, it requires expertise, but it gives you control over the responses that you get. So you can explain that this is similar to that because there is this path across the graph. And we touched on a bunch of standard similarity metrics. There's a whole episode on this. I can't remember the number, but we'll add the reference in later on. So this can be done, and it's a very powerful way of computing semantic similarity. Now, in the vector space, what happens is, you know, everything, every element in the world is represented as a vector. And we can do pick two and tell what's the distance between them, what's the semantics, but these are implicit relationships. These are, we have less control over, you know, I kind of understand that a cat is close to a dog, but it's not that I can really explain it. I cannot provide the rationale, the reasoning that the model is implemented, because it's just capturing this numerical representation. In the end, it's purely a distance function. It's like a mathematical distance, as you can think in a 2D or 3D space, except that it's a much more high dimensional space. That's the idea. These are the two approaches. And I'm going to try and show these in an example. And like I said, the first part, we're all familiar with it. If J is a graph database, we can capture the relationship, build the taxonomies, build the context around elements in the graph what's relatively new is the vector index and and this is just the screen capture of the announcement you have the links down there so there's a blog post with the announcement but probably more interestingly uh for the attendees here is the is the link to the manual as well where you can see how to create a vector index how to populateulate it, and how to create. It's pretty straightforward. I mean, we will add properties to the nodes that are vectors, that are long vectors, numeric vectors, and we will index them using the vector index. Once we have it there, we can do pretty performance searches uh based on based on that so that's that's exactly what we're what we're going to be doing so with that let me stop uh with the presentation and as always i've already started my sandbox here uh fresh one it's already created and i've been extremely diligent this time and as you can see there's already all the content there on the on the github repo so you will see that we have all the all the elements so i'll probably go a bit quicker on the construction of the graph and population of the indexes and all that and then focus more on on comparing the two types of of semantics similarity and and semantic search so yeah in the and if you watch this and have any questions then um please please uh use use the chat while you can uh and i'll i'll have a eye uh or two uh edit and uh it will read out your questions absolutely so uh yeah so you the decipher if you want to code along, but then more importantly all the resources are here. So we're going to be using ontologies that are already in this folder and the data folder contains the data and even the embeddings that have been created. But let me take you step by step. So like I said, we have a completely empty database. And the first thing we're going to do is import a collection of articles, unstructured data, right? And you will find that in the data directory. So if you go there, that's a collection of technical articles in the form of a CSV file. So you'll have the URI, the body, the title, and a few extra fields. So that's pretty straightforward. I won't spend too much time on it. A very quick load CSV that will just create a single node. So that's what it looks like. So load CSV of the file, and it creates articles and populates the title, the body, and the timestamp of the articles. So we have a bunch of them, small set, and we can see that if we click on any of them, we have all the details here, right? The time, the title, the URI, and the body, which is the full text of the article, if they happen to be, you know, referenceable. So if you click on it, you will get of the article if they happen to be, you know, referenceable. So, if you click on it, you will get to the article itself, which is pretty handy. But yeah, so we have our articles. So, the first layer is there. And now what we're going to do is add some context. And again, I'm going to run this quick, but we have used in the past something like Wikidata for that. In Wikidata, these are technical articles that talk about programming languages, that talk about databases, that talk about systems. So, I thought, well, Wikidata is a super rich knowledge graph with a definition of loads and loads of categories that form a great taxonomy. And I'm going to use that. And again, this is in your resources folder. If you go to the ontology directory, you will see that there's a DBpedia extract, and you have the query on how to produce it. So if you're curious, you can grab the Sparkle, run it on the Wikidata Sparkle endpoint, and that produces an export that I've represented in SCoS. SCoS is a language to create knowledge organizations, taxonomies, basically. And that's sufficient for me. As you know, we can import SCoS as one of the RDF-based languages, and this type of thing we handle with Neo-Semantics. So the next thing that I'm going to do is, as usual, create the constraint so that Neo-Semantics doesn't complain for a change. Alex, instead of the error thrown I created up front. So there you go. So we have an index for the URIs so that it takes care of the linkage based on the URIs. The next thing is I will go with all the defaults in the configuration. Again, very briefly, in NeoSemantics you can make a number of choices in terms of how you want your RDF imported. In this case, I'll go with the basics and I'll say that I don't care too much about preserving the vocabularies but um i will go ahead and input um oops sorry i will input the ontology with a single line instruction as course because that's the vocabulary that we're using import fetch we pass the url of the file that i just showed the remember that it's the raw version not this one because these are the human readable and it's html you have to go with the raw one so if you pass that to Neo4j and you indicate that it's total serialization, in a few seconds, you should get it imported into your graph. So now we have the taxonomy formed by 11,000 roughly entities linked to each other, and we can see what that looks like. So that's a small fragment, but let's zoom here. So we have, make it a bit bigger. It's using the Wikidata IDs, which are not very user-friendly. So let's go with the preferred label, which is probably much nicer. There you go. And we can make it a bit bigger to make it readable so you see there's the notion of a programming language and that has multiple subcategories subclasses so again this is the kind of thing that you can tweak in the configuration by default neo semantics will create a a subclass of relationship that we call SCO to make it compact. But if you're more comfortable with all the other name, you can configure that too. And the elements in the taxonomy are labeled as classes. So there you go. So you can keep expanding and these will form a large network. Ideally, it should be close to a tree, but not necessarily, but it's a rich classification of the systems, databases, programming languages space. So, we have the two parts. We have the articles, we have the taxonomy, the classification. There's a little step of data cleaning that we have to do which is uh um if you think of a tree uh a classification right at the taxonomy uh you don't expect that to find things like uh shortcuts like uh for example you have um um as girdles would like this. Yeah, yeah. But the reason, I mean, as you know, Wikidata is a collaborative repository, collaborative knowledge graph. So it's not always perfect, right? It's an evolution. And sometimes you find, I mean, it's not bad. I mean, it's just redundant information, right? So you have the fact that, I don't know, since we're going with oranges and apples, I mean, if you have a blood orange, these red oranges is a subcategory of orange, which is a subcategory of fruit. If I, you know, that normally forms a sequence of subclass of relationships. But, you know, if I could redundantly say that this blood orange, the red orange, is also a subcategory of fruit, which is something that I can't unill because this subclass of is transitive. So that's exactly what I'm doing with this. I'm saying, well, if you find a shortcut of this type, I don't really need it. I mean, it doesn't hurt, and I could keep it there. But I'm a bit of a perfectionist here, and I like to have my tree clean. It's not necessarily a tree, but I don't need these redundant relationships. And I remove a few thousand of them. Still keeping, no, I remove 500 of them. Right? So there you go. So now we have the two components. We have the context, the graph, and we have the articles. Now what we need to do is extract the entities that are mentioned in this natural language text and make them explicit so that they connect with my graph and they form this contextualized representation that we were talking about before. Now because you know it would have been a super long episode I'm going to save you from having to do that and I've already but we you know know, remember Alex, we did that in episode two, I believe. So if you want all the details, you can go there and have a look at that. What we're going to do today is go to the data directory, and you will see that there's extracted entities files. So what I've done is I've done it for you already. So for each of the articles, we've run the process of, effectively what we use, if you remember, is using the natural language processing wrappers that are in Neo4j. So, and by this, you can use your preferred, I mean, these are wrappers for the three main cloud, you know, natural language services of the three main cloud providers. I believe we use the Google Cloud one so basically with uh simple instructions in um in in cypher in apoc you can invoke you can take that string invoke this uh natural language processing entity extraction procedure from google and it will return a collection of entities that we will then represent as nodes so in order to save us the time of having to do that, I'm giving you already the list of entities that were extracted. And it's very interesting that the Google service, in particular, brings them back annotated with the Wikidata Wikipedia URIs, which is great, again, to simplify the process of linking. Because as you know, because i've mentioned it several times the linkage i mean the entity extraction is one thing but the entity linking is is not an easy one right sometimes you have to apply heuristics sometimes you have to approximate but in this case because we have them annotated with the uris then they just uh stitch like like like glue and that's that's perfect so here what I'm doing. I'm taking this file where I have my extracted entities. Basically what I have as you have seen is for this article, identify with the URI, these are the, in this case there's two entities. It's about RethinkDB and about NoSQL databases. For the second one we we have another two pair of entities. What I'm doing is scanning the CSV file and making those nodes. I will have to match the article on the URI. I will match the class that will exist, hopefully already. If it doesn't, we'll lose that. But in this case, I'm just picking the ones that link nicely to the ontology that I have. If I find the match, then I'll link them with that refers to and again that's my choice of naming but you're free to to call it the way you prefer so this should be a relatively quick one and what we have now as you can see is our old articles now hopefully enriched with let me make this a bit bigger how do we we do that? Here, we can zoom it in. And we have the note representing the article, which is the blue one. We have a number of entities that have been extracted through this process. So this article, I don't know if it's readable, but it's about Python, Zookeeper, SQL, Java, and some others. And these are already connected to our taxonomy. There you go. So that's how we've kind of closed the gap between the two. Make sense? So that's the kind of a super fast step-by-step description of how this works, but you can look more into the details of the actual entity extraction. So, okay, we have all the elements we need now to start using these to calculate semantic similarity, at least graph-based. But we have the other part, and the other part is the vector-based. So what we're going to do is now we have our articles nicely connected to entities and classified in a taxonomy or contextualized in a graph. But at the same time, I'm curious to see what happens if I take the body of the article that I use for entity extract and make explicit here. I'm going to take that and I'm going to embed it. I'm going to vectorize it. And again, here, I'll simplify it. I think I'll share the code because it's pretty straightforward. In this case, I've used the OpenAI API where you pass a text and it returns an embedding. So I have it in a Python notebook. Again, I've done this in advance and there you go. So the kind of thing that you get is for each article, you get an embedding. So I've done that so that we don't have to to run it live and and save some time and now just like the other uh data elements we have it here so we have the um article embeddings and this is in json format no particular reason i just felt like you know creating json this time and similar structure so for each each of the articles, I have the article URI and I have the vector resulting of embedding of the body of the article, the text. So what I'm going to do is parse this file and enrich my graph with a new property that will represent the body of the article but in the form of a vector. Make sense? Yeah, very cool. But because i want it indexed i'm going to first create my my index and that's a very simple instruction that you will find it all in the link that i shared before to the documentation we're creating a vector index i'm giving it to the description oh fantastic thanks for that So I'll give it a name. That's going to be my article embeddings index. And it's based on nodes of type article. So I'm going to build it out of nodes of type article and a property that will be called embedding. It doesn't exist yet, but I'm going to populate it as I said. So this is the similarity that I'm using cosine in this case, and the size of the vector. So pretty straightforward, just like any other index, it's already built in. And the next thing is, okay, let's enrich this article description with the vector representation that I'm going to get from my JSON file that I just showed. So again, remember that there was a pair article id vector so all I have to do actually let me show you what that looks like because you know it was a it was a json file and if you are familiar with the apoc load json it's just another one of these super useful functions in apoc that will let you point at a source of json and and just parse it right so if i uh return in this case it's just a list so if i return value dot id i think was the first element so there was value dot id and value dot vector these are the two value dot vector. Let's show the first one. I don't need to show more. That should give us an idea of what we have. There you go. So we have, it's been truncated, but we have the URI of the article and the vector. So that's exactly what I'm going to store, right? So instead of putting it, printing it out, what I'm going to do is I'm going to look up the article by ID. Right? I'll find the article and then I'll add a new property called embedding where I will put it out. What I'm going to do is I'm going to look up the article by ID. Right? I'll find the article and then I'll add a new property called embedding where I will put the vector. And these will automatically be indexed in our vector index because I've already created that and indicated that that's the property that's meant to be indexed. So if I do that, again, it's a pretty quick process. In this case, again, you will have to generate the embeddings. You could do it all in one go, and here you have to pick your preferred... I mean, I've used OpenAI, but you can use Vertex. You can use your preferred model, and you could perfectly do it on the fly, right? As you write to the graph or in batches, you can scan the notes, the articles in the Neo4j graph, take the text of the property, call the embedding function, and store directly. I've done it in these intermediate steps so that you can use it, and it's already done there for you. But I'll share the code on how to do that with OpenAI because that might be useful for people there. Perfect. So now we have it all, right? So we have the articles contextualized with our graph and also embedded and indexed in our vector index. So what are we going to do? We're going to start computing similarity. Let's get started. So I'm going to take two articles and I look them up by a title. I can look them up by a URI, but I thought it would be more readable like this. So I'm looking for an article that's called Performance Testing Neo4j Database Using Bolt and JMeter, and I'm taking another one, which is Couchbase GeoSearch with ASP.NET Core. So these are two articles that have a property. Each one of them have a property called embedding. So we can use the function in GDS that's cosine similarity. That takes two vectors and returns literally the distance between them, the similarity between us, distance and similarity, what inverse from each other. But it says it's as simple as this. So this will tell me, okay, the distance between these two similarities, sorry, between these two is 0.74. And that has a meaning. I mean, that has a meaning in context, right? If you put it, you know, compared to others, you will be able to tell this is more or less similar than another one. But that's what you get. And that's what we were saying before. It's great. It's super powerful. but it will not give us the explanation of why these two articles are similar or how they are similar. So we have to trust the accuracy of the model that we have used to embed. And again, I've used here a general purpose model, like OpenAI, I can't remember which one of the models was it, but you could perfectly use a model that's trained on this type of data and maybe get more accurate embedding. So this is, I'm not saying that this is the perfect solution, but I want people to get a clear understanding of how that works. So we have a way now of given to articles, take the vectors and say, that's how similar they are. And that's super powerful because ultimately what we want to do is say, because you're reading these, you might want to read this other one, or I'm interested in things that are similar to these. So that's a vector search. And as you can see, what I'm doing is just using the, not search in this case, it's just vector similarity. We'll get to search in a minute. And as you can see, what I'm doing is just using the, not search in this case, it's just vector similarity. We'll get to search in a minute. What would it be like in the graph-based approach? So you can do a number of things. So as you can remember here, we want to follow explicit paths and understand how these things are connected. So I take the same two articles and now I say well articles are connected to my classification through the refers to relationship so from my article I can get to some concepts some semantic descriptions that are connected hopefully through my context my taxonomy my ontology so i could explore that graph so here these are just uh possibilities and what i'm doing is using the path similarity implementation that we also talked about in in the in one of the previous episodes right the one that i uh mentioned here path similarity is one of the simplest ones. 16. Excellent, thank you. You got my back there. So essentially, you will find how long the path connecting the two elements in a taxonomy is, and based on that, determine roughly how similar or how distant they are. So that's what this function does. So we pass the two uh uh concepts and uh well if you can remember there was an option a configuration option when i mean we cannot assume that uh the taxonomy the the context is a perfect tree and all all the elements have a common ancestor like a thing like a root element maybe that root element. Maybe that is the case, maybe that is not. So the implementation, the definition of this algorithm, this similarity metric, says that when you can, by default, assume that for the root elements of the multiple taxonomies that you might find, you can assume that there's a virtual, a simulated common root. But in this case, I don't want that to be the case. If there's not an explicit path, don't make up this simulated route. So, this is a configuration option. And what I'm doing, if I run this, is I'm going to get, well, first a similarity metric, but then more interestingly is the explanation. And you probably saw in the second part that my path similarity not only gives me the metric based on the formula that you have seen here, which is this calculation, but I can also compute the path and return it because that is the explanation that I'm after. So I know that these are similar points too. and again, you have to put it in context. It's I can't remember this one if it was zero to one. But again, the idea is that these two articles are similar because they happen to one of them refer to Neo4j, the other one to Couchbase, that if you follow in the taxonomy, Neo4j is a graph database, which is a database management system. And on the other side, Neapuche is a graph database, which is a database management system. And on the other side, Couchbase is a document-oriented database, which is a data management system. So there you go. You have a path, a common ancestor there, which explains why these two are similar. And that's the curated aspect. So we are not relying on whatever data has been used to train the large language model that we have used to embed our data we know exactly the rationale the reasoning behind the similarity between these two concepts to these two articles we have a more visual representation of it so you probably remember from chapter 16 that there's a variant of that method, path similarity path, that returns not only the value, the metric, but it returns the actual path connecting the elements. And that's probably a much more visual way of understanding. And you see that Couchbase, this article, which I probably should make a little bit bigger so that we capture a little bit more of the title. This article refers to couch-based server, which is like we saw a document, follow the arrows, there you go. So that's the similarity path, and that's the explainability angle, which is extremely important, and that's something that we normally miss from from the um um vector base you know more uh sub symbolic as we call it where we don't uh make explicit the the the knowledge but rather learn it in the form of a vector so i'll take a pause here making sense so far yeah sorry yeah it it makes a lot of sense it's uh it's i think it's a great way of explaining these two approaches towards similarities or towards finding similarities in your data. And no matter what you have, you can use either of those two systems to find it out and to discover if there are similarities. But if you're happy with not quite knowing what came about, then... Exactly, because I'm sure you've heard about the problem of hallucinations and these kinds of degenerative AI models. Typically, by definition, they're generative, right? So sometimes you want them to be creative, right? And that's great. But sometimes you want them to be accurate and not only accurate, but explainable. Or you want to have control over the information that's used to provide an answer. And that's why approaches like retrieval augmented generation are being so popular, right? When you pass the question to your LLM, but at the same time, you pass the context required to answer that question. But, you know, without going into this, we might cover that in future episodes. Here, I really, really wanted people, because it's important to understand what we mean. And when we say semantic search, yes, semantic search is any type of search or similarity that goes beyond the purely syntactic or string-based one. So here we're talking about meaning. And I can tell you that this article, whether it mentions couch-based or not, I don't really care. I know that these two are related because there is a common topic. They talk about database management systems, even though that's not explicitly mentioned. So semantic, sorry, not semantic, but textual similarity would never pick that up. So that's semantic search, but one is very different from the other. And again, I'm not saying that one is better than the other. The answer is probably better together, but this is the foundation and I think the explanation of it. And yeah, and I guess that, you know, I'm conscious of time and I don't know if there's going to be questions, but the next step would be here, we've taken two articles and say, well, what's the similarity, how similar or how different they are. But normally what we do is effectively do the search and that's where the vector index comes into play. So what i have normally is okay i um i have um let's say just one of the articles and in this case i'm looking it up by uri but i could do exactly the same as before this is the one about testing of j with j meter so what i'm doing here is okay i have one article and and I have the content embedded and I want to find which are the other articles in its proximity. And this is semantic search. I'm not giving you two articles and telling me how close they are. I'm telling you, based on the index, and that's where the new feature is super powerful, tell me which are the closest one. And that's what I'm doing here. I'm calling my vector index and I'm saying, use the article embeddings, which are the closest one. And that's what I'm doing here. I'm calling my vector index, and I'm saying, use the article embeddings, which is the one that we just created, and give me the top five based on the embedding of this article. So don't follow the path like we're doing before. Just take the embedding and give me the five closest one, excluding, of course, the article itself. I don't want it to be returned because if I did, I'm going to get one that's identical to itself, and that's not very useful. But this gives me a list of all the similar articles based, but as you can see, the article itself is a perfect match, which is not very useful. So if I remove it, I get the top four, and I get a metric. And yes, it does make sense of course you know this is an article about neo4j at least the first two that talks about neo4j that even the even the the the text search would have picked that up now the question is what happens with this one and and why do we think that's accurate and that probably is but you know we we kind of have to trust the model that has generated these embedding. And the question is, can I trust it? And for future episodes, I mean, which are my options? Can I train my own model? Of course you can, but it's expensive. It's a huge effort. Should I fine tune it? Should I retrieve or meant to generate? There's many options. But here, let's understand what this semantic search returns and that's what we're getting and if i did the the equivalent based on the on the other approach uh let me show that to you because here i would get um something similar in this case what we're doing and and notice that here i'm not doing proper search and I get back to that in a minute, but I'm doing the comparison and I'm getting a similarity metric, but I'm getting also, and I'm not listing it here, but I could get for each one of them the path that explains why these two articles happen to be related. And that's the explainability, the control. And just to complete the explanation, and you will see how that impacts the the results and how i have control over the results these are the answers based on the taxonomy that i've imported from wikidata but let's say hey i'm an expert in my domain so i know that uh there's a the notion of some of software stacks right i know that there's the notion of software stacks. I know that there's the GRAND stack that combines Node.js, Neo4j, GraphQL. There's the LAMP stack that combines, what was it, the MEAN stack that combines ExpressJS, MongoDB. So let's say I have additional knowledge. I know more things about my domain, and I want to use that. I want my semantic similarity to take that into account. So I take this ontology, or this additional knowledge representation, and I'm going to add it to my model. I mean, nothing stops me from keeping adding additional elements of information. Remember that this is all curated. I can bring them in and out as needed so in this case i'm i'm doing an input uh request pretty much like what we did with the with the neo semantics before but in this case it's not a scos representation it's uh i think it uses owl right there you go so it's owl classes it uses still the labels uh from scos but. So it's OWL classes. It uses still the labels from Scots. But it's a different type of representation. Doesn't matter. All I have to pass is the reference and the serialization format. And this will bring additional elements. I mean, I could have been too quick to import, but I could have previewed it first. If I want to do a preview of what this brings, that's exactly what I was showing before. Let me make that a bit bigger. And we have things like there's the notion of a software stack, and we have three of them. I mean, there's three mini taxonomies that describe the LAMP stack, the GRAND stack, and the MEAN stack. So it is another way of providing additional context for my search. So maybe if you're reading about PHP, you want to also read about Perl because there happens to be a technology stack that combines the two. And because I know that now, I'm offering you a new path for exploration. I'm providing you additional context, and I control that. That's the powerful thing. I decide what information is used to return that answer. And if I remove that, that's been persisted in the graph, and I rerun exactly this query, I'm going to run it again. Now I should get new results, hopefully. And there you go. So now you see that suddenly it's jumped to the third position, the one on GraphQL that was not there before why well because now my search my path exploration is finding that there's a stronger connection between these two because now they have graphql in common which is part of the grand stack see what how that goes right so this gives you the control and the power and the curation of the knowledge that's used for the semantic similarity whereas the other one relies on the the the model and the data that has been used to train train the model but again important the really a powerful approach comes from combining the two and and and the first step is to understand them properly so that was the objective of today's session so I'm going to take a pause I've talked a lot but I hope it was useful and and and that uh triggers some some questions hopefully that was the yeah it is it is um one thing I I want to add and maybe it's also I think probably I'll end it and end in a question um but when you said it's better together so I think it makes sense if if you stay with the picture of the fruit that came to me earlier. Okay, let's think about tomato. Tomato is botanically a fruit, but obviously you don't want to put it in a fruit salad because it's not really right. So if you have it in a classical ontology, it probably would land up under fruit because it's botanically categorized as a fruit. But if you look for something for, let's say, a fruit salad, if you go hardcore into the semantic search pattern through, it might come up as a suggestion, maybe not top ranked, but potentially down there. But in a vector search, potentially it would grab it still for, not for your fruit salad, but for a vegetable kind of based thing, because it would still pick it up as it is locationally closer to a cucumber, which is obviously not a fruit. Yeah. So if that makes any sense. Absolutely. Yeah. And yeah, and that we, things are becoming a bit more subtle because, you know, they're semantically related. So a tomato is close to an apple. Yeah. But if your question is more, and we didn't get into how to answer natural language questions, because another thing that we could do and that we can do with large language models is, hey, I'm making a salad, what should I do? And in that context, maybe the training data provides additional information on whether you should put that or not in a salad. So the fact that they are semantically close, that's a fact. And then on the on the um symbolic approach and the knowledge representation approach the graph approach i've shown the you know the most basic construct in an ontology which is a classification a taxonomy but there's many more i mean you can express disjointness i mean you can overlay multiple ontology so you can say yes the two category two instances of the same category because they're both fruits but uh in the same way i've added the technology stack i could say hey in in the context of a salad these two things should be disjoint or that or maybe this isn't compatible with that so you can keep enriching that and again these um implementations of of semantic similarity graph-based semantic similarity, are probably the most rudimentary and ones that can be the starting point, but the important thing is that you have the controller. So you can traverse the graph in whichever way you want. So you can say, find a connection across my graph, but block going through nodes of a particular type. So really, the powerful thing is you can use standard metrics, but you can also totally own that process and determine how you want similarity to be computed. Because whatever you come up with, you will be able to explain it. And you can say this was discarded because it went through a node of type salad and salad is a blocker for this type for for a for an orange for example i've seen an orange in a salad sometimes yeah you can keep adding context and and um and what i've shown is is really the the the most basic type of representation which is a classification yeah cool thank cool. Thank you very much, Jesus. There were a couple of questions, so let's take them on. Peter asks, would you normally have multiple paths between articles? Absolutely, absolutely. And I kind of quickly went over that, but yes, I mean, absolutely, because remember that an article connected to a number of entities, a number of things. So an article was about Neo4j, about Java, and then for each of these concepts you will have a path, so you will have a number of them. Again, another option for you to determine how you want to combine them. You want to add the more paths, probably the more paths means the stronger the similarity. Maybe you want to average them, maybe take into account. So yes, absolutely. There will be, and as you keep adding, because that's what we did when we included the technology stack. So you're offering new paths for exploration. So the more knowledge, the more curated the knowledge about the domain, the more control and the more additional paths you can offer for exploration and the richer the semantic similarity will be. So big yes to that. Absolutely. There will be multiple paths between articles. A comment from Mark, using the context to explain to skeptical managers is critical. I think that's very true. If your black box has something and somebody asks you about how did you come up with that 20% similarity is like, well, my machine said something and printed it out in the end. And maybe this case sounded like a non-critical one because worst case, you recommend an article that's not relevant and no one will click on the link. But think of a more important decision. Think? Think of an investment, think of a medical decision based on other examples. You really want in some cases explainability is a must. Yeah. So totally agree. And one more question from Ryan. I want to figure out a way to do this for property records including spatial similarity where then dave suggested maybe k nearest neighbor uh if that could be something to to use the the the actually the it's it's approximate uh nearest neighbors i mean it's kind of a different implementation and and i i mean on that i i will have to do a bit more research, but maybe the documentation you'll find. But it's based on nearest neighbors, the search on the vector index. But the question was whether it can be combined with spatial similarity. Exactly. I mean, the spatial is not yet another index. So again, the more information you make explicit in the graph, the more elements you have to... maybe your similarity in the end could be a combination of these contextual connections between entities, but you can bring into that the geographical, the geospatial proximity as well. So absolutely. So yet another another element to to bring into the into the the the semantic similarity comparison yeah absolutely and one more from dave um vector index is the community edition or where do you get uh to able to play with vector index vector search i think it's currently a beta feature so it's available in in all the editions um obviously jesus you just showed it on on sandbox um if you download you need to download the latest version but not not not 512 but at least you need to it was 11 11 exactly anything after 5 11 um will will will have it um as a beta functionality and then you can explore. Yeah, absolutely. So that's, yeah, like I said, I use Sandbox and you have it there. So approximate nearest neighbors. Yeah. Thanks, Mark. That's what's implemented. So, yeah. Cool. I think that is a good way to end the episode. Thank you very much, Jesus, for presenting this, for going hands-on again and showing the code, especially with the vector search. Very interesting and good to see it hands-on how it works, particularly because it's such an interesting topic and so many people are wanting to know more, but I think there's a little bit of a gap between what can it do, how does it work? What can it do for me? How can I use it? And what do I want to use it for exactly? And I think hopefully we clarified a little bit. and and what what do i want to use it for exactly um and i think that's that's uh i think we hopefully we clarified a little bit uh and on that topic as always all everything we've used is already in github so please go and play with it and let us know how your experience goes and yeah enjoy Yeah. Cool. With that, let's close it out. We'll be back in November, a date to be confirmed, but we'll look to be back in the regular schedule first Tuesday of the month. So that would technically be the 7th of November. Let's see. We might be traveling that week. So we need to find a way to see how we can how we can get that get that to work but um i'm sure we find we find a way um between uh now and then um obviously notes on the 26th of october so please uh check it out have a look uh register for it and next tuesday on the 10th um the final road to notes workshops is taking place uh this one on geospatial data with will lion so if that is something you're interested in check it out tuesday 5 p.m central european time so yeah early afternoon in europe early morning in the in the states and somewhat evening ish in asia So if that works for you, register now and hope to see you on Tuesday. Yeah and with that thank you Jesus, thank you everybody for sticking around on a Friday afternoon if it's your afternoon or Friday morning for Mark, early morning indeed and yeah thank you Jesus again. Thank you Alex and see you all next time. Next time exactly, take care bye.
Going Meta - Ep 21: Vector-based Semantic Search and Graph-based Semantic Search
Episode 21 of Going Meta - A Series on Graphs, Semantics and Knowledge 

Jesús Barrasa: https://twitter.com/BarrasaDV
Repository: https://github.com/jbarrasa/goingmeta
Previous episodes: https://neo4j.com/video/going-meta-a-series-on-graphs-semantics-and-knowledge/

Recap of 20 Episodes Going Meta: https://medium.com/neo4j/20-episodes-of-going-meta-a-recap-5a0ccd689c6c
Vector Search: https://bit.ly/4cuZoeS
Cypher manual: https://bit.ly/4cjICzf
Semantic Similarities (Ep 16): https://youtube.com/live/WwTxlyjY35I
Semantic Search. A worked example (Ep 2): https://youtube.com/live/y6eCKIRsA64

0:00 Intro & Welcome
6:40 Vector-based Semantic Search and Graph-based Semantic Search Overview
19:45 Prep the Graph
35:30 Computing Similarities
50:55 Q&A
59:21 Wrap-Up

Check out https://community.neo4j.com/ for questions and discussions around Neo4j.

#neo4j #graphdatabase #semantic #knowledgegraph #rdf #semantics