 Here we go. Hello, good morning, everybody. Good evening, good afternoon, and thank you for joining another Neo4j Live today about the Knowledge Graph Builder app that my two colleagues Niels and Morgan have developed over the past. How long has it been now? A couple of weeks, maybe? Months? Yeah, I would say months. months? Yeah. So that has been a recent addition to the GenAI ecosystem, I would say, and all the interesting projects that came about end of last year when everything turned into a GenAI application. Obviously, we also added one, and you probably know Niels from an earlier stream. Maybe you know him from an earlier stream. He is the mastermind behind the NeoDash application. It's a dashboarding tool. We showcased that a couple of times in this series and also on nodes. So welcome, Niels. So good to have you back, kind of, today. And then we have uh we have morgan as well morgan is is is new to the stream but nevertheless even more welcome to to having you here today you are um where are you based actually morgan in uh in france um no so i'm based in london i'm french uh from the north of france but i'm based in london yeah yeah cool um and you uh what's your what's your story maybe you want to share a few words of uh of of what you what you do and how maybe how did this uh this collaboration uh come to come to yeah yeah sure absolutely um so as you said right my name is morgan i'm part of new 4j professional professional services um i started my journey with new 4j five years ago now, so quite a long time, probably the same as you, Nils. And so my job is basically to guide and advise on how to use Neo4j in the best way possible, not just obviously on the function and feature we provide at Neo4j, but also how to make it work within your ecosystem or environment. And obviously lately I've been focusing a lot on, obviously as we all know, AI and innovations. So that's what I do, yeah. Yeah, cool. And then this, I don't know, we can dive right in, but maybe what was the most, maybe before a little bit to get to know everybody a little bit better is like this hype, this everything Gen AI, everything LLM kind of. How was this? How did it feel for you? How was it? Did you feel like immediately something has to happen, something needs to add? You know, maybe Niels, did the first thing you saw, i need to add this to neil dash um yeah i i mean i can i can start so for me this was really um a feeling like you know there's a lot of hype there's a lot of talk but i want to make something real i want to make something that people can use and actually use that to to to create some value and not just slides and words that was my idea yeah probably morgan you're the same yeah yeah i'm totally the same on that i mean obviously we saw all the talks on the slide on how to how we should work in theory but we never have actually have like a real uh demo or actually working solution from hand to hand uh i mean in the past like years when it all started. And the way I see it really is basically just the way we saw like Google for the search engine like years ago. It's so nice tool. Obviously, everyone has to use it. And if you don't use it, obviously, you're going to be left behind. So, yeah. Yeah. And it's good to see something that I think that's a good point. There's something that you can actually try out something that you can see that you can, that you can then explore yourself a little bit and then that can, that actually works. You know, it's not just as you said, it's not just some slides that, you know, always very, very, paper is very patient as they say in German. So it's, you know, you can, you can, you can, you can put as much as you like on there, but you never have to do anything about it. But if you see something there and if something then comes to life a little bit and you see some results directly or you can execute something and then you say, yeah, this is cool. This works. This gives me feedback, gives something back. So that's pretty cool. How did you two then, how did the collaboration, do you want to say a few words about how did that happen? Do you want to take it or should I take it? Yeah, no, you go ahead. Yeah. So basically it all started from an initiative from Christopher Crosby and Michael Langer, which you're going to see in the slide. Actually, we do have a small slide with the team on this application. But so basically the idea was, okay, so we do have large language models, which obviously we can use for a lot of different things. In Neo4j, we obviously are trying to innovate on that aspect as well. So that's why you see we do support vector search for example, vector indexes and integration with those. So now the idea was to make it work from hand to hand. And we're going to discuss about that in the first few slides. Neo4j can be used as the knowledge graph, right? Which means basically take all of your knowledge, put it in a graph. All the documentation, documents you have on your company, on your processes, on your application, whatever documentation you have, just put it as a knowledge graph. So you have the knowledge represented as node and you have the relationship connected between each node. And then the idea was, okay, now that we have that, what if we create embeddings on top of it? And based on the embeddings, we can actually try to use that information, the knowledge from unstructured, into what we call GraphRAG, Ritual Augmented Generation, which is basically ask the large language model, which doesn't know about your knowledge, they don't know about what you have internally, your documentation, your processes, but feed them that information from your knowledge graph and get obviously better results, transparency and explainability. Pretty cool. Pretty cool. That sounds very exciting. Before we dive in, I just want to say hi to everybody who's watching live. Good to see. I see people from Lake Constance. I see a couple from Paris, Texas, Seattle. Good morning, Rob. So that's an early morning for you. So hopefully you have your coffee ready and we can entertain your morning here. Chicago, London, Cambridge, Massachusetts, Bengal, Luro in India. Hi, welcome. Geneva. So Bordeaux, Rome. So all over the world. So thank you very much for watching. If you have any comments, any questions, anything that's not quite clear or anything you would like to know a little bit more about during this showcase today, then please let us know. Type it in chat. Use the chat. It's supposed to be interactive. So if you have any question, anything is burning under your nails and kind of like, oh, I want to know more about this or that or how it's done or something else, then, yeah, you know how to do. Use chat. But hi, everybody. Cool. Yeah, should we dive in? You said you had a couple of slides to outline the project. So should we start with that or what do you prefer to do? Yes. Yeah, I'll start with slides. I know I'm contradicting myself starting with slides, but I just wanna quickly set the stage and I promise it will be very short. Yeah, yeah, that's cool. All right, let me share my screen and hope that goes well. Can you see it all right? Not yet. No, I don't see your screen. OK, wait a second. Something is wrong here. How about now? Oh, yeah. Here we go. Perfect. All right. Cool. So super quickly, to set the stage about what we're doing here, I promise, as I said, it will be short and we'll get right into the demo and show you something really cool. As you already heard, so Morgan and I, we work together in the field team at Neo4j, which means that we are engineering together with our customers, building Neo4j solutions. So this could be anything from data modeling, querying, visualization, or even integrations with LLMs. So we like to build solutions on top of Neo4j to make our customers happy. And what we talk about today, I will do a super quick intro about Neo4j to make our customers happy. What we talk about today, I will do a super quick intro about Neo4j and LLMs, how people typically use that together. Then Morgan will take over and dive a little bit more into the technical details, how we built this demo. We'll do a nice demo of building a knowledge graph and then also a chat interface on top of the knowledge graph that you're creating. Finally, we'll do a super quick wrap up of everything that we talked about today. So intro to Neo4j and LLMs. So Neo4j, obviously, we're all about graphs. But we're also doing recently a lot of integrations with generative AI and large language models. So what does this mean? There's a lot of things you can do with GenAI, but for us in Neo4j, this really mainly focuses on two big areas. The first one being GenAI tailored to your organization. So this means how can I actually use this technology, these large language models, with my own data, answering questions on my own documents, my own tables, my own graphs, right? And that turns out it's not so easy. I mean, it's easy to spin up JetGPT and ask some questions about public data, but making it work with your own data is quite tricky. So we're going to be showing some stuff called grounding, which means hooking up a large language model into a knowledge graph and using that knowledge graph to answer questions about your own data. And that's what we call, as I said, grounding or graph rag as Morgan already alluded to. So that's kind of on the front end side, like how do you consume a graph with which NAI, but we also have the number two, which is the backend side. How can I build a graph with Gen AI? So using large language models to create knowledge graphs. And especially what we talk about is natural language entity recognition, NLP, right? So finding in a document of unstructured text entities and turning that into a graph. And this is also something that we'll show. And these two kind of logically follow one after the other, right? If you create your graph with a large language model, you have your knowledge graph, and you can use that same knowledge graph to answer questions in natural language from users. So two and one, together they kind of make up this recipe of a Gen-AI plus graph use case. And then number three, I think something that we will also show is that we have all these partnerships with cloud providers like Azure or GCP or AWS, but also a lot of integrations with open source tools like LandChain and many, many others. So we're really well integrated with any kind of provider of LLM or tooling around that, which makes it super easy for you to build your own Neo4j plus GenAI solution. We love to open source things, so also the tool that Morgan will be showing later is completely open source for you to use and try out with your own data. Drilling down a little bit more on what I said, these two buckets, right? First, the jumpstarting knowledge graph at llm or building a graph using our flames models we have two subcategories there the first one is the very very first step of data creation turning your entities and relationships creating your entities and relationships from a unstructured data into a graph so we built this kind of reference architecture this flow and we'll have a better picture of that later on to turn your documents your unstructured text into a graph the second one here is curating knowledge graph with query generation so once you have your initial graph maybe you want to make some changes and you could even do that with natural language you could say okay fix this fix this node here connect to this this other node. So that's step two in the process. Then on the front end side, as I said, this is all about chat interfaces, natural language interface. How do I consume data from my knowledge graph? So this is the famous example, chatbots, that we all love. Natural language queries, I don't wanna write Cypher, I don't wanna write SQL, I wanna ask a normal human question to my data, to my graph and get the answers from there. So that's something that we will also show. And then the graph powered search, which is grounding. How do I grind? How do I ground my search in reality? How do I ground my possibly hallucinating LLM into a trusted knowledge graph? So these are the four main architectures that we see when we're building these kind of solutions for our customers. So as I said, we're going to be drilling into some of these today. So these two is the one that Morgan will be talking about later. And just to give you a feeling for what that looks like, what are the components involved in this entity extraction flow? So this is a very high level diagram of what that looks like, what are the components involved in this entity extraction flow. So this is a very high level diagram of what that looks like. You got on the left here, you've got your data sources. So you have unstructured data, as I said, documents, text, even videos, images can be as crazy as you want, but you can also have structured data that you insert into your knowledge graph. Then using your large language model, you extract entities from these documents and turn that into a graph, and you can use a wide variety of tools for that. So your data, your documents get turned into a graph, and you can also create vectors on that and use our vector index to create that, of course. So this is all about building graphs right um as i said once you have that in place you go on to the next step which is building a natural language interface and again you've got a whole orchestra of tools here a lang chain llama index any kinds of tools here to to build your own interface on your own graph and this is again where we're really nicely integrated with these tools. You'll see some on the list here. We have APOC procedures, open source APOC procedures that you can use to call LLM providers. Again, there's a lot of stuff here. You can build your own UI with Streamlit or even with JavaScript, any types of things. We're trying to make it so easy, as easy as possible for you to build a Graph plus LLM integration. So this is a high level of review of what our GNI landscape looks like. And I'll just now hand over to Morgan, who's gonna zoom in onto this specific project that we're talking about today. Thanks, Niels. Maybe I can just take it for a night and share my screen. Let me do that here. Screen, here you go. Can you see that? And before I even start actually talking about the VISA, just a couple of questions, actually interesting questions in the chat on the transparency and explainability as well as what's the difference between graph rag and vector rag. I'm not going to answer that right now, but you're going to see during the demo, you're going to have the answer to those two questions. So going to the next one here. So what I'm going to show you in the demo here, and just going back to the premise Niels made, it's not going to take long. I just have like three slides, just obviously to give you some context on the application, how you can use it and how we made it, right? So the application which we're going to demo is basically a simple and fast demo, basically application to onboard text to graph. So basically any text you have from unstructured documents like PDF or from cloud storage or any document you have into a knowledge graph it's usable for end users meaning that you're going to see in the demo you can use the application load your knowledge in a graph and start asking questions to the chatbot with zero code without any knowledge around how to code or do anything and it's also a good starting point for developers because as nils was mentioning everything is open source we're going to link all the github repository and all the sources so you can basically just take it and you can extend it as you want and i'm going to show you as well how i made some extensions already on some part of it and then obviously uh what's what's the aim goal is basically to demonstrate how graph rag compared to vector rag is powerful in combination with llm genii and a knowledge graph on neo4j so how does it work well i'm gonna go quickly through that because Niels already gave all the pointers. But basically what we do is we load all your documents, which can be, for example, PDF, as we were mentioning. We do the split on it, so we create chunks of that document. We use the LLM to extract nodes northern entities on the chunks we merge that northern relationship into the neo4j knowledge graph and then we use a chatbot interface to do graph rag the architecture is as you can see here on the screen from left to right so you do have data sources which can be as again as i was saying document pdf videos anything you want uh we actually have this what we call lm graph builder which is the application i'm going to show you so you do have two sides you do have a react.js front-end application and you do have a python fast api back-end obviously we do have the api layer in between to make the front-end communicate with the back-end so you can extend it as you want as well and then we have the large language model which we do support quite a few models which you're going to see the demo. On the knowledge graph aspect, so we do have Neo4j RR, obviously you can use any Neo4j database you want, you can run it locally on-premise or you can use our self hosted Neo4j RR. You do have Neo4j Bloom, which is the end users to show the graph. You're gonna see when we create the knowledge graph, I'm gonna show you nice visualization to see the the graph. You're gonna see when we create the knowledge graph, I'm gonna show you nice visualization to see the knowledge graph. You do have the graph database, so Neo4j, and then we do have graph data science because we are using embedding and then similarity on the chunks. So remember when I said, we take a document, we split it into different chunks, then on the chunks, what we do is we generate embeddings and then we create similarities based on the embeddings between chunks. And then what we have is we do have the chat interface, which is also part of the demo, which is all around ReactJS front end and then communicating with the python fast api um in terms of end user application and use case you can see here on the right we do have few obviously use cases application that we see where you can use this but obviously it's not exhaustive and you can come up with basically anything you want really where you have your knowledge and you want to ask questions about your knowledge so in this one for in this list we do have obviously faq boats we do have supply chain which i'm going to show you pharma pharmaceutical industry health assistance all these kind of things. So as we were saying, all the projects, so everything, the front end, the back end, the whole solution is open source. So you can see the GitHub repository here on the top. Obviously we're gonna share all of that so you can access it. And we do have a couple of more links to make it easy for you to find all that application. As I was saying, the frontend is made with ReactJS, so you can connect to Neo4j. You can see you can load local file, PDF file, you can load from S3 or Google Cloud Storage, or you can load YouTube and Wikipedia sources. And we do have a chatbot which is integrated within that application. On the backend, which is met again with Python fast API, what we do is we do store all the data. So every knowledge that you pass, we do store that in Neo4j. So meaning you're gonna see that in the next slide, but meaning the documents, the chunks and the entities, everything is stored on Neo4j. We do the chunking as part of the backend. We create the embeddings on the chunks. We do the entity extraction. So meaning you have a chunk of a document, we detect and extract the entities with the node and relationship in that backend as well. And we do KNN on the the chunks so you are able to see how similar chunks are connected to each other's everything is running on docker images so you can simply download the github repository do docker compose up with your environment and you will be able to run it locally or you can use the one we include which you're going to see in the next slides um the step-by-step process so i was obviously mentioning it a bit earlier but the way it works right you are going to have the front end where you can drop files pdf youtube videos whatever what's going to happen then when you hit the generate graph is we're gonna basically take that document store it as a node in Neo4j with different information. Based on this document, we are going to apply a chunking strategy. So each document with langchain loaders, and then we do split the document into different chunks. So one document can have one, two, three, five, ten chunks. So one document can have one, two, three, five, ten chunks. Then once we have the chunks, obviously we store them into Neo4j, into the graph database, and we connect them to the document that we just created. We do create the similarity based on KNN between every chunks. So you are able to see if two chunks are very similar to what they're saying you were able to create a similar relationship between them we generate embeddings on top of the chunks and we do have a vector index on those embeddings and then what we do to extract the entities on that chunk so for each chunks what we do is we do we use the LLM graph transformer or divbot graph transformer to detect those entities and relationships. And then once we obviously detect them, we load them into the graph and connect them into the chunks. So regarding the team, and then we can move into the demo. So either we're saying the PMs on the applications are Michael Enger and Christopher Crosby. Mostly the team who done all the development or the contractors called Persistent. So you can see the teams here in the middle. And then they do have a few helps from different person in your 4g so you can see too much one but which you might know if you read his blog post and all the work is done on length Shannon or the geni and LLM integration then you can see myself nice Oscar and Jim alright so let's go into the demo now. So what I'm gonna show you today is, so obviously here I'm using Confluence for the documentation aspect, but it can be anything, right? So my idea was to show you, let's say you have Confluence or any other tools, Notion, for example, or anything else, where you have all your knowledge, knowledge about your industry or your application, your APIs, your development processes, whatever you have, right? In this case, I do have it on Confluence. So I'm gonna show you a supply chain demo basically. So you can see, I do have few pages. This introduction for example is about a fake company called Graphac.me, which is based in Brussels, Belgium. They are making a bunch of different products and they do have supply chain processes and strategy. On the product specification, we can see we do have two fake products, which is EcoPack and WebRestoreustor eco pack is basically a packaging solution produced in bangladesh and then we do have robusto but is in china which is a small component polymer component so we do have obviously more pages here i'm going to load all of that into the into new 4g with that application and then we're going to be able to answer questions. Now, the reason we took that demo about supply chain is because, not sure if you saw that in the news, but there is a new law which has been approved last week, which is CSDDD, which is going to have an impact on the supply chain companies. And that has been approved, not voted yet, but approved last week. So obviously, this is something which is really recent. And if you're asking the LLM, you wouldn't have this kind of knowledge, right? You wouldn't have both, actually, neither the knowledge on this fake company or this new law. So what I'm going to do now is i'm gonna show you the ui of the application so this is what the graph builder looks like um so you have here on the left the menu to load your data from different data sources so you can see this drag and drop or browse here to load your p PDF files from local file system. You can load YouTube videos, you can load Wikipedia sources, or you can load PDF files from an S3 bucket or a Google Cloud Storage. So obviously for the Cloud Storage, if you want to make it work, you need to provide your access key and secret keys, but then you basically just pass the bucket name and then going to list all the PDF file within that bucket. Now what I'm going to do is, if I can show you here what I've done on the Confluence page, I basically just exported all my pages into PDF. So if I open just one PDF here, just to show you that, it's really just the same thing that I showed you earlier, but in the PDF format. What I did is I extracted this article from Forbes as well. Let me show you that here. So that was from last week. Forbes made an article about this CSDD law, which has just been approved on the 24th of April. And they have some comment and link to different sources here. So I'm gonna load all of that. So I'm simply just like gonna drag all of that. First of all, obviously I'm gonna connect to Neo4j, to a Neo4j database. If you don't have a Neo4j database, you can click this link. And what it does is basically take you to Neo4j Aura, our self-hosted version. So in this case here, I do have a database. So I won't have to create one. But if you don't have one, you can head over here. You can create a free database for free. You don't need to provide any credit card or anything like that what's what's nice about this is once you created your aura database you might be familiar with that but you can download a text file which will contain your environment configuration which is the database url the username and password all these kind of things you do see here a small drop zone where you can drag and drop an env file well it does support the one you get from your folder rs so in fact i can just take this one drop it here and you can see username password and uae or autofill so that's really cool let's connect to this database you can see obviously no data is present, I don't have anything on this database, it's basically just an empty database. Now what I'm going to do is, as I was saying, I'm going to drag all those PDFs, so my Confluence pages and this article from Forbes, into drag and drop. And you can see obviously it's uploading the files, so you can see different things, we can see the name of the files, you can see the status, you can see whether we were successful in uploading the file, you can see the size, the type, which you can see we do have different types, the sources, so in this case we are local, the model, which we're going to talk about that in a second to create the knowledge graph, and the node relations, durations, and processes when we create the knowledge graph and the node relations durations and processes on when we create the knowledge graph now what i'm gonna do is just to showcase the other stuff i do have a youtube video which is this one here which is talking about uh the law obviously csdd uh it's a five minute video which is talking about um different things on what what's the impact of the laws how this company is gonna change the way of working all these kind of things so I'm just gonna grab the your head from this YouTube video I'm gonna go here click on YouTube and you can see you really just have to paste the link YouTube link you can click Submit and you go and if we go back here you can see the YouTube video is here on the top so it does get the name right from the YouTube video and you can see in this case here we do have YouTube as a source you can click on it and it's going to take you to the YouTube video I'm also gonna just to showcase here I'm gonna show you how to load sources from Wikipedia so in the way it works is you can provide different Wikipedia sources separated by a comma so in this case I do have two Wikipedia pages I do have this one about the law itself and I also have this one fromadesh and you can see why it's because um if i go back to here you can see here i do have like some product um who are produced in bangladesh so i just wanted to showcase you how the graph database is going to make these links between all different things so what i did is basically just get the sources from those two wikipedia sources so bangl Wikipedia sources, so Bangladesh and then VizLaw, and hit submit. It's going to take just a couple of seconds just to load it from Wikipedia, getting all the different data from those sources, and there you go. You can see now we created two other documents, so one for Bangladesh and one for Vizslo, which are taking their sources from Wikipedia. And you can see again, we do have the link, so we can click on it. Now, on the bottom of the screen here, you can see the list of different models we do support. So we do have DivBot, we do have Gemini 1.0 Pro and 1.5, and we do have OpenAI 3.5 and 4. And by the way, I didn't mention it, but all this application, as I was saying, it's open source, so you can rent locally. But we also provide a self-hosted version that you can play and try on your hand. So we're going to link that after the demo. But then, obviously, you can play a bit with different models obviously we do recommend for your testing or playing with the data sets with your data set to use 3.5 because of C's cheaper in terms of tokens price per token in this case I'm just gonna use GPT-4 because all the size or fairly small and on that so I'm gonna click on generate graph here and you can see it's gonna start processing all the different documents it's not it's all we see in a thread and environment running in parallel so we do process multiple nodes at the same time multiple chunks of each document so you might see some documents finishing there you go so you can see one for example here which completed faster than the others and you can see for example here as i was saying you do have different informations on the knowledge that we built so you can see for this specific document we created 14 nodes five relations and it took us 15 seconds now you can click on the small glass here it's going to show you the the basically the graph the knowledge graph which has been generated based on that document but I'm going to come back to that once everything is loaded because you're going to see a better view of everything there you go so you can see in less than what couple of minutes maybe we loaded all those pdf document youtube video and wikipedia sources into new ford so we do have quite a few nodes what you can do now is click on show graph and what i'm gonna do is I'm gonna remove the chunks here and you can see different information so you can see obviously different document different chunks well I'm gonna look at here let me see if I do for for number you go you can see the main reason why I wanted to have this this Wikipedia sources from from Bangladesh is because you can see here we created a node country which is Bangladesh. We get all the different sources from the Wikipedia sources so you can see all of that is basically coming from Wikipedia. What you can see here on the right is we do have a product RoboStore from a company called GraphAcme. Sorry for the coloring here on the white screen, that's something we probably want to change. But this one is basically the second product, which is the the EcoPack product. So you can see GraphAcme from the conference page does produce two two product this one you can see is produced produced in Bangladesh so going back to the question on obviously graph rag and vector search well that's something which you can see already which is quite nice when you have your knowledge graph into graph and not not just vector database is because then you can link all the data you have in your knowledge whether it's structured or unstructured into all your different sources as you can see from my knowledge from Confluence being linked to some knowledge from Wikipedia. All right so that's for the knowledge graph kind of constructions um now what i want to show you is the second aspect of the demo which is about the chatbot interface so if you click on this button here you can see we do have this uh small pop-up which uh appear on the screen so obviously you do have different information here, you do have the source here which link back to Neo4j but you can start obviously asking questions here. Now what I'm going to do is, because I see it's already 37, I'm going to do both into instead of doing the demo here, I'm going to show you the same chatbot interface but in the bigger larger screen because then I can show you what you can do in terms of extensions as well so this chatbot interface is basically the exact same as the one you see here on the screen but I just added some extensions to it so what I'm going to do here I'm going to ask the first question which is going back to our conference pages you can see we do have different product we do have performance criteria so what I'm going to do is I'm going to ask what are the performance criteria of this product and where is it produced and who produce it there you go so now what's going to happen is sorry what's going to happen is we're going to basically create we're using basically graphag to create the graph structure, the chunks, based on the MNNs, and then we're gonna provide a response from different sources, from unstructured information and from certain informations. There you go. So there's a small glitch here on the sources, but let me ask another question just to showcase that. So you can see obviously it gets it right, EcoPak is being biodegradable, moisture-resistant, durable, etc., whatever, which is basically what we have here. You can see the production in Bangladesh and produced by GrafAgme. So if I look at here, it's correct. It's produced in Bangladesh and GraphAqm. Now let's say we don't know who this company is. So let's ask him who is GraphAqm. Here you go. and it should be in a couple of seconds we should have the response okay there you go and yeah so you can see this one was just a small glitch. I probably have to have a look at it. But then once you have the sources, going back to the explainability and transparency, you can see what we do is basically we list the sources from the question you ask. So for example, here we do have the question, who is GraphAcme? So we do, as I was saying, we do create the structured and unstructured information. So you can see it's giving a lot of different information, not just from one document, but from two documents in this case, with different information which you can fact check. But then you can see it's actually listing the sources from where it got its information. it's actually listing the sources from where it got his information. Now, going back to the new law which was approved last week, what I'm going to do is I'm going to ask him... okay so we got it... impacted by the new csd low and let's say and if so how and let's see what he gets now while it's actually generating that what i can show you as well is uh talking about the extension you can see here a bunch of different icons here um So what I had is basically turn up and turn down in terms of providing feedback on a response. You can also generate the response if you are not happy with the response and there is a text-to-speech as well. So what I'm gonna do is wait for the response here. Okay, here you go. So you can see we do get the response from a different documents here. It's obviously gonna list that GraphAcme is impacted by this new rule of course, because we all look at it in Europe in brussels belgium which was part of one conference pages um but and but we do produce uh product in bangladesh and china um so that's how uh what i'm going to do is i'm just going to stop sharing quickly and share my screen again because i want to share share with you the audio for the text to speech just to show you that you go. And I can't, I can't share the audio with this full screen. Sorry about that. But you can see if I go back here to this one here, if I click on play, we're going to basically have text-to-speech on that as well for accessibility. GraphAcme is a company that manufactures and supplies sustainable packaging solutions, including the EcoPack and components for automated warehouse systems. Their products, such as the EcoPack made from jute and bamboo, are designed for eco-friendly packaging and home decor, while their RoboStore components are intended for industrial use in automated storage systems. GraphAchemy is headquartered in Brussels, Belgium, and operates production facilities in Bangladesh and China, focusing on quality assurance, customization for bulk orders, and compliance with industry standards. All right. Probably should have taken a shorter one, but here you go. Then so what we're going to do now is so obviously we do have whether the we do get information on the EcoPak or another product on the company, whether we are impacted and all of that. That's really cool. But now obviously, now that we know we are impacted what i want to check is uh has this law been approved because i want to know uh if it's already there or not so i'm gonna ask when uh and obviously maybe we can ask also uh what is the current status of that law what is the current status of that law. And obviously, as we were mentioning here, while it's generating the response, this whole demo here is made on this fake supply chain data set, apart from the law, obviously the law was a real thing from last week, but you can apply that to any knowledge you want, right? It doesn't have to be just about supply chain, any knowledge you have internally at your company or even within your environment, you can just do the same thing here. Okay, so you can see the new law was approved. It has been approved on April 24, 2024. And then obviously giving a bunch more different information on that. Now what I want to do, the last thing I want to show you as well is about the multi-op questions. So what are multi-op questions is um something that you can actually test your when you bring the chatbot um uh what you want to test is this multi-op questions is basically a way of double checking that not only does get the context but it's also being able to process reserved based on different informations so what I'm going to do here is you can see this product, which is RoboStore, which is a component, does have this dimension. And you can see this echo pack, which is the packaging, does have the dimension, which is lower than the RoboStore. So obviously with one echo pack, you cannot fit RoboStore into that. So I'm gonna ask him exactly that and see what he respond to that. So I'm gonna say, can the RoboStore be wrapped inside the echo pack for shipping purposes? And let's see how we reply to that while it's generating the response well I can show you as well is a couple of things so obviously this is the good a proposed tree so again everything is open source so go ahead get this repository you can clone it you can play with it locally give it a star if you want to if you want to you can so fork it, you can play with it locally, give it a star if you want to. You can also fork it if you want to extend it. You do have this GenII ecosystem and all the links, I believe, Niels are going to show you that in the next slide when we're going to go back to the presentation. But this GenII ecosystem, which basically lists every initiative we have at Neo4j about GenIA and LLM. And you can see in the example project, we do have the LLM graph builder, but that's not the only thing we have. But if you click on this one, basically, and you scroll down, you have some explanation how it works. We're still seeing the chat bot only. Oh, ah, yeah, because I'm sharing the tab, sorry. Well, in the meantime, we do have the response from the chatbot. So let's go through that one. So you can see, no, obviously it cannot fit because the dimension or incompatible. So yeah, that was the demo for the chat interface. Now going back to what I was saying without changing my screen, the, this GitHub repository, which is linked in the slide and you're gonna have, you're gonna be able to access it. It's all open source as I was saying. You can give it a star, you can fork it to extend it and you can play with it locally. If you don't want to run it locally, you do have this page, GenII ecosystem, which is again listing all the initiatives we do have at New4j. And in example projects, we don't have just this project, obviously, but we do have this one here, which lists all the documentation, how to run it, etc cetera. But it also comes with the self-hosted version. So this link here is publicly available. So you can just open it and you can play and do the same thing I just did here with this link. Now, last thing I wanna show, and then I'm gonna give it back to you, Niels, is something we are working on, which is right now, as I was saying, we do have, whenever you load your data, we do process it, we do create the chunks. Based on that, we are going to create entities. What we are working on here on the development branch is to be able to come up with your taxonomy. So meaning that you will be able to come up with your taxonomy. So, meaning that you will be able to provide what node labels you want to create only and what relationship only. So, instead of letting the LLM and our basically strategy to come up with entities, you will be able to restrict that. So, you can either just come here and say I want to create a fake company for Nopal and you can either just come here and say, I want to create a fake company for an Apple, and you can create your own labels. Or if you click on use existing schema, sorry, here I'm not connected to Neo4j, so I'm just going to refresh. Here you go. Because you need to be connected to your database. If you click on use existing schema, it's going to pull the schema from your database. If I click on that, you're going to see it's getting all the labels from the database and all the relationship so you can come up with your own taxonomy um and that's pretty much it from me i'm going to hand over to you news now cool awesome yeah thanks a lot this is a really really cool demo and just from my side the big takeaways here is that this this demo i mean anyone can use this there's no code involved Yeah, thanks a lot. This is a really, really cool demo. And just from my side, the big takeaways here is that this demo, I mean, anyone can use this. There's no code involved. You don't even have to provide a model, the LLM in first model for you. You don't have to write ciphers. So my grandmother can use this in a way if she knows how to upload a PDF, which I think is pretty impressive for a demo like this. Of course, it can be customized. As Morgan said, you can provide a model and give it something to hold on, or custom queries, or agents, or this kind of stuff. So it can be customized. And I think what's also interesting is what you showed, where you combine these different PDFs and combine different data sources into one graph is what we do quite often, right? Creating a knowledge graph from different data sources and ask questions that are across data source, which is very cool. And then the last thing that I know we talked about at Salt Morgan two weeks ago about explainability, right? You give the sources for each question. That's something that a lot of our customers ask. They say, look, we think LLMs are fantastic, but we want to make sure that the information that comes back is actually true. So it's super valuable to provide the link, the PDF link. Exactly. Well, actually, I probably didn't show that as well in the demo, but so you saw when the chatbot was replying, it was giving the source of the pdf document but if the sources are from youtube video or the wikipedia sources that you loaded you're going to have the link you're going to even have the link to the wikipedia page or the youtube video that is referring to yeah cool i think we're almost out of time. I'm just going to do my very last wrap up here. And I think this is already a lot of stuff that Morgan has said. This demo is open source and it's actually built of open source components. If you're making your own Neo4j plus LLM solution, you can take this as a base or you can take the ingredients for this and build your own thing, whether that's just the back end or the front end or both. Right. So the website that Morgan showed, neo4j.com slash lab slash Gen-AI ecosystem lists all these components that you can use for your own solutions here. Right. So we showed you the LLM graph builder, which is the UI plus the back end. And under the hood, this is using stuff like back to research, the epoch-gen-AI procedures, langchain, the graph transformer. This is all sitting under the hood, hidden from the user. But all these ingredients that you could use to build your own solution here. And that's it for us. Thank you so much for joining. I think we have a couple minutes for some questions, maybe. Yeah, that would be cool. I have lots of questions. So maybe we can take some time and go through them. And I don't go by chronicle order, so sorry, but I'll pick a couple and then we see how it goes. But one that just came in that I think maybe is interesting from 804. Do you plan to add more sources? Like they are mentioning Excel files. Is there something that you think is in the future for the app? Yeah, so that's not something we have considered so far, but that's something we can obviously think about. So I can't make any premise here, but as I was saying, right, the cool thing is it's all open source. So if anyone to contribute or even like if you want to extend it as well, yeah, we can do that as well. Cool. Another question comes in from LinkedIn. They want to know if it's possible to see the cipher that's kind of like generated in the back. Do you think that is something you want to do or or not um yeah so i think we need a bit more information on uh which cipher is it for the um for the question uh that is being asked so because going back to um to the processes right that we have for the data loading part uh obviously, we don't have proper cipher generation because we're using different frameworks from Langchain. As I was saying, we do create the node for the document, then we split the document into chunks, and then all the process to extract the entities. So I think the question is more on the chatbot. So for the chatbot, what we do is we do ask the question from structured and unstructured sources. And maybe the question is from that aspect. And yeah, we do have the cipher to do the retrieval basically for the augmented generation retrieval. And we do have the cipher query which we provide based on the chunk and document that is basically best fit into the question. Cool. Jeremy asks, the references when you showed the chatbot's answers were basically all PDFs instead of something from Wikipedia. Is that because the PDFs have the same info and get higher matching? So like a tiered thing like PDF, is there some tiered or some kind of value towards which sources or was this accidental? Yeah, no, I think there's so all, so the PDF in terms of type, the PDF are going to have the type PDF. I think the question is more for the sources from Wikipedia and YouTube, which all show as type text. And yeah, the main reason is because what we do is from Wikipedia, obviously, we get the data as text, right? So we do flag the type as text because, well, basically, it's just a bunch of text that we get from Wikipedia. From YouTube, what we do is we do scan the transcript which in the end of the day is text text i think the question is do we order i mean because it cannot you cannot show all the older older sources all the time but it showed pdfs preferably is there like a weighted algorithm as well that says, okay, use PDFs before using Wikipedia? Or was this just an accident from the result list? Yeah, it was just from the result list, which is something I have to have a look at. Cool. Rob wants to know what's happening actually when you click the thumbs up thumbs down buttons. Okay so let me show you that I can share my screen again just quickly. So remember what I was saying is because it's all open source you can just grab it and you can extend well that's basically what I did here right you can see the one which is included right now doesn't have any feedback loop. So the one I showed you here, which is this one here, it's actually the same in terms of response and interactivity, you're gonna have the same. But what I did is I added those feedbacks. So it's kind of an extension, which is not part of the application here. It's something I did here just for the demo, same for the text to speech. So the temp hub is not doing of the application. Here it's something I did here just for the demo. Same for the text to speech. So the Thumb Hub is not doing anything for now. It's really just an icon here because I still need to think about how to make it work. The Thumb Down, so what I did is if you click on it, it's gonna show you a pop-up and I'm still working on that. But basically the idea is you can provide a text feedback. So for example, you are not happy with this text because for some reasons you think the name is wrong. You can just come up with your feedback saying, the name of the company is not GraphArc, but GraphOrAwesome, for example. graph but graph or awesome and if you click on save to the graph what is going to happen is it's going to save that feedback into the graph as a feedback node it's going to connect it to the chunk which is related to the question you were gonna you're giving a feedback to as you can see here feedback saved and we do generate I And we do generate, I mean, we do. So what I do is, in this extension, is I do generate the embeddings for that feedback as well. So that later on, if you ask a question, which is similar to this chunk or similar to this response, it's going to be able to retrieve the feedback you just provided. Yeah. Okay. That's cool. One question from Alonso. Can you use local models so you don't have to rely on sending something you don't want to send outside of your local private network to the internet, basically? Yeah, absolutely. So right now, for both the knowledge construction and the chatbot we do have the models we show you um so we do have open ai we do have chimney and if but um obviously you if you come up with your own for example if you're running on olama for instance um obviously you cannot take the application, run it locally, just click on the button, it's going to work. You're going to have to do the integration because we are not doing it. But in theory, you can just take it and make it work. Cool. An interesting question from Jacob. Because you showed this with the relationships and nodes and the labels, can you provide a schema to use as a guide or basically as a baseline before you actually generate the knowledge graph? Not right now, but it's basically come down to what I was showing on this development environment, which is this setting here on the top. So before you even load your data, you can go over here on the settings and you can come up with the schema or the taxonomy on which you want to create your knowledge graph you can create your own stuff so my own label or relationships or you can delete stuff or you can create it from from the graph using existing schema. Cool. There are lots of questions, but maybe another one here from Brian. Is there a limit on scale? Can you upload 100,000 documents? You can wait. Yeah, I would say it's about the weight and the limits you have on your API key. So when we load data, obviously, as I was saying, we do create the chunks, and then we create embeddings on the chunks, and then we create entities based on the model you're using. Now, if you're using, for instance, Gemini or OpenAI or anything like that, you're going to have the limits from OpenAI, and obviously you're going to have the cost and OpenAI. And obviously, you're going to have the cost and the price. Yeah, that makes sense. TikiBoy asks if you have different structured and unstructured data, so PDFs or like your buckets. There are three buckets you showed. Is there a difference in the process or is this just working seamlessly um so it's all working seamlessly i would say the only difference is going to be on the extraction of the data so as you saw we do have different sources we do have youtube wikipedia pdf and then google cloud storage in the end of the the the end goal is basically the same, is to have a document, a document object, and then split it into chunks of text, basically. So what's going to change is if you load data from your Google Cloud Storage, we do only support PDF files, so after we took all the PDF files, listed the PDF files, the process is going to be the same, because it's the same as if you drag and drop pdf files but for youtube and wikipedia as i was saying we get the text from the transcription or from the wikipedia pages yeah cool there are some more questions um but i don't think i don't know what else do you see Ollama yeah lots of discussions on Ollama so yeah I mean if you're interested in Ollama if that is something you want to do especially for local execution then as Morgan said you can happy to contribute I guess if you are Ollama expert then probably this is easy for you and then you can get it hooked up quickly and then you know maybe in you and then you can get it hooked up quickly and then maybe in the next edition you can see the Ollama working. Lots of love, especially for the project. Lots of hugs and high fives. So really, really good for what you did, what you built here. So that's really cool. Thanks. And I would say, so going back to this application, so we do have the self-hosted public version that you can use. So I would say if you really like the small demo here and you're interested in that, just grab the link, the self-hosted version, use it, just load whatever knowledge you want into it. We don't store anything on our end. So obviously everything is going to be stored on your Neo4j database and just play with it. And if you don't want to run it with our self-hosted, just grab the repository, run it locally. But yeah, just play with it and give us any feedback. Yeah, exactly. That's a good point if you if you discover something if you have uh something like like a bug or if you see something that you'd like to like to see added then the community is there and maybe you can suggest it or maybe you build it yourself and and add it to to the project so that would be cool and it's it's really cool to see that would be cool um and it's it's really cool to see um and uh great great stuff so thank you very much for for your work uh morgan and niels on on the application and for the presentation today for the cool demo for uh for the cool uh you know intro and and for the work you did and yeah here see the first fork already happened so So J Banks, cool. Good luck with that. Have fun and let us know what you make of it. So yeah, that's what we like to see. That's cool. Yeah, before I let you all go, I like to say next stream on 14th of May. So in two weeks time, it will be with David on vector search. So we go a little bit more into detail on how to work with vectors in Neo4j. So that's maybe an interesting session, a little bit like a, not necessarily follow up, but goes into the same direction as a little bit where we talked today. Next up, obviously, call for papers for nodes. So until 15th of June, you have time to submit your graph stories for notes 2024, which is happening later this year. So if you are interested in presenting, if you want to showcase something cool, like what we've seen today or something completely different, there's a broad range of topics we're going to looking for, not only Gen AI stuff, but also anything's on graph applications, graph visualizations, and more. So if there's something that's interesting to you and you'd like to show the world, then please do submit. We like to hear from you on notes. And if you are now wondering what to do next, then check out the community, either on the forum or on our Discord server. Check out the Graph Academy for learn lessons and courses and do get your free instance and play around with Neo4j and graphs. So yeah, with that, I let you go. Thank you very much again. Thank you, Morgan and Niels for the cool demo, for the cool application. I hope you all have uh have a have a good time with playing with it and uh and see what you can do with with the knowledge graph um application builder and yeah i hope to see you soon at another live stream um until then take care everybody and see you soon thank you bye-bye bye
Neo4j Live: Knowledge Graph Builder App
Let's turn PDF documents into dynamic knowledge graphs stored in Neo4j with an easy to use app!

This innovative application harnesses the formidable capabilities of OpenAI's GPT and Diffbot LLM to seamlessly extract nodes, relationships, and properties from PDF text content. Using the advanced Langchain framework, we transform this information into a structured knowledge graph, empowering you to visualize and explore your data like never before.

Learn how to effortlessly upload files from your local machine or S3 bucket and select the perfect LLM model to craft your knowledge graph.

Guests: Niels De Jong & Morgan Senechal, Neo4j

Github: https://github.com/neo4j-labs/llm-graph-builder
Neo4j GenAI: https://bit.ly/4eRHS6g
Neo4j GenAI Ecosystem: https://bit.ly/4cpAY6D
Neo4j AuraDB: https://dev.neo4j.com/discover-aura
Public App: https://llm-graph-builder.neo4jlabs.com/

0:00 Intro
8:30 Neo4j + LLMs
15:30 Knowledge Graph Builder
24:25 Demo 
48:20 Summary
51:00 Q&A
1:02:35 Wrap-Up


#neo4j #graphdatabase #llm #knowledgegraph #langchain #openai #pdf #graphrag