 Hello everybody, welcome to our Raghack live streams. Raghack is a free global hackathon happening from September 3rd to 16th. RAG is a technique to get a large language model to answer questions based off your data. We have more than 25 live streams teaching you how to build RAG apps with different Microsoft technologies plus 10 cash prizes for the best RAG apps in each category. So we're super excited to have you join us the best RAG apps in each category. So we're super excited to have you join us for this RAG Hack. And now a word from our friends at Reactor. Hi everyone. Thanks for joining us for the next session of our RAG Hack Intro to Graph RAG. My name is Anna. I'm an event planner for Reactor joining you from Redmond, Washington. Before we start, I do have some quick housekeeping. Please take a moment to read our code of conduct. My name is Anna. I'm an event planner for Reactor joining you from Redmond, Washington. Before we start, I do have some quick housekeeping. Please take a moment to read our code of conduct. We seek to provide a respectful environment for both our audience and presenters. While we absolutely encourage engagement in the chat, we ask that you please be mindful of your commentary, remain professional and on topic. Keep an eye on that chat we'll be dropping helpful links and checking for questions for our presenters to answer our session is being recorded it'll be available to view on demand at the youtube channel for the microsoft reactor with that i'd love to turn it over to our presenters thanks so much for joining us today. Thank you, Anna. Hi, I'm John Alexander, and with me is... Orvamody. Sorry about that. That was a little awkward. Today, we are going to talk about intro to GraphRag. And I'm going to take the first part of this and start off with a more traditional way to leverage knowledge-based retrieval augmented generation and we're going to highlight using python with um lang chain and neo4j and then apura what are you going to talk about thanks john i'm going to talk about microsoft's implementation of Graph Rag, the research that we've done within our MSR org, and how you can use that via a Python package or via an Azure Solution Accelerator. Very excited to share. That's great. And so we're going to jump in now and talk about what's next. So I'm going to start with some graph fundamentals, and then we're going to get into how we query the graph and walk along it using a language called Cypher from Neo4j. And that language really is kind of akin to ASCII art, the way it displays the different ways that we relate to entities entities things like that so you might find that interesting then we're going to jump into a little bit more about specifically how lang chain and neo4j work together and some of the extensions that neo4j has created to work with these with laying chain and then aprva is going to take you on a magical ride to Microsoft Graph Rag and how that can just change your life for the better. Is there anything else you want to add to that or we good to go? Let's get started. Let's rock it out. All right. So when we think about, let's go here, let's talk about graph fundamentals. And so, you know, graph retrieval augmented generation is gaining momentum, and it's really becoming a powerful addition to traditional vector search retrieval methods. And you know, this approach leverages the structured nature of graph databases, which organize data as we'll see as nodes and relationships. And this helps to really build in the depth and contextuality of retrieved information. And so let's start off with nodes and talk about those a little bit. And when we're thinking about this, these describe entities, really nouns, person, place, thing, the discrete objects of a domain. we're going to use tom hanks a little known actor and we're our first example is going to be using a graph movie database which is kind of cool now when we're working with nodes they have zero or more labels to define what kind of nodes they are. Tom is a person and an actor. He could be a director, you know, whatever that is, it's what kind of nodes they are. And then they can also have in Neo4j properties, which are key value pairs. In this case, there's a name property. And then when he was born. So in it, so nodes aren't very powerful by themselves. We need relationships, and everybody needs relationships, right? So relationships connect a source node and a target node, and they have a direction, one direction. And they have, actually, the direction is the ingoing, the outgoing, not the type there. But they have one direction, they're ingoing or outgoing. And then they have a type. In this case, the type is Tom Hanks acted in a movie. And they have properties. In this case, the property for the relationship is the role for that. And so the relationship defines, well, this defines how those two nodes work together or how they're related. And so if we combine this together this is a graph database and it uses these graph structures these nodes and relationships and edges and properties to represent and store data and when we think about complex relationships and connections it really gives us a way to express that a little bit better or actually better than traditional relational databases and this is a screenshot of um a tool called bloom in neo4j that allows you to see the graph itself. So really it's a knowledge base that has everything. The model is a graph structured model that represents and operates on data that way. Or we can also think of it as our entities are interconnected and then defined by a schema. And so the power of knowledge graphs gives us the ability to capture those conflicts or relationships and concepts that we've talked about to give us more semantic meaning about what's going on. And it makes it easy-ish to combine multiple data sources because it's schema-less. And then the query language that we have, Cypher, as we'll talk about, is specifically created to help us find the patterns inside the knowledge graph. And really, there's a term that helps us, and it's called ontology, and it provides the foundation, foundational blueprint that guides how everything is represented in the knowledge graph, how the entities and the relationships are pulled together. And so now let's take a look at this. I'm going to grab a quick drink here. And we're going to start with the cipher and kind of look at the graph and kind of see some query types, things like that. So I'm going to drop out of the slides real quick. And so I've put some prereqs in here. There's also a GitHub repo. The link to it is in the resource part of the slides, which will be available. But we're just going to be working with what's called the default movies database in Neo4j. And I'm going to do it locally, but you could use a cloud instance to do that. We've loaded up our libraries here. And let's talk a moment about what Cypher actually is. It's a query language specifically for climbing, walking, traversing the graph, and it really, it's similar to SQL, but it's designed for graph databases, and it, like I said before, it's really designed to express those graph patterns that we can then use to really kind of tease out what we want to find in terms of the answers to the questions about our data. So the first thing we'll do is connect to the graph database, and we do that using the Langchain Neo4 Graph Connector. And now with Cypher, notice that we have a couple of keywords here that we're using. In this case, I want to match all the nodes in the graph. I want to grab those and return those. And so let's run that. And we can see through our query of our graph that we have 169 nodes. Well, that's okay. But now we want to match certain entities using the node label, find all the movies, everything that represents as a movie, not as an actor or anything like that, and return the count of those movies as the number of movies there. So we can see that we have 38 movies in there. And then we can also take a look at people in there. And we have a number of people in there, actors, directors, writers, things like that. And now we can also match specific entities based on their properties. Look for this property inside of an entity and we'll use a friendly name in this case tom to return that and so we can see here we have our entity of person as tom and our properties there So we're walking through that graph. So we can match a single movie using the title property, kind of the same way there. Tom Hanks acted in Apollo 13, but we don't know that yet. So we can also then start doing things with ranges, just like you would in any other data, but it's kind of good to see how we would do that there i'm matching all the movies and i'm calling it 90s where they were released between 1990 and 2000 so let's see that query and notice that we're returning the 90s title there, we could also say, and let's rerun that. And so we can see when things were run there and all the 90s movies, you've probably seen some of them. Now we can also do something interesting and here we're showing actually the relationship where we're wanting to say look for this person with this property and we want to find out all the relationships acted called acted in which is our type and what is um the movie what is the what is the appropriate movie where the person acted in that movie with tom hanks and so we'll see that n Lane, Jim Broadbent, all sorts of stuff there. Julia Roberts, Helen Hunt. Okay. Now here's even something even more that we can use to say traverse how far out in the graph we want to go with this range here. And you may have heard of the game called Six Degrees with Kevin Bacon. And basically the thought is Kevin Bacon has been in so many movies, you can pretty much find anyone within six hops of him. Well, in order to save time, I'm going to just use three hops. And I pulled this from the NeoGraph examples repo, which we've also linked to, which is a good place to get started if you want to know more. So we can see here, and they've basically also got a mechanism in here called Hollywood, that anybody that's within three hops, whether it's movie or actor, that have worked with Kevin Bacon, we can see that, and we can then use that as well. So it brings all of that back okay so what we've seen here is that we can use cipher the query language to be able to traverse the graph and to also use start using lang chain to be able to well actually um to be able to do that okay and this once again this notebook is in in the repo and also there are several links to web pages or web resources including the the repo but then also i've got or we've put in the link to a little bit more about cypher and walking through that and the more about the graph database concepts because we've you know only got a limited amount of time but all sorts of different things here and then we've also got um relevant links for microsoft implementation of graph reg as well, let's keep going. One of the other things that is a great resource is in the Medium blog, in the Neo4j developer blog, Tomas Frontag, I probably said that wrong, has a great, really great series of blog posts and then also some really interesting datasets and notebooks that you can take advantage of and the one that we're gonna be working through I've kind of modified but we're gonna take an example of that and walk through line chain and neo4j to just kind of see what's going on there so let's keep going. But these are some really good resources for further learning that are in the links. I just wanted to touch base with that. Okay. So one thing I do want to show is this is Neo4j desktop real quick. And we've got our example project here and this is our movie dbms and if we want to look at the link itself there are several apps that we can use here including the browser where we can interact just on the web and put cypher in there directly the one i want to take a look at right now though is called neo4j bloom and what that allows you to do is view the graph itself and then you can also work with it and play with it and you can also edit it right online as well and here it comes Let's pull that up. And now I'm going to go ahead and see our graph here. And notice we can see, like, for example, Keanu Reeves, how he's acted in the matrix reloaded, the replacements, all sorts of different things there. We can hop around. We can see when it comes to the matrix, the Wachowskis. I've said that. Oops. There we go. There we go. Okay. Ah. Here we go so we've got a few good men here we've got all sorts of different things in here the matrix there we go and you can see that there are relationships that are going in both directions some are going to movies acted in directed produced and so we've got a lot of different relationships that we can work with and that's a little bit about bloom and the example project in neo4j desktop i also have links for installing this either in the cloud or or on the desktop and then how to get started with it as well okay so let's go back over to the slides and let me start a different database real quick as well. Let's start up the slides. Here we go. And so one of the things to think about, let's go back to the other slide. There we go. All right. Is the knowledge graph, and this is really, I'm just going to go through this quickly. It's for more reading. If you want to come back to it, I thought I'd put it in here. It's a little wordy, but we have we have concepts categories or types of information within the domain you know healthcare if that's the case we might have disease symptom medication patient and specific entities that belong to a class might be diabetes might be an instance of the class disease aspirin might be a class instance of the class medication we have properties and attributes for example in medication has dosage and then we have the relationship and they could be either hierarchical relationships like is a or more specific association like treats headache or symptoms or things like that. And then we also have constraints that can help us bring things back like something distinct or every, and it also, excuse me, it also allows us to set rules. For example, like every medication must have a dosage so we can set that in there. Okay. So when we think about Langchain and Neo4j, we've got some interesting additions to the capabilities of Langchain. And you've probably seen all about Langchain in various sessions that you've gone through during Raghack. And so now we've already seen that we have a database connector that we can work with our graph with. We have a graph document that allows us to have the types and relationships or entities and relationships in there and in the form of a graph. Then we also have the new graph transformer, which allows you to transform documents into graph-based documents using a large language model. And we'll see that as well. And then Neo4j allows you to do graph-based keyword searches and allows you to do vector searches on the same data as well. And so we're going to take a look at the vector store, the Langtang Neo4j vector store, but it also requires the Neo4j Python package, which I've included in the requirements file on the repo as well. Okay. So let's jump into this. Come in here. And notice here we've got our Neo4j database. We've also got a utilities Python file here for configuring And then connect to our Neo4j driver. And then we'll go ahead and load our libraries. And then connect to our Neo4j driver. And then we'll go ahead and load our libraries. And then connect to our Neo4j driver. And then connect to our Neo4j driver. And then connect to our Neo4j driver. So we'll go ahead and load our libraries. And then connect to our Neo4j database. And then we can, the way this works is we can, through an empty database, we can use the Wikipedia loader from Langchain to go out and grab raw documents, much like we would with any other way we're using data within our retrieval augmented generation application. And then we wanted to find our chunking strategy, split the documents into chunks based on that chunking strategy. Now, this is where it changes. We're going to construct a graph based on those retrieved documents. And we're using the LLM graph transformer to do that. There's also an online version of this that you can just drop your documents in on Neo4j as well. But this allows you to use the same type of thing right from within LangChain and so basically we use we set up our LLM here and then we create our LLM graph transformer and then we use it to convert the documents that we just chunked out to graph documents and through this then we then add those documents to the graph through the add graph documents method. And the base entity label assigns an additional entity label to each node. So we have further metadata about it. And then the include source parameter, it links the nodes back to their originating documents. So also right now the LLM graph transformer only supports open AI and Mistra function calling models. So keep that in mind. Now, once we do that, we can then inspect that graph with a library called y files and it's a neat little library you may have used it before we can show the graph there and we are looking for 50 mentions of our relationships between people and movies so let's run that okay it's live radio so that's always good let's make sure this loaded I forgot to actually run that, so that's okay. Let's take a moment to do that and what i used to tell um clients when we were working with things the slower an application is the more powerful it is so now we have our documents defined defined there. Create the graph based on that and take a look at it. Oops, I just ran that again. Okay. Should be okay. And we can see our graph here and we can see all the relationships from Queen Elizabeth we pulled out a Wikipedia and it automatically created the relationships, pulled out the entities on many different events and people that were around Elizabeth. So now that we've created the graph, let's go ahead and work with, see some more sophisticated things with Neo4j. And in this case, we're going to use hybrid retrieval. And so when the user asked a question, our rag retriever is going to use both keyword and vector searches to search unstructured text and combine it with knowledge graph information and then that collected data is given to the llm as the new context and it's going to generate that answer so let's take a look at that and so the way we can create the vector index the vector index we can use the from existing graph method and that's going to add both of our keyword and vector retrieval to our documents and then we can use it for a hybrid retrieval approach run that and then we have a graph retriever here that we're using with a query and we're going to use that to identify relevant entities in the input because it's important to be able to pull those out to then extract that and then create the relationships from and so we're using the lang chain um expression language with the with structured out method to achieve this and so in this case we're creating a class called entities and then a description there and our chat prompt template telling it to extract organization person entities from the text use the given format to extract information from the following question okay let's run that that. And then now we're going to test it. And I know that Amelia Earhart's in there, but really what we're just trying to do is to make sure that in the question, we can use the retriever to detect entities. So let's do that. And we can see that, yes, it detected the entities. So now that we can detect entities in the question, we're going to use a full-text index to map those entities to the knowledge graph. And so we're going to define a full-text index first, and then a function that's going to generate full-text queries that also will allow some misspelling and then once we have that our structured retriever is going to collect the neighborhood of entities mentioned in the question and so we're grabbing our query nodes we're gonna say with node we're gonna match those with the neighbor and then pull that information back we're gonna limit that to 50 so let's go ahead and do that and then we can use that to print out our entities and then retrieve the neighborhood of relevancy there. So we can see Elizabeth had relationships with Lady Jane Grey, James VI of Scotland, Mary I, member of House of Tudor, and then also Queen Elizabeth was a symbolic object, and courtiers commissioned paintings for Queen Elizabeth things like that so we can see those relationships they're all structured there and now we're going to take these these two retrievers and create a final context that we're going to pass to our llm and we can see here here's our search query there and our structured data for our structured retriever our unstructured data and then our document we're just going to go ahead and concatenate that and join it and then now we've we have ourG retrieval component for the hybrid search. And then we're going to go ahead and do a more advanced RAG technique, which is query rewriting. And notice here that in our template, we're saying, given the following conversation and a follow-up question, rephrase the follow-up question to be a standalone in its original language so we have our chat history here our follow-up input and then we're using that as a constant called condense question prompt here we're setting up our chat history and then we're setting up our search quiz our search query here and checking to see if it includes chat history. We're going to condense that with the follow-up question. So now, once we've done that, we want to introduce a prompt that leverages the context provided by that integrated hybrid retriever we just created to produce our response to complete our implementation of our rag chain. So got a quick prompt in here, use natural language and be concise. If you don't know the answer, you can say, I don't know. And we're going to run that through our chain. And so let's go ahead, pull that in, and then which house did Elizabeth belong to? Elizabeth first belonged to the house of Tudor. Now that put some history in our chat history, and let's see if it will rewrite this query and get the context from our chat history and let's see if it will rewrite this query and get the context from our chat history so we did that it did our query rewrite and it shows that elizabeth was born on the 7th of september in 1533. so what we've seen here is a way to use neo4j and lang chain to not only use traditional vector search capabilities within rag but also use full keyword searches um to be able to transverse the graph that way combine that and then send that to the llm and bring back the information, the pertinent information. Okay. So let's go back to the slides real quick. And there we go. And now it's time to take it to the next level with Microsoft's implementation of GraphRag. And so do you want me to run the slides or do you? Yeah, so we can just talk about this real quickly. So and thank you, John, for that awesome background. I think for folks who aren't that familiar with knowledge graphs, that's a great way to get started. And the key that we want to kind of take home here is the relationships matter, right? In life and in data, right? So knowledge graphs gives us this ability to get insights and reasoning through relationships. So when we talk about graph rag and knowledge graphs for rag, there are a few things that we see as distinctions between that and baseline right so if we think about baseline rag it's a great way for an llm to be able to go out and find information typically on specific questions and bring it back to reason over and give you an answer, right? And it's using vector similarity search. So you have your corpus of data. It gets vectorized. The embeddings happen. The chunkings happen. And usually it's just a semantic similarity search that's coming back and giving you some answers. So if you had three pictures of cats and one is a live cat and one is Garfield and one is an abstract Pablo Picasso painting of a cat, before we had semantic similarity search, before we had the ability to have LLMs identify these sorts of things, they might not have come back as if you did a search of tell me all about, or show me different pictures of cats. What we're doing with knowledge graphs is now taking that to another level and basically saying relationships between things. So you can tell me what three different cats look like, but what if I have this enormous corpus of data and there's all these interrelated portions and entities that John talked about and the relationships between them. What if I want to find and have reasoning over higher meaning in that, themes in that data, large topics, narratives in that data. That's something that even semantic similarity can't do. Semantic similarity, if you ask what the top five themes of this data are, it's going to look for something that sits in that vector space, but it's not going to be able to find an answer. That's where GraphRAG and Knowledge Graph or RAG kind of come in. So now I'll switch over to my screen here. Okay. And I'd like to give you a little bit of background. So Microsoft GraphRag is a project that was started out of our Microsoft Research Organization by a group of very smart researchers who deal in knowledge graphs every day. And basically it was to answer this question of, well, if traditional RAG is not giving me the answers that I want, and I have these relationships in my data, how do I get reasoning over that? So unlike traditional RAG, what's happening in graph RAG is this. The corpus that you're interested in reasoning over is pre-processed. And by pre-processing, I mean it is sent to an LLM, and an LLM looks over that entire corpus of data and given some instructions, it's able to create the knowledge graph for that data. Okay, so, and we're going to go through an example of this, but basically what's happening is the LLM is very smart, right? And it's able to look at this entire corpus of data and based on what you as the humans want it to extract it can say i'm going to extract these entities i'm going to extract these relationships i'm going to extract these nodes i'm going to create a knowledge graph all by myself you don't need to do this human let me take care of the hard work and based on that i'm going to be able to give you the ability to query on that knowledge graph and get this higher level reasoning that we talked about, right? And that higher level reasoning can be the things that we discussed in terms of thematic sorts of things, looking at much bigger picture sorts of questions that even semantic similarity can't bring back to us. So you'll, we'll have a can't bring back to us. So you'll have a number of resources that are available. So you'll be able to get to these websites that we show here. Let's switch over and let me run a little bit of a demo here. So everybody can see my screen. What I'm going to do is I'm here on my on my local PC. And nothing special about it. It's it's just running Windows Subsystems for Linux. So this is this is Ubuntu now and there's a couple prerequisites and I hope they're relatively easy ones that everybody has. You need to have Python. 3.1.0 or higher and you need to have some access to an LLM. In my case, I'm using Azure OpenAI. You can also use OpenAI. Once you have those two things and your laptop here, then we're good to go. So the first thing we're going to do is we are going to create a virtual environment just so I can show you like how we would run through this entire thing, right? So I will create this virtual environment. And once it happens, we will activate it. Now we are in our virtual environment and real simply we are going to pip install GraphRack. And you will see it is going up, collecting dependencies, installing all of this. And you will see there is a number of interesting modules that are working on the back end here. Some are data science modules, some are visualization ones, some are customized modules. But GraphRag is an end to end process. So the indexing part of this, the knowledge graph creation and the querying are all included in this one big package. And truthfully, that's a misnomer because it's not that big. It's a very portable sort of package. So we're gonna let all of this install. And once that's complete, give it another minute or two here. We'll get started. So while this is completing, I'll tell you a few things about the models that we use within here. So typically, the types of models that we use are GPT-4 class models, right? We've tried different models and things like that, but the best ones have been the GPT-4 class models and in particular we're going to be using GPT-4-O-Mini, which is just as fast as the big GPT-4 models, but much cheaper, somewhat more efficient, but it reduces your token cost by quite a bit. So we're done with our installation here. We're going to make a directory here, GraphRack, that already exists. So we'll make a directory called GraphRack2. Okay, so let's go into that directory. And the first thing I'm going to do to initialize this is just give it a simple command. And this is what's happening for me to initialize my graph rag installation. Okay. So you see I'm using graph.index in it and my root directory is right here where we are. And then in a minute here, what it will do is tell me that it has initialized this in the local directory. Alright, so initialized project, if I do a quick LS, I see it's created a prompts directory and a settings.yaml. I need to copy a different settings.yaml in here and let's take a look at what's in that file. Okay, so at the very top of this file, I have my pointer to an API key right when I don't expose an API key I use my API key actually my Azure context because I'm logged into Azure with my account and it can pull the API key without me showing anything here I'm using Azure open AI chat and the model I'm using is GPT for a mini if we looked further into this file, you'd see an embeddings model, text embedding small. You'd see some other things as well, but there is definitely some flexibility here. So I have my settings set. Now I need to initialize some environment variables. So these environment variables are telling me, okay, where is your OpenAI base? What API version are you using? Your deployments, et cetera, right? And my key is blank there, which is fine. So now what we want to do is get some data in here. OK, so I'm going to make an input directory, which GraphRack typically looks at to look for files. And we're going to bring in some data in here. And what we're going to do is bring in a book called, A Christmas Carol. So for those of you who are familiar with Charles Dickens, classic English author, he wrote a book called A Christmas Carol. It's about the main character, Ebenezer Scrooge, who's a little bit stingy, gets visited by some ghosts, has a big change of heart, and becomes a very good person at the end. Okay, so there's lots of interesting things going on in here. So we've brought this in here. So if we kind of look at this, we're going to take a look at what our encoding is. So GraphRack is best on narrative data and typically in UTF-8 text format. So if you have narrative data in UTF-8 text format, this is kind of what you want it to look like. And if we were to look at, if we were to kind of cat this whole book out, let's just take a quick peek there. You can see this is the entire, you know, well, we're seeing a lot of Project Gutenberg stuff here, but you can see this is A Christmas Carol, right? The entire book in its entirety because it is in the public domain. All right. So now what we're going to do is we have our data that we're going to look at. Now we want GraphRack to look at that data and to start going over with the LLM. The LLM is going to look at this book and it's going to start extracting stuff. So I'm using this graphrag.promptune and I'm using this this this flag called no entity types. So what I'm doing is letting graphrag do all the work. The LLM is going to do all the work for me here. It's going to extract a whole bunch of stuff through here. So let's let that run. Okay. So we see here it's generating domain. Let's see what it comes back with. Literature and social commentary. Wonderful. So if this came back and it told me the generated domain was finance and accounting, that doesn't really make sense for Christmas Carol, right? But LLMs, like we said, are smart. They're good at looking at text. Our research team did a great job of making this very easy to extract things from there. So now we have written a bunch of this information out into another directory here. So let's take a quick look. We have this prompts directory directory so if we look at prompt we see some interesting things in there right and what we're going to do real quick is look at this these summarized descriptions right and you'll see that like a lot of this is um you know looking at the way the prompt that the LLM use. You're an expert sociologist specializing in literature and social commentary, analyzing community dynamics. This is going to look at this entire book in this sort of way, and it's going to create a whole bunch of, the knowledge graph gets extracted and created, it's going to be based on all this, right? So, so far this has been very, very topical to our domain here, which is literature. So the next thing I'm going to do is let GraphRag run through and index my entire document here, right? And this is, again, another interesting part. We're going to see some cool stuff happen here. So you can see all these chunks that are, as the vectorization and embeddings happen, all these chunks are being extracted. We have these base text units and extracted entities that are happening. This is all happening right now on my laptop. I can't even open 10 tabs on my laptop, and I'm running an entire LLM extraction and knowledge graph creation on here, right? So as we go through, we see that this is very, very, it's starting to summarize descriptions. A lot of this is part of the knowledge graph creation and then taking those chunks and entities and putting them in certain areas. And you can see we have a lot of, again, interesting stuff going on. If I scroll up here, you can see we have some things that are the relationships that are being created here based on the things in the text. And we're going to let this go through, create its community reports. And again, these community reports are artifacts that are created alongside our K files, which the index actually sit in, and different sorts of vector entities and things like that, right? Currently, this is all sitting on, like I said, my laptop, right? All workflows completed successfully. So all of a I said, my laptop, right? All workflows completed successfully. So all of a sudden we're done, right? This entire book, let's ask it a question. So we'll ask it a pretty simple question to start, right? And this is something that semantic similarity might come back with, but let's see what happens, okay? So what's happening here is I'm querying using something called the global scope of query and it's sending this question to the LLM and based on the indexes and the other artifacts that I have right here on my laptop, it's going to answer this question. OK, so let's see what it came back with. So key relationships, the Cratchit family. So Bob Cratchit and his family, Jacob Marley, who was one of the ghosts as deceased business partner, the spirit of Christmas. This seems like a lot more of a in-depth and contextual answer than you would get from traditional rag, right? Let's ask a weirder question, right? Let's ask, what are the most, most odd occurrences in the story. Now there's no way that semantic similarity is going to do that unless there's a paragraph or section that says odd occurrences. Our smart LLM is saying, you want me to bring back odd occurrences. I know what odd is, I know what that means in semantic space. Based on the indexing that I've done and what I've extracted from this graph, this is what I want you to bring back. Right. So let's look at it like odd occurrences in the story. Several odd occurrences that challenge traditional expectations and highlight deeper themes. Okay. Supernatural elements. There's probably nothing that says supernatural in the book. I'm not sure, but I'm sure it's not said that much. Symbolization and characterization. I mean, geez, this thing is practically writing a book report for you here, right? Absolutely. Here we have these the data reports. So this is telling you what chunks it used and the attributions that it used to come back with. So as you can see here in, you know, the period of four or five minutes, we went through an entire exercise of getting, you know, setting up graph reg, getting a data set, an entire book, having the LLM run through that, creating a large graph, and then querying it. So there's some huge opportunities here. I showed you a very small example. There is also a much bigger implementation of this, and we have an Azure Solution Accelerator that's associated with this as well. So if you need to do this at scale, the good folks in our SMT org, which is strategic missions and technologies, have done this and they've worked with a number of scientific and research customers. They've run work with intelligence agencies. Imagine being an intelligence analyst or a researcher and you have all this data out there, right? Like tons of papers, tons of different news articles and things like that. And being able to look at that, consolidate that and bring that back into something. And we can do that through our solution accelerator. And again, you'll get all of the links for this. I'll just show you a little bit of kind of, we have notebooks that are associated with this that you can see, and we're running a little low on time, so I probably won't go through this entire step through here because you've already seen it. But imagine this at scale. If you're part of an enterprise organization, you have pedified worth of data, right? right anybody. Do data discovery on on research or even kind of within your corporate history and things like that. There's huge opportunities here, right? And this is where the solution accelerator comes into play to really make it something that's a little bit bigger. We use things like Azure Kubernetes service on the back end, so it's a little bit more horsepower. We use Azure AI Search as our vector embedding store, so you can use AI Search as a high horsepower embeddings engine. But just like John showed, we also create a knowledge graph that comes out of this. I'm sorry, this is a little bit messy. It's not going to be as cool, but we can kind of see in here, we see some interesting things, right? And it might be a little bit difficult to read, but we see, let me see if I can bump this up a little bit. The green place, mansion house. In the middle, it'll be all the characters like Ebenezer Scrooge and stuff like that. And outside the graph, we see that it's still picked up some other things, right? Project Gutenberg, so all the legal disclaimers, all the other things were picked up. But that's outside of our main graph, right? So we know that this graph construction, it was smart, right? It kind of knew the difference between those two. So it's very interesting in that regard um so i think we have five minutes left here so we can i don't know if we do a do we do do we do live qa here do we answer questions do you want to wrap this up in some sort of nice way john sure um i'm actually if you can talk for just a moment i've created an aka ms link and I'm trying to get it to the proper authorities, which has the repo and the slides in it. Yeah. So let me talk a little bit more and maybe what I can do is show a little bit of what my Azure subscription looks like and some of the things that get spun up. So again, as I said, we have the easy one that we showed, right? The Python implementation that you can run on any random laptop. And then we have our bigger one here, which is my Azure deployment, which has an API front end. It has AKS. It uses Cosmos DB and storage AI search. It uses managed identities. If this is something that you want to play with again all available in our open source repositories um you'll be able to hit the graph rag uh repo and see both the python and the azure solution accelerator portions of this and we welcome you to try those and we would love to get feedback from you you'll see a email alias on when when you get the links for the graph rack site aka ms forward slash graph rag you'll also be able to add issues to the github issues on on each of these repositories right hey um we have a live question if you want it. Would you recommend data from different domains and input during the graph indexing process? Yeah, so actually to get started, we recommend having data in a single domain, right? So when we think about knowledge graph creation and these relationships that get created, what we don't want is a noisy graph. And a noisy graph is one that has information that might not be relevant to this particular domain or contextual to the questions that you want to ask over it. So GraphRank does have this idea of different communities and different layers within the graph. But to start, it's always good to start in a single domain. So let's say if we look at a corporate or enterprise sort of perspective, look at finance, look at marketing, look at those separately. Look at your research separately. Look at those separately to start. Start small. So start with 50 to 100 megabytes create a graph see what that graph looks like you can use the auto prompt tuning you can use manual prompt tuning i would start with auto once you're able to iterate get bigger add more stuff index again but start small see how it works out for you iterate over that and then go from there. And really quickly, Ashish asked, what if there are images in the PDF book? Are we only indexing text? Or can we handle that? Great question. So currently, we have a roadmap item to implement multimodality. So right now, we really work best with narrative sorts of text. JSON is supported as well. CSV for sure. But when we talk about images and video and other things like that, it's possible, but typically, at least for now, we have to kind of transcribe those over. Multimodality is coming on the way. We've seen some great improvements to the llms they're able to do a lot more with with multimodality right gpt4o is a great example of that um so i would say stay tuned right um follow the repo you'll see updates that come there and we'll we'd like to move to more structured learning more structured understanding and getting getting uh more on that roadmap there okay um can we switch over to my screen real fast so this is the repo for intro to graph rag and this was put the there's an aka link in the in the chat and basically you'll have you have the notebooks there um the pdf of the slides and with all the links at the bottom but then also um and a readme i have a little setup here recommended order of notebooks and then all the resources there as well. And so thank you all very much for your questions. And I'll stay on a little bit longer. There are a couple of questions to still answer. So I'll stay on a little bit longer behind the scenes and answer questions. Thank you so much, everyone, for your time today. Yeah, same here. Thanks, everyone. Hey, everyone. Hope you enjoyed today's session. If you aren't registered yet for RAGHACK, you can register now at aka.ms slash raghack slash register, and you can introduce yourself on our forum. You can start hacking either by looking at one of our code samples or just starting from scratch. If you have any questions along the way, feel free to post them in our GitHub discussion forum and you can also attend live office hours in our Discord. Once your project is ready, make sure to submit it before September 16th at 11.59pm PT to be eligible for our cash prizes. And of course, make sure to join our upcoming live streams to learn more about RAG. Thanks for joining. Thank you, everyone, for joining. Thank you to our speakers. To learn more about this series, watch past episodes and register for future sessions. You can follow the link at the bottom of the screen or in the chat. We're always looking to improve our sessions and your experience. If you have any feedback for us, we would love to hear what you have to say. You can also find that link on the screen or in the chat. Today's event code is 23432. And we'll see you at the next one.
Intro to GraphRAG
In this presentation, you'll gain an understanding of GraphRAG, a technique that enhances document analysis and question-and-answer performance by leveraging large language models (LLMs) to create knowledge graphs. We'll explore how GraphRAG builds upon Retrieval-Augmented Generation (RAG) by using knowledge graphs instead of vector similarity for retrieval. You'll learn how to set up the environment, prepare data, and implement GraphRAG using LangChain, with practical code examples. Additionally, we'll explore some advanced features and customization options available in LangChain to optimize and tailor GraphRAG to your specific needs.

Presented by John Alexander, Content Developer and Apurva Mody, Principal Program Manager

** Part of RAGHack, a free global hackathon to develop RAG applications. Join at https://aka.ms/raghack ** 

**📌 [Check out the RAGHack 2024 series here!](https://aka.ms/RAGHack2024)**

#MicrosoftReactor #RAGHack

[eventID:23432]