 Hi, everyone. Welcome. My name is Paco Nathan, and welcome to our first ever episode of Graph Power Hour. This is a new webinar series, which we're launching here at Senzing. And we're really, really looking forward to talking with you all uh presenting speakers interviewing people getting a lot of live q a but really understanding what's going on in the knowledge graph community which is adjacent to entity resolution we call it entity resolve knowledge graphs so uh again my name is is paco nathan we'll be doing this series. I'll be your host. And probably about every four weeks or so, maybe we can go a little bit faster. Once we start understanding, we've got a new platform here that we're working with, and we're all just learning it. So please be advised, but I think it'll be kind of fun. Now, generally speaking, I will be interviewing other guests or having them make presentations, and then we'll have live Q&A along with that. For this first one, I wanted to give a talk. So, I will be the presenter this time, but next time we'll have some other people. I also want to mention that, okay, we've got a little bit of a surprise. We weren't going to put this on social media anywhere, but please, you know, stay to the end. We're going to have a drawing. And so one lucky winner is going to get a prize. We'll spin the number wheel and show about this. And we'll let you know through email afterwards, but we'll give details toward the end, basically. All right. This episode here in our first ever Graph Power Hour, we wanted to provide a talk, which has turned out to be kind of popular about GraphRag, understanding about GraphRag and how to enhance applications based on large language models by using knowledge graphs. RAG has become a really popular topic in terms of preventing hallucinations, providing updates and facts and context sensitivity for how you're working with LLMs. And then GraphRag has come along a couple of years later to show how to use graphs to really even get better facts injected, better recall, for instance, better lift on the downstream applications. And we'll cover that. I wanted to give this talk, though, because when you look at what the graph part of GraphRAG means, I mean, there's at least a half dozen different definitions. And some of them are complementary or conflicting. It really depends on who you talk with, or rather, who is talking. So this here, this talk is a presentation about GraphRag, trying to break it down, understand where the parts are, understand what graph actually means in all those different definitions, so we can compare and contrast, and start to understand some design patterns for how this is built. The main takeaway though is you do need a graph. So how do you get that? And once you have the graph, what is you do need a graph so how do you get that and once you have the graph what do you do with it um these are some considerations and in particular when we're looking at uh use cases for ai applications that might be in regulated environments or someplace where you you really have to be careful about audits and provenance and evidence and all the rest. We wanted to give some air time to approaches for those kind of more mission-critical types of applications. And so that's a lot of the work that we do at Sensing. Really about 60% of our end-use cases are in air-gapped environments, so highly regulated environments. And so I'm particularly interested in how people work with knowledge graphs and with entity resolution for enhancing the data quality in particular for the graphs and for the downstream AI applications. All right, so we'll throw in a poll at this point, if I could ask our poll failure. Ah, great, poll is active. So we'd love to know who you are. We want to try to understand our audience here. So we've got just a few different polls, but there's one that's real quick. It's multiple guests, and we'll show some results later about it. Just understand who's in our audience and really what kind of themes and all. And while you're doing that, I'll give a few blurbs here to introduce myself. For what it's worth, about 40 years ago, against all advice, I decided to study AI in graduate school. And then I went and did a deep dive based on a friend's recommendation, but everybody else was telling me it was something that would just never pan out called neural networks. And so I got involved in doing neural network hardware accelerators in the late 80s. There was this thing called the AI winter. It's actually happened a couple of few times. I've done a lot of network engineering and other software engineering throughout the times when people didn't really want to talk about AI. And then oddly enough, back in the mid-aughts, I got a strange phone call out of Seattle from a friend and ended up being a guinea pig or rather they bounced some questions off me. And then I was later pointed toward a website to sign up. So I became one of the very early users of something called Amazon Web Services. And my teams were guinea pigs for evaluating new services coming out. We were a case study for what became Elastic MapReduce. We had run one of the largest HADF instances on EC2. So I really got to see cloud evolve from the ground up and be involved in those communities. A lot of early work with big data and leading data science teams back before that was really even a term. And eventually I got into Databricks when we could all fit in one room and was the director of community evangelism for Databricks back when Spark was going through hypergrowth in like 2014, 2015. And from there, I went to O'Reilly Media and was director of something called the Learning Group. We had a team of people who were really good at working in conferences and teaching technology topics. And our team would go and coach other authors coming in to help them improve tutorials or, improve tutorials or hopefully become a bestseller, that kind of thing. Basically building a lot of learning materials. These days I am working at Senzing. I'm in developer relations for the Knowledge Graph practice area. And so again, really focused on entity resolution, but all the areas that are adjacent to entity resolution, like where is it getting used and where do these practices need to be coupled with other practices? Okay, let's see. I wonder if we're ready for the poll. Can we show? Overviews, good, awesome. Do we have the results ready for the first poll okay awesome thank you very much cool okay uh 20 software engineer 20 data science excellent uh eight percent data engineer um ops six percent product manager six percent okay excellent this gives us a really good idea of the composition and it fits really well with the kind of topics we're going to go into. I mean, this here, this presentation in particular, it's a lot of pointers out to primary sources, or other tutorials, or just whatever kind of resources to various communities. If you want to dive into Graphrag, hopefully you'll hear some really good starting points to jump off. And for the slides that we're showing, we're going to follow up with slides that have full links that you can access as a web app and click right through. I often hear people say that this style of presentation, you know, somebody will come back about two weeks later saying, hey, I just went through all the slides and all the links. So hopefully there's too much information there. I want to start out with an overview here of what is retrieval augmented generation, what is RAG, and then what is GraphRAG, how do these parts all fit together. So let's see here. What's in a name? GraphRAG is something that's definitely been in the headlines a lot about AI. The notion is that you use a knowledge graph to enhance something called retrieval augmented generation, RAG, to for grounding the results that come out of a large language model, an LLM. So basically, LLMs are, I like to think of as sort of syntax engines. They're really good at recognizing being trained on very large scale amounts of effectively syntax, sequence to sequence with attention ahead. And they're also good at generating syntax to that extent. They're not especially good at reasoning. There are other methods that are fantastic for reasoning that can be paired with them. But what we've seen is that large language models can be really fun to generate things that may or may not be entirely accurate. And if you're doing in-painting with images, that's really fantastic. It looks great. If you're trying to give answers for somebody preparing a legal brief, that's probably not a good idea, as has been shown in court. So, you know, people want to use LLM applications, but they really want them to be grounded in a particular set of facts that might be specific to some use case or domain context. And so we'll talk about RAG, but GraphRAG is an enhancement over that of really how do we get the facts in there. And it's been gaining a lot of momentum since really about a year ago. It was 2023, September, when we started to see some of the early mentions of GraphRAG. There may have been others that I can't actually find in the literature. I'd love to know if there are earlier references to it. And definitely, by the way, I'm trying to provide primary sources. If you have better primary sources to recommend over what we're showing here, I'd love to hear that. One other thing, Graphrag, it means a lot of different things. We'll cover that. It also has a lot of different terminology. So you'll probably see graph rag with or without a space. You'll see KG plus rag. You'll see graph enhanced rag. You'll see Grag. You'll see a lot of different names that are roughly speaking talking about a similar thing. And so speaking of primary sources, when we look at rag, and we'll dig into what rag is or isn't, but the dates on this go back about four years. Kevin Gu at Google and Patrick Lewis at Facebook, these are two of the papers. If you look at the contemporary papers, these first two are the ones that are usually cited as primary sources, some of the earliest primary sources. And the work out of Google and Facebook on those are really compelling, and it's picked up a whole lot. Early primary sources for Graph Rag, as I said, only really date back about a year. As far as I can find, first mention that I've seen so far was from September 6th from Face Off, the people who do nebulagraph out of China. And they had a demo that they launched in a blog post along with some open source code. And it was really super interesting. The demo wasn't working for a while. It may be up now, I'm not sure. But the blog post is linked here. And then other people picked up on it i know that uh llama index and langchain and others they had instances of graphrag running in open source tutorials not too long after that uh there's a paper more recently out of last month uh by bossy peng which is i feel one of the better survey papers about what's going on with GraphRag. So that's the other one that's linked here. And just as, just a kind of level set as a, I don't know, it's, this is like a generalization, but I like to have some type of diagrams or textual diagrams, if you will, to work off of. And I'm not saying that every application would be using everything on this diagram. It's more of a generalization kind of approximate, but these are what you would expect to see in terms of the data flows and the different components that may be there. So when you're working with RAG, there's a couple of different categories of sources. Generally speaking, you'll see some structured data sources and some unstructured data sources. And that's a truism for working with knowledge graph in general, is that you start out with, say, some schema, ontology, taxonomy, controlled vocabulary, whatever, some type of schema to try to organize. And then you bring in structured data sources, and then you bring in unstructured data sources. And probably, you know, I mean, there's kind of a Pareto ratio, so there's like an 80-20 rule. You'll probably see like 80% of your data is going to be unstructured. So that's actually a really big component. But it's important to start out from the structure and move outward so again i i at least understanding what your schema needs to be is really very important as a first step and from there bringing in structured data sources and then layering out your your unstructured data sources on top um an approach not already does it but an approach that we would recommend is take your structured data sources bring those data records into your graph uh aligning them with your schema your ontology and then uh we work in entity resolution and so the idea is that if you have two or more structured data sources and for each one of those you have two or more features of what would be PII, so private information, so like names, addresses, telephone numbers, birthdays, employer, tax id, whatever. You've got some identifiers that may or may not be unique, but we can kind of triangulate on them. So if you look across two or more data sets and you have these kind of connected features that are appearing regularly in those data sets, then we use resolution to figure out like what are the consistent entities that span across these different data sets. And so we like to think of that as a kind of semantic overlay. We're not changing the data records. Instead, we're sort of watching from the side and then overlaying on top some connective tissue, if you will. We'll say, okay, here's some entities that describe these five data records, and then here's the relations between them that we think are appropriate. Basically pointing back to evidence to support how do we organize what's going on in the graph. So that's sort of the structure at the top, going from structured data sources into a graph. What you're going to see a lot in the GraphRAG literature really indexes heavily on the unstructured data sources. So the notion is in RAG, you will take typically some text and you'll chunk it, split it into, oh, I don't know, you know, five, 12 characters or so many sentences or some sort of chunking factor. But you'll take chunks of it and then for each one of those, you'll run those through an embedding model so that the text in that chunk is projected out into a very high dimensional vector space. We can understand where all those different chunks of text fit in that vector space. That's what drops into typically a vector database. And so now we can come in with some query, some prompt, put it into that vector space as well, and figure out what chunks are the neighbors that we should supply out to a large language model. That's the idea of RAG, but with GraphRAG, we're enhancing it with this graph on top. So one of the other things we can do, as you'll see in some of the examples that are run, is as we're chunking that text, we can also run it through parsing. Run NLP tools to parse that text into sentences and then tokens, and then apply labels, parts of speech, understand where the noun phrases, where the verbs connecting them, clauses, etc. From there we can use techniques like named entity recognition and relation extraction to be able to understand how to construct what is more and more called a lexical graph. That is to say we take the parse trees from parsing the text and stitch the common elements together and you come up with a graph. And you can do some pretty interesting graph analytics on lexical graphs to be able to understand things like, you know, centrality. What's the most common phrase that's showing up across these documents? But it's a way of being able to stitch together from our structured data sources to our unstructured data sources. If you are using entity resolution, which I highly recommend, then one of the things you can do is you can derive essentially I would call it a linguistic analytic asset. Like if you're familiar with WordNet, if you're doing NLP, that would be a good category. If you're not familiar with WordNet, I can just say it's kind of like a thesaurus. Like I can look up a word and I can find synonyms that are closely related to that word and i can look and say okay well a cat is a kind of feline and maybe a subclass for a cat is a kitten so now i've got some structure some hierarchy and i can navigate that so effectively any thesaurus is a way of navigating a graph or the semantics of a graph. If you have that, then you can perform something called entity linking. So you can take the graph that you've built out of schema and ontology and your structured data sources and entity resolution, and then you can make a context-sensitive entity linking approach to pull together what you've built out of your lexical graph. And in this way, you can tie together a graph, effectively what's in your graph database with what's in your vector database and have references from the chunks in the vector database in your graph and use both of these together. That's kind of the gist in general, if you blur your eyes enough, squint hard enough, that's kind of what's going on in most of the graphic approaches. And then downstream, once you've built out your graph and your vectors, when you get a prompt coming in at runtime, then it can go a couple directions. One is you throw your prompt through your embedding model and you get some kind of vector similarity search and figure out those chunks that are closest to what the question that's being asked pull that out of the director database you end up with a ranked index of here's the chunks that i think in priority should be fed to my llm you feed that the llm it does its its syntax engine thing and sequences together the different parts of the raw text uh to a response. But effectively, what you're doing is you're converting from a prompt in raw text and a whole bunch of text chunks in raw text and then coming out with some kind of answer in text. That's the LLM trick. But we can make that better. One of the problems is right here, the vector similarity search, it has some downsides. There are definitely some trade-offs. So we can juice it up. We can make it better by going in and taking the prompt, running it through our graph. We might take a query, we might do other methods, but effectively we're going to come out with some elements of that graph that resonate, that are really close to what that prompt is talking about. And once we find those, we may be able to go in and use our thesaurus, our word net, etc., that we've constructed out of entity resolution. We may be able to go and do some fancy tricks on the graph side, such as a semantic random walk, which is to say, if I've got a node that says cat, I can walk out and find kitten and feline and lion and tiger and other things that are in it that are related to it, but may not be necessarily in the vector similarity search. So we can find other things a couple hops out. We run some graph analytics. We build up this ranked index because we have links between these nodes and the vector database. So we augment this ranked index with what we found out of the graph and feed that to the LLM and hopefully get much better kinds of results. That's what has been shown. We'll give some papers about that. At the end of the day, a lot of what's going on with RAG, roughly speaking, it's very familiar if you've done recommender systems work. And so in a lot of ways, one could overgeneralize and say that this is basically reapplying some of the techniques out of recommender systems to help improve AI applications. And frankly, look at what's going on here. You've got vector databases. Vector databases, of course, have been extremely useful for recommender systems and for that matter there's been a long history of using knowledge graphs to enhance recommender systems search engines other kinds of discovery process and certainly wherever you look at where rexs are being used commercially at scale for very important applications you're going to find knowledge graphs very large large practices. This is certainly the case at Amazon, Google, Microsoft, LinkedIn's economic graph, meta social graph, Pinterest, on and on and on and on. And I didn't even mention Twitter. And I've known some of these teams, and they're doing fantastic work in knowledge graphs. We'll probably have some of these people on later in different episodes but if you if you do want to dig into rexus what's going on there um here's a link to a white paper uh i did some work with the recommender systems group at nvidia a few years ago we did a white paper interviewing the top nvidia customers in recommender systems talking about their their views of you know where this is heading. And this was right about the time that RAG was starting to come out and still a couple of years before GraphRAG. So give you a little bit of background. Okay, I think it's good. Let's throw in another poll. Let's go to our next poll and we'll do that while I dig in here on what does graph mean. Great. Okay. So graph reg has become quite a buzzword in industry, but the graph part of it actually means different things. And I wanted to break that down because I think it's very important, especially as you're starting to build applications some of these may fit with your use case you may need to mix and match um on the one hand you can talk about graph being okay i've got a bunch of text uh chunks i've got embeddings for them i can look at the neighbors for each text chunk and see the distances and here let me actually use my hands a bit to try to describe this. So if I have a bunch of different text chunks and they're in a vector space, I can look at their neighbors and find out how far away they are. And I can start to use those relations to build out a graph. And this is something you can do with pretty much any embedding model. We'll show some examples of this later on. But the idea, particularly if you look at the Microsoft Graphrag paper, that's what they're doing or that's one of the instances of what they mean by graph. It's really building something out actually in NetworkX from the chunks of text. Another way though is to think, great, I've got a prompt. I'm going to throw my prompt up into converting it into graph queries, and then I'm going to figure out what graph elements are closest to that prompt, and then I'll take some representation of those nodes and edges and properties and whatnot as strings and feed that into the LLM. And that works well, too. Definitely Ultima and others have been going down this kind of route. Another way is to say, OK, as I showed in the diagram, there's that whole workflow there of taking the text chunks, constructing a lexical graph, and then understanding from a lexical graph what are the best chunks of text to feed into the LLM? Definitely, if you look at Neo4j, a lot of what they've done really exemplifies a lot of this. And we'll look at that in more detail. And another approach is to say, okay, I'm going to build a knowledge graph out of structured data, blend in some unstructured data, and then we will go through that resulting graph to feed whatever text we need to into the LLM. And if you look at, for instance, Esri with ArcGIS knowledge, that's I think a good example of what they're doing there. And then, you know, we could say you could take any combination of the above, but use graph neural networks to expand up the nodes at any given point. So that's bringing in an entirely different technology gnn's um they they are expensive uh they were all the rage from like 2019 to about 2022 uh until chat gpt kind of wiped out that i shouldn't say wiped out but suddenly everybody changed their research uh description um but gnas were really popular they didn't really show up in industry for a couple few years later, but then in 2024, LinkedIn and Spotify both published papers about how they have revenue varying GNNs in production. So what I want to say is there's a lot of different meanings of the word graph, but you can mix and match them. And we haven't seen enough of this built out yet, but there are some examples of being able to mix and match components. And so just to give a little bit of background, this idea of building a graph off of embeddings, if you wanna play with some code, here's a notebook from the KG Lab project, open source project about graphs that we did, I think like four years ago. If you wanna see just even a simple example using recipes from food.com, we were able to use a word embedding model and build out a graph from that. And if you want to look at how to do lexical graph construction, we've got, here's a GitHub repo that shows this is for another talk that's related to this. But there's more. Graph has multiple meanings in this context. Another use of the term graph that you will see related to LLMs would be, for instance, graph of thoughts. And so this is a little bit more on the other side, not what you're feeding into the LLM, but rather organizing what you're getting out of the LLM. And so if you've heard of chain of thoughts, the idea is you take your prompt that you're gonna ask a question to the LLM. And so if you've heard of chain of thoughts, the idea is you take your prompt that you're gonna ask a question to the LLM and you decompose it into parts and you run your large language models to get answers for each one of those parts and then you reassemble them back together. There's another thing called tree of thoughts where you start to branch it out. And then there's graph of thoughts where you can say okay well let me let me try several different methods but also keep track of the cost because some of this stuff is really expensive and so by using a graph of thoughts i can decompose my prompt and i can start to work on the different parts start to reassemble them if i get down one branch and i find out it's a dead end, I'll just backtrack into some other. And also if I get into some branch and like I'm kind of getting answers, but gosh, this has already become really expensive, then why not use some of the less expensive branches first? That's a technique out of operations research if you're familiar with ORSA, that kind of approach gets used a lot. So cost optimization for time, money, accuracy. There's some really great work. Masih Vesta is one of the leaders in this field out of ETH Zurich, and definitely keep this in mind in terms of graphs with LLMs. But there's some lighter weight versions of this too, which I think are actually really promising. And this is where you're doing more graph reasoning. And so LLMs are not the best for doing reasoning. And we can show that. There have been some great studies recently. But there are other ways of doing reasoning. So the idea here is there's a collection of research, even dating back to 2019, this paper here. I really love this one called Brack's Wife Hilary, about how you can use graphs to try to avoid confusions that would otherwise be coming out of LLMs or other machine learning. The idea is use the LLMs to create logical prepositions, put them together in a graph, and then do reasoning inside that graph to feedback on the answers that would come out of the LLM. One of my favorites, I think that this area has a lot of way to go. And just to kind of close up on that part, a couple of friends, Ben Lorca and Prashanth Rao, tried to take a stab at defining what design patterns are being used, categorizing all the different types of GraphRag. What they came out with is, gosh, this is really varied. So we'll just have a sampler of what the popular techniques are but they really do give a lot of names to these different approaches and there's a great uh interview subsequent after this article came out uh ben lorka interviewed cto from neoproj a friend of ours philip rodley and uh he actually broke it down into a completely different kind of taxonomy so there's more and more discussion about what does graph rag really mean, and I wanted to point to these resources. All right, do we have the results from the poll? Let's see if we had the most recent poll that was going on. Ah, interesting. Okay. Entity resolution in conjunction with knowledge graph practices. Great. So it's 30-ish, 30 across the board, split into thirds, basically. Yes, in production use case is 27%, evaluating 38%. Okay, that's great. Because I mean, this will be a lot of the theme for our webinar overall. Now, let me start to address, there's a lot of questions of, well, you know, if we've got RAG and that's working, why do we need to go to the extra step of having graph rag um quantify can we quantify it as doing anything better so first off i just want to point out three surveys uh chia yang from why how uh my my friend and colleague uh there and also uh well all these folks are good friends louis and i are actually working on some projects right now. Hopefully, we'll have Louis on the podcast before too long. Webinar podcast, I should say. But Louis had a comparison of different types of RAG and graphs and language back from February, which was right about the time that Microsoft was publishing about it. So this was looking at comparisons of, say, vector rag versus graph rag versus hybrid kinds of rag. And then Philip from Neo4j has Graph Rag Manifesto, which really goes into detail. And also Emil Efrem, the CEO of Neo4j, has done some talks that are up on YouTube about this, going into a lot of detail. And also Emil Efrem, the CEO of Neo4j has done some talks that are up on YouTube about this, going into a lot of detail. I highly recommend those resources, but a good starting point is Philip's article about Graphrag Manifesto. I think that these give a lot of characteristics for how can we go in and start to quantify and measure. And so some of the top papers there to analyze the lift of GraphRag. Three, there's one from LinkedIn that is showing more than 20% median per issue resolution time lift for customer support at LinkedIn. So that's really substantial in terms of the benefits. There's another one from Glasgow talking about 14% lift on trace Q&A datasets. There's another one from Glasgow talking about 14% lift on trace Q&A datasets. There's another one from Emory looking at comparing different benchmarks for multi-hop reasoning. So I think that we're starting to see this quantified. There's definitely a lot of lift coming out of using GraphRag. At the Knowledge Graph Conference KGC, which was held in Manhattan in May, I got to host a track about understanding the lift of GraphRAG. And so we had three talks. One was from John Stevens at AbbVie, the big pharma in Chicago. They were showing their work internally using chatbots based on RAG and training with internal knowledge that their people need inside pharma, showing the lift of using RAG versus training with internal knowledge that their people need inside the pharma, showing the lift of using RAG versus GraphRAG. Great talk. I think there should be a video about that out. And then Atsunas Kirikov and Pia Popov from Ontotext, they showed what they're doing at Ontotext, which is one of the graph database vendors that really jumped in early on doing GraphRAG, especially even with RDF and L semantic web directions. Also showing the lid for some of their large customers. Then most importantly, Juan Cicada and also Dean Alamong and others at Data World, they've had a couple of papers coming out really quantifying this area of generating SQL queries by using RAG versus using a graph and using graph queries first, then to understand how to structure the SQL queries. So there's a lot more that's coming out formally about that. And just the big picture takeaway is that what we're seeing here is that GraphRag is addressing a problem that we would see otherwise in LLM-based systems, commonly reported about recall. Like you might ask a question once and you get a pretty good answer. You ask the question second time, you get a so-so answer. A third time, you get something that's just bizarre. Even if you use rag you still might see some issues with recall because the similarity measures that are inherently used in vector databases they're relatively shallow so graph allows you to traverse the graph and find the things that should be included that might be a couple of hops out that are sort of non-trivial or non-obvious if you will um semantic random random walks are a great way to do that, graph ML, graph analytics, etc. But it's leveraging domain knowledge. And overall the big picture here is that we're taking machine learning models, what we can use with them, more statistical approaches, and then we're combining that with more of say symbolic AI where knowledge graph of course is you know the big area of symbolic AI right now. We can put these together into more hybrid AI, which is a much, much better approach. And I've linked here, if you follow this link, there's a talk by Franklin Harmelin about symbolic and statistical hybrid methods. I think it's a really great perspective. Okay, let's try another survey. We'll go for the next one. And I'm going to scoot through some of this just because I think we're making pretty good time. But when you're using GraphRag, obviously, one of the things you need to have is a graph. And so that begs the question, how do you get that graph? And how you build a graph and how you use it, of course, are very consequential. So, in terms of the diagram that we had before, you're basically at the build stage. You are here, looking at the build, not the use yet. So, there are some really great tutorials about this, but a lot of what you'll find in the literature, in the open source, the tutorials, it's really focusing on using LLMs to build knowledge graphs from mostly unstructured data sources, building this automatically. There's really great demos here from Neo4j, LLM Graph Builder. I think subsequently they've moved it out of labs, it's actually part of product now. But point at a set of wiki pages build a graph about it and then start asking questions i i saw an early demo of this uh a few months back and uh it's really fantastic it's amazing what can be performed there is some cautionary advice um what has this paper is from a couple months ago now but from from july and uh colleague or people at bosch nai i want to shout out to some friends of us uh but they're working with you manheim and they're able to show um you know you can go to a lot of effort to train llms and fine-tune them for a particular domain i think i would uh reference uh wali kadus chief scientist at any scale he had a great talk last year at ai conference where basically training an llm is for form and structure whereas like fine-tuning or using rag is more for fact. So it's expensive to train LLMs, less expensive to fine tune, particularly if you're using synthetic data projections, but it's even less expensive to use rag to get new facts into your system. If you do rely on the LLMs to be doing your reasoning and understanding your domain, proofably we can show that doesn't work very well. And so this is one of the early papers that I've seen really, really quantifying some of the cautionary advice about over-indexing too far on the LLM side. You do want to use something outside to try to ground it. And maybe scaling laws will prove this differently later, but all the trajectories seem to point toward this. So there are better practices than just relying exclusively on LLMs. We work in entity resolution at Sensing, and most of our customers are in regulated environments, as I mentioned. And so the notion of using an LLM to automagically generate your graph through some black box approach, yeah, that doesn't go over well, especially if you're trying to catch bad guys and take them to court, maybe try to put them in prison. It turns out that judges and juries like to see evidence. They don't like to have some magical black box just coming up with answers. And also, you know, this speaks to the idea of investigations uh when you are doing any kind of investigation it's typically very iterative uh you you make some progress you go back and do some some work you bring in new data you make more progress and i don't want to the science in general is more about investigation than it is about prediction i think that that's a problem silicon valley has had over the last several years is they really over-index on this notion of prediction and sort of a cult of prediction, but really it's about investigation and that's certainly what we see with a lot of our use cases. So if you are an enterprise team and you've built a graph that you've, you know, you've taken time to curate it, you understand where all the different pieces come from, why not use that as a core asset? Why not use that when you're doing graph rag because in these really critical kinds of applications uh you know accountability evidence provenance uh being able to do audits and the consequences downstream these are really crucial so um you know i i want to explain it just real briefly about if you are building a graph, let's break it down, as opposed to doing it as just one magic LLM thing. Why don't we break it down into pieces that we can understand and that we can sort of audit and give some feedback on the different parts. And when you jump into this, you really need an explainer because you start to use some of the natural language work. You've got terms that sound a lot alike, but they're actually very different. So named entity recognition, that's where you've got unstructured data. You parse it. You build a electrical graph. Okay, great. Now, if you've identified a noun phrase, named entity recognition is a way of giving a label to noun phrases. So I can say it's a person, place, a thing. I can say it's a name of a country or a unit of currency, et cetera. But it's a noun phrase and it's come from unstructured data. Relation extraction, RE, is a way of saying, I've got a couple of entities that are going to go into a graph. I want to put a label on the edge between them, give it some kind of semantic relation. And that's still early days, but there's some really good things coming out of research for that. Entity resolution, where I work, is completely different from NER. Instead, you're starting with structured data, and you're trying to disambiguate consistent entities across data sets, two or more data sets based on two or more features that are connected. Entity linking is a way of actually blending how you can take structured results from structured data sets and entity resolution and unstructured data and named entity recognition and blend them together into a KG. You'll also hear zero shot learning used a lot. I definitely wanna qualify that. That's where I can take a deep learning model and be able to give it a set of classes to recognize at runtime without having to go and retrain it, which is very powerful. So the entity relation, or sorry, the entity resolution that we're talking about, it's a way of generating building blocks for a knowledge graph. So we can take and resolve entities across data sets and then come out with an overlay that says, here's some entities that connect different data records together. Here's the relations we think that should be between them. We'll add some properties if you want to trace provenance, et cetera. So kind of a way of preserving evidence as a semantic overlay. And this talk, plus a few other tutorials, plus some others that we've got coming up really all filled together they segue so there is a talk about using entity resolve knowledge graphs i did a tutorial a few months back and claire sullivan my colleague also did an excellent tutorial recently uh just came out about using that same use case that same data but then working with graph rag based on lang chain uh to to show what you can do with GraphRag from structured data sources. And Claire and I are working on some other tutorials coming up. Also, there is a sister talk that goes along with this. It's about how to construct knowledge graphs from unstructured data and how to bring that into your structured data. And that was hosted by GraphGeeks a few weeks ago, and definitely highly recommend that, especially the code. It might be very useful for you. We've got some case studies about people working with Entity Resolved Knowledge Graphs. And I want to give a shout out to that, because I think it really does start to give a flavor for what these use cases look like, what the practices are. And it's a nice counterbalance to what we see in industry, which i'll talk about next okay let's see if we have a poll to run uh we'll run the next one um i want to talk a little bit about graph rag in practice and then we'll wrap up and go to questions um okay so now looking at the architecture diagrams, we're at a different part here. We're talking about usage of what we've built from graphs and vectors. And in terms of GraphRack, they're really turning out to be, you know, this set of popular open source libraries. Certainly Lama Index, which has excellent tutorials. Haystack, of course, has gained a lot of traction. You see this all over. Really like what they're doing. Actually, from a code standpoint, I kind of tend to favor a lot of Haystack. Langchain is super popular, probably arguably one of the more popular ones of these. Although Microsoft, of course, has gained a lot of headlines. But they have open source for all of these. And I'll be MC at AI conference coming up next week in San Francisco. And I'm sure we'll see a lot of the principles from these different teams presenting. I'm showing on the right-hand side here with Lama Index, they've got a class in Python called Property Graph Index. It's a way of building up a knowledge graph based on a property graph that you can then plug into RAG. And so it's a quick way of building up a graph RAG solution. Haystack, Lanchain, Microsoft, they all have some ways in a relatively short amount of code. Although I definitely have some criticism about these libraries as well. So I think that it's still early days. Most of these libraries are really big and they kind of have everything about the kitchen sink. So they tend to be adding new things all the time and you get these really huge installs. And as far as APIs, I mean, the design patterns there are still kind of being figured out. So I do expect to see a lot of refactoring for all of these and really building it out into different components and starting to find, you know, then look in detail at each of those components. So that's a caveat that I would give about working with these. And frankly, for a lot of teams in production, they build their own. And so if you want to look at building your own kind of graph reg, you know, there's components that are definitely recommended. The graph side of things, NetworkX, of course, is kind is the workhorse of graph analytics. It's not a database, but it's very useful. And there's the GPU accelerated version of it, Kugraph, which is so powerful at scale. KuzuDB is another team I work with that does property graphs. And so I've got some examples in the links here of code repos on GitHub, showing use of KuzuDB, also very friendly to Python. And then Lance DB is, for example, a vector database I like to use. Also, I think that Kuzu and Lance are both sort of the DuckDBs of their respective worlds. Ollama is a really great way to orchestrate hosting your own LLMs locally or elsewhere, but basically model orchestration. So there's these parts that you need to have if you're going to build. And I'm going to reference a book coming up here in just a minute that actually goes through step-by-step building your own GraphRag. I want to shout out to a few examples. I think that Esri is doing really well with ArcGIS knowledge and what they're doing there exemplifies when you look at RAG or GraphRAG in general, arguably the heavy lifting is because of the embedding models. And when you look at multimodal embeddings, this is where a lot of the power is. I think that ArcGIS knowledge is one of the products that exemplifies this area. How can I bring in graph elements with geospatial elements, with some video streaming, with some segmented imagery from satellites, and then some other text that's come out of news reports, and pull it all together? And so here's a case study that our team at Cinzing did working with Esri, again, to pull those pieces together and go after illegal fishing fleets. So, you know, finding a fleet that's operating off the coast of Peru, checking its beneficial ownership chain through the graph, finding out it's actually being operated out of Beijing. And by the way, if we look at the graph pattern and use some GraphML, we can find similar fleets operating elsewhere in the world because of geospatial data. Highly recommended checking that out. Another one I think that's really good to look at is WhyHow. What they've been doing in Knowledge Graph Studio is a recent release, but basically RAG native Knowledge Graphs. And very interesting, they've just opened up their SDK to be able to import triples so you can build your own graph and then bring it in. But then building out RAG solutions, Graph rag solutions graphic solutions also building out agents um another one that i think is really interesting is at news out of emergent methods really interesting ways they have been uh customizing like the phi3 llm and they also did a fine tuning on on gliner uh really really interesting work to go after news reports and understand what's sensationalized versus not where the bias is etc but bringing it all into a graph and so it's effectively leveraging llms to build up unstructured data to build up graphs to focus llms very nice feedback loop all right uh can we show some poll results if we've got the latest poll we could show that aha cool uh what experience do you have implementing graph rag in production um okay try so a lot of people are kind of trying or or just learning about it um which is fantastic. And I think this actually fits. It's still fairly early days, but there's more and more resources coming up. So to that point, let's wrap up and we'll go into some Q&A. There's some great community resources. I really love graphgeeks.org. We've been involved there. I'm actually involved with a study group understanding some of the data sources for ERKG. And we'll have more of that material on our webinar here in later episodes. But definitely recommend, check out the Discord for GraphGeeks. It's a great community. There are a lot of people who are interested, coming from different directions. We can all learn together. We've got the ERKG discussion group, Entity Resolve Knowledge Graph discussion group on LinkedIn. I'm one of the moderators. And we've had a lot of great material coming across there. I definitely welcome you to join and post. I'm curating a collection of papers, actually several different collections of papers on the hugging face, but papers particularly about knowledge graph construction. Some of these things I've mentioned, there's a lot more material there. There's a great discord that's been hosted by Neo4j but it has lang chain and lom index and haystack and all the others involved there um so definitely check out our discord about graph rag um there's also uh we cover a lot of this material on the data exchange podcast i'll be doing an episode actually we're we're coming up on that two other uh communities i really like ai alliance and mlops, both have a lot going on. Actually, some conferences coming up very, very soon. And a bookshelf, I will point out this first book here, Knowledge Graph Enhanced RAG. It's in early release on Manning Publications now. It's written by Tomas Botanek and Oscar Hane. And this one goes step-by-step with code of how to build your own GraphRack. I highly recommend getting the early release on that, checking out. Really great book. There's some others that are related here. Definitely a shout out to that. And here's conferences that are kind of in this area, which we'll be involved with. KFirstWorld coming up in a few days, AI conference next week, and then we've got Nodes and ML Ops World, et cetera. And definitely Connected Data London coming up later this this year uh and if you want to get in touch with us please please do uh here's some codes to be able to reach out we can give you more information learning about any resolution and any resolve knowledge graphs these practices um our engineering team will work with you on these. And also if you wanna stay up to date on different talks, tutorials, new editions of our webinar, et cetera, please sign up to our newsletter, but here's how to get in touch. And if you want to follow me here, please link up on LinkedIn. There's's more information also on SessionEyes and my portfolio site too. And so from here, I will mention we will be doing this webinar with guests and Q&A probably about every three, every four weeks or so, maybe a little bit more. So we'll have another one coming up in early october and we'll be announcing that very very shortly and now let's go to question and answer time uh would love to hear what you've got to say and if there are results from the i don't know if we have we covered all the polls do we need to report any results if so splash that up if not let me cut over and take a look at the Q&A. Oh, great. Okay, how familiar are you? Oh, interesting. We use, so 20% are using entity resolution from sensing in production, 6% are different kind of ER, and a lot of people haven't really encountered the ER side of this before, but we'll be covering a lot more about any resolution, how this fits in with knowledge graph practices. That's some general okay uh please post your questions to q a and then i'm going to scroll up and take a look and start to answer some of these um okay um first question uh michael asks many thousands of companies worldwide have migrated massive corpora of their customer-facing content to structured component topic document architectures. Which is the content often used for customer-facing chatbots? Hey, that's a great question. Yeah, I mean, when you look at intranets or just corporate approach in general, you find things like Confluence or SharePoint or many, many others. But there's a lot of kinds of content which can then be put forward to customers. Effectively, how can that be mined to be able to develop graphs and then from there use it in GraphRag, for instance? And short answer is, this is an excellent question. I think it's still kind of early days for a lot of the open source uh graph rag libraries certainly i haven't seen a lot of them going in this direction of like here's our sharepoint connector because that would be a lot of work and it's also kind of more of an enterprise thing and the llm people aren't quite on the enterprise wavelength just yet but we're trying to draw through line there so this is a great question i i think we'll see more of that and um i will actually research that and see if we can put anybody up here in the webinar to address that. Let's see. And another question, best description somebody recently coined about LLM generated graphs is speculated graphs versus curated, validated, managed knowledge graphs. Yeah, you know, I hear that. LLM generated graphs is speculated graphs versus curated, validated, managed knowledge graphs. Yeah, you know, I hear that. LLM generated graphs, it's a little iffy. I mean, there are a lot of great ways to build graphs. I do think you need to have kind of a backbone and understand that and really focus for domain. I don't think that the approach of let's just build a ginormous graph that does everything, I have yet to ever see that work in my years of working in knowledge graphs. So I think you need to be a little bit more focused. There are great reasons for the speculative graphs. Certainly, I mentioned about Ask News and emergent methods. I think I love what they're doing and kind of more in this method. But you do need to blend the two together for most use cases. Another question, can people who have graphs with LLM applications in production share their use case? Oh yeah, okay, great. That's just kind of general, but put that in the chat. We'll try to summarize that. Great, Christoph put in a note here about graphs with LLM apps in production. Thank you very much, Christoph. Yeah, excellent resources there from GraphAware and we hope to have you on here soon um let's see another question do you guys work with entity resolution it was structured er and unstructured okay great so there's some dialogue going on this is maybe i'm not looking in the right part i'm still learning here um ryan had a question okay i think i'm in the right place now um how do you encode time into a graph and the fact that relationships can change over time time-based graphs are are super popular especially in enterprise there's a lot of call for this um it's something you can kind of manage by having say information in properties but realistically it's a structural graph that's changing over time and so there are some notions of streaming in graphs to be able to represent this and also look at some of the machine learning to take into account the time-based changes um it's an active area of research i know of a couple of these projects um this sounds like a good topic to add maybe we can bring somebody on for it and then another question can you talk about potential use cases uh please i know you alluded to detecting fraud and maritime more examples like that would be helpful okay great um let me scroll back real quick here to um yeah if you want to see about use cases here's some different ones. They span across fin crime, maritime, and illegal fishing, looking at understanding financial SEC filings, really great one there from Deanna and Kiniviz, looking at beneficial ownership, AML, Cetra, GraphAware, Linkuris, Enterprise Knowledge, and others. Definitely working with different vendors who are expert in these kinds of use cases whether you're talking about ubo ml kyc pep esg on and on all these three-letter acronyms but we'll be bringing a lot of these people on for this webinar um and uh and i'll share this obviously in the deck there There's some good ways to start looking at that. Also come and check out on the GraphGeeks site too, because I think that we've got a lot of discussions about that kind of thing. I'll see if there's anything else off the public chat that I could mine. If not, shall we cut over to our drawing? And let me go into chat here. Okay, let's do our drawing. Okay, lucky drawing, and let me cut over, and oh, great, okay. Give me just a second. I'm going to download, where is this? Here we go. Let me download something, right right and I will talk maybe we can show the results of some of those polls while I'm doing this I'll take just a moment and grab there we go downloads this is a graph and then This is graph. And then... Okay. Okay, I have, I'm all loaded up. All right, should we spin the wheel? I think we need to have some music on this, but we're going to show... There we go. Okay, we've got our wheel of numbers here, and I'm Vanna White. Ah, 40, and the answer is 42. Okay, awesome. Well, okay, especially for certain fans of certain science fiction there, we already knew the answer was 42, right? All right, now let me go ahead and use my magical Python software here and run this. Boom, okay. We have a lucky winner and I will announce, but for the sake of privacy, I'm not gonna put any Gmail up or put any email up there. But our lucky winner is Pi Buap Tong. So congratulations, Pi. We will get a hold of you offline here. This is just going to be something that we're not going to announce or put on social media, but we're just going to show it live here. We have a special prize. There is a board game for data science, and it's really fun. It's pretty hilarious. It's called Charty Party, and it's a game of coming up with absurd explanations for charts, which is just like you would do in data science. So congrats again. Thank you, Pi, and thank you, everyone. I appreciate the questions. I'm going to look one last time to see if there's any other questions I can go into. Great. With that said, we're right on time. Thank you very much for watching, and we look forward to hearing from you and seeing you next month. Thank you.
Paco Nathan's Graph Power Hour: Understanding Graph Rag
Watch the first podcast of Paco Nathan's Graph Power Hour. This week's topic -  Understanding Graph Rag: Enhancing LLM Applications Through Knowledge Graphs.