 hello everybody good morning good evening good afternoon and thank you for joining this neo4j live cognitive sciences and dynamic graph rack and i'm very happy to have my guests this evening for me vasilija markovic hi how is it going how are you doing this this day hey thanks thanks a lot actually pretty good i just talked to thomas from from neo4j oh nice so it's the neo4j day for me and i'm pretty happy to be here yeah great great it worked worked out we we had planned this um this actually session um i think in august a couple months ago uh but uh but there was uh you know sickness involved and you couldn't you couldn't come so obviously obviously we had to move the event to today. So I'm very happy that we can still, we're able to squeeze it in before the end of the year. We have a couple of weeks left before it all wraps up. So I'm happy that we can quite not really repeat, but we can, you know, redo the events that we were supposed to happen in august so and then you had a couple of great news as well in in between so i think maybe even better that we had to push it back a little bit uh so um to get the picture a little little little rounded up uh sure thanks thanks for postponing i got really sick i was trying to take a holiday after a long, long fundraising process, which as the news are, we managed to raise 1.5 in summer. And yeah, I got sick after all of that. I think it just came out of me, the tiredness and everything. So although I was in a really beautiful place in Croatia, my eyes were like all messed up. I couldn't see much. So now I'm back. I can see and I'm happy to participate. No, that's great. That's great. And yeah, congratulations again to the round. I saw your announcement on LinkedIn. When was that? A week ago, actually, when you posted it with a couple of investors into your company. And I guess we get there when we talk a little bit more about the topic today. But yeah, it's great to have you today. Great to be here. So yeah, I'm happy to share more about us, about the approach we are taking. I've seen a lot of your, let's say, talks in the past and I think they all kind of have the same type of direction and then some are more technical some less so i'll try to kind of keep this on on higher enough level and can explain what we do and but still give some maybe examples and show to people how they can find the the resources absolutely sounds good um hello to everybody watching live and chat i see andrea i see alejandro i see walter here guzi was from the us uh giving i i think i would say quite quite uh indirectly giving a thanksgiving note so happy thanksgiving to everybody watching in in the us um he's giving he's thankful for all the great graph database knowledge neo4j gives away for free so yeah we're happy to have you today uh join joining this this uh this session before the big uh the big long weekend i guess in in the states um yeah before we dive in um in into into uh you know cognitive sciences and the dynamic dynamic graph rag and uh and cogni a little bit i guess um vasily would you mind giving a a few a few sentences about who you are, what you do, and maybe a little bit about as well how you came to the world of graphs? How did you discover graphs? How did you make that as, I would say, a pillar even of your job life nowadays? So it's always interesting to hear a little bit about backstory. Sure, sure. So I'm Vasilij, 34, originally from Montenegro. I've been in Berlin now for 10 years working and living here. I graduated with a degree in business around 2013, worked in a crypto exchange, helped scale it from zero to 10,000 users and learned how to code and build as they flows and customer funnels. I realized I like data more than I like crypto at the time. You know, everyone has the preference. I like crypto in the beginning, but then I kind of, I got sour on it a bit. And as I was moving through my career, I became a data analyst and data engineer, data product manager, mostly managing big data systems. So I worked for a couple of companies here in Berlin, Zalando, Taxx, Omeo, and I've spent time pretty much, you know, doing data model, right? So in the end, it's the problem with relational databases, although the structure is relatively clear, everyone has their own interpretation of a session and what they're doing. So as I finished that, I became a consultant, worked for a couple of other companies that had even bigger issues with these types of problems. Then I got bored from moving data from place A to B. I went back to uni, started in cognitive sciences and clinical psychology, actually. And as I was doing that, LLMs showed up. The alarm showed up in full force, of course, they were there before. And I was working on some interesting projects for some clients, and then I decided to just start working on something on my own. So the road was rocky and interesting. Yeah, build a couple of B2C apps. I realized I know nothing about building B2C apps and then gave up on that one. But what I noticed is that when i was using tools like lang chain and when i was trying to resolve things uh there and have like some stable contextual memory i wasn't really getting far and i realized then as i was doing like these cognitive science courses and learning about human memory and like let's say dynamic entities and objects that we could how we use and how we use memory to represent things. I figured, okay, maybe we could kind of model this in a different way, have like this, let's say, more probabilistic entities that can be evolution of this relational model. And I thought, what could I use to do that? And then Neo4j popped up. So I just figured like, okay, I didn't know much about graph databases at that point. I figured, let me do this because it's the best way to represent and evolve the entities as they come along. So we built the first prototype, launched with the WhatsApp chatbot in production with around 20,000 users. And then we realized, okay, all of this is wrong. We couldn't really generalize things. We had like a lot of assumptions, and then we went on this road, uh, to build cogni as a Python library. And that's where we are at now kind of evolving step by step with the approach. Yeah. Cool. That's very interesting. Like, love to hear your thoughts on, on, on entity recognition with, uh, with graph database at some point maybe, but, uh, but let's not deviate too much from the topic of today. It's a top of mind thing that we're currently looking at for our going meta series that runs in parallel. So it would be interesting to hear your thoughts maybe if you've got some time at the end. But yeah, that's a cool story for sure. Thank you for sharing that. So we see see you you starting with with crypto then going into data modeling then you know now it's now founding your own company and um i guess graph has been kind of part all the way i mean even with crypto it's not really core core part but you kind of like you had an inkling i would say to to the to the technology still without maybe even realizing so that's that's always the funny thing with graph technology or for us at least, it's like how many things actually are graph problems or could be solved with graphs or are built on graph technology, even without you realizing it, you try to manage something and you do it with traditional kind of like technologies. And then you realize, oh, actually I could have done this with a graph database or with graph technology and this would have saved me so much work that comes sometimes later. About these concepts that you learn at some point and then you start applying and like externalizing, but until you know them, we are like kind of stuck in the old way of thinking and I think I'm super happy that I started working with graphs maybe a bit late but still yeah yeah cool now that's great um yeah um let's uh let's continue um and talk about uh cognitive sciences and dynamic graph rack so i don't know if you if you have any slides or if you want to want to do a demo or if this is more, if we can talk about it some more. But I just wanted to reach out to the audience if you watch this live and if you have any questions, any comments, and please feel free to put them in chat and I can read them out either in a break session or towards the end and we can cover a couple of the additional questions that come up. Sure, so how I was thinking of doing this, kind of explain things a bit verbally, I'll show a Python notebook, and then kind of go back through the things we've done, and kind of, again, explain we have some semi-slides in that notebook that I think would be interesting to stop at. That's great. And then we are we can actually uh yeah also talk and check the questions so um cognitive sciences right and and and uh what are we actually trying to do and how does this uh let's say approach evolved and i'll tell you the story of what we were inspired with what models we used and what we were thinking and where we are going with it now. And I think this story from the beginning to this day, I think it can actually, you know, we can introduce a few concepts there. So when I started actually with this problem of modeling of these entities, I'd say in the graph store in September last year, and how can I actually create a context for LLM that can be useful and that shows the relationships between entities a context for LLM that can be useful and that shows the relationships between entities and make the LLM get to a better decision to increase the accuracy reduce the cost by already pre-processing that I was basing a lot of my thinking on these models from cognitive sciences such as Atkinson-Schiffrin and all these, let's say, computing paradigm that was in use back in the 60s, 70s in psychology. So what's Atkinson-Schiffrin model and how do these things look like? So when we talk about human memory and it's important to distinguish between the storage and then the memory processes right so the memory storage or the memory domains is something that we use every day to store certain type of information so when we hear some word or see something we usually get 100 of that information stored in in our system but it's for a very short period of time because we can't just keep all the things, the audio, factory, and other signals that we notice. We just kind of have 100% sensory memory that we get. But unless it's really activating other parts of our system, we just let go of that. That's one piece of the memory. Second piece is the short-term memory. You can usually play with that or heard about that. memory second piece is the short term memory you can usually you played with that or heard about that it's like how many you know words can you remember or numbers uh in a sequence and you know how many can you keep in your short-term memory when you're trying to copy from one paper to another the shorter memory has a bit longer span but it usually uh can't also translate into anything you're going to remember after a couple of years unless it's really eventful and it gets then stored in the long term this is what that kids and schaefer pretty much said so you have the sensory memory you have short-term memory you have the long term long-term memories where you know you actually remember things and you know what's happening and you can recollect the childhood events so this is called the episodic memory so episode of your life you know when you got married you got the kid you went to the first day of school and then uh we have this what's called semantic memory semantic memories where we are actually um remembering some facts or information about that so this is a whiteboard behind me the concept of the whiteboard concept of the wall all of these are pretty much stored there so when i was playing with the the llms in the first moment i was like okay um i need to store the data somehow and like pass it to the vector stores but because the vector stores only have the embeddings they don't have any type of conceptual mapping where would these embeddings belong i'm just searching you know and getting the data back so what i was thinking like okay how about i hard code these these models i could i put like i have a short-term memory i have long-term memory i have episodic semantic then there was a princeton paper about koala that that just came out at that point when i was exactly doing that i think i posted like two weeks before they they published the paper i was faster than princeton and probably the last time and i was like okay you know i had the same idea they just did it much better of course and uh then when i was implementing this i was implementing that in neo4j i was creating certain nodes that would represent the the memories and then i was attaching the data and decomposition i was doing with the llm structured outputs to each of these domains so what that led to is as you can imagine the the that is you know we had i had these super nodes this problem of super nodes that had like too much data associated with them and then i had like a lot of let's say these memory components that over time grew and i couldn't really do a good retrieval so as we implemented that for these whatsapp users in a chatbot and they started storing their memories and searching for them we did marginally improve their let's say reliability of vector stores by giving let's say a way to navigate through certain types of memory domains but these memory domains were still too static so the next thought I had was like, OK, because I know that in the cognitive sciences, we have different, let's say, processes, memory processes that we use every day. So we have forgetting, which would be decreasing over the amount of time, the importance of a certain event until it disappears from the memory. And these types of things could be relatively easily coded into the system. So the next attempt was like hey how could i maybe alongside each of these information points i was loading to each of the memory domains how could i actually add things about recency frequency of use of this domain you know if we so in the psycholinguistics in the cognitive lexicon which represents all the knowledge about all the words we use there is this hypothesis that all the words that are on top are there because they are frequently so frequently i'll use much more than uh oxymoron you know or some not something i see yeah this was kind of the approach like let's kind of code that into the system and kind of re-update the system so this is where starting to get the dynamic approach right we are having like adding these weights and kind of using these weights and making these relationships not like vasili owns uh uh i don't know this or um like this is happening there which is what most of these uh lm simple projects do but having these numerical relationships that are kind of increasing in size decreasing in size and that we can quantify yeah it was a lot of fun meant that's right yeah you you you add like you you store the facts but you also give them like a soft uh additional information to it that is uh you know you know maybe this is importance I think something like that so that you can add it into to afterwards say okay this was a really important event so I guess it's 10 or something or this was not as important so maybe it's a one and then it gets forgotten or maybe it gets removed even I don't know but um okay that's cool it's a it's an interesting idea yeah yeah yeah I mean like mean, like it's known in psychology and what I thought like, okay, this is not known in computer science in this world. No. We just focused on these transformers and this- The internet never forgets. So I think like, yeah, exactly. And yeah, in the end, what I was thinking about is like, how can I actually make this scale, right? Because what I saw as a biggest issue with these types of processes is we would need to process let's say millions of nodes we would need to update uh you know sub graphs of information deal with like very complex edge cases that you know require let's say some stability that you know most of these approaches that i was doing at the time i couldn't really reach right and um that kind of pushed me and my co-founder boris at the time to to think about how can we actually make this solution a bit more horizontal and a bit more let's say mature in terms of the technology stack so we can kind of come closer to production processes and doing this at scale uh we were pretty aware that like cost of open ai was kind of forbidding at that point but we assumed it's going to go down like the price of the tokens was was going to reduce and we're going to kind of move with that so we released the python library we created the first cognify pipeline that did pretty much this and then um and use neo4j as the the the store used the open ai's lms use structured outputs created these graphs and these artificial domains of memory and we even with these updates with these let's say numerical properties we would add to the to the nodes and the edges and then try to kind of quantify them we were still running into a lot of issues? Because we simply needed like to be two levels above in terms of the performance and operation to actually, you know, be able to process one terabyte of data because with 150 megabytes, you can do anything, but like with compute, you know, for big companies, it becomes already difficult. And I think one of the reasons why Neo4j is one of the rare companies that managed to solve the graph database problem is because it works at scale. And even Neo4j has issues at certain point. So you need to be careful on how you deal with that. So that led us to kind of think about the product a bit more. And we thought that we should simplify the whole interface because this now is was getting a bit too complex to a set of abstract components so one of the components and maybe i will go through the through the example now because it's maybe a good time to start introducing these elements So you can run this yourself. We have also here a Colab notebook. I'm not gonna run it now, but I'll kind of show you what we had in mind. And we will share the link. Maybe you can share the link with me afterwards and I can put it in the video description so if you just search for cogni on github you should find us so um effectively we are here introducing a couple of concepts which i now mostly talked about and i think it's already clear but this demo is effectively just talking about the some series and some job descriptions and we are trying to extract entities and match these series to this job description so in this context yeah you can see the strings we just used to kind of illustrate the point and how we try to resolve that is that we effectively have this concept of Cogni.add. So Cogni as a Python library has brought to the simple interface because we didn't want to over-complicate things. Add command takes any type of data, so PDFs, images, audio, and stores that inside of the system for query processing so we saw that a lot of issues that happen with the systems like graph track systems were not enough data engineering right so we use this library called dlt behind that creates does the json normalization standardization creates the merge ids so we have all of that data that we need to actually be able to replicate and rerun the pipeline when we need to because this is one of the these guarantees that we can run something in production and once we do this first step we can do it here as adding a list of all of these job positions, but we can also add paths to any file in the system or S3 buckets or whatever is needed. And then this data is gonna be processed, normalized, and we can actually create a graph out to it. So what we decided to do in terms of the architecture and how to approach the system was to create a set of tasks that are tying in into pipelines. So I worked a lot with Airflow in the past and in this data engineering deck space, so these graphs, the central theory graphs, we saw that this type of a use case is relatively flexible and relatively useful. Haystack is doing something like this similar, but what we try to do with these tasks is we give them a state and then we are allowing the every task to inherit from another task so let's say this classified documents task is going to pass the information about the classification to the permission checking which is going to pass that to the chunks so only the chunks with the permission are going to be extracted and we can pretty much structure and organize the data in this way so this is nothing to you but for us this was one of the key things to unlock further experimentation with this cognitive approach because we could then finally play around with different approaches without having a hard coded static system that you know probably won't work in three months because yeah we just assume something that that might not be true so cogni as such is now effectively a pipeline that is combining a set of tasks that are running in parallel in batch or sequentially depending on how you set it or async and And then what we did also is we recognized that everyone has their own use case. So we created a set of infrastructure adapters that you can just connect to at any given point. So you can use multiple databases, relational ones, vector databases, you know, LensDB, PG vector, Pineclone Quadrant. We have here also Filecore DB adapter. We have Neo4j drivers, NetworkX. So you can run this locally, but you can then also have it running in production with one of these bigger players, and it should be relatively straightforward. Yeah, we also support multiple nlms so once we did this and once we kind of build this type of a system what happened was that we figured that well this is not really working that well not in terms of like technology that partly felt confident more confident about them than just the graph generation. But in this case, we had like, let's say we were trying to do retrieval and this context, we are trying to, we can do the search of the vector engines. We can do search on for maybe summaries and then just search for the notes in the graph. But all of this is still a bit too granular for an average user, right? And it's going to take a lot of engineering to get to the good results. And we spend a lot of time ourselves doing this. So you're thinking like, okay, this is probably not gonna just work like this forever out of the box. So we needed to add again, another layer of automation. So when we talk about the, and I'll switch back to that to the screen when we talked about the approach of uh kind of storing the data in the metadata um that is effectively like so these let's say numerical classifiers or or or metrics we thought about actually a different approach. Why not actually extract all of the relevant things we can optimize for in the pipeline, outside of the pipeline itself? So I'm talking about chunking, I'm talking about which LLM we are using, which vector store, which data model we are generating in the graph and how. And how about we actually create a system that kind of automatically evolves this. So the approach there was, and this is something we'll release relatively soon, is to have this pipeline be an evolutionary pipeline, as we call it. So a dynamic pipeline that just fixes itself based on a test set it is evaluated on so let's say are you following me or like should i maybe clarify a bit no yeah talk a little bit more about that it sounds interesting but um yeah that is an interesting approach here dynamic um you know updating dynamic uh almost like a um you know, living knowledge graph or living knowledge base, and that it's constantly changing, a little bit evolving almost. Yeah, we always talk about a graph is similar to the brain because the neurons and the synapses are like nodes and edges, but we didn't think it as literally as you were thinking. So it's an interesting approach. Yeah. I mean, the nature of systems is to evolve, right? And I think the approach here was like the initial understanding is like that, you know, you come in a workplace, in two years it's a different workplace. Like 30% of people left, 30% of people like you don't know so i think like every type of a system is an evolving system you can try to kind of keep a system from changing but then you know things tend to kind of end up in fortran in a bank for 40 years but do we want to be that so the way we're thinking about this is like, hey, we have static data warehouses. There is 140,000 companies using data warehouses. I did some analysis with Google Cloud and Redshift and whatnot. How about we have a probabilistic data warehouse? How have entities that are evolving in a data warehouse where schemas and relationships continue changing based on how good of the answer they are providing to the problem, right? And in this case, this is the first step for us is like to create these data pipelines in such a way where I add like a test set that, actually a train set in the traditional ML approach with some question-answer pairs. I pull that through the pipeline um evaluate it at the end see what kind of a grab did i get what kind of a combination of search searches did i get them and how they're evaluated then i say okay this is not good enough let's loop it back right and then i just loop it back loop it back or i can launch it powerly i can do many things with that you know i can do all these like packs of computer science you know uh searches you know optimize and start from the middle start from the ends whatever like that's all these interview questions from facebook i never got that so you know i'm not gonna try now to pretend i know too much about it but like let's say that we can we can apply different methods and with these methods what we would get to is effectively a self-improving pipeline. The self-improving pipeline is then gonna stop at a certain stage when it passes the test. Now, maybe it's over-optimized. Maybe we just over-train or over-fit that on the train set. So we then pass it through the test set. If it passes the test set, then we know that this data representation that we made, this, let's say, dynamically improving data representation that's where we are adding new things, interpreting them in different way, deleting things that don't matter, is one that answers the questions from our stakeholders. And this is the way we could avoid tweaking the chunk size, tweaking all these factors tweaking the uh all these like factors in the in the the graph rack pipelines that are really time consuming and ineffective and then in the end we could also create another pipeline which is feeding the feedback of of the stakeholders to kind of create a layer above that is kind of again influencing how we are communicating results maybe you can do an interview with the analysts that are using the outputs of this data maybe you can create a chatbot that's just feeding the data back but you're pretty much again you have a feedback loop and with this feedback loop you have an improvement you have a measurable improvement and then with relatively good evals, you have, let's say, a way to move forward. So this is something that, yeah, for us is, I think, the first step towards, you know, this, if you think about a snapshot of human memory, which a current, let's say, Neo4j graph would be, moving towards this slowly moving dynamic memory, and then we are assuming, like, okay okay cost of tokens are gonna be i don't know a few cents per million tokens in a in a year and they're gonna be doing that in like you know you could produce millions of outputs millions of outputs in like a couple of seconds so we're probably gonna have ever increasing speed of these things and we're gonna move like we did with like data engineering from batch to streaming over time and things are gonna get done magically. But yeah, for us right now, like even if we can prove this as a proof of concept is already like a step forward. Yeah. That's, that's, that's, that's very cool. And you, you always have complete view on, on, on each of these, these, these processes. So that's helps a lot, I guess, to, to, uh, um, understand a little bit better what is actually happening because you are updating the data, you are reconfiguring the data, and then you're outputting the data, and then the data becomes your living and breathing almost knowledge graph or knowledge base. And then you put that data into your chatbot or other, you know, generate AI-driven systems, but you always know, okay, this has happened because of this and this changes happened because of this. And this is the evolve, evolvement. So you, you, you can always follow that. You don't have no, a viewpoint into something which, you know, you can say, yeah, something happened, but I don't know exactly what happened and I cannot explain to you why, why it happened this way. But I just know that this is what I get here as a result and um that's sometimes you know gonna be hard especially for you know more complex data i guess and more more data where it really makes is really important that it is correct and true um you're opening up a good question um what what is you know the day it's harder for the data that needs to be true. I think we've changed the paradigm, right? So we moved from like relational data that I could map one to one, and I know always like what's true or false to probabilistic systems that, you know, spin out some tokens. And that is, I think, gonna lead us to, we can probably only give a certain high percent chance of accuracy but we can never be 100 accurate so most of our job now becomes not only building these things because it's relatively simple to chain them together right and there is all these chain tools out there i think it's about like making sure that they work because when you hit 70 percent and you need to move to 90 and then we have a way to actually avoid you know your chatbot giving free coupons to to to customers the wrong reports this is the difficult yeah yeah that's that's that's good um I I was wondering because I had had a session recently on on agentic AI and I was wondering if this the approach you just outlined and with with this you know more agentic driven systems I think this is the the current trend I see a lot a lot of people talking about this but your approach is is is is to some extent it sounded to me but correct me if I'm wrong sounded as like a similar idea but you're giving it a more you know you you keep the process with you so you don't give it away as much because with agentic air just say okay i have this idea of a process this needs to be done so ai do this then i have another idea of a process i do this and then i do this and then i combine them in the end and then i have something when i have processes in between which i have no control over almost something happens and then something gets gets gets gets back to me and your approach seemed to be a little bit more you are trying to structure it and trying to give it a more of a yeah more of a controllable feature feature set exactly and I think we're trying to reduce the complexity by reducing the number of variables that could change and then in that much more in line with the data engineering and data processing practices versus like hey can this agent do something broadly undefined and solve it for me without specifying the exact needs that I have so i think um for us the major part of the work is on defining the evaluations and on making sure that the system works at the beginning and the end as it's expected where this other parts can be automated whereas i think with these agents a lot of it is um defining the the middle right this set of steps that you expect the agent to do one by one, one by one. For us, that's not as important as having a more reliable data store that the agents can write to, read from, versus just having some data store that won't work even in these use cases. So, yeah, I would say we are more like the accuracy and scalability of this data to a degree we can guarantee for versus just having a complex vertical use case that, with the advancement of these systems might be outplaced or replaced in a couple of, in a year or two or whatever's happened. Yeah, cool. There's a question from Jan, maybe I i read that out this do you have any any slides or diagrams describing which data you actually store in your system or which data you are using with with this dynamic graph rag approach uh and you know how do you come from let's say jobs and cvs or or from something else to to the knowledge. I don't know if you can show a bit of that or explain. Yeah, sure. Let me just check. I think I have another example somewhere or... Just a second. So effectively, I think the example, how we actually do it is the structured outputs. So what we do, we would give it, we use pidentic models everywhere so um you probably all have seen this um we just provide the pidentic model via instructor to open ai's api with the data and then we pretty much try to fit that in so uh let's see where the models are um but i think yeah this you can pretty much, I'll share the link here. So I would say that this is the, yeah, I'm sharing this in the chat. Perfect, yeah. Yeah, so I think for you, Jan, yeah, it's effectively just using the instructor that you see here under the hood with some Pydanti class. And do you share your screen, by the way? Because I... and how do I? Here we go. So yeah, this is an underlying thing, so I don't go through our code because there is a lot of stuff there. Yeah, effectively, let's say you define some class called user, then you pass some data to it, and then effectively you're gonna get generated, you're gonna get the pedantic class back with the data filled in based on the data you've had. So if we, let's say, feed to all these CVs, we defined the graph object, which has some nodes and edges, and that gets filled in. So this is not something that we want to continue doing in this way so we have a new project that we're working on now that's called code graph reg generation so let me see i think i would yeah i'll stop sharing the screen i'll try to find it somewhere and show it where we are kind of combining more deterministic methods and llm methods on top so for example in this case what we're trying to do is help our coding copilot build a graph representation of github repository and then enrich that and then help actually answer any questions on that github repository to this coding compile it this person is interacting with it the current way they do it is like they just create embeddings store something in the vector store and then they hope for the best with this right now well it is true with this we are trying to pretty much narrow it down a bit and give them a space for search that could make sense and then again evaluate that test that adapt that a bit more so um this um so the grab generation uh in itself is let's say a relatively complex problem and for different use cases we are thinking about different uh potential approaches that could work and that could actually make sense. I think for us right now, this is a place where we are kind of experimenting the most and where we are really trying to do as much as we can on the topic of, let's say, of the graph generation and graph retrieval. So I think I found it, so I'll share a bit more on this. So this is code graph pipeline, it's called. And then in this case, we have pretty similar set of tasks to what you saw before. But then here, here for example we have this class called source code class that can effectively represent the information about the graph that we can extract from the LLM but also generate more deterministic methods that is going to contain nodes that can be a class class instance node function function call so this is all the things you find in the typical repo and then we can do this on on different levels so as you see here we have expressions function calls we have class instances operators variables so so all of the parts of the actual code base and then we also have the files and then relationships with these files between themselves so this gets pretty complex the trade-off here is where do you actually make the how granular do you want to go we want to see where is always connecting between itself or do you want to kind of stay on the file level so the answer is be somewhere in the middle uh but that piece you know we don't want to find you we want to automate the tuning of that and then we'll see how that kind of works with these evil sets and then the things we are building right now so is this is this almost like um ontology creation where you define a bit of a structure first and then you said, okay, this is my structure, this is what I want to see, and then you populate the fields accordingly. Yeah, pretty much. We're thinking of ourselves as, you know, DBT of the the future so you want to create like data models but you want them to be ontologies you want them to to change evolve and then you want the system to fill in the blank somewhere but you also want human in the loop yeah that's cool um where did you did you would you say is is a what's a great example uh or a great application i would say even where where your system or the the concept of company where that is particularly successful or particularly good good place to to to try or to use in in a in a real as a use case or as an application setup. So I'll preface this with an explanation of how I'm seeing the space now. So who are the people using these types of tools, right? I think back like seven, eight years ago, data science was an esteemed prestigious thing. Everyone was trying to be a data scientist. Then not many people were and not even less people had like the required mathematical background i'm not a mathematician but like i know when i see one you know like a good one who's a bad one like i do have some some training in these things but i would not call myself one and then um because it was so difficult to productionize these data science systems um it was relatively a niche. And it was these types of systems were used to improve, let's say, a certain part of a company and increase revenue from 3 to 5 percent if they are doing well or decrease it by 5 percent if not. But you wouldn't see data science become a mainstream thing. I feel with yellow lamps that changed because now everyone's a data scientist. You know, you have an API and you can just do magic that you couldn't do like seven, eight years ago. You would require like three, you know, lead engineers and like data engineers, two data scientists, one analyst. So it would be much more computationally resource intensive. So with this, let's say, openness now of the system, we have a lot of people more moving into the LLMs, into graphs, into everything that's related. And they need a bit more hand-holding than people in the past, because they had to kind of go through these, let's say, complex problems and scaling issues themselves and now people still just simply assume things will work with yellow lens because you're going to throw something at the api and work do magic but then the rest of the system is probably going to collapse if you don't really maintain it or build it properly so uh what we are trying to keep to do and how we are looking at this is we are looking at people who are moving into the space. We're trying to give them tooling to find the use cases we can't even imagine. So I know a guy, he built a teleprompter tool. So when you're doing an interview, you have a teleprompter running and then he's going to analyze what both people are saying. So I'm not using that. I would be listening to you and responding to Jan's or other guys' questions. And then it would just suggest responses. I could just read. I would be like, you know, American president. I don't know. I wouldn't have to think for myself. And not saying that they don't, but I'm just saying it might be easy. And effectively uh this teleprompter tool is now making was making 1.5 million in revenue in may this year oh wow two guys you know just building a teleprompter tool and like they did a good job marketing it but i think like ai is opening things i couldn't even imagine before if you could tell me that ai is going to analyze our discussion and give me a prompter to have better responses i would be not expecting that yeah so to answer the question uh now a bit more granularly we're seeing interest from different types of verticals uh construction agriculture a lot of people who have a lot of data and are trying to kind of merge this data and analyze it and create and have better insights into the things they don't have time to read or understand and with the llms it's either unstructured data or it's a combination of structure and unstructured data so text to sql then pretty much code generation is one big use case we've seen and we've seen a couple of others so i think um yeah in in practice practice we're seeing let's say two groups one is the people moving into this space so we want to give them an easy tool to use and that's where i try to simplify it and kind of abstract out the data engineering behind because that's a difficult point and the second group is like these companies that have a lot of data a lot of pdfs or unstructured or semi-structured data they just want to put somewhere and make it useful so kind of unlock this data access yeah yeah yeah i've been talking about it yeah no that's great that was that was very informative yeah and i think that's uh that's a huge space i mean we we talked a lot about but unstructured data these past uh months i would say everybody everybody was talking about, oh yeah, I have my data set of huge PDFs or emails or some other stuff I cannot really access in my dataset, but I think we tend to forget that there's more probably on data that is actually structured, but it's still sitting somewhere in a system and it's been used been used and if you if you then enable people to actually combine the the huge amount of unstructured data which can be messy and the huge amount of structured data which also can be messy in truth so um and if you combine them together and then make something that is actually accessible and that is that is actually giving giving something um to the user to the to the developer that can then be worked with in the chat pod is, as somebody at Node said, so 2023, but it still comes to mind as one of the first things to think about in a, let's say, first stab at it, but more complex applications will come around soon and will be there. Like you said, code generation, there could be something as well that is really, really beneficial. Looking at history, looking at maybe something that has been conversational. Imagine like this teleprompter app, you and I, we talk about an idea we have about something we want to develop, but then we didn't transcribe our conversation but we maybe forgot about it and then this teleprompter app prompts it again and says hey you know you talked about this cool idea and then maybe it spins it first so yeah and my question to you is what have you seen based on all these interactions with people i'm sure you're talking to many, many people in the space recently. Yeah, lots of people, lots of conversations. So more or less these conversations here are back-end driven. So it's lots about GraphRack really. So that's the top content everybody wants to, like I said, everybody wants to put their unstructured or semi-structured or structured and unstructured data in some kind of AI system and do something with it. And then the trends follow along. So that's why I mentioned agentic AI. I think that's currently where I see a lot of conversations. People talk about this a lot and put something in and outsource almost like things you have, you would have to think about yourself or you would have to do them into a AI driven system and then gives you offloads lots of your work. Yeah, okay. That matches pretty much to what I was saying. I think the space is gonna continue evolving pretty rapidly. Exactly. And I'm super excited to see all these graph rags come to play. I've seen another five projects in the last week. I think like when we started, this didn't have a name and you know, it was just like a concept. So I think like, it's super exciting to be able to kind of participate and to do these things and especially do them if we can, you know, at that level and that scale that uh you know it's actual production systems and and we change this dynamic of like snowflakes and redshifts of the world towards something more exciting and evolving it's like i think it's a it would be a good contribution yeah definitely so that that's that's what i what i what i what i was surprised at the beginning or what i was what really thinking is unique about your approach is the dynamic part of it so you you're still even with with most of these agentic ai systems you set something up and it is relatively static at least that what it feels to me to be the case and then you know when but but things are evolving so fast and things are moving um a lot and then if you have something that can change and is is adaptable to certain different kinds of requirements so you could you could have your data in but then one user wants actually code generation because they develop something and then you that helps them do it but another user might have some completely different questions and that it's still um you know in some cases it might need a complete refractor of the of the whole data store because you need you have different questions that ask different things of i lost the audio. Is that just me or? Yeah, I can't hear anything. No, sorry, maybe it's on my side. Should I try and rejoin or? Yeah, Alex, it seems others can't hear either. I'm sure he's going to be back soon. In the meantime, how about the commercial break? We can talk about the sponsor of this today's podcast. I'm joking. I don't know. I'm just trying to, so do you guys have any questions I could answer until he fixes his audio? So, so you guys that are not stuck here. Okay, since I'm fine. Yeah, I think I see this personal knowledge graphs question. Maybe I can answer that in the meantime. So we created this concept of a user, graphs, where you can actually, I've hosted the podcast now, so, you know, I'm the captain. Welcome. So yeah, the personal knowledge graphs, we create the concept of the user, so you can store for a particular user, you can create little graphs or sub graphs that just have their information and then for particular use cases you could also do that on organization entity level so you would avoid having that these types of issues with the the graphs that effectively are too broad and contain maybe some personal data you would not want to share with others in the system. Any articles I would recommend? Yes, many. I think there is a list on our documentation. I'll double check that. If you go to Cogni and then to Cogni's documentation page, you should see a list of articles if not I'll make sure to add that page back again after the call but we had a list of around 20 30 articles that inspired us in the past any other questions or hopefully Alex comes back soon. So yeah, I think like Koala paper and then, I mean, to name a few, there was a recently one about generating graphs with the four and SQL with graphs. That was, I think uh pretty popular in the enterprise settings um so yeah i think um um yeah are you back i think i'm back my internet connection dropped no worries sorry i took the lead just okay great yeah thank you very much sorry for for that everybody sorry for dropping off all of a sudden. It was really weird. I could still hear you, and then I couldn't say anything anymore, and then I don't know what's happened. Yeah, this happens sometimes. It does happen, yeah. Yeah, so it's like I think... Yeah, would you like to go through these questions? How much time do we have? I don't know. Yeah, we have a couple more questions. I don't know. Did you discuss anything interesting? I see. Just let me quickly follow up on the wallet concepts from Don. Yeah, I think you probably talked about this. Alejandro says, I understand that there are certain parallels in cognitive neuroscience. Have researchers evaluated the impact of forgetting on learning and how could this be effectively implemented if useful yeah i think that is an interesting idea because i was wondering about this myself as well as because you mentioned the forgetting part and and i you know i joked about internet never forget so and we all want to you know store as much data as possible everything needs to be saved everything needs to be put in some kind of database, you know, data lakes, data, everything goes in. But, you know, sometimes, you know, it might make sense to get rid of some of the older stuff or some of the irrelevant stuff and get rid of some ballast to streamline the actual, the important bits or the relevant bits or the things people are really interested in so there is so much research on this topic on the forgetting on how fast do we forget what type of information do we forget how much emotional uh attachment to a certain type of information impacts it this is also very connected to the trauma research so for example you know if you see the vietnam war vets from the movies when they remember they're back in saigon in 1968 that's actually happening for them they look back to that point in time and their brain is kind of rotating in one old memory that they couldn't forget that they can't really move forward so what I'm saying is the forgetting and the research there it manifested itself already with like this time decay information, like time decay metrics that you could add to the nodes. And there was some research. Yeah. But there is many more strategies in the ways we remember and the ways we forget. And I think it's a very, very open field to kind of go through and research and bring back the learnings. I'm not too much of an expert here but definitely something that we need to check on our side a bit more and i think um in terms of the yeah um effective implementation i think it's a quantification of the relationships in the graph and then actually having a particular type of of model have something actually work. And I think, yeah, that's what I would say on this side. And I think in the context of summer internships, I'm seeing a question here. No? We are not looking for those because we simply don't have the resources now to also manage interns. But what I would encourage is you can always book a call with us if you want to, like some onboarding session and our rep is open source. So any contributions are welcome. We're going to review them, the merge into the main branch and you can definitely showcase the skills there. And then, yeah, I think that's everything I've seen. Super. Yeah, if people want to reach out, the website is probably the best place to check out. I'll post that in chat again, so you can find that, or, you know, like you said, go to the GitHub repo and put in your comments there. Yeah, what can I say? Thanks a lot for the chance to participate. If anyone's interested, give us a GitHub star if you like what we do. Try the library open. As many back tickets as you can, I think, and if you're busy, so we are still new to this, and we want to have a really good system down the line. So more than welcome to the feedback. We have a Discord channel that's not too active right now, but we'll make change that. And join talk to us ask questions we're happy to add features and take the community feedback we have a community roadmap too so you can just uh you know you're going to be prioritized there and you can see the progress and the things you open cool that's that's great thank you very much for for joining us today Vassili, and for presenting this interesting topic, interesting approach, and interesting graph idea. So I would encourage you all to check it out and have a go at it. Let us know how you come around with it and what you do with it. So that definitely is great to see. In terms of next things here on this channel, I'll be joined by Jesus again on the 3rd of December for the final going meta of this year, going meta episode four of season two. So we will talk about semantics and knowledge graphs, again, similar concept. So on the Tuesday, the 3rd of December. So if you're interested in this, then please join us and let's continue the conversation here. We are also looking at your feedback. So if you are a Neo4j developer and want to give us feedback on how you develop, how you code, how you learn new things, where you're from and all your thought processes in going into developing with a graph database. And please let us know it's a quick survey. Shouldn't take too long of your time. And it would be helpful for us to prioritize on where we should be, what we should develop and what we, what you need and what you want from us. And yeah, in general, what to do next, Graph Academy is a great place to learn. Neo4j Aura 3 is a great place to start as with a graph database as a service for free. And then we have a community page. We also have a Discord server, so you can check that out as well if you want to, while you join Cogniz Discord server, you can also join the neo4j discord server and and say hi and hang out with the community there and yeah with that everybody thank you very much for watching i hope i see you on uh tuesday if you're in the states have a happy thanksgiving enjoy your long weekend don't buy too many things on friday do, I don't know, as you like. Thank you very much again, Vassili, for joining today and for the interesting conversation.
Neo4j Live: Cognitive Sciences and Dynamic GraphRAG
Explore the intersection of cognitive sciences and dynamic graph-based Retrieval Augmented Generation (RAG)!

This livestream delves into how cognitive theories can inspire new approaches in machine learning, optimizing data storage, retrieval, and personalization. Learn about the latest advancements in combining graphs and vector stores to create semantic layers, enhancing the in-context learning pipeline, and shaping the future of AI tooling.

Guest: Vasilije Markovic 
https://www.cognee.ai/

Github: https://github.com/topoteretes/cognee
Instructor: https://github.com/instructor-ai/instructor

0:00 - Guest Introduction
2:05 - Overview of Today's Topic
5:04 - Cognitive Sciences and Dynamic GraphRAG
8:45 - Introduction to Dynamic Graph Memory Models
10:01 - Human Memory Models as Inspiration
14:02 - Dynamic Relationships in Graphs
18:29 - Demo: Cognitive Python Library
28:14 - Self-Improving Pipelines in Graph RAG
34:02 - Evolutionary Knowledge Graphs
39:10 - Code Graph Pipeline Example
47:02 - Future Trends and Dynamic Graphs
51:00 - Q&A Session: Personal Knowledge Graphs
57:05 - Closing Remarks and Upcoming Events

#neo4j #graphdatabase #graphrag #rag #genai #llm #science #cognitivescience