 So I'm Alex and I'm going to be presenting Runway today. So just a little introduction. I've been with Neo4j as a consultant for about a year and a half. In a previous life i was a neurophysiologist working in healthcare i'm based in chicago and this is my puppy named pochita so what we're doing here with this package is we're using llms to automate the graph data modeling design and creation processes with the goal of reducing the time that it takes to go from source relational data to working with that data in a graph format. And we're also trying to enable new developers to Neo4j while also providing tools that can be useful to those that already have a lot of experience working with Neo4j and graphs. So the package is currently in version 0.14 so it's in beta we're still developing but it is usable. All you need is Python 3.10 or greater. You'll need an open AI API key for most of the modules in this package and if you'd like to visualize any of the data models, you'll need GraphViz installed as well. So there's five primary modules in this package. There's data discovery, graph data modeling, there's ingestion code generation, the actual data ingest module. And then thanks to Dan Bukowski, we recently just added the Graph Exploratory Data Analysis module as well. And we'll cover all of these in the example notebooks. So as we go through these examples and you think about how you could use this package, just keep in mind the goal here is to get roughly an 80% functional graph. And so what we mean by that is that the output that you get from Runway is probably not going to 80% functional graph. And so what we mean by that is that the output that you get from Runway is probably not going to be a production graph. And so you'll need to take that and massage it a bit, maybe add some properties, add relationships and nodes where maybe it didn't exist. But out of the gate, you will be able to explore your data as a graph, and you'll be able to get insights immediately. hopefully it'll get the gears turning get you thinking about how you could make this even better um runway is not fully autonomous so there's multiple times where a human can step in and kind of change the behavior of what's going on um and what's really important here is that we still need domain expertise um runway does not know your data as well as you do. And so upfront will provide a data dictionary that takes away a lot of the assumptions that would otherwise have to be made in the modeling process. So here we'll go to our first example. And we'll zoom in a little bit. So in this notebook, we'll go with an end-to-end solution using Runway, and we're going to have a couple CSVs with just some data on countries. And so up here, we have just our imports, and then in here, we're loading our data. And so when we load data in Runway, what I would prefer to do is use this load local files function. And so when we use this function, we pass a data directory, we can pass some use cases that we want Runway to keep in mind as it's creating a data model, and then we can pass a data dictionary as well. And so we'll take a look at what this data dictionary looks like. If you notice, we're loading this from a YAML file. And so this is how I would recommend storing your data dictionary to be used with Runway. And so we see we have this primary files list, right? And so we have a file name here, it's countries.csv. And then each file has a list of columns. And so a column consists of a name, a description, and an alias. This alias is optional. And so what this specifies is that when we have multiple files, sometimes a column can be known as something else in another file. And so we can tell Runway that this ID column in countries is also known as a country ID down here. If we go to the states.csv file, we see that it has a country ID. We can also see that we can specify to ignore a column as well. And so what this does is that it lets Runway know that this column exists, but it won't be used in the modeling process. And so here we're ignoring this country ID column because this is just a foreign key, and so we don't actually want this to be used in our data modeling. And so once we have all of these declared, we can just run this, and this will load all of our data into Runway's internal table collection class. And so this just handles all the data and make sure that the data dictionaries are associated with the proper tables and so on. So just so you have an idea of what we're working with here, we can see here is just the head of the countries table and then we have states and cities as well and so this is we're going to combine all of this into a single graph data model. So once we have our data loaded we can use the discovery module and so what we do here is we declare a LLM that we want to use so So Runway has two types of LLMs. It has a discovery LLM, which we're using here, and a data modeling LLM. These both serve different functions. So that's why we have two different ones. So we're going to use GPT-40-mini, and we're going to run this in async because we have multiple files. So we can just run all of this in parallel. And so all we have to do is just pass the LLM that we want to use and give it the data. And then when we run this run async method, this is going to run some analytics using pandas. Then it's going to format a prompt, pass it to, in this case, OpenAI and generate some insights about how we can use this data in a graph and here we're specifying to do this in three calls so in three calls we'll analyze all the CSVs and then we'll collect that all into an aggregated discovery string as well that contains all the important information and that's what we're seeing here where we have unique IDs that it's found, some significant properties, we have possible labels and possible relationships, and then we also get how it's going to address the use cases that we specified. And so all of this is going to allow a very rich context for when we actually create our data model. We can also optionally save this to a markdown or text file, and we can actually load this back in as well, if you want to bring this in later for another session. And so once we have our discovery, we can import our graph data modeler. Here, we're initializing a different LLM. In this case, we're using GPT-4O, and we can also specify some model parameters as well. In this case, we're using GPT-4O. And we can also specify some model parameters as well. In this case, we're just specifying a temperature. And this is possible with both of the LLM classes. And so here, all we do is we pass in the LLM that we want to use, and then we give it that entire discovery object that we were just using in the cells above. And by doing this, Runway is able to pull in all of the important information it needs to generate data modeling prompts and interact with OpenAI. So we can run this. So we wanna create our very first model. So we say create initial model. And there's some parameters you can specify here, such as enforcing uniqueness and such, but we're just gonna leave everything at default. And so we can see that we get some nodes first, and these are all valid. So behind the scenes, we're using Pydantic and Instructor, which are Python packages that allow for a lot of advanced validation of your LLM results and your Python objects. And so we see these are all valid and then we generate some relationships as well and we see that our entire data model is valid. And we can visualize the data model that we get back since we have graph viz installed. So we can access the current model property on our graph data modeler and then we can use the visualize method on that data model. And that's what we get below. And we can see that we have a city that's located in its state, and this belongs to a country, and this country is part of a sub-region. And this is fine as a first pass, but I would like region to be its own node. And although we can do the analysis on currency that we asked about in our use cases, I would like currency to be its own node as well, because that will allow us to easily visualize the shared currencies between different countries. And so if you want to make any changes, it's just as easy as specifying a string that says what you want to change. So you just say, hey, create some currency and region nodes. And then once we already have a data model, we can just run this iterate model method and pass our corrections. And so this will attempt to apply those corrections to the most recent data model that you have in your graph data modeler class. And so we see, we receive a valid model, and this looks right. We can also visualize this again. Notice how the current model is also incremented so we're pointing at the model that we just created. And this looks the same and then we also have currency coming out of country and we have region as its own node as well. And so I like this data model much better. I feel like we can get a lot better insights out of this. So once we're happy with our data model, we can generate some ingestion code. And so to do that, we use this PyIngest config generator. And so all we need to do is pass the data model that we'd like to generate code for. And then we pass some connection credentials so that we can actually connect to our Neo4j instance when we run the ingestion code. And so all we need to do is just say, hey, generate a configuration string. And you don't need to know any of the specifics here. All you need to know is that we pass this configuration string to this Pyingest function in Neo4j Runway. And all we do is we pass that to Pyingest, hit Run, and this will ingest our data. And so we can take a look here after we ingest, and this is what that looks like. And so we can see that we have regions here in green, we have our subregions, our countries, and we can see all these currencies in yellow. And so we can easily visualize immediately that we have some currencies such as USD that are shared between countries and actually span across different sub-regions and regions. So it's very easy to immediately get insights out of this graph with the relationships that we've established. So we can also use the graph EDA module as well after we've ingested or even on a graph that you've generated without runway. And so this is going to be ran on that same country's data, although I have modified it a bit just to kind of see some of the other aspects of this graph EDA module. And so all we need is we need this Neo4j Graph class here. And so we just pass it some connection information. And then we pass this Graph instance to the Graph EDA class. And once we have that, we can see the available methods that we can run with the available methods property. And just for an example, you know, we can call database constraints and we get all the constraint information of our database. If we want to run all of these methods, we can just use the dot run method and pass it some optional method parameters, you know, for the specific methods if we want to tweak this a little bit. And if we want to refresh all the data, we can say true for refresh. All of the methods have this argument. Otherwise, it just uses the results that are in the internal cache for the Graph EDA module. And so once we have some information in the cache, we can run create EDA report and we can specify what we'd like to include there with a couple different arguments and we can view that report and here we can see that we have information on node counts relationship counts indexes etc and then we also have information like unlabeled nodes or we have disconnected nodes with the node ID. So we can go back into our database and search for these nodes explicitly and see where the issue is. Like maybe there's something wrong with our ingest pipeline, etc. We can also see node degrees and we can see nodes that might be a problem if we're trying to tune our cipher because high degree nodes could slow down our cipher queries. And if you'd like to persist this report, you can just save the report, and this will output a markdown file with all the information contained inside. So to summarize, we're using LLMs here to streamline the end-to-end data discovery to ingestion process. We're enabling new developers, hopefully, with best practices using graph databases. And we're also accelerating time to value. So hopefully this allows you to easily get your data into a graph and start playing with it on a quick timeline. So are there any questions?
NODES 2024 - LLM-Assisted Data Exploration With the Neo4j Runway Python Library
In this session, Alex Gilmore will guide you through how to use the Neo4j Runway Python library to explore your relational data as a graph. Runway abstracts communicate with LLMs to assist in data discovery and graph data model generation. It also provides methods to import data models from other tools, such as arrows.app, and automatically generate ingestion code. It can also easily load your data into Neo4j.

Get certified with GraphAcademy: https://dev.neo4j.com/learngraph
Neo4j AuraDB https://dev.neo4j.com/auradb
Knowledge Graph Builder https://dev.neo4j.com/KGBuilder
Neo4j GenAI https://dev.neo4j.com/graphrag