Graph Intelligence: Enhance Reasoning and Retrieval Using Graph Analytics - Alison & Andreas, Neo4j

0:00 [Music] Hey y'all. How are you doing? How's the day? I'm pretty excited to be here. Um, I was here last year and then we were at the summit, the agent summit not that long ago. I've definitely seen some familiar faces in all of those places. So, thank you for coming to our conversation today. Um, my name is Allison Cosette. I am with the developer relations team. You will likely recognize this face from all of the graph rag um videos on deep learning,

0:46 chilling with Andrew Ang like, you know, all the guys. Yeah. Yeah. It's just it's just how you roll. Um, so what I'm going to be talking to you about today, so was anybody in the Neo4j workshop this morning with Zach? Oh, this is great. I love it. It's like Neo4j did day here. Um, so yeah, we're going to do a few things. So that's good. Um, you may have Well, no, I'm going to just go like you didn't do that. Okay. Um, so this is our repo that we're going to be working with today. It has a Yes. So, is it is it not there? Because I can see your profile, but if I can't see your Yeah, same. I do. Uh, you know, that would not surprise me that I made

1:32 that mistake. One moment, please. Let's see my settings. Yeah, it probably I think it defaults to I know it's in here somewhere. Change visibility. Change to public. Sorry. All right. Oh, now I gota so much security. All right. [Music] Sorry about that, folks. Yes.

2:17 Confirming access. Okay. Is that better? Excellent. All right. Sorry about that. Um, so yes. Oh, so you can actually see. Yeah, I'm always in dark mode. Sorry. Can we do Can we do that? I don't know. Just these lights along here. Yeah, just the lights in front of the video screen. We'll work on that. Um, I love that people are raising their hand. This makes me very excited. Like, I love talking to people. I love being in the room with folks. Um, so this is good. So I'll tell you a little bit about what our goals are for our next, you know, 120 minutes together. What we're

3:02 focusing on, obviously a lot of you went to the intro to graph rag this morning. How many people have been working on rag applications already? Yes. Yes. Yes. Excellent. How many people have been working on graph rag already? A good number. Okay. Um, so what I'm going to be doing today is talking specifically about how you can use graph data science to improve what you're working on in your um, in your applications. The reason that we are doing this is because one of the beauties of having a graph is you now get access to new ways of thinking and understanding about the data that you have. And so my goal today is to have everybody feel comfortable with some of

3:49 the basics of the algorithms of graph. And you know hopefully while you're working through this you can see something that's going to be of interest. There might be something that you want to try when you get back to your application. Um I do this a lot and I will say the most successful workshops we have are when people bring their real world scenarios to bear. So we will be going over the code but a lot of what we're going to be doing is talking through the the con the concepts and around um graph as a tool for managing your rag and we're really just going to be talking about graph algorithms. Anybody here come from the data science side of things? Oh all right we got a few. We got a few.

4:34 Um, one of the things that I think is really interesting about the term AI engineer is, you know, somebody said, "Oh, it's just engineering." Like, it's still pipelines. I'm like, "No, it's not. It's different, right? I came in from the AI side." And I can tell you that none of us, none of us look at things the same way that we did, you know, a few years back. It isn't the data scientists on one side and people putting things into production on the other. You know, we work really closely. And so what my hope now is like the there's this concept that people say all the time that you know developers are just building the pipes and that the data scientists are worried about the quality of the water inside the pipe but with AI engineers we're all worried about the pipes and the water right so

5:19 what I really want is to give you as much as I can about the algorithms and about the data science and get us rolling. So, let me see. Where's my repo? Where's my repo? Okay. Um, yeah. So, just clone this repo. You can you'll have it. We'll we're going to walk through it. It's um it's it's really just like a clickthrough of a number of Jupyter notebooks. Um but really like it's mostly about the conversation that we're going to have today is where we're going. So, obviously everybody here like we always start with the basics, right? I call them the the GPS moments. So, let's review all the things everybody knows first. We know that in our RAG applications, what we're really trying to do is we're really we're getting better answers, right? That's what

6:06 everybody here is struggling with. Um like some of the common things I hear people struggle with are um am I getting the right data? Do I have the right chunking strategy? Um how do I handle temporal data? How do I handle data quality? Um what are what are some of your biggest like rag challenges? Anybody want to share that? What just irks you about what you've been working on? Everybody's so quiet. I should have signed up a slack channel. Yeah. Specific couple of things. One is how to establish the relationship between the data. Two is the volume of data, right? Yes. Yeah. So, we're definitely going to talk about we're going to talk about that today. Um I think one of the things that

6:52 we found is that the management is um what? Oh, sorry. Um the the comment was um how to handle the volume of the data was a really big one and the other element was understanding the relationships among the data. So did you go to this morning's workshop? Okay. Well you'll we have lots of video on it so we can we can definitely get that to you. Yes I would add like temporal relationships. Yes. Yeah. Temporal is a really interesting thing. Um I don't I didn't I don't have temporal in the slides but we can definitely talk about temporal because it's really important for sure. Um one of the things that we find is you know I I I will give you my apologies upfront. I get super excited about graphs. So I apologize in advance if I

7:39 am overly enthusiastic about what I do because I love graphs and there's so many times where something comes up and you're like oh well if you use graph you could address it this way. So I just want to, you know, keep bringing these things up and let's talk through all of them. But what we're really looking at is how do we give this complete and curated response, right? What are we using for the external data? I mean, initially everybody dropped their stuff into Pine Cone or WVA, you hit a vector store. Um, is anybody now working with multiple databases feeding into their rag system a little bit? Yeah, still primarily vectors. primarily vectors. Yeah. Yeah. I mean, everybody uses vectors because they work, right? Um, and that's the starting point, but the

8:25 thing about the vector is the vector really is I the analogy I always say is it's like you have a bunch of little index cards. And so imagine if I'm the my first day on the job and everything there is to know about my is about what I have to do is on an index card and I say, "Oh, well, what do I know about this?" And then somebody finds the index card with this little blurb. And so like I've just got these little things, but I don't understand how it all puts together, right? We don't understand what is it connected to, right? What is it connected to? What context do we need? And so what I want you to start thinking about is what are the projects that you've already been working on? When you're getting those answers back, what are some of the pieces of information that you wish your vector had? Right? Anybody have an idea of something they you don't have to share,

9:11 but like you've thought about like, oh, if only it could do this, right? If only it knew about that, I would get a much better answer. If you haven't, then I'm sure like the your business folks that you're responding to have because they always have an opinion, right? So, one of the things about the knowledge graph is that it's a way for us to bring our structured data and our unstructured data together. The other thing that it will do is it will actually find knowledge inside of your unstructured documents. Right? So, one of the things you brought up was how do we create the relationships among these documents or among these pieces of information? And there's a couple of ways that happens. One is it's really explicit, right?

9:57 There's a table of information inside this document and you can pull that out, right? Old school reax, everybody's friend. or it's something that is less explicit and more implicit and that's where the LLMs come in finding out what is in there. So in the this morning's one and if you um want to hit us up we can definitely show you how we actually extract and build the knowledge graph. What we're working with today is I have a a database already set for you folks that is a data dump that we're going to load into your pro trials. But the thing is this data comes from lots of different places. So I guess most people here are familiar with the graph. But just in case I do like the my little GPS moments. Let's talk about all the things

10:44 we already know. What is a graph? Right? Basically we start with nodes. Nodes are entities. They're your nouns. They're your items. Some people relate them to the tables in your, you know, relational database. These are your table names, right? But the important thing is the relationships among those entities. So a customer and a product, a product and a part, a you know supplier and a buyer. It's the relationships in and among these pieces of information that's where our business and where our information lives. In this case, we've got two people. One of them owns a car, one of them drives the car. They do know each other. I'm not sure why person one lives with person two, but not the other way around. Who knows? Life is interesting, right? Um so we have these relationships and this tells us what is the the story

11:30 of the interaction am between and among these and properties. This is where we get into the nitty-gritty. We get into the details. These are those extra columns within the table, right? Our relationships are our foreign keys. Our properties are the columns. The entities are the tables like roughly, right? You know, don't quote me on it, but pretty close, right? as from a conceptual point. Um these properties can live on the entities. They can also live on the relationships, right? So we can see that this person has been driving the car since this date. It's a very old car now. Um but one of the other things that we're going to see is we see underneath this car, we see a couple of things.

12:15 We've got uh the brand, the model, we have a description of the car, and what you'll see underneath is the description embedding. So, we're all familiar with the embeddings, right? The embedding is a way of taking some piece of text, putting it into, you know, many, many dimensions, and creating a vector in space that represents the semantics of whatever that text is. One of the things that's really great about a graph is that you have immediately that connection from the embedding to all of the other information. We see it in this microcosm here of the car, the embedding of the car. And you can pretty quickly see, oh, if I did a vector search on, you know, this car, right? Um, and I wanted to

13:01 know who are the people that own cars described like X. It's a pretty quick traversal. You're going to do a vector search. It's going to take you to that node, that entity, right? Similar to your chunks that you've already been working with. But immediately you already have all the relationships built in because they're already in the database, right? We don't have to we don't have to like create new tables and views. Everything's already there and it's connected. So when you go into graph rag and you're working with your retrievers, the beauty of the retriever is in this ability to traverse and traverse very quickly and get that immediate context. Right? So these are the basic components of what we're going to be working with within the database.

13:46 So we're going to start at the very beginning, right? How does this how does this work? We start with the entity. The first entity is the document and the document has a chunk, right? Pretty straightforward. It's a row in your vector store, right? But this is the interesting thing. You also have like anybody here working with agents already? MCP servers. Yeah. The other thing that we get and what we're going to be talking a lot about today is the application. I call it the application graph. Like they call it a memory graph. For me, it's what is the actual activity that's happening in the system? What is what is it doing? What are the messages going in and out? What are the context documents that are being used? Right?

14:32 Everything that we're building is moving quickly and it's moving at volume. There's masses amount massive amounts of documents and data. There's so so much volume. How do we actually manage it? What do we do? This is going to be a key part of what we talk about today, which is the actual understanding of what's happening in the application and how by monitoring and looking at what's going on in the application, you can do things like manage your documents at scale. you can understand what are the actual important documents, what's influencing outcomes, right? Um I've got some some good stories to to share with you about what people have been working on. Um one of the things you mentioned is what other elements are there. So when we look at the entirety of the graph, we have like

15:17 the memory graph or the application. it's connected to the chunks and then those chunks are connected to the domain, the products, the people, basically all the other structured pieces of your business or the structured elements that you've extracted from those unstructured documents. So what you end up with is you end up with a fully connected network of your data and your system together. So much of the way that we have approached development prior is you had your data people over here and your building people over there. We're all together now, right? We all work together. And the way that we have to build is we build from that cohesive point. We build from the data and the application at the same time. So I'm just going to show you what becomes

16:04 possible as you start taking this kind of an approach. Everything that we're looking at today is just this little bit of the graph, right? So, we're going to be looking at prompts and responses, and we're going to be looking at the context documents that they're connected to. And then from there, we're going to say, "Okay, let's let's figure out what where we go now and what we can start to understand." Anybody have any questions so far? All sounds really obvious, like why are you still talking about this? Too slow, too fast? Are we good? You're good. Okay. All right. Good pacing. Um, if I go too fast, because I'm from New York, I'm a fast talker. let me know if you're getting bored just kind of give me one of these and be like okay like let's go. Um so that's fine too. Uh but the best

16:50 thing that you can do is like when you have a question just raise your hand because everybody has the same kinds of questions. Everybody's struggling with things. And what I love about AI engineer is I get to be with the people who are doing the things that are like you are the people that are making everything happen right now, right? I mean, our entire world is in like just this incredible transformation and y'all are the people that are building it. And so I want to be here to support you in this moment because it's a pretty exciting time. So please don't be shy. Questions, comments? Yes. Oh, I will I will drop it in. It's not in right now, but I will drop it in before uh before we leave. Thanks. Yeah. All right. Um that's a good question.

17:35 Differentiate between the previous slide. Yeah. Oops. Yeah. So, they're all the same system. They're really just sort of like neighborhoods, right? They're kind of neighborhoods in the system. Um the reason that we draw it out is because oftent times people think of them as separate, right? So um this one doesn't have everything, but what we see here is we've got this has a this one has the source. So the yellow nodes are things that are coming from your application. The blues came from unstructured data. Um, we're not doing it today, but if you had structured data that you were also pulling in like a bringing in a table, bringing in some of or whether it's, you know, already structured or has been brought out of your unstructured data,

18:22 that would be just a different color. It's all one system. But the reason that we do this is just so that people conceptually understand where it's coming from and like what does it do? Because the big thing really is understanding the fact that once it's all connected, your application is talking to all of the data and then your structured and your unstructured are all working together and at any time you have access to understanding across that system. It's like you know complex systems and system thinking and all that good juiciness. So they're not separate, they're just neighborhoods. Yes. So So um when you you proposing that you have like one

19:07 massive graph? Yeah. I mean, don't start with a massive graph, right? Like, let's be honest, let's start with something small. I mean, my first piece of advice is always start small. Um, the other thing to really, as always, we always want to look at optimization. We want to make sure that we're building for the latency that we need. There there's always going to be context. Yes. Can you elaborate on that? I mentioned it this morning as well. Yeah. start small, but when you're trying to map, for example, the ontology of of an organization and be able to have an assistant have access to that. Yeah. We've always had to go big. How do you how do you how do you do small break it apart or Yeah. Um well, I mean, that's probably an ABK question. Yeah. So, some of the complexity like for for really big graphs, part of the bigness of it is not just the number of

19:53 nodes and relationships you have, but the number of labels and relationship types that you have. And for like a large organization, don't oversp specify. So you start up with like kind of generic terms, see if that works. And if you've got too many too much volume in the generic terms to segment, then you get more specific. And so and the other part about that I suppose is that you start with not all of the data, but like some of the data, work with it, do some eval, figure out if it's doing what you want to, and then refine before you suck in everything and create the entire database. So you kind of iterate on the schema with a subset of the data. It's normal data engineering I guess, right? Yeah. Do just some of the data, start generic and then get more specific. Yep.

20:38 Follow when you say sorry the um when you say um big versus small, you're talking ontology, not knowledge graph, right? Correct. Yeah. Yeah, that's right. So So talking about the ontology or the schema like the how the graph is actually laid out rather than the number of each kinds of things, right? So starting there iterating on that first and then bring in all the data. Yeah. Yeah. What are your thoughts on the raptor indexes or representing indexes in a tree based structure on based on some versus knowledge graphs? What performs better in what scenarios? That's definitely a good uh the raptor based indexes like if you represent content in more hierarchical treebased structures and then you are having summaries and a hierarchical nodes searching through those versus knowledge

21:24 graphs. Which scenarios plays out better? like we should go with wrapper indexes versus knowledge graphs or yeah the right answer is the annoying answer of it depends and so you should have your eval drive the structure right so come up with some eval what are what are you actually trying to do what are the questions people are going to ask and it's the usual thing the questions people that you're going to ask or people are going to ask should drive the data model here the ontology but that also ends up meaning do you put things into summarization over communities or do you have the knowledge graph representation of it that's going to be driven by the questions Y makes sense. Thanks. Y So if you can find your way to that workshop near forj, I've just pasted in an open AI key there. You're going to end up needing that at some point um while we get this repo updated. And I

22:11 guess at the meantime as well, I'll just answer any arbitrary questions you have. Yeah, dismbiguation. There are two channels. One is Neo4j Works of Neo4j and the other one works Neo Forj 2025. Yeah, it's neither of those really. It's the graph analytics workshop. What? No, I've done so many different workshops lately. Like it's Yeah, I'll clean up the repos. Sorry. The Slack channel. Yeah. Oh, no. So, oh, you have a Slack channel. There's a Slack channel on the AI engineers workspace. There's two. Okay. So, it should be the workshop new-j one, not the workshop new forj 2025. I didn't realize there was yet another one. Okay. All right. All right. Yeah. Thank you Devon for putting in the

22:56 the GitHub link. Hold on. Hold the phone because do this. So one of the things that we're going to end up exploring here and we we touched on this a little bit this morning um some of the motivation behind both graph rag and then also doing graph analytics on top of the graph. One of the aspects I I always like to call out and it isn't as as obvious I think until you gets pointed out and then it's super obvious is that part of what we're doing is for the core data you start with particularly with unstructured data if you're doing chunking of course that by using graph rag and graph analytics you're expanding the number of answerable questions for the same base data set that if you would say roughly for every chunk you've got you can

23:42 answer one question right and it's being linear Number of chunks equals number of questions you can answer. If you would think for every chunk you've got, you do the and this is the upper bounds. Take every pair of chunks do the cartisian product of all the chunks. Say what do these two chunks have together have in common or how are they different? Create relationships or don't create relationships in them in the upper bounds that is now you know n squared number of possible connections. Every time you do that that's new information that's possible. So now you've got an order of magnitude more, you know, questions that can be answered. Once you've got everything connected, you can then also do the community detection stuff that they touched on this morning and we'll get into maybe a little bit today as well. That now that you've got all these chunks connected, it's not just the pair-wise connections that are

24:29 new information. It's the subsets that are through those connections that also can answer new bits of information. So that if you have if I'm in the graph and you're in the graph and you're in the graph, you know, or you over there, whoever, maybe Allison, that it's not just us individually, but like how we're related and who we know, the connections between us and the connections across everybody else. All that ends up being slices of the graph that are particular to the individual places that you started. That ends up forming all this. It's not the full power set of all the subsets within the set but it's hugely amount like if you started with like 10 chunks by just going to the pair wise connections you end up with 100 answerable questions and once you start doing the the sub communities so the subset of all the sets at the upper end

25:15 it ends up being exponential. This is the power of graphs. Yes, it's two to the n instead of n squ right we do love math. Thank you ABK. Um okay so we have pushed if you have pulled you will want to refresh this poll uh this fork. So um what you're going to see is a few different things. One is uh I do have a link actually to the slides and then um actually we're not doing docker you can ignore that. Um but when we go down to the bottom in connecting to Neoforj or a DB you're going to see the link to um you're going to see the link to console neoforj.io. So we're going to have everybody go there. When you get in there, um, you're going to create a new instance. So, let

26:00 me Oops, sorry. Yeah, it's console.ne4j.io. Uh, this is in the readme in the repository. Do you not? Let me see. Is it in the slack channel? Which which Slack are we? Are we 2025? Not the 2025. We're not 2025. We're old school. We're not 2025. We're either in the future or deep in the past, but we are not right now. All right. We're the other one. Um so when you go in there, you're going to have a couple of different options for free tiers. One is um the aura free in the aura free, which is smaller, doesn't work as it doesn't have as much of the optimization as the pro trial. So we're

26:47 going to be using the pro trial today. Um I believe it's 14 days right now on the prot trial but you can if you want to work on your own project later go in and start an aura free instance as well and um that as long as you're querying it every so often like it'll stay open and it will always stay free. Yes. What's that? Is there a preference on the cloud provider? Um no it's your choice. Your choice. What was the question? Oh, sorry. The question was, "Is there a preference for cloud provider?" And I said, "No, there is not. The more you spread it out, the easier it is." We move along. Um, yeah. So, you're going to get the free trial up and running. And so, um, what that is

27:32 going to look like when you create the instance, you're going to see a couple of options. You're going to go with the Ora DB Professional. We're not taking our credit card. We're not doing anything. Eventually, it'll turn off. Um, but it just is the most robust version that we have. So, I guess it's seven days. Seven days. So, you're just going to try that for free and you're going to get that up and running. Why do we still have an ORADS button? No. Okay. All right. Don't worry about that. Yes. Oh, yeah. Um, onboarding. Sorry. Um, yeah, it just passed. I mean, it's going to ask you some questions. You can just like it does that thing where it makes you make the graph thing. Oh yeah. Sorry. Yeah. It doesn't I mean anything is

28:19 fine. Like don't tell anybody. It's a marketing thing. Like just pass through it. We want to just get you enabled. Um and then when you go through you can just use the default. It's going to default to four gigs. Um you know if you want to name your instance you may. One of the things that's going to you can definitely turn on graph analytics and vector optimization if you want. Um, and then just click accept and and it'll start up your instance and it'll take a couple of minutes. Once that instance is up and running, we'll get you to the dump and we'll get that loaded in. So, I'm just going to give everybody a couple of minutes to work on this. If you get stuck on this process, just raise your hand because I got lots of my people, my Neo4j people, wave.

29:04 Alexi, that's you, too. Wave. Yeah. Yeah. Yes. You're being he's being called on in class, right? Um he wasn't asleep, so that's good. He was just working. Um yeah, so if you get stuck on anything, just let us know. Okay. But what I can do while we're while y'all are getting that up and running is I can show you a little bit about what is inside of our database. Um the the console, it makes it really easy. Not only do you have access to your instances, you have access to graph analytics, but um you know the basic IDE, which is the query uh will be connected as well. So, got one of each. Let's see what's in the database. Oh, I No, that's right. I I emptied it because I wanted you guys to be able to

29:52 like do the dump with me. Okay. Oh, no, there it is. Okay. All right. Okay. So, we've got the query. And in the query, for those of you who love your cipher, you know, we can say return n limit 10. Um, all the cipher that we're using today, like I've already I've already got in the notebooks. Um, but what the what you'll see is when when you do a query, querying in cipher is really all about the pattern. So in this case, I'm just doing a match. N you probably can't see. I was gonna ask you now. Yeah, I'll fix it. Hold on. I gotta make this so people can see all the things. See all that? No, that's not going to work. We're gonna zoom in. Um, sorry. Yes.

30:40 Oh, okay. Okay. Um, if you already have an account Oh, yeah. Oh, you know what you can do? If you already have an account, one of the things you can do is you can log back in and if you change your email to your email plus like workshop before the at@ symbol, it'll give you a fresh account. So like if if like mine if I'm like Allison at you know Google, if I do Allison plus workshop at Google, it'll still be your email address. It'll still go to you, but it will be seen as a new account. So that's another option. Create a new account. Yeah, just create a new account. Okay. It'll make your life easy. Has anybody been able to get their theirs up and running

31:27 is? No. Yeah, maybe. It's so interesting because lately I've been noticing like people are so interested in engaging in conversation. There's uh less of that, but we definitely don't want to leave anybody behind. So, we'll let you all do that. So, but just to understand what is actually happening inside Neoforj when you're working with it. Um we run these we run these patterns, right? So it's similar to you know SQL like the good news is it's very userfriendly cipher um and we live in the age of like vibe coding and LLM so like there's plenty of help for all of this within our co-pilots um but depending on what you get back in this case you can look at it as a graph you can see the table um or you can see you know just the raw values but the the graph itself so um in this case we're looking at a particular

32:14 document which is actually you know we call call it a document. It's really the chunk. Um I will tell you that what we have in our what we the data that we're working with is from a project that we actually have live. Um and it's called agent neo and it was built a while ago now with uh by some of the guys who were helping build the first rag application on our graph data science platform um and our knowledge share there. So this has been in play for a while. But when we look at the data model itself, right, what you're going to see is what I talked about before, right? That application understanding. So we've got a session. So somebody logs in to the application, they start a conversation. They have their first message, right? This is the user prompt. Then that prompt is going to get an answer, right?

33:00 There there's a message that comes back from the assistant. And then that assistant is going to be going out and pulling in context documents, right? So it's going to do your cosign similarity. who does uklitian I don't know it's a thing I guess um does your cosign similarity and it's going to bring those documents back and you're we're going to start to see the chain of what that conversation looks like right so this is that application base and then these are these are the chunks so it's really just what we looked at in that previous view but just to show you what it actually looks like so what's in this you know lots of documents from our documentation developer blogs support Um, we, this was early on, our earliest chunking. We had

33:46 a 512 chunk size. Um, we used a lang chain recursive splitter. So, it was pretty early on. Um, let me see. Hold on a second. I want to show you. Where are my That's not what I want to show you. Where did it go? Here we go. All right. So once the the system actually starts running, what you see is this is this is actually one user having two different conversations. This is their first conversation. This is our second conversation. The first question is can you build graphs for me? And then the next one was what is GDS? And then you see the beginning of what that response was. Yes, I can build for you. GDS

34:32 stands for um and so you can just to give you an idea of how this starts to build out. Now this might look a little bit frightening but I'll walk you through what we're actually looking at. So what does it mean when we start looking in practice? So this is the same user right with those initial two conversations. Um what does graph data science uh yeah what is GDS? This is also what is GDS and then can you build graphs for me? And what we're looking at here is these are the conversations and these are the context documents that are being brought in. And what's interesting or plausible it makes sense. This person asked the same opening question. So it stands to reason that these same documents are going to be used in both of these first responses. Right? Makes sense. Then what we find is this

35:20 person's next conversation says um how can I have how can I learn more about GDS? is their follow-up question, right? And that makes sense that it's going to now share things with can you build because this person is then having their next prompt which is oh no that that one stopped after one. But what I wanted to show you is that what we see here is that in this middle conversation, the conversation started out in this area of information, right? It started out in what is GDS? What are some of the basics? But then the conversation went into something similar to someone else. And the reason that we want you to see this is you want to be able to see how are the conversations connecting to the

36:06 data itself because what we're going to start to see is we're going to start to see that it's not just about a single row. It's not just about a vector because the way the documents are used and the way people travel through your data gives you information about what they're using. The reason that becomes helpful is when you're trying to manage at scale, how do I know with all these documents? What are the ones that I should even be looking at? What is it that people are constantly going back to? What are the areas of knowledge where things are happening? And so what we're seeing here is that these elements right here were in both answers, right? So you see this first question had this first response had context and this had context. So it was in both of those. These are individuals. These over here were only in this one. Some of those were only in that one. You can imagine

36:52 like how quickly this gets very very hairy. Oh, where did it go? No, I guess that's it. I didn't have the extra one. Um so the the point is we can we can start to understand the movement of what's happening. Is there a question someone? No. Okay. Anybody have any questions so far? Ponderings? Waiting till we get to juicy things. Yes. Yes. Yes. The pink ones are the chunks. The orange is the user prompt. The tan is the um the the rag application's response. So you can already see how it's starting to connect to the system itself, right?

37:38 So let me get back on point here. We got things. We got things things. Okay. And sorry how performant is that? Yeah. So what we're going to see is we are actually we do actually have the chunk. Um our chunks are pretty small. Um I the size of the of the text doesn't really change the performance, does it? Like as far as like once it's stored in the property, it's really the traversal that is where things for most chunk sizes it's fine because most chunks are like in the K at most. It's not megabytes of chunk, right? If there's megabytes then it's like okay that becomes problematic. Yeah. I mean but it would be that for any database I would

38:23 imagine. Yep. Yeah. Um but what we see is is like you know we said we had these properties. So for this particular document, um I've run some of these things already, but we've got the document, we've got the text, and we have the embedding. The embedding is actually a property of the node itself. And that's one of the things that makes it really easy is that you don't have to have a separate vector database, right? The vector is actually stored and then the vector index is built on top. So once you get all those vectors in, you can run the vector index. And people do really interesting things with vector indexes for sure. Um, so this is where that's not what I want. This is where we start. Question. You definitely have a question. Whether

39:09 I have an answer is the only thing that's up for debate. So let's assume, right? Um, I have very large say legal documents, right, which I'm trying to do something very similar. Yeah. And I do these chunkings, you know. Yeah. Would I are you saying I would store the chunks in the store the vectors in a vector database, but I'll also store that in. Yeah. And how would that be from a performance standpoint over time? Like assuming your system continues to grow, um, wouldn't there be a performance impact? Um, I don't know. Would you know what the largest rag application is that we have in production in clients right now? It's a good question. I don't know. I know that's like, you know, tens of millions, you know, seems fine. Once you start to get to hundreds of millions, you might start to have to worry about like what

39:56 how to scale past that. Yes. And does it still query across look for similarity across all embeddings? Yeah. I mean, so it depends. So what it will do is it will look across whatever the vector index is that you have and there are people who I said they do some really interesting things with vector indices. So um I've seen people who like have multiple vector indices depending on what the use case is. So when you when you create the vector index, you actually just do a query and you say, "Okay, for all of these, create an index on all of these nodes." And so that's one way that I definitely know people have been doing it. So they'll have multiple indices that already sort of presort or prefilter what's going to be part of what's coming back from the vector index. So um I know one of the

40:41 ways I've seen people do it is from a governance perspective. So there's certain data that they want available to everybody, certain data they don't, and they manage that from the vector index. Yes, please. I'll let you go for a third. Really? Um, the other question is if we had documents that actually contained images and we wanted to somewhat have a relationship between the text and images, how would you do it here? Does that make sense? Yeah. I mean normally multimodal I think most of the time they keep the images separate and then there's a there's an embedding of the image on the node. That's right. Yeah. Yeah. So you'd put the image like large media files you would put that in an S3 bucket or something. You still wouldn't you wouldn't put that into the database. So you'd have a URL to the S3 bucket but

41:26 you can put the embedding into so on the node. Yeah. Yeah. We actually have an example of that. Yeah. I have a question for you have multilayer. Yeah. Yeah. I mean, it's not it's I mean, it's mostly just the way that the database is actually built because of the way the queries run because you're running on a pattern. um you actually I I kind of call it uh it's like GPS search where it's like start here and then go find what you need to find. Right? So I I don't know if that's the best way to

42:12 explain that, but um when you're running the query, it's going to be very specific. So when you're thinking about um like like how you're running the vector retriever itself, are you talking about like when you're actually doing the retrieval from the vectors? Yeah. So there's a whole um I don't even know how many how many retrievers there are right now. like eight different retrievers or something um in the the graph rag package. It's a Python based package. Um so there are some if you go to actually great place to go is graphrag.com. It's like an open like reference point and it talks about some of the different types of retrievers. So you can do pre-filtering, you can do postfiltering. Um you can manage it based on the index which is another like easy way to do it. Um you know sometimes folks will have them where they have different hierarchies. So like when you think

42:58 about these large legal documents, right? Um there are ways that you can actually like create the chunks but use the natural structure of the element as well. So like we have an example we use with SEC data. Every time somebody files something with the SEC, you know, paragraph one is always this, paragraph 7 is that, paragraph 9 is that. And so you'll take whatever that natural architecture is within the documents themselves and then you can you can leverage that as well. So, one of the other great things about um these entities is that you can actually have more than one label. Let me see. Oh, I got to plug in. So, one of the things that you'll see is

43:45 so for this particular this particular one, we see that it's a message, but it's also noted that it's the assistant. So you have some nodes that are that have more than one label sometimes. So it makes it so that if you wanted to go across all messages, but I know if I'm trying to understand um just the prompts from the just the assistant answers or just the prompts, I don't need all of those messages. So by adding on dual labels, you can actually say, "Oh, I just want to see these or I just want to see those." So it it it allows you to get some of that nuance out of that complexity by leveraging multiple labels on a single node. One of my favorite tricks personally, but that's just me.

44:31 Sorry. Yeah. I have a question. Ah, hi. So now like if you have one vector index but don't want to actually search the entire index, right? Yeah. So since that like let's take this text for example or like content for example. Uhhuh. you want to only search the content of few nodes via vector embedding. Is that possible? I mean, there's a number of ways that you can do that. Um, you know, you certainly could. I mean, I personally I'm a big fan of multiple indices just because I find them to be very efficient. Um, that's one way to do it. Um, there was a question earlier about temporal issues. Um you can also use like multiple vector indices to address that temporal portion as well right so

45:20 you can create an index from why is this all right it's starting up um you can create a vector index so like like if we wanted to like if we wanted to have a vector index just on you know version one nine or version 26 or whatever you could have that particular vector so let's just say I'm building building a chatbot and the I know that the customer is currently running whatever version of something, you can use that as the as the actual index that you're searching for. So that's another way that you can that you can address those kinds of things. Did that answer your question? So it's just like so it's just like multiple embeddings. I mean multiple indexes so not one index. I mean I don't know I don't know if engineering would agree with me but from

46:06 like a purely like practical like I got to hack this and get it done. That's my go-to. I mean, is there Yeah, I think that's the right approach right now with new forj. I I guess you're asking about metadata filtering within the vector index, right? Yeah. And so for us, the vector index is just an index. It's not a vector search, right? Um and so we don't do metadata filtering on the vector itself. We use the property graph itself for doing the filtering. Right? So you do it before you create the index because the index is only going to go to what's connected to it, right? Um so it's you'll still have the index, but then you can do predicates on top of the nodes you found and then you filter after that. Yeah. So, even within with once you get that, you can certainly filter in and around that as well. Y and if you look at um any of our the vector

46:52 retriever materials we have like we walk you through that. We've got we got a new ebook coming out. It's gone to print. It's happening um that specifically talks about those retrievers. All right. Multiple labels, multiple indices, all the good things. So, let's do this. So, for those of you who have, let's go back to here. Um, the other thing you're going to see on the readme is you're going to see this Neo4j dump file. Um, this is the actual dump file from from our database from last year, I think. Um, then so we're going to we're going to give that to you. So, you can download that file and I'll show you how you can drop that into your let me see. Sorry. [Music] All right. So when we go back to our

47:39 instances, so you should see instance instance 01 most likely. If you go on the right hand side of there, there are three little dots. What you're going to see is backup and restore. So you're going to click on backup and restore and you're going to take that dump file and either browse or just drag and drop it. and it's going to drop in um our our version of the Neo4j agent uh agent Neo GDS application data I guess things um so yeah so you just drag and drop it it runs pretty quick it'll load pretty quick um and then what you'll see is when you go to query you'll see this will start to populate the database information will populate and it'll show

48:25 you which nodes you have which nodes and entities you have. Um it'll bring in some of the indices. There is we did run some of the algorithms already. So I'm going to show you how to run them so you can run them on your own. Um but in the in the interest of time we we have run those already. So you'll see some of that. Some of them we'll we'll do live here. They won't be there. Is anybody able to load? Yes. Oh yeah. So on the readme on the readme in the repository if you go all the way down to the bottom where it says connect to Neoforj or a DB there's a Neoforj dump file that it'll take you to. Sorry. Yeah. And then you'll just download it

49:11 from there. What's that? Oh, sorry. So when you're in when you go to console, right, you should be seeing something like this, right? All right. And then on the right hand side for your instance, you'll see three little dots. We're going to click on those dots and you're going to go to backup and restore. And my data is yours. I give it to you. Just dumping it right on you. All right. Um yeah, it's just easier. Nobody wants to load CSVs today. We have no time for these things. Um yeah, and then from there you can go straight into the query. So when you go to query, you should see that it's populated. Is there anybody having trouble loading the data

49:57 that wants to load the data? It's still uploading. Okay. What's that? I couldn't find the if you if you did you you might have had to repole because I I if you repull the repo. Yeah, I had to update it because you know I'm well intentioned but sorry. Yes. Um when I've uh restored the dump file. Yeah. Should I expect that it will work in like few minutes? Yeah, it should. Let me see what you got. Um it keeps doing the loading. It's still loading. So that's normal. There's like a lat. It shouldn't be too slow. Oh, what is Oh, you're probably having network issues. Wi-Fi is pretty slow. Oh, sorry. Yeah, that's always the the challenge with

50:43 these things. Um, yeah, Wi-Fi. See, this is why we always have it loaded. I also have it loaded on my desktop because that happened to me recently. I was somewhere and I'm like, there's no Wi-Fi at all and I have to present. So, I also have Neo4j desktop. So, if anybody ever, you know, you want to go local, you want to stay on prem, you don't want any up in your business, you can also like stay off the cloud, go local also an option. Um, let's see, where are we? So, what are we going to do here? All right. So, I'm going to go into So, within our notebooks, we have three different notebooks. One is uh the get to know your graph. And we'll start there. I don't need this. Um, and really it's like all this is going to do is you

51:29 you'll run it, you'll do your pip install of all your requirements, and then what you'll what Oh, wait. I need to give you is the OpenAI key on here, too? Hold on. Oh, I put it in the Slack channel. Oh, it's in the Slack channel. Okay. So, anybody can find it anywhere in the conference. WhatsApp. Enjoy. Security. So, when you when you set up your database, they should have given you the opportunity to download your credentials. Yes, I've set that up. Got it. Um, what is this not not Oh, um, did you drop the So, you have to drop I sorry I skipped a spot. You do need to drop the those credentials that you downloaded into your in your env file. So you did that. This is your env.

52:17 Okay. And then just pasted it. Yeah, it should be. Let me see. Yeah, there it's right here. Yeah, you've got that. Um, what is it actually? What's the actual error? Is it just Wi-Fi? What does it say down here? Schema bolting. Yeah. Um, I was just Yeah, the URI should be in your credentials. Exactly. I was just I was just thinking I just It should be It should be I would take a peek at it. ABK, can you take a look and see what's happening here? All right. So

53:04 um yeah, so again the the API key is there. We're not using it actually that much today. Um but one of the things that we have obviously is we need to have Oh god, you can't see anything. This thing can't see anything. Oh goodness. Well, that's overwhelming. That's more reasonable. Okay, is that good? You guys can see we're good. Okay. Um yeah, so the basics obviously we're importing OS. We're bringing in our our environment variables. Um you see the Python driver, right? This is our classic Python driver. And then what I like to do is I like to just do a run query um uh function just because it just makes it easier. Like I I just like it. Um you don't have to do it this way.

53:51 You could just do drivers session, but um that's just how I like to do it. And then when you run this, it'll see that yes, you are actually connected. Let's see. Maybe I should actually run everything. Now we're connected. Okay. Um the first thing that we're going to do is we're just going to take a look at what is actually in the database. So in this um Apoch actually stands for awesome procedures on cipher which I just think is hilarious. I think there's a double meeting on Apoch too but we'll have to ask um isn't there an a another Apoch meaning something? I don't know. matrix. Yeah, some some matrixy thing. Um, but what this is going to show you is we start out with 17,000 nodes, right? We

54:38 have 774,000 relationships, um, seven different labels and 27 different relationship types. What does this tell us? What it tells us is obviously we got a lot of data. It's actually not that much really. um you know for something small um but you know like almost a million relationships and so really what you we're running this just to make sure that everything is loaded so everybody who has loaded their data are you seeing these numbers anyone anyone maybe maybe not no okay this is I will take you back one moment please hold the phone there are three notebooks inside the notebooks folder inside the repository.

55:24 So, you may have to repull it. You did. Huh? Huh? That's not right. Yeah. Yeah. This is not Can you reload this because it was updated? Yeah. That's so weird. having trouble getting the connection for Japan V file. It's easier to just copy the values into the cell where it's loading from. Yeah, you can do that, too. Yeah, that's my advice. Yep, that's actually really good advice. Um, the dump file. I'm not able to download. Uh, I'm uploading it to the Slack channel right now. Thank you. What's your name? Caleb. Caleb to the rescue.

56:10 Thank you, Caleb. Everybody, round of applause. Round of applause. You're that helpful guy at work, aren't you? You're like, "Oh, I can just do this thing. It'll make it easy for everybody." Thank you, Caleb. Nia is also Nia. Also, Nia, round of applause. We love Nia. You probably see if you're from San Francisco, Nia is all places, as is Alexi. They're around all the time. So, hopefully they're familiar faces for you all. Um, so yeah. So, the first one is just really meant to get you to connect to the graph and make sure that your connection is working. Um and just showing you how you can get some basic summary statistics. Um you know, one of the things that we can do is we can preview some of these documents. Um so again, this is our data model just to get you familiar with what

56:56 we're looking at. And then let's move on to our notebook number two is where things get get more exciting. Um, in notebook number two, what we what we're looking to do is we're looking to actually start leveraging these algorithms for where am I? Okay. So, when we talk about graphs like ultimately what we're trying to do is we're trying to help you do these things at scale. We're trying to help you be successful in production. We're trying to help you get the best possible outcome from the applications that you're building. And the way that I that I suggest is my little moniker which is connect cluster curate. To your point of how do we actually manage this at scale? What does that look like? What do we do? Um what we have in these notebooks walks

57:43 you through this very this process. The way that it works is we start out by running KN&N on similarity. So we take the embedding for each of those documents and we create uh we leverage a similarity score. So um basically we take a K of 25 we run KN&N and we understand we start making connections among very similar documents right you'll see why we do that in a minute but what this does is it sets us up for the next step which is community detection also known as clustering. Um, and what it allows you to do is it allows you to group these like items so that you can manage them together, right? It's very hard to look at any one and know what's happening, but we can

58:29 certainly look at all of the things. And then finally, what it leads us to is we curate the grounding data set via these techniques that work at scale. So this is a similarity graph of context documents um, you know, connected to their source URLs, right? So, it makes sense that there's a lot of overlap and similarity because they're coming from the same source. And you may be saying, well, if they're all the same, like, why do I need to know that they're similar? How does that help me? Well, it helps you in a few different ways. Uh, what this thing is going crazy. Um, what I first want to talk about is I want to talk talk a little bit about the math and the understanding of how community detection works, right? Um, I promise you there's no calculus and there's no quiz. Um, but when so like if we're looking at this doc, you know, we're looking at a few

59:15 different friends, a couple of friend groups, like it's pretty clear like there's one click, there's another click. It's pretty obvious, right? Like that seems reasonable. And then when we start getting bigger, like these are these are similar. And what we're seeing here, this is actually these are actually sized by page rank, which is an important score. Um, and they're still like it still seems pretty obvious, like yeah, they're clustering together. Like it's still pretty logical. Like I get it. Um but anybody who has worked with graph knows that it never looks like this. Has anybody ever actually seen anything that looks this tidy? No. Never. Because it looks like this. Right? This is what we're actually looking at. Right? In this case, what we've got is we've got all these different nodes. They're sized by page

1:00:01 rank and then the edges are weight. They're weighted edges. Right? It's very like the hairball people call it. um the bowl of spaghetti, whatever it is. And so like it's very complex. So how do we actually find things that work? How do we actually run comm Oh my god, it's so sensitive. My goodness. Um this is the boring math part, but just so that you understand like what is happening under the hood. So when you're doing clustering in graph, it's based on something called modularity optimization. And a community is high modularity, right? It's modular. Makes sense, right? When the items within or the connections within it are highly interconnected with each other and have a very low connection to things

1:00:47 outside, right? That's like the basics of the math. So, what it's going to do is the algorithm is going to go through and it's going to say, okay, for every node, is the modularity score going to go up or is it going to go down if I put the same label as my neighbor? Right? So, does it belong with this group or does it belong with that group? What does it do to the actual numbers? Right? Louvane is a particular type of algorithm and this is where it says it oh yeah it's exactly what we just talked about. So how does it actually work? So what it will do is it will start by giving a single node a label and then it does the calculation. and it says, "Okay, like let me figure out does it go up or does it go down and then it will do the first pass and it'll say, okay, these are these are the ones that are most like most connected to each other." And then it'll say, "Okay, well, let me

1:01:34 group and aggregate those and let me look at how many what you see in step two is you see the number of connections between each of those clusters, right? So, when we look at it, so you know, it's pretty straightforward. the pinks and the yellows. There's three edges between them, but there's 14 within just pink and there's two within yellow and four within green and 14 within blue, right? So, what it shows you is it's showing you that like there's a lot connected within and not so many on the outside, right? Pretty straightforward. That's pretty easy math, right? Um and then it just keeps going through and then eventually it goes down. One of the things to know, um there is another algorithm called Leiden. it is similar but it um but if you need to work with unconnected graphs so depending on

1:02:19 whether your graph is completely connected or not there are different ones but ultimately what it's doing is it's saying you know are are these thing which of these things is not like the other right so label propagation is a is another way of doing clustering and what label propagation does is it says okay I'm going to assign a random label to something or I I know these labels and then I'm going to go to the neighbors and I'm going going to say I'm going to assign what this one is to its neighbors and then eventually we have these iterations where they converge label propagation algorithm. Now the nice thing about the label propagation algorithm is it's actually one of the fastest clustering algorithms. So especially if you've got lots and lots of values and you want to run something very big, it's a very fast way to do it.

1:03:06 Um if you need something that's like highly nuanced, it's worth spending the time and the computation to use one of the other algorithms. But what I wanted to do is I just wanted to to show you what this looks like because label propagation it's and modularity does the same thing. It's really looking for density. It's looking for density in those graphs. So when we think about you know this the craziness that we see here like even in this like you can see like little pockets of density right we see these pockets of density. I always like to take the soft eye when we look at graphs. Um, and so that's what that's what's happening with the math under the hood, right? So what does this actually mean in our in our code? So what we've

1:03:51 got is let me close this out. So what we're going to do is the first thing you do is you're going to run the KN&N. So you run the KN&N on similarity. Obviously we connect as always. Um, there's a weird little thing you have to do. You have to create an empty relationship, but don't worry about that. like I gave it to you. Um, but that's a little little pe what it's probably because we don't need that. Um, but then we run the query and I just want to walk through how the query is actually built and what it looks like. So, the first thing that we have is do I have my I want I want to see this. Sorry, somebody's got a question over here. Yes. Then then over there, Caleb. There we go. Here we go. Did you have a

1:04:37 question? Yes. Yeah, [Music] graph.pro. So, just to go over like what what what the process is when you're when you're actually um running the algorithms. The first thing you need to do is you need to project a graph because most of the time when you're running the algorithm, you're not running it on the entire graph, right? We're running it on something small. So in our case we want to run k nearest neighbors just on the documents right. So what we need to do is we create this uh this projection. So here we've got we're calling the projection we're calling it doc. So the first thing you'll see is the name of the projection which is docs. What then what we look at is we look at which

1:05:22 nodes the next thing you're going to see is nodes. So for us we're using documents right? So we want documents. I want to include the property embedding.

…