Vector Search Finds Similar Facts, Graphs Establish Their Relationships

Stephen Chin built two agent-memory environments from the same original Markdown files describing his home lab. One stored the material in a vector database; the other used Cogni with Neo4j as a graph store. He then asked both systems questions that required connecting network topology, service exposure, and software status.

The vector-backed system did not identify a specific internet-exposed machine running obsolete software. Its response said it could not find the details and directed the user toward more precise information elsewhere. The graph-backed system identified tingsterland—Chin’s daughter’s Minecraft server—as a WAN-exposed guest running Debian 8 Jessie, which it flagged as out of date.

A second query asked which services bound to 0.0.0.0 on management ports were exposed beyond the LAN by pfSense. The vector-memory response suggested checking service configuration and pfSense rules. The graph-memory system returned HAProxy and OpenVPN, the two WAN-exposed services Chin expected to find.

The environment was deliberately constrained. Chin said the demo used a separate VLAN, segmented from his real network. It had been populated from information about that network but could not dynamically inspect hosts during the test; it could answer only from stored memory. The distinction he was testing was whether retrieval could assemble a relationship chain from retained facts, rather than merely return information that looked relevant.

Chin said the graph interface made that process visible. Blue nodes were vector-derived seed nodes; gray nodes were one-hop traversal results; green nodes were the nodes ultimately selected into the model context. For the management-port question, the traversal began at the pfSense router and followed its connected entities to the relevant services.

Stephen Chin does not reject vector search. He describes it as an improvement over loading every memory file into a prompt: store notes as embeddings, embed the query, and retrieve the closest material. He noted that OpenClaw includes pgvector and said his demonstration used pgvector and LanceDB.

But the score produced by vector retrieval measures similarity in vector space, not whether one entity owns another, exposes another, depends on another, or caused another state. His illustrative failure is an assistant seeking an apple and receiving an Apple laptop: a plausible semantic match, but not the object intended.

That distinction matters when an answer depends on linked facts. “Which shell is mine?” is not a request for the shell description most similar to a query; it requires an ownership relation, rendered on Chin’s slide as Clawd -[:OWNS]-> shell. A network question may likewise require a chain: a host runs a particular operating system; a service is attached to that host; and a router, firewall, or proxy makes that service reachable from the internet.

Chin’s claim is that similarity search can retrieve nearby material without natively supplying the path across it. In his account, vector-only retrieval can therefore return facts adjacent to a question without resolving it, and complex multi-hop work can also be expensive in traditional relational databases. Vectors remain useful for locating promising material; the limitation appears when identity, ownership, exposure, dependency, provenance, or another relationship is the thing the question asks the system to establish.

A graph represents entities as nodes, their typed connections as edges, and associated details as properties. Chin notes that embeddings can be stored in the graph as well. The architecture he demonstrated combines vector retrieval and graph traversal rather than treating them as alternatives.

Stephen Chin described vector search as the first step: it identifies seed nodes from which the system begins traversing. Graph search then pulls connected neighbors and ranks them by their relationship to the question. The result is not simply a nearest collection of text fragments, but a bounded subgraph organized around the connections that bear on the answer.

For a simple example, a graph can move from a person to their friend, from that friend to a stated preference, and from the preference to a location. In the homelab, the analogous traversal begins from infrastructure such as pfSense and follows relationships to services, hosts, addresses, and exposure status. The path—not just the presence of individually relevant facts—is what makes a result responsive.

Chin attributes three practical properties to this arrangement:

• Accuracy: answers are grounded in stored relationships rather than only nearest-neighbor similarity.
• Explainability: the traversal path can show how the answer was reached.
• Auditability: when an answer is wrong, a developer can inspect the nodes and edges that led to it.

Those properties create a debugging surface rather than simply a better presentation layer. Chin said a developer can inspect the retrieved subgraph, revise entity extraction, reduce duplicate nodes, or change the relationships being modeled. The agent’s context can be traced to a particular connected structure rather than treated as an opaque collection of relevant-looking passages.

The modeling prerequisites Chin identifies are compact but consequential:

• Extract the domain entities that need to be reasoned over.
• Define and store the relationships that questions will actually traverse.
• Use retrieved subgraphs to inspect why a result was selected.
• Address duplicate nodes so a traversal does not split one real entity across multiple representations.

He also argues that adoption does not require a developer to hand-write every graph query. An agent can write Cypher and assist with entity extraction, Chin said, as long as the developer has a basic model of the domain and the outcome sought. The modeling task remains consequential: the graph can only traverse relationships that have been extracted or represented.

The underlying problem Chin diagnoses begins before vector retrieval. In the OpenClaw-style arrangement he describes, an agent’s persistent state consists of Markdown files such as SOUL.md, AGENTS.md, USER.md, MEMORY.md, TOOLS.md, and dated memory files. They are readable, editable, and suitable for version control. But they are also flat files that must be loaded again to become usable.

Stephen Chin said his average agents load at least 100,000 tokens for each round as they accumulate skills, instructions, and retained material. The strategy is to load enough context in the hope that some part of it proves useful. He said that can work at small scale with a strong model, but does not hold as the retained world grows.

Skills do not automatically resolve the retrieval problem. Chin characterizes skills as folders containing SKILL.md instructions. An agent can have a useful recipe yet fail when the task is unfamiliar, choose the wrong recipe from a large library, or perform one skill without selecting the next required skill. He gives Hermes credit for reflecting after tasks and saving methods that worked, which he describes as remembering procedures rather than only facts. But a growing skill library still leaves the system to decide which recipe applies and how separate actions should compose.

Goose illustrates a related tradeoff. Chin describes Goose memory as another MCP server, with calls to remember, retrieve, and forget material stored as categorized and tagged Markdown files on disk. That makes memory pluggable and manipulable. It does not, in his framing, change the fact that saved memory remains a flat collection of files. He said additional saved memory adds tokens without creating explicit relationships, while a forget command puts deletion one tool call away.

The limitation is not Markdown itself. It is using a file stack as the only representation of a growing, connected domain. A stack can preserve facts and instructions; it does not inherently encode the route from a host to a dependency, a router to an exposed port, or one completed action to the next relevant action.

Chin’s home lab had only three or four nodes by his description, but it was sufficient to expose the difference between retrieving similar material and traversing a defined relationship chain. He said the demo also proved useful personally: after it identified the relevant issues, he patched the security holes in the lab.

For larger environments, Chin points to data centers, financial-services records, networks of companies and customer records, and other domains whose state does not fit within a model’s context window. These are illustrations of where he believes flat prompt-based memory becomes inadequate, not performance results demonstrated in the homelab test. Adding more tokens may retain more raw material, but it does not itself determine which chain of entities establishes an answer.

Stephen Chin frames graph memory as an alternative to treating the prompt as the entire working world. Vectors can locate a promising starting area. A graph can then establish whether the retrieved facts are connected through the relationships the question actually asks about. The returned subgraph can provide the context, the explanation, and a place to investigate when the answer is wrong.