Context Graph — Architecture & Data Flow

The one idea

The graph is the single source of truth

Context Graph is a labeled property graph of entities (nodes) and relations (edges), extended so that every edge can carry a RelationContext — the decision record: who, why, via which channel, under which policy, for how long. That graph is the truth. Everything else is built from it.

Around the graph sit vector indices — for entities, relations, chunks, and decisions. It is tempting to think of these as separate stores. They are not: they are projections of the graph and the source documents, kept for fast semantic lookup, and rebuildable at any time. A decision is not a row in a "decisions database" — it is an edge in the graph, and the decision index is just one projection of that edge.

◆ Why this matters

If the truth lives in one place and every index is derived, then drift is a non-event (rebuild the index), migration is a non-event (re-project into the new index), and no query surface can ever hold a "decision" the graph doesn't. Getting this wrong — treating an index as a second source of truth — is the single most expensive architectural mistake, and the one this design is built to avoid.

Foundations

The storage layer — four pluggable backends

Every stateful thing lives in one of four storage roles. Each is an interface with multiple implementations, so a deployment mixes and matches (file-based for a laptop, Neo4j + PostgreSQL for production).

Four roles, many backends. The default config runs Neo4j for the graph and file-based stores for the rest. Only the GRAPH is authoritative for entities, relations, and decisions.

Data structure

The graph & the fourth element

A node is an entity: an entity_id, an entity_type, a description (the text that gets embedded), and open properties. An edge is a relation: keywords (the relation type), a description, a weight, provenance — and, optionally, the RelationContext.

The quadruple (h, r, t, rc). Most edges are plain triples; a decision edge adds the rc. It's a property on the edge — connected and traversable, never a floating record.

RelationContext has 11 fields — the decision lineage. The important ones:

Field	Meaning
`decision_trace`	The rationale / why — the searchable heart of the decision.
`approved_by` · `approved_via`	Who decided, and through which channel (slack, email, jira, system…).
`policy_ref`	The policy it operated under.
`valid_from` · `valid_until`	Temporal validity — is it still in force?
`supporting_sentences`	Verbatim evidence quotes.
`provenance` · `confidence_score`	Source reference and extraction reliability.

The keystone

The derived indices

Four vector indices make the graph searchable by meaning. Each is a projection — embed some text drawn from the graph (or the source docs), keyed back to the node/edge it came from.

Index	What it embeds	Projected from
`entities_vdb`	each entity's `description`	graph nodes
`relationships_vdb`	each relation's `keywords + description`	graph edges (incl. decision edges)
`chunks_vdb`	raw document chunks	source documents
`decisions_vdb`	each `decision_trace`	graph edges that carry an rc

Truth at the top, projections below. The green arrows build the indices from the graph; the amber arrow is reindex_decisions() — rebuild the projections from the truth any time.

◆ The clarification we standardized on

A decision lives in two derived indices — relationships_vdb (so ordinary retrieval finds it, like any relation) and decisions_vdb (fast semantic precedent search). Both are projections of the same graph edge. That edge is where the decision is; the indices are how it's found.

Getting data in

Write paths

Data enters through several paths — all gated by the governance layers, all ending in the graph, with the indices updating alongside.

extraction ainsert()

Documents → chunk (~1200 tokens) → an LLM extracts entities, relations, and RelationContext from the prose → graph + entities/relationships/chunks indices. This is how a corpus becomes a graph, and one of the two paths to an rc (extracted).

emit emit_decision_trace()

Agent code writes a structured rc directly onto an edge at the moment a decision is made → graph edge + relationships_vdb + decisions_vdb. The second path to an rc (emitted). Runs the rules gate first.

graph CRUD /graph/entity·relation/create

Create nodes/edges directly (node → entities_vdb, edge → relationships_vdb). Used by backfill and manual curation. Give nodes a real description — that's the retrieval payload.

governed action /actions/invoke

RBAC → lifecycle → rules gate → emit_decision_trace → side effect → audit edge. Every standard operation, authorized and recorded.

web ingester /scrape

The web site processor: point it at any site and a polite, JS-capable crawler with pluggable platform connectors and an LLM site analyst (judges relevance, extracts the document URLs) pulls the real documents in — feeding the extraction path with the source URL carried through as provenance. Guide →

backfill backfill_git.py

Import an existing repo's reality — modules, author, commits, docs, and (with --code) source — then auto-reindex decisions once extraction drains. The greenfield-vs-existing on-ramp.

Extraction and emit converge. Both write the rc onto a graph edge and project it into both indices, so a decision is retrievable the same way no matter how it arrived. Bulk paths (backfill) call reindex once extraction drains.

Getting answers out

Read paths

Retrieval assembles a context from the indices, walks the graph for structure, and hands it to the LLM. The mode chooses which fabrics to consult.

Mode	Consults	Good for
`local`	entities_vdb → graph neighborhood	entity-centric questions
`global`	relationships_vdb → patterns	themes, relationships, decisions
`hybrid`	local + global	most questions
`naive`	chunks_vdb only	plain vector RAG
`mix` default	KG + chunks together	the recommended all-rounder
`bypass`	nothing — direct LLM	no-retrieval passthrough

Beyond the modes, several more read surfaces:

Smart router (/query/auto, the query_auto tool) — inspects the question and picks the retrieval mode for you (and reports why, via mode_reason), so callers never have to choose local vs global vs mix. The recommended default entry point for agents.
Raw context (/query/data) — returns the retrieved facts themselves — entities, relations, chunks, citations — without the LLM synthesis step. Often more useful than a prose answer: feeding another system, building a UI over the results, grounding a downstream agent, or debugging exactly what retrieval found. Same modes and router apply; you just get structured data instead of generated text.
CGR3 (/cgr3/query) — iterative multi-hop Retrieve → Rank → Reason for "why" questions that need several hops.
Precedent search (/graph/decisions/search) — semantic nearest-neighbours over decisions_vdb: "find past decisions like this one."
Decision filter (/graph/decisions) — structured query over rc-bearing edges by approver, channel, policy, confidence, or validity date. Reads the graph, not a search index.

◆ Because decisions are in the retrieval fabric…

a plain mix query surfaces a recorded decision like any other fact — no special path needed. (An earlier "query-blend" shim compensated for decisions living only in decisions_vdb; unifying them into relationships_vdb is what let normal retrieval do the job.)

On top of the graph

The governance layers

Five optional, per-workspace layers turn the graph from a system you reason from into one you operate from. Each is data, installed into a workspace; the core ships none of them.

Ontology

Object types + link types; validates & coerces what extraction writes. →

Rules

A pre-emit methodology gate (sim() concept matching): PASS · FLAG · REJECT. →

Actions

Typed operations bound to object types, invoked and audited. →

RBAC

Deny-by-default grants per role; permissive when absent.

Lifecycle

State machines per object type; illegal transition → 409.

Manifest

Assembles a role-scoped operating view for an agent.

Full treatment: the AI Agent Development use case walks all five as a working system.

Isolation

Multi-tenancy — workspaces

Every request carries a LIGHTRAG-WORKSPACE header. A workspace is a hard isolation boundary: Neo4j nodes are tagged with a per-workspace label, vector collections and KV stores are namespaced, and the decision index is per-workspace. One server hosts many tenants; a query in workspace A can never see workspace B — the graph, the indices, the governance config, and the decisions are all scoped.

The whole picture

How it fits together

In → graph → projections → retrieval → out. Every input path is gated by governance and lands in the graph; the indices project from it; retrieval reads the indices and walks the graph; the LLM answers, grounded and cited. The graph is always the pivot.

What changed lately

Recent additions

Decision unification — emitted decisions now project into relationships_vdb (not just decisions_vdb), so normal /query surfaces them. Both indices are now derived, with reindex_decisions() rebuilding them from the graph (fast filtered lookup + a background endpoint).
Governed operations — the Actions, RBAC, and Lifecycle layers, plus the role-scoped Manifest.
Onboarding surface — /onboard (NL → tailored config), the served /workspace/playbook + /workspace/bootstrap, and the conversational Get Started studio.
WebUI — workspace creation, Rules/Ontology editing, and the Get Started tab.

◆ The takeaway

One graph holds the truth — entities, relations, and the decisions on their edges. Everything else — four vector indices, the governance layers, the retrieval modes — is built on top of it and rebuildable from it. Keep that picture and the whole system stays legible.

See also: the Technical Field Guide, The Fourth Element for the concept, and the per-feature guides (Rules · Ontology · Actions · Web Ingester).