Two-Stage Jev Pushes Graph RAG onto the Quality-Latency Pareto Frontier
22 Oct 2026 - Cheney Zhang
Decision models are starting to take over work we used to give to LLMs in knowledge graph retrieval. We put Jev into Vector Graph RAG, our open-source project, and reorganized retrieval into two stages: filter relations, then rank the original passages. The result beat every other configuration in our comparison on evidence recall and reached the quality-latency Pareto frontier.
Getting there took more than swapping a model name. Our first attempt was fast, but the stronger LLM still retrieved better evidence. The improvement came when we changed what we asked Jev to judge.
How the project got here
Some RAG questions need evidence from several documents. A single similarity search can miss a passage because the question never mentions the entity that connects it to the answer. A knowledge graph gives retrieval a way to follow those connections.
We built Vector Graph RAG to make this kind of multi-hop retrieval easier to deploy. Milvus stores the entities, relations, and source passages. Vector search finds starting points; ID lookups follow their connections. There is no separate graph database to operate.
The original pipeline already retrieved good evidence, but it had a slow step: an LLM had to read the candidate relations and choose which ones to keep. It used one round of LLM reranking, yet the wait was noticeable when graph expansion returned a large candidate set.
In September, we tried replacing that step with Jev. In October, we added a second decision over the passages themselves. That gave us three versions of the same retrieval pipeline:
| Version | Approach | Result |
|---|---|---|
| Original | Expand the graph in Milvus, then let an LLM filter relations | Strong retrieval, with a slow reranking step |
| September: single-stage Jev | Replace the relation filter with Jev | Faster, but behind the stronger LLM baseline |
| October: two-stage Jev | Filter relations, then rank the retrieved passages | Best recall among the compared configurations |
The earlier project introduction and September evaluation are linked at the end. Here, we’ll follow the part of the pipeline that changed.
The original pipeline: follow relations to the evidence
Consider this question:
What side effects should we watch for with the first-line medication for type 2 diabetes?
Suppose one passage identifies metformin as a first-line medication, while another describes its adverse effects. The question never says “metformin.” Searching with the question alone may find general diabetes treatment pages and miss the passage organized around the drug’s name.
Before queries arrive, Vector Graph RAG extracts entities and relations from the documents. Entities become nodes, relations become edges, and each relation keeps the IDs of its source passages. Retrieval then proceeds through four steps.
1. Retrieve starting nodes and edges
At query time, we extract entities from the question and search for matching nodes. Separately, we embed the complete question and search for relations. Both paths use vector similarity, so “side effects” can match “adverse effects” without sharing the same wording.
In this example, the starting points are the diabetes and adverse-effects nodes, plus a relation about diabetes affecting blood glucose.
2. Expand the subgraph
We follow the IDs associated with those nodes and edges. One hop reaches metformin through the treatment relation and connects it to adverse effects. Another hop can bring in surrounding material, including the drug’s development history.
The missing drug name is now inside the retrieval scope. Expansion has also collected some distractions, which is why we need the next step.
3. Filter the relations
The original version gives the question and candidate relations to an LLM for reranking and filtering. For this question, we want the treatment link and the adverse-effects link. Development history and general blood-glucose information can be left out.
4. Retrieve the source passages
Finally, we use the retained relations’ source IDs to fetch the original text. In the diagram, R1 points to P01 and P03, while R2 points to P02. Those three passages become the retrieved evidence for the answering stage.
The graph helps us locate evidence across documents; the answer still gets the original passages. Entities, relations, and passages live in separate Milvus collections connected by IDs.
The expensive part appears after expansion. A query can produce hundreds of candidate relations, sometimes over a thousand. Vector search and ID lookups are quick, but the LLM still has to read those candidates and generate its selection. Removing that filter would also remove a major reason the retrieval works.
Single-stage Jev: a fast replacement with a quality gap
Our September experiment kept the pipeline intact and replaced the model in step three. Jev judged the usefulness of each candidate relation, and the program used those judgments to rank and select relations. Passage lookup stayed the same.
The change was small and the speed improvement was welcome. Retrieval quality beat GPT-4o-mini, but it did not catch GPT-5-mini. The gap was more visible on the harder multi-hop questions in MuSiQue.
This is the earlier September experiment on 500 questions each from MuSiQue and HotpotQA. The historical comparison used 1,000 questions per dataset, so this chart should be read separately from the October results below.
One possible explanation is that fast decisions over individual relations can miss a connection that a reasoning model works through. We did not isolate that explanation experimentally. What we could see in the pipeline was a more actionable problem: we had evaluated the relations, but never separately evaluated the passages they brought back.
A useful relation can point to several passages. One may only mention the relevant fact in passing; another may contain the details the question needs. Selecting the relation does not guarantee that the best passage gets one of the first few result slots.
Could a second Jev judgment fix that without giving up the speed advantage?
Two-stage Jev: from choosing relations to choosing passages
We kept entity retrieval, relation retrieval, and graph expansion. Jev then makes two rounds of decisions: the first selects relations, and the second ranks passages.
The first round preserves useful connections, including intermediate links. The metformin treatment relation cannot answer the side-effects question on its own, but it helps us reach the right evidence.
After fetching the selected relations’ sources, we add passages from a direct vector search. We merge the two sets and deduplicate them before the second round. Jev now reads the complete candidate passages and judges their usefulness for answering the question.
| Round | Input | Decision |
|---|---|---|
| First | Relations from the expanded subgraph | Which connections should lead us to source passages? |
| Second | Merged candidate passages | Which passages belong at the top of the final results? |
In the diabetes example, step four retrieved P01, which establishes the medication choice; P03, which discusses development history; and P02, which describes adverse effects. Direct passage retrieval finds P02 again and a new passage, P04, with medication precautions. Deduplication keeps one P02, and the merge adds P04. The second round ranks all four passages.
P02 should move forward because it addresses side effects. P01 identifies the first-line drug, and P04 adds relevant precautions. P03 can move down: it is a source for the treatment relation, but its development history contributes little to this answer.
The graph finds the missing connections; passage reranking allocates the limited result slots to useful text. The diagram illustrates the mechanism. We evaluated the complete two-stage recipe, including the added passage retrieval, so the results do not isolate the second ranker’s contribution.
What happened on 2,000 questions
We evaluated the fixed recipe on 1,000 MuSiQue questions and 1,000 2Wiki questions. The metric is Recall@5: the fraction of required evidence recovered within the top five passages, averaged over questions. The table gives equal weight to the two datasets.
Jev evaluation began in September 2026; the full two-stage results below were added in October 2026.
| Retrieval configuration | Average Recall@5 | Average additional model API time |
|---|---|---|
| Plain vector retrieval, NV-Embed-v2 | 73.10% | 0 s |
| HippoRAG 2 | 82.55% | About 8.25 s |
| Vector Graph RAG + GPT-6 Luna | 82.70% | 5.35 s |
| Vector Graph RAG + DeepSeek V4.1 Flash | 83.34% | 10.97 s |
| Vector Graph RAG + Jev 1.13, single-stage | 80.04% | 2.28 s |
| Vector Graph RAG + Jev 1.13, two-stage | 86.07% | 3.13 s |
| Vector Graph RAG + Jev 1.14, two-stage | 86.61% | About 4.9 s |
Latency covers additional model API time, not total query time. Jev 1.14’s roughly 4.9 seconds estimates direct API latency from a small calibration; its full run used OpenRouter. HippoRAG 2’s latency is an external reference. Luna used medium reasoning and Flash used low. Full methodology is linked below.
The largest jump came from the retrieval recipe. With Jev 1.13, moving from one stage to two raised Recall@5 from 80.04% to 86.07%, a gain of 6.03 percentage points. Jev 1.14 added another 0.54 points, reaching the highest recall in this comparison. Its full run included the initial 200-question sample and the remaining 1,800 questions; the latter also showed a positive gain, though those questions had already appeared in the historical evaluation.
Both two-stage versions sit on the displayed Pareto frontier: no compared configuration improves their recall while also reducing their plotted latency. Version 1.13 offers the faster option; 1.14 reaches the higher recall. Its estimated latency is close to Luna’s, while 1.13 has a wider speed margin. Provider load, batching, and routing affect these times, so the chart is a useful deployment comparison, not a measurement of architecture alone.
This work improves online retrieval. Extracting triples and importing documents still depend on an LLM. We have not made the whole project decision-model-only.
Where decision models can go next
Plenty of applications spend time filtering and ranking existing candidates. This experiment gives us a reason to try decision models in more of those places, and to look carefully at where each judgment belongs in the workflow.
As more vendors enter this space, there will be more approaches to compare. We will keep watching what they do well and testing where they fit.