Have you ever had a brilliant solution get completely crushed by a single comment on a technical blog post?
Just last week, on July 8, 2026, I published a post introducing the Pyramid Aggregator. It was a beautiful story of technical lineage: we took a string summation algorithm originally designed on October 13, 2016 (to optimize Google Apps Script) and applied it to modern Large Language Models (LLMs). By organizing document merging into a balanced, parallel tree topology, we successfully bypassed the AI’s “Lost in the Middle” bias and achieved a 95% execution speedup using the Antigravity Python SDK.
The paper was archived on Zenodo, and the Medium article did great. But then, a reader dropped a comment that pointed out a massive, system-level blindspot.
They wrote:
“Pyramid Aggregator is a great synthesis tool, but it assumes your input set is already perfect. In a real production system, what if your vector search fails to retrieve the root-cause documents in the first place? If the cause is missing, even a bias-free summary is just a bias-free summary of incomplete data.”
They were 100% correct.
In real-world operations—like Site Reliability Engineering (SRE) or Cybersecurity—root causes and visible symptoms are often separated by several layers of system logic. If a database tablespace fills up, a flat vector search for “database issues” will retrieve the database logs. But it will completely ignore the low-level voltage drop log from the power grid container, simply because the words “database” and “voltage” are semantically unrelated.
This is the Multi-Hop Relational Blindspot.
To solve this, we went back to the drawing board. Today, we are releasing the next evolution of our hierarchical context architecture: GAP (Graph-Anchor Pyramid). By combining Graph Retrieval (Graph-RAG), Topology-Aware Leaf Grouping, and Prompt-Level Semantic Anchoring, we have created a lossless, parallel framework that recovers 100% of causal chains in multi-document synthesis tasks—and we’ve proved it across two generations of Gemini models.
Here is the deep dive into how it works.
The Twin Bottlenecks of Hierarchical RAG
When we scale LLM context ingestion using hierarchical summarization, we face a fundamental trade-off: we want high-level semantic summaries, but we cannot afford to lose the critical, fine-grained details (like specific error codes, account IDs, or IP addresses).
Standard hierarchical pipelines break down in two places:
1. The Search Phase: The Relational Blindspot (Macro)
Flat vector databases represent documents as points in a high-dimensional space. While this is great for finding similar documents, it fails when documents are connected by dependency or causality rather than similarity.
If a query is "database connection failure", standard semantic search retrieves:
- Log A: “Module Mercury lost connection to database with error code ERR-102.” (Symptom)
- Log B: “Module Olympus database tablespace filled up with error code ERR-105.” (Symptom)
But it completely misses:
- Log C: “Module Golem crashed due to memory exhaustion with error code ERR-101.” (Root Cause)
- Log D: “Module Thor high-voltage supply fluctuated with error code ERR-103.” (Root Cause)
Because Golem and Thor don’t mention the word “database”, their embeddings are too far away. The RAG pipeline suffers from a search gap, feeding incomplete context to the generator.
2. The Synthesis Phase: Recursive Semantic Decay (Micro)
Even if we retrieve all the logs, merging them hierarchically causes another issue. Forcing intermediate nodes in the tree to output natural language summaries acts as a lossy compression (discrete bottleneck).
As summaries are merged up the tree, the model’s contextual embeddings drift. Under strict word limits (like summarizing 16 system logs in under 50 words), the LLM is forced to prune text. Crucial micro-facts (like specific module names or error codes) are washed out, leaving only generic macro summaries.
In our original Pyramid Aggregator benchmark, when forced into strict length and listing constraints, the Micro-Fact Recovery Rate dropped to 0.0%. The AI gave a great high-level summary but forgot the specific entities.
Introducing GAP: Graph-Anchor Pyramid
The GAP architecture resolves both issues by adding graph topology and static attention anchors to the hierarchical tree.
Here is the structural comparison between standard RAG, chronological tree aggregation, and GAP:
GAP operates in three unified stages:
Stage 1: Graph Retrieval (Graph-RAG)
Instead of relying on flat vector search, we represent the document corpus as a directed dependency graph $G = (V, E)$. The vertices $V$ are document chunks (logs), and the directed edges $E$ represent logical or causal dependencies (Cause $\rightarrow$ Effect).
When a query is received:
- We run a standard vector search to get a set of seed nodes ($V_{seed}$), capturing the visible symptoms.
- We traverse the dependency graph $k$-steps backward to retrieve all directed ancestor nodes ($V_{ret}$), pulling in the root causes.
This completely closes the relational blindspot.
Stage 2: Topology-Aware Leaf Grouping
Once we have retrieved the documents, we must group them to build the tree. A standard tree-reduction groups documents chronologically or sequentially. However, this mechanical slicing often cuts causal edges (called Causal Splitting), grouping the “cause” in Leaf 1 and the “effect” in Leaf 2. Because they are processed separately, their relationship is diluted before they ever meet.
GAP solves this by treating the leaf grouping as a graph partitioning problem. It groups nodes to minimize edge cuts on the retrieved subgraph. This guarantees that causally connected logs are grouped together in the same Level 1 leaf prompt, preserving their relationships in the initial, low-level summaries.
Stage 3: Prompt-Level Semantic Anchoring
To prevent recursive semantic decay, we introduce static Semantic Anchors ($A$).
- At the leaf level, we extract critical entity pairs (e.g.,
Golem (ERR-101) -> Mercury (ERR-102)). - As summaries travel up the tree, we persist and inject these anchors into a dedicated
### SEMANTIC ANCHORS ###block within the system prompt of every intermediate and root node. - We apply a constrained attention rule in the prompt, instructing the LLM to structure its summary around these anchors without allowing them to drift or decay.
This acts as a “hard link” that bypasses the lossy compression of intermediate summaries, carrying crucial facts all the way to the final root output.
Below is the comparative architectural workflow of the three evaluated pipelines:
Putting GAP to the Test: The Benchmark
We constructed a realistic system incident data set consisting of 16 logs containing 4 independent, multi-hop causal chains (e.g., memory exhaustion leading to database disconnects) mixed with background noise logs:
We evaluated three pipelines:
- Vector + Batch (Standard RAG): Vector search (retrieving symptoms + noise, missing 5 root cause logs) + one-shot batch summary.
- Graph + Chronological: Graph search (all logs retrieved) + standard chronological tree grouping (no anchors).
- GAP (Proposed): Graph search + topological grouping + persistent semantic anchors.
To make the task extremely challenging and force information loss, we applied three strict constraints:
- The final output must be strictly under 50 words.
- The output must be a single cohesive paragraph (no bullet lists or numbered lists).
- No single sentence may list more than 3 module names (preventing the AI from cheating by simply listing all names).
We ran the benchmarks across two generations of Gemini models: gemini-3.1-flash-lite (lightweight, optimized for low latency) and gemini-3.5-flash (designed for complex reasoning).
The Mind-Blowing Results
Here is the evaluation matrix:
| LLM Model | Pipeline | Causal Recall ($\text{RR}_{\text{causal}}$) | Macro Coverage ($\text{CC}_{\text{macro}}$) | Style Compliance ($\text{SR}_{\text{style}}$) | Latency (s) |
|---|---|---|---|---|---|
| gemini-3.1-flash-lite | Vector + Batch | $0.0%$ | $100.0%$ | $0.0%$ | 5.0 |
| Graph + Chronological | $0.0%$ | $100.0%$ | $100.0%$ | 5.4 | |
| GAP (Proposed) | $100.0%$ | $100.0%$ | $100.0%$ | 4.3 | |
| gemini-3.5-flash | Vector + Batch | $0.0%$ | $25.0%$ | $100.0%$ | 26.9 |
| Graph + Chronological | $0.0%$ | $50.0%$ | $100.0%$ | 48.2 | |
| GAP (Proposed) | $100.0%$ | $100.0%$ | $100.0%$ | 50.4 |
Let’s break down the metrics:
1. Causal Chain Recall: $0%$ vs. $100%$
In our experiments, the Causal Chain Recall ($\text{RR}_{\text{causal}}$) was calculated as a strict logical AND—meaning the causal link was only counted as recovered if both the cause and the effect were present in the final summary.
- Vector + Batch scored 0.0% because it missed the causes during retrieval.
- Graph + Chronological also scored 0.0%. Even though it retrieved all the files, splitting the logs across leaf nodes and the lack of anchors caused the LLM to discard the causal relationships during recursive merging.
- GAP scored a perfect 100.0% on both models. The combination of topological grouping and persistent anchoring successfully carried all causal connections to the final summary.
2. The Advanced Model “Over-Pruning” Trap
One of the most surprising findings was that the highly capable gemini-3.5-flash scored a mere 25.0% Macro Coverage under the Vector+Batch pipeline, and 50.0% under the Graph+Chrono pipeline, whereas the smaller gemini-3.1-flash-lite maintained 100.0% coverage.
Why? Advanced reasoning models have highly strict internal filters for style rules. When forced to respect the 50-word and list-exclusion limits, gemini-3.5-flash actively chose to prune entire system domains to keep the output grammatically coherent. It chose style over content.
However, when we introduced GAP, the semantic anchors acted as attention anchors that restricted the model’s pruning boundaries. Under GAP, gemini-3.5-flash maintained 100.0% Macro Coverage while fully complying with the style rules.
3. The Latency / Cost Sweet Spot
Thanks to the parallel tree-reduction architecture orchestrated by the Antigravity Python SDK, the Level 1 leaf summaries are processed concurrently.
gemini-3.1-flash-literunning GAP completed the entire pipeline in just 4.3 seconds.gemini-3.5-flashtook 50.4 seconds due to its larger reasoning overhead and API queue times.
For real-time operational systems, where every second of delay leads to financial loss, GAP paired with a lightweight model (3.1-flash-lite) represents the ultimate sweet spot: sub-5-second execution, minimal cost, and a lossless 100% causal recall.
Real-World Production Scenarios
Where does this make a difference? We mapped GAP to two critical enterprise operational architectures.
Scenario A: Real-Time SRE Root Cause Analysis
In a microservices environment, a single network glitch can trigger a cascade of thousands of downstream exceptions.
GAP traverses the infrastructure topology graph to group logs by their connection dependencies. In less than 5 seconds, it compresses the entire cascade into a single, cohesive alert message that accurately highlights the root cause, preventing alert fatigue and slashing Mean Time to Detection (MTTD).
Scenario B: Cyber Security Threat Hunting
Advanced Persistent Threat (APT) attacks execute slowly over weeks, leaving quiet footprints across different servers. Standard security alerts treat these events as isolated noise.
By modeling user accounts, IP connections, and host logs as a graph, GAP links lateral movements. The persistent semantic anchors ensure that critical details—like compromised user accounts and IP addresses—are never discarded, delivering a unified threat summary to security teams.
Conclusion: Stop Grouping Chronologically, Start Building Pyramids
If you are building production-grade RAG applications, it’s time to move beyond flat vector lookups and linear text chaining. When relationships and detail retention matter, hierarchical structures are essential.
By extending the parallel tree-reduction mathematics of the Pyramid Method with graph-aware topology and static semantic anchoring, GAP ensures that your LLMs retain 100% of causal context without sacrificing speed.
- Read the full pre-print paper: GAP: Graph-Anchor Pyramid for Resolving Relational Blindspots and Recursive Semantic Decay in Hierarchical RAG
- Explore the foundations:
Stop dumping raw texts into prompts. Build a Graph-Anchor Pyramid.
Credits: Google Cloud credits were provided to support this research.