LLMs as Graph Databases: Querying and Editing Weights Without Retraining

The central theme of the video is the revelation that Large Language Models (LLMs), specifically their Feed-Forward Networks (FFNs), physically function as graph databases. Rather than serving as a mere metaphor, the speaker demonstrates that an LLM’s weights consist of real entities, nodes, and edges. These can be mapped, queried, and manipulated using a SQL-like language called Larql, allowing developers to interact with the model programmatically without the need for traditional fine-tuning or retraining.

Key Findings and the Structure of LLMs

Three-Stage Architecture: The internal layers of an LLM (demonstrated via a probe on the Gemma 3 4B model) naturally divide into syntax (early layers understanding the query), knowledge (middle FFN layers acting as a fact repository), and output (final layers formatting token predictions).
Polysemanticity as a Constraint, Not a Bug: Individual features (edges) in the FFN often store multiple unrelated concepts or compress similar entities together. This “polysemanticity” happens because the model must project high-dimensional concepts into one-dimensional scalar activations.
Attention acts as the Navigator: Because the FFN’s knowledge graph is messy and polysemantic, the attention mechanism is essential. Attention operates in the full-dimensional space to route queries, select the correct features to activate, and suppress irrelevant noise (like ignoring “fountain” when looking for the country of France).
Graph Walk Inference: By un-encoding standard weight matrices into a “V index” graph structure, the speaker shows that LLM inference can be executed as a K-Nearest Neighbors (KNN) graph walk rather than computationally expensive standard matrix multiplications.

Significant Conclusions and Takeaways

One of the most groundbreaking demonstrations in the video is the ability to write new knowledge directly into the model’s weights. Using a simple INSERT command, the speaker added a fictional fact (that Poseidon is the capital of Atlantis). The system engineered the correct gate and down vectors, baked the fact into a free feature slot, and compiled it into standard formats like GGUF and safetensors—all without breaking existing knowledge or requiring retraining.

This fundamentally proves that the attention mechanism (the router) can be decoupled from the knowledge store (the FFN). The speaker concludes that this architectural shift has massive implications for AI efficiency. By treating the FFN as a database, knowledge could potentially be hosted remotely while attention runs locally, theoretically allowing massive multi-billion-parameter models to be executed on standard consumer laptops in the near future.

Mentoring question

How might the ability to directly query and edit an LLM’s internal facts without retraining transform the way your organization handles AI compliance, bias mitigation, and real-time knowledge updates?

Source: https://youtube.com/watch?v=8Ppw8254nLI&is=QTucYaPFG4umCNk2