We Built a Custom Graph RAG to Let AI Answer "Did That Initiative Actually Work?

On April 20th, a developer sat down and wrote something that most organisations quietly dread having built: a system designed to answer whether their initiatives actually worked. Not whether the project delivered. Not whether the team hit their OKRs on a spreadsheet somewhere. Whether the underlying needle — the one the whole initiative was theoretically pointed at — moved. The architecture they chose was a custom Graph RAG, which is a retrieval-augmented generation system that structures knowledge as a network of connected entities rather than a flat pile of documents. The idea is that relationships matter when you're asking causal questions. "Did the customer retention programme reduce churn?" is not a lookup query. It requires threading together timelines, ownership, metrics, and outcomes across data that was never designed to be connected. A graph helps. The developer's post walked through why standard RAG falls short for this kind of accountability question, and how the graph layer lets the AI trace a chain of reasoning rather than surface the nearest plausible-sounding paragraph. It is, in practice, a system that makes the evidence harder to avoid.

If confirmed as a reproducible approach, here is what this means. Enterprise teams have spent years building dashboards that answer the wrong question with great precision — outputs, not outcomes. A Graph RAG architecture that genuinely connects initiatives to downstream effects would shift the burden of proof. Retrospectives stop being narrative exercises and start being forensic ones. The people most affected are programme managers, strategy consultants, and any team that has ever declared a pilot a success on the basis of engagement metrics rather than business results. The second-order effect is more interesting: if AI can reliably ask "did this work and why," the bar for launching initiatives quietly rises, because the record is no longer buried in slide decks that nobody revisits.

Watch for other developers publishing comparable architectures or case studies that put this approach against real organisational data. A methodology this specific needs a second voice before it earns more than cautious curiosity.

NewsHive monitors these sources continuously. All signal titles above link to the original reporting.

Intelligence by NewsHive. Need help navigating what this means for your business? Contact GeekyBee →