How Kepler built verifiable AI for financial services with Claude

eddiehammond

Anthropic published a profile on what we're building at Kepler. Sharing because the architectural argument (LLM for intent, deterministic code for retrieval and computation, every number traceable to source) is the part I'd actually want HN to push on. Happy to answer questions in the thread.

hansmayer

> The duo’s answer was to build deterministic infrastructure that serves as a trust and verification layer for AI. On the one hand, very encouraging to see plain old deterministic infra w/o using slop machines. On the other hand, this is a recognition that LLMs are just additional friction in the system that we would better off without in the first place!

rossjudson

From a systems engineering standpoint, the purpose of LLMs is to construct, verify, and "push down" abstractions and deterministic layers. Deterministic layers are able to cope reliably with the law of medium numbers.

hweaHG

The people who built this were at Palantir before. How is the verifiable targeting of girls' schools in Iran by the Claude-powered Maven system going? We are living in an age of hot air.

HoyaSaxa

The title is misleading. They achieved a 94% accuracy rate which in financial services is a far cry from acceptable without a human-in-the-loop verifier.

hbcondo714

> Indexed 26M+ SEC filings But the https://kepler.ai website says 10M+

eddiehammond

Mandatory pitch - if working on this kind of problem is interesting to you, we're hiring! jobs.ashbyhq.com/kepler-ai

Txmm

Reassuring to see this approach coming out consistently. I’ve been doing the same for high volume data pipelines, extracting the deterministic actions from markdown instructions and leaving the LLM to do the analysis/act as the fluid coupling between deterministic parts. Over time you can refine this to be more and more codified, handle edge cases with agents/LLMs then turn them into first class deterministic branches too. This pattern seems to be emerging everywhere, the chain of thought and intent capture to improve it seems to be the next big thing

schmuhblaster

Shameless self-plug: https://github.com/deepclause/deepclause-sdk/ The idea is to take markdown instructions and "compile" them into a Prolog-based DSL that orchestrates both deterministic and LLM-based components. The (meta-)interpreter of the DSL automatically tracks the entire execution process, so that the final ouput becomes observable and more explainable. Still at an early stage, but I am having lots of fun with it and would love to explore possible use cases.