SecSignal
Agentic RAG over SEC filings, deployed and evaluated
A live full-stack platform that turns SEC 10-K and 10-Q filings into queryable intelligence. A LangGraph supervisor classifies each question, extracts tickers and time ranges, then dispatches to eight specialist agents in parallel, trend, comparison, anomaly, valuation, sentiment, peer group, general and web search. Results merge in a synthesizer that returns a cited answer with chart specifications, streamed to the client over SSE.
Verified: each of these is checkable in the repository
- 8
- specialist agents dispatched in parallelagents/graph.py, 8 nodes, conditional-edge routing
- Live
- deployed on Vercel and Rendersec-signal.vercel.app
- 4
- scored evaluation dimensionsevaluation/scorers.py. LLM-judge plus heuristics
- 1024-d
- embeddings via CortexEMBED_TEXT_1024, snowflake-arctic-embed-l-v2.0
Financial filings are long, structurally inconsistent, and mix narrative with numbers. A single retrieval pass over them answers shallow questions and fails on anything comparative, asking how two companies' risk disclosures diverged over three years is not one lookup, and a system that pretends it is will confidently return something wrong.
Filings are pulled from EDGAR, parsed section-aware, landed in GCS and then Snowflake, and transformed through dbt staging, intermediate and marts layers. At query time a classifier extracts tickers and a time range, validates the tickers against the warehouse, and builds an execution plan. A conditional edge dispatches that plan to as many of the eight specialist agents as it needs, which draw on structured SQL over the marts, semantic retrieval through Cortex Search, and a web-search fallback for tickers not in the corpus. A synthesizer merges their outputs into one cited answer with a confidence score.
Why it is built this way
01Plan first, then dispatch
Classification produces an explicit execution plan, and routing is a conditional edge that emits one dispatch per planned agent. Agents are chosen because the question needs them, and the plan is inspectable when an answer is wrong.
02Structured and semantic retrieval as peers
Numbers come from dbt marts through SQL; narrative comes from Cortex Search over filing text. Forcing both through one vector index would have made the numbers unreliable, which is the one thing a financial tool cannot afford.
03A web-search agent as an explicit fallback
Asked about a ticker outside the corpus, the system says so and falls back to web search rather than hallucinating from adjacent filings. Knowing the edge of your own knowledge base is a feature.
04Streaming because agent runs are slow
Typed SSE events stream classification, per-agent progress and the final answer, so the interface renders incrementally instead of blocking on a multi-agent run. Perceived latency is a design problem, not just an optimisation.
05An evaluation harness with curated cases
A runner drives a curated case suite through scorers covering answer relevance, source attribution, chart appropriateness and factual accuracy, combining LLM-as-judge with heuristic checks. It is the part of this repo I would want reviewed first.
Splitting retrieval by data type rather than by question type was the decision that made this work. My first attempt pushed everything through semantic search, which produced fluent answers with wrong numbers, the worst possible failure for a financial tool, because it is invisible until someone checks.
Next project
Lincoln AI-Judge Evals
An LLM judge, and the work of validating the judge