CareIQ
A clinical AI assistant that shows its work and its limits
A governed assistant over clinical documents on synthetic patient data, scoped and shipped solo in 48 hours. Requests pass through PHI masking, guardrails and intent classification before retrieval, then generation with inline citations, a faithfulness check against the retrieved sources, and an audit write. Every external dependency has a documented degradation path, so losing the vector store or the model API downgrades behaviour rather than breaking it.
Verified: each of these is checkable in the repository
- 4
- guardrail stages before the model is calledPHI masking, input guardrails, intent, retrieval
- 47
- patient care-gap queuesynthetic cohort in the Surface view
- 48h
- from scope to shipped, solohackathon build
Clinical text is exactly the kind of data you should not send to a model casually, and clinical answers are exactly the kind you should not accept without a citation. Any useful system here has to handle both at once, and has to be honest that it is not validated for clinical use.
A linear pipeline with explicit stages: mask protected health information, run input guardrails, classify intent, retrieve from a vector store with embeddings, call tools where the question needs live data, generate an answer with inline citations, check that the answer is faithful to the retrieved sources, and write an audit record. An MCP server fronts an external drug-label API so tool access sits behind a contract. Each dependency has a fallback, local vectors without the vector store, deterministic drafts without the model.
Why it is built this way
01Mask before anything else
PHI masking with a named-entity recogniser plus regex runs before guardrails, retrieval or generation. It is the first stage because the point is that identifiers never reach a third-party API at all.
02Citations from the model, not stitched on after
Generation uses a citations API so the model attributes claims to the retrieved passages as it writes, rather than having sources appended afterwards to look rigorous.
03Check faithfulness as a pipeline stage
A dedicated step verifies the answer is supported by what was retrieved. Retrieval quality and generation faithfulness fail in different ways, so they get measured separately.
04A documented degradation path per dependency
Losing Pinecone falls back to local vectors; losing the model API falls back to deterministic drafts. In a governed context, predictable degradation beats a 500.
05Audit as a product surface
Audit records go to BigQuery and there is a view that reads them. If nobody can inspect the log, the log is not really governance.
Ordering the stages was the whole design. My first version classified intent before masking, which meant raw clinical text reached a model call in the classification step, technically upstream of the 'real' pipeline, and completely defeating the purpose. Guardrails only work if nothing runs before them.
Next project
Support Intel
An ELT pipeline with traceable lineage into Snowflake