AI engineer focused on LLM evaluation and agent reliability: measuring where agents and small models fail, and building the tooling to catch it. I also build production AWS systems (Rust, Lambda, Bedrock, CDK) and spent 10+ years as a public-sector and regulated-industry solutions architect.
| Project | What it shows |
|---|---|
| agent-fail-eval | What context compaction silently drops from coding-agent sessions. Two harnesses — 180 real sessions across 436 compaction boundaries — with summaries omitting 96.5% of extracted user constraints (pi) and 72.7% (Claude Code): the "don't push", region and credential rules an agent then proceeds without. Heuristic extractor, human-labeled validation. |
| rag-eval-project | Evaluation-first RAG: a 10-cell ablation over 2,556 queries on a reusable metric harness, three committed phase reports and 137 offline tests — including a negative result, a reranker that hurt nDCG@5 and ships off by default. |
| rag-control-cite | Multi-tenant RAG over a permissioned regulatory corpus: exact-span citations with ACLs enforced in the retrieval legs and on the fused set — 0 leaked across 4,867 chunk checks, eval gate passing on 235 queries (hybrid recall@10 81.5%, +12.2 pp over dense). |
| agentcore-gateway | A Strands agent on Amazon Bedrock AgentCore whose tools exist only behind AgentCore Gateway — the agent imports none. Deployed by CDK, measured (cold start 14.6 s, warm p50 3.7 s), torn down. |
| full-stack-serving | An LLM serving stack where every layer must produce a number: container → k3s → OpenTelemetry, each gated by a command that re-measures it. The span-to-client TTFT gap grows ~10× from c=1 to c=64. |
More in the repos — toolshift-bench (397-task tool-calling benchmark: small models pick the right tool ~90% of the time but make a fully correct call only 41–72%), support-triage (a triage agent with human-gated state changes, deployed and torn down), privatelink-inference (a model endpoint with no public route — proven, not asserted), mcp-stateless (Go), plus research (rlm-switch, grokking-geography), a desktop app (mathmate), and a production Bedrock pipeline at eapsoftware.dev.
Python · Rust · Go · PyTorch · AWS Bedrock · Lambda · CDK · Kubernetes · MCP · DSPy
AWS Certified Generative AI Developer – Professional · AWS Certified Machine Learning – Specialty