I build agentic AI systems and the evaluation harnesses that keep them honest. Most of my work runs inside AWS GovCloud on sensitive insurance fraud data, so you cannot clone it or click a demo.
This page is the next best thing: how the systems are actually built, which parts were hard, and what the numbers mean.
Currently Senior Data Scientist at the National Insurance Crime Bureau, where I shipped the organization's first production AI agent. Ph.D., Northeastern. Open to Senior or Applied AI Engineer roles, remote or Bay Area.
Selected results
- Production agent, adoption
- Adopted org-wide across 100+ analysts, roughly 1,000 requests/day
- Pilot evaluation
- 9 / 10 across a 16-analyst structured pilot
- Retrieval surface
- Natural language over a ~3,000-table Redshift warehouse
- Fraud model, salvage swap
- 0.89 AUC, 0.84 recall, batch scoring 100M+ vehicle registration records
- Investigative output
- 200+ leads across the three-model portfolio, roughly $5M in potential fraud exposure
- Inference latency
- Cut generation from 5 minutes to 30 seconds on self-hosted Llama 3.1
- Infrastructure cost
- $100K/year removed from the AWS bill
- Statistical analysis
- 45.8M claims over two years; invalid-SSN claims carry 88 to 93% higher fraud odds
The system
A production analyst agent over a 3,000-table warehouse GovCloud
Fraud investigators needed answers from an internal warehouse during active cases. Getting them meant either writing Redshift SQL against three thousand tables or filing a ticket and waiting a day. Neither works when the case is moving.
I designed and built the agent that replaced both, end to end and as the sole engineer on it. It discovers the right tables, writes Redshift-compatible SQL, surfaces pre-approved vetted queries, and summarizes investigative reports without dropping a dollar amount, a VIN, or a claim count. It runs on AWS Bedrock with Claude Sonnet 4.5, on ECS Fargate, inside GovCloud.
Retrieval was the hard part
Three thousand tables do not fit in a context window, and semantic search alone fails on exactly the tokens investigators care about. A VIN, a claim ID, and an internal acronym are all high-signal exact matches that dense embeddings blur away. Pure keyword search has the opposite problem: it cannot tell that "totaled vehicle" and "salvage" are the same question.
So retrieval runs in three phases. Domain and schema anchors narrow the field, one hop of schema-graph expansion pulls in the tables that are actually joined to those anchors, then rank-BM25 and Titan v2 dense embeddings score candidates together, cosine over a normalized matrix with a 0.2 floor and 0.7 weight on the embedding side. The Assess node then reranks the top fifty down to between three and fifteen with the model itself acting as judge, which is what stops a plausible but wrong table from reaching the SQL generator.
The graph expansion step came out of the Redshift Dependency Tracer I had built the year before. Having already mapped the warehouse's table relationships is what made the retrieval tractable.
Evals gate every release
I own the harness. It measures retrieval recall@k against a golden set, faithfulness through LLM-as-judge with citation verification, and hallucination rate. Nothing ships past a regression.
If I were starting over I would build this in week one. I spent the first month iterating on prompts qualitatively and had no way to tell whether a change was an improvement or a trade. Rough metrics from day one would have beaten good metrics from month two.
Grounding, guardrails, and access
Every turn injects 300+ internal domain acronyms, because an agent that quietly guesses what an abbreviation means is worse than one that says it does not know. Hard anti-hallucination rules forbid the model from referencing any table or column outside retrieved context. Access-aware RBAC reads per-user table permissions from Redshift admin views, so two analysts asking the same question see different results, correctly.
None of that is a feature list. In a FedRAMP-aligned environment holding sensitive PII, it is the cost of being allowed to deploy at all.
Operating it
Responses stream and cancel on disconnect. Context compaction fires on a Haiku pass once a conversation passes both 16 messages and 140K tokens, with a FIFO trim at 160K. The table catalog is built offline on EC2, pushed to S3, and downloaded at container start, so the agent never queries Redshift just to figure out what exists. Deployment goes through GitLab CI/CD on the ECS component library I wrote.
Other systems
Production fraud model portfolio GovCloud
Logistic regression · SageMaker Pipelines · MLflow · champion/challenger · 2024 to 2025
The salvage-swap model was NICB's first production ML fraud classifier, and the first model I ever shipped. Logistic regression by choice, not by default: fraud labels are rare, the training set is small, and analysts will not act on a flag they cannot interpret. It scores 100M+ vehicle registration records in batch at 0.89 AUC and 0.84 recall. Staged accidents and workers' compensation classifiers followed in 2025.
GovCloud blocked shadow and blue-green deploys, so validation ran as champion/challenger with pre-registered precision, recall, and guardrail thresholds agreed before the run. That constraint turned out to be a better habit than the thing it replaced.
The team's ML platform, built 0 to 1 GovCloud
SageMaker Pipelines · GitLab CI/CD · MLflow · DVC · Poetry · AWS Glue · 2024
I joined a team where models lived in notebooks and nothing was reproducible across people. Over 2024 I built seven practices that the team still runs on, including the first SageMaker Pipelines deployment on AWS GovCloud, which required partnering with AWS to enable a service the region did not support yet.
The related work I am most pleased with is unglamorous: auto-shutdown policies, real dev and prod separation, and right-sizing took $100K a year off the AWS bill, which is what paid for the agent.
Redshift Dependency Tracer GovCloud
Python · Poetry · first internal package on the team · 2024
A visual tool that maps table relationship graphs across ~3,000 tables, traces source and dependent tables, generates access-aware create, replace, refresh, and delete SQL, and searches column descriptions by keyword. It replaced sifting through code by hand. Its retrieval approach became the foundation the agent's graph expansion is built on.
Llama 3.1 inference tuning, then deleting the GPUs
P4d.24xlarge, 8x A100, NVLink · tensor parallelism · KV-cache tuning · 2024 to 2025
The 2024 report summarizer ran on self-hosted Llama 3.1 and was too slow to use. vLLM and TensorRT-LLM had thin GovCloud support at the time, so I profiled it manually. Most of the time was not model compute, it was kernel selection and a single-GPU bottleneck. Multi-GPU tensor parallelism, kernels matched to our real sequence lengths and batch shapes, and KV-cache tuning took generation from five minutes to thirty seconds.
A year later, when the workload shifted from summarization to agents, I migrated the stack to Bedrock with prompt caching and decommissioned the GPU infrastructure entirely. I sat on that decision for about two months longer than I should have, because I was not convinced a managed API counted as serious infrastructure. It did.
SSN threat analysis
Hypothesis testing · distributional modeling · 45.8M claims
Member insurers believed invalid Social Security numbers signaled fraud but had no measurement. Across 45.8M claims over two years, claims with invalid SSNs turned out to be roughly twice as likely to become questionable-claim referrals (88% higher odds) or subject-of-investigation events (93% higher odds). Failed-SSN claims are about half a percent of volume at twice the base fraud rate, which makes the signal useful precisely because it is rare: a high-precision filter touching under 1% of claims.
Scope of experience
Worth being direct about, since it saves everyone a screening call. My serving experience is batch at large volume plus a conversational agent at roughly a thousand requests a day. I have not run online inference at high QPS, distributed training, or a feature store. I have evaluated fine-tuning and chosen against it every time so far; I have not shipped a fine-tuned model. What I have done a lot of is take a working prototype and make it survive real users inside an environment that audits everything.
Background
Ph.D., Interdisciplinary Engineering, Northeastern University, 2025. Dissertation on stochastic and behavioral modeling: Markov-chain occupancy, probabilistic habitual models, and logistic-regression feedback across 1,400+ simulated agents, improving prediction accuracy 50% over classical baselines. Twelve peer-reviewed papers, 408 citations. Also an M.S. in Energy Systems from Northeastern and a B.S. in Mechanical Engineering from the University of Mississippi.
AWS Certified Machine Learning Engineer, Associate (2025) · AWS Certified Machine Learning, Specialty (2024) · AWS Certified Cloud Practitioner (2024). Promoted to Senior Data Scientist, December 2025. Exemplary performance rating, 2024 and 2025.
Contact
The fastest way to reach me is email. I am most interested in agentic systems, evaluation infrastructure, and production LLM work where shipping reliably is the hard part, at Senior or Applied AI Engineer level, remote or in the Bay Area.
On the GovCloud markers. Work tagged above was built inside a restricted environment on sensitive data. Everything on this page is limited to what I can describe publicly. Implementation details, data, and internal specifics are not mine to share, and I will hold that line about your systems too.