David B.
Model Evaluation Engineer Golden Sets, CI/CD Gates, LLM Red-Teaming,
Ship LLM features with measurable confidence — not vibes. I'm a Model Evaluation Engineer (EDI / AI Quality Engineer). I build the evaluation infrastructure that lets teams go/no-go with evidence: frozen test sets, bootstrap confidence intervals, 3-scenario reporting, CI/CD gates, drift monitoring, and compliance. What I deliver (async-first, minimal sync): - Golden sets that don't rot — versioned, SHA-256 frozen, kappa ≥ 0.8, adversarial, refresh cadence - Harnesses that run in CI/CD — pytest + Instructor (schema enforcement) + judge/coherence 2nd-pass + bootstrap CI - Gates that map to business risk — 3-scenario (pessimistic/base/optimistic) reporting, regression suites, shadow/canary - Compliance built-in — Ley 25.326 (Argentina), GDPR, EU AI Act prep, PII minimization, DPA/SCC management - Red-teaming that finds real holes — prompt injection, PII leakage, jailbreak scenarios, adversarial sets Recent proof: - MultiOCR (Concentrix/labIA): Field F1 0.971, ANLS 0.984, Schema OK 100%, latency p95 165s, cost $0.0129 — measured over 3-scenario bootstrap - AySA MVP: 4-week build, golden set 300+ emails (kappa≥0.8), dual-track ESI/EDI, self-hosted Qwen, frozen test - Open source: ai-eval-harness — golden set manager, Instructor prompts, judge/coherence, bootstrap CI, GitHub Actions gates Stack: Python, Instructor/Pydantic, vLLM/Ollama, MLflow, Evidently, pytest, GitHub Actions, Docker, PSI/KL drift, MLflow. Work style: 100% remote, async-first, minimal sync (GMT-3, overlap US/EU mornings). Deliver repos + dashboards + runbooks. Sync only when needed. Engagement types: Fractional retainer (20–30h/week), project-based harness build, eval audit, red-team audit, compliance package. - AySA MVP: 4-week build, golden set 300+ emails (kappa≥0.8), dual-track ESI/EDI, self-hosted Qwen, frozen test - Open source: ai-eval-harness — golden set manager, Instructor prompts, judge/coherence, bootstrap CI, GitHub Actions gates Stack: Python, Instructor/Pydantic, vLLM/Ollama, MLflow, Evidently, pytest, GitHub Actions, Docker, PSI/KL drift, MLflow. Work style: 100% remote, async-first (GMT-3, overlap US/EU mornings). Deliver repos + dashboards + runbooks. Sync only for kickoff/readout. Engagement types: Fractional retainer (20–30h/week), project-based harness build, eval audit, red-team audit, compliance package.
Earnings and projects
An estimate from an earlier read
Billed in all: $13,614$14k
Across 22 contracts
Earnings, last 30 days
No data since Nov 5, 2026.