AI-Powered Clinical Documentation for Radiation Oncology
A HIPAA-compliant tool that converts uploaded medical records into structured physician notes in minutes — replacing a workflow that costs doctors hours every day.
Browser mockup: VeloNote interface — file upload panel, generated report
Client
Dr. Brian Lawenda
Radiation Oncologist
Scope
AI Engineering
OCR Pipeline Architecture
HIPAA Infrastructure
Product Strategy
My Role
Led product strategy and AI architecture for a clinical documentation tool in active use on real patient records — navigating HIPAA compliance, multi-model OCR, and healthcare adoption.
Doctors spend hours assembling notes that should take minutes
Every consultation requires assembling imaging reports, lab results, pathology, surgical notes, and referral letters — from different providers, different systems, different formats — into a single structured note.
Most doctors either write it themselves (burns time), hire a scribe (burns money), or paste records into ChatGPT (burns trust if compliance finds out). None of these options are good.
The three current options: Manual (slow), Scribe (expensive), ChatGPT (not compliant)
Upload records. Get a structured note. Paste into the EMR.
Doctors upload medical records — screenshots, scanned pages, PDFs — and the system extracts content using OCR, feeds it into a medical LLM, and generates a fully structured consultation note. Minutes instead of hours. Fully HIPAA-compliant.
Flow: Upload Files → OCR Extraction → LLM Generation → Structured Note → EMR
Multi-model fallback flow diagram
A multi-model fallback system that never silently fails
Mistral OCR is fast and cheap, but it hallucinates — fabricating thousands of lines of text from a single scanned page. Instead of accepting bad output, we built a validation layer.
Mistral processes first. Haiku validates against the original image. If flagged, Opus Vision re-processes with full image comprehension. If that also fails, the file gets excluded and flagged in the UI. Reports always generate. A single bad file never crashes the batch.
3-4 min
Initial processing time (30 files)
~1 min
After parallel batching optimization
57 files
Largest batch processed without a hiccup
Synthesis, not copy-paste
The LLM correlates dates across documents, identifies which provider said what, pulls staging from pathology, treatment history from oncology notes, and imaging findings from radiology — into a single coherent narrative.
Sections that can't be populated are marked "Information not visible in provided documentation" rather than hallucinated. Drag-and-drop section reordering saves per-user. Chat-based refinement after generation.
Generated consultation note — structured sections, clean formatting
AWS conformance pack dashboard — 97% compliance score
Not a ChatGPT wrapper with a privacy policy
Built on AWS with HIPAA compliance as a hard constraint from day one. No medical records stored in databases — everything processes in-memory and is discarded after the session.
The system runs on the doctor's own AWS account. They own the infrastructure, the data, and the access controls. 97% HIPAA conformance pack score. VP of Compliance approved VeloNote for clinical use.
$0.15
Cost per standard report (10 files)
~66%
Cost reduction via model migration
97%
HIPAA conformance pack score
A doctor said it changed his life.
A medical oncologist — not even the original client — had been spending over an hour per note and hadn't completed all his notes in a single day since starting six months earlier. After using VeloNote, he finished every note that day. He called the next morning to say it was the first good night's sleep he'd had since joining the practice.
Compliance Unlocked
Healthcare products live and die on compliance approval. The VP of Compliance reviewed the documentation and approved VeloNote for clinical use.
The Real Friction
The first paying customer didn't convert — not because the AI was bad, but because his EMR required a snipping tool per page. The bottleneck wasn't AI quality. It was integration depth.
OCR Hallucinations
Mistral would produce thousands of lines of fabricated text from a single scan. Building the multi-model fallback — and calibrating it against false flags — became the core technical differentiator.
Building in production on real patient records
AWS rate limits on a new account
The client's Bedrock quota was 180x lower than our dev account. Reports randomly failed. Weeks of back-and-forth with AWS support to get limits raised.
Every fix risked breaking something
No staging environment. No automated tests. A fix for one pathology format could silently break date extraction on another. Regression was the constant enemy.
Three time zones, one doctor
Development happened in gaps between patient appointments. Bugs reported via Slack screenshots between visits. The pressure to not break anything was constant.
Adoption, not just functionality
The product works. Doctors are impressed. But getting them to change their workflow — that's the barrier. A working product is table stakes. Adoption requires near-zero friction.
Clinical AI documentation, live and compliance-approved
Multi-model OCR pipeline — Mistral, Haiku, and Opus working in sequence with automated validation
Speed optimized from 3-4 minutes to ~1 minute for 30+ files
Cost reduced by ~66% through model migration (Opus 4.1 → 4.5)
97% HIPAA conformance — compliance-approved for clinical use
Chat-based note refinement, drag-and-drop templates, follow-up report generation
Full infrastructure transferred to client's own AWS, GCP, and GitHub accounts