What AI Coding Tools Actually Promise in a HIPAA-Compliant World

A developer on your team is debugging a scheduling algorithm. They paste a function into an AI coding tool that references a patient’s Medicaid eligibility window. Did that data just get retained? Logged? Used for training? For most engineering leaders, the honest answer is: they don’t know. 

That knowledge gap has a cost. IBM’s Cost of a Data Breach Report found that 63% of organizations lack formal AI governance policies, and as AI tooling becomes embedded in everyday development workflows, the exposure only grows. 

The term you’ll encounter most is “Zero Data Retention” (ZDR), and while it’s genuinely important, it’s widely misunderstood. After a deep review of how Claude Code, OpenAI Codex, and Google Vertex AI actually handle data, here’s what ZDR means in practice, where each tool falls short, and what healthcare technology teams and engineers need to do now. 

What ZDR Actually Means (And What It Doesn’t) 

Zero Data Retention sounds self-explanatory: send a prompt, get a response, nothing is stored. The reality is more nuanced. True ZDR means the provider processes your data in real time and immediately discards it: no prompt logging, no retained outputs, no training use. What major providers actually offer is contractual ZDR, available only to enterprise customers who specifically negotiate for it, with important carve-outs. 

Every major provider retains data for at least some period under safety monitoring obligations. Even under ZDR agreements, if a session is flagged for a Terms of Service violation, inputs and outputs can be held for up to two years. ZDR is not an absolute guarantee of ephemerality for every request. More critically, ZDR and HIPAA compliance are related but distinct obligations. You can have one without the other, and neither is automatic. 

Claude Code: The Nuances Healthcare Teams Need 

Claude Code is among the most powerful agentic coding tools currently available, but its HIPAA story comes with complexity that teams need to understand before deployment. Anthropic’s ZDR offering is real, but it is explicitly tied to the Anthropic API via direct commercial contract. 

Claude Code bundled seats are not currently covered as part of Anthropic’s HIPAA-ready offering. If you want Claude Code usage covered for HIPAA-related services, you must contact Anthropic’s account team directly. It’s not a standard, self-serve path. 

There are additional exceptions that matter in healthcare contexts. Files uploaded via the Files API are retained until explicitly deleted, regardless of any ZDR agreement, which is a meaningful risk when uploading lab reports or care plans. If metrics logging is enabled for cost tracking, that

data is explicitly exempted from ZDR and may be retained. And critically, ZDR does not apply to Claude accessed through AWS Bedrock or Google Vertex AI. Those platforms offer their own data protections through architectural isolation and cloud BAAs, which is a legitimate HIPAA compliance path, but a different one. 

OpenAI Codex: Enterprise ZDR Exists, But Read the Fine Print 

OpenAI has a solid ZDR story for enterprise API customers, and Codex supports organizations with ZDR enabled at the ChatGPT Enterprise tier. But standard API endpoints retain inputs and outputs for up to 30 days for abuse monitoring, and ZDR is not self-serve. Enterprise customers must specifically request and receive approval. 

The cloud versus local distinction is critical: Codex cloud, where tasks run in OpenAI-hosted containers, is incompatible with ZDR. ZDR effectively only applies to the local Codex CLI and IDE extension within ZDR-approved organizations. If your team delegates work to Codex cloud agents, that traffic is not ZDR-protected even in a ZDR-enabled org. 

The biggest risk for healthcare organizations: consumer-facing ChatGPT products, including Free, Plus, Pro, and Team tiers, are not HIPAA-eligible under any circumstances. Developers using personal ChatGPT accounts on healthcare codebases are operating outside any Business Associate Agreement (BAA) or ZDR protection, with no contractual safeguard of any kind. Research shows that 38% of employees have shared confidential company data with unapproved AI tools, often through personal or unsanctioned accounts. 

Vertex AI: Structural Privacy, Not Contractual 

Google Vertex AI takes a fundamentally different approach. Rather than leading with ZDR as a contractual add-on, Google’s AI/ML Privacy Commitment is built into the platform: Google won’t use your data to train or fine-tune models without prior permission. This applies to all managed models on Vertex, including Claude accessed through Model Garden. Vertex supports HIPAA workloads, and Google’s BAA covers Vertex. HIPAA support is not a special add-on tier in the same way it is with other providers. 

One nuance worth noting: by default, published Gemini models cache customer data in-memory to reduce latency, with a 24-hour Time to Live, meaning before the data automatically deletes or expires. This caching can be disabled at the project level, but you have to know how to do it. 

Where ZDR Breaks Down in Agent Workflows 

Healthcare organizations are already moving aggressively into production deployment. McKinsey research shows that roughly 50% of U.S. healthcare organizations are already using generative AI, and more than 80% have deployed at least one use case to end users. That scale of adoption means the following failure modes aren’t edge cases; they’re happening in production.

For teams building production agents that handle care coordination, eligibility workflows, or intake pipelines, the core of agent-building on Claude’s API is ZDR-eligible. The Messages API processes data in real time with no storage after response. But several common patterns silently break ZDR posture. 

Code execution sandboxes retain container data for up to 30 days and are not ZDR-eligible. The Batch API, commonly used for nightly eligibility recalculations or bulk claim processing, is explicitly not ZDR-eligible, with a standard 29-day retention policy. Beta features are categorically excluded from ZDR arrangements until explicitly documented otherwise. 

The architectural implication: route all PHI-adjacent inference through the Messages API, avoid the Batch API for regulated data, and treat any beta feature as non-ZDR by default. 

What Healthcare Technology Teams Should Do Now 

The complexity above isn’t a reason to avoid these tools. It’s a reason to implement them deliberately. Here’s what that looks like in practice: 

● Audit every AI tool your development team is using. Assume that if it’s not explicitly covered by your BAA, it’s not HIPAA-safe for work that touches PHI or PHI-adjacent code. ● Understand that ZDR is a contractual relationship, not a product feature. You need a signed agreement, the right organizational tier, and explicit opt-in. 

● Know your compliance posture for each routing path. Cloud-managed inference through Bedrock or Vertex gives you a different compliance posture than going direct to model providers, not necessarily better or worse. Understand what you have, and document it. 

● Implement technical controls that don’t depend on memory. Deny rules, code review gates, network isolation; these are the controls that hold under deadline pressure. ● Stay current. Anthropic updated its consumer data retention policies significantly in late 2025. OpenAI has been iterating on ZDR documentation. Google Vertex has expanded HIPAA coverage. The compliance assumptions from six months ago may already be outdated. 

Healthcare continues to face the costliest data breaches of any industry, averaging $7.42 million per incident according to IBM. ZDR is real, and for healthcare organizations, it matters enormously. But it is not automatic, not universal, and not the same as HIPAA compliance. Put privacy architecture first, and the powerful AI tooling follows naturally. 

Anand Chandrasekaran is the AI Chief/Principal Engineer at Arya Health, a healthcare staffing platform serving home health agencies across the United States. Arya Health builds AI-powered systems for clinician recruitment, scheduling, and EMR integration.