HIPAA-Compliant Voice AI: The Architecture Questions to Ask Before You Sign a BAA

Bottom line. "HIPAA-compliant voice AI" is mostly marketing. A signed Business Associate Agreement (BAA) is necessary but insufficient. The actual compliance question is the data-flow architecture: what gets transcribed, where the transcript sits, which LLM API processes it, whether your data trained the model, and which subcontractors signed BAAs down the chain. This article walks the architecture, names the failure points, and gives you the questions to ask before you sign.

Healthcare practices evaluating voice AI hear the same pitch from every vendor: "we're HIPAA-compliant, we'll sign a BAA, you're protected." Sometimes that's true. Often the claim is technically correct but operationally incomplete, because HIPAA compliance is not a checkbox at the vendor layer — it's an end-to-end property of how protected health information (PHI) flows through the system.

We're building Sawy, an AI receptionist that will launch Q3 2026 with healthcare practices as one of our target verticals. We have skin in the game. We're also writing this article because the existing top-ranked content on the keyword is sales copy from vendors who don't explain the architecture — and a practice manager who signs a BAA without understanding the architecture is signing something they cannot meaningfully audit.

The audience for this article is the practice manager, IT director, or owner who has to make the decision and answer to a HIPAA officer (theirs or a future auditor). It is not legal advice; consult your compliance counsel for the formal review. It is the engineering-grade explanation of what to look for.

The 30-second answer

A voice AI is HIPAA-compliant when every component that touches PHI is covered by a BAA, the data-flow architecture minimizes PHI exposure, and the vendor can produce an audit trail of who accessed what and when. The single signed BAA at the front door does not satisfy these requirements on its own.

The five architecture questions that separate real compliance from compliance theater:

1. Which LLM API processes the call content, and is the vendor on that LLM's BAA-eligible tier?
2. Where are the audio recordings and transcripts stored, for how long, with what encryption?
3. Was the vendor's model trained on customer call data, and what's the opt-out posture?
4. Which subcontractors are in the data path (telephony, STT, hosting), and are they each under BAA?
5. What is the incident-response and breach-notification timeline?

If a vendor can't answer all five in writing, the BAA is hollow.

What HIPAA actually requires of voice AI

Two definitions to anchor everything:

• PHI (Protected Health Information): any individually identifiable health information transmitted or maintained in any form. A voice recording of a patient discussing their symptoms is PHI. A transcript is PHI. The phone number plus appointment time is PHI.

• Business Associate (BA): any organization that creates, receives, maintains, or transmits PHI on behalf of a covered entity. A voice AI vendor is a Business Associate. So is the LLM API the vendor uses. So is the cloud hosting provider storing transcripts. Every BA in the chain must have a BAA with the entity above it.

This is where the marketing claim "HIPAA-compliant" breaks down. A vendor can sign a BAA with you, and still be violating HIPAA — because they don't have BAAs in place with their own subcontractors, or because they're using an LLM API in a non-BAA-eligible tier, or because they're training their model on your data without authorization.

The HIPAA Privacy Rule and Security Rule (45 CFR Part 160 and Part 164) impose obligations on both you (the Covered Entity) and the BA. The Security Rule's technical safeguards — access controls, audit logging, transmission security, encryption — apply to every BA in the chain. The Breach Notification Rule (45 CFR §164.400-414) requires notification within 60 days of discovery, with specific content and form requirements.

For the regulatory primary source, see the HHS Summary of the HIPAA Security Rule and the BA contracts overview. The form provisions are non-negotiable; the implementation is where vendors vary.

The standard voice AI data flow (and where PHI lives at each stage)

Most voice AI products follow the same pipeline. Each stage is a place where PHI exists in the system, and each stage requires its own controls.

| Stage | What happens | Where PHI lives | Risk | |---|---|---|---| | 1. Telephony ingress | Call hits a phone number, audio streams in | Audio packets in transit | Transmission encryption (TLS/SRTP) | | 2. Speech-to-text | Audio converted to text via STT model | Audio + transcript on STT provider's infrastructure | STT provider must be under BAA | | 3. LLM processing | Transcript sent to LLM API to generate response | Transcript at LLM provider | LLM API must be on BAA-eligible tier | | 4. Text-to-speech | LLM response converted back to audio | Generated audio at TTS provider | TTS provider under BAA if voice synthesis uses PHI | | 5. Storage | Recording + transcript saved for QA, dispute, audit | Storage system (cloud database, object store) | Encryption at rest, access controls, retention policy | | 6. Application logs | Diagnostic logs of the call | Log aggregator (Datadog, Splunk, internal) | Log scrubbing or log aggregator under BAA | | 7. Training data | Recordings used to improve the model | Vendor's training pipeline + future model weights | Opt-out controls + clear data-use posture |

The PHI exposure surface is large. Every box above is a potential compliance failure if not addressed. The questions that follow walk each one.

Architecture question 1 — Which LLM API, on which tier?

This is the single most common compliance hole in voice AI vendors and the one healthcare buyers ask about least.

Most voice AI vendors don't run their own LLM. They send the transcript to OpenAI, Anthropic, Google, or another LLM API and use the returned text to drive the response. The LLM provider is a Business Associate, and they require a BAA — but only on specific paid tiers.

• OpenAI: BAAs are available for API customers — email baa@openai.com with company and use-case details; OpenAI reviews case-by-case and typically responds within a few business days. Only the zero-retention API endpoints are HIPAA-eligible (the endpoints eligible for zero retention are listed in OpenAI's data documentation). The consumer ChatGPT product is not HIPAA-eligible; ChatGPT Enterprise and ChatGPT Edu are eligible only via sales-managed accounts. See OpenAI's BAA help article for the current process.
• Anthropic: BAA available for sales-assisted Enterprise plans and for the Claude API. As of December 2, 2025, a single BAA can cover both Claude API and Enterprise usage. Self-serve Enterprise sign-ups do not include a BAA — you need the sales-assisted path. Once HIPAA mode is enabled, it cannot be reversed from admin settings. See Anthropic's BAA documentation and HIPAA-ready Enterprise plans.
• Google (Vertex AI / Gemini): Vertex AI is HIPAA-eligible when covered by Google's BAA, with private endpoints, granular IAM, audit logging, and customer-managed encryption keys. Gemini in Workspace and the Gemini app (web and mobile) are also BAA-covered. Important exclusions: consumer Gemini, NotebookLM, and Gemini in Chrome are not under the BAA. Confirm specific products against Google Cloud's HIPAA compliance page before deployment.
• Azure OpenAI Service: HIPAA-eligible by default via the Microsoft Online Services Data Protection Addendum — no separate BAA signing required for eligible customers (Enterprise Agreement, CSP, or equivalent). Coverage is limited to text-based interactions in production. Preview features, DALL·E, and voice inputs are not currently HIPAA-compliant unless Microsoft explicitly states otherwise. See the Azure HIPAA documentation.

The question to ask the vendor: "Which LLM provider processes our call transcripts, and can you produce the BAA between you and that provider?" If they say "we don't use a third-party LLM, we run our own" — ask which model and where it's hosted. If they say "we use OpenAI" — verify it's the Enterprise tier with a BAA, not the consumer API.

If the vendor cannot produce the upstream BAA on request, the chain is broken. Your BAA with the vendor is not sufficient because PHI is leaving the vendor and going to a third party who hasn't signed.

Architecture question 2 — Recording and transcript storage

Every voice AI captures audio recordings and text transcripts for at least three purposes: real-time response generation, post-call QA and dispute resolution, and audit logging. PHI lives in storage from the moment the call ends until it's destroyed.

Questions to ask:

• Where is it stored? Cloud provider, region, dedicated infrastructure vs. shared. AWS GovCloud and Azure for Government are higher assurance than commercial cloud; many vendors use commercial cloud, which is fine if encryption and access controls meet Security Rule requirements.
• What encryption? At rest (AES-256 minimum), in transit (TLS 1.2+), and key management (customer-managed keys are stronger than vendor-managed).
• How long is the retention? A vendor that stores recordings indefinitely is creating an ever-growing breach surface. Best practice: configurable retention with a 30-90 day default and immediate destruction on request.
• Who has access? Internal vendor staff (engineering, support, ML team) should have logged, audited access only with a documented business need. Ask for the access policy.
• Can you produce an audit log? If you can't get a per-record access log on request, the access controls aren't real.

The pattern to watch for: vendors that retain recordings "for model improvement" by default, with opt-out buried in the settings. That's training-data risk hidden inside storage policy. See question 3.

Architecture question 3 — Was your model trained on customer call data?

This is the question vendor sales pages do not answer voluntarily. It is also the question most likely to determine whether your PHI is irretrievably mixed into a model's weights with no practical way to extract it.

There are three possible postures:

1. No customer data used for training, ever. Cleanest. The vendor trains on synthetic data, licensed datasets, or pre-trained foundation models from upstream providers. Customer recordings are used only for live response and storage, never for model improvement.

2. Customer data used for training only with explicit opt-in. Acceptable if the opt-in is clear, granular, and revocable. Risky if "opt-in" is a checkbox buried in the ToS that the practice manager clicked without reading.

3. Customer data used for training by default with opt-out. The default risk-bearing posture. PHI flows into model weights without the practice's deliberate consent. Even if the model isn't retrained immediately, the data is in the vendor's training pipeline and the exposure surface is large.

The question to ask: "Is our call data used to train, fine-tune, or evaluate your models? If so, is it opt-in or opt-out, and can you provide written confirmation of our posture?" Get it in writing in the BAA or as a separate addendum.

The honest part: even a vendor that says "no customer data used for training" still has the data on their systems while it's stored. The point is that the data isn't being incorporated into the model in a way that can't be undone. Training-data mixing is the irreversible state.

Architecture question 4 — The BAA chain

A voice AI is not one vendor. It is typically 5-8 services stitched together. Every one of those services that touches PHI is a Business Associate, and the chain must be intact.

The typical chain for a voice AI:

You (Covered Entity)
  └── BAA ── Voice AI vendor
                ├── BAA ── Telephony provider (Twilio, Telnyx)
                ├── BAA ── STT provider (Deepgram, AssemblyAI, or built-in)
                ├── BAA ── LLM API (OpenAI, Anthropic, Google, Azure)
                ├── BAA ── TTS provider (ElevenLabs, Cartesia, or built-in)
                ├── BAA ── Cloud hosting (AWS, GCP, Azure)
                └── BAA ── Logging/observability (if PHI in logs)

The vendor is required to either have BAAs with every downstream service that touches PHI, or to engineer the system so the downstream service never sees PHI. Either is acceptable; no BAA + service sees PHI is a compliance failure.

The question to ask: "Please list every third-party service in your data path that touches PHI, and confirm BAA coverage for each." A vendor that has thought about this can produce the list in 24 hours. A vendor that hasn't will hedge.

The common failure: telephony providers like Twilio have HIPAA-eligible offerings, but a vendor on Twilio's standard tier may not be covered. Same with Deepgram, AssemblyAI, and most STT vendors. Tier matters. Ask which tier the vendor is on.

Architecture question 5 — Incident response and breach notification

HIPAA Breach Notification Rule requires the Covered Entity to notify affected individuals (and HHS, and sometimes media) within 60 days of discovery of a breach. If the breach happens at the vendor (which is now common), the vendor has to notify you fast enough for you to meet your obligation.

Questions to ask:

• What is the vendor's breach-discovery process? Continuous monitoring, periodic audit, customer-reported? Continuous monitoring is the right answer.
• What is the vendor's notification SLA? 24 hours is best practice; anything over 7 days is operationally risky given the 60-day clock.
• What documentation does the vendor provide on a breach? You need: scope of records affected, dates of breach and discovery, nature of unauthorized access, mitigation steps. Less than this and you can't fulfill your own notification obligation.
• Has the vendor had a breach before? If yes, ask for the post-mortem (redacted if needed). If no, ask what they would do if they did.

The pattern that fails: a vendor with no breach-response playbook discovers an exposure, takes three weeks to investigate, then notifies the customer with a vague description that doesn't give the practice enough information to notify patients within the legal window.

How to evaluate a vendor — a practical checklist

This is the document you take into a vendor evaluation. Print it. Fill it in for each vendor.

| Question | Vendor 1 | Vendor 2 | Vendor 3 | |---|---|---|---| | Signed BAA with us? | | | | | LLM provider used? | | | | | LLM provider tier (BAA-eligible)? | | | | | Upstream BAA with LLM provider? | | | | | Storage location + encryption? | | | | | Retention policy + destruction-on-request? | | | | | Customer data used for model training? Opt-in/out posture? | | | | | Full subcontractor list (telephony, STT, TTS, hosting, logs)? | | | | | BAA in place with each subcontractor? | | | | | Audit log access (per-record, on demand)? | | | | | Breach notification SLA to customer? | | | | | Last security audit (SOC 2 Type II preferred)? | | | | | Penetration test cadence? | | | | | Voice cloning safeguards? | | | |

A vendor that can answer all 14 of these in writing within 5 business days is a vendor with mature HIPAA operations. A vendor that can't answer 5 of them in 2 weeks is not ready to be your Business Associate.

A small experiment: 10 voice AI vendor pages reviewed for HIPAA architecture disclosure

To pressure-test how much information practices actually get from public vendor pages, we reviewed the public-facing HIPAA-marketing pages of 10 voice AI vendors as of May 2026. We scored each on whether they disclose (publicly, without a sales call) the seven facts that matter most:

1. Which LLM provider they use
2. The LLM provider's BAA-eligible tier
3. Their storage location and encryption posture
4. Default retention period
5. Customer-data training opt-in vs opt-out posture
6. Full subcontractor list with BAA status
7. Breach-notification SLA

Result on our 10-vendor sample:

| Disclosed | Score | |---|---| | 7 of 7 facts disclosed publicly | 0 vendors | | 4-6 of 7 disclosed | 1 vendor | | 1-3 of 7 disclosed | 4 vendors | | 0 of 7 disclosed (only "we are HIPAA compliant") | 5 vendors |

What the sample shows: Half the voice AI vendors marketing "HIPAA compliance" do not publicly disclose any of the seven architecture facts that determine whether their compliance claim is operationally meaningful. This is not necessarily evidence of non-compliance — many vendors are compliant and simply don't put the details on the marketing page — but it does mean buyers cannot evaluate compliance from public materials alone. Every evaluation needs a direct conversation, written answers, and the full BAA chain documentation.

Methodology caveat: We did not name specific vendors in this article because the marketing pages change and we don't want a snapshot to be misread as a current verdict. If you want help running this checklist against specific vendors you're evaluating, the best AI receptionist buyer's guide has the broader comparison, but the HIPAA-specific evaluation is a conversation with each vendor's compliance team.

The "compliance theater" pattern to watch for

A few specific patterns that should raise flags:

• "HIPAA-compliant" with no detail. The vendor's HIPAA page is one paragraph that says they will sign a BAA and they take security seriously. No architecture, no subcontractor disclosure, no LLM-tier confirmation.

• "Enterprise-grade encryption." Filler language. Ask which algorithm, which key management approach, and where keys are stored.

• "SOC 2 certified." SOC 2 is a security control framework, not a HIPAA certification. It's correlated with maturity, but not equivalent. A SOC 2 Type II report (continuous, audited) is meaningfully stronger than Type I (point-in-time).

• "BAA available on request." Means the vendor is willing to sign one, not that they've thought about what BAAs they need with their upstream providers. Ask follow-ups.

• Voice cloning marketed without controls. If the vendor offers custom voice cloning trained on staff voices, ask about consent, audit, and the risk of voice misuse. Voice biometrics are PHI when associated with health context.

• "We don't store any data." Unlikely to be literally true. Storage is required for audit logging and dispute resolution under most security frameworks. Push for what they actually mean.

When voice AI is the wrong tool for a healthcare practice

To stay honest, voice AI is not the answer for every practice. Specifically:

• Crisis-line and suicide-risk practices. Calls require licensed human first-touch in many jurisdictions and in clinical best practice. AI as overflow to a triage human is the most you should consider, and only with explicit clinical sign-off.
• Complex psychiatric or medical triage above MA scope. If your front-desk decision tree requires nursing or clinical judgment beyond scheduling, AI is wrong as first-touch.
• Practices with patient demographics who will not engage with AI. Some patient populations (elderly with hearing impairment, severe anxiety populations, populations with strong cultural preference for human voice) will hang up. Test before you switch.
• Practices with active OCR investigations or recent HIPAA enforcement actions. Adding a new BA mid-investigation creates evidentiary complexity. Resolve first.

For everyone else — a dental practice, a dermatology clinic, a primary care office, an urgent care, a mental health practice with capacity for AI-handled scheduling — voice AI is operationally viable if (and only if) the architecture is sound.

FAQ

Is voice AI HIPAA-compliant?

Voice AI can be HIPAA-compliant when (1) every component touching PHI is covered by a BAA, (2) the data-flow architecture minimizes PHI exposure, and (3) the vendor can produce audit logs and breach-response documentation on demand. "HIPAA-compliant voice AI" as a marketing claim from a vendor is not the same as a genuinely compliant deployment in your practice. The compliance is a property of the end-to-end system, not a vendor checkbox.

Do I need a BAA with my AI receptionist provider?

Yes. The AI receptionist provider is a Business Associate under HIPAA — they create, receive, and transmit PHI on behalf of your practice. A signed BAA is required. It is also not sufficient on its own; ask for the upstream BAAs the provider has with their LLM, STT, hosting, and telephony subcontractors.

What's the difference between a HIPAA-compliant AI and a regular AI?

A HIPAA-compliant deployment has: BAAs in place with every downstream service, encryption at rest and in transit, configurable retention with destruction-on-request, opt-out (or opt-in) for customer data being used in model training, audit logging, breach-notification SLA, and clear documentation of the full data-flow architecture. A non-compliant deployment lacks one or more of these. The model itself is rarely the differentiator; the deployment configuration is.

Can I use OpenAI or ChatGPT for healthcare?

The consumer ChatGPT product is not HIPAA-eligible. OpenAI offers BAAs for API customers via baa@openai.com, but only the zero-retention API endpoints are HIPAA-eligible. ChatGPT Enterprise and ChatGPT Edu can be HIPAA-eligible via sales-managed accounts. Azure OpenAI Service (Microsoft's deployment of OpenAI models) is HIPAA-eligible by default via the Microsoft Online Services DPA — a different and often simpler path than going through OpenAI directly. The right answer for your practice depends on which deployment your vendor uses; ask the question and confirm with OpenAI's BAA documentation.

How much does HIPAA-compliant voice AI cost?

Category pricing for HIPAA-eligible voice AI runs higher than non-HIPAA voice AI because the underlying LLM tier (Enterprise) costs more and the vendor's operational overhead (compliance staff, audit, BAA management) is real. Expect $200-$1,500 per month for small-to-mid practice volumes from compliant vendors. The biggest mistake is buying the cheapest vendor and discovering they're using a non-BAA LLM tier in production. The savings vanish the first time a patient complaint triggers a compliance review.