Privacy with AI

The design, test evidence and limits of the Protostar AI Secure AI Gateway

A Protostar AI technical white paper. Author: Nick Soro, Founder, Protostar AI, LLC. October 2026. Version 2.0.

Abstract

The Protostar AI Secure AI Gateway sits between an application and the AI models it calls. It classifies each request, replaces identifiers with stand-in values, sends the request to a model tier that the data class permits, restores the real values after the answer returns, and writes a hash-chained audit record.

One rule carries the privacy claim. Regulated data, class C1, is served only by a self-hosted model in a network that has no route to the internet. That rule does not depend on detecting identifiers. A test that turns detection off completely still shows no C1 data reaching an external model.

Detection is the second layer. On an adversarial test set the deployed detector leaked nothing. On a harder set built from the cases it cannot reach, it missed 44% when the caller did not declare the data class and 0% when the caller did. This paper gives both numbers, the method behind them, and the limits that follow.

1. Scope and status

This paper describes the gateway as built and deployed in October 2026. It states what was measured and when. It does not describe intentions as facts.

The deployed instance is operated by Protostar AI in a United States AWS region. The stack is defined as containers and infrastructure code, so the same design can be deployed into a customer’s own cloud account.

2. Threat model

The design defends confidential data against six adversaries. The table states what each one can obtain and which control answers it.

Three cases are outside the design. The gateway cannot protect data on a compromised customer system before the request is made. It cannot protect against compromise of the cloud account that hosts it. It cannot protect text that a person has already typed into a different AI product.

3. Architecture

Every request enters through one service, the gateway, and passes through one function that enforces the rules in a fixed order. There is no second path to a model.

3.1 The life of a request

The gateway handles a chat request in ten steps, always in this order.

1. Authenticate. The API key is hashed and looked up. A revoked or expired key is refused.
2. Authorize and throttle. The key must carry the chat scope. A key over its per-minute limit is refused.
3. Classify. The gateway classifies the text of every message in the conversation, not only the last one, because the whole conversation is sent to the model.
4. Route. The policy table picks the tier. A second, independent check confirms the tier is allowed for the class. A mismatch stops the request.
5. Replace identifiers. Every message is rewritten with stand-in values before any model is called, including the self-hosted one.
6. Record the intent to leave the boundary. If the chosen tier is external, the gateway writes an audit record first. If that write fails, the external call does not happen.
7. Call the model.
8. Restore. The stand-ins in the answer are replaced with the real values, inside the gateway process.
9. Flag invented values. Any run of four or more digits that is in the answer and not in the request is listed for the caller. It is flagged and never removed.
10. Audit and respond. The completion record is written. The response carries the answer, the data class, the tier used, and the list of identifiers that were replaced.

4. Data classes and routing

4.1 Three classes

The routing table is three lines of code:

C1: [Tier 1]
C2: [Tier 1, Tier 2]
C3: [Tier 1, Tier 2, Tier 3]

4.2 How a request gets its class

The gateway assigns a class from what it finds in the text. A Social Security number, medical record number, account number, credit card number or credential makes the text C1. Any other identifier makes it C2.

Two rules make the result conservative.

• A caller can raise the class and cannot lower it. If the caller declares C3 and the gateway finds a Social Security number, the request is C1.
• Text with no declared class is never C3. It is at least C2. C3 is reached only when the caller declares it and the gateway finds no identifier.

Declaring C1 is the strongest control a caller has, because it does not depend on detection. An application that knows it handles patient data declares C1 on every such request, and the request cannot leave Tier 1.

4.3 The per-tenant ceiling

Each tenant has a ceiling: the highest tier it may use. A tenant with no ceiling configured gets Tier 1. A configuration error therefore produces no external traffic.

The ceiling can only remove tiers. It is applied after the routing table and never adds a tier to a class. A tenant whose ceiling is Tier 2 still has every C1 request served on Tier 1.

4.4 Other routing rules

A request that asks for output in a fixed schema is served on Tier 1, because only the self-hosted model enforces the schema during generation.

Tier 3 requires three things at once: the request opts in, the class is C3, and a provider with a no-retention agreement is configured. In production no provider is configured, so the tier cannot be reached.

5. Finding identifiers

5.1 Two detectors, combined

The gateway takes the union of both. Each layer covers a gap in the other. The entity model did not flag a bare 123-45-6789 in testing, and the pattern for it always does. The patterns cannot recognize an untitled person’s name, and the entity model can.

Both layers run on CPU inside the gateway container. The models are built into the container image and nothing is downloaded at run time, because the container has no internet route.

5.2 Not masking what is not an identifier

A detector that masks too much destroys the meaning of clinical text. Two independent reviews rejected an early version because it masked drug doses (“250 mg”) and clinical adjectives (“the patient, febrile,”). The current detector removes these false matches from the entity model’s output:

• bare years, durations and frequencies tagged as dates;
• drug names, anatomy terms, units and currency codes tagged as names or places;
• medical eponyms, clinical acronyms and form labels such as “Date of Birth:”.

A bare year is kept on purpose. The HIPAA Safe Harbor method removes all elements of a date except the year (45 CFR 164.514(b)(2)).

One suppression rule is deliberately narrow. A short clinical code such as “C43” is left unmasked only when it stands alone. When the entity model tags only the first fragment of a longer identifier, suppressing that fragment would leave the whole identifier in clear text.

5.3 Overlapping matches

When two detected spans overlap, the gateway masks the union of both. Discarding either span would leave part of an identifier in clear text.

5.4 Coverage of the 18 HIPAA Safe Harbor identifiers

Ages are also masked. One narrow gap remains: Safe Harbor requires suppressing a year that implies an age over 89, and a bare birth year of that kind is not yet treated specially.

6. Replacing and restoring identifiers

6.1 The request vault

Each request gets its own vault: a table, held in the gateway’s memory, that maps each stand-in to the real value. The vault is never written to disk, never logged and never sent anywhere. It is discarded when the request ends. The model clients have no access to it, so they receive stand-ins by construction.

Within one request the same real value always maps to the same stand-in. The model can therefore tell that two mentions refer to the same person.

6.2 Stand-ins a model will use

A bracketed token such as [NAME_001] makes a model treat the value as missing. It refuses the task or writes around the gap. The gateway instead substitutes a realistic value of the same kind: a name for a name, a date for a date.

The stand-ins are chosen so they cannot be real:

• Social Security numbers use the 900 area, which is never assigned;
• telephone numbers use the reserved 555-01xx range;
• email addresses use example.com.

Names are drawn from pools arranged so that no two people in a request share a first or a last name. A reply that says “Dr. Avery” therefore restores to exactly one person. The pools give unique full names for up to 400 people in one request.

6.3 Restoring

After the model returns, the gateway replaces every stand-in in one pass, longest first. A single pass means a restored value is never scanned again, so a real value that happens to look like a stand-in is not replaced twice.

Restoration is best effort. If the model rewrites a stand-in beyond a simple abbreviation, the real value may not be put back in the prose. This affects the correctness of the answer and not privacy: the real value never left the gateway. The response also carries the full list of replaced identifiers, which is authoritative whatever the prose says.

6.4 Hidden characters

Text can carry characters that do not display: zero-width characters, direction overrides, and the Unicode tag block. They can hide an instruction to a model or a tracking mark that travels with copied text. The gateway strips these characters from every message going in and every answer coming out. Ordinary letters in every script, accents, tabs and line breaks are kept.

6.5 The system prompt is not a privacy control

The gateway adds a system prompt suited to the tenant’s domain: general, clinical, financial or legal. It reduces bad output. Nothing in the design relies on it to keep data inside the boundary.

7. Splitting a request across providers

For C3 work sent to external providers, the gateway can divide a request into fragments at sentence boundaries, send different fragments to different providers, and join the results inside the boundary. No single provider receives the whole request.

The gateway does this only when it has at least two providers and the request yields at least two fragments. Otherwise it sends the request to one provider and the audit record says the request was not split. The record never claims a split that did not happen.

This function depends on Tier 3. It has been tested with mock providers and is not in use in production.

8. The audit log

8.1 A chain of hashes

Each audit record contains the hash of the record before it:

hash = SHA-256(previous_hash + record_as_sorted_JSON)

Changing or deleting any record changes every hash after it. Verification walks the chain and reports the first break.

Each tenant has its own chain, stored in PostgreSQL. Two requests from one tenant that arrive together are written one after the other under a database lock, so the chain cannot fork.

8.2 What a record holds

No record holds message text, a real identifier, or an API key. The number of identifiers replaced is stored as a count.

The intent record exists because of one failure case. If an external call is made and the completion record then fails to write, the log would have no trace that data left. Writing the intent first closes that case.

8.3 Deleting old records without breaking the chain

Retention rules require deleting old records, and deletion would normally break the chain. The gateway first writes a seal that records the hash of the last deleted record and how many were deleted, then deletes them. The seals form their own chain. Verification checks the seals, then continues from the latest seal through the remaining records.

The remaining records stay verifiable, and the fact and size of each purge is itself tamper-evident. Purging is an explicit operator action. It is not triggered by requests, and it is switched off in the current deployment.

9. Deployment

9.1 A network with no way out

The gateway, the model and the database run in private subnets across two availability zones. The route table for those subnets has no default route: no internet gateway and no NAT gateway. A process in those subnets cannot open a connection to the internet, whatever the code does.

The services reach the cloud provider only through private endpoints. The reachability rules are narrow:

The model’s weights are built into its container image, so the model needs no network to serve.

9.2 Components

API keys are stored as SHA-256 hashes, so the database does not hold a usable key.

9.3 Failing closed

The gateway refuses to start in two cases: when a production configuration still has the development API key, and when more than one tenant is configured without the database that holds per-tenant audit chains.

Three behaviours at run time follow the same rule.

• A tenant missing from the ceiling configuration gets Tier 1.
• A malformed ceiling entry is skipped, so a typing error can only lower a tenant’s ceiling.
• If the audit record of intent cannot be written, the external call is not made.

When a model fails, the caller receives an error with no upstream detail, because a provider’s error text can echo the request.

10. Evaluation

Every figure in this section names the detector that produced it and the date it was measured.

10.1 The automated test suite

The suite held 444 passing tests and 5 skipped on 4 October 2026. The 5 skipped tests need a live database or a live Bedrock model, and they run when those are present.

Two tests carry the central claim.

• Randomized simulation. 2,000 randomized requests, each with random identifiers and a random declared class, run through the real request path. The test asserts that no detectable identifier reached a model client, that every C1 request was served on Tier 1, that values were restored, and that the audit chain verified. All 2,000 passed.
• Containment with detection off. The request path runs with detection disabled, so nothing is masked, and with an external provider configured and opted in. C1 data still reached only Tier 1.

The second test is the basis for the statement that C1 containment does not depend on detection.

10.2 Detection on an internal corpus

The internal corpus has 36 examples with 76 labelled identifiers in 9 categories. It is synthetic. It guards against regressions and is not evidence about real text.

Measured 28 July 2026. Scoring was at entity level with overlap matching, the method of the i2b2/n2c2 de-identification tasks.

10.3 Adversarial test

The adversarial corpus holds identifiers written to evade detection: digits spelled out, emails written as “name at clinic dot org”, rare names in lowercase, hyphenated numbers. Each identifier is scored twice. Was it masked? If it was missed, was the text still classed C1 and held on Tier 1? A leak is an identifier that was missed and was also free to leave.

Committed corpus: 14 texts, 17 identifiers. Deployed detector, measured 28 July 2026.

Hard corpus: 22 texts, 25 identifiers. This corpus was built from the cases the detector cannot reach without masking doses and counts: rare names in unusual positions and short ambiguous numbers.

The 44% is the figure a reviewer should weigh. When the caller does not declare the class and the tenant is allowed to use Tier 2, detection is the only barrier, and on the hardest cases it stops about half. When the caller declares C1, the leak rate is 0% whatever the detector finds. The same result holds with the entity model switched off.

10.4 Does the answer survive?

A separate test checks that replacing and restoring identifiers does not damage the task. Over 12 tasks run through the full path with a deterministic test model, no real identifier reached the model, every required identifier was restored, and every task succeeded. With a live Claude model on Bedrock, no real identifier reached the model and task success was between 92% and 100%.

10.5 Public benchmarks

Detection was scored on five public corpora on 11 October 2026, on two detectors: the deployed one (patterns plus an entity model on CPU) and the evaluation one, which adds a transformer entity model. Scores are as the corpora are labelled. F1 is the harmonic mean of precision and recall.

The deployed detector has no organisation-name layer, which is most of its recall gap on TAB and Nemotron. Adding one is in progress and will be measured before it ships.

A sixth corpus, ai4privacy, was scored in July 2026 and has been withdrawn: its licence permits academic and non-commercial use only, so it is no longer used.

The precision figures count each detection layer’s match separately. Where two layers find the same identifier, both count, so precision is higher than a count of masked spans would give. The method is kept for comparability with earlier runs and will be published alongside a merged-span view.

Four choices hold these scores below what tuning could reach, and each is deliberate.

1. A bare year is not masked, following HIPAA Safe Harbor. This accounts for 52 of the 64 remaining misses on MultiGraSCCo. A stricter per-tenant profile that masks bare years and ages over 89 is available; it scores 0.952 on MultiGraSCCo and 0.865 on TAB with the evaluation detector, at a cost of about 0.005 on the clinical and finance corpora.
2. All organisation and place names are masked, including ones a corpus does not label. This lowers precision, most of all on TAB, where 78% of the false positives are courts and states that the corpus marks as not identifying.
3. Job titles, professions and demographic attributes are not claimed. This is the whole recall gap on Nemotron-PII.
4. Short bare numbers are not masked, because doing so masks doses and counts. A rule that masked them was built, measured, and removed: it masked heart rates and lab values.

Two of these corpora cannot reach 0.99 with any detector. Gretel left 288 real dates in the sample unannotated, so a detector that masks every date, as HIPAA requires, is scored wrong on each. TAB anonymises the person rather than the document, marks institutions as not to be masked, and its own annotators disagree on 1,788 spans; the system in the TAB paper, trained on TAB itself, reaches about 0.875.

Two clinical corpora with the highest published scores, i2b2/n2c2 2014 and the PhysioNet de-identification set, require credentialed access. The loaders are written. Neither has been run and no score is claimed.

Published recall for de-identification of real free text is about 0.96 to 0.98. No system reaches 1.0, and this one does not.

10.6 Self-hosted model against cloud model

Six identical prompts were run through both tiers of the deployed gateway on 23 July 2026: a summary, clinical reasoning, data extraction, a plain-language rewrite, a dose calculation and a policy question.

A grader that did not know which model wrote which answer scored them. The two material errors were both from the self-hosted model: a summary with claims not in the source, and a statement that treated encryption as a de-identification method. Both models calculated the dose correctly.

10.7 Review and live checks

Each change to detection is reviewed by a second reviewer who writes independent probes and reruns the suite. Two such reviews rejected detector changes that raised benchmark scores by masking clinical content. A change that moves any adversarial result is rejected, and the pass thresholds in the tests are never lowered.

Each deployment is checked against the running service: a C1 request on a tenant with a Tier 2 ceiling must still be served on Tier 1 with nothing leaving.

11. Limits

Detection is incomplete. Section 10.3 gives the measured gap. The classifier uses the same detectors, so a missed identifier can also lower the class. The control for regulated data is to declare C1.

The exposed case is specific. It is a request with no declared class, from a tenant allowed to use Tier 2, containing an identifier the detectors miss. That identifier reaches Amazon Bedrock unmasked, under the AWS Business Associate Addendum.

C1 is the slowest path. The self-hosted model reads a prompt at about 17 tokens per second on CPU. A 525-token prompt with no cached prefix took 31 seconds to read and 41 seconds in total. Later turns that share the prefix took 1 to 5 seconds. The most sensitive data therefore gets the slowest service, and no setting changes that without giving up the network isolation. Faster hardware for Tier 1 is a cost decision that has not been taken.

The self-hosted model is weaker. Section 10.6 gives the measured difference.

Restoration is best effort. Section 6.3 explains when a stand-in may remain in the answer.

Some identifier types are not covered. License plates, biometric data and images are not detected. Several types are covered only when labelled or coded. Section 5.4 lists them.

Token counts and costs are estimates. The gateway estimates tokens at four characters each. The figures are for reporting and are not billing-grade.

Rate limiting is per process. It is correct for the current single-instance deployment and would need a shared store before scaling out.

Tier 3 and request splitting are not in production. They are described here as built and tested, not as operating.

No independent audit has been performed. There is no SOC 2 report, no ISO certificate and no third-party penetration test. Written incident response, disaster recovery and workforce policies are planned and not yet in place. Public benchmark results are evidence about public corpora and are a proxy for performance on a customer’s own text.

12. Standards

Protostar AI holds no certification. The gateway is designed to the standards below, and each has a written mapping from its requirements to the controls in this paper, with gaps marked.

13. Conclusion

The gateway makes one strong claim and several measured ones. The strong claim is that regulated data declared as C1 does not leave a network that has no route out. It rests on a three-line routing table, a network without a default route, and a test that passes with detection switched off.

The measured claims concern detection. It stopped every identifier in the committed adversarial set and about half of the hardest set. That is why detection is the second layer and declaration is the first.

For a workload that handles regulated data, the practice that follows is short: declare C1, keep the conversation prefix stable so the self-hosted model stays fast, and read the audit chain.

Protostar AI, LLC. For more information, contact hello@protostarai.com.