BASALT · JOURNAL
De-identifying PDFs under HIPAA without leaving traces
De-identifying a PDF under HIPAA is two problems wearing one name. The first is deciding which information has to go, which is a regulatory judgment. The second is getting it out of the file so it cannot be recovered, which is a file format problem that most PDF tools handle badly.
This article is mostly about the second. It assumes you already know, or will decide with counsel or a qualified expert, what counts as an identifier in your data. Nothing here is legal advice, and whether a given document is de-identified is your determination, not a tool's.
The two methods, honestly summarized
The Privacy Rule provides two paths to de-identification.
Safe Harbor requires removing a specified set of 18 categories of identifiers of the individual and of relatives, employers, and household members, and additionally requires that the covered entity have no actual knowledge that the remaining information could be used alone or in combination to identify the individual. The categories cover the ones you would expect, including names, most geographic subdivisions smaller than a state, dates directly related to an individual other than year, telephone and fax numbers, email addresses, Social Security numbers, medical record and health plan numbers, account and certificate numbers, device identifiers, URLs and IP addresses, biometric identifiers, full-face photographs and comparable images, and a catch-all for any other unique identifying number, characteristic, or code. The exact wording and its qualifications matter, and you should work from the regulation text and HHS guidance rather than any summary, including this one.
Expert Determination is the alternative. A person with appropriate knowledge of and experience with generally accepted statistical and scientific principles and methods applies those methods, determines that the risk is very small that the information could be used alone or in combination with other reasonably available information to identify an individual, and documents the methods and results of that analysis.
Safe Harbor is mechanical, which makes it easy to audit and often over-removes. Expert Determination is contextual, which makes it more useful for research data and more work to defend. A PDF tool does not choose between them. It executes whichever removal you specify.
The distinction that matters for the rest of this article: both methods presume the identifiers are actually gone from the artifact you release. A document that displays a black box while the name sits in the content stream has satisfied neither.
What survives in a PDF after a careless redaction
PHI in a clinical PDF is rarely in one place. The same patient name can appear in the page text, in the document title, in a comment, in a form field, in an attached lab report, and in the invisible text layer produced when the scan was OCR'd. Removing one instance removes one instance.
The structures to account for:
Content stream operands. Text is painted by text-showing operators whose operands are character codes. Drawing a rectangle over them appends a fill operation and leaves the codes in place. Anything that extracts text reads the codes.
OCR text layers. A scanned chart run through OCR carries invisible text positioned under the image. Destroying the pixels in a region without destroying that text leaves the PHI fully searchable. This is a leading cause of failed medical record redactions and is covered in redacting scanned documents.
Metadata and XMP. The document information dictionary and the XMP packet regularly carry the original filename, which in health systems is very often the patient name, an MRN, or both, plus the authoring account and export timestamps. See removing PDF metadata on a Mac.
Annotations. Reviewer comments and sticky notes store their text in annotation objects independent of the page.
Form fields. Intake and consent forms store field values in the AcroForm dictionary separately from the rendered appearance. Flattening the appearance does not clear the value.
Embedded attachments. An attached CSV or DICOM-derived file inside the PDF is untouched by any page edit.
Optional content groups. A layer that is switched off is still present in the file.
Incremental update chains. PDFs can be saved by appending new objects and a new cross-reference table while leaving the old bytes in place. A viewer shows the newest state. A parser can reach the earlier one. Redacting and then saving incrementally produces a file that still contains the original object.
Hidden data in PDFs walks through each of these.
Images are PHI too
Full-face photographs and comparable images are named in the Safe Harbor list, and imaging in a PDF is not text. A black rectangle over a photograph is a drawing operation layered above an intact image XObject. The original image bytes are still embedded and can be extracted with any PDF tool.
Correct handling means editing the image itself: destroying the pixels in the marked region and re-encoding the image so that the bytes written to the file no longer contain the original samples. The same applies to a signature block, a wristband barcode, or a monitor display captured in a photograph. See redacting images inside a PDF.
The disclosure risk in the tool itself
Uploading a document containing PHI to a web-based PDF service is a disclosure to whoever runs that service. Whether that is permissible depends on whether the service is a business associate under a compliant agreement, and most consumer PDF sites are not. The upload happens before any redaction is applied, so the exposure is of the unredacted record.
The cleanest answer is to do the work on a machine you already control, with a tool that has no network path off the device. If nothing is transmitted, there is no processor and no agreement to negotiate for this step. Cloud PDF tools and the risk they carry covers the practicalities.
Verify before release
A tool reporting success is asserting its own correctness from its own data structures. The check that means something re-opens the saved file with an independent parser and looks for what should be gone.
At minimum: extract all text and search for the removed identifiers, then read the metadata, every annotation value, every form field value, and every embedded attachment and search those too. Do it on the file you are about to send, not on an intermediate. If any identifier appears, the file is not ready. How to verify a PDF redaction sets out the mechanics.
Recording a hash of the released file gives you a way to establish months later that the file under discussion is the file you produced.
Frequently asked questions
Is redaction enough for HIPAA de-identification?
Redaction is the mechanism, not the standard. De-identification under the Privacy Rule requires either removing the identifiers specified by the Safe Harbor method with no actual knowledge that the remainder could identify the individual, or an Expert Determination that the re-identification risk is very small. Redaction is how you carry out the removal, but whether the result meets either standard is your determination with counsel or a qualified expert.
How many identifiers does HIPAA Safe Harbor require removing?
Safe Harbor lists 18 categories of identifiers that must be removed, covering the individual and also relatives, employers, and household members, and it additionally requires no actual knowledge that the remaining information could identify the person. The categories include names, most geographic subdivisions smaller than a state, dates related to an individual other than year, contact details, various account and record numbers, biometric identifiers, and full-face images, along with a catch-all for other unique identifying codes. Work from the regulation text rather than any summary.
What is the difference between Safe Harbor and Expert Determination?
Safe Harbor is a mechanical rule: remove the listed categories of identifiers and confirm you have no actual knowledge that the remainder identifies anyone. Expert Determination instead relies on a person with appropriate statistical and scientific expertise concluding that the re-identification risk is very small and documenting the methods and results. Safe Harbor is easier to audit and tends to over-remove; Expert Determination preserves more analytic value but requires a defensible written analysis.
Does a black box over patient information satisfy HIPAA?
No. A drawn black rectangle is an added fill operation that changes only what is rendered, leaving the underlying text-showing operators and their character codes intact in the file. The patient information remains extractable by copy and paste or any command line parser, so it has not been removed from the record you disclosed.
Can I redact a scanned medical record safely?
Yes, but only if the process destroys both the image pixels within the marked region and the invisible OCR text layer positioned beneath them. Scanned charts are routinely made searchable by adding that text layer, and erasing pixels alone leaves the name or MRN fully readable to any extractor. Verify by extracting text from the finished file and searching for the identifiers.
Is it safe to use an online tool to redact PHI?
Uploading a document containing protected health information discloses it to whoever operates the service, which is generally permissible only if that operator is a business associate under a compliant agreement. Most consumer PDF websites are not. The upload also happens before any redaction, so what is exposed is the unredacted record. Local processing avoids the question.
Where does PHI hide in a PDF besides the visible text?
PHI commonly persists in document metadata and the XMP packet, particularly in the original filename, in annotation and comment values, in form field values stored separately from their rendered appearance, in embedded file attachments, in hidden optional content layers, in stale page thumbnails, in OCR text layers under scans, and in earlier document states left behind by incremental saves. Each stores text independently of the page you are looking at.
How do I document that a PDF was de-identified?
Record which regions were removed, on which pages, under which method, by whom, and on what date, and store a cryptographic hash of the released file alongside that record. The hash lets you demonstrate later that a specific file is the one you produced. What documentation your organization needs is set by your policies, your legal counsel, and, for Expert Determination, the expert's own written analysis.
Doing it in Basalt
Basalt removes text by excising it from the content stream glyph by glyph and re-emitting the surviving text in place, destroys and re-encodes image pixels under a mark along with any OCR text layer beneath them, and strips metadata, XMP, attachments, hidden layers, and stale thumbnails, writing one clean generation rather than an incremental append. Before a file is saved, a built-in verifier re-opens the written bytes with an independent parser and proves the removal, and if it cannot prove it, nothing is written at all. Each job can emit a signed certificate of redaction that anyone can check with shasum and openssl without installing Basalt. The engine runs as a separate process with no network entitlement, enforced by macOS at the code-signature level, so documents never leave the Mac and nothing is transmitted to any processor. Basalt removes content and proves the removal; whether the result is de-identified under the Privacy Rule remains your determination. It is at basaltformac.com.
Redaction that proves itself
Basalt destroys the content you mark, then re-opens the file it wrote and proves the content is gone before it saves anything. Your documents never leave your Mac.