BASALT · JOURNAL
What a PDF still carries after you think you cleaned it
A PDF is a container, not a page. Alongside the marks a reader sees, it holds structured objects that describe where the file came from, what it used to say, and what is deliberately not being displayed. Hidden data in PDF files survives most cleanup attempts because nothing in a viewer's interface acknowledges that it exists.
This is an inventory of what stays behind, in rough order of how often it causes problems.
Metadata in two places that disagree
Every PDF can carry a document information dictionary with /Author, /Title, /Subject, /Keywords, /Creator, and /Producer. /Author is usually the account name of whoever created the file. /Title is often the original filename or a heading from the source document, which can be a matter name you did not intend to disclose.
Separately, there is the XMP packet, an XML block embedded in the file. XMP typically carries a document identifier and an instance identifier, which link a file to its ancestors and to every derived copy. It often includes a history of the applications that touched the document and when.
The two can disagree. Editing the info dictionary in one tool while leaving XMP untouched is a common half-measure, and extraction tools read both. Documents where the visible redactions held and the metadata named the author are a recurring pattern in public releases.
Earlier versions of the same file
PDF supports incremental updates. When a file is edited, a new set of objects and a fresh cross-reference section can be appended to the end while every original byte stays in place. Viewers read the newest cross-reference table and show you the current state.
This means a file can contain three or four earlier versions of itself. Walking back through the update chain is mechanical and documented, and it reconstructs earlier states of any page.
If a redaction was saved as an incremental update, the pre-redaction page is physically present in the file you send. No amount of correct content stream editing in the final revision helps, because the earlier revision is right there in the same bytes. A redacted file has to be written as a single generation with prior revisions discarded. You can check for this yourself by counting %%EOF markers, as described in how to check whether a PDF was really redacted.
OCR text layers under scans
A scanned document that has been made searchable carries an invisible text layer: real text drawn in rendering mode 3, positioned to line up with the picture of the words. It is why you can search a scan.
It is also a second, independent copy of everything on the page. Destroying pixels in the image without removing the corresponding glyphs from the text layer leaves the words fully extractable. The reverse also happens: removing the text layer while leaving the image means the picture of the name is still there for anyone to read.
Both copies have to be handled, and they are different operations on different objects.
Optional content groups
Optional content groups are the layers feature. Content can be assigned to a group whose visibility is controlled by a dictionary, and a group that is off renders nowhere.
Off is not gone. The content stream operators are present and the text extracts normally. Documents assembled from templates, CAD exports, and marketing files routinely carry layers nobody has ever looked at, containing draft language, internal annotations, or an earlier version of a figure.
Nothing in a normal viewing session tells you a document has layers, let alone what is on them.
Annotations, form fields, and accessibility text
Annotation objects carry a /Contents string and an appearance stream. Sticky notes, highlights with comments, and free text annotations all hold readable text that never appears in the page content and often survives a flattening step.
Form field values live in the field dictionary, separate from the widget's appearance. A field can display as blank or be painted over and still hold its value. Flattening forms is a distinct operation from flattening annotations, and tools differ on which they do.
/ActualText and /Alt store plain text equivalents of marked content for accessibility. A redaction that removes glyphs from the visible run and leaves /ActualText in place has published the words to every screen reader and extraction tool while looking correct on screen. This is one of several reasons a black box over text is not a redaction.
Embedded attachments
A PDF can carry arbitrary embedded files. Portfolios make this visible. A single embedded file specification does not.
The attachment is a complete file: a spreadsheet with the underlying figures, a source document, an email. It is not paginated, so it never comes up in page-by-page review, and it is not text on a page, so text extraction of the document body will not surface it.
Thumbnails and cached appearances
Pages can carry embedded thumbnail images. If a thumbnail was generated before an edit and not regenerated afterward, it is a small picture of the earlier state of the page. Low resolution, and often perfectly legible for a name or a number.
Appearance streams for annotations and form fields are also cached renderings. An appearance stream generated before a value changed can show the old value.
Structure that outlives the content
Bookmarks, named destinations, the document outline, and the structure tree all reference content by name or by position. A bookmark reading "Exhibit C: Patient records, Maria Alvarez" is plain text in the outline dictionary regardless of what happened to page fourteen.
The structure tree, used for tagging and accessibility, holds a parallel description of the document's logical content and can retain text and ordering information about material that was removed from the page.
Fonts and object streams
Embedded font subsets record which glyphs a document uses. A subset containing glyphs that appear nowhere on the visible pages is weak evidence about removed content, and in a small alphabet it can be more than weak.
Object streams compress many objects into one stream, which means a naive grep of a compressed PDF finds nothing while the strings are present. Decompressing with qpdf --object-streams=disable is what makes them visible. Absence of a hit in a compressed file is not evidence of anything.
What actually clears it
Cleaning this up is not one operation. Metadata and XMP have to be stripped rather than blanked. Attachments have to be removed. Optional content groups have to be resolved or deleted, not just switched off. Thumbnails and cached appearance streams have to be discarded and regenerated. Annotations and form fields have to be flattened with their values removed, not just their widgets painted. OCR layers have to be edited alongside the image data.
And the result has to be written as one generation, because every one of those removals is meaningless if the previous revision is still appended in the same file. If you are working through this on a Mac, how to redact a PDF so the text is actually gone covers the content stream side.
Frequently asked questions
What hidden data does a PDF contain?
A PDF can contain document information metadata, an XMP packet with edit history and document identifiers, earlier revisions retained by incremental updates, invisible OCR text layers under scans, switched-off optional content groups, annotation and form field values, embedded file attachments, cached page thumbnails, bookmarks and structure tree text, and font subset information. None of it is visible in a normal viewing session.
Does a PDF contain earlier versions of itself?
It can. PDF's incremental update mechanism appends a new revision to the end of the file while leaving every original byte in place, so a file edited several times may contain several earlier states of each page. Walking back through the cross-reference chain reconstructs them. Counting %%EOF markers reveals how many revisions a file holds.
Can a PDF reveal who wrote it?
Yes. The /Author field in the document information dictionary usually holds the account name of whoever created the file, and the XMP packet often adds a creating application, an edit history, and identifiers linking the file to earlier versions and to other copies. Original file paths in metadata can also expose folder names and matter names.
Does removing metadata from a PDF remove all of it?
Not necessarily. Metadata lives in two independent places, the information dictionary and the XMP packet, and many tools clear one while leaving the other. Blanking a field also differs from removing it, and identifiers can persist. Verify with exiftool -a -u -g1, which reports both stores separately.
Are embedded attachments in a PDF visible during page review?
No. An embedded file specification carries a complete file inside the PDF without adding any page, so page-by-page review never encounters it and text extraction of the document body does not surface it. List attachments explicitly with pdfdetach -list. An attached spreadsheet inside a produced PDF is a full disclosure.
What is an invisible OCR text layer and why does it matter?
An OCR text layer is real text drawn in rendering mode 3, positioned over the picture of the words so a scanned document becomes searchable. It matters because it is a second, independent copy of everything on the page. Destroying image pixels without editing the text layer leaves the words fully extractable, and vice versa.
Do hidden layers in a PDF still contain readable text?
Yes. Optional content groups control visibility only, so content assigned to a group that is switched off renders nowhere while its operators remain in the content stream and extract normally. Documents built from templates or CAD exports often carry layers holding draft language or earlier figures that nobody has reviewed.
Does printing a PDF to a new PDF remove hidden data?
Only some of it. Printing rasterizes or regenerates the page, which destroys the visible text layer and any content that was not being drawn, but metadata is commonly regenerated with the same author information, and the process costs you searchable text, tagging, bookmarks, and form structure across the entire document.
Doing it in Basalt
Basalt treats the container as part of the redaction rather than a separate cleanup task. It strips metadata, XMP, attachments, hidden layers, and stale thumbnails, handles OCR text layers alongside the image data, destroys and re-encodes image pixels under a mark, and writes the file as one generation so earlier versions cannot be recovered.
An independent parser re-opens the written file and proves the result before anything is saved, and if it cannot prove it, nothing is written. Every redaction ships with an Ed25519-signed certificate verifiable with shasum and openssl. There is also an inspect tool for looking at what a file actually contains before you decide what to remove.
The engine runs as a separate process with no network entitlement, enforced by macOS at the code signature level. No account, no telemetry, no cloud. It is $29 once for up to three Macs, free for 24 hours.
Redaction that proves itself
Basalt destroys the content you mark, then re-opens the file it wrote and proves the content is gone before it saves anything. Your documents never leave your Mac.