Method, accuracy and limits

This page exists so that you can judge how far to trust the text. Nothing here is hidden, and the numbers are measurements rather than estimates.

What this is

Every page of the Afghanistan Digital Library — 69,624 pages across 579 volumes of books and periodicals printed in Afghanistan between roughly 1871 and the 1930s — has been read by a machine and indexed for search. Until now the collection was searchable only by title. The scans and the collection are the work of New York University Libraries; the transcription is not.

How the text was made

Each page was rendered at 300 dpi and read once by a vision-language model (gemini-3.1-pro), prompted to preserve the original orthography, to keep the reading order of multi-column pages, and to mark unclear words rather than guess silently. No hand correction has been applied. The whole collection was read for about $290 of donated cloud credit.

Several plausible refinements were tested against ground truth and rejected because they did not help: cropping pages into detected regions before reading, a second fine-tuned reader, and an ensemble that reconciled two independent readings. Each is documented below.

How accurate it is

Accuracy was measured two ways, and the difference between them matters.

Against a modern printed edition of the same articles, the character error rate is about 32%. That number is misleading on its own: a modern editor normalises spelling, adds punctuation and regularises archaic forms, so much of that 32% is the distance between a 1911 lithograph and a 2008 reprint, not a misreading.

Against passages transcribed by a scholar directly from the page, the character error rate is about 7% — roughly 93 characters correct in 100. That is the honest figure for reading error. It rests on three passages, so treat it as an indication rather than a settled result.

Put plainly: the text is good enough to find a page and to read what it is about. It is not good enough to quote. Read the image.

How well search works

Search was measured the way it is actually used: take a phrase from a printed edition — clean, modern spelling — and see whether the corresponding page comes back.

Searching by subject — matching meaning rather than wording — was measured too, and is markedly worse at this particular job: 55% for a sentence, 24% for a short phrase. That is expected. Finding the page that contains a known sentence is a matching problem, and compressing a whole page to a single vector discards the exact wording that identifies it. Subject search is offered anyway, because it is the only way to browse a theme rather than chase a quotation, and no measurement here speaks to how well it does that.

So a remembered sentence usually finds its page; three or four words often will not. The index is built on character trigrams rather than whole words, which is what allows a misread word to still match, and both the text and your query are normalised for the orthographic variation that otherwise defeats Perso-Arabic search — ی/ي, ک/ك, tatweel, vowel marks and digit forms.

What is certainly wrong with it

• Reading order on dense multi-column pages is unverified. Text may be stitched together across columns in the wrong sequence.
• 72 pages returned nothing at all and are blank in the index.
• Nobody has checked any of this by hand. The accuracy figure comes from three passages of one newspaper.
• Dates, issue numbers and titles are not yet extracted, so you cannot filter by year. Volume identifiers are NYU's.
• The measurement above comes from one paper (Anis, 1947–50) and one period. Older lithographs are harder and are probably read less well.

Corrections

Every page accepts a suggested reading. Suggestions are reviewed before they appear, and are credited to whoever made them. The machine transcription is never overwritten — a correction sits beside it, so the record of what the machine read stays intact.

Private notes are exactly that: stored against your account, shown to nobody else, and never reviewed by us.

Rights

NYU states that the works are in the public domain and that the images "may be freely reproduced, distributed and transmitted by anyone for any purpose, commercial or non-commercial." The transcription produced here is released on the same terms. Please credit New York University Libraries for the scans.

What was actually done

In sequence, and with the cost of each, because the numbers are the interesting part:

1. Read the collection. All 579 volumes fetched from NYU, rendered at 300 dpi, and read page by page by a vision-language model. 67,909 pages in the batch run plus a previously-transcribed volume, for $288 of donated cloud credit.
2. Built ground truth without hand transcription. A modern printed edition of Tarzi's collected articles was re-read, normalised, and aligned to the original lithograph pages, with a model adjudicating each candidate pair. That produced 102 verified image-to-text pairs — the first aligned gold set for lithographed Kabul nasta'liq, built for the cost of the compute.
3. Measured everything against it. Four readers, two thinking levels, whole-page versus region-cropped, single-pass versus ensemble, and text-only linguistic correction at three levels of aggression.
4. Rejected most of it. Cropping pages into detected regions: no gain. A second fine-tuned reader: no gain. Ensemble adjudication: a real but tiny gain that cost more than the read itself. Post-hoc "AI cleanup": improved the score while rewriting 3–5% of already-correct text, which for a primary source is damage, not improvement.
5. Cut the cost 31×. Turning the model's reasoning budget down and switching to batch inference took the full-collection read from a projected $9,030 to $288, with no measurable loss of accuracy.
6. Indexed and published. A character-trigram index over normalised text, embeddings for subject search, page images, and this site.

The through-line: nearly every intervention that sounded like it should help did not, and the ones that mattered were found by measuring rather than by reasoning about it.

What should happen next

• Extract issue metadata. Dates, issue numbers and titles are printed on the pages but not yet pulled out. Doing so would allow filtering by year and paper, which is the single most-requested thing a periodicals archive can offer.
• More human ground truth. The 7% accuracy figure rests on three passages of one newspaper from the 1940s. Passages from the 1911–18 lithographs would tell us whether the older, harder printing reads as well.
• Verify reading order on multi-column pages. Currently unmeasured, and the most likely place for silent errors.
• Release the gold set. The 102 aligned pairs are useful to anyone working on Perso-Arabic OCR, and the alignment method transfers to any corpus with a modern reprint.
• Corrections at scale. Every suggested reading makes the next measurement better. Enough of them would allow a properly supervised model.

Taking the data, and the code

The full corpus is one JSON Lines record per page — volume, page number, text — from a single link, or a single volume from its page in the reader. No account, no rate limit, no attribution requirement beyond crediting NYU for the scans.

The pipeline behind all of this — fetching, rendering, reading, the alignment that built the ground truth, every benchmark harness, and this site — is open source on GitHub, released on the same terms as the scans.

Citing it

Afghanistan Digital Library, adl0616, p. 575. Digitised by New York University Libraries. Transcription machine-generated and unverified, retrieved YYYY-MM-DD.

If a passage matters to your argument, cite the image and transcribe it yourself.

To cite the project as a whole:

German, M. Afghan Press Archive: a searchable transcription of the Afghanistan Digital Library. Steps Ventures, 2026. afghanpress.org

← back to search · the history behind these papers →