Method, accuracy and limits

This page exists so that you can judge how far to trust the text. Nothing here is hidden, and the numbers are measurements rather than estimates.

What this is

Every page of the Afghanistan Digital Library — 69,624 pages across 580 volumes of books and periodicals printed in Afghanistan between 1873 and the 1960s — has been read by a machine and indexed for search. Until now the collection was searchable only by title. The scans and the collection are the work of New York University Libraries; the transcription is not.

How the text was made

Each page was rendered at 300 dpi and read once by a vision-language model (gemini-3.1-pro-preview), prompted to preserve the original orthography, to keep the reading order of multi-column pages, and to mark unclear words rather than guess silently. No hand correction has been applied. The whole collection was read for $288 of donated cloud credit.

Several plausible refinements were tested against ground truth and rejected because they did not help: cropping pages into detected regions before reading, a second fine-tuned reader, and an ensemble that reconciled two independent readings. Each is documented below.

How accurate it is

Start with the number to plan around. Measured on 102 pages sampled evenly across the difficulty range, in the configuration actually used to read the collection, the median character error rate is 0.36 — roughly two thirds of characters agreeing. The reference is the 2008 Farhadi typeset edition of Maqalat-e Tarzi, aligned page by page against the Siraj al-akhbar lithograph: the same author's prose, in modern type.

That number understates the reading, and it is worth knowing by how much and why. The 2008 editor normalises spelling and adds punctuation, so part of the gap is editorial distance rather than a misreading. A floor of about 0.19 was measured for that effect — but on a different pairing, Sa'di's Bustan against the Ganjoor canonical text, which is classical verse rather than journalism. Carrying it across is an analogy, not a measurement. Subtracting it entirely would imply roughly 83% of characters correct, and nobody has verified that.

You may see a figure of 93% quoted for this project. This site used to lead with it and should not have. It comes from three passages of Anis, a newspaper of the 1940s: later letterpress, later period, and the easiest printing in the collection. It says almost nothing about an 1880s Kabul lithograph, and three passages is an indication rather than a measurement.

The blunt version: no page-level human transcription of these lithographs exists anywhere, so this corpus has never been scored properly. If your institution holds even a few transcribed pages, that would fix the single biggest gap in this project.

Put plainly: the text is good enough to find a page and to read what it is about. It is not good enough to quote. Read the image.

How well search works

Search was measured the way it is actually used: take a phrase from a printed edition — clean, modern spelling — and see whether the corresponding page comes back.

Searching by subject — matching meaning rather than wording — was measured too, and is markedly worse at this particular job: 55% for a sentence, 24% for a short phrase. That is expected. Finding the page that contains a known sentence is a matching problem, and compressing a whole page to a single vector discards the exact wording that identifies it. Subject search is offered anyway, because it is the only way to browse a theme rather than chase a quotation, and no measurement here speaks to how well it does that.

So a remembered sentence usually finds its page; three or four words often will not. The index is built on character trigrams rather than whole words, which is what allows a misread word to still match, and both the text and your query are normalised for the orthographic variation that otherwise defeats Perso-Arabic search — ی/ي, ک/ك, tatweel, vowel marks and digit forms.

What is certainly wrong with it

• Reading order on dense multi-column pages is unverified. Text may be stitched together across columns in the wrong sequence.
• 4,033 pages of 69,624 (5.8%) came back as blank and carry no text. Most are genuinely blank leaves; how many are pages the reader simply failed on has not been measured, and the obvious test does not work here — these scans are dark, so blank leaves carry more ink than text pages and a brightness rule runs backwards.
• Nobody has corrected any of this by hand. No page in the corpus has been through a human reader. The 102 gold pages were used to measure the text, not to repair it.
• Issue numbers and per-page dates are not extracted. Volume-level titles, authors, publishers, dates and subjects come from NYU's own catalogue and drive browse, so you can filter by decade and subject, but not to the individual issue. Volume identifiers are NYU's. NYU serves a record for 579 of the 580 volumes: adl0277's record page is empty on their site, so that volume browses as untitled and undated.
• Dates are NYU's, and they are mixed calendars. The ADL prints the Hijri year and usually glosses it in brackets, so browse uses the Gregorian year where the record carries one and leaves the volume undated where it does not — 10 volumes. Nothing is converted between calendars here. The bulk of the collection is the 1920s (331 volumes); 79 are dated 1940 or later, which is later than this archive used to claim.
• The measurement above comes from one title and one period — Siraj al-akhbar, 1911–1918, against the 2008 Farhadi edition. It is a Kabul lithograph, which is the hard case, but a single one. How the corpus reads across other presses and decades is unmeasured.

Corrections

Every page accepts a suggested reading. Sign-in is by emailed link, so there is no password to make up: give an address, click the link it sends, and you are in.

Suggestions are reviewed before they appear and are credited to whoever made them, by name unless you ask otherwise. The machine transcription is never overwritten — a correction sits beside it, so the record of what the machine read stays intact. Corrections are released under CC0 so a library can redistribute them with the corpus.

Private notes are exactly that: stored against your account, shown to nobody else, and never reviewed by us. If you would rather not sign in at all, email mike@stepsventures.com with the volume and page number and it will be recorded against that page.

Rights

NYU states that the works are in the public domain and that the images "may be freely reproduced, distributed and transmitted by anyone for any purpose, commercial or non-commercial." The transcription produced here is released on the same terms. Please credit New York University Libraries for the scans.

What was actually done

In sequence, and with the cost of each, because the numbers are the interesting part:

1. Read the collection. All 580 volumes fetched from NYU, rendered at 300 dpi, and read page by page by a vision-language model. 67,909 pages in the batch run; the remaining 1,715 come from Siraj al-akhbar, transcribed earlier. $288 of donated cloud credit for the lot.
2. Built ground truth without hand transcription. A modern printed edition of Tarzi's collected articles was re-read, normalised, and aligned to the original lithograph pages, with a model adjudicating each candidate pair. That produced 102 verified image-to-text pairs — the first aligned gold set for lithographed Kabul nasta'liq, built for the cost of the compute.
3. Measured everything against it. Four readers, two thinking levels, whole-page versus region-cropped, single-pass versus ensemble, and text-only linguistic correction at three levels of aggression.
4. Rejected most of it. Cropping pages into detected regions: no gain. A second fine-tuned reader: no gain. Ensemble adjudication: a real but tiny gain that cost more than the read itself. Post-hoc "AI cleanup": improved the score while rewriting 3–5% of already-correct text, which for a primary source is damage, not improvement.
5. Cut the cost 31×. Turning the model's reasoning budget down and switching to batch inference took the full-collection read from a projected $9,030 to $288, with no measurable loss of accuracy.
6. Indexed and published. A character-trigram index over normalised text, embeddings for subject search, page images, and this site.

The through-line: nearly every intervention that sounded like it should help did not, and the ones that mattered were found by measuring rather than by reasoning about it.

What should happen next

• Extract issue metadata. Dates, issue numbers and titles are printed on the pages but not yet pulled out. Doing so would allow filtering by year and paper, which is the single most-requested thing a periodicals archive can offer.
• More human ground truth. The 7% accuracy figure rests on three passages of one newspaper from the 1940s. Passages from the 1911–18 lithographs would tell us whether the older, harder printing reads as well.
• Verify reading order on multi-column pages. Currently unmeasured, and the most likely place for silent errors.
• Release the gold set. The 102 aligned pairs are useful to anyone working on Perso-Arabic OCR, and the alignment method transfers to any corpus with a modern reprint.
• Corrections at scale. Every suggested reading makes the next measurement better. Enough of them would allow a properly supervised model.

Taking the data, and the code

The full corpus is one JSON Lines record per page — volume, page number, text — from a single link, or a single volume from its page in the reader. No account, no rate limit, no attribution requirement beyond crediting NYU for the scans.

The pipeline behind all of this — fetching, rendering, reading, the alignment that built the ground truth, every benchmark harness, and this site — is open source on GitHub, released on the same terms as the scans.

Citing it

Sirāj al-akhbār, page 575. Kābul, Maṭbaʻah-i Māshīn Khānah, 1911 - 1918. Afghanistan Digital Library, New York University Libraries. http://hdl.handle.net/2333.1/rn8pk0wv. Machine-transcribed text, Afghan Press Archive, https://afghanpress.org/book/adl0616/575.

Every page in the reader carries this citation with a copy button, built from NYU's own record, so you need not assemble it by hand. If a passage matters to your argument, cite the image and transcribe it yourself.

To cite the project as a whole:

German, M. Afghan Press Archive: a searchable transcription of the Afghanistan Digital Library. Steps Ventures, 2026. afghanpress.org

← back to search · the history behind these papers →