Plan · written 22 August 2026

Image-to-text for Ethiopian languages

Some sites block every programmatic fetch but render fine in a browser. For those, the agent takes screenshots, and the text on them has to be read back out. For English any major model reads text straight off an image. For Fidel nobody has measured how well they do. This plan tests that, and builds the training data to improve a model ourselves if the answer is not good enough.

The problem this answers

Several of the Ethiopian sources the project needs refuse programmatic clients: Addis Standard answers 403 to everything, including robots.txt; AllAfrica blocked the project's address after thirty requests; others serve feeds that are empty or years stale. The same pages open normally in a browser. The agent can drive a browser and take screenshots, so for those sites the capture is an image of the page, not its HTML.

None of the news sources found so far yields clean Amharic article text; the RSS summaries are useful but are not the articles. So for Amharic this is the main path to article text, not a fallback. From a screenshot of an English page, any major model reads the text directly. From a screenshot of an Amharic page in Fidel, it is not known how well any model reads the text, and the native speaker on the team reports that at least one major model struggles with Fidel even as typed input. So the chain for a blocked Amharic site is: browser screenshot, then Fidel text from the image, then English from the Fidel. The middle step is the unknown.

What we will do

1. Build the evaluation and training set from text we already hold, not from websites. Take clean Amharic text we already hold, build web pages that look like the news sites with that text in them, and screenshot those pages, across the fonts, sizes, line lengths and layouts the real sites use. The ground truth for every image is the text it was rendered from, which has no markup in it and needs no human reading. This can produce as many image-text pairs as needed at no cost.
2. Measure the off-the-shelf models on that set. Screenshot-to-text with the current models from the major labs, scored against the ground truth by word error rate. That gives a number per model for how well it reads Fidel from pixels, under controlled conditions. The native speaker's experience so far is with one model; the others may do better, and the decision is made on the numbers.
3. Decide from the number. If a model reads Fidel well enough, use it and stop. If none does, fine-tune a smaller image-to-text model on the rendered set, and measure again.
4. Only then test on real pages. Screenshots of the actual blocked sites are the last test, not the training set; they are few, and the errors left by then are few enough for the native speaker to read by hand.
5. Extend to other Ethiopian languages that use Fidel (Tigrinya) the same way, from whatever clean text exists for them.

What counts as success

A measured word error rate on rendered Fidel for each model tried, and one model chosen on that basis. Then: screenshots of a blocked Amharic page produce Fidel text that the native speaker confirms is the page's text. The English translation of that text is a separate derivative with its own provenance, under the rules on rights and provenance of anything translated.

Why this way round

The obvious way, take screenshots of the real sites and check them against the sites' HTML, does not work here: the HTML on these sites has markup inside the text (BBC Amharic wraps quoted words in span elements constantly, the cause of problem 04) and page chrome inside the body (problem 13), so the HTML is not a clean reference, and extracting one from it is its own unsolved problem in this corpus. Rendering known text to images removes the reference problem entirely. The real sites come in at the end, as a check.

State

Planned, 22 August 2026. Not started. Owner: the native Amharic speaker who is also a data scientist, with the operator. Priority after the news feeds are flowing: the team's ruling on the same day was to start simple and not block on this.

the operator's and the reviewer's discussion, 2026-08-22 · problems 04, 11 and 13 · sources (the blocked hosts) · retrieved 2026-08-22

Published August 23, 2026 at 05:40 UTC. Last updated August 23, 2026 at 14:56 UTC. Times come from this page's revision history and can be checked against it.