# OAPEN/DOAB projects

 
# Using AI and Crowdsourcing for Accessible Alt-text


## Background

Images in web pages and books can cause accessibility issues for the blind and reading-disabled. The most important mechanism to mitigate this is to provide "alt text" in an "alt" attribute to allow screen reader software to describe the images.

Project Gutenberg (PG) makes available on its website over 70,000 public domain books that have been transcribed into digital files by processes that use OCR, multi-pass proofreading, and html coding, most of which is done by Distributed Proofreaders (DP), a partner organization. Creation of alt-text descriptions of the accompanying images has lagged significantly, as this process requires special skills. A recent census of the corpus has found that over 486,000 images in the PG catalog have no alt text descriptions. In some cases, this is appropriate, as adjacent captions are provided, but the majority of the images are effectively hidden from blind and reading-disabled users.

With recent advances in machine vision and artificial intelligence, it seems timely to begin creating processes for alt text description analogous to the successful OCR/proofreading processes at DP. Large language models (LLM) such as ChatGPT have already been trained on the Project Gutenberg corpus, so they likely have the data needed to contextually describe the images contained in the text.


## Possible Solution

- Emulate DP OCR/Proofreading workflow with  AI/Copyediting/Approval workflow
- Consider context of the image.
- Keep alt-text database separate to allow for tech improvements

Here's an example from a real book:

- [An image in context](https://gutenberg.org/cache/epub/40078/pg40078-images.html#ILL_008)
- [What an AI thinks of it](ai_for_alt_text.png)

## Objectives

How well can existing Machine Vision/AI systems do in providing useful alt text for images in PG texts? How might a web application that allows volunteer editors to proofcheck existing or AI suggested descriptions best integrate with AI capabilities?

To address these questions the team will:

- Evaluate AI platforms and feasibility
- Develop accessibility guidelines for alt-text
- Build an Alt-text Editing/Approval Environment that interacts with the chosen AI


### Advisors

- Eric Hellman, Free Ebook Foundation

### Associated Organizations

- [Project Gutenberg](https://gutenberg.org)
- [Distributed Proofreaders](https://pgdp.net)

### Proposed Team

(fill this in!)

-
-
-
-
-

### More about the team 

(fill this in!)

### Resources

- [Alt-texts: The Ultimate Guide](https://axesslab.com/alt-texts/)
- [Tips and Tricks](https://www.w3.org/WAI/tutorials/images/tips/)
- [What is alt text?](https://moz.com/learn/seo/alt-text)