Using AI to Boost Accessibility and SEO · The Phoenix Files

Using AI to Boost Accessibility and SEO

This article is about putting AI to work for us by making our websites more accessible and perform better with SEO. Fly is an excellent platform for running services that prioritize accessibility and SEO.

In the fast-paced world of digital content, the adage “a picture is worth a thousand words” remains as true as ever. This is the case for Sarah, a content manager at a popular online magazine. She is racing to finalize a feature article called “The Hidden Gems of Urban Street Art,” a visually rich showcase of Toronto’s vibrant but oft-overlooked urban art scene. As the deadline looms near, she faces a dilemma: how do we ensure that these compelling images are not only eye-catching but also accessible to all readers and at the same time optimized for SEO?

Here’s an example photo from Sarah’s feature article that needs a description for accessibility and SEO:

We’ve been tasked with solving this recurring need, the need for context-aware alt tag descriptions and caption text for images that accompany an article. “Context-aware” means that the description should relate to how the image is being used and not simply a description of the image. With these two versions of an image description, we improve both SEO (Search Engine Optimization) and the accessibility of our content.

Our job is to use AI to create decent alt tag and caption text on hundreds or thousands of images.

For the AI, we will leverage both OpenAI’s ChatGPT and Anthropic’s Claude LLMs (Large Language Models) through the Elixir LangChain library to perform AI analysis on images. The same process can be applied using other languages and tools.

Let’s first understand more about what makes high quality image description text so we know what we’re aiming for.

Context matters

How the image is being used matters. The usage provides additional context for how the images should be described. Let’s consider the following image when used in different contexts.

Description with no context: Elderly man sitting on a bench by a lakeside, reading a book with mountains in the background.

Description in the context of a story about a local author: René Favre reading beside a serene lake with misty mountains in the background.

Description in the context of a travel destination: Older man sitting on a bench overlooking Lake Lucerne with mountains in the background.

How the image is used changes how it should be described. This impacts what aspects of the photo should be called out as relevant or ignored entirely.

Okay, so the description needs to be in the context of our article. What are the current best practices around alt text and captions?

What is alt text?

Alt text, or alternative text, describes images in HTML code. It’s essential for two main reasons:

  1. Accessibility: Screen readers use alt text to convey image content to visually impaired users, ensuring that everyone can fully engage with the webpage.

  2. SEO: Search engines read alt text to understand and index images, enhancing webpage visibility in search results.

In a nutshell, alt text makes images accessible and boosts their discoverability, benefiting both users and search engines. When we make our content more accessible to users and to search engines, it benefits us.

What are captions?

The <figure> and <figcaption> are semantic HTML5 elements that work together to provide context and captions for images, diagrams, or other media within a webpage.
ARIA (Accessible Rich Internet Applications) tags are special attributes added to HTML elements to enhance web accessibility.

The Mozilla documentation encourages us to use semantic HTML elements over ARIA tags whenever possible.

If at all possible, you should use the appropriate semantic HTML elements to mark up a figure and its caption — <figure> and <figcaption>.

In HTML, it looks like this:

<figure>
  <img src="image.png" alt="put image description here" />
  <figcaption>Figure 1: The caption</figcaption>
</figure>

On the webpage, the default styling places the caption below the media, although this can be modified through CSS.

How can AI help?

AI can analyze an image and provide a text description. Recent model improvements from both OpenAI and Anthropic make this much easier and more powerful.

However, unless we want the AI to run amok and wax poetic on one image and do something completely different on the next, we need to provide the set of constraints and instructions that will get us what we want. In other words, prompt engineering!

To get consistent, well performing text, we need to understand a bit more about what makes for good alt text so the AI can give us exactly what we need.

This is a list of the relevant recommendations for alt text.

  1. Concise Description: Keep the ALT text concise and to the point. Aim for a simple sentence or a fragment that conveys the essential information about the image. Typically, 125 characters or less is advisable as some screen readers may truncate longer text.
  2. Content Relevance: Focus on the information that the image conveys and its context within the page. Describe what is relevant to the content and functionality of the site. If the image contains text, such as a logo, include the text in the ALT tag.
  3. Avoid Redundancy: Do not include phrases like “image of” or “graphic of,” as screen readers often provide this context to their users. This can be redundant and unnecessarily verbose.
  4. Context Matters: Tailor the ALT text depending on where and how the image is used. The same image might need different ALT text in different contexts.
  5. Localization: When dealing with multilingual sites, make sure to translate the ALT text appropriately, keeping the same considerations in mind as for the original language.
  6. Use Proper Grammar: Even though it’s brief, ensure your ALT text uses correct spelling, grammar, and punctuation. This improves clarity and the overall user experience.

AI can help with all of these points! This dramatically simplifies the task, especially when we need to provide descriptive text for many, many images.

As for the figcaption tag, we have more freedom with that and we’ll try to pull in more context from the article. It’ll be fun!

JSON Output

For each image, we want two pieces of generated content. To make this easy on ourselves, we instruct the LLM to output the two pieces of information in a JSON object.
Specifically, we instruct it to output in the following format:

{
  "alt": "generated alt text",
  "caption": "generation caption text"
}

Making the request

Everything is ready to make the request!

Now, we’ll submit the request to the server and review the response.

Final Thought

AI made our visual content more accessible to people using assistive technology and our image content will perform better in SEO.