What Is Multimodal Markup and How Do You Implement It in 2026?

Why Multimodal Markup matters for AI visibility

There is a real difference between a model looking at an image and a crawler deciding what your page says. Current frontier models are genuinely multimodal and can read a chart when the image is handed to them in a prompt. The pipeline that builds a citation is a different operation: it fetches HTML, strips it to text or markdown, chunks it, and embeds it. In that pipeline an <img> collapses to its alt attribute and nothing else. Google says the same thing about its own indexing in plainer language: the image SEO documentation, last updated 2 March 2026, calls alt text "the most important attribute when it comes to providing more metadata for an image" and adds that Google "extracts information about the subject matter of the image from the content of the page, including captions and image titles".

That gives you a practical hierarchy, and it is worth being blunt about which layer does what. Alt text is the only thing every consumer sees, because it is the fallback baked into HTML itself, so it has to state what the image shows. The figcaption is ordinary body text: it stays next to the image in the reading order, it survives chunking with the image reference attached, and it is therefore the string most likely to be quoted back verbatim, so put the source, the date, the units and the sample size there. Surrounding context is what a chunker bundles with the image and what Google says it uses to work out subject matter, which is why an image dropped into an unrelated section earns nothing. Structured data does not usually become the sentence a model cites: ImageObject and VideoObject drive eligibility and entity resolution, the licensable badge, video rich results and key moments. Ship all four, but do not expect a schema description to rescue an image whose visible page says nothing about it.

Where the spec lives

Four primary sources and two vendor documents cover everything on this page. The vendor docs are the ones that change, so check their last-updated stamps before you copy anything.

• HTML Living Standard: the figure element - the normative content model. A figure is "flow content, optionally with a caption, that is self-contained", and the figcaption must be the first or last child, never in the middle.
• W3C WAI: Complex images - the two-part text alternative pattern for charts, diagrams and maps: a short identifier plus a full textual representation of the data. Reach for this whenever alt text alone cannot carry the numbers.
• WebVTT: The Web Video Text Tracks Format - Candidate Recommendation Draft of 20 May 2026. The file must start with the string WEBVTT, and cue timings use the format 00:11.000 followed by an arrow and 00:13.000.
• schema.org VideoObject - where transcript, caption and embeddedTextCaption are defined. Note that these are schema.org properties, not Google features, so read this alongside the Google page below.
• Google Search Central: Video structured data - the only three required properties are name, thumbnailUrl and uploadDate. Google recommends contentUrl over embedUrl, and key moments need a video of at least 30 seconds.
• Google Search Central: Image SEO best practices - alt text, descriptive filenames, placement near relevant text, and the supported formats list (BMP, GIF, JPEG, PNG, WebP, SVG and AVIF).

Implementation guidelines

These are the failures that show up on real sites once the markup is in place.

1. Alt describes, caption contextualises, and they must not be the same string. If the alt and the <figcaption> carry identical text, a screen reader announces it twice and a text extractor recovers one fact where it could have had two. Alt says what the image shows, the caption says where it came from and what it means.
2. An empty alt is a decision, a missing alt is a bug. Write alt="" on decorative images so they are explicitly skipped, and never omit the attribute. Glippy, browsers and assistive technology all treat an absent attribute differently from a deliberately empty one.
3. The transcript has to be in the HTML source, not fetched on click. A <details> element works because its contents are in the served markup whether or not it is open. A transcript that a script loads after a user interaction is invisible to every crawler and to most agent browsers.
4. Serve WebVTT same-origin, with the right MIME type. The src of a <track> must be same-origin unless the parent media element carries crossorigin, and the file must be served as text/vtt starting with the literal string WEBVTT. Get either wrong and the track fails silently with no console error in some browsers.
5. role="img" on an SVG hides everything inside it. Once you set that role, the child <text> nodes stop being exposed to assistive technology, so the <desc> has to carry every number and label you care about. Do not assume the visible labels are doing the work.
6. Structured data must agree with the visible page. The contentUrl on an ImageObject has to be the same file the <img> loads, and Google requires the marked-up video to be present and playable on the page. Mismatched or stale media URLs are the most common reason a video rich result never appears.
7. Audit the extracted text, not the rendered page. Run the URL through an HTML-to-markdown converter, or through Glippy, and read what is left. If a chart contributes nothing to that output, it will contribute nothing to a citation, no matter how good it looks in a browser.

Do this, not that

How Glippy checks this

Glippy scores this as category 16, Multimodal Content, at a category weight of 0.5, and it runs six sub-checks over the rendered DOM and the parsed JSON-LD. Image Alt Quality (25 points) counts an alt as generic if it is under five characters, is the bare word image, photo, picture or img, matches a pattern like img_12, or ends in a file extension, and it passes only when 80 percent or more of the images clear that bar. Figure Captions (20 points) is the ratio of figure figcaption to figure. Video Accessibility (20 points) passes if it finds a <track> with kind="captions" or kind="subtitles" inside a <video>, or any element whose class or id contains transcript, or a .video-description, and it counts YouTube, Vimeo and generic player iframes as video. SVG Descriptions (15 points) counts an SVG as described if it has a <title> or <desc> child, an aria-label, or role="img". Image Schema (10 points) walks the entire JSON-LD graph looking for an ImageObject at any depth, including one nested inside an Article or Product. Media Variety (10 points) rewards three or more of images, video, SVG, audio and canvas on the same page.

Two results confuse people. If a page has content images but no <figure> element anywhere, Figure Captions returns an informational result worth zero of its twenty points, which is the largest silent loss in the category; before it decides that, Glippy filters out decorative images using empty alt, role="presentation", aria-hidden, class hints such as icon, logo, avatar and banner, an ancestor of aside, header, nav, footer or button, and any declared dimension of 100 pixels or less. And SVG Descriptions is deliberately lenient: a bare role="img" passes it, so a green tick there does not mean a model can read your diagram. Add the <title> and <desc> anyway. Full per-check detail is on the Multimodal Content Checker page.

Reviewed against the primary sources on 27 August 2026. These standards move quickly, so check the linked specs before you ship.