AI glossary · Beginner · Multimodal
Multimodal
Multimodal describes the ability of AI to understand or generate several kinds of information together, such as images, sound and video, not just text.
About this term
- Level
- Beginner · Core ideas for first-time readers
- Category
- Basics
- Fields
- Multimodal
- Added
- 2026-10-07
How it works
A multimodal model processes information such as images or sound by converting it into a form that can be handled alongside text. For example, one approach is to split a photo into small pieces, represent them as numbers, and feed them into the model together with text tokens. This lets the model answer questions like "What's in this photo?"
Why it matters
Real-world information is not made up of text alone. Many tasks, such as reading documents in photos, interpreting graphs, voice conversations and summarizing videos, require multimodal capabilities. It is also a core technology in fields such as robotics and autonomous driving, where systems must see their surroundings and make decisions.
Good to know
Multimodal models can also misinterpret images or sound and produce hallucinations. In addition, the types of input and output supported differ by model. Some models can read images but cannot create them, so it is a good idea to check exactly what "multimodal" means in each case.
Example
Google's 'EmbeddingGemma 2,' covered on this site, is an embedding model that places text, images, audio and video together in a single space. Models like this enable search across different kinds of information, such as searching with text to find related photos or videos.