Esc

Multimodal

A model that takes in or produces more than one kind of data, such as text with images, audio or video.

A multimodal model encodes non-text input into the same space as text, so a picture and a sentence about it can be reasoned over together. You can paste a screenshot, a chart, a PDF page or a video and ask about it in plain language.

Support varies a lot by provider and changes often. Image input is common now. Audio and video input are not universal, and image output usually comes from a separate model. Look up what your exact model version supports and do not assume two vendors match.

Where this comes up

Prompts, configs and tutorials in the library that touch this term.