The Four Families of Generative AI Models
The phrase "AI model" has become so overloaded that it no longer communicates anything useful on its own. When a content writer says they use AI, they probably mean a text generation system. When a graphic designer says the same, they likely mean something that converts prompts into images. When a game studio says it, they might be referring to voice synthesis, motion generation, or something built in-house that doesn't fit a consumer category at all.
Generative AI (the branch of machine learning that produces new content rather than simply classifying or predicting) divides into four families based on the type of output each family produces: text, images, audio, and video. Each family uses different architectures, requires different training data, serves different workflows, and is sold through different products and pricing models. Understanding the map before you start subscribing to tools is the single most effective way to avoid wasting money and time.
Large Language Models: What They Do and What They Can't
Large language models generate text. That description undersells them, because "text" covers an enormous range: prose, code, structured data like JSON, reasoning chains, translations, summaries, and conversational responses. An LLM's core skill is predicting the most contextually appropriate continuation of a sequence, which turns out to be a surprisingly powerful general-purpose capability.
The directory of large language models catalogues dozens of options across different provider categories, from frontier closed-source models to open-weight alternatives you can run on your own hardware. The differences between them matter: context window size determines how much text a model can read and respond to in a single exchange, training data cutoffs determine how current the model's knowledge is, and specialised fine-tuning shapes how the model performs on domain-specific tasks like legal drafting or code generation.
What LLMs cannot do reliably is generate images, produce audio, or create video. Some multimodal systems accept images as input, but the output remains text. When someone is frustrated that their text AI "can't draw anything," they have encountered this boundary. They need a different model family entirely.
Image Generation Models: From Prompt to Pixel
Image generation models take a text description, and sometimes a reference image, and produce a visual output. The technical architectures underlying these systems have evolved rapidly: diffusion models, which gradually refine a noisy image toward a coherent output, now dominate the space and underlie most of the major consumer products available today.
The practical differences between image models involve aesthetic style, prompt adherence, resolution capabilities, and the granularity of control they offer. Some models produce images with a distinctive house style that users either love or find limiting. Others are more neutral and respond more literally to what the prompt specifies. The catalogued image generation models breaks down these differences with side-by-side specifications.
One distinction that beginners frequently miss is between consumer-facing products (which wrap a model behind a subscription interface) and the underlying models themselves (which can sometimes be accessed directly via API or run locally). Midjourney is a product; Flux is a model family available through multiple interfaces. Understanding which you are using matters for questions of portability, pricing, and control.
Audio Models: Text-to-Speech, Music, and Voice Cloning
Audio AI splits into three distinct capability areas, each served by different products with minimal overlap.
Text-to-speech systems convert written text into spoken audio. Modern versions have closed the quality gap with human narration substantially, producing natural-sounding speech with appropriate intonation and pacing. The primary differentiators are voice selection, cloning capability (the ability to replicate a specific person's voice from a short sample), and language support.
Music generation models produce original audio tracks from text prompts describing genre, mood, instrumentation, or tempo. The outputs range from serviceable background music to compositions with genuine structural coherence, depending on the model and the specificity of the prompt.
Voice cloning specifically deserves attention because it carries distinct legal and ethical implications that text-to-speech does not. Generating speech in a real person's voice without their consent is regulated or prohibited under an expanding set of laws in multiple jurisdictions. Creators using audio AI need to understand which capability they are reaching for and what the associated obligations are.
Video Generation: The Newest Frontier
Video generation is the youngest of the four families and still the most constrained. Current systems can produce clips of a few seconds to a couple of minutes from text prompts, with significant variation in quality, motion consistency, and adherence to the described content.
The practical state of the technology in 2026 is that short-form video generation is viable for specific applications — social content, product visualisations, animated explainers — but long-form narrative video is beyond current consumer tools. The most common failure modes involve temporal consistency (objects change shape or colour between frames), physics violations (liquids and cloth behave strangely), and face generation (human faces remain a persistent challenge).
For teams evaluating video AI, the question is not "is this ready?" in some general sense but "is it ready for my specific output type?" A thirty-second ambient loop for a website is a fundamentally different requirement than a two-minute brand film.
How Multimodal Models Blend All Four
Multimodal models are systems that can accept and produce multiple modality types in a single interaction. A model that can read an image you upload and write a description, or generate an image from a text prompt, is multimodal. Some frontier systems now accept text, image, audio, and video as inputs simultaneously.
The important caveat is that multimodal input capability does not imply multimodal output capability. A model that can see images may still only produce text. A model that can hear audio may respond only in writing. When evaluating a multimodal system, map each input modality to its corresponding output before assuming the system does what you want.
Picking the Right Model Type for Your Project
The framework for choosing between model families is straightforward: start with the output you need. If you need written content or code, you need an LLM. If you need a visual asset, you need an image generation model. If you need narration or music, you need an audio model. If you need moving footage, you need a video generation model.
The confusion most people encounter arises from approaching the question from the product side rather than the output side. They sign up for whatever has the most coverage that week and then discover it does not produce the thing they actually needed. Model families are fundamentally different technologies built for different jobs. Treating them as interchangeable is like buying a saw when you need a drill because both are power tools.
Once you have identified the right family, the secondary decisions — which specific model, which pricing tier, which interface — become much easier to evaluate because you are comparing like with like.
