AdvancedModels & Architecture
ActTail
ActTail is a research method from 2026 that lets a language model skip its least important arithmetic, with a separate budget for each part, so it loses less quality.
AI Dictionary
Models & Architecture
Glossary of AI concepts, explained simply
34 concepts
Models & Architecture
34 concepts
Advanced
BeginnerModels & Architecture
DeepSeek
DeepSeek is a Chinese AI company that makes a chatbot and AI models that anyone can download, run on their own computers and use commercially.
AdvancedModels & Architecture
DeepStack injection
DeepStack injection feeds an image into an AI model at several layers instead of only at the start, so even a small model can read details such as tables in documents.
AdvancedModels & Architecture
FlashAttention
FlashAttention makes AI language models compute faster with less memory, by making the graphics card use its slow main memory far less often.
BeginnerModels & Architecture
GPT
GPT stands for generative pre-trained transformer: OpenAI's family of AI models behind ChatGPT, which write text by predicting which words come next.
AdvancedModels & Architecture
GRPO (group relative policy optimization)
GRPO is a training method in which an AI model answers the same question several times and is rewarded for the answers that beat the group's average.
BeginnerModels & Architecture
Gemini Omni
Gemini Omni is an AI model from Google, announced in May 2026, that makes and edits videos from any mix of text, photos, video and voice.
IntermediateModels & Architecture
Gemma 4
Gemma 4 is a family of free AI models from Google (April 2026) that you run on your own computer, server or phone, so your data stays in-house.
IntermediateModels & Architecture
LoRA (low-rank adaptation)
LoRA is a cheap way to train an AI model further: the model itself stays unchanged, and you train a small adapter that tunes it to your task.
AdvancedModels & Architecture
Mamba
Mamba is a way of building AI language models that reads text with a fixed-size memory, so long texts take less memory than in a transformer.
IntermediateModels & Architecture
MiniMax-M2
MiniMax-M2 is a series of AI language models that MiniMax released from October 2025, built for coding and for agents that carry out tasks with tools.
AdvancedModels & Architecture
Nemotron-Labs Diffusion
Nemotron-Labs Diffusion is a family of downloadable NVIDIA language models (May 2026) that can write several words at once, so answers come faster.
IntermediateModels & Architecture
PEFT (parameter-efficient fine-tuning)
PEFT is a group of methods that adapt an AI model to your task by training only a small add-on or a small part of it, which makes fine-tuning much cheaper.
IntermediateModels & Architecture
RAG (retrieval-augmented generation)
RAG lets an AI assistant look things up in your own documents before it answers, so its replies rest on your information and are less often made up.
AdvancedModels & Architecture
RLHF (reinforcement learning from human feedback)
RLHF is a training step in which people compare a language model's answers, so the model learns to give the kind of answers people prefer, as with ChatGPT.
IntermediateModels & Architecture
GAN (generative adversarial network)
A GAN is an AI technique in which a forger and a checker train each other until the forger makes images that look real, such as photos of people who do not exist.
IntermediateModels & Architecture
Diffusion model
A diffusion model is a type of AI that makes images by starting from random noise and removing it step by step, guided by a text prompt.
IntermediateModels & Architecture
Foundation model
A foundation model is a large AI model trained on broad data, such as the models behind ChatGPT, that businesses adapt to many different tasks.
IntermediateModels & Architecture
Mixture-of-experts (MoE) model
A mixture-of-experts model is built from many parts of which only a few work on each word, so a very large model computes like a much smaller one.
AdvancedModels & Architecture
State space model (SSM)
A state space model is a type of AI model that reads data while keeping a fixed-size summary, so long texts take less memory than in a transformer.
IntermediateModels & Architecture
Transformer
A transformer is the type of AI model behind ChatGPT and most other chatbots: it reads a whole text at once and works out which words belong together.
IntermediateModels & Architecture
Adaptive thinking in AI
Adaptive thinking lets an AI model decide per question whether to think before it answers and for how long, so easy questions stay fast and hard ones get more care.
IntermediateModels & Architecture
Activation function
An activation function is the small rule inside a neural network that decides how much of each signal passes on, so the model can learn more than straight lines.
IntermediateModels & Architecture
Encoder-decoder architecture
An encoder-decoder is an AI design with two halves: the encoder reads the whole input, and the decoder uses it to write the output, such as a translation.
IntermediateModels & Architecture
Autoregressive generation
Autoregressive generation is how a language model writes: one small piece of text at a time, each chosen from everything written so far, until the answer is done.
IntermediateModels & Architecture
Beam search
Beam search lets an AI model keep several partial sentences open at once and pick the most likely complete one at the end, as translation and speech tools do.
IntermediateModels & Architecture
Model distillation
Model distillation trains a small AI model to copy the answers of a large one, giving you a cheaper and faster model for a set task, usually at some cost in quality.
IntermediateModels & Architecture
Perplexity in NLP
Perplexity is a score for how well a language model predicts text: the lower the number, the less the model was surprised by the words that actually came next.
AdvancedModels & Architecture
Positional encoding
Positional encoding tells a language model where each word sits in a text, so it takes word order into account: "dog bites man" is not the same as "man bites dog".
IntermediateModels & Architecture
Quantization
Quantization makes an AI model smaller by storing its numbers with less precision, so that it runs on cheaper hardware at the cost of a little quality.
AdvancedModels & Architecture
Speculative decoding
Speculative decoding lets a large AI model write faster: a small model guesses the next few words, and the large model checks all those guesses in one step.
AdvancedModels & Architecture
Bicameral Model
The Bicameral Model is a research design from May 2026 in which two AI language models work as a pair and exchange internal signals instead of text.
AdvancedModels & Architecture
KV cache
The KV cache is where a language model keeps its intermediate results while it writes, so it does not have to process all earlier text again for every new word.
AdvancedModels & Architecture
Attention mechanism
The attention mechanism lets an AI language model work out which words in a text belong together, such as which earlier word the word "it" refers to.