A prompt is the input text or set of instructions given to a Large Language Model to generate a response. Prompts range from simple questions ("What is machine learning?") to elaborate multi-part instructions with system roles, examples, context documents, and output format specifications. The quality and structure of a prompt directly determines the quality of the LLM's output — the same model can produce mediocre or excellent results depending solely on how the prompt is crafted. Prompts are composed of several possible layers: a system prompt that defines the model's behavior and persona, user prompts containing the actual request, and optional context like retrieved documents or conversation history.
Why it matters
The prompt is the primary interface between humans and LLMs. Unlike traditional software where developers write code to specify exact behavior, LLMs are steered through natural language instructions — making prompt quality the single largest variable in output quality. A well-structured prompt with clear instructions, relevant examples, and explicit output format can improve task accuracy from 60% to 95% without changing anything about the underlying model. For businesses, this means that prompt optimization is often the highest-ROI activity when building AI applications: it costs nothing to improve a prompt, yet yields dramatic quality and cost improvements.
How it works
When you send a prompt to an LLM, the text is first tokenized into a sequence of token IDs. These tokens pass through the model's layers, and the model generates a response by predicting the most likely next token given the full input context. The prompt structure matters because different parts serve different functions: the system prompt (processed first) establishes behavioral constraints and persona; the context section provides reference information; and the user message contains the specific request. The model treats the entire prompt as a continuous token sequence and generates its response as a natural continuation. Longer, more detailed prompts consume more tokens and cost more, creating a natural tension between prompt comprehensiveness and efficiency.
Example
A legal tech company building a contract analysis tool discovers the difference prompt quality makes. Their initial prompt — "Analyze this contract and find risks" — produces vague, inconsistent outputs. The optimized version includes a system prompt defining the AI's role as a commercial contract analyst, specifies the exact risk categories to check (liability caps, indemnification clauses, termination triggers, IP assignment), provides two annotated examples of previous analyses, and defines the output format as a structured table with risk level, clause reference, and recommended action. The same model that produced unreliable output with the simple prompt now delivers consistent, lawyer-reviewed-quality analysis — with the only change being a better-structured 400-token prompt.