Difference between regression and classification

Regression and classification are two of the most common jobs for an AI model that learns from examples with known answers. Regression predicts a number, such as next week's sales or a delivery time. Classification predicts a category, such as spam or not spam, or which team should handle a customer's ticket. Your question decides which one you need, and that choice shapes how the model is trained and tested.

How do they differ?

Both are forms of supervised learning: the model learns from examples that come with the right answer, the label. Google's introduction to machine learning calls them two of the most common uses of supervised learning. A regression model predicts a numeric value, such as the amount of rain in millimetres. A classification model predicts whether something belongs to a category, such as whether an email is spam.

The textbook Deep Learning by Goodfellow, Bengio and Courville calls the two tasks alike except for the format of the output: one of a fixed set of categories, often with a probability for each, or a numerical value. Its example of regression is predicting how much an insured person will claim, which insurers use to set premiums.

How are they trained and tested?

The output decides the rest. A regression model is usually trained on the mean squared or mean absolute error, and a classifier on cross-entropy, also called log loss (see loss function). In the Python library scikit-learn the same neural network comes in both forms: MLPRegressor has no activation function in its output layer and minimises the squared error, and MLPClassifier minimises cross-entropy and returns a probability per class. Decision trees and random forests come in both versions too.

The scores differ as well. A classifier is judged on how often its labels are right, with measures such as accuracy, precision and recall. A regression model is judged on how far its numbers are off. The mean absolute error is the average miss in the unit of the label, such as euros or days. The R² score compares the model with one that always predicts the average: that one scores 0, a perfect model 1, and a model that does worse than the average scores below 0.

Where does it get confusing?

• Logistic regression is used for classification. scikit-learn notes that, despite its name, it is a linear model for classification. Google's glossary counts it as a regression model, because it predicts a probability between 0 and 1; a threshold that a person chooses then turns that probability into a class.
• A number is not always an amount. The same glossary calls a model that predicts a numeric postal code a classification model.
• Many questions can be asked both ways: how much rain will fall, or whether it will rain. The decision you need to make shows which form fits.

Can a language model do both?

A large language model can sort text into categories that you describe in the prompt (zero-shot or few-shot prompting). It can also predict numbers. Vacareanu et al. (2024) gave models such as GPT-4 and Claude 3 example inputs and outputs in the prompt. On synthetic test sets, their predictions rivalled those of supervised methods such as random forests, and sometimes beat them. The authors do not suggest that language models should replace traditional regression methods.

Example

Suppose an insurer for small businesses wants two predictions for each policy. Whether a new claim needs a manual check is a classification question: yes or no. How much the customer is likely to claim in the coming year is a regression question, the textbook's own example. Both models can use the same policy data, but each learns from a different label and is judged by a different score.