Classification and Prediction in Data Mining

A bank deciding whether to approve a loan and a bank estimating how much a customer will spend next month are solving two different kinds of problem, even though both start from the same customer records. The first assigns the applicant to one of a fixed set of categories — approve or reject. The second estimates a number that could fall anywhere on a scale. Classification and prediction are the two data mining techniques built for exactly these two kinds of question, and the distinction between them is the single most useful thing to get right before choosing either one.

The Core Distinction

• Classification assigns a record to one of a predefined set of categories (also called classes or labels).
• Prediction estimates a continuous numerical value that has no fixed set of options.

"Will this loan applicant repay?" has two possible answers, known in advance — that is classification. "How much will this customer spend next month?" has no such fixed list — the answer could be ₹0 or ₹50,000 or any value in between — that is prediction, and it is typically carried out using regression techniques (covered in full depth in the lesson on regression).

Both techniques share the same underlying approach even though their outputs differ: both are built from a training dataset of past records where the answer is already known, and both produce a model that is then applied to new records where it is not.

A Note on Terminology: Two Different Meanings of "Classification"

The word "classification" is used in data mining for two genuinely different things, and mixing them up causes real confusion.

Machine learning classification — the subject of this lesson — is a predictive technique: an algorithm learns from labeled historical examples and predicts the category of a new, unlabeled record.

Data classification in the information governance and security sense is a completely different activity: labeling records by sensitivity (public, internal, confidential) so that appropriate access controls and encryption can be applied to them. This second meaning has its own five-step process (define policy, identify sensitive data, apply labels, enforce access controls, review as data grows) and its own lifecycle (creation, access control, storage, sharing, archiving, publication) — but none of this involves training a model to predict anything. It is closer to a records-management discipline than to a data mining technique.

Both are legitimately called "classification" in their respective fields, but a security team labeling documents as confidential and a data scientist building a decision tree to predict loan risk are doing unrelated work that happens to share a name. The rest of this lesson is entirely about the machine learning sense.

How Classification Works

Building and using a classifier happens in two stages.

Stage 1: Training

The algorithm learns from a training dataset, where each record (also called a sample, object, or data point) has:

• A set of input attributes (features)
• The correct class label for that record

From this data the algorithm produces a classifier — a decision tree, a mathematical formula, or a neural network, depending on the algorithm chosen. What all three forms have in common is that they encode a rule for mapping input attributes to a class label, learned entirely from the labeled examples.

Stage 2: Testing and Use

The classifier is then evaluated on test data — records it has never seen — to measure how accurately it assigns the correct label. Only after this accuracy check does it get applied to genuinely unlabeled, real-world data.

Worked Example: Bank Loan Classification

Suppose a bank wants to classify loan applicants as safe or risky based on two attributes: years employed and credit score. Simulating 200 past applicants where the true outcome is already known, training a decision tree on 70% of them, and testing on the remaining 30% gives a concrete, checkable result:

import numpy as np
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, confusion_matrix

# X: [years_employed, credit_score] for 200 applicants; y: 1 = safe, 0 = risky
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

clf = DecisionTreeClassifier(max_depth=3, random_state=42)
clf.fit(X_train, y_train)
preds = clf.predict(X_test)

print("accuracy:", round(accuracy_score(y_test, preds), 3))
print("confusion matrix:\n", confusion_matrix(y_test, preds))

Output:

accuracy: 0.817
confusion matrix:
 [[14  4]
 [ 7 35]]

The accuracy of 0.817 means the model correctly classified about 82% of the 60 applicants in the test set — applicants it never saw during training. The confusion matrix breaks that number down further: of the 18 genuinely risky applicants in the test set, 14 were correctly caught and 4 were missed (classified as safe when they were actually risky); of the 42 genuinely safe applicants, 35 were correctly identified and 7 were incorrectly flagged as risky. This breakdown matters more than the single accuracy number for a bank, because the cost of the two kinds of mistake is not the same — approving a loan that defaults (the 4 missed risky cases) is typically far more expensive than rejecting an applicant who would have repaid (the 7 misclassified safe cases).

Once trained, the model can classify a brand-new applicant instantly:

sample = np.array([[5, 700]])  # 5 years employed, credit score 700
print(clf.predict(sample))  # → 1 (Safe)

This is the entire point of building the classifier: the historical data does the work of defining the rule, so a new applicant does not have to be evaluated from scratch by a person re-deriving the pattern each time.

Binary vs. Multiclass Classification

The loan example above has exactly two possible labels — this is binary classification. Many problems require more than two categories: classifying a support ticket as billing, technical, or account-related, for instance. This is multiclass classification, and it uses the same training-then-testing structure, just with more than two labels to choose among.

Applications of Classification

• Sentiment analysis — classifying social media comments or reviews as positive, negative, or neutral.
• Document (text) classification — assigning documents to categories based on content, as in spam detection, news categorization, or topic tagging.
• Image classification — identifying what an image contains, from recognizing handwritten digits to detecting objects in security footage.

All three are supervised learning tasks in the sense used throughout this lesson: each depends on a training set where the correct label is already known.

Prediction

Prediction addresses questions with no fixed category to assign — an estimate of a continuous quantity instead. It follows the identical training-then-testing pattern as classification, but the training data pairs inputs with a numerical output rather than a class label, and the model produced is typically a regression model.

House price prediction is a standard example: given a house's number of rooms, total area, location, and age, a regression model estimates its expected price — a number, not a category. Estimating customer spending works the same way: given a customer's income, purchase history, and occupation, a model estimates how much they are likely to spend, again a number on a continuous scale.

The mechanics of building these models — simple and multiple linear regression, the least-squares fitting method, and how to interpret the resulting coefficients — are covered with full worked examples in the lesson on regression in data mining.

Preparing Data for Either Technique

Both classification and prediction depend on the same data preparation work before any model is built.

• Data cleaning — removing noise and handling missing values (often by filling with the most frequent value for categorical attributes, or the median for numeric ones, as covered in the lesson on data cleaning).
• Relevance analysis — using correlation analysis to check whether each attribute actually relates to the target being predicted, and removing attributes that do not. An irrelevant attribute does not just fail to help; it adds noise the algorithm has to sort through, which can measurably hurt accuracy.
• Normalization — scaling attribute values into a comparable range (for example, 0 to 1). This matters most for algorithms like neural networks, which are sensitive to the relative scale of different input features — an income value in the tens of thousands and an age value under 100 can otherwise dominate a model's calculations for reasons that have nothing to do with which attribute is actually more informative.
• Generalization via concept hierarchies — replacing a specific value with a broader category, such as replacing an exact city with its state or country, to reduce the number of distinct values a model has to handle.

Comparing Classification and Prediction Models

Once a model is built — of either kind — it is judged against the same handful of criteria:

The trade-off between accuracy and interpretability shows up constantly in practice: the loan example above used a decision tree specifically because a bank often needs to explain to a regulator or a rejected applicant why a decision was made, not just report that the model was 81.7% accurate on held-out data.