Data Discretization

Data discretization converts a large number of continuous values into a smaller number of intervals or groups, making the data easier to analyze, understand, and manage.

In simple terms, it transforms continuous numerical data into a finite set of ranges or categories, while trying to keep the resulting loss of information as small as possible. That trade-off is the heart of the topic: discretization always discards detail, and the goal is to discard only detail that did not matter.

Why Discretize At All

Converting precise numbers into coarse bands can seem like a step backwards. There are several reasons it is worth doing:

• Some algorithms require categorical input — the original ID3 decision tree and many association rule algorithms cannot handle continuous attributes at all.
• It reduces noise. A measurement of 72.4 and one of 72.6 almost certainly mean the same thing, and treating them as distinct values invites the model to fit meaningless variation.
• It produces results people can read. "Customers aged 25–34" is actionable; a coefficient on a continuous age variable is less so.
• It reduces the number of distinct values an algorithm must consider, which speeds up processing.

Two Main Types

1. Supervised Discretization

The class label (target variable) is used when dividing the data into intervals, so the process considers how the values relate to the outcome being predicted. Boundaries are placed where the class actually changes.

2. Unsupervised Discretization

Class labels are not used. The method depends only on the distribution of the data itself.

Supervised methods generally produce more useful intervals when a target variable exists, because they cut where it matters. Unsupervised methods are the only option when there is no label — and they are the more common starting point.

Both types use one of two strategies:

• Top-down splitting — start with one large interval and divide it into smaller ones
• Bottom-up merging — start with many small intervals and combine adjacent ones

Techniques of Data Discretization

1. Binning

Binning groups continuous values into intervals called bins. It reduces noise, simplifies the data, and helps create concept hierarchies.

For example, marks from 0–100 can be grouped as:

   0–40  → Low
  41–70  → Medium
  71–100 → High

There are two fundamental binning strategies, and the difference between them matters more than it first appears. Take these 12 customer ages:

  18, 19, 21, 22, 23, 25, 28, 30, 35, 42, 55, 78

Equal-width binning divides the value range into equal intervals. The range is 18 to 78, so three bins are 20 wide:

Equal-frequency binning puts the same number of records in each bin:

Same data, same number of bins, very different results. Equal-width produced bins of 9, 2, and 1 — one bin holding three quarters of the data and another holding a single outlier. It is simple and the boundaries are easy to explain, but skewed data ruins it, and a single extreme value (the 78) stretches the range for everything else.

Equal-frequency guarantees balanced bins, which is usually better for analysis, at the cost of boundaries that fall at arbitrary-looking numbers and can split nearly identical values into different bins.

2. Histogram Analysis

A histogram shows the frequency distribution of continuous data, helping reveal:

• Outliers
• Skewness
• Whether the distribution is roughly normal

Examining the histogram before choosing bin boundaries lets you place cuts at natural gaps in the data rather than at arbitrary positions — which is often better than either standard binning strategy applied blindly.

3. Cluster Analysis

A clustering algorithm divides the values into groups, where each cluster contains values similar to one another, and those clusters become the intervals.

This is effectively automated gap-finding: clustering places boundaries where the data is sparse, which is exactly where a boundary does least damage. It handles skewed and multi-modal distributions that trip up equal-width binning.

4. Decision Tree Analysis

Decision trees can perform discretization using a top-down splitting approach, and this method is supervised.

The process selects split points that minimize the entropy of the resulting intervals — equivalently, that maximize information gain with respect to the class label. It then recurses within each interval, continuing until a stopping criterion is met.

Because it uses the class label, the boundaries land exactly where the outcome changes, which typically makes this the most effective method when a target variable exists. The entropy and information gain calculations involved are covered in this series' lesson on decision tree induction.

5. Discretization by Correlation Analysis (ChiMerge)

This is a supervised, bottom-up merging method. The standard algorithm is ChiMerge, and it works by:

1. Treating each distinct value as its own interval initially
2. Computing the chi-square statistic for each pair of adjacent intervals
3. Merging the adjacent pair with the lowest chi-square value — the pair whose class distributions are most similar
4. Repeating until a stopping condition is reached

The logic is that if two neighbouring intervals have statistically indistinguishable class distributions, the boundary between them carries no information and should be removed. Note that the merged intervals are always adjacent and never overlap — the result is a clean partition of the value range.

Concept Hierarchy Generation

A concept hierarchy represents data in a structured order from specific to general concepts, organizing it into levels of abstraction:

  New Delhi  →  India  →  Asia
    (city)     (country)  (continent)

Concept hierarchies support data summarization and allow analysis at whichever level of detail is appropriate — sales by city when investigating a specific store, by country when reporting to a board.

Discretization generates these hierarchies for numeric attributes: individual ages become age bands, and bands can themselves be grouped into broader categories.

Types of Hierarchy Generation

Top-down mapping starts with general information and moves toward specific detail:

  Continent → Country → City

Bottom-up mapping starts from specific data and generalizes upward:

  City → Country → Continent

Discretization and Binarization

These are related but distinct operations:

• Discretization converts continuous data into intervals or categories.
• Binarization converts attributes into binary values, 0 or 1.

Example of binarization:

  Temperature above 30°C  →  1  (Hot)
  Temperature below 30°C  →  0  (Not Hot)

Binarization is effectively discretization into exactly two bins, and it is also used to convert categorical attributes into a form numeric algorithms accept — a colour attribute with three values becomes three binary columns, one per colour.

Importance of Discretization

• Simplifies complex continuous data
• Reduces noise in the dataset
• Improves the signal-to-noise ratio
• Lets algorithms that require categorical input work at all
• Speeds up processing by reducing the number of distinct values
• Makes results easier to interpret and visualize

A Caution

Discretization is irreversible and it always loses information. Two risks are worth keeping in mind.

Boundary effects. Records either side of a cut point are treated as entirely different despite being nearly identical — an applicant aged 34 and one aged 35 land in different bands and may receive different decisions.

Too few bins. Aggressive discretization can erase the very pattern you were looking for. If a relationship only appears across a narrow range of values, binning that range into one category hides it permanently.

The practical guidance is to discretize when an algorithm requires it or when interpretability genuinely demands it — not as a routine preprocessing step applied to every numeric column.

Related Concepts

Discretization appears as one of the Data Transformation techniques in this series' KDD process lesson. The entropy calculations behind decision-tree discretization are covered in the decision tree induction lesson, the clustering methods in the clustering lesson, and the chi-square statistic used by ChiMerge in the redundancy and correlation lesson.