Preprocessing in Data Mining: Data Quality and Feature Selection Strategy

The mechanics of preprocessing — imputation, outlier detection, scaling, and encoding — are covered with full worked examples in the lesson on data preprocessing techniques, and where preprocessing fits inside a full project is covered in the lesson on the data mining process. This lesson covers two things that deserve their own treatment: what "clean data" actually means in concrete, checkable terms, and the three genuinely different strategies for selecting which features to keep.

What "Data Quality" Actually Means

"Clean the data" is easy to say and hard to check, because data quality isn't one property — it's six independent ones, and a dataset can fail any of them while passing the others completely:

• Accuracy — is the value correct? (an age recorded as 250)
• Completeness — is anything missing? (a blank field where a value should exist)
• Consistency — is the same fact recorded the same way everywhere it appears?
• Timeliness — is the data current enough to still be true?
• Reliability — can the source itself be trusted?
• Clarity — is the data documented well enough to be interpreted correctly? (a column labeled amt with no stated currency or unit)

Consistency failures are the ones most likely to slip through unnoticed, because each individual value can look perfectly valid on its own. Suppose a company merges its sales system, where gender is recorded as M / F, with its support-ticket system, where the same field is recorded as 1 / 0. Neither system contains an invalid value — every record passes an accuracy check individually — but a naive merge that doesn't first map M → 1 and F → 0 (or vice versa) produces a combined dataset with four apparent categories instead of two, silently splitting what should be one group into two undercounted ones. This is exactly the kind of error the data integration step of a mining project exists to catch, and it's a failure of consistency specifically, not of accuracy, completeness, or any of the other five dimensions.

Feature Selection: Three Fundamentally Different Strategies

The lesson on data preprocessing techniques distinguishes feature selection (keeping a subset of existing features) from dimensionality reduction (constructing new combined features, as PCA does). Within feature selection itself, there are three strategies that differ in what they actually evaluate:

Filter methods score each feature using a statistical measure — computed independently of any model — such as correlation with the target or information gain, and keep the highest-scoring features. Take two features, Height in centimeters and Height in inches, describing the same people: because one is a fixed linear multiple of the other, their correlation is 1.0. A filter method would flag this immediately and let you drop one of the two without losing any information the other doesn't already carry — a fast check that requires no model training at all.

Wrapper methods, such as Recursive Feature Elimination (RFE), evaluate a candidate subset of features by actually training a model on it and measuring performance, then repeating with different subsets. This catches interactions between features that a filter method's one-feature-at-a-time scoring would miss entirely (two features that are individually weak predictors but powerful together), at the cost of retraining a model — potentially many times — for every subset considered.

Embedded methods select features as a side effect of training a single model, rather than as a separate step before or around it. LASSO regression, for example, penalizes the model for using additional features and shrinks the weight of unhelpful ones toward exactly zero during training itself; tree-based models similarly produce a usable feature-importance ranking as a natural byproduct of building the tree, covered further in the lesson on attribute selection measures.

Filter methods are the cheapest and most model-agnostic; wrapper methods are the most thorough but the most computationally expensive; embedded methods sit in between, folding the selection decision into training a model you were likely going to build anyway.

Discretization: Equal-Width vs. Equal-Frequency Binning

The lesson on data preprocessing techniques covers equal-width binning, which divides a feature's range into fixed-size intervals regardless of how many records fall into each one. Equal-frequency binning does the opposite: it divides the sorted values so that each bin holds the same number of records, regardless of how wide a range that requires. Take eight sorted age values — 18, 20, 22, 25, 30, 45, 60, 90 — split into 4 bins:

Equal-frequency (2 records per bin):
Bin 1: {18, 20}   range = 2
Bin 2: {22, 25}   range = 3
Bin 3: {30, 45}   range = 15
Bin 4: {60, 90}   range = 30

Equal-width (same 4 bins, fixed range of 18 each across 18-90):
Bin 1: {18}                  (18-36)
Bin 2: {45}                  (36-54)
Bin 3: {60}                  (54-72)
Bin 4: {90}                  (72-90)

Equal-width binning here produces badly unbalanced bins (one value in three of the four bins) because most of the data is clustered at the low end of the range with a couple of larger values stretching it out — exactly the skew pattern covered in the lesson on normalization in data mining. Equal-frequency binning avoids that imbalance by construction, at the cost of bin boundaries that no longer correspond to clean, easily explained ranges.

Where the Rest of Preprocessing Lives

Cleaning (missing values, duplicates, outlier detection via the IQR method) and the encoding/scaling techniques (one-hot, ordinal, min-max, z-score) are covered with full worked examples in the lesson on data preprocessing techniques; the additional scaling methods (decimal, robust, log, softmax) are covered in the lesson on normalization in data mining; and how all of these steps fit into the larger sequence of a project is covered in the lesson on the data mining process.