KDD vs. Data Mining

The two terms are used almost interchangeably in casual conversation, and that habit obscures a real and useful distinction. Knowledge Discovery in Databases (KDD) is the entire end-to-end process of turning raw data into knowledge someone can act on. Data mining is one specific stage inside that process — the stage where an algorithm actually searches the prepared data for patterns. Every data mining project sits inside a KDD process, but a KDD process involves a great deal of work that is not, itself, data mining.

The Eight Steps of KDD

KDD is structured as a sequence of eight steps, and following them in order on a concrete scenario shows why the distinction between "the whole process" and "the mining step" matters in practice.

Take a telecom company that wants to reduce customer churn (customers canceling their subscription).

1. Goal Identification. The problem is defined precisely before any data is touched: not "understand our customers" but "identify which currently active customers are likely to cancel within the next 60 days, so retention offers can be targeted at them." A vague goal at this step produces a technically successful project that answers a question nobody asked.

2. Data Selection. The company chooses which of its data is actually relevant — billing history, call and data usage, support ticket volume, and contract type — and excludes data that has no bearing on the goal, such as internal employee records.

3. Data Cleaning and Preprocessing. Errors and missing values in the selected data are handled — a customer record missing a contract start date, or a usage log with an impossible negative call duration — using the techniques covered in the lesson on data cleaning.

4. Data Reduction and Transformation. The cleaned data is simplified: redundant or unhelpful attributes are dropped, and remaining ones are transformed into a form the mining algorithm can use — for instance, converting "last login date" into "days since last login," a number that is directly usable, rather than a raw date that is not.

5. Choosing the Data Mining Method. The company decides that this is a classification problem — customer, in or out of the "will churn" category — rather than clustering or association rule mining, because the goal from step 1 has a specific, known target to predict.

6. Model Building — the Data Mining Step. An algorithm (a decision tree, logistic regression, or similar classifier) is actually applied to the prepared data to build a model that assigns each customer a churn likelihood. This single step is what the term "data mining" refers to precisely — pattern extraction using an algorithm applied to data that steps 1 through 5 have already prepared.

7. Evaluation and Presentation. The model's predictions are checked against known outcomes (customers who did, or did not, churn in the past), and the results are turned into a form people can use — a dashboard, a report, a ranked list of at-risk customers.

8. Knowledge Usage. The retention team acts on the output: at-risk customers identified by the model receive a targeted offer, and the effect on actual churn is tracked, which is what ultimately validates whether the entire process delivered real value.

Steps 1 through 5 and 7 through 8 are KDD work that has nothing to do with running an algorithm — they are about understanding the problem, preparing data, and acting on results. Step 6 alone is data mining.

What Data Mining Adds on Top of This

Within step 6, data mining pursues one of two distinct goals, and knowing which one a given task is aiming for changes what counts as success.

Verification starts from an existing hypothesis and checks whether the data supports it. "We believe customers with more than three support tickets in 90 days churn at a higher rate" is a hypothesis; testing it against the prepared data either confirms or refutes it. This is analysis in service of a specific, pre-stated claim.

Discovery goes the other way: no hypothesis is given in advance, and the algorithm searches the data for whatever structure is there. The classification model in step 6 above did not start from a stated theory about which factors predict churn — it discovered which factors mattered by learning from the historical data directly.

Most of the named data mining tasks are discovery-oriented by nature:

• Classification — assigning records to predefined categories (as in the churn example above; covered in full depth in its own lesson).
• Clustering — grouping similar records with no predefined categories at all.
• Regression — estimating a continuous numeric value rather than a category (also covered in its own lesson).
• Association — finding relationships between variables, as in market basket analysis.

Why the Distinction Is Worth Keeping Straight

Treating "data mining" and "KDD" as synonyms causes a specific, recurring mistake: judging an entire analytics project by the quality of its algorithm alone. In the churn example, a state-of-the-art classifier (step 6) trained on badly selected or poorly cleaned data (steps 2–4) will produce confident, well-calibrated-looking predictions that are wrong, because the algorithm can only be as good as what it was given to learn from. Most of the effort in a real KDD project — and most of the ways a project actually fails — happens in the surrounding steps, not inside the mining algorithm itself. Calling the whole project "data mining" hides where that effort, and that risk, actually lives.