Rule-Based Classification in Data Mining

A rule-based classifier assigns a record to a class using explicit IF (condition) → THEN (class) statements — IF income > 50,000 AND existing_debt < 10,000 → THEN approve_loan — rather than a computed score or a set of weighted connections. This transparency is exactly what makes rule-based classification attractive in regulated domains, per the interpretability tradeoff covered in the lesson on major issues in data mining. This lesson focuses on how those rules actually get generated in the first place, since that's the part most often left unexplained.

How Rules Get Learned: Sequential Covering

The standard approach — called sequential covering (or "separate-and-conquer") — learns rules one at a time, each one covering as much of the remaining data as accurately as possible, removing whatever it covers before learning the next rule. Tracing this through 8 small weather records (Outlook → PlayTennis) makes the mechanism concrete:

Record  Outlook    PlayTennis
1       Sunny      No
2       Sunny      No
3       Rain       Yes
4       Rain       Yes
5       Overcast   Yes
6       Sunny      Yes   <- an exception to the usual Sunny pattern
7       Rain       Yes
8       Overcast   Yes

Step 1: Across all 8 records, find the single best rule. IF Outlook = Rain THEN Yes covers records 3, 4, 7 — all three correct (3/3 = 100% accuracy). Add this rule; remove records 3, 4, 7 from consideration.

Step 2: From the remaining 5 records (1, 2, 5, 6, 8), find the next best rule. IF Outlook = Overcast THEN Yes covers 5 and 8 — both correct (2/2 = 100%). Add this rule; remove 5 and 8.

Step 3: Only records 1, 2, 6 remain. The only remaining pattern is IF Outlook = Sunny THEN No, which covers all three but is only correct for 2 of them (record 6 is actually Yes) — 2/3 ≈ 67% accuracy. Since no better rule is available from what's left, this rule is accepted anyway, as the best remaining option, and the process ends since every record is now covered.

The result is three rules of decreasing reliability — 100%, 100%, 67% — learned strongest pattern first. This is the general shape of sequential covering on any dataset: each successive rule handles a smaller, harder-to-classify remainder, which is exactly why later rules in a rule-based classifier tend to be less reliable than earlier ones, and why the order rules are checked in matters (the first matching rule typically decides the class).

Rule Evaluation Reuses Familiar Measures

Once candidate rules exist, deciding which ones are worth keeping uses the same support and confidence measures covered with a full worked example in the lesson on association rule mining — support measuring how often a rule's condition occurs at all, confidence measuring how often the predicted class actually follows when it does. Rule pruning then removes rules that are redundant or unreliable (low confidence, or covering very few records), and the remaining rules are ranked, typically by confidence or by a combination of confidence and coverage.

A Category Distinction Worth Making Explicit

It's worth separating three things that get grouped together as "types of decision rules" in some treatments of this topic, because only one of them is actually rule-based classification. Classification rules (built via sequential covering, as above, or extracted directly from a decision tree — the C4.5 algorithm covered in its own dedicated lesson builds a tree first and can convert each root-to-leaf path into an equivalent IF-THEN rule) assign records to predefined classes. Association rules, covered in full in the lesson on association rule mining, and sequential pattern rules, covered in the lesson on trends in data mining, both take the same IF-THEN surface form but serve a fundamentally different, descriptive purpose: discovering which items or events tend to co-occur, with no predefined class being predicted at all. IF milk THEN bread is a discovered association, not a classification — nothing is being assigned to a category, a co-occurrence is simply being reported. All three produce rules that look alike on the page; only the first is rule-based classification in the sense this lesson covers.

Where Rule-Based Classification Still Holds Up

Even alongside the neural network and ensemble techniques covered elsewhere in this series, rule-based classifiers remain genuinely useful specifically because of the interpretability tradeoff covered in the lesson on major issues in data mining: a rule set can be read, audited, and explained to a non-technical stakeholder directly, in a way a trained neural network's internal weights cannot be — which is precisely why rule-based approaches persist in exactly the regulated, high-stakes domains (credit decisions, medical triage) where being able to explain a specific decision matters as much as the decision's raw accuracy.