What Is Data Redundancy?

During data integration, data is collected from different sources, and the same information often ends up represented more than once in the combined dataset. This is data redundancy.

An attribute is redundant if its value can be derived from other attributes already present. In a dataset with 20 attributes, if one can be determined from the others, it contributes no new information.

Redundancy also arises from inconsistent naming of attributes across sources — the same quantity recorded under two different names becomes two columns that always agree.

An Example

Consider a dataset with three attributes:

• pizza_name
• is_veg — 1 if the pizza is vegetarian, otherwise 0
• is_nonveg — 1 if the pizza is non-vegetarian, otherwise 0

Since a pizza can only be one or the other, the two attributes are directly related:

• If is_veg = 0, the pizza must be non-veg, so is_nonveg = 1
• If is_veg = 1, then is_nonveg = 0

Either attribute can be derived from the other, so one of them is redundant and can be removed without losing any information.

This specific case — where one column is an exact function of another — is called perfect collinearity, and it causes real problems beyond wasted space. Some algorithms, particularly regression-based ones, fail outright or produce unstable coefficients when fed perfectly correlated inputs, because there is no unique way to divide the effect between two columns carrying identical information.

Why Redundancy Matters

Removing redundant attributes is not just tidying up:

• Faster processing — fewer attributes means less computation, which matters as datasets grow.
• Better model quality — correlated inputs can destabilize models and make their outputs harder to interpret.
• Lower dimensionality — fewer attributes means distance measures stay meaningful, an issue covered in this series' clustering lesson.
• Clearer results — a model citing two columns that say the same thing is harder to explain than one citing a single column.

Detecting Redundancy

Two standard methods detect relationships between attributes, and which one applies depends on the data type:

• Chi-square (χ²) test — for categorical (qualitative) attributes
• Correlation coefficient and covariance — for numeric attributes

1. The Chi-Square Test

The chi-square test applies to categorical data. A contingency table records the frequency of each combination of two attributes, and the test compares:

• Observed values — the actual frequencies in the data
• Expected values — the frequencies you would see if the attributes were completely independent

The formula sums the squared difference between observed and expected across every cell, scaled by the expected value:

  χ² = Σ (Observed − Expected)² / Expected

Expected values are computed from the row and column totals:

  Expected = (row total × column total) / grand total

A Worked Example

Suppose 1,500 customers are recorded by gender and preferred book category:

If the two attributes were independent, the expected count for male/fiction would be:

  (450 × 300) / 1500 = 90

The observed count is 250 — nearly three times higher. Computing all four cells:

  χ² = 284.44 + 71.11 + 121.90 + 30.48 = 507.94

The test checks the hypothesis that the two attributes are independent. With one degree of freedom, a χ² above 10.83 rejects independence at the 0.1% level. At 507.94, the hypothesis is rejected overwhelmingly: gender and book preference are strongly related in this data.

When the hypothesis is rejected, the attributes carry overlapping information, and one may be a candidate for removal.

2. Correlation Coefficient for Numeric Data

For numeric attributes, redundancy is detected using the correlation coefficient. The relationship between two attributes A and B is measured with Pearson's product-moment correlation coefficient:

        Σ (aᵢ − Ā)(bᵢ − B̄)
  r = ─────────────────────────
      √[Σ(aᵢ − Ā)² × Σ(bᵢ − B̄)²]

The value ranges from −1 to +1:

A Worked Example

  A = [ 2,  4,  6,  8, 10]
  B = [ 3,  6,  9, 12, 15]     →  r = +1.0000
  C = [10,  8,  9,  5,  3]     →  r = −0.9220

B is exactly 1.5 × A, so the correlation is perfect: B is entirely redundant given A. C moves in the opposite direction to A and is strongly but not perfectly related — knowing A tells you a great deal about C, though not everything.

Covariance

Covariance is the unstandardized version of the same idea — the numerator of the correlation formula, divided by the number of records:

  cov(A, B) = Σ (aᵢ − Ā)(bᵢ − B̄) / n

For the data above, cov(A, B) = 12.0 and cov(A, C) = −6.8. The sign is informative — positive means the variables move together — but the magnitude is not comparable across attribute pairs, because it depends on the units. Covariance in rupees × years is not comparable to covariance in kilograms × metres.

Correlation solves this by dividing by the standard deviations, producing a value always between −1 and +1 regardless of units. This is why correlation is used for comparing relationships and covariance mostly appears as an intermediate step.

Other Correlation Methods

• Pearson correlation — for continuous numeric variables with a roughly linear relationship
• Spearman rank correlation — used when at least one variable is a rank, or when the relationship is monotonic but not linear

Interpreting the Results

• High correlation → the attributes are strongly related, and one may be removable
• Correlation near 0 → the attributes are independent, and both should be kept
• Negative correlation → one increases as the other decreases; the strength is given by the magnitude, not the sign

Note that −0.9 indicates a stronger relationship than +0.3. It is the distance from zero that measures strength.

Two Important Cautions

Correlation does not imply causation. Two attributes may correlate because one causes the other, because a third factor drives both, or by coincidence in a small sample. For the purpose of removing redundant attributes this distinction rarely matters — but the moment a correlation is described as a finding rather than a preprocessing signal, it matters a great deal.

Zero correlation does not mean no relationship. Pearson correlation measures linear relationships only. Two attributes related in a clear curve — rising then falling — can have a correlation near zero while being entirely predictable from one another. A scatter plot reveals this instantly where the coefficient hides it, which is why examining the data visually remains worthwhile alongside computing the number.

Don't remove attributes automatically. High correlation identifies candidates for removal, not conclusions. Two strongly correlated attributes may still contribute differently to a model, and which one to keep should depend on which is more reliably measured, more interpretable, or more readily available in future data.

Related Concepts

Redundancy and correlation appear as one of the five issues in this series' lesson on data integration, where this analysis sits in the wider process. The chi-square statistic is also used by the ChiMerge algorithm covered in the discretization lesson, and the dimensionality concerns motivating attribute removal are discussed in the clustering lesson.