Estimating Treatment Effects with Linear Regression

Abstract

Estimating causal effects from observational data is challenging due to non-random treatment assignment.
In this post, we use a real-world bank marketing dataset to illustrate how linear regression can be used for causal inference under key assumptions such as conditional ignorability and overlap.

We contrast predictive vs causal modeling, highlight common pitfalls (e.g., selection bias and post-treatment controls), and provide practical diagnostics to assess whether regression estimates can be interpreted causally.
The goal is not just estimation, but building intuition for when such estimates are trustworthy.

Bank Marketing Case Study

This analysis applies causal inference concepts to the Bank Marketing dataset to estimate treatment effects in a practical setting.

Causal question

Does contacting customers by cellular instead of telephone increase the probability of subscription?

• Treatment (T): contact = cellular (1) vs telephone (0)
• Outcome (Y): subscription = yes (1) vs no (0)

Dataset (links + context)

We use the Bank Marketing dataset (Portuguese bank campaigns).

• Official source (UCI):
  https://archive.ics.uci.edu/dataset/222/bank+marketing

Each row = customer

• Features = demographics, history, campaign behavior
• Treatment = contact channel
• Outcome = subscription

📚 Reference

Facure, Matheus. Causal Inference in Python: Applying Causal Inference in the Tech Industry.

Interpretation Layer

What problem are we solving?

We are estimating the causal effect of a treatment variable on an outcome using regression under causal assumptions.

Predictive vs Causal Regression

Predictive regression asks:

If X changes, how does Y move in historical data?

Causal regression asks:

If we intervene and change Treatment, how does Y change?

This distinction is critical.

Required Assumptions

To interpret regression causally:

• No unobserved confounding (Conditional Ignorability)
• Correct functional form (or reasonable approximation)
• No post-treatment leakage
• Sufficient overlap between treated and control populations

Business Interpretation

In marketing / fintech:

• Treatment coefficient ≈ incremental lift
• Positive → treatment helps
• Negative → treatment hurts
• Near zero → no incremental value

Key Causal Assumptions Being Used

1. Conditional Ignorability

After controlling for X:

Treatment ⟂ Potential Outcomes

If violated → biased estimates

2. Overlap (Positivity)

Every user must have a non-zero probability of both treatment and control.

If violated:

• Extrapolation risk
• Unstable estimates

3. No Post-Treatment Controls

Do NOT include variables affected by treatment.

This creates:

• Collider bias
• Underestimation of treatment effect

Diagnostics — Causal Meaning

Residual Diagnostics

• Random residuals → model specification reasonable
• Patterned residuals → possible nonlinearity or missing confounders

Coefficient Stability

• Large swings across specs → weak identification or collinearity

Overlap Checks

If treated/control covariates differ heavily:

Model extrapolates → causal estimate becomes fragile

Linear regression can estimate causal effects—but only when assumptions hold.
The real skill is not fitting the model, but validating those assumptions.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import statsmodels.formula.api as smf
import os

np.random.seed(42)
FIG_DIR = "figures_ch4_bank_marketing"
os.makedirs(FIG_DIR, exist_ok=True)

!pip install ucimlrepo

Requirement already satisfied: ucimlrepo in c:\users\revan\minicondanew\lib\site-packages (0.0.7)
Requirement already satisfied: pandas>=1.0.0 in c:\users\revan\minicondanew\lib\site-packages (from ucimlrepo) (2.3.3)
Requirement already satisfied: certifi>=2020.12.5 in c:\users\revan\minicondanew\lib\site-packages (from ucimlrepo) (2025.11.12)
Requirement already satisfied: numpy>=1.26.0 in c:\users\revan\minicondanew\lib\site-packages (from pandas>=1.0.0->ucimlrepo) (2.3.5)
Requirement already satisfied: python-dateutil>=2.8.2 in c:\users\revan\minicondanew\lib\site-packages (from pandas>=1.0.0->ucimlrepo) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in c:\users\revan\minicondanew\lib\site-packages (from pandas>=1.0.0->ucimlrepo) (2025.2)
Requirement already satisfied: tzdata>=2022.7 in c:\users\revan\minicondanew\lib\site-packages (from pandas>=1.0.0->ucimlrepo) (2025.2)
Requirement already satisfied: six>=1.5 in c:\users\revan\minicondanew\lib\site-packages (from python-dateutil>=2.8.2->pandas>=1.0.0->ucimlrepo) (1.17.0)

Load public Bank Marketing data (UCI)

from ucimlrepo import fetch_ucirepo

bank = fetch_ucirepo(id=222)
df = pd.concat([bank.data.features, bank.data.targets], axis=1)

df["Y"] = (df["y"].astype(str).str.lower() == "yes").astype(int)
df["contact"] = df["contact"].astype(str).str.lower()
df = df[df["contact"].isin(["cellular","telephone"])].copy()
df["T"] = (df["contact"]=="cellular").astype(int)

df.head()

Naive regression (difference in means)

📊 How to Interpret the Treatment Coefficient

Treatment Coefficient ≈ Average Treatment Effect (ATE) if assumptions hold

Example: Coefficient = 0.12

Interpretation: If treatment is applied, outcome increases by ~0.12 units on average, holding confounders constant.

Business Translation: Expected incremental lift per treated user ≈ coefficient value

m_naive = smf.ols("Y ~ T", data=df).fit()
m_naive.summary().tables[1]

Adjusted regression with month fixed effects

📊 How to Interpret the Treatment Coefficient

Treatment Coefficient ≈ Average Treatment Effect (ATE) if assumptions hold

Example: Coefficient = 0.12

Interpretation: If treatment is applied, outcome increases by ~0.12 units on average, holding confounders constant.

Business Translation: Expected incremental lift per treated user ≈ coefficient value

num_controls = ["age","balance","campaign","pdays","previous","day"]
num_controls = [c for c in num_controls if c in df.columns]

cat_controls = ["job","marital","education","housing","loan","month","poutcome"]
cat_controls = [c for c in cat_controls if c in df.columns]

formula = "Y ~ T"
for c in num_controls:
    formula += f" + {c}"
for c in cat_controls:
    formula += f" + C({c})"

m_adj = smf.ols(formula, data=df).fit()
m_adj.summary().tables[1]

Frisch–Waugh–Lovell (FWL) theorem

📊 How to Interpret the Treatment Coefficient

Treatment Coefficient ≈ Average Treatment Effect (ATE) if assumptions hold

Example: Coefficient = 0.12

Interpretation: If treatment is applied, outcome increases by ~0.12 units on average, holding confounders constant.

Business Translation: Expected incremental lift per treated user ≈ coefficient value

🧪 Diagnostics — Causal Meaning

Residual Diagnostics:

• Random residuals → Model specification reasonable
• Patterned residuals → Possible nonlinearity or missing confounder

Coefficient Stability:

• Large swings across specs → weak identification or collinearity

Overlap Checks: If treated and control covariate distributions differ heavily: → Model extrapolates → causal estimate fragile

f_T = "T ~ age + balance + C(month)"
f_Y = "Y ~ age + balance + C(month)"

mT = smf.ols(f_T, data=df).fit()
mY = smf.ols(f_Y, data=df).fit()

df["T_res"] = mT.resid
df["Y_res"] = mY.resid

m_fwl = smf.ols("Y_res ~ T_res", data=df).fit()
m_fwl.summary().tables[1]

Heterogeneous effects (interaction with month)

📊 How to Interpret the Treatment Coefficient

Treatment Coefficient ≈ Average Treatment Effect (ATE) if assumptions hold

Example: Coefficient = 0.12

Interpretation: If treatment is applied, outcome increases by ~0.12 units on average, holding confounders constant.

Business Translation: Expected incremental lift per treated user ≈ coefficient value

m_inter = smf.ols("Y ~ T*C(month) + age + balance", data=df).fit()
m_inter.summary().tables[1]

🧠 What the Results Actually Show

Across specifications, the estimated treatment effect remains positive but modest:

• Naive regression: ~0.015
• Fully adjusted model: ~0.04
• FWL residual regression: ~0.035–0.04
• Interaction model: heterogeneous but directionally consistent

👉 This stability is important.

It suggests that:

• The initial bias in naive estimates is not extreme
• Adding controls and fixed effects refines rather than overturns the result
• The treatment effect is real but relatively small

Additionally, the increase from naive to adjusted estimates suggests the presence of negative selection bias in treatment assignment.

🔑 Key Takeaways

• Regression = adjusted comparison, not causality by default
• Controls matter → effect size increases after adjustment (bias correction)
• Month fixed effects matter → seasonality was confounding the estimate
• FWL confirms equivalence → partialling out yields consistent estimates
• Heterogeneity exists → treatment effectiveness varies across months

📊 Business Interpretation

• Estimated lift ≈ 3–4 percentage points
• Effect is statistically significant but economically modest
• Not a “silver bullet” channel — but directionally beneficial

👉 Translation:
Cellular contact improves conversion, but its impact depends on timing and context.

⚠️ When Should You Trust This?

Linear regression works well here because:

• ✅ Large sample size
• ✅ Reasonable overlap between treated and control groups
• ✅ Rich covariates capturing key confounders
• ✅ Effects approximately linear

❌ When This Would Break

Be cautious if:

• Hidden confounders exist (e.g., targeting based on unobserved intent)
• Strong nonlinear effects dominate
• Treatment assignment is highly selective
• Post-treatment variables are included as controls

🔗 Where This Fits in the Bigger Picture

Linear regression is the starting point for causal inference:

• If assumptions hold → simple, interpretable, and fast
• If assumptions weaken → move to:
  • Inverse Propensity Weighting (IPW)
  • Doubly Robust methods
  • Meta-learners (S / T / X learners)
  • Causal forests

🎯 Final Takeaway

Linear regression can recover meaningful causal effects in observational marketing data —
but its reliability comes not from the model itself, but from:

the validity of assumptions and the consistency of estimates across specifications.