Chapter 4 — Linear Regression for Causal Inference

Bank Marketing Case Study

This notebook illustrates Chapter 4 of Matheus Facure’s Causal Inference in Python using the Bank Marketing dataset.

Causal question:
Does contacting customers by cellular instead of telephone increase the probability of subscribing to a term deposit?

• Treatment T: contact = cellular (1) vs telephone (0)
• Outcome Y: subscription = yes (1) vs no (0)

Dataset (links + download options)

We use the Bank Marketing dataset (Portuguese bank direct marketing campaigns).

Official source (UCI Machine Learning Repository)

• Dataset page: https://archive.ics.uci.edu/dataset/222/bank+marketing
  (Direct downloadable files are linked on that page.)

Kaggle mirror (CSV download; requires Kaggle login)

• https://www.kaggle.com/datasets/janiobachmann/bank-marketing-dataset

Which file should you use?

• Kaggle typically provides bank-full.csv / bank.csv
• The UCI dataset provides multiple formats; this notebook loads from UCI via ucimlrepo

📘 Causal Linear Regression — Blog Interpretation Layer

What problem are we solving?

We are trying to estimate the causal effect of a treatment variable on an outcome variable using linear regression under causal assumptions.

Predictive vs Causal Regression

Predictive regression answers:

If X changes, how does Y move in historical data?

Causal regression answers:

If we intervene and change Treatment, how does Y change?

Required assumptions:

• No unobserved confounding (Conditional Ignorability)
• Correct functional form (or good approximation)
• No post-treatment leakage
• Sufficient overlap between treated and control populations

Business Interpretation

In marketing / fintech:

• Treatment coefficient ≈ Incremental lift
• Positive → treatment helps
• Negative → treatment hurts
• Near zero → no incremental value

⚠️ Key Causal Assumptions Being Used Here

1️⃣ Conditional Ignorability

After controlling for covariates X: Treatment ⟂ Potential Outcomes

If violated → biased effect estimate

2️⃣ Overlap (Positivity)

Every user has some probability of treatment and control

If violated:

• Extrapolation risk
• Unstable coefficients

3️⃣ No Post-Treatment Controls

Do NOT include variables influenced by treatment This creates collider bias or blocks part of the treatment effect

🧪 Diagnostics — Causal Meaning

Residual Diagnostics:

• Random residuals → Model specification reasonable
• Patterned residuals → Possible nonlinearity or missing confounder

Coefficient Stability:

• Large swings across specs → weak identification or collinearity

Overlap Checks: If treated and control covariate distributions differ heavily: → Model extrapolates → causal estimate fragile

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import statsmodels.formula.api as smf
import os

np.random.seed(42)
FIG_DIR = "figures_ch4_bank_marketing"
os.makedirs(FIG_DIR, exist_ok=True)

!pip install ucimlrepo

Requirement already satisfied: ucimlrepo in c:\users\revan\minicondanew\lib\site-packages (0.0.7)
Requirement already satisfied: pandas>=1.0.0 in c:\users\revan\minicondanew\lib\site-packages (from ucimlrepo) (2.3.3)
Requirement already satisfied: certifi>=2020.12.5 in c:\users\revan\minicondanew\lib\site-packages (from ucimlrepo) (2025.11.12)
Requirement already satisfied: numpy>=1.26.0 in c:\users\revan\minicondanew\lib\site-packages (from pandas>=1.0.0->ucimlrepo) (2.3.5)
Requirement already satisfied: python-dateutil>=2.8.2 in c:\users\revan\minicondanew\lib\site-packages (from pandas>=1.0.0->ucimlrepo) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in c:\users\revan\minicondanew\lib\site-packages (from pandas>=1.0.0->ucimlrepo) (2025.2)
Requirement already satisfied: tzdata>=2022.7 in c:\users\revan\minicondanew\lib\site-packages (from pandas>=1.0.0->ucimlrepo) (2025.2)
Requirement already satisfied: six>=1.5 in c:\users\revan\minicondanew\lib\site-packages (from python-dateutil>=2.8.2->pandas>=1.0.0->ucimlrepo) (1.17.0)

Load public Bank Marketing data (UCI)

from ucimlrepo import fetch_ucirepo

bank = fetch_ucirepo(id=222)
df = pd.concat([bank.data.features, bank.data.targets], axis=1)

df["Y"] = (df["y"].astype(str).str.lower() == "yes").astype(int)
df["contact"] = df["contact"].astype(str).str.lower()
df = df[df["contact"].isin(["cellular","telephone"])].copy()
df["T"] = (df["contact"]=="cellular").astype(int)

df.head()

Naive regression (difference in means)

📊 How to Interpret the Treatment Coefficient

Treatment Coefficient ≈ Average Treatment Effect (ATE) if assumptions hold

Example: Coefficient = 0.12

Interpretation: If treatment is applied, outcome increases by ~0.12 units on average, holding confounders constant.

Business Translation: Expected incremental lift per treated user ≈ coefficient value

m_naive = smf.ols("Y ~ T", data=df).fit()
m_naive.summary().tables[1]

Adjusted regression with month fixed effects

📊 How to Interpret the Treatment Coefficient

Treatment Coefficient ≈ Average Treatment Effect (ATE) if assumptions hold

Example: Coefficient = 0.12

Interpretation: If treatment is applied, outcome increases by ~0.12 units on average, holding confounders constant.

Business Translation: Expected incremental lift per treated user ≈ coefficient value

num_controls = ["age","balance","campaign","pdays","previous","day"]
num_controls = [c for c in num_controls if c in df.columns]

cat_controls = ["job","marital","education","housing","loan","month","poutcome"]
cat_controls = [c for c in cat_controls if c in df.columns]

formula = "Y ~ T"
for c in num_controls:
    formula += f" + {c}"
for c in cat_controls:
    formula += f" + C({c})"

m_adj = smf.ols(formula, data=df).fit()
m_adj.summary().tables[1]

Frisch–Waugh–Lovell (FWL) theorem

📊 How to Interpret the Treatment Coefficient

Treatment Coefficient ≈ Average Treatment Effect (ATE) if assumptions hold

Example: Coefficient = 0.12

Interpretation: If treatment is applied, outcome increases by ~0.12 units on average, holding confounders constant.

Business Translation: Expected incremental lift per treated user ≈ coefficient value

🧪 Diagnostics — Causal Meaning

Residual Diagnostics:

• Random residuals → Model specification reasonable
• Patterned residuals → Possible nonlinearity or missing confounder

Coefficient Stability:

• Large swings across specs → weak identification or collinearity

Overlap Checks: If treated and control covariate distributions differ heavily: → Model extrapolates → causal estimate fragile

f_T = "T ~ age + balance + C(month)"
f_Y = "Y ~ age + balance + C(month)"

mT = smf.ols(f_T, data=df).fit()
mY = smf.ols(f_Y, data=df).fit()

df["T_res"] = mT.resid
df["Y_res"] = mY.resid

m_fwl = smf.ols("Y_res ~ T_res", data=df).fit()
m_fwl.summary().tables[1]

Heterogeneous effects (interaction with month)

📊 How to Interpret the Treatment Coefficient

Treatment Coefficient ≈ Average Treatment Effect (ATE) if assumptions hold

Example: Coefficient = 0.12

Interpretation: If treatment is applied, outcome increases by ~0.12 units on average, holding confounders constant.

Business Translation: Expected incremental lift per treated user ≈ coefficient value

m_inter = smf.ols("Y ~ T*C(month) + age + balance", data=df).fit()
m_inter.summary().tables[1]

Key takeaways

• Regression = adjusted comparison
• Month fixed effects remove seasonality bias
• FWL explains why controls work
• Interactions show when marketing works better

🧠 When Linear Regression Works Well for Causal Inference

✅ Large sample size
✅ Good overlap
✅ Strong confounder coverage
✅ Approximately linear effect

🚫 When It Struggles

❌ Strong nonlinear HTE
❌ Hidden confounders
❌ Extreme treatment imbalance
❌ Post-treatment variable leakage

🔄 Bridge to Meta-Learners and Forests

If linear model struggles: → S-Learner / T-Learner
→ X-Learner
→ Causal Forests