SilentBias
Fairness & Bias Audit for Education
Removing protected attributes doesn't remove bias. SilentBias audits the UCI Adult Income pipeline end-to-end: does a shadow model reconstruct hidden gender from 'neutral' features, and does reweighing actually fix disparate impact or just push the ratio past the fairness threshold in the opposite direction?
- Role
- Solo — data audit, shadow-model leakage probe, mitigation, SHAP dashboard.
- Timeline
- 5 weeks
- Source
- GitHub ↗
What was hard.
Shadow-model leakage from 'neutral' features
After deleting gender from the training set, a Random Forest shadow model reconstructed it at AUC 0.94 using occupation, marital status, hours-per-week and relationship. Proxy leakage is the default, not the exception.
Reweighing without over-correcting
Sample-level reweighing shifted Disparate Impact Ratio from 0.35 (biased against one group) to 2.76 (biased the other way). Mitigation is not a one-way fix — the audit made the over-correction visible instead of hiding it.
Explanations non-experts can use
SHAP force plots shipped inside a Streamlit dashboard so a reviewer can audit any individual prediction, not just aggregate fairness metrics.
How it fits together.
Data audit → shadow-model leakage probe → mitigation → SHAP UI
- ▸Fairness metrics tracked: demographic parity, equal opportunity, disparate impact.
- ▸Every mitigation step logged with before/after — no silent changes.
What shipped.
- —Deleting protected attributes does not delete bias — proxies leak.
- —Reweighing can flip the direction of unfairness; audits must run both ways.
- —A dashboard a reviewer can audit beats a paper metric they can't.
What the evidence supports.
Scope of the evidence
The AUC 0.94 reconstruction comes from this specific setup — a Random Forest shadow model on the UCI Adult Income dataset with these features. It shows proxy leakage is real here, not that sensitive attributes can always be reconstructed with the same accuracy. Likewise, the DIR shift from 0.35 to 2.76 after reweighing demonstrates over-correction in this experiment, not an unqualified mitigation success.