Water utilities companies increasingly rely on predictive models to prioritise pipeline replacement, yet they rarely evaluate whether these decisions actually prevent failures. This study applies a counterfactual modelling approach to assess the effectiveness of <em>Brabant Water</em>’s replacement strategy in the Netherlands. Using rare-events logistic regression on pipeline and environmental characteristics, failure probabilities were estimated for pipelines replaced between 2020 and 2023 and simulated a no-replacement scenario. Comparing simulated counterfactual failures with observed post-replacement failures shows that executed replacements reduced failure rates by more than 80%. Spatial hotspot analysis further reveals that most simulated high-risk areas did not experience failures after replacement, indicating successful risk mitigation. The study demonstrates that counterfactual reasoning provides a powerful framework for evaluating infrastructure policies, provided that predictive models are well calibrated.