Case Study
Testing Process Re-engineering Project
Business Context
Freddie Mac’s Enterprise Decision Service (EDS) team is responsible for managing the shared Decision Service that powers risk assessment decisions for Freddie Mac’s customer-facing applications. This service is critical for evaluating loan risk and ensuring compliance with regulatory policies.
Challenge
When policy changes occur, the EDS team must perform an impact analysis and update the decision models accordingly. Prior to deploying these changes, the team tests the updated models by manually creating test loan files, executing them against the revised rules, and comparing the expected versus actual results. This manual process presented several challenges:
- Inefficient Process: Creating test cases took several days per sprint.
- Limited Representation: Manually created test cases did not always reflect the average loan profile of Freddie Mac’s purchased loans.
- Inconsistent Results: Operational variations that creep in during manual test case execution can lead to misleading or non-representative testing outcomes.
Solution
Recognizing an opportunity for improvement, I proposed and led the development of a utility to automate key aspects of the testing process. The utility was designed to utilize historical loan files as a test case library. It would capture key loan characteristics from hundreds of recently purchased loans, strip out Personally Identifiable Information (PII), automatically execute these loan files against both the old and new versions of the updated decision models and summarize test results (i.e. changes in edits/edit volume)
Impact
The implementation of this utility significantly improved the EDS team’s testing efficiency and effectiveness:
- Reduced Testing Time: The automation cut down testing efforts by several days per sprint, accelerating the deployment of policy changes.
- Enhanced Accuracy: Test cases were now derived from actual loan data, ensuring they accurately matched the current risk profile/loan characteristics of 'real-world' loans.
- New Risk Management Capability: The utility also enabled an entirely new capability -- it gave risk managers the ability to project the effect of risk policy changes on purchase volumes before implementation.
This innovation not only streamlined internal workflows but also enhanced strategic decision-making for risk assessment and policy planning.
🤔 Want to hear more?
Use the link below to schedule a meeting.
stakeholder management
data management
requirements gathering
Servicer Oversight Enhancement Project
How do you make data a shared asset? Learn how I helped a servicer surveillance team enhance its oversight capabilities and improved access to servicing data for cross-functional teams.
user growth
product stratetgy
system integration
User Growth Strategy Project
What part do third-party integrations play in spurring user growth? Learn how I helped a fintech company increase the size and value of its user base through strategic integration plan.