Question Time! Sometimes people ask me interesting questions
Q: My approvals rate is decreasing over time, AND my arrears rates are increasing — but we haven’t changed our credit strategy!?! What is going on? How do we unpack this, and what can we do about it??
Conventional wisdom states that if the credit approval rate decreases, I should expect to see a corresponding improvement in the arrears profile.
It can be extremely disconcerting when this doesn’t happen. In this problem statement, there are three key pieces of information:
The approvals rate is decreasing
The arrears rate is increasing, and
The Credit Team have not implemented any strategy changes.
Had the Credit Team tightened credit strategy settings, we would expect to see a decrease in the approvals rate, although, we may not have anticipated the increase in the arrears rate.
Similarly, had the approvals rate remained constant, although an increase in arrears rates may have been unpalatable, it isn’t completely unexpected given current conditions.
What seems surprising is the fact that arrears rates are increasing at the same time as approval rates are decreasing.
Let’s unpack this one step at a time.
First, let’s define the approval rate: this is the number of approved applications divided by the total number of applications. Approval rates can be measured over any period; however, for reporting purposes, they are typically calculated monthly.
It’s worth pointing out that a credit application may be referred for additional underwriting or information before it is approved. We’re interested in the number of approved applications, regardless of whether they were referred during the application process or not.
Assessing application risk
Applications can be assessed in several different ways, ranging from highly automated processes, leveraging bespoke scorecards and champion/challenger strategies, to more straightforward approaches where all applications are manually reviewed for decisioning.
Let’s look at three possible scenarios
Scenario 1:
Highly automated decisioning, where applications that are deemed too high risk are automatically declined, and applications that meet risk criteria are automatically approved, subject to affordability and regulatory compliance. All other applications are referred to underwriters for evaluation and final decisioning.
Scenario 2:
Some degree of automated decisioning; only very high-risk applications, or applications that fail policy rules, are automatically declined; all other applications are referred for manual underwriting.
Scenario 3:
No automation. All applications are manually assessed.
In all cases, in order to assess credit applications, credit providers need some way of assessing application risk.
Application risk is often estimated by using application scores or bureau scores. In many instances, a combination of application and bureau scores are used as they contain different information.
In order to unpack the declining application rate, it is useful to understand whether the application score is shifting over time. There are two ways I like to do this.
Box and Whisker Plot (Box Plots)
The Box and Whisker Plot is the more familiar chart type, however, you could also consider the Violin Plot, as it adds an extra dimension to the data visualisation — it combines a box plot with a histogram plot.
Create two datasets, the first dataset contains data where the accept rate was ‘normal’, or delivering to expectations. The second dataset contains data where the accept rate has fallen to the new, concerning, levels.
Plot Box Plots (or Violin Plots) of the two datasets.
Both Box Plots and Violin Plots show the median value of the data (the middle value), the interquartile range (represented by the box in a box plot). Whiskers extend from the box and represent the minimum and maximum values, excluding any outliers. Outliers are shown as individual points above or below the whiskers.
Once we have drawn the two plots, let’s look at them.
Is there a difference in the median value? If the median value in the second box plot is less than the median value in the first box plot, it means that 50% of data is scoring lower now than it was.
Look at the shape of the Violin Plots. How is the data distributed? And have a look at how it is distributed when compared to the cut off score.
If either of these plots show that the median score is lower than it was, and we’re seeing lower inter quartile ranges and the violin plot is looking more pear-shaped; that means that we seem to be attracting a higher risk application population than we were historically attracting.
And this is where conversations can get very thorny, particularly with Marketing.
Rank Order Applications
Another way to look at the application data is to create quintiles, based on the first dataset. Essentially, we’re rank ordering all the data, and then we create five equal-sized groups. Next, we apply the same groupings (based on score ranges) to the second set of data and whether the population has shifted.
Using the first dataset, determine the accept rate associated with each risk group. What we would expect to see is that the best quintile of applications has the highest accept rate, while the worst quintile has the lowest accept rate. We would also expect to see a steady decrease in accept rate as we progress from the highest scoring quintile to the lowest scoring one. Then do the same thing for the second dataset.
It would be reasonable to expect the same (or very similar) accept rates by quintile. For example, if the top quintile has an 80% accept rate in the first dataset, it would be reasonable for it to also be 80% in the second dataset.
Similarly, if the highest risk quintile has a 10% accept rate in the first dataset, it would be reasonable for it to have a 10% accept rate in the second dataset.
In the event that the accept rates are very different across corresponding quintiles, that would be worth further investigation.
And so what?
Well, the first dataset represents what normal used to look like, when accept rates and performance were within expectations.
If this analysis shows that the number of applications in the lowest group in the recent data sample represents 30% of applications (as opposed to the 20% that it used to represent), we can start to see why not only is our accept rate reducing, but our performance is also deteriorating as the overall risk of both the application population and subsequently approved account population is scoring lower, indicating that, overall the population risk has increased.
I realise that this is not particularly intuitive, however, once we have unpacked the shifts in the population, and understand the associated implications, not only on the approval rate, but also on Operations Teams, as underwriters have more applications to process, and delinquency rates and Arrears Management Operations Team, as more accounts roll into arrears, we see the impact of closely monitoring application risk.
Of course, once we have identified that there is a shift in the application score, then we need to investigate the key drivers of that shift.
Have we changed something in our Marketing approach? Are applications coming in via new channels? Have there been any variable changes that may impact the score?
Credit risk strategies are never as easy as ‘set and forget’. They should be actively monitored and managed to ensure that any changes are picked up and investigated in a timely manner.
Comments