Does complex AI improve microloan decisions?
Not in this study. On 2,500 resolved MENA microloans, CatBoost and regularised logistic regression had statistically indistinguishable discrimination. Alternative data added little measurable lift, while a headline fairness screen missed a substantial rural–urban error-rate gap.
The short version
- What happened
- CatBoost reached a test ROC-AUC of 0.783, but regularised logistic regression reached 0.780 and the difference was not statistically significant; logistic regression was also best calibrated.
- Who it affects
- Borrowers need notice, correction rights and human appeal routes when behavioural or institutional data shape a lending decision.
- What remains unproven
- The study uses 2,500 previously approved loans from five unnamed MENA institutions; country and institution effects cannot be inspected independently.
Keep in mind
The study uses 2,500 previously approved loans from five unnamed MENA institutions; country and institution effects cannot be inspected independently.
Rejected applicants have no observed repayment outcome, so the paper cannot establish fair access or performance for the full applicant population.
The final test set contains 500 loans and 136 defaults, limiting precision for gender-by-region and other subgroup estimates.
All modelling is retrospective. No lender used the models prospectively, and borrower welfare, appeals, distress and over-indebtedness were not measured.
SHAP describes model behaviour rather than causal effects, and correlated or institution-derived variables can make attributions misleading.
Source analysis · 1 linked source. Primary analysed source date: 9 October 2026.
Read the full report · 8 min The evidence, methods, implications and limitations
The answer: simpler scoring matched the complex models
A more complex credit-scoring model did not produce a statistically credible improvement over regularised logistic regression in this dataset. CatBoost recorded the highest held-out discrimination, with a receiver-operating-characteristic area under the curve of 0.783 and a 95% confidence interval from 0.741 to 0.825. Logistic regression reached 0.780. A DeLong comparison gave p=0.59, and none of the gradient-boosting systems significantly outperformed the linear benchmark.
That is practically important because a transparent scorecard is easier for staff, auditors and applicants to interrogate than a large ensemble. Logistic regression also had the best calibration: its predicted probabilities more closely matched observed default rates. The result does not prove that simple models always win. It shows that, for 2,500 resolved microloans from five unnamed institutions in one regional portfolio, extra model complexity did not buy a clear increase in discrimination.
The fairness result was equally cautionary. Approval-proxy rates looked similar enough to pass a commonly used four-fifths screening heuristic across gender, region and their intersections. Yet rural borrowers who actually defaulted were much more likely than urban defaulters to be incorrectly predicted as repayers. A single parity ratio therefore concealed an error pattern that could expose rural borrowers to over-indebtedness and lenders to additional loss.[1]
What the researchers compared
The de-identified data covered 2,500 loans made during 2022–2023 by participating microfinance institutions in the Middle East and North Africa. All applications in the analysis had been approved and had reached a known outcome. Default meant at least 90 days past due. There were 682 defaults, or 27.28% of the sample. Borrowers were 55.48% male and 44.52% female; 55.56% of cases were rural and 44.44% urban.
Nine supervised algorithms were tested: regularised logistic regression, decision tree, random forest, gradient boosting, a radial-basis support-vector machine, k-nearest neighbours, XGBoost, LightGBM and CatBoost. The researchers used a stratified 80/20 split: 2,000 loans for training and 500 for the final test, with 136 defaults in the test set. Hyperparameters were chosen inside the training data using five-fold cross-validation rather than by looking at the test results.
They supplemented that split with five-fold cross-validation repeated five times on the training data, formal pairwise AUC tests and 1,000 bootstrap resamples for test confidence intervals. The fixed decision threshold was 0.50. That design is stronger than reporting one favourable accuracy number, although a 500-loan test set still leaves uncertainty around subgroup estimates and cannot substitute for validation at another institution or time.[1]
Alternative data had signal, but little incremental value
The study separated traditional institutional risk variables from three non-traditional indicators: mobile-money transaction frequency, utility-payment behaviour and savings-group membership. In the full CatBoost model, credit-score category, a missing utility-payment record and repayment history were the largest SHAP attributions. Savings-group membership and income also contributed. The ordering was highly stable across bootstrap samples and broadly consistent across tree models.
Feature importance does not answer whether those extra variables improve decisions beyond information the institution already holds. The authors therefore repeated the leading models without the three alternative indicators. Adding them raised CatBoost ROC-AUC from 0.775 to 0.783, a difference of 0.008 with a 95% bootstrap interval from -0.010 to 0.027. The DeLong p-value was 0.36. Logistic regression rose from 0.770 to 0.780, also without a significant difference.
Those findings can coexist: a feature can influence predictions inside a model while adding little new discrimination because its information overlaps with existing variables. This matters for consent and proportionality. Collecting more behavioural data is not automatically justified by a colourful explanation plot. A lender should first show that each proposed data source adds material, reproducible value for the target population and that its use is lawful, understandable and contestable.[1]
One fairness test passed while another exposed a gap
At the 0.50 threshold, the model's favourable prediction was treated as an approval proxy. Predicted approval rates were 78.5% for men and 82.1% for women, producing a disparate-impact ratio of 0.956. The urban and rural rates were 78.3% and 81.5%, for a ratio of 0.961. All gender-by-region intersections were also above the 0.80 heuristic, with a minimum-to-maximum ratio of 0.881.
Error rates told a different story. Among borrowers who ultimately defaulted, 68.0% of rural cases received a favourable prediction compared with 49.2% of urban cases, a gap of 18.8 percentage points. That is not evidence that rural applicants should be denied more often. It is evidence that the model and threshold behaved differently across groups and that a parity screen focused on predicted approvals did not capture the difference.
The denominator is also restricted. Because the dataset contains only granted loans, the analysis cannot show how the original institutions treated rejected applicants or whether a new model would expand access for people with thin files. This is the selective-label problem: outcomes are known only for people who passed an earlier decision. Calling the resulting model inclusive would require evidence about the full applicant pool, not only performance within approved borrowers.[1]
What this means for borrowers, lenders and regulators
For a microfinance institution, the defensible near-term option is the model that provides adequate discrimination, good calibration and explanations staff can use—provided it survives external testing. On this evidence, regularised logistic regression belongs on that shortlist. Model choice should be joined to threshold policy: the paper reports modest default recall at the fixed threshold, ranging from 0.22 to 0.55 across models, so operational consequences can change sharply when the cut-off moves.
For borrowers, explanation must extend beyond a list of influential features. People need to know what data were used, which information can be corrected, how the threshold affected the decision and how to reach a human reviewer. Utility records, mobile activity and group membership may reflect infrastructure, location or social circumstances as well as creditworthiness. Treating an attribution as a causal reason risks hardening existing disadvantage into an apparently technical decision.
For regulators and risk committees, the central lesson is to audit several fairness concepts at the deployed threshold and repeat the audit after model or population change. Selection rates, true-positive rates, false-positive rates, predictive value and calibration answer different questions and may conflict. Monitoring must also track harm after approval, including arrears, refinancing, distress and complaints, rather than equating a model-approved loan with financial inclusion.[1]
What would change the assessment
Confidence would rise with external validation at named institutions across multiple MENA countries, later economic periods and genuinely thin-file applicant groups. A prospective study should compare the simple and complex models under the same lending policy, predefine thresholds and record approval, repayment, over-indebtedness and appeal outcomes. It should include rejected applicants through a design that can address missing counterfactual outcomes rather than assuming the approved portfolio represents everyone seeking credit.
The rural error gap needs replication with larger subgroup denominators and investigation of infrastructure, income measurement, loan purpose, institution and geography. Researchers should test whether it persists after calibration or threshold changes and whether attempts to reduce it create other harms. Independent access to a privacy-protected dataset would make the result more auditable; the current loan-level data are restricted to the five institutions, although the analysis code is supplied.
For now, the study supports a restrained conclusion: in this portfolio, complexity did not beat a well-regularised transparent benchmark, and alternative data did not deliver a clear incremental gain. It also demonstrates why a fairness badge based on one ratio is unsafe. The authors reported no research funding and no competing interests; Middle East University helped pay the publication charge.[1]