Center for Financial Innovation and Stability

Notes from the Vault
Larry D. Wall
October 2018

The evaluation of retail credit applications is a good example of decisions once made by humans that are increasingly delegated to machines. The changes needed to facilitate a greater reliance on machines go well beyond merely substituting a machine for a human. Similarly, the consequences of greater reliance on machines go far beyond mere displacement of a few credit reporting and analysis jobs. This post reviews some of the developments that facilitated greater reliance on machines—and the consequences of those developments—for retail credit. The post continues with a look to the future, focusing on two evolving technological developments: blockchain technology for storing data and machine learning for analyzing data.

A brief history of credit reporting and scoring
The decision by a lender to grant credit to a borrower depends in large part on the lender's evaluation of the likelihood a borrower will repay in accordance with the loan contract. In order to make this evaluation, the lender first obtains and then evaluates information about the borrower. The lender could rely solely on information obtained from a borrower, but often the lender obtains additional information from third parties.

According to a paper by Federal Reserve Bank of Philadelphia economist Robert M. Hunt, consumer credit bureaus emerged in the late 19th century and were typically cooperative or nonprofit ventures organized by local merchants to pool credit information (and also to assist in debt collection). As consumer mobility increased, it became useful to share information across different geographies. This need for information sharing was met by the creation in 1906 of a trade organization known as Associated Credit Bureaus Inc. (or ACB) for most of its life. Hunt reports that ACB developed "the procedures, formats and definitions that enabled the sharing of credit files between agencies across the country."

Computerization of credit information started in the 1950s, according to a paper by Florida International University professor Kenneth Lipartito. However, Hunt observes that credit bureaus' technology remained largely that of (paper) filing cabinets and telephones until the mid-1960s, when bureaus in the larger cities automated, starting with Los Angeles in 1965. Nevertheless, once automation started, the high fixed cost of automating encouraged consolidation.

The creation of electronic databases provided the information needed for automated analysis of consumer credit. Lipartito argues that the demand for an automated approach grew as banks entered the credit card industry and needed the ability both to decide whether to issue a card to a given individual and then whether to approve credit purchases by that individual. The ultimate solution to this problem started with the formation of the firm Fair Isaac in 1956. Martha A. Poon's dissertation provides a history of Fair Isaac. Poon, a PhD student at the University of California at San Diego, notes that Fair Isaac applied statistical analysis to estimate the odds of "applicants' future payment behavior." She chronicles the improvement of the statistical analysis, with Fair Isaac using increasingly sophisticated methods of calculating scores.

The credit bureau business in the United States is currently dominated by three national credit bureaus: Experian, Equifax, and TransUnion, according to a post by Jeannette N. Bennett, a St. Louis Fed economic education specialist. Along with Fair Isaac's FICO credit score, each of the three national credit bureaus also calculates its own credit score. Additionally, Bennett reports, "Over 400 smaller, regional, or industry-specific credit bureaus exist in the United States."

Along with technology and borrowers' needs, an important factor shaping the development of credit bureaus and credit scores has been legal requirements. These have varied over time, but the issues became increasingly important as the bureaus and scores became more crucial. The modern legal and regulatory environment grew out of two pieces of legislation from the 1970s. Congress responded to the concerns about data accuracy and its uses with the passage in 1970 of the Fair Credit Reporting Act and its subsequent amendments.¹ Congress addressed the concerns about the use of certain types of data with the Equal Credit Opportunity Act of 1974 and its subsequent amendments. According to a post by the U.S. Department of Justice, among other things the 1974 Act "prohibits creditors from discriminating against credit applicants on the basis of race, color, religion, national origin, sex, marital status, age…"

Although the sharing of credit data is intended to improve credit decisions, the consequences of data sharing go far beyond allowing lenders to make more informed credit granting decisions. Modern credit scoring models not only allow lenders to decide whether a borrower is creditworthy, they also allow lenders to set interest rates and credit limits that reflect the riskiness of individual borrowers. In doing so, these models expand the set of consumers who could obtain credit.

The development of credit sharing and credit scoring increased competition in loan markets in at least three ways. First, access to the data and scores allowed lenders to operate in areas where they did not maintain a physical presence.² Second, banks gradually determined there was not a large difference between a personal loan to a small business owner and a business loan to that owner. As a result, they developed credit granting models for small businesses based on credit scores.³ Third, the development of credit scoring facilitated the development of loan securitization, or the process of pooling consumer loans, structuring their cash flows, and selling securities backed by these cash flows. This development allowed loans to be originated by nonbank financial institutions, which previously had limited access to funding.

Thus, the emergence of consumer credit bureaus expanded the set of information available to lenders, and the development of credit scores provided lenders with a way of relating that data to the credit outcomes using statistical methods. The development of distributed ledger technology, especially in the form of blockchains, may provide the basis for developing even more informative data. Machine learning may also provide a way of using data to produce better forecasts of borrower behavior than traditional statistical methods. The following two sections analyze the potential contribution of these two technologies.

Blockchain
The existing credit bureaus have been criticized as suffering from a variety of weaknesses including: (a) inaccurate and incomplete data, (b) the small number of bureaus that control legitimate access to individuals' credit data rather than individuals controlling their own data, and (c) the vulnerability of centralized databases to hacking.⁴ One proposed solution for these problems has been to use blockchain technology to replace the credit bureaus.⁵ However, merely demonstrating weaknesses in the current approach is not sufficient to prove blockchain technology is needed to improve credit data. Rather, any superiority provided by a blockchain has to arise from blockchain features that cannot be replicated using other database technologies.

Thus, the question is, what is different about blockchain technology? As I observed in a prior post, there is no single definition of blockchain. As in that post, for current purposes, Wikipedia provides a good definition: "A blockchain…is a continuously growing list of records, called blocks, which are linked and secured using cryptography." In particular, what blockchain technology offers relative to other types of databases is superior protection against malicious attempts to tamper with existing records (blocks) in the database.⁶ However, as my prior post explains, tamper resistant is not the same as immutable, as there are circumstances where blocks of data on a blockchain have been rewritten or deleted.

If the problem of inaccurate or incomplete credit data arose primarily because of malicious efforts to deliberately delete or introduce errors into credit databases, blockchain technology would be the obvious candidate for solving the issue. However, the problem of inaccurate data appears to arise far more from external parties providing inaccurate data to the credit bureau. Incomplete data are an issue because not all potentially relevant data are reported. A blockchain could solve the problem of bad data from external parties if every financial transaction by all individuals were recorded on a single blockchain. But the majority of almost all individuals' transactions currently take place off a blockchain and so must be reported by external sources.⁷ The blockchain may perfectly record this reported data, but that does not prove the reported data are correct or accurate. When dealing with information from external sources, blockchains are no less (and no more) dependent upon the accuracy and completeness of such sources than other types of database. ^8,9 Whatever new data sources become available to a blockchain system could, in principle, be made available to some other type of database.¹⁰

Blockchains are also promoted as potentially giving individuals complete control over who sees what parts of their credit history. However, this control depends largely on government privacy regulation. Absent regulation, credit bureaus could still collect information directly from lenders as they do now. With regulation, credit bureaus could be required to adopt privacy controls analogous to those implemented using blockchain technology.¹¹ One way in which giving consumers' complete privacy could create a negative externality would be its effect on the modeling of credit risk. Current methods of estimating creditworthiness benefit from large amounts of data. These models are likely to become less accurate if a large fraction of individuals or even a large fraction of a particular subgroup were to withhold their information.¹²

In terms of hostile parties hacking centralized databases, protection against malicious attempts to rewrite blocks is not the same as protecting the data stored in the blocks from being read by malicious actors. A blockchain storing sensitive private information should and almost certainly would protect that information using advanced cryptography and provide individuals with private keys to access the data. However, credit bureaus using some other database technology could (and should) have been encrypting private information. Indeed, public blockchains that provide the entire history of transactions may create a weakness that would not necessarily arise under other database technologies. The problem is even if a blockchain's code is being continually updated to use the latest and most secure encryption technology when writing new blocks of data, the older blocks cannot easily be rewritten using the newer encryption technology. The same features that protect against malicious changes to old blocks also raise the cost of rewriting old blocks with new encryption.

Finally, although blockchains come with certain advantages, they also come with some costs. For example, by definition, a blockchain carries the entire history of transactions on that blockchain, which has implications for data storage and communication requirements. In terms of other properties of the blockchain, there are a variety of trade-offs depending on how the blockchain system is structured, according to a paper by Michel Rauchs from the Cambridge Centre for Alternative Finance and his coauthors. For example, a blockchain system designed for maximum tamper resistance may have to accept lower throughput capacity.¹³

Thus, blockchains may ultimately prove a superior way of storing data on individuals' creditworthiness. At this point, it is not clear blockchain technology provides substantial net benefits over other types of databases.

Machine learning
The conversion of raw credit data into useful predictions about borrower creditworthiness has relied on empirical models of consumer behavior ground in statistics. However, the development of machine learning techniques raises the potential for extracting even better estimates of individuals' creditworthiness.¹⁴ As I discuss in a recent paper (Wall 2018), the differences between traditional statistics and machine learning are more due to the goals they prioritize than to any fundamental difference in what they are trying to do. Statistics is more about testing hypotheses using data samples, while machine learning is more about obtaining the best prediction using very large data sets. The strength and weakness of the machine learning approach is it is not tied to estimating a particular model grounded in economic theory. This is a strength in that machine learning can identify relationships that are causal and had not previously been predicted by theory. But it is also a weakness because machine learning may identify correlations that are not causal and cannot be reliably exploited.

Another weakness of some machine learning algorithms for credit scoring is that it can be difficult to explain how a particular credit evaluation relates to the information in an individual's credit file. This can be problematic, in part, because if a credit decision results in a denial or less favorable terms and the data from credit reporting agencies are required by regulation, then the lender must explain the key factors that adversely affected the borrower's score. Even potentially more troublesome is that a machine learning model may inadvertently discriminate based on certain prohibited personal characteristics, as discussed in my paper with Julapa Jagtiani from the Philadelphia Fed and Todd Vermilyea from the Federal Reserve Board.

Despite these concerns, machine learning is increasingly being applied to retail credit evaluation. Indeed, banks have long used machine learning to detect credit fraud, according to an article by Bart van Liebergen, an adviser at the Institute of International Finance. More recently, some marketplace-lending firms are applying machine learning to evaluate credit applications, and recent studies suggest this usage appears to be resulting in better credit risk measurement.¹⁵ However, the real test of how much these models improve on conventional retail credit models will not come until the next recession, when we would expect to see elevated default rates.

Conclusion
The development of credit bureaus and credit scoring models has had a large impact on the provision of credit to individuals and small businesses. Whether and to what extent blockchain technology can improve the accuracy and completeness of consumer credit data remains to be seen. Machine learning is already being deployed with seeming success, albeit it may be some time before we can fully evaluate its contribution to credit risk modeling.

References

Acquisti, Alessandro, Curtis Taylor, and Liad Wagman (2016). "The Economics of Privacy." Journal of Economic Literature 54, no. 2: 442–92. https://www.aeaweb.org/articles?id=10.1257/jel.54.2.442.

Akhavein, Jalal, W. Scott Frame, and Lawrence J. White (2005). "The Diffusion of Financial Innovations: An Examination of the Adoption of Small Business Credit Scoring by Large Banking Organizations." Journal of Business 78: 577–596. https://www.jstor.org/stable/10.1086/427639?seq=1#metadata_info_tab_contents.

Wall, Larry D., 2018. "Some Financial Regulatory Implications of Artificial Intelligence." Journal of Economics and Business. Available behind a paywall at https://doi.org/10.1016/j.jeconbus.2018.05.003.

Larry D. Wall is executive director of the Center for Financial Innovation and Stability at the Atlanta Fed. The author thanks Scott Frame, Gina Pieters, Brian Robertson, and Warren Weber for helpful comments. The view expressed here are the author's and not necessarily those of the Federal Reserve Bank of Atlanta or the Federal Reserve System. If you wish to comment on this post, please email atl.nftv.mailbox@atl.frb.org.

_______________________________________

¹ See a paper by Federal Reserve Board economists Robert B. Avery, Paul S. Calem, and Glenn B. Canner, along with then University of Southern California professor Raphael W. Bostic, for a discussion of consumer data and credit reporting.

² See a paper by Purdue University professor John M. Barron and Georgetown University professor Michael E. Staten for a discussion of the value of credit scoring in allowing banks to market credit cards in new areas.

³ See Jalal Akhavein of Moody's Risk services, W. Scott Frame, a colleague of mine in the Atlanta Fed's Research Department, and Lawrence J. White, a professor at New York University (2005), which documents the growing use of small business credit scoring by large banks.

⁴ See, for example, an article by Russell Brandom, policy editor for the Verge.

⁵ See, for example, here, here, here, and here.

⁶ This analysis of blockchains also applies to the more general term of distributed ledger technology as discussed in a paper by Michel Rauchs from the Cambridge Centre for Alternative Finance and coauthors. I use the term blockchain in this article because it is more widely recognizable.

⁷ For a further discussion of the issues associated with data arising that is external to a blockchain, see subsection 4.2.2 of the paper by Michel Rauchs and coauthors.

⁸ The argument can be made because blockchains rely on a consensus mechanism for adding new data that blockchains have a built-in control for data accuracy lacking in other databases. However, the consensus mechanism generally does not provide an additional check on the accuracy of data beyond that of confirming the new transactions comply with the blockchain's protocols.

⁹ Many of the posts analyzing the potential benefits of blockchains in credit reporting, for example, see here, argue that individuals could more easily correct mistakes if credit records were stored on a blockchain. Whether this is true would depend on who is given permission to write to that blockchain and what their incentives are to correct records. Lenders are unlikely to have much confidence in credit information if individual consumers are allowed to update records the consumer views as "inaccurate."

¹⁰ A whitepaper by Bloom, a recent start-up, proposes to create a credit risk measure that includes both on-chain credit transactions and individuals staking (or vouching) for one another's identity and creditworthiness. The idea of individuals vouching for one another's creditworthiness is not new; character references that put the author's reputation at risk have long been used, as has cosigning a loan that puts the cosigner at direct financial risk. However, if Bloom's staking technology proves practical, it would allow a substantial expansion in the use of character references. Whether Bloom's approach requires the use of a blockchain or could be based on a different database technology is unclear from the whitepaper.

¹¹ For example, see my post with Steven Zitzer on the General Data Protection Regulation recently implemented in the European Union.

¹² More generally, Carnegie Mellon professor Alessandro Acquisti, Duke professor Curtis Taylor, and Illinois Institute of Technology professor Liad Wagman (2016) show the economic efficiency of enhanced privacy is mixed. Depending upon the situation, the most economically efficient solution may range from requiring complete transparency to giving individuals complete control over the use of their data.

¹³ An earlier post of mine also observes that some types of blockchain systems have significant governance issues that will need to be resolved before they could become a critical part of the financial system's infrastructure.

¹⁴ Machine learning has the potential to improve credit analysis; the improvement need not be significant in all cases. A post by Scott Zoldi, Bruce Curry, and Ethan Dornhelm of FICO notes that firm has over 25 years of experience with credit scoring, allowing it to learn a great deal about the determinants of credit risk. The authors find machine learning by itself is faster than their traditional methods but only marginally more accurate. On the other hand, traditional credit scoring techniques work best for consumers with "thick" credit files. A credit scoring model developed by VantageScore for Equifax, Experian, and TransUnion uses machine learning to better evaluate borrowers for whom less information is available, according to a recent overview of the product.

¹⁵ See a paper by Julapa Jagtiani from the Philadelphia Fed and Catharine Lemieux from the Chicago Fed showing increasing reliance by Lending Club (a marketplace lender) on its own machine learning models. Similarly, see a paper by Tetyana Balyuk from Emory University and Sergei Davydenko from the University of Toronto discussing the use of machine learning by Prosper, another marketplace lender. One reason why marketplace lenders may find machine learning more useful than FICO is that they specialize in one type of credit, unsecured loans, whereas FICO scores are designed to be predictive over a wider set of consumer loan types.