Assessing bank soundness with classification techniques Christos Ioannidis (School of Management, University of Bath), Fotios Pasiouras (School of Management, University of Bath), Constantin Zopounidis (Department of Production Engineering and Management, Technical University of Crete, Greece)
University of Bath School of Management, Working Paper Series 2009.04
This working paper is produced for discussion purposes only. The papers are expected to be published in due course, in revised form and should not be quoted without the author’s permission.
University of Bath School of Management Working Paper Series School of Management Claverton Down Bath BA2 7AY United Kingdom Tel: +44 1225 826742 Fax: +44 1225 826473 http://www.bath.ac.uk/management/research/papers.htm
2009.01
2009.02
2009.03
2009.04
Androniki Apostolakou & Gregory Jackson
2009 Corporate Social Responsibility in Western Europe: An Institutional Mirror or Substitute?
Meryem DuygunFethi & Fotios Pasiouras
Assessing Bank Performance with Operational Research and Artificial Intelligence Techniques: A Survey
Klaus E. Meyer
Corporate Strategies under Pressures of Globalization: Globalfocusing
Christos Ioannidis, Fotios Pasiouras & Constantin Zopounidis
Assessing bank soundness with classification techniques
2
Assessing bank soundness with classification techniques Christos Ioannidis1, Fotios Pasiouras1*, Constantin Zopounidis2 1 2
School of Management, University of Bath, UK
Financial Engineering Laboratory, Department of Production Engineering and Management, Technical University of Crete, Greece
Abstract The recent crisis highlighted, once again, the importance of early warning models to assess the soundness of individual banks. In the present study, we use six quantitative techniques originating from various disciplines to classify banks in three groups. The first group includes very strong and strong banks; the second one includes adequate banks, while the third group includes banks with weaknesses or serious problems. We compare models developed with financial variables only, with models that incorporate additional information in relation to the regulatory environment, institutional development, and macroeconomic conditions. The accuracy of classification of the models that include only financial variables is rather poor. We observe a substantial improvement in accuracy when we consider the country-level variables, with five out of the six models achieving out-of-sample classification accuracy above 70% on average. The models developed with multi-criteria decision aid and artificial neural networks achieve the highest accuracies. We also explore the development of stacked models that combine the predictions of the individual models at a higher level. While the stacked models outperform the corresponding individual models in most cases, we found no evidence that the best stacked model can outperform the best individual model.
Keywords: Bank, Classification, Integration, Soundness,
*
Author for correspondence. E-mails:
[email protected] (C. Ioannidis),
[email protected] (F. Pasiouras),
[email protected] (C. Zopounidis)
3
1. Introduction Bank soundness is a central theme in the agenda of policy makers. After a relatively stable period between the Second World War and the early 1970s, several countries experienced a banking crisis over the last thirty years. Caprio and Klingebiel (2003) provide information on 117 systemic banking crises that occurred in 93 countries and 51 borderline and smaller banking crises in 45 countries since the late 1970s. These crises have both direct and indirect costs for the economy. First, as documented in Caprio and Klingebiel (2003) the costs for restructuring and recapitalisation can reach 10-20% and occasionally 40-55% of GDP (e.g. Argentina, Indonesia). Second, the crises have adverse effects on the efficient operation of the market economy due to the central role of banks as financial intermediates. Such adverse developments result in reduction in investment and consumption, increases in unemployment, and disturb the flow of credit to individuals and firms causing an overall economic slowdown. To reduce the likelihood of financial instability several countries have introduced prudential regulation frameworks, making banking one of the most heavily regulated industries. Possibly the most renowned example is the 1988 Basel Accord (i.e. Basel I) that established the capital adequacy requirements and Basel II that introduced additional pillars in relation to supervisory monitoring and market discipline. Furthermore, institutions like the International Monetary Fund and the World Bank have developed and promoted checklists of “best practices” for banking regulation and supervision in an attempt to achieve financial stability and economic development (Barth et al., 2001; 2005). However, the ongoing crisis that started in the US in 2007 revealed that despite these regulatory efforts, crises can still occur and spread rapidly around the world. The recent events generated a new round of discussions regarding the adequacy of the regulatory environment as well as numerous studies that attempt to explain the reasons behind the crises and how they could be avoided in the future. The recent crisis highlighted, once again, the importance of early warning models to forecast banking crises and assess the soundness of individual banks. The first strand of the literature that deals with early warning models examines systemic banking crisis at the country level (e.g. Davis and Karim, 2008). However, there are a number of problems associated with these studies. First, owing to data availability they focus on the 1980s and the 1990s, when we experienced the bulk of banking crises, their results may not be applicable to the modern financial environment. 4
Second, these studies concentrate on emerging market economies due to the higher frequency of crises in these economies in the past (Bell and Pain, 2000) whilst the current crisis started from developed countries like the US and the UK. In addition, there are notable differences in the dates attributed to the banking crises (Bell and Pain, 2000), making their empirical modelling problematic. Finally, dating is also problematic when there are successions of crises episodes as later crises can be extensions or re-emergences of previous financial distress rather than individual events (Caprio and Klingebiel, 1996; Davis and Karim, 2008). The second strand of the literature focuses on quantitative models that predict individual bank failures (e.g. Canbas et al., 2005; Lanine and Vander Vennet, 2006). These studies have the advantage that bank level can provide more rich datasets and additional information compared to aggregate data used in country studies. Nevertheless, a drawback that is also applicable to the country level studies, is that they concentrate on the classification of banks in two groups, failed and non-failed. Obviously, this classification of banks as “bad” or “good” reduces the usefulness of the model. Given the above, we model bank soundness at the bank level; however, we follow the approach of Gaganis et al. (2006) and classify banks in three groups. The first group contains very strong or strong banks; the second one contains adequate banks, while the third group contains banks with weaknesses or serious problems. By focusing on non-failed banks and distinguishing between these three groups the model can be useful in reducing the expected cost of bank failure, either by minimizing the costs to the public or by taking actions to prevent failure. Ravi Kumar and Ravi (2007) also mention that “As a bank or firm becomes more and more insolvent, it gradually enters a danger zone. Then, changes to its operations and capital structure must be made in order to keep it solvent” (p. 1). Obviously, the models developed in the present study can be used to monitor changes in the status of banks from one year to another and provide especially an early warning system when a bank gradually deteriorates from the group of strong banks to the one with serious problems. We differentiate our work from Gaganis et al. (2006) and other studies in three important respects. First, we compare, to the best of our knowledge, for the first time the classification accuracy of models that include indicators of the regulatory framework such as restrictions on bank activities and the three pillars of Basel II (i.e. capital requirements, supervisory monitoring, market discipline) with the accuracy of 5
models developed with financial variables only.1Second, we compare various advanced techniques such as artificial neural networks, multi-criteria decision aid, classification and regression trees, and nearest neighbours.Third, we investigate the use of a meta-classifier that combines the estimation of the individual models in an integrated model. Applications in other problems in finance such as the default of non-financial firms and approval of credit cards (e.g. Jo and Han, 1996; Doumpos, 2002; Doumpos and Zopounidis, 2007) have shown that this approach can provide promising results. However, these studies focus on the two-group classification and non-banking institutions. Our problem may be considered as more complex, both due to its three-group dimension as well as the dynamic nature of banking. Thus, the results obtained in past studies are not necessarily applicable to bank soundness, and we aim to examine the effectiveness of this approach in the present study. The rest of the paper is as follows: Section 2 presents the sample and the variables used in the study, while Section 3 outlines the classification techniques. Section 4 discusses the empirical results, and Section 5 concludes the study.
2. Sample and variables 2.1 Sample Following Gaganis et al. (2006) and Demirguc-Kunt et al. (2008) we measure bank soundness using financial strength ratings. As Demirguc-Kunt et al. (2008) mention, ratings provide a comprehensive measure of the ability of a bank to meet its obligations to depositors and other creditors and it can more a more accurate indicator of bank soundness than individual measures such as non-performing loans or Zscores. In the present study, we use the Fitch Individual bank ratings which are based on an A to E scale and represent Fitch’s view on the likelihood that the bank would fail, and therefore require support to prevent it from defaulting. As our purpose is not to explain or replicate the ratings of Fitch, but rather to use them as the basis for the development of a general model to assess the soundness of banks, we classify the banks in three broad groups. The first consists of banks with ratings A and B, the second with banks with rating C, and the third with banks rated D and E. Hence, 1
Demirguc-Kunt et al. (2008) and Pasiouras et al. (2006) have also considered various regulations in their analyses, however, their studies are more descriptive in the sense that they use regression techniques to reveal the impact of regulations on bank soundness and they do not develop classification models.
6
banks in Group 1 can be characterized as very strong or strong banks, banks in Group 2 can be characterized as adequate banks, and those in Group 3 can be characterized as banks with weaknesses or serious problems.2 The dataset consists of 944 banks from 78 countries with available data and Fitch individual bank ratings in Bankscope database. The ratings were obtained in end 2008, while the bank specific characteristics correspond to end 2007 or March 2008 depending on the date of publication of the annual reports. The distribution of banks in the three groups is as follows: 447 (Group 1), 275 (Group 2), and 222 (Group 3). To ensure the proper estimation and validation of the models, we randomly select two thirds from each group for training purposes (i.e. a total of 629 banks) and we keep the remaining banks for out-of-sample evaluation (i.e. total of 315 banks). The definitions of Fitch, along with the coding used in the present study and the number of banks in the training and holdout samples appear in Table 1.
[Insert Table 1 Around Here]
2.2. Variables Credit agencies, researchers, and bank regulators tend to evaluate banks’ performance on the basis of the CAMEL model that stands for the acronyms of Capital, Asset quality, Management, Earnings, and Liquidity. We follow the same approach and select financial variables that proxy the four of the five dimensions, as well as size.3 As mentioned in the introduction, we also use regulatory and other country-level variables. We discuss these variables below, while additional information on the regulatory variables is available in Appendix A.
2
We are not interested in replicating all the ratings of Fitch for two reasons. First, this approach allows us to avoid (at least to some extent) problems associated with the timely adjustment of ratings. For instance, a delay in a downgrade from A to B or from D to E would have no impact in assessing the overall soundness of a bank as we do. Furthermore small errors of judgment in the assignment of ratings such as rating an A/B or B bank as A would also had no impact on our model. Obviously, large errors of judgment could make a difference but we have no reason to believe that Fitch would classify let us say an E bank as A and visa versa. Second, the heterogeneous sample used in our study, consisting of 944 banks from 78 countries could have an adverse effect on the classification ability of the model. As discussed in the introduction, the developed model could be useful in several occasions. Furthermore, it could be useful in assessing the overall soundness of banks not rated by Fitch. 3 Management has not been included in the analysis due to its qualitative nature and the subjective analysis that is required. We also considered the inclusion of off-balance-sheet items, however due to missing values for around 25% of the observations this variable was excluded from the analysis. It would also be interesting to include variables related to corporate governance and internal control, however such data were not available in our case. We hope that future research will improve upon this.
7
2.2.1. Bank-level variables The importance of capital strength has been emphasized and became subject to regulation by the Basel Committee, in the form of capital requirements ratios. There is a number of reasons for this. For example, capital serves as the last line of defence against the risk of bank’s insolvency, as any losses a bank suffers could be potentially written off against capital. Even in the case that insolvency becomes unavoidable, capital protects to some degree depositors, creditors and investors (Le Bras and Andrews, 2004). Furthermore, as mentioned by Theodore (1999) capital allows the leveraging of a bank’s growth and diversification, and a tight solvency position would be an obstacle to do so. Therefore, we use the equity to total assets ratio (EQAS) as an indicator of capital strength.4 Profits allow companies to implement their investment strategies and grow, and unsurprisingly past studies have found profitability to be negatively related to the probability of failure (e.g. Wheelock and Wilson, 2000; Lanine and Vander Vennet, 2006). However, profits are not only affected by the ability of managers to generate revenues but from their ability to manage costs as well. In fact, bank managers perceive expenses management as being more directly under their control, devoting particular emphasis on cost cutting when necessary. Therefore, we use the return on average assets (ROAA) to measure profitability and the cost to income ratio as an indicator of the efficiency in expenses management (COST). Bad asset quality may have a negative impact on bank profitability, by reducing interest income and by increasing provisioning costs, thus decreasing net profits.5 Empirical evidence suggests that banks with lower asset quality are more likely to fail (Wheelock and Wilson, 2000). We therefore use the loan loss provisions to net interest revenue ratio a measure of asset quality.6 Because provisions depend on the probability of loans becoming non-performing, higher provisions usually indicate higher probability of non-performing ratios and lower asset quality.
4
Probably the employment of a risk-weighted ratio such as the Tier 1 ratio would be more appropriate. However, due to a very large number of missing values for Tier 1, we rely on EQAS that is considered one of the basic ratios whose use dates back to the 1900s, and is still being used in many recent studies in banking (Golin, 2001). 5 Loan loss provisions, which include general provisions applied on a statistical basis to all loans and specific provisions designed to account for the probability of losses among problem loans, are amounts set aside from earnings to adjust for the potential decline in the value of the bank’s loan assets. 6 One can easily argue that loan loss reserves or problem loans are more adequate variables of loan asset quality. Unfortunately, it was not possible to consider these items due to large numbers of missing values.
8
As the recent crisis revealed, liquidity can become a very important problem for banks especially when there is reluctance for interbank borrowings and depositors demand a higher rate for their savings. In the worlds of Golin (2001): “it is critical that a bank guard carefully against liquidity risk-the risk that it will not have sufficient current assets such as cash and quickly saleable securities to satisfy current obligations e.g. those of depositors – especially during times of economic stress”. We use the liquid assets to customer and short term funding ratio (LIQ) that shows the percentage of customer and short term funding that could be met if they were withdrawn suddenly.7 Bank size can also influence bank soundness due to differences between small and large banks in terms of credit constraints, diversification, and depth in management (Lennox, 1999; Falkenstein et al., 2000). We measure bank size using the logarithm of total assets (LOGAS).
2.2.2. Regulatory variables Capital requirements have received increase attention in recent years, with the introduction of Basel II. As discussed above, bank capital provides enhanced security for depositors and is the focus of the regulatory authorities. Nevertheless, some studies indicate that capital requirements may actually increase risk-taking behaviour (e.g. Koehn and Santomero, 1980; Besanko and Kanatas, 1996; Blum, 1999) and others provide mixed results regarding the contribution of capital requirements to the bank’s risk taking. For instance, Kendall (1992) suggests that higher capital requirements may cause riskier bank behaviour at some points in time, but do not imply a trend toward a riskier banking system. Beatty and Gron (2001) indicate that capital regulatory variables have significant effects for low-capital banks but not necessarily for other banks. Barth et al. (2004) indicate that while higher stringent capital requirements are associated with fewer non-performing loans, capital stringency is not robustly linked to banking crises when controlling for other supervisory – regulatory policies. An index, CAPRQ, that accounts for both initial and overall capital stringency is used as proxy for capital requirements. This index takes values between 0 and 8 with higher values indicating higher stringency.
7
Liquid assets refer to short-term assets that can be easily converted into cash, such as cash itself, deposits with the central bank, and interbank deposits among others.
9
The second pillar of Basel II discusses the main principles of the supervisory review process. One important point is the ability of regulators to intervene at an early stage and prevent capital falling below the minimum levels or request disclosures depends on their power. Indeed, Fernandez and Gonzalez (2005) report that in countries with low accounting and auditing requirements a more stringent disciplinary capacity of supervisors over management action appears to be useful in risk reduction. However, powerful supervision can also be related to corruption or impede bank operations (Barth et al., 2004). For instance, Barth et al. (2002) show that more powerful government supervisors are associated with higher levels of non-performing loans, while Barth et al. (2003) find that official government power is particularly harmful to bank development in countries with closed political systems. We use SPOWER that is an index of the power of supervisory agencies indicating the extent to which they can take specific actions against bank management and directors, shareholders, and bank auditors. SPOWER can take values between 0 and 14 with higher values indicating higher supervisory power. The third pillar of Basel II and other regulatory proposals (e.g. European Shadow Financial Regulatory Committee, 2000) point to the contribution, that policy makers identify as desirable, of the market participants in the supervision of banks as a complement to capital requirements and the supervisory process. De Ceuster and Masschelein (2003) and Hamalainen et al. (2005) review various studies discussing that the potential of risk-monitoring through market discipline can be achieved through increased disclosure, supplementary information from credit agencies, the creation of mandatory subordinated debt holders, and the absence of bailout concepts.8 Therefore, we use MDISC that indicates the degree to which banks are 8
Barth et al. (2004) find that regulations that encourage and facilitate private monitoring of banks are associated with greater bank development and lower net interest margins and non-performing loans, although they find no evidence that regulations that foster private monitoring reduce the likelihood of suffering major banking crises. Nevertheless, Demirguc-Kunt et al. (2008) find that countries where banks have to report regular and accurate financial data to regulators and market participants have sounder banks. Similarly, Fernandez and Gonzalez (2005) report that accounting and auditing systems are complements for minimum capital requirements and substitutes for restrictions on bank activities and official discipline, suggesting that accounting and auditing systems can be effective devices to counteract tendencies for firm risk-taking associated with bank safety nets. Calomiris (1999) argues that requiring banks to maintain minimum ratios of subordinated debt relative to insured debt or relative to risky assets while regulating other features of the subordinated debt (e.g. maturity, maximum allowable yield) imposes market discipline and limits risk-taking. Finally, deposit insurance schemes may encourage excessive risk-taking behaviour (Merton, 1977; Demirguc-Kunt and Kane, 2002). The main reason is that depositors will have no incentives to monitor bank managers, who can take on riskier investments (or reduce bank’s capital) knowing that depositors are protected in the event of a failure. Empirical evidence confirms that an explicit deposit insurance scheme, in the absence of strong
10
forced to disclose accurate information to the public and whether there are incentives to increase private monitoring (e.g. absence of deposit insurance, subordinated debt). Higher values of MDISC indicate the potential for enhanced market discipline. Another favourite policy making tool is restrictions on bank activities. Barth et al. (2004) outline several theoretical reasons for restricting bank activities (e.g. increased risk exposure, creation of complex and powerful banks that will be difficult to monitor) as well as alternative reasons for allowing banks to participate in a broad range of activities (e.g. increase in the franchise value of banks, opportunities for income diversification). The results of their study indicate that restricting bank activities is negatively associated with bank development and stability. Barth et al. (2001) and Barth (2008) also find that greater regulatory restrictions on bank activities are associated with higher probability of suffering a major banking crisis and bank fragility, respectively. In contrast, Fernandez and Gonzalez (2005) find that stricter restrictions on bank activities are effective at reducing risk, although the authors indicate that restrictions are effective at controlling risk only when information disclosure and auditing requirements are poorly developed. We use ACTRS as an indicator of the level of restrictions on banks’ activities. It is determined by considering whether securities, insurance, real estate activities, and ownership of nonfinancial firms is unrestricted, permitted, restricted or prohibited. ACTRS can take values between 0 and 4 with higher values indicating higher restrictions.
2.2.3. Other country-level variables Banks will obviously be affected by the overall environment of the country in which they operate, with a number of aspects relating to the environment having an important influence on their soundness. For example, Barth et al. (2004) find that better-developed private property rights and greater political openness mitigate the negative association of moral hazard and bank fragility. We use PRIGHTS an as indicator of the protection of property rights.9 As discussed in Demirguc-Kunt and Detragiache (1998) financial liberalization may increase banking sector fragility due to increased opportunities for excessive risk-taking and fraud. Therefore, following, banking regulations tends to increase the probability of banking crises (Demirguc-Kunt and Detragiache, 2002). 9 PRRIGHTS is the property rights index from Heritage Foundation. It indicates the ability to accumulate private property, secured by clear laws that are fully enforced by the stage. It can take values between 0 (i.e. private property is outlawed and all property belongs to the state) and 100 (i.e. private property is guaranteed by the government).
11
Demirguc-Kunt and Detragiache (1998) and Davis and Karim (2008) among others, we use the ratio of domestic credit to private sector over GDP (CRGDP) to proxy for financial liberalization. Several studies document a negative relationship between real GDP growth and the probability or hazard rate of banking crisis (e.g. Noy, 2004; Davis and Karim, 2008; Evrensel, 2008). Davis and Karim (2008) mention that GDP growth cannot only reduce non-performing loans it can also delay banking crises due to procyclicality. Following these studies, we use the real GDP growth (GDPGR) as an overall indicator of economic growth. A large body of the literature supports the view that increased competition in the deposit market decreases bank charter value and induces bank managers to increase risk (e.g. Keeley, 1990). Gehrig (1998) focuses on the loan market and shows that competition decreases screening efforts, thus worsening the quality of the loan portfolio. However, Boyd and de Nicolo (2005) challenge the above views and show that banks become more risky as their markets become more concentrated. The econometric evidence is mixed. Liang (1989) reports that market concentration and bank risk are positively related. Similarly, De Nicolo et al. (2004) find that highly concentrated banking markets faced levels of systemic risk potentially higher than less concentrated markets during the 1993-2000 period, and this relationship strengthened between 1997 and 2000. In contrast, Beck et al. (2006) report that more concentrated national systems are subject to a lower probability of systemic banking crises. As a rough measure of competition, we use CONC that is the percentage of assets held by the three largest commercial banks relative to the total assets of the commercial banking sector within the country. Finally, to assess the liquidity of the banking sector in total we use the ratio of bank credit to bank deposits (CRDEP). The ratio shows the percentage of deposits that is tied up in loans. Therefore, higher ratios may indicate that banks have less of a cushion to fund their growth and to protect themselves against a sudden recall of their funding. We now proceed to the estimation of seven models, parametric and non-parametric, that allocate individual banks to the appropriate group using the information sets defined above. The initial classification utilises a randomly selected sample from each group. The final third is used to establish the validation of the estimated models. We
12
begin by providing a brief description of the classification techniques and we then present the estimation and validation results.
3. Classification techniques 3.1. UTilités Additives DIScriminantes (UTADIS) The UTADIS approach develops an additive utility function that is used to score the banks and decide upon their classification. The utility function has the following general form: n
U ( x ) = ∑ wi u i′ ( g i ) ∈ [0,1] i =1
where wi is the weight of criterion gi (the criteria weights sum up to 1) and u i′ ( g i ) is the corresponding marginal utility function normalized between 0 and 1. The marginal utility functions provide a mechanism for decomposing the aggregate result (global utility) in terms of individual assessment to the criterion level. To avoid the estimation of both the criteria weights and the marginal utility functions, it is possible to use the transformation u i ( g i ) = wi u i′ ( g i ) . Since u i′ ( g i ) is normalized between 0 and 1, it becomes obvious that u i ( g i ) ranges in the interval [0, wi]. In this way, the additive utility function is simplified to the following form which provides an aggregate score U ( x ) for each bank along all criteria (i.e. variables): n
U ( x) = ∑ u i ( g i ) ∈ [0,1] i =1
To classify banks in their original group, it is necessary to estimate the utility thresholds u1 , u 2 ,..., u q −1 , defined in the global utility scale (i.e. between 0 and 1) that distinguish the set of q ordered groups (C1 , C 2 ,...C q ) . Comparing the global utilities with the utility thresholds, the classification is achieved by using the relations:
⇒ x ∈ C1 . . . . . . . . . . . . . . . . . . . . .................... u k ≤ U ( x) < u k −1 ⇒ x ∈ C k .................... . . . . . . . . . . . . . . . . . . . . U ( x ) < u q −1 ⇒ x ∈ C q
U ( x ) ≥ u1
13
The estimation of the additive value function and the cut-off thresholds is performed through linear programming techniques. The objective of the method is to develop the additive value model so that the above classification rules can reproduce the predetermined grouping of the banks as accurately as possible. Therefore, a linear programming formulation is employed to minimize the sum of all violations of the above classification rules for all the banks in the training sample. Detailed description of the mathematical programming formulation can be found in the work of Doumpos and Zopounidis (2004).
3.2. Artificial Neural Network (ANN) The applications of ANN in finance problems started in the 1990s and over the years gained popularity as an alternative to traditional statistical techniques due to their flexibility and their non-parametric nature.10 There are numerous NN architectures, learning methods and parameters available to the operational researcher (see Smith and Gupta, 2000). In the present study we develop a feed-forward ANN model. Typically, computing is performed by a collection of processing elements (i.e. neurons), connected in several layers. The basic computational structure consists of input, hidden and output layers. The input layers represent the variables and output layer represents the output data. All layers between the input and output layers are known as hidden layers. The interconnecting weight values are adjusted and updated during the training stage to achieve minimal overall training error between the actual and calculated output vectors. After the training has been completed the ANN can be used for prediction (i.e. out of sample) purposes.
3.3. Classification And Regression Trees (CART) In the case of CART (Breiman et al., 1984), instead of developing classification functions or network architectures a binary decision tree is developed. The main idea behind CART is simple: at each brand node, the best splitting value for 10
One of the early applications in bank bankruptcy can be found in Tam (1991). More recent applications deal among others with the forecasting of exchange rates (Zhang and Hu, 1998), the prediction of mutual funds performance (Indro et al., 1999), forecasting of option price (Yao et al., 2000), auditing (Gaganis, 2009), and the evaluation of consumer loans (Malhotra and Malhotra, 2003). Wong and Selvi (1998) identify 66 neural network applications in finance during 1990-1996. In a more general review Wong et al. (2000) identify 302 papers with applications in various domains of business between 1994-1998.
14
each independent variable is determined, and the sample split is based on the best of these values. This can be accomplished through a set of if-then split conditions that permit accurate prediction or classification of cases. For each parent node, the left child node corresponds to the points that satisfy the condition, and the right child node corresponds to the points that do not satisfy the condition. Given the hierarchical nature of classification trees, these splits are selected one at a time, starting with the split at the root node (i.e. the top decision node), and continuing with splits of resulting nodes until splitting stops and the child nodes, which have not been split, become terminal nodes (i.e. points on the tree beyond which no further decisions are made). In general terms, the split at each node will be found that generate the greatest improvement in predictive accuracy. Various split criteria exist such as the Gini, Symgini, Twoing, Ordered Twoing, Maximum Deviance. The Gini index is the most commonly adopted measure in the case classification problems and it was also used in the present study. This criterion defines the best splitting value as the one resulting in the smallest node heterogeneity. Consequently, it reaches a value of zero when only one class is present at a node. With priors estimated from class sizes and equal misclassification costs, the Gini index is calculated as the sum of products of all pairs of class proportions for classes present at the node. Obviously splitting could continue until all cases are correctly classified. However, this would probably result in “over-fitting” in a given sample with a reduced ability to classify accurately new observations. One way to control splitting is to continue until all terminal nodes are pure or contain no more than a specified minimum number of cases or objects. An alternative strategy is to continue until all terminal nodes are pure or contain no more cases than a specified minimum proportion of the sizes of one or more classes. The tree obtained from the above procedure can then be pruned to obtain a final tree that has close to the minimum estimated error rate. A k-fold cross validation is usually employed to perform pruning. Breiman et al. (1984) provides an extended description of this method with discussion of above issues, including theory of binary tress, splitting rules, etc.
3.4. k- Nearest Neighbours (k-NN) Nearest Neighbours is a non-parametric estimation method that classifies an object (i.e. bank) to the class of its nearest neighbour in the measurement space using some 15
kind of distance measure with the Euclidean being the most commonly used and this is employed in the present study. The modification of the nearest neighbour rule, the k-nearest neighbour (kNN) method that is employed in the present study, classifies an object (i.e. firm) to the class (i.e. Group 1, Group 2 or Group 3) more heavily represented among its k nearest neighbours. Assuming a bank x described by the feature vector < g 1 ( x ), g 2 ( x ),..., g m ( x) ) > where g r ( x ) is used to denote the values of the r th characteristic of bank x , the distance between two instances xi and x j is estimated as follows:
d (xi , x j ) ≡
∑ (g (x ) − g (x )) m
2
r
i
r
j
r =1
Then, the algorithm for approximating a discrete-valued function of the form f : ℜ n → C , where C is a finite set of classes {C1, C 2 ,..., C q }proceeds as follows: Step 1: For each training example (i.e. bank) < x, f ( x ) > , add the bank to the list of training examples. Step 2: Given a bank x to be classified, let x1 , x 2 ,..., x k denote the k instances from the training examples that are nearest to x. ^
Step 3: Return f ( x ) ← arg max c∈C
k
∑ δ (c, f (x )) , i
where δ (a, b) ) = 1 if a = b and
i =1
where δ (a, b) ) = 0 otherwise. ^
Thus, the algorithm returns the value f ( x ) as an estimate of f ( x ) , which is the most common value of f among the k training examples nearest to x.
3.5. Ordered Logistic Regression (OLR) Logistic regression has also been used in several classification studies in finance, including bankruptcy prediction and credit rating. In the present study, considering the ordinal nature of the dependent variable, we use an ordered logistic model. We discuss very briefly the ordered logit model below while more detailed discussions can be found in Powers and Xie (2000) and Borooah (2001).
16
The ordered model requires the estimation of a vector of attributes’ coefficients b and a vector of constant terms α. These parameters are used to specify the probability Pkj that an alternative xj (e.g. bank) belongs into group Ck. The constant terms are defined such that αk-1 > αk-2
… >
α2 (α1 =0). The parameters’
estimation is performed using maximum likelihood. Thus, a model of the following form is estimated:
P1 j = F ( a1 + g 'j b ) P2 j = F ( a2 + g 'j b ) − F ( a1 + g 'j b ) . . . Pkj = 1 − ( P1 j + P2 j + ... + Pk −1, j )
3.6. Multiple Discriminant Analysis (MDA) Discriminant analysis was introduced in the field of finance by Altman (1968) who used it to develop a bankruptcy prediction model. This method seeks to obtain a linear combination of the independent variables whose objective is to classify observations into mutually exclusive groups as accurately as possible by maximising the ratio of among-groups to within-groups variance. In the simple case of a two-group problem, we fit a linear equation of the following type for each firm: D = a 0 + b1 x1 + b2 x 2 + ... + bm x m
where D is the score for a given firm, a 0 is the intercept term and b1 to bm are regression coefficients. In a three-group problem, like ours, multiple discriminant analysis (MDA) is used to develop two discriminant functions. The first discriminates between Group 1 and Groups 2 and 3 combined, and the second function discriminates between Group 2 and Group 3.
3.7. Stacked generalization The stacked generalization approach was proposed by Wolpert (1992) and it allows the combination of individual classification models (i.e. base models) in an integrated model known as meta-level classifier or stacked model. This stacked model receives
17
as input the output predicted by the base models and it then provides the final classification decision. It can be developed by any one of the initially considered classification methods and in general, there are four steps for the development and validation of a stacked classification model: Step 1: A re-sampling technique is initially used to split the training sample T into p partitions and form sub-samples Ts1 and Ts 2 ( s = 1, 2,..., p ) . Wolpert (1992) proposed the use of leave-one-out fold cross validation technique, but as Doumpos (2002) and Doumpos and Zopounidis (2007) mention other re-sampling techniques such as bootstrapping or k-fold cross validation can be used. In the present study, we use a ten-fold cross validation. Step 2: For each partition s = 1, 2, ..., p , the sub-sample Ts1 is used to develop a classification model f ls (base model) using method l
( l = 1, 2, ..., L ) .
Each base
model is then used to assign an output (e.g. score, distance measure, probability, group assignment) for each alternative (i.e. bank) in validation sub-sample Ts 2 and decide for its classification. Step 3: When the re-sampling method has been completed and all p partitions have been considered, the assignments for the alternatives in every validation sub-sample Ts 2 are used to create a new training sample for the development of the stacked
generalization model. Step 4: When the stacked model has been developed through steps 1-3, it can be used to classify any new objects (i.e. banks). Thus, all the methods that are integrated in the stacked generalization model are used to assign an output for the object. Specifically, the assignment by method l is obtained by model Fl using the initial training sample T . The L different output assignments c1 , c2 , ..., cL , determined by the models F1 , F2 , ..., FL developed by all the L methods, are then combined by the developed
stacked model to obtain a final classification decision.
4. Results Table 2 presents descriptive statistics for both the training and holdout/validation samples. Table 3 presents the correct classification accuracies obtained from the base models when the financial variables are the only inputs. Panel A corresponds to the
18
training sample and Panel B to the holdout sample.11 Since the results in the training sample are usually upwards biased, we focus on the classification accuracies in the holdout sample. [Insert Tables 2 and 3 Around Here]
The accuracy of classification ranges between 51.58% and 59.51%. The highest average accuracy achieved by UTADIS, is mainly due to the ability of the model to classify very well banks belonging in Group 3. The latter is of particular interest because these banks are the ones with the highest risk. ANN performs well in classifying banks from both Group 1 and Group 3, however its very poor performance in the case of Group 2 ranks the method lower than UTADIS on average. MDA and OLR achieve quite similar accuracies on average, although these are achieved in a different way. MDA returns quite balanced accuracies between Groups 1 and 3, while in the case of OLR we observe a trade-off between Groups 1 and 3. Finally, it is clear that all methods fail to classify corrects banks in Group 2, a finding consistent with past studies (e.g. Gaganis et al., 2006; Pasiouras et al., 2007). As discussed in Gaganis et al. (2006), an explanation is that the value of the characteristics of such banks overlaps with those belonging to banks in the lower band of Group 1 and/or the upper band Group 3, making their correct classification a more difficult task. Table 4 presents corresponding accuracies when we add the regulatory and other country-specific variables in the models. The average classification accuracies now range between 64.60% and 78.45%. Thus, we observe a significant improvement in all cases, which ranges between 13.02% (k-NN) and 19.30% (MDA). Furthermore, it should be emphasized that the inclusion of additional information increases the classification accuracies in the case of Group 2 by up to 39.13% (ANN). This means, 11
The development of the models requires the selection of various parameters such as the appropriate number of nearest neighbors in k-NN, the value of the sub-intervals and other parameters in UTADIS, the number of neurons and layers in ANN, and so on. Improper selection of these parameters could result in over-fitting or under-fitting of the models. The selection of these parameters can be performed by splitting the training sample in estimation and validation subsets or by using cross-validation within the training sample. In the present study, a 10-fold cross-validation is used. More specifically, the 629 banks of the training sample are randomly divided into 10 almost equal datasets. The model is developed using 9 data sets, the 10th being used to obtain initial estimates of the error rates. The process is repeated ten times, each time using a different set for validation. Finally, the results of the 10 iterations are then averaged to calculate the error rate. The performance of the model in the validation subset is used to access its generalization ability. Then, using the selected parameters the model is estimated using the whole training sample (i.e. 629 banks) and it is tested in the holdout sample of the 315 unseen cases (i.e. banks). All the specified parameter estimates are available from the authors upon request.
19
that after considering the regulatory environment, and other country-specific characteristics, the models are in position to distinguish more accurately banks being in an intermediate situation from the ones belonging in the other two groups. Equally important is the observation that this result is not achieved as a trade-off with accuracies in other groups. Overall, UTADIS and ANN achieve again the highest average classifications and they manage to obtain quite high accuracies in all three groups. Nevertheless, five out of the six methods, manage to classify correctly more than 70%, on average, the banks in the holdout sample. To examine the statistical significance of the results, a ttest was performed to test the hypothesis that the UTADIS model outperforms the remaining models. We find evidence of statistically significant differences between UTADIS and three techniques (OLR, k-NN, CART) but not the remaining two (i.e. ANN, MDA).12
[Insert Table 4 Around Here]
As discussed in Doumpos and Zopounidis (2007), the successful implementation of the stacked generalization approach depends on the methods that are combined. That is, integrating methods which provide the same group classification will not provide any improvement in the obtained classification accuracies as the stacked model is not fed with additional information. One potential way to achieve the flow of the required information is to use methods originating from different disciplines, as we do in the present study. The correlation coefficients in Table 5 confirm that there are important differences in the group assignments among the six methods. Table 6 presents the classification accuracies obtained from the stacked models. An appropriate comparison (e.g. MDA individual vs MDA stacked) shows that in four out of the six cases, the stacked models provide higher classification accuracies than the corresponding individual models. Nevertheless, the best stacked model obtained through UTADIS cannot outperform the best individual model (also obtained through UTADIS). Furthermore, the differences between the individual models and the
12
The p-values are: 0.708 (UTADIS vs ANN), 0.003 (UTADIS vs CART), 0.000 (UTADIS vs k-NN), 0.041 (UTADIS vs OLR), 0.120 (UTADIS vs MDA).
20
stacked models are not statistically significant.13 Therefore, it seems that despite the differences in the assignments of banks in the three groups, it is not always the case that model combination can lead to superior results. One potential explanation is that as mentioned in the introduction, our problem is possibly more complex, due to the classification in three groups and the various distinguishing characteristics of the banking industry.
[Insert Tables 5 and 6 Around Here]
To be more informative about the performance of the model that achieves the highest average classification accuracy (i.e. individual UTADIS) we present in Table 7, details of the out-of-sample assignment of banks to the three groups.14 The results show that there are not many serious misclassifications in terms of classifying banks from Group 1 into Group 3 and the opposite. Specifically, only 1 out of the 74 banks actually belonging in Group 3 is misclassified in Group 1, while none of the banks actually belonging in Group 1 are misclassified in Group 3. In other words, the probability of misclassifying a bank with serious problems into the group of strong banks is very low. As mentioned before, the lower classification accuracy is observed in the case of Group 2, and we now observe that 18 banks are misclassified in Group 1 and another 13 in Group 3.
[Insert Table 7 Around Here]
5. Conclusions The current crisis has demonstrated, in the worst possible way, that banks play a central role in the economy and that their ‘well-being’ is of crucial importance for various stakeholders. In contrast to past crises, the current crisis began in developed countries and their economies have been influenced adversely. Unemployment has increased substantially, investments and consumption have decreased and all the governments are looking at possible ways to exit the crisis. Consequently, several of 13
The p-values are: 1.000 (ANN), 0.056 (CART), 0.784 (DA), 0.250 (k-NN), 1.000 (UTADIS), 0.371 (OLR). 14 To preserve space we present such details about the best model only. Similar information for the remaining models is available from the authors upon request.
21
them have already announced fiscal initiatives, that include in all but name the partial nationalisation of several banks, that increase substantially the debt to GDP ratio. Such developments illustrate the need for early warning models that will help to monitor banks and avoid similar problems in the future. Using a sample of 944 banks from 78 countries, we develop various quantitative models and examine their accuracy in classifying banks in three groups. The first group includes very strong and strong banks; the second one includes adequate banks, while the third group includes banks with weaknesses or serious problems. Our base models include basic financial variables only. Then, we develop additional models that incorporate country-specific information that relate to the regulatory framework, institutional development, banking sector characteristics, and basic macroeconomic conditions. We record important improvement in the classification accuracies when we consider these country-level variables. The best two models in terms of the average classification accuracy are developed with UTADIS and artificial neural networks. All but one of the remaining models achieved an average accuracy above 70% in the holdout sample. Finally, we investigated the development of stacked models that combine the predictions of the individual models at a higher level. While the stacked models outperformed the corresponding individual models in most cases, we find no evidence that the best stacked model can outperform the best individual model. We believe that models like the ones developed in the present study, could be useful in assessing the soundness of banks by identifying “red flags” that substantially differ from the norms of the industry, and by monitoring banks as their performance deteriorates from the group of “good” banks to the one of “bad” banks. For the purposes of the ‘prudential’ supervision of banks the UTADIS model provides a very useful tool as it does not provide for a underestimate of the ‘problem’ banks, since only one member of this group is classified in group 1. The classification captures 87% of the members of this group with a small addition of 13 banks that belong to the group above it.. More precise identification of such banks, mostly those in group 2 can provide further information for the assessment of sectoral risk. It is such risk assessment that prudent regulatory authorities require to adopt measures that contribute to financial stability. However, as a final remark it should be emphasized that our models cannot and should not replace the judgment of experienced bank supervisors, rather they 22
could assist them by providing objective information that can be useful in assessing the status of individual banks.
References Altman E.I. (1968), Financial ratios, discriminant analysis and the prediction of corporate bankruptcy, Journal of Finance, 23, 589-609. Barth J.R., Caprio G. Jr. Levine R., (2001), Financial regulation and performance: Cross-country evidence, Central Bank of Chile Working Papers, No. 118, November. Barth J.R., Caprio G. Jr., Levine R., (2005), Rethinking Bank Regulation, Bank of England and Bank of Slovenia Conference on Small Countries, Big Markets Achieving Financial Stability in Small Sophisticated Markets, Ljubljana, Slovenia, 28 October. Barth J.R., Caprio G. Jr., Levine R., (2008). Bank regulations are changing: But for better or worse? World Bank, Policy Research Working Paper 4646. Barth J.R., Caprio G. Jr., Levine R., (2003), Bank Regulation and Supervision: Lessons from a New Database, In Macroeconomic Stability, Financial Markets, and Economic Development, Jose Antonio Murillo Garza (Ed), Mexico, City: Banco de Mexico. Barth, J.R., Caprio G. Jr, Levine, R., (2004), Bank regulation and supervision: what works best? Journal of Financial Intermediation, 13, 205-248. Barth J.R., Dopico L.G., Nolle D.E., Wilcox J.A., (2002), Bank Safety and Soundness and the Structure of Bank Supervision: A Cross-Country Analysis, International Review of Finance, 3 (3/4), 163-188. Beatty A.L., Gron A., (2001), Capital, Portfolio, and Growth: Bank Behavior Under Risk-Based Capital Guidelines, Journal of Financial Services Research 20, 5-31. Beck T., Dermirguc-Kunt A., Levine R., (2006), Bank concentration, competition and crises: First results. Journal of Banking and Finance, 30, 1581-1603. Bell J., Pain D., (2000), Leading indicator models of banking crises – a critical review, Financial Stability Review, December, 113-129. Besanko D., Kanatas G., (1996), The Regulation of Bank Capital: Do Capital Standards Promote Bank Safety? Journal of Financial Intermediation, 5, 160183.
23
Blum J., (1999), Do capital adequacy requirements reduce risks in banking?, Journal of Banking and Finance, 23, 755-771. Borooah V.K., (2001), Logit and Probit: Ordered and Multinomial Models, Series: Quantitative Applications in the Social Sciences, SAGE Publications. Boyd J., De Nicolo G., (2005), The theory of bank risk taking and competition revisited. Journal of Finance, 60, 1329-1343. Breiman L., Friedman J.H., Olshen R.A, Stone C.J. (1984), Classification and Regression Trees, Pacific Glove, Wadsworth Inc. Calem P., Rob R., (1999), The Impact of Capital-Based Regulation on Bank RiskTaking, Journal of Financial Intermediation 8, 317-352. Calomiris C.W., (1999), Building an incentive-compatible safety net, Journal of Banking and Finance, 23, 1499-1519. Canbas S., Cabuk A., Kilic S.B., (2005), Prediction of commercial bank failure via multivariate statistical analysis of financial structures: The Turkish case, European Journal of Operational Research, 166, 528-546. Caprio G., Jr., Klingebiel D., (1996), Bank Insolvencies: Cross-country Experience, World Bank Policy Research Working Paper 1620, July. Caprio G., Klingebiel D., (2003), Episodes of Systemic and Borderline Financial Crises, January, World Bank, available at http://go.worldbank.org/5DYGICS7B0 Davis E.P., Karim D., (2008), Comparing early warning systems for banking crises, Journal of Financial Stability, 4, 89-120. De Ceuster, M.J.K., Masschelein N., (2003), Regulating banks through market discipline: A survey of the issues, Journal of Economic Surveys, 17, 749-766. De Nicolo G., Bartholomew P., Zaman J., Zephirin M., (2004), Bank consolidation, internationalization, and conglomeration: Trends and implications for financial risk. Financial Markets, Institutions & Instruments, 13, 173-217. Demirguc-Kunt A., Detragiache E., (1998), The Determinants of Banking Crises in Developing and Developed Countries, IMF Staff Papers, 45, 81-109. Demirguc-Kunt A., Detragiache E., Tressel T., (2008), Banking on the principles: Compliance with Basel Core Principles and bank soundness, Journal of Financial Intermediation, 17, 511-542. Demirguc-Kunt A., Detragiache E., (2002), Does deposit insurance increase banking system stability? An empirical investigation. Journal of Monetary Economics 49, 1373-1406. 24
Demirguc-Kunt A., Kane E.J., (2002) Deposit insurance around the world: Where does it work? Journal of Economic Perspectives 16, 175-195. Doumpos M., Zopounidis C., (2004), Developing sorting models using preference disaggregation analysis: An experimental investigation, European Journal of Operational Research, 154, 585-598. Doumpos M., Zopounidis C., (2007), Model combination for credit risk assessment: A stacked generalization approach, Annals of Operations Research, 151, 289-306. Doumpos M. (2002), A Stacked Generalization Framework for Credit Risk Assessment, Operational Research: An International Journal, 2, 261-278. European Shadow Financial Regulatory Committee, 2000. Internal ratings, capital standards and subordinated debt. Statement No. 7, February. Evrensel A.Y., (2008), Banking crisis and financial structure: A survival-time analysis, International Review of Economics and Finance, 17, 589-602. Falkenstein E., Boral A., Carty L.V., (2000), RiskCalc for Private Companies: Modeling Methodology”, Moody’s KMV Company, May. Fernandez A.I., Gonzalez F. (2005), How accounting and auditing systems can counteract risk-shifting of safety nets in banking: Some international evidence, Journal of Financial Stability 1, 466-500, (2005). Gaganis C., (2009), Classification Techniques for the Identification of Falsified Financial Statements: A Comparative Analysis, Working Paper, Financial Engineering Laboratory, Technical University of Crete. Gaganis C., Pasiouras F., Zopounidis C., (2006), A Multicriteria Decision Framework for Measuing Banks’ Soundness Around the World, Journal of Multi-Criteria Decision Analysis, 14, 103-111. Gehrig T., (1998), Screening, cross-border banking, and the allocation of credit. Research in Economics 52, 387-407. Golin J, (2001), The Bank Credit Analysis Handbook: A Guide for Analysts, Bankers and Investors. John Wiley & Sons (Asia) Pre Ltd. Hamalainen P., Hall M., Howcroft B., 2005. A framework for market discipline in bank regulatory design. Journal of Business Finance and Accounting 32, 183209. Indro D.C., Jiang C.X., Patuwo B.E., Zhang G.P, (1999), Predicting mutual fund performance using artificial neural networks, Omega, 27, 373-380.
25
Jo H., Han I., (1996), Integration of Case-Based Forecasting, Neural Network, and Discriminant Analysis for Bankruptcy Prediction, Expert Systems with Applications, 11, 415-422. Jo H., Han I. (1996), Integration of case-based forecasting, neural network, and discriminant
analysis
for
bankruptcy
prediction,
Expert
Systems
with
Applications, 11 (4), 415-422. Keeley M.C., 1990. Deposit insurance, risk, and market power in banking. American Economic Review 5, 1183-1200. Kendall S.B., (1992), Bank regulation under nonbinding capital guidelines, Journal of Financial Services Research 5, 275-286. Koehn M., Santomero A., (1980), Regulation of bank capital and portfolio risk, Journal of Finance 35, 1235-1244. Lanine G., Vander Vennet R., (2006), Failure prediction in the Russian bank sector with logit and trait recognition models, Expert Systems with Applications, 30, 463-478. Le Bras A., Andrews D., (2004), Bank Rating Methodology: Criteria Report, 25 May, FitchRatings, available at www.fitchratings.com. Lennox C., (1999), Identifying Failing Companies: A Re-evaluation of the Logit, Probit and DA Approaches, Journal of Economics and Business 51, 347-364. Liang N., (1989), Bank Profits, Risk and Local Market Concentration, Journal of Economics and Business 41, 297-305. Malhotra R., Malhotra D.K., (2003), Evaluating consumer loans using neural networks, Omega, 31, 83-96. Merton R.C., (1977), An analytic derivation of the cost of deposit insurance and loan guarantees. Journal of Banking and Finance, 1, 3-11. Noy I., (2004), Financial liberalization, prudential supervision, and the onset of banking crises, Emerging Markets Review, 5, 341-359. Pasiouras F., Gaganis C., Zopounidis C., (2006), The impact of bank regulations, supervision, market structure and bank characteristics on individual bank ratings: A cross-country analysis, Review of Quantitative Finance and Accounting, 27, 403-438. Powers D.A., Xie Y., (2000), Statistical Methods for Categorical Data Analysis, Academic Press.
26
Rami Kumar P., Ravi V., (2007), Bankruptcy prediction in banks and firms via statistical and intelligent techniques – A review, European Journal of Operational Research, 180, 1-28. Smith K.A., Gupta J.N.D., (2000), Neural networks in business: techniques and applications for the operations researcher, Computers and Operations Research, 27, 1023-1044. Tam K.Y. (1991), Neural network models and the prediction of bank bankruptcy, Omega, 19, 429-445. Theodore S.S., (1999), Bank Credit Risk Rating Methodology (An Analytical Framework for Banks in Developed Markets, April, Moody’s Investors Service, Global Credit Research. Wheelock D.C., Wilson P.W., (2000), Why do banks disappear? The determinants of U.S. bank failures and acquisitions, The Review of Economics and Statistics 82, 127-138. Wolpert D.H., (1992), Stacked Generalization, Neural Networks, 5, 241-259. Wong B.K., Lai V.S., Lam J., (2000), A bibliography of neural network business applications research: 1994-1998, Computers and Operations Research, 27, 1045-1076. Wong B.K., Selvi Y., (1998), Neural network applications in finance: A review and analysis of literature (1990-1996), Information and Management, 34, 129-139. Zhang G., Hu M.Y., (1998), Neural network forecasting of the British Pound/US Dollar exchange rate, Omega, 26, 495-506.
27
Table 1 - Definitions of Fitch’s Bank Individual ratings and Bank soundness Fitch Rating
A
Bank soundness as coded in the present study Group 1
B
Group 1
C
Group 2
D
Group 3
E
Group 3
Training sample
Holdout sample
298
149
183
92
148
74
Fitch Definition
A very strong bank. Characteristics may include outstanding profitability and balance sheet integrity, franchise, management, operating environment or prospects. A strong bank. There are no major concerns regarding the bank. Characteristics may include strong profitability and balance sheet integrity, franchise, management, operating environment or prospects. An adequate bank, which, however, possesses one or more troublesome aspects. There may be some concerns regarding its profitability and balance sheet integrity, franchise, management, operating environment or prospects. A bank, which has weaknesses of internal and/or external origin. There are concerns regarding its profitability and balance sheet integrity, franchise, management, operating environment or prospects. Banks in emerging markets are necessarily faced with a greater number of potential deficiencies of external origin. A bank with very serious problems, which either requires or is likely to require external support.
Note: Fitch also uses the following intermediate assignments among the major five categories: A/B, B/C, C/D, D/E
28
Table 2 – Descriptive statistics
EQAS ROAA COST PROVIS LIQ LOGAS CAPRQ SPOWER MDISC ACTRS PRIGHTS CRGDP GDPGR CRDEP CONC
Training Holdout Mean Median St. Dev Mean Median St. Dev. 9.384 8.080 6.495 10.050 8.340 9.295 1.350 1.060 2.083 1.262 1.055 1.342 3.782 2.820 9.675 3.369 2.680 2.689 15.785 11.830 33.006 15.901 12.670 15.986 17.767 11.730 35.556 16.714 10.380 24.909 7.222 7.221 0.894 7.208 7.205 0.913 5.359 5.000 1.456 5.356 5.000 1.333 11.329 12.000 2.211 11.314 12.000 2.184 6.313 6.000 0.781 6.286 6.000 0.762 2.463 2.750 0.608 2.421 2.750 0.643 65.246 70.000 24.177 67.143 70.000 24.419 127.162 126.416 70.363 135.158 139.989 69.323 4.501 3.390 2.926 4.380 3.290 2.808 1.028 0.864 0.387 1.001 0.813 0.377 51.892 47.155 20.559 51.512 47.155 20.225
Note: EQAS = equity / assets, ROAA = return on average assets, COST = non-interest expenses / average assets, PROVIS = loan loss provision / net interest revenue, LIQ = liquid assets / customer & short term funding, LOGAS = logarithm of total assets, CAPRQ = Capital requirements index, SPOWER = supervisory power index, MDISC = market discipline index, ACTRS = restrictions on bank activities index, PRIGHTS = protection of property rights index, CRGDP = credit to the private sector / GDP, GDPGR = real GDP growth, CREDEP = bank industry credit/deposits, CONC = concentration of 3 largest banks.
29
Table 3 – Classification accuracies (models with financial ratios) Group 1 Panel A: Training sample UTADIS 59.73% ANN 89.60% CART 85.57% MDA 62.40% OLR 82.55% k-NN 70.13% Panel B: Holdout sample UTADIS 62.42% ANN 89.93% CART 79.87% MDA 62.40% OLR 83.22% k-NN 62.42%
Group 2
Group 3
Average
37.16% 19.67% 33.88% 44.30% 24.04% 49.18%
84.46% 75.68% 75.68% 71.60% 56.08% 69.59%
60.45% 61.65% 65.04% 59.43% 54.23% 62.97%
33.70% 16.30% 25.00% 29.30% 27.17% 31.52%
82.43% 70.27% 63.51% 67.60% 48.65% 60.81%
59.51% 58.84% 56.13% 53.10% 53.01% 51.58%
Notes: CART: Classification And Regression Trees, ANN: Artificial Neural Networks, UTADIS: UTilités Additives DIScriminantes, k-NN: kNearest Neighbours, OLR: Ordinary Logistic Regression, MDA: Multiple Discriminant Analysis
30
Table 4 – Classification accuracies (full set of variables) Group 1 Panel A: Training sample UTADIS 83.56% ANN 92.62% CART 81.88% MDA 82.20% OLR 86.24% k-NN 100.00% Panel B: Holdout sample UTADIS 82.55% ANN 89.26% CART 75.17% MDA 85.90% OLR 88.59% k-NN 76.51%
Group 2
Group 3
67.76% 63.93% 72.13% 47.50% 44.81% 100.00%
87.16% 79.49% 82.43% 79.66% 88.51% 80.84% 81.80% 70.50% 77.70% 69.58% 100.00% 100.00%
66.30% 55.43% 53.26% 48.90% 44.57% 51.09%
86.49% 81.08% 83.78% 82.40% 77.03% 66.22%
Average
78.45% 75.26% 70.74% 72.40% 70.06% 64.60%
Notes: CART: Classification And Regression Trees, ANN: Artificial Neural Networks, UTADIS: UTilités Additives DIScriminantes, k-NN: kNearest Neighbours, OLR: Ordinary Logistic Regression, MDA: Multiple Discriminant Analysis
31
Table 5 – Correlations of group assignments Panel A: Training sample Group 1 UTADIS UTADIS 1.000 ANN 0.460** CART 0.378** MDA 0.332** OLR 0.247** k-NN 0.122* Group 2 UTADIS 1.000 ANN 0.560** CART 0.475** MDA 0.586** OLR 0.595** k-NN 0.131 Group 3 UTADIS 1.000 ANN 0.496** CART 0.325** MDA 0.376** OLR 0.510** k-NN 0.397** Panel B: Holdout sample Group 1 UTADIS UTADIS 1.000 ANN 0.469** CART 0.513** MDA 0.220** OLR 0.280** k-NN 0.212** Group 2 UTADIS 1.000 ANN 0.575** CART 0.600** MDA 0.639** OLR 0.632** k-NN 0.398** Group 3 UTADIS 1.000 ANN 0.659** CART 0.488** MDA 0.569** OLR 0.671** k-NN 0.450**
ANN
CART
MDA
1.000 0.326** 1.000 0.532** 0.337** 1.000 0.378** 0.232** 0.579** 0.192** 0.119* 0.125*
OLR
1.000 -0.023
KNN
1.000
1.000 0.423** 0.700** 0.617** 0.034
1.000 0.535** 1.000 0.483** 0.828** 1.000 -0.004 0.108 0.118
1.000 0.623** 0.750** 0.694** 0.352**
1.000 0.564** 1.000 0.562** 0.853** 1.000 0.209* 0.302** 0.256** 1.000
ANN
CART
OLR
KNN
1.000 0.353** 0.545** 0.421** 0.189*
1.000 0.348** 1.000 0.185* 0.704** 1.000 0.229** 0.200* 0.081
1.000
1.000 0.570** 0.742** 0.636** 0.241*
1.000 0.617** 1.000 0.615** 0.837** 1.000 0.379** 0.422** 0.447** 1.000
1.000 0.616** 0.780** 0.671** 0.635**
1.000 0.524** 1.000 0.523** 0.867** 1.000 0.330** 0.485** 0.453** 1.000
MDA
1.000
Notes: **Statistically significant at the 1% level (two-tailed), *Statistically significant at the 5% level (two-tailed); CART: Classification And Regression Trees, ANN: Artificial Neural Networks, UTADIS: UTilités Additives DIScriminantes, k-NN: k-Nearest Neighbours, OLR: Ordinary Logistic Regression, MDA: Multiple Discriminant Analysis
32
Table 6 - Classification accuracies of stacked models (full set of variables) Group 1 Group 2 Group 3 Average Panel A: Training sample UTADIS 85.91% 52.46% 85.23% 74.53% ANN 89.60% 45.36% 83.22% 72.72% CART 84.56% 63.39% 85.91% 77.95% MDA 83.20% 57.40% 79.90% 73.50% OLR 87.58% 50.82% 79.87% 72.76% k-NN 81.88% 48.63% 80.54% 70.35% Panel B: Holdout sample UTADIS 85.91% 60.87% 87.84% 78.20% ANN 91.28% 54.35% 79.73% 75.12% CART 86.58% 60.87% 72.97% 73.47% MDA 82.60% 60.90% 78.40% 73.97% OLR 91.95% 48.91% 74.32% 71.73% k-NN 77.85% 54.35% 77.03% 69.74% Notes: CART: Classification And Regression Trees, ANN: Artificial Neural Networks, UTADIS: UTilités Additives DIScriminantes, k-NN: kNearest Neighbours, OLR: Ordinary Logistic Regression, MDA: Multiple Discriminant Analysis
33
Table 7 – Details on the assignments of the UTADIS individual model (holdout sample)
A C T U A L
UTADIS Panel A: Assignments per group (number of banks) Group 1 Group 2 Group 3 Group 1 123 26 0 Group 2 18 61 13 Group 3 1 9 64 Panel A: Assignments per group (in %) Group 1 Group 2 Group 3 Group 1 82.55% 17.45% 0.00% Group 2 19.57% 66.30% 14.13% Group 3 1.35% 12.16% 86.49%
149 92 74
100% 100% 100%
34
Appendix A – Regulatory Variables Variable CAPRQ
Category Capital requirements
Description This variable is determined by adding 1 if the answer is yes to questions 1-6 and 0 otherwise, while the opposite occurs in the case of questions 7 and 8 (i.e. yes=0, no =1). (1) Is the minimum required capital asset ratio risk-weighted in line with Basle guidelines? (2) Does the ratio vary with market risk? (3-5) Before minimum capital adequacy is determined, which of the following are deducted from the book value of capital: (a) market value of loan losses not realized in accounting books? (b) unrealized losses in securities portfolios? (c) unrealized foreign exchange losses? (6) Are the sources of funds to be used as capital verified by the regulatory/supervisory authorities? (7) Can the initial or subsequent injections of capital be done with assets other than cash or government securities? (8) Can initial disbursement of capital be done with borrowed funds? MDISC Market This variable is determined by adding 1 if the answer is yes to questions 1-7 and 0 otherwise, while the opposite occurs in the case of discipline questions 8 and 9 (i.e. yes=0, no =1). (1) Is subordinated debt allowable (or required) as part of capital? (2) Are financial institutions required to produce consolidated accounts covering all bank and any non-bank financial subsidiaries? (3) Are off-balance sheet items disclosed to public? (4) Must banks disclose their risk management procedures to public? (5) Are directors legally liable for erroneous/misleading information? (6) Do regulations require credit ratings for commercial banks? (7) Is an external audit by certified/licensed auditor a compulsory obligation for banks? (8) Does accrued, though unpaid interest/principal enter the income statement while loan is non-performing? (9) Is there an explicit deposit insurance protection system? This variable is determined by adding 1 if the answer is yes and 0 otherwise, for each one of the following fourteen questions: (1) Does OFFPR Official the supervisory agency have the right to meet with external auditors to discuss their report without the approval of the bank? (2) Are disciplinary power auditors required by law to communicate directly to the supervisory agency any presumed involvement of bank directors or senior managers in illicit activities, fraud, or insider abuse? (3) Can supervisors take legal action against external auditors for negligence? (4) Can the supervisory authorities force a bank to change its internal organizational structure? (5) Are off-balance sheet items disclosed to supervisors? (6) Can the supervisory agency order the bank's directors or management to constitute provisions to cover actual or potential losses? (7) Can the supervisory agency suspend director’s decision to distribute dividends? (8) Can the supervisory agency suspend director’s decision to distribute bonuses? (9) Can the supervisory agency suspend director’s decision to distribute management fees? (10) Can the supervisory agency supersede bank shareholder rights and declare bank insolvent? (11) Does banking law allow supervisory agency or any other government agency (other than court) to suspend some or all ownership rights of a problem bank? (12) Regarding bank restructuring and reorganization, can the supervisory agency or any other government agency (other than court) supersede shareholder rights? (13) Regarding bank restructuring & reorganization, can supervisory agency or any other government agency (other than court) remove and replace management? (14) Regarding bank restructuring & reorganization, can supervisory agency or any other government agency (other than court) remove and replace directors? ACTRS Restrictions on The score for this variable is determined on the basis of the level of regulatory restrictiveness for bank participation in: (1) securities banks activities activities (2) insurance activities (3) real estate activities (4) bank ownership of non-financial firms. These activities can be unrestricted, permitted, restricted or prohibited that are assigned the values of 1, 2, 3 or 4 respectively. We use an overall index by calculating the average value over the four categories. Note: The individual questions and answers were obtained from the World Bank database on bank regulation and supervision (Barth et al.., 2008)
35
36
