Audit Quality Indicators: Audit Practice Meets Audit Research

Current Issues in Auditing Volume 7, Issue 2 2013 Pages A17–A23

American Accounting Association DOI: 10.2308/ciia-50581

Audit Quality Indicators: Audit Practice Meets Audit Research Roger D. Martin SUMMARY: Regulators, audit firms, and academic researchers are exploring issues related to Audit Quality Indicators (AQIs), including how to optimize their use and how to anticipate the consequences of measuring, reporting, and monitoring AQIs on a routine basis. This article describes the main topics addressed by a panel discussion and participant breakout group discussions at a 2012 symposium hosted by the Center for Audit Quality. Topics include how AQIs could be useful to audit stakeholders, what we know from practice about the current state of AQIs, and how academic research might inform the development and use of AQIs by stakeholders. Keywords: audit quality; audit quality indicators; audit practice; audit regulation.

INTRODUCTION Audit quality always has been critical and, most recently, audit quality indicators (AQIs) have been under much scrutiny. Lawmakers, regulators, issuers, financial statement users, and others have called for the development of publicly available indicators of audit quality. The Treasury Department’s Advisory Committee on the Auditing Profession recommended that the PCAOB develop AQIs and that audit firms publish these indicators (U.S. Department of the Treasury 2008). The PCAOB lists the development of AQIs as a priority project for 2013 in its most recent strategic plan (PCAOB 2012), and the Standing Advisory Group of the PCAOB discussed AQIs at its May 2013 meeting (PCAOB 2013). A common theme in this activity is the belief that audit quality indicators will benefit auditors, issuers, and financial statement users by providing indicators of audit quality that improve transparency of audit firms and their audit processes. This improved transparency would be expected to increase market participants’ ability to assess audit quality, which in turn might lead to differentiation among audit firms based on these measures, as well as increased incentives for firms to improve audit quality (PCAOB 2013; Bedard et al. 2010). AQIs

Roger D. Martin is an Associate Professor at the University of Virginia. This paper is based on Professor Martin’s participation in the Center for Audit Quality event described in the paper, along with a transcript of the panel discussion and composite notes from breakout discussion groups of participants at that event. Interpretations and summaries of comments made by the panelists and other participants are Professor Martin’s views and not quotes or direct attributions of ideas or representations by the panelists or participants at that event. Submitted: June 2013 Accepted: July 2013 Published Online: August 2013

Martin

A18

also could be used by regulators as part of their quality assessment process and to inform standard setters about challenges or deficiencies in current audit approaches revealed by monitoring AQIs over time (PCAOB 2013). The Center for Audit Quality (CAQ) included in its 2012 CAQ Symposium for Practitioners and Academics a panel discussion and breakout discussions on AQIs from the perspective of multiple stakeholders. The purpose of the symposium was to bring together about 40 audit practice leaders and 40 audit research scholars to discuss current auditing topics. The AQIs discussion was two-fold—a 75 minute panel discussion moderated by Stephen Chipman, CEO of Grant Thornton LLP, and a 45 minute brainstorming session organized in groups of about ten people, with each group including practitioners and academics. The panelists were Jim Liddy, U.S. Vice Chair of Audit and Regional Head of Audit Americas at KPMG LLP; Tim Bell, Coggin Distinguished Professor of Accounting at the University of North Florida; and Roger Martin, Associate Professor of Commerce at the University of Virginia. The purpose of this article is to summarize the main topics in the discussion at the CAQ Symposium. The panel discussion addressed AQIs from the perspective of five primary groups of stakeholders in the audit process: audit firms, audit committees, investors and creditors, audit regulators, and preparers’ management. The breakout groups were organized similarly, with each breakout group assigned to discuss AQIs from the perspective of one of the specific stakeholder groups. A transcript of the panel discussion was prepared, and a ‘‘scribe’’ was assigned to each breakout group to provide a summary of key topics discussed. Those two sources are the basis for this summary of the discussions described in this article.

STAKEHOLDERS’ PERSPECTIVES ON AQIs The structure of this section is based on the five groups of stakeholders used to organize the discussion at the symposium: audit firms, audit committees, investors and creditors, audit regulators, and preparers’ management. For each group, a summary of the main panel discussion topics is provided, followed by issues raised in the breakout groups where applicable. Audit Firms As a precursor to a discussion of indicators of audit quality, the panel first discussed audit quality in broad terms. Audit quality, from a firm’s perspective, is managed as a continuous process that identifies important matters that affect audit performance, analyzes conditions, formulates responses, and monitors and reinforces performance. Firms collect and analyze voluminous data that are assimilated to provide a basis for action. Importantly, firms’ internal processes focus on identifying root causes of issues indicative of underlying audit quality.1 Mr. Liddy of KPMG described activities in his firm’s processes that are designed to gather relevant information, review outputs of their quality monitoring processes, and formulate responses.2 A consistent message in the panel and breakout discussions was that large accounting firms employ continuous and multilevel review processes to assess and manage audit quality and to monitor those processes. 1

2

Note that this description does not pertain solely to poor audit quality, even though that is often the focus of attention in discussions of audit quality. Practitioners also identify ‘‘best practices’’ that explain exceptionally good audit quality, with the intent of learning how to apply those practices more broadly. The largest audit firms typically describe these activities in publicly available ‘‘transparency reports.’’ I refer readers to these for descriptions of the organizations and processes established by various firms to monitor and manage audit quality. For examples, see KPMG LLP (2012) and PricewaterhouseCoopers LLP (2012).

Current Issues in Auditing Volume 7, Issue 2, 2013

Martin

A19

Mr. Bell described how firms (and other stakeholders) might think about quality from a conceptual perspective and how that might provide insights into how to use AQIs to measure audit quality. Drawing on literature in the healthcare field, he cited Donabedian’s (1988) description of how to evaluate quality and, perhaps most importantly, what level of quality should be strived for: Technical performance depends on the knowledge and judgment used in arriving at the appropriate strategies of care and on skill in implementing those strategies. The goodness of technical performance is judged in comparison with the best in practice. The best in practice, in its turn, has earned that distinction because, on the average, it is known or believed to produce the greatest improvement in health . . . If the realized fraction of what is achievable is called effectiveness, the quality of technical care becomes proportionate to its effectiveness. Two consistent themes throughout the discussion were (1) how should audit performance be measured to inform the assessment of quality, and (2) how should that assessment of quality be used to manage improvements and to provide evidence of achieved quality? The panel discussion transitioned from this background to how AQIs might be used to evaluate whether audit quality is improving. Mr. Martin observed that auditors are experts in identifying diagnostic evidence and evaluating its implications, and suggested that the firms’ challenge is to apply that expertise to evaluating quality. Firms must apply their expertise and skepticism to a process that evaluates critically whether they are using the appropriate evidence to assess quality, employing adequate root cause analysis to identify strengths and weaknesses, and using that root cause analysis to manage revisions to their audit processes. The subsequent discussion focused on the challenges that firms face in ensuring that their quality monitoring processes are being applied rigorously and with adequate skepticism. Mr. Liddy (and other practitioners in breakout sessions) described how practitioners try to measure quality-related factors and manage these important processes. Audit committee members routinely ask engagement teams how they manage audit quality—consequently, those discussions already have conditioned auditors to think about and manage their own processes in the context of how to report on them to outsiders to provide confidence in how they operate. Another interesting dimension of this challenge is the observation that some of the firms’ quality assessment and monitoring efforts take place in ‘‘real time,’’ in the sense that they occur as engagements are being performed. The advantage of mid-engagement assessments and adjustments is that they improve quality as the audit is being conducted as opposed to identifying issues after they occur. Audit Committees The discussion transitioned to explore how audit committee (AC) members assess audit quality. The panelists agreed that AC members focus primarily on their specific audit engagements, although they likely also consider how broader audit quality issues faced by the profession and by their audit firm pertain to their specific circumstances. The emphasis on the role of ACs in the financial reporting process has been very positive and, while pressure on audit fees has been visible, practitioners stated that they believe AC members focus primarily on financial reporting risks, identification of significant accounting policies and judgments, and how the auditor is addressing those issues. Practitioners also described audit quality information that AC members increasingly are asking for on their engagements. While making hiring and retention decisions, AC members often focus Current Issues in Auditing Volume 7, Issue 2, 2013

Martin

A20

on issues including industry specialization, qualifications of key engagement team members, and the type and quality of service provided specifically to the AC. In terms of the oversight of engagement performance, panelists indicated that AC members consistently are focusing on key risks (and how the auditor assesses and tests for those risks); disagreements—or potential disagreements—with management; and what items were considered for disclosure, but were later determined to be unnecessary. Much of this interaction between engagement team leaders and AC members is focused on process—i.e., how did the engagement team determine that they conducted a quality audit? So, while ACs certainly focus on audit outcomes, it was suggested that they are growing more attuned to understanding how the engagement team gets to their conclusions and what evidence can be provided to indicate high audit quality. Investors and Creditors The discussion then turned to whether financial statement users—investors in particular—are interested in audit quality, and what indicators they use to assess quality. The consensus seemed to be that investors have a limited understanding of the audit process and the role of the financial statement audit. Investors also do not have access to engagement-specific audit quality information (unlike AC members). The combination of these factors typically causes investors to rely on output measures to assess audit quality, with restatements being the primary outcome indicator on which investors focus. The panelists then discussed the value of earnings quality metrics as AQIs. Earnings quality refers to the ability of reported earnings to predict a company’s future earnings (Spiceland et al. 2013, 177). A wide range of information about a company is used to predict future earnings, and earnings quality is considered to be one summary indicator of information that is provided as a result of the financial reporting and auditing process (Francis et al. 2006). To assess earnings quality, researchers have used metrics including abnormal accruals (i.e., accrual levels that differ from ‘‘average’’ for comparable firms), earnings management to meet benchmarks, and future restatements. Academic research has explored the relationship of these earnings quality metrics with audit quality, typically by identifying if, and when, poor earnings quality is associated with evidence of lower audit quality (e.g., Francis 2004, 2011). The association of earnings quality and audit quality is logical because financial statement audits are intended to provide assurance that financial statements are stated in accordance with generally accepted accounting principles (GAAP) and, thus, deemed likely to be of higher quality. But, it is difficult to evaluate how an investor uses various measures of earnings quality as an audit quality input in a valuation decision. First, if an investor is able to reliably assess earnings quality for a specific company, it is unclear what additional valuation information is gained by then inferring the quality of the audit.3 Second, the ability to assess earnings quality on a firm-specific basis is difficult. Academic research typically uses large samples of observations to draw conclusions about average associations that are statistically significant; applying highly reliable earnings quality 3

Note that, in the context of AQIs, earnings quality was discussed in terms of whether it might be an indicator of audit quality (i.e., could an investor learn about the quality of an audit by examining characteristics of the quality of earnings). An alternative perspective, although not in the context of AQIs, is whether (and how) audit quality leads to earnings quality. This research typically attempts to demonstrate that earnings quality is higher for companies with auditors that are expected to have higher quality (e.g., large firms, firms that specialize in a particular industry) than companies with auditors that are expected to have lower quality (e.g., smaller firms, firms that are not industry specialists).


Martin

A21

measurement techniques for a specific company (and, in turn, the company’s audit) is surely much more difficult. Audit Regulators For audit regulators, the role of AQIs is complicated because the regulators must consider how to evaluate audit quality from both firm-wide and engagement-specific perspectives, the explicit role of auditing standards in establishing quality expectations and responding to audit deficiencies, and how to enforce auditing standards based on evidence of poor audit quality. In the U.S., the PCAOB has responsibility for all three of these dimensions through inspections, standard setting, and enforcement. This unique intersection of responsibilities forces the PCAOB to consider most of the issues discussed thus far in this article. They must consider how to define quality relative to expectations of what is achievable, evaluate audit production processes and audit quality (which naturally leads to a consideration of appropriate AQIs), and link observed audit quality with revisions to, and enforcement of, standards that will best serve financial statement users. A challenge for all audit regulators—even those with explicit responsibilities for only standard setting (e.g., the Auditing Standards Board in the U.S., or the International Auditing and Assurance Standards Board [ IAASB]) or only enforcement (e.g., government agencies such as the SEC in the U.S.)—is how to gather evidence on audit performance that reliably assesses audit quality. Therefore, many of the issues discussed earlier (e.g., what evidence should be used, how to use that evidence to assess quality, how to validate the linkage between evidence and audit quality) apply to the regulator’s role. Academic research is fairly limited in contributing to these issues because of data limitations. Researchers have access to the same publicly available data as do all stakeholders in terms of items such as restatements, audit report ‘‘accuracy,’’ and proxies for audit quality (e.g., earnings quality). However, the PCAOB has access to observed audit quality indicators (e.g., client-specific audit deficiencies), which provides them with the ability to relate observed audit engagement performance issues and overall firm quality control practices to the assessment of audit quality that comes from this proprietary information. The PCAOB is working to identify AQIs, along with developing a proposal for how AQIs should be reported and used by stakeholders (PCAOB 2013). An important part of that process will be to validate the AQIs’ reliability—reliability in terms of measures that will be consistently and fairly stated and that are correlated with audit quality in known and predictable ways. This will be a very difficult task, but one that audit firms and audit researchers will be interested in and likely be able to assist with. Firms will be able to contribute by sharing their understanding of key audit quality drivers to aid in developing an understanding of how proposed AQIs are associated with audit quality. Academics can bring their objectivity and statistical analysis expertise to help identify and validate proposed AQIs. Preparers’ Management The discussion then turned to the last stakeholder group, management of companies issuing financial statements.4 Company management likely has two primary interests in audit quality: assuring high-quality financial reporting and that audits are conducted as efficiently and unobtrusively as possible. The discussion of this stakeholder group generally focused on the 4

Management in this context refers to management responsible for overseeing financial reporting (as opposed to those company employees responsible for the more-detailed preparation of financial statements).


Martin

A22

tension between these two motivations. From the practitioners’ view, management interests are typically aligned with the external auditor and the AC in the sense that they want the audit to be of high quality. However, management probably believes that audit quality is somewhat of a constant, so its pressure on the auditor is more likely to be in the form of pushing them to be efficient. AQIs that can serve to remind management of the important investments of time and resources that the auditor must expend might be valuable in that interaction.

CONCLUSION There are many challenges posed by developing and using AQIs to help understand and manage audit quality. The discussions at the CAQ Symposium provided context to many of those challenges, and the enthusiasm for the topic was high. The discussions generally focused on the value of AQIs from the perspective of the various stakeholders in the financial reporting process. ‘‘Value’’ was considered in terms of what a particular stakeholder might want to learn about the audit or audit process for a specific company’s financial statements. In this sense, the discussion was at a conceptual level because the focus was on how AQIs might help generally. Interestingly, almost no time was spent focusing on specific AQIs, categories of AQIs, how AQIs would be produced and reported, or possible unintended consequences of disclosing AQIs. Each of these topics is interesting and very important in its own right, of course. This is such a potentially new and ‘‘game-changing’’ topic in the realms of audit oversight and audit quality that the discussion and debate must necessarily go on at multiple levels: the high-level, conceptual perspective on the demand for and value of AQIs; the implementation level of how to identify specific AQIs and provide for their disclosure; and the specific decision-making level of the individual and how they will interpret and use AQIs. Each of these perspectives will benefit from the combined expertise of audit practitioners and academics. Practitioners have worked to understand and manage the key drivers of audit quality for some time, and their insight into how their audit quality control processes operate—and the challenges they continue to face in those processes—will be informative in the discussion of AQIs. Audit academics bring their objectivity and rigor to the challenge, and hopefully will be in a position to inform this discussion and ensure that the development of AQIs will benefit all stakeholders. Understanding how specific AQIs reflect audit performance characteristics will be part of this process, and learning how individuals from all groups of stakeholders will respond to AQIs will also be critical. The symposium provided a great example of this collaborative effort, and hopefully will be followed by many similar opportunities in the future. All stakeholders—audit firms; audit committees; investors and creditors; audit regulators and management of reporting companies—will benefit from this type of interaction.

REFERENCES Bedard, J. C., K. M. Johnstone, and E. F. Smith. 2010. Audit quality indicators: A status update on possible public disclosures and insights from audit Practice. Current Issues in Auditing 4 (1): C12–C19. Donabedian, A. 1988. The quality of care: How can it be assessed? Journal of the American Medical Association 260 (12): 1743–1745. Francis, J. R. 2004. What do we know about audit quality? The British Accounting Review 36: 345–368. Francis, J. R. 2011. A framework for understanding and researching audit quality. Auditing: A Journal of Practice & Theory 30 (May): 125–152.


Martin

A23

Francis, J., P. Olsson, and K. Schipper. 2006. Earnings quality. Foundations and Trends in Accounting 1 (4): 259– 340. KPMG LLP. 2012. KPMG LLP Transparency Report 2012. Available at: http://www.kpmg.com/US/en/about/ Documents/2012-KPMG-LLP-Transparency-Report-web.pdf PricewaterhouseCoopers LLP. 2012. PricewaterhouseCoopers LLP 2012 Transparency Report. Available at: https:// www.pwc.com/en_US/us/about-us/assets/pwc-llp-fy12-transparency-report.pdf Public Company Accounting Oversight Board (PCAOB). 2012. PCAOB Strategic Plan: Improving the Relevance and Quality of the Audit for the Protection and Benefit of Investors, 2012-2016. Available at: http://pcaobus.org/ About/Ops/Documents/Strategic%20Plans/2012-2016.pdf Public Company Accounting Oversight Board (PCAOB). 2013. Standing Advisory Group Meeting Briefing Document: Discussion – Audit Quality Indicators. Available at: http://pcaobus.org/News/Events/Documents/05152013_ SAGMeeting/Audit_Quality_Indicators.pdf Spiceland, J. D., J. F. Sepe, and M. W. Nelson. 2013. Intermediate Accounting. Seventh edition. New York, NY: McGraw-Hill/Irwin. U.S. Department of the Treasury. 2008. Advisory Committee on the Auditing Profession Final Report. Available at: http://www.treasury.gov/about/organizational-structure/offices/Documents/final-report.pdf
