Accounting Horizons Vol. 29, No. 1 2015 pp. 61–81
American Accounting Association DOI: 10.2308/acch-50921
Behind the Numbers: Insights into Large Audit Firm Sampling Policies Brant E. Christensen, Randal J. Elder, and Steven M. Glover SYNOPSIS: Changes in the audit industry after the Sarbanes-Oxley Act, including mandatory audits of internal control over financial reporting and PCAOB oversight and inspection of audit work, have potentially changed the nature and extent of audit sampling in the largest accounting firms. However, little academic evidence exists on these firms’ current audit sampling policies (Elder, Akresh, Glover, Higgs, and Liljegren 2013). As such, we administer an extensive, open-ended survey to the national office of the Big 4 and two other international accounting firms regarding their firm’s audit sampling policies. We find variation among the largest auditing firms’ policies in their use of statistical and nonstatistical sampling methods and in inputs used in the sampling applications that could result in different sample sizes. Sampling experts’ internal reviews indicate that projecting and resolving identified misstatements is one of the biggest difficulties that audit engagement teams face when using sampling techniques. Finally, we present evidence that some firms have significantly changed their approach to revenue testing due to PCAOB inspections. This evidence provides important insights into current sampling policies and presents opportunities for future research. Keywords: audit sampling; statistical sampling; materiality; audit misstatements. Data Availability: Please contact the authors.
INTRODUCTION
A
udit sampling is a fundamental audit testing procedure. Over the last two decades there have been significant changes in audit approaches, including strategic systems auditing in the 1990s (Bell, Marrs, Solomon, and Thomas 1997) and federally mandated audits of internal control over financial reporting for large public companies as a result of the Sarbanes-Oxley
Brant E. Christensen is an Assistant Professor at the University of Missouri, Randal J. Elder is a Professor at Syracuse University, and Steven M. Glover is a Professor and an Associate Dean at Brigham Young University. We thank the sampling experts from the six participating audit firms for their time and participation in this study. Brant Christensen acknowledges funding from the Deloitte Foundation and Steve Glover acknowledges funding from the K. Fred Skousen Endowed Professorship.
Submitted: February 2014 Accepted: September 2014 Published Online: September 2014 Corresponding author: Randal J. Elder Email:
[email protected]
61
62
Christensen, Elder, and Glover
Act of 2002 (SOX).1 Revisions to audit approaches have the potential to change the nature and extent of audit sampling techniques used by accounting firms. For instance, the requirement to audit internal control over financial reporting has necessarily increased the extent and importance of tests of controls, many of which are performed using sampling. Additionally, the Public Company Accounting Oversight Board (PCAOB) has identified sampling as an area needing more emphasis, and inspection reports have identified multiple issues regarding audit sampling, including small and non-representative samples and incorrect or lack of error projection, among others (PCAOB 2008, 2014). Most of the existing literature on sampling practices relates to the period before the sweeping changes of the Sarbanes-Oxley Act (e.g., Hall, Hunton, and Pierce 2002), and as such little academic evidence exists on the current audit sampling policies and practices of the largest accounting firms (Elder, Akresh, Glover, Higgs, and Liljegren 2013). This study addresses a number of research questions regarding the current state of audit sampling. We administer an extensive, open-ended survey to the national office sampling experts from each of the Big 4 firms and two other international accounting firms. As a result, our study focuses on the policies in place at the firms and not necessarily how these policies are implemented in the field. However, due in part to internal firm and federal oversight, discussions with firm experts indicate that audit teams are expected to comply with firm sampling guidance. We provide evidence on the sampling policies of six of the largest firms, compared to previous studies that are based on data from one or a few firms (e.g., Durney, Elder, and Glover 2014; Elder and Allen 2003). The analysis of firms’ policies as provided to us by their sampling experts highlights important similarities and differences among the firms’ policies. For test of controls, half of the firms surveyed use statistical sampling while the other half use nonstatistical sampling. Similarly, for tests of details some firms emphasize monetary-unit-sampling (MUS) statistical methods, while others use nonstatistical methods. The nonstatistical approaches are based on statistical theory, consistent with the requirements of auditing standards, and designed to provide sample sizes and conclusions that are consistent with statistical approaches (PCAOB AU 350.23). The heterogeneity in sampling approaches among firms is different than earlier time periods when almost all firms either followed statistical approaches (Akresh 1980), or nonstatistical approaches (Sullivan 1992). We also find variation in the sampling inputs across firms, thus resulting in different sample sizes, regardless of whether the firm follows a statistical or nonstatistical sampling approach. Depending on the level of assurance obtained from other audit procedures, differences in sample sizes raise the possibility that different levels of evidence and assurance are obtained to support audit opinions. Interestingly, we find that sampling approaches and parameters within most firms are identical for large public and smaller private companies despite the likely differences in business and engagement risk. Durney et al. (2014) indicate that error rates in the post-SOX audit environment are very low and often zero. However, we find variation in the planned level of expected error, and we also find differences in error projection methods used and how firms respond to identified errors and misstatements. Sampling experts indicate that error projection, as well as responding to and resolving identified misstatements, is one of the biggest difficulties that audit engagement teams face. Finally, we present evidence that some firms have significantly changed their approach to revenue testing due to PCAOB inspections, relying more heavily on substantive testing using sampling than other substantive testing such as analytical procedures. This study provides evidence on current sampling practices and identifies important differences in sampling policy among the largest audit firms. We identify areas for additional research, including the actual implementation of these policies by teams in the field and their impact on audit 1
Although the strategic systems auditing approach was largely abandoned following the adoption of the SarbanesOxley Act, it did represent a fundamental shift in audit testing, and elements of this approach still impact audit practice.
Accounting Horizons March 2015
Behind the Numbers: Insights into Large Audit Firm Sampling Policies
63
quality. Our findings provide insights into sampling policies and procedures that are important for researchers, educators, regulators, and practitioners to better understand the application of audit sampling theory in the current audit environment. METHODOLOGY AND PARTICIPANTS The survey asked respondents a number of open-ended questions regarding sampling policies and practices currently in place at the Big 4 and two other international accounting firms. The survey was developed in part based on the call for additional research in Elder et al. (2013). We worked in coordination with the participating firms in order to focus on the most relevant areas of sampling policy. A version was sent by email in Spring 2013 to one of the Big 4 firms for completion and feedback, after which some additional clarifications were made before distributing electronically to the other firms. Due to the specialized nature of audit sampling, and due to the focus on the firms’ official policy and methodology, we only solicited responses from the leading national-level sampling expert from each participating firm. The author team first contacted the lead national office methodology partner at each firm. We explained the research questions and requested that the firm’s sampling expert complete the survey, after which the lead partner coordinated with the firm’s sampling expert to complete the survey. All participating sampling experts work with the national office of their respective firms. On average, these experts have been employed by their firms for 23 years. Given our desire to compare current policies and methodologies at the largest firms, rather than examine working practices and application, the correct population of respondents is nationallevel experts rather than a larger sample of non-expert auditors (e.g., Hall et al. 2002).2 To maintain anonymity of the participants and the firms, the firms’ responses are not identified and we do not distinguish between the policies of Big 4 and non-Big 4 firms.3 After we collected demographic information on respondents, survey questions were organized into three sections: general sampling policies, tests of controls, and substantive tests of details. Within the sections on tests of controls and substantive tests of details, questions addressed sampling approach, sampling parameters, sample selection, sample documentation, sample evaluation, treatment of exceptions, and other specialized questions such as minimum sample sizes, infrequent controls, dual-purpose tests, and multi-location testing. The survey contained 57 questions and was designed to be comprehensive in addressing sampling policies at the firms. Based on author review and pilot feedback, every effort was made to streamline the survey where possible. Participants indicated the survey took two to four hours to complete. A copy of the survey is available from the authors on request. After initial analysis of the responses, we contacted respondents by email and posed clarifying and follow-up questions. Due to the open-ended nature of the survey, analyses are primarily descriptive. SAMPLING POLICIES Firm-Specific Sampling Policies To understand the policy-setting procedures in place at the firms, we asked respondents what group (or groups) establishes sampling policy at their respective firms. Respondents indicated that 2
3
Although responses come from only one individual per firm, this individual is the designated sampling expert for the firm and the responses reflect firm policy. Firm policy is established at the national office level and then consistently transmitted throughout the firm. As such, surveying a larger number of individuals within the firm would not necessarily provide additional insights into firm sampling policies. We note that recent research suggests that Tier 2 firms have audit quality similar to that of the Big 4 (e.g., Boone, Khurana, and Raman 2010), and all six firms are subject to annual inspections by the PCAOB.
Accounting Horizons March 2015
