In completing SOC 1 and SOC 2 examinations (and most other types of audits), there is testing involved to determine the operating effectiveness of controls. There are different types of tests that can be applied to testing controls (for more information on the five types of tests refer to our article, Five Types of Testing Methods Used During Audit Procedures), and to complete a majority of these tests there is a sampling of populations that are required. In this post, we cover what audit sampling is and provide guidance on how to apply audit sampling to get to a confident conclusion on the operating effectiveness of controls.
According to the AICPA (in SAS No. 122 AU-C Section 530), audit sampling is defined as “The selection and evaluation of less than 100 percent of the population of audit relevance such that the auditor expects the items selected (the sample) to be representative of the population and, thus, likely to provide a reasonable basis for conclusions about the population.” The entire AICPA audit sampling guide can be referenced here.
The definition from the AICPA is a little wordy, but to summarize, as auditors, the purpose of audit sampling is to allow us to do the right amount of testing to confidently determine the operating effectiveness of controls. This does not mean we can always test 100 percent, or even have the capacity to. Therefore, sampling comes into play in testing. But what is the right amount and how do you figure that out?
As auditors we need to consider three primary areas when performing audit sampling: 1) sample method, 2) the sample size, and 3) tolerable rate of deviation.
There are four main types of audit sampling methods that are used when completing tests of controls in SOC 1 and SOC 2 examinations. The type of population, how it was generated, and the size of the population can have an impact on the type of audit sampling methodology that is chosen for testing. The four main types include:
Every SOC examination should follow one or more of these sampling methods for testing of the population. A walkthrough or inquiry only would not be sufficient to test all controls.
Statistical sampling requires that samples be selected at random, generally using a tool to generate random numbers. The simple random sampling method above would be considered statistical sampling.
Non-statistical sampling allows an auditor to use professional judgment when selecting samples. Non-statistical methods make a lot of sense when a population is very small, rather than spending the time setting up a statistical sample. While non-statistical sampling allows for auditor judgment, an auditor should always be careful not to include too much bias in selecting samples.
There are a number of factors that need to be considered when determining the sample size.
The tables below (Table 1 and Table 2) are what we use as guidelines when selecting our sample sizes in our SOC 1 and SOC 2 examinations. These tables align with the guidance set forth in the audit sampling guide from the AICPA.
Table 1 is used for larger sample sizes (250 or greater in the population) and shows recommended sample sizes to get to a minimum 90% confidence level. The table includes the sample sizes for up to two deviations and takes into consideration the risk of the control.
Table 2 gives further guidance sampling on less frequent operating controls and on smaller populations (transactional).
Using the tables above a few examples would include:
The guidance from the AICPA is pretty extensive around audit sampling. SOC auditors should review their sampling methods to make sure they are aligned with the AICPA guidance when performing their examinations. Please contact us if you would like further information on sampling, testing methods, or any of the services we provide.
Nicole Hemmer started her career in 2000. She is the co-founder of Linford & Co., LLP. Prior to Linford & Co., Nicole worked for Ernst & Young in Indianapolis, Chicago, and Denver. She specializes in SOC examinations and royalty audits and loves the travel and challenge that comes with clients across all industries. Nicole loves working with her clients to help them through examinations for the first time and then working together closely after that to have successful audits.
