Determine The Sample Size [PDF]

Zitiervorschau

2/25/2018

Deloitte Technical Library

Chapter 4: Determine the sample size 4.1 Tests of details 4.2 Tests of controls 4.3 Attribute sampling 4.4 Sample size considerations for a group audit 4.5 Documentation considerations When performing audit sampling, we are required to determine a sample size sufficient to reduce sampling risk to an acceptably low level. [DTTL AAM 23005.11] Our sample size determination will vary depending on the type of tests performed. This chapter addresses how to determine sample sizes for the following types of tests: Tests of details Tests of controls

In addition, this chapter explains sample size considerations specific to group audits.

Section 4.1: Tests of details Before we discuss the table used in determining sample sizes for a test of details, we will discuss a few considerations that will have an impact on how we use the table, including: Disaggregation of the population Consideration of individually significant items Stratification of the population Profiling of the population

4.1.1: Disaggregation of the population As discussed in Section 2.3, "Disaggregate the ABCOTD into populations for testing," we may consider disaggregating the population into smaller sub-populations and designing separate substantive procedures for each sub-population if there are different characteristics within the population which would impact our planned tests of details.[DTTL AAM 23002-4.24] If we determine that sampling is appropriate for each sub-population, we then determine the sample size for each sub-population independently. For example, assume we are testing a ROMM relating to the valuation of an investment balance that consists of investments with and without readily determinable fair values. Due to the different valuation methods, the risks for these two types of investments may be different and therefore it may be appropriate to test the valuation of these types of investments as separate sub-populations. For a test of details, we determine our sample size using the sample size table in DTTL AAM Figure 230024.1 or DTTL AAM Figure 23002-4.2 (depending on whether the ROMM has been assessed as lower, higher or significant) as the tests of details sample size table (“TOD sample size table”). If the minimum sampling size of 50 (higher risk and relying on controls, multiples of PM 100x) is applicable to each sub-population, then the minimum sample size would be 100 (50 investments with readily determinable fair https://techlib.deloitteresources.com/#/toc/content

1/20

2/25/2018

Deloitte Technical Library

values and 50 investments without readily determinable fair values). [Note that further disaggregation or stratification may also be possible.] Why is this important?

Properly disaggregating a population is important in order to obtain an appropriate sample size to address an identified ROMM. Disaggregating a population is also relevant to how we evaluate the results of an audit sample, as discussed further in Chapter 6, “Evaluate the test results.”1 When a misstatement is identified in a sample, that misstatement is only projected to the disaggregated population in which the misstatement was identified.

4.1.2: Consideration of individually significant items Before determining a sample size, we also consider how to address individually significant items. Such items may include items that we determine to be high risk by virtue of size (i.e., exceeding performance materiality) or ROMM due to error or fraud. If a population is not expected to include a large number of individually significant items, then we generally will test all individually significant items. In this case, the individually significant items would not be subject to audit sampling. For example, assume we are testing investment purchases at a small entity that we do not expect to have significant purchases. In this circumstance, we may identify and test all purchases in excess of performance materiality, while subjecting the remainder of the population to audit sampling. The purchases in excess of performance materiality are not subject to audit sampling as they are 100 percent tested. If a misstatement is identified when testing the individually significant items, no extrapolation is necessary. Alternatively, if a population is expected to have a large number of individually significant items, then we may subject those items to audit sampling. For example, assume a diamond distributer sells a mix of high value and low value products, such that we would expect the sales class of transactions to contain a large number of individually significant items, but the risks associated with each are the same. In this circumstance, we may determine it is necessary to subject the individually significant items to audit sampling. We also may determine it is appropriate to stratify the population of individually significant items into a separate sub-population for sampling purposes. For example, assume we are testing the valuation of investments for an investment company in which it is common to have a relatively low performance materiality relative to the investment balance. In this circumstance, we would expect the entity to have a high volume of individually significant investments, as well as low value investments. In this circumstance, we may determine it is necessary to subject the individually significant items to audit sampling. We also may determine it is appropriate to stratify the population of individually significant items into a separate sub-population for sampling purposes. Why is this important?

Identifying and considering individually significant items in the population is important in order to properly design our sampling procedures. Properly considering individually significant items can lead to a more effective and efficient sampling approach.

https://techlib.deloitteresources.com/#/toc/content

2/20

2/25/2018

Deloitte Technical Library

Understanding how we have addressed individually significant items is also relevant to our evaluation of the results of the sample, as discussed further in Chapter 6, “Evaluate the test results.” 2

DTTL AAM

Considering the nature of the population and risks of material misstatement, we may often select larger items (because size is a characteristic of ROMM) when making our selection of items for the sample. [DTTL AAM 23002-4.39]

DTTL AAM

If the population generally isn’t expected to contain items that are expected to be greater than, for example, performance materiality, we typically select some or all of these items. We end up with two separate populations, one containing the items greater than performance materiality and then one containing items less than performance materiality. We select an appropriate sample size, based on Figure 23002-4.1 or Figure 23002-4.2, from the population containing items less than performance materiality.

For example, based on our understanding of the population’s characteristics, we may have determined that it is unusual for individual items in the population to exceed, for example, performance materiality. In these circumstances, we may choose to stratify the population into those items that exhibit that particular characteristic and those that don’t, selecting some or all of the items from the population that exhibits the characteristics of interest and selecting a sample of the population of items not exhibiting these characteristics. [DTTL AAM 23002-4.40] DTTL AAM

If the population is generally expected to be comprised of items that are greater than, for example, performance materiality, we are not necessarily required to select all of these items as the characteristic of being greater than performance materiality is not unusual. We select an appropriate sample size, based on Figure 23002-4.1 or Figure 23002-4.2, from the population. [DTTL AAM 23002-4.41]

4.1.3: Stratification of the population Section 2.4.3, “Stratification,” explains that in some circumstances we may stratify the population into smaller sub-populations. Audit efficiency may be improved if we stratify a population by dividing it into discrete sub-populations that have an identifying characteristic other than risk. The objective of stratification is to reduce the variability of items within each stratum and therefore allow sample size to be reduced without increasing sampling risk. [DTTL AAM 23002-4A.2] Populations can be stratified according to monetary value or a particular qualitative characteristic. When stratification is performed, we may spread the total determined sample size across the sub-populations, meaning that each individual sub-population does not need to have its own independently determined sample size. For example, assume we are testing the accuracy of the accounts receivable balance, which includes some large amount invoices and many small amount invoices (after excluding individually significant items which were tested 100 percent). The risk of material misstatement associated with the large and small amount https://techlib.deloitteresources.com/#/toc/content

3/20

2/25/2018

Deloitte Technical Library

invoices is the same, but we may determine it is appropriate to focus our audit procedures on the more material invoices. We may stratify the population into two sub-populations: (1) large amount invoices and (2) small amount invoices. In determining our sample size, we may allocate a portion of the total sample to each sub-population in order to result in a proportionately larger sample size for the large amount sub-population. Using the TOD sample size table, we determine the appropriate sample is 60 invoices based on the total population. If the large amount sub-population and the small amount sub-population include recorded amounts of CU86,000 and CU34,000, respectively, we might select 40 sampling units (i.e., approximately two-thirds, based on a ratio of 86 divided by 120) from the large amount sub-population and the remaining 20 sampling units from the small amount sub-population. Why is this important?

Stratification of a population is important in order to focus our audit effort on the transactions or balances of most importance to our audit procedure. Stratification can be particularly important when performing a non-statistical sampling approach, as it allows us to use our judgment and focus our selections on the populations of the most interest without inappropriately biasing our selection methodology. Understanding how we have stratified a population is also relevant to our evaluation of the results of the sample, as discussed further in Chapter 6, “Evaluate the test results.”3 If we identify a misstatement in the population that relates only to the identifying characteristics used to determine one of our sub-populations and the nature and cause of the misstatement can be found only in that sub-population, we project the misstatement only on the relevant sub-population and not the entire population.

Pitfall

Stratification of a population and not making selection(s) from each subpopulation.

For example, assume that we are performing tests of details to address a ROMM relating to the accuracy of operating expenses and have defined the population as all operating expenses recorded in the general ledger. Furthermore, we have divided the total population into three subpopulations based on identifying characteristics. Even if one of the subpopulations is immaterial, we still subject all sub-populations to sampling and allocate at least one sample selection to each of the three subpopulations. Otherwise, all sampling units would not have a chance of selection. However, we may determine during the risk assessment process that a particular sub-population does not represent a ROMM and therefore exclude it from the population to be sampled.

Note

If the only characteristic that we identify for stratification is monetary value, it may not be necessary to stratify the population if we use monetary unit sampling (MUS). MUS makes selections in a manner by which the probability of an item’s selection is proportionate to its size (i.e., the recorded amount). Therefore, when using MUS, the results of our sample selection will include a higher percentage of large monetary value items even if stratification is not performed. See Section 5.1.3, “Monetary Unit Sampling.”

https://techlib.deloitteresources.com/#/toc/content

4/20

2/25/2018

Deloitte Technical Library

4.1.4: Profiling of the population We also may determine it is appropriate to profile a population as described in Section 2.4.4, “Profiling the population.” Under the profiling approach, once our COAI have been identified, we divide the total population into sub-populations for each COAI. Any sampling units that do not have a COAI are included in a remainder population. We typically determine a sample size for each sub-population of items exhibiting COAI using the TOD sample size table. For the remainder population that does not exhibit any COAI, we typically determine the sample size as follows: The lesser of 10 items or that determined for the remainder population using the TOD sample size table, if we are able to rely on the operating effectiveness of controls. The lesser of 30 items or that determined for the remainder population using the TOD sample size table, if we are not able to rely on the operating effectiveness of controls.

For example, assume that we are performing tests of details for investment purchases using the profiling approach. We have tested the operating effectiveness of the controls over investment purchases and found them to be operating effectively. The risk we are addressing has been addressed as higher. Our performance materiality is CU250,000. We identified sub-populations with the following COAI: Investment purchase transactions that are processed differently than the normal processes for that type of instrument (Sub-population A) Purchases of a new investment type that has not been previously accounted for by the investment company and that exhibits characteristics that might require different processing than investments previously purchased (Sub-population B).

The table below demonstrates how we would calculate a profiling sample size for this scenario: Sub-population Sub-population A

Recorded Amount CU12,500,000

Profiling Sample Size 25

(50 multiples of PM) Sub-population B

CU15,000,000

30

(60 multiples of PM) Remainder Population

CU10,000,000

10

(40 multiples of PM) Total

https://techlib.deloitteresources.com/#/toc/content

87

5/20

2/25/2018

Deloitte Technical Library

Our sample sizes for the sub-populations with COAI (sub-populations A and B) were determined using the TOD sample size table. For the remainder population, we first determine the sample size based on the TOD sample size table. In this circumstance, we determined a sample size of 20 selections using the table (higher risk and relying on controls, multiples of PM 40x). As we are using the profiling approach, our determined sample size is the lesser of the number of selections determined using the table (20 selections) or 10 selections, therefore our sample size minimum would be 10 selections. Alternatively, assume the same facts as above, except that we determined 4 selections for the remainder population using the TOD sample size table (higher risk and relying on controls, multiples of PM 8x). In this circumstance, our sample size for the remainder population would be a minimum of 4 selections. Engagement teams are encouraged to use the Profiling Workpaper Template to assist in properly documenting the execution of the profiling approach. Why is this important?

Profiling a population is important in order to focus our audit procedures on the transactions or balances of most importance to our audit procedure. Profiling was not designed as a means to reduce our audit effort. Understanding how we have profiled a population is also relevant to our evaluation of the results of the sample, as discussed further in Chapter 6, “Evaluate the test results.”4 When a misstatement is identified in a sample, that misstatement is only projected to the profiled population in which the misstatement was identified.

4.1.5: Determination of the sample size If we use audit sampling, our sample sizes are determined using the table set out in DTTL AAM Figure 23002-4.1 or DTTL AAM Figure 23002-4.2 below. For ROMMs that have been assessed as lower or higher, DTTL AAM Figure 23002-4.1 should be used. For ROMMs that have been assessed as significant, DTTL AAM Figure 23002-4.2 should be used. DTTL AAM Figure 23002-4.1 Audit Sampling Sample Size Table- Lower and Higher Risks Not Relying on Controls

Relying on Controls

Population Size — Multiples of Performance Materiality Lower Risk

Higher Risk

Lower Risk

Higher Risk

1x

1

2

1

1

2x

2

3

1

1

3x

2

5

1

2

4x

3

6

1

2

https://techlib.deloitteresources.com/#/toc/content

6/20

2/25/2018

Deloitte Technical Library

Not Relying on Controls

Relying on Controls

Population Size — Multiples of Performance Materiality Lower Risk

Higher Risk

Lower Risk

Higher Risk

5x

3

8

1

3

6x

4

9

2

3

7x

5

11

2

4

8x

5

12

2

4

9x

6

14

2

5

10x

6

15

2

5

15x

9

23

3

8

20x

12

30

4

10

25x

15

38

5

13

30x

18

45

6

15

40x

24

60

8

20

50x

30

75

10

25

100x (*)

60

150

20

50

The sample sizes above represent minimum samples sizes. Engagement management may determine that, in some circumstances, it is appropriate to increase the sample sizes above those in this table. For populations in between the listed levels of performance materiality, we may interpolate to determine the appropriate sample size. (*) The engagement partner is required to consult with the engagement quality control reviewer to conclude on the approach for addressing risk(s) of material misstatement in populations that exceed 100x performance materiality (see paragraph 32 of Section 23002-4). https://techlib.deloitteresources.com/#/toc/content

7/20

2/25/2018

Deloitte Technical Library

DTTL AAM Figure 23002-4.2 Audit Sampling Sample Size Table- Significant Risks Significant Risk(!)

Population Size - Multiples of Performance Materiality

Not Relying on Controls

Relying on Controls

1x

4

2

2x

6

2

3x

10

4

4x

12

4

5x

16

6

6x

18

6

7x

22

8

8x

24

8

9x

28

10

10x

30

10

15x

46

16

20x

60

20

25x

76

26

30x

90

30

40x

120

40

50x

150

50

https://techlib.deloitteresources.com/#/toc/content

8/20

2/25/2018

Deloitte Technical Library

Significant Risk(!)

Population Size - Multiples of Performance Materiality 100x (*)

Not Relying on Controls

Relying on Controls

300

100

The sample sizes above represent minimum samples sizes. Engagement management may determine that, in some circumstances, it is appropriate to increase the sample sizes above those in this table. For populations in between the listed levels of performance materiality, we may interpolate to determine the appropriate sample size (!)For populations that contain a significant risk, we are required to perform substantive procedures that are specifically responsive to that risk (paragraph 41 of Section 13300). These specifically responsive substantive procedures frequently involve nonrepresentative selection. (*) The engagement partner is required to consult with the engagement quality control reviewer to conclude on the approach for addressing risk(s) of material misstatement in populations that exceed 100x performance materiality (see paragraph 32 of Section 23002-4).

It is important to understand the factors incorporated in these sample sizes in order to design an efficient sample plan. The remainder of this section addresses the concepts incorporated into the TOD sample size tables and how to appropriately use the tables to develop a minimum sample size for a test of details. It is also important to remember that the sample size tables above represents sample size minimums. Therefore, while a floor of a certain number of selections is set for certain higher multiples of performance materiality, we may use our professional judgment to determine whether an increase in sample size is necessary. See the “Minimum sample sizes” subsection below for further information. Pitfall

Failure to use the appropriate column and row in the TOD sample size tables when determining a sample size. Be sure to carefully check the sample size determined from the TOD sample size table.

Sample size concepts incorporated into the TOD sample size table The TOD sample size tables incorporate the following concepts relevant to determining a sample size for audit sampling: Performance materiality The amount or amounts set by the auditor at less than materiality for the financial statements as a whole to reduce to an appropriately low level the probability that the aggregate of uncorrected and undetected misstatements exceeds materiality for the financial statements as a whole. If applicable, performance materiality also refers to the amount or amounts set by the auditor at less than the materiality level or levels for particular classes of transactions, account balances, or disclosures. [DTTL AAM Glossary] https://techlib.deloitteresources.com/#/toc/content

9/20

2/25/2018

Deloitte Technical Library

Population size The population size is needed in order to select the sample size and project the sample result. The population size is incorporated into the multiples of performance materiality calculation in the TOD sample size table. The number of items in a large population often has little effect on the determination of an appropriate sample size for tests of details. However, when the population consists of a small number of very significant, but not individually significant items, we may involve the use of a sampling specialist because the concepts of audit sampling can be difficult to apply. In this circumstance, it may also be more efficient to perform non-representative selection as opposed to audit sampling.

Why is this important?

Understanding the factors that are incorporated in the TOD sample size tables enables us to better understand the table and utilize the sample size minimums as a guide to further apply our professional judgment as opposed to a mechanical application. It also assists us in evaluating the results of our tests, as described further in Chapter 6, “Evaluate the test results.”5

Multiples of performance materiality In order to use the TOD sample size tables, we first calculate the multiples of performance materiality. This is accomplished by dividing total monetary value of the population by our determined performance materiality. The multiple of performance materiality is used to determine which row of the TOD sample size table to use when determining the sample size. For example, assume we are testing fixed-asset dispositions and the total amount of dispositions in the population subject to sampling is CU2,000,000. If our determined performance materiality is CU500,000, we determine the multiples of performance materiality as CU2,000,000 divided by CU500,000, which results in 4 multiples. When using the TOD sample size table we use row “4x” to determine our sample size.

Risk of material misstatement and reliance on controls In order to determine which TOD sample size table to use and which column to use in the relevant table, we consider the ROMM associated with the test of details being performed and whether we expect the controls are operating effectively based on our testing or planned testing of the operating effectiveness of the related controls. If we use audit sampling for tests of details, we use the table set out in Figure 23002-4.1 for lower and higher risks and Figure 23002-4.2 for significant risks to assist us in determining an appropriate sample size. For lower/higher risks and Figure 23002-4.1, if we are relying on the operating effectiveness of controls we use the relevant column of the two columns to the right, under the heading “Relying on Controls”, and if we are not testing the operating effectiveness of controls we use the relevant column of the two columns to the left under the heading “Not Relying on Controls”. Similarly for significant risks, we use the right column in Figure 23002-4.2 if we are relying on the operating effectiveness of controls and the left column if we are not.

Minimum sample sizes Once we have determined the multiples of performance materiality, assessed level of ROMM, and our reliance on controls for the population subject to sampling, we use the TOD sample size table to determine a minimum sample size for the test of details. For example, assume the following facts: https://techlib.deloitteresources.com/#/toc/content

10/20

2/25/2018

Deloitte Technical Library

Performance Materiality — CU100,000 Monetary Value of Population being Tested — CU1,000,000 Risk of Material Misstatement — Lower Reliance on Controls — Yes

In this circumstance, we determine our multiples of performance materiality as 10x and use the “Relying on Controls” and “Lower Risk” third column of the TOD sample size table set out in Figure 23002-4.1 in order to determine a sample size minimum of two sample selections. It is important to remember that the TOD sample size tables only provides minimum sample sizes. In most instances, the minimum sample sizes are appropriate for our test objective, especially when experienced engagement team members have been properly involved in the risk assessment process. We are not required to document our considerations regarding why we determined the sample size minimums to be sufficient when we use the TOD sample size tables. In certain instances, increasing sample sizes above those stated in the tables may be appropriate and requires professional judgment. For example, using professional judgment, we may increase sample sizes when there is a history of errors in the population or when there are no other substantive procedures directed at the same ROMM. As the population size increases beyond the criteria set out in the TOD sample size tables (i.e., to above 100 times performance materiality), it may not be necessary to increase the sample size to obtain sufficient appropriate audit evidence in respect of the objective of the sample. However, if, when performing tests of details through audit sampling, the population that is being tested exceeds 100 times performance materiality, we are required to consult with the engagement quality control reviewer on the approach for addressing the risk(s) of material misstatement, including the extent of any tests of details. The purpose of the consultation is to consider whether the population has been appropriately identified and/or disaggregated (see Section 2.3, “Disaggregate the ABCOTD into populations for testing”), whether the assessment of risk related to the population has been appropriately understood and conclude on the appropriateness of the planned response (including determining whether other substantive procedures should be performed instead of, or in combination with, tests of details and if tests of details are to be performed, the appropriate extent of testing for such tests). [DTTL AAM 23002-4.32] Note that in some circumstances, the use of large sample sizes can have a diminishing return. In other words, if a population contains material misstatements, the misstatements are often detected after examining a relatively small number of items. Conversely, when no misstatements are detected within a relatively small number of sample items, then often none are detected throughout the remainder of the large sample. In very rare circumstances, we may determine it is appropriate to make fewer selections than the minimum sample size per the TOD sample size tables. In such circumstances, we recommend a discussion or consultation with the appropriate resources within the member firm. For example, assume that we are testing a ROMM relating to additions to property, plant, and equipment (PPE) for construction work in progress for an entity that provides wind power. In order to generate wind power, the entity builds wind plants. Based on our analysis of the population, we determined that there are three distinct sub-populations for additions to PPE, as indicated in the table below. https://techlib.deloitteresources.com/#/toc/content

11/20

2/25/2018

Deloitte Technical Library

Amount in subpopulation

Number of sampling units in sub-population

(1) Turbine Costs

CU35 million

10

(2) Machinery and Equipment

CU85 million

15

(3) Other

CU31 million

450

CU151 million

475

Sub-population

Total

Each of the sub-populations has the same risks, processes, and controls. The first and second sub-populations have a small number of high monetary value sampling units; therefore we may determine it is appropriate to test 100 percent of the sampling units in sub-populations 1 and 2 (i.e., sub-populations 1 and 2 will not be sampled). The third sub-population includes a high volume of low monetary value sampling units. If we were to use the TOD sample size table to determine the minimum sample size it would result in a minimum sample size of 60 selections (for a lower risk and not relying on controls). We analyzed the population and determined that by testing 100 percent of subpopulations 1 and 2, we have audited a significant portion of the total additions balance to construction work in process (approximately 79 percent). We also considered that the risk we are addressing is lower risk, there is no history of error, and the work in progress additions do not involve management judgment. Based on a robust analysis of the population and the related risk, the engagement team determined it was appropriate to consult with the member firm national office / NPPD to determine a reduced sample size for the “Other” sub-population that would provide sufficient audit evidence based on the extent of coverage over the entire population. Required Consultation

Leading Practice

If, when performing tests of details through audit sampling, the population that is being tested exceeds 100 times performance materiality, we shall consult with the engagement quality control reviewer on the approach for addressing the risk(s) of material misstatement, including the extent of any tests of details. The purpose of the consultation is to consider whether the population has been appropriately identified and/or disaggregated (see Section 2.3, “Disaggregate the ABCOTD into populations for testing”), whether the assessment of risk related to the population has been appropriately understood and conclude on the appropriateness of the planned response (including determining whether other substantive procedures should be performed instead of, or in combination with, tests of details and if tests of details are to be performed, the appropriate extent of testing for such tests). Excerpt from: [DTTL AAM 23002-4.32]

Consider a consultation with the appropriate resources within the member firm if contemplating a lower sample size than the minimum sample size

https://techlib.deloitteresources.com/#/toc/content

12/20

2/25/2018

Deloitte Technical Library

determined as per the TOD sample size tables.

Why is this important?

Properly designing our audit sample to be responsive to the assessed risks allows us to design a sampling approach that is both effective and efficient by focusing our audit effort on the items of most interest.

Interpolation When the calculated multiples of performance materiality fall between the multiples of performance materiality included in DTTL AAM Figure 23002-4.1 and DTTL AAM Figure 23002-4.2, we can interpolate to determine the appropriate sample size (alternatively if we do not want to interpolate, we can use the sample size indicated in the row for the next (higher) multiple of performance materiality that is listed on the table). When interpolating we use linear interpolation. For example, assume we are addressing a higher ROMM relating to a particular population related to that ROMM and relevant controls have not been tested for operating effectiveness. The population subject to sampling is CU379,000,000 and performance materiality is 30,000,000 which means that the population is 12.6333 times performance materiality. We perform the following steps: 1. Determine where the population being sampled lies between the multiples of PM provided in the audit sampling sample size table, expressed as a percentage (12.6333 – 10) / (15 – 10) = 52.67% 2. Apply the percentage calculated above to the difference between the two sample sizes provided in the audit sampling sample size table and add that to the lower of the two sample sizes provided in the table 52.67% x (23 – 15) +15 = 19.2136 3. Round up to determine the interpolated sample size 19.2136 -> 20

Note: Rounding up only needs to occur in this final step (i.e, 19.2136 -> 20). In this example we do not need to roundup the multiple (i.e., 12.6333) or the percentage (i.e., 52.67%).

Section 4.2: Tests of controls 4.2.1: Determination of the sample size If inquiries supported by inspection of documentary evidence are appropriate to provide reasonable assurance that controls are operating effectively at relevant times during the period of intended reliance, the appropriate number of selections may be that provided in the following DTTL AAM Figure 23001.1, assuming that the expected number of deviations is zero or one. https://techlib.deloitteresources.com/#/toc/content

13/20

2/25/2018

Deloitte Technical Library

DTTL AAM Figure 23001.1 (“Test of Operating Effectiveness of Controls Sample Size Table”)—Suggested sample sizes for inspection of documentation to support our inquiries for the purpose of testing the operating effectiveness of controls Nature of Control

Frequency of Performance of the Control

Number of Selections (1)

No Deviations Planned

One Deviation Planned

Manual

Many times per day

25

40

Manual

Daily

15

NA

Manual

Weekly

5

NA

Manual

Monthly

2

NA

Manual

Quarterly

1

NA

Manual

Annually

1

NA

Automated Controls

Test one instance of each automated control

General IT Controls

Follow the guidance above for manual and automated aspects of general IT controls.

(1) The sample size is dependent on the expected rate of deviations. When the control is not performed “Many times per day,” it may not be appropriate to plan for deviations.

Sample size concepts incorporated into the test of operating effectiveness of controls sample size table Expected population deviation rate The expected population deviation rate is the number of deviations we expect to find in the population. The higher the number of deviations we expect to find in our sample, the larger our sample size becomes in order to maintain the same confidence level. The test of operating effectiveness of controls sample size table above assumes zero or one deviation depending on the column used in the table. If we expect the deviation rate to exceed the tolerable rate of deviation, we generally do not plan to test the operating effectiveness of the control and modify our substantive procedures accordingly. Population size The size of the population often has little or no effect on the determination of sample size, except for relatively small populations as stated above. The test of operating effectiveness of controls sample size table has been designed to take into account the frequency of the control because the more frequently a control operates, the more sampling units there are in the population. https://techlib.deloitteresources.com/#/toc/content

14/20

2/25/2018

Deloitte Technical Library

Why is this important?

Understanding the factors that are incorporated in the test of operating effectiveness of controls sample size table assists us in evaluating the results of our tests, as described further in Chapter 6, “Evaluate the test results.”6

The remainder of this section addresses how to appropriately utilize the test of operating effectiveness of controls sample size table to determine a suggested sample size for a test of controls.

Nature of control (manual versus automated controls) In order to use the test of operating effectiveness of controls sample size table, we first must determine whether the nature of the control is manual or automated. The test of operating effectiveness of controls sample size table is designed to determine suggested sample sizes for manual controls only. If the control is a manual control, we can continue using the table to determine our sample size. If we are testing an automated control, it may be appropriate to test only one instance of each automated control when relevant general-IT controls are determined to be effective throughout the period of reliance. An automated control can be expected to function consistently unless the program (including the tables, files, or other permanent data used by the program) is changed. For further information regarding considerations related to the extent of testing of an automated control, see Section 3.4.3, “Extent of procedures,” of the DTTL Internal Control Guide.

Frequency of performance of the control The next step is to determine the frequency of the performance of the control. The frequency of the control translates to the number of times that the control operates. In most circumstances, the frequency of the control is straightforward; however, in some circumstances, determining the frequency of the control can involve the use of significant professional judgment. Consider the following examples: For example, assume we are testing the following control: On a quarterly basis, management meets and reviews quantitative and qualitative information for all reporting units to update the goodwill impairment analysis performed as of the most recent measurement date. In this circumstance, one analysis is reviewed by management on a quarterly basis; therefore, the frequency of this control is quarterly. For example, assume we are testing the following control: The accounting manager verifies that the billed revenue was properly recorded to revenue by comparing the billed revenue file to the revenue recorded in the general ledger on a monthly basis. The engagement team determined there are no planned deviations. If there is only one billed revenue file for this entity, the frequency of the control is determined to be monthly and the engagement team may determine a sample size of two selections based on the test of operating effectiveness of controls sample size table. https://techlib.deloitteresources.com/#/toc/content

15/20

2/25/2018

Deloitte Technical Library

However, if there are 20 different billed revenue files that the accounting manager verifies each month, then there are two potential approaches to sampling in this scenario: Scenario #1: If the sampling unit is defined as each month, then the population is 12 months and we may determine a sample size of two selections based on the test of operating effectiveness of controls sample size table. In this scenario, because the sampling unit is the month, we would test all 20 billed revenue files for each month selected. Scenario #2: If the sampling unit is defined as each billed revenue file, then the control operates 240 times each year (12 months x 20 billed revenue files = 240 instances). A control that operates 240 times per year is analogous to a daily control; therefore, we might determine the sample size to be 15 selections based on the test of operating effectiveness of controls sample size table. In this scenario, we would test 15 individual billed revenue files. We may spread the 15 selections across the intended period of reliance or spread throughout the intended period of reliance, but with more selections being made in the latter half of the year.

For example, assume we are testing the following control: The inventory counters use an inventory list to physically count each inventory item corresponding to their assigned, pre-printed inventory tags, and indicate the information that is required on all inventory tags, including the counter’s initials, date counted, quantity, and condition of inventory (good or poor). Although the full physical inventory count may only occur very few times per year, the sampling unit for this control is each individual count of an inventory item. The combined number of individual counts that all counters perform is often in the hundreds, if not thousands, which likely equates to a “many times per day” frequency. For populations that fall in between two rows in either of the test of operating effectiveness of controls sample size table (i.e., the frequency is not exactly daily, weekly, etc.), we use the higher suggested sample size from the table.

Expected deviations The test of operating effectiveness of controls sample size table assumes that the expected deviations are either zero or one depending on the column used. As described above, it may not be appropriate to plan for any deviation if the control is performed less often than “multiple times per day”. We may determine the expected deviations based on our professional judgment considering factors such as results of prior year’s tests, the design of internal controls, and the control environment.

Modifications to suggested sample size Once we have determined all of the elements in the test of operating effectiveness of controls sample size table, we can then determine our suggested sample size. For example, assume the following facts: Nature of the control: Manual Frequency of the control: Daily Deviations planned: Zero

In this circumstance, we determine our suggested sample size to be 15 selections.

https://techlib.deloitteresources.com/#/toc/content

16/20

2/25/2018

Deloitte Technical Library

The test of operating effectiveness of controls sample size table provides suggested minimum sample sizes; therefore, it is appropriate to use professional judgment to determine whether a larger sample size is appropriate. We may decide to increase our sample sizes in certain circumstances, such as when we are testing the following: A control that addresses multiple significant risks A common control that operates at multiple locations

In this circumstance, increasing the overall sample size allows for the possibility of identifying one or more deviations in our testing. Allowing for deviations may be appropriate as it may be practically difficult to expand our testing at a later date if a deviation is identified at one or more of the individual components or locations where we perform testing.

4.2.2: Other considerations Dual-purpose testing Dual-purpose testing means that two tests, with different purposes and objectives, are planned to be performed concurrently. This approach is sometimes taken when performing a test of details at the same time as a test of controls for a particular transaction. When determining sample sizes for the dual-purpose test, we perform the test using the larger of the two sample sizes required for the test of controls or the test of details. We also consider whether the same sample selection method is appropriate to meet the objective of both tests. For further information regarding dual-purpose testing, see Section 3.4.5, “Dual-purpose tests,” of the DTTL Internal Control Guide. Pitfall

Sample sizes for dual-purpose testing are insufficient to meet the objectives of both tests.

Use of one sample to test multiple controls In some circumstances, it may be efficient to use one sample to test the operating effectiveness of more than one control. For example, a sample of shipping documents can be traced to sales invoices and cash receipts to test various attributes such as credit approval and evidence of quantity, price, and extension checks. When using this approach, it is important to determine that the population the sample is selected from is appropriate for both controls.

Timing of tests of controls When we choose to test the operating effectiveness of controls as of an interim date, there are typically two approaches we may consider. As described in Section 3.4.2, “Timing of tests of controls,” of the DTTL Internal Control Guide, the determination of our sample size will vary depending on which approach we use:

1.

Apportion the test of controls over the year (i.e., spread the total number of selections throughout the year). Under this approach, the operating effectiveness result is determined only upon completion of the test at

https://techlib.deloitteresources.com/#/toc/content

17/20

2/25/2018

Deloitte Technical Library

year-end. Performing our testing in this manner provides the basis to support our conclusions as to the effectiveness of the controls throughout the period of intended reliance. As the testing is apportioned over the entire year, rollforward procedures are not necessary. For example, for a test of a relevant control using a sample size of 25, we may choose to perform a portion of the test at interim by selecting 20 items over the first nine months and then selecting the five remaining items in the fourth quarter. We cannot reach a conclusion on the operating effectiveness of the control at the interim date (September 30) since we did not test all 25 items. We can only reach a conclusion on the operating effectiveness of the control when our testing of all sample selections is complete at year-end. Since we sampled to cover the entire period, we are not required to perform separate rollforward procedures.

2.

Perform a complete test of the control (i.e., test all selections) at an interim date. This approach requires us to perform sufficient testing to enable us to reach a preliminary conclusion regarding the operating effectiveness of the control at the interim date. Under this approach, additional procedures are required to be performed to assess the operating effectiveness of the control during the rollforward period. The earlier in the year the interim tests are performed, the more persuasive the rollforward procedures will likely need to be. For example, for a test of a relevant control using a sample size of 25, we may choose to perform the entire test at interim by selecting 25 items over the first nine months. Therefore, we can reach a conclusion on the operating effectiveness of the control at the interim date (September 30) since we tested all 25 items. However, we need to perform separate rollforward procedures to determine whether the control continues to be effective through the fourth quarter and near to or at the as-of date.

Section 4.3: Attribute sampling Attribute sampling is a form of sampling that can be used to reach a conclusion about a population in terms of a rate of occurrence. Attribute sampling can only measure a characteristic with two outcomes (e.g., present or not present, pass or fail, yes or no). Attribute sampling is commonly used to estimate the rate of occurrence of deviations when testing documentary evidence that a control is operating effectively.

4.3.1: Attribute sampling for tests of controls Attribute sampling is commonly used when testing the operating effectiveness of a control because there are only two possible outcomes: The control is operating effectively The control is not operating effectively

When using attribute sampling to test a control, we design our sample size in accordance with the test of operating effectiveness of controls sample size table, as discussed in Section 4.2, “Tests of controls.”

4.3.2: Attribute sampling for substantive procedures As described in Section 3.2.1, “Attribute sampling,” attribute sampling may be appropriate to use for substantive testing purposes in certain circumstances. It is important to remember that when we use attribute sampling for a substantive procedure, we are only able to conclude whether the actual rate of deviation exceeds the tolerable rate of deviation. The results of an attribute sample are typically not evaluated through a monetary basis; therefore, we are unable to reach conclusions about a population in terms of monetary amounts. For further guidance on using attribute sampling for substantive procedures, see Section 4.4.2 “Attribute Sampling for Substantive Tests” of the US Audit Sampling Guide.

Section 4.4: Sample size considerations for a group audit Determining a sample size for tests of details or tests of controls for a group audit involves some special considerations as compared to a single location entity. https://techlib.deloitteresources.com/#/toc/content

18/20

2/25/2018

Deloitte Technical Library

When designing our substantive procedures for a group audit, we generally determine that it is necessary to design our audit procedures at the individual component level. Therefore, if we determine that audit sampling is appropriate for a particular test (i.e., test of details), our sample size is also determined at the component level. For example, assume an entity is a retailer of women’s clothing and has two significant components, Component A and Component B. Each component has different financial management and IT systems in place to process revenue transactions, although the products being manufactured are the same. In this circumstance, we may determine it is appropriate to perform substantive audit procedures to test revenue at each component separately, as opposed to aggregating the revenue balance together and performing testing at the group level. If we determined sampling was appropriate to address the ROMMs related to revenue at both components, we determine a separate sample size for Component A and Component B. In certain circumstances, there may be account balances that we may test in the aggregate (i.e., testing the consolidated material balance as a single population). In order to test an account balance in the aggregate, the level of centralization and consistency across components must be analyzed. When we test an account balance in the aggregate, the process we follow to determine the sample size and identify selections is consistent with that of a single location entity. Therefore, even when testing an account balance in the aggregate, we still consider whether stratification of the population is appropriate. For example, assume an entity has two significant components, Component C and Component D. The entity has a centralized process to account for payroll expenses at the corporate office location. In this circumstance, we may determine it is appropriate to test the consolidated balance in the aggregate at the group level. Therefore, if we determined sampling was appropriate to address the ROMMs related to payroll expenses, we determine one sample size for all payroll expenses at the group level and make selections from the aggregate population (which would include both Components C and D). In regards to tests of controls, entities that operate in a multi-location environment generally design their controls in one of three ways (or a combination thereof): Separate controls at each component Centralized controls performed at a single location for all components Common controls that are designed and monitored centrally, but are implemented and performed at each component in accordance with the manner in which it was designed

If the entity has separate controls at each component, we design a separate sample size for each relevant control at each component. If the entity has centralized controls, we may design one sample size based on the test of operating effectiveness of controls sample size table that covers all components. If the entity has common controls that operate across multiple components or locations, we may design one sample size that covers all components, based on the test of operating effectiveness of controls sample size table, and consider whether we need to increase the overall sample size above that suggested by the table due to a potential increased risk of ineffectiveness since the controls operate at each component or location. The same considerations as discussed above apply to components that have multiple locations.

Section 4.5: Documentation considerations https://techlib.deloitteresources.com/#/toc/content

19/20

2/25/2018

Deloitte Technical Library

We are required to follow the audit documentation requirements in DTTL AAM 00200, Audit documentation. However, as it relates to the determination of sample sizes, we may document the following: How we disaggregated, stratified, or profiled the population How we treated individually significant items when determining our sample size How we determined the appropriate column and row of the sample size table to use How we determined the sample size if the TOD sample size table was not utilized

________________________________________________ 1 This 2 This 3 This 4 This 5 This 6 This

content is pending and will be announced when issued. content is pending and will be announced when issued. content is pending and will be announced when issued. content is pending and will be announced when issued. content is pending and will be announced when issued. content is pending and will be announced when issued.

Confidential and Proprietary - for Use Solely by Authorised Personnel Copyright © 2018 Deloitte Touche Tohmatsu Limited. All rights reserved.

https://techlib.deloitteresources.com/#/toc/content

20/20