What is Statistical Significance?

In simple terms, "statistical significance" tells you that a piece of information, resulting from a study of data, is reliable; and a difference or relationship found in the statistic did not happen by chance.

Standard Deviation

The primary statistical significance test for utilization (Incumbency versus Availability) and Adverse Impact (of personnel actions) is known as calculating the "standard deviation." A standard deviation score of 2.00 or greater identifies a result as statistically significant.

Utilization:
[(Availability x Total Persons in Job Group) - Number of Protected-class Employees] / the square root of [(1 - Availability ) x Availability x Total Persons in Job Group]

Adverse Impact of positive personnel actions:
Favored Class Selection Rate - Non-favored Class Selection Rate / the square root of (Overall Selection Rate * (1 - Overall Selection Rate) * ((1 / Total non-favored) + (1 / Total favored))

Adverse Impact of negative personnel actions:
Non-favored Class Selection Rate - Favored Class Selection Rate divided by the square root of (Overall Selection Rate * (1-Overall Selection rate) * ((1/total non-favored) + (1/total favored))

Exact Binomial

For a Job Group with fewer than 30 employees, the Exact Binomial test will be used to determine the statistical significance of utilization. An exact binomial result of .05 or less indicates a result as statistically significant.

P^N + (((N/1) * P^(N -1)*Q)) + (((N * (N-1))/(1 * 2))) * (P^(N-2) * Q^2) + (((N * (N-1) * (N-2))/(1 * 2 * 3))) * (P^(N-3) * Q^3) + .... + Q^N

Where: P = The probability of being non-protected (1 - availability of the protected group)

Q = The probability of being protected (the availability)

N = The number of people in the group (i.e. the number of trials)

The aggregation of the expanded members should only include up to the number of protected members + 1 (i.e., only add the first four expansion terms when there are three protected members in the group).

In the exact binomial formula, the "^" is the symbol for an exponential.

Fisher’s Exact Test

For Adverse Impact, if there are fewer than 30 incumbents or applicants in the the Job Group, the Fisher’s Exact test will be used to calculate significance. Fisher's Exact scores of .025 or lower indicate a result is considered statistically significant. The threshold of 30 can be adjusted under: Plan Settings (with careful consideration and consultation with a statistician).

Rule of Nines

When selecting Significant Difference as the selected Utilization Rule, an alternate rule can also be selected. The alternate rule will be used instead of the significant difference rule if the group size is too small, as determined by the Rule of Nines test.

The formula is the total number in the group multiplied by the protected class availability multiplied by its complement. The result should be greater than nine to rely on standard deviation as a test of significance.

(Job Group Total) * (Protected Class Availability) * (1 – Protected Class Availability)

Example: If a Job Group has 76 employees and the availability for minorities is 7.8%, the formula would be: 76 x .078 x (1 - .078) = 5.47.

In this instance, the validity of standard deviation to measure the significance of any differences between minority availability and employment would be questionable.

This rule is used infrequently because it may establish an overly high “small group” threshold. For example, under the Rule of Nines any Job Group with less than 36 employees will score less than 9. Some statisticians would set a lower absolute threshold.