Performance Testing
BSI utilizes scenario test procedures in a setting comparable to an office environment. Two copies of a product are placed in the laboratory for testing by volunteer operators in a simulated operating environment to help determine how the product will perform in its intended normal application environment. The Performance Test evaluates the ability of a biometric device to operate against a set of performance levels, also known as Common Performance Standards (CPS).
To conduct these tests, BSI recruits and maintains a Volunteer Operator pool of 200-500 people to provide as reasonable a demographic and ethnic cross section of the general population as possible for ages 18-80 based on the US national population (as reported by the US Census Bureau, 2000 in the below graphs). Each operator visits the labs at least once each week for several rounds of testing on actual products installed on a wall or suitable surface as they would be in actual operation. During this period, each device is activated a minimum of 10,000 times. The performance measures, deterred by actual activations as opposed to theoretical computer analyses, measure False Accept (FAR), False Reject (FRR), Failure to Acquire (FTA), Failure to Enroll (FTE) and Throughput rates.
Proprietary test management software controls the test flow to assure accurate data collection and to minimize data collection errors. The following are the four main CPS each device will be tested against.
| CPS | How It's Computed | How It's Reported | Target Value |
| False Accept Rate (FAR) |
A false accept occurs when a person not enrolled on a particular device is "recognized" or accepted by it. The FAR is computed as the number of false accepts observed on a device by the total number of imposter decisions made during the test period. For a family of devices, it is the total number of false accepts on all such devices divided by the total number of imposter decisions made by all the devices. |
It is reported as a percentage of total imposter decisions made by the device OR as a percentage of all imposter decisions made on all devices in the tested family. |
≤0.5% |
| False Reject Rate* + Failure to Acquire Rate |
A false reject occurs when a person enrolled on a particular device and submitting a correct PIN is rejected or not recognized. A failure to acquire rate is the frequency that the device fails to acquire biometric information required for a decision. For a family of devices, it is the total of false rejects plus the failed attempts to acquire on all such devices divided by the total number of genuine decisions made by all the devices. |
It is measured as a percentage of total genuine decisions made by the device OR as a percentage of all genuine decisions made on all devices in the tested family. |
≤5.0%; |
| Failure to Enroll Rate |
A failure to enroll rate is the failure of the biometric system to form a proper enrollment template for an end-user. For a family of devices, it is the total of failed attempts to enroll on all such devices divided by the total number of enrollment attempts made by users on all the devices. |
It is measured as a percentage of all user enrollments attempted on an individual device OR as a percentage of all user enrollments attempted on all devices in the tested family. |
≤5.0% |