Optimizing accuracy and precision for point-of-care tests

SCOPE OF POCT

The testing performed at the point of patient care (popularly referred to as POCT) has increased in number of sites to represent the majority of clinical laboratory testing sites worldwide. The POC sites are heterogeneous in terms of patient care services, and they include hospital patient care units and clinics, physician private clinics, home health facilities, etc. Extrapolation (Fig. 1) from the 2010 Center for Medicare and Medicaid Services (CMS) database indicates that 66 % of the 215,057 registered non-exempt laboratories in United States of America (USA) are designated as performing only waived tests [1].

Further review of the CMS database shows that 52 % of the registered laboratories are physician office laboratories that are solely designated as POCT. The trend towards adoption of POCT will increase as technological advances continue to enhance the development of waiver and non-waiver test devices that are easy to adopt for POCT. We will therefore see more of the approximately 10 billion laboratory tests performed each year in the United States shifting toward the POCT.

 
FIG 1: CLIA laboratories by certificate types showing the disproportionately high number of testing sites with waiver certificate among the non-exempt testing sites (N = 215,057). Extracted from CMS CLIA database

The test results produced at the POC are being used as alternate to central laboratory (CL) test results primarily due to the notion that POCT facilitates timeliness of patient test results. POCT is thus an attractive practical alternate to the CL because, in theory, it enhances therapeutic interventions. The simplicity and the supposed determination that the devices have insignificant risk of producing erroneous results enable most POCT devices to be classified in the “waived tests” category by the US Center for Medicare and Medicaid (CMS) [2]. It is, however, not uncommon to perform some “non-waived tests” (e.g., troponin) in the POC setting.

The analytes included in the POCT menu are to a large extent defined by medical need (critical, emergency or urgent care), convenience or optimization of care continuity (outpatient physician office) and self-monitoring. Representative examples of the point-of-care (POC) analytes are pH, pCO₂, pO₂, electrolytes, lactate, glucose, cardiac markers (troponin, CKMB, BNP), PT, PTT, hemoglobin A1c, hematocrit and hemoglobin, hCG, HIV, fecal occult blood, urinalysis, etc.

QUALITY OF POCT

The use of POCT devices for disease screening, diagnosis, and monitoring of treatment has mixed reviews in the scientific literature in terms of test-result precision and accuracy [3, 4]. In the work by Steinfelder-Visscher et al, the use of a POCT device for blood gases and electrolytes is reported to have a constant deviation in the normal reference ranges for the analytes.

However, when the POC device was used in critically ill patient care testing, linear trends in deviations relative to the CL analyzer were observed for pH, pO₂ and hematocrit [3]. Thus, while POCT has an overall advantage in ensuring timeliness of test results, the accuracy and reliability of the results are often in question. For a POCT testing site to be in full compliance with the US Clinical Laboratory Improvement Amendments (CLIA 1988 – “… establishing quality standards for all laboratory testing to ensure the accuracy, reliability and timeliness of patient test results regardless of where the test was performed”), the accuracy, reliability and timeliness of all results must be addressed.

Instrumental and procedural causes of test-result inaccuracy and imprecision have been attributed to failure in all the three phases in the POCT cycle

• Preanalytical phase
• Analytical phase
• Postanalytical phase

Though information is sparse for POCT error rates, it is estimated from findings for CL that about 90 % of the quality issues in POCT are associated with the preanalytical and postanalytical phases [5].

Significant effort by manufacturers in recent years has reduced POCT device errors, especially in the analytical and postanalytical phases [6]. The improvement in the test-result quality at the analytical phase has in part been facilitated by the United States Food and Drug Administration (FDA) that requires inclusion of accuracy (and traceability data) and precision claims in the documents submitted for review and approval of the devices. Issues with the postanalytical phase have also been significantly reduced by adoption of connectivity that enables direct electronic transmission of test results, specifically in large healthcare facilities with high-volume POCT that justifies acquisition of the network system [7].

Improvements in the preanalytical phase have also been addressed by the manufacturers by introduction of device features such as requirement for operator identification, competency documentation and lockout functionalities. The success in improving the test-result quality at the preanalytical phase is, however, still dismal because of site-specific operations policies that may not implement the devices which are designed for quality improvement.

The lack of a strict regulatory guideline and external monitoring system for activities in the POC setting has continued to negatively impact test-result quality as indicated by the high rate of spurious defects that occur per procedural lapses. In this paper, the preanalytical causes of inaccuracy and imprecision of POC testing, and the impact of such errors on patient care are discussed with proposed recommendations for mitigation of the POCT errors.

EVALUATING ACCURACY AND PRECISION OF POCT

The quality of the testing at the POC setting is quantitatively evaluated in terms of test-result accuracy (closeness of agreement between a measured value and the true value) and precision (closeness of agreement between independent measurements of a quantity under the same condition). Deviation from 100 % accuracy (inaccuracy) is caused by both systematic and random errors (definition – failure of a planned action to be completed as intended or use of a wrong plan to achieve an aim), while precision issues (imprecision) are due solely to random errors.

In this review, the quality of POCT will be evaluated in terms of accuracy and precision and quantified by total error (TE). The POCT total error is deduced from the following equation that accounts for systematic (bias) and random (mainly impacts imprecision and accuracy) errors:

TE = % Bias + 1.96 (CV%) [8]

where:

Bias - indicates how a series of measurements agree with the true value;
used interchangeably with inaccuracy
CV% - standard deviation divided by mean times 100;
measure of imprecision

The equation can be used to estimate total error as follows: In a scenario where a manufacturer’s specification for a device indicates inaccuracy relative to a reference method or system of 3 % (bias) and average imprecision of 5 % (range = 2.5 – 7.5 %), the average total error will be as follows:

TE = 3 % + 1.96 (5 %) = 12.8 %

The error range for this example is 7.9 – 17.7 % (calculated from TE = 3 % + 1.96 (2.5 %), and TE = 3 % + 1.96 (7.5 %), respectively). Thus, the random errors with a resultant increase in imprecision due to failure to comply with well-designed standard operating procedure can significantly impact the quality of test results. This random error is illustrated by the work of Skeie, et al, with CV% ranging from 7-20 % when the glucose meters were operated by the patients versus 2.5-5.9 % when operated by technicians [9]. The ranges of total error for the tests performed by the patients and the technician in the Skeie et al paper are 16.7 to 40.2 % and 5.9 to 12.6 %, respectively, if we assume that the bias in the system is only 1 %.

CAUSES OF POCT ERRORS/VARIATIONS

Overall, error in healthcare has been estimated to occur in 2-4 percent of all hospitalizations [4, 10, 11]. Though the proportion of these errors attributable to POCT has not been well documented, preliminary findings indicate POCT quality concerns that are mainly related to the lack of oversight and requirements for personnel qualification/training.

The CLIA waived tests are not subject to the strict regulatory requirements (personnel, quality control, proficiency testing or other quality assurance) to which moderate complexity laboratories are subjected. The POC settings performing the waived tests are only required to follow the specifications of the assay manufacturer (manufacturer’s operating instructions). All non-waived testing performed at the POCT are, on the contrary, subject to regulatory oversight with accreditation or compliance inspection [1]. The lack of strict regulations has resulted in failure on the end-user’s part to:

In POCT, the high number and heterogeneity of testing sites, diverse educational background and technical experiences of the testing personnel, infrequency of testing, and responsibility for other non-POCT activities are concerns that should be evaluated prior to the implementation of the service. POCT typically has more testing devices than the CL (e.g., approximately 100-200 POC glucose meters compared to 2-3 main analyzers in CL), and the staff perform fewer tests per day (0-10) versus 400-800/day/staff in the CL. These compounding factors increase frequency of error that negatively impacts the quality of test results, especially at the preanalytical phase of testing.

The characteristics of the POCT specimen should also be properly understood relative to the impact on test results. These specimens are typically easy to obtain and are unprocessed or minimally processed blood. Most of the specimens are random capillary blood that can vary from collection to collection due to failure to follow proper technique. For test-result interpretation, one should pay attention to documented differences between capillary whole blood compared to venous plasma or serum. For example, the following are known causes of variations for POC glucose testing versus CL glucose [12]:

• Capillary blood glucose is 20-25 % higher than venous blood during rapid change in plasma glucose.
• Glucose concentration in plasma is ~11 % higher than whole blood: Although glucose flows freely in and out of red cells, the concentration of water (Kg/L) in plasma is ~11 % higher than red cells.
• Glucose concentration in heparinized plasma is ~5 % less than serum (possible shift of fluids from erythrocytes to plasma by anticoagulants).

Some of the common preanalytical errors that may cause systematic and/or spurious POCT results with deviation from the CL results are:

• Poor phlebotomy technique for obtaining capillary blood: shallow skin puncture (causing slow blood flow), squeezing too hard, failure to wipe away first blood (rich in clot factors) are examples of poor techniques that will impact test result.
• Differences in specimen types and sources: bias of up to +11.5 %, –6.1 % and +5.6 % have been documented [13] when we compare capillary whole-blood glucose concentrations to capillary plasma, venous whole blood and venous plasma, respectively.
• Assay-specific interfering substance (medications, food additives, etc.) – e.g., ascorbic acid, acetaminophen and dopamine.
• Exposure to ambient air causes increase in pO₂ and decrease in pCO₂.
• Delayed analysis leading to decreased pO₂ and increased pCO₂.
• Excessive heparin: dilutional effect on HCO₃^- and pCO₂.
• Air bubbles: increases pO₂ and decreases pCO₂.
• Type of syringe: pO₂ value drops more rapidly in plastic syringe.
• Use of the test device for testing for which it was not approved (screening versus diagnostic test) by the United States Food and Drug Administration (or other appropriate governmental agency).
• Lack of standard operating procedures or failure to operate as specified by the manufacturer.
• Testing staff not familiar with the operating procedure.
• 70Use of expired reagents.

The following references provide detailed analyte/device-specific discussion on causes of testing errors and their impact on patient care [14-16].

PATIENT CARE IMPACT OF POCT ERRORS

Systematic errors cause bias that can lead to either consistently positive or negative change in the test results. Since the direction of the bias is predictable, an attentive testing staff or astute clinician can usually detect these errors and institute appropriate corrective action to prevent negative impact on patient care. Random errors as cause of bias or imprecision are, on the contrary, more difficult to detect because the change in test result can be either positive or negative, with equal distribution on both sides of the mean. It is therefore more difficult to detect changes in test result due to random error, except in cases when there is repeat testing. These errors may cause one or more of the following unintended outcomes [17,18]:

• Misdiagnosis and even death
• Missed (no) diagnosis and no treatment when needed
• Incorrect diagnosis and improper treatment with harmful outcomes
• Repeat testing due to questionable results
• Costly follow-up procedures, including unnecessary surgery
• Worry and anxiety for patients and parents/families
• Additional costs to the healthcare system
• Financial hardship for the consumer public

MITIGATION OF INACCURACY AND IMPRECISION IN POCT

Errors can for the most part be avoided by ensuring that a planned action is executed as designed; failure to execute the standard operating procedure as intended or use of a wrong protocol will cause error with resultant adverse effect. Thus, the activities in all three phases (preanalytical, analytical and postanalytical) of POCT should be evaluated with attention focused on the preanalytical phase because it presents numerous opportunities for improvement. Specific strategies for improvement should include:

• Evaluate and select devices that are appropriate for the intended use (screening, diagnosis and/or monitoring therapy). The device must meet quality (accuracy and precision) specifications that are required to ensure that results are adequate for clinical diagnosis.
• Validate the devices to ensure that they perform as specified in the package insert provided by the manufacturer. Periodic bi-annual or annual revalidation of the specifications is recommended for devices that are used for diagnostic and treatment monitoring purposes.
• Develop SOPs and policies that specify training requirements for the testing personnel and recertification by annual competency checks, reagents and QC material handling, device maintenance, specimen collection (including pretest handling), QC protocols and allowable limits with corrective actions for out-of-control, defined analytical sensitivity and upper limit of linearity, etc.
• Device limitations in terms of impact of medications, clinical conditions, etc. should be available and appended to results as applicable.
• Period (annual) audit of the overall POCT program to include patient care outcomes.

CONCLUSION

The long-term use of POCT will only be viable option if all the requirements of accuracy, reliability and timeliness for all test results, irrespective of the testing site (CL or POCT) are met. To significantly improve the quality of POCT in the accuracy and reliability aspects, the preanalytical phase requires more oversight. It is especially urgent in the current medico-legal environment that each testing site re-evaluates their POCT program for accuracy and reliability based on patient care outcomes.