Method comparison or procedure comparison?

Comparing POC analyzers (methods) with LAB analyzers (methods)

The main purpose of comparison studies is to determine the analytical difference between two methods (or analyzers).

However, when comparing analyzers placed at POC with LAB equipment, different results may not only be due to the analytical difference between the analyzers but also to e.g. different sample handling procedures. The contribution from the difference in procedures may be of a larger magnitude than the contribution from the analytical difference between the two methods.

Below are given examples of how procedures may be different:

1. The sample measured at POC often needs no sample preparation as whole blood can be used directly. Storage time is shorter or eliminated, and thereby storage time and temperature play minor roles for the sample measured at POC, but still affect the sample sent to the LAB.
2. The different measuring technologies may require different ways of obtaining the sample, e.g. a whole-blood capillary sample may replace the typical venous serum/plasma sample. The physiological difference between samples obtained from capillaries, veins, or arteries varies a lot from one parameter to the next; e.g. pO₂(a) cannot be measured in a venous specimen, whereas the difference between an arterial and a venous sample is not clinically significant for sodium.
3. Implementing POC analyzers also typically means that staff that has not been previously involved in blood testing is now involved and will also be involved in method comparison studies. Avoiding preanalytical errors and training of staff are therefore even more important than before.

When an analyzer to be used in a POC setting is to be compared with the traditional LAB methods/analyzer, the ideal approach is to conduct a two-step comparison:

• The first step is to perform a comparison of the two methods by placing the analyzers side by side. If the comparison study is performed correctly and on the exact same sample, this should then reflect the analytical difference between the methods, including the necessary sample preparation (e.g. centrifugation versus no centrifugation), i.e. a method comparison.
• The next step will then be to place the POC analyzer where it is to be used and then compare it with the LAB method. The results from this test will reflect the analytical difference (which has already been quantified in the previous study) plus the difference caused by using different procedures for obtaining and handling the blood sample, i.e. a procedure comparison.

Unfortunately, step one is not necessarily conducted. This may work out well if the conclusion from the study takes the differences in procedures into consideration. If method and procedure comparison are mixed up, it may lead to erroneous conclusions about the difference between the measuring performance of the analyzers involved; a difference that is actually caused by a difference in procedures. These erroneous conclusions may have a negative impact on patient treatment for years to come.

Procedure comparisons as such may be very beneficial. For instance, they may be used to highlight the importance of correct handling techniques and be an incentive to evaluate procedures currently used, to take new procedures into use, and to (re-)train all staff involved. The outcome of such an effort will be to ensure that that uniform results are reported throughout the hospital.

Table 1 is an overview and a tool for an easy understanding of how the results from a comparison test can be evaluated, i.e. method comparison versus procedure comparison.

Table I. Method comparison or procedure comparison?
* It may be argued that a difference in sample preparation is a difference in procedure; however, the sample preparation may be a necessity, thereby making the effect difficult to quantify, and it is thus categorized here in the 'method comparison' group. The same argument can be used for sampling devices; however, here one often has the option of several sampling devices, and they are therefore grouped under 'procedure comparison'.

What to keep in mind

The following is a list of various important factors to keep in mind, of ideas on how to handle method and procedure comparisons, and a few reminders. 

Preanalytical variables in general

Numerous guidelines are available for comparison studies in the laboratory, e.g. NCCLS guidelines. Many of these also deal with preanalytical issues that must be taken into consideration, as these play a major role. It may be difficult to get a firm grip of the preanalytical variables but various publications providing valuable information on this are available [1,2,3,4].

Same sample

Strictly speaking, for a method comparison only one sample should be obtained and then split into two. Any deviation from this causes a potential difference in the sample. If the sample cannot be spilt into two, the effect of using two different samples should be quantified according to local procedures.

Sample preparation

This is primarily of importance if one method uses whole blood and the other plasma/serum. This raises two problems: 1) Conceptually, it is not the same sample that is compared. 2) It is virtually impossible to measure the samples at the same time to avoid contribution from continued metabolism or other changes.

Example: If the measurement of electrolytes on a whole-blood BGA analyzer is compared with a chemistry analyzer using serum/plasma, then the part of the sample for the chemistry analyzer needs to go through the steps of separating the sample by centrifugation. The ideal approach is to measure the sample on the whole-blood analyzer at the same time as the sample is separated.

After the sample has been separated, it should be measured immediately and not wait in line at the chemistry analyzer for 'its turn'. This will then give information on the difference between the methods. Afterwards, the more realistic test can be made, following the procedures usually adhered to.

For neonatal specimen, there is a special problem that should be addressed for chemistry analyzers: evaporation! The neonatal sample, which is stored in a microcup after centrifugation, has a large surface area compared with the total volume. Unpublished test results have shown that evaporation from microcups may increase the concentration of the parameters by up to 10 % over two hours [5].  

Sample storage time and temperature

The storage time is sometimes thought to be of importance for blood gas parameters only. Parameters that are less prone to effects from storage after having been obtained are unfortunately wrongly neglected, e.g. glucose and lactate. A study showed the following: At room temperature, whole-blood metabolism decreased glucose levels by 4.6 % and increased lactate levels by 20.6 % over 30 minutes [6]. A good overview of the allowed storage time for different samples can be found in various communications [7].

Sampling device/procedure

Some analyzers require that the samples be obtained in test tubes, others in syringes and capillaries. Some samples are obtained by needle stick in the blood vessels, others through catheters. And on top of that, different anticoagulants may be used. If possible, the differences in these procedures should be quantified.

Physiological difference

It is well known that the reference range of some parameters cannot be used interchangeably for the different sampling sites: arterial, capillary, and venous samples. Blood gases are one example. However, for some parameters it is believed that there is no difference between e.g. venous and arterial or venous and capillary samples, even though different claims may be found in the literature.

An example is glucose and bilirubin, where it is difficult to reach a conclusion from the literature about the physiological difference between the sampling sites typically used. This is probably one of the most difficult issues to deal with and the best advice is to consult the literature (even though this does not provide a clear-cut answer).

Conclusion

Performing comparison studies is a major task but also a very challenging one. It may at first seem difficult. If attention is not paid to the difference between method and procedure comparison, it may lead to erroneous conclusions that may impact patient treatment for years to come. However, with proper attention, it may turn out to be very rewarding for the entire organization involved in the daily routines connected with blood testing.

It may lead to an increase in the level of understanding for all personnel groups of the processes involved in blood testing, may cause a revision of procedures, and increase motivation in the daily work processes. These efforts should ensure that that uniform results are reported throughout the hospital.