Newsletter

Sign up for our quarterly newsletter and get the newest articles from acutecaretesting.org

Printed from acutecaretesting.org

Article

July 2015

Comparing D-dimer assays using likelihood ratios and Fagan nomograms

by Hiromi Saeki, Suzanne Ekelund
Information management Coagulation/fibrinolysis D-dimer

Clinical sensitivity, clinical specificity and predictive values are frequently used as performance measure for a biochemistry assay. When D-dimer assay performance is presented, emphasis is often on the negative predictive value (NPV).

However, predictive values are greatly influenced by the prevalence of the disease and cannot be generalized beyond the studied population.

Instead of comparing negative predictive values, likelihood ratios can be compared. Likelihood ratios have the advantage over negative predictive values that they do not depend on the prevalence of the disease.

Likelihood ratios are considered useful measures of diagnostic accuracy because they can be used to estimate posttest probabilities. When we have the pretest probability, the test result and the likelihood ratio of the test, we may find the posttest probability by using the so-called Fagan nomogram [1].

Based on the posttest probability, the corresponding NPV can be calculated. In this way by using the same pretest probability for comparison assays, the corresponding “normalized” NPVs can be directly compared.

GLOSSARY AND ABBREVIATIONS 

Abbreviation/term Meaning
DVT Deep venous thrombosis
Fagan nomogram Graphical tool for estimating how much the result of a diagnostic test changes the probability that a patient has a disease
FDA The Food and Drug Administration, a federal agency of the United States Department of Health and Human Services
LR Likelihood ratio; the ratio of the probability that a given test result occurs in an ill person to the probability that the same result occurs in a healthy person
LR+ Likelihood ratio of a positive test result; the ratio of the probability that a positive test result occurs in an ill person to the probability that a positive result occurs in a healthy person
LR– Likelihood ratio of a negative test result; the ratio of the probability that a negative test result occurs in an ill person to the probability that a negative result occurs in a healthy person
NPV Negative predictive value; fraction of test negatives who do not have disease
PE Pulmonary embolism
Posttest probability The probability that a patient has a disease after the result of a diagnostic test has become available
Pretest probability The probability that a patient has a disease before the result of a diagnostic test has become available
Prevalence Fraction of persons with disease in the tested population
Sens Sensitivity; fraction of persons with disease characterized as ill with the test in question
Spec Specificity; fraction of persons without disease characterized as healthy with the test in question
Spectrum of disease The range of disease states found in the patient population upon which the test is to be used
VTE          Venous thromboembolism                          

(includes DVT and PE)

BACKGROUND

When D-dimer assays are compared, the comparison is often based on data in literature. In this context literature includes articles in peer-reviewed journals and the information given in FDA decision summaries [2] and in manufacturer’s inserts.

The performance data that can be found here are most often sensitivity, specificity and the negative predictive value (NPV).

As D-dimer assays are used as an aid in diagnosis to rule out venous thromboembolism (VTE) – i.e. deep venous thrombosis (DVT) and pulmonary embolism (PE) – in patients with a low or moderate risk score for VTE, focus is especially on the negative predictive value, because the physicians want to be confident that the patients they rule out do not suffer from VTE.

However, the prevalence of disease has an influence on the negative predictive value, which tells us the percent of all with a negative test that are not ill.

Therefore it would be an advantage if negative predictive values were presented with information about the specific prevalence they were calculated for, i.e.

“NPV is 98 % at prevalence 7 %”. Figure 1 shows how NPV is influenced by the prevalence of disease and in addition it shows the influence that assay sensitivity also has. The influence of sensitivity is due to the increasing number of false-negative results that is the consequence of lowering the sensitivity.

FIG. 1: Influence of prevalence and sensitivity on the negative predictive value. Assay specificity in both cases is 50 %.

Instead of comparing negative predictive values, likelihood ratios can be compared. Likelihood ratios have the advantage over negative predictive values that they do not depend on the prevalence of the disease.

LIKELIHOOD RATIOS

The likelihood ratio (LR) is the ratio of two probabilities:
  • The probability that a given test result occurs in an ill person
  • The probability that the same result occurs in a healthy person

The likelihood ratio of a positive test (LR+) result is the ratio of the probability that a positive test result occurs in an ill person to the probability that a positive result occurs in a healthy person. Thus it tells us how many times it is more likely to observe a positive test result in an ill than in a healthy person. It is calculated as:

(LR+) = sensitivity / (1 – specificity)

The likelihood ratio of a negative test result (LR–) is the ratio of the probability that a negative test result occurs in an ill person to the probability that a negative result occurs in a healthy person. Thus it tells us how many times less likely it is to observe a negative test result in an ill than in a healthy person. It is calculated as:

(LR–) = (1 – sensitivity) / specificity

Diagnostic tests are ordered based on the initial assessment of the likelihood of disease (i.e. the pretest probability, which is the probability that a patient has a disease before the test result has become available).

The result of the diagnostic test will help us shift our suspicion one way or the other, and then determine the final probability of disease (the posttest probability, which is the probability that a patient has a disease after the test result has become available).

Likelihood ratios tell us how much we should shift our suspicion for a particular test result.

Table I shows the interpretation of likelihood ratios. An LR > 1 produces a posttest probability which is higher than the pretest probability. An LR < 1 produces a posttest probability which is lower than the pretest probability. If LR = 1, the test does not change the probability, whatsoever.

LR Interpretation
> 10 Large and often conclusive increase in the likelihood of disease
5-10 Moderate increase in the likelihood of disease
2-5 Small increase in the likelihood of disease
1-2 Minimal increase in the likelihood of disease
1 No change in the likelihood of disease
0.5-1.0 Minimal decrease in the likelihood of disease
0.2-0.5 Small decrease in the likelihood of disease
0.1-0.2 Moderate decrease in the likelihood of disease
< 0.1 Large and often conclusive decrease in the likelihood of disease

TABLE I: Interpretation of likelihood ratios

USING LIKELIHOOD RATIOS – SOME EXAMPLES

Likelihood ratios depend as seen on the clinical sensitivity and clinical specificity of an assay. The ability of a test to correctly identify patients with and without a specific disease should not vary between patients.

However, it has to be kept in mind that the spectrum of disease found among patients used in a study can have an influence on the clinical performance of the test [3].

If we assume no major differences between patients in compared studies, likelihood ratios are ideal to directly compare two assays. As D-dimer tests are used to rule out VTE in patients with a sufficiently low pretest probability of VTE, we will focus on the negative likelihood ratio (LR–) in the following sections.

Likelihood ratios are considered useful measures of diagnostic accuracy because they can be used to estimate posttest probabilities. When we have the pretest probability, the test result and the likelihood ratio of the test, we may find the posttest probability by using the so-called Fagan nomogram [1].

To use the Fagan nomogram we need to know the pretest probability, which is the prevalence of disease for the patient group tested. We also need to know the likelihood ratio.

When we draw a line connecting the pretest probability and the likelihood ratio and extend the line until it intersects with the posttest probability axis, then the point of intersection is the posttest probability.

Posttest probability is thus the updated probability of a disease in a patient with a specific test result.

Alternatively the posttest probability may be calculated using an Excel sheet or one of the freely available calculators on the Internet.

The pretest determination is based on clinical information and algorithms. There are several scoring templates available. The Wells score (Table II and Table III) is often used. The original scorings included three levels: ”low”, ”moderate” or ”high” probability [4]. The scorings have been simplified to include two levels: ”unlikely” or ”likely” [5].

TABLE II: Wells score for DVT [4, 5]

Original score Risk Simplified score Risk
≤ 0 low ≤ 1 DVT unlikely
1 or 2 moderate ≥ 2 DVT likely
≥ 3            high           

TABLE III: Interpretation of the Wells score for DVT [4, 5]

The prevalence of DVT in the different risk groups will depend on the actual population tested and may differ a little from site to site. The guideline for diagnosing DVT from the American College of Chest Physicians [6] estimates that the prevalence of DVT in the ”low”-risk group is ~5.0 %, in the ”moderate”-risk group it is ~17 % and in the ”high”-risk group it is ~53 %.

As mentioned above the pretest probability is identical to the prevalence. Let us use these pretest probabilities for some hypothetical assays which we will call Assay 95/70, Assay 95/50, Assay 85/70 and Assay 85/50 and let us see how we can use them for comparison.

The performance of these hypothetical assays can be seen in Table IV.

Assay Sensitivity Specificity LR+ LR–
Assay 95/70 95 % 70 % 3.2 0.07
Assay 95/50 95 % 50 % 1.9 0.10
Assay 85/70 85 % 70 % 2.8 0.21
Assay 85/50 85 % 50 % 1.7 0.30

TABLE IV: Likelihood ratios for hypothetical assays

FIG. 2: Posttest probabilities for Assay 95/50 and Assay 85/50 given that the pretest probability is 5 % (low risk for DVT)

In Fig. 2 we have compared Assay 95/50 and Assay 85/50 using the Fagan nomogram and the prevalence of DVT as 5.0 % (the estimate for the ”low”-pretest-probability group by the American College of Chest Physicians) and the negative likelihood ratios 0.10 and 0.30, respectively.

We see that the posttest probabilities given a negative test are 0.5 % and 1.6 %, respectively. Now what does that mean?

It means that if we use Assay 95/50 for a low-risk patient group with a pretest probability of 5 % and get a negative result, then we know that the probability of the patient having DVT is only 0.5 %.

Therefore we can use the D-dimer results as a diagnostic aid to rule out DVT in these patients with a negative D-dimer result [4, 5, 6, 7].

If we instead use Assay 85/50 for a low-risk patient group with a pretest probability of 5 % and get a negative result, then we know that the probability of the patient having DVT is 1.6 %. This is still acceptable for ruling out DVT in the patients with a negative D-dimer result [4, 5, 6, 7].

How does all this compare to NPV? The NPV is the fraction of test negatives who do NOT have disease. As the posttest probability is the probability that a test-negative patient has DVT, it means that:

NPV = 100 % – posttest probability 

Thus, when the posttest probability for Assay 95/50 is 0.5 %, it means that the NPV is 99.5 % (100 % – 0.5 %). The posttest probability for Assay 85/50 is 1.6 %. This corresponds to an NPV of 98.4 % (100 % – 1.6 %).

You can say that we have “normalized” the NPVs, which means that now we can compare the two assays, Assay 95/50 and Assay 85/50. The “normalized” NPVs are 99.5 % and 98.4 %, respectively.

FIG. 3: Posttest probabilities for Assay 95/70, Assay 95/50, Assay 85/70 and Assay 85/50 given that the pretest probability is 17 % (moderate risk for DVT)

In Fig. 3 we have compared all the four hypothetical assays using the Fagan nomogram and the prevalence of DVT as 17 % (estimate for the ”moderate”-pretest-probability group by the American College of Chest Physicians). We get the results seen in the Table V.

Assay Posttest probability Corresponding NPV
Assay 95/70 1.4 % 98.6 %
Assay 95/50 2.0 % 98.0 %
Assay 85/70 4.1 % 95.9 %
Assay 85/50 5.8 % 94.2 %

TABLE V: Comparison of posttest probabilities and corresponding NPVs for Assay 95/70, Assay 95/50, Assay 85/70 and Assay 85/50 given that the pretest probability is 17 % (moderate risk for DVT)

We see that when the sensitivity is the same for two compared assays, higher specificity will give a lower posttest probability and a corresponding better NPV. We also see that when specificity is the same for two compared assays, higher sensitivity will give a lower posttest probability and a corresponding better NPV.

That might have been foreseen in advance. However, when two assays are compared, they usually do not have neither the same sensitivity nor the same specificity. In such a situation likelihood ratios and a Fagan nomogram can be extremely useful.

The FDA requirement for D-dimer assays as described in the CLSI document H59-A is: NPV ≥ 97 % [7]. When this requirement is achieved, 3 % or less of those who are dismissed based on a negative D-dimer result will be ill.

We can see in Fig. 3 and in Table V that only the assays with a sensitivity of 95 % will fulfill the FDA requirement for NPV when pretest probability is 17 % (~moderate risk). That is also why the guidelines for diagnosing DVT and PE group the D-dimer assays that can be used in diagnosis, into [8, 9]:

  • Highly sensitive assays. Diagnostic sensitivity ≥ 95 %. Can be used to exclude DVT or PE in patients having a low or moderate pretest probability.
  • Moderately sensitive assays. Diagnostic sensitivity < 95 %. Can be used to exclude DVT or PE in patients having a low pretest probability.

CONCLUSION

Likelihood ratios and Fagan nomograms are useful tools for comparing D-dimer assays. By using these tools, posttest probabilities and their corresponding “normalized” NPVs that can be found and used for direct assay performance comparisons.

This gives a more reliable comparison than a comparison of NPVs reported from various studies and separate sources.

References
  1. Fagan TJ. Nomogram for Bayes theorem [Letter]. N Engl J Med 1975; 293: 275
  2. FDA 510(k) Premarket Notification database – http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfPMN/pmn.cfm (accessed June 2015)
  3. CLSI/NCCLS document EP12-A2 User Protocol for Evaluation of Qualitative Test Performance; Approved Guideline 2nd Edition. Vol. 28 No. 3. 2008
  4. Wells PS, Forgie MA. Diagnosis of deep vein thrombosis. Biomed Pharmacother 1996; 50,6-7: 235-42
  5. Venous thromboembolic diseases: the management of venous thromboembolic diseases and the role of thrombophilia testing. NICE guidelines [CG144] June 2012
  6. Bates SM, Jaeschke R, Stevens SM, Goodacre S, Wells PS, Stevenson MD, Kearon C, Schunemann HJ, Crowther M, Pauker SG, Makdissi R, Guyatt GH. American College of Chest Physicians: Diagnosis of DVT: Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines. Chest 2012; 141, 2 Suppl: e351S-418S
  7. CLSI. Quantitative D-dimer for the Exclusion of Venous Thromboembolic Disease; Approved Guideline. CLSI document H59-A. Wayne PA: Clinical and Laboratory Standards Institute; 2011
  8. Bates SM, Jaeschke R, Stevens SM, Goodacre S, Wells PS, Stevenson MD, Kearon C, Schunemann HJ, Crowther M, Pauker SG, Makdissi R, Guyatt GH. American College of Chest Physicians: Diagnosis of DVT: Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines. Chest 2012; 141, 2 Suppl: e351S-418
  9. Authors/Task Force Members, Konstantinides SV, Torbicki A, Agnelli G, Danchin N, Fitzmaurice D, Galiè N, Gibbs JS, Huisman MV, Humbert M, Kucher N, Lang I, Lankeit M, Lekakis J, Maack C, Mayer E, Meneveau N, Perrier A, Pruszczyk P, Rasmussen LH, Schindler TH, Svitil P, Vonk Noordegraaf A, Zamorano JL, Zompatori M; Authors/Task Force Members. 2014 ESC Guidelines on the diagnosis and management of acute pulmonary embolism: The Task Force for the Diagnosis and Management of Acute Pulmonary Embolism of the European Society of Cardiology (ESC) Endorsed by the European Respiratory Society (ERS). Eur Heart J 2014; 35,43: 3033-73
+ View more
References
  1. Fagan TJ. Nomogram for Bayes theorem [Letter]. N Engl J Med 1975; 293: 275
  2. FDA 510(k) Premarket Notification database – http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfPMN/pmn.cfm (accessed June 2015)
  3. CLSI/NCCLS document EP12-A2 User Protocol for Evaluation of Qualitative Test Performance; Approved Guideline 2nd Edition. Vol. 28 No. 3. 2008
  4. Wells PS, Forgie MA. Diagnosis of deep vein thrombosis. Biomed Pharmacother 1996; 50,6-7: 235-42
  5. Venous thromboembolic diseases: the management of venous thromboembolic diseases and the role of thrombophilia testing. NICE guidelines [CG144] June 2012
  6. Bates SM, Jaeschke R, Stevens SM, Goodacre S, Wells PS, Stevenson MD, Kearon C, Schunemann HJ, Crowther M, Pauker SG, Makdissi R, Guyatt GH. American College of Chest Physicians: Diagnosis of DVT: Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines. Chest 2012; 141, 2 Suppl: e351S-418S
  7. CLSI. Quantitative D-dimer for the Exclusion of Venous Thromboembolic Disease; Approved Guideline. CLSI document H59-A. Wayne PA: Clinical and Laboratory Standards Institute; 2011
  8. Bates SM, Jaeschke R, Stevens SM, Goodacre S, Wells PS, Stevenson MD, Kearon C, Schunemann HJ, Crowther M, Pauker SG, Makdissi R, Guyatt GH. American College of Chest Physicians: Diagnosis of DVT: Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines. Chest 2012; 141, 2 Suppl: e351S-418
  9. Authors/Task Force Members, Konstantinides SV, Torbicki A, Agnelli G, Danchin N, Fitzmaurice D, Galiè N, Gibbs JS, Huisman MV, Humbert M, Kucher N, Lang I, Lankeit M, Lekakis J, Maack C, Mayer E, Meneveau N, Perrier A, Pruszczyk P, Rasmussen LH, Schindler TH, Svitil P, Vonk Noordegraaf A, Zamorano JL, Zompatori M; Authors/Task Force Members. 2014 ESC Guidelines on the diagnosis and management of acute pulmonary embolism: The Task Force for the Diagnosis and Management of Acute Pulmonary Embolism of the European Society of Cardiology (ESC) Endorsed by the European Respiratory Society (ERS). Eur Heart J 2014; 35,43: 3033-73
Disclaimer

May contain information that is not supported by performance and intended use claims of Radiometer's products. See also Legal info.

No portrait of author Hiromi Saeki

 

Marketing specialist Japan 

Articles by this author
Suzanne Ekelund

 

Principal Specialist, Clinical Biochemist, MSc 
Radiometer Medical ApS 
Denmark

Articles by this author
The Benefit of Using a D-dimer Assay with a High Clinical Specificity

Register for the related webinar

The Benefit of Using a D-dimer Assay with a High Clinical Specificity

Presented by Karin Strandberg, MD, PhD, Assoc. Prof.

Join the webinar
Acutecaretesting handbook

Acute care testing handbook

Get the acute care testing handbook

Your practical guide to critical parameters in acute care testing. 

Download now
Webinar

Scientific webinars

Check out the list of webinars

Radiometer and acutecaretesting.org present free educational webinars on topics surrounding acute care testing presented by international experts.

Go to webinars

Related Articles

Sign up for the Acute Care Testing newsletter

Sign up
About this site About Radiometer Contact us Legal notice Privacy policy
This site uses cookies Read more