I am delighted to have this opportunity to present some thoughts on quality control for your consideration. Many excellent articles previously presented have described the critical elements required to design a quality control process.

Rather than put forward another iteration of how quality control should be performed in theory, I have chosen to use this opportunity to present some informal personal observations on the gap between QC theory and practice.

These are my observations, and I truly hope that many of you will stand up and prove me wrong.

I have worked in the trenches and been called in to a fifty-bed hospital at 3:00 AM to test blood gases on a critically ill patient.

I have actually heard the words (yes, from a doctor) that these front-line workers should hear every day “What you did tonight probably saved that patient’s life”.

I have had the privilege of teaching QC courses and seminars for many years now, and I know the level of dedication and care that our front-line workers bring to their job every day.

In labs that I managed, I have repeatedly seen the most junior staff give up their breaks, skip lunch and stay late because they were worried about the quality of their results.

I fear that somehow the system has failed to give these critically important front liners the fundamental understanding of quality concepts they need to always make good decisions.

THE GAP

There appears to be a huge gap between QC theory and QC practice. In my experience, the fundamental building blocks of quality control are poorly understood and applied at the front line in the laboratory.

The following vignettes from my personal experience are but a few of the many examples I have seen of the dramatic disparity between theory and reality.

1. In 1994/1995, David Plaut and I conducted a basic quiz on quality control at the start of workshops we each presented in Canada and the USA, and collated the results for a poster at AACC.

The average score was 38 %! Less than half the people who cared enough to come to a full-day workshop on QC could differentiate accuracy from precision, or identify which numerical indicators are associated with these fundamentals.

2. In 2005, I asked graduate students starting an online university course to complete a quiz where they assessed QC charts and answered questions on basic concepts.

The results did not reflect a good baseline of fundamental understanding of QC concepts. Figure 1 shows a QC chart from that quiz.

The question was: “Assume that the pattern on the chart shown is similar for all control samples for a laboratory analyte – in this instance Blood Glucose, where CLIA requirements are set at ±10 % or 6 mg/dL.

A shift in the mean occurred midway through the time period. If you were the supervisor examining the following QC chart, to what degree would you agree or disagree with the following statement: "This method exhibits a negative bias following the shift in the mean".
(Make a note of your answer; more discussion will follow later in this paper).

• I disagree completely
• I don't think so
• I do not know
• There is not enough information provided to answer this question
• I think so
• I agree completely

FIGURE 1: Shift in the mean of -3 SD; Assigned SD = Observed SD

3. In 2001, I conducted a benchmarking study involving 21 laboratories in Canada and the USA. Most participants found it difficult to define a true value and an allowable error limit for their QC samples.

More significantly, most labs reported assigned SD values on QC charts that were two, three or five times higher than the actual method SD.

Somehow the very basic premise of “put the mean and SD on the QC chart” has been lost. It has become “put numbers on the QC chart so the chart looks good and you don’t get QC flags”.

If the observed SD is 1.5 units, as in Fig. 1 above, and you assign the SD on the QC chart as 4.5 units (3 × the observed SD), as in Fig. 2 below, the same shift in the mean of -3 observed SD will appear insignificant.

QC rules are “crippled” by the all-too-common practice of artificially inflating the assigned SD.

FIGURE 2: Shift in the mean of -3 SD; Assigned SD = 3 × Observed SD

4. I recently saw a vicious cycle repeating itself where a dean felt it was not the college’s responsibility to teach the fundamentals of quality control to medical technologists, as the students would learn QC in their clinical year.

Here’s the catch: most of the staff at the clinical site had graduated from that same college!

5. I have seen good people, with the very best of intentions, design absolutely bizarre and ineffectual QC systems.

6. I’ve had a tech look me in the eye with no compulsion and say, “The only way I could get rid of the QC flags for phosphorous was to turn off all the Westgard rules”.
(You know how it works: you can’t report results if you have QC flags. His job was to get rid of the QC flags).

7. I’ve seen the oddest blank stare when I asked if patient results were verified, after observing a recurring pattern of failures on a QC chart and being told, “Oh, that’s nothing.

The reagent goes off if it’s out too long. When we see this, we just replace the reagent, rerun the QC and it’s fine.”

These dedicated technologists had not been trained to understand that the reagent may have gone off moments after their last QC run four hours ago, and clinically misleading patient results may have been reported.

8. I have seen days and dollars wasted and patient results delayed while concerned front liners investigate perceived QC problems that were of no clinical consequence.

WHERE WE SHOULD BE

FIGURE 3 is a flow chart of how quality control is theoretically implemented, when the QC process is driven by the quality requirement. Text below the chart explains how each step is performed.

This is QC in the ideal world; another flow chart (FIGURE 4) will illustrate how QC is often performed in the real world.

FIGURE 3: Theoretical quality control process

WHERE WE ARE

I believe the system has failed to give front-line laboratory staff the knowledge and tools they need to make valid quality decisions.

Unlike the theoretical Performance-Driven Quality Control model above, Fig. 4 illustrates a composite of implementations of QC that I see.

FIGURE 4: Composite of typical quality control process

DISCUSSION

Despite the plethora of scientific articles promoting performance standards and error limits in the past almost four decades now, most front-line workers I meet, not to mention a fair number of senior staff, inspectors and PT officials, still believe that:

1. “quality is good as long as all results are within ±2 SD” and
2. “it’s OK to put a mean and SD on the QC chart that do not reflect the actual current observed mean and SD of each QC sample”.

Just because your QC results are within 2 SD does not mean they are good! It just means they are the same as they were. When labs receive notice that proficiency testing shows that their results fail to meet defined standards, the first response is often “But my QC is fine!

All of the results are within ±2 SD”. That’s like being stopped by a policeman for speeding and saying “No way! I use cruise control and my speed never varied by more than 2 km/hr!” If your doctor said you were overweight or underweight, would you argue “But I always weigh this much, plus or minus no more than 2 pounds!”

Just because you get the same number does not make that number right!

So the answer to the quiz chart is: “There is not enough information provided to answer this question.” Bias is the difference between the measured mean and the true value.

In this example, you have no indication of the true value for this sample. You cannot assess quality with only a QC chart! This shift may bring the reported results closer to the true value and actually reduce bias.

Change is not always bad. You cannot assess quality if you cannot define quality!

Who is responsible for making the leap from theory to practice – implementing policies and procedures that reliably and repeatedly answer the question “Are these patient results OK?”

I can assure you from personal experience that quality control concepts were not taught well in the early seventies.

Many from this/my generation are now in senior positions, where they are responsible for designing, implementing and monitoring quality systems.

Most proficiency testing programs monitor quality by periodically looking for failure. Defects are found only if both (1) the analytical process is deficient at the time of the survey and (2) the QC process fails to detect that deficiency.

Fortunately, manufacturers ensure that analytical failures are rare. QC process failures, I suspect, are far more common. Perhaps, rather than retrospectively detecting failure that has already caused damage, more regulatory effort should be placed on examination of the validity of the QC process.

Then we may see more focus on implementation of the abundance of solid-quality theories that exist to proactively maintain systems within defined quality standards. Before you fail, examine the gap!

As I said, these are my observations, and I truly hope that many of you will stand up and prove me wrong.

If you would like to discuss this essay, or test your quality savvy with the quizzes mentioned above, log on to www.zoebrooksquality.com/harmonize.