Plays role as unique detector of systemic issues

BY SHARON EHRMEYER, PhD

Controls Sharon  Sharon Ehrmeyer, PhD, is a professor of pathology and lab medicine and director of the Clinical Laboratory Science Program at the University of Wisconsin School of Medicine and Public Health. She wrote this article on behalf of Bio-Rad Laboratories.

 

        This question, in various forms, has been around for decades, but it has never provoked a more interesting discussion than now. By our very nature, laboratory professionals tend to be technologically savvy and grasp the implications that modern computing can bring to laboratory instrumentation. If smartphones can outperform computers we used years ago, why shouldn’t the latest instrument detect errors in such a manner as to make manual procedures, like QC, obsolete?
       The need for quality test results is readily agreed upon by all, and quality test results include the characteristics of useful, accurate, precise, reliable, and timely. And likely, all would agree that new instrument designs utilizing the latest technology should be able to deliver on these characteristics. Unfortunately, it’s quite a bit more challenging when dealing with nonelectronic components. Instead of just working with programs and chips, diagnostic equipment must work with less “black and white” components—that is, human samples and reagents—both of which are made up of materials with a great deal of variability between patients, lots, and over time. It must be remembered, at least at this time, that even the most sophisticated instruments cannot be expected to monitor all aspects of testing and warn us of every possible problem.

Bio-Rad Photo4_copyProactive Tool for Systemic Oversight
       An essay by Sten Westgard compared running an instrument to operating a car; even when the dashboard warning lights aren’t flashing, the car still can break down.1 It’s just as easy to have a sense of security when the laboratory instrument is humming along with no warning lights. But like a car that can fail without warning, an instrument that is seemingly working fine may begin to report incorrect results. QC is a tool that looks at the system as a whole, even the areas that aren’t monitored by the instrument, to ensure it’s working well. For example, trends and shifts identified by QC warn the laboratory that a problem is unfolding before it becomes unacceptable. Laboratories not evaluating the analytical process, or using a QC practice that is not robust enough to detect critical analytical errors, will be unaware of potential “analytical hazards” and put their testing quality at risk.
       The continuing need for QC is well-established. The Clinical Laboratory Improvement Amendments (CLIA) and ISO standards are based on a quality management system (QMS) approach that includes widely accepted good laboratory and error-prevention practices. These incorporate “Essential Elements” for the entire testing process.2 The importance of routine QC to ensure the quality of the analytical phase of testing is recognized and included in the Essential Elements, number 9, internal and external assessment.

Quality Essential Elements

| 1 |   Documents and records
| 2 |   Organization
| 3 |   Personnel
| 4 |   Equipment
| 5 |   Purchasing and inventory
| 6 |   Process control
| 7 |   Information management
| 8 |   Occurrence management
| 9 |   Internal and external assessment
|10|   Process improvement
|11|   Customer service/satisfaction
|12|   Facilities and safety

 

 

 

 

 

 


Evaluating the Quality of the Measurement System
       The CLIA and ISO standards require the analysis of different levels of QC materials at specified intervals to evaluate the quality of the measurement system. The CLIA regulations require each laboratory to implement “control procedures that monitor the accuracy and precision of the complete analytic process and … detect errors that occur due to test system failure, adverse environmental conditions, and operator performance.”4 For most quantitative tests, CLIA has set a minimum requirement, referred to as “default QC,” where at least two different concentrations of QC materials must be assessed on days when patient testing is conducted. For qualitative tests, the requirement is for assessment of at least one positive and one negative control for each day of patient testing. CLIA also requires clinical laboratories to review the QC results before reporting patient results to ensure that only patient results within quality specifications are reported. All unacceptable QC results must be investigated and appropriate corrective actions taken before reviewing or reanalyzing samples and reporting patient results. As part of a laboratory’s ongoing quality assurance activities, CLIA mandates a retrospective review of cumulative QC data so that potential analytical problems can be identified and corrected before test-result quality is affected.
       ISO standards have incorporated QC into two documents for the clinical laboratory. ISO 15189:2012, section 5.6, states that a “laboratory shall design QC procedures that verify the attainment of the intended quality of results.”5 The intended quality of results is based on the laboratory’s quality goal or acceptable error tolerance for test results. Test sites must design QC practices to ensure that all patient results meet the stated quality goal. ISO 22870:2006 states that the “quality manager is responsible for the design, implementation, and operation of QC that ensures POCT conforms to the quality standards of the central laboratory.”6 Both ISO standards require corrective actions when QC results are unacceptable and mandate the review of QC data as part of ongoing quality assurance activities to detect and prevent potential errors.

Bio-Rad Photo1_copyCultivating Broad Consensus on QC Rules
       Concern about the speed of technological advancements outpacing QC guidance documents and regulations is shared equally by the profession, governing bodies, and manufacturers. And while there is an inherent lag time, the question has not been whether QC adds any value to the emerging technology, but rather, the question has been what is an appropriate amount and frequency. After years of considerable deliberations, the answer has come back, and it is, “It depends.”
       Through a consensus process, the Clinical and Laboratory Standards Institute (CLSI) developed a guidance document for QC based on risk management principles.7 CMS announced in 2012 that it will create a process based on risk management by which clinical laboratories can determine appropriate QC for their situations.8 With this risk-based approach, laboratories can incorporate instrument technology, which might periodically check mechanical and electrical functions and alert the user about an abnormal condition that could lead to a failure, external QC analysis, and quality assurance activities as part of their individualized quality control plan to ensure test quality. This approach has been recognized by CLIA and ISO. ISO addressed the question in the recent update of ISO 15189. Section 5.6.2.2 states, “Quality control materials shall be periodically examined with a frequency that is based on the stability of the procedure and the risk of harm to the patient from an erroneous result.”5
       Technology is changing and making its way into laboratory instruments’ quality assessments. Many of the failure modes we worried about years ago are no longer the concerns of today. But failures still occur, and even the best instruments on the market are not 100% fail-proof. QC continues to be recognized around the world by our professional associations, standards organizations, and regulatory bodies as having the unique ability to detect systemic issues that might go otherwise unnoticed. QC is not and should not be going away, and as laboratory professionals, it’s up to us to use it correctly as a tool for ensuring quality results. 

Using Quantitative Results on Qualitative Assays in Molecular Diagnostic Testing

       There are many challenges for quality control (QC) in molecular diagnostic tests. As Clark Rundell, PhD, summarized in the November 2008 issue of IVD Technology magazine, the challenges included the rapidly evolving technologies, lack of QC material, lack of quantitative test outputs, and the rapidly emerging new testing targets. While many of these issues don’t have easy solutions, some traditional QC practices can be deployed in molecular diagnostic labs to improve results quickly. One example is to use quantitative data generated from quality control samples to monitor and statistically analyze the trend to detect potential assay performance problems.
       One of the most widely used molecular tests is for Chlamydia trachomatis (CT) and Neisseria gonorrhoeae(NG). The main assays approved by FDA include Hologic® (Gen-Probe®) Aptima®, BD ProbeTec™, Roche Cobas®, Abbott Realtime, and Cepheid® Xpert®. Although they are all qualitative assays, quantitative outputs are provided by the instruments and can be used for statistic analysis.
       In Hologic’s Aptima assays, a Relative Light Units (RLU) data is generated for each test. A control material should be tested like a patient sample with every run and the RLU value should be recorded. Such results can be serially plotted on Levy-Jennings charts to monitor the test system for shifts or trends. They can also be entered in QC software to compare the results with peer groups. Statistical analysis of the RLU value over time can establish expected variations. Westgard rules can also be applied to determine when a corrective action should be taken to prevent test failure. Likewise, a Moto Score is generated for each test in BD ProbeTec assays, and a Ct value is generated for each test in qPCR-based assays including Roche Cobas, Abbott Realtime, and Cepheid Xpert.
       When selecting QC material, consider whether the material fully tests the entire test method or just a portion of it. For example, material including plasmids or synthetic nucleic acid targets can only monitor the amplification step of the assay, not the sample preparation step. By contrast, material including intact target organisms can detect issues in both sample preparation and amplification steps.
       Some controls included with the assays are used as calibrators to set cutoff value. They are part of the testing procedure rather than an independent verification of the procedure. An additional control processed like a patient sample is needed to verify the entire procedure.
       Outside the United States, there is a trend toward analyzing the quantitative results from qualitative assays. For example, the German Medical Association published “Guideline by the German Medical Association on quality assurance concerning laboratory-based medical analysis” in July 2011,1 which identifies the need to analyze qualitative assays using the quantitative values in order that “if specified limits are exceeded, measures to remove the source of error will be taken.”

 

—By David Du, MD, MS, MBA, product manager, and Stan Kwang, PhD, senior product manager, Bio-Rad Laboratories, Irvine, Calif.

 

References
1. Deutsches Arzteblatt, Year 108, Issue 30, July 1, 2011

 

 
References

1. Westgard S. Would you drive your car the way you run your QC? Available at: http://www.westgard.com/qc-drive.htm. Accessed February 14, 2013.
2. Clinical and Laboratory Standards Institute. Quality management system: a model for laboratory services; (GP26-A4). CLSI: Wayne, PA. 2011. Available at: http://www.clsi.org. Accessed February 14, 2013.
3. Clinical and Laboratory Standards Institute. Statistical quality control for quantitative measurement procedures: principles and definitions; (GP24-A3). CLSI: Wayne, PA. 2006. Available at: http://www.clsi.org. Accessed February 14, 2013.
4. Current CLIA Regulations (including all changes through 01/24/2004): Available at: http://wwwn.cdc.gov/clia/regs/toc.aspx. Accessed February 14, 2013.
5. ISO 15189 (2012): Medical laboratories – requirements for quality and competence. International Organization for Standardization (ISO) standards. Available at: http://www.iso.org/iso/home.html. Accessed February 14, 2013.
6. ISO 22870 (2006): Point-of-care testing – requirements for quality and competence. International Organization for Standardization (ISO) standards. Available at: http://www.iso.org/iso/home.html. Accessed February 14, 2013.
7. Clinical and Laboratory Standards Institute. Laboratory Quality Control Based on Risk Management. (EP23-A). CLSI: Wayne, PA. 2011. Available at: http://www.clsi.org. Accessed February 14, 2013.
8. Centers for Medicare and Medicaid Services. Details for Title: Implementing the Individualized Quality Control Plan (IQCP) for Clinical Laboratory Improvement Amen. Available at: http://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/SurveyCertificationGenInfo/Downloads/SCLetter12_20-.pdf.
Accessed February 14, 2013.

Sharon Ehrmeyer, PhD, is a professor of pathology and laboratory medicine and director of the Clinical Laboratory Science Program at the University of Wisconsin School of Medicine and Public Health, Madison, Wis. Way back when, she became interested in laboratory and POCT quality issues including the impact of mandated regulations on testing practices. Ehrmeyer writes numerous journal and web-based articles and book chapters on these interests, and she travels the world talking about them. She has served on CLSI’s (NCCLS) board of directors and Joint Commission’s Technical Advisory Committee, chaired AACC’s Government Relations Committee, and is currently the Regulatory Affairs section editor for Point of Care (the journal of near patient testing and technology). Andy Quintenz, scientific and professional affairs manager, Quality Systems Division, Bio-Rad Laboratories, is a contributing author. For more information, contact Editor Judy O’Rourke, [email protected]