Risk-based decisionmaking and appropriate analytical tools can improve lab quality

By Walt Murray

Walt Murray, ARC Experts.

Walt Murray, MasterControl Inc and ARC Experts.

Over the past few decades, companies in many industries have found themselves engaged in lawsuits resulting from alleged injuries or deaths among the users of their products. The consequent legal wrangling and financial burdens have led entire industries to seek the protection of regulatory reforms, including the adoption of quality systems standards to guide product development and manufacturing.

Already subject to regulation by definition, clinical laboratories have rarely been the targets of such legal challenges, in part because patients are required to acknowledge by way of written informed consent that their diagnosis and therapy may involve a certain level of clinical risk. But the evolution of quality systems requirements in clinical laboratories has nevertheless stipulated the addition of management analysis of an organization’s performance, including a demonstration of tangible and translatable risk reduction for the totality of the laboratory environment.

This article will not discuss the advantages and disadvantages of the two common quality schemes now approved for use in US clinical laboratories—strict quality control (QC) plans based on requirements established in the Clinical Laboratory Improvement Amendments of 1988 (CLIA), and the individualized quality control plan (IQCP) program, an alternative launched earlier this year by the Centers for Medicare and Medicaid Services (CMS).1,2 Instead, the goal of this article is to offer an independent overview of “risk” as it relates to the management of the lab environment and its activities, thereby creating opportunities for analysis of and improvement in the daily functions of the lab.

IDENTIFYING RISKS

In order to improve laboratory responses to ever-evolving oversight from regulatory bodies, accreditation bodies and professional organizations representing clinical laboratories and laboratorians have thrown their weight behind new methods for assessing the quality of laboratory testing. Working within the CLIA framework, regulators have established methods for assessing the analytical capabilities of clinical laboratories, and the training and competency of laboratory professionals. Laboratories that are unable to pass such regulatory inspections may find their accreditation to conduct business in serious jeopardy.1

A survey conducted by COLA has documented the vulnerability of both testing and quality activities conducted in clinical laboratories. The top 10 reasons for lab surveyors to issue citations include a lack of acceptable competency assessments on all staff, not implementing an IQCP or reverting to the CLIA requirements for QC, and not performing required calibration verification.3

Another analysis identifies a variety of different tasks performed during distinct phases of testing—from pre-preanalytical to post-postanalytical—and estimates the most common causes of error. In this analysis, the percentage of tests affected by errors is estimated to fall within a given range, allowing for the uniqueness of various testing methods. According to this study, the most common causes of errors in the total testing process of clinical laboratories are as follows:4

Pre-preanalytical (46%–68%). Inappropriate test request, wrong order entry, patient or specimen misidentification, sample collected from infusion route, sample collection errors (eg, hemolysis, clotting, insufficient volume), inappropriate container, handling, storage, or transportation.

Preanalytical (3%–5%). Errors in sorting and routing, pour-off, aliquoting, pipetting, labeling, or centrifugation time or speed.

Analytical (7%–13%). Equipment malfunction, sample mix-ups, endogenous or exogenous interference, undetected failure in quality control.

Postanalytical (13%–20%). Erroneous validation of analytical data, failure in reporting or addressing the report, excessive turnaround time, improper data entry or manual transcription error, failure or delay in reporting critical values.

Post-postanalytical (25%–46%). Delayed or missed reaction to laboratory reporting, incorrect interpretation, inappropriate or inadequate follow-up plan, failure to order appropriate consultation.

To verify the quality of a laboratory’s performance, regulators want to assess the results of the lab’s testing processes as well as any risks that might be incidentally created by those processes. In any lab environment, subtle changes in policy or practice can make overall performance prone to failures that do not result from the intended design or execution of the lab’s processes. When not under control, such “layers” of daily functions can readily become a significant source of unwanted events and incidents. Labs are accustomed to dealing with such events by way of compliance activities oriented toward a traditional QC approach, but they have difficulty explaining and executing management activities using a quality systems orientation.

A CLOSED-LOOP QMS

Regulators and accreditation bodies expect that clinical laboratories will compile and maintain standard operating procedures sufficient to address all aspects of a corporate quality management system. Procedures should address how to validate processes, verify conditional activities in accord with good process design, and maintain control. In a clinical laboratory environment, implementing control measures suggests steps to identify, assess, and validate potential risks to the system; and to orient actions appropriate to the situation.

The international standard on Medical Laboratories: Requirements for Quality and Competence (ISO 15189:2012) provides an organized approach to creating a quality management system for clinical laboratories.5 This standard is recognized by all clinical laboratory accreditation bodies in the United States, including COLA, the College of American Pathologists, and the Joint Commission. And use of the standard is also endorsed by public health and regulatory bodies under the umbrella of the US Department of Health and Human Services, including the Centers for Disease Control and Prevention, the Centers for Medicare and Medicaid Services, FDA, and the National Institutes of Health.

Figure1. A fishbone analytical diagram for the outcome of blend uniformity, showing ranked and color-coded risks with potential for affecting process outcomes. Click to expand.

Figure 1. Laboratory accreditation standards displayed in a quality systems (ISO 9000) flowchart format, listing relevant portions of the IS0 15189 standard for medical laboratories, and the COLA Q-31 and Joint Commission accreditation standards.

ISO 15189 incorporates guidance in two distinct areas: quality requirements and technical requirements. To manage all aspects of lab quality, the standard requires closed-loop processes to document procedures, record performance data, audit the results of the system, and perform corrective and preventive actions. Technical requirements describe processes to support the complete life cycle of testing in the laboratory environment (see Figure 1).

Internationally, clinical labs are using common standards for establishing risk-based programs that produce very similar strategic improvements. Such standards include the International Council on Harmonization’s guideline on Quality Risk Management (Q9), and the ISO standards on Medical Devices: Application of Risk Management to Medical Devices (ISO 14971) and Risk Management: Principles and Guidelines (ISO 31000).6–9

DEFINING A RISK MANAGEMENT PLAN

Table 1. Accreditation audits or agency investigations can identify failures in a laboratory’s quality system, and may require the lab to take corrective and preventive actions in order to maintain its certification. Click to expand.

Table 1. Accreditation audits or agency investigations can identify failures in a laboratory’s quality system, and may require the lab to take corrective and preventive actions in order to maintain its certification. Click to expand.

When evaluating the effectiveness of a systematic approach to risk management, it is important for surveyors to understand how the technical requirements for a lab’s testing processes are incorporated into oversight of its quality management system. Such findings are often a result of accreditation audits or agency investigations (see Table 1).

Empirical data in the form of records and information stored by a lab’s electronic data management system (EDMS) or laboratory information management system (LIMS) can provide objective evidence of the lab’s compliance with the requirements of its quality management system. But the ultimate demonstration of a lab’s ability to comply with its own quality management processes comes from matching the lab’s planned objectives to its actual activities—specifically, how the lab executes its total testing process (TTP), QC plan, or IQCP.

To be effective, a lab’s execution of its quality management plan should employ risk-based thinking to identify and clarify issues, to define corrective and preventive actions to be taken, and even to analyze tasks related to quality planning. In this regard, a risk management plan is as vital to a lab’s quality management plan (which includes the TTP), as the quality management plan is to the lab’s business plan. To be rounded out and suitable to provide testing services, a lab’s context and organizational structure should be defined in relation to all of these elements of its business and operational performance.

MANAGING RISK IN QUALITY CONTROL

Risk-based quality management encompasses a combination of control measures and well-planned preventive and contingent actions designed to address unwanted events or incidents that give rise to anticipated risks.

Clinical laboratory staff must learn to assess the potential for unwanted events and incidents systematically, with the intent of identifying risks and adopting improvements that reduce or eliminate those risks. This approach to risk reduction has been incorporated into the international standard on Medical Laboratories: Requirements for Quality and Competence (ISO 15189:2012), whose criteria are similar to those of laboratory accreditation bodies such as COLA and the Joint Commission (see Figure 1).5,10 Generally speaking, there are five phases involved in performing risk-based quality management:

  • Potential hazards assessment (PHA).
  • Harm evaluation.
  • Risk analysis.
  • Commensurate control measures.
  • Developed action(s).

As the ultimate outcomes of this process, actions are developed to eliminate or avoid identified risks. Importantly, the risk-based approach should also identify detectable parameters—such as a trending excursion on a digital logger, or suspect test results that have been captured and segregated for further analysis—that indicate when a predetermined action should be initiated. When an unwanted event causes a testing process to exceed its designated specifications or limits, the predetermined action should be initiated to eliminate the root cause of the out-of-specification outcome.

In some cases, the outcomes of a lab’s processes may be considered good or bad, depending on the lab’s state of readiness. When unforeseen risks can only become apparent during the process of deployment, for instance, a laboratory in a full state of readiness will have already anticipated the risks based on occurrences elsewhere under similar circumstances, and will have predetermined actions that can be taken to avoid or minimize unwanted consequences.

Figure 2. A bow-tie analytical diagram enables staff to identify the root causes of known risks, and to gauge the effects of implementing control measures to deal with unwanted events and their outcomes. Click to expand.

Figure 2. A bow-tie analytical diagram enables staff to identify the root causes of known risks, and to gauge the effects of implementing control measures to deal with unwanted events and their outcomes. Click to expand.

A great deal of risk-based quality management relies on the laboratory’s ability to analyze various testing scenarios and their outcomes. To make such analyses possible, labs should cultivate their capability to storyboard their testing processes in relation to unwanted events that might occur, and to prospect the potential outcomes. This process enables labs to understand and confirm their planning, so that they know what measures should be taken to deal with an unwanted event and its outcomes. A common tool used to perform this activity is known as bow-tie analysis (see Figure 2).

TOOLS FOR RISK ASSESSMENT

A variety of analytical techniques and tools are available for use in performing prospective, concurrent, or retrospective analyses as the basis for planning risk-based quality management activities. However, it is critical that such analyses be informed by current empirical and experiential data derived from periodic cyclical evaluations of the laboratory’s quality and testing processes. In order to develop metrics sufficient for evaluating a lab’s progress, it is important for the lab to set goals and objectives that offer measurable outcomes.

Risk assessment planning encompasses a number of distinct process steps that specifically address the requirements of an IQCP. Each of these steps makes use of specific tools and approaches to arrive at data or information essential for assessing and controlling risks within the context of an IQCP:

  1. PHA, with criteria for judging a risk to be within acceptable parameters.
  2. Risk evaluation according to distinct categories or types of risk.
  3. Cumulative risk analysis for harms related to hazards (process failure modes and effects analysis).
  4. Bow-tie controls analysis for management of risks, including unwanted events and incidents (see Figure 2).
  5. Risk-based decisionmaking to further a testing objective, with criteria for selecting a qualified alternative.
  6. Compilation and documentation of controls selected for use, to create a dynamic control plan as part of an IQCP.

As an alternative to the structured methods employed in traditional QC plans, an IQCP relies on sound logic and the results of risk assessment to frame a plan for reducing risk. 

Figure 3. A fishbone analytical diagram for the outcome of blend uniformity, showing ranked and color-coded risks with potential for affecting process outcomes. Click to expand.

Figure 3. A fishbone analytical diagram for the outcome of blend uniformity, showing ranked and color-coded risks with potential for affecting process outcomes. Click to expand.

Managing the operational risks that may arise in a laboratory environment requires the identification and definition of causal factors that contribute to those risks. A common tool for identifying causal factors and showing their relationship to a known risk is a risk-based fishbone analysis tool (see Figure 3). Such a graphic tool enables staff to visualize and rank causal factors that may contribute to a failure, so that they can be mitigated or managed using the lab’s IQCP.

Risks identified by way of the fishbone analysis tool can be further analyzed using a tool called process failure mode and effects analysis (PFMEA). The PFMEA technique enables staff to analyze and rank risks related to any failure mode scenario under consideration. Based on the lab’s PFMEA for each of its discrete testing areas and methods, mitigation and remediation actions can then be devised and prioritized for inclusion in the lab’s IQCP.

ANALYZING AND SCORING RISK DATA

To anticipate and address problems and unwanted incidents in the lab, staff should use a risk-based fishbone diagram and other tools to define and analyze pertinent causes and effects. Once scored and ranked, risks that are considered unacceptably high can analyzed further using tools such as PFMEA, so that priorities can be assigned for the development of control measures.

Risk analysis formally acknowledges the existence of a potential risk, and offers an initial assessment of its priority relative to other identified risks. When analyzing a risk-based audit matrix, for example, if training is scored as a high-risk, “red flag” item, then the lab’s IQCP should identify corrective and preventive actions to mitigate that risk.

Figure 4. A dynamic control plan can provide a useful way of ensuring that sound protocols for assessing a lab’s testing methodologies are included in the IQCP. Click to expand.

Figure 4. A dynamic control plan can provide a useful way of ensuring that sound protocols for assessing a lab’s testing methodologies are included in the IQCP. Click to expand.

The process is dynamic in that all risks can’t be identified, and there may be residual risk. Lab managers should review such residual risks as a regular element of the lab’s IQCP. For such analyses, the risk-based fishbone analysis and PFMEA should be updated, to account for the realignment of risk priorities in response to corrective and preventive actions (see Figure 4).

PULLING IT ALL TOGETHER

For clinical laboratories, management review is an essential component of any quality management plan, as indicated in ISO 15189 (“Management Review,” sec 4.15).5 On an operational level, an IQCP is based on risk assessment, and should be associated with real-world data about the attributes and variables being analyzed. Integrating risk management as a cycle of activities to be conducted together with management review ensures that labs can implement a plan-do-check-act (PDCA) model for creating and improving an IQCP.

One of the benefits of adopting an IQCP is the cost savings that result from developing and managing resources in accord with a systematic approach to risk-based quality management. When deciding whether an IQCP program is beneficial for their operations, labs need to consider both the costs of performing traditional QC activities, and the potential of an IQCP program to reduce QC runs based on its control of risks.

A fully developed quality management system should encompass the systematic PDCA approach to risk management, and can be expected to comply with the requirements of all regulatory bodies that expect labs to adopt an effective risk-based approach. For instance, when a laboratory is developing a control plan for its own contract labs, contractors, and suppliers, a risk-based approach covering multiple IQCPs can provide the basis for:

  1. Applying risk criticality evaluation.
  2. Conducting a failure mode and effects analysis.
  3. Using a fishbone analysis to identify causal factors for known risks.
  4. Generating a weighted risk matrix.
  5. Developing and implementing IQCPs that include predetermined triggers and actions for controlling risk.
Table 2. A dynamic control plan can provide a useful way of ensuring that sound protocols for assessing a lab’s testing methodologies are included in the IQCP. Click to expand.

Table 2. A dynamic control plan can provide a useful way of ensuring that sound protocols for assessing a lab’s testing methodologies are included in the IQCP. Click to expand.

The final phase of planning risk-based control measures comes with the development of the IQCP itself. Although control plans have been in use for decades, life sciences organizations have generally not made use of them. With the advent of increased scrutiny over the risks that arise in a laboratory environment, a dynamic control plan can provide a useful way of ensuring that sound protocols for assessing a lab’s testing methodologies are included in the IQCP. The PFMEA and test method life cycle, with a rationale and justification for the lab’s risk-based quality management plan, should also be included (see Table 2).

CONCLUSION

Laboratories should consider a number of factors when deciding whether to use a traditional QC plan or implement an IQCP. The design of processes for all phases of the testing life cycle should be evaluated using tools that can reveal risks with the potential to affect test results. Most importantly, risk assessment should be performed to determine whether it is safe to reduce testing frequency or implement abbreviated methods.

Risk-based internal auditing should be used to identify and validate problems that arise, and to define what risk control measures should be taken to address those problems. Such assessments, based on good, risk-based decisionmaking and the use of appropriate analytical tools, will enable laboratory staff to implement sound actions. Cost savings may be found as a result of such measures, but such savings may ultimately be secondary to the benefits gained through vigilant activities undertaken as part of an IQCP program.

Walt Murray is president and CEO of ARC Experts, and director of quality and compliance consulting services at MasterControl Inc. For further information, contact CLP chief editor Steve Halasey via [email protected].

REFERENCES

  1. Clinical laboratory improvement amendments of 1988. P.L. 100-578. Available at: www.gpo.gov/fdsys/pkg/statute-102/pdf/statute-102-pg2903.pdf. Accessed April 28, 2016.
  1. Center for Clinical Standards and Quality, Survey & Certification Group. Individualized quality control plan (IQCP): a new quality control (QC) option. [online] Baltimore: Centers for Medicare & Medicaid Services, 2013. Available at: www.cms.gov/medicare/provider-enrollment-and-certification/surveycertificationgeninfo/downloads/survey-and-cert-letter-13-54.pdf. Accessed April 28, 2015.
  1. COLA Resources Inc issues “top 10 list” of surveyor citations [press release]. Columbia, Md: COLA Resources Inc, 2016. Available at: www.criedu.org/cola-resources-inc-issues-top-10-list-of-surveyor-citations. Accessed May 6, 2016.
  1. Piton A. Risk-based assessment applied to QA GLP audits: how to fulfill regulatory requirements while making the best use of our common sense, knowledge, talents, and resources? Ann Ist Super Sanita. 2008;44(4):379–384. Available at: www.ncbi.nlm.nih.gov/pubmed/19352000. Accessed April 28, 2016.
  1. Medical laboratories: requirements for quality and competence [ISO 15189:2012, 3rd ed.]. Geneva, Switzerland: International Organization for Standardization, 2012. Available at: www.iso.org/iso/catalogue_detail?csnumber=56115. Accessed April 28, 2016.
  1. Quality risk management. Harmonized Tripartite Guideline Q9. Geneva, Switzerland: International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use, 2005. Available at: www.ich.org/fileadmin/public_web_site/ich_products/guidelines/quality/q9/step4/q9_guideline.pdf. Accessed April 28, 2106.
  1. Medical devices: application of risk management to medical devices [ISO 14971:2007, 2nd ed.]. Geneva, Switzerland: International Organization for Standardization, 2007. Available at: www.iso.org/iso/catalogue_detail?csnumber=38193. Accessed April 28, 2016.
  1. Risk management: principles and guidelines [ISO 31000:2009]. Geneva, Switzerland: International Organization for Standardization, 2009. Available at: www.iso.org/iso/catalogue_detail?csnumber=43170. Accessed April 28, 2016.
  1. Dallas M. “Management of risk: guidance for practitioners and the international standard on risk management, ISO 31000:2009.” Norwich, UK: The Stationery Office, 2013. Available at: www.axelos.com/CMSPages/GetFile.aspx?guid=6049b3db-9f12-4c5d-948e-a39c81fcd7d2. Accessed April 28, 2016.
  1. COLA and IQCP: introducing our individualized quality control plan (IQCP) program [online]. Columbia, Md: COLA, 2015. Available at: www.cola.org/programs-services/iqcp-program. Accessed April 28, 2016.