Experience of a laboratory at the interface of translational research and future clinical diagnostics
By Hans Attig, Dipl Ing; Kate Bailey, PhD; and Yi Kong, PhD
In the realm of basic medical research, next-generation sequencing (NGS) has rapidly become a well-established technology. Its application in the study of cancer genetics has greatly improved our understanding of the underlying molecular mechanisms and differential responses to interventions.
As our knowledge has increased, so also has our appreciation for the potential of NGS for multibiomarker testing in the clinical diagnostic context. However, there remain many obstacles to the widespread adoption of NGS in routine diagnostic testing, including the inherent complexity of the NGS workflow and the need to handle large amounts of sequencing data.
The GeneReader NGS system by Qiagen, Hilden, Germany, is a research use only (RUO) NGS workflow solution that was developed with the specific aims of addressing these challenges and helping to pave the way for future adoption of NGS in routine clinical practice.
This article presents an evaluation of the GeneReader NGS system. Assessing the performance of the system in comparison with an alternative NGS technology, we demonstrate its effectiveness and reliability for the detection of genetic mutations in tumor biopsy samples. We also assessed the ability of the system to live up to its claims to be an easy-to-operate, complete workflow solution.
The Challenges of Adoption
The application of sequencing technologies to the study of human disease has enabled scientists to gain unprecedented insights into the underlying genetic drivers of many diseases. The introduction of NGS, with its rapid turnaround times and relatively low costs, has made more-thorough genomic screening possible, resulting in an even deeper understanding of the causes of complex diseases.
With current trends suggesting that approximately one-third of us will face a cancer diagnosis at some point in our lifetimes, it is no surprise that many sequencing studies have focused on cancer.1 Research has revealed the existence of remarkable genetic complexity within even a single malignant tumor, with each tumor including subpopulations of cells containing distinct genetic alterations.2 This mosaic of genetic alterations, known as tumor heterogeneity, has implications not only for cancer development and progression, but also for responses to therapy. It is becoming increasingly apparent that in order to have the best chance of successfully combating a cancer, we must first obtain insight into the heterogeneity of an individual tumor.3
In many hospitals and clinics, single-biomarker testing has become a common practice for cancer diagnostics. Screening for mutations within the BRCA1, EGFR, or KRAS genes, for example, is routinely used to guide treatment decisions. However, single-biomarker assays are unable to capture the full genetic complexity of a tumor.
A targeted NGS approach offers a means to obtain a complete overview of the heterogeneity of a specific cancer, and with it the potential for a personalized treatment approach. However, as NGS transitions from basic research into applications providing actionable insights for cancer diagnostics, clinical laboratories face a set of challenges different from those experienced in the academic labs where high-throughput sequencing has grown up. Therefore, the transition from PCR-based single biomarker testing to implementing an NGS workflow is not simple.
To explore genomes as part of a search for new discoveries, an academic laboratory may perform lengthy whole-genome sequencing projects lasting several months. By contrast, a clinical laboratory may examine many new samples every day, each different and each requiring a different insight to guide a treatment decision. In this environment, the potential for speed and accuracy is an important attribute of any genomic system.
Differences in infrastructure also pose a challenge to easy adoption of NGS in routine diagnostics. An academic institution may have multiple centralized facilities, including instrumentation and specialized staff for daily operation, bioinformatics support, and centralized computational resources that can be leveraged for the analysis and storage of large NGS datasets. By contrast, a clinical diagnostics laboratory in a small clinic or hospital is unlikely to have such resources at its disposal, and many labs cite bioinformatics as a major bottleneck.
Further barriers standing in the way of an easy transition of the technology include workflow complexity, ensuring assays of clinical relevance and actionability, and the lengthy process of achieving regulatory clearance for any given assay. These observations are supported by the findings of a survey of clinical laboratories conducted by Qiagen, in which the most common perceived challenge for NGS was identified as complexity of data analysis, followed closely by complexity of the workflow (Figure 1). Mirroring these observations were the findings that labs desire an easy-to-use workflow and a bioinformatics solution as part of their NGS system (Figure 2).
Making the Transition
The Pathology Research Laboratory at the University Hospital of Cologne, Germany, has experience in several approaches to genetic testing and analysis, including NGS. As a pathology institute fully accredited by Deutsche Akkreditierungsstelle GmbH (DAkkS; the German accreditation body for clinical diagnostics), the lab performs research functions, serves the daily demands of university hospital physicians and patients, and assists community hospitals that need genomic testing. Such smaller laboratories face daunting challenges in adopting NGS workflows. As a university-based institute, the lab bridges the worlds of basic research and clinical research laboratories.
Placed as it is at the interface of basic and clinical science, the lab has adopted as one of its goals the assessment of new technologies for their suitability as future diagnostic tools. The GeneReader NGS system, which offers a complete bundle of all components of an NGS workflow designed to work in synchronization, is an attractive offering both for a pathology research lab and for future use in diagnostic settings (see “The GeneReader NGS System“). To see how this workflow compares with other technologies currently available on the market, we collaborated with researchers at the Pathology Research Laboratory to design a study evaluating the claims made about the GeneReader NGS system.4
To test the performance of the system, the researchers examined 50 formalin-fixed paraffin-embedded (FFPE) clinical tumor samples taken from a range of cancer types, including colorectal cancer, gastric cancer, gastrointestinal stromal tumor (GIST), lung cancer, and melanoma.4 The examinations were carried out in parallel, using the GeneReader NGS system and an alternative in-house NGS workflow from the Pathology Research Laboratory built around the Illumina MiSeq platform (Table 1). Mutational status and variant allele frequencies were verified via Sanger sequencing, the current gold standard for the field.
Additionally, the researchers assessed the usability of the GeneReader NGS system under conditions mimicking those of a routine clinical diagnostics laboratory setting, to see if the workflow solution lives up to its claims. Specifically, researchers evaluated the system’s complexity in terms of operational usability and ability to overcome the bottleneck of data analysis, and the claimed actionability and relevance of the data generated by the selected assay.4
Performance on Clinical Samples
Analysis of the 50 FFPE tumor samples on the GeneReader NGS system and the Pathology Research Laboratory’s in-house system demonstrated 100% agreement in variant calling between the two workflows. Hotspot mutations identified were fully concordant between the workflows, and the results generated were verified by Sanger sequencing.
Allele frequency mapping also showed an extremely high degree of linear correlation (R2 = 0.946). This concordance was observed across a wide range of allele frequencies, from <5% to >90% (Figure 3).4
The research demonstrated that the GeneReader NGS system performed as well as the in-house solution on clinical tumor samples, and consistently identified mutations corroborated by Sanger sequencing.4 Such results confirm the ability of the GeneReader NGS system to perform well at variant detection in clinical oncology samples.
The findings of the Pathology Research Laboratory’s study are consistent with results reported in independent studies presented at the 2017 Global Congress on Molecular Pathology, in which the GeneReader NGS system was also demonstrated to be highly effective in mutation detection for a variety of cancers and sample types.5–7 A study performed by researchers at the Dartmouth-Hitchcock Medical Center compared performance of the GeneReader NGS system to the Ion PGM by Thermo Fisher Scientific, Waltham, Mass, on a selection of FFPE tumor samples, and found 100% concordance between the two systems.5 Researchers at Weill Cornell Medicine reported similar performance on a set of 59 FFPE samples from colon, lung, and pancreatic tumors with known mutational status.6 Researchers at Austria’s CBmed Center for Biomarker Research in Medicine also found that the GeneReader NGS system offered strong performance in identification and interpretation of variants from colon cancer and melanoma FFPE and plasma samples.7 Each of these studies concluded that the Qiagen system is a reliable NGS platform yielding accurate results.
Workflow Complexity versus Ease of Use
NGS is an inherently complex technology, or set of potentially fragmented technologies. While this can be said of many laboratory workflows, the low volume and precious nature of the starting sample for a clinical laboratory, combined with the importance of each workflow step for achieving a successful outcome, make NGS unique. The fragmentation of NGS processes is largely the result of vendors specializing in individual sections of an NGS workflow (such as sample preparation, library preparation, sequencing, or bioinformatics), but being unable to offer integrated solutions for the complete process. A laboratory often has no option but to piece together the workflow using components from different vendors.
As a complete process, an NGS workflow typically takes multiple days from start to finish, with each step requiring multiple kits, steps, and operating manuals. Such a complex process offers many opportunities for human error during the handling of precious sample material (Figure 4). Combined with the formidable task of filtering the large and complex datasets generated for meaningful results, the complete process can be daunting for a lab new to NGS. With many small- to medium-sized clinical labs lacking the resources for in-house NGS technology specialists, there is a gap in the market for a single vendor providing a cohesive, end-to-end system plus technical support.
In comparing the GeneReader NGS system with the Pathology Research Laboratory’s in-house NGS workflow, researchers first looked at the constituent parts of the systems, and how they work together (Table 1).4 In-house solutions comprising components from multiple vendors typically require verification of all components, both individually and as a complete workflow. This was the case for the Pathology Research Laboratory’s workflow. Such verification is a time-consuming process and must be repeated each time a workflow component is changed (as may happen in the instance of a product discontinuation or upgrade). While this approach offers flexibility to customize the solution, clinical laboratories generally do not have the in-house experience to navigate the complexities of selecting and acquiring the many components in an NGS workflow.
By contrast, the GeneReader NGS system was developed as a bundle of all the components required for an NGS workflow, from sample and library preparation, through sequencing, to interpretation of the data (Figure 5). Instead of a laboratory having to assemble disparate technologies and make them work together, the GeneReader NGS system provides a unified workflow. The solution involves one vendor and source of support, and a single set of protocols. Synchronization and verification of the system have already been performed before it reaches the lab, representing a significant time saving (Figure 6).
Our comparisons also found operational differences between the two systems. Automation is a critical concern in NGS for clinical labs, both to ensure reliability and to minimize hands-on time for lab personnel. The GeneReader NGS system and the in-house workflow had equal total turnaround times: 5 days from sample prep to results. However, the GeneReader’s high level of automation and integration enabled a very low hands-on time—approximately 8 hours and 15 minutes—less than 12% of the total turnaround for the sequencing process (Figure 7). The in-house workflow required more manual setup and hands-on attention.
The requirement for bioinformatics expertise has often been cited as one of the major hurdles to be overcome in establishing NGS in routine clinical practice.8 A single run on a sequencing workflow typically generates gigabytes of data that require specialist knowledge to analyze and reach a meaningful result. In the case of the GeneReader NGS system, bioinformatics analysis is automated and integrated as part of the workflow. The system’s QCI Analyze software allows for streamlined analysis of the sequence data and identification of genetic aberrations. It also provides valuable intrinsic quality control measures to ensure confidence in variant calling.
The Pathology Research Laboratory reported that the reduction in hands-on time and automated bioinformatics of the GeneReader NGS system meant that the workflow could be operated by a single member of lab staff. Furthermore, that member of ‘wet-lab’ staff required no additional ‘dry-lab’ bioinformatics training and, due to the minimal time spent operating the workflow, was able to perform other lab duties in parallel. Clinical labs need qualified in-house technology specialists to run the ‘wet-lab’ aspects of any workflow, but most do not have the resources to also employ ‘dry lab’ bioinformatics specialists. Therefore, the approach employed by the GeneReader NGS system has the potential to provide staffing benefits that would be of particular value to a smaller clinical diagnostics lab.
Relevance and Actionability
To benefit from NGS for oncology insights, clinical laboratories must decide among a variety of technologies, and balance the scope of information that can be gained against the cost and time required to perform sequencing.
Options for NGS include whole-genome sequencing for the broadest possible coverage of three billion base pairs; whole-exome sequencing for analysis of the protein-coding portions (1% to 2%) of the genome; or gene panels targeting ‘hotspots’ of biomarkers and spanning up to 500 genes.
Basic researchers often choose to sequence whole genomes or exomes, because time is not a major constraint and scientists value the ability to identify a more comprehensive dataset of variants. In clinical applications, however, such large NGS datasets present a logistical problem in terms of data storage, and potential ethical issues related to data usage and security.9 Hence, a clinical laboratory is more likely to turn to gene panels to target potentially relevant, actionable information from sequencing. Gene panels are quicker, more targeted, and less expensive than whole-genome or whole-exome sequencing for detection of biomarkers linked to the development of diseases such as cancer.10
In addition to saving time, targeted sequencing with gene panels enables a laboratory to run more samples per instrument cycle and allows more straightforward interpretation of data. Targeted sequencing also provides greater sensitivity for research into variant detection of rare mutations by distributing the sequencing reads in more tightly focused genomic regions.11
In our research study, the predesigned panel used was the Qiagen QIAact Actionable Insight Tumor (AIT) panel.4 This panel interrogates 773 unique variant positions on 12 genes found in the most common cancers (ALK, BRAF, EGFR, ERBB2, ERBB3, ESR1, KIT, KRAS, NRAS, PDGFRA, PIK3CA and RAF1). The panel was designed to focus research on clinically relevant mutations included in approved therapeutic labels, professional practice guidelines, and active late-stage clinical trials.
The QIAact AIT panel is provided as a fully integrated assay for the GeneReader NGS system, including customized QCI Interpret software for tertiary analysis of variants. QCI Interpret interrogates each identified variant against a continuously updated database to provide the workflow operator with a concise report that includes information about the potential significance of identified variants. Such correct interpretation of tumor molecular changes is of the utmost importance in diagnostic applications.
While the assay described here is designated as an RUO product, the design concept is robust and demonstrates how easily actionable oncology insights for clinical diagnostics might be achieved with NGS in the future.
As NGS is a powerful platform on which multiple assays can be built and run, most laboratories adopting the technology start with one assay, but later wish to expand their test menu. In addition, as science rapidly advances and the NGS technology further improves, it is increasingly important to anticipate the eventual use of NGS for such applications as liquid biopsies.
Panels targeting lung cancer and aimed at detecting all types of variants—single-nucleotide mutations, insertions, deletions, copy-number changes, and translocations—have already been developed for use with the GeneReader NGS system. As the importance of de novo BRCA tumor mutations is recognized, a BRCA panel covering the complete BRCA 1 and 2 coding sequences, plus flanking intronic regions, is also available as part of the QIAact assay family.
Since the completion of the Human Genome Project in 2003, there has been an expectation that sequencing will lead to breakthroughs not only in our understanding of human disease, but also in finding new ways to diagnose and combat illness. The insights that the introduction of NGS has provided have brought us closer to being able to deliver on this promise than ever before. Initiated from basic research elucidating the genomic changes underlying tumorigenesis, NGS now offers the capacity to use single genomic markers to predict an individual’s likelihood of developing cancer, and even to guide practitioners in making the best treatment decisions for their patient’s specific disease.
Moving NGS technologies from basic research into routine diagnostic applications promises to provide even greater diagnostic and prognostic power. However, as discussed in this article, there are still many hurdles that must be overcome if we are to achieve widespread adoption of NGS into clinical practice. An NGS diagnostic solution must be demonstrated to be capable of highly sensitive and accurate detection of mutations; issues relating to complexity of the technology and data analysis must be resolved; and the requirement for well-designed assays targeted at generating meaningful disease insights needs to be addressed.
The GeneReader NGS systemis advertised as a complete sample-to-insight NGS workflow solution, designed to overcome the bottlenecks of NGS. Our data, generated in collaboration with the Pathology Research Laboratory, and the findings reported by other research labs, found the system to generate robust results from typically difficult to work with FFPE tumor tissue samples.4–7 Its performance was comparable both to an established in-house NGS workflow and to validated Sanger sequencing methods. The Pathology Research Laboratory also reported that the workflow lived up to its claims of being a user-friendly NGS solution, with automation resulting in relatively little hands-on time, and an integrated bioinformatics analysis pipeline that effectively removed the bottleneck of data analysis. The targeted assay used with the system was also described as containing a panel of genes and variants with an impressive degree of relevance to the analysis of clinical cancer samples.
While the GeneReader NGS system is currently an RUO solution, it bears the hallmarks of a future leader in the rapidly evolving landscape of NGS diagnostics (Figure 8). It succeeded in addressing many of the challenges faced by today’s clinical labs looking to bring NGS into their repertoire of assays. Developing a single-vendor NGS workflow with a focus on relevant assays delivering meaningful results is a design concept that we hope will make a significant impact on the clinical oncology diagnostics of the future.
Hans Attig, Dipl Ing, is senior microsystems engineer; Kate Bailey, PhD, is content marketing manager; and Yi Kong, PhD, is global marketing director, at Qiagen. For further information, contact CLP chief editor Steve Halasey via [email protected].
- Cancer Statistics [online]. Bethesda, Md: National Cancer Institute, 2017. Available at: www.cancer.gov/about-cancer/understanding/statistics. Accessed January 30, 2018.
- Fisher R, Pusztai L, Swanton C. Cancer heterogeneity: implications for targeted therapeutics. Br J Cancer. 2013;108(3):479–485; doi: 10.1038/bjc.2012.581.
- Dagogo-Jack I, Shaw AT. Tumor heterogeneity and resistance to cancer therapies. Nat Rev Clin Oncol. 2018;15(2):81–94; doi: 10.1038/nrclinonc.2017.166.
- Koitzsch U, Heydt C, Attig H, et al. Use of the GeneReader NGS system in a clinical pathology laboratory: a comparative study. J Clin Pathol. 2017;70(8):725–728; doi: 10.1136/jclinpath-2017-204342.
- Turner SA, Jung HS, Abreu FB, Peterson JD, Tsongalis GJ. Detection and classification of clinical tumor variants using the Qiagen GeneReader NGS system [abstract ST19]. Presentation at the 2017 Global Congress on Molecular Pathology, Berlin, April 3–5, 2017. J Mol Diagn. 2017;19(2):S31; doi: 10.1016/S1525-1578(17)30139-3.
- Zhang P, Fammartino A, Kong Y, Fernandes H. Application of the GeneReader NGS system in clinical testing of tumor samples [abstract TT19]. Presentation at the 2017 Global Congress on Molecular Pathology, Berlin, April 3–5, 2017. J Mol Diagn. 2017;19(2):S54–S55; doi: 10.1016/S1525-1578(17)30139-3.
- Oberauner-Wappis L, Loibner M, Ulz C, Abuja P, Zatloukal K. Application of the GeneReader NGS system in analyzing a diverse set of cancer clinical samples [abstract OTH01]. Presentation at the 2017 Global Congress on Molecular Pathology, Berlin, April 3–5, 2017. J Mol Diagn. 2017;19(2):S523; doi: 10.1016/S1525-1578(17)30139-3.
- Roy S, Coldren C, Karunamurthy A, et al. Standards and guidelines for validating next-generation sequencing bioinformatics pipelines: a joint recommendation of the Association for Molecular Pathology and the College of American Pathologists. J Mol Diagn. 2018;20(1):4–27; doi: 10.1016/j.jmoldx.2017.11.003.
- Clarke AJ. Managing the ethical challenges of next-generation sequencing in genomic medicine. Br Med Bull. 2014;111(1):17–30; doi: 10.1093/bmb/ldu017.
- Horak P, Fröhling S, Glimm H. Integrating next-generation sequencing into clinical oncology: strategies, promises, and pitfalls. ESMO Open. 2016;1(5):e000094; doi: 10.1136/esmoopen-2016-000094.
- Rehm HL, Bale SJ, Bayrak-Toydemir P, et al. ACMG clinical laboratory standards for next-generation sequencing. Genet Med. 2013;15(9):733–747; doi: 10.1038/gim.2013.92.