Skip to Main Content

Print | Bookmark | Email | Font Size: + |

August 26, 2014 - Revised: 10.05.15

Analytical Performance Specifications for Comprehensive Genomic Profiling (CM00012)

The following criteria will be used to assess the analytic performance of comprehensive genomic profiling for Next Generation Sequencing (NGS):

General Laboratory Requirements:

  1. CLIA certification and
  2. College of American Pathologists1 (CAP) accreditation* or
  3. New York State Department of Health2 (NYDOH) final test approval* and
  4. Participation in an external proficiency testing (PT) program for NGS testing that addresses all reported variant classes, when available. Any variant classes for which external PT is not available should be assessed by alternative PT, as required by CLIA.

*Note: CGS and the MolDx contractor will accept a completed application for either CAP accreditation or NYDOH test approval as provisional documentation for coverage until the final CAP or NYDOH accreditation or approval is available. When CAP/NYDOH provides the laboratory with the final accreditation/approval, the laboratory must notify CGS and MolDx and submit validation of the final accreditation/approval.

Pre-Analytical Requirements:

  1. NGS-based analysis must be validated for all sample types accepted for testing (e.g., blood, bone marrow, tissue, purified DNA, etc.) as listed in the laboratory's sample acceptance criteria1;
  2. The estimated percentage and viability of neoplastic cells in the tested sample as determined by an appropriate method (e.g., histo-morphology or flow cytometry) must be documented;
  3. When the specimen consists of a tissue sample and the percentage of neoplastic cells is determined by morphology,
    1. The review must be performed by an American Board of Pathology-certified anatomic pathologist and
    2. A representative slide of the tissue or a digital image of the slide must be archived.
  4. For specimens that require enrichment (e.g., via microdissection or coring) to achieve the requisite level of tumor cellularity and/or viability, representative slides of pre- and post-enrichment tissue sections (or digital images of the slides) must be archived.

Analytical Requirements
The following four general classes of sequence variants can cause disease:

  1. Single nucleotide variants (SNVs)
  2. Small insertions and deletions (indels)
  3. Copy number variants (CNVs) and
  4. Structural variants (SVs), such as translocations

Laboratories need only address requirements for variant classes, alteration sizes, and variant allele frequencies (VAFs) that they report or claim to report based on their validation. For example, if indels >10 bp or homozygous deletions are not reported, then these requirements do not apply. Similarly, if the laboratory only claims to report deletions ≤ 8 bp with VAFs > 20%, then only row 7 (not rows 8-10) in Table 2 apply.

1. Accuracy of sequencing performed on a reference cell line
Using its standard clinical testing protocols (including equipment and personnel), the laboratory will sequence NIST Reference Material 8398 (or the HapMap cell line NA12878) for the genes included in its panel, and compare the experimental sequence against the reference values provided (or public reference sequence available at 1000genomes.orgExternal Website). The point estimate for each reported variant class for the positive percent agreement (PPA) and positive predictive value (PPV) should exceed 99.9%.

Table 1: Accuracy of sequencing performed on a reference cell line

   

Orthogonal Reference Results

   

Positive

Negative

Total

NGS Results

Positive

A

B

A+B

Negative

C

D

C+D

Total

A+C

B+D

A+B+C+D

"Positive" = Variant called/identified when compared to current build of human genome (e.g., hg19 human genome assembly, version 37)

Positive percent agreement (PPA) = A/(A+C); Positive predictive value (PPV) = A/(A+B)

2. Accuracy and precision of sequencing performed on reference cell lines: Mixing experiments
Using its standard clinical testing protocols (including equipment and personnel), the analytic performance metrics for each variant class reported by the laboratory must be specified and satisfy the requirements below, and compare the experimental sequence against the public reference sequence available at 1000genomes.orgExternal Website. Note that the laboratory will likely need to utilize HapMap cell lines (e.g., NA12878, NA19240, NA18507, NA19129, etc.) in mixing experiments to document the analytical accuracy at different VAFs, including the stated lower limit of detection for each variant class.

Table 2: Accuracy and precision of sequencing performed on reference cell lines: Mixing experiments

Row

Variant Type

Detail

Lower 95% Confidence Intervala

PPAb

PPVb

Intermediate Precisionc

1

Single nucleotide variants

Expected VAF > 10%

≥ 99.0%

≥ 99.0%

≥ 95.0%

2

Single nucleotide variants

Expected VAF = 5-10%

≥ 95.0%

≥ 95.0%

≥ 90.0%

3

Insertions

≤ 10 bp
Expected VAF > 20%

≥ 85.0%

≥ 95.0%

≥ 80.0%

4

Insertions

≤ 10 bp
Expected VAF = 10-20%

≥ 75.0%

≥ 85.0%

≥ 75.0%

5

Insertions

11-70 bp
Expected VAF > 20%

≥ 85.0%

≥ 95.0%

≥ 80.0%

6

Insertions

11-70 bp
Expected VAF = 10-20%

≥ 75.0%

≥ 85.0%

≥ 75.0%

7

Deletions

≤ 10 bp
Expected VAF > 20%

≥ 85.0%

≥ 95.0%

≥ 80.0%

8

Deletions

≤ 10 bp
Expected VAF = 10-20%

≥ 75.0%

≥ 85.0%

≥ 75.0%

9

Deletions

11-70 bp
Expected VAF > 20%

≥ 85.0%

≥ 95.0%

≥ 80.0%

10

Deletions

11-70 bp
Expected VAF = 10-20%

≥ 75.0%

≥ 85.0%

≥ 75.0%

11

Copy number alterations – Amplifications

Ploidy < 4
Expected CN ≥ 8
> 30% tumor nuclei

≥ 90.0%

≥ 90.0%

≥ 85.0%

12

Copy number alterations – Amplifications

Ploidy < 4
Expected CN ≥ 8
20-30% tumor nuclei

≥ 60.0%

≥ 80.0%

≥ 75.0%

13

Copy number alterations – Homozygous Deletions

Ploidy < 4
Expected CN = 0
> 30% tumor nuclei

≥ 80.0%

≥ 85.0%

≥ 60.0%

14

Copy number alterations – Homozygous Deletions

Ploidy < 4
Expected CN = 0
20-30% tumor nuclei

≥ 50.0%

≥ 75.0%

≥ 50.0%

15

Translocations

≥ 20% tumor nuclei

≥ 85.0%

≥ 85.0%

≥ 90.0%

a For calculating 95% CIs, use Score or Clopper-Pearson method as described in CLSI EP12-A.

b

   

Orthogonal Reference Results

   

Positive

Negative

Total

NGS
Results

Positive

A

B

A+B

Negative

C

D

C+D

Total

A+C

B+D

A+B+C+D

"Positive" = Variant called/identified when compared to current build of human genome (e.g., hg19 human genome assembly, version 37)

Positive percent agreement (PPA) = A/(A+C); Positive predictive value (PPV) = A/(A+B)

c This is based on at least 3 different operators, runs/batches/days, manufacturing reagent lots (for critical reagents), and (if applicable) sequencers and sites. Agreement is defined as concordance at the variant level for all called (i.e., not necessarily reported but excluding no calls) variants. In other words, did each called variant agree across different operators, runs/batches/days, manufacturing reagent lots, and (if applicable) sequencers and sites?

3. Accuracy of sequencing performed on clinical samples
Using samples that include all sample types accepted for testing (e.g., blood, bone marrow, tissue, purified DNA, etc.) as listed in the laboratory's sample acceptance criteria, the laboratory will report the point estimates, sample size, and 95% confidence intervals (using the Score or Clopper-Pearson method as described in CLSI EP12-A) for PPA and PPV for each reported variant class based on at least five different variants per row in Table 3 below.

Table 3: Accuracy of sequencing performed on clinical samples

Row

Variant Type

Detail

1

Single nucleotide variants

Expected VAF > 10%

2

Single nucleotide variants

Expected VAF = 5-10%

3

Insertions

≤ 10 bp
Expected VAF > 20%

4

Insertions

≤ 10 bp
Expected VAF = 10-20%

5

Insertions

11-70 bp
Expected VAF > 20%

6

Insertions

11-70 bp
Expected VAF = 10-20%

7

Deletions

≤ 10 bp
Expected VAF > 20%

8

Deletions

≤ 10 bp
Expected VAF = 10-20%

9

Deletions

11-70 bp
Expected VAF > 20%

10

Deletions

11-70 bp
Expected VAF = 10-20%

11

Copy number alterations – Amplifications

Ploidy < 4
Expected CN ≥ 8
> 30% tumor nuclei

12

Copy number alterations – Amplifications

Ploidy < 4
Expected CN ≥ 8
20-30% tumor nuclei

13

Copy number alterations – Homozygous Deletions

Ploidy < 4
Expected CN = 0
> 30% tumor nuclei

14

Copy number alterations – Homozygous Deletions

Ploidy < 4
Expected CN = 0
20-30% tumor nuclei

15

Translocations

≥ 20% tumor nuclei

   

Orthogonal Reference Results

Positive

Negative

Total

NGS
Results

Positive

A

B

A+B

Negative

C

D

C+D

Total

A+C

B+D

A+B+C+D

"Positive" = Variant called/identified when compared to current build of human genome (e.g., hg19 human genome assembly, version 37)

Positive percent agreement (PPA) = A/(A+C); Positive predictive value (PPV) = A/(A+B)

Post-Analytical Testing Requirements

  1. The variants identified, clinical interpretations, and therapeutic recommendations must be reported by a physician, board certified in Molecular Genetic Pathology by the American Board of Pathology, or in Molecular Genetics by the American Board of Medical Genetics and Genomics, or has equivalent experience and expertise. A PhD is not a recognized Medicare provider.
  2. Each sequenced run must include a control to document that quality control metrics for the assay (including at least read depth and sequence quality) have been met for that run for all variant classes reported.
  3. The bioinformatics pipeline must exclude specimen contamination as the source of identified variants for the variant classes and variant allele frequencies (VAFs) reported3,4.

References:

  1. Aziz N, Zhao Q, Bry L, et al. College of American Pathologists' Laboratory Standards for Next-Generation Sequencing Clinical Tests. Arch Pathol Lab Med 2015;139:481-93
  2. NextGenSeq websiteExternal PDF (218 KB)
  3. Jun G, Flickinger M, Hetrick KN, et al. Detecting and estimating contamination of human DNA Samples in sequencing and array-based genotype data. Am J Hum Genet. 2012;91:839-48.
  4. Sehn J, Spencer DH, Pfeifer JD, et al. Occult specimen contamination in routine clinical NGS testing. Am J Clin Pathol. In press.

spacer

26 Century Blvd Ste ST610, Nashville, TN 37214-3685 © CGS Administrators, LLC. All Rights Reserved