Abstract
A critical requirement in neuroendocrine tumor (NET) management is a sensitive, specific and reproducible blood biomarker test. We evaluated a PCR-based 51 transcript signature (NETest) and compared it to chromogranin A (CgA), pancreastatin (PST) and neurokinin A (NKA). The multigene signature was evaluated in two groups: i) a validation set of 40 NETs and controls and ii) a prospectively collected group of NETs (n=41, 61% small intestinal, 50% metastatic, 44% currently treated and 41 age-sex matched controls). Samples were analyzed by a two-step PCR (51 marker genes) protocol and ELISAs for CgA, PST and NKA. Sensitivity comparisons included χ2, non-parametric measurements, ROC curves and predictive feature importance (PFAI) analyses. NETest identified 38 of 41 NETs. Performance metrics were: sensitivity 92.8%, specificity 92.8%, positive predictive value 92.8% and negative predictive value 92.8%. Single analyte ELISA metrics were: CgA 76, 59, 65, and 71%; PST 63, 56, 59, and 61% and NKA 39, 93, 84, and 60%. The AUCs (ROC analysis) were: NETest: 0.96±0.025, CgA: 0.67±0.06, PST 0.56±0.06, NKA: 0.66±0.06. NETest significantly outperformed single analyte tests (area differences: 0.284–0.403, Z-statistic 4.85–5.9, P<0.0001). PFAI analysis determined NETest had most value (69%) in diagnosis (CgA (13%), PST (9%), and NKA (9%)). Test data were consistent with the validation set (NETest >95% sensitivity and specificity, AUC =0.98 vs single analytes: 59–67% sensitivity, AUCs: 0.58–0.63). The NETest is significantly more sensitive and efficient (>93%) than single analyte assays (CgA, PST or NKA) in NET diagnosis. Blood-based multigene analytic measurement will facilitate early detection of disease recurrence and can predict therapeutic efficacy.
Introduction
Critical requirements for the management of gastroenteropancreatic neuroendocrine tumors (GEP-NETs) include methods that efficiently detect and monitor tumor behavior (Modlin et al. 2008a). NETs are, however, a heterogeneous group of cancers both in terms of tumor biology and the variety of bioactive products they synthesize and secrete (Modlin et al. 2008b). Some tumors exhibit extended survival, even with disseminated disease while others are rapidly metastatic with poor prognosis (Yao et al. 2008). These different behaviors reflects the diverse cells (and sites) of origin. Thus, tumor types range from histamine-secreting gastric ‘carcinoids’ (stomach) to pancreatic lesions (secreting glucagon, insulin, somatostatin or vasoactive polypeptide – ‘islet cell tumors’), as well as small intestinal ‘carcinoids’ (secretin, serotonin and a variety of tachykinins) and colorectal lesions (enteroglucagon, GLP1, and pancreatic polypeptide). NETs may therefore be considered ‘functional’ or ‘non-functional’, though such terminology is archaic and of little scientific value except as a clinical descriptor of utility in diagnosis (Modlin et al. 2011). The advantages of developing multianalyte assays with algorithmic analytic (MAAA) methodology to accurately assess a tumor group arising from many different cells and with numerous biological profiles has been described by a number of authors (Engels et al. 2013, Grimm et al. 2013). Coupling the integral aspects of disease represented by individual markers referable to a specific disease produces a significantly greater degree of quantification (Engels et al. 2013). Thus for example, examining two deeply conserved process of malignancy such as apoptosis and anaerobic glucose metabolism represented respectively by Apo10 and TKTL1 (key proteins facilitating invasion and metastasis), provides key information about tumor cell behavior not available through measurement of a single marker (Grimm et al. 2013). It is therefore clear that a MAAA strategy, if appropriately configured to interrogate a specific neoplastic process, will mathematically encompass numerous different biological integers inaccessible to a monoanalyte approach.
Two critical unmet needs were identified by the NCI NET Summit conference of 2008. The first is the inability to establish an early and accurate diagnosis and secondly the evaluation of NET therapeutic responses (Modlin et al. 2008a). The latter is mainly based on radiological Response Evaluation Criteria In Solid Tumors v1.1 (RECIST) criteria which are difficult in ‘indolent’ lesions (Eisenhauer et al. 2009). NET responses to therapies are rarely associated with early measurable changes in tumor size and represent a well-documented problem in NET disease management (Denecke et al. 2013, Neperud et al. 2013). Overall, the current criteria for the assessment of NET progression and therapeutic responses are suboptimal (Faivre et al. 2012, Castano et al. 2014). As a result, alternative imaging criteria, combining morphological and functional techniques, are under consideration (de Mestier et al. 2013). Imaging therefore has limitations in the early identification of NET disease progress and the timely establishment of therapeutic efficacy. Thus alternative approaches, radiological, nuclear medicine or co-value biomarker strategies are required to provide information of recurrent disease or therapeutic efficacy.
Alternative or complementary evaluation methods such as circulating biomarkers have been investigated. The best known is chromogranin A (CgA) which is widely used as the default biomarker for NET disease (Stridsberg et al. 1995). The protein is produced and processed as a component of the neuroendocrine cellular secretory apparatus (Taupenot et al. 2003) and exists in the blood stream as a heterogeneous antigen composition ranging from a complete protein to a series of cleavage products (Lawrence et al. 2011, Kanakis & Kaltsas 2012). Increased CgA is ‘generally considered’ to be sensitive, ∼60–90% and accurate once a NET has been identified (Modlin et al. 2010), it is, however, an ineffective first line diagnostic (Marotta et al. 2012). Measurements are usually non-specific (10–35% specificity) since CgA is elevated in other conditions including neoplasia (Wu et al. 2000), cardiac and inflammatory diseases (Sciarra et al. 2005), renal failure (Hsiao et al. 1990) and proton pump inhibitor (PPI) administration (Lawrence et al. 2011).
Pancreastatin (PST) is a derivative of CgA that inhibits both pancreatic insulin and exocrine secretion, and may stimulate gastric acid secretion (Udupi et al. 1999). It is elevated in 58–81% of NETs (Stridsberg et al. 1995, Calhoun et al. 2003) and does not appear to be elevated by PPI use (Raines et al. 2012). Despite recent interest in PST as a biomarker (Ito et al. 2012), its utility remains unclear as levels do not correlate with tumor location, are not associated with tumor functionality and may not correlate with tumor aggressiveness or survival (Calhoun et al. 2003). It has been proposed that rapid alterations may be prognostic for somatostatin analog-based therapy (Stronge et al. 2008). Medications or food ingestion, e.g. glucose (alters insulin levels) influences results (Siegel et al. 1998). In addition, this disglycemic peptide is elevated in diabetics (Valicherla et al. 2013) as well as in hyperparathyroidism (Bergenfelz et al. 2000). Different conformations of the peptide may also be masked and are unavailable for assessment rendering assay interpretation complex and controversial (O'Dorisio et al. 2010).
Neurokinin A (NKA, substance K) is a ten amino acid peptide translated from the pre-protachykinin gene with broncho-constrictive properties, in asthma (Joos et al. 2004). NKA levels are considered to have utility in evaluating carcinoid heart disease and treatment responses (Dobson et al. 2013). It has been suggested that levels may be an independent indicator of NET prognosis; >50 pmol/l associated with decreased 3 year survival rates (Turner et al. 2006). Medications for hypertension, intestinal motility and acid suppression (PPIs), as well as opiates all affect NKA levels (Lecci et al. 2006).
Given the limited accuracy of the currently available biomarkers and the known limitations of single analyte measurements in clinical science (Hood & Tian 2012, Curtis et al. 2013), we have developed a blood-based multianalyte neuroendocrine tumor specific gene transcript analysis. This is an alternative to the measurement of single analytes, e.g. CgA, PST, or NKA and is a robust, reproducible PCR-based multianalyte test for the detection of NETs. The multianalyte algorithm is based on the simultaneous measurement of 51 neuroendocrine specific marker genes in peripheral blood. It has a high sensitivity (85–98%) and specificity (93–97%) for the detection of gut NETs or ‘carcinoids’ (Modlin et al. 2013a, 2014). Based upon the scientific assessment of multianalyte methodology, this technique is superior to single analyte assays in the detection of NETs (Lewis & Yao 2013). The current study examines both an independent validation set and a prospective patient set. We investigated the multianalyte gene transcript analysis for NETs against the currently used standard biomarkers: CgA, PST and NKA using an age and sex matched sample set of NETs and controls.
Methods
Validation set (Set 1)
A validation set including 20 age-matched NETs and controls (1:1) was assessed (Table 1). Samples were not sex-matched but ages were not different between the two groups (NETs: 59.6±2.2 years vs controls: 58.7±3.4, P=NS). The ethnicity was exclusively Caucasian. The demographics of each group are included in Table 1. The majority of NETs were small intestinal (n=13, 65%), 14 (∼70%) had metastases and 13 (65%) were being treated (Sandostatin: n=11, Sandostatin+Affinitor: n=2; Novartis Pharmaceuticals). Six (30%) of the control group were taking PPIs. The diseases were: GERD: n=7, asymptomatic controls: n=5, pancreatic cysts: n=4 and inflammatory bowel disease (IBD): n=4.
Demographics (Set 1)
| Neuroendocrine tumors (n=20) | Controls (n=20) | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Sample | Gender | Age | Site | Mets | Treatment | Sample | Gender | Age | Pathology | PPI |
| N1 | F | 68 | BP | C1 | M | 68 | None | |||
| N2 | F | 54 | CUP | C2 | M | 52 | IBD | |||
| N3 | F | 44 | CUP | C3 | M | 27 | None | |||
| N4 | M | 60 | D | LIV | Sandostatin | C4 | F | 59 | GERD | |
| N5 | F | 46 | P | Local | C5 | F | 37 | IBD | ||
| N6 | M | 68 | P | LIV | Affinitor/Sandostatin | C6 | M | 75 | Cyst | Y |
| N7 | F | 59 | P | LIV | Affinitor/Sandostatin | C7 | M | 56 | GERD | Y |
| N8 | F | 46 | SI | Sandostatin | C8 | M | 44 | None | ||
| N9 | M | 71 | SI | Local | Sandostatin | C9 | M | 80 | IBD | Y |
| N10 | F | 72 | SI | LIV | C10 | F | 80 | GERD | Y | |
| N11 | F | 53 | SI | C11 | M | 47 | IBD | |||
| N12 | F | 69 | SI | LIV | Sandostatin | C12 | M | 76 | Cyst | Y |
| N13 | F | 53 | SI | Sandostatin | C13 | M | 48 | GERD | ||
| N14 | F | 58 | SI | LIV | C14 | M | 53 | Cyst | ||
| N15 | F | 47 | SI | Local | Sandostatin | C15 | F | 45 | None | |
| N16 | F | 66 | SI | Local | Sandostatin | C16 | M | 67 | GERD | |
| N17 | M | 60 | SI | LIV | Sandostatin | C17 | M | 67 | GERD | Y |
| N18 | F | 65 | SI | LIV | Sandostatin | C18 | F | 63 | GERD | |
| N19 | F | 78 | SI | LIV | Sandostatin | C19 | F | 64 | GERD | |
| N20 | M | 54 | SI | LIV | Sandostatin | C20 | M | 52 | IBD | |
| SIRT | ||||||||||
BP, bronchopulmonary; CUP, carcinoid of unknown primary; D, duodenum; P, pancreas; SI, small intestine; Mets, metastases; LIV, liver.
Matched cases and controls (Set 2)
This included prospectively collected NET patients (September–December 2013) and controls, matching the 41 cases with a control (1:1) by sex and age to within 2 years. The ethnicity was exclusively Caucasian. The demographics of each group are included in Table 2. There were no differences in sex distribution: M:F=10:31, both groups) or age between the two groups (NETs: mean 56.9, range: 31–76; controls: mean 56.4, range: 33–75) confirming appropriateness of matching. The majority of NETs were small intestinal (n=25, 61%), 20 (∼50%) had metastases (six of which were lymph node or mesenteric invasion) and 18 (44%) of the NETs were currently treated (Somatulin: n=2, Sandostatin: n=15, Sandostatin+Affinitor: n=1). No NETs were on acid inhibitory therapy. In contrast, 12 (29%) of the control group were taking PPIs. The majority (n=27, 66%) had GERD; no symptoms were noted in 13 patients (one with a past history of ovarian cancer, one with a past history of pancreatic cancer), one control had uncontrolled IBD. None were taking anti-hypertensive medication.
Demographics (Set 2)
| Neuroendocrine tumors (n=41) | Controls (n=41) | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Sample | Gender | Age | Site | Mets | Treatment | Sample | Gender | Age | Pathology | PPI |
| N1 | F | 66 | A | Local | C1 | F | 67 | None | N | |
| N2 | F | 52 | BD | C2 | F | 52 | Cyst | N | ||
| N3 | F | 64 | C | LIV | Sandostatin | C3 | F | 64 | None | N |
| N4 | F | 52 | D | Local | C4 | F | 52 | None | N | |
| N5 | F | 63 | BP | C5 | F | 63 | GERD | Y | ||
| N6 | F | 75 | BP | C6 | F | 76 | Pancreatitis | Y | ||
| N7 | F | 53 | BP | LIV | Sandostatin | C7 | F | 52 | Serous Cystadenoma | N |
| N8 | M | 33 | BP | C8 | M | 35 | None | N | ||
| N9 | F | 46 | OV | C9 | F | 48 | None | N | ||
| N10 | M | 67 | P | Sandostatin | C10 | M | 67 | GERD | Y | |
| N11 | F | 59 | P | LIV | Affinitor/Sandostatin | C11 | F | 61 | None | N |
| N12 | M | 57 | P | LIV | C12 | M | 56 | Cyst | N | |
| N13 | F | 57 | P | Sandostatin | C13 | F | 56 | Mucinous Cyst | N | |
| N14 | F | 65 | P | C14 | F | 65 | Cyst | N | ||
| N15 | F | 35 | R | C15 | F | 34 | None | N | ||
| N16 | F | 59 | SI | Bo | Sandostatin | C16 | F | 59 | Cystadenoma | Y |
| N17 | M | 68 | SI | LIV | Sandostatin | C17 | M | 70 | GERD | Y |
| N18 | F | 58 | SI | LIV | Sandostatin | C18 | F | 58 | GERD | Y |
| N19 | F | 43 | SI | LIV | C19 | F | 46 | GERD | Y | |
| N20 | M | 56 | SI | Local | Sandostatin | C20 | M | 55 | GERD | y |
| N21 | F | 65 | SI | Sandostatin | C21 | F | 65 | GERD | Y | |
| N22 | F | 43 | SI | C22 | F | 47 | GERD | Y | ||
| N23 | F | 70 | SI | C23 | F | 69 | GERD | Y | ||
| N24 | F | 46 | SI | Distant | C24 | F | 48 | Cyst | N | |
| N25 | M | 58 | SI | LIV | Somatulin | C25 | M | 57 | Cyst | N |
| N26 | F | 48 | SI | LIV | Somatulin | C26 | F | 51 | Cyst | N |
| N27 | M | 52 | SI | LIV | Sandostatin | C27 | M | 52 | None | N |
| N28 | F | 65 | SI | LIV | Sandostatin | C28 | F | 67 | None | N |
| N29 | F | 33 | SI | LIV | Sandostatin | C29 | F | 31 | None | N |
| N30 | F | 65 | SI | LIV | Sandostatin | C30 | F | 64 | None | N |
| N31 | F | 72 | SI | LIV | C31 | F | 74 | Cyst | N | |
| N32 | M | 65 | SI | LIV | C32 | M | 65 | Cystadenoma | N | |
| N33 | F | 57 | SI | LIV | C33 | F | 56 | GERD | N | |
| N34 | F | 71 | SI | LIV | C34 | F | 73 | Mucinous Cyst | N | |
| N35 | F | 39 | SI | Local | Sandostatin | C35 | F | 36 | Crohn's | N |
| N36 | F | 67 | SI | Local | C36 | F | 68 | NET (confirmed) | N | |
| N37 | F | 36 | SI | Local | C37 | F | 34 | Cyst | N | |
| N38 | M | 74 | SI | Sandostatin | C38 | M | 74 | None | N | |
| N39 | F | 55 | SI | C39 | F | 56 | GERD | N | ||
| N40 | F | 44 | SI/App | C40 | F | 47 | Cyst | Y | ||
| N41 | M | 61 | ST | LIV | C41 | M | 62 | None | N | |
A, appendix; BP, bronchopulmonary; BD, bile duct; C, colon; D, duodenum; OV, ovarian; P, pancreas; R, rectum; SI, small intestine; ST, stomach; Mets, metastases; Bo, bone; LIV, liver.
Sample collection
All samples were collected and analyzed according to a standard IRB protocol (Yale University: June 17, 2013) in accordance with the World Medical Association Declaration of Helsinki regarding ethical conduct of research involving human subjects (Modlin et al. 2013a). All individuals from whom blood was obtained were seen (June 2012–December 2013) at the School of Medicine out-patient clinics following informed consent. Blood samples (5 ml) were collected in 9 mg K2EDTA tubes (BD Vacutainer Venous Blood Collection Tubes, BD Diagnostics, Franklin, NJ, USA). Aliquots of whole blood were stored at −80 °C within 2 h of collection (samples immediately stored on ice/4 °C after sampling) per standard molecular diagnostics protocols for PCR-based studies (Raza et al. 2012). A second aliquot (2 ml) was spun (600 g, 10 min) and the plasma collected for ELISA as previously described (Modlin et al. 2009, 2013a, 2014).
Multianalyte algorithm analysis (MAAA) PCR-based test (NETest)
We used a two-step manual technique protocol (RNA isolation with cDNA production and qPCR). Transcripts (mRNA) were isolated from 1 ml EDTA-collected blood samples using the mini blood kit (Qiagen). The RNA quantity was 50 μl, the quality was >1.8 (A260:280 ratio); analysis of the RNA pattern on electrophoresis (Agilent Technologies, Santa Clara, CA, USA) RIN>5.0 (Fleige et al. 2006). The standard Qiagen isolation protocol (heme/gDNA contamination not detected) with no modifications was used. cDNA was produced from 50 μl RNA using a High Capacity Reverse transcriptase kit (Life Technologies: cDNA production 2000–2500 ng/μl) and stored at −80 °C. qPCR was performed (384-well plate, HT-7900) with the cDNA (200 ng/μl) and 16 μl of reagents/well (Universal Master Mix II with UNG, Life Technologies, triplicate wells) (50 °C 2 min, 95 °C 10 min, then 95 °C 15 s, 60 °C, 60 s for 40 cycles) as described (Modlin et al. 2013a, 2014). A NET score (0–8) is derived from the PCR data using MATLAB (R2011a, Mathworks, Natick, MA, USA) (Modlin et al. 2013b); a value ≥2 is a positive tumor score (Modlin et al. 2013a,b, 2014).
Single analyte ELISA-based tests
Chromogranin A ELISA: CgA was measured using the DAKO ELISA kit (K0025, DAKO North America, Inc., Carpinteria, CA, USA) (Modlin et al. 2009, 2013a, 2014). A cut-off of 14 Units/l (DAKO) was used as the upper limit of normal as preliminary studies based upon published papers that identified the majority (>75% of control samples, n=36) exhibit these levels (Modlin et al. 2013a).
Pancreastatin ELISA (#CSB-E09209h, CUSABIO, Atlanta, GA, USA) has an assay range of 31.25–2000 pg/ml with a sensitivity of 7.8 pg/ml. The intra- and interassay CV values are <8 and <10%, respectively. Final values calculated as pmol/l to facilitate comparisons with other assays (Stronge et al. 2008).
Neurokinin ELISA (#EIA-NEA1, Raybiotech, Norcross, GA, USA) has an assay range of 0.8–1000 pg/ml with a published sensitivity of 0.8 pg/ml (Dornan et al. 1993). The intra- and interassay CV values are <10 and <15%, respectively. In the current studies, we used a limit of detection of 0.8 pg/ml. Final values calculated as pmol/l to facilitate comparisons with other assays (Turner et al. 2006).
Prior to undertaking the prospective study, we examined patient samples (n=9) with known levels (normal or elevated) of CgA, PST and NKA. We compared values measured using clinical assays – Quest Diagnostics – for each of the three analytes to the three ELISAs (DAKO – CgA, CUSABIO – PST and RayBiotech – NKA, respectively). Concordant levels (abnormal 3/3 – 100% and not elevated 3/3 – 100%) were identified between the clinical assay and each of the three assays confirming the utility of the study assays.
Statistical analyses
Sensitivity comparisons using respectively χ2, non-parametric measurements and ROC analysis were made between the MAAA-PCR test and single analyte plasma ELISAs for detection of NET. Both Prism 6.0 for Windows (GraphPad Software, La Jolla, CA, USA, www.graphpad.com) and MedCalc Statistical Software version 12.7.7 (MedCalc Software bvba, Ostend, Belgium; http://www.medcalc.org; 2013) were utilized.
The accuracy of each of the single analyte assays was compared to the NETest using receiver operating characteristic (ROC) curve analyses (continuous variables) and the sensitivity, specificity, and the area under the curve (AUC) were calculated (MedCalc) (Hanley & McNeil 1982). AUCs were compared and the Z-statistic derived (Hanley & McNeil 1983) (MedCalc). Values are presented as mean with standard deviation (Set 1) and median with ranges (Set 2). Predictive feature importance values for each test were derived using the mean decrease in Gini coefficient, following construction of a random forest model with tenfold cross-validation (Nicodemus & Malley 2009). Variables with higher mean decrease in Gini coefficient have greater contribution to the homogeneity of the nodes and leaves in the resulting random forest (Nicodemus & Malley 2009).
Results
i) Validation set analyses (Set 1)
One hundred (20/20) percent of NETs exhibited a PCR score ≥2 (Fig. 1A) compared to one (5%) of the 20 controls (>χ2=32.4, P<4×10−10). Review of this control which had been included initially as a chronic pancreatitis patient revealed suspicious radiological findings and an elevated CA19-9 (43.7 ng/ml). The patient was therefore considered to have a possible underlying pancreatic neoplasm. An FNA was undertaken but interpreted as negative. It is therefore possible that this control actually does have a covert NET and that the positive control is in fact accurate. CgA was elevated in 45% of controls but was increased in 65% of NETs. Levels were, however, higher in NETs (mean 325±145 U/l vs 18.5±12.8, P=0.04). PST was elevated in 70% of NETs (25±4.6 pmol/l) but this was not different to controls (elevated in 45%, 18.5±1.4 pmol/l). NKA was elevated in 50% of NETs (0.42±0.27 pmol/l) but 25% of controls also exhibited elevated levels (0.08±0.03 pmol/l). The MAAA-PCR NETest test had significantly better (P<0.0001) performance metrics than any of the single analyte tests (Fig. 1B). ROC analysis confirmed the superiority of the NETest (AUC: 0.98±0.016 vs 0.58–0.63, Z-statistic =4.0–4.25, P<0.0001).
Validation set accuracy and performance metrics for the MAAA-NETest vs single analyte ELISAs for CgA, PST and NKA. (A) The NETest correctly predicted 100% of NETs and 95% of controls. CgA was elevated in 65% of cases and 45% of controls. PST was elevated in 70% of cases and 45% of controls, while NKA was elevated in 50% of cases and 25% of controls. (B) The sensitivity and specificity for the NETest were all >95%. The metrics for CgA ranged from 59 to 61%, for PST: 61–65% and for NKA: 60–67%. The AUC for PCR gene analysis was 0.98 and for CgA 0.60. For PST it was 0.58 and for NKA 0.63. The NETest outperformed the single analytes (P<0.0001). NETest, multigene test; CgA, chromogranin A; NKA, neurokinin A; PST, pancreastatin; SENS, sensitivity; SPEC, specificity. *P<0.0001 vs controls. **P<0.0001 vs CgA, PST and NKA. #P<0.001 vs single analytes (Z-statistic).
Citation: Endocrine-Related Cancer 21, 4; 10.1530/ERC-14-0190
Validation set accuracy and performance metrics for the MAAA-NETest vs single analyte ELISAs for CgA, PST and NKA. (A) The NETest correctly predicted 100% of NETs and 95% of controls. CgA was elevated in 65% of cases and 45% of controls. PST was elevated in 70% of cases and 45% of controls, while NKA was elevated in 50% of cases and 25% of controls. (B) The sensitivity and specificity for the NETest were all >95%. The metrics for CgA ranged from 59 to 61%, for PST: 61–65% and for NKA: 60–67%. The AUC for PCR gene analysis was 0.98 and for CgA 0.60. For PST it was 0.58 and for NKA 0.63. The NETest outperformed the single analytes (P<0.0001). NETest, multigene test; CgA, chromogranin A; NKA, neurokinin A; PST, pancreastatin; SENS, sensitivity; SPEC, specificity. *P<0.0001 vs controls. **P<0.0001 vs CgA, PST and NKA. #P<0.001 vs single analytes (Z-statistic).
Citation: Endocrine-Related Cancer 21, 4; 10.1530/ERC-14-0190
Validation set accuracy and performance metrics for the MAAA-NETest vs single analyte ELISAs for CgA, PST and NKA. (A) The NETest correctly predicted 100% of NETs and 95% of controls. CgA was elevated in 65% of cases and 45% of controls. PST was elevated in 70% of cases and 45% of controls, while NKA was elevated in 50% of cases and 25% of controls. (B) The sensitivity and specificity for the NETest were all >95%. The metrics for CgA ranged from 59 to 61%, for PST: 61–65% and for NKA: 60–67%. The AUC for PCR gene analysis was 0.98 and for CgA 0.60. For PST it was 0.58 and for NKA 0.63. The NETest outperformed the single analytes (P<0.0001). NETest, multigene test; CgA, chromogranin A; NKA, neurokinin A; PST, pancreastatin; SENS, sensitivity; SPEC, specificity. *P<0.0001 vs controls. **P<0.0001 vs CgA, PST and NKA. #P<0.001 vs single analytes (Z-statistic).
Citation: Endocrine-Related Cancer 21, 4; 10.1530/ERC-14-0190
ii) Prospective set analyses (Set 2)
iiA) MAAA-PCR NETest
Thirty-eight of the 41 NETs (93%) exhibited a PCR score ≥2 (positive test). The three patients with scores of two had been clinically assessed as stable disease following previous resection (right hemicolectomy (<1 cm small intestinal NET: n=1), pancreas resection (<1 cm insulinomas: n=2)) and were currently not receiving any treatment. Three (7%) of the age: sex matched control group had an elevated score; one was suspected to have a pancreatic NET, one had a pancreatic mass, subsequently confirmed to be a cyst with intraductal papillary mucinous neoplastic (IPMN) features and the last had uncontrolled IBD. The latter patient was identified during push-enteroscopy to have terminal ileal ulceration and the presence of nodules (<0.5 cm in diameter). Mucosal neuroendocrine cell hyperplasia and inflammatory infiltration were identified in the biopsies of the nodules. As expected, the percentage of elevated scores was significantly higher in the NET group compared to controls (median 4: 2–7 vs 0: 0–5, P<0.0001) (Mann–Whitney U test, two-tailed) (Fig. 2A).
Differences in NETest score, CgA levels, PST and NKA in age-sex matched NETs and controls (n=41 each). The MAAA-NETest was significantly higher (P<0.0001) in NETs compared to controls (A). CgA levels were also significantly elevated in NETs than in controls (P<0.01) (B) but PST levels did not differentiate the two groups (C). NKA levels (D) were, however, elevated in NETs (P<0.001). CON, control group; NET, neuroendocrine tumor group.
Citation: Endocrine-Related Cancer 21, 4; 10.1530/ERC-14-0190
Differences in NETest score, CgA levels, PST and NKA in age-sex matched NETs and controls (n=41 each). The MAAA-NETest was significantly higher (P<0.0001) in NETs compared to controls (A). CgA levels were also significantly elevated in NETs than in controls (P<0.01) (B) but PST levels did not differentiate the two groups (C). NKA levels (D) were, however, elevated in NETs (P<0.001). CON, control group; NET, neuroendocrine tumor group.
Citation: Endocrine-Related Cancer 21, 4; 10.1530/ERC-14-0190
Differences in NETest score, CgA levels, PST and NKA in age-sex matched NETs and controls (n=41 each). The MAAA-NETest was significantly higher (P<0.0001) in NETs compared to controls (A). CgA levels were also significantly elevated in NETs than in controls (P<0.01) (B) but PST levels did not differentiate the two groups (C). NKA levels (D) were, however, elevated in NETs (P<0.001). CON, control group; NET, neuroendocrine tumor group.
Citation: Endocrine-Related Cancer 21, 4; 10.1530/ERC-14-0190
iiB) Single analyte ELISA tests (Set 2)
CgA was elevated (>14 U/ml) in 31 (76%) of NETs, significantly more than in the control group (n=17, 41%) (median 21.6: 9.5–5235 vs 13: 5.7–109, P=0.006, Fig. 2B). Although CgA levels were elevated in the NET group with liver (n=18) and distant metastases (bone: n=2) (median: 27.2 U/l, range: 9.5–5235), this was not significantly different to lymph node/mesenteric metastases (median: 17 U/l, range: 11.4–21.6) or individuals with no documented metastases (median: 23.6 U/l, range: 10.1–137.5). A comparison of those taking somatostatin-based therapies (n=18) and those currently not on treatment (n=23) identified CgA tended to be higher in the latter group (median: 18.8 U/l, range: 9.5–5235 vs 24.6: 10.1–640). This did not achieve statistical significance (P=0.21). In the matched control group, CgA was significantly elevated in controls taking PPIs (median: 26 U/l, range: 11–93.7) compared to those not exposed to PPIs (median: 12 U/l, range: 5.7–109, P=0.0009, Mann–Whitney). Of note was the observation that seven controls with no documentation of PPIs administration exhibited elevated CgAs. It is likely that this reflects the presence of other undocumented pathology, e.g. hypertension (O'Connor 1985) not identified at blood draw.
PST was elevated (>17 pmol/l) in 26 (63%) of NETs, but this was not significantly more than in the control group (n=18) (median 17.7: 13.6–32.8 vs 16.8: 12.8–36.3, P=NS, Fig. 2C). A sub-analysis of the NET group identified that PST levels were similar in the NET group with liver (n=18) and distant metastases (bone: n=2) (median: 17.95 pmol/l, range: 13.8–29.9), with lymph node/mesenteric metastases (median: 16.95 pmol/l, range: 14.5–23.6) as well as those with no documented metastases (median: 17.1 pmol/l, range: 13.6–32.8). A comparison of those receiving somatostatin-based therapies (n=18) and those currently not on treatment (n=23) identified no differences in PST levels (median: 18.25 pmol/l, range: 13.8–23.4 vs 17.1: 13.6–32.8). In the matched control group, PST in contrast to CgA, was not significantly elevated in the PPI group (median: 16.95 pmol/l, range: 14.5–30) compared to those not receiving PPIs (median: 16.6 pmol/l, range: 12.8–36.3, P=0.34, Mann–Whitney).
NKA was elevated (>0.7 pmol/l) in 15 (37%) of NETs; this was significantly more than in the control group (n=4) (median 0.13: range 0–29 vs 0: 0–8.5, P=0.0003, Fig. 2D). Although NKA was elevated in the small intestinal NET group (median 0.63: 0–12.2) this did not reach statistical significance compared to other NETs (median 0.13: 0–29, P=0.055). Of note was the observation that while 11 (44%) of the 25 small intestinal had detectable NKA, it was also elevated in 31% of the non-SI NETs (P=NS). This is consistent with previous studies suggesting over-expression of NKA in ‘mid gut carcinoids’ (Turner et al. 2006).
A sub-analysis of the NET group identified that NKA levels were not associated with metastases (liver: n=18 and distant metastases/bone: n=2 – median: 0.57 pmol/l, range: 0–11.4, lymph node/mesenteric metastases (n=6 – median: 0.97 pmol/l, range: 0.24–4.8; no documented metastases (n=15 – median: 0.44 pmol/l, range: 0–29) and was not significantly altered by somatostatin-based therapies (SSA: median: 0.57 pmol/l, range: 0–5.3 vs 0.47: 0–29). In the matched control group, NKA was not significantly elevated in individuals receiving PPIs (median: 0 pmol/l, range: 0–8.5) compared to non-PPIs users (median: 0 pmol/l, range: 0–1.24, P=0.51, Mann–Whitney).
iiC) Performance metrics (Set 2)
The NETest test had significantly better (P<0.0001) performance metrics than any of the single analyte tests. The differences in performance metrics for differentiating a NET using the multigene test vs the single analyte assays in the 82 samples (41 age-sex matched NETs and controls) are presented in Fig. 3A. For the NETest, the sensitivity was 92.8%, specificity 92.8%, positive predictive value (PPV) 92.8% and negative predictive value (NPV) 92.8%. For the ELISA assays these ranged from sensitivities: 39–75.6%, specificities: 56.1–93%, PPVs: 59.1–84% and NPVs: 60–70.6%. To formally compare these different tests, we undertook ROC analysis. The AUCs were: NETest: 0.957±0.025, CgA: 0.67±0.06, PST: 0.56±0.06 and NKA: 0.66±0.06 (Fig. 3B and Table 3). Comparing each of the AUCs (Hanley & McNeil 1983) identified that the NETest was significantly better than any of the single analyte tests (difference between areas: 0.284–0.403, Z-statistic =4.85–5.9, P<0.0001, Table 4).
Performance metrics for the MAAA-NETest vs the single analyte ELISAs for CgA, PST and NKA in the 41 matched NETs and controls. (A) The sensitivity, specificity, PPV and NPV for the NETest were all >90%. The metrics for CgA ranged from 58.5 to 75.6%, for PST it was: 31.7–80.5% and for NKA: 39–93%. (B) Receiver operating characteristic (ROC) curves for PCR gene analysis compared to CgA, PST and NKA. The AUC for PCR gene analysis was 0.96 and for CgA 0.67. For PST it was 0.51 and for NKA it was 0.66. The NETest outperformed the single analytes. NETest, multigene test; CgA, chromogranin A; SENS, sensitivity; SPEC, specificity; PPV, positive predictive value; NPV, negative predictive value. The dotted line (A) represents 80% (standard cut-off level for biomarkers).
Citation: Endocrine-Related Cancer 21, 4; 10.1530/ERC-14-0190
Performance metrics for the MAAA-NETest vs the single analyte ELISAs for CgA, PST and NKA in the 41 matched NETs and controls. (A) The sensitivity, specificity, PPV and NPV for the NETest were all >90%. The metrics for CgA ranged from 58.5 to 75.6%, for PST it was: 31.7–80.5% and for NKA: 39–93%. (B) Receiver operating characteristic (ROC) curves for PCR gene analysis compared to CgA, PST and NKA. The AUC for PCR gene analysis was 0.96 and for CgA 0.67. For PST it was 0.51 and for NKA it was 0.66. The NETest outperformed the single analytes. NETest, multigene test; CgA, chromogranin A; SENS, sensitivity; SPEC, specificity; PPV, positive predictive value; NPV, negative predictive value. The dotted line (A) represents 80% (standard cut-off level for biomarkers).
Citation: Endocrine-Related Cancer 21, 4; 10.1530/ERC-14-0190
Performance metrics for the MAAA-NETest vs the single analyte ELISAs for CgA, PST and NKA in the 41 matched NETs and controls. (A) The sensitivity, specificity, PPV and NPV for the NETest were all >90%. The metrics for CgA ranged from 58.5 to 75.6%, for PST it was: 31.7–80.5% and for NKA: 39–93%. (B) Receiver operating characteristic (ROC) curves for PCR gene analysis compared to CgA, PST and NKA. The AUC for PCR gene analysis was 0.96 and for CgA 0.67. For PST it was 0.51 and for NKA it was 0.66. The NETest outperformed the single analytes. NETest, multigene test; CgA, chromogranin A; SENS, sensitivity; SPEC, specificity; PPV, positive predictive value; NPV, negative predictive value. The dotted line (A) represents 80% (standard cut-off level for biomarkers).
Citation: Endocrine-Related Cancer 21, 4; 10.1530/ERC-14-0190
Performance metrics
| AUC | s.e.m.a | 95% CIb | |
|---|---|---|---|
| NETest | 0.957 | 0.0249 | 0.888–0.990 |
| CgA | 0.673 | 0.0593 | 0.561–0.773 |
| PST | 0.555 | 0.0643 | 0.441–0.665 |
| NKA | 0.664 | 0.0607 | 0.551–0.765 |
Pairwise comparison of ROC curves
| Difference between AUCs | s.e.m.a | 95% CI | Z-statistic | P value | |
|---|---|---|---|---|---|
| NETest vs CgA | 0.284 | 0.0625 | 0.162 to 0.406 | 4.548 | <0.0001 |
| NETest vs PST | 0.403 | 0.0682 | 0.269 to 0.536 | 5.902 | <0.0001 |
| NETest vs NKA | 0.294 | 0.0603 | 0.175 to 0.412 | 4.867 | <0.0001 |
| CgA vs PST | 0.119 | 0.0845 | −0.0470 to 0.284 | 1.404 | 0.1602 |
| CgA vs NKA | 0.00952 | 0.0819 | −0.151 to 0.170 | 0.116 | 0.9075 |
| PST vs NKA | 0.109 | 0.0833 | −0.0541 to 0.272 | 1.31 | 0.1901 |
AUC, area under the curve.
iiD) Predictive feature importance analyses (Set 2)
Feature importance analysis (see Methods) was undertaken to compare the NETest with CgA, PST and NKA measurements in order to evaluate the importance of each to the detection of NETs. The NETest was seven times more important (69 vs 9–13%) than the contributions of any of the other single analytes in the diagnosis of NETs (Fig. 4A). A visual representation of this data is provided in the consensus heatmap which demonstrated that the NETest is most often correct in differentiating between NETs and controls (Fig. 4B). A number of controls have abnormally elevated CgA or PST (and are therefore ‘called’ NETs – this is most likely reflective of concomitant PPI use or hypertension) while NKA was undetectable in the majority of patients and controls (this due to the limit of detection of 0.8 pg/ml).
Feature importance analysis for the MAAA-NETest, and the CgA, PST and NKA in the 41 matched NETs and controls. (A) Pie chart of the individual test contribution in the detection of NETs. The importance of the NETest in the diagnosis of NETs (expressed as a mean decrease in Gini coefficient) was seven times higher than any of the single analytes. (B) Consensus heatmap of diagnosis and test. Sample classification by each of the test in comparison to diagnosis highlights that the NETest is most often correct. A number of controls have abnormally elevated CgA or PST (and are therefore called ‘NETs’). NKA is undetectable in the majority of patients or controls. Controls (blue), NETs (red).
Citation: Endocrine-Related Cancer 21, 4; 10.1530/ERC-14-0190
Feature importance analysis for the MAAA-NETest, and the CgA, PST and NKA in the 41 matched NETs and controls. (A) Pie chart of the individual test contribution in the detection of NETs. The importance of the NETest in the diagnosis of NETs (expressed as a mean decrease in Gini coefficient) was seven times higher than any of the single analytes. (B) Consensus heatmap of diagnosis and test. Sample classification by each of the test in comparison to diagnosis highlights that the NETest is most often correct. A number of controls have abnormally elevated CgA or PST (and are therefore called ‘NETs’). NKA is undetectable in the majority of patients or controls. Controls (blue), NETs (red).
Citation: Endocrine-Related Cancer 21, 4; 10.1530/ERC-14-0190
Feature importance analysis for the MAAA-NETest, and the CgA, PST and NKA in the 41 matched NETs and controls. (A) Pie chart of the individual test contribution in the detection of NETs. The importance of the NETest in the diagnosis of NETs (expressed as a mean decrease in Gini coefficient) was seven times higher than any of the single analytes. (B) Consensus heatmap of diagnosis and test. Sample classification by each of the test in comparison to diagnosis highlights that the NETest is most often correct. A number of controls have abnormally elevated CgA or PST (and are therefore called ‘NETs’). NKA is undetectable in the majority of patients or controls. Controls (blue), NETs (red).
Citation: Endocrine-Related Cancer 21, 4; 10.1530/ERC-14-0190
Discussion
Given the awareness that biomarker measurement represent a major advancing area in amplifying disease management, substantial consideration has been given to the performance criteria necessary to ensure scientific and clinical value. Circulating biomarkers must exhibit a series of attributes to be considered as effective in disease detection and monitoring (2010). These include expression in a peripheral compartment, e.g. blood; robust assays for detection and measurement of the product(s) must both be associated as specifically as possible with a particular tumor as well as differentiate normal from this specific disease. In general, performance metrics, including the AUC, should be >80% (Palmer et al. 2008). The current study demonstrates that measurement of a single tumor product (e.g. NKA) or general markers of neuroendocrine cells (CgA, PST) fail to meet these criteria for the detection of NETs. None of these single analyte measurements can be considered optimally effective in attaining these goals. Indeed, this is predictable given the fact that a single analyte produced by a specific cell type is unlikely to be broadly represented in a heterogeneous group of tumors. In terms of the mathematics of the process, a monoanalyte cannot predict numerous variables. Nevertheless, such assertions have been made for monoanalyte assays such as CgA which has been variously suggested to predict diagnosis, tumor burden, prognosis and likelihood of response to therapy. In contrast, the NETest, which measures 51 analytes defining a wide array of neuroendocrine tumor gene clusters, is highly accurate for the detection of NETs. Based upon preliminary data in 133 patients (Modlin et al. 2013b) it seems likely that further investigation will demonstrate the feasibility of the NETest in providing information relevant to assessment of tumor burden and measurement of treatment efficacy. This reflects the fact that the 51 different algorithmically assembled analytes represent numerous gene clusters specific to the neuroendocrine tumor cell and different elements of its biological behavior. As might be predicted from a cell type-specific series of gene transcripts, the test is neither up-regulated in controls (as occurs for CgA and PST) nor is it undetectable in tumors (as for NKA).
The NETest conforms to a category of assays known as MAAAs. These are procedures that utilize multiple results derived from assays of various types, including molecular pathology assays, fluorescent in situ hybridization assays and non-nucleic acid based assays (e.g. proteins, polypeptides, lipids, carbohydrates) (Faltin et al. 2013). Algorithmic analyses, using the results of these assays as well as other patient information (if used), can then be performed and are typically reported as a numeric score(s) or as a probability (Eastman et al. 2012) that can provide additional prognostic and predictive information, further aiding clinical management (Cornejo et al. 2014). The strengths of MAAAs are the incorporation of multiple informative data points as well as these are typically undertaken by a single dedicated facility (Lee & Hall 2009).
The advent of multiparameter technology has been driven by the need to understand the complexity in biological systems and high-content analytical techniques have rapidly been introduced into both research and clinical laboratories. This has largely been undertaken in response to a shift in perspective from the analysis of individual molecules to the analysis of complex biological systems (e.g. combinatorial chemistry, genomics, and proteomics) (Hiraki et al. 2014). The power of MAAA profiling is in the utilization of technical and informational synergies in parallel systems, often in real time, revealing information among related analytes that may not be fully appreciated in conventional iterative assays (Torres-Chavolla & Alocilja 2009). Beyond the rapid accumulation of data, these systems have the ability to integrate qualitative findings with robust analytical techniques for quantifying complex interactions among molecules (Salas et al. 2008). The most recognizable area has been in microarray technology. Recent examples have demonstrated the utility of transcriptome-based analysis by the use of gene profiling to identify breast cancers that will respond to treatment (Miller et al. 2009, Zembutsu et al. 2009). While such approaches encompass up to thousands of analytes, it is relevant that analysis of <100 genes in PCR-based tests are informative. Examples include the recently FDA approved 70 gene MammaPrint assay test that functions as a specific and crucial aspect of metastatic and recurrent potential for breast cancer (Kittaneh et al. 2013) as well as FibroSure (FibroTest) which is a six-panel blood test for liver damage and has become widely used for hepatitis C diagnosis and detection and can replace liver biopsy (Poynard et al. 2004). Interestingly, it is relevant that even as few as two biomarkers can be more informative than a single analyte. Overall survival (hazard ratios >2.5 vs 1.3) as well as the relapse free period are more accurately predicted by combinations of two biomarkers than single markers (HR >2.3 vs 1.3) in breast cancer (Engels et al. 2013). The utility of the biomarkers increased in an additive fashion but there are numerous examples where accuracy increases in non-additive fashion. One example is represented by the immunohistochemical classification of head and neck cancer (HNC) cell lines. An accurate classification of the latter requires a minimum of seven different immunohistochemical parameters (∼80%); staining for an individual marker is not associated with a definitive classification (random forest classification accuracy <20%!) (Hoeben et al. 2014). At a circulating level, CgA has often been used in the diagnosis of pheochromocytomas despite the cost- and time-related issues of false positives and negative results (Sawka et al. 2004). Measurements of three analytes (secretogranin II-derived EM66, CgA and the CgA-derived WE-14 peptide) have a better sensitivity than any of the single analytes (95 vs 84%) (Guillemot et al. 2014) confirming the utility of MAAA approaches. The authors suggest this could accompany routine assays of metanephrine levels for the diagnosis of pheochromocytoma. A further example is provided by the use of combinations of epitope detection in monocytes where measures of Apo10 and TKTL1 exhibited significantly better performance metrics than a single measure alone in oral squamous cell carcinoma, breast cancer and prostate cancer (average single analyte sensitivity: 93 vs 95%; specificity: 90 vs 97%) (Grimm et al. 2013). Thus, the combined detection of two independent fundamental biophysical processes allows for the sensitive and specific detection of neoplasia in a noninvasive and cost-effective way. It seems likely that the MAAA-derived values will similarly allow for a sensitive non-invasive method for detecting NETs. We consider that while of utility alone, such an approach may be rendered even more informative in combination with assessments provided by integration with objective data obtained from imaging and nuclear medicine scanning, as has been shown for HNC (Hoeben et al. 2014).
Up-regulation of CgA is understood to reflect PPI-medication (Lawrence et al. 2011) while this class of agent is not thought to influence PST levels (Raines et al. 2012). In the current study, the NETest was not elevated in any of 12 individuals receiving PPI. In contrast, plasma CgA was elevated in 10 (83%) while PST was abnormally elevated in 50% of this group. We could therefore identify no overall effect of PPIs on PST (no differences in PPI-treated and non-treated controls), which presumably reflects the heterogeneity in secretion of this peptide family following prohormone cleavage (Udupi et al. 1999). While others have identified correlations between CgA and PST (Syversen et al. 1994), we could identify no statistically significant relationship; this is consistent with previous assays (Stronge et al. 2008, O'Dorisio et al. 2010). CgA is also elevated in hypertension (Takiyyuddin et al. 1995) (particularly when not treated (O'Connor 1985)). It is relevant that under these conditions PST has been reported as increased (Sanchez-Margalet et al. 2010). This suggests that abnormal measurements of these analytes is due to a range of factors including medications and genetic influences, many of which may not be apparent at the time of blood sampling. Under these conditions, NETest levels were normal demonstrating that the multianalyte assay is not affected by factors unrelated to neuroendocrine tumor disease.
Somatostatin-based therapies are often used in the treatment of NET symptoms and more recently have been advocated as an antiproliferative agent (Wolin 2012). In the current study, 18 NETs were undergoing treatment with either Sandostatin (n=16) or Somatulin (n=2). Plasma CgA and PST levels were elevated in ∼60–65%, suggesting an inadequacy of effective dosages for symptom control in about two thirds of patients (Woltering et al. 2006). It should be noted that the NETest was elevated in all treated patients (many of whom were asymptomatic) suggesting that the multianalyte assay may be more reflective of the tumor activity rather than a measure of tumor secretion per se. Irrespective, these data suggest that the NETest may be of use in assessing the effect of SSA efficacy. Similarly, the effects of PPIs in providing false elevations of single analyte test results (CgA) are not evident. This is consistent with data demonstrating that a PCR (MAAA) multigene test was unaffected by acid suppressive therapy (Modlin et al. 2014). This discrepancy may well provide some degree of understanding of the relative lack of efficacy of NET-directed therapy (Bergsland 2013, Oberg et al. 2013, Pavel et al. 2013).
Using a number of different analytic methods of performance (performance metrics, ROC curve analysis, and predictive feature importance analysis), we demonstrated that the multigene test significantly outperformed single analyte measures in the detection of NETs. The sensitivities and specificities in this prospective cohort are similar to those previously reported by us (85–98% sensitivity and 93–97% specificity) in different sample sets (Modlin et al. 2013a, 2014). The AUC (from the ROC curve) in the current study was 0.96, similar to that previously reported (95–98%) (Modlin et al. 2013a). Each of the ELISA single analytes, in contrast, had significantly lower AUCs (ranging from 0.56 to 0.67). In contrast, ROC AUC curves for commercially available CgA RIAs (CIS-Bio and EuroDiagnostica, considered more sensitive than ELISA) generally range from 0.48 to 0.76 (Marotta et al. 2012, de Laat et al. 2013, Tohmola et al. 2013). In histopathologically confirmed NETs, using a processing-independent analysis method which is also radioimmune-based but not commercially available, the metrics range from 73–86% (sensitivity) and 86–91% (specificity) (Jensen et al. 2013).
We could identify no data in the literature for either PST or NKA which likely reflects the absence of formal studies evaluating whether these are biomarkers. Nevertheless, the AUCs were all significantly lower (Z-statistic >4.5, P<0.0001) for these markers which further raises the issue as to their utility in NETs (Marotta et al. 2012, de Laat et al. 2013). The subject is further complicated by the existence of several commercially available and laboratory-developed assays that have been developed to measure putative single NET detection analytes. In the absence of a universally accepted approach, levels of each analyte can be expected to vary substantially between test platforms, all of which will have varying sensitivities and specificities. In the current study, for example, PST levels were all <40 pmol/l irrespective of tumor or control in Set 2, while only three of 20 NETs in Set 1 had levels greater than this. Other studies have determined cut-offs of 25–50 pmol/l for ‘normal’ depending on the test platform (Stronge et al. 2008, O'Dorisio et al. 2010). We identified the same issue with NKA where cut-offs range from 0.8 pg to 2 ng/ml depending on the kit type used (Turner et al. 2006).
Irrespective of the use of a specific assay or whether a single analyte or combinations of single analytes (e.g. CgA and Neuron-Specific Enolase) are used, the multianalyte assay performed significantly better (Yao et al. 2011). Furthermore, the use of combinations of assays is not only cumbersome and time consuming but adds complex confounding variables. Our proposal is that a multianalyte gene transcript analysis for NETs is a logical advance in increasing the sensitivity and specificity of a biomarker tool is supported by others who have concluded that single analyte tests may be supplanted by PCR-based analysis of NET genes detectable in the blood transcriptome (Lewis & Yao 2013).
We conclude that a 51 panel multigene blood transcript analysis is significantly more sensitive and efficient (>93%) than any single analyte assay (CgA, PST or NKA) for NET detection in a prospectively collected age and sex matched sample set of NETs and controls. These data are consistent with data derived from the validation set and recapitulated the assay performance metrics. Furthermore, our investigations suggest that a blood-based multigene analytic measurement, like other MAAA-biomarker tests, will provide additional information that can be used for the detection of disease recurrence. It may also provide added value in the prediction of therapeutic efficacy particularly when other biomarkers e.g. CgA, provide no evidence of disease (Modlin et al. 2013b, 2014).
Declaration of interest
The authors declare that there is no conflict of interest that could be perceived as prejudicing the impartiality of the research reported.
Funding
This work was supported by Clifton Life Sciences.
References
2010 Biomarkers on a roll. Nature Biotechnology28 431 (doi:10.1038/nbt0510-431).
Bergenfelz A, Luts L, Jensen TB & Sundler F 2000 Pancreastatin plasma levels in patients with primary hyperparathyroidism. World Journal of Surgery 24 1579–1583. (doi:10.1007/s002680010281).
Bergsland EK 2013 The evolving landscape of neuroendocrine tumors. Seminars in Oncology 40 4–22. (doi:10.1053/j.seminoncol.2012.11.013).
Calhoun K, Toth-Fejel S, Cheek J & Pommier R 2003 Serum peptide profiles in patients with carcinoid tumors. American Journal of Surgery 186 28–31. (doi:10.1016/S0002-9610(03)00115-6).
Castano JP, Sundin A, Maecke HR, Villabona C, Vazquez-Albertino R, Navarro E & Oberg K 2014 Gastrointestinal neuroendocrine tumors (NETs): new diagnostic and therapeutic challenges. Cancer Metastasis Reviews 5 5.
Cornejo KM, Kandil D, Khan A & Cosar EF 2014 Theranostic and molecular classification of breast cancer. Archives of Pathology & Laboratory Medicine 138 44–56. (doi:10.5858/arpa.2012-0442-RA).
Curtis KA, Hanson DL, Kennedy MS & Owen SM 2013 Evaluation of a multiplex assay for estimation of HIV-1 incidence. PLoS ONE 8 e64201. (doi:10.1371/journal.pone.0064201).
Denecke T, Baur AD, Ihm C, Steffen IG, Tischer E, Arsenic R, Pascher A, Wiedenmann B & Pavel M 2013 Evaluation of radiological prognostic factors of hepatic metastases in patients with non-functional pancreatic neuroendocrine tumors. European Journal of Radiology 82 e550–e555. (doi:10.1016/j.ejrad.2013.06.017).
Dobson R, Burgess MI, Banks M, Pritchard DM, Vora J, Valle JW, Wong C, Chadwick C, George K & Keevil B et al. 2013 The association of a panel of biomarkers with the presence and severity of carcinoid heart disease: a cross-sectional study. PLoS ONE 8 e73679. (doi:10.1371/journal.pone.0073679).
Dornan WA, Vink KL, Malen P, Short K, Struthers W & Barrett C 1993 Site-specific effects of intracerebral injections of three neurokinins (neurokinin A, neurokinin K, and neurokinin γ) on the expression of male rat sexual behavior. Physiology & Behavior 54 249–258. (doi:10.1016/0031-9384(93)90107-Q).
Eastman PS, Manning WC, Qureshi F, Haney D, Cavet G, Alexander C & Hesterberg LK 2012 Characterization of a multiplex, 12-biomarker test for rheumatoid arthritis. Journal of Pharmaceutical and Biomedical Analysis 70 415–424. (doi:10.1016/j.jpba.2012.06.003).
Eisenhauer EA, Therasse P, Bogaerts J, Schwartz LH, Sargent D, Ford R, Dancey J, Arbuck S, Gwyther S & Mooney M et al. 2009 New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1). European Journal of Cancer 45 228–247. (doi:10.1016/j.ejca.2008.10.026).
Engels CC, Ruberta F, de Kruijf EM, van Pelt GW, Smit VT, Liefers GJ, Matsushima T, Shibayama M, Ishihara H & van de Velde CJ et al. 2013 The prognostic value of apoptotic and proliferative markers in breast cancer. Breast Cancer Research and Treatment 142 323–339. (doi:10.1007/s10549-013-2748-y).
Faivre S, Ronot M, Dreyer C, Serrate C, Hentic O, Bouattour M, Bruno O, Couvelard A, Vilgrain V & Raymond E 2012 Imaging response in neuroendocrine tumors treated with targeted therapies: the experience of sunitinib. Targeted Oncology 7 127–133. (doi:10.1007/s11523-012-0216-y).
Faltin B, Zengerle R & von Stetten F 2013 Current methods for fluorescence-based universal sequence-dependent detection of nucleic acids in homogenous assays and clinical applications. Clinical Chemistry 59 1567–1582. (doi:10.1373/clinchem.2013.205211).
Fleige S, Walf V, Huch S, Prgomet C, Sehm J & Pfaffl MW 2006 Comparison of relative mRNA quantification models and the impact of RNA integrity in quantitative real-time RT-PCR. Biotechnology Letters 28 1610–1613. (doi:10.1007/s10529-006-9127-2).
Grimm M, Schmitt S, Teriete P, Biegner T, Stenzl A, Hennenlotter J, Muhs H-J, Munz A, Nadtotschi T & König K et al. 2013 A biomarker based detection and characterization of carcinomas exploiting two fundamental biophysical mechanisms in mammalian cells. BMC Cancer 13 569. (doi:10.1186/1471-2407-13-569).
Guillemot J, Guerin M, Thouennon E, Montero-Hadjadje M, Leprince J, Lefebvre H, Klein M, Muresan M, Anouar Y & Yon L 2014 Characterization and plasma measurement of the WE-14 peptide in patients with pheochromocytoma. PLoS ONE 9 e88698. (doi:10.1371/journal.pone.0088698).
Hanley JA & McNeil BJ 1982 The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143 29–36.
Hanley JA & McNeil BJ 1983 A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 148 839–843.
Hiraki S, Rinella ES, Schnabel F, Oratz R & Ostrer H 2014 Cancer risk assessment using genetic panel testing: considerations for clinical application. Journal of Genetic Counseling 7 7.
Hoeben BA, Starmans MH, Leijenaar RT, Dubois LJ, van der Kogel AJ, Kaanders JH, Boutros PC, Lambin P & Bussink J 2014 Systematic analysis of 18F-FDG PET and metabolism, proliferation and hypoxia markers for classification of head and neck tumors. BMC Cancer 14 130. (doi:10.1186/1471-2407-14-130).
Hood L & Tian Q 2012 Systems approaches to biology and disease enable translational systems medicine. Genomics, Proteomics & Bioinformatics 10 181–185. (doi:10.1016/j.gpb.2012.08.004).
Hsiao RJ, Mezger MS & O'Connor DT 1990 Chromogranin A in uremia: progressive retention of immunoreactive fragments. Kidney International 37 955–964. (doi:10.1038/ki.1990.71).
Ito T, Igarashi H & Jensen RT 2012 Serum pancreastatin: the long sought universal, sensitive, specific tumor marker for neuroendocrine tumors? Pancreas 41 505–507. (doi:10.1097/MPA.0b013e318249a92a).
Jensen KH, Hilsted L, Jensen C, Mynster T, Rehfeld JF & Knigge U 2013 Chromogranin A is a sensitive marker of progression or regression in ileo-cecal neuroendocrine tumors. Scandinavian Journal of Gastroenterology 48 70–77. (doi:10.3109/00365521.2012.733953).
Joos GF, Vincken W, Louis R, Schelfhout VJ, Wang JH, Shaw MJ, Cioppa GD & Pauwels RA 2004 Dual tachykinin NK1/NK2 antagonist DNK333 inhibits neurokinin A-induced bronchoconstriction in asthma patients. European Respiratory Journal 23 76–81. (doi:10.1183/09031936.03.00101902).
Kanakis G & Kaltsas G 2012 Biochemical markers for gastroenteropancreatic neuroendocrine tumours (GEP-NETs). Best Practice & Research. Clinical Gastroenterology 26 791–802. (doi:10.1016/j.bpg.2012.12.006).
Kittaneh M, Montero AJ & Gluck S 2013 Molecular profiling for breast cancer: a comprehensive review. Biomarkers in Cancer 5 61–70.
de Laat JM, Pieterman CR, Weijmans M, Hermus AR, Dekkers OM, de Herder WW, van der Horst-Schrivers AN, Drent ML, Bisschop PH & Havekes B et al. 2013 Low accuracy of tumor markers for diagnosing pancreatic neuroendocrine tumors in multiple endocrine neoplasia type 1 patients. Journal of Clinical Endocrinology and Metabolism 98 4143–4151. (doi:10.1210/jc.2013-1800).
Lawrence B, Gustafsson BI, Kidd M, Pavel M, Svejda B & Modlin IM 2011 The clinical relevance of chromogranin A as a biomarker for gastroenteropancreatic neuroendocrine tumors. Endocrinology and Metabolism Clinics of North America 40 111–134. (doi:10.1016/j.ecl.2010.12.001).
Lecci A, Capriati A, Altamura M & Maggi CA 2006 Tachykinins and tachykinin receptors in the gut, with special reference to NK2 receptors in human. Autonomic Neuroscience 126–127 232–249. (doi:10.1016/j.autneu.2006.02.014).
Lee JW & Hall M 2009 Method validation of protein biomarkers in support of drug development or clinical diagnosis/prognosis. Journal of Chromatography. B, Analytical Technologies in the Biomedical and Life Sciences 877 1259–1271. (doi:10.1016/j.jchromb.2008.11.022).
Lewis MA & Yao JC 2013 Molecular pathology and genetics of gastrointestinal neuroendocrine tumours. Current Opinion in Endocrinology, Diabetes, and Obesity 4 4.
Marotta V, Nuzzo V, Ferrara T, Zuccoli A, Masone M, Nocerino L, Del Prete M, Marciello F, Ramundo V & Lombardi G et al. 2012 Limitations of chromogranin A in clinical practice. Biomarkers 17 186–191. (doi:10.3109/1354750X.2012.654511).
de Mestier L, Dromain C, d'Assignies G, Scoazec JY, Lassau N, Lebtahi R, Brixi H, Mitry E, Guimbaud R & Courbon F et al. 2013 Evaluating neuroendocrine tumors progression and therapeutic response: state of the art. Endocrine-Related Cancer 18 18.
Miller WR, Larionov A, Renshaw L, Anderson TJ, Walker JR, Krause A, Sing T, Evans DB & Dixon JM 2009 Gene expression profiles differentiating between breast cancers clinically responsive or resistant to letrozole. Journal of Clinical Oncology 27 1382–1387. (doi:10.1200/JCO.2008.16.8849).
Modlin IM, Moss SF, Chung DC, Jensen RT & Snyderwine E 2008a Priorities for improving the management of gastroenteropancreatic neuroendocrine tumors. Journal of the National Cancer Institute 100 1282–1289. (doi:10.1093/jnci/djn275).
Modlin IM, Oberg K, Chung DC, Jensen RT, de Herder WW, Thakker RV, Caplin M, Delle Fave G, Kaltsas GA & Krenning EP et al. 2008b Gastroenteropancreatic neuroendocrine tumours. Lancet Oncology 9 61–72. (doi:10.1016/S1470-2045(07)70410-2).
Modlin IM, Gustafsson BI, Drozdov I, Nadler B, Pfragner R & Kidd M 2009 Principal component analysis, hierarchical clustering, and decision tree assessment of plasma mRNA and hormone levels as an early detection strategy for small intestinal neuroendocrine (carcinoid) tumors. Annals of Surgical Oncology 16 487–498. (doi:10.1245/s10434-008-0251-1).
Modlin IM, Gustafsson BI, Moss SF, Pavel M, Tsolakis AV & Kidd M 2010 Chromogranin A – biological function and clinical utility in neuro endocrine tumor disease. Annals of Surgical Oncology 17 2427–2443. (doi:10.1245/s10434-010-1006-3).
Modlin IM, Moss SF, Gustafsson BI, Lawrence B, Schimmack S & Kidd M 2011 The archaic distinction between functioning and nonfunctioning neuroendocrine neoplasms is no longer clinically relevant. Langenbeck's Archives of Surgery 27 27.
Modlin I, Drozdov I & Kidd M 2013a The Identification of gut neuroendocrine tumor disease by multiple synchronous transcript analysis in blood. PLoS ONE 8 e63364. (doi:10.1371/journal.pone.0063364).
Modlin I, Drozdov I & Kidd M 2013b A multitranscript blood neuroendocrine tumor molecular signature to identify treatment efficacy and disease progress. Journal of Clinical Oncology 31 (Suppl) A4137.
Modlin I, Drozdov I & Kidd M 2014 Gut neuroendocrine tumor blood qPCR fingerprint assay: characteristics and reproducibility. Clinical Chemistry 52 419–429.
Neperud J, Mahvash A, Garg N, Murthy R & Szklaruk J 2013 Can imaging patterns of neuroendocrine hepatic metastases predict response yttruim-90 radioembolotherapy? World Journal of Radiology 5 241–247. (doi:10.4329/wjr.v5.i6.241).
Nicodemus KK & Malley JD 2009 Predictor correlation impacts machine learning algorithms: implications for genomic studies. Bioinformatics 25 1884–1890. (doi:10.1093/bioinformatics/btp331).
Oberg K, Casanovas O, Castano JP, Chung D, Delle Fave G, Denefle P, Harris P, Khan MS, Kulke MH & Scarpa A et al. 2013 Molecular pathogenesis of neuroendocrine tumors: implications for current and future therapeutic approaches. Clinical Cancer Research 19 2842–2849. (doi:10.1158/1078-0432.CCR-12-3458).
O'Connor DT 1985 Plasma chromogranin A. Initial studies in human hypertension. Hypertension 7 I76–I79. (doi:10.1161/01.HYP.7.3_Pt_2.I76).
O'Dorisio TM, Krutzik SR, Woltering EA, Lindholm E, Joseph S, Gandolfi AE, Wang YZ, Boudreaux JP, Vinik AI & Go VL et al. 2010 Development of a highly sensitive and specific carboxy-terminal human pancreastatin assay to monitor neuroendocrine tumor behavior. Pancreas 39 611–616. (doi:10.1097/MPA.0b013e3181c68d7a).
Palmer C, Duan X, Hawley S, Scholler N, Thorpe JD, Sahota RA, Wong MQ, Wray A, Bergan LA & Drescher CW et al. 2008 Systematic evaluation of candidate blood markers for detecting ovarian cancer. PLoS ONE 3 e2633. (doi:10.1371/journal.pone.0002633).
Pavel M, Kidd M & Modlin I 2013 Systemic treatment for “carcinoid”: what are the options? Seminars in Oncology40 84–99
Poynard T, Imbert-Bismut F, Munteanu M, Messous D, Myers RP, Thabut D, Ratziu V, Mercadier A, Benhamou Y & Hainque B 2004 Overview of the diagnostic value of biochemical markers of liver fibrosis (FibroTest, HCV FibroSure) and necrosis (ActiTest) in patients with chronic hepatitis C. Comparative Hepatology 3 8. (doi:10.1186/1476-5926-3-8).
Raines D, Chester M, Diebold AE, Mamikunian P, Anthony CT, Mamikunian G & Woltering EA 2012 A prospective evaluation of the effect of chronic proton pump inhibitor use on plasma biomarker levels in humans. Pancreas 41 508–511. (doi:10.1097/MPA.0b013e318243a0b6).
Raza A, Ali Z, Irfan J, Murtaza S & Shakeel S 2012 Analytical variables influencing the HCV RNA determination by TaqMan real-time PCR in routine clinical laboratory practice. Molecular Biology Reports 39 7421–7427. (doi:10.1007/s11033-012-1574-3).
Salas VM, Edwards BS & Sklar LA 2008 Advances in multiple analyte profiling. Advances in Clinical Chemistry 45 47–74.
Sanchez-Margalet V, Gonzalez-Yanes C, Najib S & Santos-Alvarez J 2010 Metabolic effects and mechanism of action of the chromogranin A-derived peptide pancreastatin. Regulatory Peptides 161 8–14. (doi:10.1016/j.regpep.2010.02.005).
Sawka AM, Gafni A, Thabane L & Young WF Jr 2004 The economic implications of three biochemical screening algorithms for pheochromocytoma. Journal of Clinical Endocrinology and Metabolism 89 2859–2866. (doi:10.1210/jc.2003-031127).
Sciarra A, Monti S, Gentile V, Salciccia S, Gomez AM, Pannunzi LP & Di Silverio F 2005 Chromogranin A expression in familial versus sporadic prostate cancer. Urology 66 1010–1014. (doi:10.1016/j.urology.2005.05.045).
Siegel EG, Gallwitz B, Folsch UR & Schmidt WE 1998 Effect of human pancreastatin peptide (hP-16) on oral glucose tolerance in man. Experimental and Clinical Endocrinology & Diabetes 106 178–182. (doi:10.1055/s-0029-1211973).
Stridsberg M, Oberg K, Li Q, Engstrom U & Lundqvist G 1995 Measurements of chromogranin A, chromogranin B (secretogranin I), chromogranin C (secretogranin II) and pancreastatin in plasma and urine from patients with carcinoid tumours and endocrine pancreatic tumours. Journal of Endocrinology 144 49–59. (doi:10.1677/joe.0.1440049).
Stronge RL, Turner GB, Johnston BT, McCance DR, McGinty A, Patterson CC & Ardill JE 2008 A rapid rise in circulating pancreastatin in response to somatostatin analogue therapy is associated with poor survival in patients with neuroendocrine tumours. Annals of Clinical Biochemistry 45 560–566. (doi:10.1258/acb.2008.008033).
Syversen U, Jacobsen MB, O'Connor DT, Ronning K & Waldum HL 1994 Immunoassays for measurement of chromogranin A and pancreastatin-like immunoreactivity in humans: correspondence in patients with neuroendocrine neoplasia. Neuropetides 26 201–206. (doi:10.1016/0143-4179(94)90131-7).
Takiyyuddin MA, Parmer RJ, Kailasam MT, Cervenka JH, Kennedy B, Ziegler MG, Lin MC, Li J, Grim CE & Wright FA et al. 1995 Chromogranin A in human hypertension. Influence of heredity. Hypertension 26 213–220. (doi:10.1161/01.HYP.26.1.213).
Taupenot L, Harper KL & O'Connor DT 2003 The chromogranin–secretogranin family. New England Journal of Medicine 348 1134–1149. (doi:10.1056/NEJMra021405).
Tohmola N, Itkonen O, Sane T, Markkanen H, Joenvaara S, Renkonen R & Hamalainen E 2013 Analytical and preanalytical validation of a new mass spectrometric serum 5-hydroxyindoleacetic acid assay as neuroendocrine tumor marker. Clinica Chimica Acta 428C 38–43.
Torres-Chavolla E & Alocilja EC 2009 Aptasensors for detection of microbial and viral pathogens. Biosensors and Bioelectronics 24 3175–3182. (doi:10.1016/j.bios.2008.11.010).
Turner GB, Johnston BT, McCance DR, McGinty A, Watson RG, Patterson CC & Ardill JE 2006 Circulating markers of prognosis and response to treatment in patients with midgut carcinoid tumours. Gut 55 1586–1591. (doi:10.1136/gut.2006.092320).
Udupi V, Lee HM, Kurosky A & Greeley GH Jr 1999 Prohormone convertase-1 is essential for conversion of chromogranin A to pancreastatin. Regulatory Peptides 83 123–127. (doi:10.1016/S0167-0115(99)00061-0).
Valicherla GR, Hossain Z, Mahata SK & Gayen JR 2013 Pancreastatin is an endogenous peptide that regulates glucose homeostasis. Physiological Genomics 45 1060–1071. (doi:10.1152/physiolgenomics.00131.2013).
Wolin EM 2012 The expanding role of somatostatin analogs in the management of neuroendocrine tumors. Gastrointestinal Cancer Research 5 161–168.
Woltering EA, Hilton RS, Zolfoghary CM, Thomson J, Zietz S, Go VL, Vinik AI, Vinik E, O'Dorisio TM & Mamikunian G 2006 Validation of serum versus plasma measurements of chromogranin A levels in patients with carcinoid tumors: lack of correlation between absolute chromogranin A levels and symptom frequency. Pancreas 33 250–254. (doi:10.1097/01.mpa.0000235302.73615.d4).
Wu JT, Erickson AJ, Tsao KC, Wu TL & Sun CF 2000 Elevated serum chromogranin A is detectable in patients with carcinomas at advanced disease stages. Annals of Clinical Laboratory Science 30 175–178.
Yao JC, Hassan M, Phan A, Dagohoy C, Leary C, Mares JE, Abdalla EK, Fleming JB, Vauthey JN & Rashid A et al. 2008 One hundred years after “carcinoid”: epidemiology of and prognostic factors for neuroendocrine tumors in 35,825 cases in the United States. Journal of Clinical Oncology 26 3063–3072. (doi:10.1200/JCO.2007.15.4377).
Yao JC, Pavel M, Phan AT, Kulke MH, Hoosen S, St Peter J, Cherfi A & Oberg KE 2011 Chromogranin A and neuron-specific enolase as prognostic markers in patients with advanced pNET treated with everolimus. Journal of Clinical Endocrinology and Metabolism 96 3741–3749. (doi:10.1210/jc.2011-0666).
Zembutsu H, Suzuki Y, Sasaki A, Tsunoda T, Okazaki M, Yoshimoto M, Hasegawa T, Hirata K & Nakamura Y 2009 Predicting response to docetaxel neoadjuvant chemotherapy for advanced breast cancers through genome-wide gene expression profiling. International Journal of Oncology 34 361–370.
