Abstract
Incidence of neuroendocrine neoplasia (NEN) is increasing, as is use of health-related quality of life (HRQoL) measurement in clinical trials. Following development of validated questionnaires, HRQoL is widely used to assess outcomes. This review is intended for healthcare professionals and is based on a selection of data published in the last decade. HRQoL is on par with other clinical endpoints such as performance status. Assessments in clinical trials have been particularly useful for monitoring the symptom burden of NEN, for the effects of treatments on patients’ lives, and have provided new data allied to the usual clinical endpoints. QoL expressed as quality-adjusted life years (QALYs) have become the most important primary outcome to establish cost-effectiveness in health economic evaluation. From looking at clinical trials over the last 10 years, we see that the quality of HRQoL evidence reported in published studies has improved and, in general, recent studies are likely to be more methodologically robust. Assessment of HRQoL in clinical trials is likely to become a standard part of clinical practice in NEN, as in other cancers. However, clear methods for calculating the clinical meaningfulness of changes in scores are needed. Other limitations of HRQoL measurement include lack of specificity to certain symptom sets and ease of completion and administration. An international group taking a lead on developing HRQoL research specifically in NEN patients is needed to address limitations of the evidence base. In order for greater weight to be placed on HRQoL data, agreement on optimal, validated scoring systems is needed.
Introduction
Neuroendocrine neoplasms (NEN) are a heterogeneous group of tumours thought to arise from pluripotential progenitor cells that develop neuroendocrine characteristics. Historically, gut-derived NEN were classified according to their embryological origin, into tumours of the foregut (bronchi, stomach, pancreas, gallbladder, and duodenum), midgut (jejunum, ileum, appendix, and right colon), and hindgut (left colon and rectum) (Ramage et al. 2012). In 2010, the WHO issued guidance on the classification of NENs according to histopathological characteristics (Lloyd et al. 2017). NENs are now usually classified according to primary location, grade, and hormone secretion. The term ‘NET’ (neuroendocrine tumour) is used for well-differentiated tumours and ‘NEC’ (neuroendocrine carcinoma) for poorly differentiated lesions.
NEN can be termed functioning in the presence of clinical symptoms of hormonal hypersecretion or non-functioning where symptoms, if present, may relate purely to tumour mass. In older texts, these tumours were frequently referred to as ‘carcinoid tumours’, but this term is no longer in active use. NEN are distinct from, but may cause, the carcinoid syndrome (CS), a rare clinical triad of flushing, diarrhoea, and abdominal pain specifically caused by serotonin and other mediators oversecreted by the tumour into the circulation. Approximately 20% of those diagnosed with NEN have CS (Halperin et al. 2017).
As in other cancers, NENs are stratified using mitotic index (Ki67), morphology with degree of differentiation, grade, and stage. They are best managed with a multidisciplinary approach. In general, low grade and well-differentiated tumours are managed with surgery, somatostatin analogues, intra-hepatic therapies, and radiolabelled somatostatin analogues as well as some types of chemotherapy or molecular targeted drugs. High-grade and poorly differentiated tumours are managed with systemic chemotherapy. Much of the literature described relates to lower grade NEN (now termed NET), but some have included higher grade NEN. Data on higher grade NEN are sparse.
Incidence of NEN is increasing. Latest research suggest an age standardized incidence of above 9 per 100,000 in a study using UK cancer registry data (T Genus, personal communication, (Genus et al. 2019)). Prevalence has reached 43 per 100,000, greater than any other upper gastrointestinal cancer, and NEN are present in around four times the number of people living with pancreatic cancer (https://www.cancerresearchuk.org/health-professional/cancer-statistics/statistics-by-cancer-type/pancreatic-cancer). The rise may be, in part, due to greater awareness of the disease and increased accuracy of diagnosis (Singh et al. 2017); however, a definite cause has not been found. Survival tends to be longer than in other types of tumour; G1 NETs have the highest median OS (16.2 years) among grade groups, G2 NETs had worse OS (8.3 years), and G3 and G4 NETs had the worst OS (10 months). NETs of the liver and then the stomach have the worst overall survival (Dasari et al. 2017).
Although there is a small familial risk, most NEN are sporadic. An increasing number of germline genetic mutations have been implicated in the development of certain types of NEN; in these situations, genetic screening can help to inform and counsel patients and their families (O’Shea & Druce 2017). Patients with MEN1 or MEN2 may experience issues relating to childhood diagnosis, familial risk, financial problems or anxiety pre- and post-diagnosis (Peipert et al. 2018). Some researchers have also looked at issues in MEN2, although the tumours in this condition are different to most NEN under discussion in this article (Grey & Winter 2018).
As with other disseminated malignancy, patients living with NEN experience chronic symptoms which exert a disease burden on them and their relatives. In the subgroup of patients with CS, fatigue, diarrhoea and flushing are most commonly experienced (Kaupp-Roberts et al. 2015).
Definition of quality of life (QoL)
Quality of life (QoL) is difficult to define and not easily generalizable. The term ‘eudaimonia’ was suggested by Aristotle (384–322 BC), but he recognised this can mean different things to different people (Fayers & Machin 2000). Dijkers suggests quality of life as ‘having one’s individual needs and desires fulfilled to a reasonable degree; a belief that life is offering or lacking the right balance of challenges and successes in those areas that are of personal salience; happiness and satisfaction that life is delivering all or most of what is expected or desired’ (Dijkers 2003). The terms QoL and health-related quality of life (HRQoL) are overlapping concepts, the latter being a subset of the former, but they are often used interchangeably (Fayers & Machin 2000). HRQoL is a multidimensional outcome measure for QoL issues that can be influenced by disease and treatment and is defined as an assessment of how the individual's well-being may be affected over time by a disease, disability, or disorder (Barcaccia et al. 2013).
HRQoL tools may be generic but some have been tailored for specific diseases. For NEN patients, aspects of HRQoL covered in the available tools include patient-self related domains (e.g. physical, psychosocial, and emotional) and specific illness factors with substantial detrimental impact such as pain, anaemia, fatigue, sleep problems, anxiety, depression, interference with family life, and treatment toxicities (discussed later). These domains relate closely to clinical factors such as NEN type, stage, grade, functionality, tumor growth rate, treatment availability, performance status, and comorbidities (Larsson et al. 2001, Yao et al. 2008, Beaumont et al. 2012).
Patient-reported outcome measures (PROM) are tools used to measure patient-reported outcomes (PRO). PROM can be classified as either generic or disease specific; HRQoL is a type of PROM. There is an upward trend of PROMs including HRQoL measurements being used in clinical trials and healthcare over recent years (Warsame & D’Souza 2019). The National Institutes of Health definitions of clinical endpoints in trials include survival, improvements in quality of life, relief of symptoms, and disappearance of the tumour (https://www.cancer.gov/publications/dictionaries/cancer-terms/def/endpoint). All other outcome measures including progression-free survival (PFS) are surrogates, and for this reason, specific reporting on improvements in quality of life has become more important as a measure of the success of an intervention.
HRQoL is seen as the patient’s assessment of the effect of illness and its treatment, enabling patients to feel their symptoms and psychological issues are being heard (Detmar et al. 2002). Symptom recording can be a benchmark of clinical care, particularly in patients with cancer. Patients’ symptoms can go undetected by health care providers as much as half the time, particularly between clinic visits (Pakhomov et al. 2008). There are of course limitations of HRQoL measurement, discussed later, but if we agree to try to optimise quality of life for patients we should do so in a methodical way.
Why is HRQoL important in NEN?
HRQoL measurement gives an objective assessment of well-being during disease and allows prioritisation of how a patient actually feels. It is important in managing cancer, because the process of measuring QoL increases the anticipation of patients’ changing needs and improves outcomes such as hospital visits and quality-adjusted survival and can improve communication between clinician and patient as shown in a randomised trial (Basch et al. 2016). QoL is often deprioritised in clinical trials; many therapies get routinely tested against length of life, but not against quality of life, so the priorities of the clinician and of the patient may not be aligned. The best standard of care could be seen as that which not only stabilizes or cures a cancer, but also provides the best quality of life to the patient.
In NEN there is a high prevalence of patients with a heavy symptom burden at a relatively young age, having profound effects on families, personal finances, and ability to work (Singh et al. 2017). Symptom burden is linked with quality of life (QoL); measuring and optimising this requires a specific tool so the patient can report how they feel in a way which is repeatable and comparable. Several questionnaire-based QoL or HRQoL instruments have been developed which ask the patient about their symptoms. HRQoL measures in cancer include global health status, functional constructs (physical, role, emotional, cognitive, and social), and symptoms generally relevant for patients with cancer (Aaronsen et al. 1993). QoL measures specific to each type of disease can also be used. NEN-specific HRQoL measures include endocrine, gastrointestinal, pain and treatment-related symptoms, disease-related worries, social functioning, weight loss or gain, problems receiving information, and sexual functioning (Yadegarfar et al. 2013).
HRQoL is seen as the patient’s assessment of the effect of illness and its treatment. There is evidence that care is improved by recording HRQoL, while following-up NEN patients long term, since patients feel their symptoms and psychological issues are being heard (Detmar et al. 2002). Symptom recording is a benchmark of clinical care, particularly in patients with cancer. Patients’ symptoms go undetected by health care providers as much as half the time, particularly between clinic visits (Pakhomov et al. 2008). There have been criticisms of HRQoL for not being seen as a hard clinical endpoint, however, if we agree to optimise quality of life for patients there should be a robust means of measuring it.
As with other chronic conditions, care of people with NEN can be costly and resource intensive. Using QoL as a primary or secondary endpoint in the assessment of the efficacy of therapies helps to provide evidence on efficacy of expensive novel therapies in trials which in turn helps to ensure efficient and effective resource utilisation.
HRQoL measurement and interpretation
HRQoL measurement methods
QoL can be measured with visual analogue scales and numerical scales. It is most commonly done using Likert scales, administered using paper entry, online or via an ‘app’ on a mobile phone or tablet. Paper entry mean scores have been shown to be equivalent to online data entry (Velikova et al. 2002). With respect to precision of HRQoL data, patients themselves are the most accurate record, following that are family member proxies, with healthcare workers being the least accurate proxy. Increasingly, patients are entering symptom and QoL data over the internet and through mobile phone applications. These apps can potentially empower patients, promoting behavioural changes and facilitating self-monitoring of symptoms. For people who like this, they establish a more convenient method for contacting healthcare professionals (Rincon et al. 2017).
Trials using mobile technology have suggested better recording of QoL and PROMS, which may contribute to improved survival in patients undergoing cancer therapy (Basch et al. 2016). Most of these have focused on QoL data collection in more common cancers, primarily breast and prostate (Min et al. 2014, Rincon et al. 2017). Generic cancer apps which collect standardised QoL data independent of the tumour type have limitations in conditions such as NEN, where certain hormone-related symptoms may impact significantly on quality of life and may not be recorded by a general tool. Recently, a UK NEN patient-centred app which also aids collection of HRQoL data has been developed for this reason (Bouvier et al. 2019). The instruments used for the collection of PROM data and patient functioning are given in Table 1.
Instruments for collection of PROM data and patient functioning.
| Instrument | Target or population | Reference |
|---|---|---|
| EQ-5D | Generic health | Euroqol.org |
| SF-36 | Generic health | (Brazier et al. 1992) |
| SF-12 | Generic health | (Jenkinson et al. 1997) |
| PROMIS-29 | Generic health | (Cella et al. 2010) |
| QLQ-C30 | Cancer | (Aaronson et al. 1993) |
| FACT-G | Cancer | (Heffernan et al. 2002) |
| GQLI | Gastrointestinal disorders | (Eypasch et al. 1995) |
| PAIS | Anxiety/depression | (Derogatis 1986) |
| HADS | Anxiety/depression | (Bjelland et al. 2002) |
| GHQ-12 | Anxiety/depression | (Goldberg & Blackwell 1970) |
| GINET21 | NEN specific | (Yadegarfar et al. 2013) |
| Norfolk | NEN Specific | (Vinik et al. 2009) |
| ECOG | Functional status | (Zubrod et al. 1960) |
| Karnovsky | Functional status | (Karnofsky et al. 1948) |
| Symptom scores | ||
| FACIT-fatigue | Fatigue | Facit.org |
| FACIT-D | Diarrhoea | Facit.org |
Cancer quality of life instruments
Measuring PROMs in cancer has an extensive background in the literature which is not reiterated here. There is debate as to whether the use of ‘modules’ using validated questionnaires is the best approach or whether validated questions can be picked from a bank of questions (e.g. The European Organisation for Research and Treatment of Cancer (EORTC) item bank) and used for specific cancers (www.groups.eortc/qol).
The EORTC QoL Group has published extensively on QoL and cancers, including guidelines on questionnaire development. The core questionnaire is the QLQ-C30 (Aaronson et al. 1993), a list of 30 validated questions covering five functional scales, three symptom scales, and a global health and quality-of-life scale to which modules are added for specific sites of tumour. The C30 and all modules use a four-point Likert scale from which numerical data is collected and converted into a score on a scale of 1–100. Details of the scoring algorithms have been published (www.groups.eortc/qol).
The FACIT group has developed the Functional Assessment of Cancer Therapy (FACT) system which has also been used extensively as a generic cancer instrument. There is an extensive literature relating to this (Facit.org).
NEN-specific quality of life instruments
The GINET-21 module (always used with the generic C30) was developed and validated internationally per EORTC guidelines (www.groups.eortc/qol) over many years in different languages (Yadegarfar et al. 2013). It required final validation in 253 patients with NEN. It has been used in 39 commercial trials, downloaded 313 times by academic users, and translated into 37 languages (EORTC-personal communication). Notably, the diarrhoea question is part of the C30 (described previously), not the GINET21.
In the future it may be useful to use Computer Adaptive Testing (CAT) for all HRQoL questionnaires in order to explore such single questions in more depth if a screening question proves positive. This would mean that if the patient scores positively on a screening question the algorithm triggers more detailed questions within this domain. Validation of CAT for the C30 is now complete (Petersen et al. 2018) and will be explored in the future for the GINET21. Use of CAT in this way would be useful to patients so that they are not overwhelmed with questions irrelevant to their own experience.
The Norfolk scale was developed in a similar way to the GINET (Vinik et al. 2009) and has been compared directly with it (Vinik et al. 2014) using factor analysis, which has indicated many similarities. The Norfolk QOL-NET questionnaire captured more aspects of flushing, respiratory, and cardiovascular impact. There is limited data on outcomes of therapies in gastroenteropancreatic NETs (GEP-NETs) using this instrument.
Some studies have used other instruments in NEN patients. The FACT-G, a generic cancer score, has been utilized. Beaumont et al. investigated the relationship between global QoL scores and severity of diarrhoea. They reported a clear deterioration in QoL for patients reporting more than three bowel movements per day, making the case for including a severity of diarrhoea symptom score (Beaumont et al. 2012). Recently Fact-G and Promis-29 were used before and after starting somatostatin receptor analogues (SSRA) and a subscale of questions specifically for CS was proposed as a way of obtaining more detailed information (Halperin et al. 2019).
A particular focus of QoL assessment in NEN has been on patients with CS, even though these are the minority of all patients with NEN. There are secretory syndromes other than CS that can be caused by NEN, for example, excess insulin causing seizures and behavioural disturbances and excess gastrin causing dyspepsia and gastrointestinal bleeding. A literature review has revealed a lack of specific QOL instruments that could capture the lived patient experience of these syndromes (Topping et al. 2017). Development of a separate questionnaire for pancreatic NEN and specific secretory syndromes has begun (Friend et al. 2020).
Specific psychometric evaluation of NEN QoL instruments has rarely been undertaken. Content validity, known groups comparisons, and/or response to change are normally required in order to validate an instrument for use in clinical trials or practice. The reason this has not been done in NEN is the large number of patients required for validation (>300) and relative rarity of the condition. Such evaluation does exist for the GINET21 and the Norfolk questionnaires (Vinik et al. 2009, Yadegarfar et al. 2013).
Thresholds for clinical significance
A threshold for clinical significance is a value for a question or scale at which a significant clinical impact on the patient is deemed to occur. To calculate, a threshold value (× out of 100) is determined for each scale of item of the questionnaire. This has been published for the C30 (Giesinger et al. 2016). Its main practical use is in the clinical collection of QoL data where ‘flag’ might appear in the patient’s record when a certain item meets the threshold, becoming ‘clinically significant’. This alerts the clinician to address the symptom significance with the patient in further detail.
Using QoL scores to assess change
QoL scores can be used to assess baseline status and compare between patient groups or patient types in the context of research. They can also be used to assess change in patient status either in a longitudinal setting or after an intervention. The magnitude of this change deemed to be of clinical relevance is termed minimal clinically important difference (MCID). In clinical trials, MCID is defined as the minimum clinically relevant difference in QoL that separates two groups of patients, for example, those in the treatment arm vs placebo. The MCID has been determined for the QLQ-C30 and varies between 5 and 10% age points after the mean score has been converted to a score of 1–100. To date, a MCID has not been determined for the GINET21 questions and its scales.
An important measure of the impact of a treatment is time to deterioration (TTD). This uses the MCID to determine important differences within the same patient group but observed over time, before and after an intervention. It is only when MCID is clearly documented for C30 and GINET21 in NEN patients that an accurate TTD for each scale and symptom can be correctly calculated. Time to deterioration has been used in recent NEN trials to compare two groups of patients, usually active therapy vs placebo or standard care. This is more complex than simply comparing mean scores of QoL instruments over time but overcomes the problem of ‘response shift’. This is the observation that patient groups, irrespective of intervention, have a tendency to show a change in QoL questionnaire response over time. Response shift is an expression of patients apparently becoming ‘used to’ issues related to their disease or therapy over time (Hagedoorn et al. 2002).
To calculate TTD, the MCID is decided prior to trial design and the time at which the scores of the patients in each group first falls below that MCID is calculated. A limitation of TDD is that, in some cases, the mean may increase back above baseline after the initial fall. For this reason, the variant of this – ‘time until definitive deterioration’ (TUDD) – has also been used. This is defined as deterioration below the MCID but without a rise again above baseline. There are plans to develop a ‘Response evaluation criteria in solid tumours (RECIST)’ approach to QoL data, meaning that standardised limits would be adopted for all domains of QoL, above which clinical significance would be concluded, similar to how radiological change is calculated (Anota et al. 2015).
Utility values and health economic evaluation
Quality of life expressed as quality-adjusted life years (QALYs) have become the most important primary outcome to establish cost-effectiveness in health economic evaluation (EuroQol Group 1990, Wiebe et al. 2003). They represent survival time corrected by the HRQoL experienced in that time and are anchored at 0 and 1 with ‘0’ representing death and ‘1’ representing 1 year in perfect health. One broadly applied method of obtaining QALYs is the use of a preference-based measure (PBM). These typically include a parsimonious set of generic health aspects including mental, physical, functional, and social domains applied in a variety of conditions. Their scoring differs from other HRQoL instruments presented up to this point because the health states described with the instrument are attached to utility values, that is, with the ‘value’ given to that specific health state by a sample of a target population (e.g. the general population of a country were the instrument is applied).
The most prominent and widely used generic PBM is the EuroQol-5D (EQ-5D) (EuroQol Group 1990). It is common practice to apply mapping algorithms to obtain EQ-5D utilities from non preference-based HRQOL instruments, such as the QLQ-C30. However, mapped utilities need to be treated with caution and direct ones are preferred (Crott 2018). There is ongoing discussion about whether generic PBMs are sufficiently relevant and sensitive for certain medical conditions (Wiebe et al. 2003, Krahn et al. 2007). This might be true also for NEN. Measuring utility in NEN is important in guiding decisions on patient management, for example, targeted therapy in disease progression. Although the EQ-5D has been used in some NEN trials, it has not been validated specifically in NEN patients or used to assess HRQoL losses from toxicity and adverse events related to targeted treatments. However, QALYs have been calculated for a number of NEN therapies assuming that such losses are the same as in other cancers (Mujica-Mota et al. 2018).
The recent development of a cancer-specific PBM based on the QLQ-C30, the QLQ-C10D (Quality of Life-Core 10 Dimensions), now allows the calculation of utilities from QLQ-C30 data (King et al. 2016). Value sets are available for a range of countries, such as the United Kingdom, Germany, Australia, and Canada (King et al. 2018, Kemmler et al. 2019, McTaggart-Cowan et al. 2019, Norman et al. 2019), and more are under development. Clinical validation is ongoing, including in NEN patients (personal communication with project PI).
The use of HRQoL scores in published data
HRQoL measured in NEN clinical trials
Use of QoL scoring systems in NEN trials was reviewed by Martini et al. in 2018. The CONSORT-PRO extension criteria were used to evaluate studies (Martini et al. 2018). Consort-PRO is a guideline listing criteria for the reporting of HRQoL in clinical trials. The recent publication from Setting International Standards in Analyzing Patient-Reported Outcomes and Quality of Life Endpoints Data Consortium (SISAQOL) lists criteria for planning methodology for clinical trials in cancer (Coens et al. 2020). In general, few studies adequately reported these criteria, although it is interesting to note the overall trend is of increasing adherence to guidelines on use of QoL in NEN trials. An adapted presentation of relevant NEN trials using HRQoL in the last 10 years using the same strategy from Martini 2018 is shown in Fig. 1.
Relevant NEN trials using HRQoL from the last 10 years. Minimum Standard Checklist for Evaluating HRQoL Outcomes in Clinical Trials (MSCEHOCT) including PRO-specific extensions (PSe) is important for studies using HRQoL as an endpoint in order to determine standards of these trials. Comparing trials using HRQoL evaluation as an endpoint can be done using the criteria in MSCEHOCT and PSe (Calvert et al. 2013) and was done for NEN in 2016 by Martini et al. (2018). We updated this review exercise with input from one of the study group (EMG). We collated relevant trials in GEP-NETs choosing a time period of 2009–2019 (including those previously used in Martini et al. (2018)) which included randomised controlled trials, phase two studies, and prospective studies (and excluding cross-sectional studies). Three reviewers (BEW, JKR, and EMG) reached consensus on assessment of the trials. +/−, criteria reported/not reported; N/A, not applicable. Design (type of trial): RCT, randomized controlled trial; Ph II, phase 2; OL, open label, Pros, prospective. Aim, HRQoL stated in the primary or secondary aim; Hypothesis, clear HRQoL-related hypothesis; Rationale, stated rationale for use of instrument; Validated, instrument’s validation or psychometric properties reported; Cancer-spec, cancer-specific instrument used; Admin, stated how the instrument was administered to patients; Compliance, baseline assessment performed; Timing, timing of assessments reported; Missing, missing data reported; Stats, statistical methods for missing data reported; Power, power or affect sizes stated; Presentation, presentation of results adequate (considered adequate if scoring has been performed correctly and if all assessed HRQoL domains were reported, including relevant P values); Clin Sig, clinical significance discussed; Results, HRQoL results presented and discussed.
Citation: Endocrine-Related Cancer 27, 7; 10.1530/ERC-20-0066
Relevant NEN trials using HRQoL from the last 10 years. Minimum Standard Checklist for Evaluating HRQoL Outcomes in Clinical Trials (MSCEHOCT) including PRO-specific extensions (PSe) is important for studies using HRQoL as an endpoint in order to determine standards of these trials. Comparing trials using HRQoL evaluation as an endpoint can be done using the criteria in MSCEHOCT and PSe (Calvert et al. 2013) and was done for NEN in 2016 by Martini et al. (2018). We updated this review exercise with input from one of the study group (EMG). We collated relevant trials in GEP-NETs choosing a time period of 2009–2019 (including those previously used in Martini et al. (2018)) which included randomised controlled trials, phase two studies, and prospective studies (and excluding cross-sectional studies). Three reviewers (BEW, JKR, and EMG) reached consensus on assessment of the trials. +/−, criteria reported/not reported; N/A, not applicable. Design (type of trial): RCT, randomized controlled trial; Ph II, phase 2; OL, open label, Pros, prospective. Aim, HRQoL stated in the primary or secondary aim; Hypothesis, clear HRQoL-related hypothesis; Rationale, stated rationale for use of instrument; Validated, instrument’s validation or psychometric properties reported; Cancer-spec, cancer-specific instrument used; Admin, stated how the instrument was administered to patients; Compliance, baseline assessment performed; Timing, timing of assessments reported; Missing, missing data reported; Stats, statistical methods for missing data reported; Power, power or affect sizes stated; Presentation, presentation of results adequate (considered adequate if scoring has been performed correctly and if all assessed HRQoL domains were reported, including relevant P values); Clin Sig, clinical significance discussed; Results, HRQoL results presented and discussed.
Citation: Endocrine-Related Cancer 27, 7; 10.1530/ERC-20-0066
Relevant NEN trials using HRQoL from the last 10 years. Minimum Standard Checklist for Evaluating HRQoL Outcomes in Clinical Trials (MSCEHOCT) including PRO-specific extensions (PSe) is important for studies using HRQoL as an endpoint in order to determine standards of these trials. Comparing trials using HRQoL evaluation as an endpoint can be done using the criteria in MSCEHOCT and PSe (Calvert et al. 2013) and was done for NEN in 2016 by Martini et al. (2018). We updated this review exercise with input from one of the study group (EMG). We collated relevant trials in GEP-NETs choosing a time period of 2009–2019 (including those previously used in Martini et al. (2018)) which included randomised controlled trials, phase two studies, and prospective studies (and excluding cross-sectional studies). Three reviewers (BEW, JKR, and EMG) reached consensus on assessment of the trials. +/−, criteria reported/not reported; N/A, not applicable. Design (type of trial): RCT, randomized controlled trial; Ph II, phase 2; OL, open label, Pros, prospective. Aim, HRQoL stated in the primary or secondary aim; Hypothesis, clear HRQoL-related hypothesis; Rationale, stated rationale for use of instrument; Validated, instrument’s validation or psychometric properties reported; Cancer-spec, cancer-specific instrument used; Admin, stated how the instrument was administered to patients; Compliance, baseline assessment performed; Timing, timing of assessments reported; Missing, missing data reported; Stats, statistical methods for missing data reported; Power, power or affect sizes stated; Presentation, presentation of results adequate (considered adequate if scoring has been performed correctly and if all assessed HRQoL domains were reported, including relevant P values); Clin Sig, clinical significance discussed; Results, HRQoL results presented and discussed.
Citation: Endocrine-Related Cancer 27, 7; 10.1530/ERC-20-0066
Cross-sectional studies of QoL, anxiety and depression in NEN
Cross-sectional studies of QoL in NEN vs the general population have shown reasonably good overall QoL compared to other cancers. This may explain why changes are hard to detect in some patients (Fröjd et al. 2007). Depression and anxiety scores in NEN are similar to the background population and unrelated to levels of 5HIAA in CS (Lewis et al. 2018). Depression can occur related to the degree of anaemia if alpha interferon is used for therapy and QoL can be improved using erythropoietin in this situation (Larsson & Janson 2008). In CS, high levels of serotonin production might cause tryptophan depletion in the brain, and qualitative exploration of psychological disease has been explored in NEN patients (Friend 2016). Several patients had severe anxiety and in some cases psychotic issues, but it was unclear if this was specifically due to NEN or to hormone secretion.
Cognitive impairment has been documented in patients with NEN (Pasieka et al. 2014). This was significantly worse than a matched population of patients with hepatobiliary cancers. Diarrhoea is known to affect QoL and has been examined in a large cross-sectional study of NEN patients (Beaumont et al. 2012). A retrospective analysis of 600 patients inclusive of all types of NEN showed that fatigue, loss of well-being, lack of appetite, and drowsiness were the most prominent symptoms towards the end of life (Hallet et al. 2019). Fatigue is a prominent symptom in all QoL studies in NET, is greater than the background population, and can be the most common symptom of all (Fröjd et al. 2007).
HRQoL measured in NEN clinical trials
Treatments with the best efficacy in terms of PFS and symptom control in NEN are somatostatin analogues, peptide receptor radionuclide therapy (PRRT; radiolabelled somatostatin analogues which cause cell death), everolimus, sunitinib, telotristat, chemotherapy, surgery, and liver directed therapies. Measurements of HRQoL in trials using these therapies are discussed subsequently.
1) Somatostatin analogues (SSA)
SSA are the first line and cornerstone of therapy for metastatic somatostatin receptor positive NEN, possessing both anti-secretory and anti-proliferative effects. They are the least toxic drugs and time-tested by years of daily practice to be useful for restoration of HRQoL in NEN.
Octreotide was approved for treating CS in 1988. It was shown to improve PFS in midgut NENs with small volume liver metastases in the PROMID trial (Rinke et al. 2009). Retrospective analysis of HRQoL showed improved time to deterioration (TTD) for fatigue, pain, and insomnia in the active arm compared to placebo (Rinke et al. 2019).
Lanreotide, approved in 2014, improves symptoms (Ruszniewski et al. 2016) and PFS in advanced or metastatic well-differentiated GEP-NETs. The CLARINET study (Caplin et al. 2014) demonstrated improved PFS, but QoL measured by C30 and GINET21 showed no significant difference (Ruszniewski et al. 2014). This could have been because many patients had few symptoms and non-progressive disease at trial entry. Utility values were mapped showing reduced utility in progressive disease using EQ-5D (Meng et al. 2017).
Few studies have addressed QoL with reference to tumour burden or secretory status. Halperin et al. looked at the long-term use of SSA on QoL in patients with CS with a survey study including 117 patients (FACT-G and PROMIS-29). They observed that CS burden was associated with lower QoL scores and suggested that SSA treatment duration of more than 8 years treatment vs <2.7 years may be associated with higher QoL (Halperin et al. 2018). However, this was a cross-sectional study and the two populations being compared may not have been equivalent.
2) Peptide receptor radionuclide therapy (PRRT)
Peptide receptor radionuclide therapy is a form of molecular targeted therapy which is performed by coupling a somatostatin analogue with a radionuclide emitting beta radiation. It is effective at improving symptoms, controlling tumour growth, and improving survival (Hörsch et al. 2016, Strosberg et al. 2017). PRRT is generally well tolerated, and patient outcome reports suggest favourable HRQoL outcomes (Marinova et al. 2019).
The most important prospective study of QoL using PRRT is the NETTER-1 study (Strosberg et al. 2018) in which 229 patients were randomised to PRRT vs high dose sandostatin LAR (60 mg every 28 days). Time to Deterioration (by 10 points out of 100) was significantly longer in the PRRT group in global health status, physical functioning, role functioning, fatigue, pain, diarrhoea, disease-related worries, and body image. Changes in median TTD between the two groups were clinically significant in several domains: 28.8 months vs 6.1 months for global health status and 25.2 months vs 11.5 months for physical functioning. Actual means of the QoL scores at different times were not stated and so the magnitude of mean differences at any one time point was not published as yet. Time until definitive deterioration (TUDD, definitive deterioration being a deterioration that is maintained and does not go back above baseline) was also stated and significant differences demonstrated. The changes in TTD were greater than for any other prospective study published in NEN.
In a meta-analysis of HRQoL assessment in patients undergoing PRRT for GEP-NETs (Jiménez-Fonseca et al. 2015), all 13 studies showed significant improvement in HRQoL irrespective of the radionuclide (90Y or 177Lu) used, highlighting that PRRT improves symptoms, performance status, and delays disease progression.
Non-randomised studies (Khan et al. 2011, Marinova et al. 2018) used the C30 following PRRT and showed an improvement in QoL scores compared to baseline. In a study where 144 patients were assessed at 6 years since the last PRRT, more than 50% reported improvement in pain, diarrhoea, flushing, and fatigue (Hamiditabar et al. 2017).
3) Everolimus
Everolimus (RAD001; Afinitor, Novartis International AG) is a derivative of rapamycin which selectively inhibits mTOR (mammalian target of rapamycin) complex 1, decreasing cell growth, proliferation, and survival.
The early studies of the use of everolimus in different settings were termed the ‘RADIANT’ studies (comparing everolimus to placebo in pancreatic and small bowel NEN). RADIANT-2 and -3 had no QoL measurements, but in the latest iteration, RADIANT-4 (gastrointestinal and lung NEN), FACT-G was used. The results were negative, showing no real difference between active and placebo groups, although FACT-G is not specific to NEN (Pavel et al. 2017). In view of the lack of specific HRQoL measures in trials using everolimus, the OBLIQUE (Observational Study to Assess Quality of Life in Patients with Pancreatic Neuroendocrine Tumors Receiving Treatment with Everolimus) study was performed in the United Kingdom (Ramage et al. 2019), using the C30 and GINET21. The main finding was improved physical functioning at 3 months after starting therapy. Other domains demonstrated maintained QoL over 6 months.
4) Sunitinib
Sunitinib (Sutent; Pfizer Inc.) is a multiple receptor tyrosine kinase (RTKs) inhibitor, acting on platelet-derived growth factor (PDGF-Rs) and vascular endothelial growth factor receptors (VEGFRs), which play a role in both tumour angiogenesis and tumour cell proliferation.
In a phase 2, non-randomised study in 107 pancreatic NEN patients, QoL assessed using EQ-5D & FACIT-Fatigue was preserved on sunitinib (Kulke et al. 2008). A phase 3 study (pancreatic neuroendocrine tumours (pNETs); 37.5 mg/day, continuously) showed improved PFS, OS, and response rate, with no overall difference in global HRQoL, cognitive, emotional, physical, or social functioning (EORTC QLQ-C30 + GINET21). The exception was diarrhoea which worsened on sunitinib (Raymond et al. 2011).
5) Telotristat
Telotristat ethyl (Xermelo, Lexicon Pharmaceuticals, Inc., United States) is an oral inhibitor of tryptophan hydroxylase, the step-limiting enzyme in the synthesis of serotonin. Telotristat is used in combination with SSA therapy for the treatment of adults with diarrhoea associated with gastrointestinal NEN and CS that SSA therapy alone has inadequately controlled.
The relationship between symptoms and HRQoL in patients with CS was studied in the TELESTAR study and overall there was no difference between drug and placebo arms. However, in a post-hoc analysis from the TELESTAR study, the HRQoL scores (EORTC QLQ-C30 and QLQ-GINET21) were better in responders vs non-responders (Cella et al. 2018).
Looking at the changes in weight and telotristat ethyl in 120/135 patients from the TELESTAR study, in 32.5% patients significant, dose-dependent weight gain from a low baseline was seen, in parallel with reduced diarrhoea (Weickert et al. 2018).
6) Chemotherapy trials
Chemotherapy was assessed in the BETTER phase 2 non-randomised study. Patients who were chemotherapy-naive with progressive, metastatic, and well-differentiated GEP-NETs were given bevacizumab and capecitabine for up to 24 months. Global QoL was not changed over 3 monthly assessments during the first year. This was the case in both pancreatic and GI NETs across different chemotherapy regimens and using the QLQ-C30 (Ducreux et al. 2014, Mitry et al. 2014). In the NET01 study in patients with advanced, unresectable GEP-NETs, comparing capecitabine with streptozocin in one arm and with the addition of cisplatin in the other, global health scores were similar after six cycles of therapy, using the C30 and GINET-21 (Meyer et al. 2014). There were some missing data and numbers were small in all of the chemotherapy studies; hence, conclusions on this data should be interpreted with caution.
7) Surgery and liver directed therapy trials
Despite there being more than 18 series published since 2003 related to survival after resection of liver metastases, remarkably few studies have assessed QoL as part of follow-up. Spolverato studied QoL in patients having various therapies for NEN liver metastases. No difference was seen in QoL scores between surgical and non-surgical groups (Spolverato et al. 2015). Although arterial embolization and ablation of NEN liver metastases are said to give improvement in hormone symptoms, we could find no evidence of systematic measures of QoL being performed after these therapies.
Limitations of HRQoL measurement and data
Limitation of measuring HRQoL include reliance on patients’ comprehension and writing ability, ability to read and write in English, and lack of validation in diverse populations. To solve this problem, some scoring systems have been adapted to fit other languages and cultures, but this may be difficult in NEN in view of their relative rarity and small numbers recruited in each country and culture.
Technical problems of data entry include ‘questionnaire fatigue’ due to too many questions, recall bias (patients remembering of the most recent symptoms most clearly), and missing data. These limitations are bridged by ensuring validated systems are used wherever possible and using the ideal method of administration for each patient, which may be as an App on a mobile device. Good communication and accessibility are essential for adherence to timing and methods.
We include recent data using different study designs, with heterogeneous group of populations, and a variety of QoL measures. All studies aimed to assess the effects of disease-related symptoms, treatment-associated adverse events, and psychosocial factors on patient HRQoL. One of the main limitations for studies. (doing this is the possible effects of concomitant comorbid conditions are not well acknowledged, in parallel with a lack of standardized instruments allowing proper comparisons across studies. Moreover, small sample sizes, limited cross-sectional measures, lack of gender balance, lack of information on handling missing data and on the protocols used for data collection are well-known drawbacks.
High-quality studies designed specifically for sub-groups of NEN, aiming to include a more homogeneous patient population in terms of tumour biology, specific tumour related symptoms, treatment availability or disease-related outcomes, and using NET-specific HRQoL instruments, are needed for more comprehensive assessment of findings. Moreover, investigating the impact of supportive measures as well as patient caregiver-related measures in longitudinal studies may provide additional and meaningful information on the assessment of QoL during patient's life-time with the disease.
NEN are heterogeneous and therefore some HRQoL instruments may lack specificity for subtypes. For example, in pancreatic NEN with various secreted hormones, the symptom and QoL outcomes will be similarly varied. Whether a ‘module’ can be constructed to deal with all of these or whether an approach selecting individual questions for different syndromes is needed remains a topic of debate. Some early work has been done on this (Topping et al. 2017).
Conclusions
Health-related quality of life is an important clinical endpoint in itself; it supplements and adds nuance to existing data on mortality, survival, and side effects. It also provides information on the outcomes that are of key importance to the patient. It is important for assessment of individual patient treatments and for between-group comparisons in clinical trials.
Methods and results of studies assessing HRQoL in NEN are heterogeneous and non-standardised. The disease itself, while increasing in incidence, is still uncommon, variegated, and challenging to study, lacking international collaborative trials.
To facilitate better administration in HRQoL instruments, novel ways of entering data are needed; apps usable on mobile devices may lead to significant improvement. It is possible that pictorial or video prompts included with these apps may facilitate the recording of patients’ symptoms and feelings. It is clear that MCID and thresholds for clinical importance need to be calculated for NEN-specific instruments. In addition, a computer-adapted testing model (already in use for the C30) should be developed for the GINET 21 and other questionnaires. This would enable patients with one domain of severe symptoms, for example, diarrhoea, to have this area explored in more depth without increasing the number of overall questions significantly.
Adherence to guidelines on use of HRQoL in clinical trials appears to be improving gradually but remains below the ideal standards outlined by the CONSORT criteria. An international group taking a lead on developing HRQoL research specifically in NET patients would lead to improvement in this area. To deal with many of the limitations caused by quality of results, a standardised approach including the CONSORT-PRO guidelines and the SISIQUAL group suggestions as mentioned previously should be used in the design of prospective trials in NEN. In order for greater weight to be placed on QoL data, agreement on optimal, validated scoring systems is needed.
Declaration of interest
Maralyn Druce received advisory board fees from Ipsen. Simona Grozinsky-Glasberg received advisory board fees from Novartis, Ipsen, Triple A, Pfizer, and Lexicon and research grants from Novartis and Lexicon. John K Ramage received research funding and speakers fees from Ipsen, Novartis, and Advanced Accelerators Ltd. The other authors have nothing to disclose.
Funding
This work did not receive any specific grant from any funding agency in the public, commercial, or not-for-profit sector.
References
Aaronson NK, Ahmedzai S, Bergman B, Bullinger M, Cull A, Duez NJ, Filiberti A, Flechtner H, Fleishman SB & Haes JCJMd 1993 The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. Journal of the National Cancer Institute 85 365–376. (https://doi.org/10.1093/jnci/85.5.365)
Anota A, Hamidou Z, Paget-Bailly S, Chibaudel B, Bascoul-Mollevi C, Auquier P, Westeel V, Fiteni F, Borg C & Bonnetain F 2015 Time to health-related quality of life score deterioration as a modality of longitudinal analysis for health-related quality of life studies in oncology: do we need RECIST for quality of life to achieve standardization? Quality of Life Research 24 5–18. (https://doi.org/10.1007/s11136-013-0583-6)
Ballal S, Yadav MP, Bal C, Sahoo RK & Tripathi M 2020 Broadening horizons with 225Ac-DOTATATE targeted alpha therapy for gastroenteropancreatic neuroendocrine tumour patients stable or refractory to 177Lu-DOTATATE PRRT: first clinical experience on the efficacy and safety. European Journal of Nuclear Medicine and Molecular Imaging 47 934–946. (https://doi.org/10.1007/s00259-019-04567-2)
Barcaccia B, Esposito G, Matarese M, Bertolaso M, Elvira M & De Marinis MG 2013 Defining quality of life: a wild-goose chase? Europe’s Journal of Psychology 9 185–203. (https://doi.org/10.5964/ejop.v9i1.484)
Basch E, Deal AM, Kris MG, Scher HI, Hudis CA, Sabbatini P, Rogak L, Bennett AV, Dueck AC, Atkinson TM, et al.2016 Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial. Journal of Clinical Oncology 34 557–565. (https://doi.org/10.1200/JCO.2015.63.0830)
Beaumont JL, Cella D, Phan AT, Choi S, Liu Z & Yao JC 2012 Comparison of health-related quality of life in patients with neuroendocrine tumors with quality of life in the general US population. Pancreas 41 461–466. (https://doi.org/10.1097/MPA.0b013e3182328045)
Bjelland I, Dahl AA, Haug TT & Neckelmann D 2002 The validity of the hospital anxiety and depression scale. An updated literature review. Journal of Psychosomatic Research 52 69–77. (https://doi.org/10.1016/s0022-3999(01)00296-3)
Bodei L, Cremonesi M, Grana CM, Fazio N, Iodice S, Baio SM & Paganelli G 2011 Peptide receptor radionuclide therapy with 177Lu-DOTATATE: the IEO phase I-II study. European Journal of Nuclear Medicine and Molecular Imaging 38 2125–2135. (https://doi.org/10.1007/s00259-011-1902-1)
Bouvier C, Ramage J, Newell-Price J, Virk J, Allan R, Verey P, Alaghband N & Srirajaskanthan R 2019 Development of a mobile app for patients with neuroendocrine neoplasms: a collaborative project between United Kingdom NET Society and Neuroendocrine Tumour Patient Foundation. Endocrine Abstracts 68 P5. (https://doi.org/10.1530/endoabs.68.P5)
Brazier JE, Harper R, Jones NMB, O’Cathain A, Thomas KJ, Usherwood T & Westlake L 1992 Validating the SF-36 health survey questionnaire: new outcome measure for primary care. British Medical Journal 305 160–164. (https://doi.org/10.1136/bmj.305.6846.160)
Bushnell DL, O’Dorisio TM, O’Dorisio MS, Menda Y, Hicks RJ, Van Cutsem E & Pauwels SA 2010 90Y-Edotreotide for metastatic carcinoid refractory to octreotide. Journal of Clinical Oncology 28 1652– 1659. (https://doi.org/10.1200/JCO.2009.22.8585)
Calvert M, Brundage M, Jacobsen PB, Schünemann HJ & Efficace F 2013 The CONSORT Patient-Reported Outcome (PRO) extension: implications for clinical trials and practice. Health and Quality of Life Outcomes 11 184. (https://doi.org/10.1186/1477-7525-11-184)
Caplin ME, Pavel M, Ćwikła JB, Phan AT, Raderer M, Sedláčková E, Cadiot G, Wolin EM, Capdevila J, Wall L, et al.2014 Lanreotide in metastatic enteropancreatic neuroendocrine tumors. New England Journal of Medicine 371 224–233. (https://doi.org/10.1056/NEJMoa1316158)
Cella D, Riley W, Stone A, Rothrock N, Reeve B, Yount S, Amtmann D, Bode R, Buysse D, Choi S, et al. 2010 The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. Journal of Clinical Epidemiology 63 1179–1194. (https://doi.org/10.1016/j.jclinepi.2010.04.011)
Cella D, Beaumont JL, Hudgens S, Marteau F, Feuilly M, Houchard A, Lapuerta P, Ramage J, Pavel M, Hörsch D, et al.2018 Relationship between symptoms and health-related quality-of-life benefits in patients with carcinoid syndrome: post hoc analyses from TELESTAR. Clinical Therapeutics 40 2006.e2–2020.e2. (https://doi.org/10.1016/j.clinthera.2018.10.008)
Claringbold PG, Brayshaw PA, Price RA & Turner JH 2011 Phase II study of radiopeptide 177Lu-octreotate and capecitabine therapy of progressive disseminated neuroendocrine tumours. European Journal of Nuclear Medicine and Molecular Imaging 38 302–311. (https://doi.org/10.1007/s00259-010-1631-x)
Coens C, Pe M, Dueck AC, Sloan J, Basch E, Calvert M, Campbell A, Cleeland C, Cocks K, Collette L, et al.2020 International standards for the analysis of quality-of-life and patient-reported outcome endpoints in cancer randomised controlled trials: recommendations of the SISAQOL Consortium. Lancet: Oncology 21 e83–e96. (https://doi.org/10.1016/S1470-2045(19)30790-9)
Crott R 2018 Direct mapping of the QLQ-C30 to EQ-5D preferences: a comparison of regression methods. PharmacoEconomics Open 2 165–177. (https://doi.org/10.1007/s41669-017-0049-9)
Cwikla JB, Sankowski A, Seklecka N, Buscombe JR, Nasierowska-Guttmejer A, Jeziorski KG & Walecki J 2010 Efficacy of radionuclide treatment DOTATATE Y-90 in patients with progressive metastatic gastroenteropancreatic neuroendocrine carcinomas (GEP-NETs): a phase II study. Annals of Oncology 21 787–794. (https://doi.org/10.1093/annonc/mdp372)
Dasari A, Shen C, Halperin D, Zhao B, Zhou S, Xu Y, Shih T & Yao JC 2017 Trends in the incidence, prevalence, and survival outcomes in patients with neuroendocrine tumors in the United States. JAMA Oncology 3 1335–1342. (https://doi.org/10.1001/jamaoncol.2017.0589)
Delpassand ES, Samarghandi A, Zamanian S, Wolin EM, Hamiditabar M, Espenan GD & Mojtahedi A 2014 Peptide receptor radionuclide therapy with 177Lu-DOTATATE for patients with somatostatin receptor–expressing neuroendocrine tumors. Pancreas 43 518–525. (https://doi.org/10.1097/MPA.0000000000000113)
Derogatis LR 1986 The psychosocial adjustment to illness scale (PAIS). Journal of Psychosomatic Research 30 77–91. (https://doi.org/10.1016/0022-3999(86)90069-3)
Detmar SB, Muller MJ, Schornagel JH, Wever LDV & Aaronson NK 2002 Health-related quality-of-life assessments and patient-physician communication: a randomized controlled trial. JAMA 288 3027–3034. (https://doi.org/10.1001/jama.288.23.3027)
Dijkers MP 2003 Individualization in quality of life measurement: instruments and approaches. Archives of Physical Medicine and Rehabilitation 84 (Supplement 2) S3–S14. (https://doi.org/10.1053/apmr.2003.50241)
Ducreux M, Dahan L, Smith D, O’Toole D, Lepère C, Dromain C, Vilgrain V, Baudin E, Lombard-Bohas C, Scoazec JY, et al.2014 Bevacizumab combined with 5-FU/streptozocin in patients with progressive metastatic well-differentiated pancreatic endocrine tumours (BETTER trial) – a phase II non-randomised trial. European Journal of Cancer 50 3098–3106. (https://doi.org/10.1016/j.ejca.2014.10.002)
EuroQol Group 1990 EuroQol – a new facility for the measurement of health-related quality of life. Health Policy 16 199–208. (https://doi.org/10.1016/0168-8510(90)90421-9)
Eypasch E, Williams JI, Wood-Dauphinee S, Ure BM, Schmülling C, Neugebauer E & Troidl H 1995 Gastrointestinal quality of life index: development, validation and application of a new instrument. British Journal of Surgery 82 216–222. (https://doi.org/10.1002/bjs.1800820229)
Fayers PM & Machin D 2000 Quality of Life. Chichester, UK: John Wiley & Sons, Ltd. (https://doi.org/10.1002/0470846283)
Friend L 2016 An exploration of psychological symptoms in patients with vasoactive hormone-secreting neuroendocrine tumours (carcinoid syndrome). Endocrine Abstracts 46 NETS10. (https://doi.org/10.1530/endoabs.46.NETS10)
Friend E, Gray D, Fernández Ortega P, McNamara M, Kaltsas G, Falconi M, Ćwikła J, Capdevila J, Glasberg S, Mandair D, et al.2020 Development of an EORTC quality of life questionnaire for patients with pancreatic neuroendocrine tumours: phases 1-3. Neuroendocrinology 110 (Suppl) 303. (https://doi.org/10.1159/000506496)
Fröjd C, Larsson G, Lampic C & von Essen L 2007 Health related quality of life and psychosocial function among patients with carcinoid tumours. A longitudinal, prospective, and comparative study. Health and Quality of Life Outcomes 5 18. (https://doi.org/10.1186/1477-7525-5-18)
Fröjd C, Lampic C, Larsson G & Von Essen L 2009 Is satisfaction with doctors’ care related to health- related quality of life, anxiety and depression among patients with carcinoid tumours? A longitudinal report. Scandinavian Journal of Caring Sciences 23 107–116. (https://doi.org/10.1111/j.1471-6712.2008.00596.x)
Genus TSE, Bouvier C, Wong KF, Srirajaskanthan R, Rous BA, Talbot DC, Valle JW, Khan M, Pearce N, Elshafie M, et al.2019 Impact of neuroendocrine morphology on cancer outcomes and stage at diagnosis: a UK nationwide cohort study 2013–2015. British Journal of Cancer 121 966–972. (https://doi.org/10.1038/s41416-019-0606-3)
Giesinger JM, Kuijpers W, Young T, Tomaszewski KA, Friend E, Zabernigg A, Holzner B & Aaronson NK 2016 Thresholds for clinical importance for four key domains of the EORTC QLQ-C30: physical functioning, emotional functioning, fatigue and pain. Health and Quality of Life Outcomes 14 87. (https://doi.org/10.1186/s12955-016-0489-4)
Goldberg DP & Blackwell B 1970 Psychiatric Illness in general practice: a detailed study using a new method of case identification. British Medical Journal 2 439–443. (https://doi.org/10.1136/bmj.2.5707.439)
Grey J & Winter K 2018 Patient quality of life and prognosis in multiple endocrine neoplasia type 2. Endocrine-Related Cancer 25 T69–T77. (https://doi.org/10.1530/ERC-17-0335)
Hagedoorn M, Sneeuw KCA & Aaronson NK 2002 Changes in physical functioning and quality of life in patients with cancer: response shift and relative evaluation of one’s condition. Journal of Clinical Epidemiology 55 176–183. (https://doi.org/10.1016/S0895-4356(01)00438-3)
Hallet J, Davis LE, Mahar AL, Isenberg-Grzeda E, Bubis LD, Myrehaug S, Zhao H, Beyfuss K, Moody L, Law CHL, et al.2019 Symptom burden at the end of life for neuroendocrine tumors: an analysis of 2579 prospectively collected patient-reported outcomes. Annals of Surgical Oncology 26 2711–2721. (https://doi.org/10.1245/s10434-019-07441-5)
Halperin DM, Shen C, Dasari A, Xu Y, Chu Y, Zhou S, Shih Y-CT & Yao JC 2017 Frequency of carcinoid syndrome at neuroendocrine tumour diagnosis: a population-based study. Lancet: Oncology 18 525–534. (https://doi.org/10.1016/S1470-2045(17)30110-9)
Halperin DM, Huynh L, Beaumont JL, Cai B, Totev T, Bhak RH, Duh MS, Neary MP & Cella D 2018 Impact of carcinoid syndrome symptoms and long-term use of somatostatin analogs on quality of life in patients with carcinoid syndrome: a survey study. Medicine 97 e13390. (https://doi.org/10.1097/MD.0000000000013390)
Halperin DM, Huynh L, Beaumont JL, Cai B, Bhak RH, Narkhede S, Totev T, Duh MS, Neary MP & Cella D 2019 Assessment of change in quality of life, carcinoid syndrome symptoms and healthcare resource utilization in patients with carcinoid syndrome. BMC Cancer 19 1–9. (https://doi.org/10.1186/s12885-019-5459-x)
Hamiditabar M, Ali M, Roys J, Wolin EM, O’Dorisio TM, Ranganathan D, Tworowska I, Strosberg JR & Delpassand ES 2017 Peptide receptor radionuclide therapy with 177Lu-octreotate in patients with somatostatin receptor expressing neuroendocrine tumors: six years’ assessment. Clinical Nuclear Medicine 42 436–443. (https://doi.org/10.1097/RLU.0000000000001629)
Haugland T, Veenstra M, Vatn MH & Wahl AK 2013 Improvement in stress, general self-efficacy, and health related quality of life following patient education for patients with neuroendocrine tumors: a pilot study. Nursing Research and Practice 2013 695820. (https://doi.org/10.1155/2013/695820)
Heffernan N, Cella D, Webster K, Odom L, Martone M, Passik S, Bookbinder M, Fong Y, Jarnagin W & Blumgart L 2002 Measuring health-related quality of life in patients with hepatobiliary cancers: the functional assessment of cancer therapy-hepatobiliary questionnaire. Journal of Clinical Oncology 20 2229–2239.
Hörsch D, Ezziddin S, Haug A, Gratz KF, Dunkelmann S, Miederer M, Schreckenberger M, Krause BJ, Bengel FM, Bartenstein P, et al.2016 Effectiveness and side-effects of peptide receptor radionuclide therapy for neuroendocrine neoplasms in Germany: a multi-institutional registry study with prospective follow-up. European Journal of Cancer 58 41–51. (https://doi.org/10.1016/j.ejca.2016.01.009)
Jenkinson C, Layte R, Jenkinson D, Lawrence K, Petersen S, Paice C & Stradling J 1997 A shorter form health survey: can the sf-12 replicate results from the sf-36 in longitudinal studies? Journal of Public Health 19 179–186. (https://doi.org/10.1093/oxfordjournals.pubmed.a024606)
Jiménez-Fonseca P, Carmona-Bayonas A, Martín-Pérez E, Crespo G, Serrano R, Llanos M, Villabona C, García-Carbonero R, Aller J, Capdevila J, et al.2015 Health-related quality of life in well-differentiated metastatic gastroenteropancreatic neuroendocrine tumors. Cancer Metastasis Reviews 34 381–400. (https://doi.org/10.1007/s10555-015-9573-1)
Kalinowski M, Dressler M, König A, El-Sheik M, Rinke A, Höffken H & Wagner H-J 2009 Selective internal radiotherapy with Yttrium-90 microspheres for hepatic metastatic neuroendocrine tumors: a prospective single center study. Digestion 79 137–142. (https://doi.org/10.1159/000209849)
Karnofsky DA, Abelmann WH, Craver LF & Burchenal JH 1948 The use of the nitrogen mustards in the palliative treatment of carcinoma with particular reference to bronchogenic carcinoma. Cancer 1 634– 656. (https://doi.org/10.1002/1097-0142(194811)1:4<634::AID-CNCR2820010410>3.0.CO;2-L)
Kaupp-Roberts S, Srirajaskanthan R & Ramage JK 2015 Symptoms and quality of life in gastroenteropancreatic neuroendocrine tumours. EMC Oncology334–40.
Kemmler G, Gamper E, Nerich V, Norman R, Viney R, Holzner B, King M & European Organisation for Research and Treatment of Cancer (EORTC) Quality of Life Group 2019 German value sets for the EORTC QLU-C10D, a cancer-specific utility instrument based on the EORTC QLQ-C30. Quality of Life Research 28 3197–3211. (https://doi.org/10.1007/s11136-019-02283-w)
Khan S, Krenning EP, van Essen M, Kam BL, Teunissen JJ & Kwekkeboom DJ 2011 Quality of life in 265 patients with gastroenteropancreatic or bronchial neuroendocrine tumors treated with [177Lu-DOTA0,Tyr3]Octreotate. Journal of Nuclear Medicine 52 1361–1368. (https://doi.org/10.2967/jnumed.111.087932)
King MT, Costa DSJ, Aaronson NK, Brazier JE, Cella DF, Fayers PM, Grimison P, Janda M, Kemmler G, Norman R, et al.2016 QLU-C10D: a health state classification system for a multi-attribute utility measure based on the EORTC QLQ-C30. Quality of Life Research 25 625–636. (https://doi.org/10.1007/s11136-015-1217-y)
King MT, Viney R, Simon Pickard A, Rowen D, Aaronson NK, Brazier JE, Cella D, Costa DSJ, Fayers PM, Kemmler G, et al.2018 Australian utility weights for the EORTC QLU-C10D, a multi-attribute utility instrument derived from the cancer-specific Quality of Life Questionnaire, EORTC QLQ-C30. Pharmacoeconomics 36 225–238. (https://doi.org/10.1007/s40273-017-0582-5)
Korse CM, Bonfrer JMG, Aaronson NK, Hart AAM & Taal BG 2009 Chromogranin A as an alternative to 5-hydroxyindoleacetic acid in the evaluation of symptoms during treatment of patients with neuroendocrine tumors. Neuroendocrinology 89 296–301. (https://doi.org/10.1159/000162876)
Krahn M, Bremner KE, Tomlinson G, Ritvo P, Irvine J & Naglie G 2007 Responsiveness of disease-specific and generic utility instruments in prostate cancer patients. Quality of Life Research 16 509–522. (https://doi.org/10.1007/s11136-006-9132-x)
Kulke MH, Lenz HJ, Meropol NJ, Posey J, Ryan DP, Picus J, Bergsland E, Stuart K, Tye L, Huang X, et al.2008 Activity of sunitinib in patients with advanced neuroendocrine tumors. Journal of Clinical Oncology 26 3403–3410. (https://doi.org/10.1200/JCO.2007.15.9020)
Kvols LK, Oberg KE, O’Dorisio TM, Mohideen P, de Herder WW, Arnold R & Wiedenmann B 2012 Pasireotide (SOM230) shows efficacy and tolerability in the treatment of patients with advanced neuroendocrine tumors refractory or resistant to octreotide LAR: results from a phase II study. Endocrine-Related Cancer 19 657–666. (https://doi.org/10.1530/ERC-11-0367)
Larsson G & Janson ET 2008 Anemia in patients with midgut carcinoid, treated with alpha interferon: effects by erythropoietin treatment on the perceived quality of life. European Journal of Cancer Care 17 200–204. (https://doi.org/10.1111/j.1365-2354.2007.00844.x)
Larsson G, Sjödén PO, Oberg K, Eriksson B & von Essen L 2001 Health-related quality of life, anxiety and depression in patients with midgut carcinoid tumours. Acta Oncologica 40 825–831. (https://doi.org/10.1080/02841860152703445)
Lewis AR, Wang X, Magdalani L, D’Arienzo P, Bashir C, Mansoor W, Hubner R, Valle JW & McNamara MG 2018 Health-related quality of life, anxiety, depression and impulsivity in patients with advanced gastroenteropancreatic neuroendocrine tumours. World Journal of Gastroenterology 24 671–679. (https://doi.org/10.3748/wjg.v24.i6.671)
Lloyd RV, Osamura RY, Klöppel G & Rosai J 2017 Neoplasms of the neuroendocrine pancreas. In WHO Classification of Tumours of Endocrine Organs, 4th ed., pp. 209–240. Lyon, France: International Agency for Research on Cancer.
Marinova M, Mücke M, Mahlberg L, Essler M, Cuhls H, Radbruch L, Conrad R & Ahmadzadehfar H 2018 Improving quality of life in patients with pancreatic neuroendocrine tumor following peptide receptor radionuclide therapy assessed by EORTC QLQ-C30. European Journal of Nuclear Medicine and Molecular Imaging 45 38–46. (https://doi.org/10.1007/s00259-017-3816-z)
Marinova M, Mücke M, Fischer F, Essler M, Cuhls H, Radbruch L, Ghaei S, Conrad R & Ahmadzadehfar H 2019 Quality of life in patients with midgut NET following peptide receptor radionuclide therapy. European Journal of Nuclear Medicine and Molecular Imaging 46 2252–2259. (https://doi.org/10.1007/s00259-019-04431-3)
Martín-Richard M, Massutí B, Pineda E, Alonso V, Marmol M & Castellano D 2013 Antiproliferative effects of lanreotide autogel in patients with progressive, well-differentiated neuroendocrine tumours: a Spanish, multicentre, open-label, single arm phase II study. BMC Cancer 13 427. (https://doi.org/10.1186/1471-2407-13-427)
Martini C, Buxbaum S, Rodrigues M, Nilica B, Scarpa L, Holzner B & Gamper E-M 2018 Quality of life in patients with metastatic gastroenteropancreatic neuroendocrine tumors receiving peptide receptor radionuclide therapy: information from a monitoring program in clinical routine. Journal of Nuclear Medicine 59 1566–1573. (https://doi.org/10.2967/jnumed.117.204834)
McTaggart-Cowan H, King MT, Norman R, Costa DSJ, Pickard AS, Regier DA, Viney R & Peacock SJ 2019 The EORTC QLU-C10D: the Canadian valuation study and algorithm to derive cancer-specific utilities from the EORTC QLQ-C30. MDM Policy and Practice 4 2381468319842532. (https://doi.org/10.1177/2381468319842532)
Meng Y, McCarthy G, Berthon A & Dinet J 2017 Patient-reported health state utilities in metastatic gastroenteropancreatic neuroendocrine tumours – an analysis based on the CLARINET study. Health and Quality of Life Outcomes 15 131. (https://doi.org/10.1186/s12955-017-0711-z)
Meyer T, Qian W, Caplin ME, Armstrong G, Lao-Sirieix SH, Hardy R, Valle JW, Talbot DC, Cunningham D, Reed N, et al.2014 Capecitabine and streptozocin±cisplatin in advanced gastroenteropancreatic neuroendocrine tumours. European Journal of Cancer 50 902–911. (https://doi.org/10.1016/j.ejca.2013.12.011)
Min YH, Lee JW, Shin YW, Jo MW, Sohn G, Lee JH, Lee G, Jung KH, Sung J, Ko BS, et al.2014 Daily collection of self-reporting sleep disturbance data via a smartphone app in breast cancer patients receiving chemotherapy: a feasibility study. Journal of Medical Internet Research 16 e135. (https://doi.org/10.2196/jmir.3421)
Mitry E, Walter T, Baudin E, Kurtz JE, Ruszniewski P, Dominguez-Tinajero S, Bengrine-Lefevre L, Cadiot G, Dromain C, Farace F, et al.2014 Bevacizumab plus capecitabine in patients with progressive advanced well-differentiated neuroendocrine tumors of the gastro-intestinal (GI-NETs) tract (BETTER trial) – a phase II non-randomised trial. European Journal of Cancer 50 3107–3115. (https://doi.org/10.1016/j.ejca.2014.10.001)
Norman R, Mercieca-Bebber R, Rowen D, Brazier JE, Cella D, Pickard AS, Street DJ, Viney R, Revicki D, King MT, et al.2019 U.K. utility weights for the EORTC QLU-C10D. Health Economics 28 1385–1401. (https://doi.org/10.1002/hec.3950)
O’Shea T & Druce M 2017 When should genetic testing be performed in patients with neuroendocrine tumours? Reviews in Endocrine and Metabolic Disorders 18 499–515. (https://doi.org/10.1007/s11154-017-9430-3)
Pakhomov SV, Jacobsen SJ, Chute CG & Roger VL 2008 Agreement between patient-reported symptoms and their documentation in the medical record. American Journal of Managed Care 14 530–539.
Pasieka JL, Longman RS, Chambers AJ, Rorstad O, Rach-Longman K & Dixon E 2014 Cognitive impairment associated with carcinoid syndrome. Annals of Surgery 259 355–359. (https://doi.org/10.1097/SLA.0b013e318288ff6d)
Pavel M, Unger N, Borbath I, Ricci S, Hwang T-L, Brechenmacher T & Bechter O 2016 Safety and QoL in patients with advanced NET in a phase 3b expanded access study of everolimus. Targeted Oncology 11 667–675. (https://doi.org/10.1007/s11523-016-0440-y)
Pavel ME, Singh S, Strosberg JR, Bubuteishvili-Pacaud L, Degtyarev E, Neary MP, Carnaghi C, Tomasek J, Wolin E, Raderer M, et al.2017 Health-related quality of life for everolimus versus placebo in patients with advanced, non-functional, well-differentiated gastrointestinal or lung neuroendocrine tumours (RADIANT-4): a multicentre, randomised, double-blind, placebo-controlled, phase 3 trial. Lancet Oncology 18 1411–1422. (https://doi.org/10.1016/S1470-2045(17)30471-0)
Peipert BJ, Goswami S, Yount SE & Sturgeon C 2018 Health-related quality of life in MEN1 patients compared with other chronic conditions and the United States general population. Surgery 163 205–211. (https://doi.org/10.1016/j.surg.2017.04.030)
Petersen MA, Aaronson NK, Arraras JI, Chie WC, Conroy T, Costantini A, Dirven L, Fayers P, Gamper EM, Giesinger JM, et al.2018 The EORTC CAT Core – the computer adaptive version of the EORTC QLQ-C30 questionnaire. European Journal of Cancer 100 8–16. (https://doi.org/10.1016/j.ejca.2018.04.016)
Ramage JK, Ahmed A, Ardill J, Bax N, Breen DJ, Caplin ME, Corrie P, Davar J, Davies AH, Lewington V, et al.2012 Guidelines for the management of gastroenteropancreatic neuroendocrine (including carcinoid) tumours (NETs). Gut 61 6–32. (https://doi.org/10.1136/gutjnl-2011-300831)
Ramage JK, Punia P, Faluyi O, Frilling A, Meyer T, Saharan R & Valle JW 2019 Observational study to assess quality of life in patients with pancreatic neuroendocrine tumors receiving treatment with everolimus: the OBLIQUE study (Uk Phase IV Trial). Neuroendocrinology 108 317–327. (https://doi.org/10.1159/000497330)
Raymond E, Dahan L, Raoul JL, Bang YJ, Borbath I, Lombard-Bohas C, Valle J, Metrakos P, Smith D, Vinik A, et al.2011 Sunitinib malate for the treatment of pancreatic neuroendocrine tumors. New England Journal of Medicine 364 501–513. (https://doi.org/10.1056/NEJMoa1003825)
Rincon E, Monteiro-Guerra F, Rivera-Romero O, Dorronzoro-Zubiete E, Sanchez-Bocanegra CL & Gabarron E 2017 Mobile phone apps for quality of life and well-being assessment in breast and prostate cancer patients: systematic review. JMIR mHealth and uHealth 5 e187. (https://doi.org/10.2196/mhealth.8741)
Rinke A, Müller HH, Schade-Brittinger C, Klose KJ, Barth P, Wied M, Mayer C, Aminossadati B, Pape UF, Bläker M, et al.2009 Placebo-controlled, double-blind, prospective, randomized study on the effect of octreotide LAR in the control of tumor growth in patients with metastatic neuroendocrine midgut tumors: a report from the PROMID Study Group. Journal of Clinical Oncology 27 4656–4663. (https://doi.org/10.1200/JCO.2009.22.8510)
Rinke A, Neary MP, Eriksson J, Hunger M, Doan T, Karli D & Arnold R 2019 Health-related quality of life for long-acting octreotide versus placebo in patients with metastatic midgut neuroendocrine tumors in the phase 3 PROMID trial. Neuroendocrinology 109 141–151. (https://doi.org/10.1159/000499469)
Ruszniewski P, Caplin M, Pavel M, Ćwikła J, Phan A, Raderer M, Sedlackova E, Cadiot G, Wall L, Rindi G, et al.2014 1136PD Quality of life (QOL) with lanreotide autogel (LAN) vs. placebo in patients with enteropancreatic neuroendocrine tumours (EP-NETS): results from the Clarinet phase III study. Annals of Oncology 25 (Supplement_4) iv396–iv396. (https://doi.org/10.1093/annonc/mdu345.5)
Ruszniewski P, Valle JW, Lombard-Bohas C, Cuthbertson DJ, Perros P, Holubec L, Delle Fave G, Smith D, Niccoli P, Maisonobe P, et al.2016 Patient-reported outcomes with lanreotide Autogel/Depot for carcinoid syndrome: an international observational study. Digestive and Liver Disease 48 552–558. (https://doi.org/10.1016/j.dld.2015.12.013)
Singh S, Granberg D, Wolin E, Warner R, Sissons M, Kolarova T, Goldstein G, Pavel M, Öberg K & Leyden J 2017 Patient-reported burden of a neuroendocrine tumor (NET) diagnosis: results from the first global survey of patients with NETs. Journal of Global Oncology 3 43–53. (https://doi.org/10.1200/JGO.2015.002980)
Spolverato G, Bagante F, Wagner D, Buettner S, Gupta R, Kim Y, Maqsood H & Pawlik TM 2015 Quality of life after treatment of neuroendocrine liver metastasis. Journal of Surgical Research 198 155–164. (https://doi.org/10.1016/j.jss.2015.05.048)
Strosberg J, El-Haddad G, Wolin E, Hendifar A, Yao J, Chasen B, Mittra E, Kunz PL, Kulke MH, Jacene H, et al.2017 Phase 3 trial of 177Lu-dotatate for midgut neuroendocrine tumors. New England Journal of Medicine 376 125–135. (https://doi.org/10.1056/NEJMoa1607427)
Strosberg J, Wolin E, Chasen B, Kulke M, Bushnell D, Caplin M, Baum RP, Kunz P, Hobday T, Hendifar A, et al.2018 Health-related quality of life in patients with progressive midgut neuroendocrine tumors treated with 177 Lu-Dotatate in the Phase III NETTER-1 trial. Journal of Clinical Oncology 36 2578–2584. (https://doi.org/10.1200/JCO.2018.78.5865)
Topping M, Gray D, Friend E, Davies A & Ramage J 2017 A systematic review of symptoms and quality of life issues in pancreatic neuroendocrine tumours. Neuroendocrinology 105 320–330. (https://doi.org/10.1159/000475793)
Velikova G, Brown JM, Smith AB & Selby PJ 2002 Computer-based quality of life questionnaires may contribute to doctor-patient interactions in oncology. British Journal of Cancer 86 51–59. (https://doi.org/10.1038/sj.bjc.6600001)
Vinik E, Carlton CA, Silva MP & Vinik AI 2009 Development of the Norfolk quality of life tool for assessing patients with neuroendocrine tumors. Pancreas 38 e87–e95. (https://doi.org/10.1097/MPA.0b013e31819b6441)
Vinik AI, Vinik E, Diebold A & Woltering E 2014 Measuring the relationship of quality of life and health status: including tumor burden, symptoms, and biochemical measures in patients with neuroendocrine tumors. In Management of Neuroendocrine Tumors of the Pancreas and Digestive Tract: From Surgery to Targeted Therapies: A Multidisciplinary Approach. Berlin, Germany: Springer. (https://doi.org/10.1007/978-2-8178-0430-9_14)
Vinik A, Bottomley A, Korytowsky B, Bang Y-J, Raoul J-L, Valle JW & Raymond E 2016 Patient-reported outcomes and quality of life with sunitinib versus placebo for pancreatic neuroendocrine tumors: results from an international phase iii trial. Targeted Oncology 11 815–824. (https://doi.org/10.1007/s11523-016-0462-5)
Warsame R & D’Souza A 2019 Patient reported outcomes have arrived: a practical overview for clinicians in using patient reported outcomes in oncology. Mayo Clinic Proceedings 94 2291–2301. (https://doi.org/10.1016/j.mayocp.2019.04.005)
Weickert MO, Kaltsas G, Hörsch D, Lapuerta P, Pavel M, Valle JW, Caplin ME, Bergsland E, Kunz PL, Anthony LB, et al.2018 Changes in weight associated with telotristat ethyl in the treatment of carcinoid syndrome. Clinical Therapeutics 40 952.e2–962.e2. (https://doi.org/10.1016/j.clinthera.2018.04.006)
Wiebe S, Guyatt G, Weaver B, Matijevic S & Sidwell C 2003 Comparative responsiveness of generic and specific quality-of-life instruments. Journal of Clinical Epidemiology 56 52–60. (https://doi.org/10.1016/s0895-4356(02)00537-1)
Yadegarfar G, Friend L, Jones L, Plum LM, Ardill J, Taal B, Larsson G, Jeziorski K, Kwekkeboom D, Ramage JK, et al.2013 Validation of the EORTC QLQ-GINET21 questionnaire for assessing quality of life of patients with gastrointestinal neuroendocrine tumours. British Journal of Cancer 108 301–310. (https://doi.org/10.1038/bjc.2012.560)
Yao JC, Hassan M, Phan A, Dagohoy C, Leary C, Mares JE, Abdalla EK, Fleming JB, Vauthey JN, Rashid A, et al.2008 One hundred years after ‘carcinoid’: epidemiology of and prognostic factors for neuroendocrine tumors in 35,825 cases in the United States. Journal of Clinical Oncology 26 3063–3072. (https://doi.org/10.1200/JCO.2007.15.4377)
Yao JC, Fazio N, Singh S, Buzzoni R, Carnaghi C, Wolin E & Pavel ME 2016 Everolimus for the treatment of advanced, non-functional neuroendocrine tumours of the lung or gastrointestinal tract (RADIANT-4): a randomised, placebo-controlled, phase 3 study. Lancet 387 968–977. (https://doi.org/10.1016/S0140-6736(15)00817-X)
Zandee WT, Brabander T, Blažević A, Kam BLR, Teunissen JJM, Feelders RA & de Herder WW 2019 Symptomatic and radiological response to 177Lu-DOTATATE for the treatment of functioning pancreatic neuroendocrine tumors. Journal of Clinical Endocrinology and Metabolism 104 1336–1344. (https://doi.org/10.1210/jc.2018-01991)
Zubrod CG, Schneiderman M, Frei E, Brindley C, Lennard Gold G, Shnider B, Oviedo R, Gorman J, Jones R, Jonsson U, et al. 1960 Appraisal of methods for the study of chemotherapy of cancer in man: comparative therapeutic trial of nitrogen mustard and triethylene thiophosphoramide. Journal of Chronic Diseases 11 7–33. (https://doi.org/10.1016/0021-9681(60)90137-5)
