Challenges in Design of Multicenter Trials: End points assessed longitudinally for change and monotonicity

The three cohorts studied are dissimilar in important ways but are alike in having mild or moderate severity DSPN (stage 2B and 3 [23] are excluded), and severity of neuropathy was evaluated by the same approaches and criteria. Minimal criteria for the diagnosis of DSPN were the same: i.e., Σ 5 NC nds (5 attributes of nerve conduction expressed as normal deviates) ≥95th percentile, neuropathy impairment score of lower limbs [NIS(LL)] ≥2 points, or Diabetes Control and Complications Trial (DCCT) criteria (two approaches).

The epidemiologic study patients are consenting individuals with diabetes and distal symmetric DSPN from Rochester, and later Olmsted County, Minnesota (Rochester Diabetic Neuropathy Study) (22).

The Viatris trial patients (Nathan 1, Viatris) are placebo-treated individuals from a prospective double-blind multicenter trial of oral α-lipoic acid (600 mg/day) for 4 years. A total of 38 medical centers from the U.S. and Europe participated. Entry criteria were somewhat different from the epidemiologic cohort: 1) the neuropathy impairment score of lower limbs + 7 tests (NIS[LL] + 7 tests) ≥97.5th percentile; 2) NIS(LL) ≥2 points; and 3) two additional criteria: an abnormality (>99th percentile) of at least one attribute of nerve conduction (NC) in two leg nerves tested and a total symptom score ≥5 points.

The Lilly trial patients are placebo-treated individuals of two prospective double-blind multicenter trials (B7A-MC-MBCW and MBBP) of LY 333531 (a specific protein kinase C-β inhibitor) for the symptoms of DSPN during a 1-year period. The criteria for study entry were as follows: diagnosis of DSPN, neuropathic symptoms (a symptom score >6 points), vibration detection threshold of the great toe using CASE IV ≥95th percentile, a sural nerve sensory amplitude ≥1 μV, and an A1C level <12%. A total of 71 medical centers from six countries (in the U.S., Europe, Middle East, and Asia) participated.

The same clinical instruments, end points, instructions, and reference values were used for the epidemiologic cohort, the Viatris trial, and the Lilly trial. All tests were to be done independently, without taking previous or concurrent examination and test results into account.

In the epidemiologic cohort, one of us (P.J.D.) performed all clinical evaluations. In the Viatris and Lilly trials, trained and certified (for the study) neurologists or diabetologists did the examinations. Nerve conductions were performed by neurologists or electromyography technicians under the supervision of physicians. Quantitative sensation testing was done by technicians using CASE IV (a computer-based system using standard and calibrated stimuli, predetermined and validated algorithms of testing, and null stimuli and reference values [percentiles and normal deviates] from the Rochester Diabetic Neuropathy Study cohort of healthy subjects) (24,25).

All neurological evaluation sheets (Clinical Neuropathy Assessment, see below) were sent to a central reading and quality assurance center to interactively recognize omissions and errors, to electronically calculate scores, and to enter data into an electronic database (26).

In the epidemiologic cohort, all evaluations were performed independently without availability of earlier or concurrent tests. In the Viatris trial, ∼40% of respondents reported that they had always, or sometimes, reviewed previous or concurrent medical information or tests, probably improving the estimate of reproducibility and associations. Information was not available for the Lilly trial.

Neuropathy impairment was expressed as NIS(LL) points and symptoms as neuropathy symptom and change score (NSC) severity points. Individual attributes of nerve conduction were recorded as measured values and as normal deviates (nds) (from percentiles correcting for age and other applicable variables) (25). The composite nerve conduction score (Σ 5 NC nds) consists of peroneal nerve amplitude, velocity and distal latency, tibial distal latency, and sural amplitude. The normal deviates of the measurable (when amplitude is 0, velocity and latency cannot be estimated) attributes of the five attributes listed above were summed, divided by the number of measurable attributes and multiplied by 5. Results of quantitative sensation tests (QST) and reduction of heart rate with deep breathing were given as normal deviates (from percentiles corrected for applicable variables).

For reproducibility, intraclass correlations were used. For concordance between tests, Spearman rank-order correlations were used. For change with time, neuropathic end points were regressed on time and the derived slope was tested for statistical significance.

Factors that might affect results of any trial, and therefore the extent to which results from others might be disparate from ours, include 1) methods of data analyses used, including adjustment using normative values and whether classic or Bayesian statistics are used; 2) restriction on enrollments, defining the population to which inferences are to be made; and 3) the nature of any drug effect, including the magnitude of any effect and whether the effect is to show deterioration or to reverse decline.

In patients from the untreated (the epidemiologic study) and placebo arm (the Viatris and Lilly trials), glycemic control (A1C ∼8–9%) was not ideal (Table 1). All patients studied had mild-severity DSPN. However, by the criteria of the median values of NIS(LL), Σ 5 NC nds, Σ 3 QST tests nds, and heart rate with deep breathing nds, the Viatris trial cohort was most severe, followed by the Lilly trial and then by the epidemiologic cohort. By the criteria of symptoms (NSC severity) and Σ DCCT criteria (summated values giving equal weights for decreased ankle reflexes, clinical vibration of toes, and neuropathic symptoms [from NSC]), the order of most to least severe was the Lilly trial, the Viatris trial, and the epidemiologic cohorts.

Reproducibility of neuropathy end points could only be assessed for the Viatris trial and the Lilly trial. Generally, reproducibility was in the range of r_I = 0.7–0.85 (Table 2), somewhat lower than previously published values mainly from the epidemiologic cohort.

There is considerable variability of associations between pairs of end points and among cohorts (Table 3). For many pairs of end points, the percent variability explained by the data are <10%; but for others, it was in the range of 15–35%. For the epidemiologic cohort, the Viatris trial, and the Lilly trial, neurological signs (NIS[LL]) were correlated with symptoms to a substantial degree, i.e., 6, 12, and 15% at the first and 15, 24, and 16% at the last examination, respectively. By comparison, Σ 5 NC and NIS(LL) had values of 11, 17, and 0% and 14, 19, and 1%, respectively—a low correlation in the Lilly trial at both examinations. The mean values of the percent correlations were similar between the epidemiologic cohort and the Viatris trial at first and at last evaluations. It was lower for the Lilly trial cohort. In all cases, mean correlations were higher at the last examination than at the first examination.

Estimates of monotonicity varied somewhat depending on criteria for diagnosis of DSPN (Table 4). Therefore, it was assessed using three criteria for the diagnosis of DSPN, i.e., NIS[LL] ≥2 points, Σ 5 NC nds ≥95th, and DCCT with two or more of three criteria (two varieties). By these criteria, sural sensory nerve amplitude performed best, significantly worsening eight of a possible nine times, with the ninth time not showing a significant change. Peroneal nerve motor conduction velocity showed significant worsening in two of nine cases. Heart rate change with deep breathing showed neither worsening nor improvement. Improvement was found for ankle reflexes (significantly four of nine times); NSC(lower limbs) severity also showed significant improvement (three of nine times). Summated DCCT criteria showed significant improvement in three of nine cases. Tibial motor distal latency showed significant improvement in three cases.

These results were confirmed and strengthened by assessment of longitudinal data in patients in the epidemiologic cohort with DSPN for the full duration of study (median 9.5 years) (Table 5, available from the authors upon request). The end points showing consistent monotonic worsening without improvement were as follows: sural sensory amplitude, clinical vibration of toe (significant two of four times), and Σ 3 QST nds (significant four of four times).

The possible effect was tested using Σ 5 NC nds in triplicate measures at baseline of the Viatris trial cohort patients. At first (of triplicate measures), only patients with Σ 5 NC nds values ≥95th percentile were included and the same patients assessed at the second and third occasion—all three tests done within a week of each other. Comparing Σ 5 NC nds median values of the second and third to that of the first examination, the median values were 10.4 nds, 10.7 nds (NS), and 10.2 nds (P = 0.03). Because nerve conductions in DSPN have not significantly changed in a few days, the significant improvement found at the third examination is due to restriction at first evaluation and using the same end point (without restriction) for the subsequent two evaluations.

This study does not directly address all of the possible reasons why it has been difficult to demonstrate efficacy from adjuvant treatments of DSPN but deals with issues of reproducibility, concordance between pairs of tests, monotonicity, and the rate of worsening over prolonged times in mild DSPN. Our results show that the main reason is that DSPN worsens very slowly and that many end points are not sufficiently sensitive, accurate, representative, reproducible, and especially monotonic to reliably recognize the small worsening occurring over a period of 4 years. The magnitude of worsening measured over a period of 4 years was very small and was not accompanied by increased neurological signs or symptoms, both showing improvement (albeit small) rather than worsening. One possible explanation for why only a slight deterioration was observed might be our use of the commonly used practice in controlled clinical trials to “carry last observations forward” with the consequence of underestimating actual deterioration. Here, it was not the reason for the low rate of worsening, since results were essentially unchanged with deletion of “carried forward” data. Because this slow rate of worsening was found for all three cohorts and had also been observed previously in DCCT studies (1,2), we attribute it to very slow worsening of DSPN.

Assuming that DSPN worsens very slowly and that adjuvant treatments are unlikely to improve nerve function to a greater degree than from euglycemia, end points needed to detect a difference between treatment and placebo must have excellent clinimetric characteristics. If, on the other hand, an adjuvant treatment causes a greater improvement than that from institution of euglycemia, one assumes that present monitoring approaches or analyses would be able to recognize and track improvement. Many of the end points studied were reasonably reproducible and concordant. By the criteria of monotonicity, some of the end points performed poorly. This was especially true for symptoms and neurological signs, which typically showed significant improvement rather than worsening. The failure of these clinical measures to perform well raises a serious question about the adequacy of the recently published research case definition of polyneuropathy (27). The end points that performed best were sural nerve amplitude and to a lesser degree peroneal motor conduction velocity, both showing monotonic worsening. This worsening is all the more meaningful, as we had corrected for the influence of change in age and weight. Considering the epidemiologic cohort patients with DSPN followed for the full duration of study (median 9.5 years), sural nerve amplitude continued to show significant monotonic worsening three of four times. By contrast, peroneal motor conduction velocity did not show monotonic worsening. The composite score (Σ 3 QST nds) showed monotonic worsening four of four times.

What inferences about the conduct of future therapeutic trials of DSPN can be drawn from the present analysis, and are multicenter trials doable? Several important insights about how to improve future trials can be inferred. First, since near euglycemia prevents or ameliorates DSPN, success is more likely if patients who cannot, or will not, achieve good chronic glycemic control are included. Second, end points should be chosen differing on whether patients without or with DSPN are to be recruited for study—composite scores of NC and heart rate with deep breathing emphasized for the former and the two attributes of NC and composite score of QSTs identified here for the latter. Third, studies will need to be done for long times to show a treatment effect. Perhaps the best stage of DSPN for conduct of trials is patients on the verge of developing DSPN. These patients showed monotonic worsening of composite scores of nerve conduction and, to a lesser degree, of heart rate variability with deep breathing (28). Type 1 diabetic patients are preferable to type 2 diabetic patients because there is less variability of normal test results due to their younger age and because polyneuropathy worsens to a greater degree in type 1 diabetes (25,29). However, we acknowledge that selection of other patients, other stages of complication, or other methods of analyses may be shown to improve study performance unlike that shown here.

The monotonic improvement of clinical signs and symptoms in placebo arm trial patients of controlled clinical trials demonstrated here is of concern, especially in light of their historical use in DSPN (30,31) and their importance in the recently published case definition of polyneuropathy (27). We judge the observed improvement we observed in the placebo arm of trials is mainly due to a placebo effect. Measures of neurological signs and symptoms, although good enough to demonstrate a large therapeutic response (e.g., in immune neuropathies [32,33]), are not as able to do so in DSPN with low rates of worsening.

Finally, we note that restricting entry into study by use of a percentile abnormality of an end point and then using the same end point to measure change may result in a spurious estimate of improvement. Therefore, it is advisable to either incorporate a run-in phase and subsequent baseline measurements for analyses or use different criteria for entry into study than used for purposes of follow-up.

The work was supported in part from grants received from the National Institute of Neurologic Disorders and Stroke (NS36797).

The authors thank patients and investigators of the Viatris and Eli Lilly trials, patients in the Rochester Diabetic Neuropathy Study, and Vicki Clark and Mary Lou Hunziker for their help.

Additional contributing authors are Vladimir Skljarevski, MD, Eli Lilly and Co., Indianapolis, Indiana; and Phillip A. Low, MD, P. James B. Dyck, MD, Christopher J. Klein, MD (Peripheral Neuropathy Research Laboratories), and L. Joseph Melton, III, MD (Division of Epidemiology), Mayo Clinic College of Medicine, Rochester, Minnesota.