In this issue of Diabetes Care, David Eddy and Leonard Schlessinger (1,2) present and validate Archimedes, an innovative new mathematical model that simulates the natural history of diabetes and its complications and predicts the results of clinical trials. The model itself is large and complex, dealing with an extraordinary variety of physiologic-, patient-, and health system-level variables. It is also extraordinarily opaque: diabetes and its complications are reduced to a series of complex differential equations, and although Eddy and Schlessinger provide the functional forms of the equations, they do not provide the values of the variables or the parts of the model that describe microvascular and macrovascular complications. Despite these limitations, the results are astounding. They use the model to predict 74 major outcomes from 18 clinical trials. For 71 of the 74 clinical outcomes, there were no statistically significant differences between the results calculated by the model and the results observed in the trial (2). Considering only the trials that were not used to build the model, the correlation was r = 0.99 (2).
Very few epidemiologic studies or clinical trials are able to measure disease progression and intervention effects over a lifetime. Yet it is just such information—the natural history of disease and the long-term impact of interventions on costs, quality of life, and health outcomes—that is most germane to the formulation of health policy. When such information is not available, models may be used to integrate evidence from diverse sources of varying quality to make inferences about future economic, quality of life, and health outcomes and to provide data for decision making (3).
Previously, both decision-tree and state-transition models have been used to project lifetime outcomes (3). Decision-tree models present a sequence of decisions and chance events over time. Each chance event is assigned a probability. Alternative decision strategies are evaluated by calculating their average consequences. A limitation of decision-tree models is that the probability of each chance event is static. In chronic diseases, the probability of chance events changes with age, health status, and time. For this reason, decision-tree models are not often used for modeling chronic diseases such as diabetes.
In contrast, state-transition models allocate and reallocate subjects into health states defined according to population characteristics such as age, disease stage, and treatment. Age, clinical history, and treatment are included in the model by incorporating them into the definition of the health states or into the specification of the transition probability. Transitions occur from one health state to another at defined time intervals (usually 1 year) according to the transition probabilities. In Markov models, the proportion of subjects in each health state each year is treated as certain and the transition probabilities depend on the current state. Through simulation, the number of subjects in the population passing through each state at each point in time can be estimated.
In Monte Carlo models, each possible chance event is simulated for each individual in the cohort and summary statistics are computed by accumulating counts of these events over the simulated time span for the modeled population. For example, to simulate a 5% chance of developing diabetes in a given year, the computer generates a random integer between 1 and 100, and if that integer is 5 or less, the computer program tallies the simulated person as developing diabetes in that year. Because this represents one possible experimental observation, the entire simulation is repeated many times. As the number of runs grows large, the average values approach the values that would be computed by a Markov model.
In diabetes, both Markov and Monte Carlo models have been used to describe disease progression (4–9) and to estimate the cost-effectiveness of treatments (10–12). Archimedes, a complex mathematical model without explicit structure or data inputs, represents a new type of disease model. Unfortunately, what Archimedes gains with respect to precision, it may lose with respect to simplicity and transparency. No model can provide a perfect representation of reality. The value of a model lies not only in its results, but also in its ability to reveal the logical connections between inputs and outputs. A view of the inner workings of the model, the definition of variables, the structural assumptions, and the ways that data are identified, modeled, and incorporated permits an assessment of its reasonableness. If a model becomes too complex, it becomes a “black box,” and it is difficult to know how much credence to place in its results.
Models and their results are not statements of scientific fact but aids to decision making (13). Models can be evaluated by their ability to predict outcomes when tested under hypothetical conditions in which the results should be obvious (interval validation), by their ability to predict intermediate and long-term outcomes as defined by clinical trials and epidemiologic studies (external validation), and by their ability to predict outcomes obtained by other independently developed and programmed models (between-model validation) (13). If the outputs of different models differ substantially, the modelers should cooperate and attempt to explain the reasons for the discrepancies (14).
Archimedes is a new and exciting addition to our modeling armamentarium. Clearly, its value will be enhanced when its proprietary details are published and can be examined more critically, when cost and quality-of-life data are incorporated to provide an estimate of value for money, such as cost per quality-adjusted life-year gained, and when its predictions about the future are compared with those obtained from other available diabetes models.
