The Problem with Markov Models

Markov models have become a default tool in drug development, used for modeling disease progression, treatment outcomes, and cost-effectiveness. While mathematically elegant, these models are often inadequate for capturing the complexities of drug development, which involve non-linearity, fat-tailed risks, and extreme events.

By relying on memorylessness, Gaussian assumptions, and linearity, Markov models frequently fail to reflect the reality of clinical trials and regulatory pathways. Drawing from Mandelbrot’s fractals, Taleb’s black swan theory, and Extreme Value Theory (EVT), this article critiques Markov models and explores alternatives better suited for decision-making in drug development.

The Basics of Markov Models

Markov models assume that the probability of transitioning to a future state depends solely on the current state, not on the sequence of prior states. These transitions are defined by a transition probability matrix:

$$
P =
\begin{bmatrix}
P_{11} & P_{12} & \cdots & P_{1n} \
P_{21} & P_{22} & \cdots & P_{2n} \
\vdots & \vdots & \ddots & \vdots \
P_{n1} & P_{n2} & \cdots & P_{nn}
\end{bmatrix}
$$

Each element Pij represents the probability of transitioning from state i to state j. At any given time ttt, the probabilities of being in each state are computed iteratively:

$$
P^{(t)} = P^{(t-1)} \cdot P
$$

Markov models frequently assume time homogeneity, meaning the transition probabilities Pij remain constant over time—a critical limitation in dynamic systems like drug development.

Theoretical Flaws in Markov Models

1. Memorylessness Assumption

Markov models assume that the future depends only on the present state. However, in drug development, this assumption is flawed. A patient’s outcome is influenced by their entire treatment history, including cumulative toxicities, prior adverse events, and dose adjustments. Ignoring these dependencies oversimplifies the problem and introduces bias into predictions.

2. Gaussian Distributions and Fat Tails

Markov models rely on Gaussian distributions for transitions, which underestimate the likelihood of extreme, rare events. Drug development is characterized by fat-tailed distributions, where rare but high-impact events—such as regulatory rejections, catastrophic side effects, or unexpected efficacy—play a dominant role.

Extreme Value Theory (EVT) offers a better framework for modeling such risks. The probability of an extreme event exceeding a threshold xxx can be modeled using the Generalized Pareto Distribution (GPD):

$$
P(X > x) = \left( 1 + \xi \frac{x - \mu}{\sigma} \right)^{-\frac{1}{\xi}}
$$

Where:

• μ: Location parameter (threshold for extreme events)
• σ: Scale parameter (width of the tail)
• ξ: Shape parameter (thickness of the tail)

3. Linearity and Non-Linear Cascades

Markov models presuppose linear transitions between states, but drug development is inherently non-linear. A small adverse event in early clinical trials can trigger cascading failures that halt the entire program. Non-linear dynamics, which are central to Mandelbrot’s fractal theory, cannot be adequately represented in a Markov framework.

Practical Limitations

Transition Probabilities Are Not Static

Markov models assume that transition probabilities remain constant over time. In reality, probabilities in drug development change dynamically based on new data, safety concerns, or regulatory updates. Bayesian models offer a more flexible alternative, allowing probabilities to be updated iteratively as new information becomes available:

$$
P(\theta | D) = \frac{P(D | \theta) P(\theta)}{P(D)}
$$

Where:

• P(θ∣D) is the posterior probability given the data DDD,
• P(D∣θ) is the likelihood of the data given the parameter θ\thetaθ,
• P(θ) is the prior probability of the parameter,
• P(D) is the marginal probability of the data.

Inability to Model Cascading Failures

Markov models treat transitions as isolated events, ignoring the cascading effects often seen in drug development. For example, an unexpected safety signal in Phase III trials can retroactively cast doubt on earlier findings, disrupting the entire development pipeline.

Conclusion

Markov models, while convenient, are fundamentally misaligned with the complexities of drug development. Their assumptions of memorylessness, Gaussian behavior, and linearity fail to capture the extreme risks, historical dependencies, and cascading failures that dominate this field.

By embracing alternative approaches such as EVT, Bayesian frameworks, and agent-based models, drug developers can build models that reflect reality, providing more robust and reliable insights in an uncertain world.