treating diabetes is a (machine) learnable problem

In this post I attempt to formalize my thoughts on contemporary machine learning research for Type 1 Diabetes modeling and prediction. I dream of a day where an insulin pump can locally run a model of sufficient quality in a closed-loop system such that the most a patient has to think about their pump is refilling it or recharging it. I've kept notes and thoughts to myself for a long time, but I mean to do a better job of publishing what I've been obsessed with for the past few years.

Proposition (statistical learnability under clinical risk)

Let $g_t \in \mathbb{R}_{>0}$ denote the blood-glucose value reported by a continuous glucose monitor sampled at $t \in \Delta t \cdot \mathbb{Z}_{\geq 0}$ (typically $\Delta t = 5$ min), and let $u_t, b_t, c_t \in \mathbb{R}_{\geq 0}$ denote, respectively, bolus insulin delivery, basal insulin delivery, and carbohydrate intake at time $t$. Fix a lookback horizon $k \in \mathbb{N}$ and a forecast horizon $n \in \mathbb{N}$, and write

$$\mathcal{H}_t \;=\; \big(g_{t-k:t},\; u_{t-k:t},\; b_{t-k:t},\; c_{t-k:t}\big) \;\in\; \mathcal{X} \subset \mathbb{R}^{4(k+1)}$$

for the state window, with $\mathcal{X}$ compact under physiological bounds. Glucose dynamics admit the Bergman minimal model

$$\dot G = -p_1(G - G_b) - X\,G + D(t),\\ \qquad \dot X = -p_2 X + p_3 (I - I_b),\\ \qquad \dot I = -p_4(I - I_b) + \tfrac{U(t)}{V_I},$$

with insulin/carbohydrate forcing $(U, D)$ and remote insulin action $X$. By Picard–Lindelöf together with an Itô-process embedding for the measurement noise, there exists a Borel-measurable conditional law

$$\mathcal{P}^\star(\,\cdot\,\mid \mathcal{H}_t) \;\in\; \mathscr{P}(\mathbb{R}^n_{>0})$$

for the future trajectory $g_{t+1:t+n}$ given $\mathcal{H}_t$, of which the deterministic mean response $F^\star(\mathcal{H}_t) = \mathbb{E}_{\mathcal{P}^\star}[g_{t+1:t+n} \mid \mathcal{H}_t]$ is a single moment. The clinical problem demands the full conditional law, because the relevant losses are asymmetric and tail-sensitive — minimizing the mean-squared error in $g$ is not the right objective when the cost of being wrong is itself a sharply asymmetric function of $g$.

Fix a clinical target band $[g_{\mathrm{lo}}, g_{\mathrm{hi}}]$, taking the 2019 ATTD consensus values $g_{\mathrm{lo}} = 70$ and $g_{\mathrm{hi}} = 180$ mg/dL, and a severe-hypoglycemia threshold $g_{\mathrm{sev}} = 54$ mg/dL. Define the clinical risk functional

$$\rho(g) \;=\; w_{\downarrow}\,(g_{\mathrm{lo}} - g)_+^2 \;+\; w_{\uparrow}\,(g - g_{\mathrm{hi}})_+^2, \qquad w_{\downarrow} \gg w_{\uparrow},$$

convex, zero on the target band, and asymmetric in the direction of clinical reality — severe hypoglycemia is acutely lethal on a timescale of minutes, severe hyperglycemia is chronically expensive on a timescale of years. The form is in the spirit of the Kovatchev BG-Risk Index but reduced to a one-sided quadratic for analytical convenience. The full closed-loop objective for any meal/correction policy $\pi_\phi : (\mathcal{H}_t, c_t) \mapsto \hat u_t$ and stabilizing basal estimator $\beta_\psi : \mathcal{H}_t \mapsto \hat b_t$ is then

$$\mathcal{J}_{\mathrm{clin}}(\phi, \psi) \;=\; \underbrace{\mathbb{E}\!\left[\sum_{i=1}^{n} \rho(g_{t+i}) \;\middle|\; \pi_\phi, \beta_\psi\right]}_{\text{expected in-range tracking}} \;+\; \lambda \cdot \underbrace{\mathrm{CVaR}_\alpha\!\left[\,\max_{1 \le i \le n}\, (g_{\mathrm{sev}} - g_{t+i})_+\,\right]}_{\text{severe-hypo tail penalty}},$$

with $\lambda > 0$, $\alpha \in (0,1)$ a confidence level (e.g., 0.99), and $\mathrm{CVaR}_\alpha[X] = \mathbb{E}[X \mid X \ge \mathrm{VaR}_\alpha(X)]$. The first term penalizes deviation from the target band asymmetrically; the second bounds the expected severity of the worst $1-\alpha$ fraction of hypoglycemic excursions — the formal statement of "a closed-loop controller that does not kill its user". The minimum of $\mathcal{J}_{\mathrm{clin}}$ is the clinical Bayes risk — strictly tail-aware, and a strictly stronger guarantee than $L^2$ Bayes risk on $g$ alone.

Let $\{f_\theta\}_{\theta \in \Theta}$ be a sufficiently expressive parametric family of conditional generators — diffusion or flow-matched models with MLP / transformer / state-space score networks, or quantized autoregressive heads — capable of modeling $\mathcal{P}^\star(\cdot \mid \mathcal{H}_t)$ rather than only its mean. By Hornik–Cybenko-style universal approximation of the score/drift function on compact $\mathcal{X}$, lifted to distribution approximation in 2-Wasserstein, for every $\varepsilon > 0$ there exists $\theta^\star \in \Theta$ with

$$\sup_{\mathcal{H}_t \in \mathcal{X}} \; W_2\bigl(f_{\theta^\star}(\,\cdot\,\mid \mathcal{H}_t),\; \mathcal{P}^\star(\,\cdot\,\mid \mathcal{H}_t)\bigr) \;\le\; \varepsilon.$$

Both $\rho$ (locally Lipschitz on the physiological range) and $\mathrm{CVaR}_\alpha$ (1-Lipschitz in $W_1$ , hence continuous in $W_2$ ) are continuous functionals of the conditional law under $W_2$. Convergence of $f_\theta$ to $\mathcal{P}^\star$ in $W_2$ therefore implies convergence of $\mathcal{J}_{\mathrm{clin}}$ to its infimum, and the empirical-risk minimizer of $\mathcal{J}_{\mathrm{clin}}$ on $N$ samples is consistent on $\mathcal{X}$. Hence forecasting, bolusing, and basal control on $(g, u, b, c)$ are statistically learnable under clinical risk and safety: in the population limit, the closed-loop policy achieves the clinical Bayes risk and the $\mathrm{CVaR}_\alpha$ -bounded hypoglycemia constraint simultaneously.

Whether the joint problem is additionally PAC-learnable — whether a tractable hypothesis class $\mathcal{F}$ of conditional generators exists with VC dimension $d_{\mathrm{VC}}(\mathcal{F})$ or empirical Rademacher complexity $\mathfrak{R}_N(\mathcal{F})$ growing slowly enough in $N$ to deliver uniform convergence of the risk-aware empirical loss

$$\sup_{f \in \mathcal{F}} \bigl|\widehat{\mathcal{J}}_{\mathrm{clin}}^N(f) - \mathcal{J}_{\mathrm{clin}}(f)\bigr| \;=\; \mathcal{O}\!\left(\sqrt{d_{\mathrm{VC}}/N}\right)$$

with high probability — is the open question I am currently working out. $\square$

Remark (modification). The Bergman ODE is autonomous in its coefficients while the human is not. Insulin sensitivity exhibits a circadian profile, dawn phenomenon in the early morning, blunted sensitivity in the evening, and acute sympathetic activity shifts hepatic glucose production and peripheral glucose uptake on minute scales. Both are unobserved confounders relative to $\mathcal{H}_t$, inflating the predictive variance $\mathrm{Var}(g_{t+n} \mid \mathcal{H}_t)$ above its physical lower bound. Augmenting the state with time-of-day $\tau_t \in S^1$, encoded as $\tau_t = \bigl(\cos\tfrac{2\pi t}{1440},\, \sin\tfrac{2\pi t}{1440}\bigr)$, and heart rate $h_t \in \mathbb{R}_{>0}$ strictly tightens this floor: by the law of total variance,

$$\mathbb{E}\!\left[\mathrm{Var}(g_{t+n} \mid \mathcal{H}_t, \tau_t, h_t)\right] \;=\; \mathbb{E}\!\left[\mathrm{Var}(g_{t+n} \mid \mathcal{H}_t)\right] \;-\; \mathbb{E}\!\left[\mathrm{Var}\!\left(\,\mathbb{E}[g_{t+n} \mid \mathcal{H}_t, \tau_t, h_t] \;\middle|\; \mathcal{H}_t\right)\right],$$

and the subtracted term is strictly positive unless $g_{t+n} \perp\!\!\!\perp (\tau_t, h_t) \mid \mathcal{H}_t$ — a conditional-independence statement the physiology rules out. The expected conditional variance under the augmented state is therefore strictly smaller than under $\mathcal{H}_t$ alone, the conditional law $\mathcal{P}^\star$ concentrates correspondingly tighter, and $\mathcal{J}_{\mathrm{clin}}$ inherits a strictly lower achievable floor.

Remark (architecture). Attention is the right inductive bias here: meal absorption, insulin pharmacokinetics, and exercise effects each induce variable-lag couplings between events in $\mathcal{H}_t$ and future $g$, and softmax attention

$$\mathrm{Attn}(Q, K, V) \;=\; \mathrm{softmax}\!\left(\tfrac{QK^\top}{\sqrt{d_k}}\right) V$$

learns these data-dependent kernels without committing to a fixed convolutional receptive field or recurrent memory horizon. Yet existing CGM transformers underexplore the embedding space: heterogeneous tokens — scalar glucose $g_t$, sparse event-valued carb intake $c_t$, cyclic $\tau_t \in S^1$, dense $h_t$ — are typically projected through a single shared linear lift $W_e \in \mathbb{R}^{d \times d_{\mathrm{in}}}$, collapsing the inductive priors that distinguish them. Fourier features on $\tau_t$, event-typed embeddings on $(u, b, c)$, and separate continuous streams for $(g, h)$ with cross-attention fusion are the obvious levers, and remain largely untouched in the published literature.

Remark (existing work). Two recent results bracket the proposition. GluFormer (Lutsker et al., 2024, arXiv:2408.11876) trains an autoregressive transformer on over $10^7$ CGM measurements from 10,812 adults and shows that the learned representation generalizes across 19 cohorts, 5 countries, and 8 devices — outperforming HbA1c on 12-year diabetes-onset risk stratification. As evidence that $F^\star$ admits a useful low-dimensional summary, this is dispositive: the foundation-model paradigm is viable on CGM at population scale. As a candidate for the closed-loop $(g, u, b, c)$ claim, it is misaligned on three counts. First, the training cohort is primarily non-diabetic — a glycemic regime governed by endogenous β-cell response, not by exogenous bolus pharmacokinetics. The covariate shift to T1D is exactly the regime where $u_t, b_t$ dominate $F^\star$, and where prediction errors are most clinically expensive. Second, the architecture is glucose-only autoregression; insulin and carbohydrates enter only through a separately trained multi-modal variant, and neither version conditions on $u_t, b_t$ as causal forcing terms. Third, glucose tokenization — necessary for the LM-style decoder — discards the continuous metric structure any downstream control law has to reason over.

The hybrid GRU–Transformer of Mazgouti et al. (2026, Algorithms 19(1):52) cuts the other way: small, narrow, but architecturally adjacent to what the proposition wants. A 60-step lookback at $\Delta t = 15$ min, augmented with hand-engineered features (entropy, detrended fluctuation analysis, monotonicity, sinusoidal time-of-day), yields reported 30-/60-min RMSEs of 6.65 / 8.91 mg/dL on a 12-patient T1D cohort — striking relative to the OhioT1DM literature, which clusters in the 15–22 mg/dL range at 30 min. The intuition is right: short-range recurrence composes with global attention to span the meal-response and circadian scales the Bergman model predicts. But the experimental frame is doing work the architecture is not. The cohort is $n = 12$ on a single Freestyle Libre at 15-min sampling (vs. 5-min for current Dexcom G7, 1-min for Libre 3), Savitzky–Golay smoothing is applied to the full CGM signal prior to windowing, which smooths both the model inputs and the effective prediction targets, and the inputs exclude $u_t, b_t, c_t$. The model therefore forecasts the autonomous evolution of $g$, not the controlled response to insulin and carbs. Each restriction strictly shrinks the regression problem; none is admissible under the proposition.

Remark (direction). The architectural remark above identifies the right relational primitive — variable-lag, data-dependent coupling — but not the right computational or segmentation primitive for the data regime the proposition actually demands. Softmax attention's $O(N^2)$ cost forces the windowed lookback $k$ that essentially every published CGM transformer relies on, and rules out the natural training context: months of per-patient 5-min CGM plus event-typed insulin and carb streams, on the order of $10^5$ tokens per patient before any multi-patient pretraining. Structured state-space models like Mamba-2 in particular (Dao & Gu, 2024, arXiv:2405.21060) recover $O(N)$ inference under a state-space duality that makes the recurrence 2–8× faster than the original Mamba while remaining competitive with transformers on long-sequence modeling. They are the obvious backbone once context length is no longer a budgeting decision. Their theoretical and experimental gains have been demonstrated on language and DNA; the work that remains is showing the same gains transfer to the CGM regime, where the relevant baseline is not perplexity but the clinical-risk objective above.

The deeper mismatch is hierarchical. A T1D timeline is not a uniform sequence: information density spikes at meals, boluses, and exercise, and is near-zero across overnight steady states. Fixed-stride tokenization spends capacity on the idle hours and underresolves the events that drive $F^\star$. H-Net (Hwang, Wang & Gu, 2025, arXiv:2507.07955) targets exactly this: a learned dynamic chunking operator, trained jointly with the rest of the network, that replaces tokenization with content- and context-dependent segmentation. A byte-level H-Net using Mamba-2 encoder/decoder blocks with a Transformer main network matches a token-based transformer of twice its size on language and achieves nearly $4\times$ data efficiency on DNA, a continuous biosensor stream with event-driven structure and no natural token boundaries, which is structurally the regime CGM lives in. An H-Net variant with (i) event-typed embeddings for $(u, b, c)$ in place of byte embeddings, (ii) a continuous regression head on $g$ in place of softmax over a discretized vocabulary, and (iii) chunk-boundary supervision tied to detected meal/insulin/exercise events is the architecture I am currently working out.

This framing dovetails with Large Concept Models (Barrault et al., 2024, arXiv:2412.08821), which abandon next-token prediction entirely in favor of next-concept prediction in a sentence-embedding space — a sequence roughly an order of magnitude shorter than the underlying tokens, generated either by MSE regression on the embedding or by diffusion over it. The analogy to T1D is almost too clean: the dynamic chunks H-Net would learn over $(g, u, b, c)$ are not "tokens" in any meaningful sense; they are pharmacokinetic events. A bolus is a concept with a known onset–peak–duration profile parameterized by the insulin analog. A meal is a concept with a carbohydrate-rate absorption profile dependent on macronutrient composition and gastric emptying. An exercise bout is a concept with a sympathetic-burst signature followed by sustained peripheral glucose uptake. A nocturnal basal regime is a concept whose evolution is governed by hepatic glucose production and the dawn cortisol rise. Each admits a low-dimensional latent description, and the patient's day decomposes into a sequence of such concepts plus the inter-event dynamics of $g$ between them.

A "large pharmacokinetic concept model" — H-Net's dynamic chunker producing event-segmented embeddings, a Mamba-2 backbone advancing the sequence of event embeddings, and an LCM-style head predicting the next event embedding — has two structural advantages over flat next-sample autoregression of $g$. First, the effective reasoning horizon shortens by roughly the average inter-event interval: a multi-day context collapses from $\mathcal{O}(10^4)$ samples to $\mathcal{O}(10^2)$ event tokens, which is precisely the LCM compression argument transposed onto physiology. Second, the right modeling primitive at the event level is distributional, not pointwise. The LCM team motivates diffusion over MSE because a given context may have many plausible, yet semantically different, continuations; in T1D, given identical $\mathcal{H}_t$, the next-event distribution is multimodal as the same pre-meal state can be followed by a 40 g carbohydrate intake or by a stress-driven hepatic glucose surge, and the lower tail of the predictive distribution is where hypoglycemia lives. A diffusion or quantized head over event-embedding space directly targets the conditional law $\mathcal{P}^\star(\,\cdot\,\mid \mathcal{H}_t)$ that the proposition demands: it produces samples from which both the asymmetric clinical risk $\rho$ and the $\mathrm{CVaR}_\alpha$ severe-hypo tail penalty in $\mathcal{J}_{\mathrm{clin}}$ are empirically computable, and against which the policy $\pi_\phi$ and basal estimator $\beta_\psi$ can be optimized end-to-end. This is the modeling primitive the clinical-risk formulation explicitly requires; pointwise approximators of $F^\star$ cannot resolve the lower tail of the predictive distribution, and the lower tail is the safety-critical object. The SSM/H-Net/LCM stack is the obvious vehicle for it.

Why is it obvious?

The short answer is inference cost. A transformer's attention mechanism computes a similarity score between every pair of positions in its input window — if the window has $N$ tokens, that's $N^2$ comparisons. Double the context, quadruple the compute. This is fine when you have a GPU cluster and are in no hurry; it is fatal when your inference budget is a coin-cell battery clipped to a belt. More precisely, at inference time, a transformer must retrieve and attend over its entire key-value cache on every new prediction: each fresh 5-minute glucose sample forces a pass over every prior sample in the window, so compute grows with context length even after training is done. A structured state-space model like Mamba-2 works differently: it compresses all prior history into a fixed-size hidden state (a compact summary vector) and updates that state with each new observation in constant time, $\mathcal{O}(1)$ per step regardless of how long the patient has been wearing the device. The full-history context that makes SSMs expensive to train (you still process the sequence once) becomes essentially free at inference time, because the recurrent state update costs the same whether you are processing sample 1 or sample 100,000. For a closed-loop insulin pump, this distinction is the whole ballgame. A modern pump like the Omnipod 5 or Tandem t:slim X2 runs on a microcontroller in a device worn directly against skin, with a battery measured in milliamp-hours, inside a thermal envelope where sustained heat generation means a burn, not a throttle warning. An attention-based model forecasting over weeks of 5-minute CGM history cannot run there, not because the arithmetic is hard, but because the arithmetic is quadratically too much of it. An SSM-based model, updating a fixed hidden state every five minutes, can.

The physics of this system is one of the biggest constraints. A majority of these model developments are happening inside environments of massive power and thermal capability: GPU data centers measured in hundreds of megawatts, with hot-aisle exhaust pushing 50°C. Whatever model is capable of managing diabetes to near perfection must also be capable of running inference inside a device that draws single-digit milliwatts, dissipates heat into the body it is trying to protect, and has no connection to a server. The gap between where these models are currently developed and where they eventually need to run is somewhere around eight to ten orders of magnitude in power budget. That is not a gap you close by waiting for Moore's Law. It is a gap you close by choosing the right architecture and by treating on-device deployability as a first-class design constraint rather than an afterthought for the productization team.