Study finds MoE routing telemetry can improve membership-inference attacks

A new paper asks whether routing traces from mixture-of-experts (MoE) models can reveal that an example appeared in fine-tuning data. These traces arise during inference and may be retained or exposed for purposes such as monitoring, debugging, load analysis and safety auditing. Unlike ordinary model answers, routing telemetry offers a view into internal computation, prompting the study’s privacy question. The researchers test whether combining it with output-based evidence helps infer training-set membership.

The proposed attack combines conventional signals from model outputs with aggregated features derived from routing telemetry. A membership classifier is trained on independently fine-tuned shadow models and then applied to a target model. The evaluation covers three MoE architectures and three data domains. In all nine settings, telemetry improved the attack compared with a strong ensemble using output signals alone. At a 1% false-positive rate, the reported true-positive rate increased by 2.7--9.4 percentage points.

The improvement remained observable under full fine-tuning, frozen-router training, LoRA and instruction tuning. The researchers also report that the added signal remained detectable when telemetry showed only discrete expert selections, when telemetry was restricted, or when the attack used a single shadow model. Their analysis says router-specific memorization is not necessary to produce the leakage. Instead, they attribute it to membership information entering hidden representations during fine-tuning, with routing exposing a projection of that information even when router parameters are frozen.

Perturbing telemetry reduced the additional leakage only as the data became less faithful, according to the paper. The authors characterize routing traces as an additional privacy surface for fine-tuned MoE models. For builders of fine-tuned MoE systems, this makes routing records used for monitoring or debugging a privacy consideration alongside model outputs. The abstract reports results across the tested architectures and settings, but does not state how frequently such attacks succeed in deployed systems.