Context compression is an increasingly common way to reduce the inference cost of large language models: rather than processing a long prompt token-by-token, an encoder compresses it into a small set of dense “memory” vectors that the model reads in its place. These methods are typically trained and judged on how well they preserve semantic content — reconstruction fidelity. This project asks a different question: does preserving meaning also preserve behavior — specifically, safety-relevant behavior such as refusing harmful requests?

We find that it does not. Across multiple instruction-tuned model families (Mistral-7B-Instruct-v0.2, LLaMA-2-7B-Chat) and compression architectures (ICAE and ARC), encoder-based context compression systematically weakens refusal behavior. A model that reliably declines a harmful request in full-context mode becomes substantially more likely to comply when the identical request is presented in compressed form — even though reconstruction fidelity remains high. On HarmBench, Mistral-7B-Instruct’s harmful-prompt refusal rate drops from 57.5% to 16.5% under ICAE compression; on our held-out set the drop is larger still (62.82% to 10.9%). The effect is consistent rather than incidental.

To understand why, we probe the model’s internal representations, projecting decoder activations onto its learned “refusal direction” — the linear direction associated with refusing. Compression attenuates activation along this direction, giving a mechanistic account of the degraded behavior: the signal the model uses to decide to refuse is disproportionately lost under compression, even as general meaning survives. Near-perfect reconstruction does not imply preservation of refusal-triggering activations.

We then introduce Memory Steering, a lightweight intervention that restores safety without retraining. Operating entirely in the compressed representation space at inference time and controlled by a single steering strength, Memory Steering recovers most of what compression destroys: on HarmBench it lifts harmful-prompt refusals by +41 points on Mistral-7B-Instruct (16.5% → 57.5%) and +20 points on LLaMA-2-7B-Chat (77.0% → 97.5%), returning both models to their uncompressed baselines. A gated variant preserves near-zero refusal on benign prompts, preventing over-refusal.

The broader implication is a caution for compression-aware systems: semantic preservation does not guarantee behavioral preservation. Alignment-relevant features must be preserved — or restored — explicitly, not assumed to survive because meaning does.

Presented at the 39th Canadian Conference on Artificial Intelligence (2026). Anthony Nguyen and Wenjun Lin, Digital Innovation Lab, Algoma University.

Read the paper