vLLM is an inference and serving engine for large language models. Prior to 0.32.0, the Phi-4 multimodal audio encoder in phi4mm_audio.py and phi4mm_utils.py uses eager attention by default, allowing a remote client to submit one legal long audio clip that causes MultiHeadedAttention.forward_attention to materialize and retain full attention matrices in self.attn. The scaled dot-product attention branch does not materialize these matrices. The resulting quadratic GPU-memory use can cause an out-of-memory condition and terminate EngineCore for all users. This issue is fixed in version 0.32.0.
Get the CyboWatch SIEM platform, hosting, and 24×7 SOC analysts. Send an enquiry, or call or email us to get started.