Protocol-Aware Tokenization and Architecture Co-Design for Wireless Packet Foundation Models

The paper introduces a novel approach to building foundation models for wireless packet traces by employing protocol-aware tokenization and exploring architecture co-design. The study reveals that a deeper GPT model (PLUME-DEEP) with 24 layers achieves 98.2% top-1 accuracy, significantly improving from the 12-layer version, while the Mamba-2 variant (PLUME-MAMBA) offers 96.1% accuracy with enhanced throughput and context length. The findings highlight that protocol-aware tokenization is the critical factor for performance improvement, suggesting that practitioners can optimize model deployment by balancing accuracy and speed through architectural choices.