SoftSkill: Behavioral Compression for Contextual Adaptation

The paper introduces SoftSkill, a method for behavioral compression that allows natural-language skills to initialize a compact continuous context object, refined by a trainable soft delta while keeping the base model frozen. In experiments with the Qwen3.5-4B model, a length-32 SoftSkill prefix achieved significant performance improvements on benchmarks, including an 8.3-point increase on SearchQA and a 42.1-point increase on LiveMath, while reducing the reliance on extensive Markdown skill tokens. This approach suggests a shift in how task skills can be integrated into LLMs, emphasizing the potential for compact latent controls to enhance inference efficiency and effectiveness.