ORBIT: Training-Free Multi-Attribute Behavioral Steering via Orthogonal Subspace Rotation

The article introduces ORBIT (Orthogonal Rotation-Based Intervention Technique), a training-free method for multi-attribute behavioral steering in language models, which utilizes singular value decomposition to construct a joint subspace for norm-preserving rotations. It addresses the limitations of existing methods that struggle with simultaneous steering of multiple attributes by implementing adaptive per-token gating and an optional additive boost for weak attributes. Evaluated on the TraitFactory and ToneBank benchmarks with models including Llama-3.2-3B, Qwen-2.5-7B, and Llama-3.1-8B, ORBIT demonstrates improved performance in maintaining output coherence while steering multiple attributes effectively, making it a significant advancement for practitioners in AI model control.