Toward Open Weight Models Without Risks: Separating Public and Private Capabilities in LLMs

The paper introduces Tiered Language Models (TLMs), which allow for a single set of model weights to support multiple capability levels while maintaining open access. TLMs utilize a compact secret key to modify a subset of parameters, enabling a public configuration that behaves like a standard LLM and a keyed configuration that can perform advanced tasks like acquiring new languages and following instructions. This approach mitigates risks associated with sensitive capabilities and enables selective control over model functionalities, addressing challenges faced in the deployment of open-weight LLMs.