Hierarchical Modeling of ICD Codes in EHR Foundation Models

This work introduces a novel approach to Electronic Health Record (EHR) representation learning by incorporating the hierarchical structure of ICD-10-CM codes into model architectures. The authors propose two mechanisms: augmenting BERT-style transformer diagnosis sequences with hierarchical tokens and enhancing graph-based representations with hierarchy-aware edges. Experiments on large-scale datasets MIMIC-IV and eICU demonstrate that explicitly encoding ICD hierarchy significantly improves predictive performance and transferability across datasets, highlighting the importance of hierarchy in clinical representation learning for better model generalization.