PACUTE: Phonology-, Affix-, and Character-level Understanding of Tokens for Filipino

PACUTE is a newly introduced benchmark consisting of 4,600 tasks aimed at assessing morphological understanding in Filipino, which features complex linguistic structures like infixation and reduplication. The study evaluates both open-weight LLMs and advanced commercial models, revealing that while frontier models show improved performance in recovering affixes, they still struggle with tasks involving morpheme transformations and syllabification, highlighting a significant gap in morphological composition capabilities. This benchmark is crucial for practitioners as it emphasizes the need for improved morphological handling in LLMs to better support languages with intricate morphological systems.