Bearing Syntactic Fruit with Stack-Augmented Neural Networks

This paper presents stack-augmented neural networks that demonstrate human-like syntactic generalization using only surface forms, without relying on ground-truth parse trees or extensive pre-training. The study evaluates three base architectures—transformer, simple RNN, and LSTM—augmented with two types of stacks, finding that transformers with nondeterministic stacks achieve superior performance on tasks assessing hierarchical inductive bias. This advancement suggests that such architectures may better model human syntax acquisition, offering valuable insights for psycholinguistic research and practical applications in language processing.