ANEForge: Python for direct computation on the Apple Neural Engine

ANEForge is a new Python package that enables direct programming of the Apple Neural Engine (ANE) without relying on CoreML, providing a more efficient pathway for utilizing the ANE's capabilities. It compiles a lazy tensor graph from 58 fused operators and 19 native bridge operators into a single ANE program, achieving low latency with a small fused program completing in approximately 90 microseconds. This tool is significant for practitioners as it allows for advanced model training and inference on Apple Silicon devices, including support for int8, int4, and sparse weights, making it a valuable resource for optimizing AI workloads on macOS 14 and later.