Introduction
Meta Lingua (lingua) is a lightweight LLM training and inference library from Facebook Research designed for researchers to quickly experiment with architectures, losses, and data pipelines while providing tools to analyze speed and stability.
Key Features
- End-to-end training and inference with example configs and launch scripts.
- Distributed & SLURM support through
stool, enabling multi-GPU experiments and easy job management. - Modular components for transformer architectures, optimizers, data loaders, and checkpointing.
- Built-in profiling utilities for MFU/HFU and memory tracing to analyze performance.
Use Cases
- Research prototyping: rapidly implement and evaluate new ideas at small to medium scale.
- Small-scale training: iterate on architectural choices without heavy infra dependency.
- Teaching and reproducibility: a compact, well-documented codebase for demos and reproducible experiments.
Technical Highlights
- Pure PyTorch implementation that is easy to read and modify.
- Dataclass-based configurations with CLI overrides for reproducible experiments.
- Comprehensive examples, documentation, and citation information for academic use.