RustGPT

RustGPT is a teaching-oriented project that implements a transformer-based LLM in pure Rust. It is intended for developers and researchers who want to build models from scratch and understand training and inference internals. The repository includes forward/backward passes, a training pipeline, tests, and an interactive mode.

Key Features

• Pure Rust implementation using ndarray for matrix operations.
• Training and interactive test modes for quick experiments and debugging.
• Modular codebase that illustrates Transformer blocks: attention, feed-forward, embeddings.

Use Cases

• Learning and education: understand LLM internals and training pipelines.
• Prototyping: validate custom model components and training strategies.
• Engineering experiments: explore Rust-based optimizations (SIMD, parallelism).

Technical Highlights

• Lightweight matrix operations via ndarray, avoiding heavy frameworks.
• Strong test coverage for core components, easing verification and maintenance.
• Designed for extensibility to incorporate performance improvements and advanced features.