A lightweight inference engine supporting speculative speculative decoding (SSD).

ARC Relay — WebSocket relay server for agent remote control by Axolotl AI

Visual Causal Flow

High-performance Rust extensions for Axolotl (no OOM for large datasets) - drop-in acceleration for existing installations.

Nano vLLM

Disposable Temp mail service

[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

A safetensors extension to efficiently store sparse quantized tensors on disk

inference engine for LLMs

The Prime Intellect CLI provides a powerful command-line interface for managing GPU resources across various providers

Tensors and Dynamic neural networks in Python with strong GPU acceleration

🚀 Efficient implementations of state-of-the-art linear attention models in Torch and Triton

Finetune Qwen3, Llama 4, TTS, DeepSeek-R1 & Gemma 3 LLMs 2x faster with 70% less memory! 🦥

Utils for Unsloth

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Efficient implementations of state-of-the-art sequence modeling architectures—using PyTorch and Triton.

Small scale distributed training of sequential deep learning models, built on Numpy and MPI.

FlashInfer: Kernel Library for LLM Serving

PyTorch native quantization and sparsity for training and inference

A community curated list of Rust Language streamers

Extremely fast Query Engine for DataFrames, written in Rust

Node.js dependency tracing utility

[Hackintosh] Configuration for Lenovo Thinkpad P50.

Autonomous coding agent as an SDK, IDE extension, or CLI assistant.

HelixDB is an OLTP graph-vector database built in Rust.

Rust full node implementation of the Fuel v2 protocol.

Code at the speed of thought – Zed is a high-performance, multiplayer code editor from the creators of Atom and Tree-sitter.