Pantry is free and open source. The rest of the AI stack (Iron, Butter, Syrup, Whisk, Waffle) is in private early access with pricing to be determined.
Key features
GroqMetal: dedicated bare-metal infrastructure tuned for speed and reliability
GroqCore: production-ready inference stack with no infrastructure expertise required
GroqAssured: enterprise-grade governance, auditability, and control
256 LPUs per rack with 40 PB/s SRAM bandwidth
1,000 tokens per second per user throughput
128 GB on-chip SRAM per rack
315 PFLOPS of FP8 inference compute
13 globally distributed data centers across four continents
Pantry: free, open-source vulnerability scanner for macOS and Linux
Iron: GPU kernels optimized for Apple silicon, NVIDIA CUDA, and AMD ROCm
Butter: multi-modal inference engine for text, vision and audio
Whisk: memory and context management to mix documents and context into requests
Syrup: control plane for monitoring, telemetry, access control and cost per GPU
Waffle: agent harness for building assistants
Runs on Apple silicon, NVIDIA RTX, H100, H200, B100, B200 and AMD ROCm
Local execution keeps data on your hardware by default
What makes it different
Fast inference at scale without sacrificing affordability
LPU pioneered technology integrated with NVIDIA GPUs
Fully integrated platform combining infrastructure, inference, and control
Built for lower cost per token through efficiency optimization
Private by default; data stays on your hardware
Supports multiple hardware platforms with write-once kernels
Pantry offers free, open-source vulnerability scanning and patching
Enables AI on commodity hardware people already own