Serverless inference with per-token pricing (pay for output, input, and cached tokens). Training pricing from $0.50 to $10.00 per 1M training tokens depending on model size and fine-tuning type. On-demand deployments at $0.134 to $0.334 per minute ($8 to $20 per hour) for various GPU types.
Key features
GroqMetal: dedicated bare-metal infrastructure tuned for speed and reliability
GroqCore: production-ready inference stack with no infrastructure expertise required
GroqAssured: enterprise-grade governance, auditability, and control
256 LPUs per rack with 40 PB/s SRAM bandwidth
1,000 tokens per second per user throughput
128 GB on-chip SRAM per rack
315 PFLOPS of FP8 inference compute
13 globally distributed data centers across four continents
Guided training runs
Configuration-led training
Custom training logic
Serverless inference
On-demand dedicated deployments
Reserved capacity
OpenAI compatible API
Model library with latest open models
Reinforcement learning support
Multi-region deployments
What makes it different
Fast inference at scale without sacrificing affordability
LPU pioneered technology integrated with NVIDIA GPUs
Fully integrated platform combining infrastructure, inference, and control
Drop-in replacement for closed-model APIs with cost savings of 50-75%
Instant production deployment from any training checkpoint in seconds
Optimized inference engine for industry-leading throughput and latency
Elastic and global RL inference scaling
Own the complete learning loop for specialized intelligence