Tinybox- offline AI device 120B parameters
Tiny Corp has started shipping Tinybox, a powerful hardware solution for deep learning designed to run massive AI models offline with industry-leading price-to-performance ratios.
tinygrad
We write and maintain tinygrad, the fastest growing neural network framework. It's extremely simple, and breaks down the most complex networks into 3 OpTypes:
- ElementwiseOps are UnaryOps, BinaryOps, and TernaryOps. They operate on 1-3 tensors and run elementwise (e.g., SQRT, ADD, MUL).
- ReduceOps operate on one tensor and return a smaller tensor (e.g., SUM, MAX).
- MovementOps are virtual ops that operate on one tensor and move the data around copy-free with ShapeTracker (e.g., RESHAPE, PERMUTE, EXPAND).
tinybox (now shipping)
We sell a computer called the tinybox. It comes in red, green, and soon, exa.
| Feature | red v2 | green v2 blackwell | exabox | | :--- | :--- | :--- | :--- | | FP16 FLOPS | 778 TFLOPS | 3086 TFLOPS | ~1 EXAFLOP | | GPU Model | 4x 9070XT | 4x RTX PRO 6000 Blackwell | 720x RDNA5 AT0 XL | | GPU RAM | 64 GB | 384 GB | 25,920 GB | | CPU | 32 core AMD EPYC | 32 core AMD GENOA | 120x 32 core AMD GENOA | | Price | $12,000 | $65,000 | ~$10M (2027) |
FAQ Highlights
- What is a tinybox? It is a very powerful computer for deep learning with the best performance/$. It benchmarked in MLPerf Training 4.0 against computers costing 10x as much.
- Is tinygrad faster than PyTorch? Not yet for most use cases, but it will be. It compiles custom kernels for every operation and uses lazy tensors for aggressive fusion.
- What's the goal? To accelerate. We will commoditize the petaflop and enable AI for everyone.
Source: Hacker News
