quantization

2 posts tagged with this topic.

Edge AI6 min read

Kimi K2.6: 1T Parameters, 8 H200s Required

Moonshot AI's Kimi K2.6 ships natively in INT4 at 1T parameters. Open weights, world-class agent benchmarks, and a hardware bill that starts at 8 H200s.

Ethan Marsh·April 21, 2026

Edge AI12 min read

Deploying a 70B LLM Offline at Sea

Hardware, quantization, and serving stack for running Llama 3.1 70B on a vessel with no cloud fallback. Real specs and real benchmarks.

Ethan Marsh·April 19, 2026