← All posts
Tag

quantization

2 posts tagged with this topic.

Organized network cables connected to ports in a server rack
Edge AI12 min read

Deploying a 70B LLM Offline at Sea

Hardware, quantization, and serving stack for running Llama 3.1 70B on a vessel with no cloud fallback. Real specs and real benchmarks.

Ethan Marsh·April 19, 2026