MicroLLM lab — tiny LLMs, Q4, in your browser   (styles.css)    Loading MicroLLM lab… Initializing WebGPU engine & model catalog…   On-Device AI Lab WebGPU · Zero-Server · Q4 Quantized  MicroLLM lab Run, benchmark, and compare Small Language Models (SLMs) directly in your browser with hardware-accelerated WebGPU — 100% private, zero server cost, zero accounts.  🎯 Why Small Language Models? Frontier models (GPT-4, Claude) cost millions to serve and add network latency. Compact models (25M–360M) operate as an ultra-fast, efficient edge layer:  Edge Computing & Privacy: Runs entirely on your phone or laptop. No prompt or user data ever leaves your device. Fast Triage & Task Routing: Classify queries, filter spam, and extract intent in milliseconds to decide if an expensive cloud LLM is even needed. Zero Cloud Cost: Infinite concurrency powered by client GPUs with no API bills. Ultra-Low Latency: Sub-10ms time-to-first-token for instantaneous autocomplete and real-time agents.   🔍 Technical Vocabulary Small Language Model (SLM) A compact neural model (~25M–360M parameters) designed for efficient edge intelligence and specific tasks rather than broad trivia. Q4 (4-bit Quantization) Compressing weights from 16-bit floats to 4 bits per parameter, shrinking memory footprint by 75% so 100M+ models fit in ~50–84 MB of browser memory with near-lossless generation quality. WebGPU The modern W3C standard API that executes compute shaders directly on your hardware GPU (Apple Silicon Metal, DirectX 12, Vulkan) right inside your browser window.    How to use this lab in 3 steps: 1 Load a model: Click Load on any model card below. It caches directly into your browser's private IndexedDB (no file download to disk).  2 Chat on-device: Select your loaded model and scroll down to the Chat panel to test prompts and watch live tok/s speed.  3 Benchmark & Compare: Switch to the Benchmarks tab to run objective speed/accuracy tests and generate your shareable performance certificate.    Load any of the models below to get started ↓ (https://stateofutopia.com/experiments/microllmlab/current.zip) Download zip (589 MB) (https://github.com/robss2020/microllm-lab) GitHub   Chat Benchmarks Compare & Certificate Custom eval    Active model  Loaded on this device: … (Saved in browser IndexedDB cache) Load all models Unload all    Benchmarking in progress… Preparing tests…         (Write a user prompt. Ctrl/Cmd+Enter to send.)  Generate max new (64)     Next action: Chat Benchmarks Compare & Certificate Custom eval    Objective checks (regex / exact tokens), not writing quality. A 135M model is allowed
              to fail — that is the measurement. Pick models, then run. Estimate uses your
              last tok/s if we have one.   Run suite on active model Run suite on every loaded model Run sustained speed test (256 tokens)  🏎️ Highest Peak Speed — Fastest single test  ⚡ Highest Sustained Speed — Continuous 256-tok decode  📊 Avg Sustained Speed — Across tested models  ⏱️ Total Benchmark Score — Cumulative suite wall time   Runtime per test (ms)    Accuracy per test      Next action: Chat Benchmarks Compare & Certificate Custom eval    Speed (tokens/s, sustained decode, suite wall) and accuracy (pass rate on objective tests) from runs
              in this browser. Numbers stay on this machine. Charts use the latest suite per model.  🏎️ Highest Peak Speed — Fastest single test  ⚡ Highest Sustained Speed — Continuous 256-tok decode  📊 Avg Sustained Speed — Across loaded models  ⏱️ Total Benchmark Score — Total execution wall time   Accuracy (latest suite)    Speed (mean tok/s)    Suite wall (ms)      Clear saved comparison  🏆 Verified Benchmark Certificate & Social Share Generate and download a verifiable performance certificate with your device hardware, peak and sustained tokens/second, and share your score.   Your Name or Handle (WebGPU Explorer) (Enter your name or handle)   Device & Hardware Information (Auto-detected GPU / Device (editable))   🎨 Generate Certificate    (MicroLLM Lab Benchmark Certificate)   ⬇️ Download Certificate Image (.png) 📋 Copy Image 𝕏 Share on X 💼 Share on LinkedIn 🔗 Copy Text Summary     Next action: Chat Benchmarks Compare & Certificate Custom eval    Write a benchmark in JavaScript The editor is eval() ’d in this origin, then each check runs on the
                model’s decoded text.   Example: object Example: function Eval and run    Prompt for a larger LLM Copy prompt    Function form   Next action: Chat Benchmarks Compare & Certificate Custom eval     Runtime Backend —  Model —  tok/s —  TTFT —  Wall —  Stop —  JS heap —  GPU buffers —  IndexedDB —   Adapter: not queried yet  Prefer WebGPU Falls back to WASM, then JS   Log    Inspired by (https://github.com/yangqi0/petitgpt) yangqi0/petitgpt · live (https://stateofutopia.com/experiments/microllmlab) stateofutopia.com/experiments/microllmlab · (https://stateofutopia.com/experiments/microllmlab/current.zip) current.zip (591 MB)  Serve over HTTP. Do not open as file://.