Gemma 4: Byte for byte, the most capable open models

Simon Willison’s Weblog Subscribe Sponsored by: WorkOS — Ready to sell to Enterprise clients? Build and ship securely with WorkOS. Gemma 4: Byte for byte, the most capable open models. Four new vision-capable Apache 2.0 licensed reasoning LLMs from Google DeepMind, sized at 2B, 4B, 31B, plus a 26B-A4B Mixture-of-Experts. Google emphasize "unprecedented level of intelligence-per-parameter", providing yet more evidence that creating small useful models is one of the hottest areas of research right now. They actually label the two smaller models as E2B and E4B for "Effective" parameter size. The system card explains: The smaller models incorporate Per-Layer Embeddings (PLE) to maximize parameter efficiency in on-device deployments. Rather than adding more layers or parameters to the model, PLE gives each decoder layer its own small embedding for every token. These embedding tables are large but are only used for quick lookups, which is why the effective parameter count is much smaller than the total. I don't entirely understand that, but apparently that's what the "E" in E2B means! I tried them out using the GGUFs for LM Studio. The 2B (4.41GB), 4B (6.33GB) and 26B-A4B (17.99GB) models all worked perfectly, but the 31B (19.89GB) model was broken and spat out "---\n" in a loop for every prompt I tried. The succession of pelican quality from 2B to 4B to 26B-A4B is notable: E2B: E4B: 26B-A4B: (This one actually had an SVG error - "error on line 18 at column 88: Attribute x1 redefined" - but after fixing that I got probably the best pelican I've seen yet from a model that runs on my laptop.) Google are providing API access to the two larger Gemma models via their AI Studio. I added support to llm-gemini and then ran a pelican through the 31B model using that: llm -m gemini/gemma-4-31b-it 'Generate an SVG of a pelican riding a bicycle' Pretty good, though it is missing the front part of the bicycle frame: Posted 2nd April 2026 at 6:28 pm Recent articles Mr. Chatterbox is a (weak) Victorian-era ethically trained model you can run on your own computer - 30th March 2026 Vibe coding SwiftUI apps is a lot of fun - 27th March 2026 Experimenting with Starlette 1.0 with Claude skills - 22nd March 2026 This is a link post by Simon Willison, posted on 2nd April 2026. google 400 ai 1941 generative-ai 1722 local-llms 152 llms 1688 llm 583 vision-llms 85 pelican-riding-a-bicycle 102 llm-reasoning 96 gemma 13 llm-release 186 lm-studio 18 Monthly briefing Sponsor me for $10/month and get a curated email digest of the month's most important LLM developments. Pay me to send you less! Sponsor & subscribe Disclosures Colophon © 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026