made by 0x1da49.com blog

← all reports

Top 125 Local LLM & Edge Inference Engines (May 2026)

repos

Access 125 local LLM runtimes, model quantization toolkits, Ollama extensions, and GPU inference engines released in May 2026.

Local LLMs & Inference — May 31, 2026

GitHub

Openpi-flash: Real-time inference engine for openpiHebbian-Robotics/openpi-flash

GitHub

Advanced Quantization Algorithm for LLMsintel/auto-round

GitHub

Secure-by-default Ollama Docker image with built-in auth, only 70MBhwdsl2/docker-ollama

GitHub

PFlash: 10x prefill speedup over llama.cpp at 128K on a RTX 3090Luce-Org/lucebox-hub

GitHub

SenseNova-U1 – Open-source unified understanding+generation model with no VAEOpenSenseNova/SenseNova-U1

GitHub

LLM-eval-kit: Distributed LLM evaluation framework (v0.3.0)benmeryem-tech/llm-eval-kit

GitHub

MemHub, Turn Your GPT/Claude/Gemini History into LLM-Wiki MindmapXTraceAI/memhub-llm-wiki-guide

GitHub

WebLLM is a high-performance in-browser LLM inference enginemlc-ai/web-llm

GitHub

Rotato – Node.js proxy that rotates LLM API keys on 429 errorsp32929/rotato

GitHub

NodeMind – binary document index, 48× smaller than float32 RAG, no GPU requiredQLNI/NodeMind

GitHub

Arknet – decentralized AI inference, fair launch, one binaryst-hannibal/arknet

GitHub

Apple's Sharp Running in the Browser via ONNX Runtime Webbring-shrubbery/ml-sharp-web

GitHub

Valkyr LM Inference with Realtime GuaranteesFoundation42/valkyr

GitHub

VulkanForge – 14 MB Vulkan LLM engine that runs native FP8 models on AMD (Rust)maeddesg/vulkanforge

GitHub

Llmconfig – configfile and CLI for local LLMkiliczsh/llmconfig

GitHub

Messy Model Bench Tests; Qwen3.6-27B vs. Coder-NextLight-Heart-Labs/MMBT-Messy-Model-Bench-Tests

GitHub

Train a LLM from Scratchraiyanyahya/how-to-train-your-gpt

GitHub

Smile-Serve – Inference Server for ML, ONNX, and LLMhaifengl/smile

GitHub

Open-source diagnostic for Al misalignment. Model agnostic, industry agnosticifixai-ai/diagnostic

GitHub

Train Your Own LLM from Scratchangelos-p/llm-from-scratch

GitHub

A complete Llama2 inference engine that fits in 1356 bytes of x86 assemblyrdmsr/sectorllm

GitHub

Cryptographic hashing as a transformer attention headffr1/unbounded-context-attention

GitHub

GPU-accelerated terminal environment in Zigsemos-labs/attyx

GitHub

A deterministic prime counting model with 0 error up to 10^10Zombieland123/bajak-equation

GitHub

Granite Switch - compose multiple LoRA adapters to one deployable modelgenerative-computing/granite-switch

GitHub

Appctl LLM tools for apps you haveEsubaalew/appctl

GitHub

Llama and Spec: MTP Supportggml-org/llama.cpp

GitHub

Gosd: High-performance Stable Diffusion inference in pure Go(no CGO)l8bloom/gosd

GitHub

A Model Context Protocol server for driving AmigaOS 4.1 machinesderfsss/MCP-AmigaOS4

GitHub

Describe what makes a photo "bad" and let a local LLM flag themiamnotagentleman/bad-photos-out

GitHub

Verdict – model evals on your own data, not someone else's benchmarkaevyraai/verdict

GitHub

cuda-oxide: a custom rustc backend for compiling GPU kernels in pure RustNVlabs/cuda-oxide

GitHub

DiffCAD, a FreeCAD workbench to review model changes like codeeblanshey/DiffCAD

GitHub

Local autonomous security agent powered by Qwen 2.5-7B on Kali LinuxXENOCOREGIGER31/local-model

GitHub

Diagrammer: Tell the model, get the diagram or the graphical mapIdraDev/Diagrammer

GitHub

A 429 from a quota cap and a 429 from rate-limit need different cooldownseleata/resilient-llm-router

GitHub

Nexa-gauge – Cache/cost-aware graph-based eval for LLM and RAGharnexa/nexa-gauge

GitHub

Meltdown: LLM Client Made in Python and TkMerkoba/Meltdown

GitHub

KillClawd – a sarcastic AI desktop crab by local Ollamaninjahawk/KillClawd

GitHub

AI doesn't know – it guesses. What if meaning lived outside the model?pekkalepola/colibri-clf

+ 85 more verified repositories in full dataset...

Download Repos Book — $19

Executive Intelligence Summary

During May 2026, the Repos tracking engine indexed 125 open-source repositories matching the Local LLMs & Inference cluster.

These projects represent active engineering teams and independent open-source developers shipping production code, architectural prototypes, and developer tooling.

Key Sector Telemetry & Maintainer Trends

Primary Topic Signals: llm (61), model (23), inference (15), gpu (14), local (11), agent (10)
Active Maintainers & Orgs: ggml-org (3), maddiedreese (3), kouhxp (2), achaljhawar (2), AssimilatedHuman (2)
Sample Records Displayed: 40 of 125
Remaining Available in Dataset: 85 records

3 Ways to Monetize & Leverage This Developer Intelligence

1. DevTool Sales & Technical Outbound

Reach out to repository creators who just launched tools in the Local LLMs & Inference space. Maintainers actively shipping code are prime candidates for cloud infrastructure, CI/CD automation, API credits, and developer tools.

2. High-Signal Tech Recruitment

Identify skilled engineers working on bleeding-edge projects. Sourcing talent directly from verified open-source releases gives you verified code samples and active commits.

3. Competitive Intelligence & Directory Building

Import structured repository telemetry into internal dashboards, market landscape charts, or developer directory websites to track where open-source velocity is concentrating.

Developer Usage Recipe

import pandas as pd

# Load Repos Master Index
df = pd.read_csv("repos.csv")

# Filter for May 2026 records in this sector
sector_repos = df[df["month"] == "2026-05"]
print(f"Loaded {len(sector_repos):,} open-source repositories from May 2026")

If you need the full dataset of 19,400+ developer repository records formatted for instant use in spreadsheets, pipelines, or databases, check out the Repos Master Dataset.

related reports →

2026Top 45 Local LLM & Edge Inference Engines (September 2026)

2026Top 158 Local LLM & Edge Inference Engines (August 2026)

2026Top 163 Local LLM & Edge Inference Engines (July 2026)

2026Top 121 Local LLM & Edge Inference Engines (June 2026)

2026Top 148 Local LLM & Edge Inference Engines (April 2026)