made by 0x1da49.com blog

← all reports

Top 148 Local LLM & Edge Inference Engines (April 2026)

repos

Access 148 local LLM runtimes, model quantization toolkits, Ollama extensions, and GPU inference engines released in April 2026.

Local LLMs & Inference — April 30, 2026

GitHub

Xinity Runtime: Apache 2.0 LLM inference engine for on-premise deploymentxinity-ai/xinity-ai

GitHub

LLM Gateway – Simple API format converter for LLM providersskorotkiewicz/llm-gateway

GitHub

Darce – AI coding agent for your terminal, any model, 14 kBAmerSarhan/darce-cli

GitHub

New ML inference language dropped todaym0at/jizzrug

GitHub

AgentDesk MCP: Adversarial review for LLM agent outputs (open source)Rih0z/agentdesk-mcp

GitHub

Salomi, a research repo on extreme low-bit transformer quantizationOrionsLock/SALOMI

GitHub

Composo open-sources its LLM-as-Judge technique (83.6% on RewardBench 2)composo-ai/llm-judge-criteria-ensembling

GitHub

Music-bench: An LLM benchmark for reading printed musicjlebar/music-bench

GitHub

ESLint rules that teach LLMs to write better codepertrai1/eslint-plugin-llm-core

GitHub

BlackSwanX,174 AI agents predict the future by fighting each other,run on OllamaKalki-M/BlackSwanX

GitHub

Genesis Agent – A self-modifying AI agent that runs local (Electron, Ollama)Garrus800-stack/genesis-agent

GitHub

Any-GPU: my AMD 5700 XT OC can EARN IT's rent using Debian Trixie+Vulkancochranblock/any-gpu

GitHub

LM Gate – Auth and access-control gateway for self-hosted LLM back endshkdb/lmgate

GitHub

Framecraft – Generate demo videos from a prompt using HTML Canvas/Skill/MCP/LLMvaddisrinivas/framecraft

GitHub

OpenUMA – bring Apple-style unified memory to x86 AI inference (Rust, Linux)hamtun24/openuma

GitHub

Netflix Void Model: Video Object and Interaction DeletionNetflix/void-model

GitHub

TalkType – Offline Linux Speech-to-Text (Whisper, Wayland, AppImage)ronb1964/TalkType

GitHub

AI agent skills for affiliate marketing (Markdown, works with any LLM)Affitor/affiliate-skills

GitHub

Cursor extension to track LLM cache TTLagastalver/cache-timer-extension

GitHub

Standalone TurboQuant KV Cache Inferenceg023/turboquant

GitHub

What’s missing in this chat app (90% LLM cost savings)?20centAI/20centai

GitHub

Swift package AI inference engine generated from Rust crateondeinference/onde-swift

GitHub

TurboQuant model weight compression support added to LlamacppTheTom/llama-cpp-turboquant

GitHub

TurboQuant-WASM – Google's vector quantization in the browserteamchong/turboquant-wasm

GitHub

DocMason – Agent Knowledge Base for local complex office filesjetxu-llm/docmason

GitHub

DocMason – AI Agent Knowledge Base for local complex office filesJetXu-LLM/DocMason

GitHub

ShieldStack TS – LLM security middleware for TypeScriptshujaSN/ShieldStack-TS

GitHub

vLLM introduces memory optimizations for long-context inferencevllm-project/vllm

GitHub

Void: Video Object and Interaction Deletionnetflix/void-model

GitHub

All-smi: Command-line utility for monitoring GPU/TPU/NPU hardwareslablup/all-smi

GitHub

LLM inference load balancer optimized for AMD Radeon VII GPUsjanit/viiwork

GitHub

Rig: Build modular LLM apps in Rust – 20 providers, one unified interface0xPlaygrounds/rig

GitHub

Loqi, a memory system that preserves context after LLM compactionwf802222/loqi

GitHub

/Render – 3D Model Skill for Claude Codemfranzon/render

GitHub

We made significant improvements to the Kokoro TTS trainerBovineOverlord/kvoicewalk-with-GPU-CUDA-and-GUI-queue-system

GitHub

LLM Router – MCP server that routes Claude Code tasks to cheaper modelsypollak2/llm-router

GitHub

Agents.md – a schema standard for LLM-compiled knowledge basesarturseo-geo/llm-knowledge-base

GitHub

HTML to Markdown with CSS selector & XPath annotations for LLM Scraperlightfeed/scrapedown

GitHub

I built lightweight LLM tracing tool with CLISKE-Labs/lightrace

GitHub

Pscale – Logarithmic compression for LLM knowledge basespscale-commons/pscale

+ 108 more verified repositories in full dataset...

Download Repos Book — $19

Executive Intelligence Summary

During April 2026, the Repos tracking engine indexed 148 open-source repositories matching the Local LLMs & Inference cluster.

These projects represent active engineering teams and independent open-source developers shipping production code, architectural prototypes, and developer tooling.

Key Sector Telemetry & Maintainer Trends

Primary Topic Signals: llm (85), model (19), local (17), agent (13), inference (12), gpu (12)
Active Maintainers & Orgs: skorotkiewicz (3), model-vpr (3), vllm-project (2), huggingface (2), adrianium (2)
Sample Records Displayed: 40 of 148
Remaining Available in Dataset: 108 records

3 Ways to Monetize & Leverage This Developer Intelligence

1. DevTool Sales & Technical Outbound

Reach out to repository creators who just launched tools in the Local LLMs & Inference space. Maintainers actively shipping code are prime candidates for cloud infrastructure, CI/CD automation, API credits, and developer tools.

2. High-Signal Tech Recruitment

Identify skilled engineers working on bleeding-edge projects. Sourcing talent directly from verified open-source releases gives you verified code samples and active commits.

3. Competitive Intelligence & Directory Building

Import structured repository telemetry into internal dashboards, market landscape charts, or developer directory websites to track where open-source velocity is concentrating.

Developer Usage Recipe

import pandas as pd

# Load Repos Master Index
df = pd.read_csv("repos.csv")

# Filter for April 2026 records in this sector
sector_repos = df[df["month"] == "2026-04"]
print(f"Loaded {len(sector_repos):,} open-source repositories from April 2026")

If you need the full dataset of 19,400+ developer repository records formatted for instant use in spreadsheets, pipelines, or databases, check out the Repos Master Dataset.

related reports →

2026Top 45 Local LLM & Edge Inference Engines (September 2026)

2026Top 158 Local LLM & Edge Inference Engines (August 2026)

2026Top 163 Local LLM & Edge Inference Engines (July 2026)

2026Top 121 Local LLM & Edge Inference Engines (June 2026)

2026Top 125 Local LLM & Edge Inference Engines (May 2026)