gpu-offload

让消费级显卡本地跑动 MoE 架构 30B~35B 大模型（以 Q4_K_M / IQ3_S 这类 4-bit 级量化为基准；Q2/IQ2 等低比特量化质量太低，不在本项目范围）：显存与内存混合承担（非专家权重上显存、占大头的专家权重留内存），速度远超纯 CPU；没有独显或显存不足时纯 CPU 兜底。免装 CUDA/Python/任何运行库，模型可替换。

Qwen3.8 Flash Next FP8 on 4x CMP 170HX: PP3 transformer stages with the 51B PLE table on GPU 3

Measured report + drop-in harness for running llama.cpp with the Vulkan backend on Android/Adreno (Turnip). GPU offload cuts CPU time 6.6x while token generation stays flat. Raw benchmark data, offload ladder, and a corrections log.

Bash script for 72B LLM testing on Tesla P40 with RTX 3050 offload

Proof repo: run a real LLM fully on-device in React Native (Expo + llama.rn). Offline, no cloud, sub-2s cold start.

Experimental portable GGML GPU offload for diffuse-cpp across Vulkan, Metal, and CUDA

Run LLM2Vec (Llama-3-8B) on an 8 GB GPU via RAM offload, and batch it without changing the embeddings (padding shifts them: cosine 0.76 vs 0.999956)