Inference - DEV Community

Skip to content

DEV Community

👋 Sign in for the ability to sort posts by relevant, latest, or top.

gxqwb

Nov 11

Test-Time Compute vs More Parameters: Where the Inference Budget Actually Goes

#ai #inference #reasoning #scaling

5 min read

Nov 11

Five-Axis Framework for Comparing AI Inference Chips

#ai #machinelearning #gpu #inference

13 min read

Nov 9

수락 길이는 22.8% 늘었는데 처리량은 14.0%만 늘었습니다: DSpine 카드 수치와 공개 파일 헤더로 투기 디코딩 초안 모델을 읽는 법

#ai #machinelearning #llm #inference

3 min read

gxqwb

Nov 8

Prefix Caching vs Recomputed Prompts: The Hit-Rate Break-Even

#ai #inference #performance #caching

6 min read

Nov 8

The Ridge Dies at 228 Tokens on a 4090, and KV Precision Cannot Save It

#hardware #performance #llm #inference

6 min read

gxqwb

Nov 7

Mixture-of-Depths vs Uniform Compute: The Routing Tax on Every Token

#ai #inference #performance #gpu

5 min read

Jonathan Murray for Backboard.io

Nov 7

12.8x faster for coding agents. At full precision.

#ai #llm #inference #agents

2 min read

devs

Nov 7

Hugging Face’s 70k-Word MoE Handbook After 800k GPU Hours — Why Sparsity Still Wins for Local Real-Time Tools

#moe #inference #localai #windows

4 min read

Nov 6

A 32B Model on an H100 Never Reaches Its KV Cache Crossover

#hardware #performance #llm #inference

6 min read

Nov 4

Why Pay for the Smartest Model Every Time?

#machinelearning #inference #efficiency #cost

6 min read

Jonathan Murray for Backboard.io

Nov 7

Nearly 5x the output from the same eight B200s

#ai #llm #inference #gpu

2 min read

Jonathan Murray for Backboard.io

Nov 7

Qwen3.8-27B: faster than vLLM and SGLang on one GPU

#ai #llm #inference #gpu

2 min read

Jonathan Murray for Backboard.io

Nov 7

Up to 95% more output from one GPU on Qwen3.6

#ai #llm #inference #gpu

2 min read

Nov 2

Should You Rent AI or Run Your Own?

#machinelearning #inference #cost

6 min read

云微

Nov 1

Can a language model run in Linux eBPF?

#ebpf #llm #kernel #inference

6 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.