Kernel Forge writes kernels for your model
Tuned to your GPU and checked for correctness.
Agents trace your model's hottest path, write Triton and CUDA kernel candidates, benchmark them on the target GPU and keep the fastest ones that pass correctness checks against a reference implementation. You bring the model, the GPU and a performance goal.
A real run takes about 3–4 hours of compute on the target GPU, so this page replays a sample run in about 75 seconds. Signed in? Kernel Forge is in preview in the Console as SoL Optimizer.
A Kernel Forge run in 5 steps
Read the model's architecture and measure the target GPU's memory bandwidth and compute, to find where the time goes.
The replay below shows all 11 stages of a sample run.
How a Kernel Forge run works
There aren't enough performance engineers to hand-tune every model on every chip for every workload, so Kernel Forge gives the search to agents: you bring the model, the target GPU and a performance goal.
- 1
Profile the model and the GPU
Read the model's architecture and measure the target GPU's memory bandwidth and compute, to find where the time goes.
- 2
Generate seed strategies
Rank ideas for the hot spots, such as fused attention, GEMM tuning, memory coalescing, kernel fusion and tiling.
- 3
Generate kernel candidates
Write Triton and CUDA kernels for each strategy, in several variants such as different block sizes.
- 4
Benchmark the candidates
Run every candidate on the target GPU across batch sizes and sequence lengths, against the baseline implementation.
- 5
Regenerate
Refine the leaders, from warp counts to tile sizes and register pressure, and write new variants from what the benchmarks showed.
- 6
Compile and verify correctness
Compile every kernel, compare its outputs with a reference implementation and check memory bounds and races. Kernels that fail are dropped.
- 7
Benchmark the winners
Package the kernels that passed, benchmark them on the target GPU and compare them with the baseline.
Replay a sample run
All 11 stages of a run, compressed from about 3–4 hours into about 75 seconds. Each stage shows how long it typically takes on a real run.
Llama 2 7B on an NVIDIA GeForce RTX 4090
The model and GPU are an example. Stage times are what a real run typically takes; the logs are sample logs.
11 stages that take about 3–4 hours on a real run, replayed in about 75 seconds.
Sample logs
One stage at a time, as the replay runs.
Start the replay to stream the sample logs for each stage here.Bring your model and your GPU
Tell us which model you serve, the GPU it runs on and the performance goal, and we'll talk through a Kernel Forge run with you.