Hey I'm [@naklecha](https://x.com/naklecha), the founder of [cloudy.so](https://cloudy.so): a research tool that designs experiments, learns from every result and ==keeps improving until your goal is met==. In its first week Cloudy served 20b tokens, hundreds of GPU hours and thousands of sandboxes, and ==more than half of the people who bought credits bought more within a couple of days== :) I'm clearing my calendar this week to talk to as many inference providers, kernel engineers, post-training and RL teams as I can. [Book a 15 min call](https://calendly.com/naklecha-cloudy/book) ## I'd love to chat if - You serve models at scale. Training runs end, inference runs forever, so your GPU bill is pretty much a function of how well you run inference. ==Every µs you save on a kernel can save you thousands of dollars on inference==. Cloudy is really good at writing kernels, see [[#Recent experiments I ran on Cloudy|the experiments below]]. - You want a speculative decoding draft model. In 2025 I was part of a small team that trained the one behind Groq's R1 70b (==1.6k+ tokens/sec==). I can help you train one, or Cloudy can do it. - You need a lot of tokens. I can help you generate trillions of tokens from whatever model you want, at ==the cheapest rate you've been offered, or lower==. - You want agents running your experiments. On Cloudy every subagent gets its own CPU or GPU sandbox and they all share the same storage, so it's easy to keep track of your runs. You can also plug in your own subscription so you don't pay for tokens. Happy to set it up with you. - You're stuck on InfiniBand errors. I've seen pretty much every IB error there is, and I'm happy to help for free. ## Recent experiments I ran on Cloudy ### Week 1: an inference engine faster than vLLM in 1,351 lines I gave Cloudy an inference engine called [SimpleLLM](https://cloudy.so/research/faster-than-vllm). 5,762 of its 6,747 lines were copied GPU kernels spread over 44 files. 24 hours on one H100 later, the whole engine is ==1,351 lines== with all its kernels in one 297-line Triton file, and it ==decodes 11–15% faster than vLLM== on gpt-oss-20b and 120b (and 1.4–1.8× faster than the original). | gpt-oss | Batch | Original | vLLM | SimpleLLM | | ------- | ----- | -------- | ---------- | --------- | | 20b | 1 | 264.8 | 330.9 | 381.3 | | 20b | 64 | 5,346.3 | 8,880.7 | 9,843.5 | | 120b | 1 | 183.6 | 232.9 | 267.7 | | 120b | 64 | 3,037.8 | 4,269.5 | 4,780.4 | Output tokens/sec at 1,000 tokens on one H100, median of 3 runs. ### Week 2: briefly #1 on the Open Kernel Challenge On the Open Kernel Challenge, getting into the top 10 is very doable: people sat at #3 and #7 on the muse_bf16_proj board after a single submission. #1 is a different game, because the top of the board was separated by a few microseconds. I started one Cloudy chat on Oct 26 at 22:27 UTC. It ran 464 experiments on up to 8 H200s, and ==71 hours later we were #1==, 23 µs ahead of the previous #1. | | When (UTC) | µs per step | vs SGLang | | ------------------ | ----------------- | ----------- | ---------- | | SGLang | | 4,830.8 | 1.000× | | #1 when we started | Oct 26, 22:27 | 2,951.5 | 1.637× | | ==Cloudy, #1== | ==Oct 29, 21:11== | ==2,928.1== | ==1.650×== | ![[okc-progress.svg]] It also showed that ==the more tokens we spent, the faster we climbed==. On Oct 28 Cloudy finished 120 experiments and took 40 µs off our board time. On Oct 29 it finished 191 and took off 108 µs, right at the top of the board where every microsecond is hardest to find. Cloudy manages all of this for you. It starts and stops the GPUs, runs your experiments in parallel, and you can ==track every experiment live in one place==, with its code, numbers and write-up. [Book a 15 min call](https://calendly.com/naklecha-cloudy/book)