Just-In-Time Coding

For better or for worse, JIT compiling has become dominant, i.e., the code only gets compiled during the execution of the program. AI can take this concept even further: The code only gets written during the execution of the program.*

This sounds outlandish today, but it's pretty close to possible: The average React button is ~100 tokens of Javascript/TS. With a Time-To-First-Token on Groq of ~200ms and a 500 tokens per second throughput, that's a 400ms wait time on button click (+RTT). That is slow, but not awfully so. Every button in Salesforce is slower.

And once you can JIT-code a button, you can soon do it with a page, and then with an application. If it can be done, it will be done and it will become the new standard – for better or for worse.

* Incidentally, that is how many startups work. The code only gets written when people ask for it.

August 19, 2024 – Max Rumpf