Building products with LLMs

With the current AI landscape, there are a couple of things to consider:

• LLMs are powerful but commoditising fast.
• They're great at small, specific tasks but unreliable on long ones.

Each generation has improved, but the gains between releases are getting smaller. Below we can see the success probability of current AI agents being able not to mess things up given the size of the task. As you can see, most of them really drop after the 15-20 minute mark. This is due to the contextual windows and tokens. Eventually, you simply do not have enough tokens to play around with.

With this in mind, we can also agree that LLMs are really good at figuring out and solving small specific problems - for example, when programming, your job is now to just think hard about the problem you are trying to solve, and then communicate that so that AI is able to write your code. This allows you to do work at ~7-10x pace (this is what I calculated when looking at the work I used to do, vs work I do now with AI).

It seems that the optimal way to use these is for executing small tasks with precision. This can be anything from coding, reasoning, creativity.

These are "protocols" for execution.

If you think about it, even when running a gym, or doing real estate deals - there is some sort of a sequence of steps that you take, which are your protocol of execution in order to get the work done.

I separate execution protocols into 3 steps:

1. What needs to be done?
2. How do I do what needs to be done?
3. How do I execute what needs to be done efficiently?

AI can currently output 100x quicker on the "how do I do" and "how do I do it quickly", but "what needs to be done" is what humans are still better at. Even if we delegate this to AI, humans' unique insights are what ultimately decides on whether or not that is correct.

Your decisions in business and life are based on our experience, gut feeling and the rest of the knowledge that at the time is available to us. AI is not able to get there, or at least not with the current tech. This isn't my hypothesis, but something a lot of people that I worked with and follow think as well.

This is where I believe the real leverage / moat is.

I think that building out tools is going to become obsolete because everyone will be able to build them. Development cost will literally go down to 0.

What no one can teach you, and what AI is not able to do, is duplicate your own experience and tell you what is the best to do for your individual case.

So if you have your own protocol - like launching a funnel, writing an ad or making a business decision - you simply figure out what to do, how to do it, and then do it efficiently.

What you are trying to do here is separate things between: "this is what I need to think hard about", and "this is what I am simply executing".

Then describe that process as a protocol - this way you eat complexity away with AI (by this I mean automate the stuff that is simply execution).

Then, instead of building a funnel in a week, you do it in an hour.

You now are able to scale up your service-based business 1000x simply by the way you operate. And instead of building a "tool" which you will sell to others - you use it as a competitive advantage to scale up your own. So the business is not now AI-enabled, but is AI-native. No one cares about how you do things; customers would just care that you can do things a lot quicker and cheaper than your competition.

With this, "you" become "the moat". This is what enables you to win because you are the best at one thing, which is being you.

This allows you to protect your own protocol, which is in reality unique insight. So your solution is not to enable others to do what you can, but to leverage your own AI to win.

Core idea: build tech that productises unique insights.