Jeff Keltner

One of my favorite podcasts these days is *The Future of Everything with Russ Altman. Professor Altman interviews people across Stanford University about their research. The episodes are short and cover a wide variety of topics - and I always come away having learned something interesting about the future.

For his 300th episode he interviewed Condoleezza Rice, Director of the Hoover Institute and former Secretary of State. She is also the Co-Chair of the Stanford Emerging Technology Review.

The interview was a great listen and covers a wide range of topics. And if you want more depth on areas of emerging technology from AI to Synthetic Biology to Space - check out the Stanford Emerging Technology Review.

I just finished the ~ 5 hour Acquired podcast on Rolex. If you can manage 5 hours (or ~3 hours at 1.5X), it’s worth a listen - even if you’re not a watch person. But my favorite insight came at the end when the hosts analyzed the business. They try to identify a key decision or risk that Rolex took that led to their outsized success. 

For reference, they are a relatively small player in the watch market by volume, but garner a huge portion of revenue and likely a larger portion of profits. Outsize success usually requires some sort of key decision or bet that was made differently than the industry. So what was it? But they couldn’t find any. All of the Rolex decisions were relatively rational. Nothing outlandish, no bet-the-company moments. 

So what differentiates Rolex from the competition?The combination of a consistent strategy, a focus on their core value proposition, and relentless execution.

I think the business world today too often over-values clever strategies and underrates the power of  a relentless focus on execution and continual improvement. If you want to win consistently over time, you should really prioritize the latter over the former.

At long last EU regulators are going to do something about the horrific slate of cookie banners that have descended on the web due to poor EU regulations (that have done nothing to protect privacy, best I can tell).

If you hate cookie banners (and who doesn’t), this seems like a clear win. But while the proposed solution (allowing users to specify their preference one time in their browser instead of on every site) will certainly improve everyone’s online browsing experience, it still doubles down on the horrible idea of asking regulators to play product manager.

It turns out that product management is hard. Many companies are quite bad at it. But regulators are usually worse. We would all be better off if regulators gave broad guidance and then let product companies compete and innovate to deliver the best experiences.

There’s a lot of AI news this week - but much of it kept bringing me back to the reality that ultimately models aren’t going to be defensible. That’s not to say that aren’t incredibly valuable and hard to design. But while models may ultimately create a lot of value, I think it will be hard to rely on creating a model to capture that value?

Why? Mostly because models have turned out to be somewhat undifferentiated. Every time a new model comes out with remarkable new capabilities, another matches it within a few weeks - often a more efficient model. Moreover, most models are now more than capable enough for the vast majority of every day use cases. User may prefer one model over another - but likely not enough to pay a premium for it.

So, who will be able to capture the value. I see a few potential winners.

1. Workflow Systems. Most LLMs are going to end up being utilized in context of a workflow. The systems that own those workflows (think EHRs in medicine or CRMs in sales) wil be able to capture a lot of value by orchestrating the right models and the the right prompts and their data. This could be existing players or new entrants.

2. Data Systems. For a personal agent to be useful, it needs access to your personal data. Companies with access to that data will be able to capture more of the value of enabling AI on top of it than those with just a model. Think Microsoft and Google.

3. User-Facing Winner(s). There is likely to be one big winner in the consumer-facing brand of AI. Just as Google became synonymous with search. Once that user habit is engrainged, you don’t need to be the best to maintain it. Right now, this looks like ChatGPT - though never count at Google (especailly given their existing search distribution). To win this war it’s likely you will need to build your own foundational model - but having a great model won’t be enough.

So, where does that leave the players in the space?

OpenAI is winning on 3 right now. It’s trying to tackle 2 through integrations. This framework would suggest they should lean into that hard and move fast.

Anthropic isn’t winning any of these right now either. This framework indicates they are not in a great position.

Microsoft has real advantages in 2 and to some extent 1. That may be enough to win large deals in the enterprise space. I don’t see much of a path for them on the consumer side (though they will try to leverage Windows for 2).

Grok isn’t winning on any of these at the moment. Neither is Meta.

There is a case to make that Google is doing well on all 3. GCP is a solid contender in enterprise data and workflows. Gmail / Docs has a lot of consumer data. And more people likely interact with Gemini through Google search than use ChatGPT. It does seem like Google has the most paths to success at this point - including their own model.

It seems there are more and more discussions of the possibilty of putting data centers in space. At first the idea seems crazy. I mean - why launch a bunch of computers into space on a rocket instead of just, you know, plugging them in here on Earth. But the more you think about it, the more sense it starts to make.

Data centers are challenging because they need physical space, energy, and cooling. It turns out that all three are easier in space! Space is naturally cooling, solar power is far more effecienty when being collected in space (and they can keep the satellites pointed at the sun 24 hours a day), and there is lots of, well, space in space.

I’m not sure that this will be the best way to operate data centers, but I love seeing new ideas being tested and innovation in the space. And a a scifi nerd, a data center in orbit just sounds cool!

One of the reasons I’m less worried about AI models leading to mass unemployment, is that I don’t think the models are nearly ready to take over actual jobs. All of our benchmarks look at small isolated tasks, and assume that getting better at these makes the models more capable at doing whole jobs. I disagree. And now there is some evidence to support that view.

Scale AI and the Center for AI Safety just released their Remote Labor Index. It used freelance projects on Upwork as a proxy for more realistic job needs and tested leading models to see if they could complete these tasks at a level a client would be satisfied with. Most models succeeded less than 2% of the time. That’s pretty different than the benchmarks we see for specific tasks or skills.

And I think that represents the best case scenario. Most jobs aren’t nearly as short-term as Upwork freelance projects - and don’t have such neatly defined inputs and expected outputs. That helps the team measure this concept - but it also means that 2% is more like an upper bound for the work most readily tackled by AI. It would be far lower in the real world.

If you’re interested in a more software engineering take on this argument, check out the recent post A Project Is Not a Bundle of Tasks by Steve Newman.

We would do well to focus as much on the diffusion of AI technology as the development of it.

Every time a new model comes out, there is a debate about how much smarter it is than previous models. Developers talk about all sorts of different tests and metrics to evaluate the capabilities of each model. And while we can argue about whether advances in capability are slowing down or not, it’s clear that we have mode remarkable progress.

It seems clear at this point that we are engaged in a race of sorts with China around AI. And, like the space race, I think it can be an incredibly powerful motivator - and I think it’s important to future of democracy and capitalism that we win this race. Which is why I’m worred we’re running the wrong one.

Much of our discussion about the AI race centers around either investment in building infrastructure or developing the most capable models - measured in either model capabilities or data center capacity. And this is an important part of the race. There is a good case to be made that we are largely winning in thise area (though I would love to see this competition spur greater investment in energy infrastructure in the US).

But it also misses a critical component, which is diffusion of AI technologies - how broadly we are taking advantage of these capabilities . It is far less clear that we are winning this part of the race. Often laws and regulations in the US prevent us from leveraging  many AI (or even just basic ML) technologies in critical areas like healthcare, financial services, or education. Having the best models is only useful if you can take advantage of them where it matters most.

So, as we debate how we make sure the most powerful AIs are made in America and how to regulate AI so that it can be developed and deployed safely - we also need to make sure we are thinking about how we enable reasonable deployment of these technologies broadly so that we can benefit from these amazing inventions.

Despite all the debate around Artificial General Intelligence (AGI), the concept itself feels misguided. It doesn’t reflect how AI models are actually evolving.

AI systems have jagged edges — brilliant in some areas, clueless in others. They can ace science Olympiads yet miss how many r’s are in “blueberry.”

So are they geniuses or idiots? Both. That’s why the AGI discussion misses the point. A model that still makes silly mistakes can still be transformative when used in the right context with the right guardrails. We don’t need “general intelligence” to get extraordinary value.

We should stop fixating on when AI will become “general” or “super.” What matters is what these models can do, where they fail, and how we build systems that amplify their strengths while containing their flaws.

I remain incredibly bullish on the future of AI and its impacts on the world and humanity. It’s not without risk - but the upsides are tremendous. Here’s my short version of the case for being optimistic about the ultimate impacts of AI in the world.

1) The doomers are overstating the case.

We’ve all seen The Terminator. I would be wary of giving an AI control over life or death decisions – but we do not seem to be close to either (a) giving over control of vital infrastructure or weapons to AIs or (b) AIs being capable enough to take control for themselves.

That isn’t to say that there aren’t concerns about AI - there are plenty. But I tend to think they are manageable. We’ve managed major technological transitions before, and while this one may be different, our starting prediction should be that it will look mostly like previous major technology revolutions. Disruptive, but manageable.

2) The upsides are more easily missed.

If the doomers are over-selling the case, I think many people under-appreciate the upsides we have. I’m most excited not about LLMs like ChatGPT, but what domain-specific models are doing across science, medicine, and material sciences. AlphaFold, for example, solved protein structures that had stumped scientists for decades, and DeepMind’s GNoME recently identified over two million new crystal materials that could transform batteries and semiconductors. The opportunity to accelerate discoveries and innovations across scientific domains is immense—and still largely under-appreciated by most.

3) BUT, we need to get the big decisions right

So, I think the downsides are over-played and the upsides are underplayed. But both require smart decision-making and policies. That means balancing the desire to prevent truly bad outcomes (think making bio-terrorism easier) while not stopping the positive one (like making it faster to find treatments and cures for diseases or better materials for building sustainably). 

We need regulation and policies in the goldilocks zone - not too little but not too much. That balance is achievable, though rarely linear. I’m cautiously optimistic that we’ll find sensible policy frameworks over time. In the end, my biggest uncertainty isn’t about AI’s technical trajectory—it’s about whether we humans can navigate the tradeoffs wisely enough to harness its potential without choking off innovation.

I promised my quick review, so here it is. I like it and it has become my primary browser. Though, given how much I like new toys, perhaps the better test if it still is in 3-4 weeks.

What do I like? Here are my top few items.

1) ChatGPT as default search as soon as you open a new tab is great.

2) The new set of options to switch to web, image, video, or news results is also great. This is now a real, functional Google replacement.

3) The sidebar is very useful. Having access to ChatGPT beside any web page is a game changer.

4) AgentMode with access to my. browser (so already logged into all of my sites/services) is much more useful. I’ve already used it many more times than the original version.

5) Cursor Chat, the ability to select text in any input field (like a Google Doc) and pull up ChatGPT inline seems great, but I haven’t really used it yet.

6) UX just feels simple, fresh, and sort of delightful. Definitely a nice change of pace from Chrome. Another example of OpenAI just having great product sense.

7) It seems like they are iterating quickly. There are already announcements of new features (eg open a Project in the sidebar) and some fixes (I can’t wait for the 1Password extension to work properly with the app). Seems like a good sign.

So, that’s the quick take. Very little I’m not enjoying so far. It’s been a nice replacement for Comet and Chrome for me.

If you’ve got other takes - or questions for me! - I’d love to hear them.