Orbs and Jellyware

Thorsten: I think we're all surprised by how far these agents can go in these boxes.

Quinn: Amp is now a distributed system.

Quinn: Shut your laptop, obviously, and it's going to keep running.

Quinn: Whenever we're doing something that might have a back-compat issue, first, have the agent introduce warning logs for when there might be some issue. Attachments used to be at unguessable URLs, and now you have to be authenticated. Maybe we didn't have all the right metadata stored from attachments nine months ago, so you warn if we would reject. And then you have the agent look at the warning logs over the next six hours and see what are the categories of attachments that don't have the right metadata, and how can we fix them up.

Quinn: While I was on these long flights, it started to gather all this information, and then I started to turn the vise, 'Let's actually enforce the authentication on this kind of attachment.' And the agent iteratively worked through making this safe. And so by the time I had it turn on enforcement, it had done all of the incremental steps, all the monitoring. It's the kind of thing that in the past would have occupied a developer full-time to be babysitting this.

Thorsten: Yeah. Yeah.

Thorsten: But this is wild. I think the first or second episode, we said like, 'The agents get so much better when they have feedback loops.' And now, we're talking about you deploying something to production,

Quinn: Yeah.

Thorsten: then using production as a feedback loop.

Quinn: Yeah.

Thorsten: Welcome back. Raising an Agent, Season 2, Episode 2. It's us in person in Munich. Um, the whole Amp team is here in Munich the whole week, right? We're hacking. The local dev env is dead, but local meetups are not.

Quinn: Yeah.

Quinn: Yeah, we had the whole team here, and we thought it was going to be this bustling scene, but then people cleaned it up behind us.

Thorsten: Yeah, yeah.

Thorsten: This is—this is not what it looked like. So, to recap, last time we said the local dev env is dead, right? We talked about Orbs, but the problem was last time we talked about a bunch of stuff that wasn't released yet, which was awkward. So, this was—we recorded three weeks ago, and since then we shipped—I wrote it down, this—I never write anything down. We shipped subscriptions, agent communication, so agents can send messages to each other, they can send files to each other. We shipped Puck, which is our meta-agent or the assistant in Amp. We shipped Portals. I mean, I guess you're in the process of shipping it right now. Multiplayer threads, Slack integration, secret handling, automations, schedules, webhooks. Like, we shipped a bunch of stuff.

Quinn: Yeah, we've been busy.

Thorsten: Yeah, we've been busy. Last time we said like, 'We think the local dev env is dead,' but to us, it is now dead, no?

Quinn: Yeah, we—we were using all these things ourselves, and when they all come together, then you don't want to use local dev. When you have the Portal and it is so reliable and you can just access a dev server and it's totally isolated, you don't have to worry about like ports or database contention or anything. And then, if you're doing it in an Orb, then if you can bring other people in with multiplayer, or you can have Puck go and figure out—like, I was just doing this thing where there's some threads with the—the work in progress, and I said to another thread, 'Go and get a screenshot from this thread and do it here.' And if that was running in local dev and my laptop was off or something, then I wouldn't be able to do that. So, it's all coming together.

Thorsten: Yeah.

Quinn: But we've got to do a better job of showing what it looks like when it all comes together because the dev workflow is changing more like in the last six weeks with this stuff for us than it's changed all of last year.

Thorsten: Yeah.

Thorsten: So, we're recording some other things here in Munich, and I interviewed the whole team, like everybody on the team in the last, I don't know, two, three days. We did a lot of like, um, short interviews. And everybody on the team, even people who are die—you know, hardcore, die-hard tmux fans, are now saying, 'I don't care about the local dev env anymore.' Like, this has come together in the last—you know, in my interviews, I said like, 'Oh, what has changed for you in the last four weeks or six weeks?' But it's truly been like the last three or four weeks, ever since we shipped all of this stuff.

Quinn: Yeah. And so, there's something about how we dogfood. We have 20 people on the Amp team. They are all experienced, they're opinionated, and everyone here is a co-founder. They're not going to go along with something they don't agree with. There's no bullshit here.

Thorsten: Yeah.

Quinn: And I think we've both been at companies, you know, even together, where getting people to dogfood the product can be hard. And so, there was a moment like six weeks ago when some of the team was really into Orbs, and other people, like Nicolay and Tim, they said—and Camden—were saying, 'These don't really work for me.'

Thorsten: Yeah, yeah.

Quinn: And what was so interesting is like they saw the kind of infectious enthusiasm that the other people had, but we weren't saying like, 'Oh, you must use Orbs.'

Thorsten: Yeah.

Quinn: We actually just, you know, waited a little bit, and then they all worked through for their own workflow how they can get more and more stuff into Orbs. But we were a little nervous for a while there. We were like, me, you, and Tim Lucas on the top of the Orb leaderboard.

Thorsten: Yeah, yeah, it's true.

Thorsten: Yeah, we had—I had an Orb that would query our production database and fetch the leaderboard of, you know, internal usage for Orbs. I think Tim Lucas was number one. You were—

Quinn: I was on vacation. That's why.

Thorsten: Yeah, you were on vacation. Okay, so then—but then it flipped. I was number three.

Quinn: I was still number two on vacation.

Thorsten: Yeah, exactly. But then it's like, then it dropped off. So, one thing be—before I get to the end, but so this—it was an Orb, it was a thread, and I had it query the production database, and it created this chart, and I would post it in Slack and say like, 'Guys, we've got to get the Orb usage up. Like, just try it.' You know, like if you have zero Orbs, you haven't tried it. I get, you know, just get 10, try it. And people were hesitant. So, then, this is the side note here, every day I would go in the same Orb and say like, 'Rerun the query. Rerun the query.' And it was like a tiny application to generate this chart. Like, it would have been something where it would save the thing.

Quinn: It was a tiny application, or what do you call—

Thorsten: Oh, jellyware. Yeah, yeah. We'll get to this in a second. But like, it is jellyware. So,

Quinn: Yeah.

Thorsten: I would run this every day, and the leaderboard would change, people would use it. But then I feel like something happened three weeks ago, four weeks ago, and now everybody's using it. Like, it feels like with what Camden did, they put the OIDC support into Orbs. That changed things. The Portals changed stuff. It became more stable and what not, and faster. And now, Tim Culverhouse is using Orbs. Nico is using Orbs. Um, Hamish for like the Mac app is using Orbs. It's wild. Like,

Quinn: Yeah.

Thorsten: And I—I ask everybody like, 'What has changed? Could you have predicted this?' And everybody's like, 'No, I—I thought this wouldn't work for me.' But now, everybody's like, 'I don't miss the local dev env anymore,' and everybody had their computer reset for—four weeks ago or something, or three weeks ago. I think it was even—maybe it was this. We had the computers wiped, we had to reset them all up, and people are like, 'I haven't even ported my dotfiles. I haven't set up anything yet. I haven't installed an editor yet.' So, now we're all fully in Orbs. It's—it's wild.

Quinn: Yeah. Yeah. And so, for Camden, he does a lot of our ops stuff.

Thorsten: Yeah.

Quinn: And one big question a lot of people have had is, how do you have an agent do ops stuff, especially if it's in an Orb?

Thorsten: Yeah.

Quinn: And actually, what we've come to realize, two interesting things. One is, it actually feels a lot more secure to have the agent running in an Orb with intentionally limited credentials to our GCloud, where it's read-only access, compared to being on a developer laptop, where, in theory, most developer laptops do have a way to break glass, and you might see like a password prompt, an auth prompt, and maybe that's something the agent did and it's escaping its containment. That—that is possible. And so, if you're in an Orb and it's doing ops stuff like reading the logs or reading BigQuery and it's read-only access only, that actually lets us sleep a lot more easily.

Thorsten: And one thing I noticed is, on my local machine I don't want them to install stuff. I don't want them to install with Homebrew like new packages or whatever. Like, it kind of I feel like it dirties up my system. But in an Orb, it's ephemeral, or at least like each agent or thread gets its own Orb. So they can install whatever the hell they want to get the job done. It's just an apt-get away. And that's, in hindsight, it's such an obvious unlock. You know, like, yeah, you get this you use this machine however you see fit, install whatever you think to solve this problem. And it's free to do this. Like, it's obviously the better security practice, I think.

Quinn: Yeah. Yeah, and you're so much more willing to let it do exhaustive work to prove that something is going to work. Like, one example is I had it run on a bunch of different versions of Node and Bun to make sure that it would work. And that's the kind of thing where, yes, in theory, there's probably some version managers where I could have it work in one of my worktrees on local dev, and it wouldn't conflict with anything else, but that stuff is so annoying. I just don't have to worry about it.

Quinn: Yeah. And one thing that we heard so much from our team is when the friction of starting a new thread got to zero, like basically zero, that was a meaningful shift from when it was like 1% friction. And no matter how good you make your local dev, it's impossible to eliminate all the friction. Even if you have like an awesome worktree script and it can, you know, create infinitely many, I mean, still your CPU is limited.

Thorsten: Yeah. Yeah, and you see you hear the fans turn on and port conflicts and all of this stuff. Yeah, yeah, yeah. That's one thing, I think Lewis pointed this out in our Slack like a couple weeks back. He said, "I think it might be the lack of friction that makes me create more Orbs." Like, and now when I read this, I'm like, huh, maybe. Like, it I couldn't have predicted it. I couldn't I guess I didn't say this the last time we recorded this episode, like, oh, there's so much friction with checkouts. But now when I do different checkouts, it feels like legacy. It feels like dirty. Like, uh, now I got like six different checkouts, and I got to manage the branches, and that's kind of weird, and then I got stuff lying around, and then it it feels dirty. And it feels like I CD'd into one of my checkouts yesterday, and I saw there's changed files in there. I'm like, what is this? Like, what did I do here? Do I need to revert this? What is this? It feels dirty. So there is some friction to it, and it seems like the last curve, or I don't know, like the last thing, if you take this away, then it becomes truly an unlock. And people, you know...

Quinn: Yeah.

Thorsten: I talked with everybody. I think everybody was talking about parallelism, but they also said like parallelism is not the right word to use. Like, it's something else where you, you know, parallelism you had with like six different checkouts, but now it's like you just send off these Orbs. It's so quick and so easy, and then they just stop, and they go to sleep, no cost, right? And then you go back to them whenever you want. And that's different than having these six checkouts where they kind of block future work, and you have to clean them up. So that might be another thing.

Quinn: Yeah.

Thorsten: But I want to get into one thing. This was also in the last three weeks. You started this. You you wrote in Slack, or you tweeted, I don't know. But you said like, "Well, in an Orb, the agent can do whatever it wants. It doesn't use up local resources. It doesn't block anything. And I'm not waiting for it." You're like, "I'm async anyway." And then you're like, "I ask the agent for 100% proof that what it did is correct." And let's talk about this. Like, what what do you do?

Quinn: So, okay, forget about agents. If you wanted to know 100% confidently that some change was correct, like, you know, asymptotically approaching 100%, how would you get there? And one way, in theory, is for whoever has sent you this diff to post a billion-dollar bond. And they would say, "If this is not totally correct, then you're going to keep the bond." Obviously, that, you know, that's a fantasy. You could have someone who you really trust on the team say, "I stake my personal reputation, my professional reputation on this being correct." And that's kind of how human dev works. But, you know, what does it mean for an agent to have a reputation? You know, it doesn't really carry the same kind of reputation with it.

Thorsten: Yeah.

Quinn: And what I have found really helpful is think about all the ways in which I can inspect the artifacts that it produces. So, you know, obviously, you want to look at the code, and if you see anything weird there, but then you have it record a demo. And if it struggles to make a good demo video, or if you watch it and you see that there's some jitter in the UI or there's something off, then that's that means that the time it takes you to verify it might be 7 seconds into the video, but it's worked for hours. And you're going to find that jitter way faster than any other way. Or you have it record a video and then, just from that video alone, then write the way to describe this to users. Because that kind of encapsulates all the information that will go into users, which is ultimately what matters. And all of these things that you would never want to do to a human because you'd just annoy the hell out of them...

Thorsten: Yes.

Quinn: And you wouldn't do yourself because they would be wasting time.

Thorsten: Yeah.

Quinn: And, um, another thing is just trying all the permutations. So, for us, I think we probably have a lot less permutations than a lot of other companies building a product, but we have it try an old CLI version, a current CLI version on, uh, sometimes different platforms, try the web, try the SDK, and try all these different permutations, runners as well, and make sure that that entire matrix is filled in and everything works.

Thorsten: Yeah.

Quinn: Or, you know, have a user who's on a subscription or in a workspace or individual or all of these things, and just seeing it work through that matrix. I feel bad for the agent because it's doing a lot of work, but it saves me so much time.

Thorsten: Yeah. I I mean, I wrote this, I think, like, 2 years ago or something. I wrote this post where I said like, "These are the basics. This is what I expect of everybody who I work with, an engineer." And one of the things I wrote down was before you ship something, you manually test it. And people frown. They were like, "If I tell people, have you tested it?" They go like, "I wrote unit tests," or, "I wrote an integration test." But that's not the same thing. Like, it's not it's not like you looking at this. And then if somebody opens a PR and then you go, "Have you tested this?" It's kind of like a con- I don't know. It's not, you know, like, people are like slightly, "What do you want from me? Like, of course, I tested it." But most of the time, people haven't tested it. And asking somebody to test something and is, you know, people don't like it. So then you say, "Can you make a demo of it?" And then people record a demo. And while they record a demo, they notice five things that are broken. Like, it's this is a thing. Like, once you try and do it end-to-end, you notice a lot of things.

Quinn: Or maybe you've tested it, and then you want to make one other little change.

Thorsten: Yeah. Yeah, yeah.

Quinn: And then do you really want to retest it again? If you have an agent, yes, you have it retest it all again. And it is so often that that one little change, our feeble human brains and the agent didn't realize that one little change broke it, but the agent doing exhaustive verification work uncovers that.

Thorsten: Yeah. Yeah, and I mean, that's the thing. Like, the agent does not get pissed off if I say, "Test this 10 times," or test- Maybe they do.

Quinn: Maybe they do.

Thorsten: Yeah, they don't. Maybe they do. They leave a little text file with, you know, the grievances.txt. But like, that's the thing. You ask them, "Test this end-to-end, however long it takes, I don't care." And they do it. And then it's like, you know, one objection was, "Well, how do you know it actually tested it? Does it not lie to you?" But I think that's like a trust thing. You have to just do it a bunch of times and then go, "Yeah, they don't do this anymore where they do expect true to be true in tests." Like, no, no, they actually test it. Like, they find a lot of stuff. And huge unlock. Yeah, huge. Like, yesterday morning, I was sitting here testing a big change, and the agent already tested it in that Orb. And then I was like, "It's a big change." I'm like, "Let me try this. Like, I want to test it locally." So I did it locally, and I ran into like a bu- not a bug, but like I ran into an error. And I started I did this three times. I pasted the error to the agent, and after the third time, I'm like, "What am I doing here? Like, this is dumb. Like, I I run through this, and then I give you the errors." And then I was just like, "Here's here's the errors." And I told the agent, "I'm sick of this. Go and test this end-to-end. Here's the API token, short-lived token. Do whatever it need- you need to do to test this." And it it I don't know how long it worked, but it worked for a long time, and now it works. And it's...

Quinn: And that's the mindset you have to have. It can be sometimes, frankly, tiring to always have this other loop in our own brains saying, looking at what we're doing and seeing is there a way to automate that. But I think it's really valuable for us to always have that in mind.

Thorsten: Yeah. One thing implicit here is I think we're all surprised by how far these agents can go in these boxes. This is no comparison to an agent using a browser even last year. It is much better. It's crazy.

Quinn: Yeah. Yeah.

Quinn: Okay, so there's another thing where I'm exploiting the fact that agents have infinite patience and also some of the things we've shipped. When you've got a distributed system, Amp is now a distributed system now that we have Neo and, you know, May and you can, you know, shut your laptop, obviously, and it's going to keep running. So we have to deal with back-compat and split-brain deployments. We have to do things in multiple phases.

Thorsten: Yeah.

Quinn: And people have a lot of data on Amp. We don't want to make it so you can't look at your old threads. So whenever we're doing something that might have a back-compat issue, yeah, what I have found really valuable is first have the agent introduce warning logs for when there might be some issue. Or for like attachments, for example, we made it so that attachments used to be at unguessable URLs in Amp, and now you have to be authenticated. But, you know, maybe we didn't have all the right metadata stored from attachments 9 months ago. So the first thing is like a tracer bullet. You warn if we would reject a user. And then you have the agent look at the warning logs over the next 6 hours and see what are the categories of attachments that don't have the right metadata, and how can we fix them up? And I was doing this as I was, I was on vacation in Mexico City, and then we flew to Munich, and that was running, and it was iterating, and I told it, "Only add warning logs, and keep adding them as you see more things." And while I was on these long flights, it started to gather all this information, and then I started to turn the vise a little bit, and say, "Okay, let's actually enforce the authentication on this kind of attachment here." And then, what I would do, because on a plane, you don't know if you're going to have good Wi-Fi, is first, I flipped the thread to multiplayer so that anyone else, if there was a totally unexpected problem, could jump in and see what was going on. And also, the agent would also be able to roll back that change. Now, this is a kind of dangerous thing, but it's for attachments, and it's something where if you were signed in, it would be no problem. So, it's like fairly low risk. And the agent iteratively worked through making this safe. And so, by the time I had it turn on enforcement, it had done all of the incremental steps, all the monitoring, all the like little baby steps toward tightening the vise. And this is something that, you know, took quite a while, but it's the kind of thing that in the past would have occupied a developer full-time to be babysitting this.

Thorsten: Yeah. But this is wild. I mean, we recorded our first, I think last year when, in April or something, and I remember we had, I think the first or second episode, we talked about feedback loops, right? And we said like, "The agents get so much better when they have feedback loops," meaning they run the compiler, or they run the linter, or whatever, or the tests, right? And it's like people didn't get this because back then, people are copying, pasting stuff from ChatGPT, and then they're like, "Isn't this the same as an agent?" It's like, "No, no, no, because the agent can run commands, it gets feedback loops, and then it iterates through the errors. You got to see this." And now, we're talking about you deploying something to production.

Quinn: Yeah.

Thorsten: Then using production as a feedback loop.

Quinn: Yeah. It's...

Thorsten: I mean...

Quinn: And getting production logs. You got to set that up. That's the OIDC stuff that we did in read-only. And I don't think that you would want your laptop to have a continually refreshed prod access token. That's probably right because it's your laptop, and you couldn't keep it open. In this case, I was on an airplane, and the Wi-Fi would have been bad. And you need multiplayer. You need team members to be able to jump in and see what you're doing. And actually, that's another thing. We experience the benefits of everyone at Amp. Yeah, we dogfood other things, but we're all on the same agent. And I think a big question for us is, for companies where agents are single-player tools, where everyone is using a different agent, some people, they love Claude Code, or they love Cody, or they love, you know, these other things, actually, there's benefits to everyone being on the same tool because if you're building software together, software is fundamentally a collaborative thing that you're building, then you're missing out on the ability to go and help in, you know, provide feedback to what other people are doing, nudge it along, or build on someone else's thread, or identify where you might be in conflict with what someone else is doing. And these are things that have been not even explored yet, and it feels like people assume that agents are single-player tools.

Thorsten: But I mean, we're on a higher, you know, we moved up. Like if the agent, I don't know, I don't know if we should say this out loud, but I don't think like, I guess we said like, "The coding agent is dead," back in February, right? But I think the whole like, you know, one person interacted with the agent, and then it's like, "Oh, you know, we're going to share the skills. Did you, what skill did you use?" Like, I think that's kind of like, it's not unimportant, but like, we're now at a higher level where now we're talking about patterns, you know? We're talking about, "How did you do this, Quinn?" And then it's like, "Oh, yeah, yeah, I spawned an Orb, and this is now the pattern that I use to verify stuff. You can also try this." And like, the agent is just a means to an end, which is this like figuring out what to build and how to verify it and how to share it. And now with, we have portals, you can have anything running in an Orb, we have a URL accessible to it. It's, and then you figure out like how you work with this, how do you work with Orbs, and that's now the big unlock. It's not, it's not, you know, I, I, I got to mention this like, Brett, for example, Brett is design engineer here, or engineer who does design, like, he does everything. And I asked him yesterday, like, "How do you, isn't, isn't design like a kind of thing where you want to be hands-on? Like, how do you do this in Orbs?" I'm, I, if I had to bet, if you had asked me six months ago, I would have said like, "The design will not be the thing that goes into the Orb," you know? Like, it's, it's something where I think you got to do Figma, whatever.

Quinn: Yeah, shorter leash.

Thorsten: Yeah, shorter leash, yeah. And he's like, "No, no, no, I'll, I'll just ask the agent to, if I want a new UI component, then I point it at other components, and I do like some screenshots, and I throw it in, and I say like, 'Do this and make this.' And then I ask it to create 10 variations and give me an HTML file, and then run it in a, in a portal, and then I can tweak and play around and toggle stuff, and then say like, 'I like this variation, let's do this.'" So, this is a pattern that I'm now going to use, and it's not, it's not about like, the agent doesn't, it will get the job done, it will do this, and there's not a skill or, or something. It's more about like, how do you work, you know? Like, how do you, like a higher level of like, how do you, if, if this is the task, how do you accomplish it? And I don't know, like, I think that's where we are now. Like, these, you have the verification thing, then Brett does like these components, then I think Tim Lucas does a lot of demo videos, and like, you know, these are the things that we now use. Like, it's completely, we're on a higher level of abstraction.

Quinn: Yeah. Yeah, one example of design is, um, Brett has been doing this, and Tim, and I've been doing this too, where when you have the agent change some like motion interaction,

Thorsten: Yeah.

Quinn: you can actually have it go and record a video of doing that, and you can have it record a video at like 120 frames per second. I don't know how it does it, but it's all like a virtual framebuffer, so it can do that. And then inspect every single frame to make sure that there's no jitter, there's no inconsistent animation. And what a lot of people asked for is the toggle sidebar button in Amp, to make it so that it stayed in the same place when it was showing and when it was hidden. And about four months ago, I tried to do that, and really struggled with the agent making that animation fluid. But now, when the agent was able to have that feedback loop of inspecting every single video frame, it got it, and it did it without any interaction from me.

Thorsten: Yeah. That's wild. I think, I think, we talk about this here on, yeah, the, the, you know, secret whiteboard has been flipped, but there's, there's two ends of the spectrum of software development right now. And on one end, there's traditional software development, and I think it's the old SDLC where one person writes something on their machine, and they might use an agent, but like the agent they might use as like a smarter typist or something, or like a autocomplete plus plus, you know, like close up front, and, you know, like, and then they push their changes, and then somebody reviews it, or they stamp it, and they go through all of this, and they might have like something about like the craft of software, and I still need to read every line, and, you know, all of this. And then on the other end of the spectrum, there's vibe coding, where, you know, it's a derogatory term, people say, "I don't ever read the code, it's blah, blah, blah." But then there's the middle part, which is I think what we are doing, which is Amp is polished, people love Amp, it's high quality, we have people talking about 120 frames per second and tweaking little jitters and pixels and like all of this stuff, and yet it's 100, 99 or 100% AI. And I think a lot of our customers who really thrive with Amp, or really thrive with AI, are in this thing. They accept that there's a new paradigm of software. And I think what you just described is this, what is this? Like asking an agent to record 120 frames per second and go through everything and fix the jitter, like that's ultimately like fully accepting that agents and tokens are here to stay, and this is the way to do it, and like leaning into this, and yet not lowering the bar for quality.

Quinn: Yeah. I wonder like what is the skill or discipline, that's not computer science, but there's something about the, a lot of people that do this in Amp where they have a software engineering background, where they've done all of this stuff manually, and they know what to do, they know how to inspect, they know how to push the agent. Like what, do you think that this, I don't know, requires that kind of background?

Thorsten: I think it, I don't know. I don't know, I think it requires like knowing what you want, you know? Like, it's this kind of like, you need to know about software, and you need to know about products, and you need to care about stuff, and then now you can go even further and set the bar even higher, you know? Like, but yeah, what is it? You need to have a technical background. Like, I think you need to have some sort of knowledge of this, you know, where like people think of, 'Wait, you know, what is it? I can record a profile, you know, to fix the jitter or whatever,' but you need to know what a profile is. But then now, it's like Tim Lucas would say, 'I just feed the profile to the agent and ask it to fix it,' which is like the next step. But he knows about profiles.

Quinn: Yeah.

Thorsten: Yeah. So, I don't know.

Quinn: How would someone learn to do this if they did not have a software engineering background?

Thorsten: I don't know.

Quinn: It's not, yeah.

Thorsten: Yeah, we got to rethink.

Quinn: It's like a real question for you.

Thorsten: Yeah, I don't know. Like, I think you got to rethink software from first principles and figure out like, you, you got to start with the end goal in mind, and then figure out what you want, you know, and then try and reach this and use agents or AI as much as possible. And then I think you will rediscover a lot of other things, and you will find out that you now look at them from a different angle. Like, it's not when you learn to build software, you're so stuck in the the how of building it. And and I think when now you don't have to worry about like the minutiae of it or like the little details or the syntax and all of this stuff, and and then you can take all of the methods that you might learn about and and and look at them from a different angle and use them and combine them more. But it's very strange.

Quinn: Yeah. It's tough because I still draw a lot on my background as a developer when I have the agent review the code and it makes, you know, I probably accept two-thirds of its suggestions.

Thorsten: Yeah, yeah, yeah.

Quinn: But the other third, it's stuff that I know will overcomplicate it or maybe it's not consistent with where I wanted to go. And I still find that on probably half of the types of changes I make, going and expanding the diff and actually looking at the diff,

Thorsten: Yeah.

Quinn: it's not that I'm reviewing it in the old way where I care about like the totality of the code, but I'm looking for like red flags,

Thorsten: Yeah.

Quinn: basically. And if I didn't have that intuition, I feel like I would be worse at using the agent.

Thorsten: Yeah. But I don't So, I ask everybody Well, I touched on this with a lot of people in these interviews. And everybody I ask, like, "Do you still review code? Do you still look at the lines of code?" And everybody was kind of like, "Yeah," you know. But they say like, "I still look at it," and they were describing the same thing. They they Everybody used the term architecture. "I'm I'm I'm concerned about the architecture." But I'm like, that's not really what this is. Like, it's it's might be architecture, but there's something else going on where you're just like you're trying to figure out like the the risk, the blast radius of changes, the invariants. Like, maybe that is architecture. I don't, you know, whatever your definition of architecture is. But when I think of architecture, it's like services and this talks to this and this. But there's something there where people now review things or what the agent does, and it's it's it's like high-level nudges. You know, I did something yesterday where I asked the agent to build like a new thing, basically a backend abstraction between how we run Orbs and, you know, whether we can switch out like a backend thing. And I gave it like a high-level description of what I want. I said like, "I want an HTTP interface, and then behind this, you know, it should do this, and then go and figure it out." And so I gave like a high-level instruction. It did it, and then I noticed that just by looking at the list of files that it changed, I didn't even look at the code. I only looked at, "Oh, it changed something in this directory. What is this?" And I said like, "Why do we You added a new subcommand to the CLI. I don't think we need this. I think it should do this." And then it went and worked for another hour and removed it. It's like, "Oh, yeah, you're right. We don't need this. Like, that's the wrong abstraction." What did I do here? Like, is it abstract Like, what is it? Is it architecture review? Like, it feels

Quinn: And if the Amp codebase is something that we've built up from the early days of an agent, if you didn't have that understanding of the codebase in your head,

Thorsten: Yeah, exactly. Yeah.

Quinn: Yeah.

Thorsten: Yeah, how would you do I think you still need that. I think you you still need like And it's the same thing. Like, we're going so fast now. Our rate of commits is, I don't know, tripled? I don't know. Like, it's it's a lot. We ship a lot of stuff. I mean, here, this list is eight items, and it's incomplete, and this is the last three weeks.

Quinn: I think you didn't even mention dictation.

Thorsten: Dictation Yeah, we let's talk about this in a second. But so, everything went up. Everything is faster. Changes are really easy to make. But at the same time, we also there were two months earlier this year where it was just me, and I said like, "Nobody else is going to touch this folder for the next four weeks." I'm in this. And I I basically looked at everything up close. I didn't write any code, but like, we made the foundation really solid.

Quinn: Yeah, I remember that. I think that was really important because, at the same time, I was doing some stuff on the web app, and so was Tim. And this was back in late February, early March, where the models weren't as good, and I don't think we knew how to work with them as well. And we had to actually back out a lot of the UI stuff because it was just slop on slop, and we didn't ship that to anyone.

Thorsten: Yeah. Yeah, it was slop.

Quinn: And so, yeah, in contrast, you did the, "Hey, let me build that foundation." And that worked.

Thorsten: And I de-sloppified. And it was not fun, and it wasn't easy, you know, like, really a lot of work. But now, I think that paid off. But now, when people ask me, "Well, you don't read any code anymore. Why?" It's like, we don't, but also we do. It's so in certain areas of the codebase. I don't even know what to say to people. And I think I said this on stage actually last week. I was asked like, "How do you You're not reviewing any code." And it's like, "Well, I think you got to get the foundations right, and you got to get like you need to figure out the architecture. What are the most important pieces of your application?" And then get those right, and make those invariants to the agent, document it in the AGENTS.md file or wherever. And and then you can build on top. And then the more you get to the outer edges of the codebase, the less, you know, like, who cares about the admin thing or whatever, you know? But yeah. Let's talk about dictation. It's uh it's uh we're mic'ed up here, but you're using this mic, and you you said like this was another big unlock for you in the last two weeks, no?

Quinn: Yeah. Well, yeah, using an agent is not about the skills, the techniques, the, you know, cute prompts or anything like like that for us. It's about getting what you want, what you know, out of your brain and into the agent. And also getting the ground truth. We talked about all those, you know, new feedback loops. And the fastest way to get information from your brain to the agent, the bits of information from your brain to the agent, is voice, I've found. Way faster than typing. Maybe actually video is even faster. I don't know, you know, if that's true.

Thorsten: That would That would be funny to do. Yeah.

Quinn: Yeah.

Thorsten: Here's a video of me.

Quinn: Yeah, we we can try that out. I actually saw a experimental feature that I think someone shipped overnight.

Thorsten: Oh, really?

Quinn: Yeah. We'll see. Uh and I started using Superwhisper probably about two months ago, and that was, you know, pretty good. Uh we brought dictation natively into Amp, and we did a little thing where it has the bag of words that your thread has used. So, it's got that good vocabulary that's automatic. And that means that it's just a little higher fidelity, and it's like a, you know, keystroke away. It just works. Also, like, it works on your mobile and everything just the same. And now, dictation accounts for probably 85 to 95% of my inputs, except when I'm in rooms and this room is filled with the other Amp team. But even then, I got the DJI Mic 3, and I just, you know, look around, make sure no one thinks I'm crazy, say, "I would like to..."

Thorsten: Yeah, yeah, yeah.

Quinn: And one thing is, yes, this helps you describe what you want, but it also helps me say all of these exhaustive feedback loops that I want the agent to do. And with those, you actually don't need to be so precise because it's not going to leave remnants of actual like production code. It might send it on a little bit of a goose chase, but it's going to figure it out. So, this has been a huge unlock, and getting dictation in Amp has turned on a lot of other people on the team to using dictation. But still, there's some people that basically never try it.

Thorsten: I need to try it. But I think what you're saying is I talked about this last week on stage at Laracon. I said like, the most important question you have to ask yourself when you write a prompt or talk to agents is where is the information supposed to come from? Meaning, where is the information of what I want or, you know, how is the agent supposed to know what I mean when I say, "Do X"? Like, where is that information supposed to come from? That's the most important thing. And I think the elephant in the room is a lot of developers are lazy when it comes to typing. And then it's like they try and like, "Fix the database," or whatever it is. And it's like, what does that mean? Like, this is not something you would say to another human, "Fix the database," you know, if it's not obviously broken. So, there's like this elephant in the room where people are lazy to type, but the and they do they have shortcuts and custom slash commands and and skills and whatever, but the fact is the most important thing you have to give to an agent is the information of what it's supposed to do. Like, that's it. And and and we're getting to this point where, you know, I've said this a thousand times, like, you have infinite John Carmacks, push of a button, ready to go, ready to code. They won't make mistakes again. What is it supposed to do?

Quinn: Yeah. Even John Carmack is going to struggle if you say, "Fix the database."

Thorsten: Exactly. Yeah, yeah, exactly.

Quinn: And he has no idea what you mean.

Thorsten: Exactly.

Quinn: And maybe there's something about voice where it feels like you are talking to a human more. Obviously, it's easier to say more stuff, but it it feels weird to just be like, "Fix the database. Over."

Thorsten: Yeah, yeah, yeah. Yeah, and it lowers the friction. Like, what else did we talk about? We, should we talk about jellyware?

Quinn: Jellyware.

Thorsten: Yeah. I said like, there's two ends of the software development spectrum, and in the middle, fully AI software. You know, somebody said like, what's softer than software? You know, like you have hardware, firmware, software, what's softer than software? And it's not vibe coding, and it's not vaporware, and it's not, you know, whatever. And, you know, we said like, jellyware. And there's something really interesting. I recorded a video about this yesterday, but, but I forgot to mention this. So basically, the idea is, you have high-quality software, and you fully lean into the fact that it's malleable and fast to change, and then think from first principles of what this means. I'll start with a little anecdote, which is, I had Amp on my phone build like a little ordering system, CRUD app. I, I took a photo of like the menu of like the different foods and beverages, and it, it built it. And then it built an ad...

Quinn: You were like volunteering at something over the weekend.

Thorsten: Yeah, it was a school festival. Yeah, yeah. And I, I took a photo of like the menu and I said like, "Build me a shop for this or like a calculator." And it did. And the funny thing is it built an admin interface where you had like the classic web forms and say like, "Add new product," or "Remove product," or "Change the price," or "Change the name." And I looked at it and I was like, "I'm not going to use the admin interface. What are you, like, I'm not going to use these forms. If I'm going to change what's in the application, I'm going to ask the agent. I'm going to take another photo of the menu and send it to the agent, say like, 'Make these changes.'" And I think that's this leaning into like, writing code now is faster than going to a web form and clicking five forms. The agent will get it right. So, that's one thing that's going on. And then the other thing I didn't mention in the video is, it's also accepting that bugs are now easier or faster to fix. Right? Like, it sounds like people will, you know, frown at this, but like, if you lean in, you can move faster because every change is easier, if it's broken, it's easier to fix, too. So, you can have a higher risk tolerance. And I think people have not fully leaned into this yet, you know, like, "Ah, we got to have 15 reviews." And I think the org also needs to lean into this, the processes need to change, and all of this. Like, yeah.

Quinn: Well, I think some of the most nightmarish interfaces that people use, like the expense reimbursement interface on Concur, for example, it's like, there are no bugs in that. However, it moves incredibly slowly and hasn't changed in 20 years. So, actually, people can tolerate, they want to tolerate, their revealed preference is to tolerate some number of bugs if the product is truly great.

Thorsten: Yeah. Yeah.

Quinn: And if you don't do that, your competitors are.

Thorsten: Yeah.

Quinn: And if now, that's always been true, but now it's 10 times faster to fix a bug, then that changes how you should be setting your budget.

Thorsten: Yeah. I agree. I think there's also this like software engineer circle jerk of like, "Oh, zero bugs." You know, the, the, what was it, Snow Leopard? Like macOS Snow Leopard, was it? I think it was, where they said like, "No new features, only bug fixes." And people still think of this and go like, "This is what everybody should do, only fix bugs." And I think, man, like what? Like, everybody accepts some bugs, and it's like, there's no, I don't know. If, if you have zero bugs, I guess you're moving too slow, is what I would say.

Quinn: The pain of a bug is how bad the bug is and how long it exists out there in production. And if it now exists way less time, because now when you report a bug, well, in the past, when you report a bug, usually, how are you even going to report it? You know, let's say you do report it, it's some email ticket, and you have to prove it to someone who doesn't actually understand the code, and then it gets to some product manager, they schedule it in one of the sprints, and it's probably not going to be this next sprint, and then they ship it, and then that takes a while, and then how are they going to know to follow up with you? But when we fix a bug, we just, we don't have any kind of sprint planning, and we're not the only company that's switched to this. We fix it immediately, we know to follow up, and if it's something that affects a lot of users, then we just have Amp go and figure out all the users that are affected and then send them an email giving them some credits or telling them about that. And by the way, I think people like that. But that, you know, that means that the pain of a bug is way less. And in some cases, it feels kind of like if we fix a bug and we overly self-flagellate and then we give free credits, it's almost like people know that we care, they know that they're heard, and then they send us more feedback to make the product better. So, it's this, this loop. And we would never have that if we said, "We're not going to ship anything that might have a bug." And also, no one would use our product because we'd be behind everyone.

Thorsten: Yeah. I, I mean, we do have a high bar. Like, I, you know, like, everybody is scared of shipping something that's shitty or sloppy. Like, we do have this.

Quinn: Yeah.

Thorsten: And then, I guess like, we...

Quinn: But the kind of bug reports we get, now, yes, to be clear, we have some just like, you know, bad bugs. But the kind of bug reports we get, they are things that I don't think people would report in other products.

Thorsten: Yeah.

Quinn: Because they know that they would just not, not care. Like, you know, "An older Android version in our progressive web app in this sub-interaction in the product is glitchy." People report that kind of stuff to us because we care and we can fix it.

Thorsten: Yeah. Yeah. You know, so, we work with one startup. They're really fast-moving. They're great. And you can call them at any time of the day. They're obsessive, you know, like Nathan and his team. And, um, we work also with other companies. And so, we often have like these Slack, shared Slack channels with the companies we work with, right? And it's, as a customer, as a customer, you report a bug to this and you report a bug to this other company. The experience of one company saying like 2 hours, 4 hours, 8 hours later, like, "It's fixed," versus nobody replying to you or saying, "We'll get this on the roadmap," or "It's in the backlog," or whatever. It's...

Quinn: If they say Q and then any number, like Q3, you know. By the time they fix it, we don't care anymore.

Thorsten: Yeah, exactly. You don't care anymore. And, and, I'm, the experience is, it's night and day. Like, the one, the one company like, "I want to give you more money," or "I'm happy to report more bugs." Like, "You care, you're fast to move. I'm going to report more things," which is, you know, maybe tricky, like, the more you fix bugs, the more you get reports. But it's just, completely different experience.

Quinn: Yeah. And I think if you're a company and you're not able to fix a bug in 15 minutes,

Thorsten: Yeah.

Quinn: then your competitors are. And you might not know that. You might say there's no way that we can change, but the, your customers are seeing that contrast.

Thorsten: Yeah.

Quinn: And that will ultimately push you to change.

Thorsten: Yeah.

Quinn: And for any company that is resisting or is slower in adapting to how agents can change how they work, they're going to hear that loud and clear from the market.

Thorsten: Yeah. And it's also, for the record, I think this whole, "Oh, we have more AI, so we have more bugs and like everything is enshittified." I hate the term enshittified. I think I've said this before on the record, but I think it's so dumb. What are you talking about? Like, there's not more bugs. Like, have you noticed more bugs in all, like, I don't think so. It's like people, it's insane. Like, it's, you can now more easily fix bugs. Like, it's insane to say like, "Oh, it's all slop code, there's more bugs." That's insane. Like, what you're talking about is people who have no quality bar, people who have no taste or whatever you want to call it, who don't know what they want. Like, I, I, I talked to somebody a couple weeks back and, and I'm like, "Look, like, hey, people say we have high taste, we have good product, it's high quality, and yet it's 99% made of AI. What does this mean?" It's not slop. And the guy was saying, "Well, I think slop is not AI, it's the absence of taste. It's the absence of having a quality bar. It's the absence of knowing what you want." Because if you know what you want, you can now go further and aim higher even than before. And that's what, it drives me up the wall when people say, "There's more bugs because..." Yeah, it's because you don't have a taste for good software. Like, you don't have a high bar. You don't care. You don't ship fast enough. If you can ship more, there's less bugs.

Quinn: Yeah.

Thorsten: And people need to optimize like their whole, if you have a backlog, fucking delete it. You start an Orb, start an agent for every bug that's reported, maybe do some light triaging, whatever. But like, it's now so easy to go so far and to push the bar for quality even higher. You, it's insane to not do.

Quinn: I mean, I think there's probably, there's always been companies where there is a reward for shipping new shiny stuff.

Thorsten: That's true.

Quinn: And there's not any incentive or reward for fixing or following through. And that remains.

Thorsten: Yeah, it's a company culture thing. I think everybody here is like, I mean, compared to other companies I worked at, like, even, you know, like, people here report when like the border radius is wrong or like some stuff doesn't line up in two pixels. And, and, I guess like, if you had told me like five years ago, "This is a bug report," people, "No, it's not. You know, who cares?" But I think people here truly care and then people are on it and then they fix it. And it's like kind of red alert if there's like two things, panels that don't line up and it's like two pixels off, and then everybody will jump on it and fix it. So, it's a company culture thing.

Quinn: Yeah.

Thorsten: But it's also like, we're a team of like, everybody has a different background, and I think like, if you had come to me with like a pixel, pixel type of thing like two years ago, I would have just said, "Ah, man, I don't know. Like, it's CSS, it's so hard. It's going to take me half a day." You know?

Quinn: Well, it might even be a product manager saying, "Let me think about how to prioritize that versus this other thing."

Thorsten: Yeah, maybe, maybe, maybe. Yeah, because it's effort, right? Like, it's like half a day of effort for me to fix like a CSS background border type of shadow, whatever thing. But now it's not. So, now I can fix all of this stuff.

Quinn: Yeah.

Thorsten: I don't have to know all of this, and I can see, I can judge the output. So...

Quinn: Yeah. And it might be that it works on desktop and it doesn't work on mobile. Mobile is where it's misaligned. And, you know, the company two years ago said, "We're not even going to prioritize mobile because we just don't have that budget." But now they would, and it means maybe on mobile there is that slight misalignment. For some users, they see it for two days out in the wild, or for two hours out in the wild, and it gets fixed. So, is that more bugs?

Thorsten: Yeah.

Quinn: You know, it's, it's hard to say that people are worse off because of that.

Thorsten: Yeah. All right, I think we, we got to wrap up. And I want to wrap up just as a reminder, like, what we ship. Again, this list is incomplete. We got subscriptions now, which means you can...

Quinn: Amp is way cheaper than ever before. You can use your ChatGPT sub, you can use your SpaceXAI sub, and that means with our subscriptions, you can get by for the whole month on 20 bucks a month.

Thorsten: Yeah.

Quinn: And we give really generous Orb credits. We want to make it so easy. We won't, we don't want you to have to think about Orb hours or Orb minutes.

Thorsten: Yeah.

Quinn: So, you can just go, go, go.

Thorsten: Yes. We have Puck, which is our meta agent, which is...

Quinn: The circle-y man that's...

Thorsten: The circle-y man. You might think he's unhappy, but he's not. No, he's not. Yeah.

Quinn: It's how he expresses himself.

Thorsten: But it's like your meta agent. You can ask it, you can talk... Well, I guess it's not released yet again. But like, you talk to it, you know, and can just say, 'Start a new thread, start a new Orb, find me, archive them.' Like, it's your meta agent in Amp. We do have agent-to-agent communication, which is, there's some wild stories in our Slack about this. So basically, agents can send messages, messages to each other and files. And that means you can now have Orbs that send files to each other, or an agent saying, 'I'm going to continue this in a different Orb,' and then it starts a thing and sends files to it, and then that sends back the result. It's...

Quinn: Yeah, that's how I made another thread to take these two other threads and make the news posts with screenshots from each of those. And each of those threads recorded their own videos and screenshots and sent them back. And Hamish has been using it for the native macOS app and iOS app that we'll have out soon.

Thorsten: Yeah. So, he starts amp no-tui, which is amp --no-tui on his Mac. That serves as a runner, which is not an Orb, but then he starts threads in the Orb, and they send a message to his Mac and say like, 'Use Xcode' or whatever, and, and, and, you know, 'Give me a screenshot of like the macOS app or iOS app,' and then sends them back. Um, we do have the portals, which we talked about. You can, anything you can run in Orb, you can access. Multiplayer, Slack integration, I think that's kind of, it's exciting, but it's like to us, it's already, I think, daily use. Like, people, we talk about some bugs in Slack, and then somebody tags Amp and says like, 'Fix this.' Secrets, we talked about, OIDC. Automations, you can set a timer, webhooks.

Quinn: Yeah, actually, the automations, that's how I say, after you've deployed this, then check every 30 minutes for any warning logs in the errors, you know, things like that.

Thorsten: Yeah. I have an automation that every day, it looks at our unpublished news posts and flags them if they haven't been released yet, you know? Like, it's, it's kind of like, yeah, it's good. Like, every day I get like a little thing that says, 'Do this.'

Quinn: And Kemper was just telling me he has something that every 10 minutes looks and sees if there's any emails to our email inbox that have not been responded to.

Thorsten: Oh, really? Yeah. Um, Hitesh uses it to check every, I think it's 30 minutes, Camden too, like to check for 500 errors on, in logs, or suspicious-looking logs, and then report them via Slack. So, he gets the agent to send him a Slack message when there's something showing up.

Quinn: And so, that's the example of a simple primitive, like a, a polling loop, you know, while, you know, and then sleep. And we also have webhooks, which I don't think you mentioned, and that's a way to, like, more granularly trigger or to get more information. Um, and that has its value, but it's a lot simpler just to set up something that polls.

Thorsten: Yeah.

Quinn: And I have not needed to think about what the agent is doing or how I need to tell it to set up what automation I wanted to do. And that is the holy grail. We've looked at other products out there, I won't name them, but the screen to set up an automation is a TPS report.

Thorsten: Oh my God. Yeah. And then, if I see YAML, it's over. Like, I'm not going to do it. It's, it's 2026. I'm not going to write YAML. It's insane.

Quinn: Yeah. Yeah. Automations, it's, it's...

Thorsten: Yeah, yeah.

Quinn: ...have the agent write the code. Agents are better at writing code.

Thorsten: Yeah. The other things, you mentioned primitive. So, the webhooks, Tim built this, right? It's basically, the agent gets, you know, has the ability to have a external URL, so where the outside can reach it, right? I think, I haven't checked, but I'm pretty sure yesterday I asked it to test this, right? And that means talking to another service, like a SaaS provider. And that means that thing has to talk to you to, you know, like, you have a callback. So, previously, we used ngrok or Cloudflare Tunnels, or whatever it was. Um, but the agent in the Orb can now use these URLs and basically say like, 'I got a dev server running,' then it talks to the SaaS provider, says like, 'This is my URL,' and then it talks back. Like, it's, it's like a local dev with ngrok, and it's also webhooks, and you can run them. It's crazy. It's like that one primitive of a URL, it, it unlocks a lot of stuff.

Quinn: Mhm. It's wild.

Thorsten: There's more stuff coming, um, macOS, iOS app.

Quinn: I think the big thing, we know, we hear from a lot of people, we need to show how all of this fits together. And, you can look for more streams, more demos.

Thorsten: Yeah.

Quinn: But we've gotten all of this stuff out.

Thorsten: Yeah, we need to do this. You did the streams. We have time capsules in our chronicle. You can look at them. They're really good. But basically, our biggest puzzle right now to solve is how do we like explain how we think about building software and how we use and leverage all of this? And I mean, this podcast is one form to do it, but I, I see it now as like the thing that I'm most excited about is like, we need to teach people how to rethink software from first principles and kind of educate them on how to unlock a lot of this stuff. We have a lot of features. Uh, now we got to show like what it looks like when you use them in the real world. All right.

Quinn: Yeah. That's it.

Thorsten: All right. Happy building.

Quinn: Happy building.