0:00 Brad The people that are going to excel are the engineers who have a team leader, tech leader mindset.
0:27 Scott Welcome back, everybody. We know you've been waiting for another episode of how many CTOs does it take? And in this occasion, this instance, my name is Scott Porad, and this is my trusty co-host.
0:41 Brad Brad Hefta-Gaub,
0:42 Scott But the question today is not how many CTOs does it take? The question is how many tokens does it take?
0:50 Brad how many tokens does it take?
0:54 Scott Brad, we were just chatting before we got recording here and I said we should be recording this.
1:00 Brad Yes, we should be.
1:01 Scott You got your cursor wrapped.
1:05 Brad My year
1:06 Scott I
1:06 Brad -end got review
1:06 Scott a
1:06 Brad is
1:06 Scott Spotify
1:07 Brad better.
1:07 Scott wrapped. You got a cursor wrapped. My Spotify wrapped says my Spotify age is 32. Did you get a Spotify I age?
1:15 Brad didn't get it. No. I don't use Spotify. My Spotify account is like the family account, so it's like random stuff that my wife and kids listen to.
1:26 Scott The sad thing about having your Spotify age be 32 when you're my age is at first you think, oh, that's cool. I'm listening to music 20 years younger than my age. And then you think, wait a second, 32 isn't young?
1:47 Brad You're a millennial. Your music tastes are millennial. Oh, man, that's
1:53 Scott Well,
1:53 Brad funny.
1:54 Scott tell me about your cursor wrapped.
1:58 Brad Well, I, you know, I. I was not necessarily proud of this. I was kind of like, wow, that doesn't that seems like a smaller number than I was expecting. But then I shared it with some of the devs that I work with and they were all like, what the hell? Anyway, 2.9 billion tokens in 2025. That's what I personally used within Cursor alone. 2.9 billion
2:20 Scott What
2:21 Brad tokens.
2:21 Scott is the cost of a token?
2:24 Brad Ah. That's a great question. And I'm an early enough adopter of Cursor that I'm on the legacy plan. I'm not going to call it the grandfathered plan. I'm on the legacy plan. And so I'm only paying 20 bucks a month. And like everyone else has to pay per token, but I'm like grandfathered in or excuse me, legacyed in.
2:46 Scott I wonder what the actual cost of a token is.
2:50 Brad Yeah. Yeah.
2:52 Scott Probably $10.
2:54 Brad No.
2:55 Scott I'm just kidding. I'm just kidding. But these companies are spending so much money. I read today that Microsoft spent $88 billion on AI last year.
3:05 Brad Well, that sounds about right. But then that means my 2.9 billion tokens are not nearly that much. They spent a lot more than that.
3:15 Scott Yeah.
3:15 Brad That's a lot.
3:17 Scott Well, that's cool.
3:17 Brad So, yeah, so I think there's a little, it's funny because I feel like even some of my top developers, so like one of the top developers I work with after I posted my $2.9 billion, he was like, he responded saying, oh no, I need to up my level because he'd only had 700 million tokens. So he was a little bit like, wow, like, you know, what's the map on that?
3:38 Scott I
3:38 Brad Four times more
3:38 Scott mean, this is an interesting thing because there's a lot of different ways one can. To interpret token usage. It's kind of like clocks.
3:48 Brad uh yeah maybe maybe it's a little bit like clocks i mean i would tend to agree that like as as you know you and i love to talk about engineering efficiency and and you know every time we talk about what's the right metric i always love to say it's clocks of course right because it's the worst measurement ever so i wouldn't be so i think it's a legitimate to say like uh what did you do with those tokens brad right like were there you know It's not a bad case to make that throwing around token numbers is a little bit like throwing around lines of code. So I think that's a fair point. I would say that like, you know, and we kind of hinted at this in our last episode of 2025,
4:35 Brad like, where do we think, what do I, what my opinion about where I think AI coding is going in 2026 and what, you know, I don't want to turn this into, it's not a predictions, you know, I don't want to say a predictions episode or anything like that, but I think that we're going to see a dramatic. um you know uh a k-shaped not a k-shaped economy but like a k-shaped reaction to ai coding tools right the the halves are going to grow faster and the have nots are going to fall further behind faster um and i think it's going to come down to that that central question of what you're you're scratching at which is it doesn't matter how many tokens you use it matters how effectively used the tokens that you use.
5:19 Scott I agree. And it might actually be an effective jumping off point for something that might actually be useful and practical to the 39 people who listen to this show, which is what are the things that the effective people are doing or are going to do that? will make them continue to be effective or more effective. And I have one thing in my list. I'm curious to know, you are much more of the power user here than I am. So I'm curious to know what you think those things are.
5:51 Brad You want to share your item that you think is the leading indicator of this?
5:56 Scott I see. two well actually i said one thing but i actually think it's two things the first is engineers are going to figure continue to figure out how to take large tasks and break them into smaller tasks that the agents can do one by one and i just i will come back to this like 50 times in the next year there was just this thing that kent beck it didn't just happen he put it on Probably LinkedIn is where I saw it, where he said, you know, 90% of my skills are becoming obsolete, but the remaining 10% are 10,000 times more valuable.
6:31 Brad Yep.
6:31 Scott And knowing how to break the work down to describe each step to the agent is one of those skills, which jibes with something I've said. Many times in this podcast, I'm sure, which is the advice I always give to engineers about developing their career is get to the point where you can write an implementation plan or text back or whatever you want to call it so that other people can understand your design and start collaborating with you on it is to me the most important skill to develop because then you're at a crossroads in your career. You can decide if you want to be a manager or if you want to stick with. being an IC, but until you can actually take requirements and translate them into work for someone to do,
7:18 Scott that's where you've got to get to. So that fits with one of the things I'm seeing is being able to break that work down. The other thing I'm seeing is AI is accelerating things. It's not making it free.
7:34 Brad Here
7:35 Scott And what do I mean by that? I'll give you two examples. A few examples. I'll start with one. I've probably talked about this in the podcast before because it came from earlier this year, an engineer who updated a lot of code, but it saved him tons of time. But he said it took him like four or five hours to get the prompt right, you
7:53 Brad Yeah, yeah
7:54 Scott know, of experimentation. And then it saved him probably weeks of time. We had another case that came through recently. an engineer they had about a little less than 2 000 tests and these tests were flaky and low quality for a variety of reasons and i mean he spent he spent an afternoon it was four or five hours he spent an afternoon setting up a process a set of steps for the agent to go through first by getting some third-party criteria for what what a good test looked like then evaluating each test and giving it a letter grade then asking the robot to go through and I've been doing a bunch of steps and ultimately having it refactor some of the things at the bottom or at the not at the bottom at the end again multi-step process learning how to take these big tasks and it took them a bunch of time and then an engineer I just talked to recently this I thought this was really interesting he created
8:55 Scott created an entire GitHub repo which consisted of mostly MD files.
8:59 Brad Mm-hmm.
9:01 Scott Just context.
9:02 Brad Yep, yep. This is a huge, by the way, you know, you talked about the notion of like what our best practice is. And so for sure, great documentation in an MD file in your directory trees, that is gold for the agents. They need that. Give them more of that.
9:22 Scott They needed this was not even really in the directory. free of the code it was separate he was trying to get you he was he is trying to get i'm just looking for here i had i looked at it earlier today he was trying to get um The robot to containerize something that's currently running on bare metal, like
9:45 Brad Uh-huh.
9:45 Scott an older application. And he set up a number of, and can we just tell you what these files are? He's got AI agent and Quickster Yep. MD implementation plan summary MD,
9:57 Brad Yep.
9:57 Scott containerization implementation plan MD,
9:59 Brad Yep.
10:00 Scott implementation plan checklist, containerization goals MD, migration guide MD. The previous containerization attempts MD, which because lessons learned from previous attempts, Yep. relevant confluence pages MD.
10:13 Brad Yep.
10:14 Scott kids have other repositories like
10:16 Brad Yep.
10:16 Scott and then you know just all just pages pages just describing all this context and what to do he's been working on this for like three or four days
10:24 Brad Yeah, yeah, totally, totally. So here's my, I'm going to just going to come right out and say the magic words that matter for all of this. And you know, I already said K-shape, K-shaped economy when it comes to dealing with AI coding agents. The haves are going to excel. The have-nots are going to fail faster and faster. And it all comes down to not only what you said about the concept of being able to really break down work, but I believe it's all about being a tech lead or an engineering manager. The people that are going to excel are the engineers who have a team leader, tech leader mindset.
11:09 Scott Mm
11:09 Brad But
11:09 Scott hmm.
11:09 Brad instead of leading other developers, they are leading the robots.
11:13 Scott Mm And hmm.
11:13 Brad so that's the money shot. That's the thing that matters the most. And so I will say I think the reason that I'm going to make the case that my 2.9 billion tokens are incredibly effective tokens because I was operating at a, you know. tech lead engineering manager architect level with with the robots for those 2.9 billion tokens so imagine that that was effectively you know 300 junior engineers that I had working for me and I managed them like junior engineers
11:45 Scott Yeah, one of the people on our team said you have to think about the agents as really eager interns.
11:52 Brad Yeah. And so what does that mean? So let's kind of go to the next level, right? So one, it means being really good at describing the problem space, right? Like you said, and I agree with you that like being able to break down the problem, explain the problem, capture the statement of the problem, that's what your engineer that's created all these MD files for this containerization project, that's what they've done. Right. Now, that's only the start, though, because if I just do that, but I am not good at evaluating progress and output. Right.
12:33 Scott Mm-hmm.
12:33 Brad The other thing that a tech lead or an engineering manager does is they have an understanding of what. of what right-sized progress looks like and what right-sized output looks like,
12:46 Scott Mm
12:46 Brad right?
12:46 Scott -hmm.
12:46 Brad So they can evaluate the technical output to say this is correct or this is not correct. But more importantly, or equally importantly, they're able to evaluate the progress against the goal as it's happening and course correct. Right. So what's the classic story of the really good engineering managers you've worked with in your career? They're the ones that get. compelling results out of all of their engineers. And they're never coming to you saying like, oh yeah, this engineer, you know, please get them off my team. They suck. They can't do their job, right? You don't ever hear that from the really good engineering managers, right? They're in there figuring out how to make even the worst performers perform well.
13:36 Brad And the reason they're able to do that is because they know what They know not only what good results look like, but they know what it looks like when the engineer is struggling and
13:50 Scott Mm-hmm.
13:51 Brad the AI bots struggle and they continue to struggle and they're going to continue to struggle. They're getting better, but they struggle. And so. This is why I think there's going to be a case-shaped reaction to more and more use of AI agents, and this is why measuring the number of tokens is a dangerous thing.
14:14 Scott Mm
14:14 Brad You'll
14:14 Scott -hmm.
14:14 Brad burn through a bunch of tokens in a world where you let the bots go off the rails, you're not good at evaluating their progress. They churn out. Slop, right? We all hear
14:29 Scott Mm
14:30 Brad the term AI
14:30 Scott -hmm.
14:30 Brad slop, right? And so whenever I see somebody on LinkedIn or on Twitter basically saying, oh, AI coding sucks, it just generates slop. My response to that is, I bet that person would be a horrible engineering manager. Like they only know how to evaluate their own work. They don't know how to evaluate other engineers' work.
14:52 Scott Yeah, well, there's two parts there. It's not just evaluating the work. I think the words I want to use to say evaluate the process. And what I mean by that is like, as an engineering manager, if I had an engineer. Who, you know, went for a couple of days of the easiest way to describe this would be if their tickets stayed in the same column on the board for a couple of days. It's like a sign. Actually, not even a couple of days, a day or two, you know, two stand ups in a row and like that ticket's still there. It's like, OK, that person's stuff. Something's wrong.
15:24 Brad Yep.
15:25 Scott Actually, I don't know what's wrong. Something's wrong. Right.
15:28 Brad Yep.
15:30 Scott And so it's not whether the work product that person was creating was good or not, but they're actual like they weren't making any progress.
15:38 Brad Yeah, exactly. And I think that that what I'm saying is that is a 100 percent happening with all the AI coding agents. And if you let them get over their over their skis, then and you don't and you don't course correct quickly, then they're burning tokens, they're wasting their time, they're wasting time, they're burning tokens and they're wasting your. in your time and they're increasing the likelihood that they're going to produce slop yep
16:02 Scott Okay, I have a question for you, something I have never tried or never even thought to have tried until now. If you have an engineer who's struggling with something, you say to them, you sit down with them, you talk through the problem, you can point in the right direction, you say work on this, you say let's check in after lunch, right? Let's see how it's going. Can I say to an agent, Like pick a time frame, 15 minutes,
16:25 Brad Yeah.
16:25 Scott 30 minutes. Hey, do this thing and report back to me. If you're not done in 30 minutes, just stop and
16:32 Brad Yeah.
16:32 Scott let's discuss.
16:34 Brad Yeah. So my quick answer to that is the tooling has been changing and evolving in such a way that the. I would argue that the development tool set companies, Cursor being a good example of this, they have recognized this challenge that you and I have talked about, which is that the agents can get off the rails if you give them too big of a task. And so they've been changing their tooling to do to essentially add those checkpoints. And you might recall like and if you go back into our old episodes, you might remember that I've said things like early on, like, you know. Nine months ago, one of my the first prompt that I would always give, I would say, do not write any code until you've told me your plan. Like I had explicitly say that to the agents and now cursor has plan mode. So you use plan mode instead.
17:27 Brad And so that's one of the examples of the guardrails that the tooling companies have put in to effectively give you that functionality that you're talking about. Like, hey, give me your thought, give me your implementation plan, check in with me before you move on. It's not necessary. I've not yet seen Cursor implement anything where the agent stops when it gets stuck and says, hey, I'm stuck. stuck. I would not be surprised if that doesn't show up soon because I'm confident that all of us are running into it. But as a simple example of this, I literally am. currently in the midst of burning some tokens on trying to solve this obscure like safari bug where it's like safari's use of audio and video is is is bad it's it's quite bad and um so there's some bug i'm trying to solve that has to do with you know default mac default audio device in safari person unplugged their airpods and it you
18:31 Brad goes crazy right and i've been working on this for a couple days which which effectively means i'm burning a you know millions of tokens on it and um the agent keeps coming back to solutions that i have already tried and it failed and it keeps and it keeps circling back right um and so this is a good example where i i fundamentally feel that if you're a developer who's not who would struggle at this themselves then you're going to be completely perplexed and you're going to be completely stymied by the agent running off and getting stuck like if you don't know how to unstuck yourself right then you're not going to know how to unstuck the agent right um
19:13 Scott Yeah, yeah.
19:13 Brad uh so The basic answer is yes, you can certainly include in your prompts. Hey, come back to me. Another thing that you can do, depending on the tools that you're using, like all of the tools nowadays are kind of showing their thought process. So if you're paying attention and you're watching the agent work, if you're sitting back and watching the agent work, you can see it thinking about what it's going to say next or try next, and they all have a stop button. button you know and if you're watching and it's basically you're going to start spinning off to do the same thing you got to hit that stop button and say and it's exactly what you would do with a junior engineer right you're checking in at stand-up and you're like hey you haven't made any progress on the ticket talk to me what's going on how are you what are you trying today you know and if they're basically
19:59 Brad telling you something that they talked about yesterday you're like that didn't work yesterday let's let's take this out of stand-up and let's go do a one-on-one real quick and we're going to try to figure out why you're stuck on this
20:10 Scott okay
20:10 Brad yes
20:11 Scott so to just recap and reset here a little bit if we're seeing k-shape for
20:17 Brad the yep
20:18 Scott upper branch decay i said breaking down tasks and paying attention to progress you sort of encapsulated that in the saying you sort of tech leader engineering manager skills what
20:29 Brad Yes.
20:30 Scott else do you think the people who are on the upper branch of the k are doing yeah
20:35 Brad Well, those two levers are pretty important, right?
20:37 Scott yeah
20:38 Brad They're pretty important.
20:39 Scott if all they're doing is that then they're probably pretty good
20:42 Brad Yeah, I do think that and we talked a little bit about this a couple episodes ago in the door report. I do think that there are very real. environmental attributes that make a difference. So, you know, kind of the do you have already good processes for code reviews? Do you already have good processes for change management, CICD? I personally believe that I'm a big strong believer. I'm a strong believer in linting, type checking. quality test harnesses right like the and so if you've got those things in place right then you can be a little bit more free to let the agents run and
21:25 Scott Mm-hmm.
21:27 Brad Because they can't do damage, right? Because you have other guardrails in place to make sure that they don't do damage.
21:33 Scott Mm-hmm.
21:34 Brad So I feel like that's another important point to make that, you know, how many times have you gone into an organization, right? You know, as a fractional CTO, you show up and you find out that like they don't have. They don't even have like high quality version control best practices or they don't have, you know, unit test best practices or they don't have, you know, linters or things like that.
21:55 Scott It's been a long time since I've seen no version control.
21:59 Brad Well, yeah, I mean. there's all kinds of different versions of weak version control right
22:06 Scott It's
22:06 Brad like
22:07 Scott true.
22:07 Brad your branching strategy can be you know can yeah i got it under version control but literally it's just like when we commit we commit it like put you know we don't we don't we don't know what we release we're we're saving it all in version control but we have no tagging to know what we've released or something like that
22:25 Scott Okay, here's another thought for the topic, okay? I think. A certain swashbuckler vibe.
22:33 Brad YOLO.
22:35 Scott I guess YOLO, maybe I'm not exactly sure. But there's a certain element where you have to not be afraid of failing.
22:44 Brad Agreed.
22:45 Scott Because I'm sure I would imagine for every 10 things you try to do with your junior intern robot. Some percentage of them, five of them, five of the 10 don't succeed very well, something like that. I don't
23:01 Brad throw
23:01 Scott know.
23:01 Brad them out oh
23:03 Scott Throw them out, of course. So it's a different attitude as an engineer.
23:07 Brad totally i think that again like so so so let me just play that back to you because i i think i agree with you very much that um if you have a strong competency in um evaluating correctness and evaluating
23:25 Scott Mm
23:25 Brad progress
23:25 Scott -hmm.
23:26 Brad and you combine that with a willingness to You know, the bots aren't going to complain that you gave them a bunch of tasks that you then just closed their PR without merging. You know, they're not going to get all sad because you didn't merge their PR, right? So if you're basically willing to give the bots tons of work to do and you have the ability to evaluate that. So like literally I just heard this interview with the CEO of Cursor, not just, but maybe it was a couple months ago, but he was basically talking about like. They realized that the job of the engineer has moved into it's all about code review, right?
24:05 Scott Mm
24:05 Brad So like,
24:05 Scott -hmm.
24:06 Brad when was the last time we looked at what a code review really looks like these days? And it's like, we all think of the GitHub, kind of the GitHub, green lines, red lines, you know, split view, unified view, but has have we have really stepped back and asked the question, what's the easiest way to evaluate and adjudicate quality code? And so they're doing some cool innovation in that space. case because they recognize that like how you know it's it's all about evaluation of evaluating the quality of the output right that's a huge piece of this um That's i
24:35 Scott CEO Krishnamay said that, but I feel like I said that first.
24:40 Brad mean it's not a it's it's i'm just gonna say it's it's a pretty straightforward idea
24:47 Scott You know, here's a question. It's a little bit of a change of topic, but it was already hard enough to be a good software engineer. Is this actually making it this, when I say this, I mean, is AI or AI-assisted coding actually making it harder?
25:06 Brad Man, I'm going to take the, I don't know, say the over or the under. I'm going to take the, look, I've already said this. I think it's going to be K-shaped. I think that for the people that are good at this job. they are just gonna crush it and
25:21 Scott You're really good. Yeah,
25:23 Brad the
25:23 Scott so you're
25:24 Brad people that are great and
25:25 Scott with me.
25:25 Brad the people that are weak are going to get crushed so uh here's a quick story like one of the teams i'm working with like like first time ever halfway through the sprint one of the engineers is like fuck this let's ship tomorrow like they're like let's start like we got so much shit done in the first half of the sprint let's let's let's like ship now you
25:48 Scott Yeah
25:48 Brad know and it's this sense it when you said swashbuckler that's kind of what i i'm hearing there like i've got another team that i work with where you know this one of the the team of one engineer is just like crushing it and he's like you know hey like like Like, I'm getting stuff done in an hour of what would have taken me a week before.
26:08 Scott Well, you know, it's interesting when I said squash buckler, there was an assumption in that statement that maybe I have wrong.
26:17 Brad Oh,
26:17 Scott My
26:17 Brad say it.
26:17 Scott assumption. So what I meant, my assumption was that engineers are choosing to use AI for some things, meaning use an agent to write some things and choosing to write it themselves for other things. And where the swashbucklerness came in there was saying, well, I'm going to go see if the agent can do this one
26:37 Brad No.
26:37 Scott instead of me. But you're saying 100% agent.
26:41 Brad 100% of the code is my top engineers. They are no longer writing code. 100% of the lines of code that are being checked in by my top engineers are not written by the developers. It is being managed and evaluated by the developers.
26:59 Scott Yeah
27:04 Brad I defy anyone to tell me otherwise. If you're still typing brackets and fours and ifs, you are getting left behind.
27:15 Scott So this raises an interesting question that came up this week. my company. I have just for some context, I have a weekly engineering roundtable that I host. And what that means is it started out as an office hours, the idea where people could come and ask me questions or whatever, but that didn't really ever work very well. So I mean, sometimes somebody would come and sometimes they'd have a question, but lots of times they just wanted to hear what other people wanted to have. that we're going to ask and so i changed it up a little bit and said okay we're just going to have this meeting and um anybody can bring a topic or i'm going to try to come with a topic or two every week and i'm just going to throw it out there and we'll just discuss there's no no we're not trying to accomplish anything we're just discussing and one of the topics that was discussed this week it's every tuesday it was discussed this week was in a world where
28:12 Scott The agents are writing most of the code. Does the quality of the code matter? like it doesn't matter if it's a big ball of mud if the agent knows how to change the ball of mud i mean the reason we don't like it to be a ball of mud i was more on the side of that it doesn't matter i think it does matter but i was more on the side that it doesn't matter because i was like look most of the reason we don't like spaghetti code is because it's difficult to change and difficult to understand but if the robot can understand it and can change it then what's the matter if
28:43 Brad Yeah.
28:43 Scott it's messy and and we don't like it because when we want to make big changes to it it's really difficult but if the robot can make big changes to
28:52 Brad Yeah.
28:52 Scott it like it's just still one prompt for us or whatever figuratively speaking and so It doesn't matter. This was intense debate, by the way, and I think maybe you're – I took that it doesn't matter. It sounds like you're taking that it matters.
29:05 Brad Yeah, I'm on the it matters. And I'm on the it matters for two reasons. The first simple reason is that as I just proclaimed that the top of the K are going to be all the people that are really good at evaluating quality and evaluating work product. And so it's still human beings that are accountable for the. For the code quality, I am confident that the answer is it's all about how quickly you can evaluate the content. So it does matter that it's be readable by you, the evaluator. Right. So that's reason number one. And reason number two, I think that in 2026, even I suspect between now and 2028, like, no, the robots aren't good at reading the big ball of mud. And they aren't good at the bigger projects. It's not there.
29:53 Brad Like all of our data is showing that we're starting to kind of. geek out right now like there's there's we've hit a plateau in sort of like frontier model performance um and um it's still quite good but it's kind of plateauing right now and i'm not seeing like a major breakthrough on the horizon so what that means is you've got something that's really quite good at writing things very quickly and now it all comes down to how quickly you can evaluate it so i'm going to say code quality matters
30:24 Scott Okay, so as I said a minute ago, just for the record, I'm not fully in team. It doesn't matter. But theoretically,
30:31 Brad That's your way
30:31 Scott I entertain the idea.
30:33 Brad of saying uncle?
30:34 Scott No, no, I'm just saying I want to be clear, but I do want to push harder. I do want to push harder against team. It does matter with this. Imagine, theoretically, you've got perfect and complete. test automation. Does it matter?
30:51 Brad If the code is quality, if the test automation
30:53 Scott Yeah.
30:54 Brad catches it.
30:55 Scott Yeah.
30:57 Brad Well, I mean, I've also asked the AI to go write tests for me. and i love that it can do that and it often does but guess what you better code review those tests i read i've had the ai right test that basically passed and tested the exact opposite thing that it was supposed to do it's like it's a little bit like i mean that but by the way you see that with with human engineers too right like oh i couldn't get the test to work so i changed the test when you look at it what you changed the test to actually be passing the fail case, not the...
31:32 Scott you're reminding me of this war story moment that happened a few years ago where i was looking at some code just about five years ago i was looking at some code that had been written long before i arrived this wasn't even at the company i met and i was prior company i was looking at this code that was there and i'm just i'm just staring at it and it's supposed to be doing one thing and i'm like I just think this is doing exactly the opposite of what it's supposed to be doing and I asked another person to come look at it and I was like could you just look at this like I just think this is 100% backwards and they were like yeah that's that's what's happening and I was like how how did this this code has been here for years and it's literally been like doing minus when it should be doing plus I mean like that was how simple it was right
32:22 Scott Right? And I was like, how did nobody notice this? And how did test pass? How did this whole thing get through?
32:30 Brad And what was the answer?
32:32 Scott Who knows? But I just remember staring at it. I just remember not being able
32:35 Brad There
32:35 Scott to believe
32:36 Brad were tests.
32:36 Scott my eyes.
32:36 Brad There were tests and it was passing the test. The tests were exactly wrong.
32:41 Scott Yeah, probably. I just remember staring at the code and just like not believing my own eyes, just being like. That can't be the way it is, right?
32:50 Brad I will
32:51 Scott Anyway.
32:52 Brad say, like, I do think. I you noticed that I said look you got to have you know good engineering practices that would be good for your humans as
33:01 Scott Mm
33:01 Brad well
33:02 Scott -hmm.
33:02 Brad and let the bots live in that space and I listed linting and type checking and and I will say that you know I uh Paul Brown uh one of the guests on our podcast recently uh he and I were talking about this months ago and he was kind of like Who cares about Lent? Why do you care about Lent? If the bot is able to write the code without Lent, then what do you care? And so I think it's fair that there's going to be a valid debate that's going to happen. I'm currently standing on the ground pretty firmly. I feel pretty confident. I guess I'm going to say this is a strong opinion, loosely held. And then if you can show me some data that I'm... Be happy to change my mind on this. But I think that it all comes down to evaluation of the code that's being written.
33:50 Brad And so we need to stack the deck in our favor as the human evaluators to
33:57 Scott Every
33:57 Brad evaluate.
33:57 Scott way we can. readability is one of those pieces all of it is all they're all parts of it
34:03 Brad By the way, my 2.9 billion tokens doesn't include all of the co-pilot code review requests that I click on. So now on, like all the all the teams I work on, it's like it's very clear that our practice is, you know, whoever writes the code. writes the code whether they use an agent to write or do they encrypt and decrypt the keyboard themselves the very first thing that happens is everybody clicks code review by co-pilot and then i can't count that like this you know engineers that i know that would have written the code that way written the code as best they could have the co-pilot comes back and gives some feedback and they're like oh that is actually a good thing let me consider that let me go change that code and so before we even have a human code review it it's like i won't even look at code that hasn't had co-pilot look at it now and then i will go look at the co-pilot comments and see like well were those valid things did the engineer did engineer make those changes and that gives me a sense of like that that's another piece of evaluating code quality
35:03 Brad Right. Like if the original implementation had a lot of sort of fundamental errors that were caught by the AI code review agent, I'm going to pay more close attention to the final product.
35:16 Scott I want to add one thing to this. We're talking about the ability to evaluate. We're talking about the intern, the eager intern and all these different practices. I'll throw one more little practice tactic into the sauce, small PRs.
35:33 Brad Yeah.
35:34 Scott I bet the agents are inclined. You can let the agent make large PRs, but we know I'm sure there's science behind this, but I just know it from. working with people if your pr is more than 300 lines it's hard for somebody to reason about what the heck's going on there when i when i see those prs that are 68 files and thousands of lines i'm like this is forget this right
36:02 Brad yeah totally totally well there's there's two things about that first of all let me say uh spiritually i'm 100 aligned with what you just said agree 100 um and i'm also a big fan of the 300 uh lines uh benchmark as a reasonable thing i will say that i have adjusted my mindset around that where it's 300 lines of functional code and
36:22 Scott Mm
36:22 Brad if
36:22 Scott -hmm.
36:23 Brad you've got if that 300 lines of functional code includes you know 1500 lines of of new tests written uh in the pr then i then i can i still consider that to be passing but you're right that i just got done saying how important it is to code review the tests so
36:37 Scott Yeah. Yeah, team, it matters. You need to code review the tips.
36:41 Brad Yeah, that's that's a fair point. That's a fair point. But by the way, by the way, I was going to say with one last point about your
36:47 Scott Please
36:47 Brad shape PRs, another good friend of mine who's like he's not a software developer, but he's a, you know, you know, an engineer mindset and he's been working on a project, you know, two guys in a garage type thing. And his co-founder was like ranting to him like you can't give me these 3000 line PRs and he's like, I don't. I don't know why not and I was like tell me what was in that 3000 line PR and it's like oh yeah I you know kept asking the agent for more and more and more and more and more and I was like okay that was your that was your mistake like you can't ask your co-founder to code review something that has 12 concepts in it that's that's where that was your original sin so
37:28 Scott Brad, it is a crazy new world.
37:33 Brad Yep, yep
37:34 Scott It's tectonic. It is in my entire career has been one way. If I think about the major changes in my career, which I'm not going to collect every one of them up here, but no SQL databases were a huge change. Cloud was a huge change. Um, I,
37:54 Brad The internet. I
37:55 Scott well, the internet, yeah, that was, that kind of started on the internet. I didn't really do pre-internet software. This just feels bigger than all of them. It
38:05 Brad agree.
38:05 Scott just feels bigger.
38:07 Brad I agree.
38:08 Scott It's changing the craft.
38:11 Brad I agree. I agree.
38:14 Scott It's crazy. It's blowing my mind.
38:17 Brad Well, there you go. 2026, K-shaped economy for AI coding.
38:22 Scott All right. Well, smash, rate, review, subscribe, do all that stuff, my friends.
38:28 Brad Yep.
38:28 Scott And Brad, we'll see you here next week.
38:30 Brad See you here next week.
38:31 Scott All right. Bye, everyone.
38:32 Brad Bye. This episode of How Many CTOs Does It Take Podcast was edited by Saffron Epigow. Our theme song was AI generated by Brad. Please follow us on all the social media platforms at at how many CTOs pod you can find us on YouTube LinkedIn Instagram Facebook threads X tick-tock and patreon or at our website at how many CTOs pod.com and remember to rate review and subscribe on the podcast platform of your choice thanks for listening
