0:00 Brad Well, I think the solution to that are the right guards in the software factory. And the right guards aren't humans. Like humans are never the right guards. Welcome, everybody, to another episode of the How Many CTOs Does It Take podcast. My name is Brad Hefta-Gaub, and my trusty co-host is...
0:33 Scott Scott Porad. How are you doing, Brad?
0:35 Brad I'm doing great. You just said to me that you had something interesting to talk about, and before you could utter another word, I clicked the record button because... I'm champing at the bit to hear
0:46 Scott Well,
0:47 Brad what
0:47 Scott I'll
0:47 Brad exciting
0:48 Scott tell you, things I think
0:48 Brad you want
0:48 Scott what most
0:49 Brad to talk about.
0:49 Scott of our audience, all 40, 42 of them don't know is often we get on here and we start chit-chatting and we're about 15 minutes into it and we're like, oh, why didn't we hit record on this? This is interesting. Okay, I want to talk about the fear that's coming along with AI right now and
1:06 Brad Oh, yeah,
1:07 Scott coaching people through that. I'm in particular thinking about an engineering manager who I spoke with recently, but actually there was also an engineer, an experienced engineer guy probably in his mid-40s who's been doing this 20, 25 years. He's at a big, one of the big co's. I saw him like in a backyard barbecue a couple of weekends ago. He's very fearful. He's just like, man, I just hope I can hold on until I retire. Like I'm hoping I can squeeze another 10 years out of this. this career before the robots take over but that's I'm talking about him as interested but I want I'm more interested in exploring the engineering manager part of it and
1:48 Brad Yeah.
1:48 Scott how we should as CTOs and engineering directors engineering leaders should be coaching our managers to deal with this moment I'm going to put a name on this moment I'm going to call it an adaptive moment Let me explain what I mean by that. There is a book called Adaptive Leadership by a professor from Harvard. His name is Ron Heifetz.
2:16 Brad Mm And-hmm.
2:16 Scott adaptive leadership, this is how I'm not an expert in adaptive leadership. I've been introduced to it. And but this is how I interpret it. In general, as engineering leaders, our job has been to take chaos and organize it into some sort of order so that we can get things done efficiently. But there are times when we must exhibit adaptive leadership where the world around us is changing and we must adapt to it. And then as leaders and managers, we actually don't need to stabilize the situation. We need to disrupt the situation. And so I was talking to an engineering manager recently. who is frustrated about some of the disruption that AI is creating in our organization. He's like, if we have these processes and these procedures and the way we do these things to sort of, I don't know if this is the right word,
3:10 Scott but to protect our organization, to keep things stable, right? So that we don't ship bugs or so that we don't. create problems we don't create problems right and we've spent our i spent my whole career trying to build all those systems up but now there's this huge thing out there ai and we need to adapt right and the threat is not the ai the threat is not changing Right, because it's here, it's coming, it's coming. So I don't know, I'm just thinking through how do I advise and coach and help people
3:47 Brad Well, yeah.
3:48 Scott like this guy and others navigate this moment.
3:53 Brad I hear you. I hear you. Right. Many things kind of popped in my head as you were talking about this. And I'll just start by saying, you know this, the listeners of the pod know this, that I am very bullish on the following applications of AI. And I say it this way. If you have a strong SDLC already, if you already have a software factory prior to like prior to us talking about dark factories and AI, we all were building software factories. That's what you and I have spent our careers doing. Right. And so those things that you're talking about around. what this this engineering manager described as like these systems and scaffolding and stuff to kind of keep you safe that's a software factory right
4:42 Scott Mm-hmm.
4:42 Brad and as you know i'm a huge proponent that if you already have a strong software factory then adapting it to ai is not necessarily easy but you will get The true benefits of the AI out of it if you already had a good software factory. Right. That's kind of been my position. And I've said SDLC instead of software factory. But like, you know, by that, I mean, like you already had an SDLC, that SDLC already had reasonable ways of describing specifications and coming up with implementation plans and implementing them and implementing automated testing and having automated testing associated with your CICD and you had great observability and you had the ability to sort of have those feedback loops around.
5:28 Brad you know, release to observability to feedback loop, like that's all just software factory stuff that you and I have spent, you know, our careers helping teams build, right? So the first thing I would say is, I agree with the idea that a big part of being an engineering leader for the last several decades has been about building software factories to help us be safe.
5:52 Scott Mm hmm.
5:53 Brad So I was going to posit that. And I feel like the answer that I would give to anybody who's like, gosh, I don't know what to do with all this AI is I just feel like I lean into like, yeah, that's why we had a software factory. We have our software factory. Let's adapt our factory to this new world.
6:09 Scott Yeah. So let me put. A finer point on where I think the unsafeness, the unsafe feeling was coming from.
6:17 Brad Okay.
6:17 Scott I think it was coming from a few places. We didn't have this part of the conversation I'm just reading between the lines down.
6:24 Brad Sure. By the way, asking like, what do you feel unsafe about? That's like the next question I was going to ask anyway, right? Like, where is what's the fear? What's the fear coming from?
6:33 Scott I'm going to be responsible for maintaining code that someone wrote because it's easier for somebody else to come stick their nose to have their robot come change the part of the system I'm responsible for. um i'm going to be responding that that's somebody responsible for that so what does that mean i'm going to be responsible for maintaining it for bug fixes for on call if there's problems there's fear that the design that maybe if somebody's coming coming in changing part Okay, let's take a step back. The AI is making it easier to change the system more broadly because you don't have to have the tribal or institutional knowledge about as much of the system as you needed to have before because the AI can figure it out for you. So I think all of its concerns were around,
7:21 Scott hey, there's this part of the system I'm responsible for. I'm responsible for the flux capacitor service. And Now people can come and just change this flux capacitor service. And I have fears about that. I have fears about being responsible for code my team didn't write. I have fear about, you know, being on call for that code. I have fear that that code will be poorly written and it doesn't maybe fit with the architectural vision. where my team thinks this thing is going in the future so i don't think the fears were actually about like the factory processes of like test automation and code review and things like that it was like actually some fears that were around that if that makes sense
8:04 Brad Yeah. Well, OK. First of all, I love that characterization of the fears. And I will say. That's got nothing to do with AI. Like those fears
8:13 Scott Okay.
8:13 Brad already, those fears, like literally you didn't even use the word AI to describe the fears. You said someone will come into my code and change my code and I'll be accountable for it, right? Or someone will be able to make changes that are kind of across multiple systems and when they touch my system, I'll be accountable for it. Well, that already, like that already was a problem that existed in software engineering. hearing before AI.
8:38 Scott Yes, it was. I think what AI has done to use an expression has lowered the barrier, right? Before it would have been like prohibitively, I'll use the word expensive, but I really mean prohibitively complicated for me to go like, let's just say you work on the commerce system at our little mini company that we are a little imaginary company we have and commerce is very complex. And so I would say, oh, I'm not going to go splunk around and that thing is complex. I'm going to go over to Brad and have his. have his team work on it right but now ai is like lowered that barrier
9:06 Brad Yeah,
9:07 Scott and so it makes it easier there's like an implicit barrier
9:11 Brad I
9:11 Scott there yeah
9:11 Brad think it was episode two of this podcast. It was called the Ball of Mud. And in that episode, we talked about how One of the techniques that software teams use to prevent the tragedy of the commons, which
9:26 Scott Mm
9:27 Brad is what
9:27 Scott -hmm.
9:27 Brad causes the ball of mud, is like putting up gatekeepers and forming teams around core components and saying like, no one's allowed to come touch my component if you want my component changed. then turn in a ticket and we'll we'll change the api or something right
9:45 Scott Yeah, yeah.
9:46 Brad right so i just i just remind us of that because that right there was a was part of the like oh i'm fearful of being accountable for
9:58 Scott Mm
9:58 Brad something and
9:58 Scott -hmm.
9:59 Brad someone else changed and so how do i address that fear i put up gates and i put up
10:04 Scott barbed Yeah.
10:04 Brad wire and i put up like the barbed wire is the the scary graybeard you know staff engineer who you know just scowls at the person when they come and look at the code halfway funny right you know i guess i just i guess i just feel like it's worth saying like the same old you know song remains the same like you know
10:26 Scott right well it's it's the same old problem but it's just more the pressure has become more intense and hey brad you've been able to kind of hold off the barbarians because your commerce system was complex but
10:39 Brad Yeah.
10:40 Scott it's just the barrier has gone down it's not as complex anymore because Claude can understand it and you're not going to be able to hold this off forever I need I need to somehow turn that into for you as if you're the engineering manager I need to help you not only recognize you can't hold this off forever but actually somehow turn that into a you into a positive.
11:01 Brad Yeah, and I think that's yeah, yeah, totally, totally. And that's maybe why I kind of was that's why I kind of gravitated towards the factory. Right. Because part of what you'd want to say to your team is like, let's say AI didn't exist, but you just had to say like, look, we're going to a pod structure. We're going to we're going to go to we're moving away from like this like teams that are based on the architecture of the code and we're going to move. move to pod teams that are focused on driving business results and they need to be able to work across the full stack right like that that was a that was that's a thing now that was a thing before ai right right
11:40 Scott And that is why our company has a ball of mud that we're fixing right now, by the way.
11:44 Brad but the solution to that is not don't like i i just i i hear you
11:51 Scott What
11:52 Brad I
11:52 Scott is the think solution to that? You're actually on to something extremely valuable for the 41 listeners out there. What is the solution to that?
11:57 Brad well, I think the solution to that are the right guards in the
12:03 Scott Yeah.
12:03 Brad software factory and
12:04 Scott Yeah.
12:04 Brad the right guards aren't humans like humans are never the right guards because humans are fragile humans humans come and go humans leave the organization humans get sick humans are fallible you know what's not fallible is like structured deterministic guard gates of of you know run crap tools like like uncle bob would tell us right you You know, so
12:29 Scott So let me try this on for size with you. Let me test this out. Brad, you're the engineering manager. You're feeling threatened. I think what I need to say to you is, hey, what you need to take, where you need to channel, like, first of all, your job as an engineering manager. is to make it possible so that your team can do these things and if that's creating fear for you then your second job as an engineering manager is to work with the engineers on your team and the other engineers and other engineering managers at this company to figure out what those guardrails need those non-human guardrails need to be as
13:07 Brad Yes.
13:08 Scott as we as the buzzword has been lately that what the quality gates need to be but they can't be keeping people out of your code.
13:16 Brad Exactly.
13:16 Scott That's one that's no longer available to you.
13:19 Brad Yeah, and yes, love it. I love it. I love it. And I would just say if you can bring up any evidence to this or historical accounts of this, then lean into the pre-AI historical accounts of this exact same problem because this is not a new problem.
13:40 Scott Well, it's interesting. I would say there's a – this is what I've seen, a pattern I've seen at a few companies. Small company gets started. kind of a pod structure everybody's working on everything starts to grow still might be it's a one pod there's one pod right starts to grow people are still working on that pod kind of people just working on business results small enough still that there can be communication and collaboration such that the it isn't a ball of mud the architecture stays healthy so on and so forth but as time passes one of two things Two things happen. They both happen. Actually, three things happen. They all three of them happen. First thing is minor or sometimes major architectural mistakes.
14:25 Scott I can think of many in my career where you think you can look back and say, that was not a good decision.
14:30 Brad Right.
14:30 Scott But as time passes, then the system just doesn't work as well because somebody made a mistake. Sometimes you don't design a system well. That's one problem. Second problem is you add more people and you can no longer handle it with just communication. And I'm going to guess that that number is about. 25 to 40 people is when you get to be too big for that because really with 25 to 40 people you really still have only about like a dozen sort of your more senior engineers who are really the ones keeping that glue and they can communicate and then everyone else is sort of following along once you get up to 40 you kind of now you got some new guy coming in just now he's making architectural mistakes or things that are out of line with the community shared shared
15:16 Scott idea so you get more people and then of course you have turnover right you just have people coming through your organizations bringing different ideas so they don't have it's the same problem just different as you bring in new people they don't have the same shared understanding of what this architectural vision is and so I've seen that. And then what happens is the organization says, okay, we're not going to do this pod thing anymore. We're going to switch to system oriented teams. You're going to be the commerce team and you're going to be the front end team and you're going to be the whatever team, right?
15:52 Brad Yeah
15:52 Scott And that's how we create, that's how we solve that problem. We create these artificial and then, oh, and then we push it further. Now we're going to make microservices, we're going to make services. This is going to create all these
16:02 Brad Yeah.
16:02 Scott create all these barriers to making bad architecture right one's a human barrier one's actually like a network barrier right you can still create bad architecture in a microservices world but it starts to get a little bit harder and and that's actually when you hear then you hear this other set of folks come in and say oh you can create better architecture in a microservices world we could just still have a big old monolith if we were just more disciplined about our architecture and
16:26 Brad Right.
16:27 Scott and then you come back to the previous points so um so that's a crutch we've all gone on now a cool thing that's happening the task rabbit We have a couple architects, actually, and they are working on an architecture agent that will review people's plans to see if it fits with the program. up front which is cool we'll see how well it works actually it's working right now he's got a thing it basically it reviews the plans and it it sends him a basic files a ticket in Jira for him to say look at this doesn't quite seem
17:02 Brad Yeah.
17:03 Scott right to me so the architecture agent is more like a reviewer just for him to extend his power
17:08 Brad I mean, isn't that what you would have done before AI? Wouldn't you have had humans who are responsible for reviewing the architecture before? before the implementation starts right
17:18 Scott Well,
17:18 Brad like isn't
17:18 Scott I think,
17:18 Brad that like in fact i'm pretty sure you've told me that they have that that you guys have that at task rabbit like
17:23 Scott a
17:23 Brad that's what your architect so it's one of the things your architects have in their job description sure
17:26 Scott little bit, but I think I've seen organizations move more to the walled fiefdoms first. Let's create a commerce team. Let's create a commerce service.
17:36 Brad totally
17:36 Scott That's an easier, I don't know if it's easier, it's a less failable technique. technique than having a person review the architecture.
17:46 Brad Yeah, with the theory that like what happens inside the bubble, it's OK, it can be as messy as you want inside the walls of the microservice or inside the walls of the fiefdom.
17:56 Scott Yeah.
17:56 Brad But so it's interesting, like, OK, so let me let me just say, like, let's rewind and I want to see if I'm picking up what you're laying down. If I am a if I'm an engineering manager whose primary fear about AI is that it will lower the bar, it will lower the barrier to have more people changing code that I am generally considered accountable for. and then
18:22 Scott Hmm.
18:22 Brad that will put my team at in in a treadmill situation where we're getting called in the middle of the night because of some failure that happened by somebody that wasn't in our crew does
18:35 Scott I
18:35 Brad that that is that my summarizing it correctly
18:36 Scott think that's kind of it. I mean, that's not, this was not said to me explicitly that way. Would I just kind of like think about. The feeling, you know.
18:45 Brad I think that there's an I mean, I think that there's that's also part of the concern I hear about some individual engineers who say, I don't want to write. I don't want to have the AI write code for me because if I write the code, then I know what it is. And so if later then I know that if it if I get called in the middle of night because of the failure, then it's at least my fault. Like and I'll know the code because I wrote it and I'll be able to debug it. bug it faster. And that's why I don't want to give up this control. There's a part of me that says, well, that's an understandable fear, but what it really is is a fear of loss of control.
19:21 Scott Yeah, that's right. That's right. The fear of loss of control and the control will help me prevent, help me, the engine manager, prevent.
19:29 Brad problems yeah doesn't that same doesn't that same ic or that same engineering manager also kind of get a queasy feeling in their stomach when the new intern or the new junior engineer gets assigned to their team
19:46 Scott Yeah, maybe, but you manage that differently. Right. You say, okay, here's a ticket. Start on this one. Fix this typo. Just make sure you know how the system works and you kind of, you don't put them on something big on day one. Here.
19:59 Brad You have two thoughts I have. Brad, you've got two books to write. Book number one, managing through AI, colon, what's old is new again.
20:09 Scott Yeah, well, that's what I've been saying, right?
20:10 Brad Yeah, exactly. Book number two, managing through AI, colon, it's all about the quality gates.
20:16 Scott Yeah, yeah, there you go.
20:19 Brad That's it. That's the book. There's the books.
20:20 Scott Yeah, you've summarized what my stance is on all this. I thought you were going to say. I thought you were raising the issue of fear that's like in the zeitgeist now and this podcast will come out a month later than we're recording it. And so by now, by the time this comes out, we might already be dead. The whole like AI is going to cause extermination 10% chance we're omnicide before the end of the decade.
20:52 Brad I mean, isn't this just Jurassic? classic park. We're so busy wondering if we could, we didn't think about whether we should.
20:59 Scott So I want to challenge this narrative a little bit, but I also will say like, man, I've seen AI do some really stupid shit yesterday. AI did some really stupid shit to me. And I was like, I was like, oh, this is actually where the mistake happens. Like this is maybe the Jurassic Park moment. So I had a Jira board. that i had uh added a whole bunch of new tickets to that were based on like i had done this analysis and i created like a essentially like a punch list of a bunch of stuff that needed to get done and so i'd hooked up cursor to my jira board and said please make all these jira tickets you know and assign them to a bunch of people right boom it did it i was like that's awesome yay that was great i didn't have to like
21:49 Scott like create 30 JIRA tickets right but then I noticed that they were these were all things that need to get done sooner than later and I noticed that they were at the bottom of the to-do column instead on the Kanban board instead of the top of the to-do column like these are new high priority items and so I just like naively told cursor hey can you please just move those to the top of the to-do list and then I turned away and I started doing something else And then all of a sudden, I got a pop-up message that. Zshell wanted access to my keychain and I was like what I denied you know denied you know and it popped up again I was like denied I'm like what what and I kind of look around and I find this window and I realize it's the please move the Jira tickets to the top of the to-do list agent
22:44 Scott is furiously trying to solve this problem for me and it and I stop it I go what are you doing and it was like it enumerated like well I tried to call the MCP server and the Jira MCP server didn't have a tool for me to change the order on a Kanban board so then I tried calling the Jira back-end API and I couldn't I didn't have credentials for that so I had to so then I tried to open up your Chrome browser but you denied that that and so i've just been trying to get access to your um chrome cookies so that i could just set up uh direct http calls and
23:19 Brad Jesus, it's so resourceful.
23:21 Scott it was it was so resourceful it was it was like he asked me to move these tickets to the top he said he wanted a paper clip I'm going to make as many paperclips as possible.
23:34 Brad Oh my goodness.
23:36 Scott So, yeah, so the point that my point there is there's an element of the. it could kill us all where i'm like when i see something like that happen i'm like oh yeah sure like i accidentally asked you to move the jira tickets to the top of the to-do list okay
23:51 Brad So there was an article in the paper, the Seattle Times today, I might be the only subscriber left of the Seattle Times, but I'm keeping them afloat and about the Medicare, the Federal Medicare Department, whatever it's called, Center for Medicare and Medicaid. six different states they've been running pilot programs. Washington State is one of the states. Arizona, I don't know the other states. They've been running pilot programs. Well, actually, here's an important thing to know. Recently, the law changed in our country. Previously, you did not have to get a Medicare. treatment pre-approved by Medicare. And sometime recently, apparently, according to what I learned in this article, the law changed. Now, just like a regular insurance company, you got to get things pre-approved. I don't want to get in the politics of that, but just telling you what I read in this article.
24:37 Brad So then in six states, that's, of course, very labor intensive. You've got to have people doing this, so on and so forth. Six states, they have started experimenting with using AI to approve or reject these claims. And they're actually doing it in each state with a different company. So they're piloting six different companies and six different systems, which is kind of clever, by the way. Let's see who does it best. best. But in Washington state, the system is rejecting 53% of the claims. which is too many. Like in Arizona, it was 20% of the planes. And so that's creating real world health consequences for people, right? And so where I think the omniscient is going to come from, it's the same flavor of what you were talking about with moving the geotickets thing is we just create these things without thinking about the downstream consequences of what's going to happen here.
25:26 Brad Like, it sounds great to have AI adjudicate these claims until... it doesn't rock you
25:33 Scott Yeah.
25:33 Brad know it sounds great to have ai move the jira tickets until it starts impersonating you on the internet.
25:41 Scott Right, right. Well, and so, okay, so that's the that's the version of the Oh, my God, you know, head smack forehead smacking moment of like, I just asked for this simple task and it was so resourceful that if it weren't for the fact that I knew to say no, right, like, I'm a savvy user that was like, I don't give I don't, you know. what is the there's the quad code command that's like recklessly accept all questions or something like there's like a dash dash recklessly say yes to everything or something i don't ever do that right so i've i've generally locked down the stuff and i'm very you know persnickety about like what what grants do i what rights do i grant this thing before i ask it to do something so it's interesting to me that
26:29 Scott It, so one, there's the scary part of like, it was so resourceful and it was so focused on goal driven to get the thing done that it came up with all these alternative ways to do it. Okay. Then, then there, so that's the, that's the kind of the scary side of the coin, right? That a naive question on my part and a, an absolutely naive. Agent just trying to make me happy chose a reckless approach to make me happy, right? But the other side of the coin that makes me less scared is it was just so dumb. It was it was really dumb about just why didn't it just say, hey, by the way, I tried and I couldn't do it. Do you really care that much? Like, do you really want me to do this?
27:19 Scott You know, I
27:20 Brad Yeah.
27:20 Scott was retelling this story to one of the engineers that I work with, and he just told me that he just like he didn't tell me the name of the company, but there was some large company that he was using a website for. And he asked his agent to go do something on that website and his agent inadvertently discovered that they have a classic like security flaw where with him logged in as him calling the calling in the generic API, he's able to get everybody's records.
27:50 Brad Oh, goodness.
27:51 Scott Right. So he's like, I'm going to file a I'm going to file a security claim and tell him like, hey, you've got a bug. You need to fix it. Right.
27:57 Brad Hmm?
27:57 Scott we're definitely in the age right now of all the things that we inadvertently didn't handle correctly are going to get exposed and tripped over but it's just not really thinking it's not it is so brute force and it is just stumbling its way through stuff like all these all these all these all these claims of like anthropomorphizing these things as as go as as like nefariously goal-oriented and oh creating these cultures and and passing these secret notes to each other and stuff like that it's like no these things are just naively trying to literally just do the thing you ask them to do it's
28:37 Brad It's artificial intelligence. It's artificial intelligence.
28:41 Scott yeah
28:42 Brad Actual intelligence would have said. grabbing someone's cookie to impersonate them probably is a bad idea. It's a little scary. It's a little scary to think what could happen because there's all sorts of security vulnerabilities out there that we kind of get away with because the world can't quite get to them. But these agents have enough horsepower, even increasing the horsepower that they can just keep on going after them forever.
29:05 Scott And they'll stumble on them.
29:06 Brad They'll stumble on them.
29:08 Scott But again, I think that I mean, I don't know, call me call me Pollyannish, but I just feel like. That just says, great, let's fix all the holes. Like,
29:19 Brad For
29:20 Scott let's use
29:20 Brad sure.
29:20 Scott the agents to find the holes and fix the holes.
29:23 Brad I agree. Yeah, for sure on that one.
29:25 Scott I don't know how helpful I was in the how do you address the fear of the engineering manager or the engineering leader or the engineer that feels like the AI is making.
29:35 Brad You did. You did. Look, what's old is new again. And we put in measures in place like having teams on systems or whatever to deal with some of the problems we had before. But the intensity or velocity or volume of those problems, I'm not sure which way you would describe it. has changed it happens to have changed because of ai it could have changed for any number of reasons it could have changed because your company hired a thousand engineers it
30:07 Scott Yeah.
30:07 Brad it could there's a lot of reasons it could have changed has changed and so the systems
30:11 Scott It
30:12 Brad that we
30:12 Scott could have changed because all of a sudden your product blew up and people loved it and you started getting massive traction, right?
30:19 Brad sure sure
30:20 Scott Like that's going to change it too.
30:22 Brad so the measure the systems processes measures quality gates that we put in place for that world are no longer suitable for
30:32 Scott this world well this that the mechanisms and the tools are identical you just need to implement them and you need to implement them robustly yeah
30:40 Brad in this case you have someone who's the mechanism they used was a fiefdom
30:46 Scott the mechanism that you used was a fiefdom what the fiefdom was attempting to do was a quality gate
30:52 Brad Yeah, you
30:52 Scott That's not a real quality gate. But if have you – yeah,
30:55 Brad to go. Yeah, I
30:57 Scott but if you said – no, no, that's fine. I was going to say if you zoom out and say, oh, I need a quality gate. Well, what's the definition of quality that matters to me? I want strong unit test code coverage. Great. It used to be that we would be like, shit, you're at 87% code coverage on your unit test? Wow. That's like – that's pretty like – that's nearly world class. Nope, 100%. And wait, you got away with unit test, but no integration test? Nope, 100% unit test coverage, full integration test and do coverage against your integration tests. And run your coverage reports against integration. Show that you're getting 100% code coverage in integrated paths, right? Oh, what about unit tests that don't actually assert anything meaningful? Those can produce code coverage, but they aren't actually testing anything meaningful. Okay.
31:43 Scott What would Uncle Bob say? Let's write mutation tests,
31:46 Brad right? Mm-hmm.
31:48 Scott Mutation tests, right, are the we have the unit test, we go and change the code, and the unit test should now fail. We change the code intentionally to break. Test your unit test. Do they actually fail when you change the code to be failing, right? Those are all tools and techniques that have been available to us for years. often avoided them because we felt like we didn't have the time and money to write those tests or write those quality gates and so instead we used fiefdoms and people now we have the time and energy to write all those tests let's let the agents write all those tests for us and guard really guard the code
32:29 Brad I just had a thought cross my mind that I don't quite have the time to explore today. Maybe we can save this for our next episode.
32:34 Scott Sounds good.
32:35 Brad Our business stakeholders are enamored by how much faster stuff can get written. But it might be the case that the job is actually getting harder because you have to operate or execute at a much higher level of quality and thoroughness and attention to detail than we ever. had to do before and before best practices or the aspiration now best
33:04 Scott Now that practice.
33:05 Brad practices are the floor the
33:07 Scott The floor.
33:07 Brad bar yeah
33:08 Scott Totally. Yeah, I agree 100%.
33:11 Brad And that is a major, major lift for not only most organizations, I'm going to even posit 95% of the engineers in the industry.
33:22 Scott Well, and there you go. If the fear was I'm going to become irrelevant because AI can write all this code, then the answer is if you know how to write good quality gates and build automation around your software factory in your SDLC, you just became one of the most valuable. people to hire let's
33:38 Brad Boom. That's amazing. Brad, let's end there.
33:42 Scott end right there that's
33:43 Brad Smash that subscribe button.
33:45 Scott that's great ruin Tell three
33:47 Brad your mom to listen and
33:48 Scott your tell your mom exactly let's
33:50 Brad rate
33:50 Scott get more
33:50 Brad and moms review.
33:50 Scott listen all right good
33:51 Brad Thanks, Brad. Talk
33:52 Scott talking
33:52 Brad to you next week.
33:52 Scott to you next week this episode of the how many CTOs does it take podcast was produced and edited by Saffron Heptegal our theme song was AI generated by Brad Please follow us on all the social media platforms at HowManyCTOsPod. You can find us on YouTube, LinkedIn, Instagram, Facebook, Threads, X, TikTok, and Patreon, or at our website at HowManyCTOsPod.com. And remember to rate, review, and subscribe on the podcast platform of your choice. Thanks for listening.
