Beyond Dashboards: How AI Is Redefining Developer Productivity with Adeeb Valiulla

In this episode of ShipTalk, host Dewan Ahmed sits down with Adeeb Valiulla, a leader in developer productivity and engineering excellence at Harness, to explore how AI is transforming the very definition of software productivity.

Adeeb shares his journey from building early engineering metrics systems at Sensormatic to leading developer efficiency initiatives for Fortune 500 companies. Together, they unpack how AI is changing what it means to be “productive” — from measuring outcomes instead of outputs, to ensuring governance, trust, and quality at AI scale.

If you’ve ever wondered how frameworks like DORA, SPACE, and Flow adapt in an AI-driven world, or how to separate “AI-powered” hype from truly “AI-driven” value, this conversation is for you.

If you'd like to appear on one of our episodes, connect with Dewan.

---

Adeeb's Work:

1. Comparative Analysis of Engineering Performance and Experience Frameworks: DORA, SPACE, HEART, and Beyond.
2. Developer Experience (DevEx) Measurement in the Age of AI.
3. Other Adeeb Valiulla publications.
4. Metrics That Matter podcast

Connect with Adeeb. 

Transcript

WEBVTT

1
00:00:02.440 --> 00:00:12.100
Dewan Ahmed: Good morning, good afternoon, good evening, time-appropriate greetings. My name is Dewan Ahmed. I'm your host. This is ShipTalk Podcast.

2
00:00:12.180 --> 00:00:29.000
Dewan Ahmed: Where we talk about the ins and outs, the ups and downs of software delivery. With me today, I have Adeeb Valliulla, who leads the developer productivity and efficiency space at Harness. Welcome, Adeeb!

3
00:00:29.000 --> 00:00:30.720
Adeeb Valiulla | Harness: Thank you so much, thank you for having me.

4
00:00:30.980 --> 00:00:40.850
Dewan Ahmed: I can see fall colors out of my window, in, the east coast of Canada, beautiful New Brunswick. How is there, Adeeb? Do you see fall colors?

5
00:00:41.110 --> 00:00:52.189
Adeeb Valiulla | Harness: Yeah, I'm dialing in from San Francisco. Today, we have some rain, some outer cast, no sun today, but, you know, another good day to be here.

6
00:00:52.480 --> 00:00:59.970
Dewan Ahmed: So you don't enjoy, I think, I guess, shoveling snow. Like, you're deprived of that pleasure.

7
00:01:00.580 --> 00:01:12.459
Adeeb Valiulla | Harness: That is correct. I… I had my fair share of shoveling snow, walking in the terrible winters when I was in Chicago, but now that's beyond me and enjoying San Francisco for now.

8
00:01:12.460 --> 00:01:25.240
Dewan Ahmed: Hey man, if you ever miss shoveling snow, you can just take a short flight, come over here, I'll have a driveway full of snow. But, I don't want to scare our listeners and viewers out with all the snow stories.

9
00:01:25.240 --> 00:01:43.000
Dewan Ahmed: What I do want to mention is we actually kicked off Season 4 with Nathen Harvey. We talked about DORA and the framework to measure engineering productivity. Adeeb is another leader in this space of engineering excellence, developer productivity. So, Adeeb.

10
00:01:43.230 --> 00:01:50.219
Dewan Ahmed: I want to hear about your journey. Like, how did you end up in this space? Where did your career start?

11
00:01:51.010 --> 00:02:09.289
Adeeb Valiulla | Harness: Yeah, sure. So my journey really started back in Sensomatic, where I came in as an engineer, passionate about the craft and honesty, just eager to grow in my own career as well. At one point, I asked a pretty simple question.

12
00:02:09.470 --> 00:02:13.379
Adeeb Valiulla | Harness: how do I know I am really making a difference?

13
00:02:13.880 --> 00:02:19.040
Adeeb Valiulla | Harness: And the truth was, I did not get a clear or a measurable answer.

14
00:02:19.770 --> 00:02:21.899
Adeeb Valiulla | Harness: That moment stuck with me.

15
00:02:22.320 --> 00:02:31.700
Adeeb Valiulla | Harness: I started sketching a few of my ideas on a whiteboard, how I could actually show the business

16
00:02:32.020 --> 00:02:34.200
Adeeb Valiulla | Harness: the impact I was making.

17
00:02:35.060 --> 00:02:39.080
Adeeb Valiulla | Harness: While also surfacing opportunities to improve.

18
00:02:39.690 --> 00:02:46.550
Adeeb Valiulla | Harness: Now, this was way back in 2016, long before developer productivity

19
00:02:46.710 --> 00:02:50.909
Adeeb Valiulla | Harness: Engineering intelligence, efficiency, or common language.

20
00:02:51.770 --> 00:03:03.400
Adeeb Valiulla | Harness: I had built a homegrown project that measured things like lead time, developer punctuality, delivery punctuality, quality, and efficiency.

21
00:03:03.950 --> 00:03:13.690
Adeeb Valiulla | Harness: Now, it began as a tiny experiment with just my team, my product, and it caught on pretty fast.

22
00:03:14.090 --> 00:03:15.590
Adeeb Valiulla | Harness: before long.

23
00:03:15.710 --> 00:03:27.390
Adeeb Valiulla | Harness: all the business units at Sensomedic and every engineering team at Sensomedic were using it to understand not just what we were delivering.

24
00:03:27.770 --> 00:03:31.609
Adeeb Valiulla | Harness: But how effectively they were delivering it.

25
00:03:31.930 --> 00:03:34.280
Adeeb Valiulla | Harness: So that experience was eye-opening.

26
00:03:34.900 --> 00:03:50.490
Adeeb Valiulla | Harness: It taught me measuring engineering isn't about vanity metrics, it's about crafting a shared language of impact that both engineers and business leaders can rally around.

27
00:03:52.020 --> 00:03:57.540
Adeeb Valiulla | Harness: Now, fast forward to today… That's exactly what I do at Harness.

28
00:03:57.830 --> 00:04:09.529
Adeeb Valiulla | Harness: I lead the developer productivity function, where I help Fortune 500 companies translate these engineering data into business intelligence and outcomes.

29
00:04:10.040 --> 00:04:16.040
Adeeb Valiulla | Harness: Now, the mission is the same, just… just a little bit larger on the scale.

30
00:04:16.500 --> 00:04:28.560
Adeeb Valiulla | Harness: We help organizations measure what matters, improve developer experience, and ultimately align software delivery with real business outcomes.

31
00:04:30.530 --> 00:04:39.339
Dewan Ahmed: That is fantastic to hear, and we all have… have this goal to show what we did, right? So let's say whether it's an IC,

32
00:04:39.340 --> 00:05:01.500
Dewan Ahmed: in their quarterly performance discussion, like, what did you do this quarter? Or even, let's say, a sales rep trying to understand that how sales, change quarter over quarter, and what's the relation with product. So, underneath all this, it's the engineering excellence, right? Because that sort of

33
00:05:01.500 --> 00:05:06.389
Dewan Ahmed: Creates the flywheel that touches all these different, pieces.

34
00:05:07.510 --> 00:05:22.849
Adeeb Valiulla | Harness: Absolutely, absolutely, right? You know, so at Harness, right, my role is almost like a translator. I take raw engineering signals, your tickets, your pull requests, your pipelines, incidents.

35
00:05:23.390 --> 00:05:28.419
Adeeb Valiulla | Harness: And help leaders see the bigger picture.

36
00:05:29.110 --> 00:05:32.050
Adeeb Valiulla | Harness: Are we improving time to value?

37
00:05:32.960 --> 00:05:40.069
Adeeb Valiulla | Harness: is developer experience healthy, right? Are we creating a sustainable delivery system?

38
00:05:40.760 --> 00:05:47.519
Adeeb Valiulla | Harness: It's not about just dashboards, right? It's about driving change. You can have…

39
00:05:47.960 --> 00:05:57.310
Adeeb Valiulla | Harness: 10 or 15 different metrics lined up in a dashboard, but the real value is, are we looking at the right things to drive transformations

40
00:05:57.570 --> 00:06:07.840
Adeeb Valiulla | Harness: in your organizations, right? For example, I've run engineering metrics program at cybersecurity companies, API security companies.

41
00:06:07.950 --> 00:06:18.360
Adeeb Valiulla | Harness: Gaming, hospitality, information technology companies, where leaders wanted both velocity and resilience.

42
00:06:18.730 --> 00:06:23.990
Adeeb Valiulla | Harness: Now, my team helps make sense of DORA metrics.

43
00:06:24.330 --> 00:06:28.700
Adeeb Valiulla | Harness: Augment them with space-style human factors.

44
00:06:29.420 --> 00:06:34.070
Adeeb Valiulla | Harness: and increasingly layer AI insights into the mix.

45
00:06:34.470 --> 00:06:43.009
Adeeb Valiulla | Harness: So, in short, my role is about turning engineering data into business intelligence, so leaders can make decisions

46
00:06:43.210 --> 00:06:46.120
Adeeb Valiulla | Harness: With confidence, not just gut feel.

47
00:06:47.280 --> 00:07:04.410
Dewan Ahmed: you touched on something key, everyone's favorite two-letter word these days, AI. Theme of Season 4 is actually AI meets software delivery. Now, we want to hear, how has engineering productivity, like, how you measure it, the definition, and the whole…

48
00:07:04.540 --> 00:07:10.010
Dewan Ahmed: I guess the area around it has changed now that we see AI everywhere.

49
00:07:10.960 --> 00:07:15.229
Adeeb Valiulla | Harness: Yeah, that's a great question. So, the word productivity

50
00:07:16.010 --> 00:07:27.729
Adeeb Valiulla | Harness: used to mean output, right? Your lines of codes and your commits, your velocity points, but AI has blown up that definition.

51
00:07:28.680 --> 00:07:37.200
Adeeb Valiulla | Harness: Now, productivity isn't about typing faster, it's about… Problem-solving capacity.

52
00:07:38.250 --> 00:07:41.750
Adeeb Valiulla | Harness: research backed this up, right? The recent…

53
00:07:41.960 --> 00:07:50.030
Adeeb Valiulla | Harness: 2025 DORA AI study suggests that 95% of teams are using AI.

54
00:07:50.460 --> 00:07:55.670
Adeeb Valiulla | Harness: And yes, 80% say it has improved productivity, but

55
00:07:55.800 --> 00:07:59.679
Adeeb Valiulla | Harness: 30% don't even trust the AI-generated code.

56
00:08:00.300 --> 00:08:08.320
Adeeb Valiulla | Harness: Now, similarly, a report that Harness put together in their AI and software engineering report.

57
00:08:08.730 --> 00:08:20.429
Adeeb Valiulla | Harness: Harness shows that 63% of orgs are shipping faster with AI, but 45% of AI-generated deployments

58
00:08:20.610 --> 00:08:30.579
Adeeb Valiulla | Harness: have problems. That's a significant number, right? So the definition has shifted from how fast we code to

59
00:08:30.840 --> 00:08:37.099
Adeeb Valiulla | Harness: How much value reliable software can we deliver sustainably?

60
00:08:37.659 --> 00:08:41.670
Adeeb Valiulla | Harness: augmented by AI. So that's a profound change.

61
00:08:42.210 --> 00:08:54.620
Dewan Ahmed: Totally. And to our viewers and listeners, the reports that Adibh is mentioning in his stat, we'll be sure to link in the description of the podcast, as well as the YouTube video.

62
00:08:54.620 --> 00:09:05.520
Dewan Ahmed: Now, Adeep, when you're working with engineering teams, what sort of indicators are you looking at to understand AI? Is even AI helping with those engineering productivity? Because

63
00:09:05.520 --> 00:09:13.579
Dewan Ahmed: Nowadays, we don't have an issue with less metrics. We have an issue with too many signals, so which indicators actually matter?

64
00:09:14.110 --> 00:09:25.110
Adeeb Valiulla | Harness: Fair point, you know, I look at three layers, essentially, right? The first layer is your developer metrics, which is essentially what DORA helps you to track, right? Are your lead times shrinking?

65
00:09:25.370 --> 00:09:31.969
Adeeb Valiulla | Harness: is deployment frequency up? Are you… are you delivering value to your customers frequently?

66
00:09:32.410 --> 00:09:34.600
Adeeb Valiulla | Harness: And is stability maintained?

67
00:09:34.810 --> 00:09:42.019
Adeeb Valiulla | Harness: Right? If AI helps code faster, but your change failure rate spikes.

68
00:09:42.630 --> 00:09:46.970
Adeeb Valiulla | Harness: That's not productivity. That's literally chaos.

69
00:09:47.320 --> 00:09:58.769
Adeeb Valiulla | Harness: Right? So that's the first layer. The second layer is, yes, the developer experience matters. So, are developers spending more time in flow?

70
00:09:59.410 --> 00:10:06.190
Adeeb Valiulla | Harness: Or are they bogged down by context switching between 10 different AI tools?

71
00:10:07.600 --> 00:10:18.090
Adeeb Valiulla | Harness: The real tool sprawl is killing teams with onboarding, now taking Two months and beyond.

72
00:10:18.330 --> 00:10:23.399
Adeeb Valiulla | Harness: Right, and then the third layer is around value alignment.

73
00:10:24.310 --> 00:10:25.420
Adeeb Valiulla | Harness: Sure.

74
00:10:25.750 --> 00:10:32.079
Adeeb Valiulla | Harness: you're shipping, but are you shipping features that actually move the business KPIs?

75
00:10:32.920 --> 00:10:38.010
Adeeb Valiulla | Harness: AI can flood the pipeline with code, as I mentioned earlier.

76
00:10:38.140 --> 00:10:44.079
Adeeb Valiulla | Harness: But if it's not aligned to outcomes, It's noise.

77
00:10:44.470 --> 00:10:54.740
Adeeb Valiulla | Harness: Right? So, I ask, Is AI helping developers spend more time on valuable work?

78
00:10:54.880 --> 00:10:57.269
Adeeb Valiulla | Harness: And less time on toll?

79
00:10:58.140 --> 00:11:01.249
Adeeb Valiulla | Harness: If not, we are not measuring the right things.

80
00:11:02.420 --> 00:11:10.430
Dewan Ahmed: Yeah, I'll pick up on the point you mentioned, that AI can flood the pipeline with new code, so it might… it might…

81
00:11:10.880 --> 00:11:29.439
Dewan Ahmed: makes sense that, yes, we're getting more code, we're increasing productivity, but then it also introduces risks. How are we making sure that these are good code, these are code ready to be added to production? So how should quality engineering adapt with AI?

82
00:11:30.160 --> 00:11:38.110
Adeeb Valiulla | Harness: Yeah, so… So think of AI as a speed booster on your highway.

83
00:11:38.370 --> 00:11:43.900
Adeeb Valiulla | Harness: Right? If you don't upgrade your brakes, and guardrails.

84
00:11:44.410 --> 00:11:47.990
Adeeb Valiulla | Harness: Essentially, speed just means more accidents.

85
00:11:49.120 --> 00:11:56.069
Adeeb Valiulla | Harness: Now, practices I recommend are You want to shift left testing.

86
00:11:56.220 --> 00:12:04.229
Adeeb Valiulla | Harness: with AI-powered test generation, It also increases human-in-the-loop validation.

87
00:12:05.340 --> 00:12:08.530
Adeeb Valiulla | Harness: Number 2 is your feature flags and

88
00:12:09.150 --> 00:12:15.650
Adeeb Valiulla | Harness: safety in deployments, right? So today, less than half the orgs

89
00:12:16.170 --> 00:12:24.949
Adeeb Valiulla | Harness: and the data comes from Harness's AI and software engineering reports. Use feature flags for AI-generated code.

90
00:12:25.880 --> 00:12:27.309
Adeeb Valiulla | Harness: That's a big gap.

91
00:12:29.590 --> 00:12:32.500
Adeeb Valiulla | Harness: The third one is guardrails in Pipeline.

92
00:12:32.670 --> 00:12:39.899
Adeeb Valiulla | Harness: Now, blue-green canaries, chaos testing, these are not options anymore.

93
00:12:40.430 --> 00:12:44.980
Adeeb Valiulla | Harness: These are mandatory in your software delivery lifecycle.

94
00:12:45.360 --> 00:12:51.029
Adeeb Valiulla | Harness: And most importantly, security reviews at AI scale.

95
00:12:52.230 --> 00:12:59.020
Adeeb Valiulla | Harness: Since 48% of orgs worry about increased vulnerabilities.

96
00:13:00.480 --> 00:13:14.659
Adeeb Valiulla | Harness: again, this is data from our harness report, right? You need to, in short, evolve your quality engineering from reactive testing to proactive governance.

97
00:13:16.400 --> 00:13:26.099
Dewan Ahmed: I like the term AI scale. Like, previously we talked about, doing things at planet scale, or doing things at this scale.

98
00:13:26.240 --> 00:13:33.629
Dewan Ahmed: But we haven't seen this scale. We haven't seen anything like AI scale, like the pace at which things are being…

99
00:13:33.740 --> 00:13:36.830
Dewan Ahmed: developed and deployed.

100
00:13:36.940 --> 00:13:53.519
Dewan Ahmed: That brings me to ask, like, there has to be some misconceptions around as well, right? There's definitely things that are going right, but people are thinking that, can AI really do that? So, what are some misconceptions you have heard in the engineering productivity space in the AI era?

101
00:13:54.140 --> 00:14:08.379
Adeeb Valiulla | Harness: Yeah, so, you know, I hear this a lot, and right off the bat, I feel like a lot of… a lot of people in the industry think AI immediately equals productivity by default.

102
00:14:08.520 --> 00:14:10.410
Adeeb Valiulla | Harness: Right, so that's the first myth.

103
00:14:10.650 --> 00:14:20.700
Adeeb Valiulla | Harness: The 2025 State of AI-assisted Software Development by DORA shows that AI adoption

104
00:14:21.130 --> 00:14:27.580
Adeeb Valiulla | Harness: Amplifies what you already have, which means strong teams get stronger.

105
00:14:27.920 --> 00:14:30.140
Adeeb Valiulla | Harness: Weak teams get weaker.

106
00:14:30.270 --> 00:14:35.859
Adeeb Valiulla | Harness: So, if your system is broken, AI just breaks it fast.

107
00:14:36.680 --> 00:14:44.929
Adeeb Valiulla | Harness: Okay? Second myth is the idea that individual output is the right measure.

108
00:14:45.840 --> 00:14:51.600
Adeeb Valiulla | Harness: With AI, one engineer can generate massive amount of code.

109
00:14:52.370 --> 00:14:56.340
Adeeb Valiulla | Harness: But if the team… And not S.

110
00:14:56.650 --> 00:15:02.060
Adeeb Valiulla | Harness: Secure, or deploy it, It's not productivity.

111
00:15:02.450 --> 00:15:04.290
Adeeb Valiulla | Harness: It's just waste.

112
00:15:04.470 --> 00:15:11.239
Adeeb Valiulla | Harness: So the misconception is Productivity is about output.

113
00:15:11.880 --> 00:15:18.270
Adeeb Valiulla | Harness: The truth is, productivity is about outcomes in context.

114
00:15:19.390 --> 00:15:35.050
Dewan Ahmed: Yeah, yeah, totally. And then, I think that's where engineering teams nowadays, they crave the product insights, experts like you provide, because without that, you can't fix what you can't see, what you can't understand.

115
00:15:35.050 --> 00:15:40.649
Adeeb Valiulla | Harness: And we see a lot of terms like AI-driven or AI-powered.

116
00:15:40.650 --> 00:15:48.650
Dewan Ahmed: So, what do you think the difference between an AI-powered feature versus truly an AI-driven product?

117
00:15:51.180 --> 00:15:55.090
Adeeb Valiulla | Harness: AI-powered feature, and it truly…

118
00:15:55.550 --> 00:16:03.659
Adeeb Valiulla | Harness: AI-driven product. That's an interesting, way to put it, and I love the question. So, in my opinion, AI…

119
00:16:03.940 --> 00:16:09.860
Adeeb Valiulla | Harness: Our feature is, like, an autocomplete for your IPE.

120
00:16:10.470 --> 00:16:13.749
Adeeb Valiulla | Harness: Right? It makes a task easier.

121
00:16:13.900 --> 00:16:19.370
Adeeb Valiulla | Harness: But a true AI-driven product Changes how you work.

122
00:16:19.970 --> 00:16:26.499
Adeeb Valiulla | Harness: Right? So, for example, a coding assistant that suggests snippets is a feature

123
00:16:26.910 --> 00:16:33.839
Adeeb Valiulla | Harness: But an AI-driven product would manage the entire pull request lifecycle.

124
00:16:34.300 --> 00:16:44.839
Adeeb Valiulla | Harness: So, detecting risk, generating those tests, Assigning reviewers Essentially, reshaping your workflows.

125
00:16:45.490 --> 00:16:52.790
Adeeb Valiulla | Harness: So the difference is… Are you shrinking or sprinkling AI?

126
00:16:52.980 --> 00:16:57.069
Adeeb Valiulla | Harness: Or are you re-architecting around AI?

127
00:16:57.580 --> 00:17:01.769
Adeeb Valiulla | Harness: The winner will be those who do the latter.

128
00:17:02.790 --> 00:17:17.480
Dewan Ahmed: Yeah, I can relate. So previously, whenever I needed to, let's say, create a pipeline, like a CI-CD pipeline, I'd read the docs, and then I'd try to, let's say, build the CI stage for build and push.

129
00:17:17.490 --> 00:17:31.809
Dewan Ahmed: then I'd probably have some sort of approval gate if I need that, then I'd deploy, and then I'd try to understand, okay, what are the configurations I need? Like, it would be a tedious process, like, for a production-ready pipeline, you need to look at all those details.

130
00:17:31.960 --> 00:17:42.109
Dewan Ahmed: So now with, Harness AI, I can go to, I can see, like, on Harness Platform, you have this button, Create with AI. I can say.

131
00:17:42.110 --> 00:17:53.029
Dewan Ahmed: create me a pipeline with a build stage with a push to, let's say, ECR, and then I'm gonna deploy to, to GKE, and then I need to do this.

132
00:17:53.620 --> 00:18:00.529
Adeeb Valiulla | Harness: You know, and on top of it, adding the security scanners, right, governance, as part of that pipeline.

133
00:18:00.900 --> 00:18:20.119
Adeeb Valiulla | Harness: not worrying about creating from scratch, right? So use the Harness AI feature just to create complex pipelines is the beauty, right? So you're not just making simple pipelines, this is complex pipeline with the governance added, with the security scanners added, with the testing added, right? That's the beauty of it.

134
00:18:20.330 --> 00:18:25.830
Dewan Ahmed: Yeah, and I think that's where I can relate what you said. It's not just sprinkling AI, but…

135
00:18:26.010 --> 00:18:40.560
Dewan Ahmed: thinking how everything changes with AI, because not only is it creating pipeline, as previously, when I used to debug pipeline failures, what I would do, I'd look at logs, right? Logs after logs, we'd, like, tail the logs, try to do… Absolutely.

136
00:18:40.560 --> 00:18:51.349
Dewan Ahmed: now I have this button that says, debug, or let Harness AI, like, debug it for you and find out the issue, and it parses through it, it reads the entire log.

137
00:18:51.350 --> 00:19:05.500
Dewan Ahmed: and then tells, okay, exactly, like, you might have, have, maybe a YAML validation error, or, or something silly that you… it'll need you probably hours to, to troubleshoot.

138
00:19:06.620 --> 00:19:07.370
Adeeb Valiulla | Harness: Absolutely.

139
00:19:07.960 --> 00:19:22.900
Dewan Ahmed: So, now that we think about how AI is making these changes, how do we evaluate? Like, what sort of, I guess, framework do you have to evaluate if AI is even providing real value to the engineering teams?

140
00:19:24.190 --> 00:19:33.420
Adeeb Valiulla | Harness: Yeah, so, right, with any technology or any tool, you want to make sure it's adopted, right? What impact is it making, and is it efficient, right? So…

141
00:19:33.770 --> 00:19:39.500
Adeeb Valiulla | Harness: On the adoption side, are developers voluntarily using it?

142
00:19:39.690 --> 00:19:45.169
Adeeb Valiulla | Harness: Even after the novelty wears off, that's where true adoption kicks in, right?

143
00:19:45.700 --> 00:19:53.149
Adeeb Valiulla | Harness: I talked about impact, does it… improve Throughput and stability.

144
00:19:54.250 --> 00:19:57.510
Adeeb Valiulla | Harness: The DORA research shows the best orgs.

145
00:19:57.970 --> 00:20:03.230
Adeeb Valiulla | Harness: achieve both throughput and stability, so there's a huge impact there.

146
00:20:03.540 --> 00:20:14.560
Adeeb Valiulla | Harness: And then, efficiency. Your cost efficiency. So, is it reducing downstream toil Or creating cloud bills.

147
00:20:14.690 --> 00:20:19.510
Adeeb Valiulla | Harness: which… Are going to shock your teams, right?

148
00:20:20.050 --> 00:20:32.429
Adeeb Valiulla | Harness: comes from inefficient AI code, right? So, again, the report from Harness suggests 70% of organizations worry about runway costs.

149
00:20:33.720 --> 00:20:34.980
Adeeb Valiulla | Harness: coming from AI.

150
00:20:35.740 --> 00:20:44.510
Adeeb Valiulla | Harness: So, If it fails these 3 checks, It's just a… Lipstick on a pig, right?

151
00:20:45.820 --> 00:21:00.780
Dewan Ahmed: Yeah, yeah, and on top of that, many enterprise customers, for them, data governance is the key issue, the make-or-break moment, that, what do they do with data, like their data readiness, data governance.

152
00:21:00.870 --> 00:21:18.770
Dewan Ahmed: So for those engineering leaders, and especially, like, you are one of the very few people in the industry who has worked extensively on this, so what sort of data governance and data readiness steps they need to take into account before they can include AI in their workflows?

153
00:21:19.330 --> 00:21:23.359
Adeeb Valiulla | Harness: Yeah, I mean, all this is backed by data, right? You have to have good data.

154
00:21:23.680 --> 00:21:27.099
Adeeb Valiulla | Harness: So, you start with data hygiene.

155
00:21:27.570 --> 00:21:34.410
Adeeb Valiulla | Harness: AI is only as good as the signals it learns from.

156
00:21:35.680 --> 00:21:40.780
Adeeb Valiulla | Harness: What does that mean? That means… Clean, labeled engineering data.

157
00:21:42.020 --> 00:21:45.520
Adeeb Valiulla | Harness: clear AI usage policies.

158
00:21:45.730 --> 00:21:51.330
Adeeb Valiulla | Harness: Right? Responsible AI. And then governance for model output.

159
00:21:52.410 --> 00:21:55.159
Adeeb Valiulla | Harness: Don't just trust, you gotta verify.

160
00:21:55.990 --> 00:21:59.350
Adeeb Valiulla | Harness: And don't skip platform investment.

161
00:22:00.110 --> 00:22:08.050
Adeeb Valiulla | Harness: DORA found 94% of organizations now rely on platform engineering.

162
00:22:08.770 --> 00:22:10.290
Adeeb Valiulla | Harness: As the foundation.

163
00:22:10.960 --> 00:22:20.779
Adeeb Valiulla | Harness: Without a strong internal platform, your AI adoption Will fragment and collapse.

164
00:22:22.820 --> 00:22:25.700
Dewan Ahmed: You talked about measurement,

165
00:22:25.940 --> 00:22:40.700
Dewan Ahmed: So, you'd mentioned about DORA, but you also measure the framework itself. So, we talked about DORA with Nathan Harvey, but I was super interested to hear from you that you actually measured the framework itself.

166
00:22:40.700 --> 00:22:49.039
Dewan Ahmed: So, DORA, Space, DevEx, so all these engineering excellence and performance frameworks. How do you measure the framework itself?

167
00:22:49.390 --> 00:22:57.880
Adeeb Valiulla | Harness: There are so many different frameworks. Literally, there is a framework every single day, right?

168
00:22:58.740 --> 00:23:08.780
Adeeb Valiulla | Harness: each framework kind of solves a different piece of the puzzle. You mentioned there's DORA, there's space, there's heart, there is flow.

169
00:23:09.040 --> 00:23:18.229
Adeeb Valiulla | Harness: DevEx, and I can go on and on, but I'll pick the top few, right? So, DORA is great for delivery performance.

170
00:23:19.750 --> 00:23:29.199
Adeeb Valiulla | Harness: speed, stability, It's simple, actionable, But it's narrow.

171
00:23:29.430 --> 00:23:36.749
Adeeb Valiulla | Harness: It only focuses on the lead times, deployment frequency, change, failure rate, mean time to restore.

172
00:23:37.430 --> 00:23:41.579
Adeeb Valiulla | Harness: Space, on the other hand, adds the human dimensions.

173
00:23:42.380 --> 00:23:51.619
Adeeb Valiulla | Harness: satisfaction, flow, collaboration, But it's much harder to measure consistently.

174
00:23:53.290 --> 00:23:56.790
Adeeb Valiulla | Harness: start… Again, created by Google.

175
00:23:56.990 --> 00:24:05.440
Adeeb Valiulla | Harness: Is user-focused, which means it measures how delivery impacts customer experience.

176
00:24:06.250 --> 00:24:09.859
Adeeb Valiulla | Harness: It's often overlooked, but it's very vital.

177
00:24:11.210 --> 00:24:18.270
Adeeb Valiulla | Harness: Flow framework is… is the one which connects your engineering work to business value.

178
00:24:18.450 --> 00:24:23.680
Adeeb Valiulla | Harness: It bridges the gap between your CIOs and CFOs.

179
00:24:23.910 --> 00:24:32.659
Adeeb Valiulla | Harness: Right? So… In my research paper, I map these on a quadrant.

180
00:24:33.210 --> 00:24:36.149
Adeeb Valiulla | Harness: DORA is easy to implement.

181
00:24:37.120 --> 00:24:45.590
Adeeb Valiulla | Harness: Space and floor, Space and flow are more holistic, but harder to operationalize.

182
00:24:46.400 --> 00:24:50.450
Adeeb Valiulla | Harness: The real power is combining them together.

183
00:24:53.060 --> 00:24:58.559
Adeeb Valiulla | Harness: And we'll be sure to link Adib's research paper in the description as well.

184
00:24:58.560 --> 00:25:10.490
Dewan Ahmed: So, these frameworks, right, so they were developed way before the explosion of AI tools and AI is becoming mainstream. So how should these frameworks adapt to

185
00:25:10.610 --> 00:25:17.150
Dewan Ahmed: the AI workflows, where traditional metrics might not tell the whole story.

186
00:25:19.110 --> 00:25:25.840
Adeeb Valiulla | Harness: Yeah, I feel like… There is an opportunity…

187
00:25:26.230 --> 00:25:30.009
Adeeb Valiulla | Harness: For us as an industry to extend

188
00:25:30.400 --> 00:25:39.919
Adeeb Valiulla | Harness: metrics. For example, in my opinion, DORA could track AI-assisted lead time versus the traditional lead time.

189
00:25:40.060 --> 00:25:43.280
Adeeb Valiulla | Harness: Right? If you look at space.

190
00:25:44.140 --> 00:25:52.409
Adeeb Valiulla | Harness: Space could include, cognitive load, from Cool Sprouts.

191
00:25:52.870 --> 00:25:59.099
Adeeb Valiulla | Harness: Right? Art could measure trust in AI-generated features.

192
00:26:01.440 --> 00:26:11.720
Adeeb Valiulla | Harness: what I'm alluding to is frameworks must recognize that AI changes not just speed.

193
00:26:12.520 --> 00:26:17.870
Adeeb Valiulla | Harness: But trust, governance, and the human experience side of things.

194
00:26:19.200 --> 00:26:28.690
Dewan Ahmed: Yeah, and also accountability, because if you now have a code that is generated by AI, reviewed by AI,

195
00:26:29.240 --> 00:26:43.300
Dewan Ahmed: where does the accountability line blur? Like, is it you? Is it the admin who allowed certain model? I guess that's something these frameworks would also need to highlight, right?

196
00:26:43.830 --> 00:26:56.000
Adeeb Valiulla | Harness: Absolutely, absolutely. Responsible AI policies is another, another factor to be included in, in just operationalizing your AI strategy within your organization.

197
00:26:56.460 --> 00:27:08.739
Dewan Ahmed: Yeah, yeah. And then the other question, let's say if you're a CTO, you might think that, how do I balance developer experience metrics with developer performance metrics?

198
00:27:10.140 --> 00:27:19.069
Adeeb Valiulla | Harness: Yeah, so… It's… it's about… The cause and the effect.

199
00:27:20.060 --> 00:27:24.359
Adeeb Valiulla | Harness: The developer experience is the leading indicator.

200
00:27:25.550 --> 00:27:31.970
Adeeb Valiulla | Harness: developer… Rather, delivery performance is the lagging indicator.

201
00:27:32.630 --> 00:27:39.100
Adeeb Valiulla | Harness: So, if developers report high friction, or burnout.

202
00:27:39.370 --> 00:27:40.909
Adeeb Valiulla | Harness: Which is your space.

203
00:27:41.710 --> 00:27:51.149
Adeeb Valiulla | Harness: You will see it Downstreams, as long… downstreams, as longer lead times.

204
00:27:51.570 --> 00:27:55.930
Adeeb Valiulla | Harness: Right? Or higher failure rates, which is your DORA.

205
00:27:56.790 --> 00:28:07.570
Adeeb Valiulla | Harness: Right? So… I tell the CTOs and the leaders that don't treat them as compelling dashboards.

206
00:28:08.260 --> 00:28:12.160
Adeeb Valiulla | Harness: Read them as a feedback loop.

207
00:28:12.810 --> 00:28:16.350
Adeeb Valiulla | Harness: Healthy experience fuels healthy delivery.

208
00:28:18.970 --> 00:28:26.989
Dewan Ahmed: I think that itself could be a blog, like, healthy experience, fuel, delivery, right? And I couldn't agree more, because,

209
00:28:27.200 --> 00:28:30.879
Dewan Ahmed: This… this mindset where we… we…

210
00:28:31.030 --> 00:28:50.139
Dewan Ahmed: do one time, it's a one-time thing, has an issue in itself, because it's a feedback loop, because now it's AI, tomorrow it might be something else, 20 years before, the tools change, the frameworks change, but the mindset needs to be that it's never done.

211
00:28:50.250 --> 00:28:58.340
Dewan Ahmed: We're continuously improving, seeing what works, seeing what doesn't, and then, change, change accordingly.

212
00:28:58.340 --> 00:29:15.270
Dewan Ahmed: That brings me to the last segment of our podcast, where we ask our guests on future. Now, we're not going to ask you to predict what's gonna be the winning loader number. We will ask you how they work. Also, it might be fun if you could predict that.

213
00:29:15.270 --> 00:29:23.520
Dewan Ahmed: So, how do you see the defining traits of AI-enabled software delivery organizations in, let's say, in the next 3 to 5 years?

214
00:29:23.520 --> 00:29:29.100
Adeeb Valiulla | Harness: I think, players who have…

215
00:29:29.600 --> 00:29:35.430
Adeeb Valiulla | Harness: Platform as a strategy are definitely going to have a competitive advantage.

216
00:29:35.660 --> 00:29:40.040
Adeeb Valiulla | Harness: stronger platforms that make AI safe.

217
00:29:41.250 --> 00:29:48.510
Adeeb Valiulla | Harness: and scalable will definitely be the winners, right? Platform that can help you

218
00:29:49.100 --> 00:30:02.850
Adeeb Valiulla | Harness: Automate your deployments, can automatically create your complex pipelines. We spoke about this a bit earlier. Adding your governance, adding your security as part of that pipeline.

219
00:30:03.500 --> 00:30:15.809
Adeeb Valiulla | Harness: will help organizations deliver faster, but not just faster, right? Safer, in a secure manner, and efficiently. So, platform is…

220
00:30:16.130 --> 00:30:24.110
Adeeb Valiulla | Harness: where I feel… Players will… will definitely have a good, good advantage over

221
00:30:24.630 --> 00:30:31.080
Adeeb Valiulla | Harness: or players who, don't have platform as a strategy. That's… that's the key thing.

222
00:30:31.190 --> 00:30:35.949
Adeeb Valiulla | Harness: Governance, so AI policies and guardrails.

223
00:30:37.140 --> 00:30:47.819
Adeeb Valiulla | Harness: embedded in your pipelines. That's… that's the key word here. Will significantly help AI enable software delivery.

224
00:30:48.890 --> 00:31:00.890
Adeeb Valiulla | Harness: And human-centered, which is… They'll review… they'll view your developer experience as the competitive edge.

225
00:31:01.000 --> 00:31:13.579
Adeeb Valiulla | Harness: not just a nice-to-have, right? So, developer experience is key to all of these, right? So, to summarize, I want to say platform, governance, and developer experience.

226
00:31:14.180 --> 00:31:30.729
Dewan Ahmed: Yeah, I can almost see that triangle, platform governance and developer experience. Maybe, maybe in one of the frameworks, it shows as a triangle. For initiatives, the main thing is mindset, right? Engineering leaders, they need to shift mindset, because.

227
00:31:31.190 --> 00:31:35.759
Dewan Ahmed: Things have been done for the last, let's say, 30, 40, 50 years, and it has worked.

228
00:31:35.790 --> 00:31:43.110
Adeeb Valiulla | Harness: Now, suddenly, there needs to be a change in mindset. So, for those listeners, those executives, engineering leaders.

229
00:31:43.110 --> 00:31:55.299
Dewan Ahmed: CTOs who are listening to this podcast, what would be one mindset shift that you'd, recommend to them, now, that would help them in the future?

230
00:31:56.120 --> 00:32:04.420
Adeeb Valiulla | Harness: Yeah, so, look, AI is here, it's not… it's not really the next big thing, right? It's here, it's generating a lot of code,

231
00:32:04.420 --> 00:32:17.250
Adeeb Valiulla | Harness: which is good, but it's bad in terms of the delivery process, right? If you don't have a scalable approach on the right-hand side, you are only inviting chaos, noise, and

232
00:32:17.770 --> 00:32:27.870
Adeeb Valiulla | Harness: an instability into your delivery pipeline. So, leaders need to shift their mindset from cool thinking?

233
00:32:28.250 --> 00:32:30.370
Adeeb Valiulla | Harness: Two-way system thinking.

234
00:32:31.110 --> 00:32:34.840
Adeeb Valiulla | Harness: So, buying the next AI assistant

235
00:32:35.500 --> 00:32:38.779
Adeeb Valiulla | Harness: Will not fix your systemic bottlenecks.

236
00:32:39.960 --> 00:32:46.510
Adeeb Valiulla | Harness: But investing in… Feedback loops, culture, governance.

237
00:32:47.540 --> 00:32:51.060
Adeeb Valiulla | Harness: Which are part of a platform-centered approach.

238
00:32:51.200 --> 00:32:57.179
Adeeb Valiulla | Harness: will make every AI tool more valuable.

239
00:32:59.050 --> 00:33:08.180
Dewan Ahmed: Yeah, totally, like… AI doesn't fix broken practice, broken system, it just creates,

240
00:33:08.180 --> 00:33:32.480
Dewan Ahmed: problems, at a scale. So now you have a broken problem times 100. So, totally, like, you need to have a solid practice, fix the issues, follow the usual SDLC best practices, and of course, then when you add AI at a system level, not just as a one-time, then the benefit will be there.

241
00:33:32.540 --> 00:33:47.420
Dewan Ahmed: So we heard the regular prediction, but I think our listeners want to hear one bold or unconventional prediction where AI and software delivery intersect. What would be one unconventional prediction from you?

242
00:33:48.360 --> 00:33:58.620
Adeeb Valiulla | Harness: Yeah, so… I think in 5 years, we'll stop talking about AI-assisted coding.

243
00:33:59.050 --> 00:34:05.349
Adeeb Valiulla | Harness: We have already seen a boom today, so in 5 years, you know, in my opinion.

244
00:34:05.590 --> 00:34:09.500
Adeeb Valiulla | Harness: Coding will fully be automated, for…

245
00:34:09.850 --> 00:34:19.380
Adeeb Valiulla | Harness: majority of the use cases. The real differentiator will be AI-assisted decision making.

246
00:34:20.560 --> 00:34:24.159
Adeeb Valiulla | Harness: Helping leaders choose what to build.

247
00:34:24.350 --> 00:34:26.679
Adeeb Valiulla | Harness: Not just how to build it.

248
00:34:28.139 --> 00:34:31.609
Adeeb Valiulla | Harness: I think that's… that's my next frontier.

249
00:34:32.630 --> 00:34:51.540
Dewan Ahmed: And for my developer friends who are listening to this podcast, don't feel that developer jobs are going away. You know the term, right? We just need a button to make everything automated. Someone still needs to make that button. So, the work of builders will always be there.

250
00:34:53.050 --> 00:35:09.829
Adeeb Valiulla | Harness: Absolutely, absolutely. Very well said, and, you know, thank you for this opportunity to share my experience, my expertise in the developer experience, developer productivity space to your audience. I really appreciate the time.

251
00:35:09.830 --> 00:35:19.069
Dewan Ahmed: Of course, thank you, and if our viewers and listeners want to connect with you, read up more about your work, where can they find you on the digital world?

252
00:35:19.710 --> 00:35:37.599
Adeeb Valiulla | Harness: Yeah, so I'll definitely link my LinkedIn. I have my publications on SSRN and ResearchGate. I do… I do podcasts, and I do host my own podcast, Metrics That Matter, where I… I speak to other engineering leaders, learn about their journey.

253
00:35:37.650 --> 00:35:54.900
Adeeb Valiulla | Harness: and, you know, collectively discuss and talk about just the space around developer experience, how AI is impacting our day-to-day lives. So yeah, those are some of the spaces where they can find me.

254
00:35:55.460 --> 00:36:18.269
Dewan Ahmed: Perfect. This was ShipTalk, Season 4, Episode 4 with Adeeb Valiulla, a leader in developer productivity and developer efficiency space. Hey, if you are someone as a CTO, or engineering leader, or an executive, would like to talk about engineering excellence, engineering productivity, we'll link Adeeb's podcast in the description. Today,

255
00:36:18.270 --> 00:36:22.930
Dewan Ahmed: I'm the one signing off, and we'll see you in the next episode.