When I was working for a large defense contractor, we tested rolling out AI tools for our developers.

The deployment numbers looked great: 52% increase for junior devs, 32% for mid-level, and 12% for senior engineers.

But we spent more than 80% of our time fixing bugs in deployed code(support burden). This has only gotten worse as people trust AI more, not less.

So when I see people like Rohit Agnihotri pitching charts saying we need less humans in the loop, or Chris Hughes peddling this narrative when he should know better, it tells me "expert" doesn't mean what it used to.

This is dangerous advice from people who shouldn't be positioning themselves as experts. Having a title doesn't mean you understand what happens in production where mistakes have real consequences.

If someone is telling you to reduce human oversight because AI is maturing, they don't understand the problem.

Do your own testing. Trust your own data. Be careful whose advice you're betting your systems on.

#AI #security #compliance

@kstrlworks What are "development numbers"?

@buherator deployment numbers were specifically tied to engineering team deployment frequency. That said there were more metrics that went into decisions based on defined problems we were trying to solve as a measure and control. So good question. For this division I used the following:

- deployment frequency: measure cadence of production deployments
- lead time to change: find deployment pipeline bottlenecks
- cycle time: track work duration to find process bottlenecks
- time from stg to prod: isolate pre-production friction
- code review time: is code getting to review that's wasting dev time to review as it's thoroughly wrong. Is there problems with our requirements
- defect escape rate: measure dev/testing quality
- support burden: track % sprint capacity on features vs refactoring
- blocked story time: quantify impact of blockers on throughput
- sprint commitment vs completion: measure predictability and scope creep
#kpi #softwaredevelopment #softwareengineering #metrics #programming

@kstrlworks Thanks for the detailed answer, this makes sense! (also it turns out I can't read, but you got the point)