Historical question — why did we as an industry decide that penetration tests needed to be outsourced, but that we didn't need outsourced application security tests or code reviews in the same way?

@dymaxion

IME pen tests were historically ordered after some disaster, outsourcing is reactive, when we feel we have to spend the money, and the vibe is "we clearly can't trust our team to do it." In contrast, code reviews were seen as proactive, delayable, can be done on the cheap, in house (or not at all).

@dymaxion

In my world, pentests are ordered by management, usually after a disaster, when they don't trust their team.

Code reviews are ordered by developers, management doesn't know they exist, and couldn't care less (and certainly wouldn't budget a third party to do them).

@dymaxion compliance

@kaoudis No, the pattern preceded compliance regimes being written to require it. I know because I watched it happen.

@dymaxion @kaoudis I guess back in the day this was about workforce distribution? People with pentest skills usually didn't work at places that required them + project distribution didn't justify hiring? There is also a conflict of interest if a person with pentest skills also works on the system under test.

@buherator
@kaoudis Code audits have exactly the same problem, but worse. On top of the bit where code review, especially security-oriented code review, is a largely unrelated skill to development, it's not just a conflict of interest but a practical problem. It's much harder to correctly review code you've written yourself, as you know too much about how it's supposed to work to see how it does work.

@dymaxion @kaoudis You are right, and now I think here lies the solution: with pentests you want priorities based on how an external party sees your system. Does this make sense?

(having to think about these things shows how much time passed since I started doing pentesting...)

@buherator
@kaoudis I mean, that's definitely where it started — "the attacker's point of view" and all that. That's also why back in the beginning of my appsec code review days we'd get handed a zip file of code without even an explanation of what the system did, let alone a running instance. I feel like by the mid to late aughts we'd gotten away from that, at least with competent dev shops.