Spec or code as source of truth depends on tacit knowledge

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 35:29

   one of the the mantras inside of facebook used to be that i think some something along the line that code is truth or something like that. But in general, in a lot of tech companies, if you want to know how things work, like where the truth lives, is it in documents or is it in the coded business logic? It's in the code in the business logic. And if it's well-maintained code, often well-commented or structured, you can actually see the business logic and the rules for, I don't know, for example.

2. 35:55

   See, I feel like I've got to channel my friend Charity at this point and say the truth is production. Everything else is the lies we tell ourselves. Production is truth. But I think in terms of like a definition of what the system should be doing, there is this ongoing debate about where is that definition? Is it the code or is it the spec? Brigitte Brockley did a nice article on Martin Fowler's website recently about spec-driven development.

3. 36:22

   And she talks about the idea of some companies being spec-first or spec-ankered. And a lot of it comes down to where they see the source of truth, being so a lot of companies are using things like kiro or spec kit to define a feature and then the llm executes that spec and you could Iterate on that process, right? And so a lot of people, they start off, they use the spec to write the code, and then they throw the spec away. And then you get other companies that keep the spec, and then when the feature changes, they just update the spec. And so they're sort of seeing a bit of the code and a bit of the spec as a source of truth in terms of what the system should do. And then you're now starting to get companies where they didn't even ever look at the code, and they just look at the spec.

4. 37:06

   And so we're kind of going on this spectrum. She had some great terms for these. And I think for me, it's like, is that the right path? I don't know. I think there's a lot of questions about what we would need to do to get that. We have a degree of confidence about what happens when I execute a line of Java code. Beyond the safety of the compiler and the runtime and all that stuff, there's a lot more questions to answer, I think, before the whole world is ready to say the spec is the source of truth of what our system is supposed to do.

5. 37:35

   production is always going to be the real truth so just to be clear but but i think it's a good question to just ask yourself what however you're building however using agents of where is the truth of your system right now is it more in the code do you have a spec is it scattered between the minds of people uh which goes back to the the collaboration because again a lot of the reality of a lot of startups are is a few people hold these things in their head right which is okay in the beginning but of course as you scale one of the reasons we used to love code review which is now going out of fashion but we used to share information and when someone left the company or or got ill or you know the the famous bus factor it was okay because other people understood because they reviewed they were taking part in that part of it but now we're with the ai code review is

6. 38:20

   some some some companies are like well we're not going to do it because of it's it's too much code that was always one of the superpowers of pairing right pairing you wrote if you do if you do promiscuous pairing you rotate every day when someone leaves everyone's got the same shared context. That was something people missed around one of the benefits of pairing. I think there's a lot to be said around this. I was chatting to Chris Ford. He's sort of head of software, head of technology for ThoughtWorks in Spain. And he said the biggest problem we've got around AI-driven software delivery is tacit knowledge. There's a lot of knowledge in our heads that we've built up that we don't necessarily know that we have. When you do something like a code review, You probably had looked at that and said, that doesn't really feel right.

7. 39:04

   I wouldn't do it like that. And sometimes it's subjectivity, but sometimes there's something about that that you can't quite pinpoint. Maybe you can unpack it. Maybe you can't. But it's also like you have bunches of things like that in your head about how software delivery should be done about how the business works and everything else. If we want to step up that level of abstraction to work at the spec level, we actually need as human beings to be able to unpack that tacit knowledge that we have and make sure it's in the spec.

8. 39:32

   If we really want to be hands off, I also think there's a fundamental problem that we've got in the current way a lot of people are doing software deliveries. You've got, and this comes back to the cognitive debt stuff, which is you've got a lot of people running off in different directions. I've seen a few people talking about pairing with their AI assistant, not a human. Stop pretending it is. And so you're all going off, you're doing more in a different direction.

9. 39:54

   That shared mental model of the program is breaking down and there's gaps in between. AI was supposed to free us from drudgery. right it isn't for most software developers because we're doing more work we've got more context switching we're losing that big picture so i think if we can work collectively on the specs we can rebuild that model then that absolutely could be the source of truth i still think it's going to be parts our brains and part the spec right but for that to work with there does have