Muse launched only when every behavior on a 100-row checklist was green

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 21:49

   how reliably they like, you know, check back with the user if there's things that they require guidance on. But how did you know it was ready to be released? Part of it was pretty numerical, which is like we had set, you know, go back to that spreadsheet of 100 plus things. We had set for each of those rows what threshold is launch blocking and above what threshold is good. We had many checkpoints that were like green on 80 of the rows, but red on 20 of the rows.

2. 22:17

   We knew that wasn't good enough. And we had our first model checkpoint, which was green across the board, which was a specially trained version of MewSpark 1.3. So there was the very numerical part of it. And then there was just the experiential part, which is we tried the model. We could tell that it was a lot better than anything we had iterated with before. So going back to that seven-month period you were talking about, how many people were working on the product?

3. 22:41

   Between the model work and the product work, like under 200 people total, which I guess sounds like a lot, but in big company land. It's actually a lot smaller. It's not very many. It was kind of cool looking back on this and thinking about it. We knew this was going to be the most important product. The whole executive team, everyone knew that this was probably going to be one of the most important products that Meta would ship this year.

4. 23:09

   But we had the restraint organizationally, I think, to make sure it was a really small, focused team working on it and that it didn't balloon to this, like, oh, you have like, you know, huge swaths of the company working on it because we knew it was a single work of art. Like it required strong point of view and strong focus to like pull that through into one beautiful experience. Say more about this. Why did you just call it like a single piece of art and you need a single perspective or point of view? I think one criticism of products that come from larger organizations.

5. 23:47

   And I think this is even now starting to plague a lot of the AI labs, is that they become kind of this Frankenstein, almost like Cronenberg amalgamation of a bunch of different people's points of view and visions and beliefs. And it's sort of like PM hell in some sense, where every PM has a thing that they're trying to get into the product and jam into the product. And then all of that kind of like blends together into this like smoothie almost.

6. 24:19

   People feel it as users and consumers. You can feel when a product is just like, it feels like lots of different people worked on it. And there were like clearly different people who were gold differently. So they just kind of like jammed it all together. Muse had to be the exact opposite, which is like, there's one clear point of view. There's one cohesive experience that we're trying to deliver to the world.

7. 24:43

   I love that you said that, and it just happens to be pure coincidence. I don't know when this is going to come out, but we're recording this on the 15th anniversary of Steve Jobs' death.