A Greptile score loop makes agents fix their own PRs

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 21:32

   what Greploop is? Yeah. So Greploop is a skill by Greptile, which is a code review agent company. I am a big fan of them. There are other great code review agents as well. I just like Greptile. And what the grep loop skill does is basically it's this loop. And maybe I can give an example with a PR. Whenever you file a PR with when you have Greptile installed, maybe I can find one that doesn't have a five out of five.

2. 21:59

   But essentially, just to explain this, every time you make a PR, if you have Greptile installed, you're going to get a Greptile summary and a confidence score and some feedback. Feedback, for example, says do not assume cash funding. This is feedback. This is a P1. It explains and it even gives you some artifacts to check out. What I'm really interested in, though, is this confidence score, the way they have this confidence score calculated is really, really good.

3. 22:26

   most models, your Fables, even Astra, usually the first PR they make is a four out of five. Sometimes it'll be a three out of five, but with the state of the art models, you'll get a four out of five and you'll get one piece of feedback. What the grep loop skill allows your agent to do is, and I'll actually, why don't we read the skill itself because it will explain it properly. What this skill basically does, it says, I'll read it here, iteratively improves PR until Greptile gives a five out of five confidence score with zero unresolved comments.

4. 23:04

   Triggers Greptile review, fixes all actionable comments, pushes, re-triggers review, and repeats. Use when user wants to fully optimize PR against Greptile's code review standard. So basically, to draw this, because this is, when I heard of loops at first, wasn't really a fan, but loops in this sense are great. So I'm ready to merge my feature, but I am now running Greptile. What Greptile will basically do is it gives me a score.

5. 23:31

   My agent checks the score, and if the score, let's say, is a 3 out of 5, right, what essentially happens is if I go back to my skills so we can visually understand this, what essentially happens is we go back to step 2. So it takes the feedback in. It starts to write code. It will follow step 3. It will follow before and after, and then it'll push a change. The agent will then wait for Greptile's new review.

6. 23:58

   If it's a five out of five, it stops, and it tells me the PR is ready. This is a five out of five. What's essentially the time for me now is I can look at the code if I want to, or I can merge. I basically have all the evidence I need to merge. But what this skill does is it takes Greptile, which is a code review agent, uses the score as a way to check if the code generated by my agent was good, right?

7. 24:24

   Anytime Greptile gives any sort of feedback or a score under five out of five, the grep loop fires. It takes in the feedback. It writes the code. It pushes to GitHub. Greptile does its check again. And this loop doesn't stop until you reach five iterations. My version is 10 iterations, or you get a five out of five. Okay. So basically, and all these skills are open source and free, right? They're available on GitHub?

8. 24:51

   Yeah, this is all available on MyRepo. GrepLoop, again, is their skill. I just, I literally brought it in. And you might have seen a GrepLoop apps. This one is basically for PRs with 3,000, 4,000 plus lines. These are their skills, right? The only skills that are mine that I made are code structure and evidence-driven testing and new feature. I even have the unslop skill. This is from Potato, one of the engineers from Grok cursor now.

9. 25:17

   And this just, you know, unslops the PR. Again, to your point, Astro doesn't really follow this. It tends to skip unslop, which is quite fun. But this is essentially the crux of the workflow, right? Isolate, new work tree, write code to a specific standard, prove that a bug exists if a bug is what's being fixed. If not, prove that the solution you made actually works via video. If that doesn't work, before and after screenshot, if it's not screenshotable material, give me some stats, write some tests and give me some stats to actually prove.

10. 25:50

    Push this to GitHub. If the Greptile code review agent gives a score less than five out of five, we run a loop until we get a five out of five. And then what I end up getting is responses like this.