Hacker News
points by joshstrange 2 days ago | 0 comments
Agreed. It's sad how bad GH Copilot is, and thank you for reminding me to cancel that subscription. I just realized I haven't used it in a while now.

We also settled on a custom solution for reviews. We started with reviewers that claude/codex on GH actions and those were "ok" but kind of dumb and hard to tweak (GH Actions is just miserable to debug/iterate on). We still use them occasionally (@tag them to kick off a review) but we've shifted to a piece of software that we wrote that watches GH PRs and once certain labels have been added and the ticket in our ticket tracker (not GH issues) is marked as "Ready for Review" it kicks off a review and it reviews _all_ the PRs connected to the ticket. We don't have a monorepo, yet, so a new feature might span 3-4 repos (server, client1, client2, ...) and we want that work reviewed together.

We are close to having our review code spin up a fully isolated devstack running all our, many, services so it can poke around in the UI and write custom playwright scripts to test the new functionality. We have the devstack VMs working, just gluing the last of it together, it's pretty cool to see it all come together.

[–] othmanosx 2 days ago
What about after the bot reviews, did you come up with any solutions to speed that up? I’m talking about the human part of the review
[–] joshstrange 2 days ago
We’re still experimenting. Automatically fixing obvious issues raised by the agent (at another agent’s discretion) or presenting a list of the feedback items with fix/refute suggestions are things we’ve toyed with.

Personally I have an agent review the feedback and break down each item so it’s easier to review (helps remove some of the jargon/short-hand that agents use sometimes). Then I decide what to have an agent address and what to push back on. Like I said, I’m working on finding a way to automate part of that but haven’t settled on one I like yet.

[–] othmanosx 2 days ago
I’m having issues with agents not applying the rules to begin with, during both implementing and reviewing, it just reads the rules and choses to ignore them sometimes, adding more rules just makes this more likely to happen, and having review cycles can potentially get to a state where two agents talk nonstop and increase the slop I need to read instead of reducing it.