In the tech world, a lively discussion about code review is unfolding. The days of spending 2 to 5 hours each week reviewing code might soon be a thing of the past, a goal that CodeRabbit is striving to achieve.
Code review, that love-hate process for developers, has long been a crucial method for improving code quality. However, it is notoriously time-consuming and labor-intensive. Statistics show that half of companies spend 2 to 5 hours weekly on this. Worse, if resources are scarce, code review can become a bottomless pit, consuming developers' time and energy.
Image source note: The image was generated by AI, provided by the image licensing service Midjourney
At this juncture, Harjot Gill, co-founder and CEO of CodeRabbit, stepped forward, claiming they could significantly automate the code review process with artificial intelligence. Gill is no newcomer; he previously served as Senior Technical Director at data center software company Nutanix and founded Netsil, which was acquired by Nutanix. Co-founder Gur Singh is also a seasoned veteran, having led development teams at healthcare payment platform Alegeus.
Gill confidently stated that the CodeRabbit platform uses "advanced AI reasoning" to "understand the intent behind the code" and provides developers with "actionable," "human-like" feedback. Sounds impressive, right? Gill also took a jab at traditional methods: "Traditional static analysis tools and code inspectors are rule-based, often resulting in high false positive rates, while peer reviews are time-consuming and subjective. In contrast, CodeRabbit is an AI-first platform."
However, these bold claims are filled with buzzwords, raising doubts about their authenticity. In fact, evidence suggests that AI-driven code review might not be as reliable as reviews with human involvement.
Greg Foster from Graphite shared the results of their internal experiment using OpenAI's GPT-4 for code review in a blog post. While the AI model could catch some useful things, such as minor logical errors and spelling mistakes, it also produced a high number of false positives. Foster noted that even after attempting to fine-tune the model, the number of false positives did not significantly decrease.
So, can CodeRabbit really solve these issues? Or is it just another AI marketing gimmick? Currently, we haven't seen specific performance data from CodeRabbit, so we cannot make an accurate judgment on its effectiveness.
Nonetheless, CodeRabbit's attempt reflects the tech industry's relentless pursuit of efficiency. Even if AI cannot fully replace human code review, it may provide valuable assistance in certain aspects.
In the future, we may see more AI-assisted code review tools emerge. These tools might help developers quickly identify common errors, allowing them to focus more on complex issues that require human intelligence.