AI Power Showdown: Sourcegraph Cody vs GitHub Copilot - A Deep Dive into the Future of Coding

Let's move on to the next set of tools, focusing on coding with AI. Last time, I discussed the pros and cons of GitHub Copilot.

Since then, GitHub has released Copilot Workspace, which I'll cover in a broader AI and coding article. Before that, I want to examine two more tools: Sourcegraph Cody and AWS Q (in the next article).

Both are significant enough to be chosen by enterprise companies as an AI coding copilot.

When comparing them to GitHub Copilot, I immediately think of these questions:

• Is Copilot the only one innovating, with others simply following?

• Are there tangible differences between the AI copilots?

• Why should someone choose anything other than GitHub Copilot?

Sourcegraph Cody

Sourcegraph is a tool that originated from code search and code intelligence. It enables search and analysis over large codebases and is widely recognized in the industry, as it is used by companies such as CERN, Lyft, Uber, and Atlassian.

What distinguishes Sourcegraph from simple repository searches, like those provided by GitHub?

Sourcegraph facilitates complex search queries, like structural searches, and is easier to use with features like keyword search. It also operates significantly faster than other code search tools.

One of its most significant advantages is its support for multi-repository environments. If a company has parts of its code in different repository providers or even non-git platforms (note that Sourcegraph does not support all platforms), Sourcegraph allows you to search through all the code via a single user interface. This feature is crucial as your code is akin to your data. To understand and make sense of it, you need access.

Where does the AI come in?

With Cody, Sourcegraph added AI functionality to its portfolio, similar to GitHub Copilot. Sourcegraph had a head start as it had already indexed all your code and was ready to be used as an RAG for an LLM.

This was an impressive feature. Initially, enterprise context about your code was built with embeddings and later transformed into a more RAG-type approach using the search index as a feed.

This functionality was first seen in Sourcegraph, and only later arrived at GitHub. Like Sourcegraph Search, Cody works across all your repositories, which is a significant advantage for large enterprise companies.

Where Does It Excel?

Simple: The context it knows from your repos is a big plus. Be aware that, as it was not trained on your code, it still runs into the same issue: it can create a ton of hallucinations. But as we have not figured out how to train cheaper AI models, that is as good as it gets.

Across multi-repo is also a big plus, but I did not mention another big plus so far. It is not dependent on OpenAI. Cody can be connected to different LLMs, e.g., Claude, Mistral, and LLama, so you are independent of the model. If OpenAI would have imploded when Sam was fired, no worries, switch to Claude and keep running. A new model pops up and seems to be better for your purpose. If Cody supports it, that is no problem; you can switch to it. That is awesome and might be a big plus for a future nobody can predict.

What are the issues?

GitHub Copilot has been doing an excellent job recently in adding a couple of nice features, and it will be interesting if the team from Sourcegraph can keep up with that.

The plugins for each IDE were quite buggy in the beginning, especially for JetBrains, and some functionality is still missing, which is only available in VS Code.

All of the AI vendors lack integration with CI pipelines. I would love AI functionality out of my CI, such as checking a pull request to see if it meets specific standards (e.g., SOLID). That would be awesome, and the output would be printed as a comment with code suggestions into my pull request.

Personal Experience

I work in a company that uses Sourcegraph, so I have extensive personal experience with the tool.

Autocomplete (code and tests)

It performed for a long time worse than GitHub Copilot, especially its performance. Recently, we were able to switch to Starcoder for autocomplete, which greatly enhanced the stability and performance. Again, it was a big advantage to be able to switch the Autocomplete LLM as well as the Chat LLM.

Help me fix a bug

There is not much to say here. It is an LLM, so it helps you, but it has the same communication issues as every other LLM. You can switch to another model, which might work better for your use case (check the LLM Litmus Test page for help).

Create documentation, commit messages, or pull requests

As usual, it is very good here but can create a lot of unnecessary wording. It's comparable to GitHub Copilot, with the only exception being GitHub's pull request UI, which appears more appealing than Cody's commands. However, commands tend to be more powerful. If you need help writing commands, there's a valuable tool available to help you write custom commands.

Create a new project

So, that is a bit different here from GitHub Copilot. You can move to another model, e.g., Claude Opus 3, and suddenly, as this model was trained with more recent data, it can give better answers. This is especially true if you work with new technologies or frameworks.

Conclusion

Okay, so Sourcegraph is a great tool. Its roots are in search, but that gave it a head start when feeding the LLMs with more context than just the data they were trained on. You will also have an easier way to integrate that into your landscape if you do not only use GitHub as your code repository. The biggest advantage for me is that you can switch LLMs and take the one that is more suited for your purpose.

The pricing differs, but remember that you get more than just AI. Consider their other tools, such as Code Insights or Batch Changes. Depending on the size of your organization, these tools can be beneficial in identifying problems.