Is anyone using GitHub Copilot in production?

Question

I'm piloting Copilot as part of an internal trial at my company, and I'm wondering if anyone is actually using Copilot in production or if it's hype driven by non coders. From my testing in Swift, it's certainly able to "write code", but the code it tries to generate is almost always incorrect. It definitely seems like more of a hijacker than a copilot&mdash;is this just because of the language I'm using (Swift)? Or is anyone actually writing better code faster?

solardev · Accepted Answer

I tried it for a couple months (Typescript) in production. It was about 80% accurate, and useful for some snippets, but the 20% of errors were subtle and time consuming to debug. I canceled it.
These days I will occasionally use ChatGpt to ask a coding question (alongside Stack), but automatic AI autocomplete just gets in the way for me. I turn off all autocomplete, generally. The time it saves when it's right is overshadowed by the time it takes to manually fix its errors, especially when it takes me out of the flow.
I don't doubt that it can write higher quality code than I can when properly prompted, but doing so in autocomplete and with a high error rate was more disruptive to this human than helpful. YMMV.

muzani · Answer

Yeah, our Swift team don't use it. How are you using it? Copilot doesn't seem supported by Xcode. It has the most features on VS Code.
It does an excellent job on Kotlin, especially tests. We find it's most fluent with Python, pretty good with JS and TS, but found several security mistakes on PHP, etc.
For boilerplate, accuracy seems a little higher than human. It wrote 1000 lines of mocks & test code in a day for me. But creatively, it's worse, there's some hallucination.
There's a marked increase in quality and accuracy in GPT-4. GPT-4 is high quality enough that it catches tons of mistakes in code reviews and can debug with a screenshot of a stacktrace. Copilot seems like it's a generation or two out of date.

ativzzz · Answer

I just use it as an autocomplete. I can usually guess when it will save me keystrokes (when I am doing something repetitive) and will look out for it, otherwise usually I ignore what it gives me, but sometimes it surprises me and saves me more than a few keystrokes.
My company pays for it, I don't think I'd pay for it personally since I use chatGPT when I need more advanced help and copilot is just autocomplete.
I think it's only given me full functions a handful of times (and they're usually quite wrong), most of the time it's just a method name or a super short (1-2 line) code snippet

giaour · Answer

I don't use it on my own system, but I will sometimes pair with coworkers who use it. From what I have seen, CoPilot will suggest code that is either subtly wrong or completely broken alongside some really helpful comments. (There must be some selection bias here, since I wouldn't get called in to pair for situations where CoPilot's suggestion was sufficient.)

ctoth · Answer

Lots of luck with Typescript and Python, far less with obscure languages like LambdaMOO :)

yogini · Answer

I am using it for the Nextjs application.Mostly using it as an autocomplete but it is quite good for writing repetitive code and logic. Also the new slash commands are pretty useful.

kojeovo · Answer

I've been shipping code to production that it generates for months now. It's really good for some things. But yeah a lot of the time it's wrong.

cpach · Answer

I don&rsquo;t use it, but I&rsquo;m certain it has a quite large user base.Even if they would have only 1% of all developers world-wide (for example), that&rsquo;s still a lot of users.

fullstick · Answer

I use it at work for production code. I've found it very helpful for typescript. Also very handy for writing tests.

Is anyone using GitHub Copilot in production?

Lots of luck with Typescript and Python, far less with obscure languages like LambdaMOO :)

I am using it for the Nextjs application.
Mostly using it as an autocomplete but it is quite good for writing repetitive code and logic. Also the new slash commands are pretty useful.

I've been shipping code to production that it generates for months now. It's really good for some things. But yeah a lot of the time it's wrong.

I don’t use it, but I’m certain it has a quite large user base.
Even if they would have only 1% of all developers world-wide (for example), that’s still a lot of users.

I use it at work for production code. I've found it very helpful for typescript. Also very handy for writing tests.