How can we track AI vs human-generated code in our projects?
Asked by
CuriousCoder38
Hey everyone! Our team has recently started using Amazon Q Developer, and management is keen on understanding how much of our code is generated by AI versus what our developers write manually. We're looking for a few things: 1. Methods to differentiate between Q-generated code and human-written code in our repositories. 2. Tools or techniques to quantify the ratio of AI contributions versus manual work. 3. Best practices for assessing how AI-generated code impacts overall productivity. We've already considered Amazon Q's built-in analytics, but it seems more focused on usage metrics instead of actual code tracking. We use primarily Python and JavaScript, so any tooling suggestions in that direction would be great. Thanks for your insights!
Answers
You might want to reach out to Amazon Q Developer directly. They might have insights or features that enable recognition of its own generated code which would help with your tracking efforts.
Reply to this answer
I think management is missing the point here. Instead of focusing on who wrote the code, they should look at existing engineering metrics like cycle time and defect density to see if using AI tools is changing productivity over time.
Reply to this answer
It's hard to effectively track which code is AI-generated; it's a slippery slope that kind of resembles the old 'lines of code' metric. I suggest assessing the expected effort for a task and comparing it with how quickly a dev can complete it—that might give you better efficiency insights.
Reply to this answer
Tracking AI-generated vs. human-written code is quite tricky. There's really no perfect way to tell the difference. It's much like trying to determine if code was copied from Stack Overflow and slightly modified. If an AI tool created 80% of the app and you tweaked it, who gets credit there? That's a big question!
Reply to this answer
Honestly, why does it really matter who generated the code? The key factor is who approved the changes and who's responsible for the pull request. We tackled this at my last job by focusing on integration and approval rather than tracing the origin of the code.
Reply to this answer
Here's a cheeky idea: why not have Q prefix all the names it generates? It sounds goofy, but it could work! Also, consider having developers write a comment with the original prompt at the top of their files for better tracking of AI input.
Reply to this answer
Add your answer
Answers are reviewed before publication.
Related Discussions