doc: create ai-guidelines and include to CONTRIBUTING#62105
doc: create ai-guidelines and include to CONTRIBUTING#62105RafaelGSS wants to merge 2 commits intonodejs:mainfrom
Conversation
Co-Authored-By: Beth Griggs <bethanyngriggs@gmail.com>
|
Review requested:
|
|
There may be some ideas we can borrow from https://llvm.org/docs/AIToolPolicy.html - for example "good first issue" should not be picked up by AI is a good one. |
Co-authored-by: Aditi <62544124+Aditi-1400@users.noreply.github.com> Co-authored-by: Joyee Cheung <joyeec9h3@gmail.com>
I took inspiration from https://github.com/zulip/zulip/blob/main/CONTRIBUTING.md#ai-use-policy-and-guidelines |
| * **Do not post AI-generated messages** in pull requests, issues, or the | ||
| project's communication channels. |
There was a problem hiding this comment.
What is the intended action to take when this occurs? This currently comprises a significant proportion of GH traffic on the core repo; if we are making this a guideline, I would anticipate it applying to a lot of issues/PRs.
There was a problem hiding this comment.
If that recurring, we should ping the author to stop that, if that doesn't work, we escalate it to TSC/Moderation.
There was a problem hiding this comment.
I would recommend allowing pull requests descriptions and commit messages. It's quite useful for those. I would limit this to "when responding to humans".
There was a problem hiding this comment.
I generally disagree with pull requests descriptions. I find them too verbose and stating the obvious.
There was a problem hiding this comment.
nit
| * **Do not post AI-generated messages** in pull requests, issues, or the | |
| project's communication channels. | |
| * **Do not post AI-generated messages** in pull requests, issues, or the | |
| project's communication channels. Including a quote from a chat-bot, or using | |
| such tool to rephrase (or translate) your message does not count as an | |
| "AI-generated message". |
EDIT: hum I guess that's already covered by the last item. What I find weird is that we start by saying "Don't send AI-generated messages", while we're still accepting AI-generated content 🤔
There was a problem hiding this comment.
What about something like:
| * **Do not post AI-generated messages** in pull requests, issues, or the | |
| project's communication channels. | |
| * **Do not post messages generated entirely by AI** in pull requests, issues, or the | |
| project's communication channels. |
So it emphasizes it is fully AI generated, extremely verbose descriptions that suck.
|
|
||
| * **Edit generated comments critically.** LLM-produced comments are often | ||
| verbose or inaccurate. Remove comments that simply restate what the code | ||
| does; add comments only where the logic is non-obvious. |
There was a problem hiding this comment.
I would add a "do not disappear" section. Fly-by PRs are problematic. It's ok to close PRs if one cannot pursue them further, while stalling block progress.
| * **Do not post AI-generated messages** in pull requests, issues, or the | ||
| project's communication channels. |
There was a problem hiding this comment.
I would recommend allowing pull requests descriptions and commit messages. It's quite useful for those. I would limit this to "when responding to humans".
| * **Verify accuracy** of any LLM-generated content before including it in a | ||
| PR description or comment. | ||
| * **Complete pull request templates fully** rather than replacing them with | ||
| LLM-generated summaries. |
There was a problem hiding this comment.
Do we have a template? I thought those are for issues, not PRs.
There was a problem hiding this comment.
Not strictly a template: https://github.com/nodejs/node/blob/main/.github/PULL_REQUEST_TEMPLATE.md?plain=1
| [commit message guidelines][]. | ||
|
|
||
| * **Test thoroughly.** AI-generated code must pass the full test suite and | ||
| any manually written tests relevant to the change. Existing Tests should not |
There was a problem hiding this comment.
| any manually written tests relevant to the change. Existing Tests should not | |
| any manually written tests relevant to the change. Existing tests should not |
|
|
||
| Node.js expects contributors to understand and take full responsibility for | ||
| every change they propose. Pull requests consisting of AI-generated code the | ||
| contributor has not personally understood, tested, and verified will be closed |
There was a problem hiding this comment.
nit
| contributor has not personally understood, tested, and verified will be closed | |
| contributor has not personally understood, tested, and verified shall be closed |
There was a problem hiding this comment.
Changing it from "will" to "shall" makes it mandatory.
"will" is however fine in everyday language as a statement of intention.
There was a problem hiding this comment.
What I'm trying to convey is that we cannot guarantee that such PR will definitely be closed, but it's the intention. I guess we could use might instead:
| contributor has not personally understood, tested, and verified will be closed | |
| contributor has not personally understood, tested, and verified might be closed |
There was a problem hiding this comment.
I understand your intentions and I've made an alternative suggestion in #62105 (comment) which should indicate that the likely outcome is for the PR to be closed, without actually guaranteeing this action.
|
|
||
| Pull requests that consist of AI-generated code the contributor has not | ||
| personally understood, tested, and verified waste collaborator time and | ||
| will be closed without review. |
There was a problem hiding this comment.
| will be closed without review. | |
| will be subject to closure without additional review. |
| contributor has not personally understood, tested, and verified will be closed | ||
| without review. |
There was a problem hiding this comment.
| contributor has not personally understood, tested, and verified will be closed | |
| without review. | |
| contributor has not personally understood, tested, and verified will be subject to closure | |
| without additional review. |
MikeMcC399
left a comment
There was a problem hiding this comment.
Added some minor text suggestions about PR closure
As discussed in today's TSC meeting.
cc: @nodejs/tsc @BridgeAR