We should ban AI generated content altogether.
I can do no better than to quote an excellent similar response over on SF&F
I propose that we follow other network sites in banning AI-generated copy-pasted content. This stuff is (to use a technical term) dangerous bullshit. As you've noted, it looks plausible and well-written, but to an expert eye (sometimes even a non-expert but humanly intelligent eye) it's clearly nonsense. It's one of the facts of SE that people sometimes upvote incorrect answers that look plausible - not every voter has the expertise necessary to judge whether an answer is true or false. So if we allowed this stuff, we'd run the risk of getting upvoted answers that are not only incorrect but were never even written by anyone thinking they were correct - the computer-generated equivalent of shitposting.
If you see something that looks at first glance like an answer but to a knowledgeable eye is clearly nonsense, feel free to mod-flag it.
So yes, let's have none of this nonsense, and please be proactive in flagging apparently AI-generated posts. (I have very little experience with AI-generated text or how to tell if a post is AI-generated or not, but there are mod-only spaces where I can get advice from people smarter than me at this kind of thing. Let's not discuss publicly how to detect these posts, beyond the very simple criterion mentioned above, otherwise the people posting them might make them less easily detectable.)
Using AI as a tool to assist generating good quality content might be reasonable, in some cases. I'm not proposing (yet) an outright ban on any answers where any kind of help from AI was used. But using AI, or other tools, to generate a large quantity of answers, without regard to whether those answers are correct and actually answer the question on which they are posted, is not acceptable. Content copy-pasted directly from ChatGPT output must be attributed to its source, or risk falling afoul of plagiarism rules as well as new policies against ChatGPT specifically.