ChatGPT-5 and Editing: Still a Novice Editor That Can't Be Trusted

This post is part of our Self-Publishing Series, designed to provide important information for writers considering the editorial responsibilities, benefits and costs associated with publishing their own work. For details on available editing services or to learn more about getting your writing into print, contact PJC Editing for a free editorial consultation.

ALSO IN THE SELF-PUBLISHING SERIES

What is Proofreading? An Overview of Editing's Final Stage

Creative Versus Critical Eyes: Why It's So Hard to Edit Your Own Writing

The Difference Between Writing and Self-Publishing a Book

Proofreading is More Than Spotting Spelling Errors and Tossing Typos

Problems Arise When "Someone" Edits Your Work

*****

If you are in the writing or editing business, it's hard to make it from sunrise to sunset without reading someone's opinion on AI's presence in the field. Scrolling LinkedIn, X or any of the hundreds of public writing forums, you'll find a range of carefully crafted analyses and all-caps, punctuation-free rambles on the ethical and existential dilemmas raised by ChatGPT and its AI competitors.

Let's skip past all of that and address one of the key questions related to AI and human editorial professionals: Can AI, specifically the newly released ChatGPT-5, replace human editors?

ChatGPT-5 launched last week and is supposedly "the most capable writing collaborator yet," per OpenAI. So, like I've done in the past, I handed the current version a basic editing test.

The goal is not to see if the software can edit — it can, fairly well. The goal is to determine whether or not it can be trusted to do the full job of an experienced editor. If an editor can identify errors in isolation but can't be trusted to fix and enhance the entire work, they serve no purpose to the writer. Would you trust a dentist who can read an X-ray with precision but is a little shaky on root canals?

One of the hottest AI/writing debates, especially in self-publishing groups, is whether writers should pay hundreds to thousands of dollars for professional copy-editing and proofreading services or simply pass their manuscripts, articles and business material to low-/no-cost AI programs. If ChatGPT can edit my fantasy novel for no cost, the pro-AI side says, why should I pay a copy editor three cents a word? It's a reasonable question.

For this test, I fed ChatGPT-5 a few sports-writing samples. Sports articles are fact-based and easily verifiable, which makes them great for this type of assignment. And as a longtime sports editor and writer, I know where mistakes tend to pop up.

I'll get the giant whopper out of the way first:

I gave ChatGPT-5 the same published article twice with the same prompt. ("Copy edit the following article. Highlight suggested changes but do not make the changes.") Both times, I added to the original article some basic errors that are common in these types of pieces. For example, confusing similar names/places, misspelling a name, misattributing a quote, counting errors.

The good: The edits were surprisingly strong and insightful. It picked up inconsistent spelling of names (that I inserted), flagged a location error (that I inserted) and suggested a fact-check, and provided the rationale behind its edits (some of which I didn't agree with or found confusing).

The bad: It gave completely different edits on each pass. The changes it suggested in the first read were gone or different in the second. For instance, it flagged the use of "portal" as a metaphor (GPT said "the metaphor might be overextended"), but let it slide in the next read. It also failed to pick up a blatant error — specifically, repeating the full introduction of a speaker on second mention later in the story. It missed a location error (this article cited two locations starting with H, and the "writer" swapped them by mistake in this particular reference). It also recommended that I remove a comma that didn't exist. Major red flags.

Can ChatGPT-5 edit? Yes.

Can I trust it to edit like a professional? No. Not even close.

Before that test, I had written a very short, basic summary of that day's Yankees–Astros game. I prompted ChatGPT-5 to copy edit, which it did reasonably well.

Then ChatGPT wanted to flex its muscles, which I'll admit are impressive. Or, can be impressive if a professional is carefully monitoring them.

"If you’d like, I can also give you a slightly more sports column rewrite that adds a bit more punch and narrative flair while keeping it factually tight," ChatGPT offered. I accepted.

Unfortunately it couldn't live up to its own hype. ChatGPT introduced its own fact error into this punchy rewrite.

The rest of the fluff that it suggested, including a "New York Post-style, snarky back-page article" and a "brutal, takedown-snark version" (GPT's words), while certainly well executed, were merely for show. I can't trust its output for anything beyond curious entertainment.

If you're looking to put professional-grade writing into the public marketplace, go with a vetted professional editor, human edition. ChatGPT-5 is a novice editor with a lot of flash but little polish, zero professionalism and no ability to coach you through your work.