Reformed Theological Seminary's report on Chat GPT

Status
Not open for further replies.
Thank you for sharing! Is this available for sharing with the general public? I know a few folks who would greatly benefit from this. A few quotes that caught my eye:

Verdict
• ChatGPT was able to score an A- or B+ on one of my Gospels midterms
consisting of short answer and essay questions; the answers, of course,
missed out on certain nuances I cover in class, but the overall quality
is possibly better than the average student’s
• Book/chapter summaries for virtually any book (ancient or modern)
can be easily produced by ChatGPT; it can even attempt to summarize
journal articles, but often the results are generic or nonsensical
• The engine is not (yet) able to produce documentation for papers,2
and it tends to glitch out for long requests; it could not produce, say,
a 15-page exegesis paper but could be used to piece one together if a
student structured a series of queries well
• That said, ChatGPT could be a remarkably useful for preparing study
guides or developing leads for research topics
 
ChatGPT (or any other similar service) is only as good as its programmers, and those programmers are--emphatically--mere men, flawed, limited, and sinful.

The following article, if true in the substance and no hoax in the matters which are reported, is alarming; but perhaps should not be surprising. I, for one, am more surprised at what seems like naivete, on the part of the reporter who writes, "Presumably the people who developed ChatGPT didn’t program it to lie." Well, why not? Why is it reasonable for a Christian to suppose AI, created by man, could not be programmed to lie?

 
The following article, if true in the substance and no hoax in the matters which are reported, is alarming;
Yes, it is true. I’ve been learning of many instances where it just makes up stuff. But a lawyer using ChatGPT without cite checking is like carrying nitroglycerin in a martini glass.
 
Yes, it is true. I’ve been learning of many instances where it just makes up stuff. But a lawyer using ChatGPT without cite checking is like carrying nitroglycerin in a martini glass.
As said the reporter: "ChatGPT came up with five other non-existent cases. The lawyers are in deep trouble." Trying to get clerk-work done on the cheap, I guess. If you lose your law-license, how are you going to pay for the JD degree you maybe went 6-figures in the hole to attain? Not everyone can become a successful internet pundit or a talk-show/podcast host...
 
The lawyers are in deep trouble.
Bruce,

Indeed, made quite clear if you click on the court's actual "show cause" order! Don't go into court (or class) relying on something that's not factually reliable.

I can testify, as a historian, that I've found it wrong on a number of historical details, often being quite off even on dates. I wouldn't have known how far off or wrong it was if I weren't otherwise well-versed in what it purported was factual. Caveat lector!

Peace,
Alan
 
Some professionals recently subjected ChatGPT to the CPA exam. It bombed completely. It's an impressive entertainment tool that shows off some common grace genius in its programmers, but it is unfit and unreliable for any meaningful work.
 
I have messed around with it and found it helpful. But it makes up quotes; most of the time, it incorrectly cites quotes from sources. If you ask it follow-up questions like, "Are you sure this is from the source" it may catch its error and recite. Even then, I found it cites incorrectly.

In short, its helpful to obtain information quickly on a topic. It is beneficial to generate ideas, but nobody should rely on it for facts, primarily works cited from sources. I think most universities will need to adjust their testing models.
 
Status
Not open for further replies.
Back
Top