AI and Sermon Prep - How Far is too Far?

Status
Not open for further replies.

davejonescue

Puritan Board Senior
Hello all. Was on a discussion on Facebook in a Logos forum on a post of someone who was promoting using AI to formulate Sermon Outlines and Sermon Composition. I opposed this by the opinion that Sermon Preparation should be wholly Spirit led and done by the Pastor. Surprisingly getting some push-back, and after seeing some videos, it actually seems Logos itself is pushing pretty hard for Pastors to embrace AI in sermon prep. As Pastors (which I am not) where is the use of AI going too far, if it is; and if not, what are some of the arguments for its involvement in the creation of Sermons? Will probably not be personally responding much to this, but reading and observing from those who actually do the work of Sermon Prep. Below is a video of the current and former CEO's of Logos discussing AI in Sermon Prep.

 
Last edited:
Do you mean using AI as a tool in research or merely using a LLM to write a sermon?
 
The only method of sermon preparation I endorse is:
1. Pray for illumination.
2. Read the text in Greek or Hebrew.
3. Read a commentary if you need help understanding it.
4. Pray over and meditate on each verse.
5. Write your outline and include other texts you plan to cite, as well as applications.
6. Commit it to memory. The ideas, not the exact words. Don't be reading a manuscript in the pulpit like you're an 17th century Anglican priest doing English Popish ceremonies.
7. Preach it.

We shouldn't be ordaining men that can't read a text, understand it, and apply it.
 
It seems like a way to ensure one's sermons lean left, according to recent research.

Obviously the act of interpretation should be Spirit led, but this includes the use of means, WCF 1.7. I would not include AI in the means. Garbage in, garbage out. There is a hidden element in the input. At least with commentaries you can discern the bias of the author and can factor that into your critical appraisal of what he says. With AI you are left guessing where it comes from.

But I suppose it might be interesting from an analytical pov to see what garbage it dishes up.
 
Imagine you're going steady with a nice lady and she writes you these love letters that you really value, even though they're kind of vague and generic.
Then when you're about to get married you see them all in her chatGPT history and realize she didn't write them at all.
You're devastated.

That's what this is like.
 
Imagine you're going steady with a nice lady and she writes you these love letters that you really value, even though they're kind of vague and generic.
Then when you're about to get married you see them all in her chatGPT history and realize she didn't write them at all.
You're devastated.

That's what this is like.
Off topic, I was the best man at a wedding last year where both the bride and groom opted to use AI to write their own vows. They were pretty decent. Lol

As for sermon prep...no!
 
For research? Sounds great, I genuinely believe it can be a helpful tool for that.
For writing a sermon? No AI is going to know the spiritual needs of a congregation, and no AI is being guided by the Holy Spirit. Plus, you cease to grow in your own sanctification from sermon prep if someone else is doing the prep
 
I’m sure there’s an AI chatbot that could pretend for us
Chats response:

The Puritans, who were deeply committed to their faith and the meticulous study of the Bible, would likely have a mixed reaction to using AI for sermon preparation.

1. **Concerns about Authenticity and Dependency:** The Puritans placed a high value on personal study, prayer, and divine inspiration in preparing sermons. They might express concern that using AI could lead to a dependency on technology rather than relying on God’s guidance and the Holy Spirit. They might argue that sermons should come from deep personal reflection, spiritual discipline, and a genuine encounter with Scripture.

2. **Integrity of the Message:** The Puritans emphasized the integrity and authenticity of the preacher’s message. They might worry that AI-generated content lacks the depth and spiritual insight that comes from a minister's personal experiences, convictions, and spiritual struggles.

3. **Tool vs. Divine Aid:** On the other hand, the Puritans were not entirely against the use of tools that aid in study, such as commentaries, concordances, and other scholarly works. If they viewed AI as a tool to gather information, suggest outlines, or highlight Scriptural connections, they might cautiously accept it as long as it is seen as a supplement rather than a replacement for personal spiritual labor.

4. **Discernment and Critical Engagement:** The Puritans were also known for their rigorous discernment and critical engagement with ideas. They would likely encourage pastors to carefully evaluate any material produced by AI, ensuring it aligns with sound doctrine and Scriptural truth.

Overall, while the Puritans might appreciate AI's potential to assist in research and organization, they would likely insist that it must never replace personal study, prayer, and the guiding work of the Holy Spirit in sermon preparation.
 
Current LLMs have no business anywhere near the serious exegesis/interpretation/translation of the word of God, or any legitimate theological work. Compare the minister with the AI:

The Minister's "dataset" is almost exclusively the word of God and the best theological minds in history. Consider how his dataset compares with that of Silicon Valley.

The Minister's "training" is the Holy Spirit's call and work, the seminary education, and the experience of pastoral ministry itself. Consider how this training compares with the AI's: having tech "brothers" vote on the better of two outputs over and over again.

The Minister's motive is to stand blameless before the Lord on the Last Day as he gives an account for every soul he ever pastored. The AI is only capable of making predictions on the appropriateness of the next word in a sequence.
 
A generalized LLM is a bit like an unschooling taken to the extreme with locking a child with a photographic memory and rapid reading in a library with a vast array of uncurated content on a whole range of topics for decades and then you start asking him questions and supplying little correctives here and there.

----

A bespoke LLM is very intriguing as a potential research assistant. Bob Jones University's work with Sermon Audio I suspect will bear some interesting fruit and might be a much better, if more limited resource. Their first big project was to provide audio transcription of the remaining 3000 Spurgeon sermons with the voice of the guy who'd recorded 500 since the 1980s (fully authorized).

This would actually be a very interesting area for Christians to work in building ethical and "sound worldview" tools that are somewhat limited in some ways by curating out the "more knowledge is a good thing" indiscriminate LLMs. Essentially, you'd need to give it an accelerated and focused education that's quite broad and yet with serious moral bearings. It'd be a bit like sending one incredibly capable but undiscerning and naive student to a top-flight Christian school and needing an entire faculty to nurture the child into a sound thinker.

Honestly, you might need an entire university and seminary faculty to dedicate significant efforts to keep the LLM on track, as its main project, especially as it takes in and incorporates new knowledge.

----

The theological catch is that a believer is indwelt by the Holy Spirit, but there's no reason to the think the Holy Spirit would indwell a creation of man to guide and protect its judgment. On the other hand, I think there's every reason to think that the demonic could and would use inanimate objects and creations to "move" and "function" like the ancient pagan gods sometimes might. Even a well-educated "trained only on good stuff" model could still warp.

That's more than a bit of a scary proposition when your "assistant" knows far, far more than you in a way that's fundamentally different from an encyclopedia, massive commentary set, or even seminary/university library's knowledge repository.
 
@davejonescue if you don't want your pastor preaching with ChatGPT, why are you using AI to translate books?

Is putting AI garbage into the mouths of the great pastors and theologians of the past more ethical or righteous than your pastor speaking AI garbage from the pulpit?

I don’t believe they are the same. Creating a sermon outline from scratch using AI is fundamentally different from translating an existing text from one language to another.

There are so many remarkable works, especially from the Dutch tradition, still locked away in Latin, and I would genuinely appreciate reading them, even if the translation were less than perfect.
 
I don’t believe they are the same. Creating a sermon outline from scratch using AI is fundamentally different from translating an existing text from one language to another.

There are so many remarkable works, especially from the Dutch tradition, still locked away in Latin, and I would genuinely appreciate reading them, even if the translation were less than perfect.
"Less than perfect" is an understatement.
 
The best AI translation models (I've done extensive work with DeepL. for example), even for modern English to Spanish, are still pretty bad. With theological work, they're horrible. With old theological work, they're worthless. 17th-century Latin theology? Forget about it. Not to mention the amount of editorial labor it takes to really bring these old works back to life—tracking down citations, footnoting, etc.

If a person wanted to explore these old works on his own, that's one thing, but to publish this stuff, to print it, to suggest that it's accurate—that's playing with fire. I think this is the trend that alarms and frustrates me. The translation of a theological work from the Latin requires not only an understanding of the language, it requires mastery of the theological subject matter as well as its historical context. A process where AI spits out a (poorly constructed) literal translation and someone merely edits to make sure it "sounds about right" is more likely to pollute—even do permanent harm to that locked-away body of work than it is to preserve it.

The best way to preserve this work is to train up more young men to translate with reverence and care, but if people start to believe that this is a job for AI, who's going to want to do the hard work?
 
Not to mention that this is a work which is being translated by a human translator, for RHB, to be published in short order. Which I'm sure our friends at monergism are aware of. So it's not as if the options are "AI or nothing." Actually, the options are, "AI or wait a minimal amount of time for something far better."
 
@davejonescue if you don't want your pastor preaching with ChatGPT, why are you using AI to translate books?

Is putting AI garbage into the mouths of the great pastors and theologians of the past more ethical or righteous than your pastor speaking AI garbage from the pulpit?
As a person who does books, I am sure you are aware that as an employee you do not get to make decisions above your pay-grade. While I have dabbled with DeepL translating Dutch and French, after going over those translations I have altogether given up on the idea of auto-translation for a couple years now. At best it may be profitable to get a personal gist, but it is not good enough, in my opinion, for wide-spread publication. I have a copy of Vos' 1910 Dogmatics that I got from Princeton, in Dutch, but for me there is no use in doing anything with it because I dont know Dutch. But that was back when I thought DeepL could really be something. As far as Latin goes, I would never (at least at the moment) AI translate something extensively, especially a whole work; and publish it. Researching the language, it seems there are too many areas it can be misinterpreted. This is why I only deal with English texts. This is why I have yet to deal with English texts heavy in Latin like Leighs "Body of Divinity," or Twisse "Riches of Gods Love," both texts that are pretty easy to get out, but are riddled with Latin. Most contemporary readers do not know Latin, and while a splash here or there I dont see as harmful, entire paragraphs throughout a book is too much for normal audiences and kind of a waste of time (in my opinion.) We might do Leigh in the future, but if we do it will only be by popular demand, and if I do it, all the Latin will be included as is.

But again, I dont own Monergism. And frankly some things in life are trial and error. I dont have the right to tell the owner what they can publish no more than anybody working as a side editor for RHB or Banner has the right to tell them what they can publish. If people contact Monergism, which I am sure some will, and point out to them all the places the book is in error, then I am pretty sure they will pull it. But as far as I can see they seem to be testing the reaches of the present technology, which is bound to happen. Sometimes you dont know until you try.

If I had AI translated this work, and felt such confidence in doing so, that I would deem it good for ePublishing, I would have let this board know it was out and posted it as a free title. But I didnt. Not because it may not be, but because I dont know Latin and cant verify if it is or not. Yet, such is life. That is the nature of being a subordanite, that is, you have to stay in your lane. It is not a heretical book, so it doesnt bother me they did it. But since again, I dont know Latin, there is no way I can point out what may be in error to dissuade them from doing it now, or in the future; or, if they need to be dissuaded at all.

Some of it I think also is what someone mentioned earlier. There are so many works in Latin, Dutch, etc. that have yet to be translated; that people just get antsy pants and if they see a way they may shed light on them, they do it. Until it can be proven to them that it cannot be done, then simply saying "it shouldnt" has little bearing. That is one of the issues of doing free books. Is you have to take into consideration when dealing with complaints, is if they are in fact legitimate, or if they are coming from a place where people dont want what they deem as competition, or want the "old ways" to remain. There is nothing wrong with the "old ways," only as it remains there are no new ways which surpass them.

If you think the book is in error. Contact them and point out the ways it is. You are the self-professed expert in Latin. But like others have said as well. I do not place translating books, whomever they are from, or however they are translated, on remotely the same level as a Pastor using AI weekly, to compose sermons for a living, breathing congregation.
 
Last edited:
That's rough! I've used AI in very minimal ways in a youth ministry context for illustration related things (such as "what are some examples of movies that follow X plot formula") but not at all in the irreverent ways that it sounds like this individual on the Logos forum was promoting. That is, nothing of substance (biblical, theological, or otherwise) and nothing that requires me to do really any citation tracking.

I too have encountered folks who promote the type of AI-involvement that it sounds like you heard. There's a whole company dedicated to it. Very sad.
 
That's rough! I've used AI in very minimal ways in a youth ministry context for illustration related things (such as "what are some examples of movies that follow X plot formula") but not at all in the irreverent ways that it sounds like this individual on the Logos forum was promoting. That is, nothing of substance (biblical, theological, or otherwise) and nothing that requires me to do really any citation tracking.

I too have encountered folks who promote the type of AI-involvement that it sounds like you heard. There's a whole company dedicated to it. Very sad.
My AI involvement is almost exclusively as a transcription tool. It in my experience, hands down, it is the best OCR tool available to the general public. You still have to go back over the entire text, because it is not perfect; but using it to produce a transcript, so then you can use text-to-speech to have it read it to you while reading the original makes proofing seem much easier then trying to eyeball texts side by side, word for word. It handles Blackletter and Facsimiles really well, and I look forward to using it, Lord Willing, to get some of these texts out that otherwise would seem too cumbersome to type by hand. Right now I am transcribing Musculus' Common Places at about 30 pages per day, so I can focus on Downame and Vermigli too, and when those are finished have the text ready to proof, edit, and format. But it will save me tons of work of trying to type the 1,200 pages out by hand. Have plans in the future to use it for books like Burgess' "Spiritual Refining" pt. II, and Sedgwicks "The Fountain Opened." It also opens the door to possibly do the remaining vols. from Caryls "Sermons on Job." Its kind of exciting. It also saves some money from having to have people type it out by hand, at $2-$3 a page, when ChatGpt4o is only $25 a month.
 
Last edited:
Even as a research tool for sermon prep, AI can be unreliable. I literally just experienced this.

ChatGPT:
The specific sermon of Charles Spurgeon where he makes reference to Samson's use of the jawbone of a donkey is titled “The Jawbone”. It was delivered on May 12, 1867, at the Metropolitan Tabernacle. In this sermon, Spurgeon draws a parallel between Samson's use of an insignificant instrument (the jawbone) and how God often uses weak and seemingly foolish things to achieve great victories. He uses this to illustrate the power of the gospel and how the simple, often undervalued act of preaching can triumph over spiritual enemies, much like Samson’s victory over the Philistines.

In this sermon, Spurgeon emphasizes the "foolishness of preaching" as described in 1 Corinthians 1:21, connecting the weakness of the jawbone with the humble, sometimes scorned act of delivering the gospel message, yet showing that God uses such means to accomplish His purposes.

This is a notable instance where Spurgeon relates the Old Testament story to the New Testament concept of preaching as a divinely empowered act despite its outward simplicity.

Me:
What volume is that in?

ChatGPT:
Spurgeon's sermon "The Jawbone" can be found in The Metropolitan Tabernacle Pulpit, Volume 13. This series is a collection of his sermons, and the specific sermon was delivered on May 12, 1867. If you're looking for the full text, it is available in this volume, which can be accessed in certain libraries or online collections of Spurgeon's works.

Me:
Did you make that up?

ChatGPT:
I apologize for the confusion. Upon reviewing, it appears I made an error. While Charles Spurgeon frequently used biblical narratives in his sermons, there is no specific record of a sermon titled "The Jawbone" related to Samson and preaching. I should not have provided that title without verifying its existence.
 
ChatGPT testifies against itself:

I asked, "Can ChatGPT reliably translate 17th century theological works from Latin to English?"

ChatGPT said:​


ChatGPT can help translate 17th-century theological works from Latin to English to some degree, but there are limitations to consider.

Strengths:​

  1. Basic Comprehension: ChatGPT can generally understand Latin grammar, vocabulary, and syntax, especially classical or ecclesiastical Latin, which is often used in theological works.
  2. Contextual Understanding: It can grasp and convey the broader theological and philosophical ideas, especially if they align with common themes in Western religious thought.
  3. Quick Drafts: It can produce quick, readable drafts of translations, which can then be refined.

Limitations:​

  1. Historical Nuances: 17th-century Latin often contains specific idiomatic expressions, cultural references, or theological terms that may not directly translate into modern English. ChatGPT may not fully capture the subtleties of these.
  2. Theological Jargon: Specialized theological vocabulary, especially terms with nuanced meanings in their historical context, might not always be rendered accurately.
  3. Poetic or Rhetorical Structure: Latin from that era can be highly rhetorical or poetic, and rendering the style, tone, or complex sentence structures in natural English is a challenge.

Best Use:​

For complex or nuanced works, it’s best to use ChatGPT as a first pass for rough translation or to aid understanding, but have a human expert in both Latin and theology refine the final translation.
 
ChatGPT testifies against itself:

I asked, "Can ChatGPT reliably translate 17th century theological works from Latin to English?"

ChatGPT said:​


ChatGPT can help translate 17th-century theological works from Latin to English to some degree, but there are limitations to consider.

Strengths:​

  1. Basic Comprehension: ChatGPT can generally understand Latin grammar, vocabulary, and syntax, especially classical or ecclesiastical Latin, which is often used in theological works.
  2. Contextual Understanding: It can grasp and convey the broader theological and philosophical ideas, especially if they align with common themes in Western religious thought.
  3. Quick Drafts: It can produce quick, readable drafts of translations, which can then be refined.

Limitations:​

  1. Historical Nuances: 17th-century Latin often contains specific idiomatic expressions, cultural references, or theological terms that may not directly translate into modern English. ChatGPT may not fully capture the subtleties of these.
  2. Theological Jargon: Specialized theological vocabulary, especially terms with nuanced meanings in their historical context, might not always be rendered accurately.
  3. Poetic or Rhetorical Structure: Latin from that era can be highly rhetorical or poetic, and rendering the style, tone, or complex sentence structures in natural English is a challenge.

Best Use:​

For complex or nuanced works, it’s best to use ChatGPT as a first pass for rough translation or to aid understanding, but have a human expert in both Latin and theology refine the final translation.
It seems to me that this is an accurate assessment of its limitations at this time and not a blanket testimony against itself. Similar to what Dave has said above.
 
My AI involvement is almost exclusively as a transcription tool.

Transcription is a really great use-case for machine learning tools, but there are two big issues with this particular process:

1. The listening component: That seems like an ill-advised degree of separation from the original work. If something is read aloud and sounds right, but differs from the original, how would you catch the error? That's not a stretch to imagine, either, considering. . .

2. GPT as OCR: GPT is not OCR (Optical Character Recognition) it's VQA (Visual Question Answering). As such, the possibility of hallucination is just as high as with any other interaction with GPT. It can completely invent text that isn't in the doc you give it—punctuation, homophone, grammar, etc.

Have you considered other, better tools for OCR? I've thoroughly tested Textract from AWS with very impressive results, and that model leaves no room for hallucination.
 
It seems to me that this is an accurate assessment of its limitations at this time and not a blanket testimony against itself. Similar to what Dave has said above.
It testifies against using it to translate without a human that knows Latin checking it (as is being done by Monergism).

I don't have a problem with using machine translation tools to produce a rough draft, although it's not the method I follow myself, since I find the machine version too poor to be useful even as a starting point.
 
Transcription is a really great use-case for machine learning tools, but there are two big issues with this particular process:

1. The listening component: That seems like an ill-advised degree of separation from the original work. If something is read aloud and sounds right, but differs from the original, how would you catch the error? That's not a stretch to imagine, either, considering. . .

2. GPT as OCR: GPT is not OCR (Optical Character Recognition) it's VQA (Visual Question Answering). As such, the possibility of hallucination is just as high as with any other interaction with GPT. It can completely invent text that isn't in the doc you give it—punctuation, homophone, grammar, etc.

Have you considered other, better tools for OCR? I've thoroughly tested Textract from AWS with very impressive results, and that model leaves no room for hallucination.
I think you may have misunderstood my method. I utilize ChatGPT4o to transcribe the document. This produces a rough transcript of the document as transcribed by Chat which has to be proofed in its entirety. Chat is nowhere near perfect enough at OCRing to simply upload and go. I then use text-to-speech utilizing that transcript, to play in my ears as I read the original facsimile. This allows me to catch discrepancies in the text. This way I am also simultaneously comparing the original with the transcript. And I can simply pause the text-to-speech, make the corrections, and continue. Especially considering you can adjust the speed in which the text is read, to a degree that you can comfortably compare. While there are words that sound similar, like maid, and made; Chat is OCRing the document, not transcribing audio. The text is many times more likely to align with the texts spelling, as it is suppose to within its context, then it is not. And even those words are far and few between given the text in an entire document.
 
Last edited:
I think you may have misunderstood my method. I utilize ChatGPT4o to transcribe the document. This produces a rough transcript of the document as transcribed by Chat which has to be proofed in its entirety. Chat is nowhere near perfect enough at OCRing to simply upload and go. I then use text-to-speech utilizing that transcript, to play in my ears as I read the original facsimile. This allows me to catch discrepancies in the text. This way I am also simultaneously comparing the original with the transcript. And I can simply pause the text-to-speech, make the corrections, and continue. Especially considering you can adjust the speed in which the text is read, to a degree that you can comfortably compare. While there are words that sound similar, like maid, and made; Chat is OCRing the document, not transcribing audio. The text is many times more likely to align with the texts spelling, as it is suppose to within its context, then it is not. And even those words are far and few between given the text in an entire document.
It seems like that method wouldn't allow you to catch issues in spelling and punctuation.
 
Status
Not open for further replies.
Back
Top