You are very much in the minority. There were many jokes about Carter at the time. Many comedians did impressions of him. He was not known for his great oratory. He was often referred to as a technocrat.
I was around at the time and I’ve read the literature since that era. I don’t recall such criticism and I’ve never read anything criticizing his oratory skills.
Interesting. Your memory obviously is very different, but I remember Dan Akroyd did impressions of him on SNL. A very monotone delivery was very characteristic, only to be outdone by Mondale.