Yeah, I was baffled by that too. Maybe "in church" should be "in bed"? Maybe he didn't go to the church services, but found some time every Sunday morning when the church wasn't in use to sit there and read the Bible?
... Ahaha, bingo. Hao Wang's "Reflections on Kurt Goedel":
"although he did not go to church, was religious and read the Bible in church every Sunday morning"