I have a feeling they really mean Spinoza's God (like Einstein), but this use of 'God' causes confusion for a lot of readers. Some readers might think they are referring to the God of the Bible or Koran.
Never mind that Erdős doubted God’s very existence. “You don’t have to believe in God, but you should believe in The Book,” Erdős explained to other mathematicians.
Then again, the Abrahamic God (being omniscient) wouldn't need a book either, but would simply know the perfect proof (and every imperfect proof) for every theorem.