The important point is that physics will likely never find a perfect 1:1 correspondence between nature and math because of the properties of nature (such as being too complex for the human mind to fully model), not because of the properties of math (such as the incompleteness theorem).
That is to say, even if the incompleteness theorem hadn't been true, math still wouldn't have described nature with perfect accuracy.
Not to mention, the incompleteness theorem hasn't prevented any kind of modelling in physics that I have ever heard of. Indeed, physics is often not entirely constrained by formal methods, with ad-hoc mathematical models that can be shown to work even if they are not fully formalized sometimes being preferred (such as the Dirac delta "function").
Poets, like writers, use words. They often say things in few words subject to many internal constraints, so one must pay closer attention than when reading prose. (Indeed, many people write prose about the poetry as a guide.)
Mathematicians use even fewer symbols to say things subject to even stronger internal constraints, so one must pay even closer attention than when reading poetry. (Indeed, almost all people write prose around the symbols as a guide.)
Some poets explore what happens with more, some with fewer, constraints.
Some mathematicians explore what happens with more, some with fewer, constraints.
(Viewed as a MMO game, mathematics introduced the parallel postulate in or before 300 BC and although many people said it ought to be nerfed, the nerfing wouldn't happen until 1830.)
(Edit: I am getting downvoted, so let me expand my answer a bit.
Nature is explained using Physics and Physics uses Mathematics for its models of Nature.)
This is part of Godel's discovery. Especially, his first Incompleteness Theorem.
https://en.m.wikipedia.org/wiki/G%C3%B6del%27s_incompletenes...