Yeah I found it very painful at first (like learning Vim but perhaps 5x times worse). I highly recommend using typing.io to get used to it. Since the training examples are source code, it ensures you get exposed to special characters, which is crucial if you're using your ergodox for programming.
As you can see from the table on my GitHub page, progress was slow and took several weeks to reach baseline levels. This was from maybe 45-60mins per day of dedicated practice. I see the time investment as having paid for itself tenfold already.
I also switched to an Ergodox EZ (about 2 years ago), and the switch was very painful. At first, my speed was divided by 2.
Using some typing practice, I managed to get to my usual speed on a normal keyboard and then thought: good, now I can definitely remove my old keyboard for good. My god this was wrong:
When typing what I saw before me, my brain could dedicate 100% to typing, and this was fine.
But when programming, or even writing an email, part of my brain must concentrate on the content, and my typing what very laborious, maybe half of the normal speed, with many errors. This was taxing much of my mental energy.
I kept on for a at least one month of typing practice before really making a permanent switch.
It was hard, but I do not regret it at all.
Worth noting: I thinkered with the layout quite a bit. Even moving one special character somewhere else could take me a full day to adjust, and it was painful. So I finally settled on a good, but probably not optimal layout.
As some other comment above, I do not use common keys as layer keys (something I really wanted to use) because of delays. It is not much, but I experienced much more typing errors with this.
As you can see from the table on my GitHub page, progress was slow and took several weeks to reach baseline levels. This was from maybe 45-60mins per day of dedicated practice. I see the time investment as having paid for itself tenfold already.