I've been using my new keyboard for a few months now and haven't had any latency issues. Most of the newer custom wireless keyboards run ZMK (https://zmkfirmware.dev/) and latency issues are uncommon. If you are interested in making your design wireless, consider joining the Discord.
I've been thinking about that, I've heard from a friend that has had zero issues with ZMK too, but I just don't need wireless. However, I might make my design wireless-friendly, so down the line I can unplug the Arduino and drop in a ZMK-compatible controller.