I won't be managing calendars. Sadly, your choices for that seem to be either Gmail (which I'll continue using for my calendar) or Outlook (and the Outlook calendar). I'd pay good money for a working third party calendar/contacts solution that worked well on my phone, tablet, and Windows. Google doesn't fit the bill there, either, unless you like web-only.
As for securing Gmail with yubikey... how do you gain access via a standard email client like Thunderbird in that case? Anything that can't do that is a non-starter for me.
Do Gmail's app passwords not work with yubikey? I know that with totp 2fa enabled, I can generate passwords that bypass the 2fa codes. There also might be some way of using a 2fa code in the password field as well (PayPal used to do this, you just appended your 2fa code to your password).
Nextcloud or owncloud have calendars/contacts sync iirc, but I'm not sure how well it works.
I meant I won't manage calendars with my own domains. I'll be using a personal Gmail account to manage my calendars and contacts. But no critical emails will pass through Gmail and no financial or similar accounts will be tied to it.
As for securing Gmail with yubikey... how do you gain access via a standard email client like Thunderbird in that case? Anything that can't do that is a non-starter for me.