Story points aren't useful outside the team. They're for the team to help it figure out roughly how much stuff it can do each sprint. They shouldn't leak out of the team.
I understand what you're saying - of course in some sense they're convertable. But the point is to not think about time when estimating, because if you estimate time you don't factor in things like other tasks, or holiday, or anything else. Or if you do you have to spend ages trying to account perfectly for time.
Instead, if you estimate complexity (e.g. I think this task is a 3, just as a starting point, then this task is roughly the same, so it's also a 3, then this one is similar but will take almost as much testing due to its difficulty, so we'll call it a 5, then this one is very simple, not even half as difficult as the first one, so it's a 1, etc), then try and keep that up for a few sprints, then figure out how many points fit into a sprint, you automatically factor in other factors (like "I have to log into Okta ten times a day", or "people keep getting pulled into meetings") through practical observation of what got done, and you get better at predicting what you'll be able to achieve in a sprint.
It's not perfect; it just removes the need for certain entire jobs devoted to accounting for time, which you can spend on another developer instead, while also being a reasonable measure of what you'll get done, and only takes about an hour every two weeks.
If the problem is "we're not accounting for holidays in our time estimates", I can't see how the solution could possibly be "time is obviously a flawed measure, so we'll use this other measure which has this hazy relationship with time, but we all agree that it's definitely not time, although we have trouble saying what it is"
personally, I find it much easier to say 'normally, I would get this to you by wednesday of next week, but we have that offsite and my wife's parents are visiting, so does friday work for you?'. than 'this is a 3', so, I guess this fits in this sprint?
changing units and names of things really seems like a deliberate attempt to rob the discussion of any actual meaning. just a comfortable empty formalism that masks the fact that we aren't trying to come to grips with the most difficult parts of our job
But you don't estimate holidays in ... you estimate how long it would take if someone picks on the task Monday morning and works on it full time.
If someone picks up task on Monday then has 20 other meetings - estimation is still the same, he just continues after those 20 meetings and you just don't care when estimating.
Only thing is if at the end of sprint dude is saying "I started X then I had 20 meetings so I did not make it" - well you just accept that or you don't put guy into 20 meetings.
Functionally you're just created extra steps and confusion by not calling it a time estimate, or at least something equivalent to time. Even with a real time estimate you shouldn't be planning by going "well you work 80 hours this sprint, so we plan 80 hours" - you should be doing the same consideration of looking at how many "hours" were completed in the last few sprints and plan based on that number. If we're in agreement that these numbers are for the team only then it shouldn't matter if they consistently under or over estimate the hours, nobody outside the team should know or care how many "hours" they complete in a sprint.
The confusion part is that by calling it "complexity" and saying it's not a time estimate you've muddied the waters on what it is, people will debate the definition and intentionally differentiate it from actual time. I've seen this before, the points-per-sprint never stabilizes because teams have cards where "that's a 1 point card because it's simple, but it will probably take a week". And then suddenly they're ignoring the points during planning to instead come up with an actual time estimates (which also don't work because they don't track those against multiple sprints).
