I allergic to Amoxicillin and always get irritated looks when I tell this a doctor. They usually have to search for an alternative in books, because they don't know them from the top of their heads.
This is also partly why specialists exist. An infectious disease DO/MD would know both what the alternative would be and possibly even the appropriate dose calculation off the top of their head because antibiotic management is their bread and butter.
If the physician you were at felt unable to make the right call, he could have referred you to or consulted with someone who could. An 'irritated look' might instead be a flurry of such treatment paths going through their mind.
'Can I figure this out on my own? Should I consult? Should I refer out? Should I just check UpToDate? What does the latest literature I read say to do? What was done for the last patient who had this? Has the literature changed since then? What is the hospital policy on this (if any?)? What did my microbiology class say to do for this? What do the boards think is right to do in this instance?'
I wonder if the irritated look isn't from "damn, the thing I know from experience works well won't work for this patient" rather than "damn, now I have to crack open a book".