Or the reverse: caving in and picking an Icelandic first name, but keeping Harriet as the middle name, and calling her Harriet in everyday circumstances.
I'm not an expert in Icelandic by any stretch, but I think the problem here is that names are used in declensions in everyday speech. Which means that "Harriet" isn't painless, from a linguistic point of view. It sort of "breaks" the language. It returns an error, so to speak.
This is an imperfect analogy, but imagine if I were to name my son "Michael's." Not "Michael," but "Michael's," which is essentially a possessive tense. That would create some odd headaches in certain conversational and written contexts. It's hard to understand from the perspective of English, because in most cases, we don't use declensions in proper nouns. In theory, you could probably force Icelandic to work around foreign words and names, just as English has worked around and incorporated foreign words and names. But English has a long history of doing that, and the population of English speakers worldwide is enormous and heterogeneous.
I'm opposed to the Icelandic Naming Committee's position here. Seems they could stand to be a little more flexible, and/or to recognize that this inflexibility breeds more inflexibility, recursively. But I guess that's their goal. Linguistically, they are extremely conservative and would prefer their language remain static. English is pretty much the opposite: unregulated, as it were, and constantly evolving.