If you are asking in the context of a moral framework in which monogamy is morally wrong, yes.
Otherwise, no.
If you think you can answer moral questions without reference to a particular moral framework, you are wrong—not morally wrong, just flat-out wrong.
There may be a universally correct moral framework, but unfortunately there is no way to answer the question of whether there is and which one it is that doesn't require adopting an unverifiable axiom or set of axioms that determines the result.
Some creatures are monogamous. Some creatures are not. Some creatures eat their young. Some creatures are solicitous of the young of others. Other higher primates do not seem to be monogamous, but I recall reading that there is a greater chance of the male of the species being involved if they feel they're defending their child. If so, this might/would have provided positive evolutionary pressure. Similarly, being altruistic enough to let grandparents or other older members of the tribe survive let to evolutionary advantage (knowledge transfer, for example).
It's only been a few 10K years so we're still the wonderful cro magnons that showed up in the middle east and started shoving the other Humans aside.
Otherwise, no.
If you think you can answer moral questions without reference to a particular moral framework, you are wrong—not morally wrong, just flat-out wrong.
There may be a universally correct moral framework, but unfortunately there is no way to answer the question of whether there is and which one it is that doesn't require adopting an unverifiable axiom or set of axioms that determines the result.