That sounds like a workable model for vector graphics in browsers; it's just not describing SVG's design. It very deliberately was meant to contain not just vector model, but also a dynamic/interactive environment.
"Sophisticated applications of SVG are possible by use of a supplemental scripting language which accesses SVG Document Object Model (DOM), which provides complete access to all elements, attributes and properties. A rich set of event handlers such as ‘onmouseover’ and ‘onclick’ can be assigned to any SVG graphical object. Because of its compatibility and leveraging of other Web standards, features like scripting can be done on XHTML and SVG elements simultaneously within the same Web page."
But we're explicitly criticizing SVG's design! SVG doesn't contain horrible things by accident of implementation, it was specified to contain those horrible things.
Defending against this criticism by saying the behavior is in the spec is not relevant.
From the standard: https://www.w3.org/TR/SVG11/intro.html
"Sophisticated applications of SVG are possible by use of a supplemental scripting language which accesses SVG Document Object Model (DOM), which provides complete access to all elements, attributes and properties. A rich set of event handlers such as ‘onmouseover’ and ‘onclick’ can be assigned to any SVG graphical object. Because of its compatibility and leveraging of other Web standards, features like scripting can be done on XHTML and SVG elements simultaneously within the same Web page."