SVG is future-proof, scaling elegantly to match whatever display it's being viewed on. And, if you don't care about supporting older versions of IE, you can skip the onerror, leaving you with only one asset per image to maintain.
SVG is fine for assets that can easily be described in vector form, but it's not the best choice for all image types. For example, photographs (where you might have an ultra high resolution original that you're scaling down) are not easily representable in SVG other than to embed the bitmap.
<img src = "my_icon.svg" this.onerror = "my_icon.png" />
SVG is future-proof, scaling elegantly to match whatever display it's being viewed on. And, if you don't care about supporting older versions of IE, you can skip the onerror, leaving you with only one asset per image to maintain.