I would be concerned about the privacy implication of it. Imagine if the browser sent the origin to widely used CDNs, or to Google Fonts, and that people didn't actually block Google domains on their browsers.
Also, this would not be secure by default, because you would have to change the default behavior of the server to block cross origin requests.
Browsers already send the referer header on all those CDN requests, provided that the CDN is loaded over https. If anything, the origin contains less information.