That said, please note that this is partially outdated information (even to the extent of warranting a mod edit even though it's so recent) - the parts that mention how some compressed modules are unreadable are now irrelevant.
The URL for this article is /2017/04/ (April).
More recently (July, three months later), this same group has somehow (??) managed to derive the Huffman compression tables for the previously-inaccessible modules: https://github.com/ptresearch/unME11
I haven't read anything that explicitly states this (on HN; I don't read anywhere else), but I get the impression this is the holy grail, or at least one of the major pieces of the puzzle.
I found this random comment about actually using unME11: https://news.ycombinator.com/item?id=15447841
This recent HN article suggests that it's also possible to get arbitrary remote code execution, but no details are (yet) forthcoming. https://news.ycombinator.com/item?id=15298833
Disabling Intel ME 11 via undocumented mode (ptsecurity.com)
How to hack a turned-off computer, or running unsigned code in Intel ME (blackhat.com)
Personally I am extremely curious about the upcoming blackhat presentation. If it is really true this might be very big.
Disabling the Intel Management Engine | https://news.ycombinator.com/item?id=15444607 (Oct 2017, 219 comments)
If nothing else, the BlackHat talk has stirred up interest in the Intel ME which had remained in relative obscurity for quite some time.
To be honest the more transistors components have, the more it is possible to have underlying software that can manage things or even be used as a backdoor.
Now, whether that feature should be part of every consumer CPU is a valid question and concern -- one that nobody has the answer to. Likely the reason for this is that modern versions of ME also do hardware initialisation, so it would make sense for Intel to not require manufacturers to rewrite all of that code for their consumer machines. There have been exploits in Intel ME in the past, which are quite concerning (and the fact it's proprietary is obviously a concern, given how many privileges it has over the system).
You can neuter Intel ME on old machines (pre-BootGuard) using me_cleaner, but it requires attaching a flash programmer to your motherboard. If you have coreboot you can do it from userspace.
It's not valuable at all for individual retail consumers; in the long run, SGX probably does for the entire Intel customer based, including retail, pretty much everything that the ME might have done for retail users.
But all of that is missing the point that there is a way to disable it, with the HAP or AltMeDisable bits. It's believed they were added for the US government to be able to disable Intel ME (after hardware initialisation). It's not easy (you have to reflash the firmware) because most vendor firmware doesn't allow "internal" flashing from userspace, but it is doable if you buy a $5 flash programmer and a Raspberry PI.
Intel hiding CPU features is nothing new. Especially a feature that requires you to attach a flash programmer to your motherboard in order to use it, because you need to modify the descriptor table.
If you're asking why they didn't make it easier to "disable" Intel ME (it's still used for hardware initialisation), then we go back to economics. And they'd have to co-ordinate with people that write mainboard firmware in order to make sure this feature is available for all machines that have Intel CPUs.
[Just to be clear, I'm also critical of Intel. I just don't understand the view that Intel's decisions are anything other than profit-driven. That's how all companies work.]
I'm not saying you shouldn't be able to disable it.
Finally some geek cred for the ME.