Agreed. A great many things can be expressed as a graph of sub-tasks with preconditions and dependencies that create a partial ordering, and make is pretty darn good at grinding through such graphs. You can go far with just the basics plus a couple of extra tricks like "phony" targets.
Any recommendations on learning to use make more effectively outside of just reading the docs? I feel like this is a glaring gap in my knowledge as I really only know the basics.
I'd say gnu make (dont use any other make) can be learned from the documentation, it readable. Also, it's super simple to try things yourself (unlike, say, k8s by example) : just use it in your project and use the doc as needed. I am very fond of make, to the point I did a video transcoding solution based on it and ffmpeg.
Related trick: GNU tools tend to come with Info pages - it's to man pages what a book is to a listicle. These Info pages can get you up to speed quickly, while giving you a deeper understanding of the tool at the same time.
Read a lot of other people's Makefiles. I tend to shy away from huge/overly complex ones, but there's a lot of good examples out there where I got most of my ideas/habits/patterns from.