You are generally correct in your assumption: that once the CPU comes out of reset, it will reach for a particular memory address to begin execution. Some will directly begin execution from a fixed address. A sibling pjc50 comment mentions, on x86, the CPU will be in 16-bit real-mode and begin fetching instructions from FFFF:0000. Other architectures, work slightly differently. Motorola 68k fetch the first 4 bytes from 0x00000000, loads them into the program counter register, and then jumps there to begin execution.
As you saw, the child of a pjc50's comment explains how to pass your code directly to the beginning of the CPU's execution in QEMU. If you want to do this with actual metal, various classic chips of yore (z80, 6502, 68k, etc) and their documentation are relatively easy to get. A nice thing about those older CPUs, is that their memory interfaces are rather simple compared to today. You can wire up a very basic system with the CPU, an EPROM, an SRAM chip, and maybe a couple other chips as glue logic, all on a basic breadboard. And then you really can control those first bytes of executed code, on actual metal.