ok, so how do you call malloc or syscalls with mov instructions? Do you just (assuming you turned on executable stack) use mov to write the corresponding non-mov instructions necessary to do those things to memory, then mov the address of these instructions to the ip register? im struggling a little here
These are classified as "libc things". When using mov you can still access libc like any other program. Go study movfuscator(which I linked again) if you would like a full understanding[0]. When you have to interface C with a not-very-Unixy system like Webassembly or these mainframes, you have to link to a compatible libc that remaps everything appropriately.
Malloc isn't particularly more special than any libc function. It simply supports the fantasy that the memory is unlimited in scale. It was always of a static size, you were just handing off the chore of dividing it up. You can usefully implement dynamic sizing in languages with static memory allocation by pre-allocating large arrays and then maintaining handles and liveness flags. This is, in fact, how every console game used to do it until available memory sizes got into the hundreds of megabytes. Fragmentation and the subsequent crashes from OOM presented too much of a concern before then.
Ah I thought the idea was to make the whole program use nothing but mov instructions. Still cool though. That's an interesting point too about console games, it seems to make a lot of sense if the whole machine can be dedicated to running just your program