If you use the ffmpeg library, I will just say it's extremely unwieldy and difficult to use, and has barely any official documentation or examples. GStreamer is the opposite IME.
In any case, they generally fill different niches. FFmpeg is, generally, a library containing set of A/V codecs and (de)muxers. (There are some more obscure parts, too, plus a command line tool ffmpeg(1) that gives access to most aspects of that library.) GStreamer is, generally, an A/V graph framework originally modelled after Microsoft DirectShow.
If you find FFmpeg too difficult to use, most likely what you want isn't a codec library in the first place, but some higher-level abstraction (e.g. if you just want a video on screen, most likely you don't want to muck around with low-level demuxing, but you'd rather want something that deals with clocks and sync and stuff, and FFmpeg just doesn't do that—it's not a video player). It has a lot of weaknesses, but the API is fairly straightforward for what it does.
If you use the ffmpeg library, I will just say it's extremely unwieldy and difficult to use, and has barely any official documentation or examples. GStreamer is the opposite IME.