If you do, "export GOGC=20" can help a little, but it'll still use a lot of memory.
I suspect that it's probably hard to add that functionality because you can't do the deduplication without decrypting the prior backups (or at least an index). That would also explain the memory usage JoshTriplett mentions.
It seems to be fine for my mediocre backup needs
Afaik it uses scrypt which was designed for tarsnap
I would suggest using restic. It doesn't have any weird modes, it simply encrypts everything and there isn't any weird need to specify (for instance) that you want things like metadata or filenames encrypted. Also backups are deduplicated using content-defined-chunking -- giving you pretty massive storage savings. If you really need rclone since it supports some specific feature of your backend -- restic supports using rclone as a synchronisation backend (or you can do what I do, which is to batch-upload your local restic backup every day).
You might want to crypt your nudes, but still access normal pictures unencrypted through the provider's web interface.