USB is fine for PS1/retro games; should be more than enough for AthenaEnv. The difference only matters for PS2 backups. And there're more options than those two. HDD/SSD, Ethernet, MX4SIO/SIO2SD, MMCE (SD2PSX et al).
That's true; network too. Can play digital backups (off USB/Eth) only by using POPStarter (for those unaware, POPS being Sony's PS1 emulator ripped off the single game that was officially used on). Although POPS isn't really that good (was used, experimentally, only once afterall), USB throughput isn't an issue.
FreeMcBoot on a memory card + USB flash drive (or internal disk) is a popular option to play on real hardware. Saves wear and tear on the optical drive too.
It's a good time! I feel the PS2 is an iconic member of any living room TV setup, especially with an SSD, a couple controllers, and component out. Nice to have for hangouts!
The original Phat PS2 models support internal IDE disks, though it's easy to convert them to SATA with an inexpensive adapter. Then prepare the SSD and install (I use WinHIIP, "obsolete" but works fine). It noticably improves startup and load times in games!
Some also swap the fan for a Noctua to give it a complete "quiet upgrade", but my stock fan isn't very loud.