It might not be obvious but I found that even though my profile is not "public", visitors who stumbled upon my Google Profile page could see who I was following / who was following me.
In the Edit Profile section, looks like "Display the list of people I'm following and people following me" is enabled by default.
In the Edit Profile section, looks like "Display the list of people I'm following and people following me" is enabled by default.