Access your data for all websites
Read and modify bookmarks
Get data from the clipboard
Input data to the clipboard
Access browsing history
Display notifications to you
Access recently closed tabs
Access browser tabs
Access browser activity during navigation
... this seems excessive - especially clipboard and historical items.
For example, to open a bookmark from the command line requires read access to your bookmarks in order for the plugin to present you with the list of bookmarks and eventually allow you to search or navigate it somehow. The same applies to opening something from your browsing history or resurrecting a closed tab. Tab switching requires access to the tabs. Clipboard access is for things like copying URLs or the selection in visual mode to the clipboard.
Vimium itself is open source on GitHub and you can see for yourself what it's doing with your data. The Firefox port is by one of the most active project contributors.