I have shared this app on Reddit about a year ago. Since then, the notable changes would be a major performance buff in webkit/electron, force keyboard layout, and reducing the overwhelming no. of hints to what is just "clickable".
It is open source at https://github.com/dexterleng/vimac/.
Do let me know if you have any questions!
Glad to see more people making tools around the Accessibility API. I've been working on a similar app called Shortcat (https://shortcatapp.com) almost 8 years ago (wow) but haven't had the time to properly work on it. It sounds like you're encountering a lot of the same problems I've had when I was starting out (figuring what's actionable, forcing keyboard layout etc). Let me know if you want to chat about the various problems in that space
Anyway, Shortcat is a freakin brilliant idea and I hope you keep working on it, or help this author with their project. If you think you’re going to have time to work on Shortcat again, though, I’d happily buy a copy (even if I already did before, I can’t remember if I did) and be an active bug reporter.
Unfortunately I've been rather busy the last couple of years and haven't had the capacity to work on Shortcat, and coupled with not being amazing at Obj-C/Swift/Cocoa (my major contributions are FuzzyAutocomplete and the initial POC for Semantic History in iTerm2), being frustrated with building UIs on Mac (mostly build web stuff). There's also the problem that Accessibility APIs aren't well-documented and isn't commonly used, and applications not implementing Accessibility correctly (and potentially causing crashes in Shortcat, or making the target application hang), which was super tough to deal with as a solo dev.
I did have a crack at bringing the Swift-based prototype up to date with Swift 5 on the weekend, and am investigating the feasibility of using SwiftUI which would help me on the UI front! Ideally I'd find another dev to work on Shortcat (rev-share or otherwise), and consider a more sustainable pricing model.
I think tools that depend on Accessibility APIs that are used by people who don't normally depend on Accessibility APIs can force developers to improve their AX implementation so people who do depend on good Accessibility to use applications can benefit.
It makes me happy to hear that it was good enough for you to buy another copy!
How did you discover Shortcat in the first place?
But the main thing is that Shortcat was simply the thing that I always thought should have been baked into operating systems since the idea of “search to execute” became a thing. Mostly these days I rely on Command-Shift-/ (Menubar search), but that is a poor poor substitute. Shortcat really shines in dynamic context selection, like when trying to navigate something like Outlook for Mac, where you don’t know the shortcuts and don’t care to use them, and anyway just want to select the email that contains the text you’ve already started reading. Or things like the preference menus where navigating the combination of tabs and sidebars can be especially unclear from a keyboard perspective.
Was it solved in new versions?
Yes that is fixed.
The reason for slow performance + high cpu usage on electron/webkit apps was just the sheer amount of mach ipc calls (from the many many <div> layers) needed to be made to fetch the entire UI element tree.
This has been fixed two days ago Heres the solution for those interested: https://github.com/dexterleng/vimac/pull/190
I was also using a bunch of async queues instead of just a single NSThread which likely contributed to high cpu usage, and that has also been fixed.
> warning that I am using a screen reader.
Nope, you should get this warning, although it was a one time prompt for me. This is because Accessibility is opt-in for electron apps for performance reasons, and I have to ask for it through setting the AXManualAccessibility attribute (see https://electronjs.org/docs/tutorial/accessibility#assistive...), which I guess triggers the prompt.
Awesome project by the way. This will save my wrists from a lot of pain.
Vimac had major performance issues previously when traversing the UI Element tree of Electron/Webkit apps.
I think you're mistaken. OP is referring to Discord as the webkit/electron app.
Also vimium will always be faster than vimac due to how slow the accessibility API is.
I would recommend using both for now.
The only suggestion I’d make is to have the cursor return to its old position after an action. But it’s a minor nitpick.
> cursor return to its old position after an action
My right wrist used to pain a lot because of excessive mouse usage. I had tried out various keyboard-driven apps but I didn't find any app practical enough for my needs, so I made one.
I'd like to share here my "generic" keyboard-driven navigation app for Windows:
How long did it take to familiarize yourself with the trackball?
I didn't really try other pointing devices because I thought that they won't be as good as the mouse, plus I liked the idea of using keyboard-driven navigation software.
In the fallback mode, not all motions are available, because we can't read cursor position or text field value without accessibility support.
some other useful ones are C-h/C-d (backwords/forwards delete), C-o (insert line), and the main Emacs movement keys C-n, C-p, C-b, C-f. Remapping Capslock to be a second Control makes using these regularly much more natural. Karabiner is a good app for easily remapping keys on macOS.
I think the thing which is not wonderful now is: in vim, you enter a mode, in that mode, you can do a series of navigation to get to the final destination. While in vimac, you enter a mode to do just one navigation and you're out of the mode, you have to press the key enter the mode again to do another navigation. This makes me feel not productive at all.
The accessibility API is one thing that I really miss since mostly leaving macOS for Linux. Most apps support it in at least a rudimentary way, and it allows for a bunch of neat tricks.
For everyone interested in efficient keyboard usage: You might enjoy KeyCombiner - a web app to organize, learn, and practice keyboard shortcuts.