On: Learning to code vs. learning to automate

[freetonik]

Can Auto HotKey recognize images and click on them? For example, a specific button in an app or a web page?

This is one of the features that keeps me with Keyboards maestro, which keeps me from switching away from macOS.

[nickjj → freetonik]

I'm not an AHK wizard but to my understanding, the answer is "yes" to almost every "but can it do...?" questions.

In your case, from my limited understanding it would be no problem to identify a specific button in an app and then interact with it because you would be hooking into the app's low level window title / class / handles.

AHK comes with a "spy" tool where you can just move your mouse around and it gives you low level details about every window, and then you can write AHK scripts that interact with those things.

The script would look something like:

    ControlClick, OK, "My Cool App"

That would click the OK button in a window that has a title of "My Cool App". If you Google around for "auto hotkey, click button in app" you'll find a ton of examples.

[ibly31 → nickjj]

I have limited experience with Keyboard Maestro but I do have some with AppleScript. It lets you do as you described, get windows, titles, UI elements. However this only works on native apps, you cant, for example, click a button on a webpage with a CSS selector or by scanning pixels of the window, ad far as I know. Can AHK do that? That would be super useful

[nickjj → ibly31]

While I've never done it, AHK seems to have an API for being able to interact with DOM elements.

Check the marked answer here: https://stackoverflow.com/questions/49516638/ahk-web-element...

The above answer shows both navigating to a new page that isn't open yet or interacting with an already loaded page based on the title of the tab.

[vasili111 → nickjj]

You can use Selenium with AHK. https://www.autohotkey.com/boards/viewtopic.php?t=32323

[dmortin → ibly31]

Yes, Autohotkey can search for pixels and stuff:

https://www.autohotkey.com/docs/commands/PixelSearch.htm

[vasili111 → freetonik]

Yes it is very easy to match image and click on it with AHK. For image matching use https://www.autohotkey.com/docs/commands/ImageSearch.htm for matching use bmp files! bmp files works mach better then png or jpg. For clicks there are several commands, just google them. Also take a look at https://www.autohotkey.com/docs/commands/CoordMode.htm

[dicytea → vasili111]

Is there any reason bmp would work better than png? I've always used png all this time.

[vasili111 → dicytea]

I cannot tell why it is so but bmp image matching worked for me much faster then png on several PCs.

[Thorentis → freetonik]

Yes! You can search for Bitmaps on the screen. Take a screenshot and crop to the button, save it as a .bmp, and then use the ImageSearch function (https://www.autohotkey.com/docs/commands/ImageSearch.htm).

It will return the (x,y) and you can then use the Mouse functions to interact with that location.

[vasili111 → freetonik]

Also you can use Selenium with AHK. https://www.autohotkey.com/boards/viewtopic.php?t=32323

[eitland → freetonik]

Like this?

https://www.autohotkey.com/boards/viewtopic.php?t=47778