Started with an AI agent, now doing a thing that won't scale

jnwatson · 2024-09-04T11:02:50.000000Z

About a decade ago, I had success using a computer vision package for UI E2E testing. It was called Sikuli.

This seems like Sikuli taken to the next level.

vkweb · 2024-09-06T18:34:18.000000Z

Man, Sikuli. My mind just blasted to see that people did similar stuff in 2009!

When I saw a few months ago, Open Interpreter used OpenCV to interpret GUI. Don't you think Sikuli was way way ahead of its time?

Please tell me more about the success part, if you could tell me more about what steps did Sikuli performed? how fast was it? Because when I last tried Open Interpreter that used OpenCV, it felt way too slow for UI e2e testing.

jnwatson · 2024-09-11T17:47:16.000000Z

Sikuli was at least a decade ahead of its time.

It was slow; I just threw bigger computers at it.

My use case was complicated. I was testing HSM-like workflows across multiple hosts using native UIs. It worked really well.

vkweb · 2024-09-14T16:54:36.000000Z

Yeah!

Are you still into E2E testing these days? I am currently of the thought process that using AI to create e2e tests is not the right strategy but letting AI maintain it is the right one.

Any thoughts you would like to share since you are have had more experience using AI for e2e testing than anybody I know.

egamirorrim · 2024-09-05T05:18:12.000000Z

TL;DR Guy tried to write an AI E2E acceptance test running service based on existing Gherkin, it was harder than expected. Now he's a contract traditional QA Engineer. No big ending, just a snippet of a story.

vkweb · 2024-09-06T18:35:32.000000Z

Yep, no big ending because that's the starting of something great I believe :)