
Show HN: Percy.io, a continuous visual integration tool - fotinakis
https://percy.io/
======
nerdy
This seems like a really convenient way to watch for front-end changes, but
how do things like margins work? Will my whole page be red because everything
has been shifted around or squished?

Responsive capabilities will really improve usefulness, and the absence is
somewhat of a deal-breaker. On responsive sites developers will have to
eyeball each change even when using Percy, so checking the desktop resolution
wouldn't be much additional effort relative to checking all other resolutions
for each page.

On the biz-dev side, if it's going to be a paid product it might be a good
idea to start charging ASAP!

~~~
dubcanada
Based on the screenshots, yes if you changed margin on say body, or something
high up it would cause everything to shift and be "red".

------
rich90usa
A coworker has has a small tool for accomplishing this:
[https://github.com/LesterTheTester/dfrnt](https://github.com/LesterTheTester/dfrnt)

It has been integrated with Travis CI and GitHub pull requests so that non-
passing diffs are appended as a comment to the pull request by a bot account.

If there's an intended diff, a pull request can also automatically be
submitted against the original pull request which, on merge, updates the "gold
master" images and allows the original pull request to pass.

------
meesterdude
I've used gatling on a few redesigns with success

[https://github.com/gabrielrotbart/gatling](https://github.com/gabrielrotbart/gatling)

I have some tweaks i was going to merge in that gave the diff's as %'s (so you
could ignore small differences like fonts being different) as well as animate
between the two versions. On my TODO was grabbing the actual element style
attributes and then actually just diffing that, so the differences would be
more or less spelled out.

Still, cool stuff!

------
todd3834
If you are interested in how something like this was made, there was a good
talk about building something like this at RailsConf this year:
[http://youtu.be/Q4ttqkIEM7g](http://youtu.be/Q4ttqkIEM7g)

~~~
fotinakis
It's an awesome talk! Jessica Dillion has actually helped me with Percy a bit.
:) She's great, you can see her slides here:
[http://www.slideshare.net/jessicardillon/implementing-a-
visu...](http://www.slideshare.net/jessicardillon/implementing-a-visual-css-
testing-framework)

------
Kmandr
I've been working on VisualReview
([https://github.com/xebia/VisualReview](https://github.com/xebia/VisualReview)),
an open source project for visually reviewing a (web-)application as part of a
pull-request, very similar to percy.io.

We're releasing our first version somewhere next week which includes a basic
comparison GUI and testing api's. PR workflow planned in the release after
that. Interesting to see we're not the only ones with this idea.

------
anarchitect
This looks really great. The integration with Github and CircleCI makes it a
very attractive option for us as it would slot right in to our existing
workflows.

------
balls187
I wasn't a fan of giving you github permissions _just_ to sign up for an
invite.

~~~
Sir_Cmpwn
I would have agreed with this, but the permissions they ask for are pretty
benign and clearly relevant to their mission.

~~~
andybak
I'm a member of organisations that may not be comfortable with me giving
permission by proxy.

I couldn't find a way to sign up and exclude private repos or organizations.

~~~
Sir_Cmpwn
Fair enough, but I think that's a limitation of Github rather than a problem
imposed by Percy.

~~~
balls187
Percy.io's Invite Request system is tied to github.

How is that not a problem imposed by Percy?

I'd feel more comfortable having a different mechanism to request an invite.

While the permissions don't grant Code Access, it does ask to read
Organization and Team membership. I'd expose data for every member of my
companies github organization to Percy, _JUST_ for an invite.

~~~
Sir_Cmpwn
Percy is a tool that integrates with Github to set build status on Github pull
requests. Of course it needs to tie into Github.

~~~
balls187
100 %.

However:

Sign up for Service != Sign up for Invite.

~~~
Sir_Cmpwn
Indeed, but it seems reasonable to me that they wouldn't need to make two
systems for storing user information.

------
amitaibu
Congrats @fotinakis

Check out also [http://shoov.io/](http://shoov.io/) \- there are a few blog
posts, that explain the concepts and point to examples. Apart of the visual
regression testing it also allows monitoring the live sites with your
preferred functional testing frameworks (e.g. casperJS, Behat, etc').

Open source of course :)

------
planetjones
This looks promising. Our project has a plethora of selenium tests, but
they're focused on functionality only, not appearance. We can tell there is a
table, button, link, form, etc. and click it. On that side we're reliable.

But we really let ourselves down sometimes with regressions to the look and
feel of the application.

Are there any competitor tools for this visual testing ?

~~~
werkshy
We (Harry's) have released a tool called Kontrast:
[https://github.com/harrystech/kontrast](https://github.com/harrystech/kontrast),
[http://engineering.harrys.com/2014/09/24/kontrast.html](http://engineering.harrys.com/2014/09/24/kontrast.html)

It's written in Ruby but is not really dependent on Rails or anything to run.
We use it to compare a list of URLs between the version we are currently
testing in CircleCI and our production site. Any differences are highlighted
in a nice gallery and we get a message in our Hipchat room.

------
sangnoir
Percy.io seems somewhat similar to BBC's wraith[1], albeit easier to
integrate.

1.[http://bbc-news.github.io/wraith/index.html](http://bbc-
news.github.io/wraith/index.html)

------
arenaninja
This is neat. Recently started doing Selenium tests with Scrutinizer CI, but
we've had issues where the UI is broken and we find out a day later... I'll
definitely keep an eye out for you guys

~~~
fotinakis
Woohoo! Glad to hear it. :) We have an annoucement-only mailing list if anyone
doesn't want to sign up with GitHub:
[https://percy.io/#updates](https://percy.io/#updates)

~~~
ahmetmsft
Are you an one man army? Congrats! :) This is great. How do you plan to react
to potential changes in rendering engines? Would those be shown as test
failures as well?

~~~
fotinakis
Great question! And yes. :) My plan at the moment for that is, because I know
which browser exactly it was rendered in originally, I can put a notice at the
top of the page to say something like "We just upgraded our rendering browser!
The diffs below might because of that and not because of a change you made."
\-- and then after that one build, things would be back to normal.

------
On
My favorite in this space is Applitools eyes
([https://applitools.com/](https://applitools.com/)).

Eyes has really advances image processing capabilities, which meant two huge
advantages over everything else I looked at:

1\. It's not prone to rendering differences. I can set a baseline in one
browser/OS combo, run it on other combinations and set it to only consider
real regressions as a failure.

2\. It can validate the layout of the page even with dynamic content.

------
alkonaut
Doesn't say what it does. I can deduce from the title and landing page that it
has something to do with UIs and CI. From the FAQ it seems it is web only, as
it mentions "the page" (assume that is an html page).

Is this an appropriate summary?

"It can help you visually review changes made to a web front end as part of
CI/QA"

Would be neat if you could plug any kind of front end screenshot rendering and
not just web.

------
kordless
I signed up. Clicked on the survey, clicked back, got a blank page. Did it
again, got a live page. Did it one final time, got a blank page. Tried it
again just now, blank page (but there is HTML in the page source).

------
mark2hayes
There's another tool out there that does something similar called Screenbeacon
([https://www.screenbeacon.com/](https://www.screenbeacon.com/)).

------
ecesena
There's no mention about pricing... any rough idea/estimation?

~~~
fotinakis
Hey! Good question, we're still working out the details on pricing but I want
it to be very reasonable, simple and transparent for people. Just need some
time to work through with a few initial beta testers. Stay tuned!

~~~
ecesena
Great, thanks!

------
pimlottc
Devil's advocate - won't this tend to bolster the outdated attitude that the
front end must be pixel-perfect?

~~~
proexploit
I'm not sure I consider that to be true. Why is pixel perfection out-dated?

~~~
vladharbuz
Perhaps because rigid constraints are useless with the number of different
devices out today, and layouts should instead be highly adaptive?

~~~
dubcanada
No that's wrong. The screenshots are not based on one device looking like
another. It's based on one device looking like X all the time.

For example this uses Firefox, if all I change is something in the backend. My
front end should look the exact same before the change and after.

------
joshdance
Really interested how this will work with responsive. Does it test a set size
each time? Multiple sizes?

------
cloudforcer
Has anyone used visualdiff.com? The services seem pretty similar.

~~~
fotinakis
I think they might only do live sites, the thing about Percy is that it can be
integrated earlier into your feature/browser tests for all pages, even those
behind logins.

------
jmprobert
Interesting read! Bummer to hear the news though.

------
idibidiart
I wonder, does it handle responsive UIs?

------
leoi11
would pixel by pixel diffs be an overkill?

------
cloudforcer
Has anyone used visualdiff.com?

------
siegecraft
So.. is this just a lot of wrappers around wraith? Or something else?

~~~
fotinakis
Actually it's all home grown, right now it renders the page and screenshots in
Firefox 38 ESR and then the actual diff image is produced by imagemagick's
compare function, with some special things around it to handle edge cases with
width/height differences and some others.

