6 ms·
Show HN: Percy.io, a continuous visual integration tool
- meesterdude 11y agoI've used gatling on a few redesigns with success https://github.com/gabrielrotbart/gatling https://github.com/gabrielrotbart/gatling I have some tweaks i was going to merge in that gave the diff's as %'s (so you could ignore small differences like fonts being different) as well as animate between the two versions. On my TODO was grabbing the actual element style attributes and then actually just diffing that, so the differences would be more or less spelled out. Still, cool stuff!
- planetjones 11y agoThis looks promising. Our project has a plethora of selenium tests, but they're focused on functionality only, not appearance. We can tell there is a table, button, link, form, etc. and click it. On that side we're reliable. But we really let ourselves down sometimes with regressions to the look and feel of the application. Are there any competitor tools for this visual testing ?
- mholmes680 11y agocheck eggplant / testplant (not free though)
- werkshy 11y agoWe (Harry's) have released a tool called Kontrast: https://github.com/harrystech/kontrast https://github.com/harrystech/kontrast, http://engineering.harrys.com/2014/09/24/kontrast.html http://engineering.harrys.com/2014/09/24/kontrast.html It's written in Ruby but is not really dependent on Rails or anything to run. We use it to compare a list of URLs between the version we are currently testing in CircleCI and our production site. Any differences are highlighted in a nice gallery and we get a message in our Hipchat room.
- lindig 11y agoYou can try https://app.webmate.io https://app.webmate.io. It's a hosted service for layout, regression, and cross-browser testing. Basic features are free. The diff-ing works on the DOM, is visualised in screenshots and identifies the XPath of elements that are found to be changed, removed, or added. Tests are run on browsers/vm that are part of the service, giving you access to 200+ os/browser combinations.
- johns 11y agoghostinspector.com (one of the products from my company)
- jbnicolai 11y agoThere's VisualReview[0], but only self-hosted. [0]: https://github.com/xebia/VisualReview https://github.com/xebia/VisualReview
- deleted 11y ago[deleted]
- siegecraft 11y agoSo.. is this just a lot of wrappers around wraith? Or something else?
- fotinakis 11y agoActually it's all home grown, right now it renders the page and screenshots in Firefox 38 ESR and then the actual diff image is produced by imagemagick's compare function, with some special things around it to handle edge cases with width/height differences and some others.
- arenaninja 11y agoThis is neat. Recently started doing Selenium tests with Scrutinizer CI, but we've had issues where the UI is broken and we find out a day later... I'll definitely keep an eye out for you guys
- fotinakis 11y agoWoohoo! Glad to hear it. :) We have an annoucement-only mailing list if anyone doesn't want to sign up with GitHub: https://percy.io/#updates https://percy.io/#updates
- ahmetmsft 11y agoAre you an one man army? Congrats! :) This is great. How do you plan to react to potential changes in rendering engines? Would those be shown as test failures as well?
- fotinakis 11y agoGreat question! And yes. :) My plan at the moment for that is, because I know which browser exactly it was rendered in originally, I can put a notice at the top of the page to say something like "We just upgraded our rendering browser! The diffs below might because of that and not because of a change you made." -- and then after that one build, things would be back to normal.
- todd3834 11y agoIf you are interested in how something like this was made, there was a good talk about building something like this at RailsConf this year: http://youtu.be/Q4ttqkIEM7g http://youtu.be/Q4ttqkIEM7g
- fotinakis 11y agoIt's an awesome talk! Jessica Dillion has actually helped me with Percy a bit. :) She's great, you can see her slides here: http://www.slideshare.net/jessicardillon/implementing-a-visual-css-testing-framework http://www.slideshare.net/jessicardillon/implementing-a-visu...
- balls187 11y agoI wasn't a fan of giving you github permissions just to sign up for an invite.
- Sir_Cmpwn 11y agoI would have agreed with this, but the permissions they ask for are pretty benign and clearly relevant to their mission.
- andybak 11y agoI'm a member of organisations that may not be comfortable with me giving permission by proxy. I couldn't find a way to sign up and exclude private repos or organizations.
- Sir_Cmpwn 11y agoFair enough, but I think that's a limitation of Github rather than a problem imposed by Percy.
- fotinakis 11y agoYou can sign up for the mailing list instead: https://percy.io/#updates https://percy.io/#updates And yes it's a GitHub limitation, we request some limited readonly access and no code access (we don't need it).
- balls187 11y agoPercy.io's Invite Request system is tied to github. How is that not a problem imposed by Percy? I'd feel more comfortable having a different mechanism to request an invite. While the permissions don't grant Code Access, it does ask to read Organization and Team membership. I'd expose data for every member of my companies github organization to Percy, JUST for an invite.
- Sir_Cmpwn 11y ago
- nerdy 11y agoThis seems like a really convenient way to watch for front-end changes, but how do things like margins work? Will my whole page be red because everything has been shifted around or squished? Responsive capabilities will really improve usefulness, and the absence is somewhat of a deal-breaker. On responsive sites developers will have to eyeball each change even when using Percy, so checking the desktop resolution wouldn't be much additional effort relative to checking all other resolutions for each page. On the biz-dev side, if it's going to be a paid product it might be a good idea to start charging ASAP!
- dubcanada 11y agoBased on the screenshots, yes if you changed margin on say body, or something high up it would cause everything to shift and be "red".
- rich90usa 11y agoA coworker has has a small tool for accomplishing this: https://github.com/LesterTheTester/dfrnt https://github.com/LesterTheTester/dfrnt It has been integrated with Travis CI and GitHub pull requests so that non-passing diffs are appended as a comment to the pull request by a bot account. If there's an intended diff, a pull request can also automatically be submitted against the original pull request which, on merge, updates the "gold master" images and allows the original pull request to pass.
- deleted 11y ago[deleted]
- anarchitect 11y agoThis looks really great. The integration with Github and CircleCI makes it a very attractive option for us as it would slot right in to our existing workflows.
- sangnoir 11y agoPercy.io seems somewhat similar to BBC's wraith[1], albeit easier to integrate. 1.http://bbc-news.github.io/wraith/index.html http://bbc-news.github.io/wraith/index.html
- cloudforcer 11y agoHas anyone used visualdiff.com?
- cloudforcer 11y agoHas anyone used visualdiff.com? The services seem pretty similar.
- fotinakis 11y agoI think they might only do live sites, the thing about Percy is that it can be integrated earlier into your feature/browser tests for all pages, even those behind logins.
- Kmandr 11y agoI've been working on VisualReview (https://github.com/xebia/VisualReview https://github.com/xebia/VisualReview), an open source project for visually reviewing a (web-)application as part of a pull-request, very similar to percy.io. We're releasing our first version somewhere next week which includes a basic comparison GUI and testing api's. PR workflow planned in the release after that. Interesting to see we're not the only ones with this idea.
- pimlottc 11y agoDevil's advocate - won't this tend to bolster the outdated attitude that the front end must be pixel-perfect?
- olefoo 11y agoIt depends on the project. But when you're dealing with mobile devices you don't get to test even a fraction of the models available unless you've got enough scale to have a dedicated testing lab.
- proexploit 11y agoI'm not sure I consider that to be true. Why is pixel perfection out-dated?
- vladharbuz 11y agoPerhaps because rigid constraints are useless with the number of different devices out today, and layouts should instead be highly adaptive?
- dubcanada 11y agoNo that's wrong. The screenshots are not based on one device looking like another. It's based on one device looking like X all the time. For example this uses Firefox, if all I change is something in the backend. My front end should look the exact same before the change and after.
- mark2hayes 11y agoThere's another tool out there that does something similar called Screenbeacon (https://www.screenbeacon.com/ https://www.screenbeacon.com/).
- idibidiart 11y agoI wonder, does it handle responsive UIs?
- kordless 11y agoI signed up. Clicked on the survey, clicked back, got a blank page. Did it again, got a live page. Did it one final time, got a blank page. Tried it again just now, blank page (but there is HTML in the page source).
- joshdance 11y agoReally interested how this will work with responsive. Does it test a set size each time? Multiple sizes?
- ecesena 11y agoThere's no mention about pricing... any rough idea/estimation?
- amitaibu 11y agoCongrats @fotinakis Check out also http://shoov.io/ http://shoov.io/ - there are a few blog posts, that explain the concepts and point to examples. Apart of the visual regression testing it also allows monitoring the live sites with your preferred functional testing frameworks (e.g. casperJS, Behat, etc'). Open source of course :)
- jmprobert 11y agoInteresting read! Bummer to hear the news though.
- eN4zeV4jod34Ch 11y agoThis looks like a great project! It looks pretty similar to PhantomCSS, but you're taking the pain out of hosting, which I appreciate: https://github.com/Huddle/PhantomCSS https://github.com/Huddle/PhantomCSS One of my biggest struggles is introducing development practices like regression testing to traditional designers, a problem which is compounded by a lack of good tooling. Hopefuly Percy (and other tools like it) can help to alleviate this Problem. I wonder if we could eventually even have TDD for designs. So once a component is designed in Photoshop, the first step would be to submit an image into Percy and then subsequent commits would check if the CSS had evolved to the point where it matched the original static design.
- leoi11 11y agowould pixel by pixel diffs be an overkill?
- alkonaut 11y agoDoesn't say what it does. I can deduce from the title and landing page that it has something to do with UIs and CI. From the FAQ it seems it is web only, as it mentions "the page" (assume that is an html page). Is this an appropriate summary? "It can help you visually review changes made to a web front end as part of CI/QA" Would be neat if you could plug any kind of front end screenshot rendering and not just web.
- deleted 11y ago[deleted]
- On 11y agoMy favorite in this space is Applitools eyes (https://applitools.com/ https://applitools.com/). Eyes has really advances image processing capabilities, which meant two huge advantages over everything else I looked at: 1. It's not prone to rendering differences. I can set a baseline in one browser/OS combo, run it on other combinations and set it to only consider real regressions as a failure. 2. It can validate the layout of the page even with dynamic content.