Most recently, this forced me to upgrade: https://github.com/Medium/phantomjs/issues/161 And that forced upgrade broke the cookie mock/stub code that we wrote. It took almost a day of thrashing to fix the failures, none of which could be seen in a real browser. Super irritating.
I've also had to work around weird JS issues that only affect Phantom, and corrupted screenshots making it rather hard to see what's going wrong. For our team's time-spent vs time-saved, Phantom has been a waste. Our app is very JS-heavy, but that's why I thought Phantom would be a good idea.
I think the most important kind of test is the kind that developers will actually use. If tests are hard to understand, hard to run, unreliable, or slow, then they won't get used (Selenium!).
I just want idiomatic, quick, easy-to-write tests so regressions in our app are much less likely to happen. Angular's approach met those requirements pretty well and that was a big influence to our choice to use it (two years ago, data point of one). It looks like React has a good approach to testing too.
If the Aurelia docs suggest using Jasmine to test the application, especially if they help setting up html fixtures and data, then that's encouraging. But I don't think they do. For now it's all DIY so, if I join your Aurelia team, I have to spend a bunch of time learning how your testing works? In my experience, that means your project will most likely have very little testing.
all good points -- will take this in mind when recommending phantom in the future. While I haven't run into the same # of issues, but I can absolutely see why you guys didn't like it, and the possible issues I could run into some day using it.
Also agree with writing tests that devs will actually run, though I think that is going away with the advancements in orchestration/devops. If buildbot or jenkins can run the tests then you may not have to rely on a human's willingness to run it. Speed's another matter though.
Oh and no, they don't suggest using Jasmine to set up the application (not that I read) -- that was just my assumption (that you could). And that's a very good point, if they don't pay attention to testing (and make it very obvious/easy), then it will fall by the wayside.
I've also had to work around weird JS issues that only affect Phantom, and corrupted screenshots making it rather hard to see what's going wrong. For our team's time-spent vs time-saved, Phantom has been a waste. Our app is very JS-heavy, but that's why I thought Phantom would be a good idea.
I think the most important kind of test is the kind that developers will actually use. If tests are hard to understand, hard to run, unreliable, or slow, then they won't get used (Selenium!).
I just want idiomatic, quick, easy-to-write tests so regressions in our app are much less likely to happen. Angular's approach met those requirements pretty well and that was a big influence to our choice to use it (two years ago, data point of one). It looks like React has a good approach to testing too.
If the Aurelia docs suggest using Jasmine to test the application, especially if they help setting up html fixtures and data, then that's encouraging. But I don't think they do. For now it's all DIY so, if I join your Aurelia team, I have to spend a bunch of time learning how your testing works? In my experience, that means your project will most likely have very little testing.