6 ms·

Show HN: Scaling PhantomJS With Ghost Town

32 points by tkazec 12y ago

dsl 12y ago
> a system to reliably render images at scale I'm curious what exactly "render" means in your context? Phantom.js is headless, so it would be futile to be simply rendering GIFs and JPGs within the browser I assume?
- martin-adams 12y ago
  Imagine it like a web page to PNG converter. This doesn't need a desktop/display attached to do that. That's what "render" means in a headless context.
  - dsl 12y ago
    Ah. I get it. Brain did not parse "render" as the output side, but rather the input, so I was trying to figure out why they used phantom.js as an elaborate image format converter.
m0th87 12y ago
We developed the same thing: https://github.com/dailymuse/phantom-cluster https://github.com/dailymuse/phantom-cluster It was used for prerender [1] for some time, before being changed so that clustering happened at the HTTP worker level. Scaling PhantomJS is no walk in the park. There's a lot of duct tape to get it to communicate with node.js, and it's easy for a memory leak to occur. [1] https://github.com/prerender/prerender https://github.com/prerender/prerender
- gcb0 12y ago
  you are all doing it wrong... we use phantomjs on its own. and run the test code in it. so its analogous to loading the target page, and then running your code/test in a browser dev console. you take care of concurrency and retries in your preferred shell. this also allows things like integration with Makefiles, etc. using it inside node is just masochism.
codereflection 12y ago
Can you please qualify the "neglected projects" statement? Looking at the commits on both projects show that they are not neglected. phantomjs: https://github.com/ariya/phantomjs/commits/master https://github.com/ariya/phantomjs/commits/master phantomjs-node: https://github.com/sgentle/phantomjs-node/commits/master https://github.com/sgentle/phantomjs-node/commits/master
- FootballMuse 12y ago
  At least for PhantomJS, there are 236 outstanding prs/issues since their last release Jan-26 with only 19 commits mostly dealing with Travis. Also the development update Twitter stream @PhantomJS is not very active.
- tkazec 12y ago
  PhantomJS: Extremely popular (10k+ stars!) with a comparative trickle of commits. The major flaws might be excusable, but the number of small issues and unmerged pull requests is not. phantomjs-node: Popular but badly structured and documented. Many small ignored issues.
deleted 12y ago
[deleted]
damian2000 12y ago
Interested to know if Selenium web driver integration was looked into? You can use Selenium Grid for example to run distributed jobs with a Phantom JS backend... https://github.com/detro/ghostdriver https://github.com/detro/ghostdriver
qhoc 12y ago
I wrote my own doing something similar with leveraging the casperjs layer instead. I was just too lazy to deal with phantomjs directly.
mootothemax 12y ago
Has anyone had any luck with PhantomJS and web fonts? I'd love to use PhantomJS for taking screenshots of designs supplied by third-parties to us, and changing the CSS involved by hand is not a realistic option. The issue itself appears to be nearly 2 years old: https://github.com/ariya/phantomjs/issues/10592 https://github.com/ariya/phantomjs/issues/10592 And PhantomJS's core developers appear to be putting off the fix until version 2 is released, whenever that may be: https://github.com/ariya/phantomjs/pull/12108 https://github.com/ariya/phantomjs/pull/12108 Any ideas? I'm currently investigating using SlimerJS, which uses the Gecko engine, but that involves its own different set of challenges.

32 points by tkazec 12y ago

dsl 12y ago

> a system to reliably render images at scale I'm curious what exactly "render" means in your context? Phantom.js is headless, so it would be futile to be simply rendering GIFs and JPGs within the browser I assume?

martin-adams 12y ago
Imagine it like a web page to PNG converter. This doesn't need a desktop/display attached to do that. That's what "render" means in a headless context.
- dsl 12y ago
  Ah. I get it. Brain did not parse "render" as the output side, but rather the input, so I was trying to figure out why they used phantom.js as an elaborate image format converter.

m0th87 12y ago

We developed the same thing: https://github.com/dailymuse/phantom-cluster https://github.com/dailymuse/phantom-cluster It was used for prerender [1] for some time, before being changed so that clustering happened at the HTTP worker level. Scaling PhantomJS is no walk in the park. There's a lot of duct tape to get it to communicate with node.js, and it's easy for a memory leak to occur. [1] https://github.com/prerender/prerender https://github.com/prerender/prerender

gcb0 12y ago
you are all doing it wrong... we use phantomjs on its own. and run the test code in it. so its analogous to loading the target page, and then running your code/test in a browser dev console. you take care of concurrency and retries in your preferred shell. this also allows things like integration with Makefiles, etc. using it inside node is just masochism.

codereflection 12y ago

Can you please qualify the "neglected projects" statement? Looking at the commits on both projects show that they are not neglected. phantomjs: https://github.com/ariya/phantomjs/commits/master https://github.com/ariya/phantomjs/commits/master phantomjs-node: https://github.com/sgentle/phantomjs-node/commits/master https://github.com/sgentle/phantomjs-node/commits/master

FootballMuse 12y ago
At least for PhantomJS, there are 236 outstanding prs/issues since their last release Jan-26 with only 19 commits mostly dealing with Travis. Also the development update Twitter stream @PhantomJS is not very active.
tkazec 12y ago
PhantomJS: Extremely popular (10k+ stars!) with a comparative trickle of commits. The major flaws might be excusable, but the number of small issues and unmerged pull requests is not. phantomjs-node: Popular but badly structured and documented. Many small ignored issues.

deleted 12y ago

[deleted]

damian2000 12y ago

Interested to know if Selenium web driver integration was looked into? You can use Selenium Grid for example to run distributed jobs with a Phantom JS backend... https://github.com/detro/ghostdriver https://github.com/detro/ghostdriver

qhoc 12y ago

I wrote my own doing something similar with leveraging the casperjs layer instead. I was just too lazy to deal with phantomjs directly.

mootothemax 12y ago

Has anyone had any luck with PhantomJS and web fonts? I'd love to use PhantomJS for taking screenshots of designs supplied by third-parties to us, and changing the CSS involved by hand is not a realistic option. The issue itself appears to be nearly 2 years old: https://github.com/ariya/phantomjs/issues/10592 https://github.com/ariya/phantomjs/issues/10592 And PhantomJS's core developers appear to be putting off the fix until version 2 is released, whenever that may be: https://github.com/ariya/phantomjs/pull/12108 https://github.com/ariya/phantomjs/pull/12108 Any ideas? I'm currently investigating using SlimerJS, which uses the Gecko engine, but that involves its own different set of challenges.