6 ms·

Show HN: I created a browser automation tool

176 points by iamandras 4y ago

I created this tool a few months ago to automate some of my tasks. I use it to collect prices and files and to get notified when something changes on certain websites. The task runner uses Playwright. I'm not sure if it can be useful to anybody else besides me :) Any feedback is welcome.

partloyaldemon 4y ago
I'll check it out. Thanks!
Krastan 4y ago
This is cool! Is it free? If not would be good to have pricing be visible without creating an account. Also the logo in the header looks low definition. Upping its resolution will make the website way better.
- matthberg 4y ago
  Looks like a simple enough design that vector would probably be more space efficient and have infinite resolution. The product looks interesting too!
- iamandras 4y ago
  It is free for a short period of time, but after that, it will be a paid service.
  - permalac 4y ago
    Do you have a cost plan? I've invested so much time in similar things just to not be able to afford it...
    - iamandras 4y ago
      What similar things?
    - permalac 4y ago
      My god, I'm on cellphone and did not see the costs at the bottom. Apologies.
thih9 4y ago
Is there any way to play with the product without registering? Or perhaps see a video showing it in action?
- whatatita 4y ago
  Looks like you can register without CC details, and the majority of the homepage is a video example.
  - thih9 4y ago
    > the majority of the homepage is a video example I’m not seeing any videos on the homepage (I’m using safari on iOS).
fareesh 4y ago
There's a chrome extension that records all your steps and then generates a puppeteer/playwright script for you. Then you can just edit bits of that script and run it. This is my go-to method presently. I feel like you could incorporate some of those features into this and eliminate some of the manual entry of css selectors.
- iamandras 4y ago
  Oh, interesting. What is the name of that extension?
  - Fnoord 4y ago
    Selenium allows this, and its also there for Firefox. You can even export the actions as a different language e.g. Python.
  - yareally 4y ago
    Probably this: https://chrome.google.com/webstore/detail/headless-recorder/djeegiggegleadkkbgopoonhjimgehda https://chrome.google.com/webstore/detail/headless-recorder/...
  - EddySchauHai 4y ago
    You can do it in the chrome browser tools! There's a record section. Note if you're doing this for automation testing it's going to be pretty flakey and you're better off following best practices and using custom testing selectors instead :)
    - fizzpack 4y ago
      Why is it flakey for automation testing though? In my experience that's basically targeting page elements and interacting with them. Doesn't this do the same thing?
      - EddySchauHai 4y ago
        It uses a combination of xpaths that are like body[1]div[2]div[1]p[2]button[1] so if you edit the UI it'll break, or worst case they do X,Y coordinates of clicks on a page so if the UI changes it'll break. Sometimes it'll get your ID or Name or whatever but some frameworks randomize these on compilation and some devs change them as part of refactoring. Best practice is to add test attributes to the HTML elements (such as data-testid="SubmitButton") so if the page is moved around or the button is edited the tests remain stable. There's a lot of good online sources on making UI testing somewhat more reliable :)
bcjordan 4y ago
I'd love to use this as a webpage interaction/scraping component in chains of webhooks (building on e.g. low-code step pipelines built with Pipedream, could imagine fitting with something like Zapier, too). Would require the ability to: (1) make a web request to start a run (2) collect the scraped info and return it as part of (a) a webhook call or (b) in the response of a synchronous "start run" blocked long http request It sounds like #1 is already possible. 2a might be possible if there was a call webhook step or notification option? But 2b would be the simplest (if I could make a single GET or POST request, and the response gives me some outputs from an automation session). Zapier/Pipedream integrations could be nice platform integration marketing for the tool as well.
dhux 4y ago
Can you also record video with sound?
- iamandras 4y ago
  no.
  - asot22 4y ago
    why not? seems a reasonable ask?
pmontra 4y ago
> Periodically check for changes I remember a Firefox extension that notified me with the diffs of changed pages. I don't remember exactly what I was using it for. I think something related to CVEs for some keywords. Then the service I was using shut down and I lost tracks of that extension.
- ChrisGranger 4y ago
  Perhaps Distill Web Monitor or PageProbe? https://addons.mozilla.org/en-US/firefox/addon/distill-web-monitor-ff/ https://addons.mozilla.org/en-US/firefox/addon/distill-web-m... https://addons.mozilla.org/en-US/firefox/addon/pageprobe/ https://addons.mozilla.org/en-US/firefox/addon/pageprobe/
- iamandras 4y ago
  yeah. I was considering creating a focused version for this like visualping.io
- xnx 4y ago
  Distill Web Monitor? https://addons.mozilla.org/en-US/firefox/addon/distill-web-monitor-ff/ https://addons.mozilla.org/en-US/firefox/addon/distill-web-m...
- beatthatflight 4y ago
  Update Scanner. I use it a lot to track airline prices/sales.
  - pmontra 4y ago
    It could be this one. I remember the yellow highlights. It was many years ago and definitely not a cloud service so not Distill Web Monitor. It didn't let me define fields to monitor, so not PageProbe. I wouldn't have used a cloud service. What's the point? My browser already have everything it takes to download pages and diff them. I understand that if the computer is off the browser won't monitor the page but it was ok with my use case: if it's off I won't be able to react to the CVE. And for other use cases, it's no business of anyone else but me to know what I'm monitoring.
ramoz 4y ago
Curious what your pricing strategy is here? The free offering is cool for sure. Guess you'll learn how serious users may want to actually use the service, what that means for your unit costs, and how it all scales in your favor? or sort of already figured that out?
- dieselgate 4y ago
  Not sure if you’re asking about Author’s long term pricing strategy but the tiered pricing is on the site
  - ramoz 4y ago
    more about how the author came up with the pricing and if/how they developed projections around it. There also wasnt a custom offering, so was wondering why not and if the author felt it wouldnt be financially effective.
anon115 4y ago
landing page is nice
BeetleB 4y ago
Only tangentially related, but... I use Firefox. Anyone know of a way I can have it close all tabs once a day? Either via an external script or an extension?
- beatthatflight 4y ago
  Selenium would do that, in python, no worries. Alternatively, have a batch script that just runs (Assuming windows) taskkill /f /im firefox.exe which would kill it. Then have firefox startup with no tabs loaded.
- yjftsjthsd-h 4y ago
  Not scripted, but ctrl-shift-w will do it manually in a single keystroke. (Per window)
blackdogie 4y ago
Best of luck. Two things that have tripped me up 1. "and save the value of this attribute", it wasn't clear that this could only be certain values, text, id. 2. API I can list tasks, get the status of a task, start tasks via API, but I don't se how I can read the results / stored data
melony 4y ago
Rocket ride is the name of Stripe's API demo app. Might want to change the name, this is bad for your SEO.
lorey 4y ago
Really interesting tool. I've always found the task of finding the right CSS selectors tiresome, so I created tool which automatically extracts the right selectors for you (and builds a scraper as well). Open Source on GitHub: https://github.com/lorey/mlscraper https://github.com/lorey/mlscraper
- iamandras 4y ago
  I'll check this repo, thx. It all started as an experiment with Playwright. And I also wanted to try whether I can build a full app, not just some simple scripts.
Heavywater 4y ago
This is similar tools: https://github.com/huginn/huginn https://github.com/huginn/huginn https://n8n.io/ https://n8n.io/
Ladyady 4y ago
Are you no longer accepting new users?

176 points by iamandras 4y ago

partloyaldemon 4y ago

I'll check it out. Thanks!

Krastan 4y ago

This is cool! Is it free? If not would be good to have pricing be visible without creating an account. Also the logo in the header looks low definition. Upping its resolution will make the website way better.

matthberg 4y ago
Looks like a simple enough design that vector would probably be more space efficient and have infinite resolution. The product looks interesting too!
iamandras 4y ago
It is free for a short period of time, but after that, it will be a paid service.
- permalac 4y ago
  Do you have a cost plan? I've invested so much time in similar things just to not be able to afford it...
  - iamandras 4y ago
    What similar things?
  - permalac 4y ago
    My god, I'm on cellphone and did not see the costs at the bottom. Apologies.

thih9 4y ago

Is there any way to play with the product without registering? Or perhaps see a video showing it in action?

whatatita 4y ago
Looks like you can register without CC details, and the majority of the homepage is a video example.
- thih9 4y ago
  > the majority of the homepage is a video example I’m not seeing any videos on the homepage (I’m using safari on iOS).

fareesh 4y ago

There's a chrome extension that records all your steps and then generates a puppeteer/playwright script for you. Then you can just edit bits of that script and run it. This is my go-to method presently. I feel like you could incorporate some of those features into this and eliminate some of the manual entry of css selectors.

iamandras 4y ago
Oh, interesting. What is the name of that extension?
- Fnoord 4y ago
  Selenium allows this, and its also there for Firefox. You can even export the actions as a different language e.g. Python.
- yareally 4y ago
  Probably this: https://chrome.google.com/webstore/detail/headless-recorder/djeegiggegleadkkbgopoonhjimgehda https://chrome.google.com/webstore/detail/headless-recorder/...
- EddySchauHai 4y ago
  You can do it in the chrome browser tools! There's a record section. Note if you're doing this for automation testing it's going to be pretty flakey and you're better off following best practices and using custom testing selectors instead :)
  - fizzpack 4y ago
    Why is it flakey for automation testing though? In my experience that's basically targeting page elements and interacting with them. Doesn't this do the same thing?
    - EddySchauHai 4y ago
      It uses a combination of xpaths that are like body[1]div[2]div[1]p[2]button[1] so if you edit the UI it'll break, or worst case they do X,Y coordinates of clicks on a page so if the UI changes it'll break. Sometimes it'll get your ID or Name or whatever but some frameworks randomize these on compilation and some devs change them as part of refactoring. Best practice is to add test attributes to the HTML elements (such as data-testid="SubmitButton") so if the page is moved around or the button is edited the tests remain stable. There's a lot of good online sources on making UI testing somewhat more reliable :)

bcjordan 4y ago

I'd love to use this as a webpage interaction/scraping component in chains of webhooks (building on e.g. low-code step pipelines built with Pipedream, could imagine fitting with something like Zapier, too). Would require the ability to: (1) make a web request to start a run (2) collect the scraped info and return it as part of (a) a webhook call or (b) in the response of a synchronous "start run" blocked long http request It sounds like #1 is already possible. 2a might be possible if there was a call webhook step or notification option? But 2b would be the simplest (if I could make a single GET or POST request, and the response gives me some outputs from an automation session). Zapier/Pipedream integrations could be nice platform integration marketing for the tool as well.

dhux 4y ago

Can you also record video with sound?

iamandras 4y ago
no.
- asot22 4y ago
  why not? seems a reasonable ask?

pmontra 4y ago

> Periodically check for changes I remember a Firefox extension that notified me with the diffs of changed pages. I don't remember exactly what I was using it for. I think something related to CVEs for some keywords. Then the service I was using shut down and I lost tracks of that extension.

ChrisGranger 4y ago
Perhaps Distill Web Monitor or PageProbe? https://addons.mozilla.org/en-US/firefox/addon/distill-web-monitor-ff/ https://addons.mozilla.org/en-US/firefox/addon/distill-web-m... https://addons.mozilla.org/en-US/firefox/addon/pageprobe/ https://addons.mozilla.org/en-US/firefox/addon/pageprobe/
iamandras 4y ago
yeah. I was considering creating a focused version for this like visualping.io
xnx 4y ago
Distill Web Monitor? https://addons.mozilla.org/en-US/firefox/addon/distill-web-monitor-ff/ https://addons.mozilla.org/en-US/firefox/addon/distill-web-m...
beatthatflight 4y ago
Update Scanner. I use it a lot to track airline prices/sales.
- pmontra 4y ago
  It could be this one. I remember the yellow highlights. It was many years ago and definitely not a cloud service so not Distill Web Monitor. It didn't let me define fields to monitor, so not PageProbe. I wouldn't have used a cloud service. What's the point? My browser already have everything it takes to download pages and diff them. I understand that if the computer is off the browser won't monitor the page but it was ok with my use case: if it's off I won't be able to react to the CVE. And for other use cases, it's no business of anyone else but me to know what I'm monitoring.

ramoz 4y ago

Curious what your pricing strategy is here? The free offering is cool for sure. Guess you'll learn how serious users may want to actually use the service, what that means for your unit costs, and how it all scales in your favor? or sort of already figured that out?

dieselgate 4y ago
Not sure if you’re asking about Author’s long term pricing strategy but the tiered pricing is on the site
- ramoz 4y ago
  more about how the author came up with the pricing and if/how they developed projections around it. There also wasnt a custom offering, so was wondering why not and if the author felt it wouldnt be financially effective.

anon115 4y ago

landing page is nice

BeetleB 4y ago

Only tangentially related, but... I use Firefox. Anyone know of a way I can have it close all tabs once a day? Either via an external script or an extension?

beatthatflight 4y ago
Selenium would do that, in python, no worries. Alternatively, have a batch script that just runs (Assuming windows) taskkill /f /im firefox.exe which would kill it. Then have firefox startup with no tabs loaded.
yjftsjthsd-h 4y ago
Not scripted, but ctrl-shift-w will do it manually in a single keystroke. (Per window)

blackdogie 4y ago

Best of luck. Two things that have tripped me up 1. "and save the value of this attribute", it wasn't clear that this could only be certain values, text, id. 2. API I can list tasks, get the status of a task, start tasks via API, but I don't se how I can read the results / stored data

melony 4y ago

Rocket ride is the name of Stripe's API demo app. Might want to change the name, this is bad for your SEO.

lorey 4y ago

Really interesting tool. I've always found the task of finding the right CSS selectors tiresome, so I created tool which automatically extracts the right selectors for you (and builds a scraper as well). Open Source on GitHub: https://github.com/lorey/mlscraper https://github.com/lorey/mlscraper

iamandras 4y ago
I'll check this repo, thx. It all started as an experiment with Playwright. And I also wanted to try whether I can build a full app, not just some simple scripts.

Heavywater 4y ago

This is similar tools: https://github.com/huginn/huginn https://github.com/huginn/huginn https://n8n.io/ https://n8n.io/

Ladyady 4y ago

Are you no longer accepting new users?