8 ms·

pretty bad precedent, id argue that anything thats publicly accesible is free to be scraped. So unless they did something illegal via bypassing security, then t

pretty bad precedent, id argue that anything thats publicly accesible is free to be scraped. So unless they did something illegal via bypassing security, then this sets a really bad tone for internet archival, web scraping and data collection in the future.

walthamstow 2y ago
As I understand it, scraping data read-only is fine (Google Flights, Skyscanner etc) but using automated processes to book tickets on behalf of customers without sending them to the Ryanair site is not fine.
- rob74 2y ago
  And here I was thinking that read-only was part of the definition of scraping?
  - tgv 2y ago
    The article mentions reselling.
  - rsynnott 2y ago
    Yeah, it's not a terribly well-written article, and _Ryanair_ is certainly trying to push the line that this is about scraping, but it's hard to imagine that the _resale_ thing wasn't a significant part of the case.
- skissane 2y ago
  If the ruling (which I haven't read) says automated booking via screen scraping is illegal: wouldn't a workaround be to replace the automated process with a human in a low-cost country?
  - hellweaver666 2y ago
    That sounds like a very Amazon-esque solution!
  - arghwhat 2y ago
    Sure, but that's what you want, not what booking.com wants. Booking.com wants to charge you for the service, and if they're not part of the transaction they'd have to get money out-of-band, e.g. as a subscription for the price finding service. Remember that this lawsuit is between two large companies both trying to get your money.
    - skissane 2y ago
      > Booking.com wants to charge you for the service, and if they're not part of the transaction they'd have to get money out-of-band I think you misunderstand what I meant. To be more specific: if the ruling says that automated booking via screen scraping is illegal – what's to stop Booking.com hiring warm bodies in low cost countries, replacing their fully automated solution with a semi-automated solution to dodge the ruling, and continuing to charge their customers for that service?
      - arghwhat 2y ago
        The ruling does not say anything about screen scraping or automation, and the verdict also holds if warm bodies in low cost countries were used. What was ruled was that: 1. That Booking.com "intentionally directed, encouraged or induced Etraveli to access the myRyanair portion of Ryanair's website without authorization" 2. That "Etraveli recklessly caused Damage to a protected computer by way of such access to the myRyanair portion of Ryanair's website without authorization" 3. That "Etraveli caused both Damage to a protected computer and Loss by way of such access to the myRyanair portion of Ryanair's website without authorization" 4. That booking.com "knowingly and with intent to defraud, directed, encouraged, or induced a third party to access the myRyanair portion of Ryanair's website without authorization and by means of such conduct furthered the intended fraud and obtained something of value for booking.com" 5. That "the object of the fraud and the thing of value obtained by Booking.com [was] only the use of the myRyanair portion of Ryanir's website"
  - whazor 2y ago
    Or performing screen scraping in a country where it is not illegal.
- reddalo 2y ago
  I'm not sure Google Flights actually scrapes RyanAir. I think they connect to API services such as Amadeus.
  - lesuorac 2y ago
    Google owns ITA which has all of the pricing data. https://en.wikipedia.org/wiki/ITA_Software https://en.wikipedia.org/wiki/ITA_Software
    - chx 2y ago
      Not Ryanair, no. Wizz Air got semi-recently hooked into the GDS systems but Ryanair no.
sva_ 2y ago
In fact I wrote my own RyanAir scraper to get the best prices on flights (it is trivial to implement, they barely have any rate limits - in particular if you use the API endpoint where you search for flights departing from a specific airport). But I guess my scraping is a lot less, as I'm only looking for a few flights
- chrisjj 2y ago
  Same here. Except my rate was sufficient to trigger throttling.
  - sva_ 2y ago
    Which API requests did you use?
    - chrisjj 2y ago
      HTTP POST :)
arghwhat 2y ago
This lawsuit is not about scraping, it is about booking.com acting as a reseller of ryanair products without a reseller agreement - to which they mention that booking.com adds their own profits to the transaction and makes ryanair unable to communicate with the real customer - through "unauthorized access" (scraping). What we on hackernews would consider scraping is not covered by this lawsuit, and ryanair's vendetta is not against scraping but "pirate online travel agencies" (resellers).
- wcedmisten 2y ago
  Would a restaurant have the same case against Doordash? Seems like a fairly common business model
  - arghwhat 2y ago
    Depends. Established, well-behaved food delivery apps have agreements with the restaurants and have direct integration - not scraping. They take absurd margins, but that's a separate issue. When the food delivery apps "scrape", it's sometimes okay, but often not: The food offered by a place might be made to be eaten immediately, in which case a 60 minute delivery might guarantee a horrible experience. The food might not even be safe to transport by intermediate handlers, such as if the food is not packaged in sealed containers. In both cases, the food place ends up with dissatisfied customers and bad reputation for something they neither did nor wanted to do. John Oliver had an interesting video on the topic: https://www.youtube.com/watch?v=aFsfJYWpqII https://www.youtube.com/watch?v=aFsfJYWpqII
- ericd 2y ago
  Why is the CFAA mentioned, then? That’s historically been used as a bludgeon against scraping. “A US court ruled that Booking.com violated the Computer Fraud and Abuse Act by accessing part of Ryanair's website without permission, court documents showed, a ruling the airline said would help end unauthorised screen scraping by booking sites.”
  - lesuorac 2y ago
slightwinder 2y ago
It's not public, it's just generally accessible but with rules. It's similar to how a physical shop is accessible to everyone by default, but the owner still has the right to refuse business and deny access if you behave bad. The problem is that you can't easily deny companies like booking-com access to a website, as they can circumvent any technical barrier.
chrisjj 2y ago
Simple scraping is one thing. Scraping to resell in breach of agreed terms is quite another.

walthamstow 2y ago

As I understand it, scraping data read-only is fine (Google Flights, Skyscanner etc) but using automated processes to book tickets on behalf of customers without sending them to the Ryanair site is not fine.

rob74 2y ago
And here I was thinking that read-only was part of the definition of scraping?
- tgv 2y ago
  The article mentions reselling.
- rsynnott 2y ago
  Yeah, it's not a terribly well-written article, and _Ryanair_ is certainly trying to push the line that this is about scraping, but it's hard to imagine that the _resale_ thing wasn't a significant part of the case.
skissane 2y ago
If the ruling (which I haven't read) says automated booking via screen scraping is illegal: wouldn't a workaround be to replace the automated process with a human in a low-cost country?
- hellweaver666 2y ago
  That sounds like a very Amazon-esque solution!
- arghwhat 2y ago
  Sure, but that's what you want, not what booking.com wants. Booking.com wants to charge you for the service, and if they're not part of the transaction they'd have to get money out-of-band, e.g. as a subscription for the price finding service. Remember that this lawsuit is between two large companies both trying to get your money.
  - skissane 2y ago
    > Booking.com wants to charge you for the service, and if they're not part of the transaction they'd have to get money out-of-band I think you misunderstand what I meant. To be more specific: if the ruling says that automated booking via screen scraping is illegal – what's to stop Booking.com hiring warm bodies in low cost countries, replacing their fully automated solution with a semi-automated solution to dodge the ruling, and continuing to charge their customers for that service?
    - arghwhat 2y ago
      The ruling does not say anything about screen scraping or automation, and the verdict also holds if warm bodies in low cost countries were used. What was ruled was that: 1. That Booking.com "intentionally directed, encouraged or induced Etraveli to access the myRyanair portion of Ryanair's website without authorization" 2. That "Etraveli recklessly caused Damage to a protected computer by way of such access to the myRyanair portion of Ryanair's website without authorization" 3. That "Etraveli caused both Damage to a protected computer and Loss by way of such access to the myRyanair portion of Ryanair's website without authorization" 4. That booking.com "knowingly and with intent to defraud, directed, encouraged, or induced a third party to access the myRyanair portion of Ryanair's website without authorization and by means of such conduct furthered the intended fraud and obtained something of value for booking.com" 5. That "the object of the fraud and the thing of value obtained by Booking.com [was] only the use of the myRyanair portion of Ryanir's website"
- whazor 2y ago
  Or performing screen scraping in a country where it is not illegal.
reddalo 2y ago
I'm not sure Google Flights actually scrapes RyanAir. I think they connect to API services such as Amadeus.
- lesuorac 2y ago
  Google owns ITA which has all of the pricing data. https://en.wikipedia.org/wiki/ITA_Software https://en.wikipedia.org/wiki/ITA_Software
  - chx 2y ago
    Not Ryanair, no. Wizz Air got semi-recently hooked into the GDS systems but Ryanair no.

sva_ 2y ago

In fact I wrote my own RyanAir scraper to get the best prices on flights (it is trivial to implement, they barely have any rate limits - in particular if you use the API endpoint where you search for flights departing from a specific airport). But I guess my scraping is a lot less, as I'm only looking for a few flights

chrisjj 2y ago
Same here. Except my rate was sufficient to trigger throttling.
- sva_ 2y ago
  Which API requests did you use?
  - chrisjj 2y ago
    HTTP POST :)

arghwhat 2y ago

This lawsuit is not about scraping, it is about booking.com acting as a reseller of ryanair products without a reseller agreement - to which they mention that booking.com adds their own profits to the transaction and makes ryanair unable to communicate with the real customer - through "unauthorized access" (scraping). What we on hackernews would consider scraping is not covered by this lawsuit, and ryanair's vendetta is not against scraping but "pirate online travel agencies" (resellers).

wcedmisten 2y ago
Would a restaurant have the same case against Doordash? Seems like a fairly common business model
- arghwhat 2y ago
  Depends. Established, well-behaved food delivery apps have agreements with the restaurants and have direct integration - not scraping. They take absurd margins, but that's a separate issue. When the food delivery apps "scrape", it's sometimes okay, but often not: The food offered by a place might be made to be eaten immediately, in which case a 60 minute delivery might guarantee a horrible experience. The food might not even be safe to transport by intermediate handlers, such as if the food is not packaged in sealed containers. In both cases, the food place ends up with dissatisfied customers and bad reputation for something they neither did nor wanted to do. John Oliver had an interesting video on the topic: https://www.youtube.com/watch?v=aFsfJYWpqII https://www.youtube.com/watch?v=aFsfJYWpqII
ericd 2y ago
Why is the CFAA mentioned, then? That’s historically been used as a bludgeon against scraping. “A US court ruled that Booking.com violated the Computer Fraud and Abuse Act by accessing part of Ryanair's website without permission, court documents showed, a ruling the airline said would help end unauthorised screen scraping by booking sites.”
- lesuorac 2y ago

slightwinder 2y ago

It's not public, it's just generally accessible but with rules. It's similar to how a physical shop is accessible to everyone by default, but the owner still has the right to refuse business and deny access if you behave bad. The problem is that you can't easily deny companies like booking-com access to a website, as they can circumvent any technical barrier.

chrisjj 2y ago

Simple scraping is one thing. Scraping to resell in breach of agreed terms is quite another.