Part II: Contagion of Scraping – Has It Gone Too Far?

Part II: Contagion of Scraping – Has It Gone Too Far? (September 30, 2013)

To pick back up where we left off last week – is the scraping of data legal or illegal? If we take a look back over the history of scraping, it appears that the practice started in a “legal gray area,” and that it was initially deemed to simply be a “nuisance.” However, in 2000, eBay, the online auction company, successfully applied the “trespass to chattels” theory to obtain a preliminary injunction against Bidder’s Edge (“BE”), an "aggregator" of auction listings. Ultimately, the lawsuit was settled out of court so it never came to a head, but the legal precedent was set after the courts found that although BE's interference was not substantial, "any intermeddling with or use of another's personal property" established BE's possessory interference with eBay's chattel. In other words, if other auction aggregators started to “crawl” the eBay site, it could potentially lead to the point of denying effective access to eBay's customers.

In 2001 however, a travel agency sued a competitor who had “scraped” its prices from its Web site to help the rival set its own prices in the case known as EF Cultural Travel BV v. Explorica. The judge ruled that the fact that this scraping was not welcomed by the site’s owner was not sufficient to make it “unauthorized access” for the purpose of federal hacking laws. The competitor had used bots to gather all of the agency's prices and travel data, using it to set up their own website with competitive pricing.

The judge ruled that the competitor's use of the agency's information did not constitute hacking. Instead, it ruled that it was fair use of information that the travel agency had made public. Two years later, the legal standing for eBay v. Bidder’s Edge was implicitly overruled in “Intel v. Hamidi,” a case interpreting California’s common law trespass to chattels.

Over the next several years, the courts ruled time and time again that simply putting a “do not scrape us” on a website’s terms of service was not enough to warrant a legally binding agreement. In order for a website owner to enforce the “term,” a user must explicitly agree or consent to the terms. Based on the courts’ ruling, scrapers continued to operate under the status quo – and there was no change in the norms in the business. 

In 2009, Facebook won one of the first copyright suits against a web scraper. Following the court’s ruling, numerous lawsuits were filed to tie any web scraping with a direct copyright violation and clear monetary damages. For example, the most recent case involves Associated Press v. Meltwater, where the courts concluded that even small percentages, sometimes “as little as 4.5% of the content,” are significant enough to not fall under fair use.

Fast forward to today’s online environment where Internet gurus have grown nervous as the “beat of the drums” against scraping has grown louder over the past 12 months. With the revelation of the National Security Agency’s breaches of privacy (thanks to Snowden) and suspicion of the government by U.S. citizens reaching new highs, there have been a number of calls to institute and/or implement scraping laws to protect all types of data on websites, not only personal data. In mid-August 2013, while most people were on the beach or by a pool, a federal judge in San Fransisco ruled that Craigslist can invoke a controversial anti-hacking law to stop a start-up known called 3Taps from gaining access to its website. 3Taps had argued that the law in question, known as the Computer Fraud and Abuse Act (“CFAA”), should only apply to non-public information protected by passwords or firewalls – not free, public data found on sites like Craigslist. Judge Charles Breyer of the District Court in San Francisco disagreed with this view, ruling that 3Taps had accessed Craigslist’s website “without authorization” under the plain meaning of the CFAA. While Craigslist is a public website, the company blocked 3Taps from accessing the site and also issued a cease-and-desist letter in order to stop the start-up from collecting its classified data and making it available to others. After Craigslist blocked it, 3Taps turned to so-called “IP rotation technology” (tools that disguised its identity) to continue to visit the site and scrape data. The judge ruled that this “IP-masking” was enough to violate the anti-hacking statute, dismissing concerns that this ruling would criminalize ordinary Internet use.

As mentioned in last week’s blog, ~30% of the travel industry website traffic is traced to web scraping bots, according to a study by Distil Networks. If the courts were to reverse course and overrule the precedent set for the travel industry in EF Cultural Travel BV v. Explorica, there could be a dramatic shift and change in consumer’s ability to gain access to the best available travel deals available in the market.