Course


What is web scraping?

Learn about web scraping and legals.

Web scraping

Web scraping is the process of gathering information from the Internet. Even copy-pasting the lyrics of your favorite song is a form of web scraping! However, the words “web scraping” usually refer to a process that involves automation. Some websites don’t like it when automatic scrapers gather their data, while others don’t mind.

Despite the demonstrable power of web scraping, issues pertaining to its legitimacy have somewhat shrouded its benefits in illegality. ‘Breaching and Entering: When Data Scraping Should Be a Federal Computer Hacking Crime’, Myra F. Din.

Image

There exists a number of tools (Beautiful Soup, Scrapy, Selenium web driver and so on) that can help you to process the HTML data after downloading it from a page. If you’re scraping a page respectfully for educational purposes (which is this course!), then you’re unlikely to have any problems. Still, it’s a good idea to do some research on your own and make sure that you’re not violating any Terms of Service before you start a large-scale project. To learn more about the legal aspects of web scraping, check out Legal Perspectives on Scraping Data From The Modern Web.