Dealing with JavaScript

Together with HTML and CSS, JavaScript forms the third and final core building block of the modern web. We’ve already seen JavaScript appearing occasionally throughout this book, and it’s time that we take a closer look at it. As we’ll soon see in this chapter, our sturdy requests plus Beautiful Soup combination is no longer a viable approach to scrape JavaScript-heavy pages. As such, this chapter will introduce you to another library, Selenium, which is used to automate and control a full-fledged web browser.