|
via Udemy |
Go to Course: https://www.udemy.com/course/advanced-web-scraping-with-python-using-scrapy-splash/
Certainly! Here's a comprehensive review and recommendation for the Coursera course on Web Scraping with Python using Scrapy & Splash: --- **Course Review: Advanced Web Scraping with Python using Scrapy & Splash** This course offers an in-depth, project-based approach to mastering web scraping with Python, focusing on powerful tools like Scrapy and Splash. Designed for individuals with prior knowledge of web scraping, this course dives straight into real-world applications, making it ideal for learners who are already familiar with the basics. **What You Will Learn:** - **Requests Chaining:** Understanding the importance of request order and how to manage sequences for successful data extraction. - **Website Analysis:** Techniques to analyze websites beforehand to determine the best scraping approach, significantly impacting performance. - **Splash Optimization:** Tips on optimizing Splash scripts by eliminating unnecessary requests, which helps prevent common errors like 504 Gateway Timeout. - **Cluster Building:** Strategies to create a scalable infrastructure using multiple Splash instances with load balancing via HAProxy. - **Heavy Data Processing:** Using Input & Output processors for data cleaning to ensure high-quality results. - **Real-Time Data Fetching:** Leveraging ScrapyRT to build spiders capable of real-time data extraction. - **Data Presentation:** Creating minimalist web applications with Flask to showcase scraped data, adding professionalism and user-friendliness. - **Bypassing ReCaptcha:** Techniques to mimic human browsing behavior, enhancing your ability to access protected websites. - **Structured Spiders & Desktop Apps:** Building well-structured spiders and developing desktop applications with Tkinter for easier client delivery, especially beneficial for freelancers. **Strengths:** - The course is highly practical, with each section focusing on a different website and scraping dilemma, offering hands-on experience. - It emphasizes performance optimization and scalable infrastructure, crucial for handling large-scale scraping projects. - The inclusion of GUI development and web app showcase adds valuable skills for delivering professional solutions to clients. - The instructor’s approach is direct and focused, skipping theoretical fluff to emphasize actionable skills. **Who is this course for?** This course is best suited for intermediate to advanced web scrapers who already understand the basics of scraping, Scrapy, Splash, and XPath. Beginners with no foundational knowledge may find it challenging due to the course’s focus on complex, real-world concepts. **Would I recommend this course?** Yes, if you are looking to elevate your web scraping skills, particularly for high-performance and client-ready projects. The emphasis on building user-friendly tools and scalable systems makes it a valuable investment for freelancers and professionals aiming to stand out in the web scraping industry. **Final Verdict:** This course is an excellent resource for those looking to move beyond beginner tutorials and develop sophisticated, efficient web scrapers using Scrapy and Splash. Its project-based approach ensures you gain practical skills that can be immediately applied to real-world challenges. Be prepared to commit focus and effort, as the material is intense but rewarding. --- Would you like me to help craft a shorter summary or promotional blurb for sharing this review?
Hi there & welcome to the most advanced online resource on Web Scraping with Python using Scrapy & Splash. This course is fully project-based means pretty much on each section we gonna scrape a different website & tackle a different web scraping dilemma also rather than focusing on the basics of Scrapy & Splash we gonna dive straight forward into real-world projects, this also means that this course is absolutely not suitable for beginners with no background on web scraping, Scrapy, Splash & XPath expressions.--This courses covers a variety of topics such as:--Requests chaining, like how the requests must be sent in a certain order otherwise they won't be fulfilled at all.How to analyze a website before scraping it, this is an important step to do since it helps a lot in choosing the right tools to scrape a website & it literally has a huge impact on the performance of your final product.How to optimize Splash scripts by reducing/aborting all the unnecessary requests that have nothing to do with the data points you're going to scrape, this is an important thing to do if you care about the performance of Splash as it is the key to bypass 504 Gateway Timeout HTTP errors in Splash. We gonna also cover how to build a Cluster of Splash instances with a load balancer(HAProxy) rather than having one fully overloaded Splash instance this also helps in bypassing 504 Gateway Timeout errors. Heavy data processing, you'll understand how Input & Output processors work so you'll be able to use them in order to clean the scraped data points as this will ensure the quality of your feeds.We'll use ScrapyRT (Scrapy RealTime) to build spiders that can fetch data in real-time.Showcase the scraped data points in a minimalist web app using ScrapyRT & Flask, this is extremely helpful for web scraping freelancers.Bypass Google ReCaptcha, please don't get me wrong on this point, I don't mean that we will solve it using Scrapy, instead, I'm gonna show you a technique that I use frequently to fool websites and let them think that the request is sent using a browser & was performed by a human being!Build clean & well-structured spidersFinally, we gonna build a Desktop app using Tkinter, the app will fetch & execute all the available spiders in your Scrapy project, you can also choose the feed type, feed location & name, this is also extremely helpful & important if you're a web scraping freelancer, it is always a good idea to deliver to your client a desktop app rather than installing Scrapy on his machine & stuff like that.This course is straight to the point, there's no "foobar" or "quotes to toscrape dot com" as other courses do so make sure you have a good level of focus & lot of determination & motivation.By the end of this course, you'll sharpen your skills in web scraping using Scrapy & Splash, you'll be able to write clean & high performing spiders that differentiate you from others, this also means if you're a web scraping freelancer you'll get more offers since you can deliver "User-Friendly" spiders with a Graphical User Interface(GUI) or web apps that fetch data in real-time. So join me on this course & let's harvest the web together!