UseScraper blog
Learn how to crawl and scrape the Internet
![](https://framerusercontent.com/images/toIxqM2nUDvX0DZqQ8Awv50JrFQ.png)
Introduction Web Scraping with C# 2024
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Oct 19, 2023
![](https://framerusercontent.com/images/gkygEN8PjUYEww6sfOmB0NN2r0.png)
Making HTTP Requests with Axios
![](https://framerusercontent.com/images/VHVyXGH5H9boyIvYPsuGhTSwgQ.png)
Lukas Jansen
Aug 24, 2023
![](https://framerusercontent.com/images/NimRTD1EMebr06nk7c7lz3nck.png)
How to crawl a website sitemap and scrape all pages with with Scrapy and Python
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Feb 27, 2024
![](https://framerusercontent.com/images/keWz16AKEDOCs53DLHRp0jaQM.png)
Scraping APIs with JavaScript and Node.js
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
May 19, 2023
![](https://framerusercontent.com/images/Yt5ug86sX2mFnMmyQrgoCLlUuw.png)
Common Web Scraping Challenges
![](https://framerusercontent.com/images/VHVyXGH5H9boyIvYPsuGhTSwgQ.png)
Lukas Jansen
Oct 6, 2023
![](https://framerusercontent.com/images/6xbEQ5b9EUwwFMfqLuxMlDQ6H0.png)
Data Extraction and Parsing with JavaScript
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Aug 19, 2023
![](https://framerusercontent.com/images/woTcSW8sxId8K7NjudggAQjRE.png)
Storing and Managing Scraped Data
![](https://framerusercontent.com/images/H2hhcKb4fcZ6A2ndp79i9zSzRE.png)
Caitlin O'Connor
Feb 1, 2024
![](https://framerusercontent.com/images/YEeNh09irIpOcb7izWwMK0EmDA.png)
Introduction Web Scraping with Go 2024
![](https://framerusercontent.com/images/VHVyXGH5H9boyIvYPsuGhTSwgQ.png)
Lukas Jansen
Mar 1, 2024
![](https://framerusercontent.com/images/ZpncP95C8PRAyGmCJEt8xbbpOD8.png)
Scraping Dynamic Websites with Selenium
![](https://framerusercontent.com/images/H2hhcKb4fcZ6A2ndp79i9zSzRE.png)
Caitlin O'Connor
Oct 15, 2023
![](https://framerusercontent.com/images/tHu024XYL4SBtpGc4Bf1qzOBA.png)
Proxies and Rotating IPs for Web Scraping with Python
![](https://framerusercontent.com/images/VHVyXGH5H9boyIvYPsuGhTSwgQ.png)
Lukas Jansen
Nov 29, 2023
![](https://framerusercontent.com/images/zFKqatIh73U0EwZ8XnqHyDJCYL4.png)
Web Scraping FAQs
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Nov 25, 2023
![](https://framerusercontent.com/images/Z4q2FyG3OocSeVehfzfQUCBF4Q.png)
Installing Python and Required Libraries
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Aug 14, 2023
![](https://framerusercontent.com/images/36LXq1kC4wA4a88Nj3jhXQsLa0.png)
Introduction Web Scraping with Ruby 2024
![](https://framerusercontent.com/images/VHVyXGH5H9boyIvYPsuGhTSwgQ.png)
Lukas Jansen
Dec 25, 2023
![](https://framerusercontent.com/images/ZT4XCKBGt2vZHyKyKSyloDnBP4.png)
Making HTTP Requests with Python
![](https://framerusercontent.com/images/H2hhcKb4fcZ6A2ndp79i9zSzRE.png)
Caitlin O'Connor
Feb 23, 2024
![](https://framerusercontent.com/images/X0KPUE1o8srqDG3Lfdk50FlP0.png)
Web Crawling with Python
![](https://framerusercontent.com/images/VHVyXGH5H9boyIvYPsuGhTSwgQ.png)
Lukas Jansen
Nov 30, 2023
![](https://framerusercontent.com/images/VylzhJzOHIgBuN0pgqF8YxrmoPo.png)
The best 5 proxies for web scraping in 2024
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Mar 5, 2023
![](https://framerusercontent.com/images/u0XORBDOti1pYxbE5iuMFsj2LI.png)
Dealing with Rate Limiting and Throttling
![](https://framerusercontent.com/images/H2hhcKb4fcZ6A2ndp79i9zSzRE.png)
Caitlin O'Connor
Mar 6, 2023
![](https://framerusercontent.com/images/EF2pTLyWLMcjFuGoUYtFatj4.png)
Puppeteer vs Playwright - Which is best for browser automation?
![](https://framerusercontent.com/images/H2hhcKb4fcZ6A2ndp79i9zSzRE.png)
Caitlin O'Connor
Dec 31, 2023
![](https://framerusercontent.com/images/oduYUo2WgwMWM9Jwh4ycfoKa8g.png)
Headless Browsing with Puppeteer
![](https://framerusercontent.com/images/H2hhcKb4fcZ6A2ndp79i9zSzRE.png)
Caitlin O'Connor
Feb 10, 2024
![](https://framerusercontent.com/images/RDj8EDrgqKpA391goITp3bvNVRE.png)
Handling Anti-Scraping Measures in JavaScript
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Dec 11, 2023
![](https://framerusercontent.com/images/2rPcC1hEc0XakiIVnFnnsRG4qA.png)
Web Crawling with JavaScript and Node.js
![](https://framerusercontent.com/images/VHVyXGH5H9boyIvYPsuGhTSwgQ.png)
Lukas Jansen
Dec 15, 2023
![](https://framerusercontent.com/images/viURlSrB1GF1rgMzhclePZRHS8.png)
Setting up Node.js Environment for Web Scraping
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Nov 18, 2023
![](https://framerusercontent.com/images/lhhZ66usbaENxNHwZkVSqMhYs.png)
Web Scraping vs Web Crawling
![](https://framerusercontent.com/images/VHVyXGH5H9boyIvYPsuGhTSwgQ.png)
Lukas Jansen
Jan 9, 2023
![](https://framerusercontent.com/images/hIILBAWIGDzw68Mq7qx9FWa413Q.png)
The 5 best open source web crawlers in 2024
![](https://framerusercontent.com/images/H2hhcKb4fcZ6A2ndp79i9zSzRE.png)
Caitlin O'Connor
Sep 28, 2023
![](https://framerusercontent.com/images/QuicRO8ArxN7URaAgGrzqOMPoFw.png)
Scraping APIs and Authenticated Resources
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Apr 25, 2023
![](https://framerusercontent.com/images/v2Ct7br2I84bmp5CLI9GuYtWv4.png)
Scraping at Scale and Performance Optimization
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Jan 10, 2024
![](https://framerusercontent.com/images/Y7Le3r79GdRRCtZ2b5NaxIWptE.png)
Handling CAPTCHAs and Other Anti-Scraping Measures
![](https://framerusercontent.com/images/VHVyXGH5H9boyIvYPsuGhTSwgQ.png)
Lukas Jansen
Jun 27, 2023
![](https://framerusercontent.com/images/dbJU6slMvA3y3lQlFoK96mY3A.png)
How to load custom cookies in a Python request
![](https://framerusercontent.com/images/VHVyXGH5H9boyIvYPsuGhTSwgQ.png)
Lukas Jansen
Mar 22, 2023
![](https://framerusercontent.com/images/WlsdVPETCDLBi3cdoYzOAQmWSm0.png)
Scraping Best Practices and Optimization in JavaScript
![](https://framerusercontent.com/images/VHVyXGH5H9boyIvYPsuGhTSwgQ.png)
Lukas Jansen
Mar 12, 2023
![](https://framerusercontent.com/images/wVrGVWQLk15JNqfZirRSIsSuzpc.png)
Extracting Data with Cheerio
![](https://framerusercontent.com/images/H2hhcKb4fcZ6A2ndp79i9zSzRE.png)
Caitlin O'Connor
Dec 17, 2023
![](https://framerusercontent.com/images/wcnI9eldOflAOH6wBE3EbfyhB4.png)
Parallel and Distributed Web Scraping
![](https://framerusercontent.com/images/H2hhcKb4fcZ6A2ndp79i9zSzRE.png)
Caitlin O'Connor
Oct 10, 2023
![](https://framerusercontent.com/images/XD37nJjy6gf2gwlpDtU2fVT9mo.png)
Parsing HTML with BeautifulSoup
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Mar 7, 2024
![](https://framerusercontent.com/images/Q36gtY2TXgMdzNAPED7cDDZo7E.png)
Debugging and Troubleshooting Web Scraping Issues
![](https://framerusercontent.com/images/VHVyXGH5H9boyIvYPsuGhTSwgQ.png)
Lukas Jansen
Feb 25, 2024
![](https://framerusercontent.com/images/90DIKN8VmiyRJhWbjeTNtHslk.png)
Dealing with Anti-Scraping Measures in Python
![](https://framerusercontent.com/images/H2hhcKb4fcZ6A2ndp79i9zSzRE.png)
Caitlin O'Connor
Jan 26, 2024
![](https://framerusercontent.com/images/x5kPQ9sNIAu72SidpIGIg5bG7T8.png)
Take a screenshot of any website with Puppeteer and Python
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Jan 18, 2024
![](https://framerusercontent.com/images/4RIMMAfTzwX6fta1Nz76XKrVk.png)
Introduction to Web Scraping with Python
![](https://framerusercontent.com/images/VHVyXGH5H9boyIvYPsuGhTSwgQ.png)
Lukas Jansen
Dec 30, 2023
![](https://framerusercontent.com/images/bV6CJyFgKtmIFO5DjKOHpvYDKhs.png)
Regular Expressions for Web Scraping
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Dec 5, 2023
![](https://framerusercontent.com/images/caa5MB0v5SHDfDRZG2dfKiFQ.png)
Data Extraction and Parsing with Python
![](https://framerusercontent.com/images/H2hhcKb4fcZ6A2ndp79i9zSzRE.png)
Caitlin O'Connor
Dec 2, 2023
![](https://framerusercontent.com/images/0jhGa46tJ95MgVeRMFra13XnoB0.png)
Web Scraping Use Cases
![](https://framerusercontent.com/images/H2hhcKb4fcZ6A2ndp79i9zSzRE.png)
Caitlin O'Connor
Nov 30, 2023
![](https://framerusercontent.com/images/U2EvCMv7pdU7CbxR04sq2PgZ4M.png)
Parsing HTML with Cheerio
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Nov 29, 2023
![](https://framerusercontent.com/images/wGs6ZazzwNyj7uLUrDDFNnXW4.png)
LangChain ChatGPT RAG with scraped website content
![](https://framerusercontent.com/images/lYe6j3Kdio8rRPxGrnnqv5Wk5Y.jpeg)
Damien Tanner
Nov 21, 2023
![](https://framerusercontent.com/images/tEMHQrsxAI6eSB9SYX66CnZWMgM.png)
How to upload a website's content into an OpenAI GPT's knowledge
![](https://framerusercontent.com/images/lYe6j3Kdio8rRPxGrnnqv5Wk5Y.jpeg)
Damien Tanner
Nov 20, 2023
![](https://framerusercontent.com/images/NNXNlFyqHR7CtbUWmAQ6HOFtKvY.png)
Say 👋 to UseScraper Crawler
![](https://framerusercontent.com/images/lYe6j3Kdio8rRPxGrnnqv5Wk5Y.jpeg)
Damien Tanner
Nov 20, 2023
![](https://framerusercontent.com/images/37ZMua2tujOWqxKiXGcLZjD64pc.png)
Scraping Websites with Complex Navigation
![](https://framerusercontent.com/images/VHVyXGH5H9boyIvYPsuGhTSwgQ.png)
Lukas Jansen
Nov 1, 2023
![](https://framerusercontent.com/images/Y51a1SE3aLOHsf3Ghzl04Rt06M.png)
Proxies and Rotating IPs for Web Scraping with JavaScript
![](https://framerusercontent.com/images/H2hhcKb4fcZ6A2ndp79i9zSzRE.png)
Caitlin O'Connor
Oct 30, 2023
![](https://framerusercontent.com/images/Rzr7LppTRTA4f8MhoVn16Abdv8.png)
What are Headless browsers and how do they work?
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Oct 27, 2023
![](https://framerusercontent.com/images/2vze4vpA0Lxu7gzBauL8Mcxhf4.png)
Handling AJAX and JavaScript-rendered Content
![](https://framerusercontent.com/images/VHVyXGH5H9boyIvYPsuGhTSwgQ.png)
Lukas Jansen
Sep 21, 2023
![](https://framerusercontent.com/images/9ozOBEWwaAP8oudIbeFabfoN0WA.png)
Building a Web Scraping API with Node.js
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Sep 16, 2023
![](https://framerusercontent.com/images/bd3KWrUrKSuVBholBKmwAkdtrEk.png)
Navigating and Searching HTML with Cheerio
![](https://framerusercontent.com/images/H2hhcKb4fcZ6A2ndp79i9zSzRE.png)
Caitlin O'Connor
Aug 11, 2023
![](https://framerusercontent.com/images/RBUs5AixqYW03kvLIdbBLHMo4b4.png)
Scraping Websites with Client-Side Rendering
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Jul 25, 2023
![](https://framerusercontent.com/images/L7X76TFhnuPWCwFSbzV41VD55g.png)
Overview of Web Scraping Tools and Libraries
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Jul 23, 2023
![](https://framerusercontent.com/images/lKx9DIXtGUaHYybzdtwiL9ZLlyU.png)
Introduction Web Scraping with PHP 2024
![](https://framerusercontent.com/images/H2hhcKb4fcZ6A2ndp79i9zSzRE.png)
Caitlin O'Connor
Jul 4, 2023
![](https://framerusercontent.com/images/JgwEy1F7AG9H1XsBnbMVt1BNLu4.png)
Scraping Best Practices and Optimization in Python
![](https://framerusercontent.com/images/H2hhcKb4fcZ6A2ndp79i9zSzRE.png)
Caitlin O'Connor
Jun 25, 2023
![](https://framerusercontent.com/images/XVpJT2gvU87xuT8OzOWc8PBwyA.png)
Introduction Web Scraping with Java 2024
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Jun 4, 2023
![](https://framerusercontent.com/images/G67AUn00vyLKBpkvM95bf76j3I.png)
Navigating and Searching HTML with BeautifulSoup
![](https://framerusercontent.com/images/VHVyXGH5H9boyIvYPsuGhTSwgQ.png)
Lukas Jansen
May 22, 2023
![](https://framerusercontent.com/images/10HHJT31oD4aoWkZIHqgiwhzqEA.png)
Scraping Dynamic Websites with Puppeteer
![](https://framerusercontent.com/images/H2hhcKb4fcZ6A2ndp79i9zSzRE.png)
Caitlin O'Connor
May 11, 2023
![](https://framerusercontent.com/images/Fx02CqC5Tlo0Zvz6LyKGrLSbunU.png)
Scraping Websites with Infinite Scrolling
![](https://framerusercontent.com/images/H2hhcKb4fcZ6A2ndp79i9zSzRE.png)
Caitlin O'Connor
May 11, 2023
![](https://framerusercontent.com/images/f9YcGSEjkzTvHDcMqRrHs11NHvw.png)
Top tips for bypassing Cloudflare when web scraping in 2024
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Apr 30, 2023
![](https://framerusercontent.com/images/TUyAi2C4ulalciCK1ADNEZye8U.png)
The 5 best open source web scraping tools in 2024
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Apr 23, 2023
![](https://framerusercontent.com/images/fPMfISybMStHMpAJMfWxAEOD4oY.png)
Handling JavaScript with Python
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Mar 22, 2023
![](https://framerusercontent.com/images/lxytOfI7Aa5BmGrEJyriucdcNeo.png)
Rotating IP Addresses and User Agents
![](https://framerusercontent.com/images/VHVyXGH5H9boyIvYPsuGhTSwgQ.png)
Lukas Jansen
Mar 20, 2023
![](https://framerusercontent.com/images/3ADsxXky27QtTUfzCZdTmBGKfaA.png)
Extracting Data with BeautifulSoup
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Mar 20, 2023
![](https://framerusercontent.com/images/DBDbPP6BQEEdWMNl5JJptoDfow.png)
Legalities and Ethics of Web Scraping
![](https://framerusercontent.com/images/H2hhcKb4fcZ6A2ndp79i9zSzRE.png)
Caitlin O'Connor
Mar 16, 2023
![](https://framerusercontent.com/images/Eec4i6REcszuDLZc2fh5fp7OyfY.png)
Using Proxies for Web Scraping
![](https://framerusercontent.com/images/VHVyXGH5H9boyIvYPsuGhTSwgQ.png)
Lukas Jansen
Feb 28, 2023
![](https://framerusercontent.com/images/60WME0ZJ1UBo7PPwS1NnYEtmg.png)
Comparison of Web Scraping Tools and Libraries
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Jan 16, 2023
![](https://framerusercontent.com/images/yDEzV4gnvFmcyvwm88qgjlWcbk.png)
Introduction to Web Scraping with JavaScript and Node.js
![](https://framerusercontent.com/images/kmSwHbJVLIKagYRi6BPiNhvmIMc.png)
Ethan Whitaker
Jan 13, 2023
![](https://framerusercontent.com/images/Z4tZ17Ov7nVrQJ3JnPLpu4BeM.png)
What is Web Scraping?
![](https://framerusercontent.com/images/VHVyXGH5H9boyIvYPsuGhTSwgQ.png)
Lukas Jansen
Jan 12, 2023
![](https://framerusercontent.com/images/nFDLwljfQXu8xzJcU0Acj0jR9GY.png)
Scraping APIs with Python
![](https://framerusercontent.com/images/H2hhcKb4fcZ6A2ndp79i9zSzRE.png)
Caitlin O'Connor
Jan 4, 2023