Member-only story
How to Scrape Newegg.com for laptop price and spec via Python Scrapy and import the data into SQLite ?
Published in
3 min readDec 2, 2020
Purpose
Search and grab the laptop information in the newegg website via Scrapy and store the data into the SQLite so that we can easily play around with the data.
Tool
- Python package: scrapy/ sqlite3/json
- SQLite
Target Website
Steps
- Let’s set up the Scrapy environment first.
- You can refer to my another post to set up the environment. LINK
2. Go through the structure of the website to locate where the target elements are.
- When you inspect the website, you can pick price of an example and look it up across the source codes. You will find the information located in the tag of <class=”item-cell”>.
