Project file N°11
Catalogue Scrapers — Six Retailers, One Import Format
A set of Python scrapers for furniture and decor retailers on five different platforms — WooCommerce, Shopify, ikas (Next.js), Wix and a custom stack — each producing the same Matrixify-ready Excel file.
Stack
- Python
- requests
- BeautifulSoup
- Selenium
- openpyxl
- Matrixify
The problem
Product data for a new catalogue lived on six retailer sites with no exports. Every site was built differently, and the store needed one consistent import format.
What we built
- 01Selenium where pages render with JavaScript; JSON blobs (__NEXT_DATA__, Wix product data) parsed directly where they exist
- 02Per-variant prices, accordion sections and descriptions recovered from wherever each platform keeps them
- 03Shared rules across every scraper: price threshold, sold-out skip, de-duplication across categories, category mapping
- 04Unit and currency conversion (cm to inch, kg totals, TRY to USD) before writing the import file
Outcome
- —Six timestamped Excel files in the same column layout, imported into Shopify with Matrixify
Next project →