Fouani Store Clone
A full-featured e-commerce clone of Fouani Store with automated data scraping, fast product synchronization, and comprehensive admin controls for managing scraping, users, products, carousels, orders, and payments.
Admin Scraping Interface
Scraping Management
Real-time scraping progress monitor with controls for initiation and updates.
Scraping Management
Real-time scraping progress monitor with controls for initiation and updates.
Project Overview
Developed a complete e-commerce clone of Fouani Store using Next.js and MongoDB, featuring automated web scraping for product synchronization. The system achieves full product scraping in under 10 minutes and includes a powerful admin dashboard for managing all aspects of the platform without terminal commands.
Automated Scraping
Fast, efficient product data scraping with seamless database integration.
Admin Management
Intuitive controls for users, products, orders, and payments.
Performance Optimization
Optimized for speed with real-time updates and efficient data handling.
Key Features
Automated Product Scraping
One-click scraping system that fetches and synchronizes all products from the source in under 10 minutes, including images, descriptions, prices, and variants.
Admin Scraping Controls
User-friendly admin interface to initiate, monitor, and manage scraping processes without any command-line interaction.
Scraped Product Management
Comprehensive tools for editing, categorizing, and optimizing scraped products, with bulk update capabilities.
Carousel Management
Admin controls for creating and managing homepage carousels using scraped product images and custom content.
User Management
Role-based access control for managing user accounts, permissions, and activity logs.
Order Processing
Complete order management system with status tracking, shipping integration, and customer notifications.
Payment Handling
Secure payment processing with multiple gateways and transaction verification.
Performance Monitoring
Built-in analytics for scraping speed, system performance, and e-commerce metrics.
Tech Stack
Frontend
- Next.js 14
- React
- TypeScript
- Tailwind CSS
- Shadcn UI
- Zustand
- React Query
Backend
- Node.js
- Express
- MongoDB
- Mongoose
- JWT Authentication
Scraping & Storage
- Puppeteer/Cheerio
- Cloudinary
- MongoDB Atlas
- Cron Jobs
Tools & Utilities
- Framer Motion
- React Hook Form
- Zod Validation
- Axios
- Date-fns
Technical Challenges & Solutions
Fast Scraping Implementation
Challenge:
Needed to scrape thousands of products efficiently without getting blocked, completing in under 10 minutes.
Solution:
Utilized headless browser automation with Puppeteer, implemented parallel processing, and added smart rate limiting. Optimized data extraction with selectors and direct API calls where possible.
Outcome:
Achieved consistent scraping times under 10 minutes for full catalog sync, with 99% data accuracy.
Admin Scraping Management
Challenge:
Required intuitive admin controls for scraping without technical knowledge or terminal access.
Solution:
Developed a web-based scraping dashboard with progress tracking, error handling, and one-click initiation. Integrated WebSockets for real-time updates during scraping.
Outcome:
Enabled non-technical admins to manage scraping processes effortlessly, reducing operational overhead by 80%.
Data Synchronization & Integrity
Challenge:
Maintaining data consistency between scraped content, database, and frontend while handling updates.
Solution:
Implemented delta syncing for updates only, with validation schemas and duplicate detection. Used MongoDB transactions for atomic updates and Cloudinary for media handling.
Outcome:
Ensured 100% data integrity with minimal downtime during syncs, handling up to 10,000 products efficiently.