ES ← Back to Portfolio

Toy Sentinel

Event-Driven Retail Intelligence Platform

A distributed monitoring platform that continuously tracks collectible products across multiple e-commerce retailers, detects inventory and pricing changes, and automatically delivers real-time alerts to subscribers via Telegram.

Role Founder & Lead Developer
Status Active — Production
Subscribers 200+
Node.js MongoDB Redis Telegram API Web Scraping Event-Driven Architecture Distributed Systems

The Problem

Collectible figures occupy a uniquely difficult retail niche. Popular items sell out within hours of listing. Price drops are temporary and unannounced. Restocks happen without notice across multiple retailers simultaneously.

Collectors who want to stay informed face a tedious manual process:

  • Checking multiple retailer websites repeatedly throughout the day.
  • Manually comparing prices across Amazon, MercadoLibre, and specialty stores.
  • Missing restocks or discounts because no notification system exists.
  • Discovering sold-out products minutes after they listed.

The problem is not availability of information — it is the speed and effort required to act on it.

The Solution

Toy Sentinel eliminates manual monitoring entirely. The platform continuously scrapes product listings from multiple retail sources, detects meaningful changes — new listings, price drops, restocks — and publishes structured notifications to Telegram channels where subscribers receive them in real time.

Collectors no longer check websites. The platform watches on their behalf and alerts them the moment something relevant happens — before inventory runs out.

The result is a system that runs continuously in production, serves 200+ subscribers daily, and demonstrates what separates a working distributed system from a weekend demo.

System Architecture

The platform is built as a pipeline of decoupled components. Scraping, change detection, and notification delivery are intentionally separated — each can fail, retry, or scale independently.

Retail Sites
Amazon · MercadoLibre · Specialty Stores
Scrapers
Product Parser
Normalizes to common model
MongoDB Store
Product state & price history
Change Engine
Diffs current vs. previous state
Event Generation
New listing · Price drop · Restock
Redis Queue
Decouples scraping from delivery
Telegram Consumer
Rate-limited async delivery
Telegram Channels → Users

Engineering Challenges

Retail Websites Change Frequently

Each retailer exposes product data differently — different HTML structures, dynamic loading patterns, anti-scraping measures, and inconsistent field naming. Layouts change without notice and silently break extraction logic.

Solution

Built retailer-specific extraction strategies encapsulated behind a common product interface. Each scraper normalizes output to the same internal model, so downstream components are fully isolated from retailer-specific changes.

Duplicate and Noisy Notifications

Without safeguards, minor price fluctuations or repeated scrapes of the same listing trigger multiple alerts. Subscribers receiving duplicate notifications quickly lose trust in the platform and unsubscribe.

Solution

Implemented event deduplication logic in the Change Engine. Events are only generated when a meaningful threshold is crossed — not on every scrape cycle — and are fingerprinted to prevent re-publication of identical events.

Telegram Rate Limits Under Load

When multiple products change simultaneously, naively pushing all notifications at once hits Telegram's API rate limits, causing message delivery failures and requiring complex retry logic across the system.

Solution

Introduced a Redis-backed message queue between event generation and delivery. The Telegram consumer processes the queue asynchronously at a controlled rate, fully decoupling throughput from scraping cadence.

Key Features

🔍
Multi-Source Monitoring
Independent scraper modules for Amazon, MercadoLibre, and specialty retailers — each normalized to a shared product model.
Event-Driven Detection
Price changes generate events rather than immediate notifications, enabling batching, filtering, and prioritization before delivery.
📉
Price History Tracking
Products maintain full pricing history, enabling discount detection, trend analysis, and accurate savings percentage calculations.
📦
Restock Alerts
Detects when previously out-of-stock products become available again and immediately queues a notification.
🚦
Queue-Based Delivery
Redis queue decouples scraping from message delivery, preventing rate-limit failures and enabling retry logic.
📲
Telegram Distribution
Structured alerts delivered to Telegram channels with product image, prices, savings, retailer info, and direct purchase link.

Notification Format

Each alert is structured to give subscribers everything they need to make a purchase decision instantly — without opening a browser.

Toy Sentinel Telegram channel showing live price drop notifications

Notifications include product image, current and previous price, savings percentage, retailer, stock status, and a direct purchase link — everything needed to act immediately.

Platform Metrics

Toy Sentinel is a live production system, not a demo. These metrics reflect real operational scale.

200+
Active subscribers across channels
24/7
Continuous automated monitoring
Multi
Retailers monitored simultaneously
Real-time
Alert delivery on event detection

Unlike portfolio demos that simulate production conditions, Toy Sentinel runs continuously, serves real users daily, and has been maintained through retailer structure changes, Telegram API updates, and infrastructure failures.

Technologies & Architecture

Scraping & Extraction
Node.js Custom Scrapers HTML Parsing Retailer-Specific Adapters
Event Processing
Change Detection Engine Event Deduplication Redis Queue Async Consumers
Persistence
MongoDB Price History Store Product State Model
Distribution
Telegram Bot API Rate Limiting Channel Management
Operations
Long-Running Services Error Recovery Monitoring

Lessons Learned

1
Decoupling is not an optimization — it's a reliability requirement. Connecting scraping directly to notification delivery means a Telegram API failure stops the entire pipeline. The Redis queue was not added for scale; it was added because the system couldn't survive without it.
2
Websites are not stable APIs. Every retailer integration has broken at least once due to an unannounced layout change. Treating scrapers as fragile and writing them to fail loudly — rather than silently producing corrupt data — is the only sustainable approach.
3
Notification quality determines subscriber retention. A duplicate notification, a false positive, or a price that was already corrected by the time the user clicks erodes trust immediately. The Change Engine's deduplication logic exists to protect the subscriber experience, not just system efficiency.
4
Production is a different problem than development. Running a system 24/7 surfaced failure modes that no test suite covers: memory leaks in long-running scrapers, transient network failures mid-page, Telegram session expiry, and MongoDB connection pool exhaustion under load.
5
Event-driven architecture scales user value, not just throughput. The biggest win from the event model was not performance — it was the ability to add new event types (restocks, new listings) without modifying the delivery or scraping layers. The architecture made the product easier to extend.
6
Real users are the best observability tool. With 200+ subscribers receiving daily alerts, users report broken notifications faster than any monitoring dashboard. Subscriber feedback became an early warning system for retailer changes and pipeline failures.