Real-Time E-Commerce Analytics Platform
Internal tool for multi-store metrics dashboard
An internal analytics dashboard aggregating metrics from multiple Shopify stores. Built to validate cursor-based pagination, caching strategies, and defense-in-depth authentication patterns before broader rollout.
The Problem
Managing multiple e-commerce stores meant constantly switching between admin panels to understand business performance. With high-volume stores processing over a thousand orders daily, there was no unified view of metrics across the portfolio.
- Manual logins to multiple Shopify admin panels to check metrics
- No aggregated view across all stores for leadership
- High-volume stores hitting API limits with naive polling
- Office TVs couldn't display real-time dashboards securely
Constraints
- Must handle enterprise-scale order volumes without hitting rate limits
- Display on shared office screens requires URL-based authentication
- Near real-time data (within 90 seconds) for business decisions
- Support for multiple concurrent viewers without server degradation
The Approach
Built a Next.js 14 application with defense-in-depth authentication, cursor-based GraphQL pagination for high-volume data, and intelligent caching to reduce API load. Tested patterns that could scale to more stores.
Why GraphQL cursor pagination over offset pagination?
Shopify's REST API caps at 250 orders per page with no filtering. GraphQL with cursor pagination lets us fetch only today's orders efficiently, handling 1000+ orders/day stores by paginating through multiple requests.
Why three layers of authentication?
TV displays need URL-based auth (query param), but enterprise security requires IAM. Defense-in-depth: Cloud IAM for primary auth, middleware for session management, API route validation for defense against bypasses.
Why 90-second cache instead of real-time WebSocket?
Real-time adds complexity and Shopify doesn't push webhooks for all metrics. 90-second server cache with client-side jitter (0-10s random delay) prevents thundering herd while achieving ~75% cache hit rate.
Why in-memory cache over Redis?
Single Cloud Run instance sufficient for current load. Documented migration path to Redis (Upstash) when scaling needed. In-memory simpler to deploy and debug initially.
Key Tradeoffs
In-memory caching limits horizontal scaling
Current load (50+ concurrent users) handled by single instance. Migration to Redis documented and ready when needed. Avoided premature infrastructure complexity.
Conversion rate is estimated (1% industry average) not actual
Shopify Analytics API requires special permissions and complex setup. For dashboard purposes, estimated conversion provides directional value. Real data available as future enhancement.
Fixed 3x3 grid layout constrains store count
Optimized for TV display use case. Adding/removing stores requires code change. Acceptable because store portfolio is stable.
Implementation Highlights
Dynamic pagination based on store volume
Each store has configured daily order estimates. System calculates pages needed (ceil(orders/250)) and fetches in parallel. High-volume stores automatically get more API calls.
Client jitter for cache efficiency
Each client adds 0-10 second random delay to refresh interval. Spreads load across cache window, preventing all 50+ clients from hitting server simultaneously at interval boundary.
Exponential backoff for rate limits
Shopify 429 responses trigger automatic retry with 1s, 2s, 4s delays. Failed stores don't block others, dashboard shows partial data with clear error indicators.
Seven-color theme system
CSS variable-based theming with 7 schemes (cloud-dancer, dusty-jewel, wellness-mint, thermal-glow, midnight, serene-pastels, earthy-neutrals). Allows brand-appropriate display customization.
Hourly trend visualization
Recharts sparklines show 20-hour revenue trends per store. Aggregates orders by hour, enabling quick identification of unusual patterns or outages.
Outcomes
What I Learned
- Cursor pagination handles high-volume stores gracefully where offset pagination would fail.
- In-memory caching works for POC but would need Redis for production multi-instance deployment.
- Defense-in-depth auth (IAM + middleware + API validation) adds complexity but catches edge cases.