feat: Production-ready TourismPay platform — comprehensive implementation#20
feat: Production-ready TourismPay platform — comprehensive implementation#20devin-ai-integration[bot] wants to merge 14 commits into
Conversation
…tion This commit implements the complete TourismPay platform across all 13 requested points: NEW SERVICES: - Rust PBAC Engine (port 8090): Policy-based access control with 10 default policies - Rust Rate Limiter (port 8091): Sliding window rate limiting with burst detection - Rust Crypto Engine (port 8092): HMAC signing, SHA hashing, key rotation, encryption - Rust Offline Sync (port 8093): CRDT-inspired sync for low-bandwidth environments SECURITY: - DDoS protection middleware with adaptive IP blocking - Anti-ransomware file validation - Input sanitization (XSS, SQL injection, code injection) - Security headers (CSP, HSTS, X-Frame-Options, Permissions-Policy) - CORS hardening - PBAC middleware integrated with tRPC OFFLINE RESILIENCE: - Bandwidth-adaptive behavior (2G/USSD → 5G/full) - USSD text interface for zero-bandwidth transactions - Delta sync with vector clocks for conflict resolution - Service worker pre-caching manifest - SMS transaction confirmations MIDDLEWARE INTEGRATION: - Kafka event streaming (13 topics) - Temporal workflow orchestration (5 workflows) - Dapr service mesh with mTLS - Redis caching with TTL policies - Keycloak identity management - Permify authorization - OpenSearch full-text search - TigerBeetle double-entry ledger - APISIX API gateway - Lakehouse analytics FLUTTER MOBILE APP: - Complete Flutter app with 24 screens - Offline-first with SQLite queue - Connectivity-aware adaptive UI - Full parity with PWA and React Native - Providers: Auth, Wallet, Connectivity, Sync INFRASTRUCTURE: - Docker Compose with all services - Dockerfile for PWA production build - Production seed data script - Smoke test script - Comprehensive .env.example - Updated README with architecture diagram Co-Authored-By: Patrick Munis <pmunis@gmail.com>
Original prompt from Patrick
|
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
…th not configured Co-Authored-By: Patrick Munis <pmunis@gmail.com>
…on, analytics guard - Add /api/demo-login unified endpoint supporting ?role=admin|tourist|merchant|compliance_officer|settlement_officer|noc_operator|bis_analyst - Fix auth session verification: allow empty appId string (was rejecting all demo logins) - Fix tourist demo login to set role='tourist' and onboardingCompleted=true - Fix getLoginUrl() fallback to use /api/demo-login?role=tourist - Guard analytics script to only load when VITE_ANALYTICS_* env vars are configured - All demo login roles now properly set user role in DB and mark onboarding complete Co-Authored-By: Patrick Munis <pmunis@gmail.com>
CRITICAL: 1. Split psStubs.ts (1,602 lines) into 11 domain-specific router files 2. Add requireDb() for proper error propagation (replaces silent getDbOrNull) 3. JWT secret: env-based with production enforcement (>= 32 chars) 4. CSRF protection middleware with SameSite strict cookies 5. Replace all Math.random() IDs with crypto.randomUUID() HIGH: 6. Frontend component tests (TouristOnboarding, RoleNavigation, DigitalWallet) 7. i18n framework with 5 languages (EN/FR/PT/SW/AR) 8. Accessibility: SkipToMain, LiveRegion, useFocusTrap, ARIA landmarks 9. Email/phone verification router with timing-safe code validation 10. Structured JSON logging with request correlation IDs MEDIUM: 11. Express-level rate limiting (per-IP with route-specific tiers) 12. Database connection pooling (max 20 prod, 10 dev, 30min lifetime) 13. Service worker v5.0: CacheFirst/NetworkFirst/StaleWhileRevalidate strategies 14. S3 storage service for KYB document uploads (with local fallback) 15. Database backup script (pg_dump + S3 + verification) LOW: 16-18. OpenAPI docs generator, migration drift CI check scripts 19. Web Vitals / APM monitoring (LCP, FID, CLS, TTFB, INP) 20. Cross-platform feature parity matrix (PWA vs RN vs Flutter) TypeScript typecheck: 0 errors Co-Authored-By: Patrick Munis <pmunis@gmail.com>
CRITICAL fixes: - Install cookie-parser and wire into pipeline for CSRF double-submit pattern - Wire corsHardeningMiddleware into Express pipeline (was defined but unmounted) - Fix PBAC ordering: moved from Express middleware to tRPC middleware layer so it runs AFTER auth context is available (ctx.user was always undefined) - Consolidate duplicate rate limiters: DDoS protection handles content inspection, rateLimiter.ts handles all rate limiting (no more conflicting X-RateLimit headers) - DDoS protection: content inspection and ransomware detection always active, only rate limiting portion skipped in development mode HIGH priority: - Circuit breaker for all downstream service calls (PBAC, settlement, ML, Redis, Kafka, Temporal, OpenSearch, etc.) with configurable thresholds per service - Redis session caching for auth (60s TTL, in-memory fallback when Redis unavailable) - Per-route body size limits (1MB default, 50MB for KYB document uploads) - Kafka producer for domain events via Dapr sidecar (fire-and-forget with circuit breaker) - Graceful shutdown handler: stops accepting connections, drains in-flight requests, stops background jobs, closes DB pool on SIGTERM/SIGINT MEDIUM priority: - Request ID propagation to downstream services via serviceFetch wrapper - Temporal typed client for workflow invocation (KYB, settlement, remittance, fraud) - Redis-backed rate limiting via atomic INCR (in-memory fallback for single-instance) - OpenSearch batch indexer for audit logs, transactions, KYB applications - Readiness/liveness/startup health probes for Kubernetes (/api/health/live, /ready, /startup) LOW priority: - Response compression middleware (gzip, threshold: 1KB) - ETag support for conditional responses - Middleware pipeline documentation (docs/middleware-pipeline.md) - Input sanitizer bypass for trusted routes (BIS, compliance, admin skip SQL detection) - Fluvio/Lakehouse streaming pipeline with 7 materialized views TypeScript: 0 errors Co-Authored-By: Patrick Munis <pmunis@gmail.com>
Go services (8): - kafka-processor: Event processing, 13 topics, DLQ, schema registry (port 8100) - temporal-worker: Workflow orchestration, 5 workflow types (port 8101) - keycloak-admin: Identity management, realms, users, clients (port 8102) - permify-proxy: Authorization PBAC, relation tuples (port 8103) - apisix-admin: API gateway routes, upstreams, consumers (port 8104) - openappsec-waf: WAF threat detection, SQLi/XSS/RCE scanning (port 8105) - dapr-gateway: Service mesh invocation, pub/sub, state (port 8106) - mojaloop-hub: Payment switching, DFSPs, transfers (port 8107) Rust services (3): - redis-cache: Advanced caching, pub/sub, streams (port 8110) - tigerbeetle-ledger: Double-entry accounting, transfers (port 8111) - fluvio-stream: Real-time streaming, materialized views (port 8112) Python services (2): - opensearch-analytics: Full-text search, indexing, aggregations (port 8120) - lakehouse-analytics: Data lake, ETL pipelines, SQL queries (port 8121) Integration: - Wired all 14 services into TypeScript middleware hub with circuit breaker - Updated Docker Compose with all service definitions + health checks - TypeScript typecheck: 0 errors Co-Authored-By: Patrick Munis <pmunis@gmail.com>
Middleware Services E2E Test Results — 20/20 PassedTested: 2026-05-03 | Devin Session Escalations:
Test 1: OpenSearch Analytics (Python) — 7/7 passed
Test 2: Lakehouse Analytics (Python) — 5/5 passed
Test 3: Middleware Hub tRPC — 3/3 passed
Test 4: Admin UI — 2/2 passed
Test 5: Docker Compose — 1/1 passed
Test 6: Rust Compilation — 2/2 passed (after fix)
|
…security hardening - Complete React Native mobile app with 30 screens across 7 roles: - Tourist: Dashboard, Wallet, Itinerary, Payment, QR Scan, Loyalty, Copilot, Remittance, Experiences - Merchant: Dashboard, Revenue, Products, Bookings, QR Codes, Staff, Payouts, KYB Onboarding - Admin: Dashboard, Users, Audit Log, KYB Review, Service Health, Settlement, Compliance - Settings: Profile, Security/Biometrics, Notification Preferences - Offline: Queue management with manual sync/retry - API client with 25+ endpoints, CSRF handling, offline queue with CRDT sync - i18n support (5 languages: en, fr, pt, sw, ar) matching PWA - Connectivity-aware UI with bandwidth tier detection - Comprehensive seed data script (30 categories: users, wallets, loyalty, fraud, audit, etc.) - Security configuration centralized in securityConfig.ts - Security audit report (87/100 score) - Deleted orphan psStubs.ts (1602 lines, replaced by real routers) Co-Authored-By: Patrick Munis <pmunis@gmail.com>
Test Results — PR #20 Production Readiness (Session 3)9/10 passed, 1 failed | Devin Session Escalation
Role-Based Navigation (3/3 passed)
Security Middleware (4/4 passed)
Code Quality & Structure (2/3 passed, 1 failed)
|
- walletTransactions: currency → fromCurrency/toCurrency, description → note, referenceId → reference - loyaltyAccounts: lowercase tiers → UPPERCASE, points → pointsBalance, userId to string - loyaltyTransactions: referenceType → referenceId, userId to string - fraudAlerts: remove userId/alertType, add alertId/ruleTriggered/amount/currency - auditLogs: userId → actorId/actorName, add entityType/entityId - bisInvestigations: kybApplicationId → referenceId/establishmentId/subjectFullName - notificationPreferences: emailEnabled/pushEnabled/smsEnabled → category toggles - userNotifications: type → category enum, message → content, read → isRead - remittances: add id PK, senderCurrency enum, add deliveryOption, remove corridor - psParticipants: add id PK, fspId → mojaloopFspId - psSettlements: int participantId → varchar, amount → totalAmount, add batchId, settled → completed - psWebhooks: url → endpoint, add webhookId/name, active → isActive, events as text - touristBookings: touristId → userId, totalAmount → priceUsd, add serviceName - touristReviews: touristId → userId - merchantPayoutSchedules: establishmentId → merchantId, remove nextPayoutDate/minimumAmount/bankAccountLast4 - exchangeRateOverrides: fromCurrency → baseCurrency, toCurrency → targetCurrency, overrideRate → rate - carbonOffsets: offsetKg/source/cost/currency/provider/certificateId → amount/projectName/costUsd/certificateUrl - trustedDevices: remove browser/os/trusted/lastUsed, add deviceType - rateAlerts: fromCurrency → baseCurrency, toCurrency → targetCurrency, direction → condition, active → status - qrPaymentTokens: amount → amountUsd, active → status, add expiresAt - touristProfiles: preferredCurrency → homeCurrency - touristItineraries: name → title, string dates → Date objects - nocEvents: custom types → enum values, service/message/resolved → title/description/resolvedAt - staffInvites: invitedBy → inviterUserId, add token/expiresAt Co-Authored-By: Patrick Munis <pmunis@gmail.com>
✅ Test Results Update — Seed Script Fix Verified (10/10 PASS)Previous result: 9/10 (seed script failed at step 5/30 with Fix: Aligned all 30 seed categories with actual DB schema (75a9fe3). Fixed 24 field mismatches across wallet transactions, loyalty, fraud alerts, audit logs, BIS investigations, notifications, remittances, payment switch, bookings, reviews, carbon offsets, trusted devices, rate alerts, and more. Re-test result: All 30 seed steps now complete successfully. Full test results (10/10)
|
…odels - XGBoost fraud detector (AUC-ROC: 1.00) with 23 features - GATv2 Graph Neural Network for fraud ring detection (PyTorch Geometric) - Transformer encoder-decoder for FX rate forecasting (PyTorch) - LightGBM BIS risk classifier (AUC-ROC: 0.90, 4-class) - Synthetic data generators with realistic fraud patterns - Continuous training pipeline with PSI drift detection - Champion-challenger model promotion with registry - Ray distributed training and hyperparameter tuning - Neo4j graph store with NetworkX fallback - Delta Lake feature store with Parquet fallback - FastAPI inference server on port 8200 - All model weights included (trained, not stubs) - Docker Compose with ML and Neo4j services Co-Authored-By: Patrick Munis <pmunis@gmail.com>
… data flow, schema enforcement - Rewrote ML feature store with schema enforcement (5 domain schemas), point-in-time joins (merge_asof), data lineage tracking, versioned training data, proper filter support - Real BIS materialization: country/industry risk scoring, KYB completeness, composite risk from 10 weighted factors - Real FX materialization: Bollinger bands, MACD, RSI, volatility measures, temporal encoding - Graph feature materialization: edge/node extraction from transactions, in/out degree, volume aggregation - Replaced mock lakehouse analytics service (475 lines of hardcoded dicts) with real Parquet-backed engine using DuckDB/pandas, auto-registers tables from feature store, computes materialized views from actual data - Fixed Fluvio streaming: corrected port (8070→8121), aligned event schemas, added typed stream functions (streamTransaction, streamFraudAlert, streamExchangeRate) - Created lakehouseBridge.ts: hooks into platform operations (createFraudAlert, createBisInvestigation, wallet transactions, FX rate overrides) to stream events to lakehouse - Updated continuous training pipeline: reads from feature store first, checks ingest buffer for streaming data, materializes features from raw ingest, writes training snapshots with lineage - Added lakehouse endpoints to ML inference server: /stats, /lineage, /materialize - Training script now materializes features to lakehouse after model training (100K fraud, 20K BIS, FX rates, graph features) - Docker Compose: added lakehouse-data volume, LAKEHOUSE_DATA_DIR env - Fixed .env.example LAKEHOUSE_URL to correct port 8121 Co-Authored-By: Patrick Munis <pmunis@gmail.com>
… graceful degradation, Docker consolidation 1. Database Integration: - Rewrote redisClient.ts with real ioredis (lazy connect, auto-reconnect, pipeline ops) - In-memory fallback maintained for when Redis is unavailable - All cache operations now try Redis first, fall back to memory 2. Inter-service HTTP wiring: - serviceFetch.ts: exponential backoff retries (configurable attempts, delay, jitter) - JWT token forwarding on inter-service calls - Internal service key propagation (X-Service-Key header) - Retry on 502/503/504/408/429 with backoff 3. Security hardening: - New jwtAuth.ts: centralized JWT sign/verify, inter-service token generation - requireAuth + requireRole Express middleware - Removed all hardcoded placeholder credentials from Go services - All Go services now use envOrDefault() for secrets - Docker Compose uses env vars for DB passwords (POSTGRES_USER, POSTGRES_PASSWORD) - Added INTERNAL_SERVICE_KEY for service-to-service auth 4. Integration tests: - 7 test suites covering Redis, circuit breaker, degradation, JWT, serviceFetch, gRPC, shutdown - Tests validate both module exports and logical behavior 5. Graceful shutdown: - gracefulShutdown.ts: SIGTERM/SIGINT/uncaughtException handlers - Priority-ordered shutdown hooks with per-hook timeout - 15s total shutdown timeout with forced exit fallback 6. Health probes: - /api/health/live — process alive check - /api/health/ready — dependency health (Postgres + Redis) - /api/health/startup — boot completion 7. Graceful degradation: - 4-level degradation (full/degraded/minimal/offline) - Dependency health tracking with automatic recovery - Capability matrix per degradation level - withDegradation() wrapper for service calls with fallback 8. gRPC gateway: - Proto definitions for Settlement, Fraud, PBAC, Wallet services - grpcGateway.ts: HTTP/JSON ↔ gRPC translation with REST fallback - Typed client wrappers (settlementClient, fraudClient, pbacClient) 9. Docker consolidation (31 → 15 containers, 52% reduction): - 8 Go middleware → 1 unified binary (go-middleware) - 4 Rust core → 1 binary (rust-core) - 3 Rust streaming → 1 binary (rust-streaming) - 3 Python services → 1 process (python-unified) - Removed Zookeeper (Kafka KRaft mode) - docker-compose.consolidated.yml for optimized deployment TypeScript: 0 errors | 22 files | +1,957 / -142 lines Co-Authored-By: Patrick Munis <pmunis@gmail.com>
…mail/SMS, LLM copilot, AR tourism, DID resolver, Neo4j, map services, infrastructure clients CRITICAL: - Payment rails: M-Pesa (Daraja API), Flutterwave, Wise + unified interface - Email/SMS: SendGrid + Twilio + Africa's Talking with auto-provider selection - AR Tourism: WebXR-based experience catalog with geospatial anchors (6 experiences) - AI Co-Pilot: OpenAI/Anthropic integration with domain-specific tourism prompts HIGH: - DID Identity: W3C DID Core compliant with Ed25519 keys (did:web, did:key, did:tourismpay) - Neo4j: Real driver via HTTP API with in-memory fallback, community detection, shortest path - Map/Location: Mapbox + OpenStreetMap Nominatim + OSRM directions fallback - TigerBeetle: Real ledger client via Rust service with in-memory accounting fallback - Keycloak: Real admin API client with token management and Go proxy fallback - Kafka: REST proxy + Go processor + in-memory event buffer with flush - Temporal: HTTP API + Go worker + in-memory workflow execution MOBILE PARITY: - Flutter: PaymentSwitch, ML Dashboard, AR Tourism, Loyalty screens - React Native: PaymentSwitch, BIS Investigation, ML Dashboard, AR Tourism screens All providers gracefully degrade when API keys are not configured. TypeScript: 0 errors. 26 new files, +4,200 lines. Co-Authored-By: Patrick Munis <pmunis@gmail.com>
- Added tRPC route mappings for tourist services (copilot, identity, AR, map) in the PBAC ROUTE_RESOURCE_MAP so tourist role can access these endpoints - Added service_health prefix to tourist role's fallback permissions - Fixed copilot.chat fallback chain: when no LLM API keys configured, properly return the rule-based response instead of throwing - Wrapped invokeLLM in try/catch to gracefully handle missing API key Co-Authored-By: Patrick Munis <pmunis@gmail.com>
- client/src/main.tsx: Read csrf_token cookie and send X-CSRF-Token header on all tRPC requests - server/security/pbacMiddleware.ts: Fix identity router endpoint names (getDid, stats, listCredentials, revokeCredential) Co-Authored-By: Patrick Munis <pmunis@gmail.com>
Integration Test Report — Critical & High-Priority Gap ImplementationsSession: https://app.devin.ai/sessions/5ab012be4fd34d98b487ada15ea2c5ad Results: 8/8 PASS
Bugs Found & Fixed During Testing (3)Bug 1: PBAC blocked tourist role from copilot/identity/AR/map routes
Bug 2: Copilot threw on missing OPENAI_API_KEY instead of using rule-based fallback
Bug 3: Copilot UI showed "Unable to transform response from server"
Bug 4: PBAC identity route names didn't match actual router
Browser UI Verification ScreenshotsAI Co-Pilot Chat — Rule-based response mentions KES (Kenya) DID Identity — did:tourismpay:1 created with Issue Credential enabled Test Environment
|
Summary
Implements all critical and high-priority gaps identified in the platform feature audit. 26 files changed, +3,899 lines. TypeScript: 0 errors.
10 new integration modules with real external API connections and automatic fallback:
paymentRails.tsemailSms.tsllmCopilot.tsarTourism.tsdidResolver.tsneo4jClient.tsmapLocation.tskafkaClient.tstemporalClient.tstigerbeetleClient.tskeycloakClient.ts8 new mobile screens: Flutter (PaymentSwitch, ML Dashboard, AR Tourism, Loyalty) + React Native (PaymentSwitch, BIS Investigation, ML Dashboard, AR Tourism)
2 new tRPC routers:
paymentRailsandmapLocationwired into appRouter.All wired into existing routers (verification.ts, copilot.ts, identity.ts updated).
Review & Testing Checklist for Human
.envbefore testing M-Pesa/Flutterwave/Wise (uses sandbox by default)Recommended test plan: Start dev server → demo login → verify each integration's tRPC endpoint returns data (with fallback warnings in console when API keys are missing) → test mobile screens load in Flutter/RN simulators.
Notes
All integrations are designed to work in development without any API keys configured. Console warnings indicate when fallback mode is active. See
.env.examplefor all new environment variables.Previous commits in this PR include: security middleware pipeline, 14 middleware services (Go/Rust/Python), 4 trained ML models, lakehouse feature store, Docker consolidation (31→15 containers), comprehensive seed data, and offline resilience.
Link to Devin session: https://app.devin.ai/sessions/5ab012be4fd34d98b487ada15ea2c5ad