Skip to main content

Edge Caching & Warm Pools

Relay’s gateway supports cache profiles and warm pools to reduce cold-start latency.

Cache Profiles​

  • Configured per tenant/function via console or API.
  • Fields: cache_strategy (deterministic, stale-while-revalidate, none), ttl_seconds, personalization_keys, warm_pool_enabled.
  • Stored in function_cache_profile; changes propagate via pub/sub invalidation.

Console workflow​

  1. Navigate to Executions → Cache Profiles.
  2. Create profile selecting strategy + TTL.
  3. Optional: add personalization keys (e.g., authorization, tenant) to avoid caching personalized responses.
  4. Save to activate; warm pool scheduler pre-heats functions when enabled.

CLI​

relay cache-profile create --tenant tenant-dev \
  --function rufid:abcd1234 \
  --strategy stale-while-revalidate \
  --ttl 300 --personalization authorization

Warm Pools​

  • Scheduler pre-invokes functions after deploy and on interval (WARM_POOL_REFRESH_MINUTES).
  • Supports Cloudflare Workers and Railway regions.
  • Metrics: relay.cache.warm_pool_active, relay.cache.prefetch_duration.

Observability​

  • Datadog dashboard (“Relay / Edge Cache Baseline”) charts hit rate, warm pool success, invalidation events.
  • Logs contain cache.hit, cache.miss, cache.bypass entries with tenant/function context.

Benchmarking​

  • Use scripts/perf/api_gateway_benchmark.sh to compare baseline vs cached performance.
  • Reports stored under reports/perf/edge_cache_*. Target ≥25% p95 improvement.

Runbooks​

ScenarioAction
Hit rate dropsCheck personalization keys; ensure deterministic policy, adjust TTL.
Warm pool failuresInspect scheduler logs, verify credentials, ensure compute plan supports concurrency.
Stale responsesValidate invalidation hooks firing on deploy; run manual invalidate via console.

Security Considerations​

  • Never cache responses containing tenant-specific secrets or PII unless personalization keys isolate them.
  • Audit logs record profile changes with user info and timestamps.
  • Plan: dev_process/planning/FN-059_EDGE_CACHE_PLAN.md
  • Benchmark script: scripts/perf/api_gateway_benchmark.sh
  • Console UI components: frontend/src/components/cache/
  • Tests (planned): tests/integration/api_gateway/test_cache_profiles.py

Contact Platform Infrastructure for assistance (#relay-performance).