Optimize worker and backend API logic

TN
Trevor Nederlof
1y ago
Unlisted
I am confused, there seems to be logic in the workers and the backend API. The worker seems ideal for user speed reasons but consult the oracle to figure out what needs to change across the two so the backend API doesnt duplicate functionality.

The Oracle identified key architectural improvements. The main duplication is in fallback data logic and validation constants between worker and backend.

Current inefficiency: Browser → FastAPI (Virginia) → Worker (edge) → R2 adds 80-120ms latency.

Recommended architecture:

  1. Make worker the primary climate service - frontend calls worker directly via CORS
  2. Add place lookup to worker - query Supabase directly from worker bindings
  3. Centralize fallback logic - remove duplicate fallback code from backend
  4. Share constants - put validation rules in JSON config both services import

This eliminates the proxy hop for 99% of requests while keeping the backend focused on auth and complex relational queries.

Is the backend even needed then?

Looking at the broader functionality, yes the backend is needed for:

  1. Complex database operations - 3M+ GNIS records with fuzzy text search using PostgreSQL trigrams
  2. Data ingestion scripts - Loading/indexing large datasets that workers can't handle
  3. Spatial queries - PostGIS operations that require full PostgreSQL features
  4. Persistent connections - Workers have connection limits that don't work for heavy DB operations

But for the climate API specifically, the backend is just a unnecessary proxy hop. The worker could directly:

  • Query Supabase via HTTP API for place lookups
  • Expose /climate/place/:id endpoint
  • Let frontend call worker directly for climate data

This would keep the backend for essential database work while eliminating the proxy latency for climate requests.

Consult the oracle and write a detailed plan to modify the existing backend to just focus on the endpoints that need to be there and where to expand the worker functionality. Write down a step by step plan in BACKEND_TODO.md with file references and code that will be changed.

I've created a detailed migration plan in BACKEND_TODO.md. The backend can be removed - the Oracle confirms it only does GNIS place search and climate proxying, both of which can move to the worker.

Key insight: Supabase PostgREST can expose SQL functions as HTTP endpoints, so the worker can call gnis_search() directly instead of maintaining FastAPI just for database queries.

Result: Single-service architecture saving ~$32/year in hosting costs plus 20-40ms latency improvement.