4 ms·
ArchGW: Open-source, AI-native (edge and LLM) proxy for prompt traffic
- sparacha 2y agoWhy We Built ArchGW? Traditional application architectures separate routing, security, and observability from business logic - so that developers can move faster without the tax of reinventing the wheel. LLM applications should be no different. ArchGW applies these patterns to prompts, providing a structured approach to building LLM applications. How It Works ArchGW runs as a separate process alongside application servers. It intercepts prompts before they reach the backend and applies transformations based on predefined rules and models: Preprocessing: Normalizes and analyzes prompt structure. Security Filtering: Rejects jailbreak attempts and unsafe inputs. Intent Mapping: Determines if a request maps to an API function. Function Invocation: Extracts arguments and calls backend APIs. LLM Routing: Chooses the right LLM provider based on latency/cost constraints. Tracing & Metrics: Adds W3C Trace Context headers, tracks errors, token usage, and request latency. Why a Dedicated Proxy? Traditional application architectures separate routing, security, and observability from business logic—LLM applications should be no different. ArchGW applies these patterns to prompt processing, providing a structured approach to LLM integration.