Building a Production AI Platform on AWS

AWS is the most common deployment target for enterprise AI platforms — and for good reason. The breadth of managed services, the compliance certifications, and the global infrastructure make it the default choice for regulated industries. Here's the architecture we recommend for a production-grade enterprise AI platform on AWS.

Architecture Overview

The platform consists of five layers: compute (EKS), AI services (OpenAI via private endpoint or Bedrock), data (RDS PostgreSQL with pgvector), caching (ElastiCache Redis), and storage (S3). All components run within a VPC with no public internet access except through an Application Load Balancer.

Compute: Amazon EKS

We deploy the application on Amazon EKS (Elastic Kubernetes Service) using managed node groups. EKS gives us portability — the same Kubernetes manifests work on Azure AKS or GCP GKE if requirements change. We use Karpenter for node autoscaling, which responds to pending pods within seconds rather than minutes.

• →Application pods: 2–10 replicas, t3.medium or c5.large depending on load
• →Worker pods for document ingestion: spot instances to reduce cost by 70%
• →GPU nodes for local model inference: g4dn.xlarge with NVIDIA T4
• →Cluster autoscaler handles scale-to-zero for non-production environments

Database: RDS PostgreSQL + pgvector

We use a single RDS PostgreSQL instance with the pgvector extension for both relational data and vector storage. This simplifies the architecture significantly — one database to manage, one backup strategy, one set of credentials. For most enterprise deployments up to ~10M document chunks, pgvector on a db.r6g.xlarge instance handles the load comfortably.

-- Enable pgvector
CREATE EXTENSION IF NOT EXISTS vector;

-- Documents table with vector embeddings
CREATE TABLE documents (
  id          UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  content     TEXT NOT NULL,
  embedding   vector(1536),
  metadata    JSONB,
  source      TEXT,
  created_at  TIMESTAMPTZ DEFAULT NOW()
);

-- HNSW index for fast approximate nearest neighbour search
CREATE INDEX ON documents 
USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 64);

Caching: ElastiCache Redis

Redis serves two purposes: caching LLM responses for identical queries (significant cost saving for FAQ-style use cases) and storing conversation history for multi-turn AI assistants. We use a cache-aside pattern — check Redis first, fall through to the LLM on miss, write the response back to Redis with a 1-hour TTL.

Security Configuration

• →All services in private subnets — no direct internet access
• →API keys stored in AWS Secrets Manager, rotated automatically
• →IAM roles for service accounts (IRSA) — no static credentials in pods
• →VPC endpoints for S3 and Secrets Manager — traffic never leaves AWS network
• →CloudTrail enabled for all API calls — full audit trail
• →GuardDuty enabled for threat detection

Infrastructure as Code

The entire infrastructure is defined in Terraform. This means the environment is reproducible, version-controlled, and auditable. We maintain separate Terraform workspaces for dev, staging, and production. Promotion between environments is a pull request, not a manual process.

Cost Optimisation

A well-architected enterprise AI platform on AWS doesn't have to be expensive. Use spot instances for batch workloads, reserved instances for baseline compute, and implement aggressive caching to reduce LLM API costs. A 500-person enterprise deployment typically runs at $3,000–$8,000/month on AWS — a fraction of the productivity value it delivers.