Horizontal vs Vertical Scaling: When to Scale Up vs Scale Out

Your app is slow. Traffic is growing. Do you get a bigger server or add more servers? This decision shapes your entire architecture.

The Two Approaches#

Vertical Scaling (Scale Up)#

Make the server bigger — more CPU, RAM, faster SSD.

Before: 4 vCPU, 16GB RAM → After: 32 vCPU, 128GB RAM

Horizontal Scaling (Scale Out)#

Add more servers behind a load balancer.

Before: 1 server → After: 4 servers behind load balancer

Comparison#

When to Scale Vertically#

1. Database — PostgreSQL, MySQL scale vertically well (read replicas are separate)
2. Single-threaded workloads — faster CPU helps more than more CPUs
3. Quick fix — buy time before re-architecting for horizontal
4. Team < 5 — horizontal scaling adds operational complexity
5. Cost is low — going from 2 vCPU to 8 vCPU is cheap

Vertical Scaling on Cloud#

Rule: If your workload fits on a 16-64 vCPU machine, vertical is simpler.

When to Scale Horizontally#

1. Stateless APIs — each request can go to any server
2. Web servers — static content, API endpoints
3. Workers — background job processors
4. Traffic is spiky — auto-scale up during peaks, down during troughs
5. High availability required — survive server failures

Making Your App Horizontally Scalable#

The key requirement: stateless servers.

❌ Stateful (can't scale horizontally):
  Server stores sessions in memory
  Server stores uploads in local filesystem
  Server caches in local memory

✓ Stateless (can scale horizontally):
  Sessions in Redis
  Files in S3
  Cache in Redis/Memcached
  Config from environment variables

Auto-Scaling Patterns#

Target Tracking#

Scale based on a metric target:

Target: CPU utilization = 60%
Current: 80% (2 instances)
Action: Add 1 instance → 3 instances → CPU drops to ~53%

Step Scaling#

Different actions at different thresholds:

CPU > 70% → add 1 instance
CPU > 85% → add 3 instances
CPU < 30% → remove 1 instance

Scheduled Scaling#

Predictable traffic patterns:

Mon-Fri 9am: scale to 10 instances
Sat-Sun: scale to 3 instances
Black Friday: scale to 50 instances

Scaling Metrics#

Scaling Different Components#

Stateless API Servers → Horizontal#

Load Balancer → API Server 1 (auto-scaled)
              → API Server 2
              → API Server 3
All share: Redis (sessions), PostgreSQL (data), S3 (files)

Databases → Vertical + Read Replicas#

Writes → Primary DB (vertical: 64 vCPU)
Reads  → Read Replica 1
       → Read Replica 2
       → Read Replica 3

Cache → Horizontal (Cluster)#

Redis Cluster:
  Shard 1: keys a-m
  Shard 2: keys n-z
  Each shard: primary + replica

Queue Workers → Horizontal#

SQS Queue → Worker 1 (auto-scaled by queue depth)
          → Worker 2
          → Worker N

Cost Analysis#

Small App (1000 req/sec)#

At small scale, vertical is often cheaper (no load balancer cost).

Medium App (10K req/sec)#

Horizontal becomes cheaper as you scale.

Large App (100K req/sec)#

Vertical isn't an option. Must scale horizontally.

Architecture Examples#

Simple SaaS (Vertical Start)#

Client → Nginx → Rails App (1 large server)
                      → PostgreSQL (RDS, vertical scaling)
                      → Redis (single instance)

Scale vertically until you can't. Then extract services.

Production SaaS (Horizontal)#

Client → CloudFront (CDN)
       → ALB → API (auto-scaling group, 3-20 instances)
             → Workers (auto-scaling, 2-10 by queue depth)
       → RDS Primary (writes) + 2 Read Replicas
       → ElastiCache Redis Cluster (3 shards)
       → S3 (files)

Generate your scaling architecture →

Decision Framework#

Can one server handle the load?
  Yes → Scale vertically (simpler)
  No → Is the workload stateless?
    Yes → Scale horizontally (add servers)
    No → Make it stateless first (externalize state)
         Then scale horizontally

Need high availability?
  Yes → Horizontal (minimum 2 servers, different AZs)
  No → Vertical is fine for now

Summary#

1. Start vertical — it's simpler and cheaper at small scale
2. Go horizontal when you hit hardware limits, need HA, or have spiky traffic
3. Stateless is the prerequisite — externalize sessions, files, cache
4. Auto-scale on the right metric — CPU for compute, queue depth for workers
5. Databases scale differently — vertical + read replicas, then sharding
6. Cost is linear with horizontal — but adds operational complexity

---

Design scalable architectures at codelit.io.