Numbers to Know for System Design Interviews

System design interviews require candidates to make grounded engineering decisions based on modern hardware capabilities. Technology moves at an extraordinary pace, yet many interview study resources rely on hardware benchmarks from over a decade ago. Relying on obsolete numbers leads candidates to over engineer systems, introducing unnecessary distributed complexity when a single modern server could easily handle the workload.

In this guide, we break down the fundamental metrics every software engineer must know for modern system design interviews. We cover raw hardware limits, storage throughput, network latency, database capacity, and how to apply these numbers effectively during calculations.

The 5 Step System Design Framework

When evaluating hardware limits and system capacity during an interview, follow this five step framework to structure your solution:

1. Requirements & Estimations: Quantify user traffic, data payloads, query volumes, and storage footprints.
2. Core Entities & Schema Design: Model primary application domain objects and choose appropriate storage engines.
3. API Design: Establish clear application contracts for client server communication.
4. High Level Design: Sketch architectural components connecting clients, gateways, services, caches, and databases.
5. Deep Dives & Real World Production Case Studies: Analyze production bottlenecks, hardware constraints, and scaling trade offs.

Step 1: Requirements & Estimations

Accurate estimation requires aligning functional application goals with modern physical hardware capabilities.

Modern Hardware Reference Limits

Modern enterprise servers provide vast compute and storage capacity on a single physical host:

• Compute & Memory: Standard general purpose cloud nodes (such as AWS M6i.32xlarge) ship with 512 GiB RAM and 128 vCPUs. Memory optimized hosts scale up to 4 TB RAM (X1e.32xlarge) or even 24 TB RAM (U-24tb1.metal). Datasets that previously required multi node memory clusters often fit entirely within a single physical server.
• Disk Storage: Local high performance NVMe SSD instances (such as AWS i3en.24xlarge) offer up to 60 TB of raw local storage with hundreds of thousands of IOPS. Density optimized storage hosts (D3en.12xlarge) yield 336 TB of HDD capacity. Cloud object storage services (like AWS S3) provide virtually infinite capacity, handling petabyte workloads seamlessly.
• Networking: Datacenter network interfaces deliver 25 Gbps to 100 Gbps bandwidth per node. Inter availability zone network latency within the same geographic region stays around 1 to 2 milliseconds, while intra availability zone latency operates sub millisecond. Cross region network round trips range from 50 to 150 milliseconds.

Estimation Formulas and Walkthrough

To derive exact cluster sizing, use standard conversion formulas:

Daily Requests = Daily Active Users (DAU) * Operations Per User
Average QPS = Daily Requests / 86,400 seconds
Peak QPS = Average QPS * 2 (or 3 depending on traffic distribution)
Storage Per Year = Daily Writes * Average Payload Size * 365 days

Consider a global web service with the following scale targets:

• Active Daily Users: 100 million DAU
• Read Requests: 50 reads per user daily (5 billion total daily reads)
• Write Requests: 5 writes per user daily (500 million total daily writes)

Calculating throughput and storage demands:

• Average Read QPS: 5,000,000,000 / 86,400 ≈ 57,870 QPS
• Peak Read QPS: 57,870 * 2 ≈ 115,740 QPS
• Average Write QPS: 500,000,000 / 86,400 ≈ 5,787 QPS
• Peak Write QPS: 5,787 * 2 ≈ 11,574 QPS
• Daily Storage Ingestion: 500,000,000 writes * 1 KB payload = 500 GB daily (182.5 TB annually)

Under modern hardware specifications, 11,500 write QPS and 182.5 TB annual growth can be supported by a small database primary cluster backed by high IOPS NVMe arrays, eliminating the requirement to implement complex database sharding schemes on day one.

Step 2: Core Entities & Schema Design

Understanding hardware capacity directly influences database selection and entity schema structure.

Storage Engine Performance Characteristics

Schema Definition

Below is an optimized relational database schema designed for high throughput telemetry and resource metric logging:

CREATE TABLE systems (
    system_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    hostname VARCHAR(255) NOT NULL,
    environment VARCHAR(50) NOT NULL,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
);

CREATE TABLE hardware_metrics (
    metric_id BIGSERIAL PRIMARY KEY,
    system_id UUID NOT NULL REFERENCES systems(system_id),
    cpu_utilization NUMERIC(5, 2) NOT NULL,
    memory_used_bytes BIGINT NOT NULL,
    disk_iops INT NOT NULL,
    network_rx_bytes BIGINT NOT NULL,
    network_tx_bytes BIGINT NOT NULL,
    recorded_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
);

CREATE INDEX idx_metrics_system_time ON hardware_metrics(system_id, recorded_at DESC);

Step 3: API Design

Application programming interfaces expose hardware operational status and metric collection endpoints over clean HTTP contracts.

Fetch Hardware Operational Status

• HTTP Method: GET
• Path: /api/v1/systems/{system_id}/status
• Response Body:

{
  "system_id": "9b1deb4d-3b7d-4bad-9bdd-2b0d7b3dcb6d",
  "hostname": "prod-app-node-01",
  "status": "healthy",
  "hardware": {
    "total_memory_bytes": 549755813888,
    "available_memory_bytes": 214748364800,
    "cpu_cores": 128,
    "network_bandwidth_gbps": 25
  },
  "timestamp": "2026-06-29T21:15:00Z"
}

Submit Telemetry Metric Batch

• HTTP Method: POST
• Path: /api/v1/telemetry/submit
• Request Body:

{
  "system_id": "9b1deb4d-3b7d-4bad-9bdd-2b0d7b3dcb6d",
  "metrics": [
    {
      "cpu_utilization": 42.5,
      "memory_used_bytes": 335007449088,
      "disk_iops": 14200,
      "network_rx_bytes": 1048576000,
      "network_tx_bytes": 2097152000
    }
  ]
}

Step 4: High Level Design

A modern high level architecture leverages stateless compute clusters, distributed in memory caching, and durable primary relational storage backed by enterprise NVMe disk drives.

Architecture Diagram Generation Prompt: A modern enterprise system architecture diagram illustrating client HTTP requests flowing through an API Gateway into a pool of stateless application servers hosted on AWS M6i nodes. The application layer interacts with a Redis cluster for sub millisecond caching and reads or writes to a primary PostgreSQL relational database paired with secondary read replicas, configured with high performance NVMe SSD storage arrays.

Component Breakdown

1. API Gateway: Terminates incoming TLS connections, handles rate limiting, and authenticates requests.
2. Stateless Application Tier: Runs application logic on AWS M6i instances. Horizontal scaling adjusts instances dynamically based on CPU utilization and incoming network throughput.
3. In Memory Cache Cluster (Redis): Absorbs high frequency read queries, serving responses in sub millisecond timeframes and reducing database read load.
4. Primary Database (PostgreSQL): Manages core transactional writes and relational state. Uses high IOPS NVMe SSD storage to process thousands of concurrent write transactions per second.
5. Read Replicas: Asynchronously replicate data from the primary instance to serve heavy read traffic cleanly.

Step 5: Deep Dives & Real World Production Case Studies

Understanding how numbers translate into real world production engineering is critical for passing senior and staff level system design interviews.

Deep Dive 1: Caching Mechanics & Memory Management

In memory caches like Redis store data structures directly in RAM. A modern 512 GiB RAM host dedicated to caching can store hundreds of millions of key value pairs.

• Memory Overhead Calculation: An average key value pair of 1 KB requires approximately 1.2 KB of RAM due to internal Redis memory overhead tracking. A cache holding 100 million items occupies roughly 120 GB RAM, fitting easily inside a single large server.
• Hit Ratios and Eviction: When designing caches, aim for a cache hit ratio above 90 percent. Use memory eviction policies like AllKeys LRU (Least Recently Used) to manage memory cleanly when reaching node allocation limits.

Deep Dive 2: Database Throughput & Vertical Scaling Capacity

Many candidates prematurely default to database sharding when encountering moderate traffic volumes. Modern hardware changes this calculus completely.

• Single Host Throughput: A single PostgreSQL instance on modern enterprise hardware can handle between 10,000 and 30,000 write transactions per second and over 100,000 read QPS when queries hit indexed columns in memory.
• NVMe Storage Limits: Enterprise NVMe drives provide over 500,000 random read IOPS and 100,000+ write IOPS. Historical guidelines suggesting database partitioning at 100 GB are obsolete. Modern databases comfortably manage multi terabyte storage volumes on a single server before requiring physical database sharding.

Deep Dive 3: Application Server Concurrency & Connection Management

Stateless application servers handle incoming network sockets and application threads.

• Thread Allocation: Memory consumption per thread (typically 1 MB to 2 MB stack allocation in standard environments) limits total concurrent OS threads. Modern asynchronous non blocking I/O frameworks (such as Netty, Node.js, or Go goroutines) multiplex tens of thousands of concurrent network connections on minimal memory footprints.
• Connection Pooling: Database connection pools (such as PgBouncer) must be placed in front of relational databases to prevent backend connection exhaustion. Keep backend database connection counts between 100 and 500 active connections to maximize CPU cache efficiency and prevent thread context switching.

Deep Dive 4: Streaming & Message Broker Bandwidth

Distributed streaming platforms like Apache Kafka process sequential events at extreme speeds.

• Disk Sequential I/O: Kafka writes log segments to disk sequentially. Sequential disk writes on modern SSD arrays achieve speeds over 1 GB/s, matching or exceeding network ingestion rates.
• Broker Capacities: A single modern Kafka broker equipped with a 25 Gbps network interface can process over 100,000 events per second while saturating network interfaces without disk I/O bottlenecks.

Interview Cheat Sheet & Quick Reference

Keep these fundamental system benchmarks handy during system design interviews:

Common System Design Interview Pitfalls

Avoid these common mistakes when discussing system scale and hardware performance:

1. Premature Sharding: Proposing complex horizontal database sharding schemes for moderate workloads (such as 2,000 write QPS). Modern single primary databases handle these volumes cleanly without distributed sharding overhead.
2. Overestimating Network Latency: Assuming internal cloud datacenter calls take tens of milliseconds. Inter AZ calls execute in 1 to 2 milliseconds.
3. Ignoring Vertical Scaling Efficiency: Architecting overly complex distributed clusters when upgrading host instance sizing (vertical scaling) provides a far simpler, cost effective solution.
4. Neglecting Connection Limits: Forgetting to include database proxy connection pools when scaling stateless application instances out to dozens of nodes.