📝 Topics Covered

1. 1. Performance vs. Scalability
2. 2. Key Performance Metrics
   • 2.1 Latency vs. Response Time
   • 2.2 Latency vs. Throughput
   • 2.3 Bandwidth vs. Throughput
3. 3. System Component Performance Indicators
4. 4. Understanding the CAP Theorem
   • 4.1 Consistency (C)
   • 4.2 Availability (A)
   • 4.3 Partition Tolerance (P)
5. 5. The CAP Theorem Trade-Offs
   • 5.1 CP Systems (Consistency + Partition Tolerance)
   • 5.2 AP Systems (Availability + Partition Tolerance)
   • 5.3 CA Systems (Consistency + Availability)
6. 6. Faults, Failures, and Fault Tolerance

1. Performance vs. Scalability

• Performance: Refers to the responsiveness and speed of a system. It measures how quickly a system can execute its intended function for a single user within given time constraints.
• Scalability: Refers to the capability of a system to increase or decrease its performance proportionally under an increased or decreased workload.

[!TIP] A simple rule of thumb:

• If you have a performance problem, your system is slow for a single user.
• If you have a scalability problem, your system is fast for a single user but slow under heavy concurrent load.

A highly scalable system can handle growing demand and increasing traffic without experiencing a significant degradation in performance.

2. Key Performance Metrics

To successfully measure, analyze, and optimize a system’s responsiveness, we track four primary network and computing metrics:

2.1 Latency vs. Response Time

• Latency: The delay for data to pass from one point on a network to another. It represents the time spent purely on the network transition.
• Response Time: The total time it takes for a system to fully respond to a request, including network transit and application server computational processing.

$$\text{Response Time} = \text{Latency} + \text{Processing Time}$$

2.2 Latency vs. Throughput

• Throughput: The absolute number of actions or transactions performed per unit of time.
• System Goal: Always aim for maximal throughput with acceptable latency thresholds.

$$\text{Throughput} = \frac{\text{Work Done}}{\text{Time Taken}}$$

2.3 Bandwidth vs. Throughput

• Bandwidth: The maximum potential data capacity of a network link (how much data can theoretically travel in a given time).
• Throughput: The actual amount of data successfully transmitted across the link under real-world constraints.

3. System Component Performance Indicators

Each tier of a distributed system has dedicated telemetry parameters that must be monitored using APM tools (e.g. New Relic, Datadog, SolarWinds):

4. Understanding the CAP Theorem

The CAP Theorem (originally formulated as Brewer’s Theorem) states that in a distributed database system, it is impossible to simultaneously provide more than two of the three following guarantees:

4.1 Consistency (C)

Consistency guarantees that every read receives the most recent write or an error.

All clients see the exact same data at the same time, regardless of which node they connect to. For a write to be marked as successful, the update must be fully replicated across all database nodes in the system before confirming the write.

4.2 Availability (A)

Availability guarantees that every non-failing request receives a non-error response, without guaranteeing that it contains the most recent version of the data.

An available system remains fully operational and responsive 100% of the time, even if some nodes are experiencing high latency or internal glitches.

4.3 Partition Tolerance (P)

Partition Tolerance guarantees that the system continues to operate despite arbitrary message loss or network partition failures between nodes.

In the physical world, networks will eventually experience packet loss or connection dropouts. A partition-tolerant system replicates data sufficiently across combinations of nodes to withstand these outages.

5. The CAP Theorem Trade-Offs

Because hardware networks are imperfect, Partition Tolerance (P) is a non-negotiable requirement for distributed systems. Therefore, system design boils down to choosing between Consistency (C) and Availability (A) when a network partition occurs.

5.1 CP Systems (Consistency + Partition Tolerance)

A CP database prioritizes strict data consistency. If a network partition occurs, the system will reject or block writes to nodes that cannot safely replicate data, sacrificing Availability (A) to prevent returning stale data.

• Core Behavior: Refuses to process requests on isolated nodes, throwing an error or timing out.
• Common Examples: Apache HBase, Redis (clustered), MongoDB, Spanner.

5.2 AP Systems (Availability + Partition Tolerance)

An AP database prioritizes high availability. If a network partition occurs, all nodes remain fully open for writes and reads, but disconnected nodes will return stale data until the partition is resolved and nodes are re-synchronized.

• Core Behavior: Returns available data immediately, sacrificing strict Consistency (C) for eventual consistency.
• Common Examples: Apache Cassandra, Apache CouchDB, DynamoDB.

5.3 CA Systems (Consistency + Availability)

A CA database guarantees consistency and availability across all nodes but cannot handle network partitions.

• Core Behavior: If a partition occurs, the system fails completely.
• Common Examples: Single-node Relational Databases (e.g. PostgreSQL, MySQL, MariaDB) that run on a single machine where network partitions between nodes are not applicable.

6. Faults, Failures, and Fault Tolerance

Designing resilient systems requires distinguishing between faults and failures:

• Fault: An individual component behaving incorrectly or failing (e.g. disk crash, memory leak, network lag).
• Failure: The complete system failing to deliver the required service to the end user.
• Fault Tolerance: The capability of a system to continue operating without interruption or user-facing failure in the presence of localized component faults.

6.1 Fault Resiliency Matrix

• Out of Memory (OOM) Fault: Hardware runs out of memory under heavy traffic loads.
  • Fault Tolerance Strategy: Configure dynamic horizontal Auto-Scaling to distribute the request load.
• Hardware Disk / Node Fault: Physical server crashes or power fails.
  • Fault Tolerance Strategy: Implement database and server Replication across separate availability zones.
• Software Code Exception: A runtime bug occurs in the backend handler.
  • Fault Tolerance Strategy: Implement standard try-catch fallback blocks and return a graceful error response to the user instead of letting the application crash.

7. References & Further Reading

• Educative Articles: Understanding the CAP Theorem Mechanics
• Design Guides: Distributed Performance Telemetry Primer
• Network Primers: Latency vs Throughput vs Bandwidth Analysis