Web Infrastructure & Internet Fundamentals

What is the Internet, and how do computers communicate over it?

The Internet is a vast network of interconnected computers and devices that communicate with each other using standardized rules called protocols.
Think of it as a global highway system that connects millions of smaller roads (networks) — allowing data to travel between any two points in the world.

How communication works:

  1. When you send an email or open a website, your device breaks the data into small chunks called packets.
  2. Each packet is labeled with the sender and receiver’s IP addresses (like digital mailing addresses).
  3. These packets travel through a series of routers, switches, and networks to reach the destination.
  4. Once all packets arrive, they’re reassembled into the original message or webpage.

This is made possible by core communication protocols like:

  • TCP (Transmission Control Protocol): Ensures data is delivered reliably and in order.
  • IP (Internet Protocol): Decides how packets are addressed and routed to their destination.

Together, this system (TCP/IP) forms the foundation of how data moves across the Internet.

What is a client and what is a server?

In any Internet interaction, there are two key roles: clients and servers.

  • Client: The client is the device or software that requests information.
    Examples:
    • A web browser (like Chrome or Safari) is a client when it requests a webpage.
    • A mobile app is a client when it requests data from a backend system.
  • Server: The server is the system that responds to the request and provides the requested information or service.
    Examples:
    • A web server hosts website files (HTML, CSS, images, etc.).
    • A database server stores and retrieves user data.

Analogy:

Think of a restaurant — the client is the customer placing an order, and the server is the kitchen delivering the meal.

So when you type “http://www.google.com” your browser (client) sends a request to Google’s servers, which process the request and send back the search results page.

What is an IP address, and why is it important?

An IP address (Internet Protocol address) is a unique numerical identifier assigned to every device connected to the Internet.

It acts like a digital address, helping computers find and communicate with each other.

For example:

  • A website like google.com might resolve to an IP address like 142.250.190.14.
  • When you visit that website, your browser uses the IP address (not the domain name) to find and connect to Google’s server.

There are two main types of IP addresses:

  • Public IP: The address visible on the Internet (like your Wi-Fi router’s address).
  • Private IP: Used within local networks (like your laptop or phone inside your home network).

Why it’s important for PMs:

Understanding IPs helps you grasp how user requests are routed, how services identify unique devices, and how security systems (like firewalls or VPNs) manage network access.

What is the difference between IPv4 and IPv6?

The Internet Protocol comes in two major versions — IPv4 and IPv6.

FeatureIPv4IPv6
Address Format32-bit128-bit
Example192.168.1.12001:0db8:85a3:0000:0000:8a2e:0370:7334
Total Addresses~4.3 billion~340 undecillion (3.4×10³⁸)
Header ComplexitySimpleMore complex but efficient
AdoptionOlder, widely usedNewer, slowly replacing IPv4

Why IPv6 was created?

When the Internet began, IPv4 seemed sufficient. But as billions of devices (phones, IoT, laptops, etc.) came online, we started running out of IPv4 addresses.
IPv6 was introduced to solve this address exhaustion problem by offering vastly more combinations.

For PMs

Knowing IPv6 matters if your product targets global scalability, IoT, or regions where IPv6 adoption is high — it ensures future compatibility and network reach.

What is a domain name?

A domain name is the human-readable address of a website — it’s what you type into your browser to visit a site.
Since computers identify each other using IP addresses (e.g., 142.250.190.14), domain names make it easier for people to remember and access websites.

For example:

  • Instead of typing 142.250.190.14, you simply type http://www.google.com.
  • The browser then finds the correct server behind that name.

A domain name typically has three parts:

  • www → a subdomain (optional)
  • google → the main domain
  • .com → the top-level domain (TLD)

Other TLDs include .org, .in, .net, .io, etc.

PM insight: Understanding domain names helps you manage brand presence, SEO strategy, and product environments (e.g., using staging.product.com vs www.product.com).

What is DNS (Domain Name System), and why do we need it?

The Domain Name System (DNS) is like the Internet’s phonebook. It translates human-friendly domain names (like www.google.com) into machine-friendly IP addresses (like 142.250.190.14) that computers use to locate each other.

Without DNS, you’d have to remember numeric IP addresses for every website — which would be impractical.

Why we need DNS:

  • Humans remember names, not numbers.
  • Websites can change IP addresses, but the domain name stays constant — DNS handles these changes automatically.
  • It provides flexibility and reliability by distributing lookups across many DNS servers worldwide.

Analogy: Just like you use someone’s name to find their phone number in your contacts, your browser uses DNS to find the IP address of a website.

Explain DNS to a five-year-old

Imagine you want to visit your friend’s house, but you don’t know where they live.
You only know their name — let’s say your friend’s name is Google.

So, you ask your parents for help.

They look in a big address book that has everyone’s name and their home address written next to it.
They find your friend’s name — Google → 142.250.190.14 — and then tell you the address so you can go there.

That big address book is like DNS.

When you type a website’s name, like google.com, your computer doesn’t know where it is — so it asks DNS, ‘Hey, where does Google live?’ DNS finds the right address (the IP address) and tells your computer where to go.

So DNS is basically the Internet’s address book that helps your computer find where each website lives!

What happens when you type a URL http://www.google.com in your browser?

1. You type the URL and press Enter

  • Example: www.google.com
  • Your browser sees this as a request to access a website.

2. Browser checks cache

  • First, the browser checks if it has the IP address for www.google.com stored in its DNS cache.
  • If it finds it, it can skip some steps.
  • If not, it moves to the next step.

3. DNS lookup

  • The browser asks a DNS (Domain Name System) server to translate the human-readable domain (www.google.com) into an IP address (like 142.250.190.78), which computers use to communicate.
  • Where this DNS server “lives”:
    1. Browser/OS cache: Your computer first checks its own cache.
    2. Recursive DNS server: Usually provided by your ISP or public services like Google (8.8.8.8) or Cloudflare (1.1.1.1).
    3. Root DNS server: If needed, these servers direct queries to the correct Top-Level Domain (TLD) server like .com.
    4. TLD DNS server: Knows which DNS server handles the specific domain (google.com).
    5. Authoritative DNS server: Finally, the server that actually holds the IP address for www.google.com.
  • The response travels back through the chain, giving your browser the IP to connect to.
  • DNS servers are physically located all over the world in data centers and cloud networks to respond quickly depending on your location.

4. Browser establishes a connection

  • Most websites now use HTTPS, which means a secure connection.
  • Your browser uses TCP/IP to contact the server at that IP address.
  • Steps include:
    1. TCP handshake: Browser and server agree on a connection.
    2. TLS handshake: If HTTPS, they set up encryption to secure the data.

5. Browser sends an HTTP request

  • Browser sends a request to the server saying:
    "Hey, I want the page at / (the homepage of www.google.com)."
  • This request includes headers like browser type, language, and cookies.

6. Server processes the request

The server receives the request and decides what content to send back:

  • HTML of the page
  • CSS styles
  • JavaScript
  • Images, videos, etc.

7. Server sends an HTTP response

  • The server responds with the requested content, often compressed for speed.
  • It includes a status code:
    • 200 OK → Page found and sent
    • 404 Not Found → Page doesn’t exist
    • 500 Internal Server Error → Server has a problem

8. Browser renders the page

Executes JavaScript to make the page interactive.

Browser reads the HTML, CSS, and JavaScript.

Builds the DOM (Document Object Model).

Loads additional resources like images, fonts, and scripts.

PM takeaway: Understanding this flow helps diagnose user issues (e.g., site not loading due to DNS propagation delay) or coordinate launches involving new domains or subdomains.

What is HTTP, and how does it enable web communication?

HTTP (HyperText Transfer Protocol) is the set of rules that defines how data is exchanged between a web browser (client) and a web server. It’s the language of the web — every time you visit a website, your browser and the server communicate using HTTP.

How HTTP enables web communication (step-by-step):

  1. You enter a URL or click a link
    • Example: http://www.example.com
  2. Browser (client) sends an HTTP request
    • The browser asks the server for a specific resource, such as a web page or image.
  3. Server receives the request and processes it
    • The web server looks for the requested file (index.html) or runs the necessary code to generate it.
  4. Server sends an HTTP response
    • The response includes:
      • A status code (e.g., 200 OK, 404 Not Found)
      • The content (like HTML, text, images, etc.)
  5. Browser displays the content
    • The browser reads the HTML, applies styles, and shows the webpage to you.

In summary

  • This simple exchange is what allows web pages to load and function.
  • HTTP is a request–response protocol.
  • The browser requests a resource, and the server responds with the required data.
What is HTTPS, and how is it different from HTTP?

HTTPS (HyperText Transfer Protocol Secure) is a secure version of HTTP — the protocol used for communication between your browser and a web server.

The “S” stands for Secure, meaning that the data exchanged between the browser and the server is encrypted to protect it from being intercepted or modified.

How HTTPS Works

  1. You enter a URL starting with https://
    • Example: https://www.google.com
  2. Browser establishes a secure connection
    • Before any data is exchanged, the browser and server perform a TLS handshake (Transport Layer Security).
    • This process sets up encryption keys and verifies the server’s identity using a digital certificate.
  3. Data is encrypted and exchanged
    • All information (like login details, payment info, or messages) is encrypted before being sent.
    • Even if someone intercepts the data, they can’t read it.
  4. Server sends a response
    • The browser decrypts the response and displays the website safely.

Key Differences: HTTP vs HTTPS

FeatureHTTPHTTPS
Full formHyperText Transfer ProtocolHyperText Transfer Protocol Secure
SecurityData is sent in plain textData is encrypted using TLS
Port usedPort 80Port 443
CertificateNo certificate neededRequires an SSL/TLS certificate
Use caseNon-sensitive data or internal testingSensitive data like logins, payments, personal info

In summary

  • Almost all modern websites use HTTPS to protect users and build trust.
  • HTTP focuses on communication.
  • HTTPS focuses on secure communication — ensuring privacy, integrity, and authenticity of data.
What is a TLS handshake, and what steps are involved in establishing a secure connection between a browser and a web server?

The TLS handshake (Transport Layer Security handshake) is the process that happens when your browser first connects to a website using HTTPS.

It’s how the browser and server agree on how to communicate securely — by verifying each other, choosing encryption methods, and sharing keys safely.

Step-by-Step: How the TLS Handshake Works

1. Browser says hello

  • Your browser sends a message called “Client Hello” to the server.
  • It includes:
    • The TLS version it supports (e.g., TLS 1.3)
    • A list of encryption algorithms it can use
    • A random number (used later to create encryption keys)

2. Server says hello back

  • The server replies with “Server Hello.”
  • It chooses the best encryption method from the browser’s list.
  • It sends:
    • Its own random number
    • Its digital certificate (to prove its identity)
    • The public key inside the certificate

3. Browser verifies the server

  • The browser checks the digital certificate:
    • Was it issued by a trusted authority (like DigiCert or Let’s Encrypt)?
    • Has it expired or been tampered with?
    • Does it match the website domain?
  • If it passes all checks, the browser trusts the server.

4. Session key creation

  • The browser and server use their exchanged random numbers and keys to create a shared secret — a unique session key.
  • This key is used to encrypt all data for the session.

5. Secure connection established

  • Both sides confirm they can encrypt and decrypt data correctly.
  • Once confirmed, the secure connection begins — now all communication is encrypted and safe.

In short

The TLS handshake ensures that:

  • You’re talking to the real website (authentication)
  • Both sides agree on encryption methods
  • A shared secret key is safely created for secure communication

After this handshake, normal HTTP data (like page requests and responses) flows through the encrypted tunnel, forming HTTPS.

What is SSL/TLS, and why is it important for security?

SSL (Secure Sockets Layer) and TLS (Transport Layer Security) are security protocols that protect data sent between your browser and a web server.

  • SSL was the original version developed in the 1990s.
  • TLS is the newer, more secure version — it replaced SSL.
  • Today, when people say “SSL certificate,” they usually mean a TLS certificate (the term “SSL” just stuck around).

How SSL/TLS Works

When you visit a website using HTTPS, SSL/TLS:

  1. Encrypts data
    • Converts information (like passwords or credit card details) into unreadable code during transmission.
  2. Authenticates the server
    • Ensures you’re communicating with the real website, not an imposter.
  3. Maintains data integrity
    • Prevents data from being changed or corrupted while in transit.

Why SSL/TLS Is Important for Security

PurposeDescription
ConfidentialityKeeps user data private by encrypting it so hackers can’t read it.
AuthenticationVerifies the website’s identity through a trusted certificate authority (CA).
IntegrityDetects if data has been tampered with during transfer.
TrustBrowsers show a lock icon (🔒) for HTTPS sites, signaling users it’s safe.

In summary

  • Without SSL/TLS, anyone on the network could see or modify the data being transmitted.
  • SSL/TLS secures the communication channel between your browser and the web server.
  • It ensures that the data you send and receive is private, authentic, and safe from attackers.
What is a Web Server?

A web server is a computer system or software that stores, processes, and delivers web pages to users over the Internet when they request them through a browser.

In short, it’s the place where websites live and the software that sends those websites to your browser when you type a URL.

How a Web Server Works (Step-by-Step)

  1. User makes a request
    • You type a URL (like www.example.com) in your browser or click a link.
    • The browser sends an HTTP request to the web server.
  2. Server receives and processes the request
    • The web server checks what the user is asking for — a web page, image, video, etc.
  3. Server finds the requested resource
    • It locates the file in its storage (e.g., index.html, about.html).
  4. Server sends an HTTP response
    • The web server sends the resource back to the browser along with an HTTP status code (like 200 OK).
  5. Browser displays the content
    • Your browser renders the page and shows it on your screen.

Common Web Server Software

  • Apache HTTP Server (by Apache Software Foundation)
  • Nginx (pronounced “Engine-X”)
  • Microsoft IIS (Internet Information Services)
  • LiteSpeed, Caddy, etc.

In summary

A web server is both:

  • Hardware: The physical machine that stores website data.
  • Software: The program that understands HTTP requests and sends responses.

It acts as the middleman between your browser and the website’s content, ensuring you see the correct web page when you ask for it.

What is the difference between a web server and an application server?

Although both web servers and application servers deliver content to users, they serve different purposes in the web architecture.

1. Web Server

A web server is mainly responsible for handling HTTP requests and serving static content — such as HTML pages, images, CSS, and JavaScript files.

Example:

When you visit www.example.com/index.html, the web server directly sends that file to your browser.

Common Web Servers:

  • Apache HTTP Server
  • Nginx
  • Microsoft IIS

Key Features:

  • Handles HTTP protocol
  • Serves static content efficiently
  • Can forward complex requests to an application server

2. Application Server

An application server is designed to run and manage application logic — it processes dynamic requests (like user logins, payments, or database queries).

Example:

When you log in to a website, your username and password are sent to the application server, which verifies them in the database and returns a result.

Common Application Servers:

  • Apache Tomcat
  • JBoss / WildFly
  • WebLogic
  • WebSphere

Key Features:

  • Executes server-side code (Java, Python, PHP, etc.)
  • Connects with databases and other backend systems
  • Generates dynamic content before sending it to the web server

3. Key Differences

AspectWeb ServerApplication Server
Main RoleServes static contentRuns business logic and dynamic content
Content TypeHTML, CSS, JS, imagesProcessed data, API responses
Protocols UsedMainly HTTP/HTTPSHTTP plus others (RMI, JMS, etc.)
Example SoftwareNginx, Apache HTTP ServerTomcat, JBoss, WebLogic
InteractionSends files directly to clientsWorks with databases, creates responses for web servers

4. How They Work Together

In many modern systems:

  • The web server handles client requests and static files.
  • It forwards complex, dynamic requests to the application server, which processes the logic and sends the result back.
  • The web server then delivers that result to the user’s browser.

In summary

  • Together, they make up the backbone of modern web applications.
  • A web server serves static files and handles basic web requests.
  • An application server handles business logic and dynamic content generation.
What is web hosting, and what role does it play in delivering websites?

Web hosting is a service that provides the space and resources on a server where your website’s files, databases, and content are stored — so people can access your website on the Internet.

In simple terms, web hosting is like renting a space on the Internet for your website to live.

How Web Hosting Works (Step-by-Step)

1. You Create a Website

  • You design and build your website using web technologies like HTML (structure), CSS (styling), JavaScript (interactivity), and sometimes databases (like MySQL or MongoDB) for storing content and user data.
  • At this stage, all your files — web pages, images, videos, and scripts — exist locally on your computer.
  • However, for others to see it, your website must be placed on a server that’s always connected to the Internet.

2. You Upload It to a Web Host’s Server

  • A web hosting company provides you with space on a powerful, always-on computer (server) located in a data center.
  • This server is connected to high-speed Internet with redundant power supply, cooling, and backup systems, ensuring your website stays available 24/7.
  • You upload your website files to this server using:
    • FTP (File Transfer Protocol) or
    • A control panel provided by your hosting company (like cPanel or Plesk).
  • Once uploaded, your files are stored in a folder that the web server software (like Apache or Nginx) can access.

3. Domain Name Connects to the Hosting Server

  • You register a domain name (like www.example.com) from a domain registrar.
  • Your hosting provider gives you the server’s IP address (like 192.0.2.1), which uniquely identifies where your website lives on the Internet.
  • You update your domain’s DNS (Domain Name System) records to point to your web host’s server.
    • For example, the A Record of your domain might point to your hosting server’s IP.
  • Now, when someone types www.example.com in their browser, the DNS system finds which server hosts your website and directs the user there.

4. Server Delivers Your Website

  • Once the browser reaches your host’s server, it sends an HTTP request asking for your website’s files.
  • The web server software (like Apache, Nginx, or IIS) processes this request.
    • If the file is static (like an image or HTML page), it sends it directly.
    • If it’s dynamic (like a login page or an online store), the application server and database process the request and generate the correct content.
  • The server then sends back an HTTP response containing your website’s data to the browser.

5. Website Loads in the Browser

  • The browser receives the response from the hosting server.
  • It downloads all necessary files — HTML, CSS, JavaScript, images, fonts, etc.
  • It builds and displays the web page by:
    • Parsing the HTML to create a DOM (Document Object Model)
    • Applying CSS for styling
    • Running JavaScript for interactivity (buttons, animations, etc.)
  • The end user now sees your fully functional website on their screen — no matter where they are in the world.

Bonus: Behind the Scenes — What Web Hosting Also Handles

  • Security: Protects your website using firewalls, SSL/TLS certificates, and malware scanning.
  • Performance: Uses caching, load balancing, and CDNs (Content Delivery Networks) to deliver pages faster.
  • Maintenance: Takes care of server updates, uptime monitoring, and backups.
  • Scalability: Lets you upgrade your plan (CPU, RAM, bandwidth) as your traffic grows.

In summary

Web hosting is what makes your website accessible globally. It connects three essential components:

  • Your website files
  • Your domain name
  • The web hosting server

Together, they ensure that when someone types your website’s address, it’s located, loaded, and displayed reliably every time.

What is caching, and how does it make websites faster?

Caching is a technique used to store copies of frequently accessed data (like web pages, images, or files) so that it can be delivered to users much faster the next time they request it. Instead of fetching the same information repeatedly from the original server, caching serves it from a temporary storage location that’s closer to the user or easier to access.

Here’s a detailed step-by-step explanation of how caching makes websites faster:

1. Without caching (the slow way)

When a user visits a website for the first time:

  • The browser sends a request to the website’s server.
  • The server processes the request, fetches data from databases, and sends the response back.
  • This process takes time — especially if the server is far away or under heavy load.

Every time a user revisits a page, the same process happens again — unless caching is used.

2. With caching (the fast way)

Caching saves a copy of the website’s content (like images, stylesheets, or even complete pages) after the first visit.
So next time:

  • The browser or a nearby cache can serve the data instantly, without contacting the main server.
  • This reduces load times dramatically and saves bandwidth.

3. Types of Caching

a. Browser Cache

  • The user’s browser (like Chrome or Safari) stores copies of web pages, images, and scripts locally.
  • When the user revisits the same site, the browser loads the cached version instead of downloading everything again.

b. Server Cache

  • The website’s server stores pre-built versions of pages.
  • Instead of re-generating each page from scratch for every request, it serves the stored version, saving time and processing power.

c. CDN (Content Delivery Network) Cache

  • CDNs store cached copies of a website’s content in servers distributed around the world.
  • When a user visits the site, content is served from the nearest location — reducing latency and improving speed.

4. Benefits of Caching

  • Faster Load Times: Users get pages almost instantly.
  • Reduced Server Load: Fewer requests reach the origin server.
  • Better User Experience: Smooth navigation keeps users engaged.
  • Bandwidth Savings: Cached data avoids repeated downloads.

Example

When you visit http://www.amazon.in, your browser caches product images and layout files. Next time you open Amazon, those files load from your local cache — only new or changed content is fetched from Amazon’s servers.

What is a CDN (Content Delivery Network), and how does it improve performance?

A CDN (Content Delivery Network) is a system of distributed servers that deliver web content (like images, videos, scripts, or entire pages) to users based on their geographic location.

Its main goal is to make websites load faster, handle more traffic, and remain available even under heavy load.

How It Works

  1. Your website’s content is copied and stored (cached) on multiple CDN servers worldwide — called edge servers.
  2. When a user visits your site, the CDN automatically serves the content from the nearest server instead of the main origin server.
  3. This reduces the physical distance data travels, leading to faster load times and less network congestion.

Example:

  • Your origin server is in the US.
  • A user in India requests your website.
  • Instead of loading data from the US, the CDN delivers it from the nearest Indian CDN server — much faster.

How a CDN Improves Performance

BenefitExplanation
Reduced LatencyContent is served from the closest server, minimizing travel time.
Faster Load TimesStatic assets (images, CSS, JS) load quicker when cached locally.
Better ReliabilityIf one server fails, traffic automatically reroutes to another.
ScalabilityCan handle high traffic spikes (e.g., product launches, live events).
Reduced Origin LoadOffloads traffic from your main server, preventing overload.
Enhanced SecurityProtects against DDoS attacks and supports HTTPS at scale.

Real-World Examples

  • Cloudflare
  • Akamai
  • Amazon CloudFront
  • Google Cloud CDN
  • Fastly

In short:

A CDN improves web performance by:

  • Ensuring faster, safer, and more reliable content delivery.
  • Bringing content closer to users,
  • Reducing load on your origin server, and
What is Rate Limiting, and where is it used?

Rate limiting is the process of controlling how often an action can occur within a specific period of time. It sets a limit on how many requests, operations, or actions a user or system can perform — for example, “no more than 100 actions per minute.”

Purpose

Rate limiting is used to:

  • Prevent overload on systems and servers
  • Protect against abuse (like spam or brute-force attacks)
  • Ensure fair usage among users
  • Maintain consistent performance

Where It’s Used

Rate limiting is a general concept applied across many systems, such as:

AreaExample Use
APIsLimit how many requests a user can make per minute
WebsitesRestrict the number of login attempts to prevent hacking
Email ServicesLimit outgoing emails to avoid spam
Messaging AppsControl how many messages a user can send
NetworksLimit data packets to manage bandwidth and avoid congestion

In short:

Rate limiting is a traffic management technique that keeps systems stable, secure, and fair by restricting how frequently users or devices can perform actions.

How does rate limiting work, and what are the common techniques used to implement it?

At its core, rate limiting tracks how many actions (like API calls, login attempts, or messages) a user performs over time — and blocks or delays further actions once the limit is reached.

Step-by-Step Process

  1. Define a limit:
    The system sets a rule — for example:
    “Each user can make 100 requests per minute.”
  2. Track usage:
    The system monitors how many requests each user (or IP address) makes within that time window.
  3. Check the count:
    Every time a request comes in, the system checks:
    • Has the user exceeded their limit?
    • If not, the request is allowed and the counter increases.
    • If yes, the request is blocked or delayed.
  4. Reset after the time window:
    Once the time period (e.g., 1 minute) ends, the counter resets and the user can send new requests again.

Example

Let’s say an API allows 10 requests per minute.

TimeRequest #Action
0s1✅ Allowed
10s5✅ Allowed
50s10✅ Allowed
55s11❌ Blocked (limit reached)
60sCounter resets
61s1✅ Allowed again

When the limit is hit, the system may respond with an error like:

HTTP 429 Too Many Requests
Retry-After: 30

Common Implementation Methods

MethodHow It WorksNotes
Fixed WindowCounts requests in fixed time blocks (e.g., every minute)Simple but may allow bursts at window edges
Sliding WindowTracks requests over a moving time frameSmoother, more accurate limiting
Token BucketEach user gets a set number of “tokens”; each request consumes oneAllows small bursts while enforcing average rate
Leaky BucketRequests are processed at a steady rate; excess is droppedEnsures even traffic flow

In short:

Rate limiting monitors and restricts the frequency of actions over time. It helps systems stay stable, fair, and protected — especially during traffic spikes or misuse.

What is the difference between latency and throughput? Explain with simple examples.

What is Latency?

Latency is the time delay between a request being made and the corresponding response being received. It measures how fast a single operation is completed.

In simple terms:

Latency = “How long does it take to get one task done?”

Example: When you click “Buy Now” on Amazon, the time it takes for the confirmation page to appear = Latency

Measured in: milliseconds (ms) or seconds (s)

Goal: Minimize latency → faster response time.

What is Throughput?

Throughput refers to the number of requests or operations a system can handle per unit of time. It measures how many tasks can be completed in a given period.

In simple terms:

Throughput = “How many tasks can be done per second?”

Example: If a system can process 1,000 “Buy Now” requests every second → Throughput = 1,000 requests/sec

Measured in: requests per second (RPS), transactions per second (TPS), or Mbps (for networks)

Goal: Maximize throughput → handle more traffic efficiently.

Latency vs Throughput — The Key Difference

AspectLatencyThroughput
DefinitionTime taken to process a single requestNumber of requests processed per second
Measure ofSpeed (response time)Capacity (volume)
AnalogyHow fast one car completes a lapHow many cars pass the finish line per minute
GoalReduce delayIncrease processing rate
Typical UnitMilliseconds (ms)Requests per second (RPS)

Example Analogy

Imagine a restaurant:

  • Latency: How long it takes to serve one customer.
  • Throughput: How many customers are served in one hour.

You can have:

  • Low latency but low throughput: One chef cooking super-fast, but only one customer at a time.
  • High throughput but high latency: Many chefs serving hundreds of customers, but each one waits longer.

Relationship Between Latency & Throughput

They are inversely related — improving one can sometimes hurt the other:

  • If you try to process too many requests (high throughput), the response time (latency) might increase due to overload.
  • If you focus on serving each request faster (low latency), the total capacity (throughput) may be lower.

Efficient systems aim to balance both, depending on the use case.

Real-World Examples

  1. Streaming Services (e.g., Netflix):
    • Low latency: When you click “Play,” video starts quickly.
    • High throughput: Millions of users streaming simultaneously.
  2. Payment Systems:
    • Low latency is critical (user shouldn’t wait too long).
    • High throughput ensures many transactions can be processed during peak hours.
  3. APIs or Web Servers:
    • Latency = Response time per API call.
    • Throughput = Total number of API calls handled per second.

Summary

MetricFocusIdeal GoalExample Metric
LatencySpeed per requestLower = better200 ms per request
ThroughputRequests handledHigher = better10,000 RPS

In One Line:

Latency measures how fast you can respond; throughput measures how much you can handle.