Network Engineering Deep Dive — From Cables to Cloud

Table of contents

The Physical Reality of the Internet

We talk about “the cloud” like it’s magic. But at the end of the day, the internet is just a bunch of cables, routers, and switches. When you send a WhatsApp message from Mumbai to someone in New York, that data travels through fiber optic cables on the ocean floor.

Here’s a mind-blowing fact: there are over 550 active submarine cables carrying 99% of intercontinental data. These cables are laid on the ocean floor, sometimes by specialized ships, and they cost hundreds of millions of dollars to build.

Physical Layer (Layer 1)

Copper vs Fiber vs Wireless

Copper (Ethernet):

• Cat5e: Up to 1 Gbps, 100m max distance
• Cat6: Up to 10 Gbps, 55m max distance
• Cat6a: Up to 10 Gbps, 100m max distance
• Cheapest, easiest to install, but limited distance and speed

Fiber Optic:

• Single-mode: Up to 100 Gbps, 100km+ distance
• Multi-mode: Up to 10 Gbps, 2km distance
• Uses light instead of electrical signals
• Immune to electromagnetic interference
• More expensive, harder to install

Wireless (Wi-Fi):

• Wi-Fi 6 (802.11ax): Up to 9.6 Gbps theoretical
• Wi-Fi 7 (802.11be): Up to 46 Gbps theoretical
• Convenient, but slower and less reliable than wired

Real-world analogy: Copper is like a regular road — works for most things, limited speed. Fiber is like a highway — fast, long distance, but expensive to build. Wireless is like a dirt road — convenient but unpredictable.

How Data Becomes Signals

When you send data, it needs to be converted to a physical signal:

Digital Data:    0 1 1 0 1 0 0 1
                 ↓
Copper:          Voltage changes (high/low)
Fiber:           Light pulses (on/off)
Wireless:        Radio waves (modulated frequency)

This is why wireless is slower — encoding data into radio waves and decoding it takes more processing than simply detecting voltage changes in a wire.

Data Link Layer (Layer 2)

MAC Addresses

Every network device has a unique MAC address burned into its hardware:

Example MAC Address: 00:1A:2B:3C:4D:5E
                     ├── Vendor Prefix ──┤├── Device ID ──┤

MAC addresses work within a local network. When your laptop wants to talk to your router, it uses MAC addresses. But MAC addresses can’t route across the internet — that’s what IP addresses are for.

ARP: The MAC-to-IP Translator

When your laptop knows the IP address of a device on the same network but doesn’t know its MAC address, it uses ARP (Address Resolution Protocol):

1. Laptop: "Who has 192.168.1.1? Tell 192.168.1.100" (broadcast)
2. Router: "192.168.1.1 is at 00:1A:2B:3C:4D:5E" (unicast reply)
3. Laptop: Now knows the MAC address, can send data

This is why ARP spoofing is dangerous — if an attacker tricks your laptop into thinking their MAC address belongs to the router, they can intercept all your traffic.

Network Layer (Layer 3) — Routing

How Routers Think

Routers are the postal workers of the internet. They receive packets, look at the destination IP, and figure out the best next hop.

Your Computer (192.168.1.100)
        ↓
Home Router (192.168.1.1) — "I don't know how to reach 142.250.80.46,
                            let me forward to my ISP"
        ↓
ISP Router — "142.250.80.46 is in Google's network, let me route there"
        ↓
Google's Router — "This is for 142.250.80.46, deliver locally"
        ↓
Google's Server

Routing Algorithms

Static Routing: Manual configuration. Simple but doesn’t adapt to failures.

Dynamic Routing: Routers learn paths automatically.

OSPF (Open Shortest Path First):

• Link-state protocol
• Each router knows the entire network topology
• Calculates shortest path using Dijkstra’s algorithm
• Used within a single organization (interior gateway protocol)

BGP (Border Gateway Protocol):

• Path-vector protocol
• Connects autonomous systems (AS) — like ISPs and large companies
• The “protocol of the internet” — how different networks talk to each other
• BGP determines the path your data takes across the internet

AS1 (Your ISP) ←→ AS2 (Another ISP) ←→ AS3 (Google's Network)
         ↕                                    ↕
    AS4 (Cloudflare)                  AS5 (CDN Edge)

Real-world analogy: OSPF is like GPS navigation within a city — it knows all the streets and finds the shortest route. BGP is like flight planning between cities — it knows which airlines (networks) connect where.

Subnetting and CIDR

IP addresses are divided into networks and hosts:

192.168.1.100/24
│              │ │
│              │ └── 24 bits for network, 8 bits for hosts
│              └──── Network: 192.168.1.0
└─────────────────── Host: 100

This means:
- Network: 192.168.1.0
- Usable hosts: 192.168.1.1 - 192.168.1.254
- Broadcast: 192.168.1.255

Common subnet masks:

• /24 (255.255.255.0) — 254 hosts
• /16 (255.255.0.0) — 65,534 hosts
• /8 (255.0.0.0) — 16,777,214 hosts

Transport Layer — Port Numbers

Port numbers are like apartment numbers in a building. The IP address gets you to the building, the port number gets you to the specific application.

Well-Known Ports (0-1023):
  80  → HTTP
  443 → HTTPS
  22  → SSH
  21  → FTP
  53  → DNS

Registered Ports (1024-49151):
  3306 → MySQL
  5432 → PostgreSQL
  8080 → HTTP Alt
  3000 → Development servers

Dynamic/Ephemeral Ports (49152-65535):
  Used by client applications temporarily

When your browser connects to a web server, it uses a random ephemeral port (like 54321) and connects to port 80 or 443 on the server.

Firewalls: The Gatekeepers

Firewalls inspect network traffic and decide what to allow or block.

Types of Firewalls

Packet Filtering Firewall:

• Looks at individual packets
• Checks source/destination IP, port, protocol
• Fast but stateless — doesn’t track connections

Rules:
  Allow TCP from any to 10.0.0.1:443
  Allow TCP from 10.0.0.0/24 to 10.0.0.1:22
  Deny all other traffic

Stateful Firewall:

• Tracks connection states
• Knows if a packet is part of an established connection
• More secure than packet filtering

Connection Table:
  ESTABLISHED: 192.168.1.100:54321 → 142.250.80.46:443
  TIME_WAIT:    192.168.1.100:54322 → 142.250.80.46:80

Application Layer Firewall (WAF):

• Inspects HTTP traffic
• Blocks SQL injection, XSS, etc.
• Used by Cloudflare, AWS WAF

Next-Generation Firewall (NGFW):

• Combines all of the above
• Adds intrusion prevention, SSL inspection, application awareness
• Enterprise-grade

iptables Example

# Allow incoming SSH
iptables -A INPUT -p tcp --dport 22 -j ACCEPT

# Allow incoming HTTP/HTTPS
iptables -A INPUT -p tcp --dport 80 -j ACCEPT
iptables -A INPUT -p tcp --dport 443 -j ACCEPT

# Allow established connections
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Drop everything else
iptables -A INPUT -j DROP

VPNs: Encrypted Tunnels

A VPN creates an encrypted tunnel between your device and a VPN server. All traffic goes through this tunnel, making it appear as if you’re browsing from the VPN server’s location.

How VPNs Work

Without VPN:
Your Computer → ISP → Internet → Destination

With VPN:
Your Computer → Encrypted Tunnel → VPN Server → Internet → Destination

Your ISP sees encrypted traffic going to the VPN server but can’t see what’s inside. The destination sees the VPN server’s IP, not yours.

WireGuard: The Modern VPN

WireGuard is simpler and faster than traditional VPNs like OpenVPN:

# WireGuard config
[Interface]
PrivateKey = <your-private-key>
Address = 10.0.0.2/24
DNS = 1.1.1.1

[Peer]
PublicKey = <server-public-key>
Endpoint = vpn-server.com:51820
AllowedIPs = 0.0.0.0/0

WireGuard uses modern cryptography (ChaCha20, Curve25519) and has a much smaller codebase (~4,000 lines vs OpenVPN’s ~100,000), making it easier to audit and faster to run.

Cloud Networking

VPC (Virtual Private Cloud)

A VPC is your own isolated network in the cloud:

Internet
    ↓
Internet Gateway
    ↓
┌─────────────────────────────────────┐
│  VPC (10.0.0.0/16)                 │
│                                     │
│  ┌─────────────────┐  ┌──────────┐ │
│  │ Public Subnet   │  │ Private  │ │
│  │ (10.0.1.0/24)   │  │ Subnet   │ │
│  │                 │  │(10.0.2.0 │ │
│  │ Load Balancer   │  │  /24)    │ │
│  │ NAT Gateway     │  │          │ │
│  └─────────────────┘  │ App      │ │
│                       │ Servers  │ │
│                       │ Database │ │
│                       └──────────┘ │
└─────────────────────────────────────┘

Public subnet: Directly accessible from the internet (load balancers, bastion hosts).

Private subnet: No direct internet access (databases, application servers). Uses NAT gateway for outbound internet.

Security Groups

Security groups are virtual firewalls for cloud resources:

Security Group: web-servers
  Inbound:
    TCP 80 from 0.0.0.0/0
    TCP 443 from 0.0.0.0/0
    TCP 22 from 10.0.0.0/16 (internal only)
  Outbound:
    All traffic allowed

Security Group: database
  Inbound:
    TCP 5432 from sg-web-servers (reference other SG)
  Outbound:
    None (database doesn't need to initiate connections)

Practical Debugging Commands

When something goes wrong, these are your tools:

# Check if a host is reachable
ping google.com

# Trace the route to a destination
traceroute google.com

# Check DNS resolution
nslookup google.com
dig google.com

# Check what ports are open
nmap -sT google.com

# Check your network interface
ip addr show

# Check routing table
ip route show

# Check active connections
ss -tuln

# Capture network traffic
tcpdump -i eth0 port 80

# Check if a port is listening
netstat -tuln | grep :80

Conclusion

The internet is a stack of abstractions. At the bottom, it’s light pulses in fiber optic cables. Above that, MAC addresses and Ethernet frames. Then IP addresses and routing. Then TCP/UDP for reliable delivery. Then HTTP, WebSocket, and application protocols.

Understanding each layer makes you a better engineer. When something breaks, you know exactly where to look. When you’re designing a system, you understand the tradeoffs at every level.

The next time you deploy a web application, you’ll know exactly what happens between “git push” and your user seeing it live.