User Tools

Site Tools


skills:network

Network & Messaging Cheat Sheet

Network Models

OSI Model TCP/IP Model Purpose
Layer 7: Application Application Application protocols
Layer 6: Presentation Application Serialization, encryption
Layer 5: Session Application Session management
Layer 4: Transport Transport End-to-end communication
Layer 3: Network Internet Routing
Layer 2: Data Link Link Local network communication
Layer 1: Physical Physical Signal transmission

Layer 7: Application

Purpose: Define application rules and message formats.

Application-layer protocols

These protocols operate at the Application Layer (Layer 7) and define how applications exchange data.

Protocol Full Name Typical Transport Layer Common Use Cases
HTTP HyperText Transfer Protocol TCP Web pages, REST APIs
HTTPS HTTP over TLS/SSL TCP Secure web pages, secure APIs
WebSocket Full-duplex web communication protocol TCP Real-time chat, notifications, live updates
RESP Redis Serialization Protocol TCP Communication between Redis clients and servers
AMQP Advanced Message Queuing Protocol TCP Message brokers such as RabbitMQ
Kafka Protocol Apache Kafka binary protocol TCP Event streaming and distributed messaging
SMTP Simple Mail Transfer Protocol TCP Sending email
FTP File Transfer Protocol TCP File uploads and downloads
DNS Domain Name System UDP (mostly), TCP (sometimes) Domain name resolution
MQTT Message Queuing Telemetry Transport TCP IoT devices and lightweight messaging

Notes

* These are application-layer protocols, not transport protocols. * Most application protocols rely on TCP because they require reliable and ordered delivery. * DNS primarily uses UDP for low latency, but switches to TCP for large responses and zone transfers. * HTTPS is essentially:

HTTPS = HTTP + TLS + TCP

* WebSocket starts with an HTTP handshake, then upgrades to a persistent TCP connection.

Client
  ↓
Application Protocol (HTTP, MQTT, AMQP, RESP)
  ↓
Transport Protocol (TCP or UDP)
  ↓
IP
  ↓
Ethernet / Wi-Fi

Examples

HTTP/1.1 → TCP → IP → Ethernet

WebSocket → TCP → IP → Ethernet

DNS → UDP → IP → Ethernet

MQTT → TCP → IP → Ethernet

Kafka → TCP → IP → Ethernet

HTTPS → TLS → TCP → IP → Ethernet 

Examples of messages:

GET /orders HTTP/1.1
XREAD BLOCK 2000 STREAMS orders $
Fetch Request

Layer 6: Presentation

Purpose: Format, serialize, compress, and encrypt data.

Examples:

* JSON * XML * Protobuf * Avro * UTF-8 * Gzip * TLS/SSL

Example:

HTTP + TLS = HTTPS

Layer 5: Session

Purpose: Establish, maintain, and terminate sessions.

Examples:

* HTTP Keep-Alive * WebSocket Session * Session IDs * Authentication Sessions

Layer 4: Transport

Purpose: Deliver data between processes.

Feature TCP UDP
Full Name Transmission Control Protocol User Datagram Protocol
Connection Type Connection-oriented Connectionless
Reliability Reliable delivery with ACK and retransmission Best effort, no delivery guarantee
Message Ordering Preserves packet order No ordering guarantee
Error Recovery Automatic retransmission No retransmission
Flow Control Yes (sliding window) No
Congestion Control Yes No
Overhead Higher Lower
Latency Higher Lower
Speed Slower due to reliability mechanisms Faster due to minimal overhead
Header Size 20–60 bytes 8 bytes
Communication Style Continuous connection between client and server Independent datagrams
Typical Use Cases Web applications, databases, messaging systems Real-time communication, gaming, streaming
Common Protocols and Applications Transport Layer
HTTP/1.1 TCP
HTTP/2 TCP
WebSocket TCP
Redis (RESP) TCP
RabbitMQ (AMQP) TCP
Apache Kafka Protocol TCP
SMTP TCP
FTP TCP
SSH TCP
DNS UDP (primarily), TCP (zone transfer and large responses)
VoIP UDP
Online Games UDP
WebRTC UDP
QUIC UDP
HTTP/3 QUIC over UDP
Summary TCP UDP
Main Goal Reliability and correctness Speed and low latency
Best For Data that must not be lost Data that must arrive quickly
Example File download, API requests, email Video call, live stream, multiplayer game
Real-world Analogy Registered mail with delivery confirmation Radio broadcast

Layer 3: Network

Purpose: Route packets between networks.

Protocols:

* IPv4 * IPv6 * ICMP

Example IP:

192.168.1.10

Devices:

* Router

Purpose: Transfer frames within a local network.

Protocols:

* Ethernet * Wi-Fi * ARP

Example MAC address:

AA:BB:CC:DD:EE:FF

Devices:

* Switch

Layer 1: Physical

Purpose: Transmit raw bits.

Examples:

* Network cables * Fiber optics * Radio waves * Electrical signals

010101010101

Connection vs Protocol

Protocol   = language
Connection = communication channel

Examples:

HTTP      + TCP
WebSocket + TCP
RESP      + TCP
AMQP      + TCP
Kafka     + TCP
HTTP/3    + QUIC + UDP

Messaging Systems

System Application Protocol Transport Communication Model
Redis Streams RESP TCP Pull + Long Polling
Redis Pub/Sub RESP TCP Push
RabbitMQ AMQP TCP Push
Kafka Kafka Protocol TCP Pull + Long Polling

Who Calls Who?

Database Polling

Worker → Database

Worker repeatedly asks:

SELECT * FROM jobs
WHERE STATUS = 'pending';

Redis Streams

Worker → Redis

Worker sends:

XREAD BLOCK 2000

Redis waits up to 2 seconds, then replies.

Redis Pub/Sub

Worker → Redis (SUBSCRIBE)
Redis → Worker (messages)

Redis pushes messages to subscribers.

RabbitMQ

Worker → RabbitMQ (basic.consume)
RabbitMQ → Worker (messages)

RabbitMQ pushes messages.

Kafka

Worker → Kafka (Fetch Request)
Kafka → Worker (response)

Consumer repeatedly sends fetch requests.

TCP Connection Behavior

System Persistent TCP Repeated Requests Server Push
Database Polling Optional Yes No
Redis Streams Yes Yes (XREAD) No
Redis Pub/Sub Yes No Yes
RabbitMQ Yes No Yes
Kafka Yes Yes (Fetch Request) No

Redis Streams

XREAD BLOCK

Example:

XREAD BLOCK 2000 STREAMS orders $

Meaning:

* Read from stream orders * Wait up to 2000 ms if there is no data * Return immediately when a message arrives * Reuse the same TCP connection

Flow:

TCP Connection #1
├── XREAD BLOCK 2000
├── XREAD BLOCK 2000
├── XREAD BLOCK 2000
└── ...

Kafka

Fetch Request

Consumer sends:

Fetch messages from:
- topic: orders
- partition: 0
- offset: 123

Kafka:

* Waits for new data * Returns immediately if data exists * Returns empty response after timeout * Keeps using the same TCP connection

Flow:

TCP Connection #1
├── Fetch Request
├── Fetch Request
├── Fetch Request
└── ...

HTTP vs WebSocket

Feature HTTP/1.1 HTTP/2 WebSocket
TCP Yes Yes Yes
Persistent Connection Keep-Alive Yes Yes
Multiplexing No Yes N/A
Client Initiates Yes Yes Initial only
Server Push Anytime No No
Full Duplex No No Yes

HTTP/3

HTTP/1.1 → TCP
HTTP/2   → TCP
HTTP/3   → QUIC → UDP

Key Takeaways

* TCP provides reliable communication. * Application protocols define message meaning. * Redis Streams and Kafka use pull + long polling. * Redis Pub/Sub and RabbitMQ use push. * WebSocket provides bidirectional communication. * HTTP/2 is not WebSocket. * HTTP/3 uses QUIC over UDP.

skills/network.txt · Last modified: by phong2018