User Tools

Site Tools


skills:network

This is an old revision of the document!


Network & Messaging Cheat Sheet

Network Models

OSI Model TCP/IP Model Purpose
Layer 7: Application Application Application protocols
Layer 6: Presentation Application Serialization, encryption
Layer 5: Session Application Session management
Layer 4: Transport Transport End-to-end communication
Layer 3: Network Internet Routing
Layer 2: Data Link Link Local network communication
Layer 1: Physical Physical Signal transmission

Layer 7: Application

Purpose: Define application rules and message formats.

Application-layer protocols

These protocols operate at the Application Layer (Layer 7) and define how applications exchange data.

Protocol Full Name Typical Transport Layer Common Use Cases
HTTP HyperText Transfer Protocol TCP Web pages, REST APIs
HTTPS HTTP over TLS/SSL TCP Secure web pages, secure APIs
WebSocket Full-duplex web communication protocol TCP Real-time chat, notifications, live updates
RESP Redis Serialization Protocol TCP Communication between Redis clients and servers
AMQP Advanced Message Queuing Protocol TCP Message brokers such as RabbitMQ
Kafka Protocol Apache Kafka binary protocol TCP Event streaming and distributed messaging
SMTP Simple Mail Transfer Protocol TCP Sending email
FTP File Transfer Protocol TCP File uploads and downloads
DNS Domain Name System UDP (mostly), TCP (sometimes) Domain name resolution
MQTT Message Queuing Telemetry Transport TCP IoT devices and lightweight messaging

Notes

* These are application-layer protocols, not transport protocols. * Most application protocols rely on TCP because they require reliable and ordered delivery. * DNS primarily uses UDP for low latency, but switches to TCP for large responses and zone transfers. * HTTPS is essentially:

HTTPS = HTTP + TLS + TCP

* WebSocket starts with an HTTP handshake, then upgrades to a persistent TCP connection.

Client
  ↓
Application Protocol (HTTP, MQTT, AMQP, RESP)
  ↓
Transport Protocol (TCP or UDP)
  ↓
IP
  ↓
Ethernet / Wi-Fi

Examples

HTTP/1.1 → TCP → IP → Ethernet

WebSocket → TCP → IP → Ethernet

DNS → UDP → IP → Ethernet

MQTT → TCP → IP → Ethernet

Kafka → TCP → IP → Ethernet

HTTPS → TLS → TCP → IP → Ethernet 

Examples of messages:

GET /orders HTTP/1.1
XREAD BLOCK 2000 STREAMS orders $
Fetch Request

Layer 6: Presentation

Purpose: Format, serialize, compress, and encrypt data.

Examples:

* JSON * XML * Protobuf * Avro * UTF-8 * Gzip * TLS/SSL

Example:

HTTP + TLS = HTTPS

Layer 5: Session

Purpose: Establish, maintain, and terminate sessions.

Examples:

* HTTP Keep-Alive * WebSocket Session * Session IDs * Authentication Sessions

Layer 4: Transport

Purpose: Deliver data between processes.

Transport-layer protocols.

TCP

* Connection-oriented * Reliable delivery * Ordered messages * Retransmission support

Used by:

* HTTP/1.1 * HTTP/2 * WebSocket * Redis * RabbitMQ * Kafka

UDP

* Connectionless * Faster * No delivery guarantee

Used by:

* DNS * VoIP * Online games * WebRTC * QUIC (HTTP/3)

Layer 3: Network

Purpose: Route packets between networks.

Protocols:

* IPv4 * IPv6 * ICMP

Example IP:

192.168.1.10

Devices:

* Router

Purpose: Transfer frames within a local network.

Protocols:

* Ethernet * Wi-Fi * ARP

Example MAC address:

AA:BB:CC:DD:EE:FF

Devices:

* Switch

Layer 1: Physical

Purpose: Transmit raw bits.

Examples:

* Network cables * Fiber optics * Radio waves * Electrical signals

010101010101

Connection vs Protocol

Protocol   = language
Connection = communication channel

Examples:

HTTP      + TCP
WebSocket + TCP
RESP      + TCP
AMQP      + TCP
Kafka     + TCP
HTTP/3    + QUIC + UDP

Messaging Systems

System Application Protocol Transport Communication Model
Redis Streams RESP TCP Pull + Long Polling
Redis Pub/Sub RESP TCP Push
RabbitMQ AMQP TCP Push
Kafka Kafka Protocol TCP Pull + Long Polling

Who Calls Who?

Database Polling

Worker → Database

Worker repeatedly asks:

SELECT * FROM jobs
WHERE STATUS = 'pending';

Redis Streams

Worker → Redis

Worker sends:

XREAD BLOCK 2000

Redis waits up to 2 seconds, then replies.

Redis Pub/Sub

Worker → Redis (SUBSCRIBE)
Redis → Worker (messages)

Redis pushes messages to subscribers.

RabbitMQ

Worker → RabbitMQ (basic.consume)
RabbitMQ → Worker (messages)

RabbitMQ pushes messages.

Kafka

Worker → Kafka (Fetch Request)
Kafka → Worker (response)

Consumer repeatedly sends fetch requests.

TCP Connection Behavior

System Persistent TCP Repeated Requests Server Push
Database Polling Optional Yes No
Redis Streams Yes Yes (XREAD) No
Redis Pub/Sub Yes No Yes
RabbitMQ Yes No Yes
Kafka Yes Yes (Fetch Request) No

Redis Streams

XREAD BLOCK

Example:

XREAD BLOCK 2000 STREAMS orders $

Meaning:

* Read from stream orders * Wait up to 2000 ms if there is no data * Return immediately when a message arrives * Reuse the same TCP connection

Flow:

TCP Connection #1
├── XREAD BLOCK 2000
├── XREAD BLOCK 2000
├── XREAD BLOCK 2000
└── ...

Kafka

Fetch Request

Consumer sends:

Fetch messages from:
- topic: orders
- partition: 0
- offset: 123

Kafka:

* Waits for new data * Returns immediately if data exists * Returns empty response after timeout * Keeps using the same TCP connection

Flow:

TCP Connection #1
├── Fetch Request
├── Fetch Request
├── Fetch Request
└── ...

HTTP vs WebSocket

Feature HTTP/1.1 HTTP/2 WebSocket
TCP Yes Yes Yes
Persistent Connection Keep-Alive Yes Yes
Multiplexing No Yes N/A
Client Initiates Yes Yes Initial only
Server Push Anytime No No
Full Duplex No No Yes

HTTP/3

HTTP/1.1 → TCP
HTTP/2   → TCP
HTTP/3   → QUIC → UDP

Key Takeaways

* TCP provides reliable communication. * Application protocols define message meaning. * Redis Streams and Kafka use pull + long polling. * Redis Pub/Sub and RabbitMQ use push. * WebSocket provides bidirectional communication. * HTTP/2 is not WebSocket. * HTTP/3 uses QUIC over UDP.

skills/network.1781702716.txt.gz · Last modified: by phong2018