go:stdlib:reading_writing_patterns
Table of Contents
Reading/Writing patterns in Go: one-shot vs stream vs buffered
Goal
Help you choose the right approach when reading/writing data in Go:
- One-shot (read/write all at once)
- Stream (unbuffered) (read/write piece by piece)
- Stream (buffered with bufio) (stream + RAM buffer for speed)
1) One-shot (read all / write all)
Meaning: load the whole content into memory (RAM) in one go.
Common functions
- `os.ReadFile(path)` → `[]byte`
- `io.ReadAll(r io.Reader)` → `[]byte`
- `os.WriteFile(path, data, perm)` → write all bytes at once
When to use
- File/data is small enough to fit comfortably in RAM.
- You want simplest code.
Example
b, err := os.ReadFile("data.txt") if err != nil { return err } // use b...
2) Stream (unbuffered)
Meaning: read/write gradually (chunks). You do NOT load everything into RAM.
Common functions / methods
- `f.Read(p []byte)` / `f.Write(p []byte)` (methods on `*os.File`, `net.Conn`, etc.)
- `io.Copy(dst, src)` (stream copy)
- `io.CopyN(dst, src, n)` (copy exactly n bytes)
- `io.ReadFull(r, buf)` (fill buf or error)
When to use
- Large files, network streams, unknown size.
- You want low memory usage.
Example (read chunk-by-chunk)
buf := make([]byte, 4096) for { n, err := f.Read(buf) if n > 0 { // process buf[:n] } if err == io.EOF { break } if err != nil { return err } }
3) Stream (buffered with bufio)
Meaning: still stream-based, but `bufio` keeps an in-memory buffer to reduce system calls. It reads/writes larger blocks internally, while you consume/produce smaller pieces.
Common functions
- `bufio.NewReader®` / `bufio.NewReaderSize(r, size)`
- `bufio.NewWriter(w)` / `bufio.NewWriterSize(w, size)`
- Reader helpers: `ReadString('\n')`, `ReadBytes('\n')`, `ReadLine()`
- Writer helpers: `WriteString(…)`, `Write(…)`, then `Flush()`
When to use
- Reading lines (text), lots of small reads/writes.
- Network/file I/O where you read/write frequently in small pieces.
Example (read lines)
rd := bufio.NewReader(f) for { line, err := rd.ReadString('\n') if len(line) > 0 { // use line } if err == io.EOF { break } if err != nil { return err } }
Example (buffered writer)
wr := bufio.NewWriter(f) _, _ = wr.WriteString("hello\n") _ = wr.Flush() // important!
4) Stream token/line scanning (bufio.Scanner)
Meaning: convenient tokenization (often line-by-line), great for simple text parsing.
Common functions
- `scanner := bufio.NewScanner®`
- `scanner.Scan()` (loop)
- `scanner.Text()` / `scanner.Bytes()`
- `scanner.Split(bufio.ScanLines)` (default) or other split funcs
When to use
- Simple line-based reading.
- Quick parsing tasks.
Pitfall
`Scanner` has a default max token size. For very long lines, configure buffer or use `bufio.Reader`.
Quick decision table
| Your case | Recommended |
|---|---|
| Small file, want simplest | `os.ReadFile` / `os.WriteFile` |
| Large/unknown size stream | `io.Copy` or `Read` loop |
| Many small reads/writes | `bufio.Reader` / `bufio.Writer` |
| Read line-by-line simply | `bufio.Scanner` (or `Reader.ReadString`) |
Why bufio is faster (in one sentence)
Because it reduces expensive OS calls by doing fewer, bigger reads/writes under the hood.
Related pages
Hard words (English)
- one-shot /ˌwʌn ˈʃɑːt/: làm một phát (đọc/ghi hết)
- stream /striːm/: luồng dữ liệu
- chunk /tʃʌŋk/: khối dữ liệu
- buffer /ˈbʌfər/: bộ đệm (vùng RAM tạm)
- unbuffered /ʌnˈbʌfərd/: không dùng bộ đệm
- token /ˈtoʊkən/: đơn vị tách (mảnh dữ liệu)
- system call /ˈsɪstəm kɔːl/: lời gọi hệ điều hành
- under the hood /ˈʌndər ðə hʊd/: ở bên trong (cơ chế nội bộ)
go/stdlib/reading_writing_patterns.txt · Last modified: by phong2018
