===== Reading/Writing patterns in Go: one-shot vs stream vs buffered =====
==== Goal ====
Help you choose the right approach when reading/writing data in Go:
* **One-shot** (read/write all at once)
* **Stream (unbuffered)** (read/write piece by piece)
* **Stream (buffered with bufio)** (stream + RAM buffer for speed)
==== 1) One-shot (read all / write all) ====
**Meaning:** load the whole content into memory (RAM) in one go.
=== Common functions ===
* `os.ReadFile(path)` → `[]byte`
* `io.ReadAll(r io.Reader)` → `[]byte`
* `os.WriteFile(path, data, perm)` → write all bytes at once
=== When to use ===
* File/data is small enough to fit comfortably in RAM.
* You want simplest code.
=== Example ===
b, err := os.ReadFile("data.txt")
if err != nil { return err }
// use b...
==== 2) Stream (unbuffered) ====
**Meaning:** read/write gradually (chunks). You do NOT load everything into RAM.
=== Common functions / methods ===
* `f.Read(p []byte)` / `f.Write(p []byte)` (methods on `*os.File`, `net.Conn`, etc.)
* `io.Copy(dst, src)` (stream copy)
* `io.CopyN(dst, src, n)` (copy exactly n bytes)
* `io.ReadFull(r, buf)` (fill buf or error)
=== When to use ===
* Large files, network streams, unknown size.
* You want low memory usage.
=== Example (read chunk-by-chunk) ===
buf := make([]byte, 4096)
for {
n, err := f.Read(buf)
if n > 0 {
// process buf[:n]
}
if err == io.EOF {
break
}
if err != nil {
return err
}
}
==== 3) Stream (buffered with bufio) ====
**Meaning:** still stream-based, but `bufio` keeps an in-memory **buffer** to reduce system calls.
It reads/writes larger blocks internally, while you consume/produce smaller pieces.
=== Common functions ===
* `bufio.NewReader(r)` / `bufio.NewReaderSize(r, size)`
* `bufio.NewWriter(w)` / `bufio.NewWriterSize(w, size)`
* Reader helpers: `ReadString('\n')`, `ReadBytes('\n')`, `ReadLine()`
* Writer helpers: `WriteString(...)`, `Write(...)`, then `Flush()`
=== When to use ===
* Reading lines (text), lots of small reads/writes.
* Network/file I/O where you read/write frequently in small pieces.
=== Example (read lines) ===
rd := bufio.NewReader(f)
for {
line, err := rd.ReadString('\n')
if len(line) > 0 {
// use line
}
if err == io.EOF {
break
}
if err != nil {
return err
}
}
=== Example (buffered writer) ===
wr := bufio.NewWriter(f)
_, _ = wr.WriteString("hello\n")
_ = wr.Flush() // important!
==== 4) Stream token/line scanning (bufio.Scanner) ====
**Meaning:** convenient tokenization (often line-by-line), great for simple text parsing.
=== Common functions ===
* `scanner := bufio.NewScanner(r)`
* `scanner.Scan()` (loop)
* `scanner.Text()` / `scanner.Bytes()`
* `scanner.Split(bufio.ScanLines)` (default) or other split funcs
=== When to use ===
* Simple line-based reading.
* Quick parsing tasks.
=== Pitfall ===
`Scanner` has a default max token size. For very long lines, configure buffer or use `bufio.Reader`.
==== Quick decision table ====
^ Your case ^ Recommended ^
| Small file, want simplest | `os.ReadFile` / `os.WriteFile` |
| Large/unknown size stream | `io.Copy` or `Read` loop |
| Many small reads/writes | `bufio.Reader` / `bufio.Writer` |
| Read line-by-line simply | `bufio.Scanner` (or `Reader.ReadString`) |
==== Why bufio is faster (in one sentence) ====
Because it reduces expensive OS calls by doing fewer, bigger reads/writes under the hood.
==== Related pages ====
* [[go:stdlib:bufio|bufio package]]
* [[go:stdlib:io|io package]]
* [[go:stdlib:os_open|os.Open]]
* [[go:stdlib:os_readfile|os.ReadFile]]
* [[go:stdlib:os_writefile|os.WriteFile]]
==== Hard words (English) ====
* **one-shot** /ˌwʌn ˈʃɑːt/: làm một phát (đọc/ghi hết)
* **stream** /striːm/: luồng dữ liệu
* **chunk** /tʃʌŋk/: khối dữ liệu
* **buffer** /ˈbʌfər/: bộ đệm (vùng RAM tạm)
* **unbuffered** /ʌnˈbʌfərd/: không dùng bộ đệm
* **token** /ˈtoʊkən/: đơn vị tách (mảnh dữ liệu)
* **system call** /ˈsɪstəm kɔːl/: lời gọi hệ điều hành
* **under the hood** /ˈʌndər ðə hʊd/: ở bên trong (cơ chế nội bộ)