===== Reading/Writing patterns in Go: one-shot vs stream vs buffered ===== ==== Goal ==== Help you choose the right approach when reading/writing data in Go: * **One-shot** (read/write all at once) * **Stream (unbuffered)** (read/write piece by piece) * **Stream (buffered with bufio)** (stream + RAM buffer for speed) ==== 1) One-shot (read all / write all) ==== **Meaning:** load the whole content into memory (RAM) in one go. === Common functions === * `os.ReadFile(path)` → `[]byte` * `io.ReadAll(r io.Reader)` → `[]byte` * `os.WriteFile(path, data, perm)` → write all bytes at once === When to use === * File/data is small enough to fit comfortably in RAM. * You want simplest code. === Example === b, err := os.ReadFile("data.txt") if err != nil { return err } // use b... ==== 2) Stream (unbuffered) ==== **Meaning:** read/write gradually (chunks). You do NOT load everything into RAM. === Common functions / methods === * `f.Read(p []byte)` / `f.Write(p []byte)` (methods on `*os.File`, `net.Conn`, etc.) * `io.Copy(dst, src)` (stream copy) * `io.CopyN(dst, src, n)` (copy exactly n bytes) * `io.ReadFull(r, buf)` (fill buf or error) === When to use === * Large files, network streams, unknown size. * You want low memory usage. === Example (read chunk-by-chunk) === buf := make([]byte, 4096) for { n, err := f.Read(buf) if n > 0 { // process buf[:n] } if err == io.EOF { break } if err != nil { return err } } ==== 3) Stream (buffered with bufio) ==== **Meaning:** still stream-based, but `bufio` keeps an in-memory **buffer** to reduce system calls. It reads/writes larger blocks internally, while you consume/produce smaller pieces. === Common functions === * `bufio.NewReader(r)` / `bufio.NewReaderSize(r, size)` * `bufio.NewWriter(w)` / `bufio.NewWriterSize(w, size)` * Reader helpers: `ReadString('\n')`, `ReadBytes('\n')`, `ReadLine()` * Writer helpers: `WriteString(...)`, `Write(...)`, then `Flush()` === When to use === * Reading lines (text), lots of small reads/writes. * Network/file I/O where you read/write frequently in small pieces. === Example (read lines) === rd := bufio.NewReader(f) for { line, err := rd.ReadString('\n') if len(line) > 0 { // use line } if err == io.EOF { break } if err != nil { return err } } === Example (buffered writer) === wr := bufio.NewWriter(f) _, _ = wr.WriteString("hello\n") _ = wr.Flush() // important! ==== 4) Stream token/line scanning (bufio.Scanner) ==== **Meaning:** convenient tokenization (often line-by-line), great for simple text parsing. === Common functions === * `scanner := bufio.NewScanner(r)` * `scanner.Scan()` (loop) * `scanner.Text()` / `scanner.Bytes()` * `scanner.Split(bufio.ScanLines)` (default) or other split funcs === When to use === * Simple line-based reading. * Quick parsing tasks. === Pitfall === `Scanner` has a default max token size. For very long lines, configure buffer or use `bufio.Reader`. ==== Quick decision table ==== ^ Your case ^ Recommended ^ | Small file, want simplest | `os.ReadFile` / `os.WriteFile` | | Large/unknown size stream | `io.Copy` or `Read` loop | | Many small reads/writes | `bufio.Reader` / `bufio.Writer` | | Read line-by-line simply | `bufio.Scanner` (or `Reader.ReadString`) | ==== Why bufio is faster (in one sentence) ==== Because it reduces expensive OS calls by doing fewer, bigger reads/writes under the hood. ==== Related pages ==== * [[go:stdlib:bufio|bufio package]] * [[go:stdlib:io|io package]] * [[go:stdlib:os_open|os.Open]] * [[go:stdlib:os_readfile|os.ReadFile]] * [[go:stdlib:os_writefile|os.WriteFile]] ==== Hard words (English) ==== * **one-shot** /ˌwʌn ˈʃɑːt/: làm một phát (đọc/ghi hết) * **stream** /striːm/: luồng dữ liệu * **chunk** /tʃʌŋk/: khối dữ liệu * **buffer** /ˈbʌfər/: bộ đệm (vùng RAM tạm) * **unbuffered** /ʌnˈbʌfərd/: không dùng bộ đệm * **token** /ˈtoʊkən/: đơn vị tách (mảnh dữ liệu) * **system call** /ˈsɪstəm kɔːl/: lời gọi hệ điều hành * **under the hood** /ˈʌndər ðə hʊd/: ở bên trong (cơ chế nội bộ)