Java Buffered Streams
What Buffering Means
A buffer is a block of memory placed between a program and a slow data source or destination. Instead of passing each small read or write straight to the operating system, the buffered stream collects data in memory and moves it in large blocks:
- Buffered input: one large read from the source fills the buffer; many small
read()calls are then served from memory. - Buffered output: many small
write()calls fill the buffer; one large write sends it to the destination.
Every read or write that reaches the operating system (a system call) has a fixed overhead, regardless of size. Buffering turns millions of tiny system calls into a few hundred large ones.
Java has four buffered stream classes:
| Class | Wraps | Adds |
|---|---|---|
BufferedInputStream | An InputStream | Buffered byte reading, mark/reset support |
BufferedOutputStream | An OutputStream | Buffered byte writing |
BufferedReader | A Reader | Buffered character reading, readLine() (BufferedReader lesson) |
BufferedWriter | A Writer | Buffered character writing, newLine() (BufferedWriter lesson) |
This lesson focuses on the two byte streams and on the buffering rules that apply to all four. The default buffer size is 8,192 bytes (or characters); a size can be passed to the constructor.
Measuring the Difference
import java.io.*;
import java.nio.file.*;
public class Speed {
static long writeBytes(OutputStream out, int count) throws IOException {
long start = System.nanoTime();
for (int i = 0; i < count; i++) out.write(i);
out.flush();
return (System.nanoTime() - start) / 1_000_000;
}
static long readBytes(InputStream in) throws IOException {
long start = System.nanoTime();
while (in.read() != -1) { }
return (System.nanoTime() - start) / 1_000_000;
}
public static void main(String[] args) throws IOException {
int n = 2_000_000;
try (OutputStream raw = new FileOutputStream("raw.bin")) {
System.out.println("unbuffered write: " + writeBytes(raw, n) + " ms");
}
try (OutputStream buf = new BufferedOutputStream(new FileOutputStream("buffered.bin"))) {
System.out.println("buffered write: " + writeBytes(buf, n) + " ms");
}
try (InputStream raw = new FileInputStream("raw.bin")) {
System.out.println("unbuffered read: " + readBytes(raw) + " ms");
}
try (InputStream buf = new BufferedInputStream(new FileInputStream("buffered.bin"))) {
System.out.println("buffered read: " + readBytes(buf) + " ms");
}
System.out.println("files identical: " + (Files.mismatch(Path.of("raw.bin"), Path.of("buffered.bin")) == -1));
}
}
Output from one run (times vary with hardware, operating system and JVM warm-up, but the ratio is consistently large):
unbuffered write: 657 ms
buffered write: 59 ms
unbuffered read: 472 ms
buffered read: 52 ms
files identical: true
Both versions write and read the same 2,000,000 bytes one at a time and produce identical files. The only difference is the buffer: FileOutputStream.write(int) and FileInputStream.read() make one system call per byte, while the buffered versions make one per 8 KB — about 245 calls instead of 2 million. That's why the buffered versions are roughly ten times faster here, and the gap grows with slower storage or network connections.
When Buffering Helps — and When It Doesn't
Buffering helps when code performs many small operations:
- reading or writing single bytes or characters,
- parsing a format field by field (
DataInputStream.readInt()reads 4 bytes at a time), - writing output in small pieces as it's generated (rows, lines, tags),
- object serialization, which makes many tiny writes.
It adds little when code already works in large blocks:
in.read(buffer)/out.write(buffer, 0, n)with an 8 KB or larger array,transferTo,Files.copy,Files.readAllBytes,Files.readString,Files.write,- streams that are buffered already:
System.in,Files.newBufferedReader/Writer,BufferedReader/BufferedWriter. Wrapping a buffered stream in another buffer just copies the data an extra time.
How BufferedOutputStream Decides When to Write
import java.io.*;
import java.nio.file.*;
public class Flush {
public static void main(String[] args) throws IOException {
Path p = Path.of("events.bin");
BufferedOutputStream out = new BufferedOutputStream(new FileOutputStream(p.toFile()), 16);
out.write(new byte[10]);
System.out.println("after 10 bytes: file has " + Files.size(p));
out.write(new byte[10]);
System.out.println("after 20 bytes: file has " + Files.size(p) + " (buffer of 16 overflowed)");
out.write(new byte[100]);
System.out.println("after 120 bytes: file has " + Files.size(p) + " (large write went straight through)");
out.flush();
System.out.println("after flush: file has " + Files.size(p));
out.close();
}
}
Output:
after 10 bytes: file has 0
after 20 bytes: file has 10 (buffer of 16 overflowed)
after 120 bytes: file has 120 (large write went straight through)
after flush: file has 120
With a deliberately tiny 16-byte buffer, the rules become visible:
- The first 10 bytes fit in the buffer, so nothing reaches the file yet.
- The next 10 don't fit (10 + 10 > 16). The buffer's existing 10 bytes are written to the file, then the new 10 bytes are stored in the now-empty buffer.
- A write at least as large as the buffer bypasses it. The buffer's 10 waiting bytes are written first (keeping the order correct), and then the 100 bytes are written directly — copying them through the buffer would only waste time.
flush()writes whatever is still buffered (nothing here).close()flushes and then closes the file.
If the program ended without flush() or close(), any bytes still in the buffer would be lost — the file would be silently truncated. This is the main risk of buffered output, and why buffered streams always belong in try-with-resources.
mark and reset with BufferedInputStream
FileInputStream and socket streams can't go back. BufferedInputStream can, because the bytes are still in its buffer: mark(limit) remembers the current position, and reset() returns to it, as long as no more than limit bytes have been read since the mark.
import java.io.*;
public class MarkLimit {
public static void main(String[] args) throws IOException {
byte[] data = new byte[100];
for (int i = 0; i < data.length; i++) data[i] = (byte) i;
BufferedInputStream in = new BufferedInputStream(new ByteArrayInputStream(data), 8);
in.mark(4);
in.readNBytes(3);
in.reset();
System.out.println("reset within limit, next byte: " + in.read());
in.mark(4);
in.readNBytes(50);
try {
in.reset();
} catch (IOException e) {
System.out.println("reset after reading past the limit: " + e.getMessage());
}
}
}
Output:
reset within limit, next byte: 0
reset after reading past the limit: Resetting to invalid mark
- Reading 3 bytes after
mark(4)stays within the limit, soreset()goes back and the next byte is0again. - Reading 50 bytes after
mark(4)exceeds the limit; the buffer has moved on, andreset()throwsIOException. The buffer grows if needed to honour the mark limit, so choose a limit that covers what you actually read ahead — usually just a few bytes.
The main use is peeking: look at the first bytes of a stream to decide how to process it, then go back so the real reader sees the stream from the beginning.
Practical Example: Opening Plain or Compressed Logs Transparently
Log rotation compresses old log files (app.log.1.gz) while the current one stays plain (app.log). A tool that searches logs should handle both without the user caring which is which. GZIP data always starts with the bytes 1F 8B, so peeking at the first two bytes is enough to decide — but the GZIP decoder needs to read those same bytes again, which is exactly what mark/reset allow.
import java.io.*;
import java.nio.charset.StandardCharsets;
import java.nio.file.*;
import java.util.zip.*;
public class OpenLog {
/** Opens a log file for reading, transparently decompressing it if it is gzip-compressed. */
static BufferedReader openLog(Path file) throws IOException {
InputStream raw = new BufferedInputStream(Files.newInputStream(file));
try {
raw.mark(2);
int b1 = raw.read(), b2 = raw.read();
raw.reset();
boolean gzip = b1 == 0x1F && b2 == 0x8B;
InputStream content = gzip ? new GZIPInputStream(raw) : raw;
return new BufferedReader(new InputStreamReader(content, StandardCharsets.UTF_8));
} catch (IOException | RuntimeException e) {
raw.close();
throw e;
}
}
public static void main(String[] args) throws IOException {
String log = "09:00 INFO started\n09:01 ERROR db timeout\n09:02 INFO recovered\n";
Files.writeString(Path.of("app.log"), log);
try (OutputStream gz = new GZIPOutputStream(Files.newOutputStream(Path.of("app.log.1.gz")))) {
gz.write("08:00 INFO started\n08:30 ERROR disk full\n".getBytes(StandardCharsets.UTF_8));
}
for (String name : new String[] {"app.log", "app.log.1.gz"}) {
try (BufferedReader r = openLog(Path.of(name))) {
long errors = r.lines().filter(l -> l.contains(" ERROR ")).count();
System.out.println(name + ": " + errors + " error(s)");
}
}
}
}
Output:
app.log: 1 error(s)
app.log.1.gz: 1 error(s)
Step by step:
BufferedInputStreammakes peeking possible. The file stream itself can't go back, but the buffered wrapper keeps the first bytes in memory.- Peek two bytes, then reset.
mark(2)covers exactly the two bytes read. Afterreset(), the stream is back at the very beginning, so nothing is lost for the reader that follows. - Decide by content, not by name. If the bytes are the GZIP signature, the stream is wrapped in a
GZIPInputStream; otherwise it's used as it is. A renamed or misnamed file is still handled correctly. (TheGZIPInputStreamconstructor reads the header immediately, and it can do so because the bytes were pushed back.) - The rest of the chain is the same for both cases:
InputStreamReaderdecodes UTF-8 andBufferedReaderprovides lines, so the calling code — countingERRORlines — doesn't know or care whether the file was compressed. - Closing on failure. If anything fails after the file was opened (for example a corrupt GZIP header), the
catchblock closes the file before rethrowing, so no file handle leaks. On success, ownership passes to the caller, whose try-with-resources closes the whole chain — closing theBufferedReadercloses every stream beneath it.
Files.probeContentType couldn't do this reliably, since it usually looks only at the file name. Peeking at the content with a buffered stream is the robust technique, used by tools such as zgrep and by libraries that auto-detect image and archive formats.
Buffered Streams and Threads
BufferedInputStream and BufferedOutputStream are thread-safe in the sense that their methods are internally synchronized, so two threads writing to the same buffered stream won't corrupt its buffer. They don't make sequences of writes atomic, though: if two threads each write several pieces of a record, the pieces can interleave. Code that shares one output stream between threads should build each complete record first and write it with a single call, or coordinate the writes explicitly.