Java NIO Basics

Su Suriya Ravichandran Updated 18 Nov 2026
14 min read ·Lesson 147 of 306

What NIO Is

NIO ("New I/O") is a second set of I/O APIs that Java added alongside the original java.io streams. It arrived in two stages:

ReleasePackage(s)What it added
Java 1.4 (2002) — NIOjava.nio, java.nio.channels, java.nio.charsetBuffers, channels, charsets, memory-mapped files, file locking, non-blocking network I/O with selectors
Java 7 (2011) — NIO.2java.nio.filePath, Files, directory walking, file attributes, symbolic links, WatchService

The previous lessons already used NIO.2 (Path and Files). This lesson covers the original NIO building blocks underneath it: buffers and channels.

Why they exist: classic streams move data one byte or one array at a time, in one direction, blocking until the operation finishes. That model is simple, but it doesn't allow some things high-performance programs need:

  • reading and writing at any position in a file without reopening it,
  • reusing one block of memory for many operations, including memory outside the Java heap,
  • mapping a file directly into memory,
  • locking regions of a file against other processes,
  • handling thousands of network connections with a few threads (non-blocking I/O).

Most application code never touches buffers or channels directly — Files.readString and BufferedReader are simpler. But frameworks and servers (Netty, Tomcat, Kafka, databases) are built on them, and you need them for random-access binary files, large file processing, and anything performance-critical.

Buffers

A buffer is a fixed-size block of memory for one primitive type — ByteBuffer, CharBuffer, IntBuffer, LongBuffer and so on. ByteBuffer is by far the most important, because channels read and write bytes.

A buffer tracks three positions:

PropertyMeaning
capacityTotal size, fixed when the buffer is created
positionIndex of the next element to read or write
limitThe first index that must not be read or written

remaining() is limit - position: how much can still be read or written. Every relative put or get advances the position. The key to using buffers is understanding how a few methods move these markers.

import java.nio.ByteBuffer;

public class Buf {
    static void show(String step, ByteBuffer b) {
        System.out.printf("%-22s position=%d limit=%d capacity=%d remaining=%d%n",
            step, b.position(), b.limit(), b.capacity(), b.remaining());
    }

    public static void main(String[] args) {
        ByteBuffer buf = ByteBuffer.allocate(8);
        show("allocate(8)", buf);

        buf.put((byte) 'J').put((byte) 'a').put((byte) 'v').put((byte) 'a');
        show("put 4 bytes", buf);

        buf.flip();
        show("flip()", buf);

        System.out.println("get: " + (char) buf.get() + (char) buf.get());
        show("get 2 bytes", buf);

        buf.compact();
        show("compact()", buf);

        buf.clear();
        show("clear()", buf);
    }
}

Output:

allocate(8)            position=0 limit=8 capacity=8 remaining=8
put 4 bytes            position=4 limit=8 capacity=8 remaining=4
flip()                 position=0 limit=4 capacity=8 remaining=4
get: Ja
get 2 bytes            position=2 limit=4 capacity=8 remaining=2
compact()              position=2 limit=8 capacity=8 remaining=6
clear()                position=0 limit=8 capacity=8 remaining=8

Step by step:

  1. allocate(8) creates an empty buffer in write mode: position 0, limit = capacity.
  2. put writes 4 bytes; the position moves to 4.
  3. flip() switches to read mode: the limit is set to the current position (4 — the amount of data written), and the position goes back to 0. Now get reads exactly the data that was put in.
  4. get reads two bytes; the position moves to 2. Two bytes (va) remain unread.
  5. compact() moves the unread bytes to the start of the buffer and switches back to write mode, with the position after them (2). Use it when you've only partly consumed the data and want to add more after it.
  6. clear() resets to write mode with position 0 and limit = capacity. It doesn't erase the bytes; they're simply overwritten by the next writes.

The most common buffer bug is forgetting flip() after filling a buffer: the position stays at the end of the data, so a following read or channel write sees nothing (or garbage after the data). The cycle to remember: fill → flip → drain → clear (or compact) → fill again.

Other useful methods:

  • ByteBuffer.wrap(byte[]) creates a buffer around an existing array, already in read mode (position 0, limit = array length).
  • get(int index) and put(int index, byte b) are absolute operations that don't move the position.
  • rewind() sets the position to 0 without changing the limit, to read the same data again.

Multi-byte values and byte order

ByteBuffer can read and write numbers, which makes it the standard tool for binary formats and protocols:

import java.nio.*;

public class Order {
    public static void main(String[] args) {
        ByteBuffer big = ByteBuffer.allocate(4).putInt(0x0A0B0C0D);
        ByteBuffer little = ByteBuffer.allocate(4).order(ByteOrder.LITTLE_ENDIAN).putInt(0x0A0B0C0D);
        System.out.printf("big-endian:    %02X %02X %02X %02X%n", big.get(0), big.get(1), big.get(2), big.get(3));
        System.out.printf("little-endian: %02X %02X %02X %02X%n", little.get(0), little.get(1), little.get(2), little.get(3));
        System.out.println("native order:  " + ByteOrder.nativeOrder());

        ByteBuffer header = ByteBuffer.allocate(14).putShort((short) 7).putLong(1_794_635_670_123L).putInt(42);
        header.flip();
        System.out.println("read back: version=" + header.getShort() + " time=" + header.getLong() + " count=" + header.getInt());
    }
}

Output (the native order is LITTLE_ENDIAN on x86 and most ARM machines):

big-endian:    0A 0B 0C 0D
little-endian: 0D 0C 0B 0A
native order:  LITTLE_ENDIAN
read back: version=7 time=1794635670123 count=42
  • A ByteBuffer is big-endian by default (most significant byte first), which matches network protocols and Java's own DataOutputStream. order(ByteOrder.LITTLE_ENDIAN) switches to the order used by formats like BMP, ZIP and most CPU architectures.
  • putShort, putInt, putLong, putDouble and the matching get methods convert numbers to and from 2, 4 and 8 bytes. The 14-byte header is 2 + 8 + 4 bytes, and reading it back in the same order with the same byte order returns the original values.

Heap and direct buffers

ByteBuffer.allocate(n) creates a buffer backed by a normal Java byte[] on the heap. ByteBuffer.allocateDirect(n) allocates memory outside the heap, which the operating system can read and write directly, avoiding an extra copy during I/O. Direct buffers are more expensive to create and are freed only when the garbage collector reclaims the buffer object, so they're best allocated once and reused — the pattern network servers use. For ordinary code, heap buffers are fine.

Channels

A channel is a connection to an I/O source — a file, a socket — that reads into and writes from buffers. Unlike streams, a single channel can often both read and write, and a FileChannel can work at any position in the file.

import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.charset.StandardCharsets;
import java.nio.file.*;
import static java.nio.file.StandardOpenOption.*;

public class Chan {
    public static void main(String[] args) throws IOException {
        Path file = Path.of("channel.txt");
        try (FileChannel ch = FileChannel.open(file, CREATE, TRUNCATE_EXISTING, WRITE)) {
            ByteBuffer buf = ByteBuffer.wrap("Hello from FileChannel\n".getBytes(StandardCharsets.UTF_8));
            int written = ch.write(buf);
            System.out.println("written " + written + " bytes, channel size " + ch.size());
        }

        try (FileChannel ch = FileChannel.open(file, READ)) {
            ByteBuffer buf = ByteBuffer.allocate(5);
            StringBuilder text = new StringBuilder();
            int reads = 0;
            while (ch.read(buf) != -1) {
                reads++;
                buf.flip();
                text.append(StandardCharsets.UTF_8.decode(buf));
                buf.clear();
            }
            System.out.print("read in " + reads + " calls: " + text);
        }
    }
}

Output:

written 23 bytes, channel size 23
read in 5 calls: Hello from FileChannel
  • FileChannel.open(path, options...) opens a channel with the same open options as Files methods. FileInputStream, FileOutputStream and RandomAccessFile also offer getChannel().
  • write(buffer) writes the buffer's remaining bytes and advances the buffer's position. A FileChannel writes all of them in practice, but the general WritableByteChannel contract allows partial writes, so robust code loops while (buf.hasRemaining()) — especially for network channels.
  • read(buffer) fills the buffer from the channel and returns the number of bytes read, or -1 at the end. With a deliberately tiny 5-byte buffer, the 23 bytes take 5 reads.
  • The read loop shows the buffer cycle in action: read (fill) → flip() → decode (drain) → clear().
  • Decoding 5-byte chunks separately works here only because the text is ASCII. A multi-byte UTF-8 character could be split across two chunks; correct code then uses a CharsetDecoder with compact() to carry the incomplete bytes over to the next read — or simply uses a Reader, which handles this for you.

Random access: reading and writing at any position

FileChannel has read(buffer, position) and write(buffer, position), which work at an absolute file position. This makes fixed-size record files straightforward: record n starts at byte n × recordSize, so any record can be read or updated without touching the others.

import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.charset.StandardCharsets;
import java.nio.file.*;
import static java.nio.file.StandardOpenOption.*;

public class Records {
    static final int NAME_BYTES = 20;
    static final int RECORD_SIZE = 4 + NAME_BYTES + 4;   // id + name + stock = 28 bytes

    static void write(FileChannel ch, int index, int id, String name, int stock) throws IOException {
        ByteBuffer buf = ByteBuffer.allocate(RECORD_SIZE);
        buf.putInt(id);
        byte[] nameBytes = name.getBytes(StandardCharsets.UTF_8);
        if (nameBytes.length > NAME_BYTES) throw new IllegalArgumentException("name too long: " + name);
        buf.put(nameBytes).put(new byte[NAME_BYTES - nameBytes.length]);
        buf.putInt(stock);
        buf.flip();
        ch.write(buf, (long) index * RECORD_SIZE);
    }

    static String read(FileChannel ch, int index) throws IOException {
        ByteBuffer buf = ByteBuffer.allocate(RECORD_SIZE);
        ch.read(buf, (long) index * RECORD_SIZE);
        buf.flip();
        int id = buf.getInt();
        byte[] nameBytes = new byte[NAME_BYTES];
        buf.get(nameBytes);
        String name = new String(nameBytes, StandardCharsets.UTF_8).replace("\0", "");
        int stock = buf.getInt();
        return id + " " + name + " stock=" + stock;
    }

    public static void main(String[] args) throws IOException {
        Path file = Path.of("inventory.dat");
        try (FileChannel ch = FileChannel.open(file, CREATE, TRUNCATE_EXISTING, READ, WRITE)) {
            write(ch, 0, 101, "Gel pen", 40);
            write(ch, 1, 102, "Notebook", 25);
            write(ch, 2, 103, "Laptop bag", 3);
            System.out.println("file size: " + ch.size() + " bytes");

            write(ch, 1, 102, "Notebook", 24);          // update record 1 in place
            for (int i = 0; i < ch.size() / RECORD_SIZE; i++) {
                System.out.println("record " + i + ": " + read(ch, i));
            }
        }
    }
}

Output:

file size: 84 bytes
record 0: 101 Gel pen stock=40
record 1: 102 Notebook stock=24
record 2: 103 Laptop bag stock=3

How it works:

  1. Each record is exactly 28 bytes: a 4-byte int id, a 20-byte name field and a 4-byte int stock. Three records make an 84-byte file.
  2. The name is padded with zero bytes to fill its 20-byte field; names longer than 20 bytes (in UTF-8) are rejected, because they'd overwrite the next field. When reading, the padding (\0 characters) is removed.
  3. The position is calculated as index * RECORD_SIZE, cast to long first, because file positions are long and an int multiplication could overflow for large files.
  4. The update rewrites only record 1's 28 bytes — the stock changes from 25 to 24 while the rest of the file is untouched. With text formats like CSV, changing one value means rewriting the whole file.
  5. The channel is opened with READ and WRITE, so one channel serves both purposes.

Databases and search engines store data in fixed-size pages in exactly this way, at a much larger scale. java.io.RandomAccessFile offers the same seek-and-read model with seek(position), readInt() and writeInt() methods, and is common in older code.

Fast copying with transferTo

try (FileChannel in = FileChannel.open(source, READ);
     FileChannel out = FileChannel.open(target, CREATE, TRUNCATE_EXISTING, WRITE)) {
    long position = 0, size = in.size();
    while (position < size) {
        position += in.transferTo(position, size - position, out);
    }
}

transferTo asks the operating system to move bytes directly from one channel to another. On many systems this avoids copying the data into Java memory at all (zero-copy), which is how web servers send static files efficiently. It may transfer fewer bytes than requested in one call, hence the loop. Files.copy uses similar optimisations internally.

Memory-Mapped Files

A channel can map a region of a file into memory. The resulting MappedByteBuffer reads and writes the file as if it were an array: the operating system loads pages of the file on demand and writes changes back.

import java.io.IOException;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.file.*;

public class Mapped {
    public static void main(String[] args) throws IOException {
        Path log = Files.writeString(Path.of("access.log"), "GET /a 200\nGET /b 404\nPOST /c 200\nGET /d 500\n".repeat(25_000));
        try (FileChannel ch = FileChannel.open(log)) {
            MappedByteBuffer map = ch.map(FileChannel.MapMode.READ_ONLY, 0, ch.size());
            int lines = 0, errors = 0;
            for (int i = 0; i < map.limit(); i++) {
                if (map.get(i) != '\n') continue;
                lines++;
                byte statusClass = map.get(i - 3);          // each line ends with a 3-digit status code
                if (statusClass == '4' || statusClass == '5') errors++;
            }
            System.out.println(ch.size() + " bytes mapped, " + lines + " lines, " + errors + " error responses");
        }
    }
}

Output:

1125000 bytes mapped, 100000 lines, 50000 error responses
  • ch.map(READ_ONLY, 0, size) maps the whole file. FileChannel.open(path) with no options opens it for reading.
  • The loop scans the bytes with absolute get(i) calls. At each newline, the byte three positions earlier is the first digit of the status code; 4xx and 5xx responses are counted as errors (2 of every 4 lines).
  • No read calls or buffer management are needed, and the operating system handles caching, which makes mapping very fast for scanning large files or for random access to them.

Things to know:

  • A single mapping is limited to Integer.MAX_VALUE bytes (about 2 GB); larger files are mapped in several regions. (Java 22's Foreign Function & Memory API removes this limit.)
  • The mapping stays valid even after the channel is closed, until the buffer is garbage-collected. On Windows, a mapped file can't be deleted while mapped.
  • READ_WRITE mode writes changes back to the file; force() ensures they reach the disk.

File Locking

A FileLock coordinates access to a file between processes — for example, to make sure only one instance of a scheduled job runs at a time on a server.

import java.io.IOException;
import java.nio.channels.*;
import java.nio.file.*;
import static java.nio.file.StandardOpenOption.*;

public class Lock {
    public static void main(String[] args) throws IOException {
        Path lockFile = Path.of("nightly-report.lock");
        try (FileChannel ch = FileChannel.open(lockFile, CREATE, WRITE);
             FileLock lock = ch.tryLock()) {
            if (lock == null) {
                System.out.println("another process is generating the report; exiting");
                return;
            }
            System.out.println("lock acquired, shared=" + lock.isShared() + ", valid=" + lock.isValid());
            try (FileChannel second = FileChannel.open(lockFile, WRITE)) {
                second.tryLock();
            } catch (OverlappingFileLockException e) {
                System.out.println("same JVM, second tryLock: OverlappingFileLockException");
            }
        }
        System.out.println("lock released");
    }
}

Output:

lock acquired, shared=false, valid=true
same JVM, second tryLock: OverlappingFileLockException
lock released
  • tryLock() acquires an exclusive lock on the whole file and returns immediately: a FileLock on success, or null if another process holds a lock. lock() waits until the lock becomes available instead.
  • FileLock is AutoCloseable; closing it (or the channel, or the process exiting) releases the lock. try-with-resources handles null resources, so the null case needs no special treatment.
  • Locks are held on behalf of the whole JVM, so they don't coordinate threads within one program: a second attempt in the same JVM throws OverlappingFileLockException. Use java.util.concurrent locks between threads.
  • On most Unix systems file locks are advisory: they only affect programs that also use locking. A program that just opens and writes the file isn't stopped.
  • lock(position, size, shared) locks a region, and shared = true requests a shared (read) lock that other processes can also hold.

Charsets, Encoders and Decoders

java.nio.charset is where character encodings live — the Charset objects you pass to readers, writers and Files methods come from here. You can also convert between characters and bytes directly:

import java.nio.*;
import java.nio.charset.*;

public class Cs {
    public static void main(String[] args) throws CharacterCodingException {
        Charset utf8 = StandardCharsets.UTF_8;
        ByteBuffer bytes = utf8.encode("Chennai ₹500");
        System.out.println("encoded to " + bytes.remaining() + " bytes");
        CharBuffer chars = utf8.decode(bytes);
        System.out.println("decoded: " + chars);

        CharsetDecoder strict = utf8.newDecoder()
            .onMalformedInput(CodingErrorAction.REPORT)
            .onUnmappableCharacter(CodingErrorAction.REPORT);
        try {
            strict.decode(ByteBuffer.wrap(new byte[] {'o', 'k', (byte) 0xE9}));
        } catch (MalformedInputException e) {
            System.out.println("strict decoder: MalformedInputException, length " + e.getInputLength());
        }
        System.out.println("lenient decode: " + utf8.decode(ByteBuffer.wrap(new byte[] {'o', 'k', (byte) 0xE9})));
    }
}

Output:

encoded to 14 bytes
decoded: Chennai ₹500
strict decoder: MalformedInputException, length 1
lenient decode: ok�
  • charset.encode(String) returns a ByteBuffer (11 ASCII characters plus 3 bytes for ₹ = 14 bytes); charset.decode(ByteBuffer) returns a CharBuffer.
  • These convenience methods replace invalid input with the replacement character � silently. A CharsetDecoder configured with CodingErrorAction.REPORT throws instead, which is what Files.readString does internally. Use the strict form whenever corrupted text must be detected rather than passed on.

Where NIO Fits

NeedUse
Read or write a whole small fileFiles.readString, Files.writeString, Files.readAllBytes
Process a large text file line by lineFiles.newBufferedReader, Files.lines
Copy or move filesFiles.copy, Files.move
Read/write at arbitrary positions, fixed-size recordsFileChannel with positional read/write
Scan very large files fast, or random access over themMemory-mapped MappedByteBuffer
Ensure only one process runs a jobFileChannel.tryLock()
Binary protocols and formatsByteBuffer with getInt/putInt and an explicit byte order
Thousands of network connectionsNon-blocking channels with a Selector (Networking section), or a framework such as Netty

The rule of thumb: start with Files and streams; reach for buffers and channels when you need positional access, mapping, locking or maximum throughput.

0 Comments

Reviewed before they appear

No comments yet.

Java
Java Introduction Java Features Java Syntax Java Comments Java Statements Java Output Java Variables Java Data Types Java Type Casting Java Operators Java Strings Java Numbers Java Math Java Booleans Java User Input Java Conditions Java If Else Java Switch Java While Loop Java For Loop Java Break Java Continue Java Arrays Java Methods Java Parameters Java Return Values Java Scope Java Recursion Install Java Install JDK Java IDEs Java JDK Java JRE Java JVM Java Compiler Java File Compile Java Run Java JShell Basics Java Packages Java OOP Java Classes Java Objects Class Methods Constructors Constructor Overload this Keyword static Keyword final Keyword Encapsulation Inheritance Polymorphism Abstraction Access Modifiers super Keyword Method Overload Method Override Instanceof Operator Nested Classes Inner Classes Anonymous Classes Enums Records Array Basics Array Length Loop Arrays Multidimensional Arrays Array Copy Array Sorting Array Search ArrayList LinkedList HashSet TreeSet HashMap TreeMap Queue Interface Deque Interface Stack Class Iterator List Interface Set Interface Map Interface Collections Class Comparable Comparator Generics Basics Generic Classes Generic Methods Wildcards String Basics String Length String Methods String Concatenation String Compare String Search String Replace String Split String Format StringBuilder StringBuffer Text Blocks Regular Expressions Number Types Integer Class Float Class Double Class BigInteger BigDecimal Math Class Random Numbers Number Format Date Basics LocalDate LocalTime LocalDateTime ZonedDateTime Date Formatting Date Parsing Period Class Duration Class Java Instant Class Java Exception Basics Java try...catch Java Multiple Catch Blocks Java finally Block Java throw Keyword Java throws Keyword Java Checked Exceptions Java Unchecked Exceptions Java Custom Exceptions Java try-with-resources Java Exception Chaining Java File Basics Java Create Files Java Read Files Java Write Files Java Delete Files Java File Paths Java File Class Java BufferedReader Java BufferedWriter Java FileInputStream Java FileOutputStream Java Object Streams Java Directory Handling Java NIO Basics Java Files API Java Input Java Scanner Class Java Console Input Java PrintStream Java InputStream Java OutputStream Java Reader Class Java Writer Class Java Buffered Streams Java Data Streams Java Generics Introduction Java Generic Types Java Generic Methods Java Bounded Types Java Wildcards Java Type Erasure Java Generic Interfaces Java Generic Collections Java Lambda Basics Java Lambda Syntax Java Functional Interfaces Java Method References Java Stream API Java Stream Creation Java Stream Filter Java Stream Map Java Stream Reduce Java Stream Sort Java Stream Collect Java Collectors Class Java Optional Class Java Parallel Streams Java Threads: Thread Basics Create a Thread in Java Java Thread Class Java Runnable Interface Java Thread States Java Thread Sleep Java Thread Join Java Thread Priority Java Synchronization Java Thread Safety Java Race Conditions Java Deadlocks Java Executor Service Java Thread Pools Java Callable Interface Java Future Interface Java CompletableFuture Java Concurrent Collections Java Atomic Classes Java Virtual Threads Java Networking Basics Java URL Class Java URI Class Java HttpClient Java HTTP Requests Java HTTP Responses Java Socket Basics Java ServerSocket Java DatagramSocket Java WebSocket Basics Java JDBC Basics Java JDBC Setup Java Database Connection Java JDBC Connection Interface Java JDBC Statement Java PreparedStatement Java ResultSet Java JDBC CRUD Operations Java JDBC Transactions Java JDBC Batch Update Java Connection Pooling with HikariCP Java JPA Basics: Entities, EntityManager and JPQL Java Hibernate Basics: SessionFactory, Session and HQL Package in Java: Basics Explained with Examples Create Package in Java: Step-by-Step Guide with Examples Import Package in Java: The import Statement Explained Access Control in Java: Packages, Classes and Members JAR File in Java: Create, Run and Inspect JAR Files Java Modules: Introduction to the Java Module System module-info in Java: Every Module Directive Explained Java Module Path: How --module-path Finds Your Modules Java Classpath: How -cp and CLASSPATH Find Your Classes Java Annotations: Basics of @ Metadata in Java Java Built-in Annotations: @Override, @Deprecated and More Java Custom Annotations: Create Your Own @interface Java Annotation Targets: @Target and ElementType Explained Java Reflection API: Inspect and Use Classes at Runtime Java Class Metadata: Names, Modifiers, Hierarchy and Generic Java Dynamic Objects: Create Objects Dynamically With Reflec Java Dynamic Proxy: Intercept Method Calls With Proxy and In Java Security Java Secure Coding Java Password Hashing Java Input Validation Java Cryptography Basics Java KeyStore Basics Java TLS Basics Java SecureRandom Java JUnit Basics Java JUnit Test Methods JUnit Assertions JUnit Test Fixtures JUnit Parameterized Tests Mockito in Java Java Mock Objects Java Integration Testing Java Code Coverage Maven in Java Create a Maven Project in Java Maven Dependencies Maven Plugins Gradle in Java Gradle Build Gradle Dependencies Build Executable JAR in Java Build WAR File in Java HTTP Basics in Java Java Servlet Basics Servlet Lifecycle in Java HttpServletRequest in Java HttpServletResponse in Java JSP in Java JSTL in JSP REST API in Java JSON in Java Java Web Application Spring Framework Basics Spring Boot Tutorial Spring MVC Spring Data JPA Spring Security Spring Boot REST API Jakarta EE Basics Java Deployment JAR Deployment Java WAR Deployment Run Java as a Service Docker Java: Dockerize a Java Application Java Environment Variables and Configuration Java Logging Basics Java Production Setup Java JVM Options Java JVM Monitoring Java Hello World App Java Calculator App Java Number Guess Game Java Student Manager Java Library System Java Bank App Java Todo App Java File Manager Java Quiz App Java REST API Project Java Chat Server Java Employee Manager Java E-commerce API
Ask about this post
AI Ask about this post

Ask questions about Java NIO Basics and get answers drawn from it.

Signed-in readers only.