Java Production Setup
A Java production setup is everything around your JAR that keeps it running safely for real users: a dedicated user, a clean folder layout, external configuration, tuned JVM flags, health checks, graceful shutdown, a reverse proxy with HTTPS and a deploy script that can roll back. Getting these pieces right is what turns "it works on my laptop" into a service that survives crashes, reboots and bad releases.
This lesson builds a complete Java production setup for the CourseServer app used in Java Deployment and Java Logging. Examples use JDK 21 on Ubuntu 24.04, but the same steps work on any systemd-based Linux server and on JDK 25.
What a Java Production Setup Includes
| Piece | Purpose | Covered in |
|---|---|---|
| Dedicated system user | The app cannot touch other users' files | This lesson |
Release folders + current symlink | Instant switch and instant rollback | This lesson |
| Config and secrets outside the JAR | Same build for dev, staging and prod | This lesson |
| JVM options file | Fixed heap, GC logs, heap dumps | This lesson, Java JVM Options |
| Health and readiness endpoints | Proxies and scripts know when the app is usable | This lesson |
| Graceful shutdown | No half-finished requests on restart | This lesson |
| systemd service | Start on boot, restart on crash | Run Java as a Service |
| Nginx + HTTPS | TLS, compression, one public port | This lesson |
| Firewall | Only SSH, HTTP and HTTPS are reachable | This lesson |
| Deploy script | Repeatable releases with automatic rollback | This lesson |
Prepare the Server for Java in Production
Install only the JRE (runtime) on the server. Compilers and build tools stay on your build machine or CI.
sudo apt update
sudo apt install -y openjdk-21-jre-headless nginx curl
java -version
openjdk version "21.0.9" 2025-10-21
OpenJDK Runtime Environment (build 21.0.9+10-Ubuntu-124.04)
OpenJDK 64-Bit Server VM (build 21.0.9+10-Ubuntu-124.04, mixed mode, sharing)
Next, create a system user that owns the app and cannot log in, plus the folders it needs.
# system user with no shell and no home login
sudo useradd --system --no-create-home --shell /usr/sbin/nologin courseapp
# app code, config, logs and data in separate places
sudo mkdir -p /opt/courseserver/releases
sudo mkdir -p /etc/courseserver/secrets
sudo mkdir -p /var/log/courseserver /var/lib/courseserver/dumps
sudo chown -R courseapp:courseapp /opt/courseserver /var/log/courseserver /var/lib/courseserver
sudo chown -R root:courseapp /etc/courseserver
sudo chmod 750 /etc/courseserver /etc/courseserver/secrets
ls -ld /opt/courseserver /etc/courseserver /var/log/courseserver
drwxr-xr-x 3 courseapp courseapp 4096 Nov 18 10:02 /opt/courseserver
drwxr-x--- 3 root courseapp 4096 Nov 18 10:02 /etc/courseserver
drwxr-xr-x 2 courseapp courseapp 4096 Nov 18 10:02 /var/log/courseserver
How it works:
openjdk-21-jre-headlessis the runtime without GUI libraries, which keeps the server small.- The
courseappuser has no shell, so even if someone breaks into the app, they cannot open a login session as that user. - Code (
/opt), configuration (/etc), logs (/var/log) and data (/var/lib) follow the standard Linux layout. Backups and permissions become simple. /etc/courseserveris owned byrootbut readable by thecourseappgroup. The app can read its config but cannot change it.
Organize Java Releases With a current Symlink
Each version gets its own folder. A symlink named current points to the version that is running. Switching versions is a single, atomic ln command.
sudo mkdir -p /opt/courseserver/releases/1.3.0
sudo cp course-server-1.3.0.jar /opt/courseserver/releases/1.3.0/course-server.jar
sudo ln -sfn /opt/courseserver/releases/1.3.0 /opt/courseserver/current
sudo chown -h courseapp:courseapp /opt/courseserver/current
tree /opt/courseserver
/opt/courseserver
├── current -> /opt/courseserver/releases/1.3.0
└── releases
└── 1.3.0
└── course-server.jar
How it works:
- The service always starts
/opt/courseserver/current/course-server.jar, so it never needs editing when you release. ln -sfnreplaces the existing link in one step. There is never a moment wherecurrentis missing.- Old release folders stay on disk. Rolling back is pointing
currentat the previous folder and restarting.
Store Java Configuration and Secrets Outside the JAR
The same JAR must run in every environment. Settings go in an environment file, and passwords go in separate files that only the app can read, as explained in Java Environment Variables.
# /etc/courseserver/app.env (non-secret settings)
sudo tee /etc/courseserver/app.env > /dev/null <<'EOF'
APP_ENV=prod
APP_PORT=8080
APP_DB_URL=jdbc:postgresql://10.0.0.12:5432/gocourse
APP_DB_USER=course_api
LOG_LEVEL=INFO
EOF
# the password lives in its own file, readable only by root and courseapp
echo -n 'S3cure-Pa55' | sudo tee /etc/courseserver/secrets/db_password > /dev/null
sudo chown root:courseapp /etc/courseserver/app.env /etc/courseserver/secrets/db_password
sudo chmod 640 /etc/courseserver/app.env /etc/courseserver/secrets/db_password
The app reads a secret from NAME_FILE when it is set, and falls back to the plain NAME variable for local development.
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
public class Secrets {
/** Reads NAME_FILE (preferred in production) or NAME (handy in development). */
static String readSecret(String name) throws IOException {
String file = System.getenv(name + "_FILE");
if (file != null) {
return Files.readString(Path.of(file)).strip();
}
String value = System.getenv(name);
if (value == null || value.isBlank()) {
throw new IllegalStateException(
"Missing secret " + name + " (set " + name + " or " + name + "_FILE)");
}
return value;
}
public static void main(String[] args) throws IOException {
String password = readSecret("APP_DB_PASSWORD");
System.out.println("DB password loaded, length = " + password.length());
}
}
APP_DB_PASSWORD_FILE=/etc/courseserver/secrets/db_password java Secrets.java
java Secrets.java
DB password loaded, length = 11
Exception in thread "main" java.lang.IllegalStateException: Missing secret APP_DB_PASSWORD (set APP_DB_PASSWORD or APP_DB_PASSWORD_FILE)
at Secrets.readSecret(Secrets.java:15)
at Secrets.main(Secrets.java:22)
How it works:
- Environment variables can leak through
ps e, crash reports and child processes. A file with640permissions is safer for passwords. - The
_FILEconvention matches Docker and Kubernetes secrets, so the same code works in containers. - The app fails fast with a clear message when a secret is missing, instead of starting and failing on the first database call.
- The program prints only the length, never the value. Secrets must never reach the logs.
Set Production JVM Options in One File
Keep JVM flags in an argument file instead of a long command line. The java launcher reads every line of a file passed as @path.
# /etc/courseserver/jvm.options
-Xms512m
-Xmx512m
-XX:+UseG1GC
-XX:+ExitOnOutOfMemoryError
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/var/lib/courseserver/dumps
-Xlog:gc*:file=/var/log/courseserver/gc.log:time,uptime,level,tags:filecount=5,filesize=10m
-Duser.timezone=UTC
java @/etc/courseserver/jvm.options -XshowSettings:vm -version 2>&1 | head -n 3
VM settings:
Max. Heap Size: 512.00M
Using VM: OpenJDK 64-Bit Server VM
How it works:
- Equal
-Xmsand-Xmxreserve the whole heap at startup, so memory use is predictable and the server will not run out of RAM later. -XX:+ExitOnOutOfMemoryErrorstops a broken JVM immediately, and systemd restarts it. A JVM that keeps running after anOutOfMemoryErroris usually unreliable.- The heap dump and GC log give you evidence after a crash. Each flag is explained in detail in Java JVM Options.
-Duser.timezone=UTCmakes timestamps identical on every server, whatever the machine's local zone.
Add Health and Readiness Endpoints
Production tools need two different answers from your app:
| Endpoint | Question | Returns 200 when |
|---|---|---|
/health/live | Is the process alive? | The HTTP server responds at all |
/health/ready | Can it handle real traffic? | Caches are loaded and the database is reachable |
package com.gocourse.app;
import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpServer;
import java.io.IOException;
import java.io.OutputStream;
import java.net.InetSocketAddress;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicBoolean;
public class CourseServer {
private static final AtomicBoolean READY = new AtomicBoolean(false);
public static void main(String[] args) throws Exception {
int port = Integer.parseInt(System.getenv().getOrDefault("APP_PORT", "8080"));
// bind to localhost only: Nginx is the public entry point
HttpServer server = HttpServer.create(new InetSocketAddress("127.0.0.1", port), 0);
server.setExecutor(Executors.newFixedThreadPool(8));
server.createContext("/health/live", ex -> send(ex, 200, "{\"status\":\"UP\"}"));
server.createContext("/health/ready", ex -> {
if (READY.get()) send(ex, 200, "{\"status\":\"READY\"}");
else send(ex, 503, "{\"status\":\"NOT_READY\"}");
});
server.createContext("/api/courses", ex -> send(ex, 200, "[\"java\",\"go\",\"html\"]"));
// graceful shutdown on SIGTERM (systemctl stop / restart)
Runtime.getRuntime().addShutdownHook(new Thread(() -> {
READY.set(false);
System.out.println("Shutdown: draining in-flight requests");
server.stop(5); // stop accepting, wait up to 5 s for running requests
System.out.println("Shutdown: complete");
}));
server.start();
System.out.println("Listening on 127.0.0.1:" + port);
warmUp();
READY.set(true);
System.out.println("Ready to serve traffic");
}
static void warmUp() throws InterruptedException {
Thread.sleep(3000); // simulate loading caches and checking the database
}
static void send(HttpExchange exchange, int status, String json) throws IOException {
byte[] body = json.getBytes(StandardCharsets.UTF_8);
exchange.getResponseHeaders().set("Content-Type", "application/json");
exchange.sendResponseHeaders(status, body.length);
try (OutputStream out = exchange.getResponseBody()) {
out.write(body);
}
}
}
java -jar course-server.jar &
sleep 1; curl -s -w ' %{http_code}\n' http://127.0.0.1:8080/health/ready
sleep 3; curl -s -w ' %{http_code}\n' http://127.0.0.1:8080/health/ready
kill %1
Listening on 127.0.0.1:8080
{"status":"NOT_READY"} 503
Ready to serve traffic
{"status":"READY"} 200
Shutdown: draining in-flight requests
Shutdown: complete
How it works:
- During the 3-second warm-up the process is alive but not ready. A deploy script or load balancer waits for
200from/health/readybefore sending users. - Binding to
127.0.0.1means the Java port is invisible from the internet. Only Nginx on the same machine can reach it. killsendsSIGTERM, the same signalsystemctl stopsends. The JVM runs the shutdown hook, which stops accepting connections and lets running requests finish.
Note: A shutdown hook does not run on
kill -9(SIGKILL). systemd sendsSIGTERMfirst and waitsTimeoutStopSecbefore usingSIGKILL, so give the hook enough time.
Run the Java App With a Hardened systemd Service
The unit file ties everything together: user, config, secrets, JVM options and restart rules. Lesson Run Java as a Service explains systemd basics; this version adds sandboxing.
# /etc/systemd/system/courseserver.service
[Unit]
Description=CourseServer (GOCOURSE API)
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=courseapp
Group=courseapp
WorkingDirectory=/opt/courseserver
EnvironmentFile=/etc/courseserver/app.env
Environment=APP_DB_PASSWORD_FILE=/etc/courseserver/secrets/db_password
ExecStart=/usr/bin/java @/etc/courseserver/jvm.options -jar /opt/courseserver/current/course-server.jar
SuccessExitStatus=143
Restart=on-failure
RestartSec=5
TimeoutStopSec=30
LimitNOFILE=65536
# sandboxing
NoNewPrivileges=true
PrivateTmp=true
ProtectHome=true
ProtectSystem=strict
ReadWritePaths=/var/log/courseserver /var/lib/courseserver
[Install]
WantedBy=multi-user.target
sudo systemctl daemon-reload
sudo systemctl enable --now courseserver
systemctl status courseserver --no-pager
● courseserver.service - CourseServer (GOCOURSE API)
Loaded: loaded (/etc/systemd/system/courseserver.service; enabled; preset: enabled)
Active: active (running) since Wed 2026-11-18 10:30:12 UTC; 8s ago
Main PID: 41872 (java)
Tasks: 27 (limit: 4558)
Memory: 96.4M (peak: 97.1M)
CPU: 1.912s
CGroup: /system.slice/courseserver.service
└─41872 /usr/bin/java @/etc/courseserver/jvm.options -jar /opt/courseserver/current/course-server.jar
How it works:
SuccessExitStatus=143tells systemd that exit code 143 (128 +SIGTERM) is a normal stop, not a failure.ProtectSystem=strictmakes the whole file system read-only for the app, except the folders listed inReadWritePaths. A compromised app cannot modify system files or its own JAR.LimitNOFILE=65536raises the open-file limit. Every socket is a file, so busy servers need more than the default.Restart=on-failurecombined with-XX:+ExitOnOutOfMemoryErrorturns a memory crash into a 5-second restart.
Put Nginx in Front of Java With HTTPS
Nginx handles HTTPS, slow clients and compression, and forwards clean requests to Java on localhost.
# /etc/nginx/sites-available/courseserver
server {
listen 80;
server_name api.gocourse.example;
location / {
proxy_pass http://127.0.0.1:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Request-Id $request_id;
proxy_read_timeout 60s;
}
# health checks are for local tools only
location /health/ {
allow 127.0.0.1;
deny all;
proxy_pass http://127.0.0.1:8080;
}
}
sudo ln -s /etc/nginx/sites-available/courseserver /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx
sudo apt install -y certbot python3-certbot-nginx
sudo certbot --nginx -d api.gocourse.example --redirect
curl -s https://api.gocourse.example/api/courses
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
Successfully deployed certificate for api.gocourse.example to /etc/nginx/sites-enabled/courseserver
Congratulations! You have successfully enabled HTTPS on https://api.gocourse.example
["java","go","html"]
How it works:
certbot --nginxgets a free Let's Encrypt certificate, adds thelisten 443 sslblock and, with--redirect, sends all HTTP traffic to HTTPS. It also installs a timer that renews the certificate automatically.X-Request-Id $request_idgives every request a unique ID. The MDC code from Java Logging picks it up, so Nginx and Java log lines can be matched.X-Forwarded-Prototells the app the original request was HTTPS, which frameworks such as Spring Boot need for correct redirect URLs.
Open Only the Ports You Need
sudo ufw allow OpenSSH
sudo ufw allow 'Nginx Full'
sudo ufw enable
sudo ufw status
Status: active
To Action From
-- ------ ----
OpenSSH ALLOW Anywhere
Nginx Full ALLOW Anywhere
OpenSSH (v6) ALLOW Anywhere (v6)
Nginx Full (v6) ALLOW Anywhere (v6)
How it works:
Nginx Fullopens ports 80 and 443. Port 8080 is not listed, so even if the app were bound to all interfaces, the firewall would block it.- Always allow
OpenSSHbeforeufw enable, or you can lock yourself out of a remote server.
Real-World Example: A Java Deploy Script With Automatic Rollback
This script installs a new version, switches the symlink, restarts the service and waits for readiness. If the new version is not ready within 30 seconds, it switches back to the previous version and prints the last log lines.
#!/usr/bin/env bash
# deploy.sh — usage: ./deploy.sh course-server-1.4.0.jar 1.4.0
set -euo pipefail
JAR="$1"
VERSION="$2"
APP_HOME=/opt/courseserver
RELEASE_DIR="$APP_HOME/releases/$VERSION"
HEALTH_URL=http://127.0.0.1:8080/health/ready
KEEP_RELEASES=3
previous=$(readlink -f "$APP_HOME/current" || true)
echo "==> Installing $VERSION"
sudo mkdir -p "$RELEASE_DIR"
sudo cp "$JAR" "$RELEASE_DIR/course-server.jar"
sudo chown -R courseapp:courseapp "$RELEASE_DIR"
echo "==> Switching current -> $VERSION"
sudo ln -sfn "$RELEASE_DIR" "$APP_HOME/current"
sudo systemctl restart courseserver
echo "==> Waiting for readiness"
for second in $(seq 1 30); do
if curl -fsS "$HEALTH_URL" > /dev/null 2>&1; then
echo "==> $VERSION is READY after ${second}s"
# delete all but the newest releases
ls -1dt "$APP_HOME"/releases/* | tail -n +$((KEEP_RELEASES + 1)) | xargs -r sudo rm -rf
exit 0
fi
sleep 1
done
echo "!! $VERSION did not become ready, rolling back"
if [[ -n "$previous" && "$previous" != "$RELEASE_DIR" ]]; then
sudo ln -sfn "$previous" "$APP_HOME/current"
sudo systemctl restart courseserver
echo "!! Rolled back to $(basename "$previous")"
fi
sudo journalctl -u courseserver -n 5 --no-pager
exit 1
A good release:
./deploy.sh course-server-1.4.0.jar 1.4.0
==> Installing 1.4.0
==> Switching current -> 1.4.0
==> Waiting for readiness
==> 1.4.0 is READY after 4s
A release that crashes because a new required secret is missing:
./deploy.sh course-server-1.5.0.jar 1.5.0
==> Installing 1.5.0
==> Switching current -> 1.5.0
==> Waiting for readiness
!! 1.5.0 did not become ready, rolling back
!! Rolled back to 1.4.0
Nov 18 11:02:41 web1 java[42210]: Exception in thread "main" java.lang.IllegalStateException: Missing secret APP_SMTP_PASSWORD (set APP_SMTP_PASSWORD or APP_SMTP_PASSWORD_FILE)
Nov 18 11:02:41 web1 systemd[1]: courseserver.service: Main process exited, code=exited, status=1/FAILURE
Nov 18 11:02:41 web1 systemd[1]: courseserver.service: Failed with result 'exit-code'.
Nov 18 11:02:46 web1 systemd[1]: Started courseserver.service - CourseServer (GOCOURSE API).
Nov 18 11:02:50 web1 java[42267]: Ready to serve traffic
How it works:
- Remember the old version.
readlink -fresolvescurrentto the real folder before anything changes, so the script knows where to roll back. - Atomic switch.
ln -sfnpointscurrentat the new folder in one step, andsystemctl restartsendsSIGTERM, so the old process drains its requests through the shutdown hook. - Readiness, not just "started". The loop calls
/health/readyonce per second.curl -ftreats503as a failure, so the script waits until warm-up is really finished. - Automatic rollback. If 30 seconds pass without
200, the previous folder becomescurrentagain and the service restarts. Users see a short blip instead of an outage, and the journal lines explain exactly why the release failed. - Disk cleanup. Only the three newest release folders are kept, so repeated deploys never fill the disk while still leaving room for fast rollbacks.