HugeGraph Docker Cluster Guide

Overview

Docker Compose provides a quick way to run a complete HugeGraph distributed cluster (PD + Store + Server) on Linux or macOS.

Prerequisites

• Docker Engine 20.10+ or Docker Desktop 4.x+
• Docker Compose v2
• For a three-node cluster on macOS, allocate at least 12 GB of memory (Settings → Resources → Memory). Adjust resources to suit other platforms.

Tested environments: Linux (native Docker) and macOS (Docker Desktop on ARM M4).

Compose Files

The HugeGraph repository provides four Compose files in docker/:

Standalone uses hugegraph/hugegraph:${HUGEGRAPH_VERSION:-latest}; HStore uses matching hugegraph/pd, hugegraph/store, and hugegraph/server tags. Hubble is selected separately by ${HUBBLE_IMAGE:-hugegraph/hubble:latest}.

The following steps assume you have cloned the hugegraph repository, or at least its docker/ directory.

HStore and HA instructions use current master Compose files and require PD, Store, and Server images built from the same master source. Do not combine these files with HStore component images tagged 1.7.0. The standalone example separately pins hugegraph/hugegraph:1.7.0.

Authentication Environment

This .env template applies to current master Compose. Angle-bracket values and replace-with-your-password are placeholders: replace them before loading the file or starting services. Follow the master authentication environment instructions to generate .env, then replace the administrator password placeholder:

1HUGEGRAPH_ADMIN_PASSWORD='replace-with-your-password'
2HUGEGRAPH_AUTH_TOKEN_SECRET='<32 random bytes, e.g. openssl rand -hex 32>'
3HG_PD_AUTH_SECRET_KEY='<24 random bytes, e.g. openssl rand -hex 24>'

For master HStore and HA topologies, HG_PD_AUTH_SECRET_KEY is the PD REST Basic password, shared by all PD nodes, Server, and Hubble. Current master PD Docker images require it. Hubble does not read .env directly: generate the local configuration for its topology using this value before startup, as described below. Do not commit .env or generated .local.properties files.

A nonempty HUGEGRAPH_ADMIN_PASSWORD enables Server authentication through PASSWORD; Hubble detects this mode through the Server API. It initializes the built-in admin password only on first initialization. An unset or empty value disables authentication and is suitable only for trusted local environments.

Current master Compose passes HUGEGRAPH_AUTH_TOKEN_SECRET as HG_SERVER_AUTH_TOKEN_SECRET. A shared secret lets master Server replicas verify the same tokens and retain token validity after container recreation; HA Compose requires an explicit secret. The standalone hugegraph/hugegraph:1.7.0 entrypoint enables authentication through PASSWORD but ignores HG_SERVER_AUTH_TOKEN_SECRET, so this .env variable does not fix its JWT secret. For a stable JWT secret in 1.7.0, explicitly set auth.token_secret in its authentication graph configuration, conf/graphs/hugegraph.properties, and persist that file across container replacement. See the authentication guide for graph paths and secret generation.

Changing HUGEGRAPH_ADMIN_PASSWORD later does not rotate an existing password; use the user API.

Standalone Quick Start

This section pins the standalone image hugegraph/hugegraph:1.7.0. HStore/HA examples use master-built images; do not reuse this version for them.

1cd "$(git rev-parse --show-toplevel)/docker"
2# Standalone RocksDB Server image example
3HUGEGRAPH_VERSION=1.7.0 docker compose -f docker-compose.yml up -d --wait

Verify:

1curl -fsS http://localhost:8080/versions
2curl -fsS http://localhost:8088/about        # Hubble

Hubble binds only to host loopback (127.0.0.1:8088) by default. Set HUBBLE_PUBLISH_HOST only behind an HTTPS reverse proxy and trusted network controls.

Minimal HStore Quick Start (Current Master Images)

Build PD, Store, and HStore Server images from the HugeGraph Server master repository root:

1cd "$(git rev-parse --show-toplevel)"
2docker build -f hugegraph-pd/Dockerfile -t hugegraph/pd:local .
3docker build -f hugegraph-store/Dockerfile -t hugegraph/store:local .
4docker build -f hugegraph-server/Dockerfile-hstore -t hugegraph/server:local .

From the repository root, enter docker/, load .env, and generate Hubble configuration for the minimal topology. A read-only bind mount cannot create a valid missing hstore.local.properties; generate it before startup:

1cd "$(git rev-parse --show-toplevel)/docker"
2set -a; . ./.env; set +a
3./set-hubble-pd-password.sh hstore
4HUGEGRAPH_VERSION=local HUGEGRAPH_PULL_POLICY=never \
5  docker compose -f docker-compose-hstore.yml up -d --wait

Verify:

1set -a; . ./.env; set +a
2curl -fsS http://localhost:8620/v1/health    # PD
3curl -fsS http://localhost:8520/v1/health    # Store
4curl -fsS http://localhost:8080/versions     # Server
5curl -fsS http://localhost:8088/about        # Hubble
6curl -fsS -u "hg:${HG_PD_AUTH_SECRET_KEY:?Load .env first}" \
7  http://localhost:8620/v1/stores       # Check registered Stores with PD REST authentication

To build this topology from local source instead of pulling images, add the development override and keep both files in all subsequent lifecycle commands:

1cd "$(git rev-parse --show-toplevel)/docker"
2set -a; . ./.env; set +a
3./set-hubble-pd-password.sh hstore
4docker compose -f docker-compose-hstore.yml -f docker-compose.dev.yml up -d --build --wait

The override builds images tagged dev. If you already built local images above, start with the base Compose file and HUGEGRAPH_VERSION=local; keep the tag choices consistent.

Three-Node Cluster Quick Start (Current Master Images)

HA Compose has no source-build override. First build hugegraph/pd:local, hugegraph/store:local, and hugegraph/server:local from the same master source as in the minimal HStore section, then load .env in docker/, generate HA Hubble configuration, and start:

1cd "$(git rev-parse --show-toplevel)/docker"
2set -a; . ./.env; set +a
3./set-hubble-pd-password.sh hstore-ha
4HUGEGRAPH_VERSION=local \
5  docker compose -f docker-compose-3pd-3store-3server.yml up -d --wait

HA Hubble bind-mounts conf/hubble/hstore-ha.local.properties; generate this before startup. It is separate from the minimal topology’s hstore.local.properties, but both use the same HG_PD_AUTH_SECRET_KEY from .env.

The default startup order is:

1. PD nodes start first and must pass /v1/health checks.
2. Store nodes start after all PD nodes are healthy.
3. Server nodes start after all Store and PD nodes are healthy.

Verify the cluster: PD /v1/health and /v1/ready probes require no authentication. Store registration and partition queries are protected PD REST management APIs: use Basic username hg and HG_PD_AUTH_SECRET_KEY from .env.

 1set -a; . ./.env; set +a
 2curl -fsS http://localhost:8620/v1/health      # PD health check
 3curl -fsS http://localhost:8520/v1/health      # Store health check
 4curl -fsS http://localhost:8080/versions        # Server
 5for port in 8620 8621 8622; do
 6  curl -fsS "http://localhost:${port}/v1/ready" | grep -q '"ready":true' || exit 1
 7done
 8curl -fsS -u "hg:${HG_PD_AUTH_SECRET_KEY:?Load .env first}" \
 9  http://localhost:8620/v1/stores          # Authenticated Store registration query
10curl -fsS -u "hg:${HG_PD_AUTH_SECRET_KEY:?Load .env first}" \
11  http://localhost:8620/v1/partitions      # Authenticated partition query
12curl -fsS http://localhost:8088/about           # Hubble

With authentication enabled, graph listing should reject anonymous access and accept the administrator:

1set -a; . ./.env; set +a
2curl -o /dev/null -w '%{http_code}\n' \
3  http://localhost:8080/graphspaces/DEFAULT/graphs                      # Expect 401
4curl -o /dev/null -w '%{http_code}\n' -u "admin:${HUGEGRAPH_ADMIN_PASSWORD}" \
5  http://localhost:8080/graphspaces/DEFAULT/graphs                      # Expect 200

The other Server nodes listen on 8081 and 8082; the remaining PD and Store nodes use 8621/8622 and 8521/8522.

Environment Variable Reference

The following tables describe current master entrypoints and Compose files, not the standalone hugegraph/hugegraph:1.7.0 environment-variable contract. PD and Store build SPRING_APPLICATION_JSON from their variables and log selected nonsensitive settings at startup. PD secrets are excluded from that summary. Use docker logs to check listed settings; an absent secret in the logs does not establish whether it was applied. Server instead rewrites keys in conf/graphs/hugegraph.properties and conf/rest-server.properties.

PD Variables

Deprecated aliases: GRPC_HOST → HG_PD_GRPC_HOST, RAFT_ADDRESS → HG_PD_RAFT_ADDRESS, RAFT_PEERS → HG_PD_RAFT_PEERS_LIST, PD_INITIAL_STORE_LIST → HG_PD_INITIAL_STORE_LIST. Old names are mapped only when new names are unset, with a warning. Missing required variables cause the entrypoint to exit with code 2.

Store Variables

Deprecated aliases: PD_ADDRESS → HG_STORE_PD_ADDRESS, GRPC_HOST → HG_STORE_GRPC_HOST, RAFT_ADDRESS → HG_STORE_RAFT_ADDRESS.

Server Variables

Unlike PD and Store, Server has no mandatory entrypoint variables: only set values are written to configuration. Distributed deployment nevertheless requires at least HG_SERVER_BACKEND and HG_SERVER_PD_PEERS.

Master builds that include authentication property fixes preserve spaces, backslashes and Unicode in explicit signing keys. Use the same key on every Server replica. When upgrading from an image that misread such a key, explicitly supply the intended HG_SERVER_AUTH_TOKEN_SECRET again; retaining the old configuration also retains its misread key. If the active key changes, sign in again to obtain new tokens. The 1.7.0 image limitation above still applies.

Older images could initialize a Unicode administrator password incorrectly when the initializer inherited the C locale. Upgrading or setting PASSWORD again does not repair an existing account’s stored password. A working administrator must change it through the user API. This fix provides no automatic recovery when no administrator identity is usable; preserve the existing data and arrange account recovery before upgrading.

Deprecated aliases: BACKEND → HG_SERVER_BACKEND, PD_PEERS → HG_SERVER_PD_PEERS.

wait-storage.sh waits up to 300 seconds for an Up Store. This duration is hardcoded and cannot be changed through environment variables.

In master builds, HStore startup gives partition polling and the final /v1/partitions diagnostic separate WAIT_PARTITION_TIMEOUT_S budgets. Earlier builds bound polling only; the final diagnostic request has no timeout. With the default value, a stalled final response times out after 120 seconds; setting the value to 0 disables GNU timeout’s limit. The entrypoint waits for wait-partition.sh before waiting for the Java launcher. If Java exits while partition polling is still waiting, that poll can consume its full timeout before the entrypoint observes the exit. If polling succeeds, the diagnostic step can then consume a second full timeout before observation finishes; signal-triggered shutdown remains interruptible.

HG_SERVER_INIT_STORE_ENABLED accepts only the case-insensitive boolean values recognized by HugeConfig: y, t, yes, on, true, n, f, no, off, false. Other values, including 0 and 1, terminate the entrypoint.

After successful initialization, the entrypoint writes docker/init_complete; later starts skip reinitialization but still invoke bin/init-store.sh to revalidate configuration each time initialization is disabled.

Compose Variables

Compose files, rather than entrypoints, read these variables:

Port Reference

Published ports in the three-node cluster:

Standalone publishes only 8080 and 8088; minimal HStore publishes 8620 (PD REST), 8520 (Store REST), 8080, and 8088. PD Raft uses internal port 8610, unpublished in all topologies.

Troubleshooting

1. Container OOM exit (code 137): Increase Docker Desktop memory beyond 12 GB or adjust JVM memory for the killed process.

2. Raft election timeout: Verify every PD node has the same HG_PD_RAFT_PEERS_LIST. Check connectivity with docker exec hg-pd0 ping pd1.

3. Partition allocation incomplete: Load .env in docker/, then check Store registration with PD REST Basic authentication:

   1set -a; . ./.env; set +a
   2curl -fsS -u "hg:${HG_PD_AUTH_SECRET_KEY:?Load .env first}" http://localhost:8620/v1/stores

   PD can complete partition allocation once all three Stores report "state":"Up".

4. Connection refused: Use container hostnames (pd0, store0) in HG_* variables rather than 127.0.0.1.

5. Unexpected retained data: docker compose down keeps named volumes. To delete topology data too, use docker compose down -v.

Runtime logs: docker logs <container-name> (for example, docker logs hg-pd0) shows logs without entering containers. In images built from current master, the PD, Store and both Server images (hugegraph/hugegraph and the HStore hugegraph/server) set STDOUT_MODE=true and send service logs to stdout. For the Server, WARN and above from Hadoop, ZooKeeper, SOFA, Netty and Commons also reaches stdout; INFO from Hadoop, Netty and Commons and the audit and slow-query logs stay in files, and ZooKeeper and SOFA log nothing below WARN. /hugegraph-server/logs holds hugegraph-server.log, JVM crash logs (hs_err_pid*.log) and out-of-memory heap dumps (heapdump_*/java_pid*.hprof, one directory per start, so the Server and the JVMs it starts get separate files unless a JVM reuses an earlier PID from that start; if the directory cannot be created, dumps go to /hugegraph-server/logs itself). Both images declare VOLUME /hugegraph-server, so docker restart and a Compose recreate keep them; docker compose down leaves that anonymous volume behind unattached, and a later down -v does not remove it (note the volume name from docker inspect before down and delete it with docker volume rm <name>; docker volume prune would also delete every other unused anonymous volume on the Docker host), so use a named volume or bind mount to keep them reachable. Kubernetes starts a new container on every restart, so mount an emptyDir or a PersistentVolumeClaim at /hugegraph-server/logs. Give each pod its own volume, or a per-pod directory on a shared PVC (subPathExpr: $(POD_NAME), with POD_NAME passed in from metadata.name through the Downward API; the Helm chart sets neither): crash files get distinct names, but the log files themselves have fixed names. Pods on different nodes can share one PVC only if its storage supports read-write mounts from several nodes (typically ReadWriteMany); a ReadWriteOnce claim is writable from one node only, so otherwise give each pod its own PVC. The Helm chart does not mount a logs volume yet. A heap dump can be as large as the JVM heap, and every start uses a new directory, so a Server that keeps running out of memory fills the volume; size it for the dumps you want to keep, keep it within any ephemeral-storage limit or emptyDir sizeLimit (eviction deletes the emptyDir with the dump), and do not use medium: Memory. A failing liveness probe can restart the container as soon as about (failureThreshold - 1) × periodSeconds + timeoutSeconds after a dump starts, and the kubelet kills it at most terminationGracePeriodSeconds after that; make that time longer than writing a heap dump takes, or a restart in the middle of a dump leaves a truncated file. The launcher never deletes dumps or heapdump_* directories, because a computer-job JVM can go on using one after its Server exits; each start that reaches Java leaves one, empty unless something ran out of memory. Delete old ones once no HugeGraph JVM from that launch is running, never the newest one of a running Server. To turn dumps off for every JVM, set JAVA_TOOL_OPTIONS=-XX:-HeapDumpOnOutOfMemoryError in the container’s environment (docker run -e, an environment: entry on the Server service in Compose, which the shipped Compose files do not set, or env in Kubernetes; exporting it in the host shell has no effect), which keeps the image’s default JAVA_OPTS; putting the flag in JAVA_OPTS affects the Server JVM only and replaces that default (-XX:+UseContainerSupport -XX:MaxRAMPercentage=50 ...), so repeat those flags.

Version scope: the paragraph above describes current master images. No 1.7.0 or older image sets STDOUT_MODE, including the standalone hugegraph/hugegraph:1.7.0 used above: docker logs shows only entrypoint output, and service logs stay in logs/ inside the container (logs/hugegraph-server.log for the Server).

Container Monitoring and Health Checks

Version scope: This section describes current master Docker images; these behaviors are not included in 1.7.0 images. Use the master-built local images above or a latest image containing these changes.

Process Monitoring

Previously, all three entrypoints ended with tail -f /dev/null, keeping containers alive even after Java crashed. Because containers never exited, Docker’s restart: unless-stopped policy did not trigger.

Entrypoints now monitor Java directly:

• PD and Store: The entrypoint passes -d false to the startup script, which replaces itself with Java using exec. When Java exits, the container exits immediately and Docker’s restart policy applies.
• Server: The entrypoint owns the foreground launcher and waits for Java. After readiness, it preserves Java’s exit status, including 0, 42, and 137 for SIGKILL. SIGTERM and SIGINT request cooperative shutdown, reap owned readiness/partition checks, and return 143 and 130 respectively after shutdown.
• PID 1 in every image is dumb-init, which forwards Docker signals to the entrypoint.

Server Startup Timeout

HG_SERVER_STARTUP_TIMEOUT_S sets the Docker entrypoint’s REST readiness deadline; the default is 120 seconds. Values must be decimal integers without leading zeros, from 1 to 86400 seconds. Empty strings, 0, negative numbers, fractions, and out-of-range values log an error and exit with code 1.

The entrypoint also passes this value to bin/start-hugegraph.sh -t; the foreground launcher itself waits for Java, without polling REST. A startup deadline failure or Java exiting with 0 before readiness returns 1; an early nonzero Java exit keeps its status. The deadline excludes earlier storage initialization and backend readiness waits. Readiness and partition checks remain interruptible during shutdown. A deadline failure requests termination and returns without an indefinite wait; an unresponsive Java process relies on container termination for final cleanup.

For example, from the HugeGraph repository’s docker/ directory, extend the standalone Server wait to 300 seconds (Compose passes this variable to the container):

1HG_SERVER_STARTUP_TIMEOUT_S=300 HUGEGRAPH_VERSION=latest \
2  docker compose -f docker-compose.yml up -d --wait

This setting is independent of Docker health-check start_period, interval, timeout, and retries. Those determine when a container becomes unhealthy; extending health-check grace periods does not extend the startup script deadline. Adjusting this variable does not update health-check settings, so check both for slow startup.

Health-Check Endpoints

All four master Docker images include HEALTHCHECK. docker ps displays health status. Failures during the 90-second startup grace period do not count; three consecutive failures afterward mark a container unhealthy.

Compose defines additional health checks, so --wait and depends_on: condition: service_healthy do not depend on image-level checks. Compose uses shorter startup periods (30–120 seconds depending on service and topology) and more retries.

Note: The cron-based -m true monitoring option in start-hugegraph.sh is for VM/bare-metal deployments. Docker images neither install nor use it; use built-in HEALTHCHECK and Docker restart policies.