HugeGraph-PD Quick Start

1 HugeGraph-PD Overview

HugeGraph-PD (Placement Driver) is the metadata management component of HugeGraph’s distributed version, responsible for managing the distribution of graph data and coordinating storage nodes. It plays a central role in distributed HugeGraph, maintaining cluster status and coordinating HugeGraph-Store storage nodes.

PD keeps cluster metadata in an embedded RocksDB store under pd.data-path and replicates it across PD nodes with Raft, so a 3-node or 5-node PD cluster keeps serving while a minority of nodes is down. On top of that it registers and activates Store nodes, allocates and rebalances partitions, tracks Store heartbeats, and answers service discovery queries from Store and Server.

PD listens on three ports:

2 Prerequisites

2.1 Requirements

• Operating System: Linux or macOS (Windows has not been fully tested)
• Java version: ≥ 11
• Maven version: ≥ 3.5.0

3 Deployment

There are two ways to deploy the HugeGraph-PD component:

• Method 1: Download the tar package
• Method 2: Compile from source

3.1 Download the tar package

Apache downloads currently provide the complete 1.7.0 binary bundle containing PD, Store, and Server, without a separate PD binary. Verify version, signatures, and SHA512 on the official download page. This historical incubating release retains incubating in archive and extracted directory names:

# Historical release 1.7.0 retains incubating in filenames and directories
wget https://downloads.apache.org/hugegraph/1.7.0/apache-hugegraph-incubating-1.7.0.tar.gz
tar zxf apache-hugegraph-incubating-1.7.0.tar.gz
cd apache-hugegraph-incubating-1.7.0/apache-hugegraph-pd-incubating-1.7.0

3.2 Compile from source

# 1. Clone the source code
git clone https://github.com/apache/hugegraph.git

# 2. Build the project
cd hugegraph
mvn clean install -DskipTests=true

# 3. After a successful build, the PD directory and packages are located at
#    hugegraph-pd/apache-hugegraph-pd-{version}          (unpacked PD distribution)
#    hugegraph-pd/apache-hugegraph-pd-{version}.tar.gz   (PD only package, Linux build hosts only)
#    target/apache-hugegraph-{version}.tar.gz            (PD + Store + Server package)

To build only the PD distribution and the modules it depends on:

mvn clean package -pl hugegraph-pd/hg-pd-dist -am -DskipTests

The unpacked distribution contains just three directories: bin (start and stop scripts), conf (application.yml, application.yml.template, log4j2.xml, verify-license.json) and lib (the hg-pd-service jar).

3.3 Docker Deployment

HG_PD_* mappings and /v1/ready are features of current master; the Server 1.7.0 source tag lacks these Docker mappings and readiness API. The following examples require images built from current master and tagged local. For release 1.7.0 assets, use bundled configuration and version-specific instructions:

# Run from the HugeGraph repository root
docker build -f hugegraph-pd/Dockerfile -t hugegraph/pd:local .

To run master-built PD alone, first set a deployment-specific secret and replace example addresses with addresses reachable by the container:

export HG_PD_AUTH_SECRET_KEY="$(openssl rand -hex 24)"
docker run -d \
  -p 8620:8620 -p 8686:8686 -p 8610:8610 \
  -e HG_PD_GRPC_HOST=192.168.1.10 \
  -e HG_PD_RAFT_ADDRESS=192.168.1.10:8610 \
  -e HG_PD_RAFT_PEERS_LIST=192.168.1.10:8610 \
  -e HG_PD_INITIAL_STORE_LIST=192.168.1.20:8500 \
  -e HG_PD_AUTH_SECRET_KEY="$HG_PD_AUTH_SECRET_KEY" \
  -v /path/to/data:/hugegraph-pd/pd_data \
  --name hugegraph-pd \
  hugegraph/pd:local

The master entrypoint requires all five mandatory variables, builds SPRING_APPLICATION_JSON overrides, then starts PD. Other settings come from image conf/application.yml; JAVA_OPTS is passed to the JVM. Deprecated GRPC_HOST, RAFT_ADDRESS, RAFT_PEERS, and PD_INITIAL_STORE_LIST remain supported with warnings.

Docker HEALTHCHECK polls /v1/health every 15 seconds, confirming only REST listener liveness, not a Raft quorum. Compose uses the same liveness check; verify /v1/ready separately before starting Store. Containers run Java in the foreground and exit when Java exits; automatic restart requires a Docker restart policy. Inspect logs with docker logs <container-name>.

See the Store Docker section and docker/README.md for master Compose and the minimal topology. When starting the full topology with Hubble, first generate the untracked Hubble configuration described in that README.

4 Configuration

PD startup reads installation conf/application.yml. Match this file to the installed version: 1.7.0 configuration or master configuration. Master leaves auth.secret-key empty, causing protected REST calls to be rejected until configured. Startup does not read application.yml.template.

4.1 Configuration Reference

These tables were checked against hg-pd-dist configuration and Java defaults at Server master commit 2f827d6e8c9c62ae858f2fc122b3a192d015e2f4; they do not describe every 1.7.0 behavior. Use original tagged configuration with release binaries and master configuration with master builds.

gRPC and REST

application.yml.template also carries grpc.netty-server.max-inbound-message-size: 100MB, but PD sets the gRPC server’s inbound message limit to 1 GB in code, so that key has no effect.

Raft

PD core

Store management

Partitions

The initial partition count is derived from these two values and the size of pd.initial-store-list:

initial partitions = store count * partition.store-max-shard-count / partition.default-shard-count

Discovery, license and metrics

Thread pools

4.2 Single-node configuration

For master-built distributions, start with conf/application.yml, adjust per-node gRPC/Raft addresses and data paths, and set auth.secret-key. The following values are for development and testing. One PD node provides no majority fault tolerance; partition.default-shard-count: 1 means one replica per partition.

grpc:
  host: 127.0.0.1
  port: 8686
server:
  port: 8620
raft:
  address: 127.0.0.1:8610
  peers-list: 127.0.0.1:8610
pd:
  data-path: ./pd_data
  initial-store-count: 1
  initial-store-list: 127.0.0.1:8500
partition:
  default-shard-count: 1
auth:
  secret-key: replace-with-deployment-secret

Generate a deployment secret with openssl rand -hex 24; Docker supplies it through HG_PD_AUTH_SECRET_KEY. See REST API authentication for the different 1.7.0 behavior.

4.3 Three-node cluster configuration

For a production cluster run 3 or 5 PD nodes, an odd number so Raft always has a quorum. A 3-node cluster tolerates one node failure. raft.peers-list must list every node and must be byte-for-byte identical on all of them, while grpc.host and raft.address differ per node.

Node 1 (192.168.1.10):

grpc:
  host: 192.168.1.10
  port: 8686
server:
  port: 8620
raft:
  address: 192.168.1.10:8610
  peers-list: 192.168.1.10:8610,192.168.1.11:8610,192.168.1.12:8610
pd:
  data-path: /data/pd
  initial-store-count: 3
  initial-store-list: 192.168.1.20:8500,192.168.1.21:8500,192.168.1.22:8500
partition:
  default-shard-count: 3

Node 2 (192.168.1.11) and node 3 (192.168.1.12) use the same file with grpc.host and raft.address changed to their own address:

# Node 2
grpc:
  host: 192.168.1.11
raft:
  address: 192.168.1.11:8610
  peers-list: 192.168.1.10:8610,192.168.1.11:8610,192.168.1.12:8610

# Node 3
grpc:
  host: 192.168.1.12
raft:
  address: 192.168.1.12:8610
  peers-list: 192.168.1.10:8610,192.168.1.11:8610,192.168.1.12:8610

To put all three PD nodes on one machine for testing, give each node its own pd.data-path and its own ports, for example raft 8610/8611/8612, gRPC 8686/8687/8688 and REST 8620/8621/8622.

In Docker bridge networking the same configuration comes from environment variables and uses container hostnames instead of IP addresses:

# Share the same secret across PD nodes; inject it from private environment variables
HG_PD_AUTH_SECRET_KEY: ${HG_PD_AUTH_SECRET_KEY:?set HG_PD_AUTH_SECRET_KEY}

# pd0
HG_PD_GRPC_HOST: pd0
HG_PD_RAFT_ADDRESS: pd0:8610
HG_PD_RAFT_PEERS_LIST: pd0:8610,pd1:8610,pd2:8610
HG_PD_INITIAL_STORE_LIST: store0:8500,store1:8500,store2:8500
HG_PD_INITIAL_STORE_COUNT: 3

# pd1
HG_PD_GRPC_HOST: pd1
HG_PD_RAFT_ADDRESS: pd1:8610
HG_PD_RAFT_PEERS_LIST: pd0:8610,pd1:8610,pd2:8610

# pd2
HG_PD_GRPC_HOST: pd2
HG_PD_RAFT_ADDRESS: pd2:8610
HG_PD_RAFT_PEERS_LIST: pd0:8610,pd1:8610,pd2:8610

5 Start and Stop

5.1 Start PD

In the PD installation directory, execute:

./bin/start-hugegraph-pd.sh

The script requires a JDK of at least version 11 on PATH or in JAVA_HOME, and it exits without doing anything if it finds a Java process already using this installation’s conf directory.

Supported flags:

The -d flag controls daemon mode:

• -d true (default): run as a background daemon; the script returns immediately.
• -d false: run in foreground. The script execs Java, so the container or supervisor process IS Java. Use this when running under Docker or a process supervisor (systemd, supervisord) so crashes are detected and the service is restarted automatically.

Each flag also has an environment variable equivalent: DAEMON, GC_OPTION, USER_OPTION and OPEN_TELEMETRY. Setting JAVA_OPTIONS replaces the computed heap settings entirely; otherwise the script sizes the heap between 512 MB and 32 GB from available memory. Setting STDOUT_MODE=true leaves the JVM output on stdout instead of redirecting it to logs/hugegraph-pd-stdout.log, which is what the Docker image does.

After successful startup, you can see logs similar to the following in logs/hugegraph-pd-stdout.log:

YYYY-mm-dd xx:xx:xx [main] [INFO] o.a.h.p.b.HugePDServer - Started HugePDServer in x.xxx seconds (JVM running for x.xxx)

The process id is written to bin/pid.

5.2 Stop PD

In the PD installation directory, execute:

./bin/stop-hugegraph-pd.sh

The script reads bin/pid, sends the process a termination signal, waits up to 30 seconds for it to exit, and removes the pid file. If bin/pid is missing it reports that and exits successfully.

6 Startup Order in a Distributed Cluster

Start the components in this order:

1. All PD nodes. Form the Raft group and elect a leader. Check GET /v1/ready on each node; start Store only after HTTP 200 with ready:true. /v1/health checks REST listener liveness only.
2. All Store nodes. Register with PD; wait for /v1/stores to show every target Store as Up.
3. All Server nodes. Start after PD and Store are ready.

Master Compose waits on PD /v1/health through Store depends_on: condition: service_healthy, and on Store REST liveness before Server. Server also polls PD /v1/stores for an Up Store before starting. Therefore, docker compose up --wait does not replace PD Raft readiness and Store registration checks.

PD is also the last component to stop: shut down Server, then Store, then PD.

7 Verification

7.1 REST API Authentication

Current master requires HTTP Basic Authorization for all PD REST paths except /actuator/*, /v1/health, /v1/ready, and /v1/prom/targets/*. Username must be hg, store, hubble, or vermeer, and password must equal auth.secret-key. A missing secret or wrong password returns HTTP 401. Query Stores using the same secret configured at PD startup:

{"status": -1, "error": "Unauthorized"}

curl -u "store:${HG_PD_AUTH_SECRET_KEY:?Set the PD deployment secret first}" \
  http://localhost:8620/v1/stores

bin/wait-storage.sh uses the same credentials through PD_AUTH_USER and PD_AUTH_PASSWORD. Release 1.7.0 checks only internal service usernames and does not compare passwords; do not apply this older behavior to master builds.

7.2 Health Checks

Unauthenticated GET /v1/health returns 200 with an empty body, confirming only PD REST listener startup:

curl -i http://localhost:8620/v1/health

Spring Boot Actuator also provides a readable health response:

curl http://localhost:8620/actuator/health

Actuator {"status":"UP"} does not confirm an available PD Raft leader. Master GET /v1/ready returns HTTP 200 with ready:true when the PD Raft node is active and sees a leader; otherwise it returns HTTP 503 with ready:false:

curl -i http://localhost:8620/v1/ready

7.3 Cluster and member status

Check the PD members and which node is the Raft leader:

curl -u "store:${HG_PD_AUTH_SECRET_KEY:?Set the PD deployment secret first}" \
  http://localhost:8620/v1/members

The response carries pdList, the elected pdLeader, numOfService, numOfNormalService and a stateCountMap. In a healthy 3-node PD cluster numOfService and numOfNormalService are both 3 and exactly one member has role: "Leader".

GET /v1/cluster returns the same member list together with the Store list, graph list and overall cluster state, and GET / returns a short summary (leader address, cluster state, member count, store count, graph count, partition count).

7.4 Store status

You can also verify Store node status through the PD API:

curl -u "store:${HG_PD_AUTH_SECRET_KEY:?Set the PD deployment secret first}" \
  http://localhost:8620/v1/stores

If the response shows state as Up, the corresponding Store node is running normally. The example below shows a single Store node. In a healthy 3-node deployment, the storeId list should contain three IDs, and stateCountMap.Up, numOfService, and numOfNormalService should all be 3.

{
  "message": "OK",
  "data": {
    "stores": [
      {
        "storeId": 8319292642220586694,
        "address": "127.0.0.1:8500",
        "raftAddress": "127.0.0.1:8510",
        "version": "",
        "state": "Up",
        "deployPath": "/Users/{your_user_name}/hugegraph/apache-hugegraph-incubating-1.7.0/apache-hugegraph-store-incubating-1.7.0/lib/hg-store-node-1.7.0.jar",
        "dataPath": "./storage",
        "startTimeStamp": 1754027127969,
        "registedTimeStamp": 1754027127969,
        "lastHeartBeat": 1754027909444,
        "capacity": 494384795648,
        "available": 346535829504,
        "partitionCount": 0,
        "graphSize": 0,
        "keyCount": 0,
        "leaderCount": 0,
        "serviceName": "127.0.0.1:8500-store",
        "serviceVersion": "",
        "serviceCreatedTimeStamp": 1754027127000,
        "partitions": []
      }
    ],
    "stateCountMap": {
      "Up": 1
    },
    "numOfService": 1,
    "numOfNormalService": 1
  },
  "status": 0
}

7.5 Other REST endpoints

All paths below are relative to http://<pd-host>:8620 and need the Basic header from section 7.1 unless noted.

The two log endpoints take a time range as {"startTime": "...", "endTime": "..."}; yyyy-MM-dd HH:mm:ss and yyyy-MM-dd are among the accepted formats.

PD registers its own meters under the hg prefix, so /actuator/prometheus exposes hg_up, hg_graphs, hg_stores and hg_terms alongside the standard JVM metrics, plus per-graph partition and size meters once graphs exist.