HugeGraph (OLTP)

• 1: HugeGraph Server Quick Start
• 2: HugeGraph-PD Quick Start
• 3: HugeGraph-Store Quick Start

1 - HugeGraph Server Quick Start

1 HugeGraph Server Overview

apache/hugegraph is the main repository for the HugeGraph graph database. Its top-level modules include hugegraph-server, hugegraph-pd, and hugegraph-store. This page describes the hugegraph-server module and the service it runs.

The hugegraph-server module contains hugegraph-core, hugegraph-api, hugegraph-dist, and storage adapters. Core implements the property graph model, transactions, and TinkerPop interfaces. API provides the HTTP service and delegates client requests to Core. Graph data is stored in RocksDB (the default standalone backend), HStore (distributed), or HBase.

⚠️ Version note: This page follows HugeGraph 1.7.0 through the master branch and covers only RocksDB, HStore, and HBase. For other legacy backends and their configuration, see the HugeGraph 1.5.x documentation.

Naming: HugeGraph means the overall project or main repository, hugegraph-server is the Server module in that repository, and HugeGraphServer is the Java class for the service process. This page uses “Server service” for a running graph database service.

2 Dependency for Building/Running

2.1 Install Java 11 (JDK 11)

The hugegraph-server module in HugeGraph 1.7.0 is compiled with Java 11. Running and building it from source require Java 11 or later.

Before continuing, run java -version to confirm your JDK version.

Java 8 is no longer supported starting from 1.7.0. bin/hugegraph-server.sh refuses to start on anything older than Java 11.

The security check is on by default and installs HugeSecurityManager, which needs Java 11 to 23. JDK 24 removed the Security Manager (JEP 486), so on Java 24 or later you must start the service with the check disabled: bin/start-hugegraph.sh -s false.

Building from source also needs Maven 3.5.0 or later.

3 Deploy

There are four ways to deploy the Server service:

• Method 1: Use Docker container (Convenient for Test/Dev)
• Method 2: Download the binary tarball
• Method 3: Source code compilation
• Method 4: One-click deployment

Do not expose Gremlin, Cypher, or other query endpoints directly to the public Internet. In production, enable authentication and authorization, restrict network access, and retain audit logs. See the Security Guide for deployment guidance.

3.1 Use Docker container (Convenient for Test/Dev)

You can refer to the Docker deployment guide.

You can use docker run -itd --name=server -p 8080:8080 -e PASSWORD=xxx hugegraph/hugegraph:1.7.0 to quickly start a Server instance using the RocksDB backend.

Optional:

1. You can use docker exec -it server bash to enter the container for troubleshooting or other maintenance operations.
2. You can use docker run -itd --name=server -p 8080:8080 -e PRELOAD="true" hugegraph/hugegraph:1.7.0 to preload a built-in sample graph at startup. You can verify it through the RESTful API. See 5.1.4 for details.
3. You can use -e PASSWORD=xxx to enable authentication mode and set the admin password. See Config Authentication for details.

If you use Docker Desktop, you can set the options as follows:

Note: The Docker Compose files use bridge networking (hg-net) and work on Linux and Mac (Docker Desktop). For the 3-node distributed cluster on Mac (Docker Desktop), allocate at least 12 GB of memory (Settings → Resources → Memory). On Linux, Docker uses host memory directly.

If you want a single, unified setup for multiple HugeGraph services, you can use docker compose. Four compose files are available in the docker/ directory:

cd hugegraph/docker
# Keep the version aligned with the latest release, for example 1.x.0
HUGEGRAPH_VERSION=1.7.0 docker compose -f docker-compose.yml up -d --wait

The standalone topology publishes the Server on port 8080 and Hubble on 127.0.0.1:8088. HUGEGRAPH_VERSION selects the Server, PD, and Store image tags; Hubble is selected separately with HUBBLE_IMAGE.

The compose files read the administrator password from HUGEGRAPH_ADMIN_PASSWORD and the JWT secret from HUGEGRAPH_AUTH_TOKEN_SECRET, normally kept in a docker/.env file. A non-empty HUGEGRAPH_ADMIN_PASSWORD turns authentication on, and Hubble detects that mode by itself. With plain docker run, pass -e PASSWORD=xxx instead.

See docker/README.md for the full setup guide.

Note:

1. HugeGraph Docker images are provided as a convenient way to start HugeGraph quickly, but they are not official ASF distribution artifacts. You can find more details in the ASF Release Distribution Policy.

2. We recommend using a release tag (such as 1.7.0 or 1.x.0) for stable deployments. Use the latest tag only if you want the newest features still under development.

3.2 Download the binary tarball

You could download the binary tarball from the download page of the ASF site like this:

# 1.7.0 is a historical release from the incubation period, so its file name still includes "incubating"
wget https://downloads.apache.org/hugegraph/1.7.0/apache-hugegraph-incubating-1.7.0.tar.gz
tar zxf apache-hugegraph-incubating-1.7.0.tar.gz

# (Optional) verify the integrity with SHA512 (recommended)
shasum -a 512 apache-hugegraph-incubating-1.7.0.tar.gz
curl https://downloads.apache.org/hugegraph/1.7.0/apache-hugegraph-incubating-1.7.0.tar.gz.sha512

3.3 Source code compilation

Please ensure that the wget/curl commands are installed before compiling the source code

Download HugeGraph source code in either of the following 2 ways (so as the other HugeGraph repos/modules):

• download the stable/release version from the ASF site
• clone the unstable/latest version by GitBox(ASF) or GitHub

# Way 1. download release package from the ASF site
wget https://downloads.apache.org/hugegraph/{version}/apache-hugegraph-incubating-src-{version}.tar.gz
tar zxf *hugegraph*.tar.gz

# (Optional) verify the integrity with SHA512 (recommended)
shasum -a 512 apache-hugegraph-incubating-src-{version}.tar.gz
curl https://downloads.apache.org/hugegraph/{version}/apache-hugegraph-incubating-{version}-src.tar.gz.sha512

# Way2 : clone the latest code by git way (e.g GitHub)
git clone https://github.com/apache/hugegraph.git

Compile and generate tarball

cd *hugegraph
# (Optional) use "-P stage" param if you build failed with the latest code(during pre-release period)
mvn package -DskipTests -ntp

A successful build includes the following line:

[INFO] BUILD SUCCESS

After a successful build, the generated distribution is the *hugegraph-*.tar.gz file in the repository root.

The default build bundles the rocksdb, hbase, and hstore backend modules, and records them in the backends option of the backend.properties resource inside the hugegraph-dist jar. To build a smaller distribution that carries RocksDB only, add -Drocksdb-only:

mvn package -DskipTests -ntp -Drocksdb-only

# download toolchain binary package, it includes loader + tool + hubble
# please check the latest version (e.g. here is 1.7.0)
wget https://downloads.apache.org/hugegraph/1.7.0/apache-hugegraph-toolchain-incubating-1.7.0.tar.gz
tar zxf *hugegraph-*.tar.gz

# enter the tool's package
cd *hugegraph*/*tool* 

bin/hugegraph deploy -v {hugegraph-version} -p {install-path} [-u {download-path-prefix}]

4 Config

If you need to quickly start HugeGraph just for testing, then you only need to modify a few configuration items (see next section). For detailed configuration introduction, please refer to configuration document and introduction to configuration items

5 Startup

5.1 Use a startup script to startup

Startup is divided into “first startup” and “non-first startup”. On the first startup, you need to initialize the backend database before starting the service.

If the service was stopped manually, or needs to be started again for any other reason, you can usually start it directly because the backend database is persistent.

When HugeGraphServer starts, it connects to the backend storage and checks its version information. If the backend has not been initialized, or if it was initialized with an incompatible version (for example, old-version data), HugeGraphServer will fail to start and report an error.

If you need to access HugeGraphServer externally, modify the restserver.url configuration item in rest-server.properties (the default is http://127.0.0.1:8080) and change it to the machine name or IP address.

Since the configuration (hugegraph.properties) and startup steps required by various backends are slightly different, the following will introduce the configuration and startup of each backend one by one.

Note: Configure Server Authentication before starting HugeGraphServer if you need Auth mode (especially for production or public network environments).

5.1.1 Distributed Storage (HStore)

backend=hstore
serializer=binary

# PD service address, multiple PD addresses are separated by commas, configure PD's RPC port
pd.peers=127.0.0.1:8686,127.0.0.1:8687,127.0.0.1:8688

# Simple example (with authentication)
gremlin.graph=org.apache.hugegraph.auth.HugeFactoryAuthProxy

# Specify storage backend hstore
backend=hstore
serializer=binary
store=hugegraph

# pd config
pd.peers=127.0.0.1:8686

usePD=true
# load the hugegraph.properties above from the graphs directory; the source default is false
graph.load_from_local_config=true

# notice: must have this conf in 1.7.0
pd.peers=127.0.0.1:8686,127.0.0.1:8687,127.0.0.1:8688
# If auth is needed
# auth.authenticator=org.apache.hugegraph.auth.StandardAuthenticator

usePD=true
restserver.url=http://127.0.0.1:8081
gremlinserver.url=http://127.0.0.1:8181
pd.peers=127.0.0.1:8686

rpc.server_host=127.0.0.1
rpc.server_port=8091

server.id=server-1
server.role=master

usePD=true
restserver.url=http://127.0.0.1:8082
gremlinserver.url=http://127.0.0.1:8182
pd.peers=127.0.0.1:8686

rpc.server_host=127.0.0.1
rpc.server_port=8092

server.id=server-2
server.role=worker

host: 127.0.0.1
port: 8181

host: 127.0.0.1
port: 8182

cd *hugegraph-${version}
bin/init-store.sh

bin/start-hugegraph.sh

curl http://localhost:8081/graphspaces/DEFAULT/graphs
# Should return: {"graphs":["hugegraph"]}

bin/stop-hugegraph.sh

cd hugegraph/docker
HUGEGRAPH_VERSION=1.7.0 docker compose -f docker-compose-3pd-3store-3server.yml up -d --wait

# Server configuration, shared by server0, server1 and server2
HG_SERVER_BACKEND: hstore
HG_SERVER_PD_PEERS: pd0:8686,pd1:8686,pd2:8686
HG_SERVER_CLUSTER: hg
HG_SERVER_USE_PD: "true"
HG_SERVER_MIN_FREE_MEMORY: "0"
HG_SERVER_INIT_STORE_ENABLED: "false"
HG_SERVER_REQUIRE_AUTH_TOKEN_SECRET: "true"
STORE_REST: store0:8520
# per node, for example on server0
HG_SERVER_REST_URL: http://server0:8080

curl http://localhost:8080/versions
curl http://localhost:8620/v1/stores

5.1.2 RocksDB / ToplingDB

backend=rocksdb
serializer=binary
rocksdb.data_path=.
rocksdb.wal_path=.

cd *hugegraph-${version}
bin/init-store.sh

bin/start-hugegraph.sh
Starting HugeGraphServer in daemon mode...
Connecting to HugeGraphServer (http://127.0.0.1:8080/graphs)....OK
Started [pid 21614]

5.1.3 HBase

backend=hbase
serializer=hbase

# hbase backend config
hbase.hosts=localhost
hbase.port=2181
# Note: recommend to modify the HBase partition number by the actual/env data amount & RS amount before init store
# it may influence the loading speed a lot
#hbase.enable_partition=true
#hbase.vertex_partitions=10
#hbase.edge_partitions=30

cd *hugegraph-${version}
bin/init-store.sh

bin/start-hugegraph.sh
Starting HugeGraphServer in daemon mode...
Connecting to HugeGraphServer (http://127.0.0.1:8080/graphs)....OK
Started [pid 21614]

5.1.4 Create an example graph when startup

Pass the -p true argument when starting the script to enable preload, which creates a sample graph.

bin/start-hugegraph.sh -p true
Starting HugeGraphServer in daemon mode...
Connecting to HugeGraphServer (http://127.0.0.1:8080/graphs)......OK

And use the RESTful API to request HugeGraphServer and get the following result:

> curl "http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices" | gunzip

{"vertices":[{"id":"2:lop","label":"software","type":"vertex","properties":{"name":"lop","lang":"java","price":328}},{"id":"1:josh","label":"person","type":"vertex","properties":{"name":"josh","age":32,"city":"Beijing"}},{"id":"1:marko","label":"person","type":"vertex","properties":{"name":"marko","age":29,"city":"Beijing"}},{"id":"1:peter","label":"person","type":"vertex","properties":{"name":"peter","age":35,"city":"Shanghai"}},{"id":"1:vadas","label":"person","type":"vertex","properties":{"name":"vadas","age":27,"city":"Hongkong"}},{"id":"2:ripple","label":"software","type":"vertex","properties":{"name":"ripple","lang":"java","price":199}}]}

This indicates the successful creation of the sample graph.

5.1.5 Startup script options

bin/start-hugegraph.sh accepts the following options. Every one of them takes a value, so write -d false, not a bare -d.

bin/stop-hugegraph.sh accepts -m true|false (default true), which controls whether the cron monitor task is removed along with the service.

5.2 Use Docker to startup

In 3.1 Use Docker container, we introduced how to deploy hugegraph-server with Docker. You can also switch storage backends or preload a sample graph by setting the corresponding parameters.

5.2.1 Create an example graph when starting a server

Set the environment variable PRELOAD=true when starting Docker so that sample data is loaded during startup.

1. Use docker run

   Use docker run -itd --name=server -p 8080:8080 -e PRELOAD=true hugegraph/hugegraph:1.7.0

2. Use docker-compose

   Create a docker-compose.yml file like the following and set PRELOAD=true in the environment. example.groovy is a predefined script used to preload sample data. If needed, you can mount a new example.groovy script to change the preload data.

   version: '3'
   services:
     server:
       image: hugegraph/hugegraph:1.7.0
       container_name: server
       environment:
         - PRELOAD=true
         - PASSWORD=xxx
       volumes:
         - /path/to/yourscript:/hugegraph-server/scripts/example.groovy
       ports:
         - 8080:8080

   Use docker compose up -d to start the container.

And use the RESTful API to request HugeGraphServer and get the following result:

> curl "http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices" | gunzip

{"vertices":[{"id":"2:lop","label":"software","type":"vertex","properties":{"name":"lop","lang":"java","price":328}},{"id":"1:josh","label":"person","type":"vertex","properties":{"name":"josh","age":32,"city":"Beijing"}},{"id":"1:marko","label":"person","type":"vertex","properties":{"name":"marko","age":29,"city":"Beijing"}},{"id":"1:peter","label":"person","type":"vertex","properties":{"name":"peter","age":35,"city":"Shanghai"}},{"id":"1:vadas","label":"person","type":"vertex","properties":{"name":"vadas","age":27,"city":"Hongkong"}},{"id":"2:ripple","label":"software","type":"vertex","properties":{"name":"ripple","lang":"java","price":199}}]}

This indicates that the sample graph was created successfully.

6. Access server

6.1 Service startup status check

Use jps to see a service process

jps
6475 HugeGraphServer

curl request RESTfulAPI

echo `curl -o /dev/null -s -w %{http_code} "http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices"`

Return 200, which means the server starts normally.

6.2 Request Server

The RESTful API of HugeGraphServer includes various types of resources, typically including graph, schema, gremlin, traverser and task.

• graph contains vertices、edges
• schema contains vertexlabels、 propertykeys、 edgelabels、indexlabels
• gremlin contains various Gremlin statements, such as g.v(), which can be executed synchronously or asynchronously
• traverser contains various advanced queries including shortest paths, intersections, N-step reachable neighbors, etc.
• task contains query and delete with asynchronous tasks

6.2.1 Get vertices and its related properties in hugegraph

curl http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices

explanation

1. Since there are many vertices and edges in the graph, for list-type requests, such as getting all vertices, getting all edges, etc., the server will compress the data and return it, so when use curl, you get a bunch of garbled characters, you can redirect to gunzip for decompression. It is recommended to use the Chrome browser + Restlet plugin to send HTTP requests for testing.

   curl "http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices" | gunzip

2. The current default configuration of HugeGraphServer can only be accessed locally, and the configuration can be modified so that it can be accessed on other machines.

   vim conf/rest-server.properties
   
   restserver.url=http://0.0.0.0:8080

response body:

{
    "vertices": [
        {
            "id": "2lop",
            "label": "software",
            "type": "vertex",
            "properties": {
                "price": [
                    {
                        "id": "price",
                        "value": 328
                    }
                ],
                "name": [
                    {
                        "id": "name",
                        "value": "lop"
                    }
                ],
                "lang": [
                    {
                        "id": "lang",
                        "value": "java"
                    }
                ]
            }
        },
        {
            "id": "1josh",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": [
                    {
                        "id": "name",
                        "value": "josh"
                    }
                ],
                "age": [
                    {
                        "id": "age",
                        "value": 32
                    }
                ]
            }
        },
        ...
    ]
}

For the detailed API, please refer to RESTful-API

You can also visit localhost:8080/swagger-ui/index.html to check the API.

When using Swagger UI to debug the API provided by HugeGraph, if HugeGraph Server turns on authentication mode, you can enter authentication information on the Swagger page.

Currently, HugeGraph supports setting authentication information in two forms: Basic and Bearer.

7 Stop Server

cd apache-hugegraph-incubating-1.7.0/apache-hugegraph-server-incubating-1.7.0
bin/stop-hugegraph.sh

8 Debug Server with IntelliJ IDEA

Please refer to Setup Server in IDEA

2 - HugeGraph-PD Quick Start

1 HugeGraph-PD Overview

HugeGraph-PD (Placement Driver) is the metadata management component of HugeGraph’s distributed version, responsible for managing the distribution of graph data and coordinating storage nodes. It plays a central role in distributed HugeGraph, maintaining cluster status and coordinating HugeGraph-Store storage nodes.

PD keeps cluster metadata in an embedded RocksDB store under pd.data-path and replicates it across PD nodes with Raft, so a 3-node or 5-node PD cluster keeps serving while a minority of nodes is down. On top of that it registers and activates Store nodes, allocates and rebalances partitions, tracks Store heartbeats, and answers service discovery queries from Store and Server.

PD listens on three ports:

2 Prerequisites

2.1 Requirements

• Operating System: Linux or macOS (Windows has not been fully tested)
• Java version: ≥ 11
• Maven version: ≥ 3.5.0

3 Deployment

There are two ways to deploy the HugeGraph-PD component:

• Method 1: Download the tar package
• Method 2: Compile from source

3.1 Download the tar package

Download the latest version of HugeGraph-PD from the Apache HugeGraph official download page:

# 1.7.0 is a historical release from the incubation period, so its file and directory names still include "incubating"
wget https://downloads.apache.org/hugegraph/1.7.0/apache-hugegraph-incubating-1.7.0.tar.gz
tar zxf apache-hugegraph-incubating-1.7.0.tar.gz
cd apache-hugegraph-incubating-1.7.0/apache-hugegraph-pd-incubating-1.7.0

3.2 Compile from source

# 1. Clone the source code
git clone https://github.com/apache/hugegraph.git

# 2. Build the project
cd hugegraph
mvn clean install -DskipTests=true

# 3. After a successful build, the PD directory and packages are located at
#    hugegraph-pd/apache-hugegraph-pd-{version}          (unpacked PD distribution)
#    hugegraph-pd/apache-hugegraph-pd-{version}.tar.gz   (PD only package, Linux build hosts only)
#    target/apache-hugegraph-{version}.tar.gz            (PD + Store + Server package)

To build only the PD distribution and the modules it depends on:

mvn clean package -pl hugegraph-pd/hg-pd-dist -am -DskipTests

The unpacked distribution contains just three directories: bin (start and stop scripts), conf (application.yml, application.yml.template, log4j2.xml, verify-license.json) and lib (the hg-pd-service jar).

3.3 Docker Deployment

The HugeGraph-PD Docker image is available on Docker Hub as hugegraph/pd.

Note: The following steps assume you have already cloned or pulled the HugeGraph main repository locally, or at least have its docker/ directory available.

Use the docker compose setup to deploy the complete 3-node cluster (PD + Store + Server):

cd hugegraph/docker
# Keep the version aligned with the latest release, for example 1.x.0
HUGEGRAPH_VERSION=1.7.0 docker compose -f docker-compose-3pd-3store-3server.yml up -d

A single PD plus a single Store and Server is also available as docker-compose-hstore.yml.

To run a single PD node via docker run, configuration is provided via environment variables:

docker run -d \
  -p 8620:8620 \
  -p 8686:8686 \
  -p 8610:8610 \
  -e HG_PD_GRPC_HOST=<your-ip> \
  -e HG_PD_RAFT_ADDRESS=<your-ip>:8610 \
  -e HG_PD_RAFT_PEERS_LIST=<your-ip>:8610 \
  -e HG_PD_INITIAL_STORE_LIST=<store-ip>:8500 \
  -v /path/to/data:/hugegraph-pd/pd_data \
  --name hugegraph-pd \
  hugegraph/pd:1.7.0

Environment variable reference:

The entrypoint refuses to start if any of the four required variables is missing, and it turns the values above into a SPRING_APPLICATION_JSON override, so the packaged conf/application.yml does not need editing. Any key not covered by an HG_PD_* variable keeps the value from that file. JAVA_OPTS is passed through to the JVM.

Note: In Docker bridge networking, use container hostnames (e.g. pd0) for HG_PD_GRPC_HOST and HG_PD_RAFT_ADDRESS instead of IP addresses.

Deprecated aliases: GRPC_HOST, RAFT_ADDRESS, RAFT_PEERS, PD_INITIAL_STORE_LIST still work but log a deprecation warning. Use the HG_PD_* names for new deployments.

The image ships a HEALTHCHECK that polls GET /v1/health on port 8620 every 15 seconds, with a 90 second start period and 3 retries, so docker ps reports real PD health. The entrypoint runs the start script with -d false, so the container process is Java itself and Docker’s restart policy fires when it dies. The image also sets STDOUT_MODE=true, so docker logs <container-name> (e.g. docker logs hg-pd0) shows the PD log without exec-ing into the container.

See docker/README.md for the full cluster setup guide.

4 Configuration

The main configuration file for PD is conf/application.yml. This is the file the distribution ships:

spring:
  application:
    name: hugegraph-pd

management:
  metrics:
    export:
      prometheus:
        enabled: true
  endpoints:
    web:
      exposure:
        include: "*"

logging:
  config: 'file:./conf/log4j2.xml'

license:
  verify-path: ./conf/verify-license.json
  license-path: ./conf/hugegraph.license

grpc:
  # gRPC port for cluster mode
  port: 8686
  # Change to the actual local IPv4 address when deploying
  host: 127.0.0.1

server:
  # REST service port
  port: 8620

pd:
  # Storage path
  data-path: ./pd_data
  # Auto-expansion check cycle (seconds)
  patrol-interval: 1800
  # Minimum number of Store nodes required for cluster availability
  initial-store-count: 1
  # Store configuration information, format is IP:gRPC port
  initial-store-list: 127.0.0.1:8500

raft:
  # Raft address of this node
  address: 127.0.0.1:8610
  # Raft addresses of all PD nodes in the cluster
  peers-list: 127.0.0.1:8610

store:
  # Store offline time (seconds). After this time, the store is considered permanently unavailable
  max-down-time: 172800
  # Whether to enable store monitoring data storage
  monitor_data_enabled: true
  # Monitoring data interval
  monitor_data_interval: 1 minute
  # Monitoring data retention time
  monitor_data_retention: 1 day

partition:
  # Default number of replicas per partition
  default-shard-count: 1
  # Default maximum number of replicas per machine
  store-max-shard-count: 12

conf/application.yml.template is a second, unused copy with placeholders ($GRPC_PORT$, $RAFT_ADDRESS$ and so on) for deployment tooling that generates the file. PD always reads conf/application.yml, which the start script passes as -Dspring.config.location.

4.1 Configuration reference

Keys not present in conf/application.yml fall back to the built-in default listed below. Keys with no built-in default must be present, otherwise PD fails to start.

gRPC and REST

application.yml.template also carries grpc.netty-server.max-inbound-message-size: 100MB, but PD sets the gRPC server’s inbound message limit to 1 GB in code, so that key has no effect.

Raft

PD core

Store management

Partitions

The initial partition count is derived from these two values and the size of pd.initial-store-list:

initial partitions = store count * partition.store-max-shard-count / partition.default-shard-count

Discovery, license and metrics

Thread pools

4.2 Single-node configuration

The shipped conf/application.yml already is a working single-node configuration. It is meant for development and testing: one PD node has no Raft quorum to lose, and partition.default-shard-count: 1 keeps a single replica per partition.

grpc:
  host: 127.0.0.1
  port: 8686
server:
  port: 8620
raft:
  address: 127.0.0.1:8610
  peers-list: 127.0.0.1:8610
pd:
  data-path: ./pd_data
  initial-store-count: 1
  initial-store-list: 127.0.0.1:8500
partition:
  default-shard-count: 1

4.3 Three-node cluster configuration

For a production cluster run 3 or 5 PD nodes, an odd number so Raft always has a quorum. A 3-node cluster tolerates one node failure. raft.peers-list must list every node and must be byte-for-byte identical on all of them, while grpc.host and raft.address differ per node.

Node 1 (192.168.1.10):

grpc:
  host: 192.168.1.10
  port: 8686
server:
  port: 8620
raft:
  address: 192.168.1.10:8610
  peers-list: 192.168.1.10:8610,192.168.1.11:8610,192.168.1.12:8610
pd:
  data-path: /data/pd
  initial-store-count: 3
  initial-store-list: 192.168.1.20:8500,192.168.1.21:8500,192.168.1.22:8500
partition:
  default-shard-count: 3

Node 2 (192.168.1.11) and node 3 (192.168.1.12) use the same file with grpc.host and raft.address changed to their own address:

# node 2
grpc:
  host: 192.168.1.11
raft:
  address: 192.168.1.11:8610
  peers-list: 192.168.1.10:8610,192.168.1.11:8610,192.168.1.12:8610

# node 3
grpc:
  host: 192.168.1.12
raft:
  address: 192.168.1.12:8610
  peers-list: 192.168.1.10:8610,192.168.1.11:8610,192.168.1.12:8610

To put all three PD nodes on one machine for testing, give each node its own pd.data-path and its own ports, for example raft 8610/8611/8612, gRPC 8686/8687/8688 and REST 8620/8621/8622.

In Docker bridge networking the same configuration comes from environment variables and uses container hostnames instead of IP addresses:

# pd0
HG_PD_GRPC_HOST: pd0
HG_PD_RAFT_ADDRESS: pd0:8610
HG_PD_RAFT_PEERS_LIST: pd0:8610,pd1:8610,pd2:8610
HG_PD_INITIAL_STORE_LIST: store0:8500,store1:8500,store2:8500
HG_PD_INITIAL_STORE_COUNT: 3

# pd1
HG_PD_GRPC_HOST: pd1
HG_PD_RAFT_ADDRESS: pd1:8610
HG_PD_RAFT_PEERS_LIST: pd0:8610,pd1:8610,pd2:8610

# pd2
HG_PD_GRPC_HOST: pd2
HG_PD_RAFT_ADDRESS: pd2:8610
HG_PD_RAFT_PEERS_LIST: pd0:8610,pd1:8610,pd2:8610

5 Start and Stop

5.1 Start PD

In the PD installation directory, execute:

./bin/start-hugegraph-pd.sh

The script requires a JDK of at least version 11 on PATH or in JAVA_HOME, and it exits without doing anything if it finds a Java process already using this installation’s conf directory.

Supported flags:

The -d flag controls daemon mode:

• -d true (default): run as a background daemon; the script returns immediately.
• -d false: run in foreground. The script execs Java, so the container or supervisor process IS Java. Use this when running under Docker or a process supervisor (systemd, supervisord) so crashes are detected and the service is restarted automatically.

Each flag also has an environment variable equivalent: DAEMON, GC_OPTION, USER_OPTION and OPEN_TELEMETRY. Setting JAVA_OPTIONS replaces the computed heap settings entirely; otherwise the script sizes the heap between 512 MB and 32 GB from available memory. Setting STDOUT_MODE=true leaves the JVM output on stdout instead of redirecting it to logs/hugegraph-pd-stdout.log, which is what the Docker image does.

After successful startup, you can see logs similar to the following in logs/hugegraph-pd-stdout.log:

YYYY-mm-dd xx:xx:xx [main] [INFO] o.a.h.p.b.HugePDServer - Started HugePDServer in x.xxx seconds (JVM running for x.xxx)

The process id is written to bin/pid.

5.2 Stop PD

In the PD installation directory, execute:

./bin/stop-hugegraph-pd.sh

The script reads bin/pid, sends the process a termination signal, waits up to 30 seconds for it to exit, and removes the pid file. If bin/pid is missing it reports that and exits successfully.

6 Startup Order in a Distributed Cluster

Start the components in this order:

1. All PD nodes. They form the Raft group and elect a leader. Wait until every node answers GET /v1/health.
2. All Store nodes. Each Store registers with PD over gRPC, and PD activates the ones listed in pd.initial-store-list. Wait until GET /v1/stores reports "state": "Up" for every Store.
3. All Server nodes. A Server reads pd.peers and depends on PD reporting at least one live Store before partitions can be assigned.

The Docker Compose topologies enforce exactly this. Store containers wait on PD’s /v1/health healthcheck through depends_on with condition: service_healthy, Server containers wait the same way on the Store healthcheck, and the Server entrypoint then polls PD’s /v1/stores until a Store reports Up before it starts HugeGraph.

PD is also the last component to stop: shut down Server, then Store, then PD.

7 Verification

7.1 REST API authentication

Except for /actuator/*, /v1/health and /v1/prom/targets/*, every PD REST path requires an HTTP Basic Authorization header whose user name is one of the internal service names hg, store, hubble or vermeer. A request without the header is answered with:

{"status": -1, "error": "Unauthorized!"}

The password is not validated yet, so any value works. The Server’s own bin/wait-storage.sh uses store:admin and lets you override it with PD_AUTH_USER and PD_AUTH_PASSWORD, so the examples below use the same credentials:

curl -u store:admin http://localhost:8620/v1/stores

Warning: This check is only meant to separate HugeGraph’s own components from other traffic. Do not expose the PD REST or gRPC ports to an untrusted network. Restrict them with firewall rules or security groups, and keep raft.ip-whitelist.enabled on so the Raft port only accepts the configured peers.

7.2 Health check

GET /v1/health needs no credentials and is what the Docker healthcheck uses. It answers 200 with an empty body:

curl -i http://localhost:8620/v1/health

The Spring Boot actuator endpoint also works and is more readable:

curl http://localhost:8620/actuator/health

If it returns {"status":"UP"}, it indicates that the PD service has been successfully started.

7.3 Cluster and member status

Check the PD members and which node is the Raft leader:

curl -u store:admin http://localhost:8620/v1/members

The response carries pdList, the elected pdLeader, numOfService, numOfNormalService and a stateCountMap. In a healthy 3-node PD cluster numOfService and numOfNormalService are both 3 and exactly one member has role: "Leader".

GET /v1/cluster returns the same member list together with the Store list, graph list and overall cluster state, and GET / returns a short summary (leader address, cluster state, member count, store count, graph count, partition count).

7.4 Store status

You can also verify Store node status through the PD API:

curl -u store:admin http://localhost:8620/v1/stores

If the response shows state as Up, the corresponding Store node is running normally. The example below shows a single Store node. In a healthy 3-node deployment, the storeId list should contain three IDs, and stateCountMap.Up, numOfService, and numOfNormalService should all be 3.

{
  "message": "OK",
  "data": {
    "stores": [
      {
        "storeId": 8319292642220586694,
        "address": "127.0.0.1:8500",
        "raftAddress": "127.0.0.1:8510",
        "version": "",
        "state": "Up",
        "deployPath": "/Users/{your_user_name}/hugegraph/apache-hugegraph-incubating-1.7.0/apache-hugegraph-store-incubating-1.7.0/lib/hg-store-node-1.7.0.jar",
        "dataPath": "./storage",
        "startTimeStamp": 1754027127969,
        "registedTimeStamp": 1754027127969,
        "lastHeartBeat": 1754027909444,
        "capacity": 494384795648,
        "available": 346535829504,
        "partitionCount": 0,
        "graphSize": 0,
        "keyCount": 0,
        "leaderCount": 0,
        "serviceName": "127.0.0.1:8500-store",
        "serviceVersion": "",
        "serviceCreatedTimeStamp": 1754027127000,
        "partitions": []
      }
    ],
    "stateCountMap": {
      "Up": 1
    },
    "numOfService": 1,
    "numOfNormalService": 1
  },
  "status": 0
}

7.5 Other REST endpoints

All paths below are relative to http://<pd-host>:8620 and need the Basic header from section 7.1 unless noted.

The two log endpoints take a time range as {"startTime": "...", "endTime": "..."}; yyyy-MM-dd HH:mm:ss and yyyy-MM-dd are among the accepted formats.

PD registers its own meters under the hg prefix, so /actuator/prometheus exposes hg_up, hg_graphs, hg_stores and hg_terms alongside the standard JVM metrics, plus per-graph partition and size meters once graphs exist.

3 - HugeGraph-Store Quick Start

1 HugeGraph-Store Overview

HugeGraph-Store is the storage node component of HugeGraph’s distributed version, responsible for actually storing and managing graph data. It works in conjunction with HugeGraph-PD to form HugeGraph’s distributed storage engine, providing high availability and horizontal scalability.

Each Store node keeps graph data in RocksDB and replicates it with Raft (JRaft): every partition is a separate Raft group, so a partition survives the loss of a minority of its replicas. Store nodes do not know about each other directly. They register with PD, receive their partition assignment from PD, and report state back over a heartbeat. HugeGraph-Server reaches Store over gRPC after looking up partition locations in PD.

2 Prerequisites

2.1 Requirements

• Operating System: Linux or macOS (Windows has not been fully tested)
• Java version: ≥ 11 (enforced by the build and re-checked by bin/start-hugegraph-store.sh)
• Maven version: ≥ 3.5.0
• Deploy HugeGraph-PD first for multi-node deployment

3 Deployment

There are two ways to deploy the HugeGraph-Store component:

• Method 1: Download the tar package
• Method 2: Compile from source

3.1 Download the tar package

Download the latest version of HugeGraph-Store from the Apache HugeGraph official download page:

# 1.7.0 is a historical release from the incubation period, so its file and directory names still include "incubating"
wget https://downloads.apache.org/hugegraph/1.7.0/apache-hugegraph-incubating-1.7.0.tar.gz
tar zxf apache-hugegraph-incubating-1.7.0.tar.gz
cd apache-hugegraph-incubating-1.7.0/apache-hugegraph-store-incubating-1.7.0

3.2 Compile from source

# 1. Clone the source code
git clone https://github.com/apache/hugegraph.git

# 2. Build the project
cd hugegraph
mvn clean install -DskipTests=true

# 3. After a successful build, the Store directory and complete distribution package are located at
#    hugegraph-store/apache-hugegraph-store-{version}
#    target/apache-hugegraph-{version}.tar.gz

To build Store alone instead of the whole repository, build hugegraph-struct first, because Store depends on it:

mvn install -pl hugegraph-struct -am -DskipTests
mvn clean package -pl hugegraph-store/hg-store-dist -am -DskipTests

The assembled directory contains only bin/, conf/ and lib/hg-store-node-{version}.jar.

3.3 Docker Deployment

The HugeGraph-Store Docker image is available on Docker Hub as hugegraph/store.

Note: The following steps assume you have already cloned or pulled the HugeGraph main repository locally, or at least have its docker/ directory available.

Two compose files include Store:

cd hugegraph/docker
# Keep the version aligned with the latest release, for example 1.x.0

# Minimal distributed deployment
HUGEGRAPH_VERSION=1.7.0 docker compose -f docker-compose-hstore.yml up -d --wait

# Or the multi-node cluster
HUGEGRAPH_VERSION=1.7.0 docker compose -f docker-compose-3pd-3store-3server.yml up -d

To run a single Store node via docker run:

docker run -d \
  -p 8520:8520 \
  -p 8500:8500 \
  -p 8510:8510 \
  -e HG_STORE_PD_ADDRESS=<pd-ip>:8686 \
  -e HG_STORE_GRPC_HOST=<your-ip> \
  -e HG_STORE_RAFT_ADDRESS=<your-ip>:8510 \
  -v /path/to/storage:/hugegraph-store/storage \
  --name hugegraph-store \
  hugegraph/store:1.7.0

Environment variable reference:

The entrypoint turns these into a SPRING_APPLICATION_JSON overlay on top of conf/application.yml, then runs bin/start-hugegraph-store.sh -d false -j "$JAVA_OPTS". Any key not covered by the table above still has to be edited in conf/application.yml, or supplied through your own SPRING_APPLICATION_JSON.

Image details:

• JAVA_OPTS defaults to -XX:+UnlockExperimentalVMOptions -XX:+UseContainerSupport -XX:MaxRAMPercentage=50 -XshowSettings:vm
• STDOUT_MODE=true, so Java logs go to the container stdout instead of logs/hugegraph-store-server.log
• HEALTHCHECK calls GET http://localhost:8520/v1/health every 15s after a 90s start period
• The image declares EXPOSE 8520; publish 8500 and 8510 yourself when Server or other Store nodes need to reach the container from outside the Docker network

Note: In Docker bridge networking, use container hostnames (e.g. store0) for HG_STORE_GRPC_HOST instead of IP addresses.

Deprecated aliases: PD_ADDRESS, GRPC_HOST, RAFT_ADDRESS still work but log a deprecation warning. Use the HG_STORE_* names for new deployments.

4 Configuration

Store reads two files from conf/:

• application.yml, the main configuration file (PD address, ports, Raft, data path)
• application-pd.yml, pulled in by spring.profiles.include: pd in application.yml, holding the RocksDB memory settings and the Actuator exposure

4.1 application.yml

This is the file shipped in the distribution package:

pdserver:
  # PD service address, multiple PD addresses separated by commas
  address: localhost:8686

management:
  metrics:
    export:
      prometheus:
        enabled: true
  endpoints:
    web:
      exposure:
        include: "*"

grpc:
  # grpc service address
  host: 127.0.0.1
  port: 8500
  netty-server:
    max-inbound-message-size: 1000MB
raft:
  # raft cache queue size
  disruptorBufferSize: 1024
  address: 127.0.0.1:8510
  max-log-file-size: 600000000000
  # Snapshot generation interval, in seconds
  snapshotInterval: 1800
server:
  # rest service address
  port: 8520

app:
  # Storage path, support multiple paths, separated by commas
  data-path: ./storage
  #raft-path: ./storage

spring:
  application:
    name: store-node-grpc-server
  profiles:
    active: default
    include: pd

logging:
  config: 'file:./conf/log4j2.xml'
  level:
    root: info

4.2 application-pd.yml

management:
  metrics:
    export:
      prometheus:
        enabled: true
  endpoints:
    web:
      exposure:
        include: "*"

rocksdb:
  # rocksdb total memory usage, force flush to disk when reaching this value
  total_memory_size: 32000000000
  # memtable size used by rocksdb
  write_buffer_size: 32000000
  # For each rocksdb, the number of memtables reaches this value for writing to disk.
  min_write_buffer_number_to_merge: 16

4.3 Configuration reference

“Shipped” is the value in the two files above. “Code default” is what the node falls back to when the key is absent, and is the value to rely on for keys the template does not list.

Core

Raft

Storage and labels

RocksDB

Any other option defined in org/apache/hugegraph/rocksdb/access/RocksDBOptions.java can be added under the same rocksdb: block, for example rocksdb.max_background_jobs, rocksdb.level0_file_num_compaction_trigger or rocksdb.bloom_filter_bits_per_key.

Thread pools

Query pushdown

Background jobs

Built-in PD mode

Only for single-node development and debugging, activated by app.fake-pd: true. The node then plays PD’s role itself and ignores pdserver.address.

Diagnostics

4.4 Per-node changes

For multi-node deployment, you need to modify the following configurations for each Store node:

1. grpc.host and grpc.port (the address other components dial)
2. raft.address (Raft protocol address)
3. server.port (REST port)
4. app.data-path (data storage path)

pdserver.address is the same on every node, it lists the whole PD cluster.

5 Start and Stop

5.1 Start Store

Ensure that the PD service is already started, then in the Store installation directory, execute:

./bin/start-hugegraph-store.sh

The script accepts four flags:

Daemon mode:

• -d true (default): run as a background daemon. The script returns immediately and writes the Java pid to bin/pid.
• -d false: run in the foreground. The script execs Java, so the container or supervisor process is Java itself. Use this under Docker or a process supervisor (systemd, supervisord) so crashes are detected and the service is restarted automatically.

JVM memory, unless you set JAVA_OPTIONS yourself: -Xms512m, and -Xmx set to half the free memory, clamped to the 512 MB to 2048 MB range. The script also adds -XX:MetaspaceSize=256M, a heap dump on out-of-memory into logs/, and a rolling GC log at logs/gc.log. Production nodes normally need a much larger heap, so pass one explicitly, for example -j "-Xmx32g -Xms32g".

The script refuses to start if ulimit -n or ulimit -u is below 1024, and it preloads jemalloc on x86_64 and arm64 when the shared object can be downloaded and verified.

After successful startup, you can see logs similar to the following in logs/hugegraph-store-server.log:

YYYY-mm-dd xx:xx:xx [main] [INFO] o.a.h.s.n.StoreNodeApplication - Started StoreNodeApplication in x.xxx seconds (JVM running for x.xxx)

5.2 Stop Store

In the Store installation directory, execute:

./bin/stop-hugegraph-store.sh

The script reads bin/pid, signals that process, and waits up to 30 seconds for it to exit before removing the pid file. If bin/pid is missing it exits without doing anything.

5.3 Restart Store

./bin/restart-hugegraph-store.sh

It sources the stop script and then the start script, and forwards the flags from section 5.1.

5.4 Startup order

1. PD first. Each Store’s grpc.host:grpc.port should appear in PD’s pd.initial-store-list, otherwise PD registers the node in Pending state instead of bringing it to Up, and partition assignment never finishes.
2. Store next. A Store started before PD is reachable is not fatal: the heartbeat thread keeps retrying registration and logs store heartbeat error: PD UNREACHABLE until PD answers.
3. HugeGraph-Server last, once every Store node reports state: "Up". Server needs the partitions in place before it can initialize or open a graph.

The compose files encode the same order with depends_on: condition: service_healthy: Store waits for every PD healthcheck, and Server waits for every Store healthcheck.

6 Multi-Node Deployment Example

Below is a configuration example for a three-node deployment:

6.1 Three-Node Configuration Reference

• 3 PD nodes
  • raft ports: 8610, 8611, 8612
  • rpc ports: 8686, 8687, 8688
  • rest ports: 8620, 8621, 8622
• 3 Store nodes
  • raft ports: 8510, 8511, 8512
  • rpc ports: 8500, 8501, 8502
  • rest ports: 8520, 8521, 8522

6.2 Store Node Configuration

For the three Store nodes, the main configuration differences are as follows:

Node A:

grpc:
  port: 8500
raft:
  address: 127.0.0.1:8510
server:
  port: 8520
app:
  data-path: ./storage-a

Node B:

grpc:
  port: 8501
raft:
  address: 127.0.0.1:8511
server:
  port: 8521
app:
  data-path: ./storage-b

Node C:

grpc:
  port: 8502
raft:
  address: 127.0.0.1:8512
server:
  port: 8522
app:
  data-path: ./storage-c

All nodes should point to the same PD cluster:

pdserver:
  address: 127.0.0.1:8686,127.0.0.1:8687,127.0.0.1:8688

And every PD node should list all three Store gRPC addresses:

pd:
  initial-store-list: 127.0.0.1:8500,127.0.0.1:8501,127.0.0.1:8502

6.3 Docker Distributed Cluster Configuration

The distributed Store cluster definition is included in docker/docker-compose-3pd-3store-3server.yml. Each Store node gets its own hostname and environment variables:

# store0, published as 8500 (gRPC), 8510 (Raft), 8520 (REST)
HG_STORE_PD_ADDRESS: pd0:8686,pd1:8686,pd2:8686
HG_STORE_GRPC_HOST: store0
HG_STORE_GRPC_PORT: "8500"
HG_STORE_REST_PORT: "8520"
HG_STORE_RAFT_ADDRESS: store0:8510
HG_STORE_DATA_PATH: /hugegraph-store/storage

# store1, published as 8501, 8511, 8521
HG_STORE_GRPC_HOST: store1
HG_STORE_RAFT_ADDRESS: store1:8510

# store2, published as 8502, 8512, 8522
HG_STORE_GRPC_HOST: store2
HG_STORE_RAFT_ADDRESS: store2:8510

The container ports stay 8500/8510/8520 on every node, only the published host ports differ. The PD nodes set HG_PD_INITIAL_STORE_LIST: store0:8500,store1:8500,store2:8500 to match.

Store nodes start only after all PD nodes pass healthchecks (/v1/health), enforced via depends_on: condition: service_healthy.

To view runtime logs for a running Store container use docker logs <container-name> (e.g. docker logs hg-store0).

See docker/README.md for the full setup guide.

7 Verify Store Service

Confirm that the Store service is running properly:

curl http://localhost:8520/actuator/health

If it returns {"status":"UP"}, it indicates that the Store service has been successfully started.

GET /v1/health is the lighter check used by the Docker image and the compose files. It answers HTTP 200 with an empty body, so use curl -fsS and check the exit code rather than the output:

curl -fsS http://localhost:8520/v1/health && echo OK

7.1 Store REST endpoints

The Store node exposes these read-only endpoints on server.port:

Actuator and Prometheus are reachable because the shipped configuration sets management.endpoints.web.exposure.include: "*" and management.metrics.export.prometheus.enabled: true.

The node also serves maintenance endpoints that change state or run heavy work: PUT /-/state, GET /-/cleaner, GET /v1/partition/dump/{id}, GET /v1/partition/clean/{id}, POST /v1/compat?id=<partition>, GET /v1/arthasstart, POST /raft/options, and the /fix/* and /test/* groups. Use them only for troubleshooting, and keep the REST port off untrusted networks.

7.2 Check registration from PD

You can also check Store node status through the PD API:

curl -u store:admin http://localhost:8620/v1/stores

PD requires basic auth on its REST port. The user name must be one of hg, store, hubble, vermeer, and the password is not validated yet. A call with no credentials returns {"status":-1,"error":"Unauthorized!"}. Only /v1/health, /actuator/* and /v1/prom/targets/* are exempt.

If Store is configured successfully, the response should include status information for the current node, and state: "Up" means the node is running normally. A node stuck at Pending is usually missing from PD’s pd.initial-store-list.

The example below shows a single Store node. If all three nodes are configured correctly and running, the storeId list should contain three IDs, and stateCountMap.Up, numOfService, and numOfNormalService should all be 3.

{
  "message": "OK",
  "data": {
    "stores": [
      {
        "storeId": 8319292642220586694,
        "address": "127.0.0.1:8500",
        "raftAddress": "127.0.0.1:8510",
        "version": "",
        "state": "Up",
        "deployPath": "/Users/{your_user_name}/hugegraph/hugegraph-store/apache-hugegraph-store-{version}/lib/hg-store-node-{version}.jar",
        "dataPath": "./storage",
        "startTimeStamp": 1754027127969,
        "registedTimeStamp": 1754027127969,
        "lastHeartBeat": 1754027909444,
        "capacity": 494384795648,
        "available": 346535829504,
        "partitionCount": 0,
        "graphSize": 0,
        "keyCount": 0,
        "leaderCount": 0,
        "serviceName": "127.0.0.1:8500-store",
        "serviceVersion": "",
        "serviceCreatedTimeStamp": 1754027127000,
        "partitions": []
      }
    ],
    "stateCountMap": {
      "Up": 1
    },
    "numOfService": 1,
    "numOfNormalService": 1
  },
  "status": 0
}