Quick Start

• 1: HugeGraph (OLTP)
  • 1.1: HugeGraph Server Quick Start
  • 1.2: HugeGraph-PD Quick Start
  • 1.3: HugeGraph-Store Quick Start
• 2: HugeGraph ToolChain
  • 2.1: Graph visualization
  • 2.2: HugeGraph-Hubble Quick Start
  • 2.3: Graph import
    • 2.3.1: Import Graph Data with SeaTunnel Sink
  • 2.4: HugeGraph-Loader Quick Start
  • 2.5: Graph export and migration
    • 2.5.1: Export and Migrate Graph Data with SeaTunnel Source
  • 2.6: Tools Quick Start
  • 2.7: HugeGraph-Spark-Connector Quick Start
• 3: HugeGraph-AI
  • 3.1: HugeGraph-LLM
  • 3.2: HugeGraph-ML
  • 3.3: HugeGraph-LLM Workflow
  • 3.4: Configuration Reference
  • 3.5: HugeGraph-LLM REST API
  • 3.6: Vermeer Python Client
• 4: HugeGraph Computing (OLAP)
  • 4.1: HugeGraph-Vermeer Quick Start
  • 4.2: HugeGraph-Computer Quick Start
  • 4.3: HugeGraph-Computer Configuration Reference
• 5: HugeGraph Client
  • 5.1: HugeGraph-Java-Client
  • 5.2: HugeGraph Python Client Quick Start
  • 5.3: HugeGraph Go Client Quick Start

1 - HugeGraph (OLTP)

DeepWiki provides real-time updated project documentation with more comprehensive and accurate content, suitable for quickly understanding the latest project information.

📖 https://deepwiki.com/apache/hugegraph

GitHub Access: https://github.com/apache/hugegraph

1.1 - HugeGraph Server Quick Start

1 HugeGraph Server Overview

apache/hugegraph is the main repository for the HugeGraph graph database. Its top-level modules include hugegraph-server, hugegraph-pd, and hugegraph-store. This page describes the hugegraph-server module and the service it runs.

The hugegraph-server module contains hugegraph-core, hugegraph-api, hugegraph-dist, and storage adapters. Core implements the property graph model, transactions, and TinkerPop interfaces. API provides the HTTP service and delegates client requests to Core. Graph data is stored in RocksDB (the default standalone backend), HStore (distributed), or HBase.

⚠️ Version note: This page follows HugeGraph 1.7.0 through the master branch and covers only RocksDB, HStore, and HBase. For other legacy backends and their configuration, see the HugeGraph 1.5.x documentation.

Naming: HugeGraph means the overall project or main repository, hugegraph-server is the Server module in that repository, and HugeGraphServer is the Java class for the service process. This page uses “Server service” for a running graph database service.

2 Dependency for Building/Running

2.1 Install Java 11 (JDK 11)

The hugegraph-server module in HugeGraph 1.7.0 is compiled with Java 11. Running and building it from source require Java 11 or later.

Before continuing, run java -version to confirm your JDK version.

Java 8 is no longer supported starting from 1.7.0. bin/hugegraph-server.sh refuses to start on anything older than Java 11.

The security check is on by default and installs HugeSecurityManager, which needs Java 11 to 23. JDK 24 removed the Security Manager (JEP 486), so on Java 24 or later you must start the service with the check disabled: bin/start-hugegraph.sh -s false.

Building from source also needs Maven 3.5.0 or later.

3 Deploy

There are four ways to deploy the Server service:

1. Use a Docker container for test or development.
2. Download the binary tarball.
3. Compile the source code.
4. Use the legacy one-click deployment tool.

Do not expose Gremlin, Cypher, or other query endpoints directly to the public Internet. In production, enable authentication and authorization, restrict network access, and retain audit logs. See the Security Guide for deployment guidance.

3.1 Use Docker container (Convenient for Test/Dev)

You can refer to the Docker deployment guide.

You can use docker run -itd --name=server -p 8080:8080 -e PASSWORD=xxx hugegraph/hugegraph:1.7.0 to quickly start a Server instance using the RocksDB backend.

Optional:

1. You can use docker exec -it server bash to enter the container for troubleshooting or other maintenance operations.
2. You can use docker run -itd --name=server -p 8080:8080 -e PRELOAD="true" hugegraph/hugegraph:1.7.0 to preload a built-in sample graph at startup. You can verify it through the RESTful API. See 5.1.4 for details.
3. You can use -e PASSWORD=xxx to enable authentication mode and set the admin password. See Config Authentication for details.

If you use Docker Desktop, you can set the options as follows:

Note: The Docker Compose files use bridge networking (hg-net) and work on Linux and Mac (Docker Desktop). For the 3-node distributed cluster on Mac (Docker Desktop), allocate at least 12 GB of memory (Settings → Resources → Memory). On Linux, Docker uses host memory directly.

If you want a single, unified setup for multiple HugeGraph services, you can use docker compose. Four compose files are available in the docker/ directory:

cd hugegraph/docker
# Keep the version aligned with the latest release, for example 1.x.0
HUGEGRAPH_VERSION=1.7.0 docker compose -f docker-compose.yml up -d --wait

The standalone topology publishes the Server on port 8080 and Hubble on 127.0.0.1:8088. HUGEGRAPH_VERSION selects the Server, PD, and Store image tags; Hubble is selected separately with HUBBLE_IMAGE.

The compose files read the administrator password from HUGEGRAPH_ADMIN_PASSWORD and the JWT secret from HUGEGRAPH_AUTH_TOKEN_SECRET, normally kept in a docker/.env file. A non-empty HUGEGRAPH_ADMIN_PASSWORD turns authentication on, and Hubble detects that mode by itself. With plain docker run, pass -e PASSWORD=xxx instead.

See docker/README.md for the full setup guide.

Note:

1. HugeGraph Docker images are provided as a convenient way to start HugeGraph quickly, but they are not official ASF distribution artifacts. You can find more details in the ASF Release Distribution Policy.

2. We recommend using a release tag (such as 1.7.0 or 1.x.0) for stable deployments. Use the latest tag only if you want the newest features still under development.

3.2 Download the binary tarball

You could download the binary tarball from the download page of the ASF site like this:

# 1.7.0 is a historical release from the incubation period, so its file name still includes "incubating"
wget https://downloads.apache.org/hugegraph/1.7.0/apache-hugegraph-incubating-1.7.0.tar.gz
tar zxf apache-hugegraph-incubating-1.7.0.tar.gz

# (Optional) verify the integrity with SHA512 (recommended)
shasum -a 512 apache-hugegraph-incubating-1.7.0.tar.gz
curl https://downloads.apache.org/hugegraph/1.7.0/apache-hugegraph-incubating-1.7.0.tar.gz.sha512

3.3 Source code compilation

Please ensure that the wget/curl commands are installed before compiling the source code

Download HugeGraph source code in either of the following 2 ways (so as the other HugeGraph repos/modules):

• download the stable/release version from the ASF site
• clone the unstable/latest version by GitBox(ASF) or GitHub

# Way 1. download release package from the ASF site
wget https://downloads.apache.org/hugegraph/{version}/apache-hugegraph-incubating-src-{version}.tar.gz
tar zxf *hugegraph*.tar.gz

# (Optional) verify the integrity with SHA512 (recommended)
shasum -a 512 apache-hugegraph-incubating-src-{version}.tar.gz
curl https://downloads.apache.org/hugegraph/{version}/apache-hugegraph-incubating-{version}-src.tar.gz.sha512

# Way2 : clone the latest code by git way (e.g GitHub)
git clone https://github.com/apache/hugegraph.git

Compile and generate tarball

cd *hugegraph
# (Optional) use "-P stage" param if you build failed with the latest code(during pre-release period)
mvn package -DskipTests -ntp

A successful build includes the following line:

[INFO] BUILD SUCCESS

After a successful build, the generated distribution is the *hugegraph-*.tar.gz file in the repository root.

The default build bundles the rocksdb, hbase, and hstore backend modules, and records them in the backends option of the backend.properties resource inside the hugegraph-dist jar. To build a smaller distribution that carries RocksDB only, add -Drocksdb-only:

mvn package -DskipTests -ntp -Drocksdb-only

# download toolchain binary package, it includes loader + tool + hubble
# please check the latest version (e.g. here is 1.7.0)
wget https://downloads.apache.org/hugegraph/1.7.0/apache-hugegraph-toolchain-incubating-1.7.0.tar.gz
tar zxf *hugegraph-*.tar.gz

# enter the tool's package
cd *hugegraph*/*tool* 

bin/hugegraph deploy -v {hugegraph-version} -p {install-path} [-u {download-path-prefix}]

4 Config

If you need to quickly start HugeGraph just for testing, then you only need to modify a few configuration items (see next section). For detailed configuration introduction, please refer to configuration document and introduction to configuration items

5 Startup

5.1 Use a startup script to startup

Startup is divided into “first startup” and “non-first startup”. On the first startup, you need to initialize the backend database before starting the service.

If the service was stopped manually, or needs to be started again for any other reason, you can usually start it directly because the backend database is persistent.

When HugeGraphServer starts, it connects to the backend storage and checks its version information. If the backend has not been initialized, or if it was initialized with an incompatible version (for example, old-version data), HugeGraphServer will fail to start and report an error.

If you need to access HugeGraphServer externally, modify the restserver.url configuration item in rest-server.properties (the default is http://127.0.0.1:8080) and change it to the machine name or IP address.

Since the configuration (hugegraph.properties) and startup steps required by various backends are slightly different, the following will introduce the configuration and startup of each backend one by one.

Note: Configure Server Authentication before starting HugeGraphServer if you need Auth mode (especially for production or public network environments).

5.1.1 Distributed Storage (HStore)

backend=hstore
serializer=binary

# PD service address, multiple PD addresses are separated by commas, configure PD's RPC port
pd.peers=127.0.0.1:8686,127.0.0.1:8687,127.0.0.1:8688

# Simple example (with authentication)
gremlin.graph=org.apache.hugegraph.auth.HugeFactoryAuthProxy

# Specify storage backend hstore
backend=hstore
serializer=binary
store=hugegraph

# pd config
pd.peers=127.0.0.1:8686

usePD=true
# load the hugegraph.properties above from the graphs directory; the source default is false
graph.load_from_local_config=true

# notice: must have this conf in 1.7.0
pd.peers=127.0.0.1:8686,127.0.0.1:8687,127.0.0.1:8688
# If auth is needed
# auth.authenticator=org.apache.hugegraph.auth.StandardAuthenticator

usePD=true
restserver.url=http://127.0.0.1:8081
gremlinserver.url=http://127.0.0.1:8181
pd.peers=127.0.0.1:8686

rpc.server_host=127.0.0.1
rpc.server_port=8091

server.id=server-1
server.role=master

usePD=true
restserver.url=http://127.0.0.1:8082
gremlinserver.url=http://127.0.0.1:8182
pd.peers=127.0.0.1:8686

rpc.server_host=127.0.0.1
rpc.server_port=8092

server.id=server-2
server.role=worker

host: 127.0.0.1
port: 8181

host: 127.0.0.1
port: 8182

cd *hugegraph-${version}
bin/init-store.sh

bin/start-hugegraph.sh

curl http://localhost:8081/graphspaces/DEFAULT/graphs
# Should return: {"graphs":["hugegraph"]}

bin/stop-hugegraph.sh

cd hugegraph/docker
HUGEGRAPH_VERSION=1.7.0 docker compose -f docker-compose-3pd-3store-3server.yml up -d --wait

# Server configuration, shared by server0, server1 and server2
HG_SERVER_BACKEND: hstore
HG_SERVER_PD_PEERS: pd0:8686,pd1:8686,pd2:8686
HG_SERVER_CLUSTER: hg
HG_SERVER_USE_PD: "true"
HG_SERVER_MIN_FREE_MEMORY: "0"
HG_SERVER_INIT_STORE_ENABLED: "false"
HG_SERVER_REQUIRE_AUTH_TOKEN_SECRET: "true"
STORE_REST: store0:8520
# per node, for example on server0
HG_SERVER_REST_URL: http://server0:8080

curl http://localhost:8080/versions
curl http://localhost:8620/v1/stores

5.1.2 RocksDB / ToplingDB

backend=rocksdb
serializer=binary
rocksdb.data_path=.
rocksdb.wal_path=.

cd *hugegraph-${version}
bin/init-store.sh

bin/start-hugegraph.sh
Starting HugeGraphServer in daemon mode...
Connecting to HugeGraphServer (http://127.0.0.1:8080/graphs)....OK
Started [pid 21614]

5.1.3 HBase

backend=hbase
serializer=hbase

# hbase backend config
hbase.hosts=localhost
hbase.port=2181
# Note: recommend to modify the HBase partition number by the actual/env data amount & RS amount before init store
# it may influence the loading speed a lot
#hbase.enable_partition=true
#hbase.vertex_partitions=10
#hbase.edge_partitions=30

cd *hugegraph-${version}
bin/init-store.sh

bin/start-hugegraph.sh
Starting HugeGraphServer in daemon mode...
Connecting to HugeGraphServer (http://127.0.0.1:8080/graphs)....OK
Started [pid 21614]

5.1.4 Create an example graph when startup

Pass the -p true argument when starting the script to enable preload, which creates a sample graph.

bin/start-hugegraph.sh -p true
Starting HugeGraphServer in daemon mode...
Connecting to HugeGraphServer (http://127.0.0.1:8080/graphs)......OK

And use the RESTful API to request HugeGraphServer and get the following result:

> curl "http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices" | gunzip

{"vertices":[{"id":"2:lop","label":"software","type":"vertex","properties":{"name":"lop","lang":"java","price":328}},{"id":"1:josh","label":"person","type":"vertex","properties":{"name":"josh","age":32,"city":"Beijing"}},{"id":"1:marko","label":"person","type":"vertex","properties":{"name":"marko","age":29,"city":"Beijing"}},{"id":"1:peter","label":"person","type":"vertex","properties":{"name":"peter","age":35,"city":"Shanghai"}},{"id":"1:vadas","label":"person","type":"vertex","properties":{"name":"vadas","age":27,"city":"Hongkong"}},{"id":"2:ripple","label":"software","type":"vertex","properties":{"name":"ripple","lang":"java","price":199}}]}

This indicates the successful creation of the sample graph.

5.1.5 Startup script options

bin/start-hugegraph.sh accepts the following options. Every one of them takes a value, so write -d false, not a bare -d.

bin/stop-hugegraph.sh accepts -m true|false (default true), which controls whether the cron monitor task is removed along with the service.

5.2 Use Docker to startup

In 3.1 Use Docker container, we introduced how to deploy hugegraph-server with Docker. You can also switch storage backends or preload a sample graph by setting the corresponding parameters.

5.2.1 Create an example graph when starting a server

Set the environment variable PRELOAD=true when starting Docker so that sample data is loaded during startup.

1. Use docker run

   Use docker run -itd --name=server -p 8080:8080 -e PRELOAD=true hugegraph/hugegraph:1.7.0

2. Use docker-compose

   Create a docker-compose.yml file like the following and set PRELOAD=true in the environment. example.groovy is a predefined script used to preload sample data. If needed, you can mount a new example.groovy script to change the preload data.

   version: '3'
   services:
     server:
       image: hugegraph/hugegraph:1.7.0
       container_name: server
       environment:
         - PRELOAD=true
         - PASSWORD=xxx
       volumes:
         - /path/to/yourscript:/hugegraph-server/scripts/example.groovy
       ports:
         - 8080:8080

   Use docker compose up -d to start the container.

And use the RESTful API to request HugeGraphServer and get the following result:

> curl "http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices" | gunzip

{"vertices":[{"id":"2:lop","label":"software","type":"vertex","properties":{"name":"lop","lang":"java","price":328}},{"id":"1:josh","label":"person","type":"vertex","properties":{"name":"josh","age":32,"city":"Beijing"}},{"id":"1:marko","label":"person","type":"vertex","properties":{"name":"marko","age":29,"city":"Beijing"}},{"id":"1:peter","label":"person","type":"vertex","properties":{"name":"peter","age":35,"city":"Shanghai"}},{"id":"1:vadas","label":"person","type":"vertex","properties":{"name":"vadas","age":27,"city":"Hongkong"}},{"id":"2:ripple","label":"software","type":"vertex","properties":{"name":"ripple","lang":"java","price":199}}]}

This indicates that the sample graph was created successfully.

6. Access server

6.1 Service startup status check

Use jps to see a service process

jps
6475 HugeGraphServer

curl request RESTfulAPI

echo `curl -o /dev/null -s -w %{http_code} "http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices"`

Return 200, which means the server starts normally.

6.2 Request Server

The RESTful API of HugeGraphServer includes various types of resources, typically including graph, schema, gremlin, traverser and task.

• graph contains vertices、edges
• schema contains vertexlabels、 propertykeys、 edgelabels、indexlabels
• gremlin contains various Gremlin statements, such as g.v(), which can be executed synchronously or asynchronously
• traverser contains various advanced queries including shortest paths, intersections, N-step reachable neighbors, etc.
• task contains query and delete with asynchronous tasks

6.2.1 Get vertices and its related properties in hugegraph

curl http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices

explanation

1. Since there are many vertices and edges in the graph, for list-type requests, such as getting all vertices, getting all edges, etc., the server will compress the data and return it, so when use curl, you get a bunch of garbled characters, you can redirect to gunzip for decompression. It is recommended to use the Chrome browser + Restlet plugin to send HTTP requests for testing.

   curl "http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices" | gunzip

2. The current default configuration of HugeGraphServer can only be accessed locally, and the configuration can be modified so that it can be accessed on other machines.

   vim conf/rest-server.properties
   
   restserver.url=http://0.0.0.0:8080

response body:

{
    "vertices": [
        {
            "id": "2lop",
            "label": "software",
            "type": "vertex",
            "properties": {
                "price": [
                    {
                        "id": "price",
                        "value": 328
                    }
                ],
                "name": [
                    {
                        "id": "name",
                        "value": "lop"
                    }
                ],
                "lang": [
                    {
                        "id": "lang",
                        "value": "java"
                    }
                ]
            }
        },
        {
            "id": "1josh",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": [
                    {
                        "id": "name",
                        "value": "josh"
                    }
                ],
                "age": [
                    {
                        "id": "age",
                        "value": 32
                    }
                ]
            }
        },
        ...
    ]
}

For the detailed API, please refer to RESTful-API

You can also visit localhost:8080/swagger-ui/index.html to check the API.

When using Swagger UI to debug the API provided by HugeGraph, if HugeGraph Server turns on authentication mode, you can enter authentication information on the Swagger page.

Currently, HugeGraph supports setting authentication information in two forms: Basic and Bearer.

7 Stop Server

cd apache-hugegraph-incubating-1.7.0/apache-hugegraph-server-incubating-1.7.0
bin/stop-hugegraph.sh

8 Debug Server with IntelliJ IDEA

Please refer to Setup Server in IDEA

1.2 - HugeGraph-PD Quick Start

1 HugeGraph-PD Overview

HugeGraph-PD (Placement Driver) is the metadata management component of HugeGraph’s distributed version, responsible for managing the distribution of graph data and coordinating storage nodes. It plays a central role in distributed HugeGraph, maintaining cluster status and coordinating HugeGraph-Store storage nodes.

PD keeps cluster metadata in an embedded RocksDB store under pd.data-path and replicates it across PD nodes with Raft, so a 3-node or 5-node PD cluster keeps serving while a minority of nodes is down. On top of that it registers and activates Store nodes, allocates and rebalances partitions, tracks Store heartbeats, and answers service discovery queries from Store and Server.

PD listens on three ports:

2 Prerequisites

2.1 Requirements

• Operating System: Linux or macOS (Windows has not been fully tested)
• Java version: ≥ 11
• Maven version: ≥ 3.5.0

3 Deployment

There are two ways to deploy the HugeGraph-PD component:

• Method 1: Download the tar package
• Method 2: Compile from source

3.1 Download the tar package

Download the latest version of HugeGraph-PD from the Apache HugeGraph official download page:

# 1.7.0 is a historical release from the incubation period, so its file and directory names still include "incubating"
wget https://downloads.apache.org/hugegraph/1.7.0/apache-hugegraph-incubating-1.7.0.tar.gz
tar zxf apache-hugegraph-incubating-1.7.0.tar.gz
cd apache-hugegraph-incubating-1.7.0/apache-hugegraph-pd-incubating-1.7.0

3.2 Compile from source

# 1. Clone the source code
git clone https://github.com/apache/hugegraph.git

# 2. Build the project
cd hugegraph
mvn clean install -DskipTests=true

# 3. After a successful build, the PD directory and packages are located at
#    hugegraph-pd/apache-hugegraph-pd-{version}          (unpacked PD distribution)
#    hugegraph-pd/apache-hugegraph-pd-{version}.tar.gz   (PD only package, Linux build hosts only)
#    target/apache-hugegraph-{version}.tar.gz            (PD + Store + Server package)

To build only the PD distribution and the modules it depends on:

mvn clean package -pl hugegraph-pd/hg-pd-dist -am -DskipTests

The unpacked distribution contains just three directories: bin (start and stop scripts), conf (application.yml, application.yml.template, log4j2.xml, verify-license.json) and lib (the hg-pd-service jar).

3.3 Docker Deployment

The HugeGraph-PD Docker image is available on Docker Hub as hugegraph/pd.

Note: The following steps assume you have already cloned or pulled the HugeGraph main repository locally, or at least have its docker/ directory available.

Use the docker compose setup to deploy the complete 3-node cluster (PD + Store + Server):

cd hugegraph/docker
# Keep the version aligned with the latest release, for example 1.x.0
HUGEGRAPH_VERSION=1.7.0 docker compose -f docker-compose-3pd-3store-3server.yml up -d

A single PD plus a single Store and Server is also available as docker-compose-hstore.yml.

To run a single PD node via docker run, configuration is provided via environment variables:

docker run -d \
  -p 8620:8620 \
  -p 8686:8686 \
  -p 8610:8610 \
  -e HG_PD_GRPC_HOST=<your-ip> \
  -e HG_PD_RAFT_ADDRESS=<your-ip>:8610 \
  -e HG_PD_RAFT_PEERS_LIST=<your-ip>:8610 \
  -e HG_PD_INITIAL_STORE_LIST=<store-ip>:8500 \
  -v /path/to/data:/hugegraph-pd/pd_data \
  --name hugegraph-pd \
  hugegraph/pd:1.7.0

Environment variable reference:

The entrypoint refuses to start if any of the four required variables is missing, and it turns the values above into a SPRING_APPLICATION_JSON override, so the packaged conf/application.yml does not need editing. Any key not covered by an HG_PD_* variable keeps the value from that file. JAVA_OPTS is passed through to the JVM.

Note: In Docker bridge networking, use container hostnames (e.g. pd0) for HG_PD_GRPC_HOST and HG_PD_RAFT_ADDRESS instead of IP addresses.

Deprecated aliases: GRPC_HOST, RAFT_ADDRESS, RAFT_PEERS, PD_INITIAL_STORE_LIST still work but log a deprecation warning. Use the HG_PD_* names for new deployments.

The image ships a HEALTHCHECK that polls GET /v1/health on port 8620 every 15 seconds, with a 90 second start period and 3 retries, so docker ps reports real PD health. The entrypoint runs the start script with -d false, so the container process is Java itself and Docker’s restart policy fires when it dies. The image also sets STDOUT_MODE=true, so docker logs <container-name> (e.g. docker logs hg-pd0) shows the PD log without exec-ing into the container.

See docker/README.md for the full cluster setup guide.

4 Configuration

The main configuration file for PD is conf/application.yml. This is the file the distribution ships:

spring:
  application:
    name: hugegraph-pd

management:
  metrics:
    export:
      prometheus:
        enabled: true
  endpoints:
    web:
      exposure:
        include: "*"

logging:
  config: 'file:./conf/log4j2.xml'

license:
  verify-path: ./conf/verify-license.json
  license-path: ./conf/hugegraph.license

grpc:
  # gRPC port for cluster mode
  port: 8686
  # Change to the actual local IPv4 address when deploying
  host: 127.0.0.1

server:
  # REST service port
  port: 8620

pd:
  # Storage path
  data-path: ./pd_data
  # Auto-expansion check cycle (seconds)
  patrol-interval: 1800
  # Minimum number of Store nodes required for cluster availability
  initial-store-count: 1
  # Store configuration information, format is IP:gRPC port
  initial-store-list: 127.0.0.1:8500

raft:
  # Raft address of this node
  address: 127.0.0.1:8610
  # Raft addresses of all PD nodes in the cluster
  peers-list: 127.0.0.1:8610

store:
  # Store offline time (seconds). After this time, the store is considered permanently unavailable
  max-down-time: 172800
  # Whether to enable store monitoring data storage
  monitor_data_enabled: true
  # Monitoring data interval
  monitor_data_interval: 1 minute
  # Monitoring data retention time
  monitor_data_retention: 1 day

partition:
  # Default number of replicas per partition
  default-shard-count: 1
  # Default maximum number of replicas per machine
  store-max-shard-count: 12

conf/application.yml.template is a second, unused copy with placeholders ($GRPC_PORT$, $RAFT_ADDRESS$ and so on) for deployment tooling that generates the file. PD always reads conf/application.yml, which the start script passes as -Dspring.config.location.

4.1 Configuration reference

Keys not present in conf/application.yml fall back to the built-in default listed below. Keys with no built-in default must be present, otherwise PD fails to start.

gRPC and REST

application.yml.template also carries grpc.netty-server.max-inbound-message-size: 100MB, but PD sets the gRPC server’s inbound message limit to 1 GB in code, so that key has no effect.

Raft

PD core

Store management

Partitions

The initial partition count is derived from these two values and the size of pd.initial-store-list:

initial partitions = store count * partition.store-max-shard-count / partition.default-shard-count

Discovery, license and metrics

Thread pools

4.2 Single-node configuration

The shipped conf/application.yml already is a working single-node configuration. It is meant for development and testing: one PD node has no Raft quorum to lose, and partition.default-shard-count: 1 keeps a single replica per partition.

grpc:
  host: 127.0.0.1
  port: 8686
server:
  port: 8620
raft:
  address: 127.0.0.1:8610
  peers-list: 127.0.0.1:8610
pd:
  data-path: ./pd_data
  initial-store-count: 1
  initial-store-list: 127.0.0.1:8500
partition:
  default-shard-count: 1

4.3 Three-node cluster configuration

For a production cluster run 3 or 5 PD nodes, an odd number so Raft always has a quorum. A 3-node cluster tolerates one node failure. raft.peers-list must list every node and must be byte-for-byte identical on all of them, while grpc.host and raft.address differ per node.

Node 1 (192.168.1.10):

grpc:
  host: 192.168.1.10
  port: 8686
server:
  port: 8620
raft:
  address: 192.168.1.10:8610
  peers-list: 192.168.1.10:8610,192.168.1.11:8610,192.168.1.12:8610
pd:
  data-path: /data/pd
  initial-store-count: 3
  initial-store-list: 192.168.1.20:8500,192.168.1.21:8500,192.168.1.22:8500
partition:
  default-shard-count: 3

Node 2 (192.168.1.11) and node 3 (192.168.1.12) use the same file with grpc.host and raft.address changed to their own address:

# node 2
grpc:
  host: 192.168.1.11
raft:
  address: 192.168.1.11:8610
  peers-list: 192.168.1.10:8610,192.168.1.11:8610,192.168.1.12:8610

# node 3
grpc:
  host: 192.168.1.12
raft:
  address: 192.168.1.12:8610
  peers-list: 192.168.1.10:8610,192.168.1.11:8610,192.168.1.12:8610

To put all three PD nodes on one machine for testing, give each node its own pd.data-path and its own ports, for example raft 8610/8611/8612, gRPC 8686/8687/8688 and REST 8620/8621/8622.

In Docker bridge networking the same configuration comes from environment variables and uses container hostnames instead of IP addresses:

# pd0
HG_PD_GRPC_HOST: pd0
HG_PD_RAFT_ADDRESS: pd0:8610
HG_PD_RAFT_PEERS_LIST: pd0:8610,pd1:8610,pd2:8610
HG_PD_INITIAL_STORE_LIST: store0:8500,store1:8500,store2:8500
HG_PD_INITIAL_STORE_COUNT: 3

# pd1
HG_PD_GRPC_HOST: pd1
HG_PD_RAFT_ADDRESS: pd1:8610
HG_PD_RAFT_PEERS_LIST: pd0:8610,pd1:8610,pd2:8610

# pd2
HG_PD_GRPC_HOST: pd2
HG_PD_RAFT_ADDRESS: pd2:8610
HG_PD_RAFT_PEERS_LIST: pd0:8610,pd1:8610,pd2:8610

5 Start and Stop

5.1 Start PD

In the PD installation directory, execute:

./bin/start-hugegraph-pd.sh

The script requires a JDK of at least version 11 on PATH or in JAVA_HOME, and it exits without doing anything if it finds a Java process already using this installation’s conf directory.

Supported flags:

The -d flag controls daemon mode:

• -d true (default): run as a background daemon; the script returns immediately.
• -d false: run in foreground. The script execs Java, so the container or supervisor process IS Java. Use this when running under Docker or a process supervisor (systemd, supervisord) so crashes are detected and the service is restarted automatically.

Each flag also has an environment variable equivalent: DAEMON, GC_OPTION, USER_OPTION and OPEN_TELEMETRY. Setting JAVA_OPTIONS replaces the computed heap settings entirely; otherwise the script sizes the heap between 512 MB and 32 GB from available memory. Setting STDOUT_MODE=true leaves the JVM output on stdout instead of redirecting it to logs/hugegraph-pd-stdout.log, which is what the Docker image does.

After successful startup, you can see logs similar to the following in logs/hugegraph-pd-stdout.log:

YYYY-mm-dd xx:xx:xx [main] [INFO] o.a.h.p.b.HugePDServer - Started HugePDServer in x.xxx seconds (JVM running for x.xxx)

The process id is written to bin/pid.

5.2 Stop PD

In the PD installation directory, execute:

./bin/stop-hugegraph-pd.sh

The script reads bin/pid, sends the process a termination signal, waits up to 30 seconds for it to exit, and removes the pid file. If bin/pid is missing it reports that and exits successfully.

6 Startup Order in a Distributed Cluster

Start the components in this order:

1. All PD nodes. They form the Raft group and elect a leader. Wait until every node answers GET /v1/health.
2. All Store nodes. Each Store registers with PD over gRPC, and PD activates the ones listed in pd.initial-store-list. Wait until GET /v1/stores reports "state": "Up" for every Store.
3. All Server nodes. A Server reads pd.peers and depends on PD reporting at least one live Store before partitions can be assigned.

The Docker Compose topologies enforce exactly this. Store containers wait on PD’s /v1/health healthcheck through depends_on with condition: service_healthy, Server containers wait the same way on the Store healthcheck, and the Server entrypoint then polls PD’s /v1/stores until a Store reports Up before it starts HugeGraph.

PD is also the last component to stop: shut down Server, then Store, then PD.

7 Verification

7.1 REST API authentication

Except for /actuator/*, /v1/health and /v1/prom/targets/*, every PD REST path requires an HTTP Basic Authorization header whose user name is one of the internal service names hg, store, hubble or vermeer. A request without the header is answered with:

{"status": -1, "error": "Unauthorized!"}

The password is not validated yet, so any value works. The Server’s own bin/wait-storage.sh uses store:admin and lets you override it with PD_AUTH_USER and PD_AUTH_PASSWORD, so the examples below use the same credentials:

curl -u store:admin http://localhost:8620/v1/stores

Warning: This check is only meant to separate HugeGraph’s own components from other traffic. Do not expose the PD REST or gRPC ports to an untrusted network. Restrict them with firewall rules or security groups, and keep raft.ip-whitelist.enabled on so the Raft port only accepts the configured peers.

7.2 Health check

GET /v1/health needs no credentials and is what the Docker healthcheck uses. It answers 200 with an empty body:

curl -i http://localhost:8620/v1/health

The Spring Boot actuator endpoint also works and is more readable:

curl http://localhost:8620/actuator/health

If it returns {"status":"UP"}, it indicates that the PD service has been successfully started.

7.3 Cluster and member status

Check the PD members and which node is the Raft leader:

curl -u store:admin http://localhost:8620/v1/members

The response carries pdList, the elected pdLeader, numOfService, numOfNormalService and a stateCountMap. In a healthy 3-node PD cluster numOfService and numOfNormalService are both 3 and exactly one member has role: "Leader".

GET /v1/cluster returns the same member list together with the Store list, graph list and overall cluster state, and GET / returns a short summary (leader address, cluster state, member count, store count, graph count, partition count).

7.4 Store status

You can also verify Store node status through the PD API:

curl -u store:admin http://localhost:8620/v1/stores

If the response shows state as Up, the corresponding Store node is running normally. The example below shows a single Store node. In a healthy 3-node deployment, the storeId list should contain three IDs, and stateCountMap.Up, numOfService, and numOfNormalService should all be 3.

{
  "message": "OK",
  "data": {
    "stores": [
      {
        "storeId": 8319292642220586694,
        "address": "127.0.0.1:8500",
        "raftAddress": "127.0.0.1:8510",
        "version": "",
        "state": "Up",
        "deployPath": "/Users/{your_user_name}/hugegraph/apache-hugegraph-incubating-1.7.0/apache-hugegraph-store-incubating-1.7.0/lib/hg-store-node-1.7.0.jar",
        "dataPath": "./storage",
        "startTimeStamp": 1754027127969,
        "registedTimeStamp": 1754027127969,
        "lastHeartBeat": 1754027909444,
        "capacity": 494384795648,
        "available": 346535829504,
        "partitionCount": 0,
        "graphSize": 0,
        "keyCount": 0,
        "leaderCount": 0,
        "serviceName": "127.0.0.1:8500-store",
        "serviceVersion": "",
        "serviceCreatedTimeStamp": 1754027127000,
        "partitions": []
      }
    ],
    "stateCountMap": {
      "Up": 1
    },
    "numOfService": 1,
    "numOfNormalService": 1
  },
  "status": 0
}

7.5 Other REST endpoints

All paths below are relative to http://<pd-host>:8620 and need the Basic header from section 7.1 unless noted.

The two log endpoints take a time range as {"startTime": "...", "endTime": "..."}; yyyy-MM-dd HH:mm:ss and yyyy-MM-dd are among the accepted formats.

PD registers its own meters under the hg prefix, so /actuator/prometheus exposes hg_up, hg_graphs, hg_stores and hg_terms alongside the standard JVM metrics, plus per-graph partition and size meters once graphs exist.

1.3 - HugeGraph-Store Quick Start

1 HugeGraph-Store Overview

HugeGraph-Store is the storage node component of HugeGraph’s distributed version, responsible for actually storing and managing graph data. It works in conjunction with HugeGraph-PD to form HugeGraph’s distributed storage engine, providing high availability and horizontal scalability.

Each Store node keeps graph data in RocksDB and replicates it with Raft (JRaft): every partition is a separate Raft group, so a partition survives the loss of a minority of its replicas. Store nodes do not know about each other directly. They register with PD, receive their partition assignment from PD, and report state back over a heartbeat. HugeGraph-Server reaches Store over gRPC after looking up partition locations in PD.

2 Prerequisites

2.1 Requirements

• Operating System: Linux or macOS (Windows has not been fully tested)
• Java version: ≥ 11 (enforced by the build and re-checked by bin/start-hugegraph-store.sh)
• Maven version: ≥ 3.5.0
• Deploy HugeGraph-PD first for multi-node deployment

3 Deployment

There are two ways to deploy the HugeGraph-Store component:

• Method 1: Download the tar package
• Method 2: Compile from source

3.1 Download the tar package

Download the latest version of HugeGraph-Store from the Apache HugeGraph official download page:

# 1.7.0 is a historical release from the incubation period, so its file and directory names still include "incubating"
wget https://downloads.apache.org/hugegraph/1.7.0/apache-hugegraph-incubating-1.7.0.tar.gz
tar zxf apache-hugegraph-incubating-1.7.0.tar.gz
cd apache-hugegraph-incubating-1.7.0/apache-hugegraph-store-incubating-1.7.0

3.2 Compile from source

# 1. Clone the source code
git clone https://github.com/apache/hugegraph.git

# 2. Build the project
cd hugegraph
mvn clean install -DskipTests=true

# 3. After a successful build, the Store directory and complete distribution package are located at
#    hugegraph-store/apache-hugegraph-store-{version}
#    target/apache-hugegraph-{version}.tar.gz

To build Store alone instead of the whole repository, build hugegraph-struct first, because Store depends on it:

mvn install -pl hugegraph-struct -am -DskipTests
mvn clean package -pl hugegraph-store/hg-store-dist -am -DskipTests

The assembled directory contains only bin/, conf/ and lib/hg-store-node-{version}.jar.

3.3 Docker Deployment

The HugeGraph-Store Docker image is available on Docker Hub as hugegraph/store.

Note: The following steps assume you have already cloned or pulled the HugeGraph main repository locally, or at least have its docker/ directory available.

Two compose files include Store:

cd hugegraph/docker
# Keep the version aligned with the latest release, for example 1.x.0

# Minimal distributed deployment
HUGEGRAPH_VERSION=1.7.0 docker compose -f docker-compose-hstore.yml up -d --wait

# Or the multi-node cluster
HUGEGRAPH_VERSION=1.7.0 docker compose -f docker-compose-3pd-3store-3server.yml up -d

To run a single Store node via docker run:

docker run -d \
  -p 8520:8520 \
  -p 8500:8500 \
  -p 8510:8510 \
  -e HG_STORE_PD_ADDRESS=<pd-ip>:8686 \
  -e HG_STORE_GRPC_HOST=<your-ip> \
  -e HG_STORE_RAFT_ADDRESS=<your-ip>:8510 \
  -v /path/to/storage:/hugegraph-store/storage \
  --name hugegraph-store \
  hugegraph/store:1.7.0

Environment variable reference:

The entrypoint turns these into a SPRING_APPLICATION_JSON overlay on top of conf/application.yml, then runs bin/start-hugegraph-store.sh -d false -j "$JAVA_OPTS". Any key not covered by the table above still has to be edited in conf/application.yml, or supplied through your own SPRING_APPLICATION_JSON.

Image details:

• JAVA_OPTS defaults to -XX:+UnlockExperimentalVMOptions -XX:+UseContainerSupport -XX:MaxRAMPercentage=50 -XshowSettings:vm
• STDOUT_MODE=true, so Java logs go to the container stdout instead of logs/hugegraph-store-server.log
• HEALTHCHECK calls GET http://localhost:8520/v1/health every 15s after a 90s start period
• The image declares EXPOSE 8520; publish 8500 and 8510 yourself when Server or other Store nodes need to reach the container from outside the Docker network

Note: In Docker bridge networking, use container hostnames (e.g. store0) for HG_STORE_GRPC_HOST instead of IP addresses.

Deprecated aliases: PD_ADDRESS, GRPC_HOST, RAFT_ADDRESS still work but log a deprecation warning. Use the HG_STORE_* names for new deployments.

4 Configuration

Store reads two files from conf/:

• application.yml, the main configuration file (PD address, ports, Raft, data path)
• application-pd.yml, pulled in by spring.profiles.include: pd in application.yml, holding the RocksDB memory settings and the Actuator exposure

4.1 application.yml

This is the file shipped in the distribution package:

pdserver:
  # PD service address, multiple PD addresses separated by commas
  address: localhost:8686

management:
  metrics:
    export:
      prometheus:
        enabled: true
  endpoints:
    web:
      exposure:
        include: "*"

grpc:
  # grpc service address
  host: 127.0.0.1
  port: 8500
  netty-server:
    max-inbound-message-size: 1000MB
raft:
  # raft cache queue size
  disruptorBufferSize: 1024
  address: 127.0.0.1:8510
  max-log-file-size: 600000000000
  # Snapshot generation interval, in seconds
  snapshotInterval: 1800
server:
  # rest service address
  port: 8520

app:
  # Storage path, support multiple paths, separated by commas
  data-path: ./storage
  #raft-path: ./storage

spring:
  application:
    name: store-node-grpc-server
  profiles:
    active: default
    include: pd

logging:
  config: 'file:./conf/log4j2.xml'
  level:
    root: info

4.2 application-pd.yml

management:
  metrics:
    export:
      prometheus:
        enabled: true
  endpoints:
    web:
      exposure:
        include: "*"

rocksdb:
  # rocksdb total memory usage, force flush to disk when reaching this value
  total_memory_size: 32000000000
  # memtable size used by rocksdb
  write_buffer_size: 32000000
  # For each rocksdb, the number of memtables reaches this value for writing to disk.
  min_write_buffer_number_to_merge: 16

4.3 Configuration reference

“Shipped” is the value in the two files above. “Code default” is what the node falls back to when the key is absent, and is the value to rely on for keys the template does not list.

Core

Raft

Storage and labels

RocksDB

Any other option defined in org/apache/hugegraph/rocksdb/access/RocksDBOptions.java can be added under the same rocksdb: block, for example rocksdb.max_background_jobs, rocksdb.level0_file_num_compaction_trigger or rocksdb.bloom_filter_bits_per_key.

Thread pools

Query pushdown

Background jobs

Built-in PD mode

Only for single-node development and debugging, activated by app.fake-pd: true. The node then plays PD’s role itself and ignores pdserver.address.

Diagnostics

4.4 Per-node changes

For multi-node deployment, you need to modify the following configurations for each Store node:

1. grpc.host and grpc.port (the address other components dial)
2. raft.address (Raft protocol address)
3. server.port (REST port)
4. app.data-path (data storage path)

pdserver.address is the same on every node, it lists the whole PD cluster.

5 Start and Stop

5.1 Start Store

Ensure that the PD service is already started, then in the Store installation directory, execute:

./bin/start-hugegraph-store.sh

The script accepts four flags:

Daemon mode:

• -d true (default): run as a background daemon. The script returns immediately and writes the Java pid to bin/pid.
• -d false: run in the foreground. The script execs Java, so the container or supervisor process is Java itself. Use this under Docker or a process supervisor (systemd, supervisord) so crashes are detected and the service is restarted automatically.

JVM memory, unless you set JAVA_OPTIONS yourself: -Xms512m, and -Xmx set to half the free memory, clamped to the 512 MB to 2048 MB range. The script also adds -XX:MetaspaceSize=256M, a heap dump on out-of-memory into logs/, and a rolling GC log at logs/gc.log. Production nodes normally need a much larger heap, so pass one explicitly, for example -j "-Xmx32g -Xms32g".

The script refuses to start if ulimit -n or ulimit -u is below 1024, and it preloads jemalloc on x86_64 and arm64 when the shared object can be downloaded and verified.

After successful startup, you can see logs similar to the following in logs/hugegraph-store-server.log:

YYYY-mm-dd xx:xx:xx [main] [INFO] o.a.h.s.n.StoreNodeApplication - Started StoreNodeApplication in x.xxx seconds (JVM running for x.xxx)

5.2 Stop Store

In the Store installation directory, execute:

./bin/stop-hugegraph-store.sh

The script reads bin/pid, signals that process, and waits up to 30 seconds for it to exit before removing the pid file. If bin/pid is missing it exits without doing anything.

5.3 Restart Store

./bin/restart-hugegraph-store.sh

It sources the stop script and then the start script, and forwards the flags from section 5.1.

5.4 Startup order

1. PD first. Each Store’s grpc.host:grpc.port should appear in PD’s pd.initial-store-list, otherwise PD registers the node in Pending state instead of bringing it to Up, and partition assignment never finishes.
2. Store next. A Store started before PD is reachable is not fatal: the heartbeat thread keeps retrying registration and logs store heartbeat error: PD UNREACHABLE until PD answers.
3. HugeGraph-Server last, once every Store node reports state: "Up". Server needs the partitions in place before it can initialize or open a graph.

The compose files encode the same order with depends_on: condition: service_healthy: Store waits for every PD healthcheck, and Server waits for every Store healthcheck.

6 Multi-Node Deployment Example

Below is a configuration example for a three-node deployment:

6.1 Three-Node Configuration Reference

• 3 PD nodes
  • raft ports: 8610, 8611, 8612
  • rpc ports: 8686, 8687, 8688
  • rest ports: 8620, 8621, 8622
• 3 Store nodes
  • raft ports: 8510, 8511, 8512
  • rpc ports: 8500, 8501, 8502
  • rest ports: 8520, 8521, 8522

6.2 Store Node Configuration

For the three Store nodes, the main configuration differences are as follows:

Node A:

grpc:
  port: 8500
raft:
  address: 127.0.0.1:8510
server:
  port: 8520
app:
  data-path: ./storage-a

Node B:

grpc:
  port: 8501
raft:
  address: 127.0.0.1:8511
server:
  port: 8521
app:
  data-path: ./storage-b

Node C:

grpc:
  port: 8502
raft:
  address: 127.0.0.1:8512
server:
  port: 8522
app:
  data-path: ./storage-c

All nodes should point to the same PD cluster:

pdserver:
  address: 127.0.0.1:8686,127.0.0.1:8687,127.0.0.1:8688

And every PD node should list all three Store gRPC addresses:

pd:
  initial-store-list: 127.0.0.1:8500,127.0.0.1:8501,127.0.0.1:8502

6.3 Docker Distributed Cluster Configuration

The distributed Store cluster definition is included in docker/docker-compose-3pd-3store-3server.yml. Each Store node gets its own hostname and environment variables:

# store0, published as 8500 (gRPC), 8510 (Raft), 8520 (REST)
HG_STORE_PD_ADDRESS: pd0:8686,pd1:8686,pd2:8686
HG_STORE_GRPC_HOST: store0
HG_STORE_GRPC_PORT: "8500"
HG_STORE_REST_PORT: "8520"
HG_STORE_RAFT_ADDRESS: store0:8510
HG_STORE_DATA_PATH: /hugegraph-store/storage

# store1, published as 8501, 8511, 8521
HG_STORE_GRPC_HOST: store1
HG_STORE_RAFT_ADDRESS: store1:8510

# store2, published as 8502, 8512, 8522
HG_STORE_GRPC_HOST: store2
HG_STORE_RAFT_ADDRESS: store2:8510

The container ports stay 8500/8510/8520 on every node, only the published host ports differ. The PD nodes set HG_PD_INITIAL_STORE_LIST: store0:8500,store1:8500,store2:8500 to match.

Store nodes start only after all PD nodes pass healthchecks (/v1/health), enforced via depends_on: condition: service_healthy.

To view runtime logs for a running Store container use docker logs <container-name> (e.g. docker logs hg-store0).

See docker/README.md for the full setup guide.

7 Verify Store Service

Confirm that the Store service is running properly:

curl http://localhost:8520/actuator/health

If it returns {"status":"UP"}, it indicates that the Store service has been successfully started.

GET /v1/health is the lighter check used by the Docker image and the compose files. It answers HTTP 200 with an empty body, so use curl -fsS and check the exit code rather than the output:

curl -fsS http://localhost:8520/v1/health && echo OK

7.1 Store REST endpoints

The Store node exposes these read-only endpoints on server.port:

Actuator and Prometheus are reachable because the shipped configuration sets management.endpoints.web.exposure.include: "*" and management.metrics.export.prometheus.enabled: true.

The node also serves maintenance endpoints that change state or run heavy work: PUT /-/state, GET /-/cleaner, GET /v1/partition/dump/{id}, GET /v1/partition/clean/{id}, POST /v1/compat?id=<partition>, GET /v1/arthasstart, POST /raft/options, and the /fix/* and /test/* groups. Use them only for troubleshooting, and keep the REST port off untrusted networks.

7.2 Check registration from PD

You can also check Store node status through the PD API:

curl -u store:admin http://localhost:8620/v1/stores

PD requires basic auth on its REST port. The user name must be one of hg, store, hubble, vermeer, and the password is not validated yet. A call with no credentials returns {"status":-1,"error":"Unauthorized!"}. Only /v1/health, /actuator/* and /v1/prom/targets/* are exempt.

If Store is configured successfully, the response should include status information for the current node, and state: "Up" means the node is running normally. A node stuck at Pending is usually missing from PD’s pd.initial-store-list.

The example below shows a single Store node. If all three nodes are configured correctly and running, the storeId list should contain three IDs, and stateCountMap.Up, numOfService, and numOfNormalService should all be 3.

{
  "message": "OK",
  "data": {
    "stores": [
      {
        "storeId": 8319292642220586694,
        "address": "127.0.0.1:8500",
        "raftAddress": "127.0.0.1:8510",
        "version": "",
        "state": "Up",
        "deployPath": "/Users/{your_user_name}/hugegraph/hugegraph-store/apache-hugegraph-store-{version}/lib/hg-store-node-{version}.jar",
        "dataPath": "./storage",
        "startTimeStamp": 1754027127969,
        "registedTimeStamp": 1754027127969,
        "lastHeartBeat": 1754027909444,
        "capacity": 494384795648,
        "available": 346535829504,
        "partitionCount": 0,
        "graphSize": 0,
        "keyCount": 0,
        "leaderCount": 0,
        "serviceName": "127.0.0.1:8500-store",
        "serviceVersion": "",
        "serviceCreatedTimeStamp": 1754027127000,
        "partitions": []
      }
    ],
    "stateCountMap": {
      "Up": 1
    },
    "numOfService": 1,
    "numOfNormalService": 1
  },
  "status": 0
}

2 - HugeGraph ToolChain

HugeGraph Toolchain includes the Java and Go clients, Loader, Hubble, Tools, Spark Connector, and SeaTunnel Sink/Source. Choose an entry by the task you need to complete, then open the component guide for its configuration and commands.

Testing Guide: For running toolchain tests locally, please refer to HugeGraph Toolchain Local Testing Guide

DeepWiki provides real-time updated project documentation with more comprehensive and accurate content, suitable for quickly understanding the latest project information.

📖 https://deepwiki.com/apache/hugegraph-toolchain

Source repository: apache/hugegraph-toolchain

2.1 - Graph visualization

Use Hubble when you need to view graphs in a browser, run Gremlin, or manage graph connections. Hubble provides a visual interface for graph data, schemas, and tasks.

• Visualize with Hubble

2.2 - HugeGraph-Hubble Quick Start

1 HugeGraph-Hubble Overview

⚠️ Security notice: Hubble listens on plain HTTP. Do not expose it to the public Internet or untrusted networks; terminate HTTPS in front of it and restrict access with IP/port allowlists. Hubble keeps no account database of its own: when the connected HugeGraph Server has authentication enabled, Hubble shows a sign-in page and forwards the credentials to the Server; when the Server allows anonymous access, there is no sign-in and the account pages are hidden.

Version note: This page follows hugegraph-toolchain master. Features that depend on newer Server, PD or Store versions are marked below and are unavailable on older Servers.

Testing Guide: For running HugeGraph-Hubble tests locally, please refer to HugeGraph Toolchain Local Testing Guide

HugeGraph-Hubble is HugeGraph’s web management interface. It connects to one HugeGraph Server, either directly or through PD in a distributed cluster, manages GraphSpaces, graphs and schemas, imports data, runs Gremlin and Cypher queries and built-in graph algorithms, and visualizes the results.

The platform mainly includes the following modules:

Graph Overview

Graph Overview lists GraphSpaces (in PD mode) and graphs. It creates, clones and clears graphs, loads demo graphs, opens the graph detail page with statistics and schema, and jumps to the query workbench.

Metadata Modeling

Metadata Modeling manages PropertyKeys, VertexLabels, EdgeLabels and IndexLabels of one graph, in list and graph views. Schema Templates keep reusable Groovy schemas per GraphSpace that can be applied when a graph is created.

Data Import

The data import page is intended for small-scale trials. For bulk or production imports, use HugeGraph Loader.

Data Sources register FILE, HDFS, JDBC and KAFKA sources. Import tasks are configured in four steps and can run once, on a cron schedule, or continuously for Kafka.

Graph Query

Graph Query runs Gremlin and Cypher statements in immediate or asynchronous mode and displays results as a graph (2D or 3D), a table, or JSON. It keeps execution records and favorite statements.

Built-in Graph Algorithms

Built-in Graph Algorithms provides forms for the Server’s OLTP traverser APIs (interactive exploration) and for OLAP jobs (cluster batch computation through HugeGraph Computer or Vermeer).

Async Tasks

Async Tasks lists background tasks such as Gremlin and Cypher tasks, algorithm tasks, metadata removal, index creation and rebuild, and Vermeer load or compute tasks, with detail, cancel and delete actions.

System and Operations

System and Operations covers the personal profile, account management with GraphSpace permission presets, and, in PD mode, the cluster overview and node details.

1.1 Compatibility

Hubble detects the authentication mode and the capabilities of the connected Server; there is no separate authentication switch in Hubble. The supported combinations are:

Use matching Server, PD and Store minor versions in a distributed cluster.

2 Deploy

There are three ways to deploy hugegraph-hubble

• Use Docker (Convenient for Test/Dev)
• Download the Toolchain binary package
• Source code compilation

Hubble runs on Java 11: the backend is compiled with java.version=11 and the Docker image is based on eclipse-temurin:11-jre. bin/start-hubble.sh only checks that a java binary is on the PATH, so make sure the right JDK is selected.

2.1 Use docker (Convenient for Test/Dev)

Special Note: Hubble no longer asks for the Server host and port on the web page. The Server address comes from conf/hugegraph-hubble.properties: server.direct_url when pd.enabled=false, or PD discovery through pd.peers when pd.enabled=true. Inside the container 127.0.0.1 refers to the hubble container itself, so the packaged default server.direct_url=http://127.0.0.1:8080 does not reach a Server running in another container.

If hubble and server are in the same docker network, we recommend using the container_name (in our example, it is server) as the hostname, and 8080 as the port. Or you can use the host IP as the hostname, and the port is configured by the host for the server.

The image copies the packaged distribution to /hubble, rewrites server.host=0.0.0.0 and clears dashboard.address in /hubble/conf/hugegraph-hubble.properties, exposes port 8088 and runs ./bin/start-hubble.sh -f in the foreground.

Prepare a hugegraph-hubble.properties that points at your Server and keeps the container listening on all interfaces:

server.host=0.0.0.0
server.port=8088
pd.enabled=false
server.direct_url=http://server:8080

Then start hubble with that file mounted over the packaged configuration:

docker run -itd --name=hubble -p 8088:8088 \
  -v "$PWD/hugegraph-hubble.properties:/hubble/conf/hugegraph-hubble.properties" \
  hugegraph/hubble:1.7.0

Alternatively, you can use Docker Compose to start hubble. Additionally, if hubble and the graph is in the same Docker network, you can access the graph using the container name of the graph, eliminating the need for the host machine’s IP address.

Use docker-compose up -d, docker-compose.yml is following:

version: '3'
services:
  server:
    image: hugegraph/hugegraph:1.7.0
    container_name: server
    environment:
      - PASSWORD=xxx
    ports:
      - 8080:8080

  hubble:
    image: hugegraph/hubble:1.7.0
    container_name: hubble
    ports:
      - 8088:8088
    volumes:
      - ./hugegraph-hubble.properties:/hubble/conf/hugegraph-hubble.properties

Note:

1. The docker image of hugegraph-hubble is a convenience release to start hugegraph-hubble quickly, but not official distribution artifacts. You can find more details from ASF Release Distribution Policy.

2. Recommend to use release tag (like 1.7.0) for the stable version. Use latest tag to experience the newest functions in development.

2.2 Download the Toolchain binary package

hubble is in the toolchain project. First, download the binary tar tarball

export VERSION=1.7.0
export ARCHIVE="apache-hugegraph-toolchain-incubating-${VERSION}"
wget "https://downloads.apache.org/hugegraph/${VERSION}/${ARCHIVE}.tar.gz"
tar -xvf "${ARCHIVE}.tar.gz"
cd "${ARCHIVE}/apache-hugegraph-hubble-incubating-${VERSION}"

Edit conf/hugegraph-hubble.properties so that the Server address is correct, then run hubble

bin/start-hubble.sh

start-hubble.sh accepts the following options:

The script starts the JVM with -Xms512m -Dfile.encoding=UTF-8 -Dhubble.home.path=<install dir>, writes the PID to bin/pid, logs to logs/hugegraph-hubble.log and waits up to 30 seconds for http://<server.host>:<server.port>/about to answer before it returns.

The packaged default is server.host=localhost, so the service only accepts loopback connections until you change it. After startup, open http://<host>:8088.

Run bin/stop-hubble.sh to stop the service. It sends SIGTERM first so that the shutdown hooks pause running load tasks and close the embedded H2 database cleanly, and only escalates to SIGKILL if the process is still alive after STOP_TIMEOUT seconds (environment variable, default 30).

2.3 Source code compilation

Hubble’s build uses frontend-maven-plugin in hugegraph-hubble/hubble-dist/pom.xml to install Node.js v18.20.8 and Yarn v1.22.21, so neither tool needs to be installed beforehand. JDK 11 and Maven are required.

Download the toolchain source code.

git clone https://github.com/apache/hugegraph-toolchain.git

Compile hubble. It depends on the loader and client, so you need to build these dependencies in advance during the compilation process (you can skip this step later).

cd hugegraph-toolchain
python -m pip install -r hugegraph-hubble/hubble-dist/assembly/travis/requirements.txt
mvn install -pl hugegraph-client,hugegraph-loader -am -Dmaven.javadoc.skip=true -DskipTests -ntp
cd hugegraph-hubble
mvn -e compile package -Dmaven.javadoc.skip=true -Dmaven.test.skip=true -ntp
cd apache-hugegraph-hubble-*

Run hubble

bin/start-hubble.sh -d

For frontend work, run yarn dev inside hubble-fe. The backend POM does not configure spring-boot:run, so run org.apache.hugegraph.HugeGraphHubble from hubble-be/target/classes with -Dhubble.home.path pointing at a writable directory instead.

3 Platform Workflows

The home page groups the modules into three journeys: Graph Overview, Graph Import and Graph Query. It also shows whether Hubble runs in PD / cluster mode or in non-PD standalone mode. The module usage process of the platform is as follows:

4 Platform Instructions

4.1 Graph Management

In PD mode, [Graph Space Management] lists all GraphSpaces of the cluster and can create or edit one, including its alias, optional Kubernetes namespace and compute task, and resource limits. In non-PD standalone mode there is exactly one GraphSpace named DEFAULT and the GraphSpace list is skipped.

4.1.1 Graph creation

Under the graph management module, click [New Graph] and fill in the graph name, an optional alias, an optional schema template and optional sample data. The graph name is unique inside its GraphSpace and cannot be changed after creation.

Create graph by filling in the content as follows:

Special Note: The Server connection is not configured on this page. It comes from conf/hugegraph-hubble.properties, through server.direct_url or PD discovery; see section 2.1 for the Docker hostname rules. Graph creation is only offered when the connected Server exposes it (REST API 0.67 or later); on older Servers the graph list is read-only.

4.1.2 Graph Access

Realize the information access to the graph space. After entering, you can perform operations such as multidimensional query analysis, metadata management, data import, and algorithm analysis of the graph. [Open Graph Studio] opens the query workbench, [Metadata Config] opens the schema pages, and the graph detail page shows vertex and edge statistics together with the schema.

4.1.3 Graph management

1. The graph list has a card view and a list view. Search matches the graph name.
2. Per-graph actions are View Schema (with [Export Groovy Schema]), Metadata Config, Clone Graph (schema only, or schema and data), Clear Schema and Data, Delete, and, in PD mode, Set Default.
3. [Sample data and resources] builds a demo graph inside the current graph: the Red Chamber demo graph, the People & Software demo graph, or the Tiny Movie Rank demo. These demos add missing schema and elements only and never clear existing data.

4.2 Metadata Modeling (list + graph mode)

4.2.1 Module entry

Open [Metadata Config] from the graph list, or the metadata page of a graph at /graphspace/<graphspace>/graph/<graph>/meta. The page has five tabs, Property, Vertex Type, Edge Type, Vertex Index and Edge Index, and a switch between list view and graph view.

4.2.2 Property type

4.2.2.1 Create type

1. Fill in or select the property name, data type, and cardinality to complete the creation of the property.
2. Created properties can be used as properties of vertex type and edge type.

List mode:

Graph mode:

4.2.2.2 Management

1. You can delete a single item or delete it in batches in the property list. A property that is still used by a vertex or edge type cannot be deleted.
2. Deleting metadata runs as an asynchronous task; check Async Tasks for its progress.

4.2.3 Vertex type

4.2.3.1 Create type

1. Fill in or select the vertex type name, ID strategy, associated properties, primary key properties, vertex style, content displayed below the vertex in the query result, and index information: including whether to create a type index, and the specific content of the property index, complete the vertex type creation.

List mode:

Graph mode:

4.2.3.2 Administration

1. Editing operations are available. The vertex style, associated properties, vertex display content, and property index can be edited, and the rest cannot be edited. In graph mode, double-click a vertex type to edit it.

2. You can delete a single item or delete it in batches.

4.2.4 Edge Types

4.2.4.1 Create

1. Fill in or select the edge type name, the type (Normal, Parent or Sub, for edge type hierarchies), start point type, end point type, associated properties, whether to allow multiple connections, edge style, content displayed below the edge in the query result, and index information: including whether to create a type index, and the specific content of the property index, complete the creation of the edge type.

List mode:

Graph mode:

4.2.4.2 Administration

1. Editing operations are available. Edge styles, associated properties, edge display content, and property indexes can be edited, and the rest cannot be edited, the same as the vertex type.
2. You can delete a single item or delete it in batches.

4.2.5 Index Types

Displays vertex and edge indexes for vertex types and edge types. Secondary, range, search and unique indexes are supported.

4.2.6 Schema Templates

[Schema templates] at /graphspace/<graphspace>/schema keeps a reusable template library for the current GraphSpace. Example templates ship with Hubble and can be used, removed or restored; they are not stored on the Server until you save one. User templates hold Groovy schema on the Server and can be created, edited and deleted. When you create a graph, an existing template can be selected so that its schema is applied right away.

4.3 Data Import

Note: currently, we recommend to use hugegraph-loader to import data formally. The built-in import of hubble is used for testing and getting started.

The usage process of data import is as follows:

4.3.1 Module entrance

Left navigation, under Graph Import: [Data Sources] and [Data Import].

4.3.2 Data sources

1. [Data Sources] registers where an import task reads from. Four source types are supported: FILE (local upload), HDFS, Kafka and JDBC.
2. For a FILE source, upload the files that need to be composed. The accepted formats come from upload_file.format_list, which defaults to csv and txt.
3. The single file and total size limits default to 1 GB and 10 GB, and unfinished uploads are discarded after upload_file.max_uploading_time, which defaults to 12 hours.

4.3.3 Create task

1. [Data Import] > [Create Task] configures an import in four steps: Basic Information, Select Source Fields, Select Mapping Fields and Schedule.
2. Basic Information takes the task name (1 to 48 Chinese characters, letters, digits or _), the target GraphSpace and graph, the source type and the data source.
3. Multiple import tasks can be created and imported in parallel.

4.3.4 Setting up data mapping

1. Set up data mapping for the selected source, including file settings and type settings

2. File settings: check or fill in whether to include the header, separator, encoding format and other settings of the source itself, all set the default values, no need to fill in manually

3. Type setting:

   1. Vertex map and edge map:

      【Vertex Type】: Select the vertex type, and map the column data of the source for its ID;

      【Edge Type】: Select the edge type and map the column data of the source to the ID column of its start point type and end point type;

   2. Mapping settings: map the column data of the source to the properties of the selected vertex type. Here, if the property name is the same as the header name of the file, the mapping property can be automatically matched, and there is no need to manually fill in the selection.

   3. After completing the setting, the setting list will be displayed before proceeding to the next step. It supports the operations of adding, editing and deleting mappings.

Fill in the settings map:

Mapping list:

4.3.5 Import data

The last step chooses when the task runs: Run Once for a one-off import, Scheduled with a Quartz cron expression such as 0 0/5 * * * ?, or Realtime for a Kafka source.

1. Import settings

• The import setting parameter items are as shown in the figure below, all set the default value, no need to fill in manually

2. Import details

• Run a task from the task list to start the import, and pause, edit or delete it from the same list
• The execution history of a task provides the execution instance ID, the number of imported records, the average rate in records per second, the import duration and the status of each run
• If the import fails, you can view the specific reason

4.4 Graph Query

4.4.1 Module entry

Left navigation, under Graph Query: [GQL Traversal].

4.4.2 Multi-graphs switching

The top bar carries the current GraphSpace and graph, so you can flexibly switch the operation space of multiple graphs without leaving the page.

4.4.3 Graph Analysis and Processing

HugeGraph supports Gremlin, a graph traversal query language of Apache TinkerPop3. Gremlin is a general graph database query language. By entering Gremlin statements and clicking execute, you can perform query and analysis operations on graph data, and create and delete vertices/edges, modify vertex/edge properties, etc. When the connected Server supports Cypher, a Cypher tab is offered next to Gremlin. A Text2GQL tab is present as a user interface preview only: it is not connected to a model or a query service, and nothing entered there is sent or executed.

Each statement can run in one of two modes. Immediate returns the result inline and suits analyses that finish within about 30 seconds; Async submits a task instead, and its progress and result appear under Async Tasks. Ctrl/Command + Enter runs the current statement.

After the query, below is the graph result display area, which provides 3 kinds of graph result display modes: [Graph Mode], [Table Mode], [Json Mode]. The graph canvas can be rendered in 2D or 3D.

⚠️ SEC Reminder: Hubble allows the direct input and execution of native Gremlin query statements on the web interface, which grants users relatively high operational privileges. Please avoid exposing the Hubble service to public network environments. It is recommended to ensure that the graph database server has enabled the Authentication System (Auth) combined with an IP Whitelist for strict permission control when in use, preventing unauthorized access or malware execution risks.

Support zoom, center, full screen, layout and style configuration, legend, minimap, undo and redo, and export operations. The canvas can be exported as JSON, CSV or an image, and a previously exported canvas can be imported again.

【Picture Mode】

【Table mode】

【Json mode】

4.4.4 Data Details

Click the vertex/edge entity to view the data details of the vertex/edge, including vertex/edge type, vertex ID, attribute and corresponding value, expand the information display dimension of the graph, and improve the usability.

4.4.5 Multidimensional Path Query of Graph Results

In addition to the global query, an in-depth customized query and hidden operations can be performed for the vertices in the query result to realize customized mining of graph results.

Right-click a vertex, and the menu entry of the vertex appears, which can be displayed, inquired, hidden, etc.

• Expand: Click to display the vertices associated with the selected point.
• Query: By selecting the edge type and edge direction associated with the selected point, and then selecting its attributes and corresponding filtering rules under this condition, a customized path display can be realized.
• Hide: When clicked, hides the selected point and its associated edges.

Double-clicking a vertex also displays the vertex associated with the selected point.

4.4.6 Add vertex/edge

4.4.6.1 Added vertex

In the graph area, two entries can be used to dynamically add vertices, as follows:

1. Click on the graph area panel, the Add Vertex entry appears
2. Click the first icon in the action bar in the upper right corner

Complete the addition of vertices by selecting or filling in the vertex type, ID value, and attribute information.

The entry is as follows:

Add the vertex content as follows:

4.4.6.2 Add edge

Right-click a vertex in the graph result to add the outgoing or incoming edge of that point.

4.4.7 Execute the query of records and favorites

1. Record each query record at the bottom of the graph area, including: query time, execution type, content, status, time-consuming, as well as [collection] and [load] operations, to achieve a comprehensive record of graph execution, with traces to follow, and Can quickly load and reuse execution content
2. Provides the function of collecting sentences, which can be used to collect frequently used sentences, which is convenient for fast calling of high-frequency sentences.

4.5 Async Tasks

4.5.1 Module entry

Left navigation, under Graph Query: [Async Tasks].

4.5.2 Task Management

1. Provide unified management and result viewing of asynchronous tasks. The task types are:

• gremlin: Gremlin tasks
• cypher: Cypher tasks
• computer-dis: algorithm tasks
• remove_schema: remove metadata
• create_index: create an index
• rebuild_index: rebuild the index
• vermeer-task:load: Vermeer graph load tasks
• vermeer-task:compute: Vermeer graph compute tasks

2. The list displays the asynchronous task information of the current graph, including task ID, task name, task type, creation time, time-consuming, status, operation, and realizes the management of asynchronous tasks. The list refreshes every 5 seconds.
3. Support filtering by task type and status
4. Support searching for task ID and task name
5. A running task can be cancelled, and asynchronous tasks can be deleted one by one or in batches

4.5.3 Gremlin asynchronous tasks

1. Create a task

• The graph query module supports two execution modes, immediate query and asynchronous task; if the user switches to the asynchronous mode, after clicking execute, an asynchronous task will be created in the asynchronous task center. A Cypher statement creates a Cypher task in the same way;

2. Task submission

• After the task is submitted successfully, the graph area returns the submission result and task ID

3. Mission details

• Provide [View] entry, you can jump to the task details to view the specific execution of the current task After jumping to the task center, the currently executing task line will be displayed directly

Click to view the entry to jump to the task management list, as follows:

4. View the results

• The results are displayed in the form of JSON, and a compact result can be expanded inline

4.5.4 Algorithm tasks

Batch algorithms submitted from [Built-in Graph Algorithms] land here as algorithm tasks, and so do Vermeer graph load and compute tasks. Find a task by ID in the list and open it to follow its progress and result. See section 4.6 for the algorithm forms themselves.

4.5.5 Delete metadata, rebuild index

1. Create a task

• In the metadata modeling module, when deleting metadata, an asynchronous task for deleting metadata can be created

• When editing an existing vertex/edge type operation, when adding an index, an asynchronous task of creating an index can be created

2. Task details

• After confirming/saving, you can jump to the task center to view the details of the current task

4.6 Built-in Graph Algorithms

[Built-in Graph Algorithms] under Graph Query provides parameter forms for the algorithms the Server exposes, grouped by intent: explore neighborhoods, find paths and connections, compare and rank, measure importance, find communities, and analyze graph structure. Each algorithm links to its official API documentation.

Two execution modes are offered:

• Interactive exploration runs the Server’s OLTP traverser APIs and returns results directly. It covers K-out and K-neighbor, shortest path in its single source, weighted, and multi-node forms, paths and all paths, customized and template paths, rings and rays, crosspoints and customized crosspoints, same neighbors, Jaccard similarity, fusiform similarity, Adamic-Adar, resource allocation, egonet, and the rank and neighbor rank APIs.
• Cluster batch computation submits an asynchronous job over the whole graph and reports the result under Async Tasks. It covers PageRank and personal PageRank, degree, closeness and betweenness centrality, K-core, weakly connected components, label propagation, Louvain, triangle count, cluster coefficient, rings detection, subgraph matching and links, with Vermeer variants where the deployment provides Vermeer.

Batch algorithms need a HugeGraph Computer environment, including Kubernetes when the deployment requires it. When Computer cannot be reached, the page says so instead of submitting the task.

4.7 Sign-in and account management

When the connected Server has authentication enabled, Hubble opens the sign-in page at /login. Sign in with a HugeGraph Server account: Hubble forwards the credentials to the Server and keeps the returned token for the browser session, and it stores no accounts of its own. Login attempts are throttled, so after the first three failures for the same account and address, further attempts back off starting at 5 seconds and doubling up to 600 seconds. When the Server allows anonymous access, /login redirects to the home page and the profile and account pages are hidden.

[My Profile] shows the account details and changes the password. [Account Management] is available to accounts that may manage accounts or GraphSpace members. It creates accounts and assigns one of four access presets: Super Administrator, GraphSpace Read-only, GraphSpace Read-write and GraphSpace Administrator. Low-level role, target, access and belong records are not exposed in the interface.

4.8 Cluster operations

In PD mode, [System & Operations] adds [Cluster Overview] and [Node details] for accounts with the matching capabilities. Cluster Overview shows the topology, per-tier node status and cluster facts such as stores online, PD leader, capacity, data size, graphs, partitions and replicas. Node details lists every discovered node with filters for type and status, and opens a node profile with its metrics, leader role and Raft shards. Node details is also available in standalone mode; Cluster Overview needs PD.

An optional external dashboard can be linked from the navigation page through dashboard.address. It is a separate monitoring entry, and leaving it unconfigured does not affect Cluster Overview or Node details.

5 Configuration

HugeGraph-Hubble can be configured through the conf/hugegraph-hubble.properties file.

5.1 Service Configuration

5.2 Server and PD

5.3 Gremlin Query Limits

These settings control query result limits to prevent memory issues:

5.4 File Upload

These keys are not written into the packaged file; add them to override the defaults.

5.5 Cluster Operations

These keys drive the Cluster Overview and Node details pages.

The operations.store.allowed_targets default covers local testing only. A production deployment must list every trusted Store scheme, host and port explicitly, because discovery never adds an origin to this allowlist. HTTPS origins keep their configured hostname for TLS SNI and certificate verification. Supply the PD and Store passwords through a protected deployment configuration rather than the packaged file.

2.3 - Graph import

Choose an import tool when you need to write file, database, or message data into HugeGraph. Use Loader for a direct import, SeaTunnel Sink when an existing Source, Transform, and Sink pipeline should be reused, or Spark Connector from a Spark job.

• Import data with Loader
• Import graph data with SeaTunnel Sink
• Read and write graph data with Spark Connector

2.3.1 - Import Graph Data with SeaTunnel Sink

SeaTunnel connects data sources such as databases and Kafka to HugeGraph. The connector has two parts: Source reads data and Sink writes data^[1][2], with SeaTunnel transform components available between them. To export or migrate data from HugeGraph, see the SeaTunnel Source export and migration guide.

Version requirement: This guide targets SeaTunnel 3.0+. All examples use mappings

Click a diagram to view the original size.

1 Loader, Tools, and SeaTunnel

HugeGraph-Loader is suited to direct imports from common data sources. HugeGraph-Tools focuses on standalone graph management, backup, and export. SeaTunnel organizes a job as Source → Transform → Sink, so you can reuse existing connectors, transforms, and data pipelines.

Table legend

✅ Supported natively; ⚠️ conditional support or requires an extra component/external platform; ❌ not provided

SeaTunnel covers Loader’s graph-import and Tools’ export and migration scenarios in one expandable pipeline, and it also supports SQL-CDC and dozens of input and output types. Loader and Tools normally run on one machine, while SeaTunnel supports both standalone and distributed deployments and scales with data and task volume. Tools’ schedule-backup can create a crontab entry, but it does not provide unified workflow orchestration and resource management.

Existing Spark/Flink daily jobs

Both HugeGraph Source and Sink list SeaTunnel Engine (Zeta), Spark, and Flink as supported engines in SeaTunnel 3.0.0-release. If you express the daily job as a SeaTunnel job and submit it to that engine, records can move directly from Source to Transform to Sink without an intermediate file. If you keep the existing Spark/Flink DAG, SeaTunnel does not automatically take over its in-memory DataFrame or stream. Adapt it into a SeaTunnel job or expose the data through a Source connector

Loader and Tools are focused, direct, and quick to start. Use Loader for a direct graph import; use Tools for backup, restore, export, or daily operations. If a SeaTunnel job already exists, adding HugeGraph to that pipeline is usually simpler. For higher import throughput, Loader’s bypass-server path and other import optimizations are a better fit; measured peaks of 1-2 million records/s require a specific backend, data set, and hardware configuration and are not a general performance guarantee. For new SeaTunnel jobs, use 3.0+ and mappings. Recheck the connector configuration when using another version.

2 Prepare the environment

2.1 Get SeaTunnel 3.0+

The SeaTunnel 3.0+ setup guide lists JDK 8 and JDK 11 as supported. This guide uses JDK 11 and sets JAVA_HOME. Clone the SeaTunnel 3.0+^[4] branch and build a distribution by following the upstream development setup guide^[5]:

git clone --branch 3.0.0-release https://github.com/apache/seatunnel.git
cd seatunnel
./mvnw clean package -pl seatunnel-dist -am -Dmaven.test.skip=true

Extract the binary package from seatunnel-dist/target/. Run the remaining commands from the extracted SeaTunnel installation directory. When updating the feature set, switch to another version as needed. Keep the engine and connector plugins from the same build, and do not mix different plugin versions.

This guide uses the bundled Zeta engine in local mode^[6][7]. Check that connectors/ contains HugeGraph and the JDBC or Kafka connector required by each example^[11][12]. If a custom build does not include them, add the plugins produced by that same build. The JDBC examples also require the MySQL driver JAR in lib/, with driver class com.mysql.cj.jdbc.Driver.

2.2 Prepare HugeGraph and data sources

Start HugeGraph Server and create a graph for testing. The examples use the hugegraph graph in the DEFAULT graph space. Adjust these names to match the server configuration; graph space names are case-sensitive. If authentication is enabled, provide username and password in the HugeGraph Source and Sink configurations.

The following graph model is shared by the JDBC and Kafka examples. mappings creates missing PropertyKey, VertexLabel, and EdgeLabel definitions by default; existing schema definitions must be compatible.

The mysql, kafka, and hugegraph host names in the examples are placeholders. Replace them with addresses reachable from the SeaTunnel runtime. Inside a container, 127.0.0.1 points to that container; services on the same Docker network can use their service names. Set host to a host name or IP address, and set the port separately.

3 Import from a relational database (sql2graph)

Use two jobs for this import: write the person table as vertices first, then write the knows table as edges. Both edge endpoints will already exist when the edge job runs.

3.1 Import vertices

Prepare the sample data in the MySQL demo database and grant the configured account read access:

CREATE TABLE person (
  name VARCHAR(64) PRIMARY KEY,
  age INT NOT NULL
);
INSERT INTO person VALUES ('marko', 29), ('vadas', 27);

Save the following as config/sql2graph-person.conf and replace the database user name and password:

env {
  job.mode = "BATCH"
}

source {
  Jdbc {
    url = "jdbc:mysql://mysql:3306/demo?useSSL=false&serverTimezone=UTC"
    driver = "com.mysql.cj.jdbc.Driver"
    username = "seatunnel"
    password = "change_me"
    query = "SELECT name, age FROM person ORDER BY name"
  }
}

sink {
  HugeGraph {
    host = "hugegraph"
    port = 8080
    graph_name = "hugegraph"
    graph_space = "DEFAULT"
    batch_failure_fallback = false
    mappings = [
      {
        type = "VERTEX"
        label = "person"
        idStrategy = "PRIMARY_KEY"
        idFields = ["name"]
        properties = ["name", "age"]
      }
    ]
  }
}

./bin/seatunnel.sh --config ./config/sql2graph-person.conf -m local

Check the result in Hubble or Gremlin. You should find marko and vadas with their ages:

g.V().hasLabel('person').valueMap('name', 'age')

idFields = ["name"] uses the name to generate the primary key. Importing the same name again writes to the same vertex. properties lists the source fields to write.

3.2 Import edges

Prepare the relation table. Its two endpoint fields correspond to person.name from the vertex job:

CREATE TABLE knows (
  source_name VARCHAR(64) NOT NULL,
  target_name VARCHAR(64) NOT NULL,
  since INT NOT NULL
);
INSERT INTO knows VALUES ('marko', 'vadas', 2010);

env {
  job.mode = "BATCH"
}

source {
  Jdbc {
    url = "jdbc:mysql://mysql:3306/demo?useSSL=false&serverTimezone=UTC"
    driver = "com.mysql.cj.jdbc.Driver"
    username = "seatunnel"
    password = "change_me"
    query = "SELECT source_name, target_name, since FROM knows ORDER BY source_name, target_name"
  }
}

sink {
  HugeGraph {
    host = "hugegraph"
    port = 8080
    graph_name = "hugegraph"
    graph_space = "DEFAULT"
    batch_failure_fallback = false
    check_vertex = true
    mappings = [
      {
        type = "EDGE"
        label = "knows"
        sourceConfig = {
          label = "person"
          idFields = ["source_name"]
        }
        targetConfig = {
          label = "person"
          idFields = ["target_name"]
        }
        fieldMapping = {
          source_name = "name"
          target_name = "name"
        }
        properties = ["since"]
      }
    ]
  }
}

After the vertex job succeeds, run the edge job:

./bin/seatunnel.sh --config ./config/sql2graph-knows.conf -m local

The following query should return a knows edge from marko to vadas with since set to 2010:

g.V().has('person', 'name', 'marko').outE('knows').where(inV().has('name', 'vadas')).valueMap()

sourceConfig and targetConfig identify the endpoint fields. fieldMapping maps them to the vertex primary key name, and properties = ["since"] writes only the edge property. The example enables check_vertex = true and disables per-record fallback after a batch failure (batch_failure_fallback = false), so a missing endpoint or write failure causes the job to fail.

If the relation table has only numeric foreign keys while the graph uses names as primary keys, join the names in SQL before passing the records to the Sink. See MySQL CDC Source^[13] for MySQL CDC integration.

4 Import from Kafka (kafka2graph)

Kafka is useful for a continuous stream of events. Create the user-events topic and publish the following JSON message. Each message becomes one person vertex:

{"name":"marko","age":29}

Save the following as config/kafka2graph.conf:

env {
  job.mode = "STREAMING"
  checkpoint.interval = 10000
  sink.flush.interval = 5000
}

source {
  Kafka {
    bootstrap.servers = "kafka:9092"
    topic = "user-events"
    consumer.group = "hugegraph-import"
    start_mode = "earliest"
    format = "json"
    schema = {
      fields = {
        name = "string"
        age = "int"
      }
    }
  }
}

sink {
  HugeGraph {
    host = "hugegraph"
    port = 8080
    graph_name = "hugegraph"
    graph_space = "DEFAULT"
    batch_failure_fallback = false
    mappings = [
      {
        type = "VERTEX"
        label = "person"
        idStrategy = "PRIMARY_KEY"
        idFields = ["name"]
        properties = ["name", "age"]
      }
    ]
  }
}

./bin/seatunnel.sh --config ./config/kafka2graph.conf -m local

Use the Gremlin query from section 3.1 to check the data. The streaming job keeps running. checkpoint.interval saves job state every 10 seconds, while sink.flush.interval asks Zeta to flush every 5 seconds so a small number of messages does not wait for a full batch.

HugeGraph Sink writes with at-least-once semantics, so recovery can replay records. PRIMARY_KEY sends the same name to the same vertex, but it does not make every update exactly-once. Scheduled flushing is provided by Zeta and does not apply to Spark or Flink engines.

5 Common configuration and troubleshooting

The following table applies to the SeaTunnel 3.0+ version used by this guide:

Use these checks when a job fails:

• mappings is unknown or HugeGraph Source is missing: Check that the engine and HugeGraph connector come from the same SeaTunnel 3.0+ build.
• Connection failure: Check the host, port, graph space, authentication details, and whether the SeaTunnel runtime can reach the service.
• Schema incompatibility: Check the ID strategy, property types, and edge endpoints. Automatic creation does not change an existing PRIMARY_KEY label into CUSTOMIZE_STRING.
• Small Kafka batches do not appear promptly: Confirm that the job uses Zeta and set sink.flush.interval in env. In this version, batch_interval_ms is retained only for compatibility and cannot replace it.

6 Choosing a tool

Choose a tool based on the work to complete. Use Tools for graph management, Gremlin, backup, or cloning. Use Loader for a direct graph import. Choose SeaTunnel when you need to reuse a Source, Transform, and Sink pipeline. For SeaTunnel graph reads and migrations, prepare the environment using the SeaTunnel 3.0+ version used by this guide.

7 References

HugeGraph connectors

^[1] HugeGraph Source
^[2] HugeGraph Sink

Configuration and deployment

^[3] HOCON job configuration
^[4] SeaTunnel 3.0.0-release branch
^[5] SeaTunnel development setup
^[6] SeaTunnel local deployment

Execution engines

^[7] SeaTunnel Engine Overview
^[8] SeaTunnel Spark Engine
^[9] SeaTunnel Flink Engine
^[10] Connector V2 multi-engine support

Data source connectors

^[11] JDBC Source
^[12] Kafka Source
^[13] MySQL CDC Source

Legacy compatibility

^[14] SeaTunnel 2.3.13 HugeGraph Sink

Legacy version note

This guide targets SeaTunnel 3.0+. Its Source, mappings, and graph migration examples do not apply to 2.3.13. That legacy version provides only the HugeGraph Sink, uses schema_config, and requires the graph schema to be created in advance. If you must use 2.3.13, follow the official Sink documentation^[14] instead of copying this guide’s configuration

2.4 - HugeGraph-Loader Quick Start

1 HugeGraph-Loader Overview

HugeGraph-Loader is the data import component of HugeGraph, which can convert data from various data sources into graph vertices and edges and import them into the graph database in batches.

Currently supported data sources include:

• Local disk file or directory, supports TEXT, CSV and JSON format files, supports compressed files
• HDFS file or directory supports compressed files
• Mainstream relational databases, such as MySQL, PostgreSQL, Oracle, SQL Server
• Kafka topic
• An existing HugeGraph graph, used to copy data from one graph into another

Local disk files and HDFS files support resumable uploads.

It will be explained in detail below.

Note: HugeGraph-Loader requires HugeGraph Server service, please refer to HugeGraph-Server Quick Start to download and start Server

Testing Guide: For running HugeGraph-Loader tests locally, please refer to HugeGraph Toolchain Local Testing Guide

2 Get HugeGraph-Loader

HugeGraph-Loader is available in the following three ways:

• Use docker image (Convenient for Test/Dev)
• Download the compiled tarball
• Clone source code then compile and install

2.1 Use Docker image (Convenient for Test/Dev)

We can deploy the loader service using docker run -itd --name loader hugegraph/loader:1.7.0. For the data that needs to be loaded, it can be copied into the loader container either by mounting -v /path/to/data/file:/loader/file or by using docker cp.

Alternatively, to start the loader using docker-compose, the command is docker-compose up -d. An example of the docker-compose.yml is as follows:

version: '3'

services:
  server:
    image: hugegraph/hugegraph:1.7.0
    container_name: server
    ports:
      - 8080:8080

  hubble:
    image: hugegraph/hubble:1.7.0
    container_name: hubble
    ports:
      - 8088:8088

  loader:
    image: hugegraph/loader:1.7.0
    container_name: loader
    # mount your own data here
    # volumes:
      # - /path/to/data/file:/loader/file

The specific data loading process can be referenced under 4.5 User Docker to load data

Note:

1. The docker image of hugegraph-loader is a convenience release to start hugegraph-loader quickly, but not official distribution artifacts. You can find more details from ASF Release Distribution Policy.

2. Recommend to use release tag(like 1.7.0) for the stable version. Use latest tag to experience the newest functions in development.

2.2 Download the compiled archive

Download the latest version of the HugeGraph-Toolchain release package:

export VERSION=1.7.0
export ARCHIVE="apache-hugegraph-toolchain-incubating-${VERSION}"
wget "https://downloads.apache.org/hugegraph/${VERSION}/${ARCHIVE}.tar.gz"
tar zxf "${ARCHIVE}.tar.gz"

2.3 Clone source code to compile and install

Clone the latest version of HugeGraph-Loader source package:

# 1. get from github
git clone https://github.com/apache/hugegraph-toolchain.git

# 2. Download a released source package
export VERSION=1.7.0
export ARCHIVE="apache-hugegraph-toolchain-incubating-${VERSION}"
wget "https://downloads.apache.org/hugegraph/${VERSION}/${ARCHIVE}-src.tar.gz"

mvn install:install-file -Dfile=./ojdbc8.jar -DgroupId=com.oracle -DartifactId=ojdbc8 -Dversion=12.2.0.1 -Dpackaging=jar

Compile and generate tar package:

cd hugegraph-toolchain
mvn clean package -pl hugegraph-loader -am -DskipTests -ntp

3 How to use

The basic process of using HugeGraph-Loader is divided into the following steps:

• Write graph schema
• Prepare data files
• Write input source map files
• Execute command import

3.1 Construct graph schema

This step is the modeling process. Users need to have a clear idea of ​​their existing data and the graph model they want to create, and then write the schema to build the graph model.

For example, if you want to create a graph with two types of vertices and two types of edges, the vertices are “people” and “software”, the edges are “people know people” and “people create software”, and these vertices and edges have some attributes, For example, the vertex “person” has: “name”, “age” and other attributes, “Software” includes: “name”, “sale price” and other attributes; side “knowledge” includes: “date” attribute and so on.

graph model example

After designing the graph model, we can use groovy to write the definition of schema and save it to a file, here named schema.groovy.

// Create some properties
schema.propertyKey("name").asText().ifNotExist().create();
schema.propertyKey("age").asInt().ifNotExist().create();
schema.propertyKey("city").asText().ifNotExist().create();
schema.propertyKey("date").asText().ifNotExist().create();
schema.propertyKey("price").asDouble().ifNotExist().create();

// Create the person vertex type, which has three attributes: name, age, city, and the primary key is name
schema.vertexLabel("person").properties("name", "age", "city").primaryKeys("name").ifNotExist().create();
// Create a software vertex type, which has two properties: name, price, the primary key is name
schema.vertexLabel("software").properties("name", "price").primaryKeys("name").ifNotExist().create();

// Create the knows edge type, which goes from person to person
schema.edgeLabel("knows").sourceLabel("person").targetLabel("person").ifNotExist().create();
// Create the created edge type, which points from person to software
schema.edgeLabel("created").sourceLabel("person").targetLabel("software").ifNotExist().create();

Please refer to the corresponding section in hugegraph-client for the detailed description of the schema.

3.2 Prepare data

The data sources currently supported by HugeGraph-Loader include:

• local disk file or directory
• HDFS file or directory
• Partial relational database
• Kafka topic
• An existing HugeGraph graph

3.2.1 Data source structure