Skip to content

This is the multi-page printable view of this section. .

Return to the regular view of this page.

Documentation

Apache HugeGraph Documentation

Apache HugeGraph includes graph database, graph computing, and graph AI components. The HugeGraph core engine manages property graphs, transactions, and real-time queries; Computer and Vermeer run graph algorithms; and HugeGraph-AI provides GraphRAG, graph machine learning, and a Python client.

Quick Navigation by Scenario

I want to…Start here
Run graph queries (OLTP)HugeGraph Server Quick Start
Run graph algorithms (OLAP)HugeGraph Computing
Build Graph + AI applicationsHugeGraph-AI
Batch import dataHugeGraph Loader, SeaTunnel
Visualize and manage graphsHubble Web UI

Ecosystem Overview

┌─────────────────────────────────────────────────────────────────┐
│                  Apache HugeGraph Ecosystem                      │
├─────────────────────────────────────────────────────────────────┤
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────────┐  │
│  │ HugeGraph   │  │ Vermeer*    │  │ HugeGraph-AI            │  │
│  │ Core Engine │  │ Computer    │  │ (GraphRAG/ML/Python)    │  │
│  │ (OLTP)      │  │ (Java BSP)  │  │                         │  │
│  └─────────────┘  └─────────────┘  └─────────────────────────┘  │
│         │               │                    │                   │
│  ┌──────┴───────────────┴────────────────────┴──────────────┐   │
│  │              HugeGraph Toolchain                          │   │
│  │  Hubble (UI) | Loader | Client (Java/Go/Py) | Tools      │   │
│  └───────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────┘
  • Vermeer is the default graph computing entry; Computer runs Java BSP jobs.

Core Components

  • HugeGraph Core Engine (OLTP): Exposes REST APIs through HugeGraph Server and supports Gremlin and Cypher queries
  • HugeGraph Toolchain: Includes Java/Go clients, Loader, Hubble, Spark Connector, and Tools; the Python client is maintained in HugeGraph-AI, and a Rust client is under development
  • HugeGraph Computer: Contains the distributed Computer engine and the in-memory Vermeer engine
  • HugeGraph-AI: Includes GraphRAG, graph machine learning, the Python client, and the Vermeer Python client

Deployment Modes

ModeCore ComponentsSuitable ScenariosData Scale
StandaloneServer + RocksDBDevelopment, testing, Graph AI, and small to medium-scale production environments≤ 2 TB
DistributedServer + PD + Store (HStore)Production, horizontal scaling, and multi-replica deployment≤ 1 PB

See the system introduction and the corresponding quick-start guides for each component’s scope and startup instructions.

1 - Apache HugeGraph Introduction

What Is Apache HugeGraph?

Apache HugeGraph is an easy-to-use, efficient, general-purpose open-source full-stack graph system (GitHub). It covers three major areas: graph databases (OLTP real-time queries), graph computing (OLAP large-scale analysis), and graph AI (GraphRAG and graph machine learning).

HugeGraph supports fast storage and queries for tens of billions of vertices and edges, with strong OLTP performance. Its graph engine is compatible with Apache TinkerPop 3 and supports both Gremlin and Cypher (the OpenCypher standard).

Typical use cases: deep relationship exploration, association analysis, path search, feature extraction, community detection, and knowledge graphs. Application areas: network security, telecom anti-fraud, financial risk control, advertising and recommendations, social networks, and intelligent Q&A.

Ecosystem Overview

┌────────────────────────────────────────────────────────────────────┐
│            Apache HugeGraph - Full-Stack Graph System             │
├──────────────────┬────────────────────┬────────────────────────────┤
│  Graph DB (OLTP) │    Graph Compute   │          Graph AI          │
│  HugeGraph       │  Vermeer (Memory)  │       HugeGraph-AI         │
│  Server          │  Computer (Dist.)  │     GraphRAG / GNN / Py    │
├──────────────────┴────────────────────┴────────────────────────────┤
│                       HugeGraph Toolchain                          │
│ Hubble | Loader | Client (Java/Go/Python; Rust WIP) | Spark | Tools│
│ SeaTunnel 3.0+: Source + Sink                                      │
└────────────────────────────────────────────────────────────────────┘

HugeGraph Server (OLTP Graph Engine)

HugeGraph Server is the OLTP engine and service entry point for the graph database. It handles property graph modeling, transaction processing, query execution, and API access. Graph data is stored in the configured RocksDB, HStore, or HBase backend.

  • Property graph and schema: Manages VertexLabel, EdgeLabel, PropertyKey, and IndexLabel definitions
  • Query languages: Supports Gremlin (TinkerPop 3) and Cypher (OpenCypher)
  • REST API: Provides endpoints for schemas, graph data, queries, tasks, and operations
  • Indexes and queries: Supports exact, range, and compound-condition queries
  • Storage backends: Versions 1.7.0 through master primarily support RocksDB (standalone), HStore (distributed), and HBase

The main modules include hugegraph-core, the storage backend modules, and hugegraph-api. Core implements the graph model, transactions, and query logic; backend modules connect to specific storage systems; and the API module provides HTTP access. Current REST resource paths include the graph space and graph name, for example:

/graphspaces/{graphspace}/graphs/{graph}

Standalone deployments commonly use RocksDB. Distributed deployments use HStore: PD manages cluster metadata and partition scheduling, while Store persists graph data and replicas. HBase can be used as a separate storage backend.

HugeGraph Toolchain

HugeGraph Toolchain provides clients, data import, visual management, Spark integration, and command-line operations. Together, these tools cover the main stages of a graph application’s lifecycle, from data ingestion to routine management.

ModulePurpose
ClientWraps schema management, graph data reads and writes, Gremlin, and Traverser APIs; supports Java, Python, and Go, with a Rust client under development
LoaderReads data from local files, HDFS, JDBC, Kafka, or another graph, converts it into vertices and edges, and imports it into HugeGraph in batches
HubbleProvides a web management interface for graph connections, schemas, data import, Gremlin queries, and visual results
Spark ConnectorReads and writes HugeGraph data in Spark jobs for offline big-data processing
SeaTunnel SinkImports data into HugeGraph through SeaTunnel 3.0+; see the Source guide for exports and migrations
ToolsProvides command-line operations for deployment, graph management, backup and restore, and Gremlin execution

Graph Computing Engines (OLAP)

The HugeGraph-Computer repository provides two complementary OLAP graph computing engines. Start with Vermeer for general graph algorithms; use Computer when you need the Java BSP/Pregel computing model:

  • Vermeer (default entry): Written in Go, it uses a master-worker architecture and primarily performs in-memory computation. It provides REST APIs, gRPC, and a web UI, and is suitable for fast small- and medium-scale graph analysis.
  • Computer: Written in Java, it implements the distributed BSP/Pregel computing model and can run on Kubernetes, YARN, or local processes. It can spill data to disk when memory thresholds are exceeded and is suitable for larger graph computing workloads.

Both engines can read HugeGraph data, but their runtime architectures, resource requirements, configuration, and algorithm interfaces differ.

HugeGraph-AI (Graph + AI)

HugeGraph-AI connects graph technology with large language models and graph machine learning frameworks. The repository uses Python 3.10 or later and manages its workspace with uv. Its main modules are:

  • hugegraph-llm: Provides GraphRAG, knowledge graph construction, natural-language queries, and Text2Gremlin
  • hugegraph-ml: Provides models for node classification, graph classification, graph embeddings, link prediction, and fraud detection
  • hugegraph-python-client: Manages schemas, graph data, and Gremlin queries from Python
  • vermeer-python-client: Calls Vermeer graph computing services from Python

HugeGraph-AI Quick Start

Deployment Modes

ModeCore ComponentsSuitable ScenariosData Scale
Standalone (OLTP)Server + RocksDBDevelopment, testing, Graph AI, and small to medium-scale production environments≤ 2 TB
Distributed (OLTP)Server + PD + Store (HStore)Production, horizontal scaling, and multi-replica deployment≤ 1 PB

Graph computing is an OLAP workload. Its capacity and resource requirements depend on the selected engine, graph structure, and algorithm, and do not use the OLTP storage capacity figures above.

Where to Start

GoalDocumentation
Start the graph database and run queriesServer Quick Start
Import data in batchesLoader, SeaTunnel
Manage graphs through a web interfaceHubble
Run graph algorithmsHugeGraph Computing
Build GraphRAG or graph machine learning applicationsHugeGraph-AI

Community

WeChat QR Code

2 - Download Apache HugeGraph

Instructions:

  • Use published packages. Java 11 is recommended for Server 1.7.0; see the Server startup guide for other supported JDKs. Check each component’s versioned quick start for Toolchain, AI, Vermeer, and other runtime requirements.
  • To verify downloads, use the corresponding hash (SHA512), signature, and Project Signature Verification KEYS.
  • Instructions for checking hash (SHA512) and signatures are on the Validate Release page, and you can also refer to ASF official instructions.
  • Note: The version numbers of all components of HugeGraph have been kept consistent, and the version numbers of Maven repositories such as client/loader/hubble/common are the same. You can refer to these for dependency references maven example.
  • Compatibility note: after HugeGraph graduated in January 2026, download paths moved from /incubator/hugegraph to /hugegraph. Historical release file names may still include -incubating-.
  • To build from source, refer to build from source.

The src suffix identifies a source archive that must be compiled according to the component guide; its directory cannot be used like a binary distribution to run startup scripts. Master may include unreleased configuration or APIs. When using release packages below or versioned container images, check the applicability notes in the documentation.

Latest Version

Source archives are the official Apache Software Foundation releases; binary packages are convenience builds made from them. All files are served from ASF mirrors, with signatures and checksums hosted on downloads.apache.org. Source archives generated automatically by GitHub are not ASF releases.


Archived Versions

Note: 1.3.0 is the last major version compatible with Java8, please switch to or migrate to Java11 as soon as possible (lower versions of Java have potentially more SEC risks and performance impacts). Starting from version 1.5.0, a Java11 runtime environment is required.

1.5.0

1.3.0

1.2.0

1.0.0

3 - Quick Start

Choose the quick-start guide for Server, Toolchain, graph computing, or HugeGraph-AI according to your needs. Each component is released independently, so check the runtime requirements and version of the corresponding repository before installation.

3.1 - HugeGraph (OLTP)

This section covers the HugeGraph Server standalone quick start and the PD and Store guides needed for distributed deployments.

See the DeepWiki project overview. For installation, configuration, and runtime behavior, use the documentation and code for the relevant HugeGraph version.

Start with the Server standalone quick start, which distinguishes ASF 1.7.0 release packages from builds of master. HStore deployments also require PD and Store.

GitHub Access: https://github.com/apache/hugegraph

3.1.1 - HugeGraph Server Quick Start

1 HugeGraph Server Overview

apache/hugegraph is the main repository for the HugeGraph graph database. Its top-level modules include hugegraph-server, hugegraph-pd, and hugegraph-store. This page describes the hugegraph-server module and the service it runs.

The hugegraph-server module contains hugegraph-core, hugegraph-api, hugegraph-dist, and storage adapters. Core implements the property graph model, transactions, and TinkerPop interfaces. API provides the HTTP service and delegates client requests to Core. Graph data is stored in RocksDB (the default standalone backend), HStore (distributed), or HBase.

⚠️ Version scope: Download examples use the ASF HugeGraph 1.7.0 release. Source builds, configuration defaults, and startup behavior follow apache/hugegraph master. A master build is not an ASF release artifact, even if its version number matches 1.7.0. This guide covers RocksDB, HStore, and HBase; see 1.5.x documentation for other legacy backends.

Naming: HugeGraph means the overall project or main repository, hugegraph-server is its Server module, and HugeGraphServer is the Java service class. Server below means the running graph database service.

flowchart TD
    A{Choose installation source}
    A -->|Release| R["HugeGraph 1.7.0 archive"]
    R --> V[Verify SHA-512]
    V --> X[Extract and use bundled configuration]
    A -->|Development| M["master source"]
    M --> B["mvn package -DskipTests"]
    B --> C[Check RocksDB configuration in conf/graphs]
    C --> I[bin/init-store.sh]
    I --> S["bin/start-hugegraph.sh -p true"]
    S --> Q["Request /versions and graph vertices"]

2 Dependency for Building/Running

2.1 Install Java 11 (JDK 11)

The hugegraph-server module in HugeGraph 1.7.0 is compiled with Java 11. Running and building it from source require Java 11 or later.

Before continuing, run java -version to confirm your JDK version.

Java 8 is no longer supported starting from 1.7.0. bin/hugegraph-server.sh refuses to start on anything older than Java 11.

The security check is on by default and installs HugeSecurityManager, which needs Java 11 to 23. JDK 24 removed the Security Manager (JEP 486), so on Java 24 or later you must start the service with the check disabled: bin/start-hugegraph.sh -s false.

Building from source also needs Maven 3.5.0 or later.

3 Deploy

There are four ways to deploy the Server service:

  1. Use a Docker container for test or development.
  2. Download the binary tarball.
  3. Compile the source code.
  4. Use the legacy one-click deployment tool.
Warning

Production requires authentication and restricted network access

HugeGraph disables user authentication by default. Enable authentication and authorization, maintain an IP allowlist, and grant minimum permissions. Do not expose Gremlin, Cypher, or other query APIs directly to the public Internet. Retain Server audit-*.log and restrict read access. auth.audit_log_rate limits per-user output rate rather than turning audit logging on or off. See the security guide.

3.1 Use Docker container (Convenient for Test/Dev)

You can refer to the Docker deployment guide.

You can use docker run -itd --name=server -p 8080:8080 -e PASSWORD=xxx hugegraph/hugegraph:1.7.0 to quickly start a Server instance using the RocksDB backend.

Optional:

  1. You can use docker exec -it server bash to enter the container for troubleshooting or other maintenance operations.
  2. You can use docker run -itd --name=server -p 8080:8080 -e PRELOAD="true" hugegraph/hugegraph:1.7.0 to preload a built-in sample graph at startup. You can verify it through the RESTful API. See 5.1.4 for details.
  3. You can use -e PASSWORD=xxx to enable authentication mode and set the admin password. See Config Authentication for details.

If you use Docker Desktop, you can set the options as follows:

Docker Desktop settings for a HugeGraph container

Note: The Docker Compose files use bridge networking (hg-net) and work on Linux and Mac (Docker Desktop). For the 3-node distributed cluster on Mac (Docker Desktop), allocate at least 12 GB of memory (Settings → Resources → Memory). On Linux, Docker uses host memory directly.

Use docker compose to manage multiple HugeGraph services from one configuration. Four Compose files are available in the docker/ directory:

TopologyCompose fileServices
Standalone (start here)docker-compose.yml1 RocksDB Server + 1 Hubble
Minimal HStore (current master)docker-compose-hstore.yml1 PD + 1 Store + 1 Server + 1 Hubble; locally built master images required
HA reference (current master)docker-compose-3pd-3store-3server.yml3 PD + 3 Store + 3 Server + 1 Hubble; locally built master images required
Source build override for the minimal HStore topologydocker-compose.dev.yml(used together with docker-compose-hstore.yml)
cd hugegraph/docker
# Keep the version aligned with the latest release, for example 1.x.0
HUGEGRAPH_VERSION=1.7.0 docker compose -f docker-compose.yml up -d --wait

The standalone topology publishes the Server on port 8080 and Hubble on 127.0.0.1:8088. HUGEGRAPH_VERSION selects the Server, PD, and Store image tags; Hubble is selected separately with HUBBLE_IMAGE.

The standalone docker-compose.yml example uses hugegraph/hugegraph:1.7.0; its docker/conf/hubble/standalone.properties is bundled in source. Compose reads the administrator password from HUGEGRAPH_ADMIN_PASSWORD and JWT secret from HUGEGRAPH_AUTH_TOKEN_SECRET, normally in docker/.env. A nonempty password enables authentication, automatically detected by Hubble. With docker run, use -e PASSWORD=xxx. Master HStore Compose files cannot be combined with 1.7.0 PD/Store/Server release images.

JWT version differences: Master-built Server maps HUGEGRAPH_AUTH_TOKEN_SECRET to HG_SERVER_AUTH_TOKEN_SECRET and writes startup configuration; preserve the secret across recreation. The 1.7.0 entrypoint handles older variables such as PASSWORD, ignoring HG_SERVER_AUTH_TOKEN_SECRET. To keep a stable JWT secret in 1.7.0, persist auth.token_secret in that version’s conf/graphs/hugegraph.properties. With a randomly generated default, do not assume issued JWTs survive Server restarts.

See docker/README.md for the full setup guide.

Note:

  1. HugeGraph Docker images are provided as a convenient way to start HugeGraph quickly, but they are not official ASF distribution artifacts. You can find more details in the ASF Release Distribution Policy.

  2. We recommend using a release tag (such as 1.7.0 or 1.x.0) for stable deployments. Use the latest tag only if you want the newest features still under development.

3.2 Download a Released Archive

This example uses HugeGraph 1.7.0. For other versions, confirm the release and filenames in the ASF download directory. A master source build is not a released binary.

download-release.sh
cd /path/to/downloads
wget https://downloads.apache.org/hugegraph/1.7.0/apache-hugegraph-incubating-1.7.0.tar.gz
wget https://downloads.apache.org/hugegraph/1.7.0/apache-hugegraph-incubating-1.7.0.tar.gz.sha512
sha512sum -c apache-hugegraph-incubating-1.7.0.tar.gz.sha512
tar zxf apache-hugegraph-incubating-1.7.0.tar.gz

Extract only after SHA-512 verification succeeds. You can also verify the PGP signature following Apache release procedures:

wget https://downloads.apache.org/hugegraph/1.7.0/apache-hugegraph-incubating-1.7.0.tar.gz.asc
wget https://downloads.apache.org/hugegraph/KEYS
gpg --import KEYS
gpg --verify apache-hugegraph-incubating-1.7.0.tar.gz.asc apache-hugegraph-incubating-1.7.0.tar.gz

Trust publisher keys only after verifying them through Apache project channels. Signature verification can complement SHA-512 checks.

3.3 Build from master Source

These commands build the development branch, not the 1.7.0 release.

Install wget or curl before building.

Download HugeGraph source:

build-from-source.sh
git clone --branch master https://github.com/apache/hugegraph.git

Compile and package:

cd hugegraph
# (Optional) use "-P stage" param if you build failed with the latest code(during pre-release period)
mvn package -DskipTests -ntp

A successful build logs:

[INFO] BUILD SUCCESS

The build creates an aggregate master archive under repository-root target/ (the current source version property is 1.7.0):

target/apache-hugegraph-1.7.0.tar.gz

The filename version comes from source revision; it does not make the build an ASF release artifact. The build also creates repository-root apache-hugegraph-1.7.0/, containing a Server directory named by that module’s final.name, currently apache-hugegraph-server-1.7.0. If retaining only the archive, first run tar zxf target/apache-hugegraph-1.7.0.tar.gz, then enter the Server directory.

Default builds bundle rocksdb, hbase, and hstore modules, listed in backends within backend.properties in the hugegraph-dist JAR. For a RocksDB-only distribution, add -Drocksdb-only:

mvn package -DskipTests -ntp -Drocksdb-only
Outdated tools

3.4 One-click deployment (Outdated)

HugeGraph-Tools provides a one-click deployment command that downloads, extracts, configures, and starts the Server service and HugeGraph-Hubble. These tools are included in the HugeGraph-Toolchain distribution.

Of course, you should download the tarball of HugeGraph-Toolchain first.

# download toolchain binary package, it includes loader + tool + hubble
# please check the latest version (e.g. here is 1.7.0)
wget https://downloads.apache.org/hugegraph/1.7.0/apache-hugegraph-toolchain-incubating-1.7.0.tar.gz
tar zxf *hugegraph-*.tar.gz

# enter the tool's package
cd *hugegraph*/*tool* 

note: ${version} is the version, The latest version can refer to Download Page, or click the link to download directly from the Download page

The general entry script for HugeGraph-Tools is bin/hugegraph, Users can use the help command to view its usage, here only the commands for one-click deployment are introduced.

bin/hugegraph deploy -v {hugegraph-version} -p {install-path} [-u {download-path-prefix}]

{hugegraph-version} is the Server service and HugeGraphStudio version; see conf/version-mapping.yaml for supported mappings. {install-path} is the installation directory, while {download-path-prefix} optionally overrides the tarball download location. For example, deploy version 0.6 with bin/hugegraph deploy -v 0.6 -p services.

4 Configuration

Server archives include conf/rest-server.properties, conf/gremlin-server.yaml, and conf/graphs/hugegraph.properties; source builds assemble them from hugegraph-server/hugegraph-dist/src/assembly/static/conf/. Edit them inside the extracted Server directory without a separate generation step. Defaults vary by release; the following defaults and startup behavior describe master.

For standalone RocksDB, confirm backend=rocksdb and serializer=binary in conf/graphs/hugegraph.properties. Normal Server initialization scans conf/graphs/ and loads local graphs without requiring graph.load_from_local_config=true. This option defaults to false and controls constructor preloading and rescanning on reload().

See the configuration guide and option reference for details.

5 Startup

5.1 Use a startup script to startup

Startup is divided into “first startup” and “non-first startup”. On the first startup, you need to initialize the backend database before starting the service.

If the service was stopped manually, or needs to be started again for any other reason, you can usually start it directly because the backend database is persistent.

When HugeGraphServer starts, it connects to the backend storage and checks its version information. If the backend has not been initialized, or if it was initialized with an incompatible version (for example, old-version data), HugeGraphServer will fail to start and report an error.

If you need to access HugeGraphServer externally, modify the restserver.url configuration item in rest-server.properties (the default is http://127.0.0.1:8080) and change it to the machine name or IP address.

Since the configuration (hugegraph.properties) and startup steps required by various backends are slightly different, the following will introduce the configuration and startup of each backend one by one.

Warning

Configure authentication before production startup

Enable Server authentication and authorization with a strong, nondefault administrator password, an IP allowlist, and minimum business-user permissions.

5.1.1 Distributed Storage (HStore)

Expand/collapse distributed storage configuration and startup

Distributed storage, introduced after HugeGraph 1.5.0, uses HugeGraph-PD and HugeGraph-Store for distributed data storage and computing.

First deploy PD and Store; see the PD Quick Start and Store Quick Start.

After PD and Store are running:

  1. Edit Server hugegraph.properties:
backend=hstore
serializer=binary

# PD RPC addresses, separated by commas
pd.peers=127.0.0.1:8686,127.0.0.1:8687,127.0.0.1:8688
pd.cluster=hg
# Simple example with authentication
gremlin.graph=org.apache.hugegraph.auth.HugeFactoryAuthProxy

# Required HStore backend
backend=hstore
serializer=binary
store=hugegraph

# pd config
pd.peers=127.0.0.1:8686
pd.cluster=hg

The distribution includes conf/graphs/hstore.properties.template; copy it over conf/graphs/hugegraph.properties and adjust pd.peers.

This example uses cluster=hg in rest-server.properties and pd.cluster=hg in graph configuration, matching the master Compose cluster below. These are separate keys for the Server process and graph respectively, not one setting. Select an appropriate cluster name. Likewise, pd.peers in each file has its own configuration scope; this example supplies the same PD RPC addresses to both.

The backend selects the task scheduler: HStore uses distributed scheduling; others use local scheduling. task.scheduler_type is unnecessary and is ignored with a warning when retained for compatibility.

  1. Edit Server rest-server.properties:
usePD=true
# Use cluster hg, matching pd.cluster in graph configuration
cluster=hg
# Server process PD addresses; replace with actual PD RPC addresses
pd.peers=127.0.0.1:8686,127.0.0.1:8687,127.0.0.1:8688

# Optional in local tests; authentication is mandatory in production
# auth.authenticator=org.apache.hugegraph.auth.StandardAuthenticator

These addresses support multiple Server processes on one machine; 127.0.0.1 is local only. Across hosts, use routable IPs or DNS for restserver.url, gremlinserver.url, pd.peers, Gremlin host, and rpc.server_host. Advertised RPC addresses and ports must be mutually reachable.

For multiple Servers, edit each node’s rest-server.properties:

Node 1 (master):

usePD=true
restserver.url=http://127.0.0.1:8081
gremlinserver.url=http://127.0.0.1:8181
pd.peers=127.0.0.1:8686

rpc.server_host=127.0.0.1
rpc.server_port=8091

server.id=server-1
server.role=master

Node 2 (worker):

usePD=true
restserver.url=http://127.0.0.1:8082
gremlinserver.url=http://127.0.0.1:8182
pd.peers=127.0.0.1:8686

rpc.server_host=127.0.0.1
rpc.server_port=8092

server.id=server-2
server.role=worker

Also configure each node’s gremlin-server.yaml ports:

Node 1:

host: 127.0.0.1
port: 8181

Node 2:

host: 127.0.0.1
port: 8182

Start Server:

bin/start-hugegraph.sh

Distributed startup order:

  1. Start HugeGraph-PD
  2. Start HugeGraph-Store
  3. Start Server

PD and Store manage HStore metadata and storage, so init-store skips that backend. With authentication enabled, init-store still creates the built-in admin account. If storage already holds this account, set init_store.enabled=false in rest-server.properties to skip the entire step, as Docker HStore topologies do.

Verify startup:

curl http://localhost:8081/graphspaces/DEFAULT/graphs
# Expected: {"graphs":["hugegraph"]}

Stop in reverse order:

  1. Stop Server
  2. Stop HugeGraph-Store
  3. Stop HugeGraph-PD
bin/stop-hugegraph.sh
Docker HStore HA Cluster (Current Master)

docker-compose-3pd-3store-3server.yml uses master HG_PD_*, HG_STORE_*, and HG_SERVER_* variables unsupported by 1.7.0 release images. Build PD, Store, and Server from the same master source, all tagged local:

# Run from the HugeGraph repository root
docker build -f hugegraph-pd/Dockerfile -t hugegraph/pd:local .
docker build -f hugegraph-store/Dockerfile -t hugegraph/store:local .
docker build -f hugegraph-server/Dockerfile-hstore -t hugegraph/server:local .

cd docker

Create the authentication environment in docker/ and generate HStore Hubble configuration. Replace the example administrator password; because .env uses single quotes, the password must contain neither single quotes nor newlines. The script generates random JWT and PD secrets and writes the PD secret to two untracked .local.properties files mounted by HStore Compose:

Warning

umask 077 protects the new .env, but set-hubble-pd-password.sh sets generated .local.properties permissions to 0644, making the PD secret readable by other local users. This example assumes trusted local users. On shared hosts, set ownership and read permissions for the Hubble runtime user so Hubble can read the files while unauthorized users cannot. Keeping files untracked does not replace file permission controls.

(
  set -eu
  command -v openssl >/dev/null
  jwt_secret="$(openssl rand -hex 32)"
  pd_secret="$(openssl rand -hex 24)"
  umask 077
  test ! -e .env || {
    echo ".env already exists; edit it instead of overwriting it" >&2
    exit 1
  }
  printf "HUGEGRAPH_ADMIN_PASSWORD='%s'\nHUGEGRAPH_AUTH_TOKEN_SECRET='%s'\nHG_PD_AUTH_SECRET_KEY='%s'\n" \
    'replace-with-your-password' "${jwt_secret}" "${pd_secret}" > .env
  HG_PD_AUTH_SECRET_KEY="${pd_secret}" ./set-hubble-pd-password.sh hstore
  HG_PD_AUTH_SECRET_KEY="${pd_secret}" ./set-hubble-pd-password.sh hstore-ha
)

Start HA from docker/. HUGEGRAPH_VERSION=local selects the locally built PD, Store, and Server images; Compose pull_policy: missing uses existing local tags first. HUBBLE_IMAGE still selects Hubble independently:

set -a
. ./.env
set +a
HUGEGRAPH_VERSION=local docker compose \
  -f docker-compose-3pd-3store-3server.yml \
  up -d --wait pd0 pd1 pd2 store0 store1 store2 server0 server1 server2 hubble

Generate HG_PD_AUTH_SECRET_KEY once for new data directories and reuse it on restart or when retaining data. Do not commit .env or generated conf/hubble/*.local.properties. Services communicate through container hostnames on the hg-net bridge. Server replicas need the shared JWT secret as well as the administrator password; set HUGEGRAPH_AUTH_TOKEN_SECRET. See the Docker cluster guide and docker/README.md for all variables.

Verify the cluster:

curl -fsS http://localhost:8080/versions
curl -fsS -u "hg:${HG_PD_AUTH_SECRET_KEY:?Load .env first}" \
  http://localhost:8620/v1/stores

Master PD /v1/stores requires Basic authentication; this uses the generated, loaded HG_PD_AUTH_SECRET_KEY as the hg password.

Inspect runtime logs with docker logs <container-name>, such as docker logs hg-pd0.

See docker/README.md for variables, ports, and troubleshooting.

5.1.2 RocksDB / ToplingDB

The minimal standalone flow below uses a master build and preloads sample data to verify graph reads and writes.

Expand/collapse RocksDB configuration and startup

RocksDB is embedded and needs no separate deployment. GCC ≥ 4.3.0 (GLIBCXX_3.4.10) is required; upgrade first if needed.

Master conf/graphs/hugegraph.properties already sets backend=rocksdb and serializer=binary. Keep the defaults if using default data directories; otherwise check your backend and paths before initialization.

Initialize storage on first startup or after adding graph configuration under conf/graphs/:

cd apache-hugegraph-1.7.0/apache-hugegraph-server-1.7.0
bin/init-store.sh

Start Server and preload the built-in sample graph:

bin/start-hugegraph.sh -p true

The startup script polls /graphs at configured restserver.url and exits nonzero on timeout or process failure. Check service version, then read sample vertices:

curl -fsS http://127.0.0.1:8080/versions
curl --compressed -fsS \
  http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices

The first response should contain versions; the second should contain vertices with sample names such as marko and lop. A process check or HTTP status alone does not prove backend and business request health.

ToplingDB (Beta): A high-performance RocksDB alternative; see the ToplingDB Quick Start.

5.1.3 HBase

Click to expand/collapse HBase configuration and startup methods

users need to install HBase by themselves, requiring version 2.0 or above,download link

Update hugegraph.properties

backend=hbase
serializer=hbase

# hbase backend config
hbase.hosts=localhost
hbase.port=2181
# Note: recommend to modify the HBase partition number by the actual/env data amount & RS amount before init store
# it may influence the loading speed a lot
#hbase.enable_partition=true
#hbase.vertex_partitions=10
#hbase.edge_partitions=30

Initialize the database (required on the first startup, or a new configuration was manually added under ‘conf/graphs/’)

cd *hugegraph-${version}
bin/init-store.sh

Start server

bin/start-hugegraph.sh
Starting HugeGraphServer in daemon mode...
Connecting to HugeGraphServer (http://127.0.0.1:8080/graphs)....OK
Started [pid 21614]

5.1.4 Create an example graph when startup

Pass the -p true argument when starting the script to enable preload, which creates a sample graph.

bin/start-hugegraph.sh -p true
Starting HugeGraphServer in daemon mode...
Connecting to HugeGraphServer (http://127.0.0.1:8080/graphs)......OK

And use the RESTful API to request HugeGraphServer and get the following result:

> curl "http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices" | gunzip

{"vertices":[{"id":"2:lop","label":"software","type":"vertex","properties":{"name":"lop","lang":"java","price":328}},{"id":"1:josh","label":"person","type":"vertex","properties":{"name":"josh","age":32,"city":"Beijing"}},{"id":"1:marko","label":"person","type":"vertex","properties":{"name":"marko","age":29,"city":"Beijing"}},{"id":"1:peter","label":"person","type":"vertex","properties":{"name":"peter","age":35,"city":"Shanghai"}},{"id":"1:vadas","label":"person","type":"vertex","properties":{"name":"vadas","age":27,"city":"Hongkong"}},{"id":"2:ripple","label":"software","type":"vertex","properties":{"name":"ripple","lang":"java","price":199}}]}

This indicates the successful creation of the sample graph.

5.1.5 Startup script options

bin/start-hugegraph.sh accepts the following options. Every one of them takes a value, so write -d false, not a bare -d.

OptionValuesDefaultPurpose
-dtrue, falsetrueDaemon mode. With -d false the script stays in the foreground and forwards SIGTERM/SIGINT to the server.
-gzgc or ZGComit for G1GCGarbage collector to use. Only ZGC is accepted, any other value aborts the startup. ZGC needs Java 11 or later.
-mtrue, falsefalseInstall the cron-based monitor task (bin/start-monitor.sh). For VM and bare-metal deployments only.
-ptrue, falsefalsePreload the sample graph, as in 5.1.4.
-strue, falsetrueRun with the security check (HugeSecurityManager) enabled. It requires Java 11 to 23 and a readable conf/java-security.properties.
-jJVM optionsemptyExtra JVM options appended to the server command line.
-tseconds30How long to wait for the service to answer before reporting a failed startup.
-ytrue, falsefalseEnable the OpenTelemetry agent for traces.

bin/stop-hugegraph.sh accepts -m true|false (default true), which controls whether the cron monitor task is removed along with the service.

5.2 Use Docker to startup

In 3.1 Use Docker container, we introduced how to deploy hugegraph-server with Docker. You can also switch storage backends or preload a sample graph by setting the corresponding parameters.

5.2.1 Create an example graph when starting a server

Set the environment variable PRELOAD=true when starting Docker so that sample data is loaded during startup.

  1. Use docker run

    Use docker run -itd --name=server -p 8080:8080 -e PRELOAD=true hugegraph/hugegraph:1.7.0

  2. Use docker-compose

    Create a docker-compose.yml file like the following and set PRELOAD=true in the environment. example.groovy is a predefined script used to preload sample data. If needed, you can mount a new example.groovy script to change the preload data.

    version: '3'
    services:
      server:
        image: hugegraph/hugegraph:1.7.0
        container_name: server
        environment:
          - PRELOAD=true
          - PASSWORD=xxx
        volumes:
          - /path/to/yourscript:/hugegraph-server/scripts/example.groovy
        ports:
          - 8080:8080

    Use docker compose up -d to start the container.

And use the RESTful API to request HugeGraphServer and get the following result:

> curl "http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices" | gunzip

{"vertices":[{"id":"2:lop","label":"software","type":"vertex","properties":{"name":"lop","lang":"java","price":328}},{"id":"1:josh","label":"person","type":"vertex","properties":{"name":"josh","age":32,"city":"Beijing"}},{"id":"1:marko","label":"person","type":"vertex","properties":{"name":"marko","age":29,"city":"Beijing"}},{"id":"1:peter","label":"person","type":"vertex","properties":{"name":"peter","age":35,"city":"Shanghai"}},{"id":"1:vadas","label":"person","type":"vertex","properties":{"name":"vadas","age":27,"city":"Hongkong"}},{"id":"2:ripple","label":"software","type":"vertex","properties":{"name":"ripple","lang":"java","price":199}}]}

This indicates that the sample graph was created successfully.

6. Access server

6.1 Service startup status check

Use jps to see a service process

jps
6475 HugeGraphServer

curl request RESTfulAPI

curl -fsS http://127.0.0.1:8080/versions

HTTP 2xx with a JSON versions field confirms REST responsiveness. HugeGraphServer in jps confirms only process existence; verify backend health by preloading the sample graph and requesting vertices as above.

6.2 Request Server

The RESTful API of HugeGraphServer includes various types of resources, typically including graph, schema, gremlin, traverser and task.

  • graph contains vertices、edges
  • schema contains vertexlabels、 propertykeys、 edgelabels、indexlabels
  • gremlin contains various Gremlin statements, such as g.v(), which can be executed synchronously or asynchronously
  • traverser contains various advanced queries including shortest paths, intersections, N-step reachable neighbors, etc.
  • task contains query and delete with asynchronous tasks
curl http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices

explanation

  1. Server can compress vertex, edge, and other list responses. Use curl --compressed for automatic decompression:

    curl --compressed -fsS "http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices"
  2. The default listener is 127.0.0.1, inaccessible directly from other machines.

    vim conf/rest-server.properties
    
    restserver.url=http://0.0.0.0:8080
Warning

Changing the Server listener

Setting 0.0.0.0 listens on every interface. In production, enable authentication and authorization, an IP allowlist, and minimum permissions; retain and protect audit logs. Do not expose Gremlin or Cypher directly to the public Internet.

response body:

{
    "vertices": [
        {
            "id": "2lop",
            "label": "software",
            "type": "vertex",
            "properties": {
                "price": [
                    {
                        "id": "price",
                        "value": 328
                    }
                ],
                "name": [
                    {
                        "id": "name",
                        "value": "lop"
                    }
                ],
                "lang": [
                    {
                        "id": "lang",
                        "value": "java"
                    }
                ]
            }
        },
        {
            "id": "1josh",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": [
                    {
                        "id": "name",
                        "value": "josh"
                    }
                ],
                "age": [
                    {
                        "id": "age",
                        "value": 32
                    }
                ]
            }
        },
        ...
    ]
}

For the detailed API, please refer to RESTful-API

You can also visit localhost:8080/swagger-ui/index.html to check the API.

HugeGraph RESTful API endpoints in Swagger UI

When using Swagger UI to debug the API provided by HugeGraph, if HugeGraph Server turns on authentication mode, you can enter authentication information on the Swagger page.

Authorize button in the HugeGraph Swagger UI

Currently, HugeGraph supports setting authentication information in two forms: Basic and Bearer.

Basic and Bearer credential fields in the Swagger UI authorization dialog

7 Stop Server

cd apache-hugegraph-1.7.0/apache-hugegraph-server-1.7.0
bin/stop-hugegraph.sh

8 Debug Server with IntelliJ IDEA

Please refer to Setup Server in IDEA

3.1.2 - HugeGraph-PD Quick Start

1 HugeGraph-PD Overview

HugeGraph-PD (Placement Driver) is the metadata management component of HugeGraph’s distributed version, responsible for managing the distribution of graph data and coordinating storage nodes. It plays a central role in distributed HugeGraph, maintaining cluster status and coordinating HugeGraph-Store storage nodes.

PD keeps cluster metadata in an embedded RocksDB store under pd.data-path and replicates it across PD nodes with Raft, so a 3-node or 5-node PD cluster keeps serving while a minority of nodes is down. On top of that it registers and activates Store nodes, allocates and rebalances partitions, tracks Store heartbeats, and answers service discovery queries from Store and Server.

PD listens on three ports:

PortDefaultConfigured byUsed by
gRPC8686grpc.portStore and Server clients
REST8620server.portManagement, health checks, metrics
Raft8610raft.addressThe other PD nodes only

2 Prerequisites

2.1 Requirements

  • Operating System: Linux or macOS (Windows has not been fully tested)
  • Java version: ≥ 11
  • Maven version: ≥ 3.5.0

3 Deployment

There are two ways to deploy the HugeGraph-PD component:

  • Method 1: Download the tar package
  • Method 2: Compile from source

3.1 Download the tar package

Apache downloads currently provide the complete 1.7.0 binary bundle containing PD, Store, and Server, without a separate PD binary. Verify version, signatures, and SHA512 on the official download page. This historical incubating release retains incubating in archive and extracted directory names:

# Historical release 1.7.0 retains incubating in filenames and directories
wget https://downloads.apache.org/hugegraph/1.7.0/apache-hugegraph-incubating-1.7.0.tar.gz
tar zxf apache-hugegraph-incubating-1.7.0.tar.gz
cd apache-hugegraph-incubating-1.7.0/apache-hugegraph-pd-incubating-1.7.0

3.2 Compile from source

# 1. Clone the source code
git clone https://github.com/apache/hugegraph.git

# 2. Build the project
cd hugegraph
mvn clean install -DskipTests=true

# 3. After a successful build, the PD directory and packages are located at
#    hugegraph-pd/apache-hugegraph-pd-{version}          (unpacked PD distribution)
#    hugegraph-pd/apache-hugegraph-pd-{version}.tar.gz   (PD only package, Linux build hosts only)
#    target/apache-hugegraph-{version}.tar.gz            (PD + Store + Server package)

To build only the PD distribution and the modules it depends on:

mvn clean package -pl hugegraph-pd/hg-pd-dist -am -DskipTests

The unpacked distribution contains just three directories: bin (start and stop scripts), conf (application.yml, application.yml.template, log4j2.xml, verify-license.json) and lib (the hg-pd-service jar).

3.3 Docker Deployment

HG_PD_* mappings and /v1/ready are features of current master; the Server 1.7.0 source tag lacks these Docker mappings and readiness API. The following examples require images built from current master and tagged local. For release 1.7.0 assets, use bundled configuration and version-specific instructions:

# Run from the HugeGraph repository root
docker build -f hugegraph-pd/Dockerfile -t hugegraph/pd:local .

To run master-built PD alone, first set a deployment-specific secret and replace example addresses with addresses reachable by the container:

export HG_PD_AUTH_SECRET_KEY="$(openssl rand -hex 24)"
docker run -d \
  -p 8620:8620 -p 8686:8686 -p 8610:8610 \
  -e HG_PD_GRPC_HOST=192.168.1.10 \
  -e HG_PD_RAFT_ADDRESS=192.168.1.10:8610 \
  -e HG_PD_RAFT_PEERS_LIST=192.168.1.10:8610 \
  -e HG_PD_INITIAL_STORE_LIST=192.168.1.20:8500 \
  -e HG_PD_AUTH_SECRET_KEY="$HG_PD_AUTH_SECRET_KEY" \
  -v /path/to/data:/hugegraph-pd/pd_data \
  --name hugegraph-pd \
  hugegraph/pd:local
VariableRequiredDefaultConfiguration keyDescription
HG_PD_GRPC_HOSTYesNonegrpc.hostAdvertised gRPC hostname/IP; use container hostnames within a container network.
HG_PD_RAFT_ADDRESSYesNoneraft.addressThis PD node’s Raft address.
HG_PD_RAFT_PEERS_LISTYesNoneraft.peers-listRaft addresses of every PD node, including this node.
HG_PD_INITIAL_STORE_LISTYesNonepd.initial-store-listExpected Store gRPC addresses.
HG_PD_AUTH_SECRET_KEYYesNoneauth.secret-keyREST Basic password shared by all PD REST clients.
HG_PD_GRPC_PORTNo8686grpc.portgRPC service port.
HG_PD_REST_PORTNo8620server.portREST API port.
HG_PD_DATA_PATHNo/hugegraph-pd/pd_datapd.data-pathMetadata path.
HG_PD_INITIAL_STORE_COUNTNo1pd.initial-store-countMinimum Store count needed for cluster availability.
HG_PD_ACTUATOR_EXPOSURENohealth,metrics,prometheusmanagement.endpoints.web.exposure.includePublic Actuator endpoint allowlist; * is forbidden.

The master entrypoint requires all five mandatory variables, builds SPRING_APPLICATION_JSON overrides, then starts PD. Other settings come from image conf/application.yml; JAVA_OPTS is passed to the JVM. Deprecated GRPC_HOST, RAFT_ADDRESS, RAFT_PEERS, and PD_INITIAL_STORE_LIST remain supported with warnings.

Docker HEALTHCHECK polls /v1/health every 15 seconds, confirming only REST listener liveness, not a Raft quorum. Compose uses the same liveness check; verify /v1/ready separately before starting Store. Containers run Java in the foreground and exit when Java exits; automatic restart requires a Docker restart policy. Inspect logs with docker logs <container-name>.

See the Store Docker section and docker/README.md for master Compose and the minimal topology. When starting the full topology with Hubble, first generate the untracked Hubble configuration described in that README.

4 Configuration

PD startup reads installation conf/application.yml. Match this file to the installed version: 1.7.0 configuration or master configuration. Master leaves auth.secret-key empty, causing protected REST calls to be rejected until configured. Startup does not read application.yml.template.

4.1 Configuration Reference

These tables were checked against hg-pd-dist configuration and Java defaults at Server master commit 2f827d6e8c9c62ae858f2fc122b3a192d015e2f4; they do not describe every 1.7.0 behavior. Use original tagged configuration with release binaries and master configuration with master builds.

gRPC and REST

KeyMaster distribution valueBuilt-in defaultDescription
grpc.host127.0.0.1none, requiredAddress this PD advertises for gRPC. Store and Server connect here, so set it to a reachable IPv4 address or hostname, never 127.0.0.1 or 0.0.0.0, in a distributed deployment.
grpc.port8686none, requiredgRPC port.
server.port8620none, requiredREST API port. Also the port reported in Raft member information.

application.yml.template also carries grpc.netty-server.max-inbound-message-size: 100MB, but PD sets the gRPC server’s inbound message limit to 1 GB in code, so that key has no effect.

Raft

KeyMaster distribution valueBuilt-in defaultDescription
raft.address127.0.0.1:8610none, requiredRaft address of this node as host:port. Must be unique per node and must appear in raft.peers-list.
raft.peers-list127.0.0.1:8610none, requiredComma separated Raft addresses of every PD node, including this one. Must be identical on all nodes.
raft.enablenot settrueWhen true, metadata writes go through the Raft state machine. When false, PD writes straight to its local store with no replication.
raft.ip-whitelist.enablednot settrueWhen true, the Raft RPC port accepts connections only from the addresses resolved from raft.peers-list; other clients are dropped and logged as Blocked connection from <ip>. The allowlist is re-resolved when the peer list changes, but a peer that keeps its hostname and changes IP (a restarted container, for example) needs a PD restart.
raft.snapshotIntervalnot set300Seconds between Raft snapshots.
raft.rpc-timeoutnot set10000Raft RPC connect, request and install-snapshot timeout, in milliseconds.

PD core

KeyMaster distribution valueBuilt-in defaultDescription
pd.data-path./pd_datanone, requiredMetadata directory. Holds the RocksDB store in rocksdb/ and the Raft log, metadata and snapshots in pd_raft/.
pd.patrol-interval1800300Seconds between patrol runs, which check partition health across stores and rebalance partition counts.
pd.initial-store-count13Minimum number of active Store nodes. Below this the cluster state becomes Cluster_Not_Ready and the cluster is treated as unavailable. Set it to the number of stores you deploy.
pd.initial-store-list127.0.0.1:8500emptyComma separated Store gRPC addresses (ip:port) that are activated automatically when they register. An entry may also carry a group id as store_address/group_id.
pd.cluster_idnot set1Cluster id, used to keep separate PD clusters apart.

Store management

KeyMaster distribution valueBuilt-in defaultDescription
store.keepAlive-timeoutnot set300Seconds without a heartbeat after which a Store is treated as temporarily unavailable and its partition leaders move to other replicas.
store.max-down-time1728001800Seconds after which a Store is treated as permanently unavailable and its replicas are reallocated to other machines.
store.monitor_data_enabledtruefalseWhether to persist Store monitoring samples.
store.monitor_data_interval1 minute1 minuteSampling interval, written as <number> <unit> with unit one of second, minute, hour, day, month, year. The number defaults to 1 when omitted.
store.monitor_data_retention1 day1 dayHow long monitoring samples are kept, same format as above.

Partitions

KeyMaster distribution valueBuilt-in defaultDescription
partition.default-shard-count13Number of replicas per partition. Use 3 for a production cluster.
partition.store-max-shard-count1224Maximum number of partition replicas one Store holds.

The initial partition count is derived from these two values and the size of pd.initial-store-list:

initial partitions = store count * partition.store-max-shard-count / partition.default-shard-count

Discovery, license and metrics

KeyMaster distribution valueBuilt-in defaultDescription
discovery.heartbeat-try-countnot set3Number of missed heartbeats after which a registered client’s discovery entry is deleted.
license.verify-path./conf/verify-license.jsonNone; required configuration keyThe PD distribution includes this JSON file; current master has no runtime read of this configuration option.
license.license-path./conf/hugegraph.licenseNone; required configuration keyNo license file is bundled at this path. Internal gRPC putLicense writes uploads here. Master REST GET /v1/license returns an empty object; a missing file does not block startup.
auth.secret-keyEmptyEmptyMaster PD REST Basic password for usernames hg, store, hubble, or vermeer. Docker requires it; bare-metal PD can start with an empty secret, but rejects protected REST calls.
management.metrics.export.prometheus.enabledtrueSpring Boot defaultExposes /actuator/prometheus.
management.endpoints.web.exposure.includehealth,metrics,prometheusSpring Boot exposes only health by defaultActuator allowlist; these endpoints bypass the PD REST Basic interceptor.
logging.configfile:./conf/log4j2.xmlnoneLog4j2 configuration. Writes logs/hugegraph-pd.log, logs/hugegraph-pd_raft.log and logs/audit-hugegraph-pd.log.

Thread pools

KeyBuilt-in defaultDescription
thread.pool.grpc.core600Core size of the pool that serves gRPC calls.
thread.pool.grpc.max1000Maximum size of that pool.
thread.pool.grpc.queueunboundedQueue capacity of that pool.
job.uninterruptibleThreadPool.core0Core size of the background metadata job pool. A value of 0 or less means half the available processors.
job.uninterruptibleThreadPool.max256Maximum size of that pool.
job.uninterruptibleThreadPool.queueunboundedQueue capacity of that pool.

4.2 Single-node configuration

For master-built distributions, start with conf/application.yml, adjust per-node gRPC/Raft addresses and data paths, and set auth.secret-key. The following values are for development and testing. One PD node provides no majority fault tolerance; partition.default-shard-count: 1 means one replica per partition.

grpc:
  host: 127.0.0.1
  port: 8686
server:
  port: 8620
raft:
  address: 127.0.0.1:8610
  peers-list: 127.0.0.1:8610
pd:
  data-path: ./pd_data
  initial-store-count: 1
  initial-store-list: 127.0.0.1:8500
partition:
  default-shard-count: 1
auth:
  secret-key: replace-with-deployment-secret

Generate a deployment secret with openssl rand -hex 24; Docker supplies it through HG_PD_AUTH_SECRET_KEY. See REST API authentication for the different 1.7.0 behavior.

4.3 Three-node cluster configuration

For a production cluster run 3 or 5 PD nodes, an odd number so Raft always has a quorum. A 3-node cluster tolerates one node failure. raft.peers-list must list every node and must be byte-for-byte identical on all of them, while grpc.host and raft.address differ per node.

Node 1 (192.168.1.10):

grpc:
  host: 192.168.1.10
  port: 8686
server:
  port: 8620
raft:
  address: 192.168.1.10:8610
  peers-list: 192.168.1.10:8610,192.168.1.11:8610,192.168.1.12:8610
pd:
  data-path: /data/pd
  initial-store-count: 3
  initial-store-list: 192.168.1.20:8500,192.168.1.21:8500,192.168.1.22:8500
partition:
  default-shard-count: 3

Node 2 (192.168.1.11) and node 3 (192.168.1.12) use the same file with grpc.host and raft.address changed to their own address:

# Node 2
grpc:
  host: 192.168.1.11
raft:
  address: 192.168.1.11:8610
  peers-list: 192.168.1.10:8610,192.168.1.11:8610,192.168.1.12:8610

# Node 3
grpc:
  host: 192.168.1.12
raft:
  address: 192.168.1.12:8610
  peers-list: 192.168.1.10:8610,192.168.1.11:8610,192.168.1.12:8610

To put all three PD nodes on one machine for testing, give each node its own pd.data-path and its own ports, for example raft 8610/8611/8612, gRPC 8686/8687/8688 and REST 8620/8621/8622.

In Docker bridge networking the same configuration comes from environment variables and uses container hostnames instead of IP addresses:

# Share the same secret across PD nodes; inject it from private environment variables
HG_PD_AUTH_SECRET_KEY: ${HG_PD_AUTH_SECRET_KEY:?set HG_PD_AUTH_SECRET_KEY}

# pd0
HG_PD_GRPC_HOST: pd0
HG_PD_RAFT_ADDRESS: pd0:8610
HG_PD_RAFT_PEERS_LIST: pd0:8610,pd1:8610,pd2:8610
HG_PD_INITIAL_STORE_LIST: store0:8500,store1:8500,store2:8500
HG_PD_INITIAL_STORE_COUNT: 3

# pd1
HG_PD_GRPC_HOST: pd1
HG_PD_RAFT_ADDRESS: pd1:8610
HG_PD_RAFT_PEERS_LIST: pd0:8610,pd1:8610,pd2:8610

# pd2
HG_PD_GRPC_HOST: pd2
HG_PD_RAFT_ADDRESS: pd2:8610
HG_PD_RAFT_PEERS_LIST: pd0:8610,pd1:8610,pd2:8610

5 Start and Stop

5.1 Start PD

In the PD installation directory, execute:

./bin/start-hugegraph-pd.sh

The script requires a JDK of at least version 11 on PATH or in JAVA_HOME, and it exits without doing anything if it finds a Java process already using this installation’s conf directory.

Supported flags:

FlagValuesDefaultDescription
-dtrue, falsetrueDaemon mode. See the note below.
-gzgc, ZGCnot setGarbage collector. Leave the flag off for the default G1GC. Any other value, g1 included, aborts the start.
-jJVM optionsemptyExtra JVM options, for example -j "-Xmx8g -Xms8g".
-ytrue, falsefalseAttach the OpenTelemetry Java agent. The agent is downloaded into plugins/ on first use, its MD5 is verified, and traces are exported over gRPC to http://127.0.0.1:4317.

The -d flag controls daemon mode:

  • -d true (default): run as a background daemon; the script returns immediately.
  • -d false: run in foreground. The script execs Java, so the container or supervisor process IS Java. Use this when running under Docker or a process supervisor (systemd, supervisord) so crashes are detected and the service is restarted automatically.

Each flag also has an environment variable equivalent: DAEMON, GC_OPTION, USER_OPTION and OPEN_TELEMETRY. Setting JAVA_OPTIONS replaces the computed heap settings entirely; otherwise the script sizes the heap between 512 MB and 32 GB from available memory. Setting STDOUT_MODE=true leaves the JVM output on stdout instead of redirecting it to logs/hugegraph-pd-stdout.log, which is what the Docker image does.

After successful startup, you can see logs similar to the following in logs/hugegraph-pd-stdout.log:

YYYY-mm-dd xx:xx:xx [main] [INFO] o.a.h.p.b.HugePDServer - Started HugePDServer in x.xxx seconds (JVM running for x.xxx)

The process id is written to bin/pid.

5.2 Stop PD

In the PD installation directory, execute:

./bin/stop-hugegraph-pd.sh

The script reads bin/pid, sends the process a termination signal, waits up to 30 seconds for it to exit, and removes the pid file. If bin/pid is missing it reports that and exits successfully.

6 Startup Order in a Distributed Cluster

Start the components in this order:

  1. All PD nodes. Form the Raft group and elect a leader. Check GET /v1/ready on each node; start Store only after HTTP 200 with ready:true. /v1/health checks REST listener liveness only.
  2. All Store nodes. Register with PD; wait for /v1/stores to show every target Store as Up.
  3. All Server nodes. Start after PD and Store are ready.
sequenceDiagram
    participant Operator
    participant PD
    participant Store
    participant Server
    Operator->>PD: GET /v1/ready (check each PD)
    PD-->>Operator: HTTP 200 and ready=true
    Operator->>Store: Start Store
    Store->>PD: gRPC registration and heartbeat
    Operator->>PD: GET /v1/stores (Basic authentication)
    PD-->>Operator: Every target Store is Up
    Operator->>Server: Start Server
    Server->>PD: gRPC partitions and discovery
    Server->>Store: gRPC graph data access

Master Compose waits on PD /v1/health through Store depends_on: condition: service_healthy, and on Store REST liveness before Server. Server also polls PD /v1/stores for an Up Store before starting. Therefore, docker compose up --wait does not replace PD Raft readiness and Store registration checks.

PD is also the last component to stop: shut down Server, then Store, then PD.

7 Verification

7.1 REST API Authentication

Current master requires HTTP Basic Authorization for all PD REST paths except /actuator/*, /v1/health, /v1/ready, and /v1/prom/targets/*. Username must be hg, store, hubble, or vermeer, and password must equal auth.secret-key. A missing secret or wrong password returns HTTP 401. Query Stores using the same secret configured at PD startup:

{"status": -1, "error": "Unauthorized"}
curl -u "store:${HG_PD_AUTH_SECRET_KEY:?Set the PD deployment secret first}" \
  http://localhost:8620/v1/stores

bin/wait-storage.sh uses the same credentials through PD_AUTH_USER and PD_AUTH_PASSWORD. Release 1.7.0 checks only internal service usernames and does not compare passwords; do not apply this older behavior to master builds.

Warning

Protect Server and PD ports separately in production

Enable Server authentication and authorization, an IP allowlist, and minimum graph API permissions; retain and protect Server audit-*.log. These controls do not protect PD. Master PD REST uses auth.secret-key, whereas 1.7.0 checks usernames only. Keep raft.ip-whitelist.enabled enabled for configured peers and restrict PD REST/gRPC to trusted networks. Unauthenticated /v1/health, /v1/ready, and Actuator probes require network restrictions too. Protect master PD audit logs at logs/audit-hugegraph-pd.log.

7.2 Health Checks

Unauthenticated GET /v1/health returns 200 with an empty body, confirming only PD REST listener startup:

curl -i http://localhost:8620/v1/health

Spring Boot Actuator also provides a readable health response:

curl http://localhost:8620/actuator/health

Actuator {"status":"UP"} does not confirm an available PD Raft leader. Master GET /v1/ready returns HTTP 200 with ready:true when the PD Raft node is active and sees a leader; otherwise it returns HTTP 503 with ready:false:

curl -i http://localhost:8620/v1/ready

7.3 Cluster and member status

Check the PD members and which node is the Raft leader:

curl -u "store:${HG_PD_AUTH_SECRET_KEY:?Set the PD deployment secret first}" \
  http://localhost:8620/v1/members

The response carries pdList, the elected pdLeader, numOfService, numOfNormalService and a stateCountMap. In a healthy 3-node PD cluster numOfService and numOfNormalService are both 3 and exactly one member has role: "Leader".

GET /v1/cluster returns the same member list together with the Store list, graph list and overall cluster state, and GET / returns a short summary (leader address, cluster state, member count, store count, graph count, partition count).

7.4 Store status

You can also verify Store node status through the PD API:

curl -u "store:${HG_PD_AUTH_SECRET_KEY:?Set the PD deployment secret first}" \
  http://localhost:8620/v1/stores

If the response shows state as Up, the corresponding Store node is running normally. The example below shows a single Store node. In a healthy 3-node deployment, the storeId list should contain three IDs, and stateCountMap.Up, numOfService, and numOfNormalService should all be 3.

{
  "message": "OK",
  "data": {
    "stores": [
      {
        "storeId": 8319292642220586694,
        "address": "127.0.0.1:8500",
        "raftAddress": "127.0.0.1:8510",
        "version": "",
        "state": "Up",
        "deployPath": "/Users/{your_user_name}/hugegraph/apache-hugegraph-incubating-1.7.0/apache-hugegraph-store-incubating-1.7.0/lib/hg-store-node-1.7.0.jar",
        "dataPath": "./storage",
        "startTimeStamp": 1754027127969,
        "registedTimeStamp": 1754027127969,
        "lastHeartBeat": 1754027909444,
        "capacity": 494384795648,
        "available": 346535829504,
        "partitionCount": 0,
        "graphSize": 0,
        "keyCount": 0,
        "leaderCount": 0,
        "serviceName": "127.0.0.1:8500-store",
        "serviceVersion": "",
        "serviceCreatedTimeStamp": 1754027127000,
        "partitions": []
      }
    ],
    "stateCountMap": {
      "Up": 1
    },
    "numOfService": 1,
    "numOfNormalService": 1
  },
  "status": 0
}

7.5 Other REST endpoints

All paths below are relative to http://<pd-host>:8620 and need the Basic header from section 7.1 unless noted.

Method and pathDescription
GET /Brief cluster statistics: leader, state, member count, store count, graph count, partition count
GET /v1/healthLiveness only; HTTP 200 does not imply Raft readiness; no authentication
GET /v1/readyMaster readiness: HTTP 200 with ready:true means an active node that sees a leader; no authentication
GET /v1/clusterFull cluster statistics: PD members, stores, graphs, partitions
GET /v1/membersPD member list with roles and the elected leader
POST /v1/members/changeChange the Raft peer list, body {"peerList": "..."}
GET /v1/storesRegistered Store nodes with state and per-store statistics
GET /v1/store/{storeId}One Store node
POST /v1/store/{storeId}Update a Store’s state, body {"storeState": "..."}
DELETE /v1/store/{storeId}Remove a Store from the cluster
POST /v1/store/logStore state change log, body {"startTime": "...", "endTime": "..."}
GET /v1/storesAndStatsRaw Store metadata, for debugging
GET /v1/store_monitor/{storeId}Store monitoring samples as text
GET /v1/store_monitor/json/{storeId}Store monitoring samples as JSON
GET /v1/shardsEvery shard of every partition, with store id, role, state and progress
GET /v1/shardGroupsShard groups
GET /v1/shardGroupsCacheShard groups from PD’s in-memory cache
GET /v1/shardLeadersPartition leaders grouped by Store raft address
GET /v1/balanceLeadersRebalance partition leaders across Stores
GET /v1/partitionsPartition list with state and statistics
GET /v1/highLevelPartitionsPartitions with per-graph key counts and data sizes
GET /v1/partitionsAndStatsRaw partition metadata, for debugging
POST /v1/partitions/logPartition change log, body {"startTime": "...", "endTime": "..."}
GET /v1/resetPartitionStateReset the state of every partition
GET /v1/graphsGraph list
GET /v1/graph/**One graph by name
POST /v1/graph/**Update a graph’s partition count, body {"partitionCount": N}
GET /v1/graph/partitionSizeRangeMinimum and maximum partition count the cluster accepts
GET /v1/graph-spacesGraph space list
GET /v1/graph-spaces/**One graph space
POST /v1/graph-spaces/**Update a graph space
POST /v1/registryRegister a service instance for discovery
POST /v1/registryInfoQuery registered instances
GET /v1/allInfoAll registered instances
GET /v1/licenseLegacy endpoint; current master returns an empty object
GET /v1/license/machineInfoIP and MAC addresses seen by the license check
GET /v1/task/patrolStoresRun the store patrol task now
GET /v1/task/patrolPartitionsRun the partition patrol task now
GET /v1/task/balancePartitionsRebalance partitions across Stores
GET /v1/task/splitPartitionsRun automatic partition splitting now
GET /v1/task/balanceLeadersRebalance partition leaders
GET /v1/task/compactInstruct Store nodes to compact the RocksDB files of their partitions
GET /v1/prom/targets/{appName}Prometheus service discovery targets, no authentication required
GET /v1/prom/targets-allPrometheus targets for all app types
GET /v1/prom/sd_configPrometheus HTTP service discovery config
GET /actuator/healthSpring Boot health, no authentication required
GET /actuator/metricsSpring Boot metrics, no authentication required
GET /actuator/prometheusPrometheus scrape endpoint, no authentication required

The two log endpoints take a time range as {"startTime": "...", "endTime": "..."}; yyyy-MM-dd HH:mm:ss and yyyy-MM-dd are among the accepted formats.

PD registers its own meters under the hg prefix, so /actuator/prometheus exposes hg_up, hg_graphs, hg_stores and hg_terms alongside the standard JVM metrics, plus per-graph partition and size meters once graphs exist.

3.1.3 - HugeGraph-Store Quick Start

1 HugeGraph-Store Overview

HugeGraph-Store is the storage node component of HugeGraph’s distributed version, responsible for actually storing and managing graph data. It works in conjunction with HugeGraph-PD to form HugeGraph’s distributed storage engine, providing high availability and horizontal scalability.

Each Store node keeps graph data in RocksDB and replicates it with Raft (JRaft): every partition is a separate Raft group, so a partition survives the loss of a minority of its replicas. Store nodes do not know about each other directly. They register with PD, receive their partition assignment from PD, and report state back over a heartbeat. HugeGraph-Server reaches Store over gRPC after looking up partition locations in PD.

2 Prerequisites

2.1 Requirements

  • Operating System: Linux or macOS (Windows has not been fully tested)
  • Java version: ≥ 11 (enforced by the build and re-checked by bin/start-hugegraph-store.sh)
  • Maven version: ≥ 3.5.0
  • Deploy HugeGraph-PD first for multi-node deployment

3 Deployment

There are two ways to deploy the HugeGraph-Store component:

  • Method 1: Download the tar package
  • Method 2: Compile from source

3.1 Download the tar package

Apache downloads currently provide the complete 1.7.0 binary bundle containing PD, Store, and Server, without a separate Store binary. Verify version, signatures, and SHA512 on the official download page. This historical incubating release retains incubating in archive and directory names:

# Historical release 1.7.0 retains incubating in filenames and directories
wget https://downloads.apache.org/hugegraph/1.7.0/apache-hugegraph-incubating-1.7.0.tar.gz
tar zxf apache-hugegraph-incubating-1.7.0.tar.gz
cd apache-hugegraph-incubating-1.7.0/apache-hugegraph-store-incubating-1.7.0

3.2 Compile from source

# 1. Clone the source code
git clone https://github.com/apache/hugegraph.git

# 2. Build the project
cd hugegraph
mvn clean install -DskipTests=true

# 3. After a successful build, the Store directory and complete distribution package are located at
#    hugegraph-store/apache-hugegraph-store-{version}
#    target/apache-hugegraph-{version}.tar.gz

To build Store alone instead of the whole repository, build hugegraph-struct first, because Store depends on it:

mvn install -pl hugegraph-struct -am -DskipTests
mvn clean package -pl hugegraph-store/hg-store-dist -am -DskipTests

The assembled directory contains only bin/, conf/ and lib/hg-store-node-{version}.jar.

3.3 Docker Deployment

HG_STORE_* mappings are current master features, absent from Server tag 1.7.0. These Docker examples require PD, Store, and HStore Server images built from the same current master source with matching local tags:

# Run from the HugeGraph repository root
docker build -f hugegraph-pd/Dockerfile -t hugegraph/pd:local .
docker build -f hugegraph-store/Dockerfile -t hugegraph/store:local .
docker build -f hugegraph-server/Dockerfile-hstore -t hugegraph/server:local .

Before first startup, create .env in docker/ following the authentication environment instructions. The procedure uses umask 077 and refuses to overwrite existing files. Keep an existing .env and add missing entries. Never regenerate HG_PD_AUTH_SECRET_KEY for initialized data directories. Although .gitignore excludes this file, do not commit it. In every new shell, load the same file from docker/ before Compose or PD REST commands:

# Run from the HugeGraph repository root
cd docker
set -a; . ./.env; set +a
: "${HG_PD_AUTH_SECRET_KEY:?Set HG_PD_AUTH_SECRET_KEY in .env}"
# Start PD, Store, and Server without optional Hubble
HUGEGRAPH_VERSION=local HUGEGRAPH_PULL_POLICY=never \
  docker compose -f docker-compose-hstore.yml \
  up -d --wait pd store server

Load the same secret before subsequent restarts or PD REST calls; do not regenerate it for initialized directories. Explicitly include HUGEGRAPH_VERSION=local and HUGEGRAPH_PULL_POLICY=never in every Compose lifecycle command.

To start Hubble, use the topology-specific untracked configuration: minimal docker-compose-hstore.yml mounts conf/hubble/hstore.local.properties; HA docker-compose-3pd-3store-3server.yml mounts conf/hubble/hstore-ha.local.properties. The README initialization creates both files with the shared PD secret. If missing or requiring updates, load existing .env from docker/ and run:

set -a; . ./.env; set +a
./set-hubble-pd-password.sh hstore      # Minimal topology
./set-hubble-pd-password.sh hstore-ha  # HA topology

Multi-node service names are pd0–pd2, store0–store2, and server0–server2. This master Compose file also requires matching locally built images; do not mix HUGEGRAPH_VERSION=1.7.0 release images with master Compose.

To run master-built Store alone, replace example addresses with actual routable Store and PD addresses:

docker run -d \
  -p 8520:8520 -p 8500:8500 -p 8510:8510 \
  -e HG_STORE_PD_ADDRESS=192.168.1.10:8686 \
  -e HG_STORE_GRPC_HOST=192.168.1.20 \
  -e HG_STORE_RAFT_ADDRESS=192.168.1.20:8510 \
  -v /path/to/storage:/hugegraph-store/storage \
  --name hugegraph-store \
  hugegraph/store:local

Current Master Docker Environment Variables:

VariableRequiredDefaultConfiguration keyDescription
HG_STORE_PD_ADDRESSYesNonepdserver.addressComma-separated PD gRPC addresses.
HG_STORE_GRPC_HOSTYesNonegrpc.hostAdvertised hostname/IP; use a container hostname within a container network.
HG_STORE_RAFT_ADDRESSYesNoneraft.addressThis node’s Raft address.
HG_STORE_GRPC_PORTNo8500grpc.portgRPC service port.
HG_STORE_REST_PORTNo8520server.portREST API port.
HG_STORE_DATA_PATHNo/hugegraph-store/storageapp.data-pathData path.

The master entrypoint writes these values into SPRING_APPLICATION_JSON, overriding image conf/application.yml. Other options still come from that file and application-pd.yml; Java runs in the foreground. Deprecated PD_ADDRESS, GRPC_HOST, and RAFT_ADDRESS remain supported with warnings.

PD and Store Docker HEALTHCHECK use /v1/health, confirming only local REST responses. Compose depends_on uses the same startup gate; separately verify PD /v1/ready and Up Store state in /v1/stores. Store defaults to STDOUT_MODE=true; its Dockerfile declares only EXPOSE 8520, so publish gRPC 8500 and Raft 8510 separately if needed outside the Docker network.

4 Configuration

Store reads two files under conf/. Release 1.7.0 sources: application.yml, application-pd.yml. Master sources: application.yml, application-pd.yml. Master places RocksDB settings in the second file, included through spring.profiles.include: pd; startup selects application.yml.

  • application.yml: Main PD address, ports, Raft, and data path settings.
  • application-pd.yml: Included through spring.profiles.include: pd; RocksDB memory and Actuator exposure.

4.1 application.yml

The following tables were checked against distribution values and Java defaults at Server master commit 2f827d6e8c9c62ae858f2fc122b3a192d015e2f4. Use tagged configuration for release 1.7.0 without replacing it with another version’s files.

4.2 application-pd.yml

This file contains only rocksdb and Actuator exposure settings.

Warning

Protect Server and Store ports separately in production

Enable Server authentication and authorization, an IP allowlist, and minimum graph API permissions; retain and protect Server audit-*.log. These settings do not protect Store: master exposes all Actuator endpoints without PD-style Basic authentication. Restrict Store REST, gRPC, and Raft to cluster nodes and trusted operations networks.

4.3 Configuration reference

“Master distribution value” comes from the two master files above. “Code default” is the fallback when a key is absent; rely on code defaults for options omitted from the distribution files.

Core

KeyMaster distribution valueCode defaultMeaning
pdserver.addresslocalhost:8686requiredPD gRPC endpoints, comma separated. Store registers itself here and receives its partition assignment. Must be PD’s grpc.port, not its REST port.
grpc.host127.0.0.1requiredAddress this node advertises for its own gRPC service. Set it to a routable IP or hostname, 127.0.0.1 is only usable for a single-machine setup.
grpc.port8500requiredgRPC port. Server and the Store client connect here.
grpc.netty-server.max-inbound-message-size1000MBgRPC defaultMaximum size of a single inbound gRPC message. Bound by the grpc-spring-boot-starter Netty server.
grpc.server.wait-timenot set3600Seconds a scan stream waits for the client to consume a page before the server aborts it.
server.port8520requiredREST and Actuator port. Also reported to PD as the rest.port label.

Raft

KeyMaster distribution valueCode defaultMeaning
raft.address127.0.0.1:8510requiredRaft service address of this node, host:port. Must be reachable from every other Store node. There is no peer list to configure: PD tells each node which peers belong to a partition’s Raft group.
raft.disruptorBufferSize10240Raft task queue size. 0 derives it from rocksdb.total_memory_size, by rounding that size in GB to the nearest power of two and multiplying by 32.
raft.max-log-file-size60000000000050000000000Maximum byte size of Raft logs.
raft.snapshotInterval1800300Seconds between Raft snapshots.
raft.snapshotLogIndexMarginnot set0Minimum applied-index distance since the last snapshot before a snapshot is actually written. 0 disables the distance check.
raft.rpc-timeoutnot set10000Raft RPC timeout in milliseconds.
raft.metricsnot settrueCollect JRaft node metrics, readable at /metrics/raft.
raft.useRocksDBSegmentLogStoragenot settrueStore Raft logs in the RocksDB segment log storage.
raft.maxSegmentFileSizenot set67108864Segment log file size in bytes (64 MB).
raft.maxReplicatorInflightMsgsnot set256Maximum in-flight replication requests per follower.
raft.maxEntriesSizenot set256Maximum number of entries in one AppendEntries request.
raft.maxBodySizenot set524288Maximum byte size of one AppendEntries request.
ave-logEntry-size-rationot set0.95Smoothing ratio used to estimate the average log entry size. Note that this key sits at the top level, not under raft.

Storage and labels

KeyMaster distribution valueCode defaultMeaning
app.data-path./storagestoreRocksDB data directory. Multiple paths separated by commas spread partitions over several disks.
app.raft-pathcommented outemptyDirectory for Raft logs and snapshots. Falls back to app.data-path when empty.
app.fake-pdnot setfalseBuilt-in PD mode for standalone testing. Do not use it in production.
app.placeholder-sizenot set10Size in GB of a placeholder file created in each data path at startup, so space can be freed in an emergency. 0 disables it.
app.label.<name>not setnoneArbitrary key/value labels sent to PD in the store heartbeat. The node adds rest.port on its own.

RocksDB

KeyMaster distribution valueCode defaultMeaning
rocksdb.total_memory_size3200000000051539607552Memory budget shared by all RocksDB instances on this node. When absent or 0, the node uses the JVM max heap instead.
rocksdb.write_buffer_size3200000033554432Memtable size in bytes. When absent or 0, the node uses total_memory_size / 1000.
rocksdb.min_write_buffer_number_to_merge1616Number of memtables merged together before a flush.
rocksdb.write_buffer_rationot set0.66Share of total_memory_size given to the write cache. The rest becomes the block cache.

Any other option defined in org/apache/hugegraph/rocksdb/access/RocksDBOptions.java can be added under the same rocksdb: block, for example rocksdb.max_background_jobs, rocksdb.level0_file_num_compaction_trigger or rocksdb.bloom_filter_bits_per_key.

Thread pools

KeyCode defaultMeaning
thread.pool.grpc.core600Core threads serving gRPC requests.
thread.pool.grpc.max1000Maximum gRPC threads.
thread.pool.grpc.queue2147483647gRPC task queue capacity.
thread.pool.scan.core128Core threads serving scans. 0 means 4 times the CPU count.
thread.pool.scan.max1000Maximum scan threads.
thread.pool.scan.queue0Scan task queue capacity.

Query pushdown

KeyCode defaultMeaning
query.push-down.threads1500Thread pool size for pushed-down queries.
query.push-down.fetch_batch20000Rows fetched per request.
query.push-down.fetch_timeout300000Fetch timeout in milliseconds.
query.push-down.memory_limit_count50000Row limit for in-memory operations such as sorting.
query.push-down.index_size_limit_count50000Index sst file size limit in kB.

Background jobs

KeyCode defaultMeaning
job.interruptableThreadPool.core128Core threads of the TTL cleaner pool. 0 means the CPU count.
job.interruptableThreadPool.max256Maximum threads of the TTL cleaner pool. 0 means 4 times the CPU count.
job.interruptableThreadPool.queue2147483647Queue capacity of the TTL cleaner pool.
job.uninterruptibleThreadPool.core0Core threads of the engine’s uninterruptible job pool. 0 means the CPU count.
job.uninterruptibleThreadPool.max256Maximum threads of the uninterruptible job pool.
job.uninterruptibleThreadPool.queue2147483647Queue capacity of the uninterruptible job pool.
job.cleaner.batch.size10000Keys deleted per batch by the TTL cleaner.
job.start-time0Hour of day (0 to 23) at which the daily TTL cleanup runs. Values outside that range fall back to 19.

Built-in PD mode

Only for single-node development and debugging, activated by app.fake-pd: true. The node then plays PD’s role itself and ignores pdserver.address.

KeyCode defaultMeaning
fake-pd.store-list''gRPC addresses of the Store nodes in the fake cluster.
fake-pd.peers-list''Raft addresses of the same nodes.
fake-pd.partition-count3Number of partitions.
fake-pd.shard-count3Replicas per partition.

Diagnostics

KeyCode defaultMeaning
arthas.telnetPort8566Arthas telnet port, used when /v1/arthasstart is called.
arthas.httpPort8565Arthas HTTP port.
arthas.ip0.0.0.0Arthas bind address.
arthas.disabledCommandsjadArthas commands to disable.

4.4 Per-node changes

For multi-node deployment, you need to modify the following configurations for each Store node:

  1. grpc.host and grpc.port (the address other components dial)
  2. raft.address (Raft protocol address)
  3. server.port (REST port)
  4. app.data-path (data storage path)

pdserver.address is the same on every node, it lists the whole PD cluster.

5 Start and Stop

5.1 Start Store

Ensure that the PD service is already started, then in the Store installation directory, execute:

./bin/start-hugegraph-store.sh

The script accepts four flags:

FlagValuesDefaultDescription
-dtrue, falsetrueDaemon mode. See below.
-gZGC, zgcnot setGarbage collector. Omit the flag for G1, which is the default. Any value other than ZGC or zgc aborts the start, including g1, even though the script’s own usage line suggests it.
-jJVM options stringemptyExtra JVM options, for example -j "-Xmx16g -Xms8g".
-ytrue, falsefalseAttach the OpenTelemetry Java agent, downloading it into plugins/ on first use, and export traces to 127.0.0.1:4317.

Daemon mode:

  • -d true (default): run as a background daemon. The script returns immediately and writes the Java pid to bin/pid.
  • -d false: run in the foreground. The script execs Java, so the container or supervisor process is Java itself. Use this under Docker or a process supervisor (systemd, supervisord) so crashes are detected and the service is restarted automatically.

JVM memory, unless you set JAVA_OPTIONS yourself: -Xms512m, and -Xmx set to half the free memory, clamped to the 512 MB to 2048 MB range. The script also adds -XX:MetaspaceSize=256M, a heap dump on out-of-memory into logs/, and a rolling GC log at logs/gc.log. Production nodes normally need a much larger heap, so pass one explicitly, for example -j "-Xmx32g -Xms32g".

The script refuses to start if ulimit -n or ulimit -u is below 1024, and it preloads jemalloc on x86_64 and arm64 when the shared object can be downloaded and verified.

After successful startup, you can see logs similar to the following in logs/hugegraph-store-server.log:

YYYY-mm-dd xx:xx:xx [main] [INFO] o.a.h.s.n.StoreNodeApplication - Started StoreNodeApplication in x.xxx seconds (JVM running for x.xxx)

5.2 Stop Store

In the Store installation directory, execute:

./bin/stop-hugegraph-store.sh

The script reads bin/pid, signals that process, and waits up to 30 seconds for it to exit before removing the pid file. If bin/pid is missing it exits without doing anything.

5.3 Restart Store

./bin/restart-hugegraph-store.sh

It sources the stop script and then the start script, and forwards the flags from section 5.1.

5.4 Startup order

  1. All PD nodes first. Each Store’s grpc.host:grpc.port must appear in PD pd.initial-store-list, or the node remains Pending instead of Up, blocking partition allocation. On master, verify HTTP 200 with ready:true from every PD /v1/ready first; /v1/health is liveness only.
  2. Store next. Registration retries while PD is unreachable, logging store heartbeat error: PD UNREACHABLE.
  3. Server last. Every Store should report Up; partitions must be ready before opening or initializing graphs.

Master Compose uses depends_on: condition: service_healthy for local liveness, but PD and Store Docker checks only call /v1/health; they do not confirm Raft quorum or Store registration. Server waits for PD to report at least one Up Store. Operators should still check PD /v1/ready and /v1/stores.

6 Multi-Node Deployment Example

Below is a configuration example for a three-node deployment:

6.1 Three-Node Configuration Reference

  • 3 PD nodes
    • raft ports: 8610, 8611, 8612
    • rpc ports: 8686, 8687, 8688
    • rest ports: 8620, 8621, 8622
  • 3 Store nodes
    • raft ports: 8510, 8511, 8512
    • rpc ports: 8500, 8501, 8502
    • rest ports: 8520, 8521, 8522

6.2 Store Node Configuration

For the three Store nodes, the main configuration differences are as follows:

Node A:

grpc:
  port: 8500
raft:
  address: 127.0.0.1:8510
server:
  port: 8520
app:
  data-path: ./storage-a

Node B:

grpc:
  port: 8501
raft:
  address: 127.0.0.1:8511
server:
  port: 8521
app:
  data-path: ./storage-b

Node C:

grpc:
  port: 8502
raft:
  address: 127.0.0.1:8512
server:
  port: 8522
app:
  data-path: ./storage-c

All nodes should point to the same PD cluster:

pdserver:
  address: 127.0.0.1:8686,127.0.0.1:8687,127.0.0.1:8688

And every PD node should list all three Store gRPC addresses:

pd:
  initial-store-list: 127.0.0.1:8500,127.0.0.1:8501,127.0.0.1:8502

6.3 Docker Distributed Cluster Configuration

The distributed Store cluster definition is included in docker/docker-compose-3pd-3store-3server.yml. Each Store node gets its own hostname and environment variables:

# store0, published as 8500 (gRPC), 8510 (Raft), 8520 (REST)
HG_STORE_PD_ADDRESS: pd0:8686,pd1:8686,pd2:8686
HG_STORE_GRPC_HOST: store0
HG_STORE_GRPC_PORT: "8500"
HG_STORE_REST_PORT: "8520"
HG_STORE_RAFT_ADDRESS: store0:8510
HG_STORE_DATA_PATH: /hugegraph-store/storage

# store1, published as 8501, 8511, 8521
HG_STORE_GRPC_HOST: store1
HG_STORE_RAFT_ADDRESS: store1:8510

# store2, published as 8502, 8512, 8522
HG_STORE_GRPC_HOST: store2
HG_STORE_RAFT_ADDRESS: store2:8510

The container ports stay 8500/8510/8520 on every node, only the published host ports differ. The PD nodes set HG_PD_INITIAL_STORE_LIST: store0:8500,store1:8500,store2:8500 to match.

Store nodes start only after all PD nodes pass healthchecks (/v1/health), enforced via depends_on: condition: service_healthy.

To view runtime logs for a running Store container use docker logs <container-name> (e.g. docker logs hg-store0).

See docker/README.md for the full setup guide.

7 Verify Store Service

Confirm that the Store service is running properly:

curl http://localhost:8520/actuator/health

Actuator {"status":"UP"} confirms local Store application health, without proving registration as Up in PD.

GET /v1/health is the lighter check used by the Docker image and the compose files. It answers HTTP 200 with an empty body, so use curl -fsS and check the exit code rather than the output:

curl -fsS http://localhost:8520/v1/health && echo OK

7.1 Store REST endpoints

The Store node exposes these read-only endpoints on server.port:

MethodPathDescription
GET/v1/healthLiveness probe, HTTP 200 with an empty body
GET/actuator/healthSpring Boot Actuator health, {"status":"UP"}
GET/actuator/prometheusPrometheus scrape endpoint
GET/Node summary, leaderCount and partitionCount
GET/-/stateOperator-set STARTING, ONLINE, or STOPPING flag; distinct from PD registration state
GET/-/echo?name=<text>Echo check
GET/-/scanState of the running scan streams
GET/v1/partitionsAll Raft groups on this node with per-partition metrics. Add ?flags=accurate for exact key counts, which is slower.
GET/v1/partition/{id}One Raft group by partition id, including role, leader, peers and committed index
GET/metrics/systemHost CPU and memory metrics
GET/metrics/driveDisk metrics for the data paths
GET/metrics/raftJRaft node metrics, needs raft.metrics: true

Master Store configuration exposes all Actuator endpoints with management.endpoints.web.exposure.include: "*" and enables Prometheus. These endpoints lack PD REST Basic protection; restrict Store REST network access.

The node also serves maintenance endpoints that change state or run heavy work: PUT /-/state, GET /-/cleaner, GET /v1/partition/dump/{id}, GET /v1/partition/clean/{id}, POST /v1/compat?id=<partition>, GET /v1/arthasstart, POST /raft/options, and the /fix/* and /test/* groups. Use them only for troubleshooting, and keep the REST port off untrusted networks.

7.2 Check registration from PD

You can also check Store node status through the PD API:

curl -u "store:${HG_PD_AUTH_SECRET_KEY:?Set the PD deployment secret first}" \
  http://localhost:8620/v1/stores

For master builds, PD REST usernames must be hg, store, hubble, or vermeer, and the password must equal PD auth.secret-key. HG_PD_AUTH_SECRET_KEY configures PD and Server in Compose. /v1/health, /v1/ready, /actuator/*, and /v1/prom/targets/* require no authentication. Release 1.7.0 checks only usernames, without comparing passwords; do not use that old behavior for master builds.

If Store is configured successfully, the response should include status information for the current node, and state: "Up" means the node is running normally. A node stuck at Pending is usually missing from PD’s pd.initial-store-list.

The example below shows a single Store node. If all three nodes are configured correctly and running, the storeId list should contain three IDs, and stateCountMap.Up, numOfService, and numOfNormalService should all be 3.

{
  "message": "OK",
  "data": {
    "stores": [
      {
        "storeId": 8319292642220586694,
        "address": "127.0.0.1:8500",
        "raftAddress": "127.0.0.1:8510",
        "version": "",
        "state": "Up",
        "deployPath": "/Users/{your_user_name}/hugegraph/hugegraph-store/apache-hugegraph-store-{version}/lib/hg-store-node-{version}.jar",
        "dataPath": "./storage",
        "startTimeStamp": 1754027127969,
        "registedTimeStamp": 1754027127969,
        "lastHeartBeat": 1754027909444,
        "capacity": 494384795648,
        "available": 346535829504,
        "partitionCount": 0,
        "graphSize": 0,
        "keyCount": 0,
        "leaderCount": 0,
        "serviceName": "127.0.0.1:8500-store",
        "serviceVersion": "",
        "serviceCreatedTimeStamp": 1754027127000,
        "partitions": []
      }
    ],
    "stateCountMap": {
      "Up": 1
    },
    "numOfService": 1,
    "numOfNormalService": 1
  },
  "status": 0
}

7.3 Minimal Graph Read/Write Verification

After Store appears as Up in PD, start an HStore-backed Server connected to the same PD cluster. See the Server Quick Start. Once GET /versions responds and the default hugegraph graph is loaded, create a property key, vertex label, and vertex, then read it:

SERVER_URL=http://localhost:8080
GRAPH_URL="$SERVER_URL/graphspaces/DEFAULT/graphs/hugegraph"

# Names must be unique within the graphspace; choose new names if they already exist
curl -fsS -X POST "$GRAPH_URL/schema/propertykeys" \
  -H 'Content-Type: application/json' \
  -d '{"name":"pd_store_demo_name","data_type":"TEXT","cardinality":"SINGLE","properties":[]}'

curl -fsS -X POST "$GRAPH_URL/schema/vertexlabels" \
  -H 'Content-Type: application/json' \
  -d '{"name":"pd_store_demo_vertex","id_strategy":"CUSTOMIZE_STRING","properties":["pd_store_demo_name"],"primary_keys":[],"nullable_keys":[]}'

curl -fsS -X POST "$GRAPH_URL/graph/vertices" \
  -H 'Content-Type: application/json' \
  -d '{"id":"pd-store-demo-1","label":"pd_store_demo_vertex","properties":{"pd_store_demo_name":"stored in HStore"}}'

curl -fsS "$GRAPH_URL/graph/vertices/%22pd-store-demo-1%22"

The final response should contain vertex ID pd-store-demo-1 and property pd_store_demo_name. If Server authentication is enabled, add credentials to each request.

3.2 - HugeGraph ToolChain

HugeGraph Toolchain includes the Java and Go clients, Loader, Hubble, Tools, Spark Connector, and SeaTunnel Sink/Source. Choose an entry by the task you need to complete, then open the component guide for its configuration and commands.

TaskStart hereBest for
Visualize graphsHubbleViewing and managing graphs in a Web UI
Import graph dataLoader, SeaTunnel Sink, Spark ConnectorImporting data directly or connecting an existing pipeline
Export or migrate graph dataTools, SeaTunnel SourceBackup, export, cross-graph migration, and continuous reads

Testing Guide: For running toolchain tests locally, please refer to HugeGraph Toolchain Local Testing Guide

DeepWiki provides real-time updated project documentation with more comprehensive and accurate content, suitable for quickly understanding the latest project information.

📖 https://deepwiki.com/apache/hugegraph-toolchain

Source repository: apache/hugegraph-toolchain

3.2.1 - Graph visualization

Hubble provides a Web interface for HugeGraph data, schema, queries, and imports. Start with a standalone deployment without PD, then read the distributed differences when needed.

3.2.1.1 - Manage an HStore Cluster with Hubble

Connect HStore and PD to Hubble and understand GraphSpaces, Schema templates, cluster topology, and node metrics.

This guide covers the differences between HStore + PD and standalone RocksDB. For modeling, importing data, and querying, see the Hubble standalone guide. This guide follows Toolchain master.

The main repository’s docker/docker-compose-hstore.yml already combines PD, Store, Server, and Hubble. Follow the adjacent Docker README to prepare .env and generated Hubble local configuration, then start it from docker/; do not maintain another deployment YAML. The settings below explain connection differences and do not replace the README’s PD credential and service-readiness requirements.

The screenshots use Hubble built from Toolchain 1.8.0 with Server, PD, and Store 1.7.0. Hubble currently returns the static value 3.0.0 from /about, which does not identify its build version. This pairing describes the screenshot environment; latest is mutable. Metrics and permission APIs vary by version.

Connect to a distributed cluster

Hubble still manages graph data through the Server graph API. In distributed mode, it discovers Servers through PD and collects cluster information from PD and Store. Hubble does not read or write graph data directly in Store.

For the official Compose deployment, prepare .env in the main repository’s docker/ directory following its README, then generate the Hubble configuration:

set -a
. ./.env
set +a
./set-hubble-pd-password.sh hstore

The script reads the loaded HG_PD_AUTH_SECRET_KEY and writes the PD operations password to the host file docker/conf/hubble/hstore.local.properties. Edit this generated file to customize connection or operations settings, preserving operations.pd.password so it matches PD’s secret. Do not replace it with the tracked .example template. Compose mounts it read-only at /hubble/conf/hugegraph-hubble.properties inside the container; do not edit it there. Do not commit the generated file or .env. Running the script again overwrites the generated file, so reapply custom settings afterward.

Start the services from the same docker/ directory and confirm Server registration and Store readiness:

docker compose -f docker-compose-hstore.yml up -d --wait

For separately deployed services, follow the PD deployment guide and HStore deployment guide. A source or binary Hubble deployment instead uses the package’s conf/hugegraph-hubble.properties. These settings match the official minimal topology; use backend-reachable addresses for other deployments:

pd.enabled=true
cluster=hg
pd.peers=pd:8686
pd.server=pd:8620
SettingPurposeBundled value
pd.enabledExplicitly enable PD mode; server.direct_url is not used in this mode.false
clusterCluster name used for Server discovery; it must match the registration.hg
pd.peersPD gRPC addresses, separated by commas.127.0.0.1:8686
pd.serverPD REST address for cluster operations, not a list of gRPC peers.127.0.0.1:8620

Do not interchange ports 8686 and 8620, or retain 127.0.0.1 for connections between containers. The bundled file explicitly sets pd.enabled=false, while the Java fallback for a missing key is true; set it explicitly in either deployment. Restart source or binary Hubble deployments after configuration changes. For Compose, recreate the Hubble container from docker/ after editing or regenerating the host file so the read-only bind mount loads it again; retain the original project name and all -f arguments:

docker compose -f docker-compose-hstore.yml up -d --force-recreate hubble

You do not enter a Server host and port for each graph in the UI.

Organize graphs and permissions with GraphSpaces

A GraphSpace groups graphs, Schema templates, and access permissions. For example, create sales and research spaces so each team can work with its own graphs. Modeling, importing, and querying within a space follow the standalone guide.

Create and adjust a GraphSpace

With Server authentication enabled, creating, editing, and deleting GraphSpaces require super administrator access. Start with a globally unique GraphSpace name, an optional display alias, and the maximum graph count. For example, use research as the API identifier and “Research_graph” as the display alias. The name cannot change after creation. The alias and description can change; aliases do not participate in URLs or permission matching. Choose GraphSpace administrators from existing accounts.

“Advanced deployment and resource limits” includes CPU and memory limits for graph query/write services and asynchronous compute tasks, plus the storage capacity limit. These are deployment and quota settings, not current usage or a promise to resize Docker containers when the form is saved. Defaults are 100 graphs, 64 CPU cores and 128 GB of memory for each of the graph and compute services, and 1000000 GB of storage. Keep the defaults for a container trial. Configure Kubernetes namespaces, Operator images, and algorithm images only for the corresponding deployment or compute use.

Select a space before opening a graph. Check the current graph after switching spaces, especially when spaces contain identically named graphs. The list reflects account access. If a space is missing, check membership permissions before creating more graphs.

GraphSpace creation and advanced resource limits

Reuse Schema templates

User-defined Schema templates require PD mode and persist reusable Groovy Schema within a GraphSpace. When several business graphs share vertex labels, edge labels, and indexes, save the model as a template and select it when creating subsequent graphs. For example, a user template in research can be reused for new graphs in that space. After switching spaces, select a template belonging to the new space.

User templates differ from the built-in sample templates in the main guide: built-in templates help explore preset models, while user templates preserve your own models for future use. A template is not an import of data; loading sample data is a separate graph creation choice. With Server authentication enabled, creating a user template requires write access to its space. Updating or deleting it also requires ownership or the corresponding administrative permission. Anonymous mode does not enforce per-user permissions or template ownership checks. Standalone mode does not provide user template management.

Assign access to a GraphSpace

For account creation, login, and personal details, see the standalone guide. Distributed mode adds “Manage GraphSpace members” to assign permissions between existing accounts and selected spaces. Servers supporting permission presets provide these common choices:

PresetIntended userScope
GraphSpace read-onlyUsers who inspect and query graph dataSelected spaces.
GraphSpace read-writeUsers who model, import, and maintain graph dataSelected spaces.
GraphSpace administratorOwners managing a space’s members and graph resourcesSelected spaces; does not grant GraphSpace creation/editing or cluster operations access.
Super administratorOperators managing accounts, GraphSpaces, and the clusterGlobal; assign according to responsibilities.

An account may have different access in different spaces, such as read-write in research and read-only in sales. Accounts and space memberships are managed separately; removing a member does not delete the global account. After creating an account, continue directly to assigning space access. When changing an existing membership, check the selected space and preset and preserve permissions still needed in other spaces. Enabling pd.enabled grants no permissions; legacy custom roles on older Servers are not interchangeable with these presets. See Server authentication and authorization.

GraphSpace members and access presets

Locate problems through the cluster overview

The overview presents Server → PD → Store in one topology. Switch to the node list to filter by type, status, or name. PD Leader, online Store count, graphs, partitions, replicas, and data size help identify cluster scale and nodes requiring attention. Partitions and replicas describe data distribution. Data size is observed usage, distinct from a GraphSpace’s configured storage limit.

Start with cluster and source status, then inspect an affected node. UP indicates a successful collection from the source; DEGRADED indicates a cluster or partial source issue, and DOWN indicates that the corresponding probe failed. Graphs may remain queryable when topology is available but some metrics are missing. The UI distinguishes unsupported, unavailable, stale, and failed collections and includes observation times. An empty value is not zero, and an older value is not a current observation.

Cluster topology, node status, and capacity

Read node details

NodeMain observationsUse
ServerAvailability, JVM/CPU/memory, and backend metricsCheck the query/write entrypoint and resource pressure.
PDLeader/Follower role, status, and runtime metrics supplied upstreamCheck the metadata/scheduling entrypoint; do not infer a role when Leader information is absent.
StorePartition and Leader partition counts, system/disk metrics, Raft group and enabled group countsInspect storage nodes and replica service and compare distribution across nodes.
Store system, drive, Raft, and partition metrics

The PD Leader and Store Leader partitions serve different purposes: PD coordinates cluster metadata, while a Store leads the respective data partition’s Raft group. Hubble displays upstream observations and does not replace a full Raft replica consistency check. Available metrics vary by component version.

With Server authentication enabled, cluster operations require a super administrator (ADMIN level). GraphSpace administrators and ordinary members do not inherit cluster access. Use container isolation, a trusted HTTPS entrypoint, Server authentication, and network allowlists; avoid publishing Hubble or component ports directly.

Configure operations access

Hubble’s backend uses the PD/Store operations credentials, separately from the Server account used to log in through the browser. For Compose, edit the generated host file docker/conf/hubble/hstore.local.properties; other deployments use the Hubble package configuration. The script-generated PD password must match the secret in .env. Configure the Store service account when Store authentication is enabled. Do not include passwords in documentation, screenshots, or committed configuration:

SettingBundled defaultConfiguration
operations.pd.username / operations.pd.passwordUsername hubble, empty passwordMatch PD operations REST authentication.
operations.store.username / operations.store.passwordUsername hubble, empty passwordSet the service account when upstream Store REST authentication is enabled.
operations.store.allowed_targets[http://127.0.0.1:8520,http://[::1]:8520]List trusted Store metric origins.

For example, when a containerized Store advertises store:8520, set:

operations.store.allowed_targets=[http://store:8520]

For multiple nodes, list each trusted origin. Every entry must use http or https with an explicit port and no path, credentials, or wildcard, and must match the Store metric target returned by PD. Adding an origin to this list does not register or discover a node.

If topology is available but metrics are incomplete, check backend connectivity and authentication to PD REST, the Store REST/metric targets returned by PD, and their match with the allowlist. A partially available overview means some sources could not be collected; it does not imply that all graph APIs are unavailable.

3.2.2 - Graph Visualization with Hubble: Standalone Quick Start

Visualize graph data with Hubble and RocksDB Server: start with Docker, explore schema, import CSV, and run Gremlin queries.

Hubble is the HugeGraph Web management and graph visualization interface. Use one workspace to manage schema, import data, run queries, and switch between graph, table, and JSON results. This guide uses standalone RocksDB Server + Hubble, without PD or Store.

For HStore, read the shared operations here first, then follow the distributed supplement. This guide follows Toolchain master (currently 1.8.0); Docker latest is mutable, so check the actual running versions.

Warning

Do not expose Hubble or Server directly to the public network. In production, use HTTPS, containers, authentication and authorization, and an access allowlist.

Start the standalone pair

Use the main repository’s docker/docker-compose.yml instead of writing another Compose file. It already combines RocksDB Server and Hubble, with networking, health checks, and data volumes. See the adjacent README for deployment details.

git clone --branch master --single-branch --depth 1 https://github.com/apache/hugegraph.git
cd hugegraph/docker

If you already have the main repository, enter its docker/ directory. Compose mounts conf/hubble/standalone.properties from that directory. It sets pd.enabled=false and server.direct_url=http://server:8080; both services communicate over one Docker network, without a Server address configured per graph. Do not download only the YAML and start it from another directory: relative configuration files may be missing.

Hubble defaults to host loopback port 8088; Server publishes 8080. For a trial on your machine only, change Server’s ports entry to 127.0.0.1:8080:8080 to avoid exposing the anonymous API to other machines.

Choose an unused project name for this trial and keep these variables in the same terminal. Restore this project name if you use another terminal.

export HUGEGRAPH_VERSION=latest
export HUBBLE_IMAGE=hugegraph/hubble:latest
export HUBBLE_DEMO_PROJECT="hubble-demo-$(date +%Y%m%d-%H%M%S)"
docker compose ls
docker compose -p "$HUBBLE_DEMO_PROJECT" -f docker-compose.yml pull
docker compose -p "$HUBBLE_DEMO_PROJECT" -f docker-compose.yml up -d --wait
docker compose -p "$HUBBLE_DEMO_PROJECT" -f docker-compose.yml ps
curl -fsS http://127.0.0.1:8080/versions

Once services are healthy, open http://127.0.0.1:8088. In a fresh directory without HUGEGRAPH_ADMIN_PASSWORD, Server allows anonymous access and Hubble opens the home page directly. For authentication, follow the Docker README to configure an administrator password and JWT secret in .env, then sign in with a Server account. Hubble has no separate account database. Personal and account-management pages depend on the authentication mode and your permissions. Do not overwrite an existing .env.

Use latest to try current features, and pin a published image version or digest for production. Images are convenience distributions; official release archives are on the download page. The algorithm and account screenshots use Hubble built from Toolchain 1.8.0 with Server 1.7.0. Hubble currently returns the static value 3.0.0 from /about, which does not identify the build version. This pairing describes those screenshots, not a fixed meaning of latest. Available controls depend on Server capabilities. Compose’s server-data and hubble-data retain graph data and Hubble metadata respectively. For further persistence and production settings, see the Server deployment guide.

Home: find the right starting point

Home groups the workspace into graph overview, data preparation, and graph queries. Use it to understand the workflow, then return to any section through the sidebar. The top graph selector determines the target of queries, schema operations, and async tasks; check the current graph after changing pages. Standalone mode has only the DEFAULT GraphSpace and needs no PD configuration.

SectionPurpose
Graph Overview and detailsSelect a graph, load samples, inspect its size, then model or query it
Schema configurationDefine properties, vertex/edge labels, and indexes
GQL TraversalWrite queries and explore graph, table, and JSON results
Built-in AlgorithmsExplore neighbors, paths, and similarity with parameter forms
Async TasksTrack background queries, schema changes, and index operations
Data Source ManagementUpload files or configure external readers
Data ImportMap source fields to the graph model and run or schedule ingestion
Profile and Account ManagementUpdate personal details/passwords and manage accounts or space members according to permissions
OperationsInspect Server nodes; PD mode also includes cluster overview and PD/Store nodes

Graph Overview and details: get to know a graph

Select the default graph hugegraph in Graph Overview. The overview provides graph entry points and action menus. Graph details show schema and data statistics, with routes to modeling, data preparation, and queries. Statistics describe overall size; update them after importing or modifying data.

Load the People & Software Demo Graph from the graph’s More actions menu. Samples add their schema and missing elements without clearing existing data. Start with an empty graph to avoid conflicting schema names. The remaining examples explore the people and software in this graph.

Graph overview with the person/software sample

Graph creation depends on Server capabilities. Its form accepts a name, optional alias, and schema or sample; it does not configure a Server host or account per graph. The connection comes from Hubble configuration. User-defined schema templates require PD mode; see the distributed supplement.

Schema modeling: define the shape of your data

Schema determines valid properties and relationships, vertex ID generation, and query indexes. Open the graph’s schema configuration. List view is useful for maintaining definitions; graph view helps explain how labels connect.

DefinitionWhat to decide
PropertiesData type and cardinality; distinguish numbers from text
Vertex labelsProperties, nullable properties, ID strategy, and primary keys
Edge labelsSource/target labels, properties, frequency, and sort keys
Vertex / edge indexesIndex type and fields that match filtering and range queries

In the person/software sample, person generates IDs from the primary key name, with nullable age and city. software has custom numeric IDs, and created connects people to software. This matches the Loader example.

Vertex labels with primary-key and numeric ID strategies

For a new model, define properties, vertex labels, edge labels, then indexes. Associated-property and index information help inspect dependencies. Schema deletion and index creation/rebuild may submit background tasks. Acceptance of an operation is only the first step; confirm its final status in Async Tasks.

GQL workspace: query and explore relationships

Open GQL Traversal and confirm hugegraph is selected. The workspace puts the editor and results together, with immediate or async execution, query favorites, and reusable execution history. Start with this Gremlin query:

g.V().hasLabel('person').valueMap()

Inspect person properties in table or JSON view. To visualize the people-to-software relationships, run:

g.V().hasLabel('person').outE('created').inV().path()
Gremlin path query and graph result

Graph results support 2D / 3D. Click a vertex or edge to inspect its ID, label, and properties; double-click a vertex to expand its neighbors. Layout, styling, and filtering help highlight relevant relationships, while export helps share results. Use New to create elements, or edit existing data when authorized. Layout, colors, and display limits only change presentation; adding/editing elements and Gremlin writes change Server data.

Use Ctrl / Command + Enter to execute. Immediate mode suits small explorations; submit long queries asynchronously and avoid returning an entire large graph. Cypher is available only when Server supports it. Text2GQL is currently a UI preview with no model or query service connected; it cannot generate executable queries.

Built-in algorithms: explore with parameter forms

Use Built-in Algorithms when you prefer a form to writing traversal code. Search for an algorithm and supply its parameters. Neighbor exploration answers what surrounds a vertex, path algorithms connect two vertices, and similarity/ranking algorithms compare or select vertices. Start with a known vertex ID and limit direction, edge labels, depth, and result size before attempting broader computation.

Forms provide parameter guidance and documentation links, and restore common parameters when navigating away and back. Results use graph or algorithm-specific panels. Consult the help and documentation links beside the algorithm title for definitions and parameters. While the parameter form is focused, Ctrl / Command + Enter runs the current algorithm.

OLAP batch algorithms require external compute services such as Computer or Vermeer. The two containers in this example provide online graph operations, not those compute services.

For example, select K-neighbor (GET) with source=1:marko, max_depth=1, and limit=20. After parameter validation, use the run button on the card or the form shortcut to explore one-hop neighbors.

Built-in neighbor algorithm parameters and graph result

Async Tasks: confirm background results

Async Tasks lists background work for the current graph, including async queries and some schema/index operations. Filter by task type and status, inspect IDs, creation times, and execution states, open successful query results, or expand failure information. Completed task records can be deleted where the interface permits. These tasks are separate from import execution history.

For example, submit g.V().count() asynchronously, confirm success in the list, then inspect the returned count. A successful submission means the request was accepted, not that computation or indexing has finished. Use task errors and Server logs together when diagnosing failures.

Data Source Management: prepare the input

A data source defines where data comes from and how to parse it, and can be referenced by import tasks. Hubble supports FILE, HDFS, JDBC, and Kafka, each with its own path, connection, or subscription settings. FILE is an easy starting point: upload a file, configure its format, delimiter, encoding, and header, and check column names before mapping. Hubble configuration controls upload limits and permitted extensions.

Save this UTF-8 file as people.csv for the person example:

name,age,city
docs_alice,28,Beijing
docs_bob,32,Shanghai

Create a FILE data source and upload it. Select CSV (comma separation and UTF-8 by default), with column names name,age,city. Header, delimiter, and encoding belong to the data source, not mapping settings. The source fields must match the file.

Data Import: turn fields into a queryable graph

Data Import converts source rows into vertices and edges. Its four configuration sections identify the target, select fields, map them to the graph, and choose execution timing. Use the preceding data source to create a person import into DEFAULT / hugegraph:

SectionSettings for this example
Basic InformationTarget graph, new data source, and a recognizable task name
Source FieldsSelect name, age, and city, moving them to the selected field list
Mapping FieldsAdd a person vertex mapping; use Auto Match for same-name properties, then verify types
ScheduleChoose one-time execution; confirmation submits the task immediately

person uses PRIMARY_KEY, so do not select a separate ID column. Custom ID strategies require an ID column; AUTOMATIC lets Server generate IDs, while PRIMARY_KEY derives them from mapped primary-key properties. Edge mappings need source/target fields that follow the corresponding vertex ID rules.

The task list manages configuration and execution entry points; execution history in task details shows each instance’s state, count, and errors. Periodic schedules and real-time Kafka tasks are also available; choose an execution mode compatible with the source. After completion, verify the result in the GQL workspace:

g.V().hasLabel('person').has('name', within('docs_alice', 'docs_bob')).valueMap()

The result should include both new people. Import counts are not necessarily counts of newly created vertices: reruns may update existing elements, and header processing can affect reader counts. If the import fails, check source fields, numeric types, nullable properties, and target schema. Use Hubble for small trials and HugeGraph Loader for production bulk ingestion.

Profile and account permissions

Hubble uses Server authentication and accounts, with no separate user database. Anonymous mode hides Profile and Account Management. After authentication is enabled, Profile shows the current account’s details and permissions and allows changing your password. Editing details such as a nickname requires Server support for the personal-profile API. Changing a password ends the current session; sign in again with the new password.

Standalone account management

In standalone mode, the administrator can create, inspect, edit, delete, and batch-create accounts. Ordinary standalone accounts created through Hubble receive read, write, delete, and execute permissions across all graphs; they are not read-only or isolated to one graph. Configure finer resource permissions through Server authentication and authorization rather than relying on GraphSpace presets.

GraphSpace permissions in PD mode

With a Server supporting default-role APIs, global accounts and space access can be managed separately. The administrator manages global accounts; a space administrator manages members only within authorized spaces. Ordinary members do not receive account-management or operations entry points. The following presets are for PD mode, and are not universally editable on older or standalone Servers:

PresetScope and purpose
SUPER_ADMINGlobal account, GraphSpace, and operations management; grant or revoke super-administrator access
GS_ADMINManage authorized spaces and their members, without granting other-space or global super-administrator access
GS_READ_WRITERead and write graph data within authorized spaces, without managing global accounts
GS_READ_ONLYRead graph data within authorized spaces, without writes

An account may have different permissions in different spaces. Select a space before adding an existing account or changing member permissions. Before replacing custom permissions with a preset, inspect the grants that need to be retained; complex permissions may not match a single preset. After changes, refresh permission context or sign in again and verify menus and space selection. Server still validates every request. Older Servers may hide or disable unsupported operations; a visible button alone does not establish resource authorization. See the distributed supplement for space management.

Authenticated account list in PD mode

Operations: inspect the standalone Server

Standalone Operations provides Node Information for Server only, with no PD/Store nodes or cluster overview. Search nodes, filter health status, and open node details to inspect available version, system, JVM, and Server-backend metrics. When metrics are unavailable or stale, use collection state and the last successful observation time; a missing value is neither zero nor proof of health.

Anonymous mode can read operations information. With authentication enabled, operations are available only to administrators with the capability. This is an observation and diagnosis interface, not a start/stop or scaling console. The distributed supplement covers cluster overview and the PD/Store node hierarchy.

Keyboard shortcuts and graph interactions

Use the topbar shortcut-help button to see key bindings. Their scope differs: typing ? in an input does not trigger global help.

ActionKey or gestureScope
Open / close shortcut help?Outside inputs and editors
Execute queryCtrl / Command + EnterQuery editor
Run current algorithmCtrl / Command + EnterAlgorithm parameter form
Toggle graph fullscreenFClick to focus the graph canvas first; not a global binding
Inspect element detailsClick a vertex or edgeGraph result
Expand neighboring relationshipsDouble-click a vertexGraph result

Diagnose connection and result issues

SymptomCheck first
The Hubble page does not openCheck container status and docker compose -p "$HUBBLE_DEMO_PROJECT" -f docker-compose.yml logs hubble
The page opens but graphs are unavailableServer health, a server.direct_url reachable from Hubble, and a shared network
Login appears or write actions are missingServer authentication and account permissions; Hubble has no independent authentication switch
Expected data is missingCurrent graph, sample load result, matching labels/properties; distinguish canvas display from stored data
No cluster overviewThis example has no PD; see the distributed supplement

Configuration can affect displayed query size. gremlin.suffix_limit defaults to 250 and supplies .limit(N) appended to applicable Gremlin queries; it is not a universal hard limit. gremlin.vertex_degree_limit (100) and gremlin.edges_total_limit (500) constrain expansion. FILE uploads allow csv,txt by default, with 1 GB per file and 10 GB total. Override upload_file.* settings when needed.

Stop the trial or build from source

When finished, run this in the main repository’s docker/ directory:

docker compose -p "${HUBBLE_DEMO_PROJECT:?}" -f docker-compose.yml down --volumes

This removes the project’s containers, network, named volumes, and anonymous volume, losing the sample data and Hubble import tasks. To retain data, omit --volumes when taking the deployment down and reuse the same project name when starting it again.

To obtain an exact master build, use JDK 11 and Maven. The Maven plugin installs the required Node/Yarn; you do not need to install them separately. These commands skip tests:

git clone --branch master --single-branch https://github.com/apache/hugegraph-toolchain.git
cd hugegraph-toolchain
mvn install -pl hugegraph-client,hugegraph-loader -am -Dmaven.javadoc.skip=true -DskipTests -ntp
cd hugegraph-hubble
mvn package -Dmaven.javadoc.skip=true -DskipTests -ntp
cd apache-hugegraph-hubble-*
# Edit conf/hugegraph-hubble.properties with the correct Server URL
bin/start-hubble.sh

The bundled configuration binds to localhost:8088. bin/stop-hubble.sh requests graceful shutdown before forcing termination on timeout. For development and testing, see the Toolchain local test guide.

3.2.3 - Graph import

Choose an import tool when you need to write file, database, or message data into HugeGraph. Use Loader for a direct import, SeaTunnel Sink when an existing Source, Transform, and Sink pipeline should be reused, or Spark Connector from a Spark job.

3.2.3.1 - Import Graph Data with SeaTunnel Sink

SeaTunnel connects data sources such as databases and Kafka to HugeGraph. The connector has two parts: Source reads data and Sink writes data[1][2], with SeaTunnel transform components available between them. To export or migrate data from HugeGraph, see the SeaTunnel Source export and migration guide.

Version requirement: This guide targets SeaTunnel 3.0+. All examples use the mappings configuration available in SeaTunnel 3.0+.

Loader imports data directly with graph mappings; SeaTunnel 3.0+ combines Source, Transform, and Sink, and both support JDBC, Kafka, and graph data

Click a diagram to view the original size.

1 Loader, Tools, and SeaTunnel

HugeGraph-Loader is suited to direct imports from common data sources. HugeGraph-Tools focuses on standalone graph management, backup, and export. SeaTunnel organizes a job as Source → Transform → Sink, so you can reuse existing connectors, transforms, and data pipelines.

Table legend

✅ Supported natively; ⚠️ conditional support or requires an extra component/external platform; ❌ not provided

ComparisonLoaderToolsSeaTunnel
Task coverage✅ Direct graph imports✅ Backup, restore, and export✅ Import, export, and migration with composable Source, Transform, and Sink stages
Job configurationJSON mapping file describing the source, vertices, and edgesCommand-line options and operationsHOCON job file[3] combining Source, Transform, and Sink
Default deployment✅ Standalone CLI; ⚠️ Spark Loader can extend it✅ Standalone CLI✅ Standalone; ✅ distributed
Execution engine⚠️ Mainly CLI; Spark Loader is a separate extension❌ Does not provide a Spark/Flink execution engine✅ HugeGraph Source and Sink support Zeta, Spark, and Flink[1][2][7][8][9][10]
Frontend and observability❌ No built-in frontend; inspect CLI logs❌ No built-in frontend; inspect CLI logs✅ Built-in Web UI job panel for task status and runtime information
Input and output⚠️ Focused on graph imports and common files, JDBC, Kafka, and similar sources⚠️ Focused on graph data and backup files in common storage✅ Dozens of connectors, including JDBC, Kafka, and SQL-CDC
Scheduling and resource management❌ No unified cross-task scheduling or resource allocation❌ No unified cross-task scheduling or resource allocation⚠️ Can integrate with DolphinScheduler for scheduling and task management
Simplicity✅ Focused and simple; a future binary CLI will make quick use easier✅ Direct commands for standalone operations⚠️ More runtime components, suited to long-lived data pipelines
High-throughput import✅ Supports bypass-server and other optimizations; measured peaks can reach 1-2 million records/s with specific backends and hardware, so benchmark the actual setup⚠️ Focuses on backup and export rather than bulk-import throughput✅ Scales throughput through parallelism, distributed engines, and connectors

SeaTunnel covers Loader’s graph-import and Tools’ export and migration scenarios in one expandable pipeline, and it also supports SQL-CDC and dozens of input and output types. Loader and Tools normally run on one machine, while SeaTunnel supports both standalone and distributed deployments and scales with data and task volume. Tools’ schedule-backup can create a crontab entry, but it does not provide unified workflow orchestration and resource management.

Existing Spark/Flink daily jobs

Both HugeGraph Source and Sink list SeaTunnel Engine (Zeta), Spark, and Flink as supported engines in SeaTunnel 3.0+. If you express the daily job as a SeaTunnel job and submit it to that engine, records can move directly from Source to Transform to Sink without an intermediate file. If you keep the existing Spark/Flink DAG, SeaTunnel does not automatically take over its in-memory DataFrame or stream. Adapt it into a SeaTunnel job or expose the data through a Source connector

Loader and Tools are focused, direct, and quick to start. Use Loader for a direct graph import; use Tools for backup, restore, export, or daily operations. If a SeaTunnel job already exists, adding HugeGraph to that pipeline is usually simpler. For higher import throughput, Loader’s bypass-server path and other import optimizations are a better fit; measured peaks of 1-2 million records/s require a specific backend, data set, and hardware configuration and are not a general performance guarantee. For new SeaTunnel jobs, use 3.0+ and mappings. Recheck the connector configuration when using another version.

2 Prepare the environment

2.1 Get SeaTunnel 3.0+

Use JDK 11 and set JAVA_HOME; the official deployment guide lists Java 8 and 11 as prerequisites. Choose a 3.0+ binary distribution from the official release page[4] and verify its checksum or signature using the files linked there. The commands below use 3.0.0 as an example; set SEATUNNEL_VERSION to the release you downloaded.

export SEATUNNEL_VERSION=3.0.0
tar -xzf "apache-seatunnel-${SEATUNNEL_VERSION}-bin.tar.gz"
cd "apache-seatunnel-${SEATUNNEL_VERSION}"

The binary distribution does not include connector plugins. In config/plugin_config, select the connectors needed by the examples:

--seatunnel-connectors--
connector-hugegraph
connector-jdbc
connector-kafka
--end--

Install the matching released plugins using the upstream plugin installation instructions[5]:

sh bin/install-plugin.sh "${SEATUNNEL_VERSION}"

Run the remaining commands from this installation directory and keep the engine and connector plugins at the same version. This guide uses the bundled Zeta engine in local mode[6][7]. Check that connectors/ contains HugeGraph, JDBC, and Kafka as needed[11][12]. The JDBC examples also require the MySQL driver JAR in lib/, with driver class com.mysql.cj.jdbc.Driver.

2.2 Prepare HugeGraph and data sources

Start HugeGraph Server and create a graph for testing. The examples use the hugegraph graph in the DEFAULT graph space. Adjust these names to match the server configuration; graph space names are case-sensitive. If authentication is enabled, provide username and password in the HugeGraph Source and Sink configurations.

The following graph model is shared by the JDBC and Kafka examples. mappings creates missing PropertyKey, VertexLabel, and EdgeLabel definitions by default; existing schema definitions must be compatible.

Graph elementName and properties
Propertiesname is Text; age and since are Int
Vertexperson, primary key name, properties name and age
Edgeknows, from person to person, property since

The mysql, kafka, and hugegraph host names in the examples are placeholders. Replace them with addresses reachable from the SeaTunnel runtime. Inside a container, 127.0.0.1 points to that container; services on the same Docker network can use their service names. Set host to a host name or IP address, and set the port separately.

3 Import from a relational database (sql2graph)

Use two jobs for this import: write the person table as vertices first, then write the knows table as edges. Both edge endpoints will already exist when the edge job runs.

The person table creates marko and vadas vertices; the knows table creates a directed edge with since 2010 through endpoint fields

3.1 Import vertices

Prepare the sample data in the MySQL demo database and grant the configured account read access:

CREATE TABLE person (
  name VARCHAR(64) PRIMARY KEY,
  age INT NOT NULL
);
INSERT INTO person VALUES ('marko', 29), ('vadas', 27);

Save the following as config/sql2graph-person.conf and replace the database user name and password:

env {
  job.mode = "BATCH"
}

source {
  Jdbc {
    url = "jdbc:mysql://mysql:3306/demo?useSSL=false&serverTimezone=UTC"
    driver = "com.mysql.cj.jdbc.Driver"
    username = "seatunnel"
    password = "change_me"
    query = "SELECT name, age FROM person ORDER BY name"
  }
}

sink {
  HugeGraph {
    host = "hugegraph"
    port = 8080
    graph_name = "hugegraph"
    graph_space = "DEFAULT"
    batch_failure_fallback = false
    mappings = [
      {
        type = "VERTEX"
        label = "person"
        idStrategy = "PRIMARY_KEY"
        idFields = ["name"]
        properties = ["name", "age"]
      }
    ]
  }
}
./bin/seatunnel.sh --config ./config/sql2graph-person.conf -m local

Check the result in Hubble or Gremlin. You should find marko and vadas with their ages:

g.V().hasLabel('person').valueMap('name', 'age')

idFields = ["name"] uses the name to generate the primary key. Importing the same name again writes to the same vertex. properties lists the source fields to write.

3.2 Import edges

Prepare the relation table. Its two endpoint fields correspond to person.name from the vertex job:

CREATE TABLE knows (
  source_name VARCHAR(64) NOT NULL,
  target_name VARCHAR(64) NOT NULL,
  since INT NOT NULL
);
INSERT INTO knows VALUES ('marko', 'vadas', 2010);
Expand the configuration and save it as config/sql2graph-knows.conf
env {
  job.mode = "BATCH"
}

source {
  Jdbc {
    url = "jdbc:mysql://mysql:3306/demo?useSSL=false&serverTimezone=UTC"
    driver = "com.mysql.cj.jdbc.Driver"
    username = "seatunnel"
    password = "change_me"
    query = "SELECT source_name, target_name, since FROM knows ORDER BY source_name, target_name"
  }
}

sink {
  HugeGraph {
    host = "hugegraph"
    port = 8080
    graph_name = "hugegraph"
    graph_space = "DEFAULT"
    batch_failure_fallback = false
    check_vertex = true
    mappings = [
      {
        type = "EDGE"
        label = "knows"
        sourceConfig = {
          label = "person"
          idFields = ["source_name"]
        }
        targetConfig = {
          label = "person"
          idFields = ["target_name"]
        }
        fieldMapping = {
          source_name = "name"
          target_name = "name"
        }
        properties = ["since"]
      }
    ]
  }
}

After the vertex job succeeds, run the edge job:

./bin/seatunnel.sh --config ./config/sql2graph-knows.conf -m local

The following query should return a knows edge from marko to vadas with since set to 2010:

g.V().has('person', 'name', 'marko').outE('knows').where(inV().has('name', 'vadas')).valueMap()

sourceConfig and targetConfig identify the endpoint fields. fieldMapping maps them to the vertex primary key name, and properties = ["since"] writes only the edge property. The example enables check_vertex = true and disables per-record fallback after a batch failure (batch_failure_fallback = false), so a missing endpoint or write failure causes the job to fail.

If the relation table has only numeric foreign keys while the graph uses names as primary keys, join the names in SQL before passing the records to the Sink. See MySQL CDC Source[13] for MySQL CDC integration.

4 Import from Kafka (kafka2graph)

Kafka is useful for a continuous stream of events. Create the user-events topic and publish the following JSON message. Each message becomes one person vertex:

{"name":"marko","age":29}

Save the following as config/kafka2graph.conf:

env {
  job.mode = "STREAMING"
  checkpoint.interval = 10000
  sink.flush.interval = 5000
}

source {
  Kafka {
    bootstrap.servers = "kafka:9092"
    topic = "user-events"
    consumer.group = "hugegraph-import"
    start_mode = "earliest"
    format = "json"
    schema = {
      fields = {
        name = "string"
        age = "int"
      }
    }
  }
}

sink {
  HugeGraph {
    host = "hugegraph"
    port = 8080
    graph_name = "hugegraph"
    graph_space = "DEFAULT"
    batch_failure_fallback = false
    mappings = [
      {
        type = "VERTEX"
        label = "person"
        idStrategy = "PRIMARY_KEY"
        idFields = ["name"]
        properties = ["name", "age"]
      }
    ]
  }
}
./bin/seatunnel.sh --config ./config/kafka2graph.conf -m local

Use the Gremlin query from section 3.1 to check the data. The streaming job keeps running. checkpoint.interval saves job state every 10 seconds, while sink.flush.interval asks Zeta to flush every 5 seconds so a small number of messages does not wait for a full batch.

HugeGraph Sink writes with at-least-once semantics, so recovery can replay records. PRIMARY_KEY sends the same name to the same vertex, but it does not make every update exactly-once. Scheduled flushing is provided by Zeta and does not apply to Spark or Flink engines.

5 Common configuration and troubleshooting

The following table applies to the SeaTunnel 3.0+ version used by this guide:

ConfigurationPurpose
host, portSet the HugeGraph host and port
graph_name, graph_spaceSelect an existing graph and graph space
mappingsDefine how input fields become vertices or edges
propertiesList the source fields written by each mapping
schema_save_modemappings creates missing schema by default; existing schema must still be compatible
batch_sizeNumber of records per batch; default 500
env.sink.flush.intervalZeta scheduled flush interval in milliseconds
check_vertexCheck edge endpoints; the edge job in this guide sets it to true
batch_failure_fallback[2]Defaults to true, so a failed batch falls back to record-by-record retries, capped by max_insert_errors; the examples explicitly set false so a batch failure stops the job
max_insert_errorsNumber of failed records that record-by-record fallback may skip; default 500, -1 for unlimited, and only applies when batch_failure_fallback is enabled

Use these checks when a job fails:

  • mappings is unknown or HugeGraph Source is missing: Check that the engine and HugeGraph connector come from the same SeaTunnel 3.0+ build.
  • Connection failure: Check the host, port, graph space, authentication details, and whether the SeaTunnel runtime can reach the service.
  • Schema incompatibility: Check the ID strategy, property types, and edge endpoints. Automatic creation does not change an existing PRIMARY_KEY label into CUSTOMIZE_STRING.
  • Small Kafka batches do not appear promptly: Confirm that the job uses Zeta and set sink.flush.interval in env. In this version, batch_interval_ms is retained only for compatibility and cannot replace it.

6 Choosing a tool

Choose a tool based on the work to complete. Use Tools for graph management, Gremlin, backup, or cloning. Use Loader for a direct graph import. Choose SeaTunnel when you need to reuse a Source, Transform, and Sink pipeline. For SeaTunnel graph reads and migrations, prepare the environment using the SeaTunnel 3.0+ version used by this guide.

Choosing a tool: Tools for graph management, Loader for direct imports, and SeaTunnel for reusable data pipelines

7 References

HugeGraph connectors

[1] HugeGraph Source
[2] HugeGraph Sink

Configuration and deployment

[3] HOCON job configuration
[4] SeaTunnel 3.0+ release download
[5] Connector plugin installation
[6] SeaTunnel local deployment

Execution engines

[7] SeaTunnel Engine Overview
[8] SeaTunnel Spark Engine
[9] SeaTunnel Flink Engine
[10] Connector V2 multi-engine support

Data source connectors

[11] JDBC Source
[12] Kafka Source
[13] MySQL CDC Source

3.2.4 - HugeGraph-Loader Quick Start

Bulk import graph data into HugeGraph with Loader from files, HDFS, relational databases, Kafka, and other HugeGraph graphs.

This guide follows Toolchain master (currently 1.8.0). Released packages and Docker latest may differ from source; check the version you use and build from source for unreleased functionality.

1 HugeGraph-Loader Overview

HugeGraph-Loader is the data import component of HugeGraph, which can convert data from various data sources into graph vertices and edges and import them into the graph database in batches.

Currently supported data sources include:

  • Local disk file or directory, supports TEXT, CSV and JSON format files, supports compressed files
  • HDFS file or directory supports compressed files
  • Mainstream relational databases, such as MySQL, PostgreSQL, Oracle, SQL Server
  • Kafka topic
  • An existing HugeGraph graph, used to copy data from one graph into another

Local disk files and HDFS files support resumable uploads.

It will be explained in detail below.

Note: HugeGraph-Loader requires HugeGraph Server service, please refer to HugeGraph-Server Quick Start to download and start Server

Testing Guide: For running HugeGraph-Loader tests locally, please refer to HugeGraph Toolchain Local Testing Guide

2 Get HugeGraph-Loader

HugeGraph-Loader is available in the following three ways:

  • Use docker image (Convenient for Test/Dev)
  • Download the compiled tarball
  • Clone source code then compile and install

2.1 Use Docker image (Convenient for Test/Dev)

We can deploy the loader service using docker run -itd --name loader hugegraph/loader:latest. For the data that needs to be loaded, it can be copied into the loader container either by mounting -v /path/to/data/file:/loader/file or by using docker cp.

Alternatively, to start the loader using docker-compose, the command is docker-compose up -d. An example of the docker-compose.yml is as follows:

This combination is for local anonymous testing; the Server API is published only on the local host.

version: '3'

services:
  server:
    image: hugegraph/hugegraph:latest
    container_name: server
    ports:
      - 127.0.0.1:8080:8080

  loader:
    image: hugegraph/loader:latest
    container_name: loader
    # mount your own data here
    # volumes:
      # - /path/to/data/file:/loader/file

The specific data loading process can be referenced under 4.5 User Docker to load data

Note:

  1. The docker image of hugegraph-loader is a convenience release to start hugegraph-loader quickly, but not official distribution artifacts. You can find more details from ASF Release Distribution Policy.

  2. Pin a published version tag or image digest in production. latest is mutable and does not guarantee alignment with master.

2.2 Download the compiled archive

Choose a published Toolchain archive from the download page and extract it. A release may not include all master functionality; use the source build below for the implementation described here.

2.3 Clone source code to compile and install

Clone the master branch:

git clone --branch master --single-branch https://github.com/apache/hugegraph-toolchain.git

Compile and generate tar package:

cd hugegraph-toolchain
mvn clean package -pl hugegraph-loader -am -DskipTests -ntp

For Oracle input, obtain a JDBC driver JAR compatible with your Oracle/JDK versions and place it in the extracted Loader lib/ directory. The startup script loads JARs there; installing a driver only in the local Maven repository does not put it on Loader’s runtime classpath.

3 How to use

The basic process of using HugeGraph-Loader is divided into the following steps:

  • Write graph schema
  • Prepare data files
  • Write input source map files
  • Execute command import

3.1 Construct graph schema

This step is the modeling process. Users need to have a clear idea of ​​their existing data and the graph model they want to create, and then write the schema to build the graph model.

For example, if you want to create a graph with two types of vertices and two types of edges, the vertices are “people” and “software”, the edges are “people know people” and “people create software”, and these vertices and edges have some attributes, For example, the vertex “person” has: “name”, “age” and other attributes, “Software” includes: “name”, “sale price” and other attributes; side “knowledge” includes: “date” attribute and so on.

Example graph with person and software vertices connected by knows and created edges

graph model example

After designing the graph model, we can use groovy to write the definition of schema and save it to a file, here named schema.groovy.

// Create some properties
schema.propertyKey("name").asText().ifNotExist().create();
schema.propertyKey("age").asInt().ifNotExist().create();
schema.propertyKey("city").asText().ifNotExist().create();
schema.propertyKey("date").asText().ifNotExist().create();
schema.propertyKey("price").asDouble().ifNotExist().create();

// Create the person vertex type, which has three attributes: name, age, city, and the primary key is name
schema.vertexLabel("person").properties("name", "age", "city").primaryKeys("name").ifNotExist().create();
// Create a software vertex type, which has two properties: name, price, the primary key is name
schema.vertexLabel("software").properties("name", "price").primaryKeys("name").ifNotExist().create();

// Create the knows edge type, which goes from person to person
schema.edgeLabel("knows").sourceLabel("person").targetLabel("person").properties("date").ifNotExist().create();
// Create the created edge type, which points from person to software
schema.edgeLabel("created").sourceLabel("person").targetLabel("software").ifNotExist().create();

Please refer to the corresponding section in hugegraph-client for the detailed description of the schema.

3.2 Prepare data

The data sources currently supported by HugeGraph-Loader include:

  • local disk file or directory
  • HDFS file or directory
  • Partial relational database
  • Kafka topic
  • An existing HugeGraph graph
3.2.1 Data source structure
3.2.1.1 Local disk file or directory

The user can specify a local disk file as the data source. If the data is scattered in multiple files, a certain directory is also supported as the data source, but multiple directories are not supported as the data source for the time being.

For example, my data is scattered in multiple files, part-0, part-1 … part-n. To perform the import, it must be ensured that they are placed in one directory. Then in the loader’s mapping file, specify path as the directory.

Supported file formats include:

  • TEXT
  • CSV
  • JSON

TEXT is a text file with custom delimiters, the first line is usually the header, and the name of each column is recorded, and no header line is allowed (specified in the mapping file). Each remaining row represents a record, which will be converted into a vertex/edge; each column of the row corresponds to a field, which will be converted into the id, label or attribute of the vertex/edge;

An example is as follows:

id|name|lang|price|ISBN
1|lop|java|328|ISBN978-7-107-18618-5
2|ripple|java|199|ISBN978-7-100-13678-5

CSV is a TEXT file with commas , as delimiters. When a column value itself contains a comma, the column value needs to be enclosed in double quotes, for example:

marko,29,Beijing
"li,nary",26,"Wu,han"

The JSON file requires that each line is a JSON string, and the format of each line needs to be consistent.

{"source_name": "marko", "target_name": "vadas", "date": "20160110", "weight": 0.5}
{"source_name": "marko", "target_name": "josh", "date": "20130220", "weight": 1.0}
3.2.1.2 HDFS file or directory

Users can also specify HDFS files or directories as data sources, all of the above requirements for local disk files or directories apply here. In addition, since HDFS usually stores compressed files, loader also provides support for compressed files, and local disk file or directory also supports compressed files.

Currently supported compressed file types include: GZIP, BZ2, XZ, LZMA, SNAPPY_RAW, SNAPPY_FRAMED, Z, DEFLATE, LZ4_BLOCK, LZ4_FRAMED, ORC, and PARQUET.

3.2.1.3 Mainstream relational database

The loader also supports some relational databases as data sources, and currently supports MySQL, PostgreSQL, Oracle, and SQL Server.

However, the requirements for the table structure are relatively strict at present. If association query needs to be done during the import process, such a table structure is not allowed. The associated query means: after reading a row of the table, it is found that the value of a certain column cannot be used directly (such as a foreign key), and you need to do another query to determine the true value of the column.

For example, Suppose there are three tables, person, software and created

// person schema
id | name | age | city
// software schema
id | name | lang | price
// created schema
id | p_id | s_id | date

If the id strategy of person or software is specified as PRIMARY_KEY when modeling (schema), choose name as the primary key (note: this is the concept of vertex-label in hugegraph), when importing edge data, the source vertex and target need to be spliced ​​out. For the id of the vertex, you must go to the person/software table with p_id/s_id to find the corresponding name. In the case of the schema that requires additional query, the loader does not support it temporarily. In this case, the following two methods can be used instead:

  1. The id strategy of person and software is still specified as PRIMARY_KEY, but the id column of the person table and software table is used as the primary key attribute of the vertex, so that the id can be generated by directly splicing p_id and s_id with the label of the vertex when importing an edge;
  2. Specify the id policy of person and software as CUSTOMIZE, and then directly use the id column of the person table and the software table as the vertex id, so that p_id and s_id can be used directly when importing edges;

The key point is to make the edge use p_id and s_id directly, don’t check it again.

3.2.2 Prepare vertex and edge data
3.2.2.1 Vertex Data

The vertex data file consists of data line by line. Generally, each line is used as a vertex, and each column is used as a vertex attribute. The following description uses CSV format as an example.

  • person vertex data (the data itself does not contain a header)
Tom,48,Beijing
Jerry,36,Shanghai
  • software vertex data (the data itself contains the header)
name,price
Photoshop,999
Office,388
3.2.2.2 Edge data

The edge data file consists of data line by line. Generally, each line is used as an edge. Some columns are used as the IDs of the source and target vertices, and other columns are used as edge attributes. The following uses JSON format as an example.

  • knows edge data
{"source_name": "Tom", "target_name": "Jerry", "date": "2008-12-12"}
  • created edge data
{"source_name": "Tom", "target_name": "Photoshop"}
{"source_name": "Tom", "target_name": "Office"}
{"source_name": "Jerry", "target_name": "Office"}

3.3 Write data source mapping file

3.3.1 Mapping file overview

The mapping file of the input source is used to describe how to establish the mapping relationship between the input source data and the vertex type/edge type of the graph. It is organized in JSON format and consists of multiple mapping blocks, each of which is responsible for mapping an input source. Mapped to vertices and edges.

Specifically, each mapping block contains an input source and multiple vertex mapping and edge mapping blocks, and the input source block corresponds to the local disk file or directory, HDFS file or directory and relational database are responsible for describing the basic information of the data source, such as where the data is, what format, what is the delimiter, etc. The vertex map/edge map is bound to the input source, which columns of the input source can be selected, which columns are used as ids, which columns are used as attributes, and what attributes are mapped to each column, the values ​​of the columns are mapped to what values ​​of attributes, and so on.

In the simplest terms, each mapping block describes: where is the file to be imported, which type of vertices/edges each line of the file is to be used as which columns of the file need to be imported, and the corresponding vertices/edges of these columns. what properties, etc.

Note: The format of the mapping file before version 0.11.0 and the format after 0.11.0 has changed greatly. For the convenience of expression, the mapping file (format) before 0.11.0 is called version 1.0, and the version after 0.11.0 is version 2.0. And unless otherwise specified, the “map file” refers to version 2.0.

Click to expand/collapse the skeleton of the map file for version 2.0
{
  "version": "2.0",
  "structs": [
    {
      "id": "1",
      "input": {
      },
      "vertices": [
        {},
        {}
      ],
      "edges": [
        {},
        {}
      ]
    }
  ]
}

Two versions of the mapping file are given directly here (the above graph model and data file are described)

Click to expand/collapse the mapping file for version 2.0
{
  "version": "2.0",
  "structs": [
    {
      "id": "1",
      "skip": false,
      "input": {
        "type": "FILE",
        "path": "vertex_person.csv",
        "file_filter": {
          "extensions": [
            "*"
          ]
        },
        "format": "CSV",
        "delimiter": ",",
        "date_format": "yyyy-MM-dd HH:mm:ss",
        "time_zone": "GMT+8",
        "skipped_line": {
          "regex": "(^#|^//).*|"
        },
        "compression": "NONE",
        "header": [
          "name",
          "age",
          "city"
        ],
        "charset": "UTF-8",
        "list_format": {
          "start_symbol": "[",
          "elem_delimiter": "|",
          "end_symbol": "]"
        }
      },
      "vertices": [
        {
          "label": "person",
          "skip": false,
          "id": null,
          "unfold": false,
          "field_mapping": {},
          "value_mapping": {},
          "selected": [],
          "ignored": [],
          "null_values": [
            ""
          ],
          "update_strategies": {}
        }
      ],
      "edges": []
    },
    {
      "id": "2",
      "skip": false,
      "input": {
        "type": "FILE",
        "path": "vertex_software.csv",
        "file_filter": {
          "extensions": [
            "*"
          ]
        },
        "format": "CSV",
        "delimiter": ",",
        "date_format": "yyyy-MM-dd HH:mm:ss",
        "time_zone": "GMT+8",
        "skipped_line": {
          "regex": "(^#|^//).*|"
        },
        "compression": "NONE",
        "header": null,
        "charset": "UTF-8",
        "list_format": {
          "start_symbol": "",
          "elem_delimiter": ",",
          "end_symbol": ""
        }
      },
      "vertices": [
        {
          "label": "software",
          "skip": false,
          "id": null,
          "unfold": false,
          "field_mapping": {},
          "value_mapping": {},
          "selected": [],
          "ignored": [],
          "null_values": [
            ""
          ],
          "update_strategies": {}
        }
      ],
      "edges": []
    },
    {
      "id": "3",
      "skip": false,
      "input": {
        "type": "FILE",
        "path": "edge_knows.json",
        "file_filter": {
          "extensions": [
            "*"
          ]
        },
        "format": "JSON",
        "delimiter": null,
        "date_format": "yyyy-MM-dd HH:mm:ss",
        "time_zone": "GMT+8",
        "skipped_line": {
          "regex": "(^#|^//).*|"
        },
        "compression": "NONE",
        "header": null,
        "charset": "UTF-8",
        "list_format": null
      },
      "vertices": [],
      "edges": [
        {
          "label": "knows",
          "skip": false,
          "source": [
            "source_name"
          ],
          "unfold_source": false,
          "target": [
            "target_name"
          ],
          "unfold_target": false,
          "field_mapping": {
            "source_name": "name",
            "target_name": "name"
          },
          "value_mapping": {},
          "selected": [],
          "ignored": [],
          "null_values": [
            ""
          ],
          "update_strategies": {}
        }
      ]
    },
    {
      "id": "4",
      "skip": false,
      "input": {
        "type": "FILE",
        "path": "edge_created.json",
        "file_filter": {
          "extensions": [
            "*"
          ]
        },
        "format": "JSON",
        "delimiter": null,
        "date_format": "yyyy-MM-dd HH:mm:ss",
        "time_zone": "GMT+8",
        "skipped_line": {
          "regex": "(^#|^//).*|"
        },
        "compression": "NONE",
        "header": null,
        "charset": "UTF-8",
        "list_format": null
      },
      "vertices": [],
      "edges": [
        {
          "label": "created",
          "skip": false,
          "source": [
            "source_name"
          ],
          "unfold_source": false,
          "target": [
            "target_name"
          ],
          "unfold_target": false,
          "field_mapping": {
            "source_name": "name",
            "target_name": "name"
          },
          "value_mapping": {},
          "selected": [],
          "ignored": [],
          "null_values": [
            ""
          ],
          "update_strategies": {}
        }
      ]
    }
  ]
}

Click to expand/collapse the mapping file for version 1.0
{
  "vertices": [
    {
      "label": "person",
      "input": {
        "type": "file",
        "path": "vertex_person.csv",
        "format": "CSV",
        "header": ["name", "age", "city"],
        "charset": "UTF-8"
      }
    },
    {
      "label": "software",
      "input": {
        "type": "file",
        "path": "vertex_software.csv",
        "format": "CSV"
      }
    }
  ],
  "edges": [
    {
      "label": "knows",
      "source": ["source_name"],
      "target": ["target_name"],
      "input": {
        "type": "file",
        "path": "edge_knows.json",
        "format": "JSON"
      },
      "field_mapping": {
        "source_name": "name",
        "target_name": "name"
      }
    },
    {
      "label": "created",
      "source": ["source_name"],
      "target": ["target_name"],
      "input": {
        "type": "file",
        "path": "edge_created.json",
        "format": "JSON"
      },
      "field_mapping": {
        "source_name": "name",
        "target_name": "name"
      }
    }
  ]
}

The 1.0 version of the mapping file is centered on the vertex and edge, and sets the input source; while the 2.0 version is centered on the input source, and sets the vertex and edge mapping. Some input sources (such as a file) can generate both vertices and edges. If you write in the 1.0 format, you need to write an input block in each of the vertex and edge mapping blocks. The two input blocks are exactly the same; and the 2.0 version only needs to write input once. Therefore, compared with version 1.0, version 2.0 can save some repetitive writing of input.

In the bin directory of hugegraph-loader-{version}, there is a script tool mapping-convert.sh that can directly convert the mapping file of version 1.0 to version 2.0. The usage is as follows:

bin/mapping-convert.sh struct.json

A struct-v2.json will be generated in the same directory as struct.json.

The bin directory also ships utf8-bom-to-utf8.sh, which strips the UTF-8 BOM from a single data file, or from every file under a directory. It is useful when a CSV or TEXT file exported by a Windows tool fails to parse because its first header column carries an invisible BOM:

bin/utf8-bom-to-utf8.sh /path/to/file-or-dir
3.3.2 Input Source

Input sources are currently divided into five categories: FILE, HDFS, JDBC, KAFKA and GRAPH, which are distinguished by the type node. We call them local file input sources, HDFS input sources, JDBC input sources, KAFKA input sources and GRAPH input source, which are described below.

3.3.2.1 Local file input source
  • id: The id of the input source. This field is used to support some internal functions. It is not required (it will be automatically generated if it is not filled in). It is strongly recommended to write it, which is very helpful for debugging;
  • skip: whether to skip the input source, because the JSON file cannot add comments, if you do not want to import an input source during a certain import, but do not want to delete the configuration of the input source, you can set it to true to skip it, the default is false, not required;
  • input: input source map block, composite structure
    • type: an input source type, file or FILE must be filled;
    • path: the path of the local file or directory, an absolute path or a path relative to the Loader process working directory (not the mapping file). An absolute path is recommended; required;
    • file_filter: filter files with compound conditions from path, compound structure, currently only supports configuration extensions, represented by child node extensions, the default is “*”, which means to keep all files;
    • format: the format of the local file, the optional values are CSV, TEXT and JSON, which must be uppercase, the default is CSV, optional;
    • header: column names; if omitted, the first data line supplies them. With an explicit header, an identical first line is still skipped by default (see has_header). JSON does not require a header; optional;
    • has_header: for CSV and TEXT, the first line of every file is dropped when it is identical to the header, so a header repeated in each part file of a directory is not imported as data. Set this to false to turn that check off when the first line of a file is real data that happens to equal the header, optional;
    • delimiter: The column delimiter of the file line. The default depends on format: a comma "," for CSV and a tab "\t" for TEXT; a CSV file accepts no other delimiter. The JSON file does not need to be specified, optional;
    • charset: the encoded character set of the file, the default is UTF-8, optional;
    • date_format: custom date format, the default value is yyyy-MM-dd HH:mm:ss, optional; if the date is presented in the form of a timestamp, this item must be written as timestamp (fixed writing);
    • extra_date_formats: a list of fallback date formats, tried when a value does not match date_format, empty by default, optional;
    • time_zone: Set which time zone the date data is in, the default value is GMT+8, optional;
    • skipped_line: The line to be skipped, compound structure, currently only the regular expression of the line to be skipped can be configured, described by the child node regex. The default regex is (^#|^//).*|, which skips lines starting with # or // and empty lines; to keep such lines, set regex to a pattern that matches nothing, optional;
    • compression: The compression format of the file, the optional values ​​are NONE, GZIP, BZ2, XZ, LZMA, SNAPPY_RAW, SNAPPY_FRAMED, Z, DEFLATE, LZ4_BLOCK, LZ4_FRAMED, ORC and PARQUET, the default is NONE, which means a non-compressed file, optional; for ORC and PARQUET the header is matched case-insensitively;
    • list_format: When a column of the file (non-JSON) is a collection structure (the Cardinality of the PropertyKey in the corresponding figure is Set or List), you can use this item to set the start character, separator, and end character of the column, compound structure :
      • start_symbol: The start character of the collection structure column (the default value is the empty string "", JSON format currently does not support specification)
      • elem_delimiter: the delimiter of the collection structure column (the default value is |, and it must differ from delimiter; JSON format currently only supports native , delimiter)
      • end_symbol: the end character of the collection structure column (the default value is the empty string "", the JSON format does not currently support specification)
      • ignored_elems: the elements dropped after the column is split, the default value is [""], so empty elements are ignored
3.3.2.2 HDFS input source

The nodes and meanings of the above local file input source are basically applicable here. Only the different and unique nodes of the HDFS input source are listed below.

  • type: input source type, must fill in hdfs or HDFS, required;
  • path: the path of the HDFS file or directory, it must be the absolute path of HDFS, required;
  • core_site_path: the path of the core-site.xml file of the HDFS cluster, the key point is to specify the address of the NameNode (fs.default.name) and the implementation of the file system (fs.hdfs.impl), required;
  • hdfs_site_path: the path of the hdfs-site.xml file of the HDFS cluster, optional;
  • dir_filter: when path is a directory, decides which sub-directories are walked into, compound structure, optional:
    • include_regex: only directories whose name matches this regular expression are read, empty by default, which places no restriction;
    • exclude_regex: directories whose name matches this regular expression are skipped, empty by default;
  • kerberos_config: how to authenticate against a Kerberos-secured HDFS cluster, compound structure, optional:
    • enable: whether to authenticate with Kerberos, the default is false;
    • krb5_conf: the path of the krb5.conf file, required when enable is true;
    • principal: the Kerberos principal, required when enable is true;
    • keytab: the path of the keytab file, required when enable is true;
3.3.2.3 JDBC input source

As mentioned above, it supports multiple relational databases, but because their mapping structures are very similar, they are collectively referred to as JDBC input sources, and then use the vendor node to distinguish different databases.

  • type: input source type, must fill in jdbc or JDBC, required;
  • vendor: database type, optional options are [MySQL, PostgreSQL, Oracle, SQLServer], case-insensitive, required;
  • driver: the JDBC driver class, optional; when it is left out, the default driver of the vendor listed in the tables below is used;
  • url: the url of the database that jdbc wants to connect to, required;
  • database: the name of the database to be connected, required;
  • schema: The name of the schema to be connected, different databases have different requirements, and the details are explained below;
  • table: the name of the table to be connected, at least one of table or custom_sql is required;
  • custom_sql: custom SQL statement, at least one of table or custom_sql is required;
  • username: username to connect to the database, required;
  • password: password for connecting to the database, required;
  • where: an extra condition appended to the generated select statement, written without the where keyword, optional;
  • batch_size: The size of one page when obtaining table data by page, the default is 500, optional;

MYSQL

NodeFixed value or common value
vendorMYSQL
drivercom.mysql.cj.jdbc.Driver
urljdbc:mysql://127.0.0.1:3306

schema: nullable, if filled in, it must be the same as the value of database

POSTGRESQL

NodeFixed value or common value
vendorPOSTGRESQL
driverorg.postgresql.Driver
urljdbc:postgresql://127.0.0.1:5432

schema: nullable, default is “public”

ORACLE

NodeFixed value or common value
vendorORACLE
driveroracle.jdbc.driver.OracleDriver
urljdbc:oracle:thin:@127.0.0.1:1521

schema: nullable, the default value is the username in upper case

SQLSERVER

NodeFixed value or common value
vendorSQLSERVER
drivercom.microsoft.sqlserver.jdbc.SQLServerDriver
urljdbc:sqlserver://127.0.0.1:1433

schema: required

3.3.2.4 Kafka input source
  • type: input source type, kafka or KAFKA, required;
  • bootstrap_server: the list of kafka bootstrap servers, required;
  • topic: the topic to subscribe to, required;
  • group: group of Kafka consumers, required;
  • from_beginning: sets auto.offset.reset to earliest when true, or latest when false (default). It applies only when no valid committed offset exists; otherwise consumption resumes from the committed offset. Optional;
  • format: format of each message, options are CSV, TEXT and JSON, must be uppercase, required;
  • header: column name of each column of a message; no header line is read from the topic, so it has to be given for CSV and TEXT, while JSON messages do not need it;
  • delimiter: delimiter of the message columns, used by TEXT only, since CSV always splits on ,, optional;
  • charset: encoding charset of the messages, default is UTF-8, optional;
  • date_format: customized date format, default value is yyyy-MM-dd HH:mm:ss, optional; if the date is presented in the form of timestamp, this item must be written as timestamp (fixed);
  • extra_date_formats: a customized list of another date formats, empty by default, optional; each item in the list is an alternate date format to the date_format specified date format;
  • time_zone: set which time zone the date data is in, default is GMT+8, optional;
  • skipped_line: current master KafkaReader does not implement this filter. Kafka messages go directly to the parser, so configuring this field does not skip blank or comment messages; preprocess them upstream or in the consumer to avoid parse failures;
  • batch_size: the maximum number of records fetched in one poll (max.poll.records), default is 500, optional;
  • early_stop: the record pulled from Kafka broker at a certain time is empty, stop the task, default is false, only for debugging, optional;
3.3.2.5 GRAPH input Source

The GRAPH input source reads vertices and edges out of another HugeGraph graph, reached through HugeGraph-PD, and writes them into the target graph. When a mapping file contains a GRAPH input source, every input source in it that is not skipped has to be a GRAPH input source as well, and the loader puts the target graph into RESTORING mode for the duration of the import.

  • type: Data source type; must be filled in as graph or GRAPH (required);
  • graphspace: Source graphSpace name (required);
  • graph: Source graph name (required);
  • username: HugeGraph username; the --username command-line option is used when this is empty;
  • password: HugeGraph password; the --password command-line option is used when this is empty;
  • selected_vertices: the vertex labels to copy, each item written as {"label": "...", "properties": [...], "query": {...}}, where properties narrows the copied properties and query is an optional filter passed to the source graph;
  • ignored_vertices: the vertex labels to skip, each item written as {"label": "...", "properties": [...]};
  • selected_edges: the edge labels to copy, items have the same shape as in selected_vertices;
  • ignored_edges: the edge labels to skip, items have the same shape as in ignored_vertices;
  • pd-peers: HugeGraph-PD node addresses of the source cluster; the --pd-peers option is used when this is empty;
  • meta-endpoints: Meta service endpoints of the source cluster; the --meta-endpoints option is used when this is empty;
  • cluster: Source cluster name; the --cluster option is used when this is empty;
  • batch_size: Batch size for reading data from the source graph; default is 500;
3.3.3 Vertex and Edge Mapping

The nodes of vertex and edge mapping (a key in the JSON file) have a lot of the same parts. The same parts are introduced first, and then the unique nodes of vertex map and edge map are introduced respectively.

Nodes of the same section

  • label: label to which the vertex/edge data to be imported belongs, required;
  • skip: whether to skip this vertex/edge mapping while the input source and the other mappings stay active, the default is false, optional;
  • field_mapping: Map the column name of the input source column to the attribute name of the vertex/edge, optional;
  • value_mapping: map the data value of the input source to the attribute value of the vertex/edge, optional;
  • selected: select some columns to insert, other unselected ones are not inserted, cannot exist at the same time as ignored, optional;
  • ignored: ignore some columns so that they do not participate in insertion, cannot exist at the same time as selected, optional;
  • null_values: You can specify some strings to represent null values, such as “NULL”. If the vertex/edge attribute corresponding to this column is also a nullable attribute, the value of this attribute will not be set when constructing the vertex/edge, optional ;
  • update_strategies: If the data needs to be updated in batches in a specific way, you can specify a specific update strategy for each attribute (see below for details), optional;
  • unfold: Whether to unfold the column, each unfolded column will form a row with other columns, which is equivalent to unfolding into multiple rows; for example, the value of a certain column (id column) of the file is [1,2,3], The values ​​of other columns are 18,Beijing. When unfold is set, this row will become 3 rows, namely: 1,18,Beijing, 2,18,Beijing and 3,18, Beijing. Note that this will only expand the column selected as id. Default false, optional;

Update strategy supports 8 types: (requires all uppercase)

  1. Value accumulation: SUM
  2. Take the greater of the two numbers/dates: BIGGER
  3. Take the smaller of two numbers/dates: SMALLER
  4. Set property takes union: UNION
  5. Set attribute intersection: INTERSECTION
  6. List attribute append element: APPEND
  7. List/Set attribute delete element: ELIMINATE
  8. Override an existing property: OVERRIDE

Note: If the newly imported attribute value is empty, the existing old data will be used instead of the empty value. For the effect, please refer to the following example

// The update strategy is specified in the JSON file as follows
{
  "vertices": [
    {
      "label": "person",
      "update_strategies": {
        "age": "SMALLER",
        "set": "UNION"
      },
      "input": {
        "type": "file",
        "path": "vertex_person.txt",
        "format": "TEXT",
        "header": ["name", "age", "set"]
      }
    }
  ]
}

// 1. Write a line of data with the OVERRIDE update strategy (null means empty here)
'a b null null'

// 2. Write another line
'null null c d'

// 3. Finally we can get
'a b c d'   

// If there is no update strategy, you will get
'null null c d'

Note : After adopting the batch update strategy, the number of disk read requests will increase significantly, and the import speed will be several times slower than that of pure write coverage (at this time HDD disk [IOPS](https://en.wikipedia .org/wiki/IOPS) will be the bottleneck, SSD is recommended for speed)

Unique Nodes for Vertex Maps

  • id: Specify a column as the id column of the vertex. When the vertex id policy is CUSTOMIZE, it is required; when the id policy is PRIMARY_KEY, it must be empty;

Unique Nodes for Edge Maps

  • source: Select certain columns of the input source as the id column of source vertex. When the id policy of the source vertex is CUSTOMIZE, a certain column must be specified as the id column of the vertex; when the id policy of the source vertex is When PRIMARY_KEY, one or more columns must be specified for splicing the id of the generated vertex, that is, no matter which id strategy is used, this item is required;
  • target: Specify certain columns as the id columns of target vertex, similar to source, so I won’t repeat them;
  • unfold_source: Whether to unfold the source column of the file, the effect is similar to that in the vertex map, and will not be repeated;
  • unfold_target: Whether to unfold the target column of the file, the effect is similar to that in the vertex mapping, and will not be repeated;

3.4 Execute command import

After preparing the graph model, data file, and input source mapping relationship file, the data file can be imported into the graph database.

The import process is controlled by commands submitted by the user, and the user can control the specific process of execution through different parameters.

3.4.1 Parameter description
ParameterDefault valueRequired or notDescription
-f or --fileYPath to configure script
-g or --graphhugegraphGraph name
--graphspaceDEFAULTGraph space name
-s or --schemaSchema file path; optional when the Schema already exists
-h or --host or -ilocalhostAddress of HugeGraphServer
-p or --port8080Port number of HugeGraphServer
--usernamenullWhen HugeGraphServer enables permission authentication, the username of the current graph
--passwordnullWhen HugeGraphServer enables permission authentication, the password of the current graph
--create-graphfalseWhether to automatically create the graph if it does not exist
--tokennullWhen HugeGraphServer has enabled authorization authentication, the token of the current graph
--protocolhttpProtocol for sending requests to the server, optional http or https
--pd-peersPD service node addresses
--pd-tokenToken for accessing PD service
--meta-endpointsMeta information storage service addresses
--directfalseStore direct mode is disabled in current master; imports still use Server API. Keep the default
--route-typeNODE_PORTRoute selection method (optional values: NODE_PORT / DDS / BOTH)
--clusterhgCluster name
--trust-store-fileWhen the request protocol is https, the client’s certificate file path
--trust-store-passwordWhen the request protocol is https, the client certificate password
--clear-all-datafalseWhether to clear the original data on the server before importing data
--clear-timeout240Timeout for clearing the original data on the server before importing data
--incremental-modefalseWhether to use the breakpoint resume mode; only input sources FILE and HDFS support this mode. Enabling this mode allows starting the import from where the last import stopped
--failure-modefalseWhen failure mode is true, previously failed data will be imported. Generally, the failed data file needs to be manually corrected and edited before re-importing
--batch-insert-threadsCPUsBatch insert thread pool size (CPUs is the number of logical cores available to the current OS)
--single-insert-threads8Size of single insert thread pool
--max-conn4 * CPUsThe maximum number of HTTP connections between HugeClient and HugeGraphServer; while it is left at its default, it is raised automatically to 4 * --batch-insert-threads
--max-conn-per-route2 * CPUsThe maximum number of HTTP connections for each route between HugeClient and HugeGraphServer; while it is left at its default, it is raised automatically to 2 * --batch-insert-threads
--batch-size500The number of data items in each batch when importing data
--max-parse-errors1The maximum number of data parsing errors allowed (per line); the program exits when this value is reached
--max-insert-errors500The maximum number of data insertion errors allowed (per row); the program exits when this value is reached
--timeout60Timeout (seconds) for insert result return
--shutdown-timeout10Waiting time for multithreading to stop (seconds)
--retry-times3Maximum number of retries after a timeout
--retry-interval10Interval before retry (seconds)
--check-vertexfalseWhether to check if the vertices connected by the edge exist when inserting the edge
--print-progresstrueWhether to print the number of imported items in real time on the console
--dry-runfalseEnable this mode to only parse data without importing; usually used for testing
--help or -helpfalsePrint help information
--parser-threads or --parallel-countmax(2,CPUs/2)Number of parallel read pipelines; --parallel-count is deprecated
--start-file0Start file index for partial loading
--end-file-1End file index for partial loading
--scatter-sourcesfalseScatter multiple sources for I/O optimization
--cdc-flush-interval30000The flush interval for Flink CDC
--cdc-sink-parallelism1The sink parallelism for Flink CDC
--max-read-errors1The maximum number of read error lines before exiting
--max-read-lines-1LThe maximum number of read lines, task stops when reached
--test-modefalseWhether the loader works in test mode
--use-prefilterfalseWhether to filter vertex in advance
--short-idMap a customized vertex ID to a shorter generated ID, written as label:field:type, where type is one of boolean, byte, int, long, float, double, text, blob, date and uuid; repeat the option to cover several labels
--vertex-edge-limit-1LThe maximum number of vertex’s edges
--sink-typetruespark-loader only: true writes through the HugeGraph server API, false generates HFiles and bulk-loads them into HBase
--vertex-partitions64The number of partitions of the HBase vertex table, used with --sink-type false
--edge-partitions64The number of partitions of the HBase edge table, used with --sink-type false
--vertex-table-nameHBase vertex table name, used with --sink-type false
--edge-table-nameHBase edge table name, used with --sink-type false
--hbase-zk-quorumHBase ZooKeeper quorum, used with --sink-type false
--hbase-zk-portHBase ZooKeeper port, used with --sink-type false
--hbase-zk-parentHBase ZooKeeper parent, used with --sink-type false
--restorefalseSet graph mode to RESTORING
--backendhstoreThe backend store type when creating graph if not exists
--serializerbinaryThe serializer type when creating graph if not exists
--scheduler-typedistributedThe task scheduler type when creating graph if not exists
--batch-failure-fallbacktrueWhether to fallback to single insert when batch insert fails

The loader prints its usage and exits when it is given fewer than three arguments, so -f struct.json on its own is not enough.

3.4.2 Breakpoint Continuation Mode

Usually, the Loader task takes a long time to execute. If the import interrupt process exits for some reason, and next time you want to continue the import from the interrupted point, this is the scenario of using breakpoint continuation.

The user sets the command line parameter –incremental-mode to true to open the breakpoint resume mode. The key to breakpoint continuation lies in the progress file. When the import process exits, the import progress at the time of exit will be recorded. Recorded in the progress file, the progress file is located in the ${struct} directory, the file name is like load-progress_${timestamp}, ${struct} is the prefix of the mapping file, and ${timestamp} is the start of the import moment, formatted as yyyyMMdd-HHmmss. For example, for an import task started at 2019-10-10 12:30:30, the mapping file used is struct-example.json, then the path of the progress file is the same as struct-example.json Sibling struct-example/load-progress_20191010-123030. When the directory holds several progress files, the resumed import reads the last one in name order, which is the most recent one.

Note: The generation of progress files is independent of whether –incremental-mode is turned on or not, and a progress file is generated at the end of each import.

If the data file formats are all legal and the import task is stopped by the user (CTRL + C or kill, kill -9 is not supported), that is to say, if there is no error record, the next import only needs to be set to Continue for the breakpoint.

But if the limit of –max-read-errors, –max-parse-errors or –max-insert-errors is reached because too much data is invalid or network abnormality is reached, Loader will record these original rows that failed into the failure file, after the user modifies the data lines in the failure file, set –failure-mode to true to import these “failure files” as input sources (does not affect the normal file import), Of course, if there is still a problem with the modified data line, it will be logged again to the failure file (don’t worry about duplicate lines, they are dropped when the file is closed). Failure mode lifts the three error limits above, so the whole failure file is scanned.

Each input source, that is each item of structs in the mapping file, gets its own failure file. The file is named after the id of that input source with the suffix .error and is stored in the ${struct}/failure-data directory. Every failed line is written as a pair of lines: a tip line starting with #### READ ERROR:, #### PARSE ERROR: or #### INSERT ERROR:, followed by the original data line. When the input source has a header, that header is written as JSON to a sibling ${id}.header file, so the failure file can be read back with the right columns. For example, if the mapping file has an input source with id 1 holding a vertex mapping person and an input source with id 3 holding an edge mapping knows, each of which has some error lines, you will see the following files in the ${struct}/failure-data directory when the Loader exits:

  • 1.error: the failed lines of input source 1, each preceded by its tip line
  • 1.header: the header of input source 1, written only when that input source has a header
  • 3.error: the failed lines of input source 3
  • 3.header: the header of input source 3

A .error file that turns out to be empty is deleted when the Loader exits, so only the input sources that really had failed lines leave a file behind. In incremental mode new failures are appended to the existing file, otherwise the file is rewritten from scratch.

3.4.3 logs directory file description

The log and error data during program execution will be written into the hugegraph-loader.log file.

3.4.4 Execute command

Run bin/hugegraph-loader.sh and pass in parameters

bin/hugegraph-loader.sh -g {GRAPH_NAME} -f ${INPUT_DESC_FILE} -s ${SCHEMA_FILE} -h {HOST} -p {PORT}

The script runs the JVM under JAVA_HOME when that variable is set, and java from the PATH otherwise. It passes the contents of the JVM_OPTS environment variable, then -Xmx10g and the class path built from lib/, to that JVM, so JVM_OPTS is the place to add JVM flags. Logging is configured by conf/log4j2.xml.

3.4.5 Minimal Local-File Import

First start HugeGraph Server and ensure the current user can access hugegraph in graphspace DEFAULT, create schema, and write data. If authentication is enabled, add the appropriate --username, --password, or --token. This example requires only local files, without Kafka, HDFS, or JDBC.

From the extracted Loader installation directory, create data, schema, and a version 2.0 mapping file:

mkdir -p demo-loader

cat > demo-loader/people.csv <<'EOF'
docs-smoke-alice,29
docs-smoke-bob,31
EOF

cat > demo-loader/schema.groovy <<'EOF'
schema.propertyKey("docs_name").asText().ifNotExist().create()
schema.propertyKey("docs_age").asInt().ifNotExist().create()
schema.vertexLabel("docs_smoke_person").properties("docs_name", "docs_age").primaryKeys("docs_name").ifNotExist().create()
EOF

cat > demo-loader/struct.json <<'EOF'
{
  "version": "2.0",
  "structs": [
    {
      "id": "people",
      "input": {
        "type": "FILE",
        "path": "demo-loader/people.csv",
        "format": "CSV",
        "header": ["docs_name", "docs_age"]
      },
      "vertices": [
        {"label": "docs_smoke_person"}
      ],
      "edges": []
    }
  ]
}
EOF

The CSV has no header row; input.header in the mapping supplies column names. input.path is relative to the Loader process working directory, so keep running from the installation directory. If Server runs in another container, set --host to a Server address reachable by Loader:

bin/hugegraph-loader.sh \
  --graphspace DEFAULT --graph hugegraph \
  --file demo-loader/struct.json --schema demo-loader/schema.groovy \
  --host 127.0.0.1 --port 8080

Expected result: two vertices and zero edges imported. This example uses dedicated schema names to avoid conflicts with a preloaded person label. In Hubble, run g.V().hasLabel('docs_smoke_person').values('docs_name') to confirm docs-smoke-alice and docs-smoke-bob.

4 Complete example

Given below is an example in the example directory of the hugegraph-loader package. (GitHub address)

4.1 Prepare data

Vertex file: example/file/vertex_person.csv

marko,29,Beijing
vadas,27,Hongkong
josh,32,Beijing
peter,35,Shanghai
"li,nary",26,"Wu,han"
tom,null,NULL

Vertex file: example/file/vertex_software.txt

id|name|lang|price|ISBN
1|lop|java|328|ISBN978-7-107-18618-5
2|ripple|java|199|ISBN978-7-100-13678-5

Edge file: example/file/edge_knows.json

{"source_name": "marko", "target_name": "vadas", "date": "20160110", "weight": 0.5}
{"source_name": "marko", "target_name": "josh", "date": "20130220", "weight": 1.0}

Edge file: example/file/edge_created.json

{"source_name": "marko", "target_id": 1, "date": "2017-12-10", "weight": 0.4}
{"source_name": "josh", "target_id": 1, "date": "2009-11-11", "weight": 0.4}
{"source_name": "josh", "target_id": 2, "date": "2017-12-10", "weight": 1.0}
{"source_name": "peter", "target_id": 1, "date": "2017-03-24", "weight": 0.2}

4.2 Write schema

Click to expand/collapse the schema file: example/file/schema.groovy
schema.propertyKey("name").asText().ifNotExist().create();
schema.propertyKey("age").asInt().ifNotExist().create();
schema.propertyKey("city").asText().ifNotExist().create();
schema.propertyKey("weight").asDouble().ifNotExist().create();
schema.propertyKey("lang").asText().ifNotExist().create();
schema.propertyKey("date").asText().ifNotExist().create();
schema.propertyKey("price").asDouble().ifNotExist().create();

schema.vertexLabel("person").properties("name", "age", "city").primaryKeys("name").nullableKeys("age", "city").ifNotExist().create();
schema.vertexLabel("software").useCustomizeNumberId().properties("name", "lang", "price").ifNotExist().create();

schema.indexLabel("personByAge").onV("person").by("age").range().ifNotExist().create();
schema.indexLabel("personByCity").onV("person").by("city").secondary().ifNotExist().create();
schema.indexLabel("personByAgeAndCity").onV("person").by("age", "city").secondary().ifNotExist().create();
schema.indexLabel("softwareByPrice").onV("software").by("price").range().ifNotExist().create();

schema.edgeLabel("knows").sourceLabel("person").targetLabel("person").properties("date", "weight").ifNotExist().create();
schema.edgeLabel("created").sourceLabel("person").targetLabel("software").properties("date", "weight").ifNotExist().create();

schema.indexLabel("createdByDate").onE("created").by("date").secondary().ifNotExist().create();
schema.indexLabel("createdByWeight").onE("created").by("weight").range().ifNotExist().create();
schema.indexLabel("knowsByWeight").onE("knows").by("weight").range().ifNotExist().create();

person uses name as its primary key. Tom’s age and city are removed by null_values, so both properties must be nullable. software uses custom numeric IDs: mapping id supplies the vertex ID and created.target_id supplies the edge’s target ID. Run this example in an empty graph without these schema labels; ifNotExist() does not replace incompatible existing definitions.

4.3 Write the input source mapping file example/file/struct.json

Click to expand/collapse the input source mapping file example/file/struct.json
{
  "vertices": [
    {
      "label": "person",
      "input": {
        "type": "file",
        "path": "example/file/vertex_person.csv",
        "format": "CSV",
        "header": ["name", "age", "city"],
        "charset": "UTF-8",
        "skipped_line": {
          "regex": "(^#|^//).*"
        }
      },
      "null_values": ["NULL", "null", ""]
    },
    {
      "label": "software",
      "input": {
        "type": "file",
        "path": "example/file/vertex_software.txt",
        "format": "TEXT",
        "delimiter": "|",
        "charset": "GBK"
      },
      "id": "id",
      "ignored": ["ISBN"]
    }
  ],
  "edges": [
    {
      "label": "knows",
      "source": ["source_name"],
      "target": ["target_name"],
      "input": {
        "type": "file",
        "path": "example/file/edge_knows.json",
        "format": "JSON",
        "date_format": "yyyyMMdd"
      },
      "field_mapping": {
        "source_name": "name",
        "target_name": "name"
      }
    },
    {
      "label": "created",
      "source": ["source_name"],
      "target": ["target_id"],
      "input": {
        "type": "file",
        "path": "example/file/edge_created.json",
        "format": "JSON",
        "date_format": "yyyy-MM-dd"
      },
      "field_mapping": {
        "source_name": "name"
      }
    }
  ]
}

4.4 Command to import

sh bin/hugegraph-loader.sh -g hugegraph -f example/file/struct.json -s example/file/schema.groovy

After the import is complete, statistics similar to the following will appear:

vertices/edges has been loaded this time : 8/6
--------------------------------------------------
count metrics
     input read success            : 14
     input read failure            : 0
     vertex parse success          : 8
     vertex parse failure          : 0
     vertex insert success         : 8
     vertex insert failure         : 0
     edge parse success            : 6
     edge parse failure            : 0
     edge insert success           : 6
     edge insert failure           : 0

4.5 Use Docker to load data

4.5.1 Use docker exec to load data directly
4.5.1.1 Prepare data

If you just want to try out the loader, you can import the built-in example dataset without needing to prepare additional data yourself.

If using custom data, before importing data with the loader, we need to copy the data into the container.

First, following the steps in 4.1–4.3, we can prepare the data and then use docker cp to copy the prepared data into the loader container.

Suppose we’ve prepared the corresponding dataset following the above steps, stored in the hugegraph-dataset folder with the following file structure:

tree -f hugegraph-dataset/

hugegraph-dataset
├── hugegraph-dataset/edge_created.json
├── hugegraph-dataset/edge_knows.json
├── hugegraph-dataset/schema.groovy
├── hugegraph-dataset/struct.json
├── hugegraph-dataset/vertex_person.csv
└── hugegraph-dataset/vertex_software.txt

Copy the files into the container.

docker cp hugegraph-dataset loader:/loader/dataset
docker exec -it loader ls /loader/dataset

edge_created.json  edge_knows.json  schema.groovy  struct.json  vertex_person.csv  vertex_software.txt
4.5.1.2 Data loading

Taking the built-in example dataset as an example, we can use the following command to load the data.

If you need to import your custom dataset, you need to modify the paths for -f (data script) and -s (schema) configurations.

You can refer to 3.4.1-Parameter description for the rest of the parameters.

docker exec -it loader bin/hugegraph-loader.sh -g hugegraph -f example/file/struct.json -s example/file/schema.groovy -h server -p 8080

If loading a custom dataset, following the previous example, you would use:

docker exec -it loader bin/hugegraph-loader.sh -g hugegraph -f /loader/dataset/struct.json -s /loader/dataset/schema.groovy -h server -p 8080

Also update every input.path in the custom struct.json to its actual container path, such as /loader/dataset/vertex_person.csv. Changing only -f and -s does not change where Loader looks for data files.

If loader and server are in the same Docker network, you can specify -h {server_container_name}; otherwise, you need to specify the IP of the server host (in our example, server_container_name is server).

Then we can see the result:

HugeGraphLoader worked in NORMAL MODE
vertices/edges loaded this time : 8/6
--------------------------------------------------
count metrics
    input read success            : 14                  
    input read failure            : 0                   
    vertex parse success          : 8                   
    vertex parse failure          : 0                   
    vertex insert success         : 8                   
    vertex insert failure         : 0                   
    edge parse success            : 6                   
    edge parse failure            : 0                   
    edge insert success           : 6                   
    edge insert failure           : 0                   
--------------------------------------------------
meter metrics
    total time                    : 0.199s              
    read time                     : 0.046s              
    load time                     : 0.153s              
    vertex load time              : 0.077s              
    vertex load rate(vertices/s)  : 103                 
    edge load time                : 0.112s              
    edge load rate(edges/s)       : 53   

You can also use curl or hubble to observe the import result. Here’s an example using curl:

These requests use the GraphSpace-aware Server API. For an older Server, omit /graphspaces/DEFAULT from the path.

curl --compressed "http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices"
{"vertices":[{"id":1,"label":"software","type":"vertex","properties":{"name":"lop","lang":"java","price":328.0}},{"id":2,"label":"software","type":"vertex","properties":{"name":"ripple","lang":"java","price":199.0}},{"id":"1:tom","label":"person","type":"vertex","properties":{"name":"tom"}},{"id":"1:josh","label":"person","type":"vertex","properties":{"name":"josh","age":32,"city":"Beijing"}},{"id":"1:marko","label":"person","type":"vertex","properties":{"name":"marko","age":29,"city":"Beijing"}},{"id":"1:peter","label":"person","type":"vertex","properties":{"name":"peter","age":35,"city":"Shanghai"}},{"id":"1:vadas","label":"person","type":"vertex","properties":{"name":"vadas","age":27,"city":"Hongkong"}},{"id":"1:li,nary","label":"person","type":"vertex","properties":{"name":"li,nary","age":26,"city":"Wu,han"}}]}

If you want to check the import result of edges, you can use curl --compressed "http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/edges".

4.5.2 Enter the docker container to load data

Besides using docker exec directly for data import, we can also enter the container for data loading. The basic process is similar to 4.5.1.

Enter the container by docker exec -it loader bash and execute the command:

sh bin/hugegraph-loader.sh -g hugegraph -f example/file/struct.json -s example/file/schema.groovy -h server -p 8080

The results of the execution will be similar to those shown in 4.5.1.

4.6 Import data by spark-loader

The current source uses Spark 3.2.2 and Scala 2.12. Other combinations need independent verification.

The parameters of spark-loader are divided into two parts. Note: Because the abbreviations of these two-parameter names have overlapping parts, please use the full name of the parameter. And there is no need to guarantee the order between the two parameters.

Example:

sh bin/hugegraph-spark-loader.sh --master yarn \
--deploy-mode cluster --name spark-hugegraph-loader --file ./hugegraph.json \
--username admin --token admin --host xx.xx.xx.xx --port 8093 \
--graph graph-test --num-executors 6 --executor-cores 16 --executor-memory 15g

bin/hugegraph-spark-loader.sh submits org.apache.hugegraph.loader.spark.HugeGraphSparkLoader through ${SPARK_HOME}/bin/spark-submit, so SPARK_HOME has to point at a Spark installation. Every jar under lib/ is put on the class path. The Spark application name defaults to hugegraph-spark-loader and can be changed through the APP_NAME environment variable.

bin/get-params.sh splits the command line: only the options below are handed to the loader, every other argument is passed to spark-submit unchanged. The splitter matches long option names only, so short forms such as -f and -g are not recognised.

--graph --schema --host --port --username --token --protocol
--trust-store-file --trust-store-password --clear-all-data --clear-timeout
--incremental-mode --failure-mode --batch-insert-threads --single-insert-threads
--max-conn --max-conn-per-route --batch-size --max-parse-errors --max-insert-errors
--timeout --shutdown-timeout --retry-times --retry-interval --check-vertex
--print-progress --dry-run --sink-type --vertex-partitions --edge-partitions --help

--file is treated separately: with --deploy-mode cluster the mapping file is shipped to the executors through --files and the loader receives only its base name, otherwise the path is passed through as written.

In this mode the loader reads FILE, HDFS and JDBC input sources; a KAFKA or GRAPH input source is rejected.

By default (--sink-type true) each Spark partition opens a HugeClient and writes vertices and edges through the HugeGraph server API. With --sink-type false the loader generates HFiles and bulk-loads them into HBase instead, taking the table names and ZooKeeper settings from --vertex-table-name, --edge-table-name, --hbase-zk-quorum, --hbase-zk-port, --hbase-zk-parent, --vertex-partitions and --edge-partitions.

The current source uses Flink 1.13.5 with flink-connector-mysql-cdc 2.2.1 and Scala 2.12. Other combinations need independent verification.

bin/hugegraph-flinkcdc-loader.sh submits org.apache.hugegraph.loader.flink.HugeGraphFlinkCDCLoader through ${FLINK_HOME}/bin/flink run, so FLINK_HOME has to be set. The job captures MySQL change events with Flink CDC and applies them to the graph, which keeps the graph in step with the source tables.

The mapping file uses the same format as for the command-line loader, but every input source has to be a JDBC input source over MySQL: the loader takes url, database, table, username and password from it and parses the host and port out of url. Vertex and edge mappings work as usual. --cdc-flush-interval and --cdc-sink-parallelism in 3.4.1 apply to this mode only.

The command line is split by bin/get-params.sh in the same way as for spark-loader, with the arguments that are not loader options going to flink run.

Example:

sh bin/hugegraph-flinkcdc-loader.sh --file ./mysql-cdc.json \
--host xx.xx.xx.xx --port 8080 --graph hugegraph --username admin --token admin

3.2.5 - Graph export and migration

Choose an export or migration tool when you need to back up, export, or move data between graphs. Tools is suited to standalone operations and backups; SeaTunnel Source connects graph reads to an expandable data pipeline.

3.2.5.1 - Export and Migrate Graph Data with SeaTunnel Source

If you need to copy data from one HugeGraph graph to another, use graph2graph: HugeGraph Source reads vertices and edges from the source graph (graph A), and HugeGraph Sink writes them to the target graph (graph B), with an optional Transform in between. The data path is graph A → HugeGraph Source → (optional Transform) → HugeGraph Sink → graph B. If you need to export graph data to a file, JDBC, Kafka, or another system, use graph2any, where a downstream Sink receives the records read by HugeGraph Source. This page covers both job types.

Version requirement: This guide targets SeaTunnel 3.0+

SeaTunnel 3.0+ provides HugeGraph Source, with schema auto-discovery and multi-label reads. See the official Source documentation for parallel-scan backend requirements and configuration limits.

Before starting, complete the shared environment and configuration steps on the import page. They cover JDK, HOCON, plugin installation, and the sample graph model.

1 Migrate a HugeGraph graph (graph2graph)

The following example migrates person vertices and knows edges from a source graph. Use a separate target graph. This section uses CUSTOMIZE_STRING to preserve vertex IDs. Do not reuse the person label created earlier with PRIMARY_KEY.

These two jobs migrate only the selected labels and properties. They do not copy every source schema setting, such as indexes and TTLs. Pause writes to the source graph during the migration so both jobs read a consistent point in time. Afterward, compare vertex and edge counts and sample properties.

Regenerating a primary key can change 1:marko to 2:marko; CUSTOMIZE_STRING preserves the original ID so edge endpoints still resolve

1.1 Migrate vertices first

Source adds a ~id column for the original ID, and Sink stores it as a string. Do not declare ~id in schema.fields; manually declaring this reserved column is rejected.

Expand the configuration and save it as config/graph2graph-person.conf
env {
  job.mode = "BATCH"
}

source {
  HugeGraph {
    host = "source-hugegraph"
    port = 8080
    graph_name = "hugegraph"
    graph_space = "DEFAULT"
    label = "person"
    label_type = "VERTEX"
    schema = {
      fields = {
        name = "string"
        age = "int"
      }
    }
  }
}

sink {
  HugeGraph {
    host = "target-hugegraph"
    port = 8080
    graph_name = "hugegraph"
    graph_space = "DEFAULT"
    batch_failure_fallback = false
    mappings = [
      {
        type = "VERTEX"
        label = "person"
        idStrategy = "CUSTOMIZE_STRING"
        idFields = ["~id"]
        properties = ["name", "age"]
      }
    ]
  }
}
./bin/seatunnel.sh --config ./config/graph2graph-person.conf -m local

1.2 Migrate edges second

After the vertex job succeeds, use the ~source_id and ~target_id columns added by Source to locate endpoints. Because the previous job preserved the original IDs, these columns can refer directly to vertices in the target graph.

Expand the configuration and save it as config/graph2graph-knows.conf
env {
  job.mode = "BATCH"
}

source {
  HugeGraph {
    host = "source-hugegraph"
    port = 8080
    graph_name = "hugegraph"
    graph_space = "DEFAULT"
    label = "knows"
    label_type = "EDGE"
    schema = {
      fields = {
        since = "int"
      }
    }
  }
}

sink {
  HugeGraph {
    host = "target-hugegraph"
    port = 8080
    graph_name = "hugegraph"
    graph_space = "DEFAULT"
    check_vertex = true
    batch_failure_fallback = false
    mappings = [
      {
        type = "EDGE"
        label = "knows"
        sourceConfig = {
          label = "person"
          idFields = ["~source_id"]
        }
        targetConfig = {
          label = "person"
          idFields = ["~target_id"]
        }
        properties = ["since"]
      }
    ]
  }
}
./bin/seatunnel.sh --config ./config/graph2graph-knows.conf -m local

This example checks endpoints and makes write errors fail the job. The default check_vertex = false does not guarantee a consistent result: a missing endpoint can create a dangling edge, so a successful job is not a substitute for checking the migrated graph.

Why preserve IDs? A HugeGraph PRIMARY_KEY ID contains the internal ID of the vertex label, and that internal ID can differ between graphs. For example, a source vertex can be 1:marko, while regenerating the primary key in the target graph can produce 2:marko. Reusing the source edge endpoints after regenerating vertex IDs can connect edges to the wrong vertices. This example stores the original ID as a string, which changes the target graph’s ID strategy

When Source reads every label, omit label to read all labels of label_type (default VERTEX). It produces one output table per label. Bind each Sink mapping to its table with sourceTable, for example sourceTable = "default.person"; use the full table name shown in the Writer log for the exact value. Do not reuse the single-label configuration from this section. See the HugeGraph Source documentation for other limitations.

2 Export to another system (graph2any)

graph2any uses HugeGraph Source[1] to read vertices or edges and sends them to a downstream Sink. The example below exports person vertices to local JSON files; to export to JDBC, Kafka, or another system, replace LocalFile[2] and its options.

env {
  job.mode = "BATCH"
}

source {
  HugeGraph {
    host = "hugegraph"
    port = 8080
    graph_name = "hugegraph"
    graph_space = "DEFAULT"
    label = "person"
    label_type = "VERTEX"
    schema = {
      fields = {
        name = "string"
        age = "int"
      }
    }
  }
}

sink {
  LocalFile {
    path = "/tmp/hugegraph-export/${table_name}"
    file_format_type = "json"
  }
}

Save this as config/graph2file-person.conf and run it from the SeaTunnel installation directory:

./bin/seatunnel.sh --config ./config/graph2file-person.conf -m local

To export edges, change the Source label to an edge label, set label_type = "EDGE", and declare the edge properties in schema.fields. Source also outputs the reserved columns ~source_id, ~source_label, ~target_id, and ~target_label; write them to the file or pass them to downstream transforms as needed.

This page covers row reads and writes. It does not copy source indexes, TTLs, or other schema settings. For the complete Source options and shared environment guidance, return to the SeaTunnel graph import guide[3].

3 References

Connectors

[1] HugeGraph Source
[2] LocalFile Sink

Related guide

[3] SeaTunnel graph import guide

3.2.6 - Tools Quick Start

1 HugeGraph-Tools Overview

HugeGraph-Tools is an automated deployment, management and backup/restore component of HugeGraph.

Testing Guide: For running HugeGraph-Tools tests locally, please refer to HugeGraph Toolchain Local Testing Guide

2 Get HugeGraph-Tools

HugeGraph-Tools is included in the Toolchain distribution. You can download the distribution or build it from source.

  • Download the compiled tarball
  • Clone source code then compile and install

2.1 Download the compiled archive

Download the latest version of the HugeGraph-Toolchain package:

export VERSION=1.7.0
export ARCHIVE="apache-hugegraph-toolchain-incubating-${VERSION}"
wget "https://downloads.apache.org/hugegraph/${VERSION}/${ARCHIVE}.tar.gz"
tar zxf "${ARCHIVE}.tar.gz"
# hugegraph-tools ships inside the toolchain package, in a directory
# whose version suffix is the same as the archive's
cd "${ARCHIVE}/apache-hugegraph-tools-incubating-${VERSION}"

2.2 Clone source code to compile and install

Please ensure that the wget command is installed before compiling the source code

Download the latest version of the HugeGraph-Tools source package:

# 1. get from github
git clone https://github.com/apache/hugegraph-toolchain.git

# 2. Download a released source package
export VERSION=1.7.0
export ARCHIVE="apache-hugegraph-toolchain-incubating-${VERSION}"
wget "https://downloads.apache.org/hugegraph/${VERSION}/${ARCHIVE}-src.tar.gz"

Compile and generate tar package:

cd hugegraph-toolchain
mvn package -pl hugegraph-tools -am -DskipTests -ntp

The package is generated as hugegraph-tools/target/apache-hugegraph-tools-${version}.tar.gz, and the unpacked directory hugegraph-tools/apache-hugegraph-tools-${version} (containing bin/ and lib/) is created next to it.

3 How to use

3.1 Function overview

After decompression, enter the apache-hugegraph-tools-${version} directory, you can use bin/hugegraph or bin/hugegraph help to view the usage information, and bin/hugegraph help <sub-command> to view the usage of a single sub-command. mainly divided:

  • Graph management type, graph-mode-set, graph-mode-get, graph-list, graph-get, graph-clear, graph-create, graph-clone and graph-drop
  • Asynchronous task management type, task-list, task-get, task-delete, task-cancel and task-clear
  • Gremlin type, gremlin-execute and gremlin-schedule
  • Backup/Restore type, backup, restore, migrate, schedule-backup and dump
  • Authentication data backup/restore type, auth-backup and auth-restore
  • Install deployment type, deploy, clear, start-all and stop-all
Usage: hugegraph [options] [command] [command options]
3.2 [options]-Global Variable

options is a global variable of HugeGraph-Tools, which can be configured in hugegraph-tools/bin/hugegraph, including:

  • –graph,HugeGraph-Tools The name of the graph to operate on, the default value is hugegraph
  • –url,The service address of HugeGraph-Server, the default is http://127.0.0.1:8080
  • –user,When HugeGraph-Server opens authentication, pass username
  • –password,When HugeGraph-Server opens authentication, pass the user’s password
  • –timeout,Timeout when connecting to HugeGraph-Server, the default is 30s
  • –trust-store-file,The path of the certificate file, when –url uses https, the truststore file used by HugeGraph-Client, the default is empty, which means using the built-in truststore file conf/hugegraph.truststore of hugegraph-tools
  • –trust-store-password,The password of the certificate file, when –url uses https, the password of the truststore used by HugeGraph-Client, the default is empty, representing the password of the built-in truststore file of hugegraph-tools
  • –throw-mode, whether HugeGraph-Tools throws the exception instead of printing the error message and exiting, the default is false (mainly used by tests)

The protocol is taken from the scheme of –url: use https://... to connect over https. –trust-store-file and –trust-store-password can only be set when –url uses https, and both –user and –password must be given together or omitted together.

The above global variables can also be set through environment variables. One way is to use export on the command line to set temporary environment variables, which are valid until the command line is closed

Global VariableEnvironment VariableExample
–urlHUGEGRAPH_URLexport HUGEGRAPH_URL=http://127.0.0.1:8080
–graphHUGEGRAPH_GRAPHexport HUGEGRAPH_GRAPH=hugegraph
–userHUGEGRAPH_USERNAMEexport HUGEGRAPH_USERNAME=admin
–passwordHUGEGRAPH_PASSWORDexport HUGEGRAPH_PASSWORD=test
–timeoutHUGEGRAPH_TIMEOUTexport HUGEGRAPH_TIMEOUT=30
–trust-store-fileHUGEGRAPH_TRUST_STORE_FILEexport HUGEGRAPH_TRUST_STORE_FILE=/tmp/trust-store
–trust-store-passwordHUGEGRAPH_TRUST_STORE_PASSWORDexport HUGEGRAPH_TRUST_STORE_PASSWORD=xxxx

Another way is to set the environment variable in the bin/hugegraph script:

#!/bin/bash

# Set environment here if needed
#export HUGEGRAPH_URL=
#export HUGEGRAPH_GRAPH=
#export HUGEGRAPH_USERNAME=
#export HUGEGRAPH_PASSWORD=
#export HUGEGRAPH_TIMEOUT=
#export HUGEGRAPH_TRUST_STORE_FILE=
#export HUGEGRAPH_TRUST_STORE_PASSWORD=

bin/hugegraph also reads JAVA_HOME (a warning is printed when it is not set, and it is needed for https) and JAVA_OPTIONS (JVM options; when it is empty the script uses -Xms512m plus an -Xmx computed from the free memory of the machine).

3.3 Graph Management Type, graph-mode-set, graph-mode-get, graph-list, graph-get, graph-clear, graph-create, graph-clone and graph-drop
  • graph-mode-set, set graph restore mode
    • –graph-mode or -m, required, specifies the mode to be set, legal values include [NONE, RESTORING, MERGING, LOADING]
  • graph-mode-get, get graph restore mode
  • graph-list, list all graphs in a HugeGraph-Server
  • graph-get, get a graph and its storage backend type
  • graph-clear, clear all schema and data of a graph
    • –confirm-message or -c, required, delete confirmation information, manual input is required, double confirmation to prevent accidental deletion, “I’m sure to delete all data”, including double quotes
  • graph-create, create a new graph with configuration file
    • –name or -n, optional, the name of the new graph, default is g
    • –file or -f, the path to the graph configuration file, the content of the file is sent to HugeGraph-Server as the config of the new graph
  • graph-clone, clone an existing graph
    • –name or -n, optional, the name of the cloned graph, default is g
    • –clone-graph-name, optional, the name of the source graph to clone from, default is hugegraph
  • graph-drop, drop a graph (different from graph-clear, this completely removes the graph)
    • –confirm-message or -c, required, confirmation message “I’m sure to drop the graph”, including double quotes

graph-create, graph-clone, graph-clear and graph-drop raise –timeout to at least 300 seconds.

When you need to restore the backup graph to a new graph, you need to set the graph mode to RESTORING mode; when you need to merge the backup graph into an existing graph, you need to first set the graph mode to MERGING model.

3.4 Asynchronous task management Type,task-list、task-get、task-delete、task-cancel and task-clear
  • task-list,List the asynchronous tasks in a graph, which can be filtered according to the status of the tasks
    • –status,Optional, specify the status of the task to view, i.e. filter tasks by status, legal values include [UNKNOWN, NEW, QUEUED, RESTORING, RUNNING, SUCCESS, CANCELLED, FAILED] (case insensitive)
    • –limit,Optional, specify the number of tasks to be obtained, the default is -1, which means to obtain all eligible tasks, a value passed explicitly must be positive
  • task-get,Get detailed information about an asynchronous task
    • –task-id,Required, specifies the ID of the asynchronous task
  • task-delete,Delete information about an asynchronous task
    • –task-id,Required, specifies the ID of the asynchronous task
  • task-cancel,Cancel the execution of an asynchronous task
    • –task-id,Required, the ID of the asynchronous task to cancel
  • task-clear,Clean up completed asynchronous tasks
    • –force,Optional. When set, it means to clean up all asynchronous tasks. Unfinished ones are canceled first, and then all asynchronous tasks are cleared. By default, only completed asynchronous tasks are cleaned up
3.5 Gremlin Type,gremlin-execute and gremlin-schedule

⚠️ SEC Reminder: The execution of Gremlin depends on the actual logic of the statements, which may involve scenarios such as large-scale data modification and high-risk system calls with potential implicit hazards. Please use this tool only in secure and trusted network environments. It is imperative to configure and secure HugeGraph-Server with the Authentication System (Auth) and an IP Whitelist to restrict execution requests on the server side. Never hand over the tool or expose the execution entry to unauthorized personnel.

  • gremlin-execute, send Gremlin statements to HugeGraph-Server to execute query or modification operations, execute synchronously, and return results after completion
    • –file or -f, specify the script file to execute, UTF-8 encoding, mutually exclusive with –script
    • –script or -s, specifies the script string to execute, mutually exclusive with –file
    • –aliases or -a, Gremlin alias settings, the format is: key1=value1,key2=value2,…
    • –bindings or -b, Gremlin binding settings, the format is: key1=value1,key2=value2,…
    • –language or -l, the language of the Gremlin script, the default is gremlin-groovy

    –file and –script are mutually exclusive, one of them must be set

  • gremlin-schedule, send Gremlin statements to HugeGraph-Server to perform query or modification operations, asynchronous execution, and return the asynchronous task id immediately after the task is submitted
    • –file or -f, specify the script file to execute, UTF-8 encoding, mutually exclusive with –script
    • –script or -s, specifies the script string to execute, mutually exclusive with –file
    • –bindings or -b, Gremlin binding settings, the format is: key1=value1,key2=value2,…
    • –language or -l, the language of the Gremlin script, the default is gremlin-groovy

    –file and –script are mutually exclusive, one of them must be set

3.6 Backup/Restore Type
  • backup, back up the schema or data in a certain graph out of the HugeGraph system, and store it on the local disk or HDFS in the form of JSON
    • –format, the backup format, optional values include [json, text], the default is json
    • –all-properties, whether to back up all properties of vertices/edges, only valid when –format is text, default false
    • –label, the vertex label or edge label to be backed up, only applied when –format is text; when it is set, –huge-types must name exactly one type and that type must be vertex or edge, otherwise the command fails
    • –properties, properties of vertices/edges to be backed up, separated by commas, only valid when –format is text, valid only when backing up vertices or edges
    • –compress, whether to compress data during backup, the default is true
    • –directory or -d, the directory to store schema or data, the default is ‘./{graphName}’ for local directory, and ‘{fs.default.name}/{graphName}’ for HDFS
    • –huge-types or -t, the data types to be backed up, separated by commas, the optional value is ‘all’ or a combination of one or more [vertex, edge, vertex_label, edge_label, property_key, index_label], ‘all’ Represents all 6 types, namely vertices, edges and all schemas, ‘schema’ represents the 4 schema types [vertex_label, edge_label, property_key, index_label]
    • –log or -l, specify the log directory, the default is ./logs
    • –retry, specify the number of failed retries, the default is 3
    • –thread-num or -T, the number of threads to use, default is Math.min(10, Math.max(4, CPUs / 2))
    • –split-size or -s, specifies the size of splitting vertices or edges when backing up, the default is 1048576, and it must be at least 1048576 (1M)
    • -D, use the mode of -Dkey=value to specify dynamic parameters, and specify HDFS configuration items when backing up data to HDFS, for example: -Dfs.default.name=hdfs://localhost:9000

    If –timeout is less than 120 seconds, backup (and the backup step of migrate) uses 120 seconds

  • restore, restore schema or data stored in JSON format to a new graph (RESTORING mode) or merge into an existing graph (MERGING mode)
    • –directory or -d, the directory to store schema or data, the default is ‘./{graphName}’ for local directory, and ‘{fs.default.name}/{graphName}’ for HDFS
    • –clean, whether to delete the directory specified by –directory after the recovery map is completed, the default is false
    • –huge-types or -t, data types to restore, separated by commas, optional value is ‘all’ or a combination of one or more [vertex, edge, vertex_label, edge_label, property_key, index_label], ‘all’ Represents all 6 types, namely vertices, edges and all schemas, ‘schema’ represents the 4 schema types [vertex_label, edge_label, property_key, index_label]
    • –log or -l, specify the log directory, the default is ./logs
    • –retry, specify the number of failed retries, the default is 3
    • –thread-num or -T, the number of threads to use, default is Math.min(10, Math.max(4, CPUs / 2))
    • -D, use the mode of -Dkey=value to specify dynamic parameters, which are used to specify HDFS configuration items when restoring graphs from HDFS, for example: -Dfs.default.name=hdfs://localhost:9000

    restore command can be used only if –format is executed as backup for json restore requires the graph to be in RESTORING or MERGING mode (set it with graph-mode-set first), otherwise the command fails

  • migrate, migrate the currently connected graph to another HugeGraphServer
    • –target-graph, the name of the target graph, the default is hugegraph
    • –target-url, the HugeGraphServer where the target graph is located, the default is http://127.0.0.1:8081
    • –target-user, the username used to access the target graph
    • –target-password, the password to access the target map
    • –target-timeout, the timeout for accessing the target map
    • –target-trust-store-file, access the truststore file used by the target graph
    • –target-trust-store-password, the password to access the truststore used by the target map
    • –directory or -d, during the migration process, the directory where the schema or data of the source graph is stored. For a local directory, the default is ‘./{graphName}’; for HDFS, the default is ‘{fs.default.name}/ {graphName}’
    • –huge-types or -t, the data types to be migrated, separated by commas, the optional value is ‘all’ or a combination of one or more [vertex, edge, vertex_label, edge_label, property_key, index_label], ‘all’ Represents all 6 types, namely vertices, edges and all schemas, ‘schema’ represents the 4 schema types [vertex_label, edge_label, property_key, index_label]
    • –log or -l, specify the log directory, the default is ./logs
    • –retry, specify the number of failed retries, the default is 3
    • –thread-num or -T, the number of threads to use, default is Math.min(10, Math.max(4, CPUs / 2))
    • –split-size or -s, specify the size of the vertex or edge block when backing up the source graph during the migration process, the default is 1048576, and it must be at least 1048576 (1M)
    • -D, use the mode of -Dkey=value to specify dynamic parameters, which are used to specify HDFS configuration items when the data needs to be backed up to HDFS during the migration process, for example: -Dfs.default.name=hdfs://localhost: 9000
    • –graph-mode or -m, the mode to set the target graph when restoring the source graph to the target graph, legal values include [RESTORING, MERGING], the default is RESTORING. The target graph is switched to this mode during the migration and switched back to its original mode afterwards
    • –keep-local-data, whether to keep the backup of the source map generated in the process of migrating the map, the default is false, that is, the backup of the source map is not kept after the default migration map ends
  • schedule-backup, periodically back up the graph and keep a certain number of the latest backups (currently only supports local file systems)
    • –directory or -d, required, specifies the directory of the backup data
    • –backup-num, optional, specifies the number of latest backups to save, defaults to 3
    • –interval, an optional item, specifies the backup cycle, the format is the same as the Linux crontab format, the default is “0 0 * * *” (every day at 00:00)

    schedule-backup adds a crontab entry that runs backup -t all into {directory}/{graph}/hugegraph-backup-{yyMMddHHmm}/ and keeps only the latest –backup-num backups. A relative –directory is resolved against the hugegraph-tools home directory, and {directory}/{graph} must not exist yet

  • dump, export all vertices and edges in the graph, using the vertex vertex-edge1 vertex-edge2... JSON format by default. To customize the format, implement a Formatter subclass such as CustomFormatter under hugegraph-tools/src/main/java/org/apache/hugegraph/formatter, then select it when running the command: bin/hugegraph dump -f CustomFormatter
    • –formatter or -f, specify the formatter to use, the default is JsonFormatter
    • –directory or -d, the directory where schema or data is stored, the default is ‘./{graphName}’ for local directory, and ‘{fs.default.name}/{graphName}’ for HDFS
    • –log or -l, specify the log directory, the default is ./logs
    • –retry, specify the number of failed retries, the default is 3
    • –thread-num or -T, the number of threads to use, default is Math.min(10, Math.max(4, CPUs / 2))
    • –split-size or -s, specifies the size of splitting vertices or edges when backing up, the default is 1048576, and it must be at least 1048576 (1M)
    • -D, use the mode of -Dkey=value to specify dynamic parameters, and specify HDFS configuration items when backing up data to HDFS, for example: -Dfs.default.name=hdfs://localhost:9000
3.7 Authentication data backup/restore type
  • auth-backup, backup authentication data to a specified directory
    • –types or -t, types of authentication data to back up, separated by commas, optional value is ‘all’ or a combination of one or more [user, group, target, belong, access], ‘all’ represents all 5 types; ‘belong’ requires ‘user’ and ‘group’ to be included, ‘access’ requires ‘group’ and ’target’ to be included
    • –directory, directory to store backup data, the default is ‘./auth-backup-restore’ for local directory, and ‘{fs.default.name}/auth-backup-restore’ for HDFS (this option has no -d short form)
    • –retry, specify the number of failed retries, the default is 3
    • -D, use the mode of -Dkey=value to specify dynamic parameters, and specify HDFS configuration items when backing up data to HDFS, for example: -Dfs.default.name=hdfs://localhost:9000
  • auth-restore, restore authentication data from a specified directory
    • –types or -t, types of authentication data to restore, separated by commas, optional value is ‘all’ or a combination of one or more [user, group, target, belong, access], ‘all’ represents all 5 types; ‘belong’ requires ‘user’ and ‘group’ to be included, ‘access’ requires ‘group’ and ’target’ to be included
    • –directory, directory where backup data is stored, the default is ‘./auth-backup-restore’ for local directory, and ‘{fs.default.name}/auth-backup-restore’ for HDFS (this option has no -d short form)
    • –retry, specify the number of failed retries, the default is 3
    • –strategy, conflict handling strategy, optional values are [stop, ignore], default is stop. stop means stop restoring when encountering conflicts, ignore means ignore conflicts and continue restoring
    • –init-password, initial password to set when restoring users, required when –types includes user
    • -D, use the mode of -Dkey=value to specify dynamic parameters, which are used to specify HDFS configuration items when restoring data from HDFS, for example: -Dfs.default.name=hdfs://localhost:9000
3.8 Install the deployment type
  • deploy, one-click download, install and start HugeGraph-Server and HugeGraph-Studio
    • -v, required, specifies the HugeGraph-Server and HugeGraph-Studio version to install, must be one of the versions listed in bin/version-map.yaml (0.6, 0.7, 0.8, 0.9, 0.10), which maps it to the matching server and studio release versions
    • -p, required, specifies the installed HugeGraph-Server and HugeGraph-Studio directories
    • -u, optional, specifies the link to download the HugeGraph-Server and HugeGraph-Studio compressed packages
  • clear, clean up HugeGraph-Server and HugeGraph-Studio directories and tarballs (refuses to run while a matching server or studio process is still alive, and prompts before each removal)
    • -p, required, specifies the directory of HugeGraph-Server and HugeGraph-Studio to be cleaned
  • start-all, start HugeGraph-Server and HugeGraph-Studio with one click
    • -v, required, specifies the installed HugeGraph-Server and HugeGraph-Studio version to start, same values as deploy
    • -p, required, specifies the directory where HugeGraph-Server and HugeGraph-Studio are installed
  • stop-all, close HugeGraph-Server and HugeGraph-Studio with one click

deploy, start-all, clear and stop-all are handed by bin/hugegraph straight to the shell scripts bin/deploy.sh, bin/start-all.sh, bin/clear.sh and bin/stop-all.sh, so the global options and environment variables in 3.2 do not apply to them.

There is an optional parameter -u in the deploy command. When provided, the specified download address will be used instead of the default download address to download the tar package, and the address will be written into the ~/hugegraph-download-url-prefix file; if no address is specified later When -u and ~/hugegraph-download-url-prefix are not specified, the tar package will be downloaded from the address specified by ~/hugegraph-download-url-prefix; if there is neither -u nor ~/hugegraph-download-url-prefix, it will be downloaded from the default download address https://github.com/hugegraph

3.9 Specific command parameters

The specific parameters of each subcommand are as follows:

Usage: hugegraph [options] [command] [command options]
  Options:
    --graph
      Name of graph
      Default: hugegraph
    --password
      Password of user
    --throw-mode
      Whether the hugegraph-tools work to throw an exception
      Default: false
    --timeout
      Connection timeout
      Default: 30
    --trust-store-file
      The path of client truststore file used when https protocol is enabled
    --trust-store-password
      The password of the client truststore file used when the https protocol 
      is enabled
    --url
      The URL of HugeGraph-Server
      Default: http://127.0.0.1:8080
    --user
      Name of user
  Commands:
    graph-create      Create graph with config
      Usage: graph-create [options]
        Options:
          --file, -f
            Creating graph config file
          --name, -n
            The name of new created graph, default is g
            Default: g

    graph-clone      Clone graph
      Usage: graph-clone [options]
        Options:
          --clone-graph-name
            The name of cloned graph, default is hugegraph
            Default: hugegraph
          --name, -n
            The name of new created graph, default is g
            Default: g

    graph-list      List all graphs
      Usage: graph-list

    graph-get      Get graph info
      Usage: graph-get

    graph-clear      Clear graph schema and data
      Usage: graph-clear [options]
        Options:
        * --confirm-message, -c
            Confirm message of graph clear is "I'm sure to delete all data". 
            (Note: include "")

    graph-drop      Drop graph
      Usage: graph-drop [options]
        Options:
        * --confirm-message, -c
            Confirm message of graph clear is "I'm sure to drop the graph". 
            (Note: include "")

    graph-mode-set      Set graph mode
      Usage: graph-mode-set [options]
        Options:
        * --graph-mode, -m
            Graph mode, include: [NONE, RESTORING, MERGING]
            Possible Values: [NONE, RESTORING, MERGING, LOADING]

    graph-mode-get      Get graph mode
      Usage: graph-mode-get

    task-list      List tasks
      Usage: task-list [options]
        Options:
          --limit
            Limit number, no limit if not provided
            Default: -1
          --status
            Status of task

    task-get      Get task info
      Usage: task-get [options]
        Options:
        * --task-id
            Task id
            Default: 0

    task-delete      Delete task
      Usage: task-delete [options]
        Options:
        * --task-id
            Task id
            Default: 0

    task-cancel      Cancel task
      Usage: task-cancel [options]
        Options:
        * --task-id
            Task id
            Default: 0

    task-clear      Clear completed tasks
      Usage: task-clear [options]
        Options:
          --force
            Force to clear all tasks, cancel all uncompleted tasks firstly, 
            and delete all completed tasks
            Default: false

    gremlin-execute      Execute Gremlin statements
      Usage: gremlin-execute [options]
        Options:
          --aliases, -a
            Gremlin aliases, valid format is: 'key1=value1,key2=value2...'
            Default: {}
          --bindings, -b
            Gremlin bindings, valid format is: 'key1=value1,key2=value2...'
            Default: {}
          --file, -f
            Gremlin Script file to be executed, UTF-8 encoded, exclusive to 
            --script 
          --language, -l
            Gremlin script language
            Default: gremlin-groovy
          --script, -s
            Gremlin script to be executed, exclusive to --file

    gremlin-schedule      Execute Gremlin statements as asynchronous job
      Usage: gremlin-schedule [options]
        Options:
          --bindings, -b
            Gremlin bindings, valid format is: 'key1=value1,key2=value2...'
            Default: {}
          --file, -f
            Gremlin Script file to be executed, UTF-8 encoded, exclusive to 
            --script 
          --language, -l
            Gremlin script language
            Default: gremlin-groovy
          --script, -s
            Gremlin script to be executed, exclusive to --file

    backup      Backup graph schema/data. If directory is on HDFS, use -D to 
            set HDFS params. For example: 
            -Dfs.default.name=hdfs://localhost:9000 
      Usage: backup [options]
        Options:
          --all-properties
            All properties to be backup flag
            Default: false
          --compress
            compress flag
            Default: true
          --directory, -d
            Directory of graph schema/data, default is './{graphname}' in 
            local file system or '{fs.default.name}/{graphname}' in HDFS
          --format
            File format, valid is [json, text]
            Default: json
          --huge-types, -t
            Type of schema/data. Concat with ',' if more than one. Other types 
            include 'all' and 'schema'. 'all' means all vertices, edges and 
            schema. In other words, 'all' equals with 'vertex, edge, 
            vertex_label, edge_label, property_key, index_label'. 'schema' 
            equals with 'vertex_label, edge_label, property_key, index_label'.
            Default: [PROPERTY_KEY, VERTEX_LABEL, EDGE_LABEL, INDEX_LABEL, VERTEX, EDGE]
          --label
            Vertex label or edge label, only valid when type is vertex or edge
          --log, -l
            Directory of log
            Default: ./logs
          --properties
            Vertex or edge properties to backup, only valid when type is 
            vertex or edge
            Default: []
          --retry
            Retry times, default is 3
            Default: 3
          --split-size, -s
            Split size of shard
            Default: 1048576
          --thread-num, -T
            Threads number to use, default is Math.min(10, Math.max(4, CPUs / 
            2)) 
            Default: 0
          -D
            HDFS config parameters
            Syntax: -Dkey=value
            Default: {}

    schedule-backup      Schedule backup task
      Usage: schedule-backup [options]
        Options:
          --backup-num
            The number of latest backups to keep
            Default: 3
        * --directory, -d
            The directory of backups stored
          --interval
            The interval of backup, format is: "a b c d e". 'a' means minute 
            (0 - 59), 'b' means hour (0 - 23), 'c' means day of month (1 - 
            31), 'd' means month (1 - 12), 'e' means day of week (0 - 6) 
            (Sunday=0), "*" means all
            Default: "0 0 * * *"

    dump      Dump graph to files
      Usage: dump [options]
        Options:
          --directory, -d
            Directory of graph schema/data, default is './{graphname}' in 
            local file system or '{fs.default.name}/{graphname}' in HDFS
          --formatter, -f
            Formatter to customize format of vertex/edge
            Default: JsonFormatter
          --log, -l
            Directory of log
            Default: ./logs
          --retry
            Retry times, default is 3
            Default: 3
          --split-size, -s
            Split size of shard
            Default: 1048576
          --thread-num, -T
            Threads number to use, default is Math.min(10, Math.max(4, CPUs / 
            2)) 
            Default: 0
          -D
            HDFS config parameters
            Syntax: -Dkey=value
            Default: {}

    restore      Restore graph schema/data. If directory is on HDFS, use -D to 
            set HDFS params if needed. For 
            example:-Dfs.default.name=hdfs://localhost:9000 
      Usage: restore [options]
        Options:
          --clean
            Whether to remove the directory of graph data after restored
            Default: false
          --directory, -d
            Directory of graph schema/data, default is './{graphname}' in 
            local file system or '{fs.default.name}/{graphname}' in HDFS
          --huge-types, -t
            Type of schema/data. Concat with ',' if more than one. Other types 
            include 'all' and 'schema'. 'all' means all vertices, edges and 
            schema. In other words, 'all' equals with 'vertex, edge, 
            vertex_label, edge_label, property_key, index_label'. 'schema' 
            equals with 'vertex_label, edge_label, property_key, index_label'.
            Default: [PROPERTY_KEY, VERTEX_LABEL, EDGE_LABEL, INDEX_LABEL, VERTEX, EDGE]
          --log, -l
            Directory of log
            Default: ./logs
          --retry
            Retry times, default is 3
            Default: 3
          --thread-num, -T
            Threads number to use, default is Math.min(10, Math.max(4, CPUs / 
            2)) 
            Default: 0
          -D
            HDFS config parameters
            Syntax: -Dkey=value
            Default: {}

    migrate      Migrate graph
      Usage: migrate [options]
        Options:
          --directory, -d
            Directory of graph schema/data, default is './{graphname}' in 
            local file system or '{fs.default.name}/{graphname}' in HDFS
          --graph-mode, -m
            Mode used when migrating to target graph, include: [RESTORING, 
            MERGING] 
            Default: RESTORING
            Possible Values: [NONE, RESTORING, MERGING, LOADING]
          --huge-types, -t
            Type of schema/data. Concat with ',' if more than one. Other types 
            include 'all' and 'schema'. 'all' means all vertices, edges and 
            schema. In other words, 'all' equals with 'vertex, edge, 
            vertex_label, edge_label, property_key, index_label'. 'schema' 
            equals with 'vertex_label, edge_label, property_key, index_label'.
            Default: [PROPERTY_KEY, VERTEX_LABEL, EDGE_LABEL, INDEX_LABEL, VERTEX, EDGE]
          --keep-local-data
            Whether to keep the local directory of graph data after restored
            Default: false
          --log, -l
            Directory of log
            Default: ./logs
          --retry
            Retry times, default is 3
            Default: 3
          --split-size, -s
            Split size of shard
            Default: 1048576
          --target-graph
            The name of target graph to migrate
            Default: hugegraph
          --target-password
            The password of target graph to migrate
          --target-timeout
            The timeout to connect target graph to migrate
            Default: 0
          --target-trust-store-file
            The trust store file of target graph to migrate
          --target-trust-store-password
            The trust store password of target graph to migrate
          --target-url
            The url of target graph to migrate
            Default: http://127.0.0.1:8081
          --target-user
            The username of target graph to migrate
          --thread-num, -T
            Threads number to use, default is Math.min(10, Math.max(4, CPUs / 
            2)) 
            Default: 0
          -D
            HDFS config parameters
            Syntax: -Dkey=value
            Default: {}

    deploy      Install HugeGraph-Server and HugeGraph-Studio
      Usage: deploy [options]
        Options:
        * -p
            Install path of HugeGraph-Server and HugeGraph-Studio
          -u
            Download url prefix path of HugeGraph-Server and HugeGraph-Studio
        * -v
            Version of HugeGraph-Server and HugeGraph-Studio

    start-all      Start HugeGraph-Server and HugeGraph-Studio
      Usage: start-all [options]
        Options:
        * -p
            Install path of HugeGraph-Server and HugeGraph-Studio
        * -v
            Version of HugeGraph-Server and HugeGraph-Studio

    clear      Clear HugeGraph-Server and HugeGraph-Studio
      Usage: clear [options]
        Options:
        * -p
            Install path of HugeGraph-Server and HugeGraph-Studio

    stop-all      Stop HugeGraph-Server and HugeGraph-Studio
      Usage: stop-all

    auth-backup      null
      Usage: auth-backup [options]
        Options:
          --directory
            Directory of auth information, default is 
            './{auth-backup-restore}' in local file system or 
            '{fs.default.name}/{auth-backup-restore}' in HDFS
          --retry
            Retry times, default is 3
            Default: 3
          --types, -t
            Type of auth data to restore and backup, concat with ',' if more 
            than one. 'all' means all auth information. In other words, 'all' 
            equals with 'user, group, target, belong, access'. In addition, 
            'belong' or 'access' can not backup or restore alone, if type 
            contains 'belong' then should contains 'user' and 'group'. If type 
            contains 'access' then should contains 'group' and 'target'.
            Default: [TARGET, GROUP, USER, ACCESS, BELONG]
          -D
            HDFS config parameters
            Syntax: -Dkey=value
            Default: {}

    auth-restore      null
      Usage: auth-restore [options]
        Options:
          --directory
            Directory of auth information, default is 
            './{auth-backup-restore}' in local file system or 
            '{fs.default.name}/{auth-backup-restore}' in HDFS
          --init-password
            Init user password, if restore type include 'user', please specify 
            the init-password of users.
            Default: <empty string>
          --retry
            Retry times, default is 3
            Default: 3
          --strategy
            The strategy needs to be chosen in the event of a conflict when 
            restoring. Valid strategies include 'stop' and 'ignore', default 
            is 'stop'. 'stop' means if there a conflict, stop restore. 
            'ignore' means if there a conflict, ignore and continue to 
            restore. 
            Default: STOP
            Possible Values: [STOP, IGNORE]
          --types, -t
            Type of auth data to restore and backup, concat with ',' if more 
            than one. 'all' means all auth information. In other words, 'all' 
            equals with 'user, group, target, belong, access'. In addition, 
            'belong' or 'access' can not backup or restore alone, if type 
            contains 'belong' then should contains 'user' and 'group'. If type 
            contains 'access' then should contains 'group' and 'target'.
            Default: [TARGET, GROUP, USER, ACCESS, BELONG]
          -D
            HDFS config parameters
            Syntax: -Dkey=value
            Default: {}

    help      Print usage
      Usage: help
3.10 Specific command example
1. gremlin statement
# Execute gremlin synchronously
./bin/hugegraph --url http://127.0.0.1:8080 --graph hugegraph gremlin-execute --script 'g.V().count()'

# Execute gremlin asynchronously
./bin/hugegraph --url http://127.0.0.1:8080 --graph hugegraph gremlin-schedule --script 'g.V().count()'
2. Show task status
./bin/hugegraph --url http://127.0.0.1:8080 --graph hugegraph task-list

./bin/hugegraph --url http://127.0.0.1:8080 --graph hugegraph task-list --limit 5

./bin/hugegraph --url http://127.0.0.1:8080 --graph hugegraph task-list --status success
3. Set and show graph mode
./bin/hugegraph --url http://127.0.0.1:8080 --graph hugegraph graph-mode-set -m RESTORING

./bin/hugegraph --url http://127.0.0.1:8080 --graph hugegraph graph-mode-get

./bin/hugegraph --url http://127.0.0.1:8080 --graph hugegraph graph-list
4. Cleanup Graph
./bin/hugegraph --url http://127.0.0.1:8080 --graph hugegraph graph-clear -c "I'm sure to delete all data"
5. Backup Graph
./bin/hugegraph --url http://127.0.0.1:8080 --graph hugegraph backup -t all --directory ./backup-test
6. Periodic Backup Graph
./bin/hugegraph --url http://127.0.0.1:8080 --graph hugegraph schedule-backup -d ./backup --interval "*/2 * * * *"
7. Recovery Graph
# set graph mode
./bin/hugegraph --url http://127.0.0.1:8080 --graph hugegraph graph-mode-set -m RESTORING

# recovery graph
./bin/hugegraph --url http://127.0.0.1:8080 --graph hugegraph restore -t all --directory ./backup-test

# restore graph mode
./bin/hugegraph --url http://127.0.0.1:8080 --graph hugegraph graph-mode-set -m NONE
8. Graph Migration
./bin/hugegraph --url http://127.0.0.1:8080 --graph hugegraph migrate --target-url http://127.0.0.1:8090 --target-graph hugegraph

3.2.7 - HugeGraph-Spark-Connector Quick Start

1 HugeGraph-Spark-Connector Overview

HugeGraph-Spark-Connector uses the Spark DataFrame API to write bulk data to HugeGraph. The current implementation provides vertex and edge writers.

Reading from HugeGraph is not implemented yet: the table only implements SupportsWrite, so spark.read.format(...) is not supported. The connector supports the CUSTOMIZE and PRIMARY_KEY vertex id strategies; the AUTOMATIC strategy is rejected.

2 Environment Requirements

  • Java 8+
  • Maven 3.6+
  • Spark 3.2.x (the module is built against Spark 3.2.2 with provided scope, so your Spark runtime must supply the Spark jars)
  • Scala 2.12 (built with Scala 2.12.11)

3 Building

3.1 Build without executing tests

git clone https://github.com/apache/hugegraph-toolchain.git
cd hugegraph-toolchain
mvn clean package -pl hugegraph-spark-connector -am -DskipTests -ntp

3.2 Build with default tests

mvn clean package -pl hugegraph-spark-connector -am -ntp

Both commands produce a fat jar at hugegraph-spark-connector/target/hugegraph-spark-connector-${revision}-jar-with-dependencies.jar (Spark itself is not bundled). Pass it to spark-submit --jars when you do not manage the dependency through Maven.

4 Usage

Add the dependency to pom.xml, replacing ${revision} with the release version you use:

<dependency>
    <groupId>org.apache.hugegraph</groupId>
    <artifactId>hugegraph-spark-connector</artifactId>
    <version>${revision}</version>
</dependency>

The format string must be the full class name org.apache.hugegraph.spark.connector.DataSource; the connector does not register a short name with Spark’s DataSourceRegister service loader. When HugeGraphServer has authentication enabled, add .option("username", ...) and .option("token", ...) to the examples below.

4.1 Schema Definition Example

If we have a graph, the schema is defined as follows:

schema.propertyKey("name").asText().ifNotExist().create()
schema.propertyKey("age").asInt().ifNotExist().create()
schema.propertyKey("city").asText().ifNotExist().create()
schema.propertyKey("weight").asDouble().ifNotExist().create()
schema.propertyKey("lang").asText().ifNotExist().create()
schema.propertyKey("date").asText().ifNotExist().create()
schema.propertyKey("price").asDouble().ifNotExist().create()

schema.vertexLabel("person")
        .properties("name", "age", "city")
        .useCustomizeStringId()
        .nullableKeys("age", "city")
        .ifNotExist()
        .create()

schema.vertexLabel("software")
        .properties("name", "lang", "price")
        .primaryKeys("name")
        .ifNotExist()
        .create()

schema.edgeLabel("knows")
        .sourceLabel("person")
        .targetLabel("person")
        .properties("date", "weight")
        .ifNotExist()
        .create()

schema.edgeLabel("created")
        .sourceLabel("person")
        .targetLabel("software")
        .properties("date", "weight")
        .ifNotExist()
        .create()

4.2 Vertex Sink (Scala)

val df = sparkSession.createDataFrame(Seq(
  Tuple3("marko", 29, "Beijing"),
  Tuple3("vadas", 27, "HongKong"),
  Tuple3("Josh", 32, "Beijing"),
  Tuple3("peter", 35, "ShangHai"),
  Tuple3("li,nary", 26, "Wu,han"),
  Tuple3("Bob", 18, "HangZhou"),
)) toDF("name", "age", "city")

df.show()

df.write
  .format("org.apache.hugegraph.spark.connector.DataSource")
  .option("host", "127.0.0.1")
  .option("port", "8080")
  .option("graph", "hugegraph")
  .option("data-type", "vertex")
  .option("label", "person")
  .option("id", "name")
  .option("batch-size", 2)
  .mode(SaveMode.Overwrite)
  .save()

4.3 Edge Sink (Scala)

val df = sparkSession.createDataFrame(Seq(
  Tuple4("marko", "vadas", "20160110", 0.5),
  Tuple4("peter", "Josh", "20230801", 1.0),
  Tuple4("peter", "li,nary", "20130220", 2.0)
)).toDF("source", "target", "date", "weight")

df.show()

df.write
  .format("org.apache.hugegraph.spark.connector.DataSource")
  .option("host", "127.0.0.1")
  .option("port", "8080")
  .option("graph", "hugegraph")
  .option("data-type", "edge")
  .option("label", "knows")
  .option("source-name", "source")
  .option("target-name", "target")
  .option("batch-size", 2)
  .mode(SaveMode.Overwrite)
  .save()

4.4 Vertex Sink with PRIMARY_KEY id strategy (Scala)

For a vertex label that uses primaryKeys(...), do not set the id option: the id is spliced from the primary key columns. Columns that are not part of the schema can be dropped with ignored-fields.

val df = sparkSession.createDataFrame(Seq(
  Tuple4("lop", "java", 328L, "ISBN978-7-107-18618-5"),
  Tuple4("ripple", "python", 199L, "ISBN978-7-100-13678-5"),
)).toDF("name", "lang", "price", "ISBN")

df.write
  .format("org.apache.hugegraph.spark.connector.DataSource")
  .option("host", "127.0.0.1")
  .option("port", "8080")
  .option("graph", "hugegraph")
  .option("data-type", "vertex")
  .option("label", "software")
  .option("ignored-fields", "ISBN")
  .option("batch-size", 2)
  .mode(SaveMode.Overwrite)
  .save()

4.5 Edge Sink with mixed id strategies (Scala)

source-name and target-name follow the id strategy of their own vertex label. Below, person uses a customized string id (one column) while software uses a primary key (its name column):

val df = sparkSession.createDataFrame(Seq(
  Tuple4("marko", "lop", "20171210", 0.5),
  Tuple4("Josh", "lop", "20091111", 0.4),
  Tuple4("peter", "ripple", "20171210", 1.0),
  Tuple4("vadas", "lop", "20171210", 0.2)
)).toDF("source", "name", "date", "weight")

df.write
  .format("org.apache.hugegraph.spark.connector.DataSource")
  .option("host", "127.0.0.1")
  .option("port", "8080")
  .option("graph", "hugegraph")
  .option("data-type", "edge")
  .option("label", "created")
  .option("source-name", "source") // customize id
  .option("target-name", "name")   // primary key
  .option("batch-size", 2)
  .mode(SaveMode.Overwrite)
  .save()

Note on save modes: SaveMode.Overwrite and SaveMode.Append both insert the rows. The overwrite path does not delete existing data from the graph first.

5 Configuration Parameters

Option keys are matched case-insensitively and trimmed. data-type and label are always required; source-name and target-name are required when data-type is edge; all other options have defaults.

5.1 Client Configs

Client Configs are used to configure hugegraph-client.

ParameterDefault ValueDescription
hostlocalhostAddress of HugeGraphServer. A bare host name or IP, or a full http:// / https:// prefix
port8080Port of HugeGraphServer
graphhugegraphGraph name
protocolhttpProtocol for sending requests to the server, optional http or https
usernamenullUsername of the current graph when HugeGraphServer enables permission authentication. When unset, the graph name is used as the username
tokennullToken of the current graph when HugeGraphServer has enabled authorization authentication
timeout60Timeout (seconds) for inserting results to return
max-connCPUS * 4The maximum number of HTTP connections between HugeClient and HugeGraphServer
max-conn-per-routeCPUS * 2The maximum number of HTTP connections for each route between HugeClient and HugeGraphServer
trust-store-filenullThe client’s certificate file path when the request protocol is https. When unset under https, the connector reads conf/hugegraph.truststore under the directory given by the JVM system property connector.home.path, which must then be set
trust-store-tokennullThe client’s certificate password when the request protocol is https. When unset under https, hugegraph is used

5.2 Graph Data Configs

Graph Data Configs describe how DataFrame columns map to vertices or edges.

ParameterDefault ValueDescription
data-typeRequired. Graph data type, must be vertex or edge
labelRequired. Label to which the vertex/edge data to be imported belongs
idSpecify a column as the id column of the vertex. When the vertex id policy is CUSTOMIZE, it is required; when the id policy is PRIMARY_KEY, it must be empty. The AUTOMATIC id policy is not supported
source-nameRequired when data-type is edge. Select certain columns of the input source as the id column of source vertex. When the id policy of the source vertex is CUSTOMIZE, a certain column must be specified as the id column of the vertex; when the id policy of the source vertex is PRIMARY_KEY, one or more columns must be specified for splicing the id of the generated vertex, that is, no matter which id strategy is used, this item is required. Multiple columns are separated by , (the delimiter option does not apply here)
target-nameRequired when data-type is edge. Specify certain columns as the id columns of target vertex, similar to source-name
selected-fieldsSelect some columns to insert, other unselected ones are not inserted, cannot exist at the same time as ignored-fields
ignored-fieldsIgnore some columns so that they do not participate in insertion, cannot exist at the same time as selected-fields
batch-size500The number of data items in each batch when importing data. Applied per Spark task: each partition writer flushes its buffer to the server once it holds this many vertices/edges, and again at commit for the remainder

5.3 Common Configs

Common Configs contains some common configurations.

ParameterDefault ValueDescription
delimiter,Separator of selected-fields and ignored-fields. source-name and target-name are always split on ,

6 Notes and Limitations

  • Each Spark write task opens its own HugeClient, switches the graph to LOADING mode before writing and sets it back to NONE at commit or abort.
  • Vertex ids are limited to 128 bytes (UTF-8). This applies to customized string ids and to ids spliced from primary keys.
  • The AUTOMATIC vertex id strategy is not supported; the write fails with an IllegalArgumentException when the writer is created.
  • Properties with SET or LIST cardinality are not supported yet; only SINGLE cardinality values are converted.
  • Date properties: string values must use the format yyyy-MM-dd HH:mm:ss and are parsed in the GMT+8 time zone; numeric values are treated as epoch milliseconds.
  • Boolean properties given as strings accept true, 1, yes, y and false, 0, no, n (case-insensitive).
  • Rows whose customized string id, or any primary key value, is an empty string are skipped. A null id or primary key value raises an error instead.

7 License

The same as HugeGraph, hugegraph-spark-connector is also licensed under Apache 2.0 License.

3.3 - HugeGraph-AI

hugegraph-ai provides Python clients for HugeGraph, graph machine learning tools, and LLM tools for knowledge graph construction and GraphRAG applications.

Apache License 2.0 · Ask DeepWiki

Modules

  • hugegraph-llm: knowledge graph construction, GraphRAG, and natural-language graph queries.
  • hugegraph-ml: reads graph data from HugeGraph and runs graph learning models.
  • hugegraph-python-client: a Python SDK for managing schemas and graph data and running Gremlin queries.
  • vermeer-python-client: a Python SDK for the Vermeer graph computing service.

The repository uses a uv workspace whose members are hugegraph-llm and hugegraph-python-client. hugegraph-ml and vermeer-python-client are editable path dependencies rather than workspace members. The current repository version is 1.7.0. The client source directory is named hugegraph-python-client, but its distribution name is hugegraph-python.

Requirements

  • HugeGraph-AI root workspace: Python 3.10 or later; HugeGraph-LLM additionally requires a version below 3.12
  • HugeGraph-ML: Python 3.10 or later
  • PyPI hugegraph-python 1.5.0: Python 3.9 or later; current repository source uses Python 3.10 or later with the workspace
  • Vermeer Python client: current source requires Python 3.10 or later, although package metadata still says >=3.9; see the client guide
  • uv 0.7 or later
  • HugeGraph Server 1.5.0 or later; the current workspace client rejects detectable older versions

Optional Dependency Groups

The root project declares one extra per module plus a few combined ones:

ExtraInstalls
llmhugegraph-llm
mlhugegraph-ml
python-clienthugegraph-python (source directory hugegraph-python-client)
vermeervermeer-python-client
devpytest, pytest-cov, coverage, pylint, ruff, mypy, ty, pre-commit
nk-llmhugegraph-llm, hugegraph-python-client, and Nuitka for the compiled image
allall four module packages

hugegraph-llm itself declares a vectordb extra that adds pymilvus and qdrant-client.

Deploy with Docker Compose

The repository includes a Compose file that starts both HugeGraph Server and the RAG service:

git clone https://github.com/apache/hugegraph-ai.git
cd hugegraph-ai
cp docker/env.template docker/.env
# Edit docker/.env and set PROJECT_PATH to the absolute path of this repository
touch hugegraph-llm/.env
# Set GRAPH_URL=server:8080 in hugegraph-llm/.env and supply matching Server credentials
cd docker
docker compose -f docker-compose-network.yml up -d

Default addresses:

  • HugeGraph Server: http://localhost:8080
  • RAG service and Web UI: http://localhost:8001

Start the RAG Service from Source

git clone https://github.com/apache/hugegraph-ai.git
cd hugegraph-ai
uv sync --extra llm
source .venv/bin/activate
cd hugegraph-llm
python -m hugegraph_llm.demo.rag_demo.app

uv sync creates .venv at the repository root. Installing from the root resolves workspace members and path dependencies together. The repository does not track uv.lock; uv sync resolves the declarations and version constraints in pyproject.toml.

Install ML Dependencies

cd hugegraph-ai
uv sync --extra ml
source .venv/bin/activate
cd hugegraph-ml/src

Example scripts are under hugegraph-ml/src/hugegraph_ml/examples/.

Next Steps

3.3.1 - HugeGraph-LLM

HugeGraph-LLM supports knowledge graph construction, GraphRAG, and natural-language graph queries. Its demo service hosts Gradio and FastAPI in one process. Source launches listen locally at 127.0.0.1:8001, accessible at http://localhost:8001; explicitly use --host 127.0.0.1 for local-only access. Dockerfile.llm overrides this to 0.0.0.0:8001; do not use loopback inside the container, or published ports will not reach the service.

Warning

Production requires HugeGraph-LLM login (ENABLE_LOGIN=True, replacing USER_TOKEN and ADMIN_TOKEN) and a source IP allowlist at the firewall or network entry point. Separately enable Server authentication and authorization, retain Server audit logs (normally audit-*.log), and grant GRAPH_USER only the permissions this service needs. AI service tokens authenticate the LLM UI and API, not HugeGraph Server.

Requirements

AI-generated project documentation: Ask DeepWiki

  • Python 3.10 or 3.11 (>=3.10,<3.12)
  • uv 0.7 or later
  • HugeGraph Server 1.5.0 or later; the current workspace client rejects detectable older versions

Deploy with Docker Compose

Prepare the environment files from the HugeGraph-AI repository root:

git clone https://github.com/apache/hugegraph-ai.git
cd hugegraph-ai
cp docker/env.template docker/.env
# Edit docker/.env and set PROJECT_PATH to the absolute path of this repository
touch hugegraph-llm/.env
# Set GRAPH_URL=server:8080 and matching GRAPH_USER / GRAPH_PWD in hugegraph-llm/.env
cd docker
docker compose -f docker-compose-network.yml up -d
docker compose -f docker-compose-network.yml ps

After startup, HugeGraph Server is available at http://localhost:8080, and the RAG service and Web UI are available at http://localhost:8001.

The Compose file mounts ${PROJECT_PATH}/hugegraph-llm/.env into the container at /home/work/hugegraph-llm/.env, so the file has to exist before the container starts. The resource directory hugegraph-llm/src/hugegraph_llm/resources can be mounted the same way; the mount is commented out by default.

The application reads GRAPH_URL; Compose-provided HUGEGRAPH_HOST and HUGEGRAPH_PORT do not override it. Set GRAPH_URL=server:8080 and matching Server credentials in the container’s .env.

Container Images

Build recipeDescription
docker/Dockerfile.llmSource runtime image recipe; starts python -m hugegraph_llm.demo.rag_demo.app --host 0.0.0.0 --port 8001
docker/Dockerfile.nkNuitka binary image recipe using the nk-llm extra; starts ./app.dist/app.bin

Compose references untagged hugegraph/rag, which resolves to latest. scripts/build_llm_image.sh builds docker/Dockerfile.llm locally as hugegraph/graphrag:1.7.0. These are different images: building locally does not replace Compose’s image. To run the build, change Compose’s image to hugegraph/graphrag:1.7.0. The script builds but does not push. Both Dockerfiles expose 8001, run as non-root user work, declare a resource-directory volume, and use curl -f http://localhost:8001/ as their health check.

Deploy on Kubernetes

docker/charts/hg-llm is a Helm chart for the RAG service. It deploys the hugegraph/graphrag image and, by default, publishes a NodePort service that maps node port 8039 and service port 8080 onto container port 8001. The release name is fixed to hg-llm-service. Ingress and horizontal pod autoscaling are present but disabled by default.

The chart deploys only the RAG service, not HugeGraph Server. Set GRAPH_URL in the mounted .env to a Server address reachable from the Pod, with matching credentials.

image.tag still defaults to v0.0.1. The build script produces local hugegraph/graphrag:1.7.0; before deployment, push the image to a registry reachable by the cluster or load it onto cluster nodes, then set matching tag and pull policy values.

The .env and prompt YAML mounts are commented out in values.yaml. Prompt YAML can use a ConfigMap, but .env may contain secrets and passwords: use a Secret and change the env-config volume from configMap to secret. Create them separately:

kubectl create secret generic hugegraph-llm-env --from-file=.env=/path/to/.env
kubectl create configmap hugegraph-llm-prompt-config --from-file=/path/to/config_prompt.yaml

Uncomment the volumeMounts entries and enable the prompt ConfigMap volume/mount if needed. Helm renders these volume definitions directly from values.yaml.

Start from Source

Install dependencies through the workspace at the repository root:

git clone https://github.com/apache/hugegraph-ai.git
cd hugegraph-ai
uv sync --extra llm
source .venv/bin/activate
cd hugegraph-llm
python -m hugegraph_llm.demo.rag_demo.app

To use a custom address and port:

python -m hugegraph_llm.demo.rag_demo.app \
  --host 127.0.0.1 \
  --port 18001

Set HG_DEV_RELOAD=1 to start uvicorn with auto-reload during development.

The service stores model, HugeGraph, and login settings in hugegraph-llm/.env. Prompts are stored separately in hugegraph-llm/src/hugegraph_llm/resources/demo/config_prompt.yaml. The configuration code creates missing files with default values.

The .env location is resolved in this order: HUGEGRAPH_LLM_ENV_PATH if it is set, then hugegraph-llm/.env when the package runs from a source checkout, then .env in the current working directory.

Main Capabilities

Build RAG Indexes

The first Web UI tab splits text into a chunk vector index, extracts vertices and edges according to a schema, writes the graph to HugeGraph, and updates the vertex vector index. Input can be typed into the text tab or uploaded through the file tab, which accepts .txt, .docx, and .pdf files and allows selecting several at once. Encrypted PDFs and scanned PDFs without an extractable text layer are rejected.

The schema can be inline JSON or the name of an existing graph. Through the REST API, a graph name requires a matching client_config.graph; inline JSON neither connects to HugeGraph nor accepts client_config.

The tab also carries two generators. Graph Schema Generator derives a schema from query examples plus a few-shot example. Graph Extraction Prompt Generator writes an extraction prompt from a described scenario and a selected reference example. A Graph Extraction Split Type dropdown chooses document, paragraph, or sentence granularity before extraction.

GraphRAG

The query pipeline can combine direct LLM answers, chunk-vector retrieval, and graph retrieval. Graph retrieval first extracts keywords and matches vertices, then attempts Text2Gremlin. If generation or execution fails, it can fall back to predefined graph traversals. Request parameters control result limits, vector distance thresholds, template counts, and reranking.

The same tab has a batch back-testing panel that reads questions from an .xlsx or .csv file, answers each one, and returns a downloadable file. A template file is offered for download next to the upload control.

Knowledge graph builder

Text2Gremlin

POST /text2gremlin generates Gremlin from natural language, the graph schema, and optional examples. A custom prompt must retain {query}, {schema}, {example}, and {vertices}.

The matching UI tab can first build the example vector index from a .json or .csv file of question and Gremlin pairs. The bundled resources/demo/text2gremlin.csv is used when no file is supplied.

Graph and Admin Tools

The Graph Tools tab runs a Gremlin query directly, triggers a manual graph backup, and can initialize demo data in HugeGraph. The Admin Tools tab shows the last lines of logs/llm-server.log behind an ADMIN_TOKEN prompt, and can refresh or clear that file.

Two background tasks run for the lifetime of the process: a cron job that backs up the graph every day at 01:00, and a task that keeps vertex-id embeddings up to date.

Models and Vector Backends

Chat, information extraction, and Text2Gremlin can independently use an OpenAI-compatible endpoint, Ollama, or LiteLLM. The embedding model is configured separately and supports the same three providers. Reranking supports Cohere and SiliconFlow.

FAISS is the default vector index. CUR_VECTOR_INDEX selects Faiss, Milvus, or Qdrant, and the same choice is available in the 5. Set up the vector engine. panel of the Web UI. Milvus and Qdrant require the optional dependencies:

cd hugegraph-ai
uv sync --package hugegraph-llm --extra vectordb

See the workflow guide, the configuration reference, and the REST API for details.

Programmatic Use

The former RAGPipeline and KgBuilder classes were replaced by a pipeline scheduler. Call a flow by name through SchedulerSingleton:

from hugegraph_llm.flows.scheduler import SchedulerSingleton

scheduler = SchedulerSingleton.get_instance()
res = scheduler.schedule_flow(
    "rag_graph_only",
    query="Tell me about Al Pacino.",
    graph_only_answer=True,
    vector_only_answer=False,
    raw_answer=False,
    gremlin_tmpl_num=-1,
    gremlin_prompt=None,
)
print(res.get("graph_only_answer"))

The registered flow names are rag_raw, rag_vector_only, rag_graph_only, rag_graph_vector, text2gremlin, build_examples_index, build_vector_index, graph_extract, import_graph_data, update_vid_embeddings, get_graph_index_info, build_schema, and prompt_generate. schedule_stream_flow is the async streaming variant.

Development Checks

Install the module and the development tools from the repository root, then run the checks that mirror CI:

cd hugegraph-ai
uv sync --extra llm --extra dev
uv run ruff format --check .
uv run ruff check .

cd hugegraph-llm
SKIP_EXTERNAL_SERVICES=true uv run pytest src/tests/config/ src/tests/document/ src/tests/middleware/ \
  src/tests/operators/ src/tests/models/ src/tests/indices/ src/tests/test_utils.py -v --tb=short
SKIP_EXTERNAL_SERVICES=true uv run pytest src/tests/integration/test_graph_rag_pipeline.py \
  src/tests/integration/test_kg_construction.py src/tests/integration/test_rag_pipeline.py -v --tb=short

Git hooks are available through pre-commit:

cd hugegraph-ai
pre-commit install
pre-commit run --all-files

3.3.2 - HugeGraph-ML

HugeGraph-ML reads graph data from HugeGraph and converts it to DGL graphs for tasks such as node embedding, node classification, graph classification, link prediction and fraud detection. Model implementations are under hugegraph-ml/src/hugegraph_ml/models/.

Requirements

  • Python 3.10 or later
  • HugeGraph Server 1.5.0 or later; the current workspace client rejects detectable older versions
  • uv 0.7 or later

All server access goes through hugegraph-python-client (the pyhugegraph package) from the same repository. HugeGraph2DGL pulls vertices and edges over the Gremlin endpoint with g.V().hasLabel(...) and g.E().hasLabel(...), and the dataset importers write through the schema and batch vertex/edge APIs in batches of 500.

The repository root declares ML version constraints under [tool.uv] constraint-dependencies:

PackageConstraint
torch==2.2.0
dgl~=2.1.0
ogb~=1.3.6
torchdata~=0.7.0
catboost~=1.2.3
category-encoders~=2.6.3
numpy~=1.24.4
pandas~=2.2.3

These constraints limit versions but do not select CPU or CUDA builds. Tasks that support gpu default to -1 for CPU. GPU use requires mutually compatible CUDA builds of torch and dgl.

Installation

git clone https://github.com/apache/hugegraph-ai.git
cd hugegraph-ai
uv sync --extra ml
source .venv/bin/activate
cd hugegraph-ml/src

HugeGraph-ML is a path dependency of the root project but is not a uv workspace member. Select the ml extra at the repository root so uv installs it and its local client dependency according to workspace configuration.

Implemented Models

Every module below lives in hugegraph-ml/src/hugegraph_ml/models/. models/__init__.py re-exports nothing, so import from the module file directly.

ModelModuleEntry classUsed forPaper
AGNNagnn.pyAGNNNode classification1803.03735
APPNPappnp.pyAPPNPNode classification1810.05997
ARMAarma.pyARMA4NCNode classification1901.01343
BGNNbgnn.pyBGNNPredictorGradient boosting over node features combined with a GNN; the bundled example runs regression2101.08543
BGRLbgrl.pyBGRLSelf-supervised node embedding2102.06514
CARE-GNNcare_gnn.pyCAREGNNFraud detection2008.08692
Cluster-GCNcluster_gcn.pySAGENode classification with subgraph sampling1905.07953
C&Scorrect_and_smooth.pyMLP, CorrectAndSmooth, LabelPropagationCorrecting and smoothing base predictions2010.13993
DAGNNdagnn.pyDAGNNNode classification2007.09296
DeeperGCNdeepergcn.pyDeeperGCNNode classification with edge features2006.07739
DGIdgi.pyDGISelf-supervised node embedding1809.10341
DiffPooldiffpool.pyDiffPoolGraph classification1806.08804
GATNEgatne.pyDGLGATNEHeterogeneous network embedding1905.01669
GINgin_global_pool.pyGINGraph classification
GRACEgrace.pyGRACESelf-supervised node embedding2006.04131
GRANDgrand.pyGRANDNode classification2005.11079
JKNetjknet.pyJKNetNode classification1806.03536
MLPmlp.pyMLPClassifierDownstream classifier over learned embeddings
P-GNNpgnn.pyPGNNLink predictionyou19b
SEALseal.pyDGCNN, SEALDataLink prediction1802.09691

GIN accepts pooling values sum (default), mean, max, global_attention and set2set.

Reading Graph Data

HugeGraph2DGL in hugegraph-ml/src/hugegraph_ml/data/hugegraph2dgl.py opens a PyHugeClient and converts query results into DGL objects:

from hugegraph_ml.data.hugegraph2dgl import HugeGraph2DGL

hg2d = HugeGraph2DGL(
    url="http://127.0.0.1:8080",
    graph="hugegraph",
    user="",
    pwd="",
    graphspace=None,
)
MethodReturnsNotes
convert_graph(vertex_label, edge_label, feat_key="feat", label_key="label", mask_keys=None)dgl.DGLGraphmask_keys falls back to ["train_mask", "val_mask", "test_mask"]
convert_hetero_graph(vertex_labels, edge_labels, feat_key="feat", label_key="label", mask_keys=None)DGL heterographTakes lists of labels
convert_graph_dataset(graph_vertex_label, vertex_label, edge_label, feat_key="feat", label_key="label")HugeGraphDatasetFills info with n_graphs, max_n_nodes, n_feat_dim, n_classes
convert_graph_nx(vertex_label, edge_label)networkx.GraphUsed by P-GNN
convert_graph_with_edge_feat(vertex_label, edge_label, node_feat_key="feat", edge_feat_key="edge_feat", label_key="label", mask_keys=None)dgl.DGLGraphAlso fills edata["feat"]
convert_graph_ogb(vertex_label, edge_label, split_label)(dgl.DGLGraph, split_edge)Used by SEAL
convert_hetero_graph_bgnn(vertex_labels, edge_labels, feat_key="feat", label_key="class", cat_key="cat_features", mask_keys=None)DGL heterographUsed by BGNN

Node features land in ndata["feat"], labels in ndata["label"] and each mask in ndata[<mask key>]. NodeEmbed requires feat only; NodeClassify, NodeClassifyWithEdge and NodeClassifyWithSample require feat, label, train_mask, val_mask and test_mask and raise ValueError when one is missing.

Importing Sample Datasets

hugegraph_ml.utils.dgl2hugegraph_utils writes DGL, OGB and NetworkX datasets into HugeGraph so the conversion layer has something to read. Every function takes the same url, graph, user, pwd and graphspace arguments as HugeGraph2DGL, and most upper-case the dataset name before matching it.

FunctionAccepted datasetsLabels created
import_graph_from_dglCORA, CITESEER, PUBMED<NAME>_vertex, <NAME>_edge
import_graphs_from_dglMUTAG, COLLAB, NCI1, PROTEINS, PTC, ENZYMES, DD<NAME>_graph_vertex, <NAME>_vertex, <NAME>_edge
import_hetero_graph_from_dglACM<NAME>_<ntype>_v, <NAME>_<etype>_e
import_hetero_graph_from_dgl_no_featAMAZONGATNE<NAME>_<ntype>_v, <NAME>_<etype>_e
import_hetero_graph_from_dgl_bgnnAVAZU<NAME>_<ntype>_v, <NAME>_<etype>_e
import_graph_from_nxCAVEMAN<NAME>_vertex, <NAME>_edge
import_graph_from_dgl_with_edge_featCORA, CITESEER, PUBMED<NAME>_edge_feat_vertex, <NAME>_edge_feat_edge
import_graph_from_ogbogbl-collab, matched without upper-casing<NAME>_vertex, <NAME>_edge
import_split_edge_from_ogbogbl-collab, matched without upper-casing<NAME>_split_edge

Any other name raises ValueError("dataset not supported"). import_split_edge_from_ogb additionally requires the idx_to_vertex_id mapping and a max_nodes cap returned by the vertex import.

clear_all_data() drops every vertex and edge in the target graph. The test fixture calls it, loads CORA, MUTAG and ACM, and calls it again on teardown.

AMAZONGATNE and AVAZU are not fetched automatically. Their archive URLs are recorded in comments above import_hetero_graph_from_dgl_no_feat and import_hetero_graph_from_dgl_bgnn.

Tasks

Task classes live in hugegraph-ml/src/hugegraph_ml/tasks/. Each one takes the converted graph and a model instance.

ClassModuleEntry points
NodeEmbednode_embed.pytrain_and_embed(add_self_loop=True, lr=1e-3, weight_decay=0, n_epochs=200, patience=inf, gpu=-1) returns the graph with ndata["feat"] replaced by the embedding
NodeClassifynode_classify.pytrain(lr, weight_decay, n_epochs, patience, early_stopping_monitor, gpu) then evaluate(), which returns {"accuracy": ..., "loss": ...}
NodeClassifyWithEdgenode_classify_with_edge.pySame shape, for models that also read edata["feat"]
NodeClassifyWithSamplenode_classify_with_sample.pyCluster-GCN style training on ClusterGCNSampler partitions; runs on CPU and takes no gpu argument
GraphClassifygraph_classify.pytrain(batch_size=20, lr, weight_decay, n_epochs, patience, early_stopping_monitor, clip=2.0, gpu) over a HugeGraphDataset, split 70/20/10
DetectorCaregnnfraud_detector_caregnn.pyCARE-GNN training; evaluate() reports recall and ROC AUC and reads ndata["feature"] rather than ndata["feat"]
HeteroSampleEmbedGATNEhetero_sample_embed_gatne.pytrain_and_embed(lr=1e-3, n_epochs=200, gpu=-1)
LinkPredictionPGNNlink_prediction_pgnn.pytrain(lr, weight_decay, n_epochs, gpu)
LinkPredictionSeallink_prediction_seal.pyThe constructor calls data_prepare() itself, then train(lr=1e-3, n_epochs=200, gpu=-1)

patience defaults to float("inf"). EarlyStopping in utils/early_stopping.py monitors either loss or accuracy, keeps a copy of the best weights and restores them when training stops.

Runnable Examples

Scripts sit in hugegraph-ml/src/hugegraph_ml/examples/. From hugegraph-ml/src, run one with:

python ./hugegraph_ml/examples/dgi_example.py

Each script also exposes a function of the same name, so it can be imported and called with a smaller epoch count.

ScriptModelTaskReads
agnn_example.pyAGNNNodeClassifyCORA_vertex, CORA_edge
appnp_example.pyAPPNPNodeClassifyCORA_vertex, CORA_edge
arma_example.pyARMA4NCNodeClassifyCORA_vertex, CORA_edge
bgnn_example.pyBGNNPredictorIts own fit()AVAZU__N_v, AVAZU__E_e
bgrl_example.pyBGRLNodeEmbed, NodeClassifyCORA_vertex, CORA_edge
care_gnn_example.pyCAREGNNDetectorCaregnnAMAZON_user_v plus AMAZON_net_upu_e, AMAZON_net_usu_e, AMAZON_net_uvu_e
cluster_gcn_example.pySAGENodeClassifyWithSampleCORA_vertex, CORA_edge
correct_and_smooth_example.pyMLP from correct_and_smoothNodeClassifyCORA_vertex, CORA_edge
dagnn_example.pyDAGNNNodeClassifyCORA_vertex, CORA_edge
deepergcn_example.pyDeeperGCNNodeClassifyWithEdgeCORA_vertex, CORA_edge through convert_graph_with_edge_feat
dgi_example.pyDGINodeEmbed, NodeClassifyCORA_vertex, CORA_edge
diffpool_example.pyDiffPoolGraphClassifyMUTAG_graph_vertex, MUTAG_vertex, MUTAG_edge
gatne_example.pyDGLGATNEHeteroSampleEmbedGATNEAMAZONGATNE__N_v, AMAZONGATNE_1_e, AMAZONGATNE_2_e
gin_example.pyGINGraphClassifyMUTAG_graph_vertex, MUTAG_vertex, MUTAG_edge
grace_example.pyGRACENodeEmbed, NodeClassifyCORA_vertex, CORA_edge
grand_example.pyGRANDNodeClassifyCORA_vertex, CORA_edge
jknet_example.pyJKNetNodeClassifyCORA_vertex, CORA_edge
pgnn_example.pyPGNNLinkPredictionPGNNCAVEMAN_vertex, CAVEMAN_edge
seal_example.pyDGCNNLinkPredictionSealogbl-collab_vertex, ogbl-collab_edge, ogbl-collab_split_edge

DGI Node Embedding Example

First import DGL’s Cora dataset into HugeGraph. The name is upper-cased before use, so cora and CORA both produce the CORA_vertex and CORA_edge labels:

from hugegraph_ml.utils.dgl2hugegraph_utils import import_graph_from_dgl

import_graph_from_dgl("cora")

Read the graph and train DGI:

from hugegraph_ml.data.hugegraph2dgl import HugeGraph2DGL
from hugegraph_ml.models.dgi import DGI
from hugegraph_ml.models.mlp import MLPClassifier
from hugegraph_ml.tasks.node_classify import NodeClassify
from hugegraph_ml.tasks.node_embed import NodeEmbed

hg2d = HugeGraph2DGL()
graph = hg2d.convert_graph(vertex_label="CORA_vertex", edge_label="CORA_edge")

embed_model = DGI(n_in_feats=graph.ndata["feat"].shape[1])
embed_task = NodeEmbed(graph=graph, model=embed_model)
embedded_graph = embed_task.train_and_embed(
    add_self_loop=True, n_epochs=300, patience=30
)

classifier = MLPClassifier(
    n_in_feat=embedded_graph.ndata["feat"].shape[1],
    n_out_feat=embedded_graph.ndata["label"].unique().shape[0],
)
classify_task = NodeClassify(graph=embedded_graph, model=classifier)
classify_task.train(lr=1e-3, n_epochs=400, patience=40)
print(classify_task.evaluate())

evaluate() returns a dictionary such as {'accuracy': 0.82, 'loss': 0.5714246034622192}. The complete script is hugegraph-ml/src/hugegraph_ml/examples/dgi_example.py.

GRAND Node Classification Example

from hugegraph_ml.data.hugegraph2dgl import HugeGraph2DGL
from hugegraph_ml.models.grand import GRAND
from hugegraph_ml.tasks.node_classify import NodeClassify

hg2d = HugeGraph2DGL()
graph = hg2d.convert_graph(vertex_label="CORA_vertex", edge_label="CORA_edge")
model = GRAND(
    n_in_feats=graph.ndata["feat"].shape[1],
    n_out_feats=graph.ndata["label"].unique().shape[0],
)
task = NodeClassify(graph, model)
task.train(lr=1e-2, weight_decay=5e-4, n_epochs=2000, patience=100)
print(task.evaluate())

GRAND returns a list of logits per augmentation sample, and NodeClassify masks each element of that list before computing the loss. The complete script is hugegraph-ml/src/hugegraph_ml/examples/grand_example.py.

Troubleshooting

  • Connection failures: check the HugeGraph Server address, port, and credentials.
  • Schema mismatches: the examples use CORA_vertex and CORA_edge; pass the actual labels for your own data.
  • ValueError: Graph is missing required node attribute ...: the node classification tasks need feat, label, train_mask, val_mask and test_mask in ndata. Import a dataset that carries masks, or pass your own mask_keys to convert_graph.
  • ValueError: dataset not supported: the importer only accepts the names in the table above, and import_graph_from_ogb matches ogbl-collab without upper-casing.
  • DGL or PyTorch import failures: rerun uv sync --extra ml from the repository root and confirm that Python comes from the root .venv.
  • bgrl_example.py currently fails on import: it asks for MLP_Predictor from hugegraph_ml.models.bgrl, but that module defines the class as MLPPredictor.
  • care_gnn_example.py reads AMAZON_user_v and the three AMAZON_net_*_e edge labels. No bundled importer creates them, so load that dataset yourself before running the script.

3.3.3 - HugeGraph-LLM Workflow

This page explains the processing flow in the HugeGraph-LLM Web UI. See HugeGraph-LLM for startup instructions.

0. Configuration Panel

Above the tabs sits a collapsible configuration panel with five sections: 1. Set up the HugeGraph server., 2. Set up the LLM., 3. Set up the Embedding., 4. Set up the Reranker., and 5. Set up the vector engine.. Each section has its own apply button, and applying a change writes the supported fields back to .env. The header also shows the current prompt language.

1. Build RAG Indexes

The first tab splits documents into a chunk vector index. It also extracts vertices and edges according to a schema, writes them to HugeGraph, and maintains a vertex vector index.

flowchart TD
    A[Input document] --> B[Split text]
    B --> C[Generate chunk vectors]
    C --> D[Write vector index]
    B --> E[LLM extracts vertices and edges from schema]
    E --> F[Write to HugeGraph]
    F --> G[Update vertex vector index]

Input comes from either the text sub-tab or the file sub-tab. Uploads accept .txt, .docx, and .pdf, and several files can be selected at once.

Common operations are Import into Vector, Extract Graph Data (1), Load into GraphDB (2), and Update Vid Embedding. Load into GraphDB (2) also refreshes the vertex vector index, so the separate Update Vid Embedding step is only needed when the graph already held data. The Graph Extraction Split Type dropdown next to these buttons chooses document, paragraph, or sentence. document keeps the whole input as one unit; the other two split long documents before extraction.

The page can also inspect or clear chunk indexes, vertex indexes, and graph data. Clearing removes existing data, so first confirm that the current graph and indexes are not still used by other queries.

Two collapsed helpers sit below the main controls:

  • Graph Schema Generator takes query examples and a few-shot example and produces a schema for the Graph Schema field.
  • Graph Extraction Prompt Generator takes an expected scenario, such as social relationships or a financial knowledge graph, and a selected reference example, and produces a Graph Extract Prompt Header.

2. GraphRAG Queries

The second tab can answer directly with the LLM, use only chunk-vector retrieval, use only graph retrieval, or combine graph and vector retrieval.

flowchart TD
    Q[Question] --> V[Query chunk vector index]
    Q --> K[Extract keywords]
    K --> M[Match graph vertices]
    M --> T[Generate and execute Gremlin]
    T -->|Failure| B[Fallback to BFS graph traversal]
    T --> R[Prepare graph results]
    B --> R
    V --> S[Merge and rerank]
    R --> S
    S --> A[Generate answer]

Graph retrieval first matches HugeGraph vertices exactly by keyword and then uses vector similarity if no exact match exists. The matched vertices are passed to Text2Gremlin. If generation or execution fails, the pipeline can fall back to a predefined traversal.

Template Num controls how Text2Gremlin participates in graph retrieval:

  • A negative value skips Text2Gremlin entirely, so graph retrieval goes straight to the predefined traversal.
  • 0 generates Gremlin without any examples (zero-shot).
  • A positive value retrieves that many similar examples from the example index and uses the template-guided result. The example count is clamped to the range 0 to 10.

Other controls on this tab are Rerank method (bleu or reranker), Graph Ratio, Near neighbor first, and Query related information, plus editable Query Prompt and Keywords Extraction Prompt fields.

Below the single-question panel is a batch back-testing panel. Upload an .xlsx or .csv file of questions, set Max Lines To Show, and click Generate Answer (Batch). The answers appear in a preview table and can be downloaded as a file. A template file is offered next to the upload control.

3. Text2Gremlin

The third tab has two parts. The upper part builds the example vector index from a .json or .csv file of question and Gremlin pairs; the bundled resources/demo/text2gremlin.csv is used when no file is uploaded.

The lower part reads the graph schema, retrieves similar natural-language and Gremlin examples, fills the prompt with the question, schema, examples, and matched vertices, then generates Gremlin and optionally executes it. Number of refer examples sets how many examples are retrieved, from 0 to 10, and defaults to 2. The results appear in four fields: Gremlin with a template, Gremlin without a template, and the execution output for each.

RAG query scope selector

A custom prompt must contain {query}, {schema}, {example}, and {vertices}. The REST API rejects a request if any placeholder is missing.

4. Graph and Administration Tools

Graph Tools runs a Gremlin query directly against the configured graph, triggers a manual graph backup, and can initialize demo data in HugeGraph through a beta action. A background job also backs up the graph every day at 01:00, and a second background task keeps vertex-id embeddings up to date while the process runs.

Admin Tools is password protected. Entering the configured ADMIN_TOKEN reveals the tail of logs/llm-server.log, which refreshes every 60 seconds, along with buttons to refresh or clear it. Access is refused while ADMIN_TOKEN is empty or still set to the placeholder xxxx.

When ENABLE_LOGIN=True, the Web UI asks for basic credentials with the fixed user name rag and USER_TOKEN as the password, and the REST API requires USER_TOKEN as a Bearer token. The log endpoint additionally requires a separately configured, secure ADMIN_TOKEN.

Warning

In production, enable HugeGraph-LLM login, replace USER_TOKEN and ADMIN_TOKEN, and enforce a source IP allowlist at the firewall or network entry point. Separately enable Server authentication and authorization, retain Server audit logs (normally audit-*.log), and grant GRAPH_USER minimum required permissions. AI service tokens do not replace Server authentication.

Keywords extracted in the RAG UI

5. Prompt Language

Set LANGUAGE=EN or LANGUAGE=CN in hugegraph-llm/.env, then restart the service. This selects the language of built-in prompts; it does not translate input documents and is not a field in the /rag request body.

6. REST Calls

The Web UI and REST API use the same pipeline. For application integration, use /rag, /rag/graph, /graph/extract, and /text2gremlin; see the REST API for request formats.

3.3.4 - Configuration Reference

HugeGraph-LLM reads runtime configuration from .env and prompts from config_prompt.yaml. These files have different path-resolution rules; the prompt file is not part of .env.

The .env path is resolved in this order:

  1. HUGEGRAPH_LLM_ENV_PATH, if that environment variable is set. A leading ~ is expanded.
  2. hugegraph-llm/.env, when the package runs from a source checkout.
  3. .env in the current working directory, for an installed package.

The path is selected when the configuration module is imported. Set HUGEGRAPH_LLM_ENV_PATH before starting Python, not inside the .env file that will be loaded. Relative overrides are resolved against the process working directory.

The prompt YAML path is resolved in this order:

  1. HUGEGRAPH_LLM_PROMPT_CONFIG_PATH from the process environment, expanding a leading ~.
  2. hugegraph-llm/src/hugegraph_llm/resources/demo/config_prompt.yaml when running from source.
  3. ${XDG_CONFIG_HOME:-~/.config}/hugegraph-llm/config_prompt.yaml for an installed package.

Set path overrides before starting the process. Relative paths use its working directory. The source Docker image points PYTHONPATH at the source tree, so its default prompt path remains under hugegraph-llm/src/hugegraph_llm/resources/demo/.

Create or update files from configuration-class defaults with:

cd hugegraph-ai/hugegraph-llm
python -m hugegraph_llm.config.generate --update

--update is enabled by default, so running without arguments has the same effect. On first configuration-module import, missing .env and prompt YAML files are created from defaults. The generator asks interactively before overwriting existing files. It handles HugeGraph, administrator, LLM, index, and prompt settings without overwriting existing files silently.

.env contains keys and passwords. Do not commit it to version control.

Basic Options

SettingDefaultDescription
LANGUAGEENPrompt language: EN or CN
CHAT_LLM_TYPEopenaiAnswer model: openai, litellm, or ollama/local
EXTRACT_LLM_TYPEopenaiInformation extraction model; same choices as above
TEXT2GQL_LLM_TYPEopenaiText2Gremlin model; same choices as above
EMBEDDING_TYPEopenaiEmbedding model; same choices as above, or empty
RERANKER_TYPEemptycohere or siliconflow
KEYWORD_EXTRACT_TYPEllmllm, textrank, or hybrid
WINDOW_SIZE3TextRank window size, from 1 to 10
HYBRID_LLM_WEIGHTS0.5Weight of LLM results in hybrid mode, from 0 to 1

OpenAI-Compatible APIs

Chat, extraction, and Text2Gremlin can use different endpoints, keys, and models.

PurposeAPI baseKeyModelDefault maximum tokens
AnswerOPENAI_CHAT_API_BASEOPENAI_CHAT_API_KEYOPENAI_CHAT_LANGUAGE_MODELOPENAI_CHAT_TOKENS=8192
ExtractionOPENAI_EXTRACT_API_BASEOPENAI_EXTRACT_API_KEYOPENAI_EXTRACT_LANGUAGE_MODELOPENAI_EXTRACT_TOKENS=256
Text2GremlinOPENAI_TEXT2GQL_API_BASEOPENAI_TEXT2GQL_API_KEYOPENAI_TEXT2GQL_LANGUAGE_MODELOPENAI_TEXT2GQL_TOKENS=4096
EmbeddingOPENAI_EMBEDDING_API_BASEOPENAI_EMBEDDING_API_KEYOPENAI_EMBEDDING_MODELNot applicable

The default API base is https://api.openai.com/v1. The default language model for all three tasks is gpt-4.1-mini, and the default embedding model is text-embedding-3-small.

OPENAI_BASE_URL and OPENAI_API_KEY provide general fallback values. Embeddings also support OPENAI_EMBEDDING_BASE_URL and OPENAI_EMBEDDING_API_KEY as fallback values.

LiteLLM

PurposeAPI baseKeyModelDefault maximum tokens
AnswerLITELLM_CHAT_API_BASELITELLM_CHAT_API_KEYLITELLM_CHAT_LANGUAGE_MODELLITELLM_CHAT_TOKENS=8192
ExtractionLITELLM_EXTRACT_API_BASELITELLM_EXTRACT_API_KEYLITELLM_EXTRACT_LANGUAGE_MODELLITELLM_EXTRACT_TOKENS=256
Text2GremlinLITELLM_TEXT2GQL_API_BASELITELLM_TEXT2GQL_API_KEYLITELLM_TEXT2GQL_LANGUAGE_MODELLITELLM_TEXT2GQL_TOKENS=4096
EmbeddingLITELLM_EMBEDDING_API_BASELITELLM_EMBEDDING_API_KEYLITELLM_EMBEDDING_MODELNot applicable

The default language model is openai/gpt-4.1-mini, and the default embedding model is openai/text-embedding-3-small. Model names generally use the provider/model form; supported values depend on the LiteLLM service.

Ollama

PurposeHostPortModel
AnswerOLLAMA_CHAT_HOSTOLLAMA_CHAT_PORTOLLAMA_CHAT_LANGUAGE_MODEL
ExtractionOLLAMA_EXTRACT_HOSTOLLAMA_EXTRACT_PORTOLLAMA_EXTRACT_LANGUAGE_MODEL
Text2GremlinOLLAMA_TEXT2GQL_HOSTOLLAMA_TEXT2GQL_PORTOLLAMA_TEXT2GQL_LANGUAGE_MODEL
EmbeddingOLLAMA_EMBEDDING_HOSTOLLAMA_EMBEDDING_PORTOLLAMA_EMBEDDING_MODEL

The default host is 127.0.0.1 and the default port is 11434. Model names have no defaults; pull the required models in Ollama before use.

Reranking

SettingDefaultDescription
COHERE_BASE_URLhttps://api.cohere.com/v1/rerankCohere rerank endpoint; CO_API_URL is a fallback
RERANKER_API_KEYemptyCohere or SiliconFlow key
RERANKER_MODELemptyModel name supported by the service

HugeGraph Connection and Retrieval Limits

SettingDefaultDescription
GRAPH_URL127.0.0.1:8080HugeGraph address; it is not split into IP and port
GRAPH_NAMEhugegraphGraph name
GRAPH_USERadminUser name
GRAPH_PWDxxxPassword
GRAPH_SPACEemptyGraphSpace name
LIMIT_PROPERTYFalseWhether to limit returned properties; read as a string by the configuration class
MAX_GRAPH_PATH10Maximum graph path length
MAX_GRAPH_ITEMS30Maximum number of graph retrieval items
EDGE_LIMIT_PRE_LABEL8Result limit for each edge label
VECTOR_DIS_THRESHOLD0.9Results beyond this vector-distance threshold are ignored
TOPK_PER_KEYWORD1Candidates per keyword
TOPK_RETURN_RESULTS20Results returned after reranking

Vector Index Backend

SettingDefaultDescription
CUR_VECTOR_INDEXFaissActive vector store: Faiss, Milvus, or Qdrant
QDRANT_HOSTempty
QDRANT_PORT6333
QDRANT_API_KEYempty
MILVUS_HOSTempty
MILVUS_PORT19530
MILVUS_USERempty
MILVUS_PASSWORDempty

FAISS is local and needs no extra dependency. Selecting Milvus or Qdrant without the optional dependencies raises an error that names the missing package, so install them first:

cd hugegraph-ai
uv sync --package hugegraph-llm --extra vectordb

The same choice is available in the 5. Set up the vector engine. panel of the Web UI, which also persists the connection settings for the selected engine.

Login and Log API

SettingDefaultDescription
ENABLE_LOGINFalseWhether to require a Bearer token; read as a string by the configuration class
USER_TOKEN4321Token for the Web UI and regular APIs
ADMIN_TOKENxxxxAdministrator token used by /logs

/logs returns 403 when ADMIN_TOKEN is empty or still set to xxxx.

Warning

In production, set ENABLE_LOGIN=True, replace USER_TOKEN and ADMIN_TOKEN, and enforce a source IP allowlist at the firewall or network entry point. This protects only HugeGraph-LLM. Separately enable Server authentication and authorization, retain Server audit logs (normally audit-*.log), and grant GRAPH_USER minimum required permissions. The two services use different credentials.

Minimal OpenAI Configuration

LANGUAGE=EN
CHAT_LLM_TYPE=openai
EXTRACT_LLM_TYPE=openai
TEXT2GQL_LLM_TYPE=openai
EMBEDDING_TYPE=openai

OPENAI_API_KEY=your-api-key
OPENAI_BASE_URL=https://api.openai.com/v1
OPENAI_CHAT_LANGUAGE_MODEL=gpt-4.1-mini
OPENAI_EXTRACT_LANGUAGE_MODEL=gpt-4.1-mini
OPENAI_TEXT2GQL_LANGUAGE_MODEL=gpt-4.1-mini
OPENAI_EMBEDDING_MODEL=text-embedding-3-small

GRAPH_URL=127.0.0.1:8080
GRAPH_NAME=hugegraph
GRAPH_USER=admin
GRAPH_PWD=your-password

Configuration Loading

Configuration classes supply code defaults. During initialization, values from the selected .env are written into the process environment before the configuration objects are created, overriding same-named shell variables. Missing keys use defaults; empty values and unknown keys are ignored. The Web UI and configuration APIs can update settings and write supported fields back to .env. Restart after manual .env edits; prompt YAML is read at service startup or page load.

Keys are matched case-insensitively.

Configuration definitions are in:

  • hugegraph-llm/src/hugegraph_llm/config/llm_config.py
  • hugegraph-llm/src/hugegraph_llm/config/hugegraph_config.py
  • hugegraph-llm/src/hugegraph_llm/config/index_config.py
  • hugegraph-llm/src/hugegraph_llm/config/admin_config.py
  • hugegraph-llm/src/hugegraph_llm/config/prompt_config.py
  • hugegraph-llm/src/hugegraph_llm/config/models/base_config.py for the loading and file-sync behaviour

3.3.5 - HugeGraph-LLM REST API

The HugeGraph-LLM demo process serves both the Web UI and REST API. The default address is http://localhost:8001:

cd hugegraph-ai/hugegraph-llm
python -m hugegraph_llm.demo.rag_demo.app \
  --host 127.0.0.1 \
  --port 8001

All endpoints are POST:

PathSuccess statusPurpose
/rag200Answer a question with the selected retrieval modes
/rag/graph200Graph retrieval only, without a final answer
/graph/extract200Extract vertices and edges from text
/text2gremlin200Generate Gremlin from natural language
/config/graph201Update the HugeGraph connection
/config/llm201Update the language model
/config/embedding201Update the embedding model
/config/rerank201Update the reranker
/logs200Stream the server log

Authentication

Enable login in .env:

ENABLE_LOGIN=True
USER_TOKEN=replace-with-a-secret

Requests then require a Bearer token:

Authorization: Bearer replace-with-a-secret

The same setting puts the Gradio UI behind basic authentication, with the fixed user name rag and USER_TOKEN as the password. A wrong token returns 401 with a WWW-Authenticate: Bearer header. When ENABLE_LOGIN is left at False, every endpoint is open.

Warning

Production requires HugeGraph-LLM login and a source IP allowlist at the firewall or network entry point. This protects only the LLM UI and REST API. Separately enable Server authentication and authorization, retain Server audit logs (normally audit-*.log), and grant GRAPH_USER minimum permissions. USER_TOKEN does not replace Server authentication.

RAG

POST /rag

Returns one or more answer types according to the switches. When none is explicitly selected, only graph_only is enabled.

curl -X POST http://localhost:8001/rag \
  -H 'Content-Type: application/json' \
  -d '{
    "query": "Which movies feature Al Pacino?",
    "raw_answer": false,
    "vector_only": false,
    "graph_only": true,
    "graph_vector_answer": false,
    "max_graph_items": 30,
    "topk_return_results": 20,
    "vector_dis_threshold": 0.9,
    "topk_per_keyword": 1,
    "gremlin_tmpl_num": 1,
    "client_config": {
      "url": "127.0.0.1:8080",
      "graph": "hugegraph",
      "user": "admin",
      "pwd": "admin",
      "gs": "DEFAULT"
    }
  }'

The response contains only enabled answer fields:

{
  "query": "Which movies feature Al Pacino?",
  "graph_only": "..."
}

Other optional parameters include graph_ratio (default 0.5), rerank_method (bleu or reranker, default bleu), near_neighbor_first (default false), custom_priority_info, and the three custom prompt fields answer_prompt, keywords_extract_prompt, and gremlin_prompt. Omitting a prompt field uses the value from config_prompt.yaml.

gremlin_tmpl_num selects how Text2Gremlin runs during graph retrieval. A negative value skips Text2Gremlin and goes straight to the predefined traversal, 0 generates Gremlin without examples, and a positive value retrieves that many examples from the example index.

An empty or whitespace-only query returns 400.

POST /rag/graph

Runs graph retrieval without generating a final natural-language answer:

curl -X POST http://localhost:8001/rag/graph \
  -H 'Content-Type: application/json' \
  -d '{
    "query": "Which movies feature Al Pacino?",
    "get_vertex_only": false,
    "gremlin_tmpl_num": 1,
    "rerank_method": "bleu"
  }'

graph_recall in the response can contain query, keywords, match_vids, graph_result_flag, gremlin, graph_result, and vertex_degree_list. Set get_vertex_only=true to return immediately after vertex matching; the endpoint then replaces match_vids with the full vertex details.

An empty query returns 400, a type error in the request returns 400, and any other failure returns 500.

Graph Extraction

POST /graph/extract

An inline schema does not connect to HugeGraph:

curl -X POST http://localhost:8001/graph/extract \
  -H 'Content-Type: application/json' \
  -d '{
    "texts": ["Alice works at Acme."],
    "schema": {
      "vertexlabels": [
        {"name": "person", "properties": ["name"]},
        {"name": "company", "properties": ["name"]}
      ],
      "edgelabels": [
        {
          "name": "works_at",
          "source_label": "person",
          "target_label": "company",
          "properties": []
        }
      ]
    },
    "language": "en",
    "split_type": "sentence",
    "include_meta": true
  }'

Request fields:

FieldDefaultNotes
textsrequiredA string or an array of strings; empty or blank entries are dropped and an empty result is rejected
schemarequiredInline JSON object or string, or the name of an existing graph
example_promptprompt YAML valueExtraction prompt header
extract_typeproperty_graphOnly value currently accepted
languagezhzh or en, used for chunk splitting
split_typedocumentdocument, paragraph, or sentence
include_metafalseAdds vertex_count, edge_count, and text_count to meta
client_confignoneOnly allowed with a graph-name schema

An inline schema must be an object with vertexlabels and edgelabels lists. Every vertex label needs a non-empty name and a non-empty properties list; every edge label needs a non-empty name, source_label, and target_label. propertykeys is optional and must be a list when present.

When schema is an existing graph name, also pass client_config, and make client_config.graph match that name. client_config here accepts only graph, user, pwd, and gs; unknown fields are rejected, and there is no url field:

{
  "texts": "Alice works at Acme.",
  "schema": "hugegraph",
  "client_config": {
    "graph": "hugegraph",
    "user": "admin",
    "pwd": "admin",
    "gs": "DEFAULT"
  }
}

A successful response always contains status (always succeeded), result.vertices, result.edges, warnings, and meta. meta stays empty unless include_meta is true.

Text2Gremlin

POST /text2gremlin

curl -X POST http://localhost:8001/text2gremlin \
  -H 'Content-Type: application/json' \
  -d '{
    "query": "Find all person vertices",
    "example_num": 1,
    "output_types": ["template_gremlin", "template_execution_result"]
  }'

output_types can contain:

  • match_result
  • template_gremlin
  • raw_gremlin
  • template_execution_result
  • raw_execution_result

If omitted, only template_gremlin is returned by default. An empty array lets the implementation return all outputs. A custom gremlin_prompt must contain {query}, {schema}, {example}, and {vertices}; a missing placeholder fails request validation and names the placeholders that are absent.

example_num defaults to 0, which means no templates, and is clamped to the range 0 to 10. client_config overrides the HugeGraph connection for the request; the schema used for generation is the active graph name. An empty query returns 400, and a generation failure returns 500.

Runtime Configuration

POST /config/graph

{
  "url": "127.0.0.1:8080",
  "graph": "hugegraph",
  "user": "admin",
  "pwd": "admin",
  "gs": "DEFAULT"
}

user and pwd default to empty strings, and gs is optional.

POST /config/llm and POST /config/embedding

Both endpoints use the same request model. /config/llm sets chat_llm_type, extract_llm_type, and text2gql_llm_type to the same value; per-task types can only be set separately through .env or the Web UI. OpenAI or LiteLLM example:

{
  "llm_type": "openai",
  "api_key": "your-key",
  "api_base": "https://api.openai.com/v1",
  "language_model": "gpt-4.1-mini",
  "max_tokens": "4096"
}

Ollama requests still require the common fields; api_key and api_base can be empty strings:

{
  "llm_type": "ollama/local",
  "api_key": "",
  "api_base": "",
  "language_model": "qwen2.5:7b",
  "host": "127.0.0.1",
  "port": "11434"
}

POST /config/rerank

{
  "reranker_type": "siliconflow",
  "reranker_model": "BAAI/bge-reranker-v2-m3",
  "api_key": "your-key"
}

reranker_type accepts cohere or siliconflow. Cohere also accepts cohere_base_url.

All four configuration endpoints return 201 on success. They change the process’s active configuration and may write values back to .env. /config/llm, /config/embedding, and /config/rerank restore the previous values if applying a change raises; /config/graph does not.

client_config in /rag, /rag/graph, and /text2gremlin overrides the HugeGraph connection for one request, and only the fields actually present in the request are applied. The current implementation still changes process-global settings temporarily, so do not issue long-running requests with different connections concurrently.

Logs

POST /logs

This endpoint requires ADMIN_TOKEN in .env to be changed to a secure value. Example request body:

{
  "admin_token": "replace-with-an-admin-secret",
  "log_file": "llm-server.log"
}

log_file defaults to llm-server.log, must be a file name under logs/, and cannot be absolute, contain path separators, or resolve to . or ... Invalid names return 400.

An unset or placeholder ADMIN_TOKEN returns 403 before the token is even compared, and a wrong token returns a 403 body with the message Invalid admin_token.

The successful response is a text/plain stream that first replays the last 125 lines of the file and then follows it, in the manner of tail -f.

3.3.6 - Vermeer Python Client

vermeer-python-client is the Python SDK for Vermeer, the memory-first graph computing engine written in Go. The SDK wraps the REST API of the Vermeer master so you can list graphs, submit load and compute tasks, and read task state from Python. The import package is pyvermeer.

The module does not pin a Vermeer server version. It talks to the Vermeer master over HTTP using the endpoints listed in API Surface.

Requirements

  • Current source requires Python 3.10 or later because its type annotations use Python 3.10 features, although packaging metadata still declares >=3.9.
  • A running Vermeer master reachable over HTTP on its default port 6688. Docker deployments must publish 6688:6688; see the Vermeer quick start.
  • uv (recommended) or pip

Runtime dependencies: requests, urllib3, python-dateutil, decorator, rich, and setuptools.

Installation

The distribution name in the packaging metadata is vermeer-python-client and the version is managed independently of the repository version. The package is not published on PyPI yet, so install it from source.

From the root of the HugeGraph-AI repository, the vermeer extra installs it into the shared virtual environment:

git clone https://github.com/apache/hugegraph-ai.git
cd hugegraph-ai
uv sync --extra vermeer
source .venv/bin/activate

vermeer-python-client is wired in as an editable path dependency rather than a uv workspace member, so a plain uv sync at the repository root does not install it. You have to ask for the extra (or for --all-extras).

To install the module standalone:

git clone https://github.com/apache/hugegraph-ai.git
cd hugegraph-ai/vermeer-python-client
uv sync
source .venv/bin/activate

Connect to a Vermeer Master

from pyvermeer.client.client import PyVermeerClient

client = PyVermeerClient(
    ip="127.0.0.1",
    port=6688,
    token="",
    timeout=(0.5, 15.0),
    log_level="INFO",
)

Constructor parameters:

ParameterTypeDefaultDescription
ipstrrequiredHost name or IP address of the Vermeer master
portintrequiredREST port of the Vermeer master
tokenstrrequiredSent verbatim as the Authorization request header
timeout(float, float) or NoneNoneConnect and read timeouts in seconds
log_levelstr"INFO"Level applied to the shared VermeerClient logger

For a client running on the host, use a master address reachable from that host. For a container client, use its container-network address. The client connects only to the master HTTP API; PD and Store addresses must be reachable from the Vermeer nodes that perform loading.

Behavior worth knowing before you connect:

  • token may be an empty string when the master does not check authorization, but it cannot be None. The session raises ValueError("Vermeer Token must be provided.") in that case.
  • timeout is a (connect, read) pair. VermeerConfig has its own default of (0.5, 15.0), but the client always forwards its own argument, so omitting timeout stores None and the request waits without a deadline. Pass the pair explicitly if you want one.
  • The base URL is always built as http://{ip}:{port}/, so the client speaks plain HTTP.
  • Every request sets Content-Type: application/json and serializes params into the request body, including for GET requests.
  • The session configures up to 3 retries on HTTP 500, 502, and 504 with backoff factor 0.1. Status retries follow urllib3’s default allowed methods, which exclude POST; do not rely on automatic retries for task submission.
  • log_level sets the level of the shared logger named VermeerClient. Its console handler is fixed at INFO, so DEBUG records are not printed to the console today.

End-to-End Example

The module ships vermeer-python-client/src/pyvermeer/demo/task_demo.py. This extended example polls with a deadline, waits for loading, reads the graph, and submits PageRank. It reads PD peers and the HugeGraph password from environment variables. The worker group must match the task-space allocation; a single-worker quick start can use worker_group=$. Bind a named group to the task space first, as shown in the Vermeer quick start.

import os
import time

from pyvermeer.client.client import PyVermeerClient
from pyvermeer.structure.task_data import TaskCreateRequest

client = PyVermeerClient(
    ip="127.0.0.1",
    port=6688,
    token=os.getenv("VERMEER_TOKEN", ""),
    timeout=(0.5, 15.0),
    log_level="INFO",
)

# List tasks on the master
tasks = client.tasks.get_tasks()
print(tasks.to_dict())

# Load graph data from HugeGraph into Vermeer
create_response = client.tasks.create_task(
    create_task=TaskCreateRequest(
        task_type="load",
        graph_name="DEFAULT-example",
        params={
            "load.hg_pd_peers": os.environ["VERMEER_PD_PEERS"],
            "load.hugegraph_name": "DEFAULT/example/g",
            "load.hugegraph_username": "admin",
            "load.hugegraph_password": os.environ["HUGEGRAPH_PASSWORD"],
            "load.parallel": "10",
            "load.type": "hugegraph",
        },
    )
)
print(create_response.errcode, create_response.message)
if create_response.errcode != 0:
    raise RuntimeError(f"Could not create load task: {create_response.message}")

def wait_task(task_id, success_state, poll_timeout=300.0):
    deadline = time.monotonic() + poll_timeout
    while True:
        response = client.tasks.get_task(task_id)
        if response.errcode != 0:
            raise RuntimeError(f"Could not read task {task_id}: {response.message}")
        task = response.task
        print(task_id, task.state)
        if task.state == success_state:
            return task
        if task.state in ("error", "canceled"):
            raise RuntimeError(f"Task {task_id} ended with state {task.state}")
        remaining = deadline - time.monotonic()
        if remaining <= 0:
            raise TimeoutError(f"Task {task_id} did not finish within {poll_timeout}s")
        time.sleep(min(1.0, remaining))


# Wait for loading before reading the graph
wait_task(create_response.task.id, success_state="loaded")

# Inspect the loaded graph
print(client.graph.get_graph("DEFAULT-example").to_dict())

# Submit PageRank on the loaded graph
compute_response = client.tasks.create_task(
    create_task=TaskCreateRequest(
        task_type="compute",
        graph_name="DEFAULT-example",
        params={
            "compute.algorithm": "pagerank",
            "compute.parallel": "10",
            "compute.max_step": "10",
            "output.type": "local",
            "output.parallel": "1",
            "output.file_path": "result/pagerank",
        },
    )
)
if compute_response.errcode != 0:
    raise RuntimeError(f"Could not create compute task: {compute_response.message}")
wait_task(compute_response.task.id, success_state="complete")

Loading succeeds with loaded; computation succeeds with complete. error or canceled stops the example. Set VERMEER_PD_PEERS to a JSON array string, for example export VERMEER_PD_PEERS='["hugegraph-pd:8686"]'. The master must reach these PD addresses, and workers must reach the Store addresses returned by PD. Set VERMEER_TOKEN only if the master uses token authentication; an empty string works without it.

output.type=local writes results to the executing worker’s local filesystem with file-name prefix result/pagerank. Mount its result directory to read container output from the host. poll_timeout is the client polling deadline; requests between checks remain subject to HTTP timeouts and SDK retries, so completion can exceed that deadline. A timeout stops client waiting, not the Server task.

Never hardcode a real HugeGraph password into a script or configuration file. Use an environment variable or credential store.

Save and Run the Documentation Example

Save the complete extended example above as vermeer_client_example.py at the hugegraph-ai/ repository root. Run it with the workspace’s vermeer extra:

uv sync --extra vermeer
export VERMEER_PD_PEERS='["hugegraph-pd:8686"]'
read -s -r HUGEGRAPH_PASSWORD
export HUGEGRAPH_PASSWORD
uv run --extra vermeer python vermeer_client_example.py

Enter the password and press Enter. Replace hugegraph-pd:8686 with a PD address reachable from the Vermeer master. If token authentication is enabled, supply VERMEER_TOKEN through your local credential-management method. These commands run the extended documentation example, not the bundled demo.

Original Bundled Demo

vermeer-python-client/src/pyvermeer/demo/task_demo.py is a separate minimal example. It hardcodes client port 8688, PD address 127.0.0.1:8686, and placeholder credentials xxx. It does not read the extended example’s environment variables, poll task state, or compute PageRank. Adjust its addresses and credentials separately if running it; it does not replace the documentation commands above.

API Surface

PyVermeerClient exposes its API groups as attributes. Two groups are registered today, graph and tasks.

client.graph

MethodVermeer endpointReturns
get_graphs()GET /graphsGraphsResponse
get_graph(graph_name)GET /graphs/{graph_name}GraphResponse

client.tasks

MethodVermeer endpointReturns
get_tasks()GET /tasksTasksResponse
get_task(task_id)GET /task/{task_id}TaskResponse
create_task(create_task)POST /tasks/createTaskCreateResponse

pyvermeer/api/master.py and pyvermeer/api/worker.py contain only the license header, and neither group is registered on the client. Master and worker information is therefore not reachable from the client yet, even though MasterResponse and WorkersResponse already exist under pyvermeer/structure/.

client.send_request(method, endpoint, params) is the shared entry point behind both groups. You can call it directly to reach a Vermeer endpoint that has no wrapper yet; it returns the decoded JSON body as a plain dict.

Requests and Responses

TaskCreateRequest(task_type, graph_name, params) is serialized as {"task_type": ..., "graph": ..., "params": ...}. Note that graph_name becomes graph on the wire, which matches the payload documented for the Vermeer REST API.

Every response type extends BaseResponse and exposes errcode and message, plus a to_dict() helper. errcode is 0 on success and 1 on error; -1 means the field was missing from the response body.

  • GraphsResponse.graphs and GraphResponse.graph yield VermeerGraph objects with name, space_name, status, create_time, update_time, vertex_count, edge_count, workers, worker_group, use_out_edges, use_property, use_out_degree, use_undirected, on_disk, and backend_option.
  • TasksResponse.tasks, TaskResponse.task, and TaskCreateResponse.task yield TaskInfo objects with id, state, create_user, create_type, create_time, start_time, update_time, graph_name, space_name, params, workers, and error_message.
  • Timestamps are parsed with python-dateutil into datetime objects. An empty timestamp string becomes None.

Server returns the task type as task_type, but SDK TaskInfo.type reads type, so that property is currently empty. The SDK also drops Server statistics_result. Use TaskInfo.state for polling. This is a field-contract mismatch in current source.

Task Parameters

The client does not validate params. Keys and values are passed straight through to Vermeer, so the accepted names come from the engine, not from the SDK. For the load parameters and the parameters of the supported algorithms, see the Vermeer quick start.

The usual sequence is the same as with the REST API directly: create a load task to read the graph into Vermeer, wait for it to finish, then create computation tasks against the loaded graph.

Errors

pyvermeer.utils.exception defines four exceptions, all raised from the underlying requests or JSON failure:

ExceptionRaised when
ConnectErrorrequests.ConnectionError, the master is unreachable
TimeOutErrorrequests.Timeout, the connect or read deadline expired
JsonDecodeErrorThe response body is not valid JSON
UnknownErrorAny other failure during the request
from pyvermeer.utils.exception import ConnectError, TimeOutError

try:
    graphs = client.graph.get_graphs()
except (ConnectError, TimeOutError) as error:
    print(error)

The client does not check the HTTP status code of the response, so inspect errcode and message on the returned object to tell success from a Vermeer-side error.

Development Checks

Run the formatting and static checks from the root of the HugeGraph-AI repository:

./style/code_format_and_analysis.sh

The source lives under vermeer-python-client/src/pyvermeer/. A few structure tests are available under src/tests/structure/test_task_data.py; they do not cover HTTP API integration.

References

3.4 - HugeGraph Computing (OLAP)

The HugeGraph-Computer repository contains two graph computing systems with different deployment and runtime models. Start with Go Vermeer for general graph algorithms; use Computer when you need distributed Java BSP/Pregel computation. Both can connect to HugeGraph, but their configuration and job entry points are not interchangeable.

Default entry: Go Vermeer

flowchart TB
    Master["Master"] --> Workers["Workers"]
    Master -->|PD| PD["PD"]
    Workers -->|Scan| Store["Store"]
    Workers -.->|REST| Server["Server"]

With load.type=hugegraph, the Vermeer master queries PD for partition metadata and workers scan HStore Store partitions directly. Workers write results through the Server REST API only when output.type=hugegraph; other input sources or output settings do not require those connections.

Java Computer

flowchart TB
    Config["Job configuration"] --> Master["Master"] --> Workers["Workers"]
    Master -. BSP .-> Etcd["etcd"]
    Workers <-->|REST| Server["Server"]
    Workers <-->|HDFS| HDFS["HDFS (optional)"]

Job configuration can be submitted through the Kubernetes Operator or YARN. Workers read graph data through the HugeGraph Server REST API and can write results back according to the output configuration. With HDFS input or output, workers access HDFS directly. The master uses etcd to coordinate BSP jobs.

3.4.1 - HugeGraph-Vermeer Quick Start

Vermeer high-performance in-memory graph computing: start once, execute repeatedly, with 15+ OLAP algorithms, seconds-to-minutes execution, deployment, loading, PageRank, and community detection.

1. Overview of Vermeer

1.1 Architecture

Vermeer is a high-performance, memory-first graph computing framework written in Go: start once and execute repeatedly. It supports fast execution of 15+ OLAP algorithms, often in seconds to minutes; actual time depends on graph size, algorithm parameters, and resources. A single master currently schedules multiple workers.

The master handles communication, forwarding, and aggregation, with modest computation and resource usage. Workers store graph data and execute tasks, consuming most memory and CPU. gRPC handles internal communication; REST provides external APIs.

At startup, built-in defaults are loaded first, followed by the [default] section of config/<env>.ini under the working directory, then explicit command-line overrides. For example, --env=master reads config/master.ini. The Docker image copies repository configuration to /go/bin/config/ and works from /go/bin/. A host bind mount hides the bundled files, so it must contain every ini file used. The configuration reader does not read environment variables.

Default ports:

RoleConfiguration keyDefault addressPurpose
masterhttp_peer0.0.0.0:6688REST API for host-side clients
mastergrpc_peer0.0.0.0:6689Workers connect to master
workerhttp_peer0.0.0.0:6788Worker HTTP service
workergrpc_peer0.0.0.0:6789Worker gRPC listener, also advertised to master for node communication

The Docker examples publish only master HTTP on host loopback 127.0.0.1:6688:6688; master and worker gRPC ports stay within the container network.

flowchart LR
  Client["curl / Python client"] -->|"HTTP :6688"| Master["master"]
  Worker["worker"] <-->|"Bidirectional gRPC: master :6689, worker :6789"| Master
  Master -->|"gRPC partition lookup"| PD["HugeGraph PD"]
  Worker -->|"gRPC partition scan"| Store["HugeGraph Store"]

1.2 Running Vermeer

Warning

In production, enable Server authentication and authorization, an IP allowlist, and minimum permissions; retain audit-*.log. Server Auth does not protect independent Vermeer, PD, or Store APIs. Restrict these HTTP/gRPC ports to trusted networks and callers, and configure access controls at the Vermeer external entry point.

master.ini defaults to auth=none, disabling authentication for ordinary and administrative APIs. Local examples publish loopback only. Before remote access, enable auth=token or restrict callers through a protected network or gateway.

Both Docker options need a host configuration directory. From the Vermeer repository root, copy the supplied master.ini and worker.ini templates. The mount hides image configuration under /go/bin/config, so never mount an empty directory or your entire home directory there:

CONFIG_DIR="$HOME/vermeer-config"
mkdir -p "$CONFIG_DIR"
cp config/master.ini config/worker.ini "$CONFIG_DIR/"

Keep master HTTP/gRPC listeners at 0.0.0.0:6688 and 0.0.0.0:6689. The Compose example assigns 172.20.0.10 to master and 172.20.0.11 to worker; set these options in the copied worker.ini:

[default]
http_peer=0.0.0.0:6788
grpc_peer=172.20.0.11:6789
master_peer=172.20.0.10:6689
run_mode=worker
worker_group=$

This single-worker example uses literal worker_group=$, the general group used when no named group is bound. The repository template defaults to worker_group=default; if retaining that named group, bind it to the task space or graph before submitting tasks. For default space $DEFAULT, use:

curl --fail --show-error -X POST \
  'http://localhost:6688/admin/workers/alloc/default/%24DEFAULT'

An errcode=0 response confirms binding; with token authentication, also supply the authorization header. master_peer must reach master gRPC from the worker network. grpc_peer is both the listener and advertised worker address, so do not advertise 0.0.0.0, which other containers cannot reach. Update these IPs when changing the subnet or addresses; each additional worker needs a unique, mutually reachable grpc_peer.

  1. Option 1: Docker Compose (Recommended)

Run from the Vermeer repository root. Modify existing docker-compose.yaml or use the following example. The repository file currently mounts all of ~/ as configuration and does not publish master HTTP; replace the mount and add the port mapping before starting:

services:
  vermeer-master:
    image: hugegraph/vermeer
    container_name: vermeer-master
    ports:
      - "127.0.0.1:6688:6688"
    volumes:
      - /home/user/vermeer-config:/go/bin/config:ro
    command: --env=master
    networks:
      vermeer_network:
        ipv4_address: 172.20.0.10 # Static master address

  vermeer-worker:
    image: hugegraph/vermeer
    container_name: vermeer-worker
    volumes:
      - /home/user/vermeer-config:/go/bin/config:ro
    command: --env=worker
    networks:
      vermeer_network:
        ipv4_address: 172.20.0.11 # Static worker address

networks:
  vermeer_network:
    driver: bridge
    ipam:
      config:
        - subnet: 172.20.0.0/24 # Change the subnet as needed

Replace /home/user/vermeer-config with your actual absolute configuration directory. Changing the subnet or static IPs of vermeer_network also requires updating grpc_peer and master_peer in worker.ini. Do not use the default grpc_peer=0.0.0.0:6789 as an advertised container address.

Build and start from the project directory:

# Build the image from the Vermeer root directory
docker build -t hugegraph/vermeer .

# Start from the Vermeer root directory
docker-compose up -d
# Or use the newer CLI:
# docker compose up -d

View logs / stop / remove:

docker-compose logs -f
docker-compose down
  1. Option 2: Separate docker run Commands (Manual Network and Static IPs)

Set CONFIG_DIR to the prepared configuration directory, with grpc_peer and master_peer matching the static container addresses. Replace the example CONFIG_DIR with your actual absolute path.

Build the image:

docker build -t hugegraph/vermeer .

Create a custom bridge network (once):

docker network create --driver bridge \
  --subnet 172.20.0.0/24 \
  vermeer_network

Run master (use your absolute CONFIG_DIR and adjust IPs as needed):

CONFIG_DIR=/home/user/vermeer-config

docker run -d \
  --name vermeer-master \
  --network vermeer_network --ip 172.20.0.10 \
  -p 127.0.0.1:6688:6688 \
  -v ${CONFIG_DIR}:/go/bin/config:ro \
  hugegraph/vermeer \
  --env=master

Run worker:

docker run -d \
  --name vermeer-worker \
  --network vermeer_network --ip 172.20.0.11 \
  -v ${CONFIG_DIR}:/go/bin/config:ro \
  hugegraph/vermeer \
  --env=worker

View logs / stop / remove:

docker logs -f vermeer-master
docker logs -f vermeer-worker

docker stop vermeer-master vermeer-worker
docker rm vermeer-master vermeer-worker

# Delete the custom network if needed
docker network rm vermeer_network
  1. Option 3: Build from Source

Build following the Vermeer README.

go build

Start from the Vermeer root directory with ./vermeer --env=master and ./vermeer --env=worker01. In worker01.ini, set grpc_peer to an address bindable locally and reachable by master and other workers; point master_peer to master gRPC.

After starting master, check its HTTP port from the host:

curl --fail --show-error http://localhost:6688/graphs

Expect HTTP 200 and JSON errcode=0.

2. Task Creation REST API

2.1 Introduction

Submit a load task, wait for loading to finish, then submit a compute task. A loaded graph can support repeated computations and is not deleted after completion. Asynchronous APIs return creation results and task information, including ID, before completion; synchronous APIs wait for success or failure, so client and proxy HTTP timeouts must be sufficiently long. Query task states: loaded for successful loading, complete for successful computation, error for failure, and canceled for cancellation; other states remain waiting or running. Graphs loading or in an error state cannot be computed. Deletion requires a deletable graph state and no current usage.

Available URLs:

  • Asynchronous: POST http://master_ip:port/tasks/create; read the ID from response task.id.
  • Synchronous: POST http://master_ip:port/tasks/create/sync; returns after the task ends.
  • Query task: GET http://master_ip:port/task/{task_id}. errcode=0 indicates a successful query; inspect task.state for task status. Set a client polling deadline for asynchronous jobs; reaching it stops client waiting without canceling the server task.

2.2 Loading Graph Data

The examples list common load parameters. Each loader reads its own keys; unused keys do not change behavior.

Vermeer provides three loading methods:

  1. Local files

load.vertex_files and load.edge_files map worker address host portions to file paths read by those workers. With containers, mount data into the worker and use container paths. In the Compose example, mount a host data directory at worker /data (such as - /host/data:/data:ro) and use 172.20.0.11 from grpc_peer as the mapping key. Other deployments use the host portion of their advertised worker addresses.

Obtain a dataset such as Twitter-2010; the first twitter-2010.txt.gz file is sufficient.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "load",
 "graph": "testdb",
 "params": {
  "load.parallel": "50",
  "load.type": "local",
  "load.vertex_files": "{\"172.20.0.11\":\"/data/twitter-2010.v_[0,99]\"}",
  "load.edge_files": "{\"172.20.0.11\":\"/data/twitter-2010.e_[0,99]\"}",
  "load.use_out_degree": "1",
  "load.use_outedge": "1"
 }
}
  1. HugeGraph

Request example:

Replace request addresses, graph name, and credentials with your actual connection settings.

POST http://localhost:6688/tasks/create
{
  "task_type": "load",
  "graph": "testdb",
  "params": {
    "load.parallel": "50",
    "load.type": "hugegraph",
    "load.hg_pd_peers": "[\"<pd-address-reachable-from-vermeer>:8686\"]",
    "load.hugegraph_name": "DEFAULT/hugegraph2/g",
    "load.hugegraph_username": "admin",
    "load.hugegraph_password": "<your-password-here>",
    "load.use_out_degree": "1",
    "load.use_outedge": "1"
  }
}

Vermeer master connects to PD through load.hg_pd_peers to look up partitions; workers read data from the returned Store addresses. These services must be reachable from the corresponding Vermeer hosts or containers. Inside Docker, 127.0.0.1 identifies the container itself, not the host or another container.

  1. HDFS

Request example:

POST http://localhost:6688/tasks/create
{
  "task_type": "load",
  "graph": "testdb",
  "params": {
    "load.parallel": "50",
    "load.type": "hdfs",
    "load.hdfs_namenode": "name_node1:9000",
    "load.hdfs_conf_path": "/path/to/conf",
    "load.krb_realm": "EXAMPLE.COM",
    "load.krb_name": "user@EXAMPLE.COM",
    "load.krb_keytab_path": "/path/to/keytab",
    "load.krb_conf_path": "/path/to/krb5.conf",
    "load.hdfs_use_krb": "1",
    "load.vertex_files": "/data/graph/vertices",
    "load.edge_files": "/data/graph/edges",
    "load.use_out_degree": "1",
    "load.use_outedge": "1"
  }
}

2.3 Outputting Computation Results

Current result writers support local, hdfs, and hugegraph through output.type; none disables output. Source retains an afs constant, but current master registers no AFS loader or writer. Set output.need_statistics=1 to put statistics into task information; supported statistics operators depend on each algorithm implementation.

The examples list common computation and output parameters; supported algorithm parameters depend on current Vermeer implementations.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "pagerank",
 "compute.parallel": "10",
 "compute.max_step": "10",
 "output.type": "local",
 "output.parallel": "1",
 "output.file_path": "result/pagerank"
  }
}

output.type=local writes to the executing worker’s local filesystem. For containers, mount the output directory if the host needs to read results.

3. Supported Algorithms

3.1 PageRank

The PageRank algorithm, also known as the web ranking algorithm, is a technique used by search engines to calculate the relevance and importance of web pages (nodes) based on their mutual hyperlinks.

  • If a web page is linked to by many other web pages, it indicates that the web page is relatively important, and its PageRank value will be relatively high.
  • If a web page with a high PageRank value links to other web pages, the PageRank value of the linked web pages will also increase accordingly.

The PageRank algorithm is suitable for scenarios such as web page ranking and identifying key figures in social networks.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "pagerank",
 "compute.parallel":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/pagerank",
 "compute.max_step":"10"
 }
}

3.2 WCC (Weakly Connected Components)

The weakly connected components algorithm calculates all connected subgraphs in an undirected graph and outputs the weakly connected subgraph ID to which each vertex belongs, indicating the connectivity between points and distinguishing different connected communities.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "wcc",
 "compute.parallel":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/wcc",
 "compute.max_step":"10"
 }
}

3.3 LPA (Label Propagation Algorithm)

The label propagation algorithm is a graph clustering algorithm commonly used in social networks to discover potential communities.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "lpa",
 "compute.parallel":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/lpa",
 "compute.max_step":"10"
 }
}

3.4 Degree Centrality

The degree centrality algorithm calculates the degree centrality value of each node in the graph, supporting both undirected and directed graphs. Degree centrality is an important indicator of node importance; the more edges a node has with other nodes, the higher its degree centrality value, and the more important the node is in the graph. In an undirected graph, degree centrality is calculated based on edge information to count the number of times a node appears, resulting in the degree centrality value of the node. In a directed graph, it is based on the direction of the edges, filtering based on input or output-edge information to count the number of times a node appears, resulting in the in-degree or out-degree value of the node. It indicates the importance of each point, with more important points having higher degrees.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "degree",
 "compute.parallel":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/degree",
 "degree.direction":"both"
 }
}

3.5 Closeness Centrality

Closeness centrality is used to calculate the inverse of the shortest distance from a node to all other reachable nodes, accumulating and normalizing the value. Closeness centrality can be used to measure the time it takes for information to be transmitted from the node to other nodes. The larger the closeness centrality of a node, the closer its position in the graph is to the center, suitable for scenarios such as identifying key nodes in social networks.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "closeness_centrality",
 "compute.parallel":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/closeness_centrality",
 "closeness_centrality.sample_rate":"0.01"
 }
}

3.6 Betweenness Centrality

The betweenness centrality algorithm determines the value of a node as a “bridge” node; the larger the value, the more likely it is to be a necessary path between two points in the graph. Typical examples include mutual followers in social networks. It is suitable for measuring the degree of aggregation around a node in a community.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "betweenness_centrality",
 "compute.parallel":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/betweenness_centrality",
 "betweenness_centrality.sample_rate":"0.01"
 }
}

3.7 Triangle Count

The triangle count algorithm calculates the number of triangles passing through each vertex, suitable for calculating the relationships between users and whether the associations form triangles. The more triangles, the higher the degree of association between nodes in the graph, and the tighter the organizational relationship. In social networks, triangles indicate cohesive communities, and identifying triangles helps understand clustering and interconnections among individuals or groups in the network. In financial or transaction networks, the presence of triangles may indicate suspicious or fraudulent activities, and triangle counting can help identify transaction patterns that may require further investigation.

The output result is the Triangle Count corresponding to each vertex, i.e., the number of triangles the vertex is part of.

Note: This algorithm is for undirected graphs and ignores edge directions.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "triangle_count",
 "compute.parallel":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/triangle_count"
 }
}

3.8 K-Core

The K-Core algorithm marks all vertices with a degree of K, suitable for graph pruning and finding the core part of the graph.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "kcore",
 "compute.parallel":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/kcore",
 "kcore.degree_k":"5"
 }
}

3.9 SSSP (Single Source Shortest Path)

The single source the shortest path algorithm calculates the shortest distance from one point to all other points.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "sssp",
 "compute.parallel":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/degree",
 "sssp.source":"tom"
 }
}

3.10 KOUT

Starting from a point, get the k-layer nodes of this point.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "kout",
 "compute.parallel":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/kout",
 "kout.source":"tom",
 "compute.max_step":"6"
 }
}

3.11 Louvain

The Louvain algorithm is a community detection algorithm based on modularity. The basic idea is that nodes in the network try to traverse all neighbor community labels and choose the community label that maximizes the modularity increment. After maximizing modularity, each community is regarded as a new node, and the process is repeated until the modularity no longer increases.

The distributed Louvain algorithm implemented on Vermeer is affected by factors such as node order and parallel computation. Due to the random traversal order of the Louvain algorithm, community compression also has a certain randomness, leading to different results in multiple executions. However, the overall trend will not change significantly.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "louvain",
 "compute.parallel":"10",
 "compute.max_step":"1000",
 "louvain.threshold":"0.0000001",
 "louvain.resolution":"1.0",
 "louvain.step":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/louvain"
  }
 }

3.12 Jaccard Similarity Coefficient

The Jaccard index, also known as the Jaccard similarity coefficient, is used to compare the similarity and diversity between finite sample sets. The larger the Jaccard coefficient value, the higher the similarity of the samples. It is used to calculate the Jaccard similarity coefficient between a given source point and all other points in the graph.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "jaccard",
 "compute.parallel":"10",
 "compute.max_step":"2",
 "jaccard.source":"123",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/jaccard"
 }
}

3.13 Personalized PageRank

The goal of personalized PageRank is to calculate the relevance of all nodes relative to user u. Starting from the node corresponding to user u, at each node, there is a probability of 1-d to stop walking and start again from u, or a probability of d to continue walking, randomly selecting a node from the nodes pointed to by the current node to walk down. It is used to calculate the personalized PageRank score starting from a given starting point, suitable for scenarios such as social recommendations.

Since the calculation requires using out-degree, load.use_out_degree needs to be set to 1 when reading the graph.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "ppr",
 "compute.parallel":"100",
 "compute.max_step":"10",
 "ppr.source":"123",
 "ppr.damping":"0.85",
 "ppr.diff_threshold":"0.00001",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/ppr"
 }
}

3.14 Global Kout

Calculate the k-degree neighbors of all nodes in the graph (excluding themselves and 1~k-1 degree neighbors). Due to the severe memory expansion of the global kout algorithm, k is currently limited to 1 and 2. Additionally, the global kout algorithm supports filtering functions (parameters such as “compute.filter”:“risk_level==1”), and the filtering condition is judged when calculating the k-degree. The final result set includes those that meet the filtering condition. The algorithm’s final output is the number of neighbors that meet the condition.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "kout_all",
 "compute.parallel":"10",
 "output.type":"local",
 "output.parallel":"10",
 "output.file_path":"result/kout",
 "compute.max_step":"2",
 "compute.filter":"risk_level==1"
 }
}

3.15 Clustering Coefficient

The clustering coefficient represents the coefficient of the clustering degree of nodes in a graph. In real networks, especially in specific networks, nodes tend to establish a tightly organized relationship due to relatively high-density connection points. The clustering coefficient algorithm (Cluster Coefficient) is used to calculate the clustering degree of nodes in the graph. This algorithm is for local clustering coefficients. The local clustering coefficient can measure the clustering degree around each node in the graph.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "clustering_coefficient",
 "compute.parallel":"100",
 "compute.max_step":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/cc"
 }
}

3.16 SCC (Strongly Connected Components)

In the mathematical theory of directed graphs, if every vertex of a graph can be reached from any other point in the graph, the graph is said to be strongly connected. The parts of any directed graph that can achieve strong connectivity are called strongly connected components. It indicates the connectivity between points and distinguishes different connected communities.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "scc",
 "compute.parallel":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/scc",
 "compute.max_step":"200"
 }
}

🚧, further updates and improvements will be made at any time. Suggestions and feedback are welcome.

3.4.2 - HugeGraph-Computer Quick Start

1. Component Overview

HugeGraph-Computer is a Java distributed graph computing framework based on BSP (Bulk Synchronous Parallel), with algorithms running in iterative supersteps. Kubernetes Operator or YARN can schedule jobs; master and worker processes can also run on one machine for small trial jobs.

Computer and Vermeer, in the same repository, are separate implementations: Computer uses a Java/BSP runtime for distributed computing, while Vermeer is a Go in-memory graph computing platform with a master-worker architecture. They share HugeGraph data sources, but their deployment and job configurations are not interchangeable.

Computer reads graph data from HugeGraph or HDFS and writes results to either system. The runtime can spill some data to disk; whether a job completes still depends on input size, resources, and configuration. Disk spilling does not remove resource limits.

2. Prerequisites and Connections

Building and running require JDK 11 or later; source builds also require Maven 3.5 or later. The PageRank example needs a running HugeGraph-Server with graph data, plus etcd reachable by the master and workers.

ServiceExample address or portPurpose
HugeGraph-Serverhttp://127.0.0.1:8080Read graph data and write algorithm results; configured by hugegraph.url.
etcdhttp://127.0.0.1:2379BSP job coordination; configured by bsp.etcd_endpoints.
Computer master RPCTCP 8190Workers connect to master; port in the distribution configuration.
Computer worker data transferOS-assigned locally; K8s Operator defaults to 8099Transfer vertices and messages between workers. Advertised addresses and ports must be reachable across hosts.
MinIO (K8s manifests)HTTP 9000Input partition snapshots. Computer 1.7.0 manifests incorrectly map Service port 9000 to MinIO Console port 9090; change targetPort to 9000 before enabling snapshots. Defaults are snapshot.write=false and snapshot.load=false, so MinIO access is unnecessary when snapshots are disabled.
cert-manager (K8s)In-cluster serviceOperator manifests use cert-manager.io/v1 Certificate, Issuer, and CA injection. Install a compatible version before deploying the Operator.
HDFSCluster-specificRequired only when input or output uses HDFS.

For Kubernetes jobs, hugegraph.url must be reachable from all computing Pods; do not use a localhost address available only on your computer. If HugeGraph authentication is enabled, configure the username and password and supply matching credentials for REST queries.

Warning

In production, enable Server authentication and authorization, retain Server audit logs (normally audit-*.log), give Computer a dedicated account with only the read/write permissions required by its jobs, and configure a source IP allowlist for Server. hugegraph.username and hugegraph.password are credentials for connecting to Server, not a replacement for Server authentication.

See the Computer configuration reference for more options.

3. Obtain Source and Build a Distribution

The Apache HugeGraph download directory provides versioned Computer source archives, without separate precompiled Computer binaries. This example uses VERSION=1.7.0, whose source archive and build output names include -incubating-. For other versions, use the actual filenames in the download directory.

VERSION=1.7.0 # Replace with the release to download
ARCHIVE="apache-hugegraph-computer-incubating-${VERSION}-src.tar.gz"
DOWNLOAD_BASE="https://downloads.apache.org/hugegraph/${VERSION}"
curl -fLO "${DOWNLOAD_BASE}/${ARCHIVE}"
curl -fLO "${DOWNLOAD_BASE}/${ARCHIVE}.sha512"
curl -fLO "${DOWNLOAD_BASE}/${ARCHIVE}.asc"
curl -fLO https://downloads.apache.org/hugegraph/KEYS
shasum -a 512 -c "${ARCHIVE}.sha512"
gpg --import KEYS
gpg --verify "${ARCHIVE}.asc" "${ARCHIVE}"
tar -xzf "${ARCHIVE}"
cd "apache-hugegraph-computer-incubating-${VERSION}-src/computer"
mvn clean package -DskipTests
tar -xzf "target/apache-hugegraph-computer-incubating-${VERSION}.tar.gz"
cd "apache-hugegraph-computer-incubating-${VERSION}"

To build current master, clone and package the source:

git clone https://github.com/apache/hugegraph-computer.git
cd hugegraph-computer/computer
mvn clean package -DskipTests

The computer/computer-dist module assembles the distribution, including bin/start-computer.sh, runtime dependencies in lib/, built-in algorithms in algorithm/builtin-algorithm.jar, and default conf/computer.properties and conf/log4j2.xml. The source configuration is computer/computer-dist/src/assembly/static/conf/computer.properties. Release 1.7.0 build output includes -incubating-; current master omits it and takes its version from the POM. Use archive names matching the source you built.

4. Run PageRank Locally

Edit conf/computer.properties in the distribution directory with your HugeGraph URL, graph name, and credentials; point bsp.etcd_endpoints to reachable etcd. The default configuration selects built-in PageRankParams. Master and workers must use the same configuration and job.id; concurrent jobs need distinct job IDs.

Start processes from the distribution directory in two terminals. The script reads conf/computer.properties by default; use -c to select another file.

# Terminal 1: start master
bin/start-computer.sh -d local -r master
# Terminal 2: start worker
bin/start-computer.sh -d local -r worker

Master waits for the configured workers to register before running the job. Check both terminal outputs; the default logging configuration also writes master and worker logs under the distribution directory’s logs/. Process startup alone does not confirm completion: verify that input, superstep computation, and output all finish normally in the master log.

PageRank writes results to HugeGraph under page_rank. First ensure every target vertex label allows this property. Computer creates a DOUBLE, OLAP_COMMON property key but does not add it to vertex labels. If absent, create it:

curl --fail --request POST \
  --header 'Content-Type: application/json' \
  --data '{"name":"page_rank","data_type":"DOUBLE","cardinality":"SINGLE","write_type":"OLAP_COMMON"}' \
  'http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/propertykeys'

If the property key already exists, verify its type is DOUBLE and write_type is OLAP_COMMON. List vertex labels, then add nullable page_rank to every label being computed; replace person with the actual label:

curl --fail --compressed \
  'http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/vertexlabels'

curl --fail --request PUT \
  --header 'Content-Type: application/json' \
  --data '{"name":"person","properties":["page_rank"],"nullable_keys":["page_rank"]}' \
  'http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/vertexlabels/person?action=append'

If the current read mode hides OLAP writes, an administrator can set it to ALL:

curl --fail --request PUT \
  --header 'Content-Type: application/json' \
  --data '"ALL"' \
  'http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/graph_read_mode'

Query vertices to verify the result property:

curl --fail --compressed \
  'http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices?limit=3'

When authentication is required, add --user "$HG_USER:$HG_PASSWORD" to curl. See the graph read-mode REST API for the endpoint and permissions.

5. Run PageRank on Kubernetes

Ensure computing Pods can reach HugeGraph-Server, then install a cluster-compatible cert-manager using its official installation guide. The Computer Operator manifests deploy the Operator, etcd, and MinIO. Use the same Computer release for the CRD and Operator manifests; this example uses 1.7.0:

VERSION=1.7.0 # Use the same release for CRD and Operator manifests
kubectl apply -f "https://raw.githubusercontent.com/apache/hugegraph-computer/${VERSION}/computer/computer-k8s-operator/manifest/hugegraph-computer-crd.v1.yaml"
kubectl apply -f "https://raw.githubusercontent.com/apache/hugegraph-computer/${VERSION}/computer/computer-k8s-operator/manifest/hugegraph-computer-operator.yaml"
kubectl rollout status deployment/hugegraph-computer-operator-controller-manager \
  -n hugegraph-computer-operator-system --timeout=120s
Optional: enable MinIO snapshots (1.7.0)

The 1.7.0 MinIO Service maps S3 API port 9000 to Console port 9090. Fix this mapping before enabling snapshots and set snapshot.minio_endpoint to http://hugegraph-computer-operator-minio.hugegraph-computer-operator-system.svc:9000. Skip this when snapshots remain disabled.

kubectl patch service hugegraph-computer-operator-minio \
  -n hugegraph-computer-operator-system \
  --type=json \
  -p='[{"op":"replace","path":"/spec/ports/0/targetPort","value":9000}]'

Replace the HugeGraph URL with a service address reachable by computing Pods, then submit a HugeGraphComputerJob. This example uses the official Docker Hub image hugegraph/hugegraph-computer:latest and its built-in PageRank JAR. Custom algorithm JARs can be bundled in the image and selected with jarFile, or downloaded from HTTP(S) with remoteJarUri. The partition count must be at least the worker count.

Save the following YAML as pagerank-job.yaml, then apply it:

apiVersion: operator.hugegraph.apache.org/v1
kind: HugeGraphComputerJob
metadata:
  namespace: hugegraph-computer-operator-system
  name: pagerank-sample
spec:
  jobId: pagerank-sample
  algorithmName: page_rank
  image: hugegraph/hugegraph-computer:latest # Official Docker Hub image
  jarFile: /hugegraph/hugegraph-computer/algorithm/builtin-algorithm.jar
  pullPolicy: IfNotPresent
  workerInstances: 1
  masterCpu: 500m
  workerCpu: 500m
  masterMemory: 1Gi
  workerMemory: 2Gi
  computerConf:
    job.partitions_count: "1"
    algorithm.params_class: org.apache.hugegraph.computer.algorithm.centrality.pagerank.PageRankParams
    hugegraph.url: http://hugegraph-server:8080
    hugegraph.name: hugegraph
kubectl apply -f pagerank-job.yaml
kubectl get hcjob pagerank-sample -n hugegraph-computer-operator-system --watch

SUCCEEDED means the job completed. By default, the Operator deletes the CR and computing resources after completion. Expand the following instructions when you need to retain state or investigate a failed job.

Optional: retain job state, inspect logs, or clean up resources

The Operator defaults to AUTO_DESTROY_POD=true, so short jobs may lose their CR and Pods before inspection. Disable this before submitting the job, wait for the replacement Controller Pod to become ready, then submit. Change this setting in the Java Operator’s controller container:

kubectl set env deployment/hugegraph-computer-operator-controller-manager \
  -n hugegraph-computer-operator-system \
  --containers=controller AUTO_DESTROY_POD=false
kubectl rollout status deployment/hugegraph-computer-operator-controller-manager \
  -n hugegraph-computer-operator-system --timeout=120s

Inspect runtime information or failure logs:

kubectl logs --follow <master-pod-name> -n hugegraph-computer-operator-system
kubectl logs --follow <worker-pod-name> -n hugegraph-computer-operator-system

After checking results, delete the CR to clean up associated resources and restore the default policy:

kubectl delete hcjob pagerank-sample -n hugegraph-computer-operator-system
kubectl set env deployment/hugegraph-computer-operator-controller-manager \
  -n hugegraph-computer-operator-system \
  --containers=controller AUTO_DESTROY_POD=true

After PageRank writes to HugeGraph, set the read mode and query as in the previous section. For HDFS output, results are under output.hdfs_path_prefix/<job.id>/; filenames and partition layout depend on job configuration.

See the CRD configuration reference for all fields.

6. Built-in Algorithms and Development

Current built-in algorithms include:

  • Centrality: PageRank, Betweenness Centrality, Closeness Centrality, Degree Centrality.
  • Communities and structure: Clustering Coefficient, K-core, LPA, Triangle Count, WCC.
  • Paths and sampling: ring detection, filtered ring detection, single-source shortest path, Random Walk.

Implementations are in computer/computer-algorithm. Custom algorithms must follow the Computer API and be packaged as loadable JARs; see the Computer README for modules and development entry points.

3.4.3 - HugeGraph-Computer Configuration Reference

Computer Config Options

The defaults in the tables come from ComputerOptions.java in computer-api; explicit conf/computer.properties overrides are shown as “code default (distribution: actual value)”. Common options such as rpc.* come from HugeGraph Commons; distribution values are listed separately below.

Configuration Sources

  • Source template: computer/computer-dist/src/assembly/static/conf/computer.properties, with log4j2.xml in the same directory. Maven package copies these into distribution conf/, runtime dependencies into lib/, and built-in algorithm JARs into algorithm/. Templates are maintained in source; there is no separate configuration generator.
  • Standalone and YARN: startup scripts read distribution conf/computer.properties by default. Override it with bin/start-computer.sh -c <configuration-path>; master and workers need the same job parameters.
  • Kubernetes Operator: users supply spec.computerConf in the CRD; the Operator writes a ConfigMap mounted as computer.properties. It supplies job ID, worker count, Pod addresses, and etcd when unspecified. Unspecified or zero transfer/RPC ports become 8099/8190; transport.server_host and rpc.server_host become Pod IPs. Job startup scripts substitute only ${POD_IP}, ${HOSTNAME}, ${POD_NAME}, and ${POD_NAMESPACE}.
  • Kubernetes job images must contain the Computer runtime. Bundle algorithm JARs and select them with jarFile, or use an HTTP(S) remoteJarUri for startup download. See the Computer Quick Start and CRD table below.

Apache downloads provide versioned Computer source archives without separate precompiled binaries. Release 1.7.0 source and distribution names include -incubating-; run mvn clean package -DskipTests in the tagged computer/ project to build them. Current master omits that marker and takes its version from the POM. Source archives cannot be started as built distributions.

Warning

Empty HugeGraph credentials and example MinIO keys below are for local demonstrations. In production, enable Server authentication and authorization, retain Server audit-*.log, use a Computer account with minimum required Server permissions, and configure a Server source IP allowlist. Do not use example MinIO keys in production.


1. Basic Configuration

Core job settings for HugeGraph-Computer.

config optiondefault valuedescription
hugegraph.urlhttp://127.0.0.1:8080The HugeGraph server URL to load data and write results back.
hugegraph.namehugegraphThe graph name to load data and write results back.
hugegraph.username"" (empty)The username for HugeGraph authentication (leave empty if authentication is disabled).
hugegraph.password"" (empty)The password for HugeGraph authentication (leave empty if authentication is disabled).
job.idlocal_0001 (packaged: local_001)The job identifier on YARN cluster or K8s cluster.
job.namespace"" (empty)Optional etcd job-key prefix for namespace isolation; distinct from Kubernetes metadata.namespace and not populated automatically by the Operator.
job.workers_count1The number of workers for one graph algorithm job. In K8s, this option is set by the Operator.
job.partitions_count1The number of partitions for computing one graph algorithm job.
job.partitions_thread_nums4The number of threads for partition parallel compute.

2. Algorithm Configuration

Algorithm-specific configuration for computation logic.

config optiondefault valuedescription
algorithm.params_classComputerOptions.Null placeholder classRequired. The class used to pass algorithm parameters before the algorithm runs.
algorithm.result_classComputerOptions.Null placeholder classThe vertex value class used to store computation results.
algorithm.message_classComputerOptions.Null placeholder classThe message class passed while computing a vertex.

3. Input Configuration

Configuration for loading input data from HugeGraph or other sources.

3.1 Input Source

config optiondefault valuedescription
input.source_typehugegraph-serverThe source type to load input data, allowed values: [‘hugegraph-server’, ‘hugegraph-loader’]. The ‘hugegraph-loader’ means use hugegraph-loader to load data from HDFS or file. If using ‘hugegraph-loader’, please configure ‘input.loader_struct_path’ and ‘input.loader_schema_path’.
input.loader_struct_path"" (empty)The structure path for Loader input. It takes effect only when input.source_type=hugegraph-loader.
input.loader_schema_path"" (empty)The schema path for Loader input. It takes effect only when input.source_type=hugegraph-loader.

3.2 Input Splits

config optiondefault valuedescription
input.split_size1048576 (1 MB)The input split size in bytes.
input.split_max_splits10000000The maximum number of input splits.
input.split_page_size500The page size for streamed load input split data.
input.split_fetch_timeout300The timeout in seconds to fetch input splits.

3.3 Input Processing

config optiondefault valuedescription
input.filter_classorg.apache.hugegraph.computer.core.input.filter.DefaultInputFilterThe class to create input-filter object. Input-filter is used to filter vertex edges according to user needs.
input.edge_directionOUTThe direction of edges to load, allowed values: [OUT, IN, BOTH]. When the value is BOTH, edges in both OUT and IN directions will be loaded.
input.edge_freqMULTIPLEThe frequency of edges that can exist between a pair of vertices, allowed values: [SINGLE, SINGLE_PER_LABEL, MULTIPLE]. SINGLE means only one edge can exist between a pair of vertices (identified by sourceId + targetId); SINGLE_PER_LABEL means each edge label can have one edge between a pair of vertices (identified by sourceId + edgeLabel + targetId); MULTIPLE means many edges can exist between a pair of vertices (identified by sourceId + edgeLabel + sortValues + targetId).
input.max_edges_in_one_vertex200The maximum number of adjacent edges allowed to be attached to a vertex. The adjacent edges will be stored and transferred together as a batch unit.

3.4 Input Performance

config optiondefault valuedescription
input.send_thread_nums4The number of threads for parallel sending of vertices or edges.

4. Snapshot & Storage Configuration

HugeGraph-Computer supports snapshot functionality to save vertex/edge partitions to local storage or MinIO object storage, enabling checkpoint recovery or accelerating repeated computations.

4.1 Basic Snapshot Configuration

config optiondefault valuedescription
snapshot.writefalseWhether to write snapshots of input vertex/edge partitions.
snapshot.loadfalseWhether to load from snapshots of vertex/edge partitions.
snapshot.name"" (empty)User-defined snapshot name to distinguish different snapshots.

4.2 MinIO Integration (Optional)

MinIO can be used as a distributed object storage backend for snapshots in K8s deployments.

config optiondefault valuedescription
snapshot.minio_endpoint"" (empty)MinIO service endpoint (e.g., http://minio:9000). Required when using MinIO.
snapshot.minio_access_keyminioadminMinIO access key for authentication.
snapshot.minio_secret_keyminioadminMinIO secret key for authentication.
snapshot.minio_bucket_name"" (empty)MinIO bucket name for storing snapshot data.

Usage Scenarios:

  • Checkpoint Recovery: Resume from snapshots after job failures, avoiding data reloading
  • Repeated Computations: Load data from snapshots when running the same algorithm multiple times
  • A/B Testing: Save multiple snapshot versions of the same dataset to test different algorithm parameters

Example: Local Snapshot (in computer.properties):

snapshot.write=true
snapshot.name=pagerank-snapshot-20260201

Example: MinIO Snapshot (in K8s CRD computerConf):

computerConf:
  snapshot.write: "true"
  snapshot.name: "pagerank-snapshot-v1"
  snapshot.minio_endpoint: "http://minio:9000"
  snapshot.minio_access_key: "my-access-key"
  snapshot.minio_secret_key: "my-secret-key"
  snapshot.minio_bucket_name: "hugegraph-snapshots"

5. Worker & Master Configuration

Configuration for worker and master computation logic.

5.1 Master Configuration

config optiondefault valuedescription
master.computation_classorg.apache.hugegraph.computer.core.master.DefaultMasterComputationMaster-computation is computation that can determine whether to continue to the next superstep. It runs at the end of each superstep on the master.

5.2 Worker Computation

config optiondefault valuedescription
worker.computation_classorg.apache.hugegraph.computer.core.config.NullThe class to create worker-computation object. Worker-computation is used to compute each vertex in each superstep.
worker.combiner_classorg.apache.hugegraph.computer.core.config.NullCombiner can combine messages into one value for a vertex. For example, PageRank algorithm can combine messages of a vertex to a sum value.
worker.partitionerorg.apache.hugegraph.computer.core.graph.partition.HashPartitionerThe partitioner that decides which partition a vertex should be in, and which worker a partition should be in.

5.3 Worker Combiners

config optiondefault valuedescription
worker.vertex_properties_combiner_classorg.apache.hugegraph.computer.core.combiner.OverwritePropertiesCombinerThe combiner can combine several properties of the same vertex into one properties at input step.
worker.edge_properties_combiner_classorg.apache.hugegraph.computer.core.combiner.OverwritePropertiesCombinerThe combiner can combine several properties of the same edge into one properties at input step.

5.4 Worker Buffers

config optiondefault valuedescription
worker.received_buffers_bytes_limit104857600 (100 MB)The limit bytes of buffers of received data. The total size of all buffers can’t exceed this limit. If received buffers reach this limit, they will be merged into a file (spill to disk).
worker.write_buffer_capacity52428800 (50 MB)The initial size of write buffer that used to store vertex or message.
worker.write_buffer_threshold52428800 (50 MB)The threshold of write buffer. Exceeding it will trigger sorting. The write buffer is used to store vertex or message.

5.5 Worker Data & Timeouts

config optiondefault valuedescription
worker.data_dirs[jobs]The directories separated by ‘,’ that received vertices and messages can persist into.
worker.wait_sort_timeout600000 (10 minutes)The max timeout (in ms) for message-handler to wait for sort-thread to sort one batch of buffers.
worker.wait_finish_messages_timeout86400000 (24 hours)The max timeout (in ms) for message-handler to wait for finish-message of all workers.

6. I/O & Output Configuration

Configuration for output computation results.

6.1 Output Class & Result

config optiondefault valuedescription
output.output_classorg.apache.hugegraph.computer.core.output.LogOutputThe class to output the computation result of each vertex. Called after iteration computation.
output.result_namevalueThe value is assigned dynamically by #name() of instance created by WORKER_COMPUTATION_CLASS.
output.result_write_typeOLAP_COMMONThe result write-type to output to HugeGraph, allowed values: [OLAP_COMMON, OLAP_SECONDARY, OLAP_RANGE].

6.2 Output Behavior

config optiondefault valuedescription
output.with_adjacent_edgesfalseWhether to output the adjacent edges of the vertex.
output.with_vertex_propertiesfalseWhether to output the properties of the vertex.
output.with_edge_propertiesfalseWhether to output the properties of the edge.

6.3 Batch Output

config optiondefault valuedescription
output.batch_size500The batch size of output.
output.batch_threads1The number of threads used for batch output.
output.single_threads1The number of threads used for single output.

6.4 HDFS Output

config optiondefault valuedescription
output.hdfs_urlhdfs://127.0.0.1:9000The HDFS URL for output.
output.hdfs_userhadoopThe HDFS user for output.
output.hdfs_path_prefix/hugegraph-computer/resultsThe directory of HDFS output results.
output.hdfs_delimiter, (comma)The delimiter of HDFS output.
output.hdfs_merge_partitionstrueWhether to merge output files of multiple partitions.
output.hdfs_replication3The replication number of HDFS.
output.hdfs_core_site_path"" (empty)The HDFS core site path.
output.hdfs_site_path"" (empty)The HDFS site path.
output.hdfs_kerberos_enablefalseWhether Kerberos authentication is enabled for HDFS.
output.hdfs_kerberos_principal"" (empty)The HDFS principal for Kerberos authentication.
output.hdfs_kerberos_keytab"" (empty)The HDFS keytab file for Kerberos authentication.
output.hdfs_krb5_conf/etc/krb5.confKerberos configuration file path.

6.5 Retry & Timeout

config optiondefault valuedescription
output.retry_times3The retry times when output fails.
output.retry_interval10The retry interval (in seconds) when output fails.
output.thread_pool_shutdown_timeout60The timeout (in seconds) of output thread pool shutdown.

7. Network & Transport Configuration

Configuration for network communication between workers and master.

7.1 Server Configuration

config optiondefault valuedescription
transport.server_host127.0.0.1Worker data listener and advertised address. It must be reachable by other workers across hosts; the Operator sets it to the Pod IP.
transport.server_port0 (distribution: 0; K8s Operator: 8099)Worker data port. Locally, 0 lets the OS assign a port; the Operator uses fixed port 8099 by default for Pod communication.
transport.server_threads4The number of transport threads for server.

7.2 Client Configuration

config optiondefault valuedescription
transport.client_threads4The number of transport threads for client.
transport.client_connect_timeout3000The timeout (in ms) of client connect to server.

7.3 Protocol Configuration

config optiondefault valuedescription
transport.provider_classorg.apache.hugegraph.computer.core.network.netty.NettyTransportProviderThe transport provider, currently only supports Netty.
transport.io_modeAUTOThe network IO mode, allowed values: [NIO, EPOLL, AUTO]. AUTO means selecting the appropriate mode automatically.
transport.tcp_keep_alivetrueWhether to enable TCP keep-alive.
transport.transport_epoll_ltfalseWhether to enable EPOLL level-trigger (only effective when io_mode=EPOLL).

7.4 Buffer Configuration

config optiondefault valuedescription
transport.send_buffer_size0The size of socket send-buffer in bytes. 0 means using system default value.
transport.receive_buffer_size0The size of socket receive-buffer in bytes. 0 means using system default value.
transport.write_buffer_high_mark67108864 (64 MB)The high water mark for write buffer in bytes. It will trigger sending unavailable if the number of queued bytes > write_buffer_high_mark.
transport.write_buffer_low_mark33554432 (32 MB)The low water mark for write buffer in bytes. It will trigger sending available if the number of queued bytes < write_buffer_low_mark.

7.5 Flow Control

config optiondefault valuedescription
transport.max_pending_requests8The max number of client unreceived ACKs. It will trigger sending unavailable if the number of unreceived ACKs >= max_pending_requests.
transport.min_pending_requests6The minimum number of client unreceived ACKs. It will trigger sending available if the number of unreceived ACKs < min_pending_requests.
transport.min_ack_interval200The minimum interval (in ms) of server reply ACK.

7.6 Timeouts

config optiondefault valuedescription
transport.close_timeout10000The timeout (in ms) of close server or close client.
transport.sync_request_timeout10000The timeout (in ms) to wait for response after sending sync-request.
transport.finish_session_timeout0The timeout (in ms) to finish session. 0 means using (transport.sync_request_timeout × transport.max_pending_requests).
transport.write_socket_timeout3000The timeout (in ms) to write data to socket buffer.
transport.server_idle_timeout360000 (6 minutes)The max timeout (in ms) of server idle.

7.7 Heartbeat

config optiondefault valuedescription
transport.heartbeat_interval20000 (20 seconds)The minimum interval (in ms) between heartbeats on client side.
transport.max_timeout_heartbeat_count120The maximum times of timeout heartbeat on client side. If the number of timeouts waiting for heartbeat response continuously > max_timeout_heartbeat_count, the channel will be closed from client side.

7.8 Advanced Network Settings

config optiondefault valuedescription
transport.max_syn_backlog511The capacity of SYN queue on server side. 0 means using system default value.
transport.recv_file_modetrueWhether to enable receive buffer-file mode. It will receive buffer and write to file from socket using zero-copy if enabled. Note: Requires OS support for zero-copy (e.g., Linux sendfile/splice).
transport.network_retries3The number of retry attempts for network communication if network is unstable.

7.9 Master RPC Configuration

These options come from HugeGraph Commons RPC configuration; the distribution template explicitly sets host and port. Kubernetes Operator advertises the Pod IP and defaults an unspecified or zero port to 8190.

OptionDistribution valueDescription
rpc.server_host127.0.0.1 (K8s: Pod IP)Master RPC address, reachable by workers.
rpc.server_port8190 (K8s default: 8190)Master RPC listener port; allow worker connections in security groups and network policies.

8. Storage & Persistence Configuration

Configuration for HGKV (HugeGraph Key-Value) storage engine and value files.

8.1 HGKV Configuration

config optiondefault valuedescription
hgkv.max_file_size2147483648 (2 GB)The max number of bytes in each HGKV file.
hgkv.max_data_block_size65536 (64 KB)The max byte size of HGKV file data block.
hgkv.max_merge_files10The max number of files to merge at one time.
hgkv.temp_file_dir/tmp/hgkvThis folder is used to store temporary files during the file merging process.

8.2 Value File Configuration

config optiondefault valuedescription
valuefile.max_segment_size1073741824 (1 GB)The max number of bytes in each segment of value-file.

9. BSP & Coordination Configuration

Configuration for Bulk Synchronous Parallel (BSP) protocol and etcd coordination.

config optiondefault valuedescription
bsp.etcd_endpointshttp://localhost:2379 (distribution: http://127.0.0.1:2379)Comma-separated etcd client endpoints. The Operator uses INTERNAL_ETCD_URL only when computerConf does not specify this key.
bsp.max_super_step10 (packaged: 2)The max super step of the algorithm.
bsp.register_timeout300000 (packaged: 100000)The max timeout (in ms) to wait for master and workers to register.
bsp.wait_workers_timeout86400000 (24 hours)The max timeout (in ms) to wait for workers BSP event.
bsp.wait_master_timeout86400000 (24 hours)The max timeout (in ms) to wait for master BSP event.
bsp.log_interval30000 (30 seconds)The log interval (in ms) to print the log while waiting for BSP event.

10. Performance Tuning Configuration

Configuration for performance optimization.

config optiondefault valuedescription
allocator.max_vertices_per_thread10000Maximum number of vertices per thread processed in each memory allocator.
sort.thread_nums4The number of threads performing internal sorting.

11. System Administration Configuration

Kubernetes Operator fills or overrides these options. Avoid overriding them unless customizing the Operator or network. job.namespace is optional user configuration; see the basic configuration table.

OptionManaged byDescription
bsp.etcd_endpointsK8s OperatorUses INTERNAL_ETCD_URL only when absent from computerConf.
transport.server_hostK8s OperatorOverrides with Pod IP; workers must reach each other.
transport.server_portK8s OperatorDefaults unspecified or zero values to 8099, not a random port.
job.idK8s OperatorSet from CRD job ID.
job.workers_countK8s OperatorSet from CRD workerInstances.
rpc.server_hostK8s OperatorOverrides with master Pod IP.
rpc.server_portK8s OperatorDefaults unspecified or zero values to 8190.
rpc.remote_urlComputer startupRemoved when configuration is read; do not set as job configuration.

Why these values must match:

  • BSP/RPC: Must match deployed etcd/RPC services for coordination.
  • Job settings: Must match the CRD to provide the correct worker count.
  • Transport: Workers need mutually reachable Pod IPs and ports; the K8s default is 8099.

K8s Operator Config Options

NOTE: Option needs to be converted through environment variable settings, e.g. k8s.internal_etcd_url => INTERNAL_ETCD_URL

config optiondefault valuedescription
k8s.auto_destroy_podtrueDelete the job CR after completion or failure; CR deletion triggers cleanup of associated computing resources.
k8s.close_reconciler_timeout120The max timeout (in ms) to close reconciler.
k8s.internal_etcd_urlCode: http://127.0.0.1:2379; manifest: http://hugegraph-computer-operator-etcd.hugegraph-computer-operator-system:2379etcd URL used by Operator jobs; the supplied manifest uses the etcd Service address.
k8s.internal_minio_urlCode: http://127.0.0.1:9000; manifest: http://hugegraph-computer-operator-minio.hugegraph-computer-operator-system:9000MinIO address supplied to jobs by the Operator; needed only for MinIO snapshots.
k8s.max_reconcile_retry3The max retry times of reconcile.
k8s.probe_backlog50The maximum backlog for serving health probes.
k8s.probe_port9892The port that the controller binds to for serving health probes.
k8s.ready_check_internal1000The time interval (ms) of check ready.
k8s.ready_timeout30000The max timeout (in ms) of check ready.
k8s.reconciler_countCode: Runtime.getRuntime().availableProcessors(); manifest: 6Maximum reconciler thread count.
k8s.resync_period600000The minimum frequency at which watched resources are reconciled.
k8s.timezoneAsia/ShanghaiThe timezone of computer job and operator.
k8s.watch_namespacehugegraph-computer-operator-systemNamespace watched for custom resources; also used by the supplied manifest. Use * to watch all namespaces.

HugeGraph-Computer CRD

1.7.0 CRD: https://github.com/apache/hugegraph-computer/blob/1.7.0/computer/computer-k8s-operator/manifest/hugegraph-computer-crd.v1.yaml

specdefault valuedescriptionrequired
algorithmNameThe name of algorithm.true
jobIdThe job id.true
imageJob container image, which must include the Computer runtime. Bundle algorithm JARs or download them with remoteJarUri.true
computerConfThe map of computer config options.true
workerInstancesThe number of worker instances, it will override the ‘job.workers_count’ option.true
pullPolicyKubernetes selects Always for latest, otherwise IfNotPresent, when unsetExplicit values: Always, Never, or IfNotPresent. The CRD has no default; see image pull policy.false
pullSecretsThe pull-secrets of Image, detail please refer to: https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-podfalse
masterCpuThe cpu limit of master, the unit can be ’m’ or without unit detail please refer to: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#meaning-of-cpufalse
workerCpuThe cpu limit of worker, the unit can be ’m’ or without unit detail please refer to: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#meaning-of-cpufalse
masterMemoryThe memory limit of master, the unit can be one of Ei、Pi、Ti、Gi、Mi、Ki detail please refer to: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#meaning-of-memoryfalse
workerMemoryThe memory limit of worker, the unit can be one of Ei、Pi、Ti、Gi、Mi、Ki detail please refer to: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#meaning-of-memoryfalse
log4jXmlThe content of log4j.xml for computer job.false
jarFilePath to the algorithm JAR inside the image.false
remoteJarUriHTTP(S) algorithm JAR URL, downloaded and loaded by the startup script.false
jvmOptionsThe java startup parameters of computer job.false
envVarsplease refer to: https://kubernetes.io/docs/tasks/inject-data-application/define-interdependent-environment-variables/false
envFromplease refer to: https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/false
masterCommandbin/start-computer.shThe run command of master, equivalent to ‘Entrypoint’ field of Docker.false
masterArgs["-r master", “-d k8s”]The run args of master, equivalent to ‘Cmd’ field of Docker.false
workerCommandbin/start-computer.shThe run command of worker, equivalent to ‘Entrypoint’ field of Docker.false
workerArgs["-r worker", “-d k8s”]The run args of worker, equivalent to ‘Cmd’ field of Docker.false
volumesPlease refer to: https://kubernetes.io/docs/concepts/storage/volumes/false
volumeMountsPlease refer to: https://kubernetes.io/docs/concepts/storage/volumes/false
secretPathsThe map of k8s-secret name and mount path.false
configMapPathsThe map of k8s-configmap name and mount path.false
podTemplateSpecPlease refer to: https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-template-v1/#PodTemplateSpecfalse
securityContextPlease refer to: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/false

KubeDriver Config Options

config optiondefault valuedescription
k8s.build_image_bash_pathThe path of command used to build image.
k8s.enable_internal_algorithmtrueWhether enable internal algorithm.
k8s.framework_image_urlhugegraph/hugegraph-computer:latestThe image url of computer framework.
k8s.image_repository_passwordThe password for login image repository.
k8s.image_repository_registryThe address for login image repository.
k8s.image_repository_urlhugegraph/hugegraph-computerThe url of image repository.
k8s.image_repository_usernameThe username for login image repository.
k8s.internal_algorithm[pageRank]The name list of all internal algorithm. Note: Algorithm names use camelCase here (e.g., pageRank), but algorithm implementations return underscore_case (e.g., page_rank).
k8s.internal_algorithm_image_urlhugegraph/hugegraph-computer:latestThe image url of internal algorithm.
k8s.jar_file_dir/cache/jars/The directory where the algorithm jar will be uploaded.
k8s.kube_config~/.kube/configThe path of k8s config file.
k8s.log4j_xml_pathThe log4j.xml path for computer job.
k8s.namespacehugegraph-computer-operator-systemNamespace of the HugeGraph-Computer system.
k8s.pull_secret_names[]The names of pull-secret for pulling image.

3.5 - HugeGraph Client

The Java and Go clients are maintained in the HugeGraph Toolchain repository, while the Python client is maintained in the HugeGraph-AI repository. Their installation methods and APIs differ; see the corresponding pages for details.

3.5.1 - HugeGraph-Java-Client

1 Overview

HugeGraph Java Client translates Java APIs into REST requests to HugeGraph Server. It supports managing schemas and graph data, executing Gremlin queries, and calling Traverser APIs. See the Client API for detailed interfaces; this page shows how to use the client in a Java project.

For other languages, use the Go Client or the Python Client maintained in the HugeGraph-AI repository.

2 What You Need

  • JDK 11 (used by the current CI; the source target remains Java 8)
  • Maven 3.6+

3 How To Use

The basic steps to use HugeGraph-Client are as follows:

  • Build a new Maven project by IDEA or Eclipse
  • Add HugeGraph-Client dependency in a pom file;
  • Create an object to invoke the interface of HugeGraph-Client

See the complete example in the following section for the detail.

4 Complete Example

4.1 Build New Maven Project

Using IDEA or Eclipse to create the project:

4.2 Add Hugegraph-Client Dependency In POM

<dependencies>
    <dependency>
        <groupId>org.apache.hugegraph</groupId>
        <artifactId>hugegraph-client</artifactId>
        <!-- The complete examples below work with the published 1.7.0 client -->
        <version>1.7.0</version>
    </dependency>    
</dependencies>

The complete examples below use Schema, Graph, Gremlin and resource-cleanup APIs available in the published Client 1.7.0 and compatible with Toolchain master. Use the dependency above directly in your example project.

For new master APIs in the API reference, such as capability detection, use Client 1.8.0 from Toolchain master and change the POM version to 1.8.0. If that version is not published to Maven yet, run the following command from the Toolchain source root to install the Client and its parent POM locally. The source build still needs its dependencies available from your configured Maven repositories.

mvn install -pl hugegraph-client -am -DskipTests -Dmaven.javadoc.skip=true -ntp

Client and Server versions need not match. Check the supported server API range and required capabilities before upgrading.

4.3 Example

4.3.1 SingleExample
import java.io.IOException;
import java.util.Iterator;
import java.util.List;

import org.apache.hugegraph.driver.GraphManager;
import org.apache.hugegraph.driver.GremlinManager;
import org.apache.hugegraph.driver.HugeClient;
import org.apache.hugegraph.driver.SchemaManager;
import org.apache.hugegraph.structure.constant.T;
import org.apache.hugegraph.structure.graph.Edge;
import org.apache.hugegraph.structure.graph.Path;
import org.apache.hugegraph.structure.graph.Vertex;
import org.apache.hugegraph.structure.gremlin.Result;
import org.apache.hugegraph.structure.gremlin.ResultSet;

public class SingleExample {

    public static void main(String[] args) throws IOException {
        // If connect failed will throw a exception.
        HugeClient hugeClient = HugeClient.builder("http://localhost:8080",
                                                   "DEFAULT",
                                                   "hugegraph")
                                          .configUser("username", "password")
                                          // This is an example. In a production environment, secure credentials should be used.
                                          .build();

        SchemaManager schema = hugeClient.schema();

        schema.propertyKey("name").asText().ifNotExist().create();
        schema.propertyKey("age").asInt().ifNotExist().create();
        schema.propertyKey("city").asText().ifNotExist().create();
        schema.propertyKey("weight").asDouble().ifNotExist().create();
        schema.propertyKey("lang").asText().ifNotExist().create();
        schema.propertyKey("date").asDate().ifNotExist().create();
        schema.propertyKey("price").asInt().ifNotExist().create();

        schema.vertexLabel("person")
              .properties("name", "age", "city")
              .primaryKeys("name")
              .ifNotExist()
              .create();

        schema.vertexLabel("software")
              .properties("name", "lang", "price")
              .primaryKeys("name")
              .ifNotExist()
              .create();

        schema.indexLabel("personByCity")
              .onV("person")
              .by("city")
              .secondary()
              .ifNotExist()
              .create();

        schema.indexLabel("personByAgeAndCity")
              .onV("person")
              .by("age", "city")
              .secondary()
              .ifNotExist()
              .create();

        schema.indexLabel("softwareByPrice")
              .onV("software")
              .by("price")
              .range()
              .ifNotExist()
              .create();

        schema.edgeLabel("knows")
              .sourceLabel("person")
              .targetLabel("person")
              .properties("date", "weight")
              .ifNotExist()
              .create();

        schema.edgeLabel("created")
              .sourceLabel("person").targetLabel("software")
              .properties("date", "weight")
              .ifNotExist()
              .create();

        schema.indexLabel("createdByDate")
              .onE("created")
              .by("date")
              .secondary()
              .ifNotExist()
              .create();

        schema.indexLabel("createdByWeight")
              .onE("created")
              .by("weight")
              .range()
              .ifNotExist()
              .create();

        schema.indexLabel("knowsByWeight")
              .onE("knows")
              .by("weight")
              .range()
              .ifNotExist()
              .create();

        GraphManager graph = hugeClient.graph();
        Vertex marko = graph.addVertex(T.LABEL, "person", "name", "marko",
                                       "age", 29, "city", "Beijing");
        Vertex vadas = graph.addVertex(T.LABEL, "person", "name", "vadas",
                                       "age", 27, "city", "Hongkong");
        Vertex lop = graph.addVertex(T.LABEL, "software", "name", "lop",
                                     "lang", "java", "price", 328);
        Vertex josh = graph.addVertex(T.LABEL, "person", "name", "josh",
                                      "age", 32, "city", "Beijing");
        Vertex ripple = graph.addVertex(T.LABEL, "software", "name", "ripple",
                                        "lang", "java", "price", 199);
        Vertex peter = graph.addVertex(T.LABEL, "person", "name", "peter",
                                       "age", 35, "city", "Shanghai");

        marko.addEdge("knows", vadas, "date", "2016-01-10", "weight", 0.5);
        marko.addEdge("knows", josh, "date", "2013-02-20", "weight", 1.0);
        marko.addEdge("created", lop, "date", "2017-12-10", "weight", 0.4);
        josh.addEdge("created", lop, "date", "2009-11-11", "weight", 0.4);
        josh.addEdge("created", ripple, "date", "2017-12-10", "weight", 1.0);
        peter.addEdge("created", lop, "date", "2017-03-24", "weight", 0.2);

        GremlinManager gremlin = hugeClient.gremlin();
        System.out.println("==== Path ====");
        ResultSet resultSet = gremlin.gremlin("g.V().outE().path()").execute();
        Iterator<Result> results = resultSet.iterator();
        results.forEachRemaining(result -> {
            System.out.println(result.getObject().getClass());
            Object object = result.getObject();
            if (object instanceof Vertex) {
                System.out.println(((Vertex) object).id());
            } else if (object instanceof Edge) {
                System.out.println(((Edge) object).id());
            } else if (object instanceof Path) {
                List<Object> elements = ((Path) object).objects();
                elements.forEach(element -> {
                    System.out.println(element.getClass());
                    System.out.println(element);
                });
            } else {
                System.out.println(object);
            }
        });

        hugeClient.close();
    }
}
4.3.2 BatchExample
import java.util.ArrayList;
import java.util.List;

import org.apache.hugegraph.driver.GraphManager;
import org.apache.hugegraph.driver.HugeClient;
import org.apache.hugegraph.driver.SchemaManager;
import org.apache.hugegraph.structure.graph.Edge;
import org.apache.hugegraph.structure.graph.Vertex;

public class BatchExample {

    public static void main(String[] args) {
        HugeClient hugeClient = HugeClient.builder("http://localhost:8080",
                                                   "DEFAULT",
                                                   "hugegraph")
                                          .configUser("username", "password")
                                          // This is an example. In a production environment, secure credentials should be used.
                                          .build();

        SchemaManager schema = hugeClient.schema();

        schema.propertyKey("name").asText().ifNotExist().create();
        schema.propertyKey("age").asInt().ifNotExist().create();
        schema.propertyKey("lang").asText().ifNotExist().create();
        schema.propertyKey("date").asDate().ifNotExist().create();
        schema.propertyKey("price").asInt().ifNotExist().create();

        schema.vertexLabel("person")
              .properties("name", "age")
              .primaryKeys("name")
              .ifNotExist()
              .create();

        schema.vertexLabel("person")
              .properties("price")
              .nullableKeys("price")
              .append();

        schema.vertexLabel("software")
              .properties("name", "lang", "price")
              .primaryKeys("name")
              .ifNotExist()
              .create();

        schema.indexLabel("softwareByPrice")
              .onV("software").by("price")
              .range()
              .ifNotExist()
              .create();

        schema.edgeLabel("knows")
              .link("person", "person")
              .properties("date")
              .ifNotExist()
              .create();

        schema.edgeLabel("created")
              .link("person", "software")
              .properties("date")
              .ifNotExist()
              .create();

        schema.indexLabel("createdByDate")
              .onE("created").by("date")
              .secondary()
              .ifNotExist()
              .create();

        // get schema object by name
        System.out.println(schema.getPropertyKey("name"));
        System.out.println(schema.getVertexLabel("person"));
        System.out.println(schema.getEdgeLabel("knows"));
        System.out.println(schema.getIndexLabel("createdByDate"));

        // list all schema objects
        System.out.println(schema.getPropertyKeys());
        System.out.println(schema.getVertexLabels());
        System.out.println(schema.getEdgeLabels());
        System.out.println(schema.getIndexLabels());

        GraphManager graph = hugeClient.graph();

        Vertex marko = new Vertex("person").property("name", "marko")
                                           .property("age", 29);
        Vertex vadas = new Vertex("person").property("name", "vadas")
                                           .property("age", 27);
        Vertex lop = new Vertex("software").property("name", "lop")
                                           .property("lang", "java")
                                           .property("price", 328);
        Vertex josh = new Vertex("person").property("name", "josh")
                                          .property("age", 32);
        Vertex ripple = new Vertex("software").property("name", "ripple")
                                              .property("lang", "java")
                                              .property("price", 199);
        Vertex peter = new Vertex("person").property("name", "peter")
                                           .property("age", 35);

        Edge markoKnowsVadas = new Edge("knows").source(marko).target(vadas)
                                                .property("date", "2016-01-10");
        Edge markoKnowsJosh = new Edge("knows").source(marko).target(josh)
                                               .property("date", "2013-02-20");
        Edge markoCreateLop = new Edge("created").source(marko).target(lop)
                                                 .property("date",
                                                           "2017-12-10");
        Edge joshCreateRipple = new Edge("created").source(josh).target(ripple)
                                                   .property("date",
                                                             "2017-12-10");
        Edge joshCreateLop = new Edge("created").source(josh).target(lop)
                                                .property("date", "2009-11-11");
        Edge peterCreateLop = new Edge("created").source(peter).target(lop)
                                                 .property("date",
                                                           "2017-03-24");

        List<Vertex> vertices = new ArrayList<>();
        vertices.add(marko);
        vertices.add(vadas);
        vertices.add(lop);
        vertices.add(josh);
        vertices.add(ripple);
        vertices.add(peter);

        List<Edge> edges = new ArrayList<>();
        edges.add(markoKnowsVadas);
        edges.add(markoKnowsJosh);
        edges.add(markoCreateLop);
        edges.add(joshCreateRipple);
        edges.add(joshCreateLop);
        edges.add(peterCreateLop);

        vertices = graph.addVertices(vertices);
        vertices.forEach(vertex -> System.out.println(vertex));

        edges = graph.addEdges(edges, false);
        edges.forEach(edge -> System.out.println(edge));

        hugeClient.close();
    }
}

4.4 Run The Example

Before running Example, you need to start the Server. For the startup process, seeHugeGraph-Server Quick Start.

4.5 More Information About Client-API

SeeIntroduce basic API of HugeGraph-Client.

3.5.2 - HugeGraph Python Client Quick Start

hugegraph-python-client is the Python SDK for HugeGraph. It manages schemas, reads and writes graph data, and executes Gremlin queries. HugeGraph-LLM and HugeGraph-ML also use this client.

The module lives in the hugegraph-ai repository under hugegraph-python-client/. The import name is pyhugegraph.

Requirements

  • The client and HugeGraph-AI workspace require Python 3.10 or later; CI runs client tests on 3.10 and 3.11.
  • HugeGraph Server 1.5.0 or later. The client refuses to connect to older servers; use client v1.3.x for those.
  • uv (recommended) or pip

Runtime dependencies are decorator, requests, setuptools, urllib3 and rich.

Installation

The released package is published on PyPI as hugegraph-python:

uv pip install hugegraph-python
# Alternatively: pip install hugegraph-python

These commands do not pin a version and install the latest release; see PyPI. The source directory is hugegraph-python-client, while the distribution is hugegraph-python. Install from the HugeGraph-AI workspace for master code.

To use the latest repository code, sync the workspace from the root of the HugeGraph-AI repository. hugegraph-python-client is a workspace member exposed through the python-client extra, so plain uv sync does not pull it in:

git clone https://github.com/apache/hugegraph-ai.git
cd hugegraph-ai
uv sync --extra python-client
source .venv/bin/activate

Connect and Write Data

from pyhugegraph.client import PyHugeClient

client = PyHugeClient(
    url="http://127.0.0.1:8080",
    graph="hugegraph",
    user="admin",
    pwd="admin",
    graphspace=None,
)

schema = client.schema()
schema.propertyKey("name").asText().ifNotExist().create()
schema.propertyKey("birthDate").asText().ifNotExist().create()
schema.vertexLabel("Person").properties("name", "birthDate") \
      .usePrimaryKeyId().primaryKeys("name").ifNotExist().create()
schema.vertexLabel("Movie").properties("name") \
      .usePrimaryKeyId().primaryKeys("name").ifNotExist().create()
schema.edgeLabel("ActedIn").sourceLabel("Person").targetLabel("Movie") \
      .ifNotExist().create()

graph = client.graph()
person = graph.addVertex(
    "Person", {"name": "Al Pacino", "birthDate": "1940-04-25"}
)
movie = graph.addVertex("Movie", {"name": "The Godfather"})
edge = graph.addEdge("ActedIn", person.id, movie.id, {})

print(graph.getVertexById(person.id))
print(graph.getEdgeById(edge.id))
graph.close()

Client Parameters

PyHugeClient(url, graph, user, pwd, graphspace=None, timeout=None)

ParameterTypeDefaultDescription
urlstrrequiredBase URL of HugeGraph Server. If the value has no scheme, http:// is prepended, so 127.0.0.1:8080 also works.
graphstrrequiredGraph name. This is the second positional parameter.
userstrrequiredUsername, sent as HTTP basic auth.
pwdstrrequiredPassword, sent as HTTP basic auth.
graphspacestr or NoneNoneGraphSpace name. See below for how None is resolved.
timeouttuple[float, float] or NoneNone(connect, read) timeouts in seconds. None becomes (0.5, 15.0).

Every HTTP session retries three times with a 0.1 backoff factor on 500, 502 and 504 responses.

Server Version and GraphSpace

The client resolves GraphSpace at construction time:

  • A non-empty graphspace string turns GraphSpace mode on directly.
  • Otherwise the client sends GET {url}/versions and reads versions.core.
  • A server older than 1.5.0 raises RuntimeError asking you to upgrade the server or use client v1.3.x.
  • A server newer than 1.5.0 gets graphspace set to DEFAULT and GraphSpace mode turned on, with a warning in the log. A server at exactly 1.5.0 keeps GraphSpace mode off.
  • If the probe fails for network reasons, GraphSpace mode stays off.

The mode decides the request prefix: /graphspaces/<graphspace>/graphs/<graph>/... when GraphSpace is on, /graphs/<graph>/... when it is off.

Managers on the Client

Each accessor builds its manager lazily and gives it a dedicated HTTP session.

AccessorManagerCovers
client.schema()SchemaManagerProperty keys, vertex labels, edge labels, index labels
client.graph()GraphManagerVertex and edge CRUD, batch writes, paging
client.gremlin()GremlinManagerGremlin execution
client.graphs()GraphsManagerGraph list, graph info, config, clear data
client.traverser()TraverserManagerTraversal and path algorithms
client.variable()VariableManagerGraph variables
client.task()TaskManagerAsync task list, query, cancel, delete
client.auth()AuthManagerUsers, groups, targets, belongs, accesses
client.metrics()MetricsManagerServer metrics
client.version()VersionManagerServer version

RankManager, RebuildManager and ServicesManager also ship in pyhugegraph.api, but PyHugeClient does not expose accessors for them yet; construct them directly with a session if you need them.

Common Operations

Build the Schema

The schema builders are fluent. Call create() last, or append(), eliminate() and remove() to change an existing definition.

schema = client.schema()

# Property keys: asText/asInt/asLong/asFloat/asDouble/asBool/asByte/asBlob/asDate/asObject
# cardinality: valueSingle/valueList/valueSet
# aggregation: calcMax/calcMin/calcSum/calcOld
schema.propertyKey("age").asInt().valueSingle().ifNotExist().create()

# Vertex labels: useAutomaticId/useCustomizeStringId/useCustomizeNumberId/usePrimaryKeyId
schema.vertexLabel("person").properties("name", "age", "city") \
      .primaryKeys("name").nullableKeys("city").ifNotExist().create()

# Edge labels: link() is shorthand for sourceLabel() plus targetLabel()
schema.edgeLabel("knows").link("person", "person").multiTimes() \
      .properties("date", "city").sortKeys("date").nullableKeys("city") \
      .ifNotExist().create()

# Index labels: onV/onE, then secondary/range/search/shard/unique
schema.indexLabel("personByCity").onV("person").by("city") \
      .secondary().ifNotExist().create()

Query the Schema

schema = client.schema()
print(schema.getSchema())            # whole schema, format defaults to "json"
print(schema.getPropertyKeys())
print(schema.getVertexLabels())
print(schema.getEdgeLabels())
print(schema.getIndexLabels())

# Single definitions
print(schema.getPropertyKey("name"))
print(schema.getVertexLabel("person"))
print(schema.getEdgeLabel("knows"))
print(schema.getIndexLabel("personByCity"))

# Edge label links, formatted as "Person--ActedIn-->Movie"
print(schema.getRelations())

Read, Update and Delete Graph Data

The graph API takes property dictionaries, not chained property builders:

graph = client.graph()
graph.appendVertex(person.id, {"birthDate": "1940-04-25"})    # add properties
graph.eliminateVertex(person.id, {"birthDate": "1940-04-25"}) # drop properties
graph.appendEdge(edge.id, {"city": "Beijing"})
graph.eliminateEdge(edge.id, {"city": "Beijing"})
graph.removeEdgeById(edge.id)
graph.removeVertexById(person.id)
graph.close()

addVertex returns a VertexData with id, label, type and properties. addEdge returns an EdgeData with id, label, type, outV, outVLabel, inV, inVLabel and properties.

Vertex ids passed to the client may be strings, integers or uuid.UUID values. Booleans are rejected, and integers must fit the Java signed long range.

Batch Writes

addVertices takes (label, properties) pairs, and addEdges takes (label, out_id, in_id, out_label, in_label, properties) tuples. Both return objects that carry only the generated ids.

graph = client.graph()
vertices = graph.addVertices([
    ("person", {"name": "Alice", "age": 20}),
    ("person", {"name": "Bob", "age": 23}),
])
edges = graph.addEdges([
    ("knows", vertices[0].id, vertices[1].id, "person", "person", {"date": "2012-01-10"}),
])

Paging and Conditional Queries

graph = client.graph()

# Returns (vertices, next_page); pass next_page back in to continue
vertices, next_page = graph.getVertexByPage("person", limit=10)
vertices, next_page = graph.getVertexByPage("person", limit=10, page=next_page)

# Server-side property predicates
older = graph.getVertexByCondition("person", properties={"age": "P.gt(29)"})

# Edges by page. When vertex_id is given, direction is required
edges, next_page = graph.getEdgeByPage(label="knows", limit=10)
edges, next_page = graph.getEdgeByPage(vertex_id=person.id, direction="OUT", limit=10)

# Batch lookup by id
graph.getVerticesById([v1.id, v2.id])
graph.getEdgesById([e1.id, e2.id])

Execute Gremlin

gremlin = client.gremlin()
result = gremlin.exec("g.V().limit(5)")
print(result)

exec binds the graph and g aliases for you, based on the graph name and the resolved GraphSpace, and returns the result field of the server response. A response missing requestId, status or result raises ResponseParseError.

Traverse the Graph

TraverserManager wraps the server traverser endpoints. Its methods use snake_case.

traverser = client.traverser()

traverser.k_out(marko_id, 2)
traverser.k_neighbor(marko_id, 2)
traverser.same_neighbors(marko_id, josh_id)
traverser.jaccard_similarity(marko_id, josh_id)
traverser.shortest_path(marko_id, ripple_id, 3)
traverser.all_shortest_paths(marko_id, ripple_id, 3)
traverser.weighted_shortest_path(marko_id, ripple_id, "weight", 3)
traverser.single_source_shortest_path(marko_id, 2)
traverser.multi_node_shortest_path([marko_id, josh_id], max_depth=2)
traverser.paths(marko_id, josh_id, 2)
traverser.crosspoints(marko_id, josh_id, 2)
traverser.rings(marko_id, 3)
traverser.rays(marko_id, 2)
traverser.vertices(marko_id)
traverser.edges(edge_id)

The POST-based variants take request bodies: advanced_paths, customized_paths, template_paths, customized_crosspoints and fusiform_similarity.

Graph Variables

variable = client.variable()
variable.set("owner", "mary")
print(variable.get("owner"))
print(variable.all())
variable.remove("owner")

Async Tasks

task = client.task()
print(task.list_tasks(status="success", limit=10))
print(task.get_task(task_id))
task.cancel_task(task_id)
task.delete_task(task_id)

Server Metrics and Graph Info

metrics = client.metrics()
metrics.get_all_basic_metrics()
metrics.get_gauges_metrics()
metrics.get_counters_metrics()
metrics.get_histograms_metrics()
metrics.get_meters_metrics()
metrics.get_timers_metrics()
metrics.get_statistics_metrics()
metrics.get_system_metrics()
metrics.get_backend_metrics()

graphs = client.graphs()
graphs.get_all_graphs()
graphs.get_version()
graphs.get_graph_info()
graphs.get_graph_config()
graphs.clear_graph_all_data()   # deletes every vertex, edge and schema entry

print(client.version().version())

Authentication and Authorization

AuthManager follows the server routing: users, targets, belongs and accesses are mounted under /graphspaces/{graphspace}/auth/..., while groups stay at the server-level /auth/groups. On HugeGraph 1.7.0 and later a graphspace must be resolved, otherwise these calls raise ValueError before any request is sent.

auth = client.auth()

user = auth.create_user("test_user", "password")
auth.modify_user(user["id"], user_email="hugegraph@apache.org")
auth.get_user(user["id"])
auth.list_users(limit=10)
auth.delete_user(user["id"])

group = auth.create_group("test_group", "read only")
auth.modify_group(group["id"], group_description="updated")
auth.list_groups()
auth.delete_group(group["id"])

target = auth.create_target("target1", "hugegraph", "127.0.0.1:8080", [])
auth.update_target(target["id"], "target1", "hugegraph", "127.0.0.1:8080", [])
auth.list_targets()
auth.delete_target(target["id"])

belong = auth.create_belong(user["id"], group["id"])
auth.update_belong(belong["id"], "description")
auth.list_belongs()
auth.delete_belong(belong["id"])

access = auth.grant_accesses(group["id"], target["id"], "READ")
auth.modify_accesses(access["id"], "description")
auth.list_accesses()
auth.revoke_accesses(access["id"])

Method Naming

Manager methods written in camelCase, such as addVertex and getVertexById, also get a snake_case alias generated at construction time. graph.add_vertex(...) and graph.addVertex(...) reach the same method. The camelCase spellings are marked deprecated in the debug log, so prefer snake_case in new code.

Error Handling

Exceptions live in pyhugegraph.utils.exceptions:

ExceptionRaised when
NotAuthorizedErrorThe server answers 401
NotFoundErrorThe server answers 404, or a required argument is missing
ServerErrorAny other non-2xx response, with the server message attached
ResponseParseErrorA successful response cannot be parsed into the expected shape
ServiceUnavailableErrorThe server reports ServiceUnavailableException
InvalidParameterError, CreateError, RemoveError, UpdateError, DataFormatErrorRaised by individual builders and structures

Request and response bodies are logged with password, token and secret values redacted.

from pyhugegraph.utils.exceptions import NotFoundError

try:
    graph.getVertexById("no-such-id")
except NotFoundError:
    print("vertex missing")

API parameters may change with the HugeGraph REST API version. If an interface is incompatible, first check the REST API documentation for the current server version and the client test cases.

Development Checks

Run formatting and static checks from the root of the HugeGraph-AI repository:

./style/code_format_and_analysis.sh

Run the tests the same way CI does:

# Unit and contract tests, no server needed
uv run pytest hugegraph-python-client/src/tests -m "unit or contract"

# Integration tests against a running server
HUGEGRAPH_URL=http://127.0.0.1:8080 \
HUGEGRAPH_GRAPH=hugegraph \
HUGEGRAPH_USER=admin \
HUGEGRAPH_PASSWORD=admin \
uv run pytest hugegraph-python-client/src/tests -m "integration and hugegraph"

CI runs the integration job against the hugegraph/hugegraph:1.7.0 image. HUGEGRAPH_GRAPHSPACE is also read when you need a non-default space.

The source code and tests are under hugegraph-python-client/src/pyhugegraph/ and hugegraph-python-client/src/tests/. A runnable example is at hugegraph-python-client/src/pyhugegraph/example/hugegraph_example.py.

3.5.3 - HugeGraph Go Client Quick Start

HugeGraph Go Client is the Go SDK in the Toolchain repository. It currently provides APIs for version queries, schemas (property keys, vertex labels, and edge labels), vertices, and Gremlin. An edge data API is not implemented yet.

This module is still under development. Refer to the source code under hugegraph-client-go/api/v1 for the currently available interfaces.

Requirements

  • Go 1.19 or later
  • An accessible HugeGraph Server; examples use http://127.0.0.1:8080

Installation

Run the following command in a Go module project:

go get github.com/apache/hugegraph-toolchain/hugegraph-client-go

Initialize the Client

NewCommonClient requires Host to be an IP address and Port to be between 1 and 65535. The client always connects over plain HTTP. Leave the username and password empty when authentication is disabled; Basic Auth is sent only when both are set.

GraphSpace is applied only by the Vertex API, which then calls /graphspaces/{space}/graphs/{graph}/..., and by the default Gremlin aliases (an empty value is treated as DEFAULT). The schema entry points and Version() always call /graphs/{graph}/... and /versions, regardless of GraphSpace. Use DEFAULT for the default space; leaving GraphSpace empty makes the Vertex API fall back to the /graphs/{graph} path used by older servers.

package main

import (
	"fmt"
	"log"

	hugegraph "github.com/apache/hugegraph-toolchain/hugegraph-client-go"
)

func main() {
	client, err := hugegraph.NewCommonClient(hugegraph.Config{
		Host:       "127.0.0.1",
		Port:       8080,
		GraphSpace: "DEFAULT",
		Graph:      "hugegraph",
		Username:   "",
		Password:   "",
	})
	if err != nil {
		log.Fatal(err)
	}

	response, err := client.Version()
	if err != nil {
		log.Fatal(err)
	}
	defer response.Body.Close()

	fmt.Println(response.Versions.Version)
}

The Versions value returned by Version() includes the HugeGraph Server, Core, Gremlin, and REST API versions. The NewDefaultCommonClient() helper in the source connects to the hugegraph graph at 127.0.0.1:8080 with admin/pa authentication and a ColorLogger that prints every request and response body. Production code should normally pass an explicit configuration instead.

Configuration Options

hugegraph.Config has the following fields:

FieldTypeDescription
HoststringHugeGraph Server IP address. Host names are rejected.
PortintHugeGraph Server REST port, 1 to 65535
GraphSpacestringGraph space; only used by the Vertex API and the default Gremlin aliases. Set an empty string when not needed.
GraphstringGraph name configured on the server
UsernamestringServer username; empty string when authentication is disabled
PasswordstringServer password; empty string when authentication is disabled
Transporthttp.RoundTripperCustom HTTP transport; http.DefaultTransport when nil
Loggerhgtransport.LoggerRequest/response logger; no logging when nil

The hgtransport package ships four loggers: TextLogger (plain text), ColorLogger (terminal colors), CurlLogger (runnable curl commands), and JSONLogger (JSON lines). Each has the same fields: Output (an io.Writer), EnableRequestBody, and EnableResponseBody.

import (
	"os"

	hugegraph "github.com/apache/hugegraph-toolchain/hugegraph-client-go"
	"github.com/apache/hugegraph-toolchain/hugegraph-client-go/hgtransport"
)

client, err := hugegraph.NewCommonClient(hugegraph.Config{
	Host:  "127.0.0.1",
	Port:  8080,
	Graph: "hugegraph",
	Logger: &hgtransport.ColorLogger{
		Output:             os.Stdout,
		EnableRequestBody:  true,
		EnableResponseBody: true,
	},
})

Available Entry Points

CommonClient currently exposes the following entry points:

Entry pointPurpose
Version()Query the server version
Schema()Query the complete schema
PropertykeyCreate, GetAll, GetByName, UpdateUserdata, DeleteByName
VertexLabelCreate, GetAll, GetByName, UpdateUserdata, DeleteByName
EdgeLabelCreate, GetAll, DeleteByName
VertexCreate, BatchCreate, UpdateProperties (with WithAction: append or eliminate)
GremlinGet and Post. Post defaults language to gremlin-groovy, fills the graph/g aliases from GraphSpace and Graph, and returns the parsed result in Data. Get only returns the status code and prints the raw response to stdout.

Each operation takes functional options named With... on the operation itself, for example client.Gremlin.Post.WithGremlin(...) or client.Propertykey.GetByName.WithName(...).

resp, err := client.Gremlin.Post(
	client.Gremlin.Post.WithGremlin("g.V().limit(3)"),
)
if err != nil {
	log.Fatal(err)
}
fmt.Println(resp.StatusCode, resp.Data.Status.Code, resp.Data.Result.Data)

The Vertex operations take model.Vertex[any] values from the internal/model package. Go does not allow importing an internal package from another module, so at the moment the Vertex API can only be called from code inside the client module itself; its test file is also fully commented out.

For complete usage, see the tests in each API directory, such as version_test.go, gemlin_test.go, and vertexlabel_test.go.

4 - HugeGraph-Server Configuration

This section covers HugeGraph-Server configuration files, available options, authentication, and HTTPS settings.

Backend Configuration

4.1 - Server Startup Guide

1 Overview

Default configuration lives under conf/ in the extracted distribution. External HBase, Kerberos, and HTTPS files can use other paths; see the corresponding backend, authentication, and HTTPS guides.

The main files are gremlin-server.yaml, rest-server.properties, and hugegraph.properties.

HugeGraphServer integrates GremlinServer and RestServer, configured by gremlin-server.yaml and rest-server.properties respectively.

  • GremlinServer: Accepts Gremlin requests and invokes the graph engine.
  • RestServer: Provides REST APIs that invoke Core APIs. A Gremlin request body is forwarded to GremlinServer to operate on graph data.

The following sections describe these files.

Warning

Production requires Server authentication and network access controls

The authentication settings on this page apply to HugeGraph Server. In production, enable authentication and authorization, maintain an IP allowlist, grant minimum permissions, and retain and protect Server audit-*.log files. Commented authentication settings in the examples indicate that authentication is disabled by default.

The distribution copies default files from the Server repository into the installation directory’s conf/ folder. Options absent from these files use source-code defaults; explicit file settings override those defaults. The file-loading behavior on this page was checked against Server master commit 2f827d6. Use the configuration files and entry scripts that correspond to your installed release.

FileSource and purposeLoading behavior
conf/gremlin-server.yamlShipped with the distribution; configures Gremlin Server.Passed to Server by start-hugegraph.sh.
conf/rest-server.propertiesShipped when building the checked Server master; configures REST Server, PD, authentication, and related settings.Read by startup scripts and Server; the corresponding container entrypoint also updates supported settings before startup.
conf/graphs/hugegraph.propertiesShipped default graph configuration, using RocksDB.Scanned by both init-store.sh and application initialization, which attempts to load local graphs. graph.load_from_local_config defaults to false and controls only constructor preloading and rescanning on reload().
conf/graphs/hstore.properties.templateShipped HStore template; users create other graph configurations as needed.The HStore image renames it to hugegraph.properties during its build; distribution users can copy and customize it.

Complete defaults are in Server master: gremlin-server.yaml, rest-server.properties, hugegraph.properties, and hstore.properties.template.

The following Docker environment-variable behavior applies only to images built from master; arbitrary variable names are not converted into configuration options. Before initialization and startup, this entrypoint processes HG_SERVER_BACKEND, HG_SERVER_PD_PEERS, HG_SERVER_USE_PD, HG_SERVER_CLUSTER, HG_SERVER_REST_URL, HG_SERVER_MIN_FREE_MEMORY, HG_SERVER_AUTH_TOKEN_SECRET, HG_SERVER_INIT_STORE_ENABLED, and PASSWORD. Check the tagged entrypoint for historical release images. See Docker entrypoint.

2 gremlin-server.yaml

This example shows commonly adjusted startup options. Use the source links above for the complete serializer and plugin configuration.

The shipped host and port lines are commented out. Without explicit values, the address is 127.0.0.1:8182. Uncomment and set these options to change the listener:

conf/gremlin-server.yaml
#host: 127.0.0.1
#port: 8182
evaluationTimeout: 30000
channelizer: org.apache.tinkerpop.gremlin.server.channel.WsAndHttpChannelizer
graphs: {}
ssl: { enabled: false }

Usually, focus on channelizer, host, and port. Graphs are not loaded from the Gremlin Server graphs section. During application initialization, the REST-side manager scans the graphs directory and attempts to load local graph configurations. graph.load_from_local_config controls only constructor preloading and rescanning on reload(); its default false does not disable local loading during application initialization.

  • channelizer: The default WsAndHttpChannelizer supports WebSocket and HTTP. Gremlin Console uses WebSocket; HugeGraph Client, Loader, and Hubble use HTTP.

GremlinServer listens at 127.0.0.1:8182 by default. Set host and port to change this address.

  • host: The hostname or IP of the GremlinServer host. RestServer forwards Gremlin requests; GremlinServer is not directly exposed to users.
  • port: The GremlinServer listener port.

Set the matching gremlinserver.url=http://host:port in rest-server.properties.

3 rest-server.properties

The following example lists rest-server.properties options. The master template omits graph.load_from_local_config, whose source-code default is false. Setting it to true is optional: it enables constructor preloading and rescanning on reload(). Application initialization still scans and attempts to load local graph configurations.

# bind url
# could use '0.0.0.0' or specified (real)IP to expose external network access
restserver.url=http://127.0.0.1:8080
#restserver.enable_graphspaces_filter=false
# gremlin server url, need to be consistent with host and port in gremlin-server.yaml
#gremlinserver.url=127.0.0.1:8182

graphs=./conf/graphs
graph.load_from_local_config=true

# The maximum thread ratio for batch writing, only take effect if the batch.max_write_threads is 0
batch.max_write_ratio=80
batch.max_write_threads=0

# configuration of arthas
arthas.telnetPort=8562
arthas.httpPort=8561
arthas.ip=127.0.0.1
arthas.disabledCommands=jad

# authentication configs
#auth.authenticator=org.apache.hugegraph.auth.StandardAuthenticator
# for admin password, By default, it is pa and takes effect upon the first startup
#auth.admin_pa=pa
#auth.graph_store=hugegraph

# use pd
# usePD=true

# slow query log
log.slow_query_threshold=1000
# bytes of request body recorded as-is (may contain sensitive literals), 0 to disable
log.slow_query_body_limit=512

# jvm(in-heap) memory usage monitor, set 1 to disable it
memory_monitor.threshold=0.85
memory_monitor.period=2000
  • restserver.url: The REST listener URL. Use a specific address for remote access, or http://0.0.0.0 to listen on every interface while restricting the accessible network.
  • graphs: The graph configuration directory, defaulting to ./conf/graphs. Both init-store.sh and application initialization scan it; initialization attempts to load its properties files.
  • graph.load_from_local_config: Controls constructor preloading and rescanning on reload(), with source-code default false. It does not block local loading during application initialization and is not a security isolation switch.

The upstream template still uses arthas.telnet_port, arthas.http_port, and arthas.disabled_commands, but ServerOptions reads the camelCase names in the example. Use arthas.telnetPort, arthas.httpPort, and arthas.disabledCommands in custom configurations.

gremlinserver.url is the GremlinServer address used by RestServer, defaulting to http://127.0.0.1:8182. It must match host and port in gremlin-server.yaml. Like the template, it can omit the scheme; missing schemes receive an http:// prefix.

4 hugegraph.properties

hugegraph.properties is the default shipped graph configuration. Each additional graph needs its own properties file under conf/graphs. This example shows the key settings for the default RocksDB backend; see the source links above for the complete file.

gremlin.graph=org.apache.hugegraph.HugeFactory
backend=rocksdb
serializer=binary
store=hugegraph

The main uncommented options are:

  • gremlin.graph: The graph entry point used by GremlinServer. Leave it unchanged unless enabling authentication, which uses org.apache.hugegraph.auth.HugeFactoryAuthProxy.
  • vertex.cache_type / edge.cache_type: Cache implementation, either l1 or l2, defaulting to l2.
  • backend: The storage backend. Version 1.7.0 supports memory, rocksdb, hstore, and hbase.
  • serializer: Serialization of schemas, vertices, and edges. RocksDB uses binary.
  • store: The graph’s backend store name.
  • task.schedule_period, task.retry, task.wait_timeout: Task scheduling period in seconds, retry count, and wait timeout in seconds. HStore uses a distributed scheduler; other backends use a local scheduler. The old task.scheduler_type key is ignored.
  • search.text_analyzer / search.text_analyzer_mode: Full-text analyzer and mode. Supported analyzers include ansj, hanlp, smartcn, jieba, jcseg, mmseg4j, and ikanalyzer, each with its own mode values.
  • rocksdb.data_path: RocksDB data directory, defaulting to rocksdb-data/data; applies only when backend=rocksdb.
  • rocksdb.wal_path: RocksDB WAL directory, defaulting to rocksdb-data/wal; applies only when backend=rocksdb.

5 Multi-Graph Configuration

A Server can load multiple graphs, each with its own properties file. This example creates a RocksDB graph hugegraph_rocksdb and an in-memory graph hugegraph_memory.

[Optional]: Modify rest-server.properties

Set the graph configuration directory with graphs in rest-server.properties, defaulting to graphs=./conf/graphs. Adjust it for another directory. The example optionally enables constructor preloading and rescanning on reload() with graph.load_from_local_config=true; application initialization still reads local graph configurations from this directory:

graphs=./conf/graphs
graph.load_from_local_config=true

Under conf/graphs, create hugegraph_memory.properties and hugegraph_rocksdb.properties based on hugegraph.properties.

Modify hugegraph_memory.properties as follows:

backend=memory
serializer=text
store=hugegraph_memory

Modify hugegraph_rocksdb.properties as follows:

backend=rocksdb
serializer=binary

store=hugegraph_rocksdb

Stop Server, run init-store.sh to initialize the new graphs, then restart Server.

$ ./bin/stop-hugegraph.sh
$ ./bin/init-store.sh

Initializing HugeGraph Store...
2023-06-11 14:16:14 [main] [INFO] o.a.h.u.ConfigUtil - Scanning option 'graphs' directory './conf/graphs'
2023-06-11 14:16:14 [main] [INFO] o.a.h.c.InitStore - Init graph with config file: ./conf/graphs/hugegraph_rocksdb.properties
...
2023-06-11 14:16:15 [main] [INFO] o.a.h.StandardHugeGraph - Graph 'hugegraph_rocksdb' has been initialized
2023-06-11 14:16:15 [main] [INFO] o.a.h.c.InitStore - Init graph with config file: ./conf/graphs/hugegraph_memory.properties
...
2023-06-11 14:16:16 [main] [INFO] o.a.h.StandardHugeGraph - Graph 'hugegraph_memory' has been initialized
2023-06-11 14:16:16 [main] [INFO] o.a.h.StandardHugeGraph - Close graph standardhugegraph[hugegraph_rocksdb]
...
2023-06-11 14:16:16 [main] [INFO] o.a.h.HugeFactory - HugeFactory shutdown
2023-06-11 14:16:16 [hugegraph-shutdown] [INFO] o.a.h.HugeFactory - HugeGraph is shutting down
Initialization finished.
$ ./bin/start-hugegraph.sh

Starting HugeGraphServer in daemon mode...
Connecting to HugeGraphServer (http://127.0.0.1:8080/graphs)...OK
Started [pid 21614]

List the created graphs:

curl http://127.0.0.1:8080/graphspaces/DEFAULT/graphs

{"graphs":["hugegraph_rocksdb","hugegraph_memory"]}

Inspect a graph:

curl http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph_memory

{"name":"hugegraph_memory","backend":"memory"}
curl http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph_rocksdb

{"name":"hugegraph_rocksdb","backend":"rocksdb"}

4.2 - Server Complete Configuration Manual

Defaults in these tables are source-code defaults used when an option is not explicitly set. Distribution files can specify different values and override them; use the configuration shipped with your installed release to determine effective settings.

Gremlin Server Config Options

Corresponding configuration file gremlin-server.yaml

config optiondefault valuedescription
host127.0.0.1The host or ip of Gremlin Server.
port8182The listening port of Gremlin Server.
graphs{}Graphs are loaded dynamically by the Server; do not configure them here.
evaluationTimeout30000Gremlin script evaluation timeout in milliseconds.
channelizerorg.apache.tinkerpop.gremlin.server.channel.WsAndHttpChannelizerHandles both WebSocket and HTTP requests.
maxContentLength65536Maximum size in bytes of a request that the server accepts.
maxChunkSize8192Maximum chunk size in bytes of an HTTP request.
maxHeaderSize8192Maximum size in bytes of the HTTP request headers.
resultIterationBatchSize64Number of results returned per batch when streaming a result set.
ssl.enabledfalseWhether Gremlin Server serves over TLS.
authenticationNot configuredWhen enabling authentication, configure the authenticator, handler, and path to rest-server.properties.

Rest Server & API Config Options

Corresponding configuration file rest-server.properties

config optiondefault valuedescription
graphs./conf/graphsDirectory containing graph configuration properties files.
graph.load_from_local_configfalseControls local graph preloading in the manager constructor and rescanning on reload(). Application initialization still scans and attempts to load graphs; false does not prevent local loading.
graphs.enable_dynamic_create_droptrueWhether to enable create or drop graph dynamically.
init_store.enabledtrueWhether init-store initializes the local backend stores and the built-in admin account. Set false in distributed deployments (PD/HStore) where the storage side already owns the metadata.
server.idEmpty stringThe optional legacy id of hugegraph-server.
server.rolemasterThe role of nodes in the cluster, available types are [master, worker, computer]
server.node_idnode-id1The node id of the server.
server.node_roleworkerThe node role of the server.
server.graphspaceDEFAULTThe graph space of the server.
server.service_idDEFAULTThe service id of the server.
server.path_graphspaceDEFAULTThe default path graph space of the server.
server.start_ignore_single_graph_errortrueWhether to start ignore single graph error.
server.event_hub_threads1The event hub threads of server.
restserver.urlhttp://127.0.0.1:8080The url for listening of graph server.
ssl.keystore_fileconf/hugegraph-server.keystoreThe path of server keystore file used when https protocol is enabled.
ssl.keystore_passwordhugegraphThe password of the server keystore file when the https protocol is enabled.
white_ip.statusdisableThe status of whether enable white ip.
restserver.max_worker_threads2 * CPUsThe maximum worker threads of rest server.
restserver.task_threadsmax(4, CPUs / 2)The task threads of rest server.
restserver.min_free_memory64The minimum free memory(MB) of rest server, requests will be rejected when the available memory of system is lower than this value.
restserver.request_timeout30The time in seconds within which a request must complete, -1 means no timeout.
restserver.connection_idle_timeout30The time in seconds to keep an inactive connection alive, -1 means no timeout.
restserver.connection_max_requests256The max number of HTTP requests allowed to be processed on one keep-alive connection, -1 means unlimited.
gremlinserver.urlhttp://127.0.0.1:8182The url of gremlin server.
gremlinserver.max_route2 * CPUsThe max route number for gremlin server.
gremlinserver.timeout30The timeout in seconds of waiting for gremlin server.
batch.max_edges_per_batch2500The maximum number of edges submitted per batch.
batch.max_vertices_per_batch2500The maximum number of vertices submitted per batch.
batch.max_write_ratio70The maximum thread ratio for batch writing, only take effect if the batch.max_write_threads is 0.
batch.max_write_threads0The maximum threads for batch writing, if the value is 0, the actual value will be set to batch.max_write_ratio * restserver.max_worker_threads.
raft.group_peers127.0.0.1:8090The rpc address of raft group initial peers.
auth.authenticatorThe class path of authenticator implementation. e.g., org.apache.hugegraph.auth.StandardAuthenticator, or a custom implementation.
auth.graph_storehugegraphThe name of graph used to store authentication information, like users, only for org.apache.hugegraph.auth.StandardAuthenticator.
auth.admin_papaUsed when the Server startup path initializes admin. Explicitly set a strong password before production deployment; the public default is pa. On local persistent backends, init-store.sh prompts for the initial password.
auth.remote_urlIf the address is empty, it provide auth service, otherwise it is auth client and also provide auth service through rpc forwarding. The remote url can be set to multiple addresses, which are concat by ‘,’.
exception.allow_tracetrueWhether to allow exception trace stack.
memory_monitor.threshold0.85Threshold for JVM memory usage monitoring, 1 means disabling the memory monitoring task.
memory_monitor.period2000The period in ms of JVM memory usage monitoring, in each period we will detect the jvm memory usage and take corresponding actions.
log.slow_query_threshold1000The threshold time(ms) of logging slow query, 0 means logging slow query is disabled.
log.slow_query_body_limit512The max bytes of request body recorded in the slow query log, 0 means the body is not recorded. The recorded prefix is written as-is and may contain sensitive Gremlin or Cypher literals.

The role election options server.role_election and server.role.* apply to version 1.7.0 and earlier only. Later versions no longer read them: a server that still sets them starts normally and logs a warning for each key.

PD/Meta Config Options (Distributed Mode)

Corresponding configuration file rest-server.properties

config optiondefault valuedescription
usePDfalseWhether use pd.
pd.peers127.0.0.1:8686The pd server peers, separated with commas.
clusterhg-testThe cluster name.
pd.stores_wait_timeout300Seconds to wait for at least pd.initial-store-count active Stores before opening an HStore graph. Range: 0..2147483647; 0 disables waiting.
metrics.data_to_pdtrueWhether to report metrics data to pd.
meta.endpointshttp://127.0.0.1:2379The URL of meta endpoints. No code reads this option, so setting it has no effect; the meta connection is built from pd.peers.
meta.use_cafalseWhether to use ca to meta server.
meta.caThe ca file of meta server.
meta.client_caThe client ca file of meta server.
meta.client_keyThe client key file of meta server.

The HStore backend also reads two options from the graph configuration file {graph-name}.properties. Both default to 0, which means the value is decided by PD:

config optiondefault valuedescription
hstore.partition_count0Number of partitions, which PD controls partitions based on.
hstore.shard_count0Number of copies, which PD controls partition copies based on.

Authentication Graph Config Options

Write these options in the properties file for the graph specified by auth.graph_store, normally conf/graphs/hugegraph.properties. Configure auth.authenticator, auth.graph_store, auth.admin_pa, and auth.remote_url in the rest-server.properties table above.

config optiondefault valuedescription
auth.audit_log_rate1000.0Maximum audit records per user per second. Non-negative values are truncated to integers: 1.9 becomes 1; 0 or values below 1 suppress audit logging. Use a positive integer to retain audit records.
auth.cache_capacity10240Maximum entries per authentication cache; a non-negative integer.
auth.cache_expire600Authentication client and Server cache expiration in seconds; a non-negative integer.
auth.token_expire86400JWT lifetime in seconds; a non-negative integer.
auth.token_secret32 random bytes encoded as Base64Authentication graph setting; the default is not written back to the file. Explicitly set the same strong random value on every node to validate tokens across restarts or nodes. HS256 requires at least 32 UTF-8 bytes.

Basic Config Options

Basic Config Options and Backend Config Options correspond to configuration files:{graph-name}.properties, such as hugegraph.properties

config optiondefault valuedescription
gremlin.graphorg.apache.hugegraph.HugeFactoryGremlin entrance to create graph.
backendmemoryThe data store type. For version 1.7.0+ the allowed values are [memory, rocksdb, hstore, hbase]; the shipped conf/graphs/hugegraph.properties sets rocksdb and conf/graphs/hstore.properties.template sets hstore. Note: cassandra, scylladb, mysql, postgresql were removed in 1.7.0 (use <= 1.5.x for legacy backends).
serializertextThe serializer for backend store, built-in values are [text, binary, binaryscatter]; a backend may register its own, like hbase. The shipped graph templates set binary.
serializer.buffer_max_capacity134217728The process-wide max capacity of one serialization buffer in bytes.
storehugegraphThe backend database namespace.
store.connection_detect_interval600The interval in seconds for detecting connections, if the idle time of a connection exceeds this value, detect it and reconnect if needed before using, value 0 means detecting every time.
store.graphgThe graph table name, which store vertex, edge and property.
graphspaceDEFAULTThe graph space name.
alias.graph.idThe graph alias id.
graph.read_modeOLTP_ONLYThe graph read mode, which could be ALL | OLTP_ONLY | OLAP_ONLY.
pd.peers127.0.0.1:8686The addresses of pd nodes, separated with commas. Only used by the hstore backend.
pd.clusterhgCluster prefix used for graph-level PD metadata connections when usePD=false. With usePD=true, the REST cluster setting takes precedence. This value is bound once per process.
schema.illegal_name_regex.\s+$|~.The regex specified the illegal format for schema name.
schema.cache_capacity10000The max cache size(items) of schema cache.
schema.init_templateThe template schema used to init graph.
schema.index_rebuild_using_pushdowntrueWhether to use pushdown when to create/rebuild index.
vertex.cache_typel2The type of vertex cache, allowed values are [l1, l2].
vertex.cache_capacity10000000The max cache size(items) of vertex cache.
vertex.cache_expire600The expiration time in seconds of vertex cache.
vertex.check_customized_id_existfalseWhether to check the vertices exist for those using customized id strategy.
vertex.default_labelvertexThe default vertex label.
vertex.tx_capacity10000The max size(items) of vertices(uncommitted) in transaction.
vertex.check_adjacent_vertex_existfalseWhether to check the adjacent vertices of edges exist.
vertex.lazy_load_adjacent_vertextrueWhether to lazy load adjacent vertices of edges.
vertex.part_edge_commit_size5000Whether to enable the mode to commit part of edges of vertex, enabled if commit size > 0, 0 means disabled.
vertex.encode_primary_key_numbertrueWhether to encode number value of primary key in vertex id.
vertex.remove_left_index_at_overwritefalseWhether remove left index at overwrite.
edge.cache_typel2The type of edge cache, allowed values are [l1, l2].
edge.cache_capacity1000000The max cache size(items) of edge cache.
edge.cache_expire600The expiration time in seconds of edge cache.
edge.tx_capacity10000The max size(items) of edges(uncommitted) in transaction.
query.page_size500The size of each page when querying by paging.
query.batch_size1000The size of each batch when querying by batch.
query.ignore_invalid_datatrueWhether to ignore invalid data of vertex or edge.
query.index_intersect_threshold1000The maximum number of intermediate results to intersect indexes when querying by multiple single index properties.
query.max_indexes_available1The upper limit of the number of indexes that can be used to query.
query.dedup_optionlimitThe way to dedup data, allowed values are [limit, global].
query.trust_indexfalseWhether to trust index.
query.ramtable_edges_capacity20000000The maximum number of edges in ramtable, include OUT and IN edges.
query.ramtable_enablefalseWhether to enable ramtable for query of adjacent edges.
query.ramtable_vertices_capacity10000000The maximum number of vertices in ramtable, generally the largest vertex id is used as capacity.
query.optimize_aggregate_by_indexfalseWhether to optimize aggregate query(like count) by index.
oltp.concurrent_depth10The min depth to enable concurrent oltp algorithm.
oltp.concurrent_threadsmax(10, CPUs / 2)Thread number to concurrently execute oltp algorithm.
oltp.collection_typeECThe implementation type of collections used in oltp algorithm, allowed values are [JCF, EC, FU].
oltp.query_batch_size10000The size of each batch when executing oltp algorithm.
oltp.query_batch_avg_degree_ratio0.95The ratio of exponential approximation for average degree of iterator when executing oltp algorithm.
oltp.query_batch_expect_degree100000000The expect sum of degree in each batch when executing oltp algorithm.
rate_limit.read0The max rate(times/s) to execute query of vertices/edges.
rate_limit.write0The max rate(items/s) to add/update/delete vertices/edges.
task.schedule_period10Period time in seconds when scheduler to schedule task.
task.wait_timeout10Timeout in seconds for waiting for the task to complete, such as when truncating or clearing the backend.
task.retry0Task retry times, allowed range is [0, 3].
task.input_size_limit16777216The job input size limit in bytes.
task.result_size_limit16777216The job result size limit in bytes.
task.sync_deletionfalseWhether to delete schema or expired data synchronously.
task.ttl_delete_batch1The batch size used to delete expired data.
computer.config./conf/computer.yamlThe config file path of computer job.
k8s.operator_template./conf/operator-template.yamlThe path of operator container template.
k8s.quota_template./conf/resource-quota-template.yamlThe path of resource quota template.
search.text_analyzerikanalyzerChoose a text analyzer for searching the vertex/edge properties, available type are [ansj, hanlp, smartcn, jieba, jcseg, mmseg4j, ikanalyzer]. The shipped graph templates set jieba. If use ‘ikanalyzer’, need download jar from ‘https://github.com/apache/hugegraph-doc/raw/ik_binary/dist/server/ikanalyzer-2012_u6.jar' to lib directory
search.text_analyzer_modesmartSpecify the mode for the text analyzer, the available mode of analyzer are {ansj: [BaseAnalysis, IndexAnalysis, ToAnalysis, NlpAnalysis], hanlp: [standard, nlp, index, nShort, shortest, speed], smartcn: [], jieba: [SEARCH, INDEX], jcseg: [Simple, Complex], mmseg4j: [Simple, Complex, MaxWord], ikanalyzer: [smart, max_word]}.
snowflake.datacenter_id0The datacenter id of snowflake id generator.
snowflake.force_stringfalseWhether to force the snowflake long id to be a string.
snowflake.worker_id0The worker id of snowflake id generator.
memory.modeoff-heapThe memory mode used for query in HugeGraph.
memory.max_capacity1073741824The maximum memory capacity in bytes that can be managed for all queries in HugeGraph.
memory.one_query_max_capacity104857600The maximum memory capacity in bytes that can be managed for a query in HugeGraph.
memory.alignment8The alignment used for round memory size.
Raft Config Options (Deprecated)

The shipped graph configuration templates mark these options as deprecated. They only take effect when raft.mode=true, and raft.group_peers is read from rest-server.properties instead of the graph file.

config optiondefault valuedescription
raft.modefalseWhether the backend storage works in raft mode.
raft.safe_readfalseWhether to use linearly consistent read.
raft.path./raftlogThe log path of current raft node.
raft.use_replicator_pipelinetrueWhether to use replicator line, when turned on it multiple logs can be sent in parallel, and the next log doesn’t have to wait for the ack message of the current log to be sent.
raft.election_timeout10000Timeout in milliseconds to launch a round of election.
raft.snapshot_interval3600The interval in seconds to trigger snapshot save.
raft.snapshot_threads4The thread number used to do snapshot.
raft.snapshot_parallel_compressfalseWhether to enable parallel compress.
raft.snapshot_compress_threads4The thread number used to do snapshot compress.
raft.snapshot_decompress_threads4The thread number used to do snapshot decompress.
raft.backend_threadsCPUsThe thread number used to apply task to backend.
raft.read_index_threads8The thread number used to execute reading index.
raft.read_strategyReadOnlyLeaseBasedThe linearizability of read strategy, allowed values are [ReadOnlyLeaseBased, ReadOnlySafe].
raft.apply_batch1The apply batch size to trigger disruptor event handler.
raft.queue_size16384The disruptor buffers size for jraft RaftNode, StateMachine and LogManager.
raft.queue_publish_timeout60The timeout in second when publish event into disruptor.
raft.rpc_threadsmax(CPUs * 2, 80)The rpc threads for jraft RPC layer.
raft.rpc_connect_timeout5000The rpc connect timeout in milliseconds for jraft rpc.
raft.rpc_timeout60The general rpc timeout in seconds for jraft rpc.
raft.install_snapshot_rpc_timeout36000The install snapshot rpc timeout in seconds for jraft rpc.
raft.rpc_buf_low_water_mark10485760The ChannelOutboundBuffer’s low water mark of netty, when buffer size less than this size, the method ChannelOutboundBuffer.isWritable() will return true, it means that low downstream pressure or good network.
raft.rpc_buf_high_water_mark20971520The ChannelOutboundBuffer’s high water mark of netty, only when buffer size exceed this size, the method ChannelOutboundBuffer.isWritable() will return false, it means that the downstream pressure is too great to process the request or network is very congestion, upstream needs to limit rate at this time.

RocksDB Backend Config Options

config optiondefault valuedescription
backendMust be set to rocksdb.
serializerMust be set to binary.
rocksdb.data_pathrocksdb-data/dataThe path for storing data of RocksDB.
rocksdb.wal_pathrocksdb-data/walThe path for storing WAL of RocksDB.
rocksdb.sst_pathThe path for ingesting SST file into RocksDB.
rocksdb.data_disks[]The optimized disks for storing data of RocksDB. The format of each element: STORE/TABLE: /path/disk.Allowed keys are [g/vertex, g/edge_out, g/edge_in, g/vertex_label_index, g/edge_label_index, g/range_int_index, g/range_float_index, g/range_long_index, g/range_double_index, g/secondary_index, g/search_index, g/shard_index, g/unique_index, g/olap]
rocksdb.log_levelINFOThe info log level of RocksDB.
rocksdb.num_levels7Set the number of levels for this database.
rocksdb.compaction_styleLEVELSet compaction style for RocksDB: LEVEL/UNIVERSAL/FIFO.
rocksdb.optimize_modetrueOptimize for heavy workloads and big datasets.
rocksdb.bulkload_modefalseSwitch to the mode to bulk load data into RocksDB.
rocksdb.compression_per_level[none, none, snappy, snappy, snappy, snappy, snappy]The compression algorithms for different levels of RocksDB, allowed values are none/snappy/z/bzip2/lz4/lz4hc/xpress/zstd.
rocksdb.bottommost_compressionnoneThe compression algorithm for the bottommost level of RocksDB, allowed values are none/snappy/z/bzip2/lz4/lz4hc/xpress/zstd.
rocksdb.compressionsnappyThe compression algorithm for compressing blocks of RocksDB, allowed values are none/snappy/z/bzip2/lz4/lz4hc/xpress/zstd.
rocksdb.max_background_jobs8Maximum number of concurrent background jobs, including flushes and compactions.
rocksdb.max_subcompactions4The value represents the maximum number of threads per compaction job.
rocksdb.delayed_write_rate16777216The rate limit in bytes/s of user write requests when need to slow down if the compaction gets behind.
rocksdb.max_open_files-1The maximum number of open files that can be cached by RocksDB, -1 means no limit.
rocksdb.max_manifest_file_size104857600The max size of manifest file in bytes.
rocksdb.skip_stats_update_on_db_openfalseWhether to skip statistics update when opening the database, setting this flag true allows us to not update statistics.
rocksdb.skip_check_sst_size_on_db_openfalseWhether to skip checking sizes of all sst files when opening the database.
rocksdb.max_file_opening_threads16The max number of threads used to open files.
rocksdb.max_total_wal_size0Total size of WAL files in bytes. Once WALs exceed this size, we will start forcing the flush of column families related, 0 means no limit.
rocksdb.bytes_per_sync0Allows OS to incrementally sync SST files to disk while they are being written, asynchronously in the background. Issue one request for every bytes_per_sync written. 0 turns it off.
rocksdb.wal_bytes_per_sync0Allows OS to incrementally sync WAL files to disk while they are being written, asynchronously in the background. Issue one request for every bytes_per_sync written. 0 turns it off.
rocksdb.strict_bytes_per_syncfalseWhen true, guarantees SST/WAL files have at most bytes_per_sync/wal_bytes_per_sync bytes submitted for writeback at any given time. This can be used to handle cases where processing speed exceeds I/O speed.
rocksdb.db_write_buffer_size0Total size of write buffers in bytes across all column families, 0 means no limit.
rocksdb.log_readahead_size0The number of bytes to prefetch when reading the log. 0 means the prefetching is disabled.
rocksdb.compaction_readahead_size0The number of bytes to perform bigger reads when doing compaction. If running RocksDB on spinning disks, you should set this to at least 2MB. 0 means the prefetching is disabled.
rocksdb.row_cache_capacity0The capacity in bytes of global cache for table-level rows. 0 means the row_cache is disabled.
rocksdb.delete_obsolete_files_period21600The periodicity in seconds when obsolete files get deleted, 0 means always do full purge.
rocksdb.write_buffer_size134217728Amount of data in bytes to build up in memory.
rocksdb.max_write_buffer_number6The maximum number of write buffers that are built up in memory.
rocksdb.min_write_buffer_number_to_merge2The minimum number of write buffers that will be merged together.
rocksdb.max_write_buffer_number_to_maintain0The total maximum number of write buffers to maintain in memory for conflict checking when transactions are used.
rocksdb.memtable_bloom_size_ratio0.0If prefix-extractor is set and memtable_bloom_size_ratio is not 0, or if memtable_whole_key_filtering is set true, create bloom filter for memtable with the size of write_buffer_size * memtable_bloom_size_ratio. If it is larger than 0.25, it is santinized to 0.25.
rocksdb.memtable_whole_key_filteringfalseEnable whole key bloom filter in memtable, it can potentially reduce CPU usage for point-look-ups. Note this will only take effect if memtable_bloom_size_ratio > 0.
rocksdb.memtable_huge_page_size0The page size for huge page TLB for bloom in memtable. If <= 0, not allocate from huge page TLB but from malloc.
rocksdb.inplace_update_supportfalseAllows thread-safe inplace updates if a put key exists in current memtable and sizeof new value is smaller.
rocksdb.level_compaction_dynamic_level_bytesfalseWhether to enable level_compaction_dynamic_level_bytes, if it’s enabled we give max_bytes_for_level_multiplier a priority against max_bytes_for_level_base, the bytes of base level is dynamic for a more predictable LSM tree, it is useful to limit worse case space amplification. Turning this feature on/off for an existing DB can cause unexpected LSM tree structure so it’s not recommended.
rocksdb.max_bytes_for_level_base536870912The upper-bound of the total size of level-1 files in bytes.
rocksdb.max_bytes_for_level_multiplier10.0The ratio between the total size of level (L+1) files and the total size of level L files for all L.
rocksdb.target_file_size_base67108864The target file size for compaction in bytes.
rocksdb.target_file_size_multiplier1The size ratio between a level L file and a level (L+1) file.
rocksdb.level0_file_num_compaction_trigger2Number of files to trigger level-0 compaction.
rocksdb.level0_slowdown_writes_trigger20Soft limit on number of level-0 files for slowing down writes.
rocksdb.level0_stop_writes_trigger36Hard limit on number of level-0 files for stopping writes.
rocksdb.soft_pending_compaction_bytes_limit68719476736The soft limit to impose on pending compaction in bytes.
rocksdb.hard_pending_compaction_bytes_limit274877906944The hard limit to impose on pending compaction in bytes.
rocksdb.allow_mmap_writesfalseAllow the OS to mmap file for writing.
rocksdb.allow_mmap_readsfalseAllow the OS to mmap file for reading sst tables.
rocksdb.use_direct_readsfalseEnable the OS to use direct I/O for reading sst tables.
rocksdb.use_direct_io_for_flush_and_compactionfalseEnable the OS to use direct read/writes in flush and compaction.
rocksdb.use_fsyncfalseIf true, then every store to stable storage will issue a fsync.
rocksdb.atomic_flushfalseIf true, flushing multiple column families and committing their results atomically to MANIFEST. Note that it’s not necessary to set atomic_flush=true if WAL is always enabled.
rocksdb.format_version5The format version of BlockBasedTable, allowed values are 0~5.
rocksdb.index_typekBinarySearchThe index type used to lookup between data blocks with the sst table, allowed values are [kBinarySearch,kHashSearch,kTwoLevelIndexSearch,kBinarySearchWithFirstKey].
rocksdb.data_block_index_typekDataBlockBinarySearchThe search type used to point lookup in data block with the sst table, allowed values are [kDataBlockBinarySearch,kDataBlockBinaryAndHash].
rocksdb.data_block_hash_table_util_ratio0.75The hash table utilization ratio value of entries/buckets. It is valid only when data_block_index_type=kDataBlockBinaryAndHash.
rocksdb.block_size4096Approximate size of user data packed per block, Note that it corresponds to uncompressed data.
rocksdb.block_size_deviation10The percentage of free space used to close a block.
rocksdb.block_restart_interval16The block restart interval for delta encoding in blocks.
rocksdb.block_cache_capacity8388608The amount of block cache in bytes that will be used by RocksDB, 0 means no block cache.
rocksdb.cache_index_and_filter_blockstrueSet this option true if we’d put index/filter blocks to the block cache.
rocksdb.pin_l0_filter_and_index_blocks_in_cachetrueSet this option true if we’d pin L0 index/filter blocks to the block cache.
rocksdb.bloom_filter_bits_per_key-1The bits per key in bloom filter, a good value is 10, which yields a filter with ~ 1% false positive rate. Set bloom_filter_bits_per_key > 0 to enable bloom filter, -1 means no bloom filter (0~0.5 round down to no filter).
rocksdb.bloom_filter_block_based_modefalseIf bloom filter is enabled, set this option true to use block based filter rather than full filter.
rocksdb.bloom_filter_whole_key_filteringtrueIf bloom filter is enabled, set this option true to place whole keys in the bloom filter, else place the prefix of keys when prefix-extractor is set.
rocksdb.optimize_filters_for_hitstrueIf bloom filter is enabled, this flag allows us to not store filters for the last level. set this option true to optimize the filters mainly for cases where keys are found rather than also optimize for keys missed.
rocksdb.partition_filters_and_indexesfalseIf bloom filter is enabled, set this option true to use partitioned full filters and indexes for each sst file. This option is incompatible with block-based filters.
rocksdb.pin_top_level_index_and_filtertrueIf partition_filters_and_indexes is set true, set this option true if we’d pin top-level index of partitioned filter and index blocks to the block cache.
rocksdb.prefix_extractor_n_bytes0The prefix-extractor uses the first N bytes of a key as its prefix, it will use the full key when a key is shorter than the N. 0 means unset prefix-extractor.
K8s Config Options (Optional)

Corresponding configuration file rest-server.properties

config optiondefault valuedescription
server.use_k8sfalseWhether to use k8s to support multiple tenancy.
server.deploy_in_k8sfalseWhether to deploy server in k8s.
server.urls_to_pdhttp://0.0.0.0:8080Used as the server address reserved for PD and provided to clients, only used when starting the server in k8s.
server.k8s_urlhttps://127.0.0.1:8888The url of k8s.
server.k8s_use_cafalseWhether to use ca to k8s api server.
server.k8s_caThe ca file of k8s api server.
server.k8s_client_caThe client ca file of k8s api server.
server.k8s_client_keyThe client key file of k8s api server.
k8s.apifalseThe k8s api start status when the computer service is enabled.
k8s.namespacehugegraph-computer-systemThe namespace used for k8s work when the computer service is enabled.
k8s.kubeconfigThe k8s kube config file when the computer service is enabled.
k8s.hugegraph_urlThe hugegraph url for k8s work when the computer service is enabled.
k8s.enable_internal_algorithmtrueWhether to open k8s internal algorithm.
service.access_pd_namehgService name for server to access pd service.
service.access_pd_tokenService token for server to access pd service.
server.k8s_oltp_image127.0.0.1/kgs_bd/hugegraphserver:3.0.0The oltp server image of k8s.
server.k8s_olap_imagehugegraph/hugegraph-server:v1The olap server image of k8s.
server.k8s_storage_imagehugegraph/hugegraph-server:v1The storage server image of k8s.
server.default_oltp_k8s_namespacehugegraph-serverThe default oltp namespace for HugeGraph default graph space.
server.default_olap_k8s_namespacehugegraph-computer-systemThe default olap namespace for HugeGraph default graph space.
k8s.internal_algorithm[page-rank, degree-centrality, wcc, triangle-count, rings, rings-with-filter, betweenness-centrality, closeness-centrality, lpa, links, kcore, louvain, clustering-coefficient, ppr, subgraph-match]The names of the built-in k8s algorithms.
k8s.algorithmsSee ServerOptions.K8S_ALGORITHMSThe name:paramsClass mapping of the built-in k8s algorithms.
k8s.internal_algorithm_image_urlEmptyImage URL for built-in Kubernetes algorithms.
Arthas Diagnostic Config Options (Optional)

Corresponding configuration file rest-server.properties

config optiondefault valuedescription
arthas.telnetPort8562Arthas telnet port.
arthas.httpPort8561Arthas HTTP port.
arthas.ip0.0.0.0Arthas bind IP.
arthas.disabledCommandsjadDisabled Arthas commands, separated by commas.
RPC Server Config Options

Corresponding configuration file rest-server.properties

config optiondefault valuedescription
rpc.server_hostThe hosts/ips bound by rpc server to provide services, empty value means not enabled.
rpc.server_port8090The port bound by rpc server to provide services.
rpc.server_adaptive_portfalseWhether the bound port is adaptive, if it’s enabled, when the port is in use, automatically +1 to detect the next available port. Note that this process is not atomic, so there may still be port conflicts.
rpc.server_timeout30The timeout(in seconds) of rpc server execution.
rpc.remote_urlThe remote urls of rpc peers, it can be set to multiple addresses, which are concat by ‘,’, empty value means not enabled.
rpc.client_connect_timeout20The timeout(in seconds) of rpc client connect to rpc server.
rpc.client_reconnect_period10The period(in seconds) of rpc client reconnect to rpc server.
rpc.client_read_timeout40The timeout(in seconds) of rpc client read from rpc server.
rpc.client_retries3Failed retry number of rpc client calls to rpc server.
rpc.client_load_balancerconsistentHashThe rpc client uses a load-balancing algorithm to access multiple rpc servers in one cluster. Default value is ‘consistentHash’, means forwarding by request parameters.
rpc.protocolboltRpc communication protocol, client and server need to be specified the same value.
rpc.serializationhessian2Rpc serialization type, client and server must set the same value. Note: If you choose ‘protobuf’, you need to add the relative IDL file. (Could refer PD/Store *.proto)
rpc.config_order999Sofa-RPC configuration file loading order, the larger the more later loading.
rpc.logger_implcom.alipay.sofa.rpc.log.SLF4JLoggerImplSofa-RPC log implementation class.
HBase Backend Config Options
config optiondefault valuedescription
backendMust be set to hbase.
serializerMust be set to hbase.
hbase.hostslocalhostThe hostnames or ip addresses of HBase zookeeper, separated with commas.
hbase.port2181The port address of HBase zookeeper.
hbase.threads_max64The max threads num of hbase connections.
hbase.znode_parent/hbaseThe znode parent path of HBase zookeeper.
hbase.zk_retry3The recovery retry times of HBase zookeeper.
hbase.truncate_timeout30The timeout in seconds of waiting for store truncate.
hbase.aggregation_timeout43200The timeout in seconds of waiting for aggregation.
hbase.kerberos_enablefalseIs Kerberos authentication enabled for HBase.
hbase.kerberos_keytabThe HBase’s key tab file for kerberos authentication.
hbase.kerberos_principalThe HBase’s principal for kerberos authentication.
hbase.krb5_conf/etc/krb5.confKerberos configuration file, including KDC IP, default realm, etc.
hbase.hbase_site/etc/hbase/conf/hbase-site.xmlThe HBase’s configuration file
hbase.enable_partitiontrueIs pre-split partitions enabled for HBase.
hbase.vertex_partitions10The number of partitions of the HBase vertex table.
hbase.edge_partitions30The number of partitions of the HBase edge table.

≤ 1.5 Version Config (Legacy)

The following backend stores are no longer supported in version 1.7.0+ and are only available in version 1.5.x and earlier:

Cassandra Backend Config Options
config optiondefault valuedescription
backendMust be set to cassandra.
serializerMust be set to cassandra.
cassandra.hostlocalhostThe seeds hostname or ip address of cassandra cluster.
cassandra.port9042The seeds port address of cassandra cluster.
cassandra.connect_timeout5The cassandra driver connect server timeout(seconds).
cassandra.read_timeout20The cassandra driver read from server timeout(seconds).
cassandra.keyspace.strategySimpleStrategyThe replication strategy of keyspace, valid value is SimpleStrategy or NetworkTopologyStrategy.
cassandra.keyspace.replication[3]The keyspace replication factor of SimpleStrategy, like ‘[3]’.Or replicas in each datacenter of NetworkTopologyStrategy, like ‘[dc1:2,dc2:1]’.
cassandra.usernameThe username to use to login to cassandra cluster.
cassandra.passwordThe password corresponding to cassandra.username.
cassandra.compression_typenoneThe compression algorithm of cassandra transport: none/snappy/lz4.
cassandra.jmx_port=71997199The port of JMX API service for cassandra.
cassandra.aggregation_timeout43200The timeout in seconds of waiting for aggregation.
ScyllaDB Backend Config Options
config optiondefault valuedescription
backendMust be set to scylladb.
serializerMust be set to scylladb.

Other options are consistent with the Cassandra backend.

MySQL & PostgreSQL Backend Config Options
config optiondefault valuedescription
backendMust be set to mysql.
serializerMust be set to mysql.
jdbc.drivercom.mysql.jdbc.DriverThe JDBC driver class to connect database.
jdbc.urljdbc:mysql://127.0.0.1:3306The url of database in JDBC format.
jdbc.usernamerootThe username to login database.
jdbc.password******The password corresponding to jdbc.username.
jdbc.ssl_modefalseThe SSL mode of connections with database.
jdbc.reconnect_interval3The interval(seconds) between reconnections when the database connection fails.
jdbc.reconnect_max_times3The reconnect times when the database connection fails.
jdbc.storage_engineInnoDBThe storage engine of backend store database, like InnoDB/MyISAM/RocksDB for MySQL.
jdbc.postgresql.connect_databasetemplate1The database used to connect when init store, drop store or check store exist.
PostgreSQL Backend Config Options
config optiondefault valuedescription
backendMust be set to postgresql.
serializerMust be set to postgresql.

Other options are consistent with the MySQL backend.

The driver and url of the PostgreSQL backend should be set to:

  • jdbc.driver=org.postgresql.Driver
  • jdbc.url=jdbc:postgresql://localhost:5432/

4.3 - Built-in User Authentication and Authorization Configuration and Usage in HugeGraph

Overview

To facilitate authentication usage in different user scenarios, HugeGraph currently provides built-in authorization StandardAuthenticator mode, which supports multi-user authentication and fine-grained access control. It adopts a 4-layer design based on “User-UserGroup-Operation-Resource” to flexibly control user roles and permissions (supports multiple GraphServers).

Some key designs of the StandardAuthenticator mode include:

  • During initialization, a super administrator (admin) user is created. Subsequently, other users can be created by the super administrator. Once newly created users are assigned sufficient permissions, they can create or manage more users.
  • It supports dynamic creation of users, user groups, and resources, as well as dynamic allocation or revocation of permissions.
  • Users can belong to one or multiple user groups. Each user group can have permissions to operate on any number of resources. The types of operations include read, write, delete, execute, and others.
  • “Resource” describes the data in the graph database, such as vertices that meet certain criteria. Each resource consists of three elements: type, label, and properties. There are 18 types in total, with the ability to combine any label and properties. The internal condition of a resource is an AND relationship, while the condition between multiple resources is an OR relationship.

Here is an example to illustrate:

// Scenario: A user only has data read permission for the Beijing area
user(name=xx) -belong-> group(name=xx) -access(read)-> target(graph=graph1, resource={label: person, city: Beijing})

Configure User Authentication

By default, HugeGraph does not enable user authentication, and it needs to be enabled by modifying the configuration file.

Warning

Production requires authentication

HugeGraph disables user authentication by default. In production, enable authentication and authorization, set a strong non-default administrator password, maintain the Server IP allowlist, and grant minimum permissions. Do not expose Gremlin, Cypher, or other query endpoints directly to the public network. Standard Server configuration writes authentication-proxy audit records to audit-*.log; retain these files and restrict read access. auth.audit_log_rate limits per-user output rather than serving as a dedicated audit-log on/off switch.

You need to modify the configuration file to enable this feature. HugeGraph provides built-in authentication mode: StandardAuthenticator. This mode supports multi-user authentication and fine-grained permission control. Additionally, developers can implement their own HugeAuthenticator interface to integrate with their existing authentication systems.

HugeGraph uses HTTP Basic Authentication. The value after Basic is the Base64 encoding of username:password. With curl, pass the credentials directly through -u:

curl -u 'admin:<password>' \
  http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/vertexlabels
Warning

Protect Basic Authentication credentials

Basic Authentication only Base64-encodes username:password; it does not encrypt credentials. Use HTTPS for network transmission. See HTTPS configuration.

Warning

JWT risks in older versions

HugeGraph-Server versions before 1.5.0 have JWT-related security risks in Auth mode. Upgrade affected versions or change the JWT secretKey according to that version’s requirements.

auth.token_secret is read from the authentication graph’s configuration. The default graph is hugegraph, so its file is normally conf/graphs/hugegraph.properties; if auth.graph_store changes, use that graph’s properties file. The source-code default generates 32 random bytes encoded as Base64, but does not write the value back to the file. Independently generated defaults cannot remain consistent across restarts or Server nodes. To preserve existing tokens across restarts or validate the same token on multiple nodes, explicitly configure the same strong random secret in each authentication graph configuration.

Server encodes this string as UTF-8 for JJWT’s HS256 signer, which requires at least 32 bytes. This is a byte-length requirement; an arbitrary 32-character string need not have sufficient randomness. The command below generates 32 random bytes and Base64-encodes them locally.

# Set the same value in every Server node's authentication graph configuration
auth.token_secret=<locally-generated-secret>

Generate the secret locally and transfer it securely into the authentication graph configuration. Do not commit the actual secret or expose it in public logs:

openssl rand -base64 32

Since the default is generated at every startup, explicitly configure it when tokens must survive restarts or be accepted by multiple nodes. Token lifetime is controlled by auth.token_expire, defaulting to 86400 seconds.

Both auth.token_expire and auth.token_secret belong to the authentication graph specified by auth.graph_store. Entries with the same names in rest-server.properties do not override that graph configuration.

StandardAuthenticator Mode

The StandardAuthenticator mode supports user authentication and permission control by storing user information in the database backend. This implementation authenticates users based on their names and passwords (encrypted) stored in the database and controls user permissions based on their roles. Below is the specific configuration process (requires service restart):

Configure the authenticator and its rest-server file path in the gremlin-server.yaml configuration file:

authentication: {
  authenticator: org.apache.hugegraph.auth.StandardAuthenticator,
  authenticationHandler: org.apache.hugegraph.auth.WsAndHttpBasicAuthHandler,
  config: {tokens: conf/rest-server.properties}
}

Configure the authenticator and the graph that stores authorization data in rest-server.properties:

auth.authenticator=org.apache.hugegraph.auth.StandardAuthenticator
auth.graph_store=hugegraph
# Used when Server first creates admin; replace public default pa with a strong password in production
# For local persistent backends, init-store.sh prompts for a password when StandardAuthenticator is first initialized
#auth.admin_pa=<your-admin-password>

# Auth Client Config
# If GraphServer and AuthServer are deployed separately, you also need to specify the following configuration. Fill in the IP:RPC port of AuthServer.
# auth.remote_url=127.0.0.1:8899,127.0.0.1:8898,127.0.0.1:8897

In the above configuration, the graph_store option specifies which graph to use for storing user information. If there are multiple graphs, you can choose any of them.

In the hugegraph{n}.properties configuration file, configure the gremlin.graph information:

gremlin.graph=org.apache.hugegraph.auth.HugeFactoryAuthProxy

For authorization API usage, see the Authentication API documentation.

Custom User Authentication System

If you need to support a more flexible user system, you can customize the authenticator for extension. Simply implement the org.apache.hugegraph.auth.HugeAuthenticator interface with your custom authenticator, and then modify the authenticator configuration item in the configuration file to point to your implementation.

Switching authentication mode

When init-store.sh is run for the first time and the admin user does not yet exist, the command prompts for the administrator password. For an initialized persistent backend, init-store.sh adds the system metadata required for authentication without deleting existing graph data.

# stop the hugeGraph firstly
bin/stop-hugegraph.sh

# Initialize authentication system metadata; existing backend data is preserved
bin/init-store.sh

# start hugeGraph again
bin/start-hugegraph.sh

Use docker to enable authentication mode

For versions of the hugegraph/hugegraph image equal to or greater than 1.2.0, you can enable authentication mode while starting the Docker image.

The steps are as follows:

1. Use docker run

To enable authentication mode, add the environment variable PASSWORD=xxx (you can freely set the password) in the docker run command:

docker run -itd -e PASSWORD=xxx --name=server -p 8080:8080 hugegraph/hugegraph:1.7.0

2. Use docker-compose

Use docker-compose and set the environment variable PASSWORD=xxx:

version: '3'
services:
  server:
    image: hugegraph/hugegraph:1.7.0
    container_name: server
    ports:
      - 8080:8080
    environment:
      - PASSWORD=xxx

3. Enter the container to enable authentication mode

Enter the container first:

docker exec -it server bash
# Modify the config quickly, the modified file are save in the conf-bak folder
bin/enable-auth.sh

Then follow Switching authentication mode

4.4 - Configuring HugeGraphServer to Use HTTPS Protocol

Overview

HugeGraphServer uses HTTP by default and can be configured to use HTTPS.

Warning

HTTPS does not replace Server authentication or access control

HTTPS protects transport but does not restrict caller permissions. In production, use HTTPS and enable Server authentication and authorization, an IP allowlist, and minimum permissions. Retain and protect Server audit-*.log files. If a reverse proxy terminates TLS, also restrict network access to the Server backend port.

Warning

Use a dedicated Server certificate in production

If conf/hugegraph-server.keystore is missing at startup, the script downloads a shared demonstration keystore with the public password hugegraph. Use a dedicated certificate and keystore password in production, and restrict read access to private-key files.

Server Configuration

Modify the conf/rest-server.properties configuration file and change the schema part of restserver.url to https.

# Set the protocol to HTTPS
restserver.url=https://127.0.0.1:8080
# Server keystore file path. This default value is automatically effective when using HTTPS, and you can modify it as needed.
ssl.keystore_file=conf/hugegraph-server.keystore
# Server keystore file password. This default value is automatically effective when using HTTPS, and you can modify it as needed.
ssl.keystore_password=******

The keystore file is not shipped inside the distribution, because it carries no license declaration. When restserver.url starts with https and conf/hugegraph-server.keystore is missing, bin/start-hugegraph.sh downloads it from the binary-1.5 branch of the hugegraph-doc repository before starting the server. The password of that file is hugegraph. Both values are the defaults of ssl.keystore_file and ssl.keystore_password; users can generate their own keystore file and password and then change the two options.

Client Configuration

Using HTTPS in HugeGraph-Client

When constructing a HugeClient, pass the HTTPS-related configurations. Here’s an example in Java:

String url = "https://localhost:8080";
String graphName = "hugegraph";
HugeClientBuilder builder = HugeClient.builder(url, graphName);
// Client keystore file path
String trustStoreFilePath = "hugegraph.truststore";
// Client keystore password
String trustStorePassword = "******";
builder.configSSL(trustStoreFilePath, trustStorePassword);
HugeClient hugeClient = builder.build();

Note: Before version 1.9.0, HugeGraph-Client was created directly using the new keyword and did not support the HTTPS protocol. Starting from version 1.9.0, it changed to use the builder pattern and supports configuring the HTTPS protocol.

Using HTTPS in HugeGraph-Loader

When starting an import task, add the following options in the command line:

# HTTPS
--protocol https
# Client certificate file path. When specifying --protocol as https, the default value conf/hugegraph.truststore is automatically used, and you can modify it as needed.
--trust-store-file {file}
# Client certificate file password. When specifying --protocol as https, the default value hugegraph is automatically used, and you can modify it as needed.
--trust-store-password {password}

Under the conf directory of hugegraph-loader, there is already a default client certificate file named hugegraph.truststore, and its password is hugegraph.

Using HTTPS in HugeGraph-Tools

When executing commands, add the following options in the command line:

# Client certificate file path. When using the HTTPS protocol in the URL, the default value conf/hugegraph.truststore is automatically used, and you can modify it as needed.
--trust-store-file {file}
# Client certificate file password. When using the HTTPS protocol in the URL, the default value hugegraph is automatically used, and you can modify it as needed.
--trust-store-password {password}
# When executing migration commands and using the --target-url with the HTTPS protocol, the default value conf/hugegraph.truststore is automatically used, and you can modify it as needed.
--target-trust-store-file {target-file}
# When executing migration commands and using the --target-url with the HTTPS protocol, the default value hugegraph is automatically used, and you can modify it as needed.
--target-trust-store-password {target-password}

Under the conf directory of hugegraph-tools, there is already a default client certificate file named hugegraph.truststore, and its password is hugegraph.

How to Generate Certificate Files

This section provides an example of generating certificates. If the default certificate is sufficient or if you already know how to generate certificates, you can skip this section.

Server

  1. Generate the server’s private key and import it into the server’s keystore file. The server.keystore is for the server’s use and contains its private key.
keytool -genkey -alias serverkey -keyalg RSA -keystore server.keystore

During the process, fill in the description information according to your requirements. The description information for the default certificate is as follows:

First and Last Name: hugegraph
Organizational Unit Name: hugegraph
Organization Name: hugegraph
City or Locality Name: BJ
State or Province Name: BJ
Country Code: CN
  1. Export the server certificate based on the server’s private key.
keytool -export -alias serverkey -keystore server.keystore -file server.crt

server.crt is the server’s certificate.

Client

keytool -import -alias serverkey -file server.crt -keystore client.truststore

client.truststore is for the client’s use and contains the trusted certificate.

4.5 - Configuring the RocksDB Backend

Overview

RocksDB is an embedded LSM-tree key-value store. With the rocksdb backend, HugeGraph-Server keeps all graph data in RocksDB instances that live inside the server process, so there is no separate storage service to deploy. This is the backend used by the shipped conf/graphs/hugegraph.properties.

Since version 1.7.0 the server accepts only memory, rocksdb, hbase and hstore as the backend. The rocksdb backend stores data on the local disks of one server: it does not support shared storage, so a graph cannot be served by several servers over the same data directory. For a distributed deployment use the hstore backend with PD and Store.

The RocksDB JNI library is pinned to version 8.10.2 by hugegraph-rocksdb/pom.xml, so the on-disk format and the option semantics are those of RocksDB 8.10.

The backend driver version reported by this store is 1.11, and it is written into the meta table of the system store when the graph is initialized.

Selecting the backend

Set the backend and the serializer in the graph properties file (conf/graphs/<graph>.properties):

gremlin.graph=org.apache.hugegraph.HugeFactory

backend=rocksdb
serializer=binary

store=hugegraph

# rocksdb backend config
#rocksdb.data_path=/path/to/disk
#rocksdb.wal_path=/path/to/disk
  • backend=rocksdb selects the RocksDB store provider.
  • serializer=binary is the serializer the shipped template uses for this backend. The built-in serializers are binary, binaryscatter and text.
  • store is the database namespace of the graph, and it is also part of the graph name that the provider passes down to the store.

Run bin/init-store.sh once before the first start to create the stores, then start the server. Both bin/init-store.sh and bin/hugegraph-server.sh load the RocksDB library, so the data directories are created on the machine that runs them.

The distribution registers the option space and the store provider for each backend listed in the packaged backend.properties, whose value comes from the hugegraph.backends build property. A default build registers rocksdb, hbase, hstore; building with -Drocksdb-only activates the rocksdb-only profile and produces a distribution that registers only rocksdb. A backend that is not registered fails at startup with Not exists BackendStoreProvider.

The provider registration also adds a second name, rocksdbsst, for the store that writes SST files instead of a live database. That name is not in the list of allowed backends, so backend=rocksdbsst is rejected with backend is illegal: rocksdbsst. To load SST files into a normal rocksdb graph, use rocksdb.sst_path as described below.

Data directory layout

Two directories matter: rocksdb.data_path (default rocksdb-data/data) and rocksdb.wal_path (default rocksdb-data/wal). Relative paths resolve against the working directory of the server, which is the installation directory.

Each graph opens three stores: m for schema, g for graph data, and s for the system store. The store name is appended to both configured paths, so a default single-graph installation looks like this:

rocksdb-data/
  data/
    m/    # schema store: property keys, vertex/edge/index labels, counters
    g/    # graph store: vertices, edges, index tables, olap tables
    s/    # system store: tasks, server info, backend meta (driver version)
  wal/
    m/
    g/
    s/

Every backend table becomes a RocksDB column family inside the store it belongs to, named <database>+<table>, where the database is derived from the graph name. Column families of existing data directories are always reopened, so tables created by an older version stay readable.

Other points to keep in mind:

  • Two graphs must not share a data path. When a graph is created by cloning an existing configuration through the API, the provider appends _<newGraph> to both rocksdb.data_path and rocksdb.wal_path. Deleting such a graph deletes both directories.
  • Snapshots are created beside the data directory: the last two segments of the data path are rewritten with a prefix, so with the default paths the snapshot of the graph store goes to rocksdb-data/<prefix>_data/g. Resuming a snapshot closes the instance, deletes the data directory and moves the snapshot into its place.
  • With rocksdb.data_disks set, the tables named there are opened as separate RocksDB instances under the given paths instead of under rocksdb.data_path. The server opens up to 8 instances in parallel, waits at most 600 seconds for the open to finish and 30 seconds for sessions to close.

Path and log options

config optiondefault valuedescription
rocksdb.data_pathrocksdb-data/dataThe path for storing data of RocksDB. Must not be empty.
rocksdb.data_disks[]The optimized disks for storing data of RocksDB. The format of each element: STORE/TABLE: /path/disk. Allowed keys are [g/vertex, g/edge_out, g/edge_in, g/vertex_label_index, g/edge_label_index, g/range_int_index, g/range_float_index, g/range_long_index, g/range_double_index, g/secondary_index, g/search_index, g/shard_index, g/unique_index, g/olap]. A disk path must differ from rocksdb.data_path.
rocksdb.wal_pathrocksdb-data/walThe path for storing WAL of RocksDB. Must not be empty.
rocksdb.sst_path(empty)The path for ingesting SST file into RocksDB. Empty disables ingestion.
rocksdb.log_levelINFOThe info log level of RocksDB. Allowed values: DEBUG, INFO, WARN, ERROR, FATAL, HEADER.

Compaction and compression options

config optiondefault valuedescription
rocksdb.num_levels7Set the number of levels for this database. Range: 1 to 2^31-1.
rocksdb.compaction_styleLEVELSet compaction style for RocksDB: LEVEL/UNIVERSAL/FIFO.
rocksdb.optimize_modetrueOptimize for heavy workloads and big datasets. See “How the options are applied” below.
rocksdb.bulkload_modefalseSwitch to the mode to bulk load data into RocksDB.
rocksdb.compression_per_level[none, none, snappy, snappy, snappy, snappy, snappy]The compression algorithms for different levels of RocksDB, allowed values are none/snappy/z/bzip2/lz4/lz4hc/xpress/zstd. The list must be empty or hold exactly rocksdb.num_levels elements.
rocksdb.bottommost_compressionnoneThe compression algorithm for the bottommost level of RocksDB, allowed values are none/snappy/z/bzip2/lz4/lz4hc/xpress/zstd.
rocksdb.compressionsnappyThe compression algorithm for compressing blocks of RocksDB, allowed values are none/snappy/z/bzip2/lz4/lz4hc/xpress/zstd.

Database level options

config optiondefault valuedescription
rocksdb.max_background_jobs8Maximum number of concurrent background jobs, including flushes and compactions. Range: 1 to 2^31-1.
rocksdb.max_subcompactions4The value represents the maximum number of threads per compaction job. Range: 1 to 2^31-1.
rocksdb.delayed_write_rate16777216 (16 MB/s)The rate limit in bytes/s of user write requests when need to slow down if the compaction gets behind.
rocksdb.max_open_files-1The maximum number of open files that can be cached by RocksDB, -1 means no limit.
rocksdb.max_manifest_file_size104857600 (100 MB)The max size of manifest file in bytes.
rocksdb.skip_stats_update_on_db_openfalseWhether to skip statistics update when opening the database, setting this flag true allows us to not update statistics.
rocksdb.skip_check_sst_size_on_db_openfalseWhether to skip checking sizes of all sst files when opening the database.
rocksdb.max_file_opening_threads16The max number of threads used to open files. Range: 1 to 2^31-1.
rocksdb.max_total_wal_size0Total size of WAL files in bytes. Once WALs exceed this size, we will start forcing the flush of column families related, 0 means no limit.
rocksdb.bytes_per_sync0Allows OS to incrementally sync SST files to disk while they are being written, asynchronously in the background. Issue one request for every bytes_per_sync written. 0 turns it off.
rocksdb.wal_bytes_per_sync0Same as above for WAL files. 0 turns it off.
rocksdb.strict_bytes_per_syncfalseWhen true, guarantees SST/WAL files have at most bytes_per_sync/wal_bytes_per_sync bytes submitted for writeback at any given time. This can be used to handle cases where processing speed exceeds I/O speed.
rocksdb.db_write_buffer_size0Total size of write buffers in bytes across all column families, 0 means no limit.
rocksdb.log_readahead_size0The number of bytes to prefetch when reading the log. 0 means the prefetching is disabled.
rocksdb.compaction_readahead_size0The number of bytes to perform bigger reads when doing compaction. If running RocksDB on spinning disks, you should set this to at least 2MB. 0 means the prefetching is disabled.
rocksdb.row_cache_capacity0The capacity in bytes of global cache for table-level rows. 0 means the row_cache is disabled.
rocksdb.delete_obsolete_files_period21600 (6 hours)The periodicity in seconds when obsolete files get deleted, 0 means always do full purge. The value is converted to microseconds before it reaches RocksDB.

Memtable options

config optiondefault valuedescription
rocksdb.write_buffer_size134217728 (128 MB)Amount of data in bytes to build up in memory. Minimum 1 MB. This is per column family.
rocksdb.max_write_buffer_number6The maximum number of write buffers that are built up in memory. Range: 1 to 2^31-1.
rocksdb.min_write_buffer_number_to_merge2The minimum number of write buffers that will be merged together. Range: 1 to 2^31-1.
rocksdb.max_write_buffer_number_to_maintain0The total maximum number of write buffers to maintain in memory for conflict checking when transactions are used.
rocksdb.memtable_bloom_size_ratio0.0If prefix-extractor is set and memtable_bloom_size_ratio is not 0, or if memtable_whole_key_filtering is set true, create bloom filter for memtable with the size of write_buffer_size * memtable_bloom_size_ratio. A value larger than 0.25 is reduced to 0.25. Range: 0.0 to 1.0.
rocksdb.memtable_whole_key_filteringfalseEnable whole key bloom filter in memtable, it can potentially reduce CPU usage for point-look-ups. Note this will only take effect if memtable_bloom_size_ratio > 0.
rocksdb.memtable_huge_page_size0The page size for huge page TLB for bloom in memtable. If <= 0, not allocate from huge page TLB but from malloc.
rocksdb.inplace_update_supportfalseAllows thread-safe inplace updates if a put key exists in current memtable and sizeof new value is smaller.

Level sizing and write stall options

config optiondefault valuedescription
rocksdb.level_compaction_dynamic_level_bytesfalseWhether to enable level_compaction_dynamic_level_bytes, if it’s enabled we give max_bytes_for_level_multiplier a priority against max_bytes_for_level_base, the bytes of base level is dynamic for a more predictable LSM tree, it is useful to limit worse case space amplification. Turning this feature on/off for an existing DB can cause unexpected LSM tree structure so it’s not recommended.
rocksdb.max_bytes_for_level_base536870912 (512 MB)The upper-bound of the total size of level-1 files in bytes. Minimum 1 MB.
rocksdb.max_bytes_for_level_multiplier10.0The ratio between the total size of level (L+1) files and the total size of level L files for all L. Minimum 1.0.
rocksdb.target_file_size_base67108864 (64 MB)The target file size for compaction in bytes. Minimum 1 MB.
rocksdb.target_file_size_multiplier1The size ratio between a level L file and a level (L+1) file.
rocksdb.level0_file_num_compaction_trigger2Number of files to trigger level-0 compaction.
rocksdb.level0_slowdown_writes_trigger20Soft limit on number of level-0 files for slowing down writes.
rocksdb.level0_stop_writes_trigger36Hard limit on number of level-0 files for stopping writes.
rocksdb.soft_pending_compaction_bytes_limit68719476736 (64 GB)The soft limit to impose on pending compaction in bytes. Minimum 1 GB.
rocksdb.hard_pending_compaction_bytes_limit274877906944 (256 GB)The hard limit to impose on pending compaction in bytes. Minimum 1 GB.

File I/O options

config optiondefault valuedescription
rocksdb.allow_mmap_writesfalseAllow the OS to mmap file for writing.
rocksdb.allow_mmap_readsfalseAllow the OS to mmap file for reading sst tables.
rocksdb.use_direct_readsfalseEnable the OS to use direct I/O for reading sst tables.
rocksdb.use_direct_io_for_flush_and_compactionfalseEnable the OS to use direct read/writes in flush and compaction.
rocksdb.use_fsyncfalseIf true, then every store to stable storage will issue a fsync.
rocksdb.atomic_flushfalseIf true, flushing multiple column families and committing their results atomically to MANIFEST. Note that it’s not necessary to set atomic_flush=true if WAL is always enabled.

SST table format and block cache options

config optiondefault valuedescription
rocksdb.format_version5The format version of BlockBasedTable, allowed values are 0~5.
rocksdb.index_typekBinarySearchThe index type used to lookup between data blocks with the sst table, allowed values are [kBinarySearch, kHashSearch, kTwoLevelIndexSearch, kBinarySearchWithFirstKey].
rocksdb.data_block_index_typekDataBlockBinarySearchThe search type used to point lookup in data block with the sst table, allowed values are [kDataBlockBinarySearch, kDataBlockBinaryAndHash].
rocksdb.data_block_hash_table_util_ratio0.75The hash table utilization ratio value of entries/buckets. It is valid only when data_block_index_type=kDataBlockBinaryAndHash. Range: 0.0 to 1.0.
rocksdb.block_size4096 (4 KB)Approximate size of user data packed per block, Note that it corresponds to uncompressed data.
rocksdb.block_size_deviation10The percentage of free space used to close a block. Range: 0 to 100.
rocksdb.block_restart_interval16The block restart interval for delta encoding in blocks.
rocksdb.block_cache_capacity8388608 (8 MB)The amount of block cache in bytes that will be used by RocksDB, 0 means no block cache. A separate cache of this size is created for each column family.

Bloom filter options

The options in this group are read only when rocksdb.bloom_filter_bits_per_key is 0 or greater. With the default value of -1 there is no bloom filter and none of the other options in this table take effect, including the index and filter block caching ones.

config optiondefault valuedescription
rocksdb.bloom_filter_bits_per_key-1The bits per key in bloom filter, a good value is 10, which yields a filter with ~ 1% false positive rate. Set bloom_filter_bits_per_key > 0 to enable bloom filter, -1 means no bloom filter (0~0.5 round down to no filter).
rocksdb.bloom_filter_block_based_modefalseIf bloom filter is enabled, set this option true to use block based filter rather than full filter.
rocksdb.bloom_filter_whole_key_filteringtrueIf bloom filter is enabled, set this option true to place whole keys in the bloom filter, else place the prefix of keys when prefix-extractor is set.
rocksdb.cache_index_and_filter_blockstrueSet this option true if we’d put index/filter blocks to the block cache.
rocksdb.pin_l0_filter_and_index_blocks_in_cachetrueSet this option true if we’d pin L0 index/filter blocks to the block cache.
rocksdb.optimize_filters_for_hitstrueIf bloom filter is enabled, this flag allows us to not store filters for the last level. set this option true to optimize the filters mainly for cases where keys are found rather than also optimize for keys missed. This one is applied even when the filter is disabled.
rocksdb.partition_filters_and_indexesfalseIf bloom filter is enabled, set this option true to use partitioned full filters and indexes for each sst file. This option is incompatible with block-based filters. Enabling it also forces the index type to kTwoLevelIndexSearch and sets the metadata block size to rocksdb.block_size.
rocksdb.pin_top_level_index_and_filtertrueIf partition_filters_and_indexes is set true, set this option true if we’d pin top-level index of partitioned filter and index blocks to the block cache.
rocksdb.prefix_extractor_n_bytes0The prefix-extractor uses the first N bytes of a key as its prefix, it will use the full key when a key is shorter than the N. 0 means unset prefix-extractor.

How the options are applied

The server builds the RocksDB option objects once per store and per column family, so a change to any of the options above takes effect on the next server start.

  • rocksdb.optimize_mode=true applies presets before the values in the tables above: at the database level it raises parallelism to half of the available processors (at least one), allows concurrent memtable writes and enables the write thread adaptive yield; at the column family level it calls the RocksDB level-style and universal-style compaction presets. The explicit options are applied afterwards, so any value you set in the properties file wins over the preset.
  • rocksdb.bulkload_mode=true disables automatic compaction, raises the three level-0 triggers to the maximum integer and the two pending compaction limits to the maximum long value. Turn it off and restart after the load, otherwise compaction never runs.
  • rocksdb.block_cache_capacity=0 turns the block cache off completely rather than making it unbounded.
  • rocksdb.prefix_extractor_n_bytes greater than 0 installs a capped prefix extractor of that length.
  • Every column family uses the uint64add merge operator, which is what the counter table relies on.
  • The database is created if it is missing, and avoid_unnecessary_blocking_io and write_dbid_to_manifest are always on.

Memory notes

The caches and write buffers of RocksDB are native allocations, so they are not part of the JVM heap sizing in bin/hugegraph-server.sh. The GET /metrics/backend endpoint reports what the store uses: the memory number is the sum of the block cache usage, the pinned block cache usage, the estimated table reader memory (index and filter blocks) and the size of all memtables, taken from the RocksDB properties of every open column family.

Two option values multiply with the number of column families:

  • rocksdb.block_cache_capacity creates one cache instance per column family, so the total block cache of a server is roughly this value times the number of open tables across the m, g and s stores of every graph, plus the instances opened for rocksdb.data_disks.
  • rocksdb.write_buffer_size times rocksdb.max_write_buffer_number bounds the memtable memory of one column family. rocksdb.db_write_buffer_size caps the total across all column families of one store, and its default of 0 means there is no such cap.

rocksdb.row_cache_capacity is different: it is one cache per store, and 0 disables it.

Ingesting SST files

Setting rocksdb.sst_path turns on ingestion. When a store is opened, and again whenever tables are created, the server walks <sst_path>/<column family>/, collects every non-empty *.sst file below it and ingests those files into the matching column family. The files are moved rather than copied, so the source directory is consumed by the ingestion.

Raft mode

The RocksDB backend can still run behind the raft state machine: with raft.mode=true the store provider of any local backend is wrapped by the raft provider. The wrapper rejects backends with shared storage, so rocksdb is accepted while hbase is not. Under raft mode a RocksDB session writes with the WAL disabled and without sync, because the state machine can restore from a snapshot plus the raft log, and snapshots are supported by this backend.

Notes for anyone using it:

  • bin/init-store.sh forces raft.mode=false while it initializes the backend, so initialization never goes through raft.
  • The shipped conf/graphs/hugegraph.properties marks the raft options as deprecated. Distributed deployments of 1.7.0 and later use the hstore backend with PD and Store instead.
  • The raft peer endpoints are served under graphspaces/{graphspace}/graphs/{graph}/raft/, with list_peers, get_leader, set_leader, transfer_leader, add_peer and remove_peer. bin/raft-tools.sh wraps the same operations, but it still builds URLs without the graphspace segment, so the path has to be adjusted for a 1.7.0 server.
  • The remaining raft.* options are listed in the Server Complete Configuration Manual.

Backend capabilities

The feature flags of this backend affect what the server can push down to the store:

  • Scans by key prefix and by key range, paged queries, range conditions and order-by are supported.
  • There is no index inside RocksDB, so querying schema by name, querying by label and deleting edges by label are done by the server instead of the store.
  • Transactions are supported through RocksDB write batches.
  • Snapshots are supported, which is what raft mode and backup rely on.
  • Shared storage is not supported, so one data directory belongs to one server.
  • Olap properties are supported, and their tables are created as extra column families.
  • The store does not expire data by itself, so the server filters out elements whose TTL has passed when it reads them.
  • in, contains and contains_key conditions, aggregate properties and vertex or edge property updates in place are not supported at the store level.

Platform note for riscv64

On Linux riscv64 the RocksDB JNI library needs libatomic.so.1. bin/util.sh looks for it and adds it to LD_PRELOAD before bin/hugegraph-server.sh, bin/init-store.sh and bin/dump-store.sh start the JVM. If it is missing, those scripts stop with RISC-V RocksDB requires libatomic.so.1; install libatomic1, and installing the libatomic1 package fixes it.

4.6 - Configuring the HStore Distributed Backend

1 Overview

hstore is the distributed storage backend of HugeGraph. When a graph uses it, HugeGraph-Server keeps no graph data on its own disk. Two other processes do that work:

  • HugeGraph-PD (Placement Driver) owns the cluster metadata: the registered store list, the partition layout of every graph, the partition to store mapping, the graph schema and the schema id counters.
  • HugeGraph-Store owns the key value data itself, replicated across store nodes with Raft.

Server links a PD client and a Store client into its own process. For every read and write it asks PD which partition owns the key and which store node currently leads that partition, then sends the request directly to that store node.

The server side adapter is the hugegraph-hstore module. It registers under the backend name hstore and reports driver version 1.13.

Selecting hstore changes more than where the bytes are written. Server switches these behaviors on the backend type:

AreaWith hstoreWith a local backend
Schema storageSchema is read and written through the PD meta driverSchema lives in the m store
Schema idsAllocated by PD through the PD clientAllocated by the schema store
System storeNone, system data goes to the graph storeSeparate s store
Task schedulerdistributedlocal
Auth managerStandardAuthManagerV2StandardAuthManager
Backend version checkReads the graph storeReads the system store
init-store.shSkips the graph, PD and Store already own the metadataCreates the local store

2 Prerequisites

hstore is not self contained. A PD cluster and at least one Store node must be running before Server opens an hstore graph, and they have to be started in this order:

  1. PD, so that it can form its Raft group.
  2. Store, which registers itself with PD over gRPC. A store whose gRPC address is listed in PD’s own pd.initial-store-list goes to state Up right away. A store that is not in that list, and that PD has never seen Up or Offline before, registers as Pending and has to be activated before it serves data.
  3. Server, which then reads the store list back out of PD.

Default ports the Server side needs to know about:

ProcessgRPC portREST port
PD86868620
Store85008520

pd.peers on the Server side points at the PD gRPC port, not the REST port.

For installing and configuring the other two processes, see Install/Build HugeGraph-PD and Install/Build HugeGraph-Store.

3 Selecting the hstore backend

3.1 Graph configuration file

Set the backend in the graph properties file, for example conf/graphs/hugegraph.properties:

backend=hstore
serializer=binary
store=hugegraph
pd.peers=127.0.0.1:8686

Notes on those four keys:

  • backend=hstore selects the adapter. Since 1.7.0 the allowed values are memory, rocksdb, hbase and hstore. The value is compared case insensitively where the distribution checks it.
  • serializer=binary is required. Registering the hstore backend adds a config space and a store provider but no serializer of its own, and the adapter is written against the binary serializer. The built-in default of serializer is text, so this value has to be written out.
  • store=hugegraph is the namespace part of the name PD sees. Server opens the provider with <graphspace>/<store> and each backing store appends its own suffix, so PD ends up with one graph entry per store: DEFAULT/hugegraph/g for graph data and DEFAULT/hugegraph/m for the schema store slot. graphspace defaults to DEFAULT, while g and m are fixed.
  • pd.peers is the comma separated list of PD gRPC addresses. The adapter reads it from the graph config, not from rest-server.properties, and the graph level metadata connection uses the same value.

If the graph file does not contain pd.peers, Server copies the value from rest-server.properties into the graph config while loading the graph, provided that usePD is true or the backend is hstore. Writing the key explicitly in the graph file is still the clearer option.

3.2 rest-server.properties

# use pd
usePD=true
pd.peers=127.0.0.1:8686

usePD=true makes the Server load its metadata from PD at startup. On that path it connects the meta manager to PD, creates the built-in admin account and the default graph space, loads the graph spaces and services, creates the internal system graph (always with backend=hstore), and loads the graph configs that PD holds.

It is a separate switch from the graph level backend=hstore: a graph can use hstore with usePD left at its default of false, and Server then never opens the PD backed metadata path. The distribution’s own test startup script sets it whenever the backend is hstore.

3.3 The shipped template

The distribution ships a ready made graph file for this backend at conf/graphs/hstore.properties.template. It matches hugegraph.properties except that it sets backend=hstore, leaves pd.peers=127.0.0.1:8686 uncommented, and carries no memory management block.

The hstore Docker image applies that template for you: it deletes conf/graphs/hugegraph.properties and renames the template over it, so a container starts with the hstore backend already selected.

A locally built distribution has the hstore provider compiled in by default. The rocksdb-only Maven profile narrows the compiled backend list to rocksdb, and a distribution built that way rejects backend=hstore with Not exists BackendStoreProvider: hstore after the provider factory check.

4 hstore config options

These are the only keys in the hstore config space. They belong in the graph properties file.

config optiondefault valuedescription
hstore.partition_count0Number of partitions, which PD controls partitions based on.
hstore.shard_count0Number of copies, which PD controls partition copies based on.

4.1 hstore.partition_count

Server sends this number to PD once per graph store, the first time the store is opened, together with the graph name. A negative value is rejected at that point with The value of hstore.partition_count cannot be less than 0.

How PD reads the number:

  • 0, the default, means let PD decide. For a graph data store PD uses its own cluster wide partition total, which it derives from the number of entries in pd.initial-store-list, partition.store-max-shard-count and partition.default-shard-count. For the /m and /s stores it uses a fixed count of 1.
  • A value between 1 and that total is used as is.
  • A value above that total is clamped down to it.

The number is applied when the store is first registered with PD, so changing it later in the properties file does not repartition an existing graph.

4.2 hstore.shard_count

hstore.shard_count is declared in the hstore config space and is accepted in the properties file, but no code on the Server side reads it in this release: hstore.partition_count is the only one of the two the adapter reads. The replica count in effect is the one PD is configured with, partition.default-shard-count in PD’s application.yml.

5 Other options that only apply in hstore mode

These keys live in the shared rest-server.properties and graph properties files, but only take effect, or only change behavior, when PD and the hstore backend are in use. The source column gives the file and line in the checked Server master commit 2f827d6 where the option is declared.

config optionfiledefaultwhy it matters with hstoresource
pd.peersrest-server.properties127.0.0.1:8686PD addresses used for metadata, service discovery and the system graphServerOptions.java:195-201
pd.peers{graph}.properties127.0.0.1:8686PD addresses used by the backend adapter itselfCoreOptions.java:649-654
usePDrest-server.propertiesfalseWhether Server loads its metadata from PD at startupServerOptions.java:401-407
clusterrest-server.propertieshg-testCluster name used as the prefix of every PD metadata keyServerOptions.java:187-193
init_store.enabledrest-server.propertiestrueSet it to false in a PD/Store deployment, where the storage side already owns the metadataServerOptions.java:382-390
graph.load_from_local_configrest-server.propertiesfalseControls constructor preloading and rescanning on reload(); application initialization still scans and attempts to load local configurations, so false does not block local loadingServerOptions.java:366-372
auth.graph_storerest-server.propertieshugegraphThe graph that holds auth data, checked against the hstore backend when init-store is offServerOptions.java:602-609
graphspace{graph}.propertiesDEFAULTFirst segment of the graph name PD seesCoreOptions.java:679-685

init-store.sh never initializes an hstore graph. On the enabled path it scans conf/graphs and skips every graph whose backend is hstore. If you turn the whole step off with init_store.enabled=false, it validates instead that the admin account can still be created on the PD startup path: usePD has to be true, the auth graph has to exist locally with backend hstore, and auth.admin_pa has to be set to an explicit non-empty value. Otherwise startup fails rather than handing out the public default password.

6 How the Server finds the stores

The adapter builds its clients once per process, on the first hstore graph it opens:

  1. A PD client config from pd.peers, with the PD authority credentials and the client side partition cache enabled.
  2. The process wide PD client.
  3. The process wide store client, created from that PD client.

Creating the store client installs a PD backed partitioner as the node provider, partitioner and notifier of the store client’s node manager. That partitioner is the whole of the routing logic:

  • Point and prefix requests ask PD for the partition that owns the key, take the leader shard of that partition and send the request to that store id.
  • Code range scans walk the partitions by code until the range is covered, producing one target store per partition.
  • Whole graph scans ask PD for the active stores of the graph and fan out to every one of them.
  • Store address lookup resolves a store id to a host and port through PD.
  • Cache invalidation: when a store answers that a partition leader moved, the notifier updates the partition leader in PD’s client cache and invalidates the stale partition entry, so later requests follow the new leader.

Because the store list comes from PD rather than from configuration, a store node is added or removed by starting or stopping it against the same PD cluster. No Server side config change is needed.

7 Backend capabilities

hstore does not support every query form the local backends do. The differences visible to a user:

FeatureSupported
Scan by key prefixyes
Scan by key rangeyes
Query with range conditionyes
Query with order byyes
Query by pageyes
OLAP propertiesyes
Task and server vertexyes
Scan tokenno
Query schema by nameno
Query by labelno
Query with in conditionno
Query with containsno
Query with contains keyno
Sort results by input idsno
Delete edge by labelno
Update vertex propertyno
Update edge propertyno
Transactionno
Number typeno
Aggregate propertyno
TTLno

Sorting by input ids is off because multi node batch scans group the input keys by store and lose the global order. Vertex and edge property updates are off because the properties are stored in a single cell.

8 Verification

Once the Server is up, the backend metrics endpoint reports the number of stores that PD currently considers active:

curl http://localhost:8080/metrics/backend

The nodes value in the response is the count of active stores PD returns. A nodes value of 0 means the Server reached PD but PD has no store in state Up, which usually means the Store nodes have not registered yet, or registered as Pending because they are not in PD’s pd.initial-store-list.

4.7 - Configuring the HBase Backend

Overview

The HBase backend stores graph data in Apache HBase tables. HugeGraph acts as an HBase client only: it connects through the HBase ZooKeeper quorum, creates one HBase namespace per graph, and creates that graph’s schema, data and index tables inside it. Counting queries are answered by the HBase AggregateImplementation coprocessor, which HugeGraph attaches to every table it creates.

Note: the HBase backend is deprecated and is planned for removal in HugeGraph 2.0. New deployments should use hstore (distributed) or rocksdb (embedded, the default), and existing HBase deployments should plan a migration.

Since 1.7.0 the only backends shipped in the distribution are hstore, rocksdb, hbase and memory. The backend driver version reported by the HBase provider is 1.12.

Supported HBase Versions

The client jars are pinned to HBase 2.6.5 (hbase-endpoint plus hbase-shaded-client). HBase 2.x is required on the server side: when the detected HBase version is older than 2.0 the scan path rewrites an inclusive stop row into an exclusive one plus a trailing 0 byte, because inclusive stop rows do not work before that release. The CI job and the local Docker image both use HBase 2.6.5, so that is the version the backend is tested against.

Selecting the Backend

Edit conf/graphs/hugegraph.properties of the graph that should use HBase:

backend=hbase
serializer=hbase

# the namespace name is derived from this value
store=hugegraph

hbase.hosts=localhost
hbase.port=2181
hbase.znode_parent=/hbase

Note: serializer must be set to hbase, not to binary. The HBase serializer is a BinarySerializer subclass that drops the id prefix from row keys and writes the pre-split partition prefix that the pre-split vertex and edge tables expect. With serializer=binary neither of these applies.

Then initialize the store and start the server:

./bin/init-store.sh
./bin/start-hugegraph.sh

The default distribution is built with the backends rocksdb, hbase, hstore, so no extra jar is needed. A distribution built with the rocksdb-only Maven profile does not contain the HBase backend, and backend=hbase then fails to open with Not exists BackendStoreProvider: hbase.

All options below live in the graph properties file (conf/graphs/hugegraph.properties), not in rest-server.properties. They are registered only when the hbase backend is part of the distribution.

Connection Options

OptionDefaultDescription
hbase.hostslocalhostThe hostnames or ip addresses of HBase zookeeper, separated with commas. Must not be empty. Maps to hbase.zookeeper.quorum.
hbase.port2181The port address of HBase zookeeper, in the range 1 to 65535. Maps to hbase.zookeeper.property.clientPort.
hbase.znode_parent/hbaseThe znode parent path of HBase zookeeper. Must not be empty. Maps to zookeeper.znode.parent.
hbase.zk_retry3The recovery retry times of HBase zookeeper, in the range 0 to 1000. Maps to zookeeper.recovery.retry.
hbase.threads_max64The max threads num of hbase connections, in the range 1 to 1000. Maps to hbase.hconnection.threads.max, which HBase itself defaults to 256; the lower value is used to avoid running out of memory.

Timeout Options

OptionDefaultDescription
hbase.truncate_timeout30The timeout in seconds of waiting for store truncate. Must be positive. It applies per store, and a graph has three stores, so a truncate can take up to three times this value.
hbase.aggregation_timeout43200 (12 hours)The timeout in seconds of waiting for aggregation. Must be positive. Sets hbase.rpc.timeout on the aggregation client used by count queries.

Kerberos and HBase Site Options

OptionDefaultDescription
hbase.kerberos_enablefalseIs Kerberos authentication enabled for HBase.
hbase.krb5_conf/etc/krb5.confKerberos configuration file, including KDC IP, default realm, etc. Applied as the java.security.krb5.conf system property.
hbase.hbase_site/etc/hbase/conf/hbase-site.xmlThe HBase’s configuration file. It is added as a configuration resource on every connection, whether or not Kerberos is enabled.
hbase.kerberos_principal(empty)The HBase’s principal for kerberos authentication.
hbase.kerberos_keytab(empty)The HBase’s key tab file for kerberos authentication.

When hbase.kerberos_enable=true, HugeGraph sets hadoop.security.authentication and hbase.security.authentication to kerberos on the connection, then logs in from the keytab with the configured principal before opening the connection. A Kerberos setup therefore needs all four of hbase.krb5_conf, hbase.hbase_site, hbase.kerberos_principal and hbase.kerberos_keytab to be valid:

hbase.kerberos_enable=true
hbase.krb5_conf=/etc/krb5.conf
hbase.hbase_site=/etc/hbase/conf/hbase-site.xml
hbase.kerberos_principal=hugegraph/host@EXAMPLE.COM
hbase.kerberos_keytab=/etc/security/keytabs/hugegraph.keytab

hbase.hbase_site is read even with Kerberos disabled, so a path that does not exist is simply an empty resource. Point it at the cluster’s own hbase-site.xml when HBase settings beyond the options above are needed.

Pre-split Partition Options

OptionDefaultDescription
hbase.enable_partitiontrueIs pre-split partitions enabled for HBase. Also decides whether the backend reports support for key-prefix and key-range scans.
hbase.vertex_partitions10The number of partitions of the HBase vertex table. Must not be negative.
hbase.edge_partitions30The number of partitions of the HBase edge table. Must not be negative.

With pre-split enabled, the vertex table is created with hbase.vertex_partitions regions and each of the two edge tables with hbase.edge_partitions regions, and the serializer prefixes row keys with the partition the id hashes to.

Note: set the partition counts to match the actual data volume and the number of region servers before the store is initialized. They change the load speed considerably, and they are only applied at table creation time.

Turning hbase.enable_partition off restores plain, unprefixed row keys. In exchange the backend then reports support for key-prefix scans and key-range scans, which pre-split row keys cannot serve.

Namespace and Table Layout

Each graph maps to one HBase namespace named <graphspace>/<store>, lowercased, with / replaced by _ because an HBase namespace name may only contain alphanumeric characters and the _ character. With the defaults graphspace=DEFAULT and store=hugegraph, the namespace is default_hugegraph.

Inside that namespace a graph keeps three stores, the schema store m, the graph store g and the system store s:

StoreTables
schema (m)VL, EL, PK, IL, C, m_si
graph (g)g_v, g_oe, g_ie, g_si, g_vi, g_ei, g_ii, g_fi, g_li, g_di, g_ai, g_hi, g_ui
system (s)s_v, s_oe, s_ie, s_si, s_vi, s_ei, s_ii, s_fi, s_li, s_di, s_ai, s_hi, s_ui, M

g_v is the vertex table, g_oe and g_ie are the out-edge and in-edge tables, and the remaining g_* tables are the secondary, vertex-label, edge-label, range (int, float, long, double), search, shard and unique index tables. Every table has a single column family named f, and every table is created with the org.apache.hadoop.hbase.coprocessor.AggregateImplementation coprocessor attached. Only g_v, g_oe and g_ie are pre-split; the system store’s copies of those tables are created with a single region.

The M table in the system store holds the backend version written by init-store.sh. It is excluded when a graph is truncated, because losing it makes the version check fail on the next startup. Clearing a graph drops the tables; clearing it with the storage space included drops the whole namespace.

GET /metrics/backend reports the HBase cluster state: cluster_id, master_name, average_load, hbase_version, region_count, leaving_servers, nodes, region_servers, and a servers map with heap, disk, request and per region details for each region server. PUT /graphspaces/{graphspace}/graphs/{name}/compact asks HBase to compact every table of the graph.

Local Testing with Docker

docker/hbase in the server repository builds a standalone HBase 2.6.5 image (hugegraph/hbase:2.6.5, container name hg-hbase-test) for local development and tests. Run these from the repository root.

Start HBase for a HugeGraph server running on the host:

docker compose -p hg-hbase -f docker/hbase/docker-compose.hbase.yml build --no-cache hbase
HBASE_MASTER_HOSTNAME=localhost HBASE_REGIONSERVER_HOSTNAME=localhost \
docker compose -p hg-hbase -f docker/hbase/docker-compose.hbase.yml up -d
until docker exec hg-hbase-test nc -z localhost 2181 >/dev/null 2>&1; do sleep 2; done

Start HBase for a HugeGraph server running in a container on the same Docker network:

HBASE_HOSTNAME=hbase docker compose -p hg-hbase -f docker/hbase/docker-compose.hbase.yml up -d

The advertised hostnames matter: the container writes HBASE_MASTER_HOSTNAME and HBASE_REGIONSERVER_HOSTNAME into its hbase-site.xml on startup, falling back to HBASE_HOSTNAME (default hbase). A client that cannot resolve the advertised name fails with UnknownHostException: hbase:16000 even though ZooKeeper answers.

Ports published to the host:

PortService
2181ZooKeeper, matches the hbase.port default
16000HBase Master RPC
16010HBase Master web UI, http://localhost:16010
16020HBase RegionServer RPC
16030HBase RegionServer web UI, http://localhost:16030

Run the backend test suite against it:

mvn test -pl hugegraph-server/hugegraph-test -am -P core-test,hbase

Stop it and remove its volumes:

docker compose -p hg-hbase -f docker/hbase/docker-compose.hbase.yml down -v

The image starts ZooKeeper, the master and the region server as separate daemons and waits for the master to report a live server before it starts tailing the logs, so the first startup can take a while. Give Docker at least 4 GB of memory. The compose health check has a 90 second start period for the same reason.

Limitations

The HBase backend does not support these features:

  • Transactions. A rollback only discards the batch that has not been committed yet, and a commit writes one table at a time, so it is not atomic across tables.
  • Updating a single vertex or edge property in place, and merging vertex properties. Properties are stored in one cell, so the whole property column is rewritten.
  • Querying schema by name, and querying vertices or edges by label alone. Both would need an HBase secondary index.
  • Deleting edges by label.
  • Queries with an in condition, a contains condition or a contains_key condition.
  • Aggregate properties and OLAP properties.
  • Native number types (the supportsNumberType backend feature is off).
  • Scan tokens.
  • Key-prefix scans and key-range scans while hbase.enable_partition is true.
  • Aggregation other than count. Any other aggregate function is rejected.
  • Snapshots. Creating or resuming a backend snapshot throws UnsupportedOperationException.

Supported features include TTL on vertices and edges, paged queries, order-by queries, range conditions, and sorting by input ids.

5 - Clients and APIs

Connect to HugeGraph with the Java client, REST API, Gremlin Console, and other client libraries.

This section covers the REST API, Gremlin Console, and client libraries. The current Server REST API identifies graph resources with both a graph space and a graph name. Refer to each API page and the Server OpenAPI page for the exact paths.

5.1 - HugeGraph RESTful API

HugeGraph RESTful API reference for graph, schema, vertex, and edge operations.

This section documents the REST API on current master. For historical APIs, switch to the versioned HugeGraph 1.7 RESTful API documentation or HugeGraph 1.5 RESTful API documentation.

The default graph space is DEFAULT.

After starting Server, open http://localhost:8080/swagger-ui/index.html to view the OpenAPI page for the current version. See the usage example.

5.1.1 - Graphspace API

Graphspace REST API: Multi-tenancy and resource isolation for creating, viewing, updating, and deleting graph spaces with prerequisites and constraints.

2.0 Graphspace

HugeGraph implements multi-tenancy through graph spaces, which isolate compute/storage resources per tenant.

Prerequisites

  1. Graphspace currently only works in HStore mode.
  2. In non-HStore mode you can only use the default graphspace DEFAULT; creating/deleting/updating other graphspaces is not supported.
  3. Set usePD=true in rest-server.properties and backend=hstore in hugegraph.properties.
  4. Production requires Server authentication and authorization, and auth=true for new graph spaces. Replace the public default administrator password pa configured by auth.admin_pa.
  5. Every endpoint on this page requires PD mode. In standalone mode they answer 400 with the message GraphSpace management is not supported in standalone mode.
Warning

Restrict graph-space listing and detail endpoints

GET /graphspaces and GET /graphspaces/{graphspace} have no @RolesAllowed annotation. They are anonymous when Server authentication is disabled and lack method-level administrator checks when it is enabled. Detail responses contain dp_username and dp_password. In production, set white_ip.status=enable and maintain the IP allowlist. A gateway must restrict both paths by caller identity or role to administrators and trusted operators; restricting only source IPs still lets ordinary authenticated accounts on that network read DP credentials. Server network policy must allow only trusted gateway egress addresses to the API port, blocking direct client bypass. Record caller identity, source, and outcome at the gateway. Server audit-*.log files record authentication and authorization but do not replace gateway access auditing for these paths. Grant business accounts minimum permissions.

2.0.1 Create a graphspace

Method & Url
POST http://localhost:8080/graphspaces
Request Body

Note: CPU/memory and Kubernetes-related capabilities are not publicly available yet.

NameRequiredTypeDefaultRange/NoteDescription
nameYesStringLowercase letters, digits, underscore; must start with a letter; max length 48Graphspace name
nicknameNoStringnameMust be unique among graphspacesDisplay name of the graphspace
descriptionNoStringDescription
cpu_limitYesInt> 0CPU cores for the graphspace
memory_limitYesInt> 0 (GB)Memory quota in GB
storage_limitYesInt> 0Maximum disk usage, in GB
compute_cpu_limitNoInt0>= 0Extra HugeGraph-Computer CPU cores; falls back to cpu_limit if unset or 0
compute_memory_limitNoInt0>= 0Extra HugeGraph-Computer memory in GB; falls back to memory_limit if unset or 0
oltp_namespaceNoString""Kubernetes namespace for OLTP HugeGraph-Server
olap_namespaceNoString""Resources are merged when identical to oltp_namespaceKubernetes namespace for OLAP / HugeGraph-Computer
storage_namespaceNoString""Kubernetes namespace for HugeGraph-Store
operator_image_pathNoString""HugeGraph-Computer operator image registry
internal_algorithm_image_urlNoString""HugeGraph-Computer algorithm image registry
max_graph_numberYesInt> 0Maximum number of graphs that can be created inside the graphspace
max_role_numberNoInt0Maximum number of roles that can be created inside the graphspace
authNoBooleanfalsetrue / falseWhether to enable authentication for the graphspace
configsNoMapAdditional configuration
{
  "name": "gs1",
  "description": "1st graph space",
  "max_graph_number": 100,
  "cpu_limit": 1000,
  "memory_limit": 8192,
  "storage_limit": 1000000,
  "max_role_number": 10,
  "auth": true,
  "configs": {}
}
Response Status
201
Response Body
{
  "name": "gs1",
  "nickname": "gs1",
  "description": "1st graph space",
  "cpu_limit": 1000,
  "memory_limit": 8192,
  "storage_limit": 1000000,
  "compute_cpu_limit": 0,
  "compute_memory_limit": 0,
  "oltp_namespace": "hugegraph-server",
  "olap_namespace": "hugegraph-server",
  "storage_namespace": "hugegraph-server",
  "operator_image_path": "127.0.0.1/hugegraph-registry/hugegraph-computer-operator:3.1.1",
  "internal_algorithm_image_url": "127.0.0.1/hugegraph-registry/hugegraph-computer-algorithm:3.1.1",
  "max_graph_number": 100,
  "max_role_number": 10,
  "cpu_used": 0,
  "memory_used": 0,
  "storage_used": 0,
  "storage_percent": 0.0,
  "graph_number_used": 0,
  "role_number_used": 0,
  "auth": true,
  "creator": "admin",
  "create_time": "2024-05-01 12:00:00",
  "update_time": "2024-05-01 12:00:00"
}

2.0.2 List all graphspaces

Method & Url
GET http://localhost:8080/graphspaces
Response Status
200
Response Body
{
  "graphSpaces": [
    "gs1",
    "DEFAULT"
  ]
}

2.0.3 Get graphspace details

Params

Path parameters

  • graphspace: Graphspace name
Method & Url
GET http://localhost:8080/graphspaces/gs1
Response Status
200
Response Body
{
  "name": "gs1",
  "nickname": "gs1",
  "description": "1st graph space",
  "cpu_limit": 1000,
  "memory_limit": 8192,
  "storage_limit": 1000000,
  "oltp_namespace": "hugegraph-server",
  "olap_namespace": "hugegraph-server",
  "storage_namespace": "hugegraph-server",
  "operator_image_path": "127.0.0.1/hugegraph-registry/hugegraph-computer-operator:3.1.1",
  "internal_algorithm_image_url": "127.0.0.1/hugegraph-registry/hugegraph-computer-algorithm:3.1.1",
  "compute_cpu_limit": 0,
  "compute_memory_limit": 0,
  "max_graph_number": 100,
  "max_role_number": 10,
  "cpu_used": 0,
  "memory_used": 0,
  "storage_used": 0,
  "storage_percent": 0.0,
  "graph_number_used": 0,
  "role_number_used": 0,
  "auth": true,
  "creator": "admin",
  "create_time": "2024-05-01 12:00:00",
  "update_time": "2024-05-01 12:00:00",
  "dp_username": "gs1_dp",
  "dp_password": "a1b2c3d4e5f60718"
}

The dp_username and dp_password values above are documentation examples. Real detail responses contain the graph space’s DP credentials and must be protected as sensitive credentials.

2.0.4 Update a graphspace

auth cannot be changed once a graphspace is created.

Params

Path parameter

  • graphspace: Graphspace name

Request parameters

  • action: Must be "update"
  • update: Container for the actual fields to update (see table below)
NameRequiredTypeRange/NoteDescription
nameYesStringMust match the graphspace name in the pathGraphspace name
nicknameNoStringMust be unique among graphspacesDisplay name of the graphspace
descriptionNoStringDescription
cpu_limitYesInt> 0CPU cores for OLTP HugeGraph-Server
memory_limitYesInt> 0 (GB)Memory quota (GB) for OLTP HugeGraph-Server
storage_limitYesInt> 0Maximum disk usage, in GB
compute_cpu_limitNoInt>= 0Extra HugeGraph-Computer CPU cores; falls back to cpu_limit if unset or 0
compute_memory_limitNoInt>= 0Extra HugeGraph-Computer memory in GB; falls back to memory_limit if unset or 0
oltp_namespaceYesStringKubernetes namespace for OLTP HugeGraph-Server
olap_namespaceYesStringResources are merged when identical to oltp_namespaceKubernetes namespace for OLAP
storage_namespaceYesStringKubernetes namespace for HugeGraph-Store
operator_image_pathNoStringHugeGraph-Computer operator image registry
internal_algorithm_image_urlNoStringHugeGraph-Computer algorithm image registry
max_graph_numberYesInt> 0Maximum number of graphs
max_role_numberYesInt> 0Maximum number of roles
Method & Url
PUT http://localhost:8080/graphspaces/gs1
Request Body
{
  "action": "update",
  "update": {
    "name": "gs1",
    "description": "1st graph space",
    "cpu_limit": 2000,
    "memory_limit": 40960,
    "storage_limit": 2048,
    "oltp_namespace": "hugegraph-server",
    "olap_namespace": "hugegraph-server",
    "operator_image_path": "127.0.0.1/hugegraph-registry/hugegraph-computer-operator:3.1.1",
    "internal_algorithm_image_url": "127.0.0.1/hugegraph-registry/hugegraph-computer-algorithm:3.1.1",
    "max_graph_number": 1000,
    "max_role_number": 100
  }
}
Response Status
200
Response Body
{
  "name": "gs1",
  "nickname": "gs1",
  "description": "1st graph space",
  "cpu_limit": 2000,
  "memory_limit": 40960,
  "storage_limit": 2048,
  "oltp_namespace": "hugegraph-server",
  "olap_namespace": "hugegraph-server",
  "storage_namespace": "hugegraph-server",
  "operator_image_path": "127.0.0.1/hugegraph-registry/hugegraph-computer-operator:3.1.1",
  "internal_algorithm_image_url": "127.0.0.1/hugegraph-registry/hugegraph-computer-algorithm:3.1.1",
  "compute_cpu_limit": 0,
  "compute_memory_limit": 0,
  "max_graph_number": 1000,
  "max_role_number": 100,
  "cpu_used": 0,
  "memory_used": 0,
  "storage_used": 0,
  "storage_percent": 0.0,
  "graph_number_used": 0,
  "role_number_used": 0,
  "auth": true,
  "creator": "admin",
  "create_time": "2024-05-01 12:00:00",
  "update_time": "2024-05-01 12:30:00"
}

2.0.5 Delete a graphspace

Params

Path parameter

  • graphspace: Graphspace name
Method & Url
DELETE http://localhost:8080/graphspaces/gs1
Response Status
204

Warning: deleting a graphspace releases all resources that belong to it.

2.0.6 List all graphspaces with their details

Params

Query parameters

  • prefix: Return only the graphspaces whose name or nickname starts with this prefix
Method & Url
GET http://localhost:8080/graphspaces/profile
Response Status
200
Response Body

Each entry carries the same fields as GET /graphspaces/{graphspace} plus authed, default, create_time and update_time. authed says whether the current user may enter the graphspace: it is false when the graphspace has authentication on and the user is neither an administrator, nor a manager, nor a member of it. default is always false for now, the default-graphspace feature is not implemented yet.

[
  {
    "name": "gs1",
    "nickname": "gs1",
    "description": "1st graph space",
    "cpu_limit": 1000,
    "memory_limit": 8192,
    "storage_limit": 1000000,
    "compute_cpu_limit": 0,
    "compute_memory_limit": 0,
    "oltp_namespace": "hugegraph-server",
    "olap_namespace": "hugegraph-server",
    "storage_namespace": "hugegraph-server",
    "max_graph_number": 100,
    "max_role_number": 10,
    "cpu_used": 0,
    "memory_used": 0,
    "storage_used": 0,
    "storage_percent": 0.0,
    "graph_number_used": 0,
    "role_number_used": 0,
    "auth": true,
    "creator": "admin",
    "authed": true,
    "default": false,
    "create_time": "2024-05-01 12:00:00",
    "update_time": "2024-05-01 12:30:00"
  }
]

Default roles

Every graphspace carries four built-in roles, so that a user or a group can be given a whole set of permissions at once:

  • space: manager of the graphspace, only an administrator may grant it
  • space_member: member of the graphspace
  • analyst: analyst of the graphspace
  • observer: read-only role, it can be narrowed to a single graph by passing graph

user accepts either a user name or a group name. Whether the current user holds a default role can also be checked with GET /graphspaces/{graphspace}/auth/managers/default, see Authentication API.

2.0.7 Grant a default role

Params

Path parameter

  • graphspace: Graphspace name

Request parameters

  • user: User or group name, required
  • role: One of space, space_member, analyst, observer, required
  • graph: Graph name, optional, only taken into account with role=observer
Method & Url
POST http://localhost:8080/graphspaces/gs1/role
Request Body
{
  "user": "boss",
  "role": "analyst"
}
Response Status
201
Response Body

graph is echoed back only when the role was granted on a single graph.

{
  "user": "boss",
  "role": "analyst",
  "graphSpace": "gs1"
}

2.0.8 Check a default role

Params

Path parameter

  • graphspace: Graphspace name

Query parameters

  • user: User or group name, required
  • role: Default role name, required
  • graph: Graph name, optional, only taken into account with role=observer
Method & Url
GET http://localhost:8080/graphspaces/gs1/role?user=boss&role=analyst
Response Status
200
Response Body
{
  "check": true
}

2.0.9 Revoke a default role

Params

Path parameter

  • graphspace: Graphspace name

Query parameters

  • user: User or group name, required
  • role: Default role name, required
  • graph: Graph name, optional, only taken into account with role=observer
Method & Url
DELETE http://localhost:8080/graphspaces/gs1/role?user=boss&role=analyst
Response Status
204

Schema templates

A schema template stores a Gremlin schema script under a name, so that a new graph can be initialized with it by passing schema when the graph is created, see Graphs API. A template can be updated or deleted by its creator, by a manager of the graphspace, or by an administrator.

2.0.10 Create a schema template

Params

Path parameter

  • graphspace: Graphspace name

Request parameters

  • name: Template name, required
  • schema: Gremlin schema script, required
Method & Url
POST http://localhost:8080/graphspaces/gs1/schematemplates
Request Body
{
  "name": "template1",
  "schema": "schema.propertyKey('name').asText().ifNotExist().create();"
}
Response Status
201
Response Body
{
  "name": "template1",
  "schema": "schema.propertyKey('name').asText().ifNotExist().create();",
  "creator": "admin",
  "create": "2024-05-01 12:00:00.000",
  "create_time": "2024-05-01 12:00:00.000",
  "update": "2024-05-01 12:00:00.000",
  "update_time": "2024-05-01 12:00:00.000"
}

2.0.11 List the schema templates of a graphspace

Method & Url
GET http://localhost:8080/graphspaces/gs1/schematemplates
Response Status
200
Response Body
{
  "schema_templates": [
    "template1"
  ]
}

2.0.12 Get a schema template

Method & Url
GET http://localhost:8080/graphspaces/gs1/schematemplates/template1
Response Status
200

2.0.13 Update a schema template

Only schema can be updated, the name of a template is fixed.

Method & Url
PUT http://localhost:8080/graphspaces/gs1/schematemplates/template1
Request Body
{
  "schema": "schema.propertyKey('age').asInt().ifNotExist().create();"
}
Response Status
200

2.0.14 Delete a schema template

Method & Url
DELETE http://localhost:8080/graphspaces/gs1/schematemplates/template1
Response Status
204

5.1.2 - Schema API

Schema REST API: Query the complete schema definition of a graph, including property keys, vertex labels, edge labels, and index labels.

1.1 Schema

HugeGraph provides a single interface to get all Schema information of a graph, including: PropertyKey, VertexLabel, EdgeLabel and IndexLabel.

Method & Url
GET http://localhost:8080/graphspaces/{graphspace}/graphs/{graph_name}/schema

e.g: GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/schema
Response Status
200
Response Body
{
    "propertykeys": [
        {
            "id": 7,
            "name": "price",
            "data_type": "DOUBLE",
            "cardinality": "SINGLE",
            "aggregate_type": "NONE",
            "write_type": "OLTP",
            "properties": [],
            "status": "CREATED",
            "user_data": {
                "~create_time": "2023-05-08 17:49:05.316"
            }
        },
        {
            "id": 6,
            "name": "date",
            "data_type": "TEXT",
            "cardinality": "SINGLE",
            "aggregate_type": "NONE",
            "write_type": "OLTP",
            "properties": [],
            "status": "CREATED",
            "user_data": {
                "~create_time": "2023-05-08 17:49:05.309"
            }
        },
        {
            "id": 3,
            "name": "city",
            "data_type": "TEXT",
            "cardinality": "SINGLE",
            "aggregate_type": "NONE",
            "write_type": "OLTP",
            "properties": [],
            "status": "CREATED",
            "user_data": {
                "~create_time": "2023-05-08 17:49:05.287"
            }
        },
        {
            "id": 2,
            "name": "age",
            "data_type": "INT",
            "cardinality": "SINGLE",
            "aggregate_type": "NONE",
            "write_type": "OLTP",
            "properties": [],
            "status": "CREATED",
            "user_data": {
                "~create_time": "2023-05-08 17:49:05.280"
            }
        },
        {
            "id": 5,
            "name": "lang",
            "data_type": "TEXT",
            "cardinality": "SINGLE",
            "aggregate_type": "NONE",
            "write_type": "OLTP",
            "properties": [],
            "status": "CREATED",
            "user_data": {
                "~create_time": "2023-05-08 17:49:05.301"
            }
        },
        {
            "id": 4,
            "name": "weight",
            "data_type": "DOUBLE",
            "cardinality": "SINGLE",
            "aggregate_type": "NONE",
            "write_type": "OLTP",
            "properties": [],
            "status": "CREATED",
            "user_data": {
                "~create_time": "2023-05-08 17:49:05.294"
            }
        },
        {
            "id": 1,
            "name": "name",
            "data_type": "TEXT",
            "cardinality": "SINGLE",
            "aggregate_type": "NONE",
            "write_type": "OLTP",
            "properties": [],
            "status": "CREATED",
            "user_data": {
                "~create_time": "2023-05-08 17:49:05.250"
            }
        }
    ],
    "vertexlabels": [
        {
            "id": 1,
            "name": "person",
            "id_strategy": "PRIMARY_KEY",
            "primary_keys": [
                "name"
            ],
            "nullable_keys": [
                "age",
                "city"
            ],
            "index_labels": [
                "personByAge",
                "personByCity",
                "personByAgeAndCity"
            ],
            "properties": [
                "name",
                "age",
                "city"
            ],
            "status": "CREATED",
            "ttl": 0,
            "enable_label_index": true,
            "user_data": {
                "~create_time": "2023-05-08 17:49:05.336"
            }
        },
        {
            "id": 2,
            "name": "software",
            "id_strategy": "CUSTOMIZE_NUMBER",
            "primary_keys": [],
            "nullable_keys": [],
            "index_labels": [
                "softwareByPrice"
            ],
            "properties": [
                "name",
                "lang",
                "price"
            ],
            "status": "CREATED",
            "ttl": 0,
            "enable_label_index": true,
            "user_data": {
                "~create_time": "2023-05-08 17:49:05.347"
            }
        }
    ],
    "edgelabels": [
        {
            "id": 1,
            "name": "knows",
            "source_label": "person",
            "target_label": "person",
            "frequency": "SINGLE",
            "sort_keys": [],
            "nullable_keys": [],
            "index_labels": [
                "knowsByWeight"
            ],
            "properties": [
                "weight",
                "date"
            ],
            "status": "CREATED",
            "ttl": 0,
            "enable_label_index": true,
            "user_data": {
                "~create_time": "2023-05-08 17:49:08.437"
            }
        },
        {
            "id": 2,
            "name": "created",
            "source_label": "person",
            "target_label": "software",
            "frequency": "SINGLE",
            "sort_keys": [],
            "nullable_keys": [],
            "index_labels": [
                "createdByDate",
                "createdByWeight"
            ],
            "properties": [
                "weight",
                "date"
            ],
            "status": "CREATED",
            "ttl": 0,
            "enable_label_index": true,
            "user_data": {
                "~create_time": "2023-05-08 17:49:08.446"
            }
        }
    ],
    "indexlabels": [
        {
            "id": 1,
            "name": "personByAge",
            "base_type": "VERTEX_LABEL",
            "base_value": "person",
            "index_type": "RANGE_INT",
            "fields": [
                "age"
            ],
            "status": "CREATED",
            "user_data": {
                "~create_time": "2023-05-08 17:49:05.375"
            }
        },
        {
            "id": 2,
            "name": "personByCity",
            "base_type": "VERTEX_LABEL",
            "base_value": "person",
            "index_type": "SECONDARY",
            "fields": [
                "city"
            ],
            "status": "CREATED",
            "user_data": {
                "~create_time": "2023-05-08 17:49:06.898"
            }
        },
        {
            "id": 3,
            "name": "personByAgeAndCity",
            "base_type": "VERTEX_LABEL",
            "base_value": "person",
            "index_type": "SECONDARY",
            "fields": [
                "age",
                "city"
            ],
            "status": "CREATED",
            "user_data": {
                "~create_time": "2023-05-08 17:49:07.407"
            }
        },
        {
            "id": 4,
            "name": "softwareByPrice",
            "base_type": "VERTEX_LABEL",
            "base_value": "software",
            "index_type": "RANGE_DOUBLE",
            "fields": [
                "price"
            ],
            "status": "CREATED",
            "user_data": {
                "~create_time": "2023-05-08 17:49:07.916"
            }
        },
        {
            "id": 5,
            "name": "createdByDate",
            "base_type": "EDGE_LABEL",
            "base_value": "created",
            "index_type": "SECONDARY",
            "fields": [
                "date"
            ],
            "status": "CREATED",
            "user_data": {
                "~create_time": "2023-05-08 17:49:08.454"
            }
        },
        {
            "id": 6,
            "name": "createdByWeight",
            "base_type": "EDGE_LABEL",
            "base_value": "created",
            "index_type": "RANGE_DOUBLE",
            "fields": [
                "weight"
            ],
            "status": "CREATED",
            "user_data": {
                "~create_time": "2023-05-08 17:49:08.963"
            }
        },
        {
            "id": 7,
            "name": "knowsByWeight",
            "base_type": "EDGE_LABEL",
            "base_value": "knows",
            "index_type": "RANGE_DOUBLE",
            "fields": [
                "weight"
            ],
            "status": "CREATED",
            "user_data": {
                "~create_time": "2023-05-08 17:49:09.473"
            }
        }
    ]
}

5.1.3 - PropertyKey API

PropertyKey REST API: Define data types and cardinality constraints for all properties in the graph, serving as fundamental schema elements.

1.2 PropertyKey

Params Description:

  • name: The name of the property type, required.
  • data_type: The data type of the property type, including: bool, byte, int, long, float, double, text, blob, date, uuid. The default data type is text (Represent a string type)
  • cardinality: The cardinality of the property type, including: single, list, set. The default cardinality is single.

Request Body Field Description:

  • id: The ID value of the property type.
  • properties: The properties of the property type. For properties, this field is empty.
  • user_data: Setting the common information of the property type, such as setting the value range of the age property from 0 to 100. Currently, no validation is performed on this field, and it is only a reserved entry for future expansion.

1.2.1 Create a PropertyKey

Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/propertykeys
Request Body
{
    "name": "age",
    "data_type": "INT",
    "cardinality": "SINGLE"
}
Response Status
202
Response Body
{
    "property_key": {
        "id": 1,
        "name": "age",
        "data_type": "INT",
        "cardinality": "SINGLE",
        "aggregate_type": "NONE",
        "write_type": "OLTP",
        "properties": [],
        "status": "CREATED",
        "user_data": {
            "~create_time": "2022-05-13 13:47:23.745"
        }
    },
    "task_id": 0
}

1.2.2 Add or Remove userdata for an existing PropertyKey

Params
  • action: Indicates whether the current action is to add or remove userdata. Possible values are append (add) and eliminate (remove).
Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/propertykeys/age?action=append
Request Body
{
    "name": "age",
    "user_data": {
        "min": 0,
        "max": 100
    }
}
Response Status
202
Response Body
{
    "property_key": {
        "id": 1,
        "name": "age",
        "data_type": "INT",
        "cardinality": "SINGLE",
        "aggregate_type": "NONE",
        "write_type": "OLTP",
        "properties": [],
        "status": "CREATED",
        "user_data": {
            "min": 0,
            "max": 100,
            "~create_time": "2022-05-13 13:47:23.745"
        }
    },
    "task_id": 0
}

1.2.3 Get all PropertyKeys

Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/propertykeys
Response Status
200
Response Body
{
    "propertykeys": [
        {
            "id": 3,
            "name": "city",
            "data_type": "TEXT",
            "cardinality": "SINGLE",
            "properties": [],
            "user_data": {}
        },
        {
            "id": 2,
            "name": "age",
            "data_type": "INT",
            "cardinality": "SINGLE",
            "properties": [],
            "user_data": {}
        },
        {
            "id": 5,
            "name": "lang",
            "data_type": "TEXT",
            "cardinality": "SINGLE",
            "properties": [],
            "user_data": {}
        },
        {
            "id": 4,
            "name": "weight",
            "data_type": "DOUBLE",
            "cardinality": "SINGLE",
            "properties": [],
            "user_data": {}
        },
        {
            "id": 6,
            "name": "date",
            "data_type": "TEXT",
            "cardinality": "SINGLE",
            "properties": [],
            "user_data": {}
        },
        {
            "id": 1,
            "name": "name",
            "data_type": "TEXT",
            "cardinality": "SINGLE",
            "properties": [],
            "user_data": {}
        },
        {
            "id": 7,
            "name": "price",
            "data_type": "INT",
            "cardinality": "SINGLE",
            "properties": [],
            "user_data": {}
        }
    ]
}

1.2.4 Get PropertyKey according to name

Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/propertykeys/age

Where age is the name of the PropertyKey to be retrieved.

Response Status
200
Response Body
{
    "id": 1,
    "name": "age",
    "data_type": "INT",
    "cardinality": "SINGLE",
    "aggregate_type": "NONE",
    "write_type": "OLTP",
    "properties": [],
    "status": "CREATED",
    "user_data": {
        "min": 0,
        "max": 100,
        "~create_time": "2022-05-13 13:47:23.745"
    }
}

1.2.5 Delete PropertyKey according to name

Method & Url
DELETE http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/propertykeys/age

Where age is the name of the PropertyKey to be deleted.

Response Status
202
Response Body
{
    "task_id" : 0
}

5.1.4 - VertexLabel API

VertexLabel REST API: Define vertex types, ID strategies, and associated properties that determine vertex structure and constraints.

1.3 VertexLabel

Assuming that the PropertyKeys listed in 1.1.3 have already been created.

Params Description:

  • id: The ID value of the vertex type.
  • name: The name of the vertex type, required.
  • id_strategy: The ID strategy for the vertex type, including primary key ID, auto-generated, custom string, custom number, custom UUID. The default strategy is primary key ID.
  • properties: The property types associated with the vertex type.
  • primary_keys: The primary key properties. This field must have a value when the ID strategy is PRIMARY_KEY, and must be empty for other ID strategies.
  • enable_label_index: Whether to enable label indexing. It is disabled by default.
  • index_names: The indexes created for the vertex type. See details in section 3.4.
  • nullable_keys: Nullable properties.
  • user_data: Setting the common information of the vertex type, similar to the property type.

1.3.1 Create a VertexLabel

Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/vertexlabels
Request Body
{
    "name": "person",
    "id_strategy": "DEFAULT",
    "properties": [
        "name",
        "age"
    ],
    "primary_keys": [
        "name"
    ],
    "nullable_keys": [],
    "enable_label_index": true
}
Response Status
201
Response Body
{
    "id": 1,
    "primary_keys": [
        "name"
    ],
    "id_strategy": "PRIMARY_KEY",
    "name": "person2",
    "index_names": [
    ],
    "properties": [
        "name",
        "age"
    ],
    "nullable_keys": [
    ],
    "enable_label_index": true,
    "user_data": {}
}

Starting from version v0.11.2, hugegraph-server supports Time-to-Live (TTL) functionality for vertices. The TTL for vertices is set through VertexLabel. For example, if you want the vertices of type “person” to have a lifespan of one day, you need to set the TTL field to 86400000 (in milliseconds) when creating the “person” VertexLabel.

{
    "name": "person",
    "id_strategy": "DEFAULT",
    "properties": [
        "name",
        "age"
    ],
    "primary_keys": [
        "name"
    ],
    "nullable_keys": [],
    "ttl": 86400000,
    "enable_label_index": true
}

Additionally, if the vertex has a property called “createdTime” and you want to use it as the starting point for calculating the vertex’s lifespan, you can set the ttl_start_time field in the VertexLabel. For example, if the “person” VertexLabel has a property called “createdTime” of type Date, and you want the vertices of type “person” to live for one day starting from the creation time, the Request Body for creating the “person” VertexLabel would be as follows:

{
    "name": "person",
    "id_strategy": "DEFAULT",
    "properties": [
        "name",
        "age",
        "createdTime"
    ],
    "primary_keys": [
        "name"
    ],
    "nullable_keys": [],
    "ttl": 86400000,
    "ttl_start_time": "createdTime",
    "enable_label_index": true
}

1.3.2 Add properties or userdata to an existing VertexLabel, or remove userdata (removing properties is currently not supported)

Params
  • action: Indicates whether the current action is to add or remove. Possible values are append (add) and eliminate (remove).
Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/vertexlabels/person?action=append
Request Body
{
    "name": "person",
    "properties": [
        "city"
    ],
    "nullable_keys": ["city"],
    "user_data": {
        "super": "animal"
    }
}
Response Status
200
Response Body
{
    "id": 1,
    "primary_keys": [
        "name"
    ],
    "id_strategy": "PRIMARY_KEY",
    "name": "person",
    "index_names": [
    ],
    "properties": [
        "city",
        "name",
        "age"
    ],
    "nullable_keys": [
        "city"
    ],
    "enable_label_index": true,
    "user_data": {
        "super": "animal"
    }
}

1.3.3 Get all VertexLabels

Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/vertexlabels
Response Status
200
Response Body
{
    "vertexlabels": [
        {
            "id": 1,
            "primary_keys": [
                "name"
            ],
            "id_strategy": "PRIMARY_KEY",
            "name": "person",
            "index_names": [
            ],
            "properties": [
                "city",
                "name",
                "age"
            ],
            "nullable_keys": [
                "city"
            ],
            "enable_label_index": true,
            "user_data": {
                "super": "animal"
            }
        },
        {
            "id": 2,
            "primary_keys": [
                "name"
            ],
            "id_strategy": "PRIMARY_KEY",
            "name": "software",
            "index_names": [
            ],
            "properties": [
                "price",
                "name",
                "lang"
            ],
            "nullable_keys": [
                "price"
            ],
            "enable_label_index": false,
            "user_data": {}
        }
    ]
}

1.3.4 Get VertexLabel by name

Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/vertexlabels/person
Response Status
200
Response Body
{
    "id": 1,
    "primary_keys": [
        "name"
    ],
    "id_strategy": "PRIMARY_KEY",
    "name": "person",
    "index_names": [
    ],
    "properties": [
        "city",
        "name",
        "age"
    ],
    "nullable_keys": [
        "city"
    ],
    "enable_label_index": true,
    "user_data": {
        "super": "animal"
    }
}

1.3.5 Delete VertexLabel by name

Deleting a VertexLabel will result in the removal of corresponding vertices and related index data. This operation will generate an asynchronous task.

Method & Url
DELETE http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/vertexlabels/person
Response Status
202
Response Body
{
    "task_id": 1
}

Note:

You can use GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/tasks/1 (where “1” is the task_id) to query the execution status of the asynchronous task. For more information, refer to the Asynchronous Task RESTful API.

5.1.5 - EdgeLabel API

EdgeLabel REST API: Define edge types and relationship constraints between source and target vertices to construct graph connection rules.

1.4 EdgeLabel

Assuming PropertyKeys from version 1.2.3 and VertexLabels from version 1.3.3 have already been created.

Params Explanation

  • name: Name of the vertex type, required.
  • source_label: Name of the source vertex type, required.
  • target_label: Name of the target vertex type, required.
  • frequency: Whether there can be multiple edges between two points, can have values SINGLE or MULTIPLE, optional (default value: SINGLE).
  • properties: Property types associated with the edge type, optional.
  • sort_keys: Specifies a list of differentiating key properties when multiple associations are allowed.
  • nullable_keys: Nullable properties, optional (default: nullable).
  • enable_label_index: Whether to enable type indexing, disabled by default.

1.4.1 Create an EdgeLabel

Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/edgelabels
Request Body
{
    "name": "created",
    "source_label": "person",
    "target_label": "software",
    "frequency": "SINGLE",
    "properties": [
        "date"
    ],
    "sort_keys": [],
    "nullable_keys": [],
    "enable_label_index": true
}
Response Status
201
Response Body
{
    "id": 1,
    "sort_keys": [
    ],
    "source_label": "person",
    "name": "created",
    "index_names": [
    ],
    "properties": [
        "date"
    ],
    "target_label": "software",
    "frequency": "SINGLE",
    "nullable_keys": [
    ],
    "enable_label_index": true,
    "user_data": {}
}

Starting from version 0.11.2 of hugegraph-server, the TTL (Time to Live) feature for edges is supported. The TTL for edges is set through EdgeLabel. For example, if you want the “knows” type of edge to have a lifespan of one day, you need to set the TTL field to 86400000 when creating the “knows” EdgeLabel, where the unit is milliseconds.

{
    "id": 1,
    "sort_keys": [
    ],
    "source_label": "person",
    "name": "knows",
    "index_names": [
    ],
    "properties": [
        "date",
        "createdTime"
    ],
    "target_label": "person",
    "frequency": "SINGLE",
    "nullable_keys": [
    ],
    "enable_label_index": true,
    "ttl": 86400000,
    "user_data": {}
}

Additionally, when the edge has a property called “createdTime” and you want to use the “createdTime” property as the starting point for calculating the edge’s lifespan, you can set the ttl_start_time field in the EdgeLabel. For example, if the knows EdgeLabel has a property called “createdTime” which is of type Date, and you want the “knows” type of edge to live for one day from the time of creation, the Request Body for creating the knows EdgeLabel would be as follows:

{
    "id": 1,
    "sort_keys": [
    ],
    "source_label": "person",
    "name": "knows",
    "index_names": [
    ],
    "properties": [
        "date",
        "createdTime"
    ],
    "target_label": "person",
    "frequency": "SINGLE",
    "nullable_keys": [
    ],
    "enable_label_index": true,
    "ttl": 86400000,
    "ttl_start_time": "createdTime",
    "user_data": {}
}

1.4.2 Add properties or userdata to an existing EdgeLabel, or remove userdata (removing properties is currently not supported)

Params
  • action: Indicates whether the current action is to add or remove, with values append (add) and eliminate (remove).
Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/edgelabels/created?action=append
Request Body
{
    "name": "created",
    "properties": [
        "weight"
    ],
    "nullable_keys": [
        "weight"
    ]
}
Response Status
200
Response Body
{
    "id": 2,
    "sort_keys": [
    ],
    "source_label": "person",
    "name": "created",
    "index_names": [
    ],
    "properties": [
        "date",
        "weight"
    ],
    "target_label": "software",
    "frequency": "SINGLE",
    "nullable_keys": [
        "weight"
    ],
    "enable_label_index": true,
    "user_data": {}
}

1.4.3 Get all EdgeLabels

Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/edgelabels
Response Status
200
Response Body
{
    "edgelabels": [
        {
            "id": 1,
            "sort_keys": [
            ],
            "source_label": "person",
            "name": "created",
            "index_names": [
            ],
            "properties": [
                "date",
                "weight"
            ],
            "target_label": "software",
            "frequency": "SINGLE",
            "nullable_keys": [
                "weight"
            ],
            "enable_label_index": true,
            "user_data": {}
        },
        {
            "id": 2,
            "sort_keys": [
            ],
            "source_label": "person",
            "name": "knows",
            "index_names": [
            ],
            "properties": [
                "date",
                "weight"
            ],
            "target_label": "person",
            "frequency": "SINGLE",
            "nullable_keys": [
            ],
            "enable_label_index": false,
            "user_data": {}
        }
    ]
}

1.4.4 Get EdgeLabel by name

Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/edgelabels/created
Response Status
200
Response Body
{
    "id": 1,
    "sort_keys": [
    ],
    "source_label": "person",
    "name": "created",
    "index_names": [
    ],
    "properties": [
        "date",
        "city",
        "weight"
    ],
    "target_label": "software",
    "frequency": "SINGLE",
    "nullable_keys": [
        "city",
        "weight"
    ],
    "enable_label_index": true,
    "user_data": {}
}

1.4.5 Delete EdgeLabel by name

Deleting an EdgeLabel will result in the deletion of corresponding edges and related index data. This operation will generate an asynchronous task.

Method & Url
DELETE http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/edgelabels/created
Response Status
202
Response Body
{
    "task_id": 1
}

Note:

You can query the execution status of an asynchronous task by using GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/tasks/1 (where “1” is the task_id). For more information, refer to the Asynchronous Task RESTful API.

5.1.6 - IndexLabel API

IndexLabel REST API: Create indexes on vertex and edge properties to accelerate property-based queries and filtering operations.

1.5 IndexLabel

Assuming PropertyKeys from version 1.1.3, VertexLabels from version 1.2.3, and EdgeLabels from version 1.3.3 have already been created.

1.5.1 Create an IndexLabel

Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/indexlabels
Request Body
{
    "name": "personByCity",
    "base_type": "VERTEX_LABEL",
    "base_value": "person",
    "index_type": "SECONDARY",
    "fields": [
        "city"
    ]
}
Response Status
202
Response Body
{
    "index_label": {
        "id": 1,
        "base_type": "VERTEX_LABEL",
        "base_value": "person",
        "name": "personByCity",
        "fields": [
            "city"
        ],
        "index_type": "SECONDARY"
    },
    "task_id": 2
}

1.5.2 Get all IndexLabels

Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/indexlabels
Response Status
200
Response Body
{
    "indexlabels": [
        {
            "id": 3,
            "base_type": "VERTEX_LABEL",
            "base_value": "software",
            "name": "softwareByPrice",
            "fields": [
                "price"
            ],
            "index_type": "RANGE"
        },
        {
            "id": 4,
            "base_type": "EDGE_LABEL",
            "base_value": "created",
            "name": "createdByDate",
            "fields": [
                "date"
            ],
            "index_type": "SECONDARY"
        },
        {
            "id": 1,
            "base_type": "VERTEX_LABEL",
            "base_value": "person",
            "name": "personByCity",
            "fields": [
                "city"
            ],
            "index_type": "SECONDARY"
        },
        {
            "id": 3,
            "base_type": "VERTEX_LABEL",
            "base_value": "person",
            "name": "personByAgeAndCity",
            "fields": [
                "age",
                "city"
            ],
            "index_type": "SECONDARY"
        }
    ]
}

1.5.3 Get IndexLabel by name

Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/indexlabels/personByCity
Response Status
200
Response Body
{
    "id": 1,
    "base_type": "VERTEX_LABEL",
    "base_value": "person",
    "name": "personByCity",
    "fields": [
        "city"
    ],
    "index_type": "SECONDARY"
}

1.5.4 Delete IndexLabel by name

Deleting an IndexLabel will result in the deletion of related index data. This operation will generate an asynchronous task.

Method & Url
DELETE http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/indexlabels/personByCity
Response Status
202
Response Body
{
    "task_id": 1
}

Note:

You can query the execution status of an asynchronous task by using GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/tasks/1 (where “1” is the task_id). For more information, refer to the Asynchronous Task RESTful API.

1.5.5 Add or remove userdata for an existing IndexLabel

Only user_data can be changed this way, base_type, base_value and index_type must be left out of the request body.

Params
  • action: Indicates whether the current action is to add or remove userdata. Possible values are append (add) and eliminate (remove).
Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/indexlabels/personByCity?action=append
Request Body
{
    "name": "personByCity",
    "user_data": {
        "comment": "index on city"
    }
}
Response Status
200
Response Body
{
    "id": 1,
    "base_type": "VERTEX_LABEL",
    "base_value": "person",
    "name": "personByCity",
    "fields": [
        "city"
    ],
    "index_type": "SECONDARY",
    "user_data": {
        "comment": "index on city",
        "~create_time": "2022-05-13 13:47:23.745"
    }
}

5.1.7 - Rebuild API

Rebuild REST API: Rebuild graph schema indexes to ensure consistency between index data and graph data.

1.6 Rebuild

1.6.1 Rebuild IndexLabel

Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/jobs/rebuild/indexlabels/personByCity
Response Status
202
Response Body
{
    "task_id": 1
}

Note:

You can get the asynchronous job status by GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/tasks/${task_id} (the task_id here should be 1). See More AsyncJob RESTfull API

1.6.2 Rebulid all Indexs of VertexLabel

Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/jobs/rebuild/vertexlabels/person
Response Status
202
Response Body
{
    "task_id": 2
}

Note:

You can get the asynchronous job status by GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/tasks/${task_id} (the task_id here should be 2). See More AsyncJob RESTfull API

1.6.3 Rebulid all Indexs of EdgeLabel

Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/jobs/rebuild/edgelabels/created
Response Status
202
Response Body
{
    "task_id": 3
}

Note:

You can get the asynchronous job status by GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/tasks/${task_id} (the task_id here should be 3). See More AsyncJob RESTfull API

5.1.8 - Vertex API

Vertex REST API: Create, query, update, and delete vertex data in the graph with support for batch operations and conditional filtering.

2.1 Vertex

In vertex types, the Id strategy determines the type of the vertex Id, with the corresponding relationships as follows:

Vertex ID strategies
Id_Strategyid type
AUTOMATICnumber
PRIMARY_KEYstring
CUSTOMIZE_STRINGstring
CUSTOMIZE_NUMBERnumber
CUSTOMIZE_UUIDuuid

For the GET/PUT/DELETE API of a vertex, the id part in the URL should be passed as the id value with type information. This type information is indicated by whether the JSON string is enclosed in quotes, meaning:

  • When the id type is number, the id in the URL is without quotes, for example: xxx/vertices/123456.
  • When the id type is string, the id in the URL is enclosed in quotes, for example: xxx/vertices/"123456".

The next example requires first creating the graph schema from the following groovy script

schema.propertyKey("name").asText().ifNotExist().create();
schema.propertyKey("age").asInt().ifNotExist().create();
schema.propertyKey("city").asText().ifNotExist().create();
schema.propertyKey("weight").asDouble().ifNotExist().create();
schema.propertyKey("lang").asText().ifNotExist().create();
schema.propertyKey("price").asDouble().ifNotExist().create();
schema.propertyKey("hobby").asText().valueList().ifNotExist().create();

schema.vertexLabel("person").properties("name", "age", "city", "weight", "hobby").primaryKeys("name").nullableKeys("age", "city", "weight", "hobby").ifNotExist().create();
schema.vertexLabel("software").properties("name", "lang", "price").primaryKeys("name").nullableKeys("lang", "price").ifNotExist().create();

schema.indexLabel("personByAge").onV("person").by("age").range().ifNotExist().create();

2.1.1 Create a vertex

Method & Url

POST http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices

Request Body

request.json
{
    "label": "person",
    "properties": {
        "name": "marko",
        "age": 29
    }
}

Response Status

201

Response Body

response.json
{
    "id": "1:marko",
    "label": "person",
    "type": "vertex",
    "properties": {
        "name": "marko",
        "age": 29
    }
}

2.1.2 Create multiple vertices

Method & Url

POST http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices/batch

Request Body

[
    {
        "label": "person",
        "properties": {
            "name": "marko",
            "age": 29
        }
    },
    {
        "label": "software",
        "properties": {
            "name": "ripple",
            "lang": "java",
            "price": 199
        }
    }
]

Response Status

201

Response Body

[
    "1:marko",
    "2:ripple"
]

2.1.3 Update vertex properties

Method & Url

PUT http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices/"1:marko"?action=append

Request Body

{
    "label": "person",
    "properties": {
        "age": 30,
        "city": "Beijing"
    }
}

Note: There are three categories for property values: single, set, and list. If it is single, it means adding or updating the property value. If it is set or list, it means appending the property value.

Response Status

200

Response Body

{
    "id": "1:marko",
    "label": "person",
    "type": "vertex",
    "properties": {
        "name": "marko",
        "age": 30,
        "city": "Beijing"
    }
}

2.1.4 Batch Update Vertex Properties

Function Description

Batch update properties of vertices and support various update strategies, including:

  • SUM: Numeric accumulation
  • BIGGER: Take the larger value between two numbers/dates
  • SMALLER: Take the smaller value between two numbers/dates
  • UNION: Take the union of set properties
  • INTERSECTION: Take the intersection of set properties
  • APPEND: Append elements to list properties
  • ELIMINATE: Remove elements from list/set properties
  • OVERRIDE: Override existing properties, if the new property is null, the old property is still used

Assuming the original vertex and properties are:

{
    "vertices": [
        {
            "id": "2:lop",
            "label": "software",
            "type": "vertex",
            "properties": {
                "name": "lop",
                "lang": "java",
                "price": 328
            }
        },
        {
            "id": "1:josh",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "josh",
                "age": 32,
                "city": "Beijing",
                "weight": 0.1,
                "hobby": [
                    "reading",
                    "football"
                ]
            }
        }
    ]
}

Add vertices with the following command:

curl -H "Content-Type: application/json" -d '[{"label":"person","properties":{"name":"josh","age":32,"city":"Beijing","weight":0.1,"hobby":["reading","football"]}},{"label":"software","properties":{"name":"lop","lang":"java","price":328}}]' http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices/batch

Method & Url

PUT http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices/batch

Request Body

{
    "vertices": [
        {
            "label": "software",
            "type": "vertex",
            "properties": {
                "name": "lop",
                "lang": "c++",
                "price": 299
            }
        },
        {
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "josh",
                "city": "Shanghai",
                "weight": 0.2,
                "hobby": [
                    "swimming"
                ]
            }
        }
    ],
    "update_strategies": {
        "price": "BIGGER",
        "age": "OVERRIDE",
        "city": "OVERRIDE",
        "weight": "SUM",
        "hobby": "UNION"
    },
    "create_if_not_exist": true
}

Response Status

200

Response Body

{
    "vertices": [
        {
            "id": "2:lop",
            "label": "software",
            "type": "vertex",
            "properties": {
                "name": "lop",
                "lang": "c++",
                "price": 328
            }
        },
        {
            "id": "1:josh",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "josh",
                "age": 32,
                "city": "Shanghai",
                "weight": 0.3,
                "hobby": [
                    "reading",
                    "football",
                    "swimming"
                ]
            }
        }
    ]
}

Result Analysis:

  • The lang property does not specify an update strategy and is directly overwritten by the new value, regardless of whether the new value is null.
  • The price property specifies the BIGGER update strategy. The old property value is 328, and the new property value is 299, so the old property value of 328 is retained.
  • The age property specifies the OVERRIDE update strategy, but the new property value does not include age, which is equivalent to age being null. Therefore, the original property value of 32 is still retained.
  • The city property also specifies the OVERRIDE update strategy, and the new property value is not null, so it overrides the old value.
  • The weight property specifies the SUM update strategy. The old property value is 0.1, and the new property value is 0.2. The final value is 0.3.
  • The hobby property (cardinality is Set) specifies the UNION update strategy, so the new value is taken as the union with the old value.

The usage of other update strategies can be inferred in a similar manner and will not be further elaborated.

2.1.5 Delete Vertex Properties

Method & Url

PUT http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices/"1:marko"?action=eliminate

Request Body

{
    "label": "person",
    "properties": {
        "city": "Beijing"
    }
}

Note: Here, the properties (keys and all values) will be directly deleted, regardless of whether the property values are single, set, or list.

Response Status

200

Response Body

{
    "id": "1:marko",
    "label": "person",
    "type": "vertex",
    "properties": {
        "name": "marko",
        "age": 30
    }
}

2.1.6 Get Vertices that Meet the Criteria

Params

  • label: Vertex type
  • properties: Property key-value pairs (precondition: indexes are created for property queries)
  • keep_start_p: Default is false. When set to true, the range matching input expression will not be automatically escaped. For example, properties={"age":"P.gt(18)"} will be interpreted as an exact match, i.e., the age property is equal to the string “P.gt(18)”
  • offset: Offset, default is 0
  • limit: Maximum number of results, default is 100
  • page: Page number

All of the above parameters are optional. page can not be combined with a non-zero offset, everything else can be combined in any way.

Property key-value pairs consist of the property name and value in JSON format. Multiple property key-value pairs are allowed as query conditions. The property value supports exact matching, range matching, and fuzzy matching. For exact matching, use the format properties={"age":29}, for range matching, use the format properties={"age":"P.gt(29)"}, and for fuzzy matching, use the format properties={"city": "P.textcontains("ChengDu China")}. The following expressions are supported for range matching:

ExpressionExplanation
P.eq(number)Vertices with property value equal to number
P.neq(number)Vertices with property value not equal to number
P.lt(number)Vertices with property value less than number
P.lte(number)Vertices with property value less than or equal to number
P.gt(number)Vertices with property value greater than number
P.gte(number)Vertices with property value greater than or equal to number
P.between(number1,number2)Vertices with property value greater than or equal to number1 and less than number2
P.inside(number1,number2)Vertices with property value greater than number1 and less than number2
P.outside(number1,number2)Vertices with property value less than number1 and greater than number2
P.within(value1,value2,value3,…)Vertices with property value equal to any of the given values

Query all vertices with age 29 and label person

Method & Url

GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices?label=person&properties={"age":29}&limit=1

Response Status

200

Response Body

{
    "vertices": [
        {
            "id": "1:marko",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "marko",
                "age": 30
            }
        }
    ]
}

Paginate through all vertices, retrieve the first page (page without parameter value), limited to 3 records

Add vertices with the following command:

curl -H "Content-Type: application/json" -d '[{"label":"person","properties":{"name":"peter","age":29,"city":"Shanghai"}},{"label":"person","properties":{"name":"vadas","age":27,"city":"Hongkong"}}]' http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices/batch

Method & Url

GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices?page&limit=3

Response Status

200

Response Body

{
    "vertices": [
        {
            "id": "2:lop",
            "label": "software",
            "type": "vertex",
            "properties": {
                "name": "lop",
                "lang": "c++",
                "price": 328
            }
        },
        {
            "id": "1:josh",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "josh",
                "age": 32,
                "city": "Shanghai",
                "weight": 0.3,
                "hobby": [
                    "reading",
                    "football",
                    "swimming"
                ]
            }
        },
        {
            "id": "1:marko",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "marko",
                "age": 30
            }
        }
    ],
    "page": "CIYxOnBldGVyAAAAAAAAAAM="
}

The returned body contains information about the page number of the next page, "page": "CIYxOnBldGVyAAAAAAAAAAM". When querying the next page, assign this value to the page parameter.

Paginate and retrieve all vertices, including the next page (passing the page value returned from the previous page), limited to 3 items.

Method & Url

GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices?page=CIYxOnBldGVyAAAAAAAAAAM=&limit=3

Response Status

200

Response Body

{
    "vertices": [
        {
            "id": "1:peter",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "peter",
                "age": 29,
                "city": "Shanghai"
            }
        },
        {
            "id": "1:vadas",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "vadas",
                "age": 27,
                "city": "Hongkong"
            }
        },
        {
            "id": "2:ripple",
            "label": "software",
            "type": "vertex",
            "properties": {
                "name": "ripple",
                "lang": "java",
                "price": 199
            }
        }
    ],
    "page": null
}

At this point, "page": null indicates that there are no more pages available. (Note: When using Cassandra as the backend for performance reasons, if the returned page happens to be the last page, the page value may not be empty. When requesting the next page using that page value, it will return empty data and page = null. The same applies to other similar situations.)

2.1.7 Retrieve Vertex by ID

Method & Url

GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices/"1:marko"

Response Status

200

Response Body

{
    "id": "1:marko",
    "label": "person",
    "type": "vertex",
    "properties": {
        "name": "marko",
        "age": 30
    }
}

2.1.8 Delete Vertex by ID

Params

  • label: Vertex type, optional parameter

Delete the vertex based on ID only.

Method & Url

DELETE http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices/"1:marko"

Response Status

204

Delete Vertex by Label+ID

When deleting a vertex by specifying both the Label parameter and the ID, it generally offers better performance compared to deleting by ID alone.

Method & Url

DELETE http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices/"1:marko"?label=person

Response Status

204

5.1.9 - Edge API

Edge REST API: Create, query, update, and delete relationship data between vertices with support for batch operations and directional queries.

2.2 Edge

The modification of the vertex ID format also affects the ID of the edge, as well as the formats of the source vertex and target vertex IDs.

The EdgeId is formed by concatenating src-vertex-id + direction + label + sort-values + tgt-vertex-id, but the vertex ID types are not distinguished by quotation marks here. Instead, they are distinguished by prefixes:

  • When the ID type is number, the vertex ID in the EdgeId has a prefix L, like “L123456>1»L987654”.
  • When the ID type is string, the vertex ID in the EdgeId has a prefix S, like “S1:peter>1»S2:lop”.

The following example requires creating a graph schema based on the following groovy script:

import org.apache.hugegraph.HugeFactory
import org.apache.tinkerpop.gremlin.structure.T

conf = "conf/graphs/hugegraph.properties"
graph = HugeFactory.open(conf)
schema = graph.schema()

schema.propertyKey("name").asText().ifNotExist().create()
schema.propertyKey("age").asInt().ifNotExist().create()
schema.propertyKey("city").asText().ifNotExist().create()
schema.propertyKey("weight").asDouble().ifNotExist().create()
schema.propertyKey("lang").asText().ifNotExist().create()
schema.propertyKey("date").asText().ifNotExist().create()
schema.propertyKey("price").asInt().ifNotExist().create()

schema.vertexLabel("person").properties("name", "age", "city").primaryKeys("name").ifNotExist().create()
schema.vertexLabel("software").properties("name", "lang", "price").primaryKeys("name").ifNotExist().create()
schema.indexLabel("personByCity").onV("person").by("city").secondary().ifNotExist().create()
schema.indexLabel("personByAgeAndCity").onV("person").by("age", "city").secondary().ifNotExist().create()
schema.indexLabel("softwareByPrice").onV("software").by("price").range().ifNotExist().create()
schema.edgeLabel("knows").sourceLabel("person").targetLabel("person").properties("date", "weight").ifNotExist().create()
schema.edgeLabel("created").sourceLabel("person").targetLabel("software").properties("date", "weight").ifNotExist().create()
schema.indexLabel("createdByDate").onE("created").by("date").secondary().ifNotExist().create()
schema.indexLabel("createdByWeight").onE("created").by("weight").range().ifNotExist().create()
schema.indexLabel("knowsByWeight").onE("knows").by("weight").range().ifNotExist().create()

marko = graph.addVertex(T.label, "person", "name", "marko", "age", 29, "city", "Beijing")
vadas = graph.addVertex(T.label, "person", "name", "vadas", "age", 27, "city", "Hongkong")
lop = graph.addVertex(T.label, "software", "name", "lop", "lang", "java", "price", 328)
josh = graph.addVertex(T.label, "person", "name", "josh", "age", 32, "city", "Beijing")
ripple = graph.addVertex(T.label, "software", "name", "ripple", "lang", "java", "price", 199)
peter = graph.addVertex(T.label, "person", "name", "peter", "age", 35, "city", "Shanghai")

graph.tx().commit()
g = graph.traversal()

2.2.1 Creating an Edge

Params

Path Parameter Description:

  • graph: The graph to operate on

Request Body Description:

  • label: The edge type name (required)
  • outV: The source vertex id (required)
  • inV: The target vertex id (required)
  • outVLabel: The source vertex type (required)
  • inVLabel: The target vertex type (required)
  • properties: The properties associated with the edge. The internal structure of the object is as follows:
    1. name: The property name
    2. value: The property value
Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/edges
Request Body
{
    "label": "created",
    "outV": "1:marko",
    "inV": "2:lop",
    "outVLabel": "person",
    "inVLabel": "software",
    "properties": {
        "date": "20171210",
        "weight": 0.4
    }
}
Response Status
201
Response Body
{
    "id": "S1:marko>2>>S2:lop",
    "label": "created",
    "type": "edge",
    "outV": "1:marko",
    "outVLabel": "person",
    "inV": "2:lop",
    "inVLabel": "software",
    "properties": {
        "weight": 0.4,
        "date": "20171210"
    }
}

2.2.2 Creating Multiple Edges

Params

Path Parameter Description:

  • graph: The graph to operate on

Request Parameter Description:

  • check_vertex: Whether to check the existence of vertices (true | false). When set to true, an error will be thrown if the source or target vertices of the edge to be inserted do not exist. Default is true.

Request Body Description:

  • List of edge information
Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/edges/batch
Request Body
[
    {
        "label": "knows",
        "outV": "1:marko",
        "inV": "1:vadas",
        "outVLabel": "person",
        "inVLabel": "person",
        "properties": {
            "date": "20160110",
            "weight": 0.5
        }
    },
    {
        "label": "knows",
        "outV": "1:marko",
        "inV": "1:josh",
        "outVLabel": "person",
        "inVLabel": "person",
        "properties": {
            "date": "20130220",
            "weight": 1.0
        }
    }
]
Response Status
201
Response Body
[
    "S1:marko>1>>S1:vadas",
    "S1:marko>1>>S1:josh"
]

2.2.3 Updating Edge Properties

Params

Path Parameter Description:

  • graph: The graph to operate on
  • id: The ID of the edge to be operated on

Request Parameter Description:

  • action: The append action

Request Body Description:

  • Edge information
Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/edges/S1:marko>2>>S2:lop?action=append
Request Body
{
    "properties": {
        "weight": 1.0
    }
}

NOTE: There are three categories of property values: single, set, and list. If it is single, it means adding or updating the property value. If it is set or list, it means appending the property value.

Response Status
200
Response Body
{
    "id": "S1:marko>2>>S2:lop",
    "label": "created",
    "type": "edge",
    "outV": "1:marko",
    "outVLabel": "person",
    "inV": "2:lop",
    "inVLabel": "software",
    "properties": {
        "weight": 1.0,
        "date": "20171210"
    }
}

2.2.4 Batch Updating Edge Properties

Params

Path Parameter Description:

  • graph: The graph to operate on

Request Body Description:

  • edges: List of edge information
  • update_strategies: For each property, you can set its update strategy individually, including:
    • SUM: Only supports number type
    • BIGGER/SMALLER: Only supports date/number type
    • UNION/INTERSECTION: Only supports set type
    • APPEND/ELIMINATE: Only supports collection type
    • OVERRIDE
  • check_vertex: Whether to check the existence of vertices (true | false). When set to true, an error will be thrown if the source or target vertices of the edge to be inserted do not exist. Default is true.
  • create_if_not_exist: Currently only supports setting to true
Method & Url
PUT http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/edges/batch
Request Body
{
    "edges": [
        {
            "label": "knows",
            "outV": "1:marko",
            "inV": "1:vadas",
            "outVLabel": "person",
            "inVLabel": "person",
            "properties": {
                "date": "20160111",
                "weight": 1.0
            }
        },
        {
            "label": "knows",
            "outV": "1:marko",
            "inV": "1:josh",
            "outVLabel": "person",
            "inVLabel": "person",
            "properties": {
                "date": "20130221",
                "weight": 0.5
            }
        }
    ],
    "update_strategies": {
        "weight": "SUM",
        "date": "OVERRIDE"
    },
    "check_vertex": false,
    "create_if_not_exist": true
}
Response Status
200
Response Body
{
    "edges": [
        {
            "id": "S1:marko>1>>S1:vadas",
            "label": "knows",
            "type": "edge",
            "outV": "1:marko",
            "outVLabel": "person",
            "inV": "1:vadas",
            "inVLabel": "person",
            "properties": {
                "weight": 1.5,
                "date": "20160111"
            }
        },
        {
            "id": "S1:marko>1>>S1:josh",
            "label": "knows",
            "type": "edge",
            "outV": "1:marko",
            "outVLabel": "person",
            "inV": "1:josh",
            "inVLabel": "person",
            "properties": {
                "weight": 1.5,
                "date": "20130221"
            }
        }
    ]
}

2.2.5 Deleting Edge Properties

Params

Path Parameter Description:

  • graph: The graph to operate on
  • id: The ID of the edge to be operated on

Request Parameter Description:

  • action: The eliminate action

Request Body Description:

  • Edge information
Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/edges/S1:marko>2>>S2:lop?action=eliminate
Request Body
{
    "properties": {
        "weight": 1.0
    }
}

NOTE: This will directly delete the properties (removing the key and all values), regardless of whether the property values are single, set, or list.

Response Status
400
Response Body

It is not possible to delete an attribute that is not set as nullable.

{
    "exception": "class java.lang.IllegalArgumentException",
    "message": "Can't remove non-null edge property 'p[weight->1.0]'",
    "cause": ""
}

2.2.6 Fetching Edges that Match the Criteria

Params

Path Parameter:

  • graph: The graph to operate on

Request Parameters:

  • vertex_id: Vertex ID
  • direction: Edge direction (OUT | IN | BOTH), default is BOTH
  • label: Edge label
  • properties: Key-value pairs of properties (requires pre-built indexes for property queries)
  • keep_start_p: Default is false. When set to true, the range matching input expression will not be automatically escaped. For example, properties={"age":"P.gt(0.8)"} will be interpreted as an exact match, i.e., the age property is equal to “P.gt(0.8)”
  • offset: Offset, default is 0
  • limit: Number of queries, default is 100
  • page: Page number

Key-value pairs of properties consist of the property name and value in JSON format. Multiple key-value pairs are allowed as query conditions. Property values support exact matching and range matching. For exact matching, it is in the form properties={"weight":0.8}. For range matching, it is in the form properties={"age":"P.gt(0.8)"}. The expressions supported by range matching are as follows:

ExpressionDescription
P.eq(number)Edges with property value equal to number
P.neq(number)Edges with property value not equal to number
P.lt(number)Edges with property value less than number
P.lte(number)Edges with property value less than or equal to number
P.gt(number)Edges with property value greater than number
P.gte(number)Edges with property value greater than or equal to number
P.between(number1,number2)Edges with property value greater than or equal to number1 and less than number2
P.inside(number1,number2)Edges with property value greater than number1 and less than number2
P.outside(number1,number2)Edges with property value less than number1 and greater than number2
P.within(value1,value2,value3,…)Edges with property value equal to any of the given values
P.textcontains(value)Edges with property value containing the given value (string type)
P.contains(value)Edges with property value containing the given value (collection type)

Edges connected to the vertex person:marko(vertex_id=“1:marko”) with label knows and date property equal to “20160111”

Method & Url
GET http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/edges?vertex_id="1:marko"&label=knows&properties={"date":"P.within(\"20160111\")"}
Response Status
200
Response Body
{
    "edges": [
        {
            "id": "S1:marko>1>>S1:vadas",
            "label": "knows",
            "type": "edge",
            "outV": "1:marko",
            "outVLabel": "person",
            "inV": "1:vadas",
            "inVLabel": "person",
            "properties": {
                "weight": 1.5,
                "date": "20160111"
            }
        }
    ]
}

Paginate and retrieve all edges, get the first page (page without parameter value), limit to 2 entries

Method & Url
GET http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/edges?page&limit=2
Response Status
200
Response Body
{
    "edges": [
        {
            "id": "S1:marko>1>>S1:josh",
            "label": "knows",
            "type": "edge",
            "outV": "1:marko",
            "outVLabel": "person",
            "inV": "1:josh",
            "inVLabel": "person",
            "properties": {
                "weight": 1.5,
                "date": "20130221"
            }
        },
        {
            "id": "S1:marko>1>>S1:vadas",
            "label": "knows",
            "type": "edge",
            "outV": "1:marko",
            "outVLabel": "person",
            "inV": "1:vadas",
            "inVLabel": "person",
            "properties": {
                "weight": 1.5,
                "date": "20160111"
            }
        }
    ],
    "page": "EoYxOm1hcmtvgggCAIQyOmxvcAAAAAAAAAAC"
}

The returned body contains the page number information for the next page, "page": "EoYxOm1hcmtvgggCAIQyOmxvcAAAAAAAAAAC". When querying the next page, assign this value to the page parameter.

Paginate and retrieve all edges, get the next page (include the page value returned from the previous page), limit to 2 entries

Method & Url
GET http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/edges?page=EoYxOm1hcmtvgggCAIQyOmxvcAAAAAAAAAAC&limit=2
Response Status
200
Response Body
{
    "edges": [
        {
            "id": "S1:marko>2>>S2:lop",
            "label": "created",
            "type": "edge",
            "outV": "1:marko",
            "outVLabel": "person",
            "inV": "2:lop",
            "inVLabel": "software",
            "properties": {
                "weight": 1.0,
                "date": "20171210"
            }
        }
    ],
    "page": null
}

When "page": null is returned, it indicates that there are no more pages available.

NOTE: When the backend is Cassandra, for performance considerations, if the returned page happens to be the last page, the page value may not be empty. When requesting the next page data using that page value, it will return empty data and page = null. Similar situations apply for other cases.

2.2.7 Fetching Edge by ID

Params

Path parameter description:

  • graph: The graph to be operated on.
  • id: The ID of the edge to be operated on.
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/edges/S1:marko>2>>S2:lop
Response Status
200
Response Body
{
    "id": "S1:marko>2>>S2:lop",
    "label": "created",
    "type": "edge",
    "outV": "1:marko",
    "outVLabel": "person",
    "inV": "2:lop",
    "inVLabel": "software",
    "properties": {
        "weight": 1.0,
        "date": "20171210"
    }
}

2.2.8 Deleting Edge by ID

Params

Path parameter description:

  • graph: The graph to be operated on.
  • id: The ID of the edge to be operated on.

Request parameter description:

  • label: The label of the edge.

Deleting Edge by ID only

Method & Url
DELETE http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/edges/S1:marko>2>>S2:lop
Response Status
204

Deleting Edge by Label + ID

In general, specifying the Label parameter along with the ID to delete an edge will provide better performance compared to deleting by ID only.

Method & Url
DELETE http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/edges/S1:marko>1>>S1:vadas?label=knows
Response Status
204

5.1.10 - Traverser API

Traverser REST API: Execute complex graph algorithms and path queries including shortest path, k-neighbors, similarity computation, and advanced analytics.

3.1 Overview of Traverser API

HugeGraphServer provides a RESTful API interface for the HugeGraph graph database. In addition to the basic CRUD operations for vertices and edges, it also offers several traversal methods, which we refer to as the traverser API. These traversal methods implement various complex graph algorithms, making it convenient for users to analyze and explore the graph.

The Traverser API supported by HugeGraph includes:

  • K-out API: It finds neighbors that are exactly N steps away from a given starting vertex. There are two versions:
    • The basic version uses the GET method to find neighbors that are exactly N steps away from a given starting vertex.
    • The advanced version uses the POST method to find neighbors that are exactly N steps away from a given starting vertex. The advanced version differs from the basic version in the following ways:
      • Supports counting the number of neighbors only
      • Supports filtering by edge and vertex properties
      • Supports returning the shortest path to reach the neighbor
  • K-neighbor API: It finds all neighbors that are within N steps of a given starting vertex. There are two versions:
    • The basic version uses the GET method to find all neighbors that are within N steps of a given starting vertex.
    • The advanced version uses the POST method to find all neighbors that are within N steps of a given starting vertex. The advanced version differs from the basic version in the following ways:
      • Supports counting the number of neighbors only
      • Supports filtering by edge and vertex properties
      • Supports returning the shortest path to reach the neighbor
  • Same Neighbors: It queries the common neighbors of two vertices.
  • Jaccard Similarity API: It calculates the Jaccard similarity, which includes two types:
    • One type uses the GET method to calculate the similarity (intersection over union) of neighbors between two vertices.
    • The other type uses the POST method to find the top N vertices with the highest Jaccard similarity to a given starting vertex in the entire graph.
  • Shortest Path API: It finds the shortest path between two vertices.
  • All Shortest Paths: It finds all shortest paths between two vertices.
  • Weighted Shortest Path: It finds the shortest weighted path from a starting vertex to a target vertex.
  • Single Source Shortest Path: It finds the weighted shortest path from a single source vertex to all other vertices.
  • Multi Node Shortest Path: It finds the shortest path between every pair of specified vertices.
  • Paths API: It finds all paths between two vertices. There are two versions:
    • The basic version uses the GET method to find all paths between a given starting vertex and an ending vertex.
    • The advanced version uses the POST method to find all paths that meet certain conditions between a set of starting vertices and a set of ending vertices.
  • Customized Paths API: It traverses all paths that pass through a batch of vertices according to a specific pattern.
  • Template Path API: It specifies a starting point, an ending point, and the path information between them to find matching paths.
  • Crosspoints API: It finds the intersection (common ancestors or common descendants) between two vertices.
  • Customized Crosspoints API: It traverses multiple patterns starting from a batch of vertices and finds the intersections with the vertices reached in the final step.
  • Rings API: It finds the cyclic paths that can be reached from a starting vertex.
  • Rays API: It finds the paths from a starting vertex that reach the boundaries (i.e., paths without cycles).
  • Fusiform Similarity API: It finds the fusiform similar vertices to a given vertex.
  • Adamic-Adar API: It computes the Adamic-Adar index of two vertices.
  • Resource Allocation API: It computes the resource allocation index of two vertices.
  • Edge Existence API: It returns the edges that exist between two given vertices.
  • Count API: It counts the vertices reached after a series of traversal steps, without returning them.
  • Vertices API:
    • Batch querying vertices by ID.
    • Getting the partitions of vertices.
    • Querying vertices by partition.
  • Edges API:
    • Batch querying edges by ID.
    • Getting the partitions of edges.
    • Querying edges by partition.

3.2 Detailed Explanation of Traverser API

The usage examples provided in this section are based on the graph presented on the TinkerPop official website:

TinkerPop example graph

The data import program is as follows:

public class Loader {
    public static void main(String[] args) {
        HugeClient client = new HugeClient("http://127.0.0.1:8080", "hugegraph");
        SchemaManager schema = client.schema();
        schema.propertyKey("name").asText().ifNotExist().create();
        schema.propertyKey("age").asInt().ifNotExist().create();
        schema.propertyKey("city").asText().ifNotExist().create();
        schema.propertyKey("weight").asDouble().ifNotExist().create();
        schema.propertyKey("lang").asText().ifNotExist().create();
        schema.propertyKey("date").asText().ifNotExist().create();
        schema.propertyKey("price").asInt().ifNotExist().create();

        schema.vertexLabel("person")
              .properties("name", "age", "city")
              .primaryKeys("name")
              .nullableKeys("age")
              .ifNotExist()
              .create();

        schema.vertexLabel("software")
              .properties("name", "lang", "price")
              .primaryKeys("name")
              .nullableKeys("price")
              .ifNotExist()
              .create();

        schema.indexLabel("personByCity")
              .onV("person")
              .by("city")
              .secondary()
              .ifNotExist()
              .create();

        schema.indexLabel("personByAgeAndCity")
              .onV("person")
              .by("age", "city")
              .secondary()
              .ifNotExist()
              .create();

        schema.indexLabel("softwareByPrice")
              .onV("software")
              .by("price")
              .range()
              .ifNotExist()
              .create();

        schema.edgeLabel("knows")
              .multiTimes()
              .sourceLabel("person")
              .targetLabel("person")
              .properties("date", "weight")
              .sortKeys("date")
              .nullableKeys("weight")
              .ifNotExist()
              .create();

        schema.edgeLabel("created")
              .sourceLabel("person").targetLabel("software")
              .properties("date", "weight")
              .nullableKeys("weight")
              .ifNotExist()
              .create();

        schema.indexLabel("createdByDate")
              .onE("created")
              .by("date")
              .secondary()
              .ifNotExist()
              .create();

        schema.indexLabel("createdByWeight")
              .onE("created")
              .by("weight")
              .range()
              .ifNotExist()
              .create();

        schema.indexLabel("knowsByWeight")
              .onE("knows")
              .by("weight")
              .range()
              .ifNotExist()
              .create();

        GraphManager graph = client.graph();
        Vertex marko = graph.addVertex(T.label, "person", "name", "marko",
                                       "age", 29, "city", "Beijing");
        Vertex vadas = graph.addVertex(T.label, "person", "name", "vadas",
                                       "age", 27, "city", "Hongkong");
        Vertex lop = graph.addVertex(T.label, "software", "name", "lop",
                                     "lang", "java", "price", 328);
        Vertex josh = graph.addVertex(T.label, "person", "name", "josh",
                                      "age", 32, "city", "Beijing");
        Vertex ripple = graph.addVertex(T.label, "software", "name", "ripple",
                                        "lang", "java", "price", 199);
        Vertex peter = graph.addVertex(T.label, "person", "name", "peter",
                                       "age", 35, "city", "Shanghai");

        marko.addEdge("knows", vadas, "date", "20160110", "weight", 0.5);
        marko.addEdge("knows", josh, "date", "20130220", "weight", 1.0);
        marko.addEdge("created", lop, "date", "20171210", "weight", 0.4);
        josh.addEdge("created", lop, "date", "20091111", "weight", 0.4);
        josh.addEdge("created", ripple, "date", "20171210", "weight", 1.0);
        peter.addEdge("created", lop, "date", "20170324", "weight", 0.2);
    }
}

The vertex IDs are:

"2:ripple",
"1:vadas",
"1:peter",
"1:josh",
"1:marko",
"2:lop"

The edge IDs are:

"S1:peter>2>>S2:lop",
"S1:josh>2>>S2:lop",
"S1:josh>2>>S2:ripple",
"S1:marko>1>20130220>S1:josh",
"S1:marko>1>20160110>S1:vadas",
"S1:marko>2>>S2:lop"

3.2.1 K-out API (GET, Basic Version)

3.2.1.1 Functionality Overview

The K-out API allows you to find vertices that are exactly “depth” steps away from a given starting vertex, considering the specified direction, edge type (optional), and depth.

Params
  • source: ID of the starting vertex (required)
  • direction: Direction of traversal from the starting vertex (OUT, IN, BOTH). Optional, default is BOTH.
  • max_depth: Number of steps (required)
  • label: Edge type (optional), represents all edge labels by default
  • nearest: When nearest is set to true, it means the shortest path length from the starting vertex to the result vertices is equal to the depth, and there is no shorter path. When nearest is set to false, it means there is at least one path of length depth from the starting vertex to the result vertices (not necessarily the shortest and may contain cycles). Optional, default is true.
  • max_degree: Maximum number of adjacent edges to traverse per vertex during the query. Optional, default is 10000.
  • capacity: Maximum number of vertices to be visited during the traversal. Optional, default is 10000000.
  • limit: Maximum number of vertices to be returned. Optional, default is 10000000.
3.2.1.2 Usage Example
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/traversers/kout?source="1:marko"&max_depth=2
Response Status
200
Response Body
{
    "vertices":[
        "2:ripple",
        "1:peter"
    ]
}
3.2.1.3 Use Cases

Finding vertices that are exactly N steps away in a relationship. Two examples:

  • In a family relationship, finding all grandchildren of a person. The set of vertices that can be reached by person A through two consecutive “son” edges.
  • Discovering potential friends in a social network. For example, finding users who are two degrees of friendship away from the target user, reachable through two consecutive “friend” edges.

3.2.2 K-out API (POST, Advanced Version)

3.2.2.1 Functionality Overview

The K-out API allows you to find vertices that are exactly “depth” steps away from a given starting vertex, considering the specified steps (including direction, edge type, and attribute filtering).

The advanced version differs from the basic version of K-out API in the following aspects:

  • Supports counting the number of neighbors only
  • Supports edge attribute filtering
  • Supports returning the shortest path to the neighbor
Params
  • source: The ID of the starting vertex, required.
  • steps: Steps from the starting point, required, with the following structure:
    • direction: Represents the direction of the edges (OUT, IN, BOTH), default is BOTH.
    • edge_steps: The step set of edges, supporting label and properties filtering for the edge. If edge_steps is empty, the edge is not filtered.
      • label: Edge types.
      • properties: Filter edges based on property values.
    • vertex_steps: The step set of vertices, supporting label and properties filtering for the vertex. If vertex_steps is empty, the vertex is not filtered.
      • label: Vertex types.
      • properties: Filter vertices based on property values.
    • max_degree: Maximum adjacent edges traversed per vertex, defaulting to 10000; the parameter name degree is also accepted.
    • skip_degree: Sets the minimum number of edges to skip super vertices during the query process. If the number of adjacent edges for a vertex is greater than skip_degree, the vertex is completely skipped. Optional. If enabled, it should satisfy the constraint skip_degree >= max_degree. Default is 0 (not enabled), indicating no skipping of any vertices (Note: Enabling this configuration means that during traversal, an attempt will be made to access skip_degree edges of a vertex, not just max_degree edges. This incurs additional traversal overhead and may have a significant impact on query performance. Please enable it only after understanding the implications).
  • max_depth: Number of steps, required.
  • nearest: When nearest is true, it means the shortest path length from the starting vertex to the result vertex is equal to depth, and there is no shorter path. When nearest is false, it means there is a path of length depth from the starting vertex to the result vertex (not necessarily the shortest and can contain cycles). Optional, default is true.
  • count_only: Boolean value, true indicates only counting the number of results without returning specific results, false indicates returning specific results. Default is false.
  • with_path: When true, it returns the shortest path from the starting vertex to each neighbor. When false, it does not return the shortest path. Optional, default is false.
  • with_edge: Optional parameter, default is false:
    • When true, the result will include complete edge information (all edges in the path):
      • When with_path is true, it returns complete information of all edges in all paths.
      • When with_path is false, no information is returned.
    • When false, it only returns edge IDs.
  • with_vertex: Optional parameter, default is false:
    • When true, the result will include complete vertex information (all vertices in the path):
      • When with_path is true, it returns complete information of all vertices in all paths.
      • When with_path is false, it returns complete information of all neighbors.
    • When false, it only returns vertex IDs.
  • capacity: Maximum number of vertices to visit during traversal. Optional, default is 10000000.
  • limit: Maximum number of vertices to return. Optional, default is 10000000.
  • traverse_mode: Traversal mode. There are two options: “breadth_first_search” and “depth_first_search”, default is “breadth_first_search”.
3.2.2.2 Usage
Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/traversers/kout
Request Body
{
    "source": "1:marko",
    "steps": {
        "direction": "BOTH",
        "edge_steps": [
            {
                "label": "knows",
                "properties": {
                    "weight": "P.gt(0.1)"
                }
            },
            {
                "label": "created",
                "properties": {
                    "weight": "P.gt(0.1)"
                }
            }
        ],
        "vertex_steps": [
            {
                "label": "person",
                "properties": {
                    "age": "P.lt(32)"
                }
            },
            {
                "label": "software",
                "properties": {}
            }
        ],
        "max_degree": 10000,
        "skip_degree": 100000
    },
    "max_depth": 1,
    "nearest": true,
    "limit": 10000,
    "with_vertex": true,
    "with_path": true,
    "with_edge": true
}
Response Status
200
Response Body
{
    "size": 2,
    "kout": [
        "1:vadas",
        "2:lop"
    ],
    "paths": [
        {
            "objects": [
                "1:marko",
                "2:lop"
            ]
        },
        {
            "objects": [
                "1:marko",
                "1:vadas"
            ]
        }
    ],
    "vertices": [
        {
            "id": "1:marko",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "marko",
                "age": 29,
                "city": "Beijing"
            }
        },
        {
            "id": "1:vadas",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "vadas",
                "age": 27,
                "city": "Hongkong"
            }
        },
        {
            "id": "2:lop",
            "label": "software",
            "type": "vertex",
            "properties": {
                "name": "lop",
                "lang": "java",
                "price": 328
            }
        }
    ],
    "edges": [
        {
            "id": "S1:marko>1>20160110>S1:vadas",
            "label": "knows",
            "type": "edge",
            "outV": "1:marko",
            "outVLabel": "person",
            "inV": "1:vadas",
            "inVLabel": "person",
            "properties": {
                "weight": 0.5,
                "date": "20160110"
            }
        },
        {
            "id": "S1:marko>2>>S2:lop",
            "label": "created",
            "type": "edge",
            "outV": "1:marko",
            "outVLabel": "person",
            "inV": "2:lop",
            "inVLabel": "software",
            "properties": {
                "weight": 0.4,
                "date": "20171210"
            }
        }
    ]
}
3.2.2.3 Use Cases

Refer to 3.2.1.3.

3.2.3 K-neighbor (GET, Basic Version)

3.2.3.1 Function Introduction

Find all vertices that are reachable within depth steps, including the starting vertex, based on the starting vertex, direction, edge type (optional), and depth.

Equivalent to the union of: starting vertex, K-out(1), K-out(2), …, K-out(max_depth).

Params
  • source: ID of the starting vertex, required.
  • direction: Direction in which the starting vertex’s edges extend (OUT, IN, BOTH). Optional, default is BOTH.
  • max_depth: Number of steps, required.
  • label: Edge type, optional, default represents all edge labels.
  • max_degree: Maximum number of adjacent edges to traverse for a single vertex during the query process. Optional, default is 10000.
  • limit: Maximum number of vertices to return, also represents the maximum number of vertices to visit during traversal. Optional, default is 10000000.
3.2.3.2 Usage
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/traversers/kneighbor?source=%221:marko%22&max_depth=2
Response Status
200
Response Body
{
    "vertices":[
        "2:ripple",
        "1:marko",
        "1:josh",
        "1:vadas",
        "1:peter",
        "2:lop"
    ]
}
3.2.3.3 Use Cases

Find all vertices reachable within N steps, for example:

  • In a family relationship, find all descendants within five generations of a person. This can be achieved by traversing five consecutive “parent-child” edges from person A.
  • In a social network, discover friend circles. For example, users who can be reached by 1, 2, or 3 “friend” edges from the target user can form the target user’s friend circle.

3.2.4 K-neighbor API (POST, Advanced Version)

3.2.4.1 Function Introduction

Find all vertices that are reachable within depth steps from the starting vertex, based on the starting vertex, steps (including direction, edge type, and filter properties), and depth.

The difference from the Basic Version of K-neighbor API is that:

  • It supports counting the number of neighbors only.
  • It supports filtering edges based on their properties.
  • It supports returning the shortest path to reach the neighbors.
Params
  • source: Starting vertex ID, required.
  • steps: Steps from the starting point, required, with the following structure:
    • direction: Represents the direction of the edges (OUT, IN, BOTH), default is BOTH.
    • edge_steps: The step set of edges, supporting label and properties filtering for the edge. If edge_steps is empty, the edge is not filtered.
      • label: Edge types.
      • properties: Filter edges based on property values.
    • vertex_steps: The step set of vertices, supporting label and properties filtering for the vertex. If vertex_steps is empty, the vertex is not filtered.
      • label: Vertex types.
      • properties: Filter vertices based on property values.
    • max_degree: Maximum adjacent edges traversed per vertex, defaulting to 10000; the parameter name degree is also accepted.
    • skip_degree: Used to set the minimum number of edges to discard super vertices during the query process. When the number of adjacent edges for a vertex exceeds skip_degree, the vertex is completely discarded. This is an optional parameter. If enabled, it should satisfy the constraint skip_degree >= max_degree. Default is 0 (not enabled), which means no vertices are skipped. (Note: When this configuration is enabled, the traversal will attempt to access skip_degree edges for each vertex, not just max_degree edges. This incurs additional traversal overhead and may significantly impact query performance. Please make sure to understand this before enabling.)
  • max_depth: Number of steps, required.
  • count_only: Boolean value. If true, only the count of results is returned without the actual results. If false, the specific results are returned. Default is false.
  • with_path: If true, the shortest path from the starting point to each neighbor is returned. If false, the shortest path from the starting point to each neighbor is not returned. This is an optional parameter. Default is false.
  • with_edge: Optional parameter, default is false:
    • When true, the result will include complete edge information (all edges in the path):
      • When with_path is true, it returns complete information of all edges in all paths.
      • When with_path is false, no information is returned.
    • When false, it only returns edge IDs.
  • with_vertex: Optional parameter, default is false:
    • When true, the result will include complete vertex information (all vertices in the path):
      • When with_path is true, it returns complete information of all vertices in all paths.
      • When with_path is false, it returns complete information of all neighbors.
    • When false, it only returns vertex IDs.
  • limit: Maximum number of vertices to be returned. Also, the maximum number of vertices visited during the traversal process. This is an optional parameter. Default is 10000000.
3.2.4.2 Usage Method
Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/traversers/kneighbor
Request Body
{
    "source": "1:marko",
    "steps": {
        "direction": "BOTH",
        "edge_steps": [
            {
                "label": "knows",
                "properties": {
                    "weight": "P.gt(0.1)"
                }
            },
            {
                "label": "created",
                "properties": {
                    "weight": "P.gt(0.1)"
                }
            }
        ],
        "vertex_steps": [
            {
                "label": "person",
                "properties": {
                    "age": "P.lt(32)"
                }
            },
            {
                "label": "software",
                "properties": {}
            }
        ],
        "max_degree": 10000,
        "skip_degree": 100000
    },
    "max_depth": 1,
    "nearest": true,
    "limit": 10000,
    "with_vertex": true,
    "with_path": true,
    "with_edge": true
}
Response Status
200
Response Body
{
    "size": 4,
    "kneighbor": [
        "1:josh",
        "2:lop",
        "1:peter",
        "2:ripple"
    ],
    "paths": [
        {
            "objects": [
                "1:marko",
                "2:lop"
            ]
        },
        {
            "objects": [
                "1:marko",
                "2:lop",
                "1:peter"
            ]
        },
        {
            "objects": [
                "1:marko",
                "1:josh"
            ]
        },
        {
            "objects": [
                "1:marko",
                "1:josh",
                "2:ripple"
            ]
        }
    ],
    "vertices": [
        {
            "id": "2:ripple",
            "label": "software",
            "type": "vertex",
            "properties": {
                "name": "ripple",
                "lang": "java",
                "price": 199
            }
        },
        {
            "id": "1:marko",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "marko",
                "age": 29,
                "city": "Beijing"
            }
        },
        {
            "id": "1:josh",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "josh",
                "age": 32,
                "city": "Beijing"
            }
        },
        {
            "id": "1:peter",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "peter",
                "age": 35,
                "city": "Shanghai"
            }
        },
        {
            "id": "2:lop",
            "label": "software",
            "type": "vertex",
            "properties": {
                "name": "lop",
                "lang": "java",
                "price": 328
            }
        }
    ],
    "edges": [
        {
            "id": "S1:josh>2>>S2:ripple",
            "label": "created",
            "type": "edge",
            "outV": "1:josh",
            "outVLabel": "person",
            "inV": "2:ripple",
            "inVLabel": "software",
            "properties": {
                "weight": 1.0,
                "date": "20171210"
            }
        },
        {
            "id": "S1:marko>2>>S2:lop",
            "label": "created",
            "type": "edge",
            "outV": "1:marko",
            "outVLabel": "person",
            "inV": "2:lop",
            "inVLabel": "software",
            "properties": {
                "weight": 0.4,
                "date": "20171210"
            }
        },
        {
            "id": "S1:marko>1>20130220>S1:josh",
            "label": "knows",
            "type": "edge",
            "outV": "1:marko",
            "outVLabel": "person",
            "inV": "1:josh",
            "inVLabel": "person",
            "properties": {
                "weight": 1.0,
                "date": "20130220"
            }
        },
        {
            "id": "S1:peter>2>>S2:lop",
            "label": "created",
            "type": "edge",
            "outV": "1:peter",
            "outVLabel": "person",
            "inV": "2:lop",
            "inVLabel": "software",
            "properties": {
                "weight": 0.2,
                "date": "20170324"
            }
        }
    ]
}
3.2.4.3 Use Cases

See 3.2.3.3

3.2.5 Same Neighbors

3.2.5.1 Function Introduction

Retrieve the common neighbors of two vertices.

Params
  • vertex: ID of one vertex, required.
  • other: ID of another vertex, required.
  • direction: Direction in which the vertex expands outward (OUT, IN, BOTH). Optional, default is BOTH.
  • label: Edge type. Optional, default represents all edge labels.
  • max_degree: Maximum number of adjacent edges to traverse for each vertex during the query process. Optional, default is 10000.
  • limit: Maximum number of common neighbors to be returned. Optional, default is 10000000.
3.2.5.2 Usage Method
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/traversers/sameneighbors?vertex=%221:marko%22&other=%221:josh%22
Response Status
200
Response Body
{
    "same_neighbors":[
        "2:lop"
    ]
}
3.2.5.3 Use Cases

Find the common neighbors of two vertices:

  • In a social network, find the common followers or users both users are following.

3.2.6 Jaccard Similarity (GET)

3.2.6.1 Function Introduction

Compute the Jaccard similarity between two vertices (the intersection of the neighbors of the two vertices divided by the union of the neighbors of the two vertices).

Params
  • vertex: ID of one vertex, required.
  • other: ID of another vertex, required.
  • direction: Direction in which the vertex expands outward (OUT, IN, BOTH). Optional, default is BOTH.
  • label: Edge type. Optional, default represents all edge labels.
  • max_degree: Maximum number of adjacent edges to traverse for each vertex during the query process. Optional, default is 10000.
3.2.6.2 Usage Method
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/traversers/jaccardsimilarity?vertex="1:marko"&other="1:josh"
Response Status
200
Response Body
{
    "jaccard_similarity": 0.2
}
3.2.6.3 Use Cases

Used to evaluate the similarity or closeness between two vertices.

3.2.7 Jaccard Similarity (POST)

3.2.7.1 Function Introduction

Compute the N vertices with the highest Jaccard similarity to a specified vertex.

The Jaccard similarity is calculated as the intersection of the neighbors of the two vertices divided by the union of the neighbors of the two vertices.

Params
  • vertex: ID of a vertex, required.
  • Steps from the starting point, required. The structure is as follows:
    • direction: Direction of the edges (OUT, IN, BOTH). Optional, default is BOTH.
    • labels: List of edge types.
    • properties: Filter edges based on property values.
    • max_degree: Maximum adjacent edges traversed per vertex, defaulting to 10000; the parameter name degree is also accepted.
    • skip_degree: Used to set the minimum number of edges to skip super vertices during the query process. If the number of adjacent edges for a vertex is greater than skip_degree, the vertex is completely skipped. Optional, default is 0 (not enabled), which means no skipping. (Note: When this configuration is enabled, the traversal will attempt to access skip_degree edges of a vertex, not just max_degree edges. This incurs additional traversal overhead and may have a significant impact on query performance. Please enable it after understanding and confirming.)
  • top: Return the top N vertices with the highest Jaccard similarity for a starting vertex. Optional, default is 100.
  • capacity: Maximum number of vertices to be visited during the traversal process. Optional, default is 10000000.
3.2.7.2 Usage Method
Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/traversers/jaccardsimilarity
Request Body
{
  "vertex": "1:marko",
  "step": {
    "direction": "BOTH",
    "labels": [],
    "max_degree": 10000,
    "skip_degree": 100000
  },
  "top": 3
}
Response Status
200
Response Body
{
    "2:ripple": 0.3333333333333333,
    "1:peter": 0.3333333333333333,
    "1:josh": 0.2
}
3.2.7.3 Use Cases

Used to find the vertices in the graph that have the highest similarity to a specified vertex.

3.2.8 Shortest Path

3.2.8.1 Function Introduction

Find the shortest path between a starting vertex and a target vertex based on the direction, edge type (optional), and maximum depth.

Params
  • source: ID of the starting vertex, required.
  • target: ID of the target vertex, required.
  • direction: Direction in which the starting vertex expands (OUT, IN, BOTH). Optional, default is BOTH.
  • max_depth: Maximum number of steps, required.
  • label: Edge type, optional. Default represents all edge labels.
  • max_degree: Maximum number of adjacent edges to traverse for each vertex during the query process. Optional, default is 10000.
  • skip_degree: Used to set the minimum number of edges to skip super vertices during the query process. If the number of adjacent edges for a vertex is greater than skip_degree, the vertex is completely skipped. Optional, default is 0 (not enabled), which means no skipping. (Note: When this configuration is enabled, the traversal will attempt to access skip_degree edges of a vertex, not just max_degree edges. This incurs additional traversal overhead and may have a significant impact on query performance. Please enable it after understanding and confirming.)
  • capacity: Maximum number of vertices to be visited during the traversal process. Optional, default is 10000000.
3.2.8.2 Usage Method
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/traversers/shortestpath?source="1:marko"&target="2:ripple"&max_depth=3
Response Status
200
Response Body
{
    "path":[
        "1:marko",
        "1:josh",
        "2:ripple"
    ]
}
3.2.8.3 Use Cases

Used to find the shortest path between two vertices, for example:

  • In a social network, finding the shortest path between two users, representing the closest friend relationship chain.
  • In a device association network, finding the shortest association relationship between two devices.

3.2.9 All Shortest Paths

3.2.9.1 Function Introduction

Find all shortest paths between a starting vertex and a target vertex based on the direction, edge type (optional), and maximum depth.

Params
  • source: ID of the starting vertex, required.
  • target: ID of the target vertex, required.
  • direction: Direction in which the starting vertex expands (OUT, IN, BOTH). Optional, default is BOTH.
  • max_depth: Maximum number of steps, required.
  • label: Edge type, optional. Default represents all edge labels.
  • max_degree: Maximum number of adjacent edges to traverse for each vertex during the query process. Optional, default is 10000.
  • skip_degree: Used to set the minimum number of edges to skip super vertices during the query process. If the number of adjacent edges for a vertex is greater than skip_degree, the vertex is completely skipped. Optional, default is 0 (not enabled), which means no skipping. (Note: When this configuration is enabled, the traversal will attempt to access skip_degree edges of a vertex, not just max_degree edges. This incurs additional traversal overhead and may have a significant impact on query performance. Please enable it after understanding and confirming.)
  • capacity: Maximum number of vertices to be visited during the traversal process. Optional, default is 10000000.
3.2.9.2 Usage Method
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/traversers/allshortestpaths?source="A"&target="Z"&max_depth=10
Response Status
200
Response Body
{
    "paths":[
        {
            "objects": [
                "A",
                "B",
                "C",
                "Z"
            ]
        },
        {
            "objects": [
                "A",
                "M",
                "N",
                "Z"
            ]
        }
    ]
}
3.2.9.3 Use Cases

Used to find all shortest paths between two vertices, for example:

  • In a social network, finding all shortest paths between two users, representing all the closest friend relationship chains.
  • In a device association network, finding all shortest association relationships between two devices.

3.2.10 Weighted Shortest Path

3.2.10.1 Function Introduction

Find a weighted shortest path between a starting vertex and a target vertex based on the direction, edge type (optional), maximum depth, and edge weight property.

Params
  • source: ID of the starting vertex, required.
  • target: ID of the target vertex, required.
  • direction: Direction in which the starting vertex expands (OUT, IN, BOTH). Optional, default is BOTH.
  • label: Edge type, optional. Default represents all edge labels.
  • weight: Edge weight property, required. It must be a numeric property.
  • max_degree: Maximum number of adjacent edges to traverse for each vertex during the query process. Optional, default is 10000.
  • skip_degree: Used to set the minimum number of edges to skip super vertices during the query process. If the number of adjacent edges for a vertex is greater than skip_degree, the vertex is completely skipped. Optional, default is 0 (not enabled), which means no skipping. (Note: When this configuration is enabled, the traversal will attempt to access skip_degree edges of a vertex, not just max_degree edges. This incurs additional traversal overhead and may have a significant impact on query performance. Please enable it after understanding and confirming.)
  • capacity: Maximum number of vertices to be visited during the traversal process. Optional, default is 10000000.
  • with_vertex: true to include complete vertex information (all vertices in the path) in the result, false to only return vertex IDs. Optional, default is false.
3.2.10.2 Usage Method
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/traversers/weightedshortestpath?source="1:marko"&target="2:ripple"&weight="weight"&with_vertex=true
Response Status
200
Response Body
{
    "path": {
        "weight": 2.0,
        "vertices": [
            "1:marko",
            "1:josh",
            "2:ripple"
        ]
    },
    "vertices": [
        {
            "id": "1:marko",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "marko",
                "age": 29,
                "city": "Beijing"
            }
        },
        {
            "id": "1:josh",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "josh",
                "age": 32,
                "city": "Beijing"
            }
        },
        {
            "id": "2:ripple",
            "label": "software",
            "type": "vertex",
            "properties": {
                "name": "ripple",
                "lang": "java",
                "price": 199
            }
        }
    ]
}
3.2.10.3 Use Cases

Used to find the weighted shortest path between two vertices, for example:

  • In a transportation network, finding the transportation method that requires the least cost from city A to city B.

3.2.11 Single Source Shortest Path

3.2.11.1 Function Introduction

Starting from a vertex, find the shortest paths from that vertex to other vertices in the graph (optional with weight).

Params
  • source: ID of the starting vertex, required.
  • direction: Direction in which the starting vertex expands (OUT, IN, BOTH). Optional, default is BOTH.
  • label: Edge type, optional. Default represents all edge labels.
  • weight: Edge weight property, optional. It must be a numeric property. If not provided or the edges don’t have this property, the weight is considered as 1.0.
  • max_degree: Maximum number of adjacent edges to traverse for each vertex during the query process. Optional, default is 10000.
  • skip_degree: Used to set the minimum number of edges to skip super vertices during the query process. If the number of adjacent edges for a vertex is greater than skip_degree, the vertex is completely skipped. Optional, default is 0 (not enabled), which means no skipping. (Note: When this configuration is enabled, the traversal will attempt to access skip_degree edges of a vertex, not just max_degree edges. This incurs additional traversal overhead and may have a significant impact on query performance. Please enable it after understanding and confirming.)
  • capacity: Maximum number of vertices to be visited during the traversal process. Optional, default is 10000000.
  • limit: Number of target vertices to be queried and the number of shortest paths to be returned. Optional, default is 10.
  • with_vertex: true to include complete vertex information (all vertices in the path) in the result, false to only return vertex IDs. Optional, default is false.
3.2.11.2 Usage Method
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/traversers/singlesourceshortestpath?source="1:marko"&with_vertex=true
Response Status
200
Response Body
{
    "paths": {
        "2:ripple": {
            "weight": 2.0,
            "vertices": [
                "1:marko",
                "1:josh",
                "2:ripple"
            ]
        },
        "1:josh": {
            "weight": 1.0,
            "vertices": [
                "1:marko",
                "1:josh"
            ]
        },
        "1:vadas": {
            "weight": 1.0,
            "vertices": [
                "1:marko",
                "1:vadas"
            ]
        },
        "1:peter": {
            "weight": 2.0,
            "vertices": [
                "1:marko",
                "2:lop",
                "1:peter"
            ]
        },
        "2:lop": {
            "weight": 1.0,
            "vertices": [
                "1:marko",
                "2:lop"
            ]
        }
    },
    "vertices": [
        {
            "id": "2:ripple",
            "label": "software",
            "type": "vertex",
            "properties": {
                "name": "ripple",
                "lang": "java",
                "price": 199
            }
        },
        {
            "id": "1:marko",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "marko",
                "age": 29,
                "city": "Beijing"
            }
        },
        {
            "id": "1:josh",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "josh",
                "age": 32,
                "city": "Beijing"
            }
        },
        {
            "id": "1:vadas",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "vadas",
                "age": 27,
                "city": "Hongkong"
            }
        },
        {
            "id": "1:peter",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "peter",
                "age": 35,
                "city": "Shanghai"
            }
        },
        {
            "id": "2:lop",
            "label": "software",
            "type": "vertex",
            "properties": {
                "name": "lop",
                "lang": "java",
                "price": 328
            }
        }
    ]
}
3.2.11.3 Use Cases

Used to find the weighted shortest path from one vertex to other vertices, for example:

  • Finding the shortest travel time by bus from Beijing to all other cities in the country.

3.2.12 Multi Node Shortest Path

3.2.12.1 Function Introduction

Finds the shortest paths between pairs of specified vertices.

Params
  • vertices: Defines the starting vertices, required. It can be specified in the following ways:
    • ids: Provide a list of vertex IDs as starting vertices.
    • label and properties: If no IDs are specified, use the combined conditions of label and properties to query the starting vertices.
      • label: Vertex type.
      • properties: Query the starting vertices based on property values.

      Note: Property values in properties can be a list, indicating that the value of the key can be any value in the list.

  • step: Represents the path from the starting vertices to the destination vertices, required. The structure of the step is as follows:
    • direction: Represents the direction of the edges (OUT, IN, BOTH). Default is BOTH.
    • labels: List of edge types.
    • properties: Filters the edges based on property values.
    • max_degree: Maximum adjacent edges traversed per vertex, defaulting to 10000; the parameter name degree is also accepted.
    • skip_degree: Used to set the minimum number of edges to skip super vertices during the query process. If the number of adjacent edges for a vertex is greater than skip_degree, the vertex is completely skipped. Optional, default is 0 (not enabled), which means no skipping. (Note: When this configuration is enabled, the traversal will attempt to access skip_degree edges of a vertex, not just max_degree edges. This incurs additional traversal overhead and may have a significant impact on query performance. Please enable it after understanding and confirming.)
  • max_depth: Number of steps, required.
  • capacity: Maximum number of vertices to be visited during the traversal process. Optional, default is 10000000.
  • with_vertex: true to include complete vertex information (all vertices in the path) in the result, false to only return vertex IDs. Optional, default is false.
3.2.12.2 Usage Method
Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/traversers/multinodeshortestpath
Request Body
{
    "vertices": {
        "ids": ["382:marko", "382:josh", "382:vadas", "382:peter", "383:lop", "383:ripple"]
    },
    "step": {
        "direction": "BOTH",
        "properties": {
        }
    },
    "max_depth": 10,
    "capacity": 100000000,
    "with_vertex": true
}
Response Status
200
Response Body
{
    "paths": [
        {
            "objects": [
                "382:peter",
                "383:lop"
            ]
        },
        {
            "objects": [
                "382:peter",
                "383:lop",
                "382:marko"
            ]
        },
        {
            "objects": [
                "382:peter",
                "383:lop",
                "382:josh"
            ]
        },
        {
            "objects": [
                "382:peter",
                "383:lop",
                "382:marko",
                "382:vadas"
            ]
        },
        {
            "objects": [
                "383:lop",
                "382:marko"
            ]
        },
        {
            "objects": [
                "383:lop",
                "382:josh"
            ]
        },
        {
            "objects": [
                "383:lop",
                "382:marko",
                "382:vadas"
            ]
        },
        {
            "objects": [
                "382:peter",
                "383:lop",
                "382:josh",
                "383:ripple"
            ]
        },
        {
            "objects": [
                "382:marko",
                "382:josh"
            ]
        },
        {
            "objects": [
                "383:lop",
                "382:josh",
                "383:ripple"
            ]
        },
        {
            "objects": [
                "382:marko",
                "382:vadas"
            ]
        },
        {
            "objects": [
                "382:marko",
                "382:josh",
                "383:ripple"
            ]
        },
        {
            "objects": [
                "382:josh",
                "383:ripple"
            ]
        },
        {
            "objects": [
                "382:josh",
                "382:marko",
                "382:vadas"
            ]
        },
        {
            "objects": [
                "382:vadas",
                "382:marko",
                "382:josh",
                "383:ripple"
            ]
        }
    ],
    "vertices": [
        {
            "id": "382:peter",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "peter",
                "age": 29,
                "city": "Shanghai"
            }
        },
        {
            "id": "383:lop",
            "label": "software",
            "type": "vertex",
            "properties": {
                "name": "lop",
                "lang": "java",
                "price": 328
            }
        },
        {
            "id": "382:marko",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "marko",
                "age": 29,
                "city": "Beijing"
            }
        },
        {
            "id": "382:josh",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "josh",
                "age": 32,
                "city": "Beijing"
            }
        },
        {
            "id": "382:vadas",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "vadas",
                "age": 27,
                "city": "Hongkong"
            }
        },
        {
            "id": "383:ripple",
            "label": "software",
            "type": "vertex",
            "properties": {
                "name": "ripple",
                "lang": "java",
                "price": 199
            }
        }
    ]
}
3.2.12.3 Use Cases

Used to find the shortest paths between multiple vertices, for example:

  • Finding the shortest paths between multiple companies and their legal representatives.

3.2.13 Paths (GET, Basic Version)

3.2.13.1 Function Introduction

Finds all paths based on conditions such as the starting vertex, destination vertex, direction, edge types (optional), and maximum depth.

Params
  • source: ID of the starting vertex, required.
  • target: ID of the destination vertex, required.
  • direction: Direction in which the starting vertex expands (OUT, IN, BOTH). Optional, default is BOTH.
  • label: Edge type. Optional, default represents all edge labels.
  • max_depth: Number of steps, required.
  • max_degree: Maximum number of adjacent edges to traverse for each vertex during the query process. Optional, default is 10000.
  • capacity: Maximum number of vertices to be visited during the traversal process. Optional, default is 10000000.
  • limit: Maximum number of paths to be returned. Optional, default is 10.
3.2.13.2 Usage Method
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/traversers/paths?source="1:marko"&target="1:josh"&max_depth=5
Response Status
200
Response Body
{
    "paths":[
        {
            "objects":[
                "1:marko",
                "1:josh"
            ]
        },
        {
            "objects":[
                "1:marko",
                "2:lop",
                "1:josh"
            ]
        }
    ]
}
3.2.13.3 Use Cases

Used to find all paths between two vertices, for example:

  • In a social network, finding all possible relationship paths between two users.
  • In a device association network, finding all associated paths between two devices.

3.2.14 Paths (POST, Advanced Version)

3.2.14.1 Function Introduction

Finds all paths based on conditions such as the starting vertex, destination vertex, steps (step), and maximum depth.

Params
  • sources: Defines the starting vertices, required. The specification methods include:
    • ids: Provide the starting vertices through a list of vertex IDs.
    • label and properties: If no IDs are specified, use the label and properties as combined conditions to query the starting vertices.
      • label: Vertex type.
      • properties: Query the starting vertices based on the values of their properties.

      Note: The property values in properties can be a list, indicating that any value corresponding to the key is acceptable.

  • targets: Defines the destination vertices, required. The specification methods include:
    • ids: Provide the destination vertices through a list of vertex IDs.
    • label and properties: If no IDs are specified, use the label and properties as combined conditions to query the destination vertices.
      • label: Vertex type.
      • properties: Query the destination vertices based on the values of their properties.

      Note: The property values in properties can be a list, indicating that any value corresponding to the key is acceptable.

  • step: Represents the path from the starting vertex to the destination vertex, required. The structure of Step is as follows:
    • direction: Represents the direction of edges (OUT, IN, BOTH). The default is BOTH.
    • labels: List of edge types.
    • properties: Filters edges based on property values.
    • max_degree: Maximum adjacent edges traversed per vertex, defaulting to 10000; the parameter name degree is also accepted.
    • skip_degree: Used to set the minimum number of edges to be discarded for super vertices during the query process. When the number of adjacent edges for a vertex is greater than skip_degree, the vertex is completely discarded. Optional, if enabled, it must satisfy the constraint skip_degree >= max_degree. Default is 0 (not enabled), which means no points are skipped. (Note: When this configuration is enabled, the traversal will attempt to visit skip_degree edges of a vertex, not just max_degree edges. This incurs additional traversal overhead and may have a significant impact on query performance. Please make sure to understand before enabling it.)
  • max_depth: Number of steps, required.
  • nearest: When nearest is true, it means the shortest path length from the starting vertex to the result vertex is depth, and there is no shorter path. When nearest is false, it means there is a path of length depth from the starting vertex to the result vertex (not necessarily the shortest path and can have cycles). Optional, default is true.
  • capacity: Maximum number of vertices to be visited during the traversal process. Optional, default is 10000000.
  • limit: Maximum number of paths to be returned. Optional, default is 10.
  • with_vertex: When true, the results include complete vertex information (all vertices in the path). When false, only the vertex IDs are returned. Optional, default is false.
3.2.14.2 Usage Method
Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/traversers/paths
Request Body
{
"sources": {
  "ids": ["1:marko"]
},
"targets": {
  "ids": ["1:peter"]
},
"step": {
"direction": "BOTH",
  "properties": {
    "weight": "P.gt(0.01)"
  }
},
"max_depth": 10,
"capacity": 100000000,
"limit": 10000000,
"with_vertex": false
}
Response Status
200
Response Body
{
    "paths": [
        {
            "objects": [
                "1:marko",
                "1:josh",
                "2:lop",
                "1:peter"
            ]
        },
        {
            "objects": [
                "1:marko",
                "2:lop",
                "1:peter"
            ]
        }
    ]
}
3.2.14.3 Use Cases

Used to find all paths between two vertices, for example:

  • In a social network, finding all possible relationship paths between two users.
  • In a device association network, finding all associated paths between two devices.

3.2.15 Customized Paths

3.2.15.1 Function Introduction

Finds all paths that meet the specified conditions based on a batch of starting vertices, edge rules (including direction, edge types, and property filters), and maximum depth.

Params
  • sources: Defines the starting vertices, required. The specification methods include:
    • ids: Provide the starting vertices through a list of vertex IDs.
    • label and properties: If no IDs are specified, use the label and properties as combined conditions to query the starting vertices.
      • label: Vertex type.
      • properties: Query the starting vertices based on the values of their properties.

      Note: The property values in properties can be a list, indicating that any value corresponding to the key is acceptable.

  • steps: Represents the path rules traversed from the starting vertices and is a list of Steps. Required. The structure of each Step is as follows:
    • direction: Represents the direction of edges (OUT, IN, BOTH). The default is BOTH.
    • labels: List of edge types.
    • properties: Filters edges based on property values.
    • weight_by: Calculates the weight of edges based on the specified property. It is effective when sort_by is not NONE and is mutually exclusive with default_weight.
    • default_weight: The default weight to be used when there is no property to calculate the weight of edges. It is effective when sort_by is not NONE and is mutually exclusive with weight_by.
    • max_degree: Maximum adjacent edges traversed per vertex, defaulting to 10000; the parameter name degree is also accepted.
    • sample: Used when sampling is needed for the edges that meet the conditions of a specific step. -1 means no sampling, and the default is to sample 100 edges.
  • sort_by: Sorts the paths based on their weights. Optional, default is NONE:
    • NONE: No sorting, default value.
    • INCR: Sorts in ascending order based on path weights.
    • DECR: Sorts in descending order based on path weights.
  • capacity: Maximum number of vertices to be visited during the traversal process. Optional, default is 10000000.
  • limit: Maximum number of paths to be returned. Optional, default is 10.
  • with_vertex: When true, the results include complete vertex information (all vertices in the path). When false, only the vertex IDs are returned. Optional, default is false.
3.2.15.2 Usage Method
Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/traversers/customizedpaths
Request Body
{
    "sources":{
        "ids":[

        ],
        "label":"person",
        "properties":{
            "name":"marko"
        }
    },
    "steps":[
        {
            "direction":"OUT",
            "labels":[
                "knows"
            ],
            "weight_by":"weight",
            "max_degree":-1
        },
        {
            "direction":"OUT",
            "labels":[
                "created"
            ],
            "default_weight":8,
            "max_degree":-1,
            "sample":1
        }
    ],
    "sort_by":"INCR",
    "with_vertex":true,
    "capacity":-1,
    "limit":-1
}
Response Status
200
Response Body
{
    "paths":[
        {
            "objects":[
                "1:marko",
                "1:josh",
                "2:lop"
            ],
            "weights":[
                1,
                8
            ]
        }
    ],
    "vertices":[
        {
            "id":"1:marko",
            "label":"person",
            "type":"vertex",
            "properties":{
                "city":[
                    {
                        "id":"1:marko>city",
                        "value":"Beijing"
                    }
                ],
                "name":[
                    {
                        "id":"1:marko>name",
                        "value":"marko"
                    }
                ],
                "age":[
                    {
                        "id":"1:marko>age",
                        "value":29
                    }
                ]
            }
        },
        {
            "id":"1:josh",
            "label":"person",
            "type":"vertex",
            "properties":{
                "city":[
                    {
                        "id":"1:josh>city",
                        "value":"Beijing"
                    }
                ],
                "name":[
                    {
                        "id":"1:josh>name",
                        "value":"josh"
                    }
                ],
                "age":[
                    {
                        "id":"1:josh>age",
                        "value":32
                    }
                ]
            }
        },
        {
            "id":"2:lop",
            "label":"software",
            "type":"vertex",
            "properties":{
                "price":[
                    {
                        "id":"2:lop>price",
                        "value":328
                    }
                ],
                "name":[
                    {
                        "id":"2:lop>name",
                        "value":"lop"
                    }
                ],
                "lang":[
                    {
                        "id":"2:lop>lang",
                        "value":"java"
                    }
                ]
            }
        }
    ]
}
3.2.15.3 Use Cases

Suitable for finding various complex sets of paths, for example:

  • In a social network, finding the paths from users who have watched movies directed by Zhang Yimou to the influencers they follow (Zhang Yimou —> Movie —> User —> Influencer).
  • In a risk control network, finding the paths from multiple high-risk users to the friends of their direct relatives (High-risk user —> Direct relative —> Friend).

3.2.16 Template Paths

3.2.16.1 Function Introduction

Finds all paths that meet the specified conditions based on a batch of starting vertices, edge rules (including direction, edge types, and property filters), and maximum depth.

Params
  • sources: Defines the starting vertices, required. The specification methods include:
    • ids: Provide the starting vertices through a list of vertex IDs.
    • label and properties: If no IDs are specified, use the label and properties as combined conditions to query the starting vertices.
      • label: Vertex type.
      • properties: Query the starting vertices based on the values of their properties.

      Note: The property values in properties can be a list, indicating that any value corresponding to the key is acceptable.

  • targets: Defines the ending vertices, required. The specification methods include:
    • ids: Provide the ending vertices through a list of vertex IDs.
    • label and properties: If no IDs are specified, use the label and properties as combined conditions to query the ending vertices.
      • label: Vertex type.
      • properties: Query the ending vertices based on the values of their properties.

      Note: The property values in properties can be a list, indicating that any value corresponding to the key is acceptable.

  • steps: Represents the path rules traversed from the starting vertices and is a list of Steps. Required. The structure of each Step is as follows:
    • direction: Represents the direction of edges (OUT, IN, BOTH). The default is BOTH.
    • labels: List of edge types.
    • properties: Filters edges based on property values.
    • max_times: The number of times the current step can be repeated. When set to N, it means the starting vertices can pass through the current step 1-N times.
    • max_degree: Maximum adjacent edges traversed per vertex, defaulting to 10000; the parameter name degree is also accepted.
    • skip_degree: Used to set the minimum number of edges to discard super vertices during the query process. When the number of adjacent edges of a vertex is greater than skip_degree, the vertex is completely discarded. Optional. If enabled, it must satisfy the skip_degree >= max_degree constraint. Default is 0 (not enabled), which means no points are skipped. (Note: After enabling this configuration, traversing will attempt to access a vertex’s skip_degree edges, not just max_degree edges. This incurs additional traversal overhead and may have a significant impact on query performance. Please ensure understanding before enabling.)
  • with_ring: Boolean value, true to include cycles; false to exclude cycles. Default is false.
  • capacity: Maximum number of vertices to be visited during the traversal process. Optional, default is 10000000.
  • limit: Maximum number of paths to be returned. Optional, default is 10.
  • with_vertex: When true, the results include complete vertex information (all vertices in the path). When false, only the vertex IDs are returned. Optional, default is

false.

3.2.16.2 Usage Method
Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/traversers/templatepaths
Request Body
{
  "sources": {
    "ids": [],
    "label": "person",
    "properties": {
      "name": "vadas"
    }
  },
  "targets": {
    "ids": [],
    "label": "software",
    "properties": {
      "name": "ripple"
    }
  },
  "steps": [
    {
      "direction": "IN",
      "labels": ["knows"],
      "properties": {
      },
      "max_degree": 10000,
      "skip_degree": 100000
    },
    {
      "direction": "OUT",
      "labels": ["created"],
      "properties": {
      },
      "max_degree": 10000,
      "skip_degree": 100000
    },
    {
      "direction": "IN",
      "labels": ["created"],
      "properties": {
      },
      "max_degree": 10000,
      "skip_degree": 100000
    },
    {
      "direction": "OUT",
      "labels": ["created"],
      "properties": {
      },
      "max_degree": 10000,
      "skip_degree": 100000
    }
  ],
  "capacity": 10000,
  "limit": 10,
  "with_vertex": true
}
Response Status
200
Response Body
{
    "paths": [
        {
            "objects": [
                "1:vadas",
                "1:marko",
                "2:lop",
                "1:josh",
                "2:ripple"
            ]
        }
    ],
    "vertices": [
        {
            "id": "2:ripple",
            "label": "software",
            "type": "vertex",
            "properties": {
                "name": "ripple",
                "lang": "java",
                "price": 199
            }
        },
        {
            "id": "1:marko",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "marko",
                "age": 29,
                "city": "Beijing"
            }
        },
        {
            "id": "1:josh",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "josh",
                "age": 32,
                "city": "Beijing"
            }
        },
        {
            "id": "1:vadas",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "vadas",
                "age": 27,
                "city": "Hongkong"
            }
        },
        {
            "id": "2:lop",
            "label": "software",
            "type": "vertex",
            "properties": {
                "name": "lop",
                "lang": "java",
                "price": 328
            }
        }
    ]
}
3.2.16.3 Use Cases

Suitable for finding various complex template paths, such as personA -(Friend)-> personB -(Classmate)-> personC, where the “Friend” and “Classmate” edges can have a maximum depth of 3 and 4 layers, respectively.

3.2.17 Crosspoints

3.2.17.1 Function Introduction

Finds the intersection points based on the specified conditions, including starting vertices, destination vertices, direction, edge types (optional), and maximum depth.

Params
  • source: ID of the starting vertex, required.
  • target: ID of the destination vertex, required.
  • direction: The direction from the starting vertex to the destination vertex. The reverse direction is used from the destination vertex to the starting vertex. When set to BOTH, the direction is not considered (OUT, IN, BOTH). Optional, default is BOTH.
  • label: Edge type, optional. Default represents all edge labels.
  • max_depth: Number of steps, required.
  • max_degree: Maximum number of adjacent edges to traverse for each vertex during the query process. Optional, default is 10000.
  • capacity: Maximum number of vertices to be visited during the traversal process. Optional, default is 10000000.
  • limit: Maximum number of intersection points to be returned. Optional, default is 10.
3.2.17.2 Usage Method
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/traversers/crosspoints?source="2:lop"&target="2:ripple"&max_depth=5&direction=IN
Response Status
200
Response Body
{
    "crosspoints":[
        {
            "crosspoint":"1:josh",
            "objects":[
                "2:lop",
                "1:josh",
                "2:ripple"
            ]
        }
    ]
}
3.2.17.3 Use Cases

Used to find the intersection points and their paths between two vertices, such as:

  • In a social network, finding the topics or influencers that two users have in common.
  • In a family relationship, finding common ancestors.

3.2.18 Customized Crosspoints

3.2.18.1 Function Introduction

Finds the intersection of destination vertices that satisfy the specified conditions, including starting vertices, multiple edge rules (including direction, edge type, and property filters), and maximum depth.

Params
  • sources: Defines the starting vertices, required. The specified options include:

    • ids: Provides a list of vertex IDs as starting vertices.
    • label and properties: If no IDs are specified, uses the combined conditions of label and properties to query the starting vertices.
      • label: Type of the vertex.
      • properties: Queries the starting vertices based on property values.

      Note: Property values in properties can be a list, indicating that the value of the key can be any item in the list.

  • path_patterns: Represents the path rules to be followed from the starting vertices. It is a list of rules. Required. Each rule is a PathPattern.

    • Each PathPattern consists of a list of steps, where each step has the following structure:
      • direction: Indicates the direction of the edge (OUT, IN, BOTH). Default is BOTH.
      • labels: List of edge types.
      • properties: Filters the edges based on property values.
      • max_degree: Maximum number of adjacent edges to traverse for each vertex during the query process. Default is 10000.
      • skip_degree: Sets the minimum number of edges to discard super vertices during the query process. If the number of adjacent edges for a vertex is greater than skip_degree, the vertex is completely discarded. Optional. If enabled, it must satisfy the constraint skip_degree >= max_degree. Default is 0 (not enabled), which means no vertices are skipped. Note: When this configuration is enabled, the traversal process will attempt to visit skip_degree edges of a vertex, not just max_degree edges. This incurs additional traversal overhead and may significantly impact query performance. Please make sure you understand it before enabling.
  • capacity: Maximum number of vertices to be visited during the traversal process. Optional. Default is 10000000.

  • limit: Maximum number of paths to be returned. Optional. Default is 10.

  • with_path: When set to true, returns the paths where the intersection points are located. When set to false, does not return the paths. Optional. Default is false.

  • with_vertex: Optional. Default is false.

    • When set to true, the result includes complete vertex information (all vertices in the paths):
      • When with_path is true, it returns complete information of all vertices in the paths.
      • When with_path is false, it returns complete information of all intersection points.
    • When set to false, only the vertex IDs are returned.
3.2.18.2 Usage Method
Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/traversers/customizedcrosspoints
Request Body
{
    "sources":{
        "ids":[
            "2:lop",
            "2:ripple"
        ]
    },
    "path_patterns":[
        {
            "steps":[
                {
                    "direction":"IN",
                    "labels":[
                        "created"
                    ],
                    "max_degree":-1
                }
            ]
        }
    ],
    "with_path":true,
    "with_vertex":true,
    "capacity":-1,
    "limit":-1
}
Response Status
200
Response Body
{
    "crosspoints":[
        "1:josh"
    ],
    "paths":[
        {
            "objects":[
                "2:ripple",
                "1:josh"
            ]
        },
        {
            "objects":[
                "2:lop",
                "1:josh"
            ]
        }
    ],
    "vertices":[
        {
            "id":"2:ripple",
            "label":"software",
            "type":"vertex",
            "properties":{
                "price":[
                    {
                        "id":"2:ripple>price",
                        "value":199
                    }
                ],
                "name":[
                    {
                        "id":"2:ripple>name",
                        "value":"ripple"
                    }
                ],
                "lang":[
                    {
                        "id":"2:ripple>lang",
                        "value":"java"
                    }
                ]
            }
        },
        {
            "id":"1:josh",
            "label":"person",
            "type":"vertex",
            "properties":{
                "city":[
                    {
                        "id":"1:josh>city",
                        "value":"Beijing"
                    }
                ],
                "name":[
                    {
                        "id":"1:josh>name",
                        "value":"josh"
                    }
                ],
                "age":[
                    {
                        "id":"1:josh>age",
                        "value":32
                    }
                ]
            }
        },
        {
            "id":"2:lop",
            "label":"software",
            "type":"vertex",
            "properties":{
                "price":[
                    {
                        "id":"2:lop>price",
                        "value":328
                    }
                ],
                "name":[
                    {
                        "id":"2:lop>name",
                        "value":"lop"
                    }
                ],
                "lang":[
                    {
                        "id":"2:lop>lang",
                        "value":"java"
                    }
                ]
            }
        }
    ]
}
3.2.18.3 Use Cases

Used to query a group of vertices that have intersections at the destination through multiple paths. For example:

  • In a product knowledge graph, multiple models of smartphones, learning devices, and gaming devices belong to the top-level category of electronic devices through different lower-level category paths.

3.2.19 Rings

3.2.19.1 Function Introduction

Finds reachable cycles based on the specified conditions, including starting vertices, direction, edge types (optional), and maximum depth.

For example: 1 -> 25 -> 775 -> 14690 -> 25, where the cycle is 25 -> 775 -> 14690 -> 25.

Params
  • source: Starting vertex ID, required.
  • direction: Direction of edges emitted from the starting vertex (OUT, IN, BOTH). Optional. Default is BOTH.
  • label: Edge type. Optional. Default represents all edge labels.
  • max_depth: Number of steps. Required.
  • source_in_ring: Whether the starting point is included in the cycle. Optional. Default is true.
  • max_degree: Maximum number of adjacent edges to traverse for each vertex during the query process. Optional. Default is 10000.
  • capacity: Maximum number of vertices to be visited during the traversal process. Optional. Default is 10000000.
  • limit: Maximum number of reachable cycles to be returned. Optional. Default is 10.
3.2.19.2 Usage Method
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/traversers/rings?source="1:marko"&max_depth=2
Response Status
200
Response Body
{
    "rings":[
        {
            "objects":[
                "1:marko",
                "1:josh",
                "1:marko"
            ]
        },
        {
            "objects":[
                "1:marko",
                "1:vadas",
                "1:marko"
            ]
        },
        {
            "objects":[
                "1:marko",
                "2:lop",
                "1:marko"
            ]
        }
    ]
}
3.2.19.3 Use Cases

Used to query cycles reachable from the starting vertex, for example:

  • In a risk control project, querying individuals or devices involved in a circular guarantee that a user is connected to.
  • In a device network, discovering devices that have circular references around a specific device.

3.2.20 Rays

3.2.20.1 Function Introduction

Finds paths that diverge from the starting vertex and reach boundary vertices based on the specified conditions, including starting vertices, direction, edge types (optional), and maximum depth.

For example: 1 -> 25 -> 775 -> 14690 -> 2289 -> 18379, where 18379 is the boundary vertex, meaning there are no edges emitted from 18379.

Params
  • source: Starting vertex ID, required.
  • direction: Direction of edges emitted from the starting vertex (OUT, IN, BOTH). Optional. Default is BOTH.
  • label: Edge type. Optional. Default represents all edge labels.
  • max_depth: Number of steps. Required.
  • max_degree: Maximum number of adjacent edges to traverse for each vertex during the query process. Optional. Default is 10000.
  • capacity: Maximum number of vertices to be visited during the traversal process. Optional. Default is 10000000.
  • limit: Maximum number of non-cycle paths to be returned. Optional. Default is 10.
3.2.20.2 Usage Method
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/traversers/rays?source="1:marko"&max_depth=2&direction=OUT
Response Status
200
Response Body
{
    "rays":[
        {
            "objects":[
                "1:marko",
                "1:vadas"
            ]
        },
        {
            "objects":[
                "1:marko",
                "2:lop"
            ]
        },
        {
            "objects":[
                "1:marko",
                "1:josh",
                "2:ripple"
            ]
        },
        {
            "objects":[
                "1:marko",
                "1:josh",
                "2:lop"
            ]
        }
    ]
}
3.2.20.3 Use Cases

Used to find paths from the starting vertex to boundary vertices based on a specific relationship, for example:

  • In a family relationship, finding paths from a person to all descendants who do not have children.
  • In a device network, discovering paths from a specific device to terminal devices.

3.2.21 Fusiform Similarity

3.2.21.1 Function Introduction

Queries a batch of “fusiform similar vertices” based on specified conditions. When two vertices share a certain relationship with many common vertices, they are considered “fusiform similar vertices.” For example, if “Reader A” has read 100 books, readers who have read 80 or more of these 100 books can be defined as “fusiform similar vertices” of “Reader A.”

Params
  • sources: Starting vertices, required. Specify using:

    • ids: Provide a list of vertex IDs as starting vertices.
    • label and properties: If ids are not specified, use the combined conditions of label and properties to query the starting vertices.
      • label: Vertex type.
      • properties: Query the starting vertices based on the values of their properties.

      Note: Property values in properties can be a list, indicating that the value of the key can be any value in the list.

  • label: Edge type. Optional. Default represents all edge labels.

  • direction: Direction in which the starting vertex diverges (OUT, IN, BOTH). Optional. Default is BOTH.

  • min_neighbors: Minimum number of neighbors. If the number of neighbors is less than this threshold, the starting vertex is not considered a “fusiform similar vertex.” For example, if you want to find “fusiform similar vertices” of books read by “Reader A,” and min_neighbors is set to 100, it means that “Reader A” must have read at least 100 books to have “fusiform similar vertices.” Required.

  • alpha: Similarity, representing the proportion of common neighbors between the starting vertex and “fusiform similar vertices” to all neighbors of the starting vertex. Required.

  • min_similars: Minimum number of “fusiform similar vertices.” Only when the number of “fusiform similar vertices” of the starting vertex is greater than or equal to this value, the starting vertex and its “fusiform similar vertices” will be returned. Optional. Default is 1.

  • top: Returns the top highest similarity “fusiform similar vertices” of a starting vertex. Required. 0 means all.

  • group_property: Used together with min_groups. Returns the starting vertex and its “fusiform similar vertices” only if there are at least min_groups different values for a certain attribute of the starting vertex and its “fusiform similar vertices.” For example, when recommending “out-of-town” book buddies for “Reader A,” set group_property to the “city” attribute of readers and min_group to at least 2. Optional. If not specified, no filtering based on attributes is needed.

  • min_groups: Used together with group_property. Only meaningful when group_property is set.

  • max_degree: Maximum number of adjacent edges to traverse for each vertex during the query process. Optional. Default is 10000.

  • capacity: Maximum number of vertices to be visited during the traversal process. Optional. Default is 10000000.

  • limit: Maximum number of results to be returned (one starting vertex and its “fusiform similar vertices” count as one result). Optional. Default is 10.

  • with_intermediary: Whether to return the starting vertex and the intermediate vertices that are commonly related to the “fusiform

similar vertices.” Default is false.

  • with_vertex: Optional. Default is false.
    • true: Returns complete vertex information in the results.
    • false: Only returns vertex IDs.
3.2.21.2 Usage Method
Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/traversers/fusiformsimilarity
Request Body
{
    "sources":{
        "ids":[],
        "label": "person",
        "properties": {
            "name":"p1"
        }
    },
    "label":"read",
    "direction":"OUT",
    "min_neighbors":8,
    "alpha":0.75,
    "min_similars":1,
    "top":0,
    "group_property":"city",
    "min_group":2,
    "max_degree": 10000,
    "capacity": -1,
    "limit": -1,
    "with_intermediary": false,
    "with_vertex":true
}
Response Status
200
Response Body
{
    "similars": {
        "3:p1": [
            {
                "id": "3:p2",
                "score": 0.8888888888888888,
                "intermediaries": [
                ]
            },
            {
                "id": "3:p3",
                "score": 0.7777777777777778,
                "intermediaries": [
                ]
            }
        ]
    },
    "vertices": [
        {
            "id": "3:p1",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "p1",
                "city": "Beijing"
            }
        },
        {
            "id": "3:p2",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "p2",
                "city": "Shanghai"
            }
        },
        {
            "id": "3:p3",
            "label": "person",
            "type": "vertex",
            "properties": {
                "name": "p3",
                "city": "Beijing"
            }
        }
    ]
}
3.2.21.3 Use Cases

Used to query vertices that have high similarity with a group of vertices. For example:

  • Readers with similar book lists to a specific reader.
  • Players who play similar games to a specific player.

3.2.22 Vertices

3.2.22.1 Batch Query Vertices by Vertex IDs
Params
  • ids: List of vertex IDs to be queried.
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/traversers/vertices?ids="1:marko"&ids="2:lop"
Response Status
200
Response Body
{
    "vertices":[
        {
            "id":"1:marko",
            "label":"person",
            "type":"vertex",
            "properties":{
                "city":[
                    {
                        "id":"1:marko>city",
                        "value":"Beijing"
                    }
                ],
                "name":[
                    {
                        "id":"1:marko>name",
                        "value":"marko"
                    }
                ],
                "age":[
                    {
                        "id":"1:marko>age",
                        "value":29
                    }
                ]
            }
        },
        {
            "id":"2:lop",
            "label":"software",
            "type":"vertex",
            "properties":{
                "price":[
                    {
                        "id":"2:lop>price",
                        "value":328
                    }
                ],
                "name":[
                    {
                        "id":"2:lop>name",
                        "value":"lop"
                    }
                ],
                "lang":[
                    {
                        "id":"2:lop>lang",
                        "value":"java"
                    }
                ]
            }
        }
    ]
}
3.2.22.2 Get Vertex Shard Information

Obtain vertex shard information by specifying the shard size split_size (can be used in conjunction with Scan in 3.2.21.3 to retrieve vertices).

Params
  • split_size: Shard size, required.
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/traversers/vertices/shards?split_size=67108864
Response Status
200
Response Body
{
    "shards":[
        {
            "start": "0",
            "end": "2165893",
            "length": 0
        },
        {
            "start": "2165893",
            "end": "4331786",
            "length": 0
        },
        {
            "start": "4331786",
            "end": "6497679",
            "length": 0
        },
        {
            "start": "6497679",
            "end": "8663572",
            "length": 0
        },
        ......
    ]
}
3.2.22.3 Batch Retrieve Vertices Based on Shard Information

Retrieve vertices in batches based on the specified shard information (refer to 3.2.21.2 Shard for obtaining shard information).

Params
  • start: Shard start position, required.
  • end: Shard end position, required.
  • page: Page position for pagination, optional. Default is null, no pagination. When page is “”, it represents the first page of pagination starting from the position indicated by start.
  • page_limit: The upper limit of the number of vertices per page when retrieving vertices with pagination, optional. Default is 100000.
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/traversers/vertices/scan?start=0&end=4294967295
Response Status
200
Response Body
{
    "vertices":[
        {
            "id":"2:ripple",
            "label":"software",
            "type":"vertex",
            "properties":{
                "price":[
                    {
                        "id":"2:ripple>price",
                        "value":199
                    }
                ],
                "name":[
                    {
                        "id":"2:ripple>name",
                        "value":"ripple"
                    }
                ],
                "lang":[
                    {
                        "id":"2:ripple>lang",
                        "value":"java"
                    }
                ]
            }
        },
        {
            "id":"1:vadas",
            "label":"person",
            "type":"vertex",
            "properties":{
                "city":[
                    {
                        "id":"1:vadas>city",
                        "value":"Hongkong"
                    }
                ],
                "name":[
                    {
                        "id":"1:vadas>name",
                        "value":"vadas"
                    }
                ],
                "age":[
                    {
                        "id":"1:vadas>age",
                        "value":27
                    }
                ]
            }
        },
        {
            "id":"1:peter",
            "label":"person",
            "type":"vertex",
            "properties":{
                "city":[
                    {
                        "id":"1:peter>city",
                        "value":"Shanghai"
                    }
                ],
                "name":[
                    {
                        "id":"1:peter>name",
                        "value":"peter"
                    }
                ],
                "age":[
                    {
                        "id":"1:peter>age",
                        "value":35
                    }
                ]
            }
        },
        {
            "id":"1:josh",
            "label":"person",
            "type":"vertex",
            "properties":{
                "city":[
                    {
                        "id":"1:josh>city",
                        "value":"Beijing"
                    }
                ],
                "name":[
                    {
                        "id":"1:josh>name",
                        "value":"josh"
                    }
                ],
                "age":[
                    {
                        "id":"1:josh>age",
                        "value":32
                    }
                ]
            }
        },
        {
            "id":"1:marko",
            "label":"person",
            "type":"vertex",
            "properties":{
                "city":[
                    {
                        "id":"1:marko>city",
                        "value":"Beijing"
                    }
                ],
                "name":[
                    {
                        "id":"1:marko>name",
                        "value":"marko"
                    }
                ],
                "age":[
                    {
                        "id":"1:marko>age",
                        "value":29
                    }
                ]
            }
        },
        {
            "id":"2:lop",
            "label":"software",
            "type":"vertex",
            "properties":{
                "price":[
                    {
                        "id":"2:lop>price",
                        "value":328
                    }
                ],
                "name":[
                    {
                        "id":"2:lop>name",
                        "value":"lop"
                    }
                ],
                "lang":[
                    {
                        "id":"2:lop>lang",
                        "value":"java"
                    }
                ]
            }
        }
    ]
}
3.2.22.4 Use Cases
  • Querying vertices by ID list, which can be used for batch vertex queries. For example, after querying multiple paths in a path search, you can further query all vertex properties of a specific path.
  • Retrieving shards and querying vertices by shard, which can be used to traverse all vertices.

3.2.23 Edges

3.2.23.1 Batch Retrieve Edges Based on Edge IDs
Params
  • ids: List of edge IDs to be queried.
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/traversers/edges?ids="S1:josh>1>>S2:lop"&ids="S1:josh>1>>S2:ripple"
Response Status
200
Response Body
{
    "edges": [
        {
            "id": "S1:josh>1>>S2:lop",
            "label": "created",
            "type": "edge",
            "inVLabel": "software",
            "outVLabel": "person",
            "inV": "2:lop",
            "outV": "1:josh",
            "properties": {
                "date": "20091111",
                "weight": 0.4
            }
        },
        {
            "id": "S1:josh>1>>S2:ripple",
            "label": "created",
            "type": "edge",
            "inVLabel": "software",
            "outVLabel": "person",
            "inV": "2:ripple",
            "outV": "1:josh",
            "properties": {
                "date": "20171210",
                "weight": 1
            }
        }
    ]
}
3.2.23.2 Retrieve Edge Shard Information

Retrieve shard information for edges by specifying the shard size (split_size). This can be used in conjunction with the Scan operation described in section 3.2.22.3 to retrieve edges.

Params
  • split_size: Shard size, required field.
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/traversers/edges/shards?split_size=4294967295
Response Status
200
Response Body
{
    "shards":[
        {
            "start": "0",
            "end": "1073741823",
            "length": 0
        },
        {
            "start": "1073741823",
            "end": "2147483646",
            "length": 0
        },
        {
            "start": "2147483646",
            "end": "3221225469",
            "length": 0
        },
        {
            "start": "3221225469",
            "end": "4294967292",
            "length": 0
        },
        {
            "start": "4294967292",
            "end": "4294967295",
            "length": 0
        }
    ]
}
3.2.23.3 Batch Retrieve Edges Based on Shard Information

Batch retrieve edges by specifying shard information (refer to section 3.2.22.2 for shard retrieval).

Params
  • start: Shard starting position, required field.
  • end: Shard ending position, required field.
  • page: Page position for pagination, optional field. Default is null, which means no pagination. When page is empty, it indicates the first page of pagination starting from the position indicated by start.
  • page_limit: Upper limit of the number of edges per page for paginated retrieval, optional field. Default is 100000.
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/traversers/edges/scan?start=0&end=3221225469
Response Status
200
Response Body
{
    "edges":[
        {
            "id":"S1:peter>2>>S2:lop",
            "label":"created",
            "type":"edge",
            "inVLabel":"software",
            "outVLabel":"person",
            "inV":"2:lop",
            "outV":"1:peter",
            "properties":{
                "weight":0.2,
                "date":"20170324"
            }
        },
        {
            "id":"S1:josh>2>>S2:lop",
            "label":"created",
            "type":"edge",
            "inVLabel":"software",
            "outVLabel":"person",
            "inV":"2:lop",
            "outV":"1:josh",
            "properties":{
                "weight":0.4,
                "date":"20091111"
            }
        },
        {
            "id":"S1:josh>2>>S2:ripple",
            "label":"created",
            "type":"edge",
            "inVLabel":"software",
            "outVLabel":"person",
            "inV":"2:ripple",
            "outV":"1:josh",
            "properties":{
                "weight":1,
                "date":"20171210"
            }
        },
        {
            "id":"S1:marko>1>20130220>S1:josh",
            "label":"knows",
            "type":"edge",
            "inVLabel":"person",
            "outVLabel":"person",
            "inV":"1:josh",
            "outV":"1:marko",
            "properties":{
                "weight":1,
                "date":"20130220"
            }
        },
        {
            "id":"S1:marko>1>20160110>S1:vadas",
            "label":"knows",
            "type":"edge",
            "inVLabel":"person",
            "outVLabel":"person",
            "inV":"1:vadas",
            "outV":"1:marko",
            "properties":{
                "weight":0.5,
                "date":"20160110"
            }
        },
        {
            "id":"S1:marko>2>>S2:lop",
            "label":"created",
            "type":"edge",
            "inVLabel":"software",
            "outVLabel":"person",
            "inV":"2:lop",
            "outV":"1:marko",
            "properties":{
                "weight":0.4,
                "date":"20171210"
            }
        }
    ]
}
3.2.23.4 Use Cases
  • Querying edges based on ID list, suitable for batch retrieval of edges.
  • Retrieving shard information and querying edges based on shards, useful for traversing all edges.

3.2.24 Adamic-Adar

3.2.24.1 Function Introduction

Compute the Adamic-Adar index of two vertices: the sum of the reciprocal of the logarithm of the degree of each common neighbor.

Params
  • vertex: ID of one vertex, required.
  • other: ID of another vertex, required. It must differ from vertex.
  • direction: Direction in which the vertex expands outward (OUT, IN, BOTH). Optional, default is BOTH.
  • label: Edge type. Optional, default represents all edge labels.
  • max_degree: Maximum number of adjacent edges to traverse for each vertex during the query process. Optional, default is 10000.
  • limit: Maximum number of common neighbors taken into account. Optional, default is 10000000.
3.2.24.2 Usage Method
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/traversers/adamicadar?vertex="1:marko"&other="1:josh"
Response Status
200
Response Body

Common neighbors with a degree of 0 are skipped, so the result is 0.0 when the two vertices share no neighbor.

{
    "adamic_adar": 0.9102392266268373
}
3.2.24.3 Use Cases

Predict whether a link is likely to appear between two vertices, where rare common neighbors weigh more than popular ones.

3.2.25 Resource Allocation

3.2.25.1 Function Introduction

Compute the resource allocation index of two vertices: the sum of the reciprocal of the degree of each common neighbor.

Params
  • vertex: ID of one vertex, required.
  • other: ID of another vertex, required. It must differ from vertex.
  • direction: Direction in which the vertex expands outward (OUT, IN, BOTH). Optional, default is BOTH.
  • label: Edge type. Optional, default represents all edge labels.
  • max_degree: Maximum number of adjacent edges to traverse for each vertex during the query process. Optional, default is 10000.
  • limit: Maximum number of common neighbors taken into account. Optional, default is 10000000.
3.2.25.2 Usage Method
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/traversers/resourceallocation?vertex="1:marko"&other="1:josh"
Response Status
200
Response Body
{
    "resource_allocation": 0.3333333333333333
}
3.2.25.3 Use Cases

Link prediction, as an alternative to Adamic-Adar with a stronger penalty on high-degree common neighbors.

3.2.26 Edge Existence

3.2.26.1 Function Introduction

Return the edges that exist between a source vertex and a target vertex.

Params
  • source: ID of the source vertex, required.
  • target: ID of the target vertex, required.
  • label: Edge type. Optional, default represents all edge labels.
  • sort_values: Value of the sort keys, required for edge labels of the MULTIPLE frequency to pick one of several parallel edges. Optional, default is an empty string.
  • limit: Maximum number of edges to be returned. Optional, default is 100.
3.2.26.2 Usage Method
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/traversers/edgeexist?source="1:marko"&target="2:lop"
Response Status
200
Response Body
{
    "edges":[
        {
            "id":"S1:marko>2>>S2:lop",
            "label":"created",
            "type":"edge",
            "inVLabel":"software",
            "outVLabel":"person",
            "inV":"2:lop",
            "outV":"1:marko",
            "properties":{
                "weight":0.4,
                "date":"20171210"
            }
        }
    ]
}
3.2.26.3 Use Cases

Check whether two vertices are directly connected, and get the properties of the connecting edges in one request.

3.2.27 Count

3.2.27.1 Function Introduction

Count the vertices reached from a starting vertex after a series of traversal steps, without returning the vertices themselves.

Params
  • source: ID of the starting vertex, required.
  • steps: Steps of the traversal, required. Each step accepts the following fields:
    • direction: Direction in which the vertex expands outward (OUT, IN, BOTH). Optional, default is BOTH.
    • labels: List of edge labels of the step. Optional, default represents all edge labels.
    • properties: Property filter of the edges of the step. Optional.
    • max_degree: Maximum number of adjacent edges to traverse for each vertex in this step. Optional, default is 10000.
    • skip_degree: Threshold above which a super vertex is skipped in this step. Optional, default is 100000.
  • contains_traversed: Whether to also count the vertices reached by the intermediate steps. Optional, default is false.
  • dedup_size: Maximum number of vertices kept for deduplication, -1 means no limit. Optional, default is 1000000.
3.2.27.2 Usage Method
Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/traversers/count
Request Body
{
    "source": "1:marko",
    "steps": [
        {
            "direction": "BOTH",
            "labels": [],
            "max_degree": 100,
            "skip_degree": 100
        },
        {
            "direction": "BOTH",
            "labels": [],
            "max_degree": 100,
            "skip_degree": 100
        },
        {
            "direction": "BOTH",
            "labels": [],
            "max_degree": 100,
            "skip_degree": 100
        }
    ]
}
Response Status
200
Response Body
{
    "count": 3
}
3.2.27.3 Use Cases

Get the size of a multi-step neighborhood when only the number matters, so the vertices do not have to be serialized and transferred.

5.1.11 - Rank API

Rank REST API: Execute graph node ranking algorithms such as PageRank and Personalized PageRank for centrality analysis.

4.1 Rank API overview

Not only the Graph iteration (traverser) method, HugeGraph-Server also provide Rank API for recommendation purpose. You can use it to recommend some vertexes much closer to a vertex.

4.2 Details of Rank API

4.2.1 Personal Rank API

A typical scenario for Personal Rank algorithm is in recommendation application. According to the out edges of a vertex, recommend some other vertices that having the same or similar edges.

Here is a use case: According to someone’s reading habit or reading history, we can recommend some books he may be interested or some book pal.

For Example:

  1. Suppose we have a vertex, Person type, and named tom.He like 5 books a,b,c,d,e. If we want to recommend some book pal and books for tom, an easier idea is let’s check whoever also liked these books (common hobby based).
  2. Now, we need someone else, like neo, he like three books b,d,f. And Jay, he like 4 books c,d,e,g, and Lee, he also like 4 books a,d,e,f.
  3. For we don’t need to recommend books tom already read, the recommend-list should only contain the books Tom’s book pal already read but tom haven’t read yet. Such as book “f” and “g”, and with priority f > g.
  4. Now, we recompute Tom’s personal rank value, we will get a sorted TopN book pal or book recommend-list. (Choose OTHER_LABEL,for Only Book purpose)
4.2.1.0 Data Preparation

The case above is simple. Here we also provide a public test dataset MovieLens for use case. You should download the dataset. The load it into HugeGraph with HugeGraph-Loader. To make it simple, we ignore all properties data of user and move. only field id is enough. we also ignore the value of edge rating.

The metadata for input file and mapping file as follows:

////////////////////////////////////////////////////////////
// UserID::Gender::Age::Occupation::Zip-code
// MovieID::Title::Genres
// UserID::MovieID::Rating::Timestamp
////////////////////////////////////////////////////////////

// Define schema
schema.propertyKey("id").asInt().ifNotExist().create();
schema.propertyKey("rate").asInt().ifNotExist().create();

schema.vertexLabel("user")
      .properties("id")
      .primaryKeys("id")
      .ifNotExist()
      .create();
schema.vertexLabel("movie")
      .properties("id")
      .primaryKeys("id")
      .ifNotExist()
      .create();

schema.edgeLabel("rating")
      .sourceLabel("user")
      .targetLabel("movie")
      .properties("rate")
      .ifNotExist()
      .create();
{
  "vertices": [
    {
      "label": "user",
      "input": {
        "type": "file",
        "path": "users.dat",
        "format": "TEXT",
        "delimiter": "::",
        "header": ["UserID", "Gender", "Age", "Occupation", "Zip-code"]
      },
      "ignored": ["Gender", "Age", "Occupation", "Zip-code"],
      "mapping": {
          "UserID": "id"
      }
    },
    {
      "label": "movie",
      "input": {
        "type": "file",
        "path": "movies.dat",
        "format": "TEXT",
        "delimiter": "::",
        "header": ["MovieID", "Title", "Genres"]
      },
      "ignored": ["Title", "Genres"],
      "mapping": {
          "MovieID": "id"
      }
    }
  ],
  "edges": [
    {
      "label": "rating",
      "source": ["UserID"],
      "target": ["MovieID"],
      "input": {
        "type": "file",
        "path": "ratings.dat",
        "format": "TEXT",
        "delimiter": "::",
        "header": ["UserID", "MovieID", "Rating", "Timestamp"]
      },
      "ignored": ["Timestamp"],
      "mapping": {
          "UserID": "id",
          "MovieID": "id",
          "Rating": "rate"
      }
    }
  ]
}

Note: modify the input.path to your local path.

4.2.1.1 Function Introduction

suitable for bipartite graph, will return all vertex or a list of its correlation which related to all source vertex.

Bipartite Graph is a special model in Graph Theory, as well as a special flow in network. The strongest feature is, it split all vertex in graph into two sets. The vertex in the set is not connected. However,the vertex in two sets may connect with each other.

Suppose we have one bipartite graph based on user and things. A random walk based PersonalRank algorithm should be likes this:

  1. Choose a user u as start vertex, let’s set the initial weight to be 1.0 . Go from Vu with probability alpha to a neighbor vertex, and (1-alpha) to stay.
  2. If we decide to go outside, we would like to choose an edge, such as rating, to find a common judge.
    1. Then choose the neighbors of current vertex randomly with uniform distribution, and reset the weights with uniform distribution.
    2. Compensate the source vertex’s weight with (1 - alpha)
    3. Repeat step 2;
  3. Convergence after reaching a certain number of steps or precision, then we got a recommend-list.
Params

Required:

  • source: the id of source vertex
  • label: edge label go from the source vertex, should connect two different type of vertex

Optional:

  • alpha: the probability of going out for one vertex in each iteration,similar to the alpha of PageRank,required, value range is (0, 1], default 0.85.
  • max_degree: in query process, the max iteration number of adjacency edge for a vertex, default 10000
  • max_depth: iteration number,range [2, 5000], default 5
  • with_label:result filter,default BOTH_LABEL,optional list as follows:
    • SAME_LABEL:Only keep vertex which has the same type as source vertex
    • OTHER_LABEL:Only keep vertex which has different type as source vertex (the another part in bipartite graph)
    • BOTH_LABEL:Keep both type vertex
  • limit: max return vertex number,default 100
  • max_diff: accuracy for convergence, default 0.0001 (will implement soon)
  • sorted: whether sort the result by rank or not, true for descending sort, false for none, default true
4.2.1.2 Usage
Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/traversers/personalrank
Request Body
{
    "source": "1:1",
    "label": "rating",
    "alpha": 0.6,
    "max_depth": 15,
    "with_label": "OTHER_LABEL",
    "sorted": true,
    "limit": 10
}
Response Status
200
Response Body
{
    "2:2858": 0.0005014026017816927,
    "2:1196": 0.0004336708357653617,
    "2:1210": 0.0004128083140214213,
    "2:593": 0.00038117341069881513,
    "2:480": 0.00037005373269728036,
    "2:1198": 0.000366641614652057,
    "2:2396": 0.0003622362410538888,
    "2:2571": 0.0003593312457300953,
    "2:589": 0.00035922123055598566,
    "2:110": 0.0003466135844390885
}
4.2.1.3 Suitable Scenario

In a bipartite graph build by two different type of vertex, recommend other most related vertex to one vertex. for example:

  • Reading recommendation: find out the books should be recommended to someone first, It is also possible to recommend book pal with the highest common preferences at the same time (just like: WeChat “your friend also read xx " function)
  • Social recommendation: find out other Poster who interested in same topics, or other News/Messages you may be interested with (Such as : “Hot News” function in Weibo)
  • Commodity recommendation: according to someone’s shopping habit,find out a commodity list should recommend first, some online salesman may also be good (Such as : “You May Like” function in TaoBao)

4.2.2 Neighbor Rank API

4.2.2.0 Data Preparation
public class Loader {
    public static void main(String[] args) {
        HugeClient client = new HugeClient("http://127.0.0.1:8080", "hugegraph");
        SchemaManager schema = client.schema();

        schema.propertyKey("name").asText().ifNotExist().create();

        schema.vertexLabel("person")
              .properties("name")
              .useCustomizeStringId()
              .ifNotExist()
              .create();

        schema.vertexLabel("movie")
              .properties("name")
              .useCustomizeStringId()
              .ifNotExist()
              .create();

        schema.edgeLabel("follow")
              .sourceLabel("person")
              .targetLabel("person")
              .ifNotExist()
              .create();

        schema.edgeLabel("like")
              .sourceLabel("person")
              .targetLabel("movie")
              .ifNotExist()
              .create();

        schema.edgeLabel("directedBy")
              .sourceLabel("movie")
              .targetLabel("person")
              .ifNotExist()
              .create();

        GraphManager graph = client.graph();

        Vertex O = graph.addVertex(T.label, "person", T.id, "O", "name", "O");

        Vertex A = graph.addVertex(T.label, "person", T.id, "A", "name", "A");
        Vertex B = graph.addVertex(T.label, "person", T.id, "B", "name", "B");
        Vertex C = graph.addVertex(T.label, "person", T.id, "C", "name", "C");
        Vertex D = graph.addVertex(T.label, "person", T.id, "D", "name", "D");

        Vertex E = graph.addVertex(T.label, "movie", T.id, "E", "name", "E");
        Vertex F = graph.addVertex(T.label, "movie", T.id, "F", "name", "F");
        Vertex G = graph.addVertex(T.label, "movie", T.id, "G", "name", "G");
        Vertex H = graph.addVertex(T.label, "movie", T.id, "H", "name", "H");
        Vertex I = graph.addVertex(T.label, "movie", T.id, "I", "name", "I");
        Vertex J = graph.addVertex(T.label, "movie", T.id, "J", "name", "J");

        Vertex K = graph.addVertex(T.label, "person", T.id, "K", "name", "K");
        Vertex L = graph.addVertex(T.label, "person", T.id, "L", "name", "L");
        Vertex M = graph.addVertex(T.label, "person", T.id, "M", "name", "M");

        O.addEdge("follow", A);
        O.addEdge("follow", B);
        O.addEdge("follow", C);
        D.addEdge("follow", O);

        A.addEdge("follow", B);
        A.addEdge("like", E);
        A.addEdge("like", F);

        B.addEdge("like", G);
        B.addEdge("like", H);

        C.addEdge("like", I);
        C.addEdge("like", J);

        E.addEdge("directedBy", K);
        F.addEdge("directedBy", B);
        F.addEdge("directedBy", L);

        G.addEdge("directedBy", M);
    }
}
4.2.2.1 Function Introduction

In a general graph structure,find the first N vertices of each layer with the highest correlation with a given starting point and their relevance.

In graph words: to go out from the starting point, get the probability of going to each vertex of each layer.

Params
  • source: id of source vertex,required
  • alpha:the probability of going out for one vertex in each iteration,similar to the alpha of PageRank,required, value range is (0, 1]
  • steps: a path rule for source vertex visited,it’s a list of Step,each Step map to a layout in result,required.The structure of each Step as follows:
    • direction:the direction of edge(OUT, IN, BOTH), BOTH for default.
    • labels:a list of edge types, will union all edge types
    • max_degree:in query process, the max iteration number of adjacency edge for a vertex, default 10000
      • max_degree: Maximum adjacent edges traversed per vertex, defaulting to 10000; the parameter name degree is also accepted.
    • skip_degree: the threshold above which a super vertex is skipped in this layer, default 0 (no skipping)
    • top: retains only the top N results with the highest weight in each layer of the results, default 10, max 1000
  • capacity: the maximum number of vertexes visited during the traversal, optional, default 10000000
4.2.2.2 Usage
Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/traversers/neighborrank
Request Body
{
    "source":"O",
    "steps":[
        {
            "direction":"OUT",
            "labels":[
                "follow"
            ],
            "max_degree":-1,
            "top":100
        },
        {
            "direction":"OUT",
            "labels":[
                "follow",
                "like"
            ],
            "max_degree":-1,
            "top":100
        },
        {
            "direction":"OUT",
            "labels":[
                "directedBy"
            ],
            "max_degree":-1,
            "top":100
        }
    ],
    "alpha":0.9,
    "capacity":-1
}
Response Status
200
Response Body
{
    "ranks": [
        {
            "O": 1
        },
        {
            "B": 0.4305,
            "A": 0.3,
            "C": 0.3
        },
        {
            "G": 0.17550000000000002,
            "H": 0.17550000000000002,
            "I": 0.135,
            "J": 0.135,
            "E": 0.09000000000000001,
            "F": 0.09000000000000001
        },
        {
            "M": 0.15795,
            "K": 0.08100000000000002,
            "L": 0.04050000000000001
        }
    ]
}
4.2.2.3 Suitable Scenario

Find the vertices in different layers for a given start point that should be most recommended

  • For example, in the four-layered structure of the audience, friends, movies, and directors, according to the movies that a certain audience’s friends like, recommend movies for that audience, or recommend directors for those movies based on who made them.

5.1.12 - Variable API

Variable REST API: Store and manage key-value pairs as global variables for graph-level configuration and state management.

5.1 Variables

Variables can be used to store data about the entire graph. The data is accessed and stored in the form of key-value pairs.

5.1.1 Creating or Updating a Key-Value Pair

Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/variables/name
Request Body
{
  "data": "tom"
}
Response Status
200
Response Body
{
    "name": "tom"
}

5.1.2 Listing all key-value pairs

Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/variables
Response Status
200
Response Body
{
    "name": "tom"
}

5.1.3 Listing a specific key-value pair

Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/variables/name
Response Status
200
Response Body
{
    "name": "tom"
}

5.1.4 Deleting a specific key-value pair

Method & Url
DELETE http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/variables/name
Response Status
204

5.1.13 - Graphs API

Graphs REST API: Manage graph instance lifecycle including creating, querying, cloning, clearing, and deleting graph databases.

6.1 Graphs

Warning

In production, enable Server authentication and authorization, restrict graph management with an IP allowlist and minimum permissions, and retain audit-*.log audit records. Unauthenticated settings below are only for isolated local tests.

This page documents the Graphs API on current master. For historical paths and request bodies, use the 1.7 Graphs API or 1.5 Graphs API.

With authentication enabled, creation, cloning, deletion, clearing, display-name changes, graph-configuration reads, read-mode changes, and manual compaction require graph-space management permission (space); administrators can satisfy it through permission inheritance. Snapshot creation/restoration and data-mode changes allow graph-space managers or the graph owner. Listing, details, and data/read-mode queries use graph read permissions. Raft APIs additionally require graph-space membership.

6.1.1 List all graphs in the graphspace

Params

Path parameters

  • graphspace: Graphspace name
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs
Response Status
200
Response Body
{
    "graphs": [
        "hugegraph",
        "hugegraph1"
    ]
}

6.1.2 Get details of the graph

Params

Path parameters

  • graphspace: Graphspace name
  • graph: Graph name
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph
Response Status
200
Response Body
{
    "name": "hugegraph",
    "backend": "rocksdb"
}

6.1.3 Clear all data of a graph, include: schema, vertex, edge and index

Params

Path parameters

  • graphspace: Graphspace name
  • graph: Graph name

Query parameters

Since emptying the graph is a dangerous operation, we have added parameters for confirmation to the API to avoid false calls by users:

  • confirm_message: default by I'm sure to delete all data
Method & Url
DELETE http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/clear?confirm_message=I%27m+sure+to+delete+all+data
Response Status
204

6.1.4 Clone graph

Params

Path parameters

  • graphspace: Graphspace name
  • graph: Name of the new graph to create

Query parameters

  • clone_graph_name: name of an existed graph. To clone from an existing graph, the user can choose to transfer the configuration file, which will replace the configuration in the existing graph
Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/cloneGraph?clone_graph_name=hugegraph
Request Body [Optional]

Clone a non-auth mode graph (set Content-Type: application/json)

{
  "gremlin.graph": "org.apache.hugegraph.HugeFactory",
  "backend": "rocksdb",
  "serializer": "binary",
  "store": "cloneGraph",
  "rocksdb.data_path": "./rks-data-xx",
  "rocksdb.wal_path": "./rks-data-xx"
}

Note:

  1. The data/wal_path can’t be the same as the existing graph (use separate directories)
  2. Replace “gremlin.graph=org.apache.hugegraph.auth.HugeFactoryAuthProxy” to enable auth mode
Response Status
201
Response Body
{
    "name": "cloneGraph",
    "nickname": "cloneGraph",
    "backend": "rocksdb",
    "description": ""
}

6.1.5 Create graph

Params

Path parameters

  • graphspace: Graphspace name
  • graph: Graph name
Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph-xx
Request Body

Create a graph (set Content-Type: application/json)

gremlin.graph Configuration:

  • Auth mode: "gremlin.graph": "org.apache.hugegraph.auth.HugeFactoryAuthProxy" (required in production)
  • Non-auth mode: "gremlin.graph": "org.apache.hugegraph.HugeFactory"

Note: For HStore, configure PD correctly in HugeGraph Server; see HStore configuration. The backend selects the scheduler: HStore uses the distributed scheduler; other backends use the local scheduler.

{
  "gremlin.graph": "org.apache.hugegraph.auth.HugeFactoryAuthProxy",
  "backend": "rocksdb",
  "serializer": "binary",
  "store": "hugegraph",
  "rocksdb.data_path": "./rks-data-xx",
  "rocksdb.wal_path": "./rks-data-xx"
}
Response Status
201
Response Body
{
    "name": "hugegraph2",
    "nickname": "hugegraph2",
    "backend": "rocksdb",
    "description": ""
}

6.1.6 Delete graph and its data

Graph-space management permission (space) is required when authentication is enabled.

Params

Path parameters

  • graphspace: Graphspace name
  • graph: Graph name

Query parameters

Since deleting a graph is a dangerous operation, we have added parameters for confirmation to the API to avoid false calls by users:

  • confirm_message: default by I'm sure to drop the graph
Method & Url
DELETE http://localhost:8080/graphspaces/DEFAULT/graphs/graphA?confirm_message=I%27m%20sure%20to%20drop%20the%20graph
Response Status
204

6.1.7 List the graphs of the graphspace with their configuration

Returns one entry per graph the current user can read, each carrying the graph configuration (keys that look like passwords, secrets, tokens, credentials or private keys are left out) plus the fields below. Graphs marked as default for the current user come first.

Params

Path parameters

  • graphspace: Graphspace name

Query parameters

  • prefix: Return only the graphs whose name or nickname starts with this prefix
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/profile
Response Status
200
Response Body

default_update_time is only present when the graph is a default graph of the current user, and create_time only when the graph records one.

[
  {
    "backend": "rocksdb",
    "serializer": "binary",
    "store": "hugegraph",
    "name": "hugegraph",
    "nickname": "hugegraph",
    "graphspace_nickname": "DEFAULT",
    "default": true,
    "default_update_time": "2024-05-01 12:30:00",
    "create_time": "2024-05-01 12:00:00"
  }
]

6.1.8 Update the nickname of a graph

Params

Path parameters

  • graphspace: Graphspace name
  • graph: Graph name

Request parameters

  • action: Must be update
  • update: Container for the fields to update. name is required and must match the graph name in the path, nickname is the new display name and must be unique inside the graphspace.
Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph
Request Body
{
  "action": "update",
  "update": {
    "name": "hugegraph",
    "nickname": "MyGraph"
  }
}
Response Status
200
Response Body
{
  "hugegraph": "updated"
}

6.1.9 Manage the default graphs of the current user

A default graph is recorded per user, so the endpoints below act on behalf of the calling user. They need the authentication system, a server started in standalone mode without it answers 400 with GraphSpace management is not supported in standalone mode.

Set a graph as default
Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/default
Response Status
200
Response Body
{
  "default_graph": [
    "hugegraph"
  ]
}
Unset a default graph
Method & Url
DELETE http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/default
Response Status
200
Response Body
{
  "default_graph": []
}
Get the default graphs
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/default
Response Status
200
Response Body
{
  "default_graph": [
    "hugegraph"
  ]
}

6.1.10 Reload the graphs of the graphspace

Reloads the graphs the server holds, which is useful after the graph configuration has changed outside the server.

Params

Path parameters

  • graphspace: Graphspace name

Request parameters

  • action: Must be reload
Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/graphs/manage
Request Body
{
  "action": "reload"
}
Response Status
200
Response Body
{
  "graphs": "reloaded"
}

6.2 Conf

6.2.1 Get configuration for a graph

Params

Path parameters

  • graphspace: Graphspace name
  • graph: Graph name
Method & Url
GET
http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/conf
Response Status
200
Response Body
# gremlin entrence to create graph
gremlin.graph=org.apache.hugegraph.HugeFactory
# cache config
#schema.cache_capacity=1048576
#graph.cache_capacity=10485760
#graph.cache_expire=600

# schema illegal name template
#schema.illegal_name_regex=\s+|~.*

#vertex.default_label=vertex

backend=rocksdb
serializer=binary

store=hugegraph
...=

6.3 Mode

Allowed graph mode values are: NONE, RESTORING, MERGING, LOADING

  • None mode is regular mode
    • Not allowed to create schema with specified id
    • Not support creating vertex with id for AUTOMATIC id strategy
  • LOADING mode used to load data via hugegraph-loader.
    • When adding vertices / edges, it is not checked whether the required attributes are passed in

Restore has two different modes: Restoring and Merging

  • Restoring mode is used to restore schema and graph data to a new graph.
    • Support create schema with specified id
    • Support create vertex with id for AUTOMATIC id strategy
  • Merging mode is used to merge schema and graph data to an existing graph.
    • Not allowed to create schema with specified id
    • Support create vertex with id for AUTOMATIC id strategy

Under normal circumstances, the graph mode is None. When you need to restore the graph, you need to temporarily modify the graph mode to Restoring or Merging as needed. When you complete the restore, change the graph mode to None.

6.3.1 Get graph mode

Graph read permission (space_member or the graph owner) is required when authentication is enabled.

Params

Path parameters

  • graphspace: Graphspace name
  • graph: Graph name
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/mode
Response Status
200
Response Body
{
    "mode": "NONE"
}

Allowed graph mode values are: NONE, RESTORING, MERGING, LOADING

6.3.2 Modify graph mode.

Graph-space management permission (space) or graph ownership is required when authentication is enabled; administrators can satisfy it through permission inheritance.

Params

Path parameters

  • graphspace: Graphspace name
  • graph: Graph name
Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/mode
Request Body
"RESTORING"

Allowed graph mode values are: NONE, RESTORING, MERGING, LOADING

Response Status
200
Response Body
{
    "mode": "RESTORING"
}

6.3.3 Get graph’s read mode

Graph read permission (space_member or the graph owner) is required when authentication is enabled.

Params

Path parameters

  • graphspace: Graphspace name
  • graph: Graph name
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph_read_mode
Response Status
200
Response Body
{
    "graph_read_mode": "ALL"
}

6.3.4 Modify graph’s read mode.

Graph-space management permission (space) is required when authentication is enabled; administrators can satisfy it through permission inheritance.

Params

Path parameters

  • graphspace: Graphspace name
  • graph: Graph name
Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph_read_mode
Request Body
"OLTP_ONLY"

Allowed read mode values are: ALL, OLTP_ONLY. The API rejects OLAP_ONLY with Graph-read-mode could be ALL or OLTP_ONLY.

Response Status
200
Response Body
{
    "graph_read_mode": "OLTP_ONLY"
}

6.4 Snapshot

6.4.1 Create a snapshot

Graph-space management permission (space) or graph ownership is required when authentication is enabled; administrators can satisfy it through permission inheritance.

Params

Path parameters

  • graphspace: Graphspace name
  • graph: Graph name
Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/snapshot_create
Response Status
200
Response Body
{
    "hugegraph": "snapshot_created"
}

6.4.2 Resume a snapshot

Graph-space management permission (space) or graph ownership is required when authentication is enabled; administrators can satisfy it through permission inheritance.

Params

Path parameters

  • graphspace: Graphspace name
  • graph: Graph name
Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/snapshot_resume
Response Status
200
Response Body
{
    "hugegraph": "snapshot_resumed"
}

6.5 Compact

6.5.1 Manually compact graph

Graph-space management permission (space) is required when authentication is enabled; administrators can satisfy it through permission inheritance.

Params

Path parameters

  • graphspace: Graphspace name
  • graph: Graph name
Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/compact
Response Status
200
Response Body
{
    "nodes": 1,
    "cluster_id": "local",
    "servers": {
        "local": "OK"
    }
}

6.6 Raft

These endpoints only work when the graph runs in raft mode, see the raft.mode option in Config Options. On a graph that does not, they answer 400 with Allowed <operation> operation only when working on raft mode.

Params

Path parameters

  • graphspace: Graphspace name
  • graph: Graph name

Query parameters

  • group: Raft group name, default is default
  • endpoint: Address of the peer, in the host:port form. Required by transfer_leader, set_leader, add_peer and remove_peer.

6.6.1 List the peers of a raft group

Graph-space membership (space_member) is required when authentication is enabled; administrators can satisfy it through permission inheritance.

Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/raft/list_peers
Response Status
200
Response Body

The key of the returned object is the raft group name.

{
  "default": [
    "127.0.0.1:8281",
    "127.0.0.1:8282",
    "127.0.0.1:8283"
  ]
}

6.6.2 Get the leader of a raft group

Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/raft/get_leader
Response Status
200
Response Body
{
  "default": "127.0.0.1:8281"
}

6.6.3 Transfer the leadership of a raft group

Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/raft/transfer_leader?endpoint=127.0.0.1:8282
Response Status
200
Response Body
{
  "default": "127.0.0.1:8282"
}

6.6.4 Set the leader of a raft group

Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/raft/set_leader?endpoint=127.0.0.1:8282
Response Status
200
Response Body
{
  "default": "127.0.0.1:8282"
}

6.6.5 Add a peer to a raft group

This schedules an asynchronous task, see Task API.

Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/raft/add_peer?endpoint=127.0.0.1:8284
Response Status
200
Response Body
{
  "task_id": 1
}

6.6.6 Remove a peer from a raft group

This schedules an asynchronous task, see Task API.

Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/raft/remove_peer?endpoint=127.0.0.1:8284
Response Status
200
Response Body
{
  "task_id": 2
}

5.1.14 - Task API

Task REST API: Query and manage asynchronous task execution status for long-running operations like index rebuilding and graph traversals.

7.1 Task

7.1.1 List all async tasks in graph

Params
  • status: the status of asyncTasks, one of NEW, SCHEDULING, SCHEDULED, QUEUED, RESTORING, RUNNING, SUCCESS, CANCELLING, CANCELLED, FAILED, HANGING, DELETING, case-insensitive
  • ids: task ids to query, can be repeated. It can not be combined with status or page, and it ignores limit.
  • limit: the max number of tasks to return, default is 100
  • page: page token for pagination. When it is passed, the response carries a page field with the token of the next page.
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/tasks?status=success
Response Status
200
Response Body
{
	"tasks": [{
		"task_name": "hugegraph.traversal().V()",
		"task_progress": 0,
		"task_create": 1532943976585,
		"task_status": "success",
		"task_update": 1532943976736,
		"task_result": "0",
		"task_retries": 0,
		"id": 2,
		"task_type": "gremlin",
		"task_callable": "org.apache.hugegraph.api.job.GremlinAPI$GremlinJob",
		"task_input": "{\"gremlin\":\"hugegraph.traversal().V()\",\"bindings\":{},\"language\":\"gremlin-groovy\",\"aliases\":{\"hugegraph\":\"graph\"}}"
	}]
}

7.1.2 View the details of an async task

Params
  • with_result: whether to load the result of the task, default is true
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/tasks/2
Response Status
200
Response Body
{
	"task_name": "hugegraph.traversal().V()",
	"task_progress": 0,
	"task_create": 1532943976585,
	"task_status": "success",
	"task_update": 1532943976736,
	"task_result": "0",
	"task_retries": 0,
	"id": 2,
	"task_type": "gremlin",
	"task_callable": "org.apache.hugegraph.api.job.GremlinAPI$GremlinJob",
	"task_input": "{\"gremlin\":\"hugegraph.traversal().V()\",\"bindings\":{},\"language\":\"gremlin-groovy\",\"aliases\":{\"hugegraph\":\"graph\"}}"
}

7.1.3 Delete task information of an async task,won’t delete the task itself

Params
  • force: whether to delete the task even when it is still running, default is false
Method & Url
DELETE http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/tasks/2
Response Status
204

7.1.4 Cancel an async task, the task should be able to be canceled

If you already created an async task via Gremlin API as follows:

"for (int i = 0; i < 10; i++) {" +
    "hugegraph.addVertex(T.label, 'man');" +
    "hugegraph.tx().commit();" +
    "try {" +
        "sleep(1000);" +
    "} catch (InterruptedException e) {" +
        "break;" +
    "}" +
"}"
Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/tasks/2?action=cancel

cancel it in 10s. if more than 10s, the task may already be finished, then can’t be cancelled.

Response Status
202

Cancelling a task that is already completed or already cancelling returns 400.

Response Body

The whole task object is returned, with task_status set to cancelling or cancelled:

{
	"task_name": "for (int i = 0; i < 10; i++) {...}",
	"task_progress": 0,
	"task_create": 1532943976585,
	"task_status": "cancelling",
	"task_update": 1532943977001,
	"task_retries": 0,
	"id": 2,
	"task_type": "gremlin",
	"task_callable": "org.apache.hugegraph.api.job.GremlinAPI$GremlinJob"
}

At this point, the number of vertices whose label is man must be less than 10.

7.2 Algorithm Job

Schedules an OLAP algorithm as an asynchronous task inside the server. The task id in the response can be followed with the Task API above.

Params

Path parameters

  • graphspace: Graphspace name
  • graph: Graph name
  • name: Algorithm name. The registered algorithms are count_vertex, count_edge, degree_centrality, stress_centrality, betweenness_centrality, closeness_centrality, eigenvector_centrality, triangle_count, cluster_coefficient, lpa, louvain, weak_connected_component, fusiform_similarity, rings, k_core, page_rank and subgraph_stat. An unknown name returns 404.
Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/jobs/algorithm/page_rank
Request Body

The body is the parameter map of the algorithm, and each algorithm validates its own parameters. Pass {} to run with the defaults.

{
    "alpha": 0.15,
    "times": 10
}
Response Status
201
Response Body
{
    "task_id": 1
}

7.3 Computer Job

Schedules a HugeGraph-Computer job as an asynchronous task. The computer job runs outside the server, see HugeGraph-Computer.

Params

Path parameters

  • graphspace: Graphspace name
  • graph: Graph name
  • name: Computer name. The registered computers are page_rank, weak_connected_component, lpa, triangle_count and louvain. An unknown name returns 404.
Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/jobs/computer/page_rank
Request Body

The body is the parameter map of the computer job. Pass {} to run with the defaults.

{}
Response Status
201
Response Body
{
    "task_id": 2
}

5.1.15 - Gremlin API

Gremlin REST API: Execute Gremlin graph traversal language scripts via HTTP interface.

8.1 Gremlin

Warning

Use native query APIs safely in production

The flexibility of graph query languages such as Gremlin and Cypher can introduce security risks. Do not expose native query endpoints directly to the public network. In production, enable authentication and authorization, maintain an IP allowlist, and grant minimum permissions. Standard Server configuration writes authentication-proxy authorization records to audit-*.log; retain these files and restrict read access. auth.audit_log_rate limits per-user output rather than serving as a dedicated audit-log on/off switch.

8.1.1 Send a Gremlin statement to HugeGraphServer (GET), synchronously

Params
  • gremlin: The Gremlin statement to execute on HugeGraphServer.
  • bindings: Parameter bindings with string keys and string or numeric values, similar to MySQL prepared statements, to speed up execution.
  • language: Statement language, defaulting to gremlin-groovy.
  • aliases: Adds aliases for existing variables in a graph space.

This REST proxy cannot reliably carry a JSON-braced aliases parameter in GET queries. Current graph binding names contain hyphens and cannot be used directly as Groovy variables. This GET example uses a simple expression; use the POST example with aliases below to select a graph for traversal.

Method & Url
curl --compressed --get \
  --data-urlencode "gremlin=1+1" \
  http://127.0.0.1:8080/gremlin
Response Status
200
Response Body
{
	"requestId": "<request_id>",
	"status": {"message": "", "code": 200, "attributes": {}},
	"result": {"data": [2], "meta": {}}
}

8.1.2 Send a Gremlin statement to HugeGraphServer (POST), synchronously

Count vertices

Runnable request
curl --compressed -sS -X POST http://127.0.0.1:8080/gremlin \
  -H 'Content-Type: application/json' \
  --data-binary '{"gremlin":"g.V().count()","aliases":{"g":"__g_DEFAULT-hugegraph"}}'

/gremlin is a top-level endpoint. The traversal source for graph hugegraph in graph space DEFAULT is __g_DEFAULT-hugegraph; aliases maps it to script variable g. Adjust the alias to the Server binding name for another graph or graph space. Batch responses can use gzip; curl --compressed decompresses them automatically.

Response Status
200
Response Body
{
	"requestId": "<request_id>",
	"status": {"message": "", "code": 200, "attributes": {}},
	"result": {"data": [6], "meta": {}}
}

Vertex count depends on current graph data; 6 above is an example result.

The response structure differs from the Vertex and Edge REST APIs; clients may need to parse it explicitly.

Query edges

Request Body
{
	"gremlin": "g.E().hasLabel('created').limit(1)",
	"bindings": {},
	"language": "gremlin-groovy",
	"aliases": {
		"g": "__g_DEFAULT-hugegraph"
	}
}
Response Status
200
Response Body
{
	"requestId": "<request_id>",
	"status": {
		"message": "",
		"code": 200,
		"attributes": {}
	},
	"result": {
		"data": [{
			"id": "<edge_id>",
			"label": "created",
			"type": "edge",
			"outVLabel": "person",
			"inVLabel": "software",
			"outV": "<source_vertex_id>",
			"inV": "<target_vertex_id>",
			"properties": {
				"weight": 0.4,
				"date": "<date>"
			}
		}],
		"meta": {}
	}
}

Edge IDs, endpoints, and properties depend on current graph data.

8.1.3 Send a Gremlin statement to HugeGraphServer (POST), asynchronously

Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/jobs/gremlin

Query vertices

Request Body
{
	"gremlin": "g.V('1:marko')",
	"bindings": {},
	"language": "gremlin-groovy",
	"aliases": {}
}

Note:

Asynchronous requests cannot supply aliases. Server automatically binds graph to the current graph and g to its traversal source, and adds the URL graph name as an alias for graph. Scripts can use graph, g, or that graph name.

Response Status
201
Response Body
{
	"task_id": 1
}

Note:

Query task status with GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/tasks/1, where 1 is the task ID. See the asynchronous task REST API.

Query edges

Request Body
{
	"gremlin": "g.E().hasLabel('created').limit(1)",
	"bindings": {},
	"language": "gremlin-groovy",
	"aliases": {}
}
Response Status
201
Response Body
{
	"task_id": 2
}

Note:

Query task status with GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/tasks/2, where 2 is the task ID. See the asynchronous task REST API.

5.1.16 - Cypher API

Cypher REST API: Execute OpenCypher declarative graph query language via HTTP interface.

9.1 Cypher

The Cypher API always needs an Authorization header, either Basic or Bearer. A request without one is rejected with 401, even when the server runs without authentication. The credentials are forwarded to the Gremlin Server through conf/remote-objects.yaml.

9.1.1 Sending a cypher statement (GET) to HugeGraphServer for synchronous execution

Method & Url
GET /graphspaces/{graphspace}/graphs/{graph}/cypher?cypher={cypher}
Params

Path parameters

  • graphspace: Graphspace name
  • graph: Graph name

Query parameters

  • cypher: Cypher statement
Example
GET http://localhost:8080/graphspaces/DEFAULT/graphs/hugecypher1/cypher?cypher=match(n:person) return n.name as name order by n.name limit 1
Response Status
200
Response Body
{
    "requestId": "766b9f48-2f10-40d9-951a-3027d0748ab7",
    "status": {
        "message": "",
        "code": 200,
        "attributes": {
        }
    },
    "result": {
        "data": [
            {
                "name": "hello"
            }
        ],
        "meta": {
        }
    }
}

9.1.2 Sending a cypher statement (POST) to HugeGraphServer for synchronous execution

Method & Url
POST /graphspaces/{graphspace}/graphs/{graph}/cypher
Params

Path parameters

  • graphspace: Graphspace name
  • graph: Graph name
Body

{cypher}

  • cypher: Cypher statement

Note:

It is not in JSON format, but a plain text Cypher statement.

Example
POST http://localhost:8080/graphspaces/DEFAULT/graphs/hugecypher1/cypher
Request Body
match(n:person) return n.name as name order by n.name limit 1
Response Status
200
Response Body
{
    "requestId": "f096bee0-e249-498f-b5a3-ea684fc84f57",
    "status": {
        "message": "",
        "code": 200,
        "attributes": {
        }
    },
    "result": {
        "data": [
            {
                "name": "hello"
            }
        ],
        "meta": {
        }
    }
}

5.1.17 - Authentication API

Authentication REST API: Manage users, roles, permissions, and access control to implement fine-grained graph data security.

Version notes: This page documents current master. For historical behavior, use the 1.7 Authentication API or 1.5 Authentication API.

Users, graph-space groups, resources, memberships, and permission grants use /graphspaces/{graphspace}/auth/.... Login, logout, and token verification remain at /auth/login, /auth/logout, and /auth/verify; default graph-space roles are under /graphspaces/{graphspace}/role. Source retains top-level /auth/groups; this page’s group examples use the graph-space API.

{group_id}, {target_id}, {another_target_id}, {belong_id}, and {access_id} are placeholders. Replace them with the corresponding response id, then encode IDs as URL path components before placing them in a path.

10.1 User Authentication and Access Control

To enable authentication and related configurations, please refer to the Authentication Configuration documentation.

Overview of User Authentication and Access Control:

HugeGraph supports multi-user authentication and fine-grained access control. It adopts a 4-tier design based on “User-User Group-Operation-Resource” to flexibly control user roles and permissions. Resources describe data in the graph database, such as vertices that meet certain conditions. Each resource consists of three elements: type, label, and properties. There are a total of 18 types and combinations of any label and properties to form resources. The internal condition of a resource is an “AND” relationship, while the condition between multiple resources is an “OR” relationship. Users can belong to one or more user groups, and each user group can have permissions for any number of resources. The types of operations include read, write, delete, execute, etc. HugeGraph supports dynamically creating users, user groups, and resources, and supports dynamically assigning or revoking permissions. During the initialization of the database, a super administrator user is created, and subsequently, various role users can be created by the super administrator. If a newly created user is assigned sufficient permissions, they can create or manage more users.

Example:

user(name=boss) -belong-> group(name=all) -access(read)-> target(graph=graph1, resource={label: person, city: Beijing})
Description: User ‘boss’ has read permission for people in the ‘graph1’ graph from Beijing.

Interface Description:

The core of user authentication and access control is 5 categories: UserAPI, GroupAPI, TargetAPI, BelongAPI, AccessAPI. Alongside them, ManagerAPI grants graphspace-level manager roles, LoginAPI issues and verifies tokens, and ProjectAPI groups several graphs so that permissions can be granted for the whole set at once.

10.2 User (User) API

The user interface includes APIs for creating users, deleting users, modifying users, and querying user-related information.

10.2.1 Create User

Params
  • user_name: User name
  • user_password: User password
  • user_nickname: User nickname
  • user_phone: User phone number
  • user_email: User email
  • user_avatar: URL of the user avatar
  • user_description: User description

Both user_name and user_password are required, the rest are optional.

Request Body
{
    "user_name": "boss",
    "user_password": "******",
    "user_phone": "182****9088",
    "user_email": "123@xx.com"
}
Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/auth/users
Response Status
201 
Response Body

In the response message, the password is encrypted as ciphertext.

{
    "user_password": "******",
    "user_email": "123@xx.com",
    "user_update": "2020-11-17 14:31:07.833",
    "user_name": "boss",
    "user_creator": "admin",
    "user_phone": "182****9088",
    "id": "boss",
    "user_create": "2020-11-17 14:31:07.833"
}

10.2.2 Delete User

Params
  • id: User ID to be deleted
Method & Url
DELETE http://localhost:8080/graphspaces/DEFAULT/auth/users/test
Response Status
204

10.2.3 Modify User

Params
  • id: User ID to be modified
Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/auth/users/test
Request Body

Modify user_password and user_phone. user_name can not be changed, and when it is passed it must match the existing name.

{
    "user_name": "test",
    "user_password": "******",
    "user_phone": "183****9266"
}
Response Status
200
Response Body

The returned result is the entire user object including the modified content.

{
    "user_password": "******",
    "user_update": "2020-11-12 10:29:30.455",
    "user_name": "test",
    "user_creator": "admin",
    "user_phone": "183****9266",
    "id": "test",
    "user_create": "2020-11-12 10:27:13.601"
}

10.2.4 Query User List

Params
  • name: Return only the user with this name. When it is given, the response is a single user object instead of a list, and 404 is returned if no such user exists.
  • limit: Upper limit of the number of results returned, default is 100
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/auth/users
Response Status
200
Response Body
{
    "users": [
        {
            "user_password": "******",
            "user_update": "2020-11-11 11:41:12.254",
            "user_name": "admin",
            "user_creator": "system",
          "id": "admin",
            "user_create": "2020-11-11 11:41:12.254"
        }
    ]
}

10.2.5 Query a User

Params
  • id: User ID to be queried
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/auth/users/admin
Response Status
200
Response Body
{
    "user_password": "******",
    "user_update": "2020-11-11 11:41:12.254",
    "user_name": "admin",
    "user_creator": "system",
    "id": "admin",
    "user_create": "2020-11-11 11:41:12.254"
}

10.2.6 Query Roles of a User

Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/auth/users/boss/role
Response Status
200
Response Body
{
    "roles": {
        "hugegraph": {
            "READ": [
                {
                    "type": "ALL",
                    "label": "*",
                    "properties": null
                }
            ]
        }
    }
}

10.3 Group (Group) API

Groups grant corresponding resource permissions, and users are assigned to different groups, thereby having different resource permissions. The group interface includes APIs for creating groups, deleting groups, modifying groups, and querying group-related information.

GroupAPI remains at /auth/groups, the only group route on 1.7.0. Current master also serves /graphspaces/DEFAULT/auth/groups, added by apache/hugegraph#3096.

The GraphSpace API generates each persisted group name as ~hubble_role:v1: + base64url(graphspace) + : + 32 hex digits. For DEFAULT, the name and ID look like ~hubble_role:v1:REVGQVVMVA:<32 hex>; request group_name is only a client label. Use the ID returned by the create response below.

10.3.1 Create Group

Params
  • group_name: Required client label; the Server generates the persisted name
  • group_description: Group description
Request Body
{
    "group_name": "all",
    "group_description": "group can do anything"
}

group_name is required, but cannot customize the persisted graph-space group name generated by Server. Copy the response id for later membership, authorization, lookup, update, or deletion operations.

Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/auth/groups
Response Status
201 
Response Body
{
    "group_creator": "admin",
    "group_name": "<server-generated graphspace-scoped name>",
    "group_create": "2020-11-11 15:46:08.791",
    "group_update": "2020-11-11 15:46:08.791",
    "id": "{group_id}",
    "group_description": "group can do anything"
}

10.3.2 Delete Group

Params
  • id: Group ID to be deleted
Method & Url
DELETE http://localhost:8080/graphspaces/DEFAULT/auth/groups/{group_id}
Response Status
204

10.3.3 Modify Group

Params
  • id: Group ID to be modified
Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/auth/groups/{group_id}
Request Body

Modify group_description. On the GraphSpace form group_name is omitted here, or equal to the generated name: any other value is rejected with “The name of group can’t be updated”.

{
    "group_description": "grant"
}
Response Status
200
Response Body

The returned result is the entire group object including the modified content.

{
    "group_creator": "admin",
    "group_name": "<server-generated graphspace-scoped name>",
    "group_create": "2020-11-12 09:50:58.458",
    "group_update": "2020-11-12 09:57:58.155",
    "id": "{group_id}",
    "group_description": "grant"
}

10.3.4 Query Group List

Params
  • limit: Upper limit of the number of results returned
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/auth/groups
Response Status
200
Response Body
{
    "groups": [
        {
            "group_creator": "admin",
            "group_name": "<server-generated graphspace-scoped name>",
            "group_create": "2020-11-11 15:46:08.791",
            "group_update": "2020-11-11 15:46:08.791",
            "id": "{group_id}",
            "group_description": "group can do anything"
        }
    ]
}

10.3.5 Query a Specific Group

Params
  • id: Group ID to be queried
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/auth/groups/{group_id}
Response Status
200
Response Body
{
    "group_creator": "admin",
    "group_name": "<server-generated graphspace-scoped name>",
    "group_create": "2020-11-11 15:46:08.791",
    "group_update": "2020-11-11 15:46:08.791",
    "id": "{group_id}",
    "group_description": "group can do anything"
}

10.4 Resource (Target) API

Resources describe data in the graph database, such as vertices that meet certain criteria. Each resource includes three elements: type, label, and properties. There are 18 types in total, and the combination of any label and any properties forms a resource. The internal conditions of a resource are based on the AND relationship, while the conditions between multiple resources are based on the OR relationship.
The resource API includes creating, deleting, modifying, and querying resources.

10.4.1 Create Resource

Params
  • target_name: Name of the resource
  • target_graph: Graph of the resource
  • target_url: URL of the resource
  • target_resources: Resource definitions (list)

target_resources can include multiple target_resource, stored in the form of a list.
Each target_resource contains:

  • type: Optional value: VERTEX, EDGE, etc. Can be filled with ALL, indicating it can be a vertex or edge.
  • label: Optional value: name of a vertex or edge type. Can be filled with *, indicating any type.
  • properties: Map type, can contain multiple key-value pairs of properties. Must match all property values. Property values can support conditional ranges (e.g., age: P.gte(18)). If properties are null, it means any property is allowed. If both the property name and value are ‘*’, it also means any property is allowed.

For example, a specific resource: “target_resources”: [{“type”:“VERTEX”,“label”:“person”,“properties”:{“city”:“Beijing”,“age”:“P.gte(20)”}}]
The resource definition means: a vertex of type ‘person’ with the city property set to ‘Beijing’ and the age property greater than or equal to 20.

Request Body
{
    "target_name": "all",
    "target_graph": "hugegraph",
    "target_url": "127.0.0.1:8080",
    "target_resources": [
        {
            "type": "ALL"
        }
    ]
}
Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/auth/targets
Response Status
201 
Response Body
{
    "target_creator": "admin",
    "target_name": "all",
    "target_url": "127.0.0.1:8080",
    "target_graph": "hugegraph",
    "target_create": "2020-11-11 15:32:01.192",
    "target_resources": [
        {
            "type": "ALL",
            "label": "*",
            "properties": null
        }
    ],
    "id": "{target_id}",
    "target_update": "2020-11-11 15:32:01.192"
}

10.4.2 Delete Resource

Params
  • id: Resource Id to be deleted
Method & Url
DELETE http://localhost:8080/graphspaces/DEFAULT/auth/targets/{target_id}
Response Status
204

10.4.3 Modify Resource

Params
  • id: Resource Id to be modified
Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/auth/targets/{target_id}
Request Body

Modify the ’type’ in the resource definition.

{
    "target_resources": [
        {
            "type": "NONE"
        }
    ]
}
Response Status
200
Response Body

The response contains the entire target group object, including the modified content.

{
    "target_creator": "admin",
    "target_name": "all",
    "target_url": "127.0.0.1:8080",
    "target_graph": "hugegraph",
    "target_create": "2020-11-12 09:34:13.848",
    "target_resources": [
        {
            "type": "NONE",
            "label": "*",
            "properties": null
        }
    ],
    "id": "{target_id}",
    "target_update": "2020-11-12 09:37:12.780"
}

10.4.4 Query Resource List

Params
  • limit: Upper limit of the number of returned results.
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/auth/targets
Response Status
200
Response Body
{
    "targets": [
        {
            "target_creator": "admin",
            "target_name": "all",
            "target_url": "127.0.0.1:8080",
            "target_graph": "hugegraph",
            "target_create": "2020-11-11 15:32:01.192",
            "target_resources": [
                {
                    "type": "ALL",
                    "label": "*",
                    "properties": null
                }
            ],
            "id": "{target_id}",
            "target_update": "2020-11-11 15:32:01.192"
        },
        {
            "target_creator": "admin",
            "target_name": "grant",
            "target_url": "127.0.0.1:8080",
            "target_graph": "hugegraph",
            "target_create": "2020-11-11 15:43:24.841",
            "target_resources": [
                {
                    "type": "GRANT",
                    "label": "*",
                    "properties": null
                }
            ],
            "id": "{another_target_id}",
            "target_update": "2020-11-11 15:43:24.841"
        }
    ]
}

10.4.5 Query a Specific Resource

Params
  • id: Id of the resource to query
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/auth/targets/{target_id}
Response Status
200
Response Body
{
    "target_creator": "admin",
    "target_name": "all",
    "target_url": "127.0.0.1:8080",
    "target_graph": "hugegraph",
    "target_create": "2020-11-11 15:32:01.192",
    "target_resources": [
        {
            "type": "ALL",
            "label": "*",
            "properties": null
        }
    ],
    "id": "{target_id}",
    "target_update": "2020-11-11 15:32:01.192"
}

10.5 Association of Roles (Belong) API

The association between users and user groups allows a user to be associated with one or more user groups. User groups have permissions for related resources, and the permissions for different user groups can be understood as different roles. In other words, users are associated with roles.
The API for associating roles includes creating, deleting, modifying, and querying the association of roles for users.

Use your actual group ID from 10.3. For later requests, use the Belong response id and URL-encode > as %3E in URLs.

10.5.1 Create an Association of Roles for a User

Params
  • user: User ID
  • group: User group ID
  • belong_description: Description
Request Body
{
  "user": "boss",
    "group": "{group_id}"
}
Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/auth/belongs
Response Status
201 
Response Body
{
    "belong_create": "2020-11-11 16:19:35.422",
    "belong_creator": "admin",
    "belong_update": "2020-11-11 16:19:35.422",
  "id": "{belong_id}",
  "user": "boss",
    "group": "{group_id}"
}

10.5.2 Delete an Association of Roles

Params
  • id: ID of the association of roles to delete
Method & Url
DELETE http://localhost:8080/graphspaces/DEFAULT/auth/belongs/{belong_id}
Response Status
204

10.5.3 Modify an Association of Roles

An association of roles can only be modified for its description. The user and group properties cannot be modified. If you need to modify an association of roles, you need to delete the existing association and create a new one.

Params
  • id: ID of the association of roles to modify
Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/auth/belongs/{belong_id}
Request Body

Modify the belong_description field

{
    "belong_description": "update test"
}
Response Status
200
Response Body

The response includes the modified content as well as the entire association of roles object

{
    "belong_description": "update test",
    "belong_create": "2020-11-12 10:40:21.720",
    "belong_creator": "admin",
    "belong_update": "2020-11-12 10:42:47.265",
  "id": "{belong_id}",
  "user": "boss",
    "group": "{group_id}"
}

10.5.4 Query List of Associations of Roles

Params
  • user: Return only the associations of this user
  • group: Return only the associations of this group
  • limit: Upper limit on the number of results to return, default is 100

user and group can not be used together.

Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/auth/belongs
Response Status
200
Response Body
{
    "belongs": [
        {
            "belong_create": "2020-11-11 16:19:35.422",
            "belong_creator": "admin",
            "belong_update": "2020-11-11 16:19:35.422",
          "id": "{belong_id}",
          "user": "boss",
            "group": "{group_id}"
        }
    ]
}

10.5.5 View a Specific Association of Roles

Params
  • id: The id of the association of roles to be queried
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/auth/belongs/{belong_id}
Response Status
200
Response Body
{
    "belong_create": "2020-11-11 16:19:35.422",
    "belong_creator": "admin",
    "belong_update": "2020-11-11 16:19:35.422",
  "id": "{belong_id}",
  "user": "boss",
    "group": "{group_id}"
}

10.6 Authorization (Access) API

Grant permissions to user groups for resources, including operations such as READ, WRITE, DELETE, EXECUTE, etc. The authorization API includes: creating, deleting, modifying, and querying permissions.

Use your actual group ID from 10.3 and target ID from 10.4. For later requests, use the Access response id and URL-encode > as %3E in URLs.

10.6.1 Create Authorization (Granting permissions to user groups for resources)

Params
  • group: Group ID
  • target: Resource ID
  • access_permission: Permission grant
  • access_description: Authorization description

Access permissions:

  • READ: Read operations, including all queries such as querying the schema, retrieving vertices/edges, aggregating vertex and edge counts (VERTEX_AGGR/EDGE_AGGR), and reading the graph’s status (STATUS), variables (VAR), tasks (TASK), etc.
  • WRITE: Write operations, including creating and updating operations, such as adding property keys to the schema or adding/updating properties of vertices.
  • DELETE: Delete operations, including deleting metadata, vertices, or edges.
  • EXECUTE: Execute operations, including executing Gremlin queries, executing tasks, and executing metadata functions.
Request Body
{
    "group": "{group_id}",
    "target": "{target_id}",
    "access_permission": "READ"
}
Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/auth/accesses
Response Status
201 
Response Body
{
    "access_permission": "READ",
    "access_create": "2020-11-11 15:54:54.008",
    "id": "{access_id}",
    "access_update": "2020-11-11 15:54:54.008",
    "access_creator": "admin",
    "group": "{group_id}",
    "target": "{target_id}"
}

10.6.2 Delete Authorization

Params
  • id: The ID of the authorization to be deleted
Method & Url
DELETE http://localhost:8080/graphspaces/DEFAULT/auth/accesses/{access_id}
Response Status
204

10.6.3 Modify Authorization

Authorization can only be modified for its description. User group, resource, and permission cannot be modified. If you need to modify the authorization relationship, delete the original authorization and create a new one.

Params
  • id: The ID of the authorization to be modified
Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/auth/accesses/{access_id}
Request Body

Modify access_description

{
    "access_description": "test"
}
Response Status
200
Response Body

The response includes the modified content as well as the entire authorization object.

{
    "access_description": "test",
    "access_permission": "READ",
    "access_create": "2020-11-12 10:12:03.074",
    "id": "{access_id}",
    "access_update": "2020-11-12 10:16:18.637",
    "access_creator": "admin",
    "group": "{group_id}",
    "target": "{target_id}"
}

10.6.4 Query Authorization List

Params
  • group: Return only the authorizations of this group
  • target: Return only the authorizations on this resource
  • limit: The maximum number of results to return, default is 100

group and target can not be used together.

Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/auth/accesses
Response Status
200
Response Body
{
    "accesses": [
        {
            "access_permission": "READ",
            "access_create": "2020-11-11 15:54:54.008",
            "id": "{access_id}",
            "access_update": "2020-11-11 15:54:54.008",
            "access_creator": "admin",
            "group": "{group_id}",
            "target": "{target_id}"
        }
    ]
}

10.6.5 Query a Specific Authorization

Params
  • id: The ID of the authorization to be queried
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/auth/accesses/{access_id}
Response Status
200
Response Body
{
    "access_permission": "READ",
    "access_create": "2020-11-11 15:54:54.008",
    "id": "{access_id}",
    "access_update": "2020-11-11 15:54:54.008",
    "access_creator": "admin",
    "group": "{group_id}",
    "target": "{target_id}"
}

10.7 Graphspace Manager (Manager) API

Note: Before using the following APIs, you need to create a graphspace first. For example, create a graphspace named gs1 via the Graphspace API. The examples below assume that gs1 already exists.

Note: The manager APIs only work when the server runs in PD mode. In standalone mode they return 400 with the message GraphSpace management is not supported in standalone mode.

  1. The graphspace manager API is used to grant/revoke manager roles for users at the graphspace level, and to query the roles of the current user or other users in a graphspace. Supported role types include SPACE, SPACE_MEMBER, and ADMIN.

10.7.1 Check whether the current login user has a specific role

Params
  • type: Role type to check, required, one of SPACE, SPACE_MEMBER, ADMIN
Method & Url
GET http://localhost:8080/graphspaces/gs1/auth/managers/check?type=SPACE_MEMBER
Response Status
200
Response Body
{
  "check": true
}

10.7.2 List graphspace managers

Params
  • type: Role type, required, one of SPACE, SPACE_MEMBER, ADMIN. SPACE lists the managers of the graphspace, SPACE_MEMBER lists its members, and ADMIN lists the administrators of the whole cluster.
Method & Url
GET http://localhost:8080/graphspaces/gs1/auth/managers?type=SPACE
Response Status
200
Response Body
{
  "admins": [
    "admin"
  ]
}

10.7.3 Grant/create a graphspace manager

  • The following example grants user boss the SPACE_MEMBER role in graphspace gs1.
Params
  • user: User or group name, required
  • type: Role type, required, one of SPACE, SPACE_MEMBER, ADMIN

Granting SPACE to a user that is already a space member revokes the member role first, and the other way round. Only an administrator can grant ADMIN.

Request Body
{
  "user": "boss",
  "type": "SPACE_MEMBER"
}
Method & Url
POST http://localhost:8080/graphspaces/gs1/auth/managers
Response Status
201
Response Body
{
  "user": "boss",
  "type": "SPACE_MEMBER",
  "graphspace": "gs1"
}

10.7.4 Revoke graphspace manager privileges

  • The following example revokes the SPACE_MEMBER role of user boss in graphspace gs1.
Params
  • user: User name to revoke. The built-in admin user can not be removed from ADMIN.
  • type: Role type to revoke, one of SPACE, SPACE_MEMBER, ADMIN
Method & Url
DELETE http://localhost:8080/graphspaces/gs1/auth/managers?user=boss&type=SPACE_MEMBER
Response Status
204

10.7.5 Query roles of a specific user in a graphspace

Params
  • user: User name
Method & Url
GET http://localhost:8080/graphspaces/gs1/auth/managers/role?user=boss
Response Status
200
Response Body

The returned roles are a subset of ADMIN, SPACE and SPACE_MEMBER; NONE is returned when the user holds none of them in this graphspace.

{
  "user": "boss",
  "graphspace": "gs1",
  "roles": [
    "SPACE_MEMBER"
  ]
}

10.7.6 Check whether the current login user holds a default role

Default roles are the built-in roles of a graphspace, see Graphspace API. Valid role values are space, space_member, analyst and observer; graph is only taken into account for the observer role.

Params
  • role: Default role name, required
  • graph: Graph name, optional, only used with role=observer
Method & Url
GET http://localhost:8080/graphspaces/gs1/auth/managers/default?role=analyst
Response Status
200
Response Body
{
  "check": true
}

10.8 Login (Login) API

Besides HTTP Basic authentication, the server can hand out a JWT token that is then passed as Authorization: Bearer <token>. The login endpoints are not scoped to a graphspace.

The token is signed with the auth.token_secret option and expires after auth.token_expire seconds (default 86400). The default secret is generated randomly at startup, so set it explicitly when tokens must stay valid across a restart or must be accepted by more than one server.

10.8.1 Log in and get a token

Params
  • user_name: User name, required
  • user_password: User password, required
  • token_expire: Token lifetime in seconds, optional
Request Body
{
    "user_name": "test",
    "user_password": "******"
}
Method & Url
POST http://localhost:8080/auth/login
Response Status
200

Wrong credentials return 401.

Response Body
{
    "token": "eyJhbGciOiJIUzI1NiJ9.eyJ1c2VyX25hbWUiOiJ0ZXN0IiwidXNlcl9pZCI6InRlc3QiLCJleHAiOjE3MTIxMjM0NTZ9.PBs0iBt0PtqvLDpJvKrPHkyIzT1TICz9zJmMy8FvXVo"
}

10.8.2 Log out and invalidate the token

The token to invalidate is taken from the request header, no request body is needed.

Params

Request header

  • Authorization: Bearer <token>, required. Only the Bearer scheme is accepted, other schemes return 400.
Method & Url
DELETE http://localhost:8080/auth/logout
Response Status
204

An invalid or expired token returns 401.

10.8.3 Verify a token

Params

Request header

  • Authorization: Bearer <token>, required
Method & Url
GET http://localhost:8080/auth/verify
Response Status
200

An invalid or expired token returns 401.

Response Body
{
    "user_name": "test",
    "user_id": "test"
}

10.9 Project (Project) API

A project groups a set of graphs together with an admin group and an op group, so that permissions can be granted for the whole set at once. Creating a project also creates its project_target, project_admin_group and project_op_group, which are returned in the response but can not be set by the client.

10.9.1 Create Project

Params
  • project_name: Project name, required
  • project_description: Project description, optional

project_graphs can not be passed on creation, use the add_graph action below.

Request Body
{
    "project_name": "test_project",
    "project_description": "this is a good project"
}
Method & Url
POST http://localhost:8080/graphspaces/DEFAULT/auth/projects
Response Status
201
Response Body
{
    "project_name": "test_project",
    "project_description": "this is a good project",
    "project_target": "project_test_project",
    "project_admin_group": "project_test_project_admin",
    "project_op_group": "project_test_project_op",
    "project_create": "2024-01-10 09:30:00.000",
    "project_update": "2024-01-10 09:30:00.000",
    "project_creator": "admin",
    "id": "test_project"
}

10.9.2 Add graphs to or remove graphs from a project

Params
  • id: Project ID
  • action: add_graph to add graphs, remove_graph to remove them
Request Body
{
    "project_graphs": [
        "hugegraph"
    ]
}
Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/auth/projects/test_project?action=add_graph
Response Status
200
Response Body

The whole project object is returned, including the updated graph list.

10.9.3 Modify the description of a project

Params
  • id: Project ID

Leave action out to update the description. project_graphs must not be present in this case.

Request Body
{
    "project_description": "update desc"
}
Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/auth/projects/test_project
Response Status
200

10.9.4 Query Project List

Params
  • limit: The maximum number of results to return, default is 100
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/auth/projects
Response Status
200
Response Body
{
    "projects": [
        {
            "project_name": "test_project",
            "project_description": "this is a good project",
            "project_target": "project_test_project",
            "project_admin_group": "project_test_project_admin",
            "project_op_group": "project_test_project_op",
            "project_create": "2024-01-10 09:30:00.000",
            "project_update": "2024-01-10 09:30:00.000",
            "project_creator": "admin",
            "id": "test_project"
        }
    ]
}

10.9.5 Query a Specific Project

Params
  • id: Project ID
Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/auth/projects/test_project
Response Status
200

10.9.6 Delete Project

Params
  • id: Project ID

Remove all graphs from the project before deleting it.

Method & Url
DELETE http://localhost:8080/graphspaces/DEFAULT/auth/projects/test_project
Response Status
204

5.1.18 - Metrics API

Metrics REST API: Retrieve runtime performance metrics, statistics, and health status data of the system.

HugeGraph provides a metrics interface for obtaining monitoring information, such as statistics on each Gremlin execution time, cache size, etc. The metrics interface includes the following categories: basic metrics, statistical metrics, system metrics, and backend storage metrics.

1. Basic Metrics

1.1 Get All Basic Metrics

Params
  • type: If the passed value is json, it is returned in json format, otherwise it is returned in Promethaus format.
1.1.1 Method & Url
http://localhost:8080/metrics/?type=json
Response Status
200
Response Body
{
  "gauges": {
    "org.apache.hugegraph.backend.cache.Cache.edge-hugegraph.capacity": {
      "value": 1000000
    },
    "org.apache.hugegraph.backend.cache.Cache.edge-hugegraph.expire": {
      "value": 600000
    },
    "org.apache.hugegraph.backend.cache.Cache.edge-hugegraph.hits": {
      "value": 0
    },
    "org.apache.hugegraph.backend.cache.Cache.edge-hugegraph.miss": {
      "value": 0
    },
    "org.apache.hugegraph.backend.cache.Cache.edge-hugegraph.size": {
      "value": 0
    },
    "org.apache.hugegraph.backend.cache.Cache.instances": {
      "value": 7
    },
    "org.apache.hugegraph.backend.cache.Cache.schema-id-hugegraph.capacity": {
      "value": 10000
    },
    "org.apache.hugegraph.backend.cache.Cache.schema-id-hugegraph.expire": {
      "value": 0
    },
    "org.apache.hugegraph.backend.cache.Cache.schema-id-hugegraph.hits": {
      "value": 0
    },
    "org.apache.hugegraph.backend.cache.Cache.schema-id-hugegraph.miss": {
      "value": 0
    },
    "org.apache.hugegraph.backend.cache.Cache.schema-id-hugegraph.size": {
      "value": 17
    },
    "org.apache.hugegraph.backend.cache.Cache.schema-name-hugegraph.capacity": {
      "value": 10000
    },
    "org.apache.hugegraph.backend.cache.Cache.schema-name-hugegraph.expire": {
      "value": 0
    },
    "org.apache.hugegraph.backend.cache.Cache.schema-name-hugegraph.hits": {
      "value": 0
    },
    "org.apache.hugegraph.backend.cache.Cache.schema-name-hugegraph.miss": {
      "value": 0
    },
    "org.apache.hugegraph.backend.cache.Cache.schema-name-hugegraph.size": {
      "value": 17
    },
    "org.apache.hugegraph.backend.cache.Cache.token-hugegraph.capacity": {
      "value": 10240
    },
    "org.apache.hugegraph.backend.cache.Cache.token-hugegraph.expire": {
      "value": 600000
    },
    "org.apache.hugegraph.backend.cache.Cache.token-hugegraph.hits": {
      "value": 0
    },
    "org.apache.hugegraph.backend.cache.Cache.token-hugegraph.miss": {
      "value": 0
    },
    "org.apache.hugegraph.backend.cache.Cache.token-hugegraph.size": {
      "value": 0
    },
    "org.apache.hugegraph.backend.cache.Cache.users-hugegraph.capacity": {
      "value": 10240
    },
    "org.apache.hugegraph.backend.cache.Cache.users-hugegraph.expire": {
      "value": 600000
    },
    "org.apache.hugegraph.backend.cache.Cache.users-hugegraph.hits": {
      "value": 0
    },
    "org.apache.hugegraph.backend.cache.Cache.users-hugegraph.miss": {
      "value": 0
    },
    "org.apache.hugegraph.backend.cache.Cache.users-hugegraph.size": {
      "value": 0
    },
    "org.apache.hugegraph.backend.cache.Cache.users_pwd-hugegraph.capacity": {
      "value": 10240
    },
    "org.apache.hugegraph.backend.cache.Cache.users_pwd-hugegraph.expire": {
      "value": 600000
    },
    "org.apache.hugegraph.backend.cache.Cache.users_pwd-hugegraph.hits": {
      "value": 0
    },
    "org.apache.hugegraph.backend.cache.Cache.users_pwd-hugegraph.miss": {
      "value": 0
    },
    "org.apache.hugegraph.backend.cache.Cache.users_pwd-hugegraph.size": {
      "value": 0
    },
    "org.apache.hugegraph.backend.cache.Cache.vertex-hugegraph.capacity": {
      "value": 10000000
    },
    "org.apache.hugegraph.backend.cache.Cache.vertex-hugegraph.expire": {
      "value": 600000
    },
    "org.apache.hugegraph.backend.cache.Cache.vertex-hugegraph.hits": {
      "value": 0
    },
    "org.apache.hugegraph.backend.cache.Cache.vertex-hugegraph.miss": {
      "value": 0
    },
    "org.apache.hugegraph.backend.cache.Cache.vertex-hugegraph.size": {
      "value": 0
    },
    "org.apache.hugegraph.server.RestServer.max-write-threads": {
      "value": 0
    },
    "org.apache.hugegraph.task.TaskManager.pending-tasks": {
      "value": 0
    },
    "org.apache.hugegraph.task.TaskManager.workers": {
      "value": 4
    },
    "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.average-load-penalty": {
      "value": 922769200
    },
    "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.estimated-size": {
      "value": 2
    },
    "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.eviction-count": {
      "value": 0
    },
    "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.eviction-weight": {
      "value": 0
    },
    "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.hit-count": {
      "value": 0
    },
    "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.hit-rate": {
      "value": 0
    },
    "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.load-count": {
      "value": 2
    },
    "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.load-failure-count": {
      "value": 0
    },
    "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.load-failure-rate": {
      "value": 0
    },
    "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.load-success-count": {
      "value": 2
    },
    "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.long-run-compilation-count": {
      "value": 0
    },
    "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.miss-count": {
      "value": 2
    },
    "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.miss-rate": {
      "value": 1
    },
    "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.request-count": {
      "value": 2
    },
    "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.total-load-time": {
      "value": 1845538400
    },
    "org.apache.tinkerpop.gremlin.server.GremlinServer.sessions": {
      "value": 0
    }
  },
  "counters": {
    "favicon.ico/GET/FAILED_COUNTER": {
      "count": 1
    },
    "favicon.ico/GET/TOTAL_COUNTER": {
      "count": 1
    },
    "metrics/POST/FAILED_COUNTER": {
      "count": 1
    },
    "metrics/POST/TOTAL_COUNTER": {
      "count": 1
    },
    "metrics/backend/GET/SUCCESS_COUNTER": {
      "count": 2
    },
    "metrics/backend/GET/TOTAL_COUNTER": {
      "count": 2
    },
    "metrics/gauges/GET/SUCCESS_COUNTER": {
      "count": 1
    },
    "metrics/gauges/GET/TOTAL_COUNTER": {
      "count": 1
    },
    "metrics/system/GET/SUCCESS_COUNTER": {
      "count": 2
    },
    "metrics/system/GET/TOTAL_COUNTER": {
      "count": 2
    },
    "system/GET/FAILED_COUNTER": {
      "count": 1
    },
    "system/GET/TOTAL_COUNTER": {
      "count": 1
    }
  },
  "histograms": {
    "favicon.ico/GET/RESPONSE_TIME_HISTOGRAM": {
      "count": 1,
      "min": 1,
      "mean": 1,
      "max": 1,
      "stddev": 0,
      "p50": 1,
      "p75": 1,
      "p95": 1,
      "p98": 1,
      "p99": 1,
      "p999": 1
    },
    "metrics/POST/RESPONSE_TIME_HISTOGRAM": {
      "count": 1,
      "min": 21,
      "mean": 21,
      "max": 21,
      "stddev": 0,
      "p50": 21,
      "p75": 21,
      "p95": 21,
      "p98": 21,
      "p99": 21,
      "p999": 21
    },
    "metrics/backend/GET/RESPONSE_TIME_HISTOGRAM": {
      "count": 2,
      "min": 6,
      "mean": 12.6852124529148,
      "max": 20,
      "stddev": 6.992918475157571,
      "p50": 6,
      "p75": 20,
      "p95": 20,
      "p98": 20,
      "p99": 20,
      "p999": 20
    },
    "metrics/gauges/GET/RESPONSE_TIME_HISTOGRAM": {
      "count": 1,
      "min": 7,
      "mean": 7,
      "max": 7,
      "stddev": 0,
      "p50": 7,
      "p75": 7,
      "p95": 7,
      "p98": 7,
      "p99": 7,
      "p999": 7
    },
    "metrics/system/GET/RESPONSE_TIME_HISTOGRAM": {
      "count": 2,
      "min": 0,
      "mean": 8.942674506664073,
      "max": 40,
      "stddev": 16.665399873223066,
      "p50": 0,
      "p75": 0,
      "p95": 40,
      "p98": 40,
      "p99": 40,
      "p999": 40
    },
    "system/GET/RESPONSE_TIME_HISTOGRAM": {
      "count": 1,
      "min": 2,
      "mean": 2,
      "max": 2,
      "stddev": 0,
      "p50": 2,
      "p75": 2,
      "p95": 2,
      "p98": 2,
      "p99": 2,
      "p999": 2
    }
  },
  "meters": {
    "org.apache.hugegraph.api.API.commit-succeed": {
      "count": 0,
      "mean_rate": 0,
      "m15_rate": 0,
      "m5_rate": 0,
      "m1_rate": 0,
      "rate_unit": "events/second"
    },
    "org.apache.hugegraph.api.API.expected-error": {
      "count": 0,
      "mean_rate": 0,
      "m15_rate": 0,
      "m5_rate": 0,
      "m1_rate": 0,
      "rate_unit": "events/second"
    },
    "org.apache.hugegraph.api.API.illegal-arg": {
      "count": 0,
      "mean_rate": 0,
      "m15_rate": 0,
      "m5_rate": 0,
      "m1_rate": 0,
      "rate_unit": "events/second"
    },
    "org.apache.hugegraph.api.API.unknown-error": {
      "count": 0,
      "mean_rate": 0,
      "m15_rate": 0,
      "m5_rate": 0,
      "m1_rate": 0,
      "rate_unit": "events/second"
    },
    "org.apache.tinkerpop.gremlin.server.GremlinServer.errors": {
      "count": 0,
      "mean_rate": 0,
      "m15_rate": 0,
      "m5_rate": 0,
      "m1_rate": 0,
      "rate_unit": "events/second"
    }
  },
  "timers": {
    "org.apache.hugegraph.api.auth.AccessAPI.create": {
      "count": 0,
      "min": 0,
      "mean": 0,
      "max": 0,
      "stddev": 0,
      "p50": 0,
      "p75": 0,
      "p95": 0,
      "p98": 0,
      "p99": 0,
      "p999": 0,
      "duration_unit": "milliseconds",
      "mean_rate": 0,
      "m15_rate": 0,
      "m5_rate": 0,
      "m1_rate": 0,
      "rate_unit": "calls/second"
    },
    "org.apache.hugegraph.api.auth.AccessAPI.delete": {
      "count": 0,
      "min": 0,
      "mean": 0,
      "max": 0,
      "stddev": 0,
      "p50": 0,
      "p75": 0,
      "p95": 0,
      "p98": 0,
      "p99": 0,
      "p999": 0,
      "duration_unit": "milliseconds",
      "mean_rate": 0,
      "m15_rate": 0,
      "m5_rate": 0,
      "m1_rate": 0,
      "rate_unit": "calls/second"
    },
    "org.apache.hugegraph.api.auth.AccessAPI.get": {
      "count": 0,
      "min": 0,
      "mean": 0,
      "max": 0,
      "stddev": 0,
      "p50": 0,
      "p75": 0,
      "p95": 0,
      "p98": 0,
      "p99": 0,
      "p999": 0,
      "duration_unit": "milliseconds",
      "mean_rate": 0,
      "m15_rate": 0,
      "m5_rate": 0,
      "m1_rate": 0,
      "rate_unit": "calls/second"
    },
    "org.apache.hugegraph.api.auth.AccessAPI.list": {
      "count": 0,
      "min": 0,
      "mean": 0,
      "max": 0,
      "stddev": 0,
      "p50": 0,
      "p75": 0,
      "p95": 0,
      "p98": 0,
      "p99": 0,
      "p999": 0,
      "duration_unit": "milliseconds",
      "mean_rate": 0,
      "m15_rate": 0,
      "m5_rate": 0,
      "m1_rate": 0,
      "rate_unit": "calls/second"
    },
    ...
  }
}
1.1.2 Method & Url
http://localhost:8080/metrics/
Response Status
200
Response Body
# HELP hugegraph_info
# TYPE hugegraph_info untyped
hugegraph_info{version="0.69",
} 1.0
# HELP org_apache_hugegraph_backend_cache_Cache_edge_hugegraph_capacity
# TYPE org_apache_hugegraph_backend_cache_Cache_edge_hugegraph_capacity gauge
org_apache_hugegraph_backend_cache_Cache_edge_hugegraph_capacity 1000000
# HELP org_apache_hugegraph_backend_cache_Cache_edge_hugegraph_expire
# TYPE org_apache_hugegraph_backend_cache_Cache_edge_hugegraph_expire gauge
org_apache_hugegraph_backend_cache_Cache_edge_hugegraph_expire 600000
# HELP org_apache_hugegraph_backend_cache_Cache_edge_hugegraph_hits
# TYPE org_apache_hugegraph_backend_cache_Cache_edge_hugegraph_hits gauge
org_apache_hugegraph_backend_cache_Cache_edge_hugegraph_hits 0
# HELP org_apache_hugegraph_backend_cache_Cache_edge_hugegraph_miss
# TYPE org_apache_hugegraph_backend_cache_Cache_edge_hugegraph_miss gauge
org_apache_hugegraph_backend_cache_Cache_edge_hugegraph_miss 0
# HELP org_apache_hugegraph_backend_cache_Cache_edge_hugegraph_size
# TYPE org_apache_hugegraph_backend_cache_Cache_edge_hugegraph_size gauge
org_apache_hugegraph_backend_cache_Cache_edge_hugegraph_size 0
# HELP org_apache_hugegraph_backend_cache_Cache_instances
# TYPE org_apache_hugegraph_backend_cache_Cache_instances gauge
org_apache_hugegraph_backend_cache_Cache_instances 7
# HELP org_apache_hugegraph_backend_cache_Cache_schema_id_hugegraph_capacity
# TYPE org_apache_hugegraph_backend_cache_Cache_schema_id_hugegraph_capacity gauge
org_apache_hugegraph_backend_cache_Cache_schema_id_hugegraph_capacity 10000
# HELP org_apache_hugegraph_backend_cache_Cache_schema_id_hugegraph_expire
# TYPE org_apache_hugegraph_backend_cache_Cache_schema_id_hugegraph_expire gauge
org_apache_hugegraph_backend_cache_Cache_schema_id_hugegraph_expire 0
# HELP org_apache_hugegraph_backend_cache_Cache_schema_id_hugegraph_hits
# TYPE org_apache_hugegraph_backend_cache_Cache_schema_id_hugegraph_hits gauge
org_apache_hugegraph_backend_cache_Cache_schema_id_hugegraph_hits 0
# HELP org_apache_hugegraph_backend_cache_Cache_schema_id_hugegraph_miss
# TYPE org_apache_hugegraph_backend_cache_Cache_schema_id_hugegraph_miss gauge
org_apache_hugegraph_backend_cache_Cache_schema_id_hugegraph_miss 0
# HELP org_apache_hugegraph_backend_cache_Cache_schema_id_hugegraph_size
# TYPE org_apache_hugegraph_backend_cache_Cache_schema_id_hugegraph_size gauge
org_apache_hugegraph_backend_cache_Cache_schema_id_hugegraph_size 17
# HELP org_apache_hugegraph_backend_cache_Cache_schema_name_hugegraph_capacity
# TYPE org_apache_hugegraph_backend_cache_Cache_schema_name_hugegraph_capacity gauge
org_apache_hugegraph_backend_cache_Cache_schema_name_hugegraph_capacity 10000
# HELP org_apache_hugegraph_backend_cache_Cache_schema_name_hugegraph_expire
# TYPE org_apache_hugegraph_backend_cache_Cache_schema_name_hugegraph_expire gauge
org_apache_hugegraph_backend_cache_Cache_schema_name_hugegraph_expire 0
# HELP org_apache_hugegraph_backend_cache_Cache_schema_name_hugegraph_hits
# TYPE org_apache_hugegraph_backend_cache_Cache_schema_name_hugegraph_hits gauge
org_apache_hugegraph_backend_cache_Cache_schema_name_hugegraph_hits 0
# HELP org_apache_hugegraph_backend_cache_Cache_schema_name_hugegraph_miss
# TYPE org_apache_hugegraph_backend_cache_Cache_schema_name_hugegraph_miss gauge
org_apache_hugegraph_backend_cache_Cache_schema_name_hugegraph_miss 0
# HELP org_apache_hugegraph_backend_cache_Cache_schema_name_hugegraph_size
# TYPE org_apache_hugegraph_backend_cache_Cache_schema_name_hugegraph_size gauge
org_apache_hugegraph_backend_cache_Cache_schema_name_hugegraph_size 17
...

1.2 Get Gauges Metrics

Method & Url
http://localhost:8080/metrics/gauges
Response Status
200
Response Body
{
  "org.apache.hugegraph.backend.cache.Cache.edge-hugegraph.capacity": {
    "value": 1000000
  },
  "org.apache.hugegraph.backend.cache.Cache.edge-hugegraph.expire": {
    "value": 600000
  },
  "org.apache.hugegraph.backend.cache.Cache.edge-hugegraph.hits": {
    "value": 0
  },
  "org.apache.hugegraph.backend.cache.Cache.edge-hugegraph.miss": {
    "value": 0
  },
  "org.apache.hugegraph.backend.cache.Cache.edge-hugegraph.size": {
    "value": 0
  },
  "org.apache.hugegraph.backend.cache.Cache.instances": {
    "value": 7
  },
  "org.apache.hugegraph.backend.cache.Cache.schema-id-hugegraph.capacity": {
    "value": 10000
  },
  "org.apache.hugegraph.backend.cache.Cache.schema-id-hugegraph.expire": {
    "value": 0
  },
  "org.apache.hugegraph.backend.cache.Cache.schema-id-hugegraph.hits": {
    "value": 0
  },
  "org.apache.hugegraph.backend.cache.Cache.schema-id-hugegraph.miss": {
    "value": 0
  },
  "org.apache.hugegraph.backend.cache.Cache.schema-id-hugegraph.size": {
    "value": 17
  },
  "org.apache.hugegraph.backend.cache.Cache.schema-name-hugegraph.capacity": {
    "value": 10000
  },
  "org.apache.hugegraph.backend.cache.Cache.schema-name-hugegraph.expire": {
    "value": 0
  },
  "org.apache.hugegraph.backend.cache.Cache.schema-name-hugegraph.hits": {
    "value": 0
  },
  "org.apache.hugegraph.backend.cache.Cache.schema-name-hugegraph.miss": {
    "value": 0
  },
  "org.apache.hugegraph.backend.cache.Cache.schema-name-hugegraph.size": {
    "value": 17
  },
  "org.apache.hugegraph.backend.cache.Cache.token-hugegraph.capacity": {
    "value": 10240
  },
  "org.apache.hugegraph.backend.cache.Cache.token-hugegraph.expire": {
    "value": 600000
  },
  "org.apache.hugegraph.backend.cache.Cache.token-hugegraph.hits": {
    "value": 0
  },
  "org.apache.hugegraph.backend.cache.Cache.token-hugegraph.miss": {
    "value": 0
  },
  "org.apache.hugegraph.backend.cache.Cache.token-hugegraph.size": {
    "value": 0
  },
  "org.apache.hugegraph.backend.cache.Cache.users-hugegraph.capacity": {
    "value": 10240
  },
  "org.apache.hugegraph.backend.cache.Cache.users-hugegraph.expire": {
    "value": 600000
  },
  "org.apache.hugegraph.backend.cache.Cache.users-hugegraph.hits": {
    "value": 0
  },
  "org.apache.hugegraph.backend.cache.Cache.users-hugegraph.miss": {
    "value": 0
  },
  "org.apache.hugegraph.backend.cache.Cache.users-hugegraph.size": {
    "value": 0
  },
  "org.apache.hugegraph.backend.cache.Cache.users_pwd-hugegraph.capacity": {
    "value": 10240
  },
  "org.apache.hugegraph.backend.cache.Cache.users_pwd-hugegraph.expire": {
    "value": 600000
  },
  "org.apache.hugegraph.backend.cache.Cache.users_pwd-hugegraph.hits": {
    "value": 0
  },
  "org.apache.hugegraph.backend.cache.Cache.users_pwd-hugegraph.miss": {
    "value": 0
  },
  "org.apache.hugegraph.backend.cache.Cache.users_pwd-hugegraph.size": {
    "value": 0
  },
  "org.apache.hugegraph.backend.cache.Cache.vertex-hugegraph.capacity": {
    "value": 10000000
  },
  "org.apache.hugegraph.backend.cache.Cache.vertex-hugegraph.expire": {
    "value": 600000
  },
  "org.apache.hugegraph.backend.cache.Cache.vertex-hugegraph.hits": {
    "value": 0
  },
  "org.apache.hugegraph.backend.cache.Cache.vertex-hugegraph.miss": {
    "value": 0
  },
  "org.apache.hugegraph.backend.cache.Cache.vertex-hugegraph.size": {
    "value": 0
  },
  "org.apache.hugegraph.server.RestServer.max-write-threads": {
    "value": 0
  },
  "org.apache.hugegraph.task.TaskManager.pending-tasks": {
    "value": 0
  },
  "org.apache.hugegraph.task.TaskManager.workers": {
    "value": 4
  },
  "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.average-load-penalty": {
    "value": 9.227692E8
  },
  "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.estimated-size": {
    "value": 2
  },
  "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.eviction-count": {
    "value": 0
  },
  "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.eviction-weight": {
    "value": 0
  },
  "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.hit-count": {
    "value": 0
  },
  "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.hit-rate": {
    "value": 0.0
  },
  "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.load-count": {
    "value": 2
  },
  "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.load-failure-count": {
    "value": 0
  },
  "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.load-failure-rate": {
    "value": 0.0
  },
  "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.load-success-count": {
    "value": 2
  },
  "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.long-run-compilation-count": {
    "value": 0
  },
  "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.miss-count": {
    "value": 2
  },
  "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.miss-rate": {
    "value": 1.0
  },
  "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.request-count": {
    "value": 2
  },
  "org.apache.tinkerpop.gremlin.server.GremlinServer.gremlin-groovy.sessionless.class-cache.total-load-time": {
    "value": 1845538400
  },
  "org.apache.tinkerpop.gremlin.server.GremlinServer.sessions": {
    "value": 0
  }
}

1.3 Get Counters Metrics

Method & Url
GET http://localhost:8080/metrics/counters
Response Status
200
Response Body
{
  "favicon.ico/GET/FAILED_COUNTER": {
    "count": 1
  },
  "favicon.ico/GET/TOTAL_COUNTER": {
    "count": 1
  },
  "metrics//GET/SUCCESS_COUNTER": {
    "count": 2
  },
  "metrics//GET/TOTAL_COUNTER": {
    "count": 2
  },
  "metrics/POST/FAILED_COUNTER": {
    "count": 1
  },
  "metrics/POST/TOTAL_COUNTER": {
    "count": 1
  },
  "metrics/backend/GET/SUCCESS_COUNTER": {
    "count": 2
  },
  "metrics/backend/GET/TOTAL_COUNTER": {
    "count": 2
  },
  "metrics/gauges/GET/SUCCESS_COUNTER": {
    "count": 1
  },
  "metrics/gauges/GET/TOTAL_COUNTER": {
    "count": 1
  },
  "metrics/statistics/GET/SUCCESS_COUNTER": {
    "count": 2
  },
  "metrics/statistics/GET/TOTAL_COUNTER": {
    "count": 2
  },
  "metrics/system/GET/SUCCESS_COUNTER": {
    "count": 2
  },
  "metrics/system/GET/TOTAL_COUNTER": {
    "count": 2
  },
  "metrics/timers/GET/SUCCESS_COUNTER": {
    "count": 1
  },
  "metrics/timers/GET/TOTAL_COUNTER": {
    "count": 1
  },
  "system/GET/FAILED_COUNTER": {
    "count": 1
  },
  "system/GET/TOTAL_COUNTER": {
    "count": 1
  }
}

1.4 Get Histograms Metrics

Method & Url
GET http://localhost:8080/metrics/gauges
Response Status
200
Response Body
{
  "favicon.ico/GET/RESPONSE_TIME_HISTOGRAM": {
    "count": 1,
    "min": 1,
    "mean": 1.0,
    "max": 1,
    "stddev": 0.0,
    "p50": 1.0,
    "p75": 1.0,
    "p95": 1.0,
    "p98": 1.0,
    "p99": 1.0,
    "p999": 1.0
  },
  "metrics//GET/RESPONSE_TIME_HISTOGRAM": {
    "count": 2,
    "min": 10,
    "mean": 10.0,
    "max": 10,
    "stddev": 0.0,
    "p50": 10.0,
    "p75": 10.0,
    "p95": 10.0,
    "p98": 10.0,
    "p99": 10.0,
    "p999": 10.0
  },
  "metrics/POST/RESPONSE_TIME_HISTOGRAM": {
    "count": 1,
    "min": 21,
    "mean": 21.0,
    "max": 21,
    "stddev": 0.0,
    "p50": 21.0,
    "p75": 21.0,
    "p95": 21.0,
    "p98": 21.0,
    "p99": 21.0,
    "p999": 21.0
  },
  "metrics/backend/GET/RESPONSE_TIME_HISTOGRAM": {
    "count": 2,
    "min": 6,
    "mean": 12.6852124529148,
    "max": 20,
    "stddev": 6.992918475157571,
    "p50": 6.0,
    "p75": 20.0,
    "p95": 20.0,
    "p98": 20.0,
    "p99": 20.0,
    "p999": 20.0
  },
  "metrics/gauges/GET/RESPONSE_TIME_HISTOGRAM": {
    "count": 1,
    "min": 7,
    "mean": 7.0,
    "max": 7,
    "stddev": 0.0,
    "p50": 7.0,
    "p75": 7.0,
    "p95": 7.0,
    "p98": 7.0,
    "p99": 7.0,
    "p999": 7.0
  },
  "metrics/statistics/GET/RESPONSE_TIME_HISTOGRAM": {
    "count": 2,
    "min": 1,
    "mean": 1.4551211076264199,
    "max": 2,
    "stddev": 0.49798181193626,
    "p50": 1.0,
    "p75": 2.0,
    "p95": 2.0,
    "p98": 2.0,
    "p99": 2.0,
    "p999": 2.0
  },
  "metrics/system/GET/RESPONSE_TIME_HISTOGRAM": {
    "count": 2,
    "min": 0,
    "mean": 8.942674506664073,
    "max": 40,
    "stddev": 16.665399873223066,
    "p50": 0.0,
    "p75": 0.0,
    "p95": 40.0,
    "p98": 40.0,
    "p99": 40.0,
    "p999": 40.0
  },
  "metrics/timers/GET/RESPONSE_TIME_HISTOGRAM": {
    "count": 1,
    "min": 3,
    "mean": 3.0,
    "max": 3,
    "stddev": 0.0,
    "p50": 3.0,
    "p75": 3.0,
    "p95": 3.0,
    "p98": 3.0,
    "p99": 3.0,
    "p999": 3.0
  },
  "system/GET/RESPONSE_TIME_HISTOGRAM": {
    "count": 1,
    "min": 2,
    "mean": 2.0,
    "max": 2,
    "stddev": 0.0,
    "p50": 2.0,
    "p75": 2.0,
    "p95": 2.0,
    "p98": 2.0,
    "p99": 2.0,
    "p999": 2.0
  }
}

1.5 Get Meters Metrics

Method & Url
GET http://localhost:8080/metrics/meters
Response Status
200
Response Body
{
  "org.apache.hugegraph.api.API.commit-succeed": {
    "count": 0,
    "mean_rate": 0.0,
    "m15_rate": 0.0,
    "m5_rate": 0.0,
    "m1_rate": 0.0,
    "rate_unit": "events/second"
  },
  "org.apache.hugegraph.api.API.expected-error": {
    "count": 0,
    "mean_rate": 0.0,
    "m15_rate": 0.0,
    "m5_rate": 0.0,
    "m1_rate": 0.0,
    "rate_unit": "events/second"
  },
  "org.apache.hugegraph.api.API.illegal-arg": {
    "count": 0,
    "mean_rate": 0.0,
    "m15_rate": 0.0,
    "m5_rate": 0.0,
    "m1_rate": 0.0,
    "rate_unit": "events/second"
  },
  "org.apache.hugegraph.api.API.unknown-error": {
    "count": 0,
    "mean_rate": 0.0,
    "m15_rate": 0.0,
    "m5_rate": 0.0,
    "m1_rate": 0.0,
    "rate_unit": "events/second"
  },
  "org.apache.tinkerpop.gremlin.server.GremlinServer.errors": {
    "count": 0,
    "mean_rate": 0.0,
    "m15_rate": 0.0,
    "m5_rate": 0.0,
    "m1_rate": 0.0,
    "rate_unit": "events/second"
  }
}

1.6 Get Timers Metrics

Method & Url
GET http://localhost:8080/metrics/timers
Response Status
200
Response Body
{
  "org.apache.hugegraph.api.auth.AccessAPI.create": {
    "count": 0,
    "min": 0.0,
    "mean": 0.0,
    "max": 0.0,
    "stddev": 0.0,
    "p50": 0.0,
    "p75": 0.0,
    "p95": 0.0,
    "p98": 0.0,
    "p99": 0.0,
    "p999": 0.0,
    "duration_unit": "milliseconds",
    "mean_rate": 0.0,
    "m15_rate": 0.0,
    "m5_rate": 0.0,
    "m1_rate": 0.0,
    "rate_unit": "calls/second"
  },
  "org.apache.hugegraph.api.auth.AccessAPI.delete": {
    "count": 0,
    "min": 0.0,
    "mean": 0.0,
    "max": 0.0,
    "stddev": 0.0,
    "p50": 0.0,
    "p75": 0.0,
    "p95": 0.0,
    "p98": 0.0,
    "p99": 0.0,
    "p999": 0.0,
    "duration_unit": "milliseconds",
    "mean_rate": 0.0,
    "m15_rate": 0.0,
    "m5_rate": 0.0,
    "m1_rate": 0.0,
    "rate_unit": "calls/second"
  },
  ...
}

2.Statistical Metrics

Params
  • type: If the passed value is JSON, it is returned in JSON format, otherwise it is returned in Promethaus format.
2.1 Method & Url
GET http://localhost:8080/metrics/statistics
Response Status
# HELP hugegraph_info
# TYPE hugegraph_info untyped
hugegraph_info{version="0.69",
} 1.0
# HELP metrics_POST
# TYPE metrics_POST gauge
metrics_POST{name=FAILED_REQUEST,} 1
metrics_POST{name=MEAN_RESPONSE_TIME,} 21.0
metrics_POST{
name=MAX_RESPONSE_TIME,
} 21
metrics_POST{name=SUCCESS_REQUEST,
} 0
metrics_POST{
name=TOTAL_REQUEST,
} 1
# HELP metrics_backend_GET
# TYPE metrics_backend_GET gauge
metrics_backend_GET{name=FAILED_REQUEST,
} 0
metrics_backend_GET{
name=MEAN_RESPONSE_TIME,
} 12.6852124529148
metrics_backend_GET{
name=MAX_RESPONSE_TIME,
} 20
metrics_backend_GET{
name=SUCCESS_REQUEST,
} 2
metrics_backend_GET{name=TOTAL_REQUEST,} 2
# HELP system_GET
# TYPE system_GET gauge
system_GET{name=FAILED_REQUEST,} 1
system_GET{name=MEAN_RESPONSE_TIME,} 2.0
system_GET{name=MAX_RESPONSE_TIME,} 2
system_GET{
name=SUCCESS_REQUEST,
} 0
system_GET{name=TOTAL_REQUEST,
} 1
# HELP metrics_gauges_GET
# TYPE metrics_gauges_GET gauge
metrics_gauges_GET{name=FAILED_REQUEST,} 0
metrics_gauges_GET{name=MEAN_RESPONSE_TIME,
} 7.0
metrics_gauges_GET{
name=MAX_RESPONSE_TIME,
} 7
metrics_gauges_GET{
name=SUCCESS_REQUEST,
} 1
metrics_gauges_GET{
name=TOTAL_REQUEST,
} 1
# HELP favicon.ico_GET
# TYPE favicon.ico_GET gauge
favicon.ico_GET{name=FAILED_REQUEST,
} 1
favicon.ico_GET{
name=MEAN_RESPONSE_TIME,
} 1.0
favicon.ico_GET{name=MAX_RESPONSE_TIME,} 1
favicon.ico_GET{name=SUCCESS_REQUEST,} 0
favicon.ico_GET{
name=TOTAL_REQUEST,
} 1
# HELP metrics__GET
# TYPE metrics__GET gauge
metrics__GET{name=FAILED_REQUEST,} 0
metrics__GET{name=MEAN_RESPONSE_TIME,} 10.0
metrics__GET{name=MAX_RESPONSE_TIME,
} 10
metrics__GET{
name=SUCCESS_REQUEST,
} 2
metrics__GET{
name=TOTAL_REQUEST,
} 2
# HELP metrics_system_GET
# TYPE metrics_system_GET gauge
metrics_system_GET{name=FAILED_REQUEST,} 0
metrics_system_GET{name=MEAN_RESPONSE_TIME,
} 8.942674506664073
metrics_system_GET{
name=MAX_RESPONSE_TIME,
} 40
metrics_system_GET{name=SUCCESS_REQUEST,} 2
metrics_system_GET{name=TOTAL_REQUEST,
} 2
Response Body
200
2.2 Method & Url
GET http://localhost:8080/metrics/statistics?type=json
Response Status
200
Response Body
{
  "metrics/POST": {
    "FAILED_REQUEST": 1,
    "MEAN_RESPONSE_TIME": 21,
    "MAX_RESPONSE_TIME": 21,
    "SUCCESS_REQUEST": 0,
    "TOTAL_REQUEST": 1
  },
  "metrics/backend/GET": {
    "FAILED_REQUEST": 0,
    "MEAN_RESPONSE_TIME": 12.6852124529148,
    "MAX_RESPONSE_TIME": 20,
    "SUCCESS_REQUEST": 2,
    "TOTAL_REQUEST": 2
  },
  "system/GET": {
    "FAILED_REQUEST": 1,
    "MEAN_RESPONSE_TIME": 2,
    "MAX_RESPONSE_TIME": 2,
    "SUCCESS_REQUEST": 0,
    "TOTAL_REQUEST": 1
  },
  "metrics/gauges/GET": {
    "FAILED_REQUEST": 0,
    "MEAN_RESPONSE_TIME": 7,
    "MAX_RESPONSE_TIME": 7,
    "SUCCESS_REQUEST": 1,
    "TOTAL_REQUEST": 1
  },
  "favicon.ico/GET": {
    "FAILED_REQUEST": 1,
    "MEAN_RESPONSE_TIME": 1,
    "MAX_RESPONSE_TIME": 1,
    "SUCCESS_REQUEST": 0,
    "TOTAL_REQUEST": 1
  },
  "metrics//GET": {
    "FAILED_REQUEST": 0,
    "MEAN_RESPONSE_TIME": 10,
    "MAX_RESPONSE_TIME": 10,
    "SUCCESS_REQUEST": 2,
    "TOTAL_REQUEST": 2
  },
  "metrics/system/GET": {
    "FAILED_REQUEST": 0,
    "MEAN_RESPONSE_TIME": 8.942674506664073,
    "MAX_RESPONSE_TIME": 40,
    "SUCCESS_REQUEST": 2,
    "TOTAL_REQUEST": 2
  }
}

3.System Metrics

System metrics mainly return the machine metrics, such as memory, threads, and other information.

Method & Url
GET http://localhost:8080/metrics/system
Response Status
200
Response Body
{
  "basic": {
    "mem": 1010,
    "mem_total": 911,
    "mem_used": 239,
    "mem_free": 671,
    "mem_unit": "MB",
    "processors": 20,
    "uptime": 137503,
    "systemload_average": -1.0
  },
  "heap": {
    "committed": 911,
    "init": 254,
    "used": 239,
    "max": 3596
  },
  "nonheap": {
    "committed": 98,
    "init": 2,
    "used": 95,
    "max": 0
  },
  "thread": {
    "peak": 82,
    "daemon": 34,
    "total_started": 108,
    "count": 82
  },
  "class_loading": {
    "count": 11495,
    "loaded": 11495,
    "unloaded": 0
  },
  "garbage_collector": {
    "ps_scavenge_count": 16,
    "ps_scavenge_time": 155,
    "ps_marksweep_count": 3,
    "ps_marksweep_time": 494,
    "time_unit": "ms"
  }
}

4.Backend Metrics

HugeGraph supports multiple backend storage, with backend metrics including memory, disk, and other information.

Method & Url
GET http://localhost:8080/metrics/backend
Response Status
200
Response Body
{
  "hugegraph": {
    "backend": "rocksdb",
    "nodes": 1,
    "cluster_id": "local",
    "servers": {
      "local": {
        "mem_unit": "MB",
        "disk_unit": "GB",
        "mem_used": 0.1,
        "mem_used_readable": "103.53 KB",
        "disk_usage": 0.03,
        "disk_usage_readable": "29.03 KB",
        "block_cache_usage": 0.00359344482421875,
        "block_cache_pinned_usage": 0.00359344482421875,
        "block_cache_capacity": 304.0,
        "estimate_table_readers_mem": 0.019697189331054688,
        "size_all_mem_tables": 0.07421875,
        "cur_size_all_mem_tables": 0.07421875,
        "estimate_live_data_size": 5.536526441574097E-5,
        "total_sst_files_size": 5.536526441574097E-5,
        "live_sst_files_size": 5.536526441574097E-5,
        "estimate_pending_compaction_bytes": 0.0,
        "estimate_num_keys": 0,
        "num_entries_active_mem_table": 0,
        "num_entries_imm_mem_tables": 0,
        "num_deletes_active_mem_table": 0,
        "num_deletes_imm_mem_tables": 0,
        "num_running_flushes": 0,
        "mem_table_flush_pending": 0,
        "num_running_compactions": 0,
        "compaction_pending": 0,
        "num_immutable_mem_table": 0,
        "num_snapshots": 0,
        "oldest_snapshot_time": 0,
        "num_live_versions": 38,
        "current_super_version_number": 38
      }
    }
  }
}

5.1.19 - Other API

Other REST API: Provide auxiliary functions such as version query, API listing, exception trace switch, IP allowlist and the Arthas agent.

11.1 Other

11.1.1 View Version Information of HugeGraph

Method & Url
GET http://localhost:8080/versions
Response Status
200
Response Body
{
    "versions": {
        "version": "v1",
        "core": "1.7.0",
        "gremlin": "3.5.1",
        "api": "0.72.0.0"
    }
}

11.1.2 View the profile of the server

Returns the service name, the core version, the documentation links and the API groups served by this node.

Method & Url
GET http://localhost:8080/
Response Status
200
Response Body

The swagger_ui value is derived from restserver.url, and apis lists the API groups registered on this node, sorted by name.

{
    "service": "hugegraph",
    "version": "1.7.0",
    "doc": "https://hugegraph.apache.org/docs/",
    "api_doc": "https://hugegraph.apache.org/docs/clients/",
    "swagger_ui": "http://127.0.0.1:8080/swagger-ui/index.html",
    "apis": [
        "arthas",
        "auth",
        "cypher",
        "filter",
        "graph",
        "gremlin",
        "job",
        "metrics",
        "profile",
        "raft",
        "schema",
        "space",
        "traversers",
        "variables"
    ]
}

11.1.3 List all APIs of the server

Lists every registered resource method, grouped by API group and resource class. Each entry carries the url, the HTTP method and the query parameters with their types and default values.

Method & Url
GET http://localhost:8080/apis
Response Status
200
Response Body

The response is long, the following fragment shows the shape:

{
    "apis": {
        "schema": {
            "PropertyKeyAPI": [
                {
                    "url": "graphspaces/{graphspace}/graphs/{graph}/schema/propertykeys",
                    "method": "GET",
                    "parameters": [
                        {
                            "name": "names",
                            "type": "java.util.List<java.lang.String>",
                            "default_value": null
                        }
                    ]
                }
            ]
        }
    }
}

11.1.4 View and switch the exception trace stack

Whether the error responses of the server carry the exception stack in the exception and cause fields is decided by the exception.allow_trace option (default true). The switch below is a node-wide runtime override: while it is on, the stack is always included, no matter what the option says. GET reports the state of that override, which starts as false.

Method & Url
GET http://localhost:8080/exception/trace
Response Status
200
Response Body
{
    "trace": false
}
Method & Url
PUT http://localhost:8080/exception/trace
Request Body
true
Response Status
200
Response Body
{
    "trace": true
}

11.1.5 Manage the IP allowlist, this operation requires administrator privileges

The allowlist is only enforced when it is switched on, see the white_ip.status option (default disable).

List the allowlist
Method & Url
GET http://localhost:8080/whiteiplist
Response Status
200
Response Body
{
    "whiteIpList": [
        "127.0.0.1"
    ]
}
Add IPs to or remove IPs from the allowlist
Params
  • ips: list of IPv4 addresses
  • action: load to add, remove to delete
Method & Url
POST http://localhost:8080/whiteiplist
Request Body
{
    "ips": [
        "10.0.0.1",
        "10.0.0.2"
    ],
    "action": "load"
}
Response Status
202
Response Body

existed_ips are the addresses already in the list, added_ips are the newly added ones, and illegal_ips is only returned when some addresses are not valid IPv4 addresses. For action=remove the response carries removed_ips and non_existed_ips instead.

{
    "existed_ips": [],
    "added_ips": [
        "10.0.0.1",
        "10.0.0.2"
    ]
}
Enable or disable the allowlist
Params
  • status: true to enable, false to disable
Method & Url
PUT http://localhost:8080/whiteiplist?status=true
Response Status
200
Response Body
{
    "WhiteIpListOpen": true
}

11.1.6 Start the Arthas agent

Attaches the Arthas agent to the running server process for diagnosis. The ports, the bind IP and the disabled commands are taken from the arthas.telnetPort, arthas.httpPort, arthas.ip and arthas.disabledCommands options, see Config Options.

Method & Url
PUT http://localhost:8080/arthas
Response Status
200
Response Body

The applied Arthas configuration is returned:

{
    "arthas.telnetPort": "8562",
    "arthas.httpPort": "8561",
    "arthas.ip": "0.0.0.0",
    "arthas.disabledCommands": "jad"
}

5.2 - HugeGraph Java Client

The code in this document is written in java, but its style is very similar to gremlin(groovy). The user only needs to replace the variable declaration in the code with def or remove it directly, You can convert java code into groovy; in addition, each line of statement can be without a semicolon at the end, groovy considers a line to be a statement. The gremlin(groovy) written by the user in HugeGraph-Studio can refer to the java code in this document, and some examples will be given below.

This reference follows Toolchain master (Client 1.8.0). Prepare the matching dependency using the quickstart; newly added APIs such as capability detection are unavailable in older clients.

1 HugeGraph-Client

HugeGraph-Client is the general entry for operating graph. Users must first create a HugeGraph-Client object and establish a connection (pseudo connection) with HugeGraph-Server before they can obtain the operation entry objects of schema, graph and gremlin.

HugeGraph-Client connects to an existing graph on the server. Its builder accepts a GraphSpace; the two-argument builder, or an empty GraphSpace value, uses DEFAULT.

// HugeGraphServer address: "http://localhost:8080"
// Graph Name: "hugegraph"
HugeClient hugeClient = HugeClient.builder("http://localhost:8080", "hugegraph")
                                //.builder("http://localhost:8080", "graphSpaceName", "hugegraph")
                                  .configTimeout(20) // 20s timeout
                                  // .configUser("username", "password") // set only when server auth is enabled
                                  .build();

If the above process of creating HugeClient fails, an exception will be thrown, and the user needs to use try-catch. If successful, continue to get schema, graph and gremlin manager.

When operating through gremlin in HugeGraph-Hubble(or HugeGraph-Studio), HugeClient is not required and can be ignored.

1.1 Builder options

The builder accepts the following options. Every timeout is expressed in seconds and converted to milliseconds internally.

interfacedescriptiondefault
configUrl(String url)Server address, normally passed to builder(...) alreadyrequired
configGraph(String graph)Graph name, normally passed to builder(...) alreadyrequired
configGraphSpace(String graphSpace)GraphSpace name, a null or empty value falls back to DEFAULTDEFAULT
configUser(String username, String password)Credentials for the server, a null value is stored as an empty stringempty, no auth
configToken(String token)Token used instead of username and passwordempty
configTimeout(int seconds)Request timeout; use a positive number of seconds20
configConnectTimeout(Integer seconds)Connect timeout, left unset so that configTimeout appliesunset
configReadTimeout(Integer seconds)Read timeout, left unset so that configTimeout appliesunset
configPool(int maxConns, int maxConnsPerRoute)Connection pool sizes, passing 0 for either one restores its default4 x CPUs, 2 x CPUs
configIdleTime(int seconds)Idle connection keep-alive, must be greater than 030
configSSL(String trustStoreFile, String trustStorePassword)Truststore used for HTTPS connectionsempty
configHttpBuilder(Consumer<OkHttpClient.Builder> consumer)Callback that receives the underlying OkHttp builder for further customizationnone
graphRequired(boolean graphRequired)Whether build() rejects an empty url or graph nametrue

Current master has a unit-conversion defect in configTimeout(0): it sets 20,000 seconds instead of the default 20 seconds. Omit the call to keep the default, or use configTimeout(20) explicitly; do not use 0 to reset it.

On build(), the client reads the server API version and rejects anything outside the range [0.38, 0.81).

1.2 Operation entries

Besides schema, graph and gremlin, HugeClient exposes the following entries. The graph-scoped ones are only available when a graph name was supplied; when the client is built with an empty graph name they return null until assignGraph(graphSpace, graph) is called.

interfacereturnsscopedescription
schema()SchemaManagergraphManage PropertyKey, VertexLabel, EdgeLabel and IndexLabel
graph()GraphManagergraphAdd, query, update and delete vertices and edges, single or batch
gremlin()GremlinManagergraphRun Gremlin statements, synchronously or as an async task
cypher()CypherManagergraphRun Cypher statements, synchronously or as an async task
traverser()TraverserManagergraphRESTful traversals such as shortest path, k-out, k-neighbor and crosspoints
variables()VariablesManagergraphGet, set, list and remove graph variables
job()JobManagergraphRebuild the index of a VertexLabel, EdgeLabel or IndexLabel
task()TaskManagergraphList, get, cancel, delete and wait on async tasks
computer()ComputerManagergraphCreate, cancel, list and get computer jobs
graphs()GraphsManagergraphspaceCreate, clone, list, reload, clear and drop graphs, read and set the graph mode
graphSpace()GraphSpaceManagerserverManage GraphSpaces, see section 4
auth()AuthManagerserverManage users, groups, targets, belongs and accesses
metrics()MetricsManagerserverRead backend, system and statistics metrics
versionManager()VersionManagerserverRead the core, gremlin and API versions of the server

The client also reports what the connected server supports, so callers can branch on a capability instead of on a version string: supportsGraphSpace(), supportsCypher(), supportsGraphCreate(), supportsDefaultRole() and isServerAuthEnabled().

2 Schema

2.1 SchemaManager

SchemaManager is used to manage four kinds of schema in HugeGraph, namely PropertyKey (property type), VertexLabel (vertex type), EdgeLabel (edge type) and IndexLabel (index label). A SchemaManager object can be created for schema information definition.

The user can obtain the SchemaManager object using the following methods:

SchemaManager schema = hugeClient.schema()

Create a schema object via gremlin in HugeGraph-Hubble:

schema = graph.schema()

The definition process of the 4 kinds of schema is described below.

2.2 PropertyKey

2.2.1 Interface and parameter introduction

PropertyKey is used to standardize the property constraints of vertices and edges, and properties of properties are not currently supported.

The constraint information that PropertyKey allows to define includes: name, datatype, cardinality, aggregateType, writeType and userdata, which are introduced one by one below.

  • name: The name of the property, used to distinguish different PropertyKeys, PropertyKeys with the same name are not allowed.
interfaceparammust set
propertyKey(String name)namey
  • datatype: property value type, you must select an explicit setting from the following table that conforms to the specific business scenario:
interfaceJava Class
asText()String
asInt()Integer
asDate()Date
asUUID()UUID
asBoolean()Boolean
asByte()Byte
asBlob()Byte[]
asDouble()Double
asFloat()Float
asLong()Long
  • cardinality: Whether the property value is single-valued or multivalued, in the case of multivalued, it is divided into allowing-duplicate values and not-allowing-duplicate values. This item is single by default. If necessary, you can select a setting from the following table:
interfacecardinalitydescription
valueSingle()singlesingle value
valueList()listmulti-values that allow duplicate value
valueSet()setmulti-values that not allow duplicate value
  • aggregateType: How repeated writes of the same property are combined. The default is none, which keeps the last written value. The numeric options require a number datatype:
interfaceaggregateTypedescription
calcSum()sumaccumulate the written values
calcMax()maxkeep the greatest value
calcMin()minkeep the smallest value
calcOld()oldkeep the first written value and ignore updates

aggregateType(AggregateType type) sets the same thing directly, and AggregateType.NONE restores the default.

  • writeType: Whether the property belongs to the OLTP graph or to an OLAP computing result, and for OLAP whether it carries an index. The default is oltp:
writeTypedescription
OLTPordinary graph property
OLAP_COMMONOLAP property without index
OLAP_SECONDARYOLAP property with a secondary index
OLAP_RANGEOLAP property with a range index
interfacedescription
writeType(WriteType writeType)set the write type with the enum value
writeType(String name)set the write type by enum name
  • userdata: Users can add some constraints or additional information by themselves, and then check whether the incoming properties satisfy the constraints, or extract additional information when necessary:
interfacedescription
userdata(String key, Object value)The same key, the latter will cover the former
2.2.2 Create PropertyKey
schema.propertyKey("name").asText().valueSet().ifNotExist().create()

The syntax of creating the above PropertyKey object through gremlin in HugeGraph-Hubble is exactly the same. If the user does not define the schema variable, it should be written like this:

graph.schema().propertyKey("name").asText().valueSet().ifNotExist().create()

In the following examples, the syntax of gremlin and java is exactly the same, so we won’t repeat them.

  • ifNotExist(): Add a judgment mechanism for create, if the current PropertyKey already exists, it will not be created, otherwise the property will be created. If no ifNotExist() is added, an exception will be thrown if a property-key with the same name already exists. The same as below, and will not be repeated there.
2.2.3 Delete PropertyKey
schema.propertyKey("name").remove()
2.2.4 Query PropertyKey
// Get PropertyKey
schema.getPropertyKey("name")

// Get attributes of PropertyKey
schema.getPropertyKey("name").cardinality()
schema.getPropertyKey("name").dataType()
schema.getPropertyKey("name").name()
schema.getPropertyKey("name").userdata()

2.3 VertexLabel

2.3.1 Interface and parameter introduction

VertexLabel is used to define the vertex type and describe the constraint information of the vertex.

The constraint information that VertexLabel allows to define include: name, idStrategy, properties, primaryKeys, nullableKeys and ttl, which are introduced one by one below.

  • name: The name of the VertexLabel, used to distinguish different VertexLabels, VertexLabels with the same name are not allowed.
interfaceparammust set
vertexLabel(String name)namey
  • idStrategy: Each VertexLabel can choose its own ID strategy. There are currently three strategies to choose from, namely Automatic (automatically generated), Customize (user input) and PrimaryKey (primary attribute key). Among them, Automatic uses the Snowflake algorithm to generate ID, Customize requires the user to pass in the ID of string or number type, and PrimaryKey allows the user to select several properties of VertexLabel as the basis for differentiation. HugeGraph will be spliced and generated ID according to the value of the primary properties. idStrategy uses Automatic by default, but if the user does not explicitly set idStrategy and calls the primaryKeys(…) method to set the primary property, then idStrategy will automatically use PrimaryKey.
interfaceidStrategydescription
useAutomaticIdAUTOMATICgenerate id automatically by Snowflake algorithm
useCustomizeStringIdCUSTOMIZE_STRINGpassed id by user, must be string type
useCustomizeNumberIdCUSTOMIZE_NUMBERpassed id by user, must be number type
useCustomizeUuidIdCUSTOMIZE_UUIDpassed id by user, must be UUID type
usePrimaryKeyIdPRIMARY_KEYchoose some important prop as primary key to splice id
  • properties: define the properties of the vertex, the incoming parameter is the name of the PropertyKey.
interfacedescription
properties(String… properties)allow to pass multi properties
  • primaryKeys: When the user selects the ID strategy of PrimaryKey, several primary properties need to be selected from the properties of VertexLabel as the basis for differentiation;
interfacedescription
primaryKeys(String… keys)allow to choose multi prop as primaryKeys

Note that the selection of the ID strategy and the setting of primaryKeys have some mutual constraints, which cannot be called at will. The constraints are shown in the following table:

useAutomaticIduseCustomizeStringIduseCustomizeNumberIdusePrimaryKeyId
unset primaryKeysAUTOMATICCUSTOMIZE_STRINGCUSTOMIZE_NUMBERERROR
set primaryKeysERRORERRORERRORPRIMARY_KEY

The client itself only checks that the ID strategy is set once, so calling two of these methods on the same builder fails locally. The combinations above are validated by the server.

  • nullableKeys: For properties set by the properties(…) method, all of them are non-nullable by default, that is, the property must be assigned a value when creating a vertex, which may impose too strict integrity requirements on user data. In order to avoid such strong constraints, the user can set some properties to be nullable through this method, so that the properties can be unassigned when adding vertices.
interfacedescription
nullableKeys(String… properties)allow to pass multi props

Note: primaryKeys and nullableKeys cannot intersect, because a property cannot be both primary and nullable.

  • ttl: Time to live of the vertices of this label. The default is 0, which means they never expire. The client rejects a negative value. By default the countdown is relative to the moment the vertex is written; ttlStartTime instead names a date property of the label that the countdown is measured from.
interfacedescription
ttl(long ttl)set the time to live, 0 disables expiry
ttlStartTime(String property)name the date property the countdown starts from
  • enableLabelIndex: The user can specify whether to create an index for the label. If you don’t create it, you can’t globally search for the vertices and edges of the specified label. If you create it, you can search globally, like g.V().hasLabel('person'), g.E().has('label', 'person') query, but the performance will be slower when inserting data, and it will take up more storage space. This defaults to true.
interfacedescription
enableLabelIndex(boolean enable)Whether to create a label index
  • userdata: Users can add some constraints or additional information by themselves, and then check whether the incoming properties meet the constraints, or extract additional information when necessary.
interfacedescription
userdata(String key, Object value)The same key, the latter will cover the former
2.3.2 Create VertexLabel
// Use Automatic Id strategy
schema.vertexLabel("person").properties("name", "age").ifNotExist().create();
schema.vertexLabel("person").useAutomaticId().properties("name", "age").ifNotExist().create();

// Use Customize_String Id strategy
schema.vertexLabel("person").useCustomizeStringId().properties("name", "age").ifNotExist().create();
// Use Customize_Number Id strategy
schema.vertexLabel("person").useCustomizeNumberId().properties("name", "age").ifNotExist().create();
// Use Customize_Uuid Id strategy
schema.vertexLabel("person").useCustomizeUuidId().properties("name", "age").ifNotExist().create();

// Use PrimaryKey Id strategy
schema.vertexLabel("person").properties("name", "age").primaryKeys("name").ifNotExist().create();
schema.vertexLabel("person").usePrimaryKeyId().properties("name", "age").primaryKeys("name").ifNotExist().create();
2.3.3 Update VertexLabel

VertexLabel can append constraints, but only properties and nullableKeys, and the appended properties must also be added to the nullableKeys collection.

schema.vertexLabel("person").properties("price").nullableKeys("price").append();
2.3.4 Delete VertexLabel
schema.vertexLabel("person").remove();
2.3.5 Query VertexLabel
// Get VertexLabel
schema.getVertexLabel("name")

// Get attributes of VertexLabel
schema.getVertexLabel("person").idStrategy()
schema.getVertexLabel("person").primaryKeys()
schema.getVertexLabel("person").name()
schema.getVertexLabel("person").properties()
schema.getVertexLabel("person").nullableKeys()
schema.getVertexLabel("person").userdata()
schema.getVertexLabel("person").ttl()
schema.getVertexLabel("person").ttlStartTime()

2.4 EdgeLabel

2.4.1 Interface and parameter introduction

EdgeLabel is used to define the edge type and describe the constraint information of the edge.

The constraint information that EdgeLabel allows to define include: name, sourceLabel, targetLabel, frequency, properties, sortKeys, nullableKeys and ttl, which are introduced one by one below.

  • name: The name of the EdgeLabel, used to distinguish different EdgeLabels, EdgeLabels with the same name are not allowed.
interfaceparammust set
edgeLabel(String name)namey
  • sourceLabel and targetLabel: The names of the source and the target vertex type of the edge link. Setting both is the same as declaring one link pair.

  • link: An EdgeLabel holds a set of source and target pairs, so link(...) can be called more than once to let the same edge type connect several pairs of vertex types. Once a pair has been added this way, sourceLabel(...) and targetLabel(...) are rejected, and the sourceLabel() and targetLabel() getters only work on a label that has exactly one pair. Use links() to read them all.

interfaceparammust set
link(String sourceLabel, String targetLabel)sourceLabel, targetLabely, or set the two below
sourceLabel(String label)labely, unless link() was used
targetLabel(String label)labely, unless link() was used
  • frequency: Indicating the number of times a relationship occurs between two specific vertices, which can be single (single) or multiple (frequency), the default is single.
interfacefrequencydescription
singleTime()singlea relationship can only occur once
multiTimes()multiplea relationship can occur many times
  • properties: Define the properties of the edge.
interfacedescription
properties(String… properties)allow to pass multi props
  • sortKeys: When the frequency of EdgeLabel is multiple, some properties are needed to distinguish the multiple relationships, so sortKeys (sorted keys) is introduced;
interfacedescription
sortKeys(String… keys)allow to choose multi prop as sortKeys
  • nullableKeys: Consistent with the concept of nullableKeys in vertices.

Note: sortKeys and nullableKeys also cannot intersect.

  • ttl: Consistent with the concept of ttl in vertices, with the same ttl(long ttl) and ttlStartTime(String property) methods and the same default of 0.

  • edge label type: An EdgeLabel is normal by default. It can instead be declared as the parent of a family of edge labels, as a child of such a parent, or as a general label:

interfaceedgeLabelTypedescription
asBase()PARENTdeclare the label as a parent label
withBase(String parentLabel)SUBdeclare the label as a child of ‘parentLabel’
asGeneral()GENERALdeclare the label as a general label
  • enableLabelIndex: It is consistent with the concept of enableLabelIndex in the vertex.

  • userdata: Users can add some constraints or additional information by themselves, and then check whether the incoming properties meet the constraints, or extract additional information when necessary.

interfacedescription
userdata(String key, Object value)The same key, the latter will cover the former
2.4.2 Create EdgeLabel
schema.edgeLabel("knows").link("person", "person").properties("date").ifNotExist().create();
schema.edgeLabel("created").multiTimes().link("person", "software").properties("date").sortKeys("date").ifNotExist().create();
2.4.3 Update EdgeLabel
schema.edgeLabel("knows").properties("price").nullableKeys("price").append();
2.4.4 Delete EdgeLabel
schema.edgeLabel("knows").remove();
2.4.5 Query EdgeLabel
// Get EdgeLabel
schema.getEdgeLabel("knows")

// Get attributes of EdgeLabel
schema.getEdgeLabel("knows").frequency()
schema.getEdgeLabel("knows").sourceLabel()
schema.getEdgeLabel("knows").targetLabel()
schema.getEdgeLabel("knows").sortKeys()
schema.getEdgeLabel("knows").name()
schema.getEdgeLabel("knows").properties()
schema.getEdgeLabel("knows").nullableKeys()
schema.getEdgeLabel("knows").userdata()
schema.getEdgeLabel("knows").ttl()
schema.getEdgeLabel("knows").ttlStartTime()
schema.getEdgeLabel("knows").edgeLabelType()
// All the source and target pairs, safe to call when there is more than one
schema.getEdgeLabel("knows").links()

2.5 IndexLabel

2.5.1 Interface and parameter introduction

IndexLabel is used to define the index type and describe the constraint information of the index, mainly for the convenience of query.

The constraint information that IndexLabel allows to define include: name, baseType, baseValue, indexFields, indexType, which are introduced one by one below.

  • name: The name of the IndexLabel, used to distinguish different IndexLabels, IndexLabels with the same name are not allowed.
interfaceparammust set
indexLabel(String name)namey
  • baseType: Indicates whether to index VertexLabel or EdgeLabel, used in conjunction with the baseValue below.

  • baseValue: Specifies the name of the VertexLabel or EdgeLabel to be indexed.

interfaceparamdescription
onV(String baseValue)baseValuebuild index for VertexLabel: ‘baseValue’
onE(String baseValue)baseValuebuild index for EdgeLabel: ‘baseValue’
  • indexFields: on which fields to index, it can be a joint index for multiple columns.
interfaceparamdescription
by(String… fields)filesallow to build index for multi fields for secondary index
  • indexType: There are currently five types of indexes established, namely Secondary, Range, Search, Shard and Unique.
    • Secondary Index supports exact matching secondary index, allow to build joint index, joint index supports index prefix search
      • Single Property Secondary Index, support equality query, for example: the secondary index of the city property of the person vertex, you can use g.V().has("city", "Beijing") to query all the vertices with “city attribute value is Beijing”
      • Joint Secondary Index, supports prefix query and equality query, such as: joint index of city and street properties of person vertex, you can use g.V().has("city", "Beijing").has('street', 'Zhongguancun street ') to query all vertices of “city property value is Beijing and street property value is ZhongGuanCun”, or g.V().has("city", "Beijing") to query all vertices of “city property value is Beijing”.

      The query of Secondary Index is based on the query condition of “yes” or “equal”, and does not support “partial matching”.

    • Range Index supports for range queries of numeric types
      • Must be a single number or date attribute, for example: the range index of the age property of the person vertex, you can use g.V().has("age", P.gt(18)) to query the vertices with “age property value greater than 18” . In addition to P.gt(), also supports P.gte(), P.lte(), P.lt(), P.eq(), P.between() , P.inside() and P.outside() etc.
    • Search Index supports full-text search
      • It must be a single text property, such as: full-text index of the address property of the person vertex, you can use g.V().has("address", Text.contains('building') to query all vertices whose “address property contains a ‘building’”

      The query of the Search Index is based on the query condition of “is” or “contains”.

    • Shard Index supports prefix matching + numeric range query
      • The shard index of N properties supports range queries with equal prefixes. For example, the shard index of the city and age properties of the person vertex can use g.V().has("city", "Beijing").has ("age", P.between(18, 30))Query “city property is Beijing and all vertices whose age is greater than or equal to 18 and less than 30”.
      • When all N properties are text properties in a Shard Index, it is equivalent to Secondary Index.
      • When there is only one single number or date property in a Shard Index, it is equivalent to the Range Index.

      Shard Index can have any number or date property, but at most one range search condition can be provided when querying, and the prefix properties of the Shard Search conditions must be “equals”.

    • Unique Index supports properties uniqueness constraints, that is, the value of properties can be limited to not repeat, and joint indexing is allowed, but querying is not supported now
      • The unique index of single or multiple properties cannot be used for query, only the value of the property can be limited, and an error will be reported when there is a duplicate value.
interfaceindexTypedescription
secondary()Secondarysupport prefix search
range()Rangesupport range(numeric or date type) search
search()Searchsupport full text search
shard()Shardsupport prefix + range(numeric or date type) search
unique()Uniquesupport unique props value, not support search
2.5.2 Create IndexLabel

These examples assume a person vertex label with name and age, and a created edge label with date. First add the indexed properties below; id is a business property, separate from the internal vertex ID.

schema.propertyKey("lived").asText().ifNotExist().create();
schema.propertyKey("city").asText().ifNotExist().create();
schema.propertyKey("id").asInt().ifNotExist().create();
schema.vertexLabel("person").properties("lived", "city", "id")
      .nullableKeys("lived", "city", "id").append();

schema.indexLabel("personByAge").onV("person").by("age").range().ifNotExist().create();
schema.indexLabel("createdByDate").onE("created").by("date").secondary().ifNotExist().create();
schema.indexLabel("personByLived").onV("person").by("lived").search().ifNotExist().create();
schema.indexLabel("personByCityAndAge").onV("person").by("city", "age").shard().ifNotExist().create();
schema.indexLabel("personById").onV("person").by("id").unique().ifNotExist().create();
2.5.3 Delete IndexLabel
schema.indexLabel("personByAge").remove()
2.5.4 Query IndexLabel
// Get IndexLabel
schema.getIndexLabel("personByAge")

// Get attributes of IndexLabel
schema.getIndexLabel("personByAge").baseType()
schema.getIndexLabel("personByAge").baseValue()
schema.getIndexLabel("personByAge").indexFields()
schema.getIndexLabel("personByAge").indexType()
schema.getIndexLabel("personByAge").name()

3 Graph

3.1 Vertex

Vertices are the most basic elements of a graph, and there can be many vertices in a graph. Here is an example of adding vertices:

Vertex marko = graph.addVertex(T.LABEL, "person", "name", "marko", "age", 29);
Vertex lop = graph.addVertex(T.LABEL, "software", "name", "lop", "lang", "java", "price", 328);
  • The key to adding vertices is the vertex properties. The number of parameters of the vertex adding function must be an even number and satisfy the order of key1 -> val1, key2 -> val2 ..., and the order between key-value pairs is free .
  • The parameter must contain a special key-value pair, namely T.LABEL -> "val", which is used to define the category of the vertex, so that the program can obtain the schema definition of the VertexLabel from the cache or backend, and then do subsequent constraint checks. The label in the example is defined as person. T.LABEL is the constant "label", so the plain string works just as well.
  • If the vertex type’s ID policy is AUTOMATIC, users are not allowed to pass in id key-value pairs.
  • If the ID policy of the vertex type is CUSTOMIZE_STRING, the user needs to pass in the value of the id of the String type. The key-value pair is like: T.ID, "123456".
  • If the ID policy of the vertex type is CUSTOMIZE_NUMBER, the user needs to pass in the value of the id of the Number type. The key-value pair is like: T.ID, 123456.
  • If the ID policy of the vertex type is PRIMARY_KEY, the parameters must also contain the name and value of the properties corresponding to the primaryKeys, if not set an exception will be thrown. For example, the primaryKeys of person is name, in the example, the value of name is set to marko.
  • For properties that are not nullableKeys, a value must be assigned.
  • The remaining parameters are the settings of other properties of the vertex, but they are not required.
  • After calling the addVertex method, the vertices are inserted into the backend storage system immediately.

3.2 Edge

After added vertices, edges are also needed to form a complete graph. Here is an example of adding edges:

Edge knows1 = marko.addEdge("knows", vadas, "city", "Beijing");
  • The function addEdge() of the (source) vertex is to add an edge(relationship) between itself and another vertex. The first parameter of the function is the label of the edge, and the second parameter is the target vertex. The position and order of these two parameters are fixed. The subsequent parameters are the order of key1 -> val1, key2 -> val2 ..., set the properties of the edge, and the key-value pair order is free.
  • The source and target vertices must conform to the definitions of source-label and target label in EdgeLabel, and cannot be added arbitrarily.
  • For properties that are not nullableKeys, a value must be assigned.

Note: When frequency is multiple, the value of the property type corresponding to sortKeys must be set.

4 GraphSpace

The client can manage multiple GraphSpaces in one physical deployment, and each GraphSpace can contain multiple graphs. When no GraphSpace is specified, it uses DEFAULT.

GraphSpaces need a server of core version 1.7.0 or later. Against an older server the client falls back to a legacy profile, and hugeClient.supportsGraphSpace() returns false.

4.1 Create GraphSpace

GraphSpaceManager spaceManager = hugeClient.graphSpace();

// Define GraphSpace configuration
GraphSpace graphSpace = new GraphSpace();
graphSpace.setName("myGraphSpace");
graphSpace.setDescription("Business data graph space");
graphSpace.setMaxGraphNumber(10);  // Maximum number of graphs
graphSpace.setMaxRoleNumber(100);  // Maximum number of roles

// Create GraphSpace
spaceManager.createGraphSpace(graphSpace);

4.2 GraphSpace Interface Summary

CategoryInterfaceDescription
Manager - QuerylistGraphSpace()Get all GraphSpace names
listProfile() / listProfile(String prefix)Get GraphSpace profiles
getGraphSpace(String name)Get the specified GraphSpace
getDefault()Get the default GraphSpace
Manager - Create/UpdatecreateGraphSpace(GraphSpace)Create a GraphSpace
updateGraphSpace(GraphSpace)Update configuration
setDefault(String name)Set the default GraphSpace
Manager - DeletedeleteGraphSpace(String name)Delete the specified GraphSpace
Manager - Default rolesetDefaultRole(String name, String user, String role)Grant a default role, optionally scoped to a graph with a fourth argument
checkDefaultRole(String name, String user, String role)Check a default role, optionally scoped to a graph with a fourth argument
deleteDefaultRole(String name, String user, String role)Revoke a default role, optionally scoped to a graph with a fourth argument
GraphSpace - PropertiesgetName() / getNickname() / getDescription()Get name / nickname / description
getGraphNumberUsed() / getRoleNumberUsed()Get the number of graphs / roles in use
getCpuUsed() / getMemoryUsed() / getStorageUsed()Get the resources in use
getCreateTime() / getUpdateTime()Get the creation / update time
GraphSpace - ConfigurationsetDescription(String) / setNickname(String)Set description / nickname
setMaxGraphNumber(int) / setMaxRoleNumber(int)Set the maximum number of graphs / roles
setCpuLimit(int) / setMemoryLimit(int) / setStorageLimit(int)Set the resource quotas
setConfigs(Map<String, Object>)Set extra configuration entries

5 Simple Example

Simple examples can reference HugeGraph-Client

5.3 - Gremlin-Console

Gremlin-Console is an interactive client developed by TinkerPop. Users can use this client to perform various operations on Graph. There are two main usage modes:

  • Stand-alone offline mode
  • Client/Server mode

Note: Gremlin-Console is only for users to quickly get started and experience, it is not recommended for use in production environments.

1 Stand-alone offline mode

Since the lib directory already contains the HugeCore jar package, and HugeGraph-Server has been registered in the Console as a plug-in, the users can write a groovy script directly to call the code of HugeGraph-Core, and then hand it over to the parsing engine in Gremlin-Console for execution. As a result, the users can operate the graph without starting the Server.

Here is an example, first modify the hugegraph.properties configuration to use the Memory backend (using other backends may encounter some initialization issues):

backend=memory
serializer=text

Then enter the following command:

> ./bin/gremlin-console.sh -- -i scripts/example.groovy

         \,,,/
         (o o)
-----oOOo-(3)-oOOo-----
plugin activated: HugeGraph
plugin activated: tinkerpop.server
plugin activated: tinkerpop.utilities
plugin activated: tinkerpop.tinkergraph
main dict load finished, time elapsed 644 ms
model load finished, time elapsed 35 ms.
>>>> query all vertices: size=6
>>>> query all edges: size=6
gremlin> 

The -- here will be parsed by getopts as the last option, allowing the subsequent options to be passed to Gremlin-Console for processing. -i represents Execute the specified script and leave the console open on completion. For more options, you can refer to the source code of Gremlin-Console.

example.groovy is an example script under the scripts directory. This script inserts some data and queries the number of vertices and edges in the graph at the end.

You can continue to enter Gremlin statements to operate on the graph:

gremlin> g.V()
==>v[2:lop]
==>v[1:josh]
==>v[1:marko]
==>v[1:peter]
==>v[1:vadas]
==>v[2:ripple]
gremlin> g.E()
==>e[S1:josh>2>>S2:lop][1:josh-created->2:lop]
==>e[S1:josh>2>>S2:ripple][1:josh-created->2:ripple]
==>e[S1:marko>1>>S1:josh][1:marko-knows->1:josh]
==>e[S1:marko>1>>S1:vadas][1:marko-knows->1:vadas]
==>e[S1:marko>2>>S2:lop][1:marko-created->2:lop]
==>e[S1:peter>2>>S2:lop][1:peter-created->2:lop]
gremlin> 

For more Gremlin statements, please refer to Tinkerpop Official Website

2 Client/Server mode

Gremlin Console connects to HugeGraph Server through WebSocket. The default configuration uses WsAndHttpChannelizer, which handles both WebSocket and HTTP requests, so there is no need to switch the Channelizer.

# vim conf/gremlin-server.yaml
# ......
channelizer: org.apache.tinkerpop.gremlin.server.channel.WsAndHttpChannelizer
# ......

Confirm that host and port match the settings in remote.yaml, and then follow the steps to start HugeGraph Server.

Then enter Gremlin-Console:

> ./bin/gremlin-console.sh

         \,,,/
         (o o)
-----oOOo-(3)-oOOo-----
plugin activated: HugeGraph
plugin activated: tinkerpop.server
plugin activated: tinkerpop.utilities
plugin activated: tinkerpop.tinkergraph

To connect to the server, you need to specify the connection parameters in the configuration file, and there is a default remote.yaml file in the conf directory

# cat conf/remote.yaml
hosts: [localhost]
port: 8182
serializer: {
  className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0,
  config: {
    serializeResultToString: false,
    ioRegistries: [org.apache.hugegraph.io.HugeGraphIoRegistry]
  }
}

If the Server runs in auth mode, add the credentials to the same file:

username: admin
password: pa

The conf directory also ships remote-objects.yaml and gremlin-driver-settings.yaml, which carry the same host, port, and serializer settings.

gremlin> :remote connect tinkerpop.server conf/remote.yaml
==>Configured localhost/127.0.0.1:8182

Server-side graphs are bound under a graphspace-qualified name, so the graph hugegraph in graphspace DEFAULT is bound as DEFAULT-hugegraph and its traversal source as __g_DEFAULT-hugegraph. A bare hugegraph does not resolve on the Server, and DEFAULT-hugegraph is not a valid Groovy identifier, so a remote script reaches the traversal source through an alias. If the sample graph was preloaded when HugeGraph-Server started, a query looks like this:

gremlin> import org.apache.tinkerpop.gremlin.driver.Cluster
gremlin> cluster = Cluster.open('conf/remote.yaml')
gremlin> client = cluster.connect().alias(['g': '__g_DEFAULT-hugegraph'])
gremlin> client.submit('g.V().count()').all().get()[0].object
==>6
gremlin> client.submit('g.V().toList().size()').all().get()[0].object
==>6
gremlin> client.close(); cluster.close()

NOTE: In Client/Server mode, all operations related to the Server should be prefixed with :> . If not added, it indicates local console operations. A :> script carries no alias, so it can only use names the Server itself has bound.

For more information on the use of Gremlin-Console, please refer to Tinkerpop Official Website

6 - GUIDES

This section covers HugeGraph architecture, design, backup and restore, plugin development, security settings, and frequently asked questions.

6.1 - HugeGraph Architecture Overview

Full-Stack Components

The HugeGraph ecosystem combines a graph database, graph computing engines, graph AI, and a toolchain with distinct responsibilities. HugeGraph Server provides OLTP graph database services; HugeGraph-Computer and Vermeer are independent OLAP engines; HugeGraph-AI provides graph AI capabilities. Toolchain clients, import tools, visualization, and operations tools provide entry points for applications and operators.

HugeGraph architecture: Toolchain, AI, Computer, and Vermeer connections to Server, PD, and Store

Figure: Server uses HStore as an example backend; see the Mermaid diagram below for independent backends such as RocksDB and HBase. Vermeer and HugeGraph-Computer are independent OLAP engines. Dashed arrows indicate optional integration.

Ecosystem Integration

flowchart TB
  Apps["Applications / Operations"] --> Toolchain["Toolchain"] --> Server["Server REST API"]
  AI["AI"] -. REST .-> Server
  Computer["Computer"] -. REST .-> Server
  Vermeer["Vermeer"] -. PD .-> PD["PD"]
  Vermeer -. Scan .-> Store["Store"]
  Vermeer -. Write .-> Server

Optional connections do not mean every AI, Computer, or Vermeer task must connect to Server, PD, or Store. Computer and AI read and write through the Server REST API. Vermeer’s HugeGraph input queries PD for partition metadata and scans HStore partitions directly through Store. With HugeGraph result output, it writes results back through the Server REST API.

Server Internals

flowchart TB
  API["REST API / Gremlin"] --> Core["Core"] --> Adapter["Backend adapter"]
  Adapter --> RocksDB["RocksDB"]
  Adapter --> HBase["HBase"]
  Adapter --> HStore["HStore"]
  HStore --> PD["PD"]
  HStore --> Store["Store"]

Server REST APIs and Gremlin send requests to Core, which reads and writes through a backend adapter. RocksDB and HBase are independent backends. The HStore adapter obtains cluster metadata and partition information from PD and sends graph operations to Store nodes. PD manages metadata and partitions; it does not carry graph data reads and writes.

Current Server implementations include RocksDB, HBase, HStore, and memory backends. Memory is for testing or temporary use, not a persistent production deployment. See the Server quick start for production deployment options.

Main Toolchain components:

  • Hubble: Connects to the graph database through a web interface to manage schemas, import data, and run queries.
  • Loader: Converts multiple data sources and imports them into graph data in batches.
  • Tools: Provides command-line deployment, management, backup, and restore operations.
  • Client: Wraps Server connections, schema management, graph reads and writes, and queries. Clients are available for Java, Python, and Go; a Rust client is under development.

Independent Graph Computing Engines

  • Vermeer: Provides independent graph computing services and algorithm APIs.
  • HugeGraph-Computer: A Java distributed graph computing engine based on BSP/Pregel.

Historical Architecture

The following diagram is retained for historical reference. Its depiction of Server providing both OLTP and OLAP, and of various old backends, does not describe current master.

Expand the historical architecture diagram
Historical HugeGraph architecture, for reference only

6.2 - HugeGraph Design Concepts

1. Property Graph

Version note: This page retains early design material. The property graph model and vertex ID strategies remain consistent with current master, but figures, partitioning conclusions, and transaction descriptions have the scope stated in each section. They are not current deployment or storage implementation guarantees.

There are two common graph data representation models, namely the RDF (Resource Description Framework) model and the Property Graph (Property Graph) model. Both RDF and Property Graph are the most basic and well-known graph representation modes, and both can represent entity-relationship modeling of various graphs. RDF is a W3C standard, while Property Graph is an industry standard and is widely supported by graph database vendors. HugeGraph currently uses Property Graph.

HugeGraph uses the property graph model. This person-and-book example shows labels and properties on vertices and directed edges. It is a logical model, not an illustration of the current internal data layout.

Property graph logical model: a Person vertex reads a Book vertex; vertices and the edge have separate labels and properties

Figure: Vertices and edges each carry their own labels and properties.

Inside HugeGraph, each vertex/edge is identified by a unique VertexId/EdgeId, and the attributes are stored inside the corresponding vertex/edge. The relationship/mapping between vertices is stored through edges.

When the vertex attribute value is stored by edge pointer, if you want to update a vertex-specific attribute value, you can directly write it by overwriting. The disadvantage is that the VertexId is redundantly stored; if you want to update the attribute of the relationship, you need to use the read-and-modify method , read all attributes first, modify some attributes, and then write to the storage system, the update efficiency is low. According to experience, there are more requirements for modifying vertex attributes, but less for edge attributes. For example, calculations such as PageRank and Graph Cluster require frequent modification of vertex attribute values.

2. Graph Partition Scheme

Version note: The Edge Cut and Vertex Cut comparison is historical design background. Current PD and Store code does not define “HugeGraph uses Edge Cut” as an implementation contract. Do not use that conclusion to describe current partitioning or replication; see the HStore quick start for current distributed deployments.

For distributed graph databases, there are two partition storage methods for graphs: Edge Cut and Vertex Cut, as shown in the following figure. When using the Edge Cut method to store graphs, any vertex will only appear on one machine, while edges may be distributed on different machines. This storage method may lead to multiple storage of edges. When using the Vertex Cut method to store graphs, any edge will only appear on one machine, and each same point may be distributed to different machines. This storage method may result in multiple storage of vertices.

image

The EdgeCut partition scheme can support high-performance insert and update operations, while the VertexCut partition scheme is more suitable for static graph query analysis, so EdgeCut is suitable for OLTP graph query, and VertexCut is more suitable for OLAP graph query. Early design material described HugeGraph as using Edge Cut. That historical conclusion is retained here and does not describe the current partitioning implementation.

3. VertexId Strategy

Vertex of HugeGraph supports four ID strategies. Different VertexLabels in the same graph database can use different Id strategies. Currently, the Id strategies supported by HugeGraph are:

  • Automatic generation (AUTOMATIC): Use the Snowflake algorithm to automatically generate a globally unique Id, Long type;
  • Primary Key (PRIMARY_KEY): Generate Id through VertexLabel+PrimaryKeyValues, String type;
  • Custom (CUSTOMIZE_STRING|CUSTOMIZE_NUMBER): User-defined Id, which is divided into two types: String and Long, and you need to ensure the uniqueness of the Id yourself;
  • Custom UUID (CUSTOMIZE_UUID): User-defined Id in UUID form, you need to ensure the uniqueness of the Id yourself;

The default Id policy is AUTOMATIC, if the user calls the primaryKeys() method and sets the correct PrimaryKeys, the PRIMARY_KEY policy is automatically enabled. After enabling the PRIMARY_KEY strategy, HugeGraph can implement data deduplication based on PrimaryKeys.

  1. AUTOMATIC ID Policy
schema.vertexLabel("person")
     .useAutomaticId()
     .properties("name", "age", "city")
     .create();
graph.addVertex(T.label, "person","name", "marko", "age", 18, "city", "Beijing");
  1. PRIMARY_KEY ID policy
schema.vertexLabel("person")
     .usePrimaryKeyId()
     .properties("name", "age", "city")
     .primaryKeys("name", "age")
     .create();
graph.addVertex(T.label, "person","name", "marko", "age", 18, "city", "Beijing");
  1. CUSTOMIZE_STRING ID Policy
schema.vertexLabel("person")
     .useCustomizeStringId()
     .properties("name", "age", "city")
     .create();
graph.addVertex(T.label, "person", T.id, "123456", "name", "marko","age", 18, "city", "Beijing");
  1. CUSTOMIZE_NUMBER ID Policy
schema.vertexLabel("person")
     .useCustomizeNumberId()
     .properties("name", "age", "city")
     .create();
graph.addVertex(T.label, "person", T.id, 123456, "name", "marko","age", 18, "city", "Beijing");
  1. CUSTOMIZE_UUID ID Policy
schema.vertexLabel("person")
     .useCustomizeUuidId()
     .properties("name", "age", "city")
     .create();
graph.addVertex(T.label, "person", T.id, UUID.randomUUID(), "name", "marko","age", 18, "city", "Beijing");

If users need Vertex deduplication, there are three options:

  1. Adopt PRIMARY_KEY strategy, automatic overwriting, suitable for batch insertion of large amount of data, users cannot know whether overwriting has occurred
  2. Adopt AUTOMATIC strategy, read-and-modify, suitable for small data insertion, users can clearly know whether overwriting occurs
  3. Using the CUSTOMIZE_STRING or CUSTOMIZE_NUMBER strategy, the user guarantees the uniqueness

4. EdgeId policy

Current implementation: The four-part composition below is a historical simplification. Current EdgeId also distinguishes edge direction and parent/child edge labels. See EdgeId in the Server source; use the source for your version when relying on ID formats or persistent keys.

The EdgeId of HugeGraph is composed of srcVertexId + edgeLabel + sortKey + tgtVertexId. Among them sortKey is an important concept of HugeGraph. There are two reasons for adding sortKey to Edge as the unique ID of Edge:

  1. If there are multiple edges of the same Label between two vertices, they can be distinguished by sortKey
  2. For SuperNode nodes, edges can be sorted and truncated by sortKey.

Since EdgeId is composed of srcVertexId + edgeLabel + sortKey + tgtVertexId, HugeGraph will automatically overwrite when the same Edge is inserted multiple times to achieve deduplication. It should be noted that the properties of Edge will also be overwritten in the batch insert mode.

In addition, because HugeGraph’s EdgeId adopts an automatic deduplication strategy, HugeGraph considers that there is only one edge in the case of self-loop (a vertex has an edge pointing to itself), while a graph database that uses the AUTOMATIC strategy (TitanDB for example) considers that the graph has two edges.

The edges of HugeGraph only support directed edges, and undirected edges can be realized by creating two edges, Out and In.

5. HugeGraph transaction overview

Version scope: The isolation-level, thread-binding, and backend atomicity descriptions below are historical implementation notes. Transaction isolation and rollback boundaries depend on the backend implementation; do not treat these historical descriptions as cross-backend guarantees.

TinkerPop transaction overview

A TinkerPop transaction refers to a unit of work that performs operations on the database. A set of operations within a transaction either succeeds or all fail. For a detailed introduction, please refer to the official documentation of TinkerPop: http://tinkerpop.apache.org/docs/current/reference/#transactions

TinkerPop transaction interfaces
  • open open transaction
  • commit commit transaction
  • rollback rollback transaction
  • close closes the transaction
TinkerPop transaction specification
  • The transaction must be explicitly committed before it can take effect (the modification operation can only be seen by the query in this transaction if it is not committed)
  • A transaction must be opened before it can be committed or rolled back
  • If the transaction setting is automatically turned on, there is no need to explicitly turn it on (the default method), if it is set to be turned on manually, it must be turned on explicitly
  • When the transaction is closed, you can set three modes: automatic commit, automatic rollback (default mode), manual (explicit shutdown is prohibited), etc.
  • The transaction must be closed after committing or rolling back
  • The transaction must be open after the query
  • Transactions (non-threaded tx) must be thread-isolated, and multi-threaded operations on the same transaction do not affect each other

For more transaction specification use cases, see: Transaction Test

HugeGraph transaction implementation
  • All operations in a transaction either succeed or fail
  • A transaction can only read what has been committed by another transaction (Read committed)
  • All uncommitted operations can be queried in this transaction, including:
    • Adding a vertex can query the vertex
    • Delete a vertex to filter out the vertex
    • Deleting a vertex can filter out the related edges of the vertex
    • Adding an edge can query the edge
    • Delete edge can filter out the edge
    • Adding/modifying (vertex, edge) attributes can take effect when querying
    • Delete (vertex, edge) attributes can take effect at query time
  • All uncommitted operations become invalid after the transaction is rolled back, including:
    • Adding and deleting vertices and edges
    • Addition/modification, deletion of attributes

Example: One transaction cannot read another transaction’s uncommitted content

    static void testUncommittedTx(final HugeGraph graph) throws InterruptedException {

        final CountDownLatch latchUncommit = new CountDownLatch(1);
        final CountDownLatch latchRollback = new CountDownLatch(1);

        Thread thread = new Thread(() -> {
            // this is a new transaction in the new thread
            graph.tx().open();

            System.out.println("current transaction operations");

            Vertex james = graph.addVertex(T.label, "author",
                                           "id", 1, "name", "James Gosling",
                                           "age", 62, "lived", "Canadian");
            Vertex java = graph.addVertex(T.label, "language", "name", "java",
                                          "versions", Arrays.asList(6, 7, 8));
            james.addEdge("created", java);

            // we can query the uncommitted records in the current transaction
            System.out.println("current transaction assert");
            assert graph.vertices().hasNext() == true;
            assert graph.edges().hasNext() == true;

            latchUncommit.countDown();

            try {
                latchRollback.await();
            } catch (InterruptedException e) {
                throw new RuntimeException(e);
            }

            System.out.println("current transaction rollback");
            graph.tx().rollback();
        });

        thread.start();

        // query none result in other transaction when not commit()
        latchUncommit.await();
        System.out.println("other transaction assert for uncommitted");
        assert !graph.vertices().hasNext();
        assert !graph.edges().hasNext();

        latchRollback.countDown();
        thread.join();

        // query none result in other transaction after rollback()
        System.out.println("other transaction assert for rollback");
        assert !graph.vertices().hasNext();
        assert !graph.edges().hasNext();
    }
Principle of transaction realization
  • The server internally realizes isolation by binding transactions to threads (ThreadLocal)
  • The uncommitted content of this transaction overwrites the old data in chronological order for this transaction to query the latest version of data
  • The bottom layer relies on the back-end database to ensure transaction atomicity (for example, the batch interface of RocksDB guarantees atomicity)
Notice

REST APIs do not expose a separate transaction lifecycle interface; write APIs invoke the relevant Server-side transaction commit logic.

TinkerPop API allows open transactions, which are automatically closed when the request is completed (Gremlin Server forces close)

6.3 - HugeGraph Plugin mechanism and plug-in extension process

Background

  1. HugeGraph is not only open source and open, but also simple and easy to use. General users can easily add plug-in extension functions without changing the source code.
  2. HugeGraph supports a variety of built-in storage backends, and also allows users to extend custom backends without changing the existing source code.
  3. HugeGraph supports full-text search. The full-text search function involves word segmentation in various languages. Currently, there are 7 built-in word breakers (ansj, hanlp, smartcn, jieba, jcseg, mmseg4j, ikanalyzer), and it also allows users to expand custom word breakers without changing the existing source code.

Scalable dimension

Currently, the plug-in method provides extensions in the following dimensions:

  • backend storage
  • serializer
  • Custom configuration items
  • tokenizer

Plug-in implementation mechanism

  1. HugeGraph provides a plug-in interface HugeGraphPlugin, which supports plug-in through the Java SPI mechanism
  2. HugeGraph provides four extension registration functions as static methods on HugeGraphPlugin: registerOptions(), registerBackend(), registerSerializer(), registerAnalyzer()
  3. The plug-in implementer implements the corresponding Options, Backend, Serializer or Analyzer interface
  4. The plug-in implementer implements register()the method of the HugeGraphPlugin interface, registers the specific implementation class listed in the above point 3 in this method, and packs it into a jar package
  5. The plug-in user puts the jar package in the HugeGraph Server installation directory plugins, modifies the relevant configuration items to the plug-in custom value, and restarts to take effect

Plug-in implementation process example

1 Create a new maven project

1.1 Name the project name: hugegraph-plugin-demo
1.2 Add hugegraph-core Jar package dependencies

The Maven pom.xml is shown below. Match hugegraph.version to the target Server version; this example uses the current master version 1.7.0:

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

    <modelVersion>4.0.0</modelVersion>
    <groupId>org.apache.hugegraph</groupId>
    <artifactId>hugegraph-plugin-demo</artifactId>
    <version>1.0.0</version>
    <packaging>jar</packaging>

    <name>hugegraph-plugin-demo</name>

    <properties>
        <hugegraph.version>1.7.0</hugegraph.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.apache.hugegraph</groupId>
            <artifactId>hugegraph-core</artifactId>
            <version>${hugegraph.version}</version>
        </dependency>
    </dependencies>
</project>

2 Realize extended functions

2.1 Extending a custom backend
2.1.1 Implement the interface BackendStoreProvider
  • Realizable interfaces: org.apache.hugegraph.backend.store.BackendStoreProvider
  • Or inherit an abstract class:org.apache.hugegraph.backend.store.AbstractBackendStoreProvider

Take the RocksDB backend RocksDBStoreProvider as an example:

public class RocksDBStoreProvider extends AbstractBackendStoreProvider {

    protected String database() {
        return this.graph().toLowerCase();
    }

    @Override
    protected BackendStore newSchemaStore(HugeConfig config, String store) {
        return new RocksDBStore.RocksDBSchemaStore(this, this.database(), store);
    }

    @Override
    protected BackendStore newGraphStore(HugeConfig config, String store) {
        return new RocksDBStore.RocksDBGraphStore(this, this.database(), store);
    }

    @Override
    protected BackendStore newSystemStore(HugeConfig config, String store) {
        return new RocksDBStore.RocksDBSystemStore(this, this.database(), store);
    }

    @Override
    public String type() {
        return "rocksdb";
    }

    @Override
    public String driverVersion() {
        return "1.11";
    }
}
2.1.2 Implement interface BackendStore

The main abstract methods of BackendStore are shown below; default helper methods are omitted:

public interface BackendStore {
    // Store name
    String store();

    // Stored version
    String storedVersion();

    // Database name
    String database();

    // Get the parent provider
    BackendStoreProvider provider();

    // Get the system schema store
    SystemSchemaStore systemSchemaStore();

    // Whether it is the storage of schema
    boolean isSchemaStore();

    // Open/close database
    void open(HugeConfig config);
    void close();
    boolean opened();

    // Initialize/clear database
    void init();
    void clear(boolean clearSpace);
    boolean initialized();

    // Delete all data of database (keep table structure)
    void truncate();

    // Add/delete data
    void mutate(BackendMutation mutation);

    // Query data
    Iterator<BackendEntry> query(Query query);
    Iterator<Iterator<BackendEntry>> query(Iterator<Query> queries,
                                          Function<Query, Query> queryWriter,
                                          HugeGraph hugeGraph);
    Number queryNumber(Query query);

    // Transaction
    void beginTx();
    void commitTx();
    void rollbackTx();

    // Get metadata by key
    <R> R metadata(HugeType type, String meta, Object[] args);

    // Backend features
    BackendFeatures features();

    // Increase next id for specific type
    void increaseCounter(HugeType type, long increment);

    // Get current counter for a specific type
    long getCounter(HugeType type);
}
2.1.3 Extending custom serializers

The serializer must inherit the abstract class: org.apache.hugegraph.backend.serializer.AbstractSerializer ( implements GraphSerializer, SchemaSerializer) The main interface is defined as follows:

public interface GraphSerializer {
    BackendEntry writeVertex(HugeVertex vertex);
    BackendEntry writeOlapVertex(HugeVertex vertex);
    BackendEntry writeVertexProperty(HugeVertexProperty<?> prop);
    HugeVertex readVertex(HugeGraph graph, BackendEntry entry);
    BackendEntry writeEdge(HugeEdge edge);
    BackendEntry writeEdgeProperty(HugeEdgeProperty<?> prop);
    HugeEdge readEdge(HugeGraph graph, BackendEntry entry);
    CIter<Edge> readEdges(HugeGraph graph, BackendEntry bytesEntry);
    BackendEntry writeIndex(HugeIndex index);
    HugeIndex readIndex(HugeGraph graph, ConditionQuery query, BackendEntry entry);
    BackendEntry writeId(HugeType type, Id id);
    Query writeQuery(Query query);
}

public interface SchemaSerializer {
    BackendEntry writeVertexLabel(VertexLabel vertexLabel);
    VertexLabel readVertexLabel(HugeGraph graph, BackendEntry entry);
    BackendEntry writeEdgeLabel(EdgeLabel edgeLabel);
    EdgeLabel readEdgeLabel(HugeGraph graph, BackendEntry entry);
    BackendEntry writePropertyKey(PropertyKey propertyKey);
    PropertyKey readPropertyKey(HugeGraph graph, BackendEntry entry);
    BackendEntry writeIndexLabel(IndexLabel indexLabel);
    IndexLabel readIndexLabel(HugeGraph graph, BackendEntry entry);
}
2.1.4 Extend custom configuration items

When adding a custom backend, it may be necessary to add new configuration items. The implementation process mainly includes:

  • Add a configuration item container class and implement the interface org.apache.hugegraph.config.OptionHolder
  • Provide a singleton method public static OptionHolder instance(), and call the method when the object is initialized OptionHolder.registerOptions()
  • Add configuration item declaration, single-value configuration item type is ConfigOption, multi-value configuration item type is ConfigListOption

Take the RocksDB configuration item definition as an example:

public class RocksDBOptions extends OptionHolder {

    private RocksDBOptions() {
        super();
    }

    private static volatile RocksDBOptions instance;

    public static synchronized RocksDBOptions instance() {
        if (instance == null) {
            instance = new RocksDBOptions();
            instance.registerOptions();
        }
        return instance;
    }

    public static final ConfigOption<String> DATA_PATH =
            new ConfigOption<>(
                    "rocksdb.data_path",
                    "The path for storing data of RocksDB.",
                    disallowEmpty(),
                    "rocksdb-data/data"
            );

    public static final ConfigOption<String> WAL_PATH =
            new ConfigOption<>(
                    "rocksdb.wal_path",
                    "The path for storing WAL of RocksDB.",
                    disallowEmpty(),
                    "rocksdb-data/wal"
            );

    public static final ConfigListOption<String> DATA_DISKS =
            new ConfigListOption<>(
                    "rocksdb.data_disks",
                    false,
                    "The optimized disks for storing data of RocksDB. " +
                    "The format of each element: `STORE/TABLE: /path/disk`." +
                    "Allowed keys are [g/vertex, g/edge_out, g/edge_in, " +
                    "g/vertex_label_index, g/edge_label_index, " +
                    "g/range_int_index, g/range_float_index, " +
                    "g/range_long_index, g/range_double_index, " +
                    "g/secondary_index, g/search_index, g/shard_index, " +
                    "g/unique_index, g/olap]",
                    null,
                    String.class,
                    ImmutableList.of()
            );
}
2.2 Extend custom tokenizer

The tokenizer needs to implement the interface org.apache.hugegraph.analyzer.Analyzer, take implementing a SpaceAnalyzer space tokenizer as an example.

package org.apache.hugegraph.plugin;

import java.util.Arrays;
import java.util.HashSet;
import java.util.Set;

import org.apache.hugegraph.analyzer.Analyzer;

public class SpaceAnalyzer implements Analyzer {

    @Override
    public Set<String> segment(String text) {
        return new HashSet<>(Arrays.asList(text.split(" ")));
    }
}

3. Implement the plug-in interface and register it

The plug-in registration entry is HugeGraphPlugin.register(), the custom plug-in must implement this interface method, and register the extension items defined above inside it. The interface org.apache.hugegraph.plugin.HugeGraphPlugin is defined as follows:

public interface HugeGraphPlugin {

    String name();

    void register();

    String supportsMinVersion();

    String supportsMaxVersion();
}

And HugeGraphPlugin provides 4 static methods for registering extensions:

  • registerOptions(String name, String classPath): register configuration items
  • registerBackend(String name, String classPath): register backend (BackendStoreProvider)
  • registerSerializer(String name, String classPath): register serializer
  • registerAnalyzer(String name, String classPath): register tokenizer

The following is an example of registering the SpaceAnalyzer tokenizer:

package org.apache.hugegraph.plugin;

public class DemoPlugin implements HugeGraphPlugin {

    @Override
    public String name() {
        return "demo";
    }

    @Override
    public void register() {
        HugeGraphPlugin.registerAnalyzer("demo", SpaceAnalyzer.class.getName());
    }

    @Override
    public String supportsMinVersion() {
        return "1.7.0";
    }

    @Override
    public String supportsMaxVersion() {
        return "1.8.0";
    }
}

4. Configure SPI entry

  1. Make sure the services directory exists: hugegraph-plugin-demo/resources/META-INF/services
  2. Create a text file in the services directory: org.apache.hugegraph.plugin.HugeGraphPlugin
  3. The content of the file is as follows: org.apache.hugegraph.plugin.DemoPlugin

5. Make Jar package

Through maven packaging, execute the command in the project directory mvn package, and a Jar package file will be generated in the target directory. Copy the Jar package to the plugins directory when using it, and restart the service to take effect.

6.4 - HugeGraph Toolchain Local Testing Guide

This guide helps developers run HugeGraph toolchain tests locally.

1. Core Concepts

1.1 Core Dependency: HugeGraph Server

Integration and functional tests of the toolchain depend on HugeGraph Server, including Client, Loader, Hubble, Spark Connector, Tools, and other components.

1.2 Test Types

  • Unit Tests: Test individual functions/methods, no external dependencies required
  • API Tests (ApiTestSuite): Test API interfaces, requires running HugeGraph Server
  • Functional Tests (FuncTestSuite): End-to-end tests, require complete system environment

2. Environment Setup

2.1 System Requirements

  • Operating System: Linux / macOS (Windows use WSL2)
  • JDK: >= 11, configure JAVA_HOME
  • Maven: >= 3.6
  • Python: >= 3.11 (only required for Hubble tests)

2.2 Clone Code

git clone https://github.com/apache/hugegraph-toolchain.git
cd hugegraph-toolchain

3. Deploy Test Environment

Deployment Options

  • Script Deployment: Specify a Server commit to reproduce the server version used by CI
  • Docker Deployment: Suitable for quick checks; if tests fail, first verify compatibility between the image and Toolchain

For detailed installation instructions, refer to Community Documentation

3.1 Script Deployment

Parameter Description

  • $COMMIT_ID: Specify Server source code Git Commit ID
  • $DB_DATABASE / $DB_PASS: MySQL database name and password for Loader JDBC tests

Deployment Steps

1. Install HugeGraph Server

# Set the Server baseline; use a full commit SHA for reproducible results
export COMMIT_ID="master"

# Execute installation (script located in /assembly/travis/ directory)
hugegraph-client/assembly/travis/install-hugegraph-from-source.sh $COMMIT_ID
  • The script starts HTTP and HTTPS instances on ports 8080 and 8443 and configures admin/pa authentication.
  • Ensure both ports are available before running it.

2. Install Optional Dependencies

# Hadoop (only required for Loader HDFS tests)
hugegraph-loader/assembly/travis/install-hadoop.sh

# MySQL (only required for Loader JDBC tests)
hugegraph-loader/assembly/travis/install-mysql.sh $DB_DATABASE $DB_PASS

3. Health Check

curl -u admin:pa http://localhost:8080/graphspaces/DEFAULT/graphs
# Returns {"graphs":["hugegraph"]} indicates success

3.2 Docker Deployment

Note: Docker images may have version lag, use script deployment if encountering compatibility issues

Quick Start

docker network create hugegraph-net
docker run -itd --name=server -p 8080:8080 --network hugegraph-net hugegraph/hugegraph:latest

docker-compose Configuration (Optional)

Complete configuration example including Server, MySQL, Hadoop services (requires Docker Compose V2):

version: '3.8'

services:
  hugegraph-server:
    image: hugegraph/hugegraph:latest  # Can be replaced with a specific version, or build your own image
    container_name: hugegraph-server
    ports:
      - "8080:8080"  # HugeGraph Server HTTP port
    environment:
      # Configure HugeGraph Server parameters as needed, e.g., backend storage
      - HUGEGRAPH_SERVER_OPTIONS="-Dstore.backend=rocksdb"
    volumes:
      # If you need to persist data or mount configuration files, add volumes here
      # - ./hugegraph-data:/opt/hugegraph/data
    healthcheck:
      test: ["CMD-SHELL", "curl -f http://localhost:8080/graphspaces/DEFAULT/graphs || exit 1"]
      interval: 10s
      timeout: 3s
      retries: 5
    networks:
      - hugegraph-net
  
  # If you need JDBC tests for hugegraph-loader, you can add the following service
  #   mysql:
  #     image: mysql:5.7
  #     container_name: mysql-db
  #     environment:
  #       MYSQL_ROOT_PASSWORD: ${DB_PASS:-your_mysql_root_password} # Read from environment variable, or use default
  #       MYSQL_DATABASE: ${DB_DATABASE:-hugegraph_test_db} # Read from environment variable, or use default
  #     ports:
  #       - "3306:3306"
  #     volumes:
  #       - ./mysql-data:/var/lib/mysql # Data persistence
  #     healthcheck:
  #       test: ["CMD", "mysqladmin", "ping", "-h", "localhost", "-p${DB_PASS:-your_mysql_root_password}"]
  #       interval: 5s
  #       timeout: 3s
  #       retries: 5
  #     networks:
  #       - hugegraph-net

  # If you need Hadoop/HDFS tests for hugegraph-loader, you can add the following services
  #   namenode:
  #     image: johannestang/hadoop-namenode:2.0.0-hadoop2.8.5-java8
  #     container_name: namenode
  #     ports:
  #       - "0.0.0.0:9870:9870"
  #       - "0.0.0.0:8020:8020"
  #     environment:
  #       - CLUSTER_NAME=test-cluster
  #       - HDFS_NAMENODE_USER=root
  #       - HADOOP_CONF_DIR=/hadoop/etc/hadoop
  #     volumes:
  #       - ./config/core-site.xml:/hadoop/etc/hadoop/core-site.xml
  #       - ./config/hdfs-site.xml:/hadoop/etc/hadoop/hdfs-site.xml
  #       - namenode_data:/hadoop/dfs/name
  #     command: bash -c "if [ ! -d /hadoop/dfs/name/current ]; then hdfs namenode -format; fi && /entrypoint.sh"
  #     healthcheck:
  #       test: ["CMD", "hdfs", "dfsadmin", "-report"]
  #       interval: 5s
  #       timeout: 3s
  #       retries: 5
  #     networks:
  #       - hugegraph-net

  #   datanode:
  #     image: johannestang/hadoop-datanode:2.0.0-hadoop2.8.5-java8
  #     container_name: datanode
  #     depends_on:
  #       - namenode
  #     environment:
  #       - CLUSTER_NAME=test-cluster
  #       - HDFS_DATANODE_USER=root
  #       - HADOOP_CONF_DIR=/hadoop/etc/hadoop
  #     volumes:
  #       - ./config/core-site.xml:/hadoop/etc/hadoop/core-site.xml
  #       - ./config/hdfs-site.xml:/hadoop/etc/hadoop/hdfs-site.xml
  #       - datanode_data:/hadoop/dfs/data
  #     healthcheck:
  #       test: ["CMD", "hdfs", "dfsadmin", "-report"]
  #       interval: 5s
  #       timeout: 3s
  #       retries: 5
  #     networks:
  #       - hugegraph-net

networks:
  hugegraph-net:
    driver: bridge
volumes:
  namenode_data:
  datanode_data:

Hadoop Configuration Mounts

Create a ./config folder in the same directory as docker-compose.yml to mount Hadoop configuration files. You can skip this step if HDFS testing is not required.

📁 ./config/core-site.xml content:

<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://namenode:8020</value>
    </property>
</configuration>

📁 ./config/hdfs-site.xml content:

<configuration>
    <property>
        <name>dfs.namenode.name.dir</name>
        <value>/hadoop/hdfs/name</value>
    </property>
    <property>
        <name>dfs.datanode.data.dir</name>
        <value>/hadoop/hdfs/data</value>
    </property>
    <property>
        <name>dfs.permissions.superusergroup</name>
        <value>hadoop</value>
    </property>
    <property>
        <name>dfs.support.append</name>
        <value>true</value>
    </property>
</configuration>

Docker Operations

# Start services
docker compose up -d

# Check status
docker compose ps
lsof -i:8080  # Server
lsof -i:8020  # Hadoop
lsof -i:3306  # MySQL

# Stop services
docker compose down

4. Run Tests

Test process for each tool:

HugeGraph Toolchain Testing Process

4.1 hugegraph-client

Compile

mvn -e compile -pl hugegraph-client -Dmaven.javadoc.skip=true -ntp

Dependent Services

Start HugeGraph Server (refer to Section 3)

Server Authentication Configuration

ApiTest requires authentication. No additional configuration is needed when using the script in Section 3.1. For a manually deployed Server, the authentication settings and test credentials must match the test code.

# 1. Modify authentication mode
cp conf/rest-server.properties conf/rest-server.properties.backup
sed -i 's|#auth.authenticator=.*|auth.authenticator=org.apache.hugegraph.auth.StandardAuthenticator|' conf/rest-server.properties
grep auth.authenticator conf/rest-server.properties
sed -i 's|gremlin.graph=org.apache.hugegraph.HugeFactory|gremlin.graph=org.apache.hugegraph.auth.HugeFactoryAuthProxy|' conf/graphs/hugegraph.properties

# 2. Set password
# Note: Test code uses "pa" as default password, must match for tests to work
bin/stop-hugegraph.sh
export PASSWORD="pa"  # Set to test default password
echo -e "${PASSWORD}" | bin/init-store.sh
bin/start-hugegraph.sh

Run Tests

# Check environment
curl -u admin:pa http://localhost:8080/graphspaces/DEFAULT/graphs

# Run tests
cd hugegraph-client
mvn test -Dtest=UnitTestSuite -ntp      # Unit tests
mvn test -Dtest=ApiTestSuite -ntp       # API tests (requires Server)
mvn test -Dtest=FuncTestSuite -ntp      # Functional tests (requires Server)

Check Server log if tests fail: logs/hugegraph-server.log

4.2 hugegraph-loader

Compile

mvn install -pl hugegraph-client,hugegraph-loader -am -Dmaven.javadoc.skip=true -DskipTests -ntp

Dependent Services

  • Required: HugeGraph Server
  • Optional: Hadoop (HDFS tests), MySQL (JDBC tests); Kafka tests use Testcontainers to start Kafka and also require an available Docker daemon.

Run Tests

cd hugegraph-loader
mvn test -P unit -ntp   # Unit tests
mvn test -P file -ntp   # File tests (requires Server)
mvn test -P hdfs -ntp   # HDFS tests (requires Server + Hadoop)
mvn test -P jdbc -ntp   # JDBC tests (requires Server + MySQL)
mvn test -P kafka -ntp  # Kafka tests (requires Server + Docker daemon; starts a Kafka container)

4.3 hugegraph-hubble

Compile

mvn install -pl hugegraph-client,hugegraph-loader -am -Dmaven.javadoc.skip=true -DskipTests -ntp
cd hugegraph-hubble
mvn -e compile -Dmaven.javadoc.skip=true -ntp

Dependent Services

1. Start Server (refer to Section 3)

2. Python Environment

python -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate
python -m pip install -r hubble-dist/assembly/travis/requirements.txt

3. Build and Verify

mvn package -Dmaven.test.skip=true
# Optional: Start and verify
cd apache-hugegraph-hubble*/bin
./start-hubble.sh -d && sleep 10
curl http://localhost:8088/actuator/health
./stop-hubble.sh

Run Tests

# Unit tests
mvn test -P unit-test -pl hugegraph-hubble/hubble-be -ntp

# Legacy Python API tests (requires Server; the script installs and starts Hubble)
curl -u admin:pa http://localhost:8080/graphspaces/DEFAULT/graphs  # Check Server
cd hugegraph-hubble
./hubble-dist/assembly/travis/run-api-test.sh

# Full verification entry point used by current CI (build the Hubble tarball first)
HUBBLE_TARBALL="$(ls target/apache-hugegraph-hubble-*.tar.gz | head -n 1)"
hubble-dist/assembly/travis/verify-hubble-issue-694.sh \
  "$HUBBLE_TARBALL" http://127.0.0.1:8080

4.4 hugegraph-spark-connector

Compile

mvn install -pl hugegraph-client,hugegraph-spark-connector -am -Dmaven.javadoc.skip=true -DskipTests -ntp

Run Tests

cd hugegraph-spark-connector
mvn test -ntp  # Requires Server running

4.5 hugegraph-tools

Compile

mvn install -pl hugegraph-client,hugegraph-tools -am -Dmaven.javadoc.skip=true -DskipTests -ntp

Run Tests

cd hugegraph-tools
mvn test -Dtest=FuncTestSuite -ntp  # Requires Server running

5. Common Issues

Service Connection Issues

If Server, MySQL, or Hadoop cannot be reached:

  • Confirm services are running (Server must be on port 8080)
  • Check port usage: lsof -i:8080
  • Docker check: docker compose ps and docker compose logs

Configuration Issues

If files cannot be found or parameters are invalid:

  • Check environment variables: echo $COMMIT_ID
  • Script permissions: chmod +x hugegraph-*/assembly/travis/*.sh

HDFS Test Failures

  • Confirm NameNode/DataNode running normally
  • Check Hadoop logs
  • Verify HDFS connection: hdfs dfsadmin -report

JDBC Test Failures

  • Confirm MySQL running normally
  • Verify database connection: mysql -u root -p$DB_PASS
  • Check MySQL logs

6. References

6.5 - Backup and Restore

Description

Backup and Restore are functions of backup map and restore map. The data backed up and restored includes metadata (schema) and graph data (vertex and edge).

Backup

Export the metadata and graph data of a graph in the HugeGraph system in JSON format.

Restore

Re-import the data in JSON format exported by Backup to a graph in the HugeGraph system.

Restore has two modes:

  • In Restoring mode, the metadata and graph data exported by Backup are restored to the HugeGraph system intact. It can be used for graph backup and recovery, and the general target graph is a new graph (without metadata and graph data). for example:
    • System upgrade, first back up the map, then upgrade the system, and finally restore the map to the new system
    • Graph migration, from a HugeGraph system, use the Backup function to export the graph, and then use the Restore function to import the graph into another HugeGraph system
  • In the Merging mode, the metadata and graph data exported by Backup are imported into another graph that already has metadata or graph data. During the process, the ID of the metadata may change, and the IDs of vertices and edges will also change accordingly.
    • Can be used to merge graphs

Instructions

You can use hugegraph-tools to backup and restore the graph.

Backup

bin/hugegraph backup -t all -d data

This command backs up all the metadata and graph data of the hugegraph graph of http://127.0.0.1:8080 (the default –url) to the data directory.

Backup works in any graph mode, it does not check the graph mode

Restore

Restore has two modes: RESTORING and MERGING. Before restore, you must first set the graph mode according to your needs, the restore command fails when the graph is in any other mode.

Step 1: View and set graph mode
bin/hugegraph graph-mode-get

This command is used to view the current graph mode, including: NONE, RESTORING, MERGING, LOADING.

bin/hugegraph graph-mode-set -m RESTORING

This command is used to set the graph mode. Before Restore, it can be set to RESTORING or MERGING mode. In the example, it is set to RESTORING.

Step 2: Restore data
bin/hugegraph restore -t all -d data

This command re-imports all metadata and graph data in the data directory to the hugegraph graph at http://127.0.0.1:8080.

Step 3: Restoring Graph Mode
bin/hugegraph graph-mode-set -m NONE

This command is used to restore the graph mode to NONE.

So far, a complete graph backup and graph recovery process is over.

help

For detailed usage of backup and restore commands, please refer to the hugegraph-tools documentation.

API description for Backup/Restore usage and implementation

Backup

Backup uses the corresponding list(GET) API export of metadata and graph data, and no new API is added.

Restore

Restore uses the corresponding create(POST) API imports for metadata and graph data, and does not add new APIs.

There are two different modes for Restore: Restoring and Merging. In addition, there is a regular mode of NONE (default), the differences are as follows:

  • In None mode, the writing of metadata and graph data is normal, please refer to the function description. special:
    • ID is not allowed when metadata (schema) is created
    • Graph data (vertex) is not allowed to specify an ID when the id strategy is Automatic
  • Restoring mode, restoring to a new graph, in particular:
    • ID is allowed to be specified when metadata (schema) is created
    • Graph data (vertex) allows specifying an ID when the id strategy is Automatic
  • Merging mode, merging into a graph with existing metadata and graph data, in particular:
    • ID is not allowed when metadata (schema) is created
    • Graph data (vertex) allows specifying an ID when the id strategy is Automatic

Normally, the graph mode is None. When you need to restore the graph, you need to temporarily change the graph mode to Restoring mode or Merging mode as needed, and when the Restore is completed, restore the graph mode to None.

The implemented RESTful API for setting graph mode is as follows:

View the mode of a graph

When authentication is enabled, graph read permission is required (space_member or the graph owner).

Method & Url
GET http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/mode
Response Status
200
Response Body
{
    "mode": "NONE"
}

Legal graph modes include: NONE, RESTORING, MERGING, LOADING

Set the mode of a graph

When authentication is enabled, graph-space management permission (space) is required; administrators can satisfy it through permission inheritance.

Method & Url
PUT http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/mode
Request Body
"RESTORING"

Legal graph modes include: NONE, RESTORING, MERGING, LOADING

Response Status
200
Response Body
{
    "mode": "RESTORING"
}

6.6 - HugeGraph Docker Cluster Guide

Overview

Docker Compose provides a quick way to run a complete HugeGraph distributed cluster (PD + Store + Server) on Linux or macOS.

Prerequisites

  • Docker Engine 20.10+ or Docker Desktop 4.x+
  • Docker Compose v2
  • For a three-node cluster on macOS, allocate at least 12 GB of memory (Settings → Resources → Memory). Adjust resources to suit other platforms.

Tested environments: Linux (native Docker) and macOS (Docker Desktop on ARM M4).

Compose Files

The HugeGraph repository provides four Compose files in docker/:

FileServicesUse case
docker-compose.yml1 RocksDB Server + 1 HubbleDefault standalone Quick Start; recommended starting point
docker-compose-hstore.yml1 PD + 1 Store + 1 Server + 1 HubbleLocal distributed development
docker-compose-3pd-3store-3server.yml3 PD + 3 Store + 3 Server + 1 HubbleHA reference and evaluation
docker-compose.dev.ymlOverride onlySource-build override for the minimal HStore topology; always combine with docker-compose-hstore.yml

Standalone uses hugegraph/hugegraph:${HUGEGRAPH_VERSION:-latest}; HStore uses matching hugegraph/pd, hugegraph/store, and hugegraph/server tags. Hubble is selected separately by ${HUBBLE_IMAGE:-hugegraph/hubble:latest}.

The following steps assume you have cloned the hugegraph repository, or at least its docker/ directory.

HStore and HA instructions use current master Compose files and require PD, Store, and Server images built from the same master source. Do not combine these files with HStore component images tagged 1.7.0. The standalone example separately pins hugegraph/hugegraph:1.7.0.

Authentication Environment

This .env template applies to current master Compose. Angle-bracket values and replace-with-your-password are placeholders: replace them before loading the file or starting services. Follow the master authentication environment instructions to generate .env, then replace the administrator password placeholder:

HUGEGRAPH_ADMIN_PASSWORD='replace-with-your-password'
HUGEGRAPH_AUTH_TOKEN_SECRET='<32 random bytes, e.g. openssl rand -hex 32>'
HG_PD_AUTH_SECRET_KEY='<24 random bytes, e.g. openssl rand -hex 24>'

For master HStore and HA topologies, HG_PD_AUTH_SECRET_KEY is the PD REST Basic password, shared by all PD nodes, Server, and Hubble. Current master PD Docker images require it. Hubble does not read .env directly: generate the local configuration for its topology using this value before startup, as described below. Do not commit .env or generated .local.properties files.

A nonempty HUGEGRAPH_ADMIN_PASSWORD enables Server authentication through PASSWORD; Hubble detects this mode through the Server API. It initializes the built-in admin password only on first initialization. An unset or empty value disables authentication and is suitable only for trusted local environments.

Current master Compose passes HUGEGRAPH_AUTH_TOKEN_SECRET as HG_SERVER_AUTH_TOKEN_SECRET. A shared secret lets master Server replicas verify the same tokens and retain token validity after container recreation; HA Compose requires an explicit secret. The standalone hugegraph/hugegraph:1.7.0 entrypoint enables authentication through PASSWORD but ignores HG_SERVER_AUTH_TOKEN_SECRET, so this .env variable does not fix its JWT secret. For a stable JWT secret in 1.7.0, explicitly set auth.token_secret in its authentication graph configuration, conf/graphs/hugegraph.properties, and persist that file across container replacement. See the authentication guide for graph paths and secret generation.

Changing HUGEGRAPH_ADMIN_PASSWORD later does not rotate an existing password; use the user API.

Warning

Production component access controls

In production, enable Server authentication and authorization for graph APIs, maintain the Server IP allowlist, grant minimum permissions, and retain Server audit-*.log with restricted read access. Configure PD REST credentials and PD/Store gRPC, Raft, and REST network boundaries separately; Server Auth does not protect those ports. Expose PD/Store only to cluster nodes and trusted operations networks.

Standalone Quick Start

This section pins the standalone image hugegraph/hugegraph:1.7.0. HStore/HA examples use master-built images; do not reuse this version for them.

cd "$(git rev-parse --show-toplevel)/docker"
# Standalone RocksDB Server image example
HUGEGRAPH_VERSION=1.7.0 docker compose -f docker-compose.yml up -d --wait

Verify:

curl -fsS http://localhost:8080/versions
curl -fsS http://localhost:8088/about        # Hubble

Hubble binds only to host loopback (127.0.0.1:8088) by default. Set HUBBLE_PUBLISH_HOST only behind an HTTPS reverse proxy and trusted network controls.

Minimal HStore Quick Start (Current Master Images)

Build PD, Store, and HStore Server images from the HugeGraph Server master repository root:

cd "$(git rev-parse --show-toplevel)"
docker build -f hugegraph-pd/Dockerfile -t hugegraph/pd:local .
docker build -f hugegraph-store/Dockerfile -t hugegraph/store:local .
docker build -f hugegraph-server/Dockerfile-hstore -t hugegraph/server:local .

From the repository root, enter docker/, load .env, and generate Hubble configuration for the minimal topology. A read-only bind mount cannot create a valid missing hstore.local.properties; generate it before startup:

cd "$(git rev-parse --show-toplevel)/docker"
set -a; . ./.env; set +a
./set-hubble-pd-password.sh hstore
HUGEGRAPH_VERSION=local HUGEGRAPH_PULL_POLICY=never \
  docker compose -f docker-compose-hstore.yml up -d --wait

Verify:

set -a; . ./.env; set +a
curl -fsS http://localhost:8620/v1/health    # PD
curl -fsS http://localhost:8520/v1/health    # Store
curl -fsS http://localhost:8080/versions     # Server
curl -fsS http://localhost:8088/about        # Hubble
curl -fsS -u "hg:${HG_PD_AUTH_SECRET_KEY:?Load .env first}" \
  http://localhost:8620/v1/stores       # Check registered Stores with PD REST authentication

To build this topology from local source instead of pulling images, add the development override and keep both files in all subsequent lifecycle commands:

cd "$(git rev-parse --show-toplevel)/docker"
set -a; . ./.env; set +a
./set-hubble-pd-password.sh hstore
docker compose -f docker-compose-hstore.yml -f docker-compose.dev.yml up -d --build --wait

The override builds images tagged dev. If you already built local images above, start with the base Compose file and HUGEGRAPH_VERSION=local; keep the tag choices consistent.

Three-Node Cluster Quick Start (Current Master Images)

HA Compose has no source-build override. First build hugegraph/pd:local, hugegraph/store:local, and hugegraph/server:local from the same master source as in the minimal HStore section, then load .env in docker/, generate HA Hubble configuration, and start:

cd "$(git rev-parse --show-toplevel)/docker"
set -a; . ./.env; set +a
./set-hubble-pd-password.sh hstore-ha
HUGEGRAPH_VERSION=local \
  docker compose -f docker-compose-3pd-3store-3server.yml up -d --wait

HA Hubble bind-mounts conf/hubble/hstore-ha.local.properties; generate this before startup. It is separate from the minimal topology’s hstore.local.properties, but both use the same HG_PD_AUTH_SECRET_KEY from .env.

The default startup order is:

  1. PD nodes start first and must pass /v1/health checks.
  2. Store nodes start after all PD nodes are healthy.
  3. Server nodes start after all Store and PD nodes are healthy.

Verify the cluster: PD /v1/health and /v1/ready probes require no authentication. Store registration and partition queries are protected PD REST management APIs: use Basic username hg and HG_PD_AUTH_SECRET_KEY from .env.

set -a; . ./.env; set +a
curl -fsS http://localhost:8620/v1/health      # PD health check
curl -fsS http://localhost:8520/v1/health      # Store health check
curl -fsS http://localhost:8080/versions        # Server
for port in 8620 8621 8622; do
  curl -fsS "http://localhost:${port}/v1/ready" | grep -q '"ready":true' || exit 1
done
curl -fsS -u "hg:${HG_PD_AUTH_SECRET_KEY:?Load .env first}" \
  http://localhost:8620/v1/stores          # Authenticated Store registration query
curl -fsS -u "hg:${HG_PD_AUTH_SECRET_KEY:?Load .env first}" \
  http://localhost:8620/v1/partitions      # Authenticated partition query
curl -fsS http://localhost:8088/about           # Hubble

With authentication enabled, graph listing should reject anonymous access and accept the administrator:

set -a; . ./.env; set +a
curl -o /dev/null -w '%{http_code}\n' \
  http://localhost:8080/graphspaces/DEFAULT/graphs                      # Expect 401
curl -o /dev/null -w '%{http_code}\n' -u "admin:${HUGEGRAPH_ADMIN_PASSWORD}" \
  http://localhost:8080/graphspaces/DEFAULT/graphs                      # Expect 200

The other Server nodes listen on 8081 and 8082; the remaining PD and Store nodes use 8621/8622 and 8521/8522.

Environment Variable Reference

The following tables describe current master entrypoints and Compose files, not the standalone hugegraph/hugegraph:1.7.0 environment-variable contract. PD and Store build SPRING_APPLICATION_JSON from their variables and log selected nonsensitive settings at startup. PD secrets are excluded from that summary. Use docker logs to check listed settings; an absent secret in the logs does not establish whether it was applied. Server instead rewrites keys in conf/graphs/hugegraph.properties and conf/rest-server.properties.

PD Variables

VariableRequiredDefaultConfiguration mapping
HG_PD_GRPC_HOSTYesNonegrpc.host
HG_PD_RAFT_ADDRESSYesNoneraft.address
HG_PD_RAFT_PEERS_LISTYesNoneraft.peers-list
HG_PD_INITIAL_STORE_LISTYesNonepd.initial-store-list
HG_PD_GRPC_PORTNo8686grpc.port
HG_PD_REST_PORTNo8620server.port
HG_PD_DATA_PATHNo/hugegraph-pd/pd_datapd.data-path
HG_PD_INITIAL_STORE_COUNTNo1pd.initial-store-count
HG_PD_AUTH_SECRET_KEYYes (current master Docker)Noneauth.secret-key; PD REST Basic password, also used by Server and Hubble

Deprecated aliases: GRPC_HOST → HG_PD_GRPC_HOST, RAFT_ADDRESS → HG_PD_RAFT_ADDRESS, RAFT_PEERS → HG_PD_RAFT_PEERS_LIST, PD_INITIAL_STORE_LIST → HG_PD_INITIAL_STORE_LIST. Old names are mapped only when new names are unset, with a warning. Missing required variables cause the entrypoint to exit with code 2.

Store Variables

VariableRequiredDefaultConfiguration mapping
HG_STORE_PD_ADDRESSYesNonepdserver.address
HG_STORE_GRPC_HOSTYesNonegrpc.host
HG_STORE_RAFT_ADDRESSYesNoneraft.address
HG_STORE_GRPC_PORTNo8500grpc.port
HG_STORE_REST_PORTNo8520server.port
HG_STORE_DATA_PATHNo/hugegraph-store/storageapp.data-path

Deprecated aliases: PD_ADDRESS → HG_STORE_PD_ADDRESS, GRPC_HOST → HG_STORE_GRPC_HOST, RAFT_ADDRESS → HG_STORE_RAFT_ADDRESS.

Server Variables

Unlike PD and Store, Server has no mandatory entrypoint variables: only set values are written to configuration. Distributed deployment nevertheless requires at least HG_SERVER_BACKEND and HG_SERVER_PD_PEERS.

VariableDefaultConfiguration mapping
HG_SERVER_BACKENDTemplate value (rocksdb, or hstore in hugegraph/server)backend in conf/graphs/hugegraph.properties
HG_SERVER_PD_PEERSNonepd.peers in hugegraph.properties and rest-server.properties
HG_SERVER_USE_PDfalseusePD in rest-server.properties
HG_SERVER_CLUSTERhg-testcluster in rest-server.properties
HG_SERVER_REST_URLhttp://0.0.0.0:8080 (set in the image)restserver.url
HG_SERVER_MIN_FREE_MEMORY64(MB)restserver.min_free_memory
HG_SERVER_INIT_STORE_ENABLEDtrueinit_store.enabled; set false for PD/HStore deployments whose metadata is managed by storage
HG_SERVER_AUTH_TOKEN_SECRETGenerated when PASSWORD is setauth.token_secret in both configuration files; at least 32 bytes
HG_SERVER_REQUIRE_AUTH_TOKEN_SECRETfalseWhen true, refuse startup if PASSWORD is set without HG_SERVER_AUTH_TOKEN_SECRET
PASSWORDNoneauth.admin_pa; runs bin/enable-auth.sh to enable authentication
PRELOADNoneWhen true, preload the example graph from scripts/example.groovy
JAVA_OPTSSet in the imagePassed to bin/start-hugegraph.sh -j
HG_SERVER_STARTUP_TIMEOUT_S120 secondsPassed to bin/start-hugegraph.sh -t, range 1–86400; see Server startup timeout below
STORE_RESTstore:8520Store REST address polled by wait-partition.sh, for HStore only
HG_SERVER_PD_REST_ENDPOINTDerived from pd.peers by replacing :8686 with :8620PD REST address polled by wait-storage.sh
PD_AUTH_USERstoreBasic username for PD REST calls from wait-storage.sh
PD_AUTH_PASSWORDEmptyPD REST Basic password; must match HG_PD_AUTH_SECRET_KEY when PD REST authentication is enabled
WAIT_PARTITION_TIMEOUT_S120Partition allocation wait time in wait-partition.sh

Deprecated aliases: BACKEND → HG_SERVER_BACKEND, PD_PEERS → HG_SERVER_PD_PEERS.

wait-storage.sh waits up to 300 seconds for an Up Store. This duration is hardcoded and cannot be changed through environment variables.

HG_SERVER_INIT_STORE_ENABLED accepts only the case-insensitive boolean values recognized by HugeConfig: y, t, yes, on, true, n, f, no, off, false. Other values, including 0 and 1, terminate the entrypoint.

After successful initialization, the entrypoint writes docker/init_complete; later starts skip reinitialization but still invoke bin/init-store.sh to revalidate configuration each time initialization is disabled.

Compose Variables

Compose files, rather than entrypoints, read these variables:

VariableDefaultPurpose
HUGEGRAPH_VERSIONlatestServer, PD, and Store image tag
HUGEGRAPH_PULL_POLICYmissingImage pull_policy; use never to retain locally built images
HUBBLE_IMAGEhugegraph/hubble:latestHubble image, selected independently of HUGEGRAPH_VERSION
HUBBLE_PULL_POLICYmissingHubble image pull_policy
HUBBLE_PUBLISH_HOST127.0.0.1Host interface for publishing Hubble port 8088
HUGEGRAPH_ADMIN_PASSWORDNonePassed to Server as PASSWORD
HUGEGRAPH_AUTH_TOKEN_SECRETNoneMaster Compose passes this to Server as HG_SERVER_AUTH_TOKEN_SECRET; the 1.7.0 image ignores it
HG_PD_AUTH_SECRET_KEYNoneHStore PD REST Basic password; Compose supplies it to PD and Server and uses it to generate local Hubble configuration

Port Reference

Published ports in the three-node cluster:

ServiceHost portContainer portPurpose
pd086208620REST API
pd086868686gRPC
pd186218620REST API
pd186878686gRPC
pd286228620REST API
pd286888686gRPC
store085008500gRPC
store085108510Raft
store085208520REST API
store185018500gRPC
store185118510Raft
store185218520REST API
store285028500gRPC
store285128510Raft
store285228520REST API
server080808080Graph API
server180818080Graph API
server280828080Graph API
hubble80888088Hubble UI, bound to 127.0.0.1 by default

Standalone publishes only 8080 and 8088; minimal HStore publishes 8620 (PD REST), 8520 (Store REST), 8080, and 8088. PD Raft uses internal port 8610, unpublished in all topologies.

Troubleshooting

  1. Container OOM exit (code 137): Increase Docker Desktop memory beyond 12 GB or adjust JVM memory for the killed process.

  2. Raft election timeout: Verify every PD node has the same HG_PD_RAFT_PEERS_LIST. Check connectivity with docker exec hg-pd0 ping pd1.

  3. Partition allocation incomplete: Load .env in docker/, then check Store registration with PD REST Basic authentication:

    set -a; . ./.env; set +a
    curl -fsS -u "hg:${HG_PD_AUTH_SECRET_KEY:?Load .env first}" http://localhost:8620/v1/stores

    PD can complete partition allocation once all three Stores report "state":"Up".

  4. Connection refused: Use container hostnames (pd0, store0) in HG_* variables rather than 127.0.0.1.

  5. Unexpected retained data: docker compose down keeps named volumes. To delete topology data too, use docker compose down -v.

Runtime logs: docker logs <container-name> (for example, docker logs hg-pd0) shows logs without entering containers. Standalone hugegraph/hugegraph sets STDOUT_MODE=true and sends service logs to stdout. The HStore hugegraph/server image does not set this variable: docker logs shows only entrypoint output; inspect logs/hugegraph-server.log inside the container for service logs.

Container Monitoring and Health Checks

Version scope: This section describes current master Docker images; these behaviors are not included in 1.7.0 images. Use the master-built local images above or a latest image containing these changes.

Process Monitoring

Previously, all three entrypoints ended with tail -f /dev/null, keeping containers alive even after Java crashed. Because containers never exited, Docker’s restart: unless-stopped policy did not trigger.

Entrypoints now monitor Java directly:

  • PD and Store: The entrypoint passes -d false to the startup script, which replaces itself with Java using exec. When Java exits, the container exits immediately and Docker’s restart policy applies.
  • Server: The entrypoint waits with tail --pid=$PID -f /dev/null. SIGTERM/SIGINT traps forward Docker stop signals to Java and wait for graceful shutdown (exit 0). If Java crashes, the entrypoint exits with code 1, triggering the restart policy.
  • PID 1 in every image is dumb-init, which forwards Docker signals to the entrypoint.

Server Startup Timeout

HG_SERVER_STARTUP_TIMEOUT_S controls how long the Server startup script waits for a REST response; the default is 120 seconds. Values must be decimal integers without leading zeros, from 1 to 86400 seconds. Empty strings, 0, negative numbers, fractions, and out-of-range values log an error and exit with code 1.

The entrypoint passes this value to bin/start-hugegraph.sh -t. If Server does not become ready within the deadline or exits early, startup fails and the container exits with code 1; its restart policy may restart it. This duration excludes earlier storage initialization and backend readiness waits.

For example, from the HugeGraph repository’s docker/ directory, extend the standalone Server wait to 300 seconds (Compose passes this variable to the container):

HG_SERVER_STARTUP_TIMEOUT_S=300 HUGEGRAPH_VERSION=latest \
  docker compose -f docker-compose.yml up -d --wait

This setting is independent of Docker health-check start_period, interval, timeout, and retries. Those determine when a container becomes unhealthy; extending health-check grace periods does not extend the startup script deadline. Adjusting this variable does not update health-check settings, so check both for slow startup.

Health-Check Endpoints

All four master Docker images include HEALTHCHECK. docker ps displays health status. Failures during the 90-second startup grace period do not count; three consecutive failures afterward mark a container unhealthy.

ImageHealth-check endpointPortParameters
hugegraph/hugegraph (standalone RocksDB Server)GET /versions8080--interval=15s --timeout=10s --start-period=90s --retries=3
hugegraph/server (HStore Server)GET /versions8080Same as above
hugegraph/pdGET /v1/health8620Same as above
hugegraph/storeGET /v1/health8520Same as above

Compose defines additional health checks, so --wait and depends_on: condition: service_healthy do not depend on image-level checks. Compose uses shorter startup periods (30–120 seconds depending on service and topology) and more retries.

Note: The cron-based -m true monitoring option in start-hugegraph.sh is for VM/bare-metal deployments. Docker images neither install nor use it; use built-in HEALTHCHECK and Docker restart policies.

6.7 - FAQ

  • How to choose the back-end storage? RocksDB or distributed storage?

    HugeGraph supports multiple deployment modes. Choose based on your data scale and scenario:

    • Standalone Mode: Server + RocksDB, suitable for development, testing, Graph AI, and small to medium-scale production environments (≤ 2 TB)
    • Distributed Mode: HugeGraph-PD + HugeGraph-Store (HStore), for deployments that require horizontal scaling and multiple replicas, supporting data scales up to 1 PB

    Version 1.7.0 supports RocksDB, HStore, HBase, and Memory. Legacy backends such as Cassandra, ScyllaDB, MySQL, and PostgreSQL require version 1.5.x or earlier.

  • Prompt when starting the service: xxx (core dumped) xxx

    First confirm that the JDK version is Java 11 or later. HugeGraph 1.7.0 no longer supports Java 8.

  • The service is started successfully, but there is a prompt similar to “Unable to connect to the backend or the connection is not open” when operating the graph

    Persistent local backends such as RocksDB and HBase must be initialized with init-store before their first startup. HStore is managed by PD and Store and does not use this script.

  • Do all backends need to be executed before use init-store, and can the serialization options be filled in at will?

    Memory and HStore do not use init-store; persistent local backends such as RocksDB and HBase must be initialized before first use. The serializer must match the backend, for example RocksDB uses binary.

  • Execution init-store error: Exception in thread "main" java.lang.UnsatisfiedLinkError: /tmp/librocksdbjni3226083071221514754.so: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.10' not found (required by /tmp/librocksdbjni3226083071221514754.so)

    RocksDB requires gcc 4.3.0 (GLIBCXX_3.4.10) and above

  • The bin directory contains start-hugegraph.sh, start-restserver.sh and start-gremlinserver.sh. These scripts seem to be related to startup. Which one should be used?

    Current release packages retain only start-hugegraph.sh as the Server startup script. GremlinServer and the REST Server run in the same process.

  • Two graphs are configured, the names are hugegraph and hugegraph1, and the command to start the service is start-hugegraph.sh. Is only the hugegraph graph opened?

    The script name is unrelated to the graph name. Server scans and loads local graph configurations from conf/graphs by default; change the directory with graphs in conf/rest-server.properties. graph.load_from_local_config defaults to false and controls only constructor preloading and rescanning on reload(), not whether startup loads local graphs. Give each local graph its own configuration file. See the Server configuration guide.

  • After the service starts successfully, garbled characters are returned when using curl to query all vertices

    The batch vertices/edges returned by the server are compressed (gzip), and can be redirected to gunzip for decompression (curl http://example | gunzip), or can be sent with the postman of Firefox or the restlet plug-in of Chrome browser. request, the response data will be decompressed automatically.

  • When using the vertex Id to query the vertex through the RESTful API, it returns empty, but the vertex does exist

    Check the type of the vertex ID. If it is a string type, the “id” part of the API URL needs to be enclosed in double quotes, while for numeric types, it is not necessary to enclose the ID in quotes.

  • Vertex Id has been double quoted as required, but querying the vertex via the RESTful API still returns empty

    Check whether the vertex id contains +, space, /, ?, %, &, and = reserved characters of these URLs. If they exist, they need to be encoded. The following table gives the coded values:

    special character | encoded value
    ------------------| -------------
    +                 | %2B
    space             | %20
    /                 | %2F
    ?                 | %3F
    %                 | %25
    #                 | %23
    &                 | %26
    =                 | %3D
  • Timeout when querying vertices or edges of a certain category (query by label)

    Since the amount of data belonging to a certain label may be relatively large, please add a limit limit.

  • It is possible to operate the graph through the RESTful API, but when sending Gremlin statements, an error is reported: Request Failed(500)

    It may be that the configuration of GremlinServer is wrong, check whether the host and port of gremlin-server.yaml match the gremlinserver.url of rest-server.properties, if they do not match, modify them, and then Restart the service.

  • When using Loader to import data, a Socket Timeout exception occurs, and then Loader is interrupted

    Continuously importing data will put too much pressure on the Server, which will cause some requests to time out. The pressure on Server can be appropriately relieved by adjusting the parameters of Loader (such as: number of retries, retry interval, error tolerance, etc.), and reduce the frequency of this problem.

  • How to delete all data from a graph

    An administrator can call DELETE /graphspaces/{graphspace}/graphs/{graph}/clear?confirm_message=I'm sure to delete all data. The confirm_message query parameter must match that value exactly, otherwise the request is rejected. See the Graph API for details. This operation removes schemas, vertices, edges, and indexes.

  • The database has been cleared and init-store has been executed, but when trying to add a schema, the prompt “xxx has existed” appeared.

    There is a cache in the HugeGraphServer, and it is necessary to restart the Server when the database is cleared, otherwise the residual cache will be inconsistent.

  • An error is reported during the process of inserting vertices or edges: The max length of vertex id is 16384, but got xxx {yyy} or The max length of edge id is 65536, but got xxx {yyy}

    In order to ensure query performance, the current backend storage limits the length of the id column. The vertex id cannot exceed 16384 bytes and the edge id cannot exceed 65536 bytes. An index id longer than 32 bytes is stored as a hash instead of being rejected.

  • Is there support for nested attributes, and if not, are there any alternatives?

    Nested attributes are currently not supported. Alternative: Nested attributes can be taken out as individual vertices and connected with edges.

  • Can an EdgeLabel connect multiple pairs of VertexLabel, such as “investment” relationship, which can be “individual” investing in “enterprise”, or “enterprise” investing in “enterprise”?

    Yes. Call link(sourceLabel, targetLabel) once per pair when building the EdgeLabel; every pair is kept, so one “investment” label can cover both “individual” to “enterprise” and “enterprise” to “enterprise”. The older sourceLabel() and targetLabel() builder methods are deprecated and accept only a single pair.

  • Prompt HTTP 415 Unsupported Media Type when sending a request through RestAPI

    Content-Type: application/json needs to be specified in the request header

Other issues can be searched in the issue area of the corresponding project, such as Server-Issues / Loader Issues

6.8 - Security Report

Reporting New Security Problems with Apache HugeGraph

Warning

Production security requirements

HugeGraph disables user authentication by default. In production, enable authentication and authorization, set white_ip.status=enable, maintain the IP allowlist, and grant minimum permissions. Do not expose Gremlin, Cypher, or other query endpoints directly to the public network. See the IP allowlist API. Isolate the Server process, for example with Docker or Kubernetes.

Standard Server configuration writes authentication-proxy audit records to audit-*.log. Retain these files and restrict read access. auth.audit_log_rate controls the maximum per-user log output rate rather than serving as a dedicated audit-log on/off switch.

Warning

Vulnerability reporting scope

The community has received many reports about the flexibility of graph query languages. Until the security architecture is refactored, known risks from DSL queries executed with Auth disabled or deliberately skipped, outside an authorized session, will not be treated individually as new vulnerabilities.

Exploitation through anonymous or unauthorized access despite Auth being enabled, or bypassing an IP allowlist or escaping a container to cause serious privilege violations or system compromise, remains a high-risk security vulnerability. Please report these cases to the community.

Following ASF procedures, the HugeGraph community actively works to resolve security issues.

Report security issues privately to the dedicated security list first. See the ASF security procedures for details.

The security list handles undisclosed vulnerabilities and their resolution. Report ordinary software bugs through GitHub Issues/Discussions or the developer mailing list. Messages unrelated to security sent to the security list will be ignored.

Security mailing list: security@hugegraph.apache.org

The general vulnerability-handling process is:

  • The reporter privately sends the HugeGraph security list the affected versions, description, reproduction steps, and impact.
  • The project security team works privately with the reporter on a fix. A CVE identifier can be requested after initial confirmation.
  • The project releases an updated version of the affected software containing the fix.
  • At an appropriate time, the project discloses the general issue and how to apply the fix, following ASF rules and omitting sensitive reproduction details.
  • CVE publication and related steps follow the ASF security procedures.

Known Security Vulnerabilities (CVEs)

HugeGraph main repository (Server/PD/Store)

HugeGraph-Toolchain repository (Hubble/Loader/Client/Tools/..)

7 - Query Languages

HugeGraph supports Gremlin and Cypher. This section mainly covers Gremlin; see the Cypher API for the Cypher HTTP interface.

7.1 - HugeGraph Gremlin

Overview

HugeGraph supports Gremlin, a graph traversal query language of Apache TinkerPop3. While SQL is a query language for relational databases, Gremlin is a general-purpose query language for graph databases. Gremlin can be used to create entities (Vertex and Edge) of a graph, modify the properties of entities, delete entities, as well as perform graph queries.

Gremlin can be used to create entities (Vertex and Edge) of a graph, modify the properties of entities, and delete entities. More importantly, it can be used to perform graph querying and analysis operations.

TinkerPop Features

HugeGraph implements the TinkerPop framework, but not all TinkerPop features are implemented.

The table below lists the support status of various TinkerPop features in HugeGraph:

Graph Features

NameDescriptionSupport
ComputerDetermines if the {@code Graph} implementation supports {@link GraphComputer} based processingfalse
TransactionsDetermines if the {@code Graph} implementations supports transactions.true
PersistenceDetermines if the {@code Graph} implementation supports persisting it’s contents natively to disk.This feature does not refer to every graph’s ability to write to disk via the Gremlin IO packages(.e.g. GraphML), unless the graph natively persists to disk via those options somehow. For example,TinkerGraph does not support this feature as it is a pure in-sideEffects graph.Backend-dependent
ThreadedTransactionsDetermines if the {@code Graph} implementation supports threaded transactions which allow a transaction be executed across multiple threads via {@link Transaction#createThreadedTx()}.false
ConcurrentAccessDetermines if the {@code Graph} implementation supports more than one connection to the same instance at the same time. For example, Neo4j embedded does not support this feature because concurrent access to the same database files by multiple instances is not possible. However, Neo4j HA could support this feature as each new {@code Graph} instance coordinates with the Neo4j cluster allowing multiple instances to operate on the same database.false

Vertex Features

NameDescriptionSupport
UserSuppliedIdsDetermines if an {@link Element} can have a user defined identifier. Implementation that do not support this feature will be expected to auto-generate unique identifiers. In other words, if the {@link Graph} allows {@code graph.addVertex(id,x)} to work and thus set the identifier of the newly added {@link Vertex} to the value of {@code x} then this feature should return true. In this case, {@code x} is assumed to be an identifier data type that the {@link Graph} will accept.true
NumericIdsDetermines if an {@link Element} has numeric identifiers as their internal representation. In other words,if the value returned from {@link Element#id()} is a numeric value then this method should be return {@code true}. Note that this feature is most generally used for determining the appropriate tests to execute in the Gremlin Test Suite.false
StringIdsDetermines if an {@link Element} has string identifiers as their internal representation. In other words, if the value returned from {@link Element#id()} is a string value then this method should be return {@code true}. Note that this feature is most generally used for determining the appropriate tests to execute in the Gremlin Test Suite.true
UuidIdsDetermines if an {@link Element} has UUID identifiers as their internal representation. In other words,if the value returned from {@link Element#id()} is a {@link UUID} value then this method should be return {@code true}.Note that this feature is most generally used for determining the appropriate tests to execute in the Gremlin Test Suite.false
CustomIdsDetermines if an {@link Element} has a specific custom object as their internal representation.In other words, if the value returned from {@link Element#id()} is a type defined by the graph implementations, such as OrientDB’s {@code Rid}, then this method should be return {@code true}.Note that this feature is most generally used for determining the appropriate tests to execute in the Gremlin Test Suite.true
AnyIdsDetermines if an {@link Element} any Java object is a suitable identifier. TinkerGraph is a good example of a {@link Graph} that can support this feature, as it can use any {@link Object} as a value for the identifier. Note that this feature is most generally used for determining the appropriate tests to execute in the Gremlin Test Suite. This setting should only return {@code true} if {@link #supportsUserSuppliedIds()} is {@code true}.false
AddPropertyDetermines if an {@link Element} allows properties to be added. This feature is set independently from supporting “data types” and refers to support of calls to {@link Element#property(String, Object)}.true
RemovePropertyDetermines if an {@link Element} allows properties to be removed.true
AddVerticesDetermines if a {@link Vertex} can be added to the {@code Graph}.true
MultiPropertiesDetermines if a {@link Vertex} can support multiple properties with the same key.true
DuplicateMultiPropertiesDetermines if a {@link Vertex} can support non-unique values on the same key. For this value to be {@code true}, then {@link #supportsMetaProperties()} must also return true. By default this method, just returns what {@link #supportsMultiProperties()} returns.true
MetaPropertiesDetermines if a {@link Vertex} can support properties on vertex properties. It is assumed that a graph will support all the same data types for meta-properties that are supported for regular properties.false
RemoveVerticesDetermines if a {@link Vertex} can be removed from the {@code Graph}.true

Edge Features

NameDescriptionSupport
UserSuppliedIdsDetermines if an {@link Element} can have a user defined identifier. Implementation that do not support this feature will be expected to auto-generate unique identifiers. In other words, if the {@link Graph} allows {@code graph.addVertex(id,x)} to work and thus set the identifier of the newly added {@link Vertex} to the value of {@code x} then this feature should return true. In this case, {@code x} is assumed to be an identifier data type that the {@link Graph} will accept.false
NumericIdsDetermines if an {@link Element} has numeric identifiers as their internal representation. In other words,if the value returned from {@link Element#id()} is a numeric value then this method should be return {@code true}. Note that this feature is most generally used for determining the appropriate tests to execute in the Gremlin Test Suite.false
StringIdsDetermines if an {@link Element} has string identifiers as their internal representation. In other words, if the value returned from {@link Element#id()} is a string value then this method should be return {@code true}. Note that this feature is most generally used for determining the appropriate tests to execute in the Gremlin Test Suite.true
UuidIdsDetermines if an {@link Element} has UUID identifiers as their internal representation. In other words,if the value returned from {@link Element#id()} is a {@link UUID} value then this method should be return {@code true}.Note that this feature is most generally used for determining the appropriate tests to execute in the Gremlin Test Suite.false
CustomIdsDetermines if an {@link Element} has a specific custom object as their internal representation.In other words, if the value returned from {@link Element#id()} is a type defined by the graph implementations, such as OrientDB’s {@code Rid}, then this method should be return {@code true}.Note that this feature is most generally used for determining the appropriate tests to execute in the Gremlin Test Suite.true
AnyIdsDetermines if an {@link Element} any Java object is a suitable identifier. TinkerGraph is a good example of a {@link Graph} that can support this feature, as it can use any {@link Object} as a value for the identifier. Note that this feature is most generally used for determining the appropriate tests to execute in the Gremlin Test Suite. This setting should only return {@code true} if {@link #supportsUserSuppliedIds()} is {@code true}.false
AddPropertyDetermines if an {@link Element} allows properties to be added. This feature is set independently from supporting “data types” and refers to support of calls to {@link Element#property(String, Object)}.true
RemovePropertyDetermines if an {@link Element} allows properties to be removed.true
AddEdgesDetermines if an {@link Edge} can be added to a {@code Vertex}.true
RemoveEdgesDetermines if an {@link Edge} can be removed from a {@code Vertex}.true

Data Type Features

NameDescriptionSupport
BooleanValuestrue
ByteValuestrue
DoubleValuestrue
FloatValuestrue
IntegerValuestrue
LongValuestrue
MapValuesSupports setting of a {@code Map} value. The assumption is that the {@code Map} can contain arbitrary serializable values that may or may not be defined as a feature itselffalse
MixedListValuesSupports setting of a {@code List} value. The assumption is that the {@code List} can contain arbitrary serializable values that may or may not be defined as a feature itself. As this{@code List} is “mixed” it does not need to contain objects of the same type.false
BooleanArrayValuesfalse
ByteArrayValuestrue
DoubleArrayValuesfalse
FloatArrayValuesfalse
IntegerArrayValuesfalse
LongArrayValuesfalse
SerializableValuesfalse
StringArrayValuesfalse
StringValuestrue
UniformListValuesSupports setting of a {@code List} value. The assumption is that the {@code List} can contain arbitrary serializable values that may or may not be defined as a feature itself. As this{@code List} is “uniform” it must contain objects of the same type.true

Gremlin Steps

HugeGraph supports all steps of Gremlin. For complete reference information about Gremlin, please refer to the Gremlin official website.

StepDescriptionDocumentation
addEAdd an edge between two vertices.addE step
addVadd vertices to graph.addV step
andMake sure all traversals return values.and step
asStep modulator for assigning variables to the step’s output.as step
byStep Modulators used in conjunction with group and order.by step
coalesceReturns the first traversal that returns a result.coalesce step
constantReturns a constant value. Used in conjunction with coalesce.constant step
countReturns a count from the traversal.count step
dedupReturns values with duplicates removed.dedup step
dropDiscards a value (vertex/edge).drop step
foldActs as a barrier for computing aggregated values from results.fold step
groupGroups values based on specified labels.group step
hasUsed to filter properties, vertices, and edges. Supports hasLabel, hasId, hasNot, and has variants.has step
injectInjects values into the stream.inject step
isUsed to filter by a Boolean expression.is step
limitUsed to limit the number of items in a traversal.limit step
localLocally wraps a part of a traversal, similar to a subquery.local step
notUsed to generate the negation result of a filter.not step
optionalReturns the result of a specified traversal if it generates any results, otherwise returns the calling element.optional step
orEnsures that at least one traversal returns a value.or step
orderReturns results in the specified order.order step
pathReturns the full path of the traversal.path step
projectProjects properties as a map.project step
propertiesReturns properties with specified labels.properties step
rangeFilters based on a specified range of values.range step
repeatRepeats a step a specified number of times. Used for looping.repeat step
sampleUsed to sample results returned by the traversal.sample step
selectUsed to project the results returned by the traversal.select step
storeThis step is used for non-blocking aggregation of results returned by traversalstore step
treeAggregate the paths in vertices into a tree.tree step
unfoldUnfolds an iterator as a step.unfold step
unionMerge the results returned by multiple traversals.union step
VThese are the steps required for traversing between vertices and edges: V, E, out, in, both, outE, inE, bothE, outV, inV, bothV, and otherV.vertex steps
whereUsed to filter the results returned by a traversal. Supports eq, neq, lt, lte, gt, gte, and between operators.where step

7.2 - HugeGraph Examples

1 Overview

This example uses the TitanDB Getting Started guide as a template to demonstrate how to use HugeGraph. By comparing HugeGraph and TitanDB, you can understand the differences between them.

1.1 Similarities and Differences between HugeGraph and TitanDB

Both HugeGraph and TitanDB are graph databases based on the Apache TinkerPop3 framework. They both support the Gremlin graph query language and share many similarities in terms of usage and interfaces. However, HugeGraph is a completely new design and development, characterized by its clear code structure, richer features, and more user-friendly interfaces.

Compared to TitanDB, HugeGraph’s main features are as follows:

  • HugeGraph currently offers a comprehensive suite of tools, including HugeGraph-API, HugeGraph-Client, HugeGraph-Loader, HugeGraph-Studio, and HugeGraph-Spark. These components facilitate system integration, data loading, visual graph querying, Spark connectivity, and other functionalities.
  • HugeGraph incorporates the concepts of Server and Client, allowing third-party systems to connect via multiple methods such as JAR references, clients, and APIs. In contrast, TitanDB only supports connections via JAR references.
  • HugeGraph requires explicit schema definition, and all insertions and queries must pass strict schema validation. Implicit schema creation is not supported at the moment.
  • HugeGraph makes full use of the characteristics of the underlying storage system to achieve efficient data access, whereas TitanDB ignores the differences of the backend with a unified Kv structure.
  • HugeGraph’s update operations can be performed on-demand (e.g., updating a specific attribute), offering better performance. TitanDB uses a read-and-update approach for updates.
  • Both VertexId and EdgeId in HugeGraph support concatenation, allowing for automatic deduplication and better query performance. In TitanDB, all IDs are auto-generated and require indexing for queries.

1.2 Character Relationship Graph

This example uses the Property Graph Model to describe the relationships between characters in Greek mythology, also known as the character relationship graph. The specific relationships are shown in the diagram below.

image

In the diagram, circular nodes represent entities (Vertices), arrows represent relationships (Edges), and the content in the boxes represents attributes.

There are two types of vertices in this graph: characters and locations, as shown in the table below:

NameTypeAttributes
charactervertexname,age,type
locationvertexname

There are six types of relationships: father, mother, brother, battled, lives, and pet. The details of these relationships are as follows:

NameTypeSource Vertex LabelTarget Vertex LabelAttributes
fatheredgecharactercharacter-
motheredgecharactercharacter-
brotheredgecharactercharacter-
battlededgecharactercharactertime
petedgecharactercharacter-
livesedgecharacterlocationreason

An edge label can be linked to more than one pair of source and target vertex labels: call link(sourceLabel, targetLabel) once per pair when creating it. The deprecated sourceLabel() and targetLabel() builder methods accept only a single pair.

In this example, the original TitanDB’s monster, god, human, and demigod are all represented using the same vertex label: character in HugeGraph, with an additional type attribute to indicate the type of character. The edge labels remain consistent with the original TitanDB.

2 Graph Schema and Data Ingest Examples

HugeGraph requires explicit schema creation, which involves creating PropertyKeys, VertexLabels, and EdgeLabels in sequence. If indexing is needed, IndexLabels must also be created.

2.1 Graph Schema

The snippets assume a local Groovy script has opened HugeGraph as graph and set g = graph.traversal(). Start with the Gremlin Console local-mode example, whose scripts/example.groovy creates both variables. In a remote Console, g is a Server-side traversal-source alias and there is no local graph object; use the Schema REST API for remote schema creation.

import org.apache.tinkerpop.gremlin.structure.T
import org.apache.tinkerpop.gremlin.structure.Vertex

schema = graph.schema()
g = graph.traversal()

schema.propertyKey("name").asText().ifNotExist().create()
schema.propertyKey("age").asInt().ifNotExist().create()
schema.propertyKey("time").asInt().ifNotExist().create()
schema.propertyKey("reason").asText().ifNotExist().create()
schema.propertyKey("type").asText().ifNotExist().create()

schema.vertexLabel("character").properties("name", "age", "type").primaryKeys("name").nullableKeys("age").ifNotExist().create()
schema.vertexLabel("location").properties("name").primaryKeys("name").ifNotExist().create()

schema.edgeLabel("father").link("character", "character").ifNotExist().create()
schema.edgeLabel("mother").link("character", "character").ifNotExist().create()
schema.edgeLabel("battled").link("character", "character").properties("time").ifNotExist().create()
schema.edgeLabel("lives").link("character", "location").properties("reason").nullableKeys("reason").ifNotExist().create()
schema.edgeLabel("pet").link("character", "character").ifNotExist().create()
schema.edgeLabel("brother").link("character", "character").ifNotExist().create()

2.2 Graph Data

// add vertices
Vertex saturn = graph.addVertex(T.label, "character", "name", "saturn", "age", 10000, "type", "titan")
Vertex sky = graph.addVertex(T.label, "location", "name", "sky")
Vertex sea = graph.addVertex(T.label, "location", "name", "sea")
Vertex jupiter = graph.addVertex(T.label, "character", "name", "jupiter", "age", 5000, "type", "god")
Vertex neptune = graph.addVertex(T.label, "character", "name", "neptune", "age", 4500, "type", "god")
Vertex hercules = graph.addVertex(T.label, "character", "name", "hercules", "age", 30, "type", "demigod")
Vertex alcmene = graph.addVertex(T.label, "character", "name", "alcmene", "age", 45, "type", "human")
Vertex pluto = graph.addVertex(T.label, "character", "name", "pluto", "age", 4000, "type", "god")
Vertex nemean = graph.addVertex(T.label, "character", "name", "nemean", "type", "monster")
Vertex hydra = graph.addVertex(T.label, "character", "name", "hydra", "type", "monster")
Vertex cerberus = graph.addVertex(T.label, "character", "name", "cerberus", "type", "monster")
Vertex tartarus = graph.addVertex(T.label, "location", "name", "tartarus")

// add edges
jupiter.addEdge("father", saturn)
jupiter.addEdge("lives", sky, "reason", "loves fresh breezes")
jupiter.addEdge("brother", neptune)
jupiter.addEdge("brother", pluto)
neptune.addEdge("lives", sea, "reason", "loves waves")
neptune.addEdge("brother", jupiter)
neptune.addEdge("brother", pluto)
hercules.addEdge("father", jupiter)
hercules.addEdge("mother", alcmene)
hercules.addEdge("battled", nemean, "time", 1)
hercules.addEdge("battled", hydra, "time", 2)
hercules.addEdge("battled", cerberus, "time", 12)
pluto.addEdge("brother", jupiter)
pluto.addEdge("brother", neptune)
pluto.addEdge("lives", tartarus, "reason", "no fear of death")
pluto.addEdge("pet", cerberus)
cerberus.addEdge("lives", tartarus)

graph.tx().commit()

2.3 Indices

HugeGraph by default automatically generates IDs. However, if a user specifies the primaryKeys field list for a VertexLabel through primaryKeys, the ID strategy for that VertexLabel will automatically switch to the primaryKeys strategy. Once the primaryKeys strategy is enabled, HugeGraph generates VertexId by concatenating vertexLabel+primaryKeys, which allows for automatic deduplication. Additionally, there is no need to create extra indexes to use the properties in primaryKeys for fast querying. For example, both “character” and “location” have the primaryKeys("name") attribute, so without creating additional indexes, vertices can be queried using g.V().hasLabel('character').has('name','hercules').

3 Graph Traversal Examples

3.1 Traversal Query

1. Find the grandfather of hercules

g.V().hasLabel('character').has('name','hercules').out('father').out('father')

It can also be done using the repeat method:

g.V().hasLabel('character').has('name','hercules').repeat(__.out('father')).times(2)

2. Find the name of Hercules’s father

g.V().hasLabel('character').has('name','hercules').out('father').value('name')

3. Find the characters with age > 100

g.V().hasLabel('character').has('age',gt(100))

4. Find who are pluto’s cohabitants

g.V().hasLabel('character').has('name','pluto').out('lives').in('lives').values('name')

5. Find pluto can’t be his own cohabitant

pluto = g.V().hasLabel('character').has('name', 'pluto')
g.V(pluto).out('lives').in('lives').where(is(neq(pluto))).values('name')

// use 'as'
g.V().hasLabel('character').has('name', 'pluto').as('x').out('lives').in('lives').where(neq('x')).values('name')

6. Pluto’s Brothers

pluto = g.V().hasLabel('character').has('name', 'pluto').next()
// where do pluto's brothers live?
g.V(pluto).out('brother').out('lives').values('name')

// which brother lives in which place?
g.V(pluto).out('brother').as('god').out('lives').as('place').select('god','place')

// what is the name of the brother and the name of the place?
g.V(pluto).out('brother').as('god').out('lives').as('place').select('god','place').by('name')

It is recommended to use HugeGraph-Hubble to execute the above code visually. Additionally, the code can be executed through various other methods such as HugeGraph-Client, HugeGraph-Api, GremlinConsole, and GremlinDriver.

3.2 Summary

HugeGraph currently supports Gremlin syntax, and users can implement various query requirements through Gremlin / REST-API.

8 - PERFORMANCE

⚠️ The performance chapter is being reworked

The benchmark, API performance, and Loader performance reports in this chapter are based on historical versions, test methods, or test environments. Their results do not represent current releases and should not be used directly for version comparisons or capacity planning. The test methods, software and hardware environments, and reports will be updated consistently, and the old reports will then be archived.

Available benchmark pages:

8.1 - HugeGraph BenchMark Performance

Note:

The current performance metrics are based on an earlier version. The latest version has significant improvements in both performance and functionality. We encourage you to refer to the most recent release featuring autonomous distributed storage and enhanced computational push down capabilities. Alternatively, you may wait for the community to update the data with these enhancements.

The HugeGraph 0.4.4 historical benchmark in this directory uses data collected in 2018 and is provided for historical reference only.

1 Test environment

1.1 Hardware information

CPUMemory网卡磁盘
48 Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz128G10000Mbps750GB SSD

1.2 Software information

1.2.1 Test cases

Testing is done using the graphdb-benchmark, a benchmark suite for graph databases. This benchmark suite mainly consists of four types of tests:

  • Massive Insertion, which involves batch insertion of vertices and edges, with a certain number of vertices or edges being submitted at once.
  • Single Insertion, which involves the immediate insertion of each vertex or edge, one at a time.
  • Query, which mainly includes the basic query operations of the graph database:
    • Find Neighbors, which queries the neighbors of all vertices.
    • Find Adjacent Nodes, which queries the adjacent vertices of all edges.
    • Find the Shortest Path, which queries the shortest path from the first vertex to 100 random vertices.
  • Clustering, which is a community detection algorithm based on the Louvain Method.
1.2.2 Test dataset

Tests are conducted using both synthetic and real data.

The size of the datasets used in this test is not mentioned.

NameNumber of VerticesNumber of EdgesFile Size
email-enron.txt36,691367,6614MB
com-youtube.ungraph.txt1,157,8062,987,62438.7MB
amazon0601.txt403,3933,387,38847.9MB
com-lj.ungraph.txt399796134681189479MB

1.3 Service configuration

  • HugeGraph version: 0.5.6, RestServer and Gremlin Server and backends are on the same server

    • RocksDB version: rocksdbjni-5.8.6
  • Titan version: 0.5.4, using thrift+Cassandra mode

    • Cassandra version: cassandra-3.10, commit-log and data use SSD together
  • Neo4j version: 2.0.1

The Titan version adapted by graphdb-benchmark is 0.5.4.

2 Test results

2.1 Batch insertion performance

Backendemail-enron(30w)amazon0601(300w)com-youtube.ungraph(300w)com-lj.ungraph(3000w)
HugeGraph0.6295.7115.24367.033
Titan10.15108.569150.2661217.944
Neo4j3.88418.93824.890281.537

Instructions

  • The data scale is in the table header in terms of edges
  • The data in the table is the time for batch insertion, in seconds
  • For example, HugeGraph(RocksDB) spent 5.711 seconds to insert 3 million edges of the amazon0601 dataset.
Conclusion
  • The performance of batch insertion: HugeGraph(RocksDB) > Neo4j > Titan(thrift+Cassandra)

2.2 Traversal performance

2.2.1 Explanation of terms
  • FN(Find Neighbor): Traverse all vertices, find the adjacent edges based on each vertex, and use the edges and vertices to find the other vertices adjacent to the original vertex.
  • FA(Find Adjacent): Traverse all edges, get the source vertex and target vertex based on each edge.
2.2.2 FN performance
Backendemail-enron(3.6w)amazon0601(40w)com-youtube.ungraph(120w)com-lj.ungraph(400w)
HugeGraph4.07245.11866.006609.083
Titan8.08492.507184.5431099.371
Neo4j2.42410.53711.609106.919

Instructions

  • The data in the table header “()” represents the data scale, in terms of vertices.
  • The data in the table represents the time spent traversing vertices in seconds.
  • For example, HugeGraph uses the RocksDB backend to traverse all vertices in amazon0601, and search for adjacent edges and another vertex, which takes a total of 45.118 seconds.
2.2.3 FA performance
Backendemail-enron(30w)amazon0601(300w)com-youtube.ungraph(300w)com-lj.ungraph(3000w)
HugeGraph1.54010.76411.243151.271
Titan7.36193.344169.2181085.235
Neo4j1.6734.7754.28440.507

Explanation

  • The data size in the header “()” is based on the number of vertices.
  • The data in the table is the time it takes to traverse the vertices in seconds.
  • For example, HugeGraph with RocksDB backend traverses all vertices in the amazon0601 dataset, and it looks up adjacent edges and other vertices, taking a total of 45.118 seconds.
Conclusion
  • Traversal performance: Neo4j > HugeGraph(RocksDB) > Titan(thrift+Cassandra)

2.3 Performance of Common Graph Analysis Methods in HugeGraph

Terminology Explanation
  • FS (Find Shortest Path): finding the shortest path between two vertices
  • K-neighbor: all vertices that can be reached by traversing K hops (including 1, 2, 3…(K-1) hops) from the starting vertex
  • K-out: all vertices that can be reached by traversing exactly K out-edges from the starting vertex.
FS performance
Backendemail-enron(30w)amazon0601(300w)com-youtube.ungraph(300w)com-lj.ungraph(3000w)
HugeGraph0.4940.1033.3648.155
Titan11.8180.239377.709575.678
Neo4j1.7191.8001.9568.530

Explanation

  • The data in the header “()” represents the data scale in terms of edges
  • The data in the table is the time it takes to find the shortest path from the first vertex to 100 randomly selected vertices in seconds
  • For example, HugeGraph using the RocksDB backend to find the shortest path from the first vertex to 100 randomly selected vertices in the amazon0601 graph took a total of 0.103s.
Conclusion
  • In scenarios with small data size or few vertex relationships, HugeGraph outperforms Neo4j and Titan.
  • As the data size increases and the degree of vertex association increases, the performance of HugeGraph and Neo4j tends to be similar, both far exceeding Titan.
K-neighbor Performance
VertexDepthDegree 1Degree 2Degree 3Degree 4Degree 5Degree 6
v1Time0.031s0.033s0.048s0.500s11.27sOOM
v111Time0.027s0.034s0.115s1.36sOOM–
v1111Time0.039s0.027s0.052s0.511s10.96sOOM

Explanation

  • HugeGraph-Server’s JVM memory is set to 32GB and may experience OOM when the data is too large.
K-out performance
VertexDepth1st Degree2nd Degree3rd Degree4th Degree5th Degree6th Degree
v1Time0.054s0.057s0.109s0.526s3.77sOOM
Degree10133245350,8301,128,688
v111Time0.032s0.042s0.136s1.25s20.62sOOM
Degree1021149441131502,629,970
v1111Time0.039s0.045s0.053s1.10s2.92sOOM
Degree101402555508251,070,230

Explanation

  • The JVM memory of HugeGraph-Server is set to 32GB, and OOM may occur when the data is too large.
Conclusion
  • In the FS scenario, HugeGraph outperforms Neo4j and Titan in terms of performance.
  • In the K-neighbor and K-out scenarios, HugeGraph can achieve results returned within seconds within 5 degrees.

2.4 Comprehensive Performance Test - CW

DatabaseSize 1000Size 5000Size 10000Size 20000
HugeGraph(core)20.804242.099744.7801700.547
Titan45.790820.6332652.2359568.623
Neo4j5.91350.267142.354460.880

Explanation

  • The “scale” is based on the number of vertices.
  • The data in the table is the time required to complete community discovery in seconds. For example, if HugeGraph uses the RocksDB backend and operates on a dataset of 10,000 vertices, and the community aggregation is no longer changing, it takes 744.780 seconds.
  • The CW test is a comprehensive evaluation of CRUD operations.
  • In this test, HugeGraph, like Titan, did not use the client and directly operated on the core.
Conclusion
  • Performance of community detection algorithm: Neo4j > HugeGraph > Titan

8.2 - HugeGraph-API Performance

The HugeGraph API performance test mainly tests HugeGraph-Server’s ability to concurrently process RESTful API requests, including:

  • Single insertion of vertices/edges
  • Batch insertion of vertices/edges
  • Vertex/Edge Queries

For the performance test of the RESTful API of each release version of HugeGraph, please refer to:

Updates coming soon, stay tuned!

8.2.1 - v0.5.6 Stand-alone(RocksDB)

Note:

The current performance metrics are based on an earlier version. The latest version has significant improvements in both performance and functionality. We encourage you to refer to the most recent release featuring autonomous distributed storage and enhanced computational push down capabilities. Alternatively, you may wait for the community to update the data with these enhancements.

1 Test environment

Compressed machine information:

CPUMemory网卡磁盘
48 Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz128G10000Mbps750GB SSD,2.7T HDD
  • Information about the machine used to generate loads: configured the same as the machine that is being tested under load.
  • Testing tool: Apache JMeter 2.5.1

Note: The load-generating machine and the machine under test are located in the same local network.

2 Test description

2.1 Definition of terms (the unit of time is ms)

  • Samples: The total number of threads completed in the current scenario.
  • Average: The average response time.
  • Median: The statistical median of the response time.
  • 90% Line: The response time below which 90% of all threads fall.
  • Min: The minimum response time.
  • Max: The maximum response time.
  • Error: The error rate.
  • Throughput: The number of requests processed per unit of time.
  • KB/sec: Throughput measured in terms of data transferred per second.

2.2 Underlying storage

RocksDB is used for backend storage, HugeGraph and RocksDB are both started on the same machine, and the configuration files related to the server remain as default except for the modification of the host and port.

3 Summary of performance results

  1. The speed of inserting a single vertex and edge in HugeGraph is about 1w per second
  2. The batch insertion speed of vertices and edges is much faster than the single insertion speed
  3. The concurrency of querying vertices and edges by id can reach more than 13000, and the average delay of requests is less than 50ms

4 Test results and analysis

4.1 batch insertion

4.1.1 Upper limit stress testing
Test methods

The upper limit of stress testing is to continuously increase the concurrency and test whether the server can still provide services normally.

Stress Parameters

Duration: 5 minutes

Maximum insertion speed for vertices:
image
in conclusion:
  • With a concurrency of 2200, the throughput for vertices is 2026.8. This means that the system can process data at a rate of 405360 per second (2026.8 * 200).
Maximum insertion speed for edges
image
Conclusion:
  • With a concurrency of 900, the throughput for edges is 776.9. This means that the system can process data at a rate of 388450 per second (776.9 * 500).

4.2 Single insertion

4.2.1 Stress limit testing
Test Methods

Stress limit testing is a process of continuously increasing the concurrency level to test the upper limit of the server’s ability to provide normal service.

Stress parameters
  • Duration: 5 minutes.
  • Service exception indicator: Error rate greater than 0.00%.
Single vertex insertion
image
Conclusion:
  • With a concurrency of 11500, the throughput is 10730. This means that the system can handle a single concurrent insertion of vertices at a concurrency level of 11500.
Single edge insertion
image
Conclusion:
  • With a concurrency of 9000, the throughput is 8418. This means that the system can handle a single concurrent insertion of edges at a concurrency level of 9000.

4.3 Search by ID

4.3.1 Stress test upper limit
Testing method

Continuously increasing the concurrency level to test the upper limit of the server’s ability to provide service under normal conditions.

stress parameters
  • Duration: 5 minutes
  • Service abnormality indicator: error rate greater than 0.00%
Querying vertices by ID
image
Conclusion:
  • Concurrency is 14,000, throughput is 12,663. The concurrency capacity for querying vertices by ID is 14,000, with an average delay of 44ms.
Querying edges by ID
image
Conclusion:
  • Concurrency is 13,000, throughput is 12,225. The concurrency capacity for querying edges by ID is 13,000, with an average delay of 12ms.

8.2.2 - v0.5.6 Cluster(Cassandra)

Note:

The current performance metrics are based on an earlier version. The latest version has significant improvements in both performance and functionality. We encourage you to refer to the most recent release featuring autonomous distributed storage and enhanced computational push down capabilities. Alternatively, you may wait for the community to update the data with these enhancements.

1 Test environment

Compressed machine information

CPUMemory网卡磁盘
48 Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz128G10000Mbps750GB SSD,2.7T HDD
  • Starting Pressure Machine Information: Configure the same as the compressed machine.
  • Testing tool: Apache JMeter 2.5.1.

Note: The machine used to initiate the load and the machine being tested are located in the same data center (or server room)

2 Test Description

2.1 Definition of terms (the unit of time is ms)

  • Samples – The total number of threads completed in this scenario.
  • Average – The average response time.
  • Median – The median response time in statistical terms.
  • 90% Line – The response time below which 90% of all threads fall.
  • Min – The minimum response time.
  • Max – The maximum response time.
  • Error – The error rate.
  • Throughput – The number of transactions processed per unit of time.
  • KB/sec – The throughput measured in terms of data transmitted per second.

2.2 Low-Level Storage

A 15-node Cassandra cluster is used for backend storage. HugeGraph and the Cassandra cluster are located on separate servers. Server-related configuration files are modified only for host and port settings, while the rest remain default.

3 Summary of Performance Results

  1. The speed of a single vertex and edge insertion in HugeGraph is 9000 and 4500 per second, respectively.
  2. The speed of bulk vertex and edge insertion is 50,000 and 150,000 per second, respectively, which is much higher than the single insertion speed.
  3. The concurrency for querying vertices and edges by ID can reach more than 12,000, and the average request delay is less than 70ms.

4 Test Results and Analysis

4.1 Batch Insertion

4.1.1 Pressure Upper Limit Test
Test Method

Continuously increase the concurrency level to test the upper limit of the server’s ability to provide services.

Pressure Parameters

Duration: 5 minutes.

Maximum Insertion Speed of Vertices:
image
Conclusion:
  • At a concurrency level of 3500, the throughput of vertices is 261, and the amount of data processed per second is 52,200 (261 * 200).
Maximum Insertion Speed of Edges:
image
Conclusion:
  • At a concurrency level of 1000, the throughput of edges is 323, and the amount of data processed per second is 161,500 (323 * 500).

4.2 Single Insertion

4.2.1 Pressure Upper Limit Test
Test Method

Continuously increase the concurrency level to test the upper limit of the server’s ability to provide services.

Pressure Parameters
  • Duration: 5 minutes.
  • Service exception mark: Error rate greater than 0.00%.
Single Insertion of Vertices:
image
Conclusion:
  • At a concurrency level of 9000, the throughput is 8400, and the single-insertion concurrency capability for vertices is 9000.
Single Insertion of Edges:
image
Conclusion:
  • At a concurrency level of 4500, the throughput is 4160, and the single-insertion concurrency capability for edges is 4500.

4.3 Query by ID

4.3.1 Pressure Upper Limit Test
Test Method

Continuously increase the concurrency and test the upper limit of the pressure that the server can still provide services normally.

Pressure Parameters
  • Duration: 5 minutes
  • Service exception flag: error rate greater than 0.00%
Query by ID for vertices
image
Conclusion:
  • The concurrent capacity of the vertex search by ID is 14500, with a throughput of 13576 and an average delay of 11ms.
Edge search by ID
image
Conclusion:
  • For edge ID-based queries, the server’s concurrent capacity is up to 12,000, with a throughput of 10,688 and an average latency of 63ms.

8.3 - HugeGraph-Loader Performance

Note:

The current performance metrics are based on an earlier version. The latest version has significant improvements in both performance and functionality. We encourage you to refer to the most recent release featuring autonomous distributed storage and enhanced computational push down capabilities. Alternatively, you may wait for the community to update the data with these enhancements.

Use Cases

When the number of graph data to be batch inserted (including vertices and edges) is at the billion level or below, or the total data size is less than TB, the HugeGraph-Loader tool can be used to continuously and quickly import graph data.

Performance

The test uses the edge data of website.

RocksDB single-machine performance (Update: multi-raft + rocksdb cluster is supported now)

  • When the label index is turned off, 228k edges/s.
  • When the label index is turned on, 153k edges/s.

Cassandra cluster performance

  • When label index is turned on by default, 63k edges/s.

8.4 - HugeGraph 0.4.4 Benchmark

⚠️ Historical data — do not use it to evaluate current releases

This page records a HugeGraph 0.4.4 benchmark whose data was collected in 2018. The test environment, software versions, and implementation differ substantially from current releases, so these results are provided for historical reference only and must not be used to assess current performance.

A benchmark report for a current release will be added later, after which this page will be archived. Use the updated report for current performance conclusions.

1 测试环境

1.1 硬件信息

CPUMemory网卡磁盘
48 Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz128G10000Mbps750GB SSD

1.2 软件信息

1.2.1 测试用例

测试使用graphdb-benchmark,一个图数据库测试集。该测试集主要包含4类测试:

  • Massive Insertion,批量插入顶点和边,一定数量的顶点或边一次性提交

  • Single Insertion,单条插入,每个顶点或者每条边立即提交

  • Query,主要是图数据库的基本查询操作:

    • Find Neighbors,查询所有顶点的邻居
    • Find Adjacent Nodes,查询所有边的邻接顶点
    • Find Shortest Path,查询第一个顶点到100个随机顶点的最短路径
  • Clustering,基于Louvain Method的社区发现算法

1.2.2 测试数据集

测试使用人造数据和真实数据

本测试用到的数据集规模
名称vertex数目edge数目文件大小
email-enron.txt36,691367,6614MB
com-youtube.ungraph.txt1,157,8062,987,62438.7MB
amazon0601.txt403,3933,387,38847.9MB

1.3 服务配置

  • HugeGraph版本:0.4.4,RestServer和Gremlin Server和backends都在同一台服务器上
  • Cassandra版本:cassandra-3.10,commit-log 和data共用SSD
  • RocksDB版本:rocksdbjni-5.8.6
  • Titan版本:0.5.4, 使用thrift+Cassandra模式

graphdb-benchmark适配的Titan版本为0.5.4

2 测试结果

2.1 Batch插入性能

Backendemail-enron(30w)amazon0601(300w)com-youtube.ungraph(300w)
Titan9.51688.123111.586
RocksDB2.34514.07616.636
Cassandra11.930108.709101.959
Memory3.07715.20413.841

说明

  • 表头"()“中数据是数据规模,以边为单位
  • 表中数据是批量插入的时间,单位是s
  • 例如,HugeGraph使用RocksDB插入amazon0601数据集的300w条边,花费14.076s,速度约为21w edges/s
结论
  • RocksDB和Memory后端插入性能优于Cassandra
  • HugeGraph和Titan同样使用Cassandra作为后端的情况下,插入性能接近

2.2 遍历性能

2.2.1 术语说明
  • FN(Find Neighbor), 遍历所有vertex, 根据vertex查邻接edge, 通过edge和vertex查other vertex
  • FA(Find Adjacent), 遍历所有edge,根据edge获得source vertex和target vertex
2.2.2 FN性能
Backendemail-enron(3.6w)amazon0601(40w)com-youtube.ungraph(120w)
Titan7.72470.935128.884
RocksDB8.87665.85263.388
Cassandra13.125126.959102.580
Memory22.309207.411165.609

说明

  • 表头”()“中数据是数据规模,以顶点为单位
  • 表中数据是遍历顶点花费的时间,单位是s
  • 例如,HugeGraph使用RocksDB后端遍历amazon0601的所有顶点,并查找邻接边和另一顶点,总共耗时65.852s
2.2.3 FA性能
Backendemail-enron(30w)amazon0601(300w)com-youtube.ungraph(300w)
Titan7.11963.353115.633
RocksDB6.03264.52652.721
Cassandra9.410102.76694.197
Memory12.340195.444140.89

说明

  • 表头”()“中数据是数据规模,以边为单位
  • 表中数据是遍历边花费的时间,单位是s
  • 例如,HugeGraph使用RocksDB后端遍历amazon0601的所有边,并查询每条边的两个顶点,总共耗时64.526s
结论
  • HugeGraph RocksDB > Titan thrift+Cassandra > HugeGraph Cassandra > HugeGraph Memory

2.3 HugeGraph-图常用分析方法性能

术语说明
  • FS(Find Shortest Path), 寻找最短路径
  • K-neighbor,从起始vertex出发,通过K跳边能够到达的所有顶点, 包括1, 2, 3…(K-1), K跳边可达vertex
  • K-out, 从起始vertex出发,恰好经过K跳out边能够到达的顶点
FS性能
Backendemail-enron(30w)amazon0601(300w)com-youtube.ungraph(300w)
Titan11.3330.313376.06
RocksDB44.3912.221268.792
Cassandra39.8453.337331.113
Memory35.6382.059388.987

说明

  • 表头”()“中数据是数据规模,以边为单位
  • 表中数据是找到从第一个顶点出发到达随机选择的100个顶点的最短路径的时间,单位是s
  • 例如,HugeGraph使用RocksDB查找第一个顶点到100个随机顶点的最短路径,总共耗时2.059s
结论
  • 在数据规模小或者顶点关联关系少的场景下,Titan最短路径性能优于HugeGraph
  • 随着数据规模增大且顶点的关联度增高,HugeGraph最短路径性能优于Titan
K-neighbor性能
顶点深度一度二度三度四度五度六度
v1时间0.031s0.033s0.048s0.500s11.27sOOM
v111时间0.027s0.034s0.1151.36sOOM–
v1111时间0.039s0.027s0.052s0.511s10.96sOOM

说明

  • HugeGraph-Server的JVM内存设置为32GB,数据量过大时会出现OOM
K-out性能
顶点深度一度二度三度四度五度六度
v1时间0.054s0.057s0.109s0.526s3.77sOOM
度10133245350,8301,128,688
v111时间0.032s0.042s0.136s1.25s20.62sOOM
度1021149441131502,629,970
v1111时间0.039s0.045s0.053s1.10s2.92sOOM
度101402555508251,070,230

说明

  • HugeGraph-Server的JVM内存设置为32GB,数据量过大时会出现OOM
结论
  • FS场景,HugeGraph性能优于Titan
  • K-neighbor和K-out场景,HugeGraph能够实现在5度范围内秒级返回结果

2.4 图综合性能测试-CW

数据库规模1000规模5000规模10000规模20000
Titan45.943849.1682737.1179791.46
Memory(core)41.0771825.905**
Cassandra(core)39.783862.7442423.1366564.191
RocksDB(core)33.383199.894763.8691677.813

说明

  • “规模"以顶点为单位
  • 表中数据是社区发现完成需要的时间,单位是s,例如HugeGraph使用RocksDB后端在规模10000的数据集,社区聚合不再变化,需要耗时763.869s
  • “*“表示超过10000s未完成
  • CW测试是CRUD的综合评估
  • 后三者分别是HugeGraph的不同后端,该测试中HugeGraph跟Titan一样,没有通过client,直接对core操作
结论
  • HugeGraph在使用Cassandra后端时,性能略优于Titan,随着数据规模的增大,优势越来越明显,数据规模20000时,比Titan快30%
  • HugeGraph在使用RocksDB后端时,性能远高于Titan和HugeGraph的Cassandra后端,分别比两者快了6倍和4倍

9 - Contribution Guidelines

Read the contribution process before submitting code or documentation. Separate pages cover committer nominations, mailing-list subscriptions, and release validation. Contributor agreements follow the official ASF ICLA/CCLA process.

9.1 - Contribute to the HugeGraph Community

Choose How to Contribute

You can report problems through GitHub Issues, or contribute code, tests, or documentation. Before starting a substantial change, consider opening an issue that explains its scope to avoid duplicated work.

The following example uses apache/hugegraph. The same process applies to other HugeGraph repositories, but follow each repository’s README.md, AGENTS.md, and CI configuration for its build and test commands.

Prepare the Repository

Fork the HugeGraph repository on GitHub

Fork apache/hugegraph on GitHub, then clone your fork:

git clone https://github.com/<your-name>/hugegraph.git
cd hugegraph
git remote add upstream https://github.com/apache/hugegraph.git
git fetch upstream master

Do not develop directly on master. Use a separate branch for each change:

git switch master
git merge --ff-only upstream/master
git switch -c fix/<short-description>

Make and Verify Changes

HugeGraph Server code is under hugegraph-server/. For example, the core module is located at:

hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/

Current top-level Maven modules are listed below. Server’s hugegraph-server/hugegraph-hstore/ storage adapter is distinct from the standalone hugegraph-store/ storage-node project.

PathSubmodules
hugegraph-server/hugegraph-core/, hugegraph-api/, hugegraph-example/, hugegraph-dist/, hugegraph-test/, hugegraph-rocksdb/, hugegraph-hbase/, hugegraph-hstore/
hugegraph-pd/hg-pd-common/, hg-pd-client/, hg-pd-core/, hg-pd-service/, hg-pd-dist/, hg-pd-cli/, hg-pd-grpc/, hg-pd-test/
hugegraph-store/hg-store-common/, hg-store-client/, hg-store-core/, hg-store-node/, hg-store-dist/, hg-store-cli/, hg-store-grpc/, hg-store-rocksdb/, hg-store-test/
hugegraph-commons/hugegraph-common/, hugegraph-rpc/
hugegraph-cluster-test/hugegraph-clustertest-minicluster/, hugegraph-clustertest-dist/, hugegraph-clustertest-test/
Independent root moduleshugegraph-struct/, install-dist/

These names come from the root and subproject pom.xml files. Use the structure of the branch you are working on.

Run the tests directly related to your change first. Common Server test commands include:

# Core tests with the in-memory backend
mvn test -pl hugegraph-server/hugegraph-test -am -P core-test,memory

# API tests with the RocksDB backend
mvn test -pl hugegraph-server/hugegraph-test -am -P api-test,rocksdb

# Format files and verify compilation
mvn editorconfig:format
mvn clean compile -Dmaven.javadoc.skip=true

GitHub requires a username and token for Git authentication instead of a username and password. Create a personal access token at https://github.com/settings/tokens:

Authenticate Git pushes with a personal access token

When adding a third-party dependency, also update the license information included in the distribution:

  1. Add the dependency’s license file to hugegraph-server/hugegraph-dist/release-docs/licenses/.
  2. Update hugegraph-server/hugegraph-dist/release-docs/LICENSE. If the dependency includes a NOTICE file, update NOTICE as well.
  3. Run hugegraph-server/hugegraph-dist/scripts/dependency/regenerate_known_dependencies.sh to update the known-dependency list.

Submit a Pull Request

GitHub no longer accepts an account username and password for pushing code. If you need a personal access token, create one with the necessary permissions in token settings.

Make sure the email address used for your commits is associated with your GitHub account. Use email settings:

Verify your commit email on GitHub

Use the type(module): message format for commit messages, for example:

git add <changed-files>
git commit -m "fix(core): handle empty vertex query"
git push -u origin fix/<short-description>

Then open a pull request from your fork branch to apache/hugegraph:master. Explain the problem, the implementation, and the validation commands you actually ran. Include screenshots for UI changes.

Address Review Feedback

If CI fails or a reviewer requests changes, continue committing and pushing to the same branch. Rebase when you need to synchronize with upstream:

git fetch upstream master
git rebase upstream/master
git push --force-with-lease

Do not overwrite the remote branch with plain --force. After all CI and review requirements are satisfied, a project maintainer will merge the pull request.

Contributor agreements follow the official ASF process. See the Contributor Agreement.

9.2 - Subscribe to the Community Mailing List

HugeGraph uses dev@hugegraph.apache.org for development and usage discussions. Subscribe before sending messages to the list; messages from non-subscribers may be rejected.

Subscribe

  1. From the email address you want to subscribe, send a message with any subject and body to dev-subscribe@hugegraph.apache.org.
  2. Reply directly to the confirmation message.
  3. After receiving the subscription confirmation, you can send messages to dev@hugegraph.apache.org.

If the confirmation does not arrive, check your spam folder and automatic mail categories. If it is still missing, wait a while and send the subscription message again.

You can browse public messages in the ASF Mailing List Archives without subscribing.

Unsubscribe

  1. From your subscribed address, send a message to dev-unsubscribe@hugegraph.apache.org.
  2. Reply directly to the confirmation message from dev-help@hugegraph.apache.org.
  3. Unsubscription is complete when you receive a message whose subject contains GOODBYE.

For general ASF mailing-list guidance, see Apache Mailing Lists.

9.3 - Validate Apache Release

Note: this doc will be updated continuously. Use Java 11 for runtime verification. Since version 1.5.0, components other than the client no longer support Java 8.

Graduation note: Apache HugeGraph graduated in January 2026. Official release voting is now completed within the HugeGraph community (PMC binding votes on dev@hugegraph.apache.org), and no longer requires Incubator general@incubator.apache.org approval.

Verification

When the internal temporary release and packaging work is completed, other community developers ( especially PMC) need to participate in verification based on ASF release policy and checklist references:

  • ASF release policy
  • Incubator checklist (historical reference) To ensure the “correctness + completeness” of someone’s published version, here requires **everyone ** to participate as much as possible, and then explain which items you have checked in the subsequent email reply.(The following are the core items)

1. prepare

If there is no svn or gpg or wget environment locally, it is recommended to install it first (windows recommend using WSL2 environment, or at least git-bash), also make sure to install java (prefer Java 11) and maven software

# 1. install svn
# ubuntu/debian
sudo apt install subversion -y
# MacOS
brew install subversion
# To verify that the installation was successful, execute the following command:
svn --version

# 2. install gpg
# ubuntu/debian
sudo apt-get install gnupg -y
# MacOS
brew install gnupg
# To verify that the installation was successful, execute the following command:
gpg --version

# 3. install wget (we will enhance it later, like use `curl`)
# ubuntu/debian
sudo apt-get install wget -y
# MacOS
brew install wget

# 4. Download the hugegraph-svn directory 
# For version number, pay attention to fill in the verification version
svn co https://dist.apache.org/repos/dist/dev/hugegraph/1.x.x/
# (Note) If svn downloads a file very slowly, 
# you can consider wget to download a single file, as follows (or consider using a proxy)
wget https://dist.apache.org/repos/dist/dev/hugegraph/1.x.x/apache-hugegraph-toolchain-incubating-1.x.x.tar.gz

2. check hash value

First you need to check the file integrity of the source + binary package, Verify by shasum to ensure that it is consistent with the hash value published on apache/GitHub (Usually sha512), Here is the same as the last step of 0x02 inspection.

execute the following command:
for i in *.tar.gz; do echo $i; shasum -a 512 --check  $i.sha512; done

3. check gpg signature

This is to ensure that the published package is uploaded by a reliable person. Assuming tom signs and uploads, others should download A’s public key and then perform signature confirmation.

Related commands:

# 1. Download project trusted public key to local (required for the first time) & import
curl  https://downloads.apache.org/hugegraph/KEYS > KEYS
gpg --import KEYS

# After importing, you can see the following output, which means that x user public keys have been imported
gpg: /home/ubuntu/.gnupg/trustdb.gpg: trustdb created
gpg: key BA7E78F8A81A885E: public key "imbajin (apache mail) <jin@apache.org>" imported
gpg: key 818108E7924549CC: public key "vaughn <vaughn@apache.org>" imported
gpg: key 28DCAED849C4180E: public key "coderzc (CODE SIGNING KEY) <zhaocong@apache.org>" imported
...
gpg: Total number processed: x
gpg:               imported: x

# 2. Trust release users (trust n username mentioned in voting mail, if more than one user, 
#      just repeat the steps in turn or use the script below)
gpg --edit-key $USER # input the username, enter the interactive mode
gpg> trust
...output options..
Your decision? 5 # select 5
Do you really want to set this key to ultimate trust? (y/N) y # slect y, then q quits trusting the next user

# (Optional) You could also use the command to trust one user in non-interactive mode:
echo -e "5\ny\n" | gpg --batch --command-fd 0 --edit-key $USER trust
# Or trust all currently imported GPG public keys (review them carefully first):
for key in $(gpg --no-tty --list-keys --with-colons | awk -F: '/^pub/ {print $5}'); do
  echo -e "5\ny\n" | gpg --batch --command-fd 0 --edit-key "$key" trust
done


# 3. Check the signature (make sure there is no Warning output, every source/binary file prompts Good Signature)
#Single file verification
gpg --verify xx.asc xxx-src.tar.gz
gpg --verify xx.asc xxx.tar.gz # Note: without the bin/binary suffix

# One-click shell traversal verification (recommended)
for i in *.tar.gz; do echo $i; gpg --verify $i.asc $i ; done

First confirm the overall integrity/consistency, and then confirm the specific content (key)

4. Check the archive contents

Check the contents of the archive downloaded from preparation work. Divided into two aspects: source code package + binary package, The source code package is stricter, it can be said that the core part (Because it is longer, For a complete list refer to the official Wiki)

A. source package

After decompressing *hugegraph*src.tar.gz, Do the following checks:

  1. package/folder naming should match the release line (historical releases may still contain incubating), and no empty files/folders
  2. LICENSE + NOTICE exist and the content is normal; DISCLAIMER is required for historical incubating artifacts
  3. does not exist binaries (without LICENSE)
  4. The source code files all contain the standard ASF License header (this could be done with the Maven-MAT plugin)
  5. Check whether the pom.xml version number of each parent/child module is consistent (and meet expectations)
  6. Finally, make sure the source code works/compiles correctly
# prefer to use/switch to `java 11` for the following operations (compiling/running) (Note: `Computer` only supports `java >= 11`)
# java --version

# try to compile in the Unix env to check if it works well (-P is optional)
mvn clean package -DskipTests -Dcheckstyle.skip=true -P stage
B. binary package

After decompressing xxx-hugegraph.tar.gz, perform the following checks:

  1. package/folder naming should match the release line (historical releases may still contain incubating)
  2. LICENSE and NOTICE file exists and the content is normal (DISCLAIMER applies to historical incubating artifacts)
  3. start server
# hugegraph-server
bin/start-hugegraph.sh

# hugegraph-loader
bin/hugegraph-loader.sh -g hugegraph -f example/file/struct.json -s example/file/schema.groovy

# hugegraph-hubble
bin/start-hubble.sh

more reference official website: https://hugegraph.apache.org/docs/quickstart

Note: If a third-party dependency is introduced in the binary package, you need to update the LICENSE and add the third-party dependent LICENSE; if the third-party dependent LICENSE is Apache 2.0, and the corresponding project contains NOTICE, you also need to update Our NOTICE file

5. Check the official website and GitHub and other pages

  1. Make sure that the official website at least meets apache website check, and no circular links, etc.
  2. Update download link and release notes updated
  3. …

Mail Template

After the check & test, you should reply to the mail with the following content: (normal devs & PMC)

[] +1 approve

[] +0 no opinion

[] -1 disapprove with the reason
+1 (non-binding)
I checked:
1. Download link/tag in mail are valid
2. Checksum and GPG signatures are OK
3. LICENSE & NOTICE & DISCLAIMER are exist
4. Build successfully on XX OS & Version XX
5. No unexpected binary files
6. Date is right in the NOTICE file
7. Compile from source is fine under JavaXX
8. No empty file & directory found
9. Test running XXX service OK
10. ....

and the PMC members should reply with binding, it’s important for summary the valid votes:

+1 (binding)
I checked:
1. Download link/tag in mail are valid
2. Checksum and GPG signatures are OK
3. LICENSE & NOTICE & DISCLAIMER are exist
4. Build successfully on XX OS & Version XX
5. No unexpected binary files
6. Date is right in the NOTICE file
7. Compile from source is fine under JavaXX
8. No empty file & directory found
9. Test running XX process OK
10. ....

9.4 - Setup Server in IDEA (Dev)

Scope: These IDEA instructions are based on the HugeGraph 1.2.0 release tag and retained as historical debugging steps. For current source layout and verification commands, see the contribution process and Server quick start. Do not infer the current module list from this page.

Background

The Quick Start section provides instructions on how to start and stop HugeGraph-Server using scripts. In this guide, we will explain how to run and debug HugeGraph-Server on the Linux platform using IntelliJ IDEA.

The core steps for local startup are the same as starting with scripts:

  1. Initialize the database backend by executing the InitStore class to initialize the graph.
  2. Start HugeGraph-Server by executing the HugeGraphServer class to load the initialized graph information and start the server.

Before proceeding with the following process, make sure that you have cloned the source code of HugeGraph and have configured the development environment, such as Java 11 & you could config your local environment with this config-doc

git clone https://github.com/apache/hugegraph.git

Steps

1. Copy Configuration Files

To avoid the impact of configuration file changes on Git tracking, it is recommended to copy the required configuration files to a separate folder. Run the following command to copy the files:

cp -r hugegraph-server/hugegraph-dist/src/assembly/static/scripts \
      hugegraph-server/hugegraph-dist/src/assembly/static/conf \
      path-to-your-directory

Replace path-to-your-directory with the path to the directory where you want to copy the files. Run the command from the repository root, the hugegraph-dist module lives under the top-level hugegraph-server directory.

The following ToplingDB steps apply only to historical custom builds. Current master contains neither preload-topling.sh nor the ToplingDB module; skip this note when debugging current master.

2. Configure InitStore to initialize the graph

First, you need to configure the database backend in the configuration files. In this example, we will use RocksDB. Open path-to-your-directory/conf/graphs/hugegraph.properties and configure it as follows:

backend=rocksdb
serializer=binary
rocksdb.data_path=.
rocksdb.wal_path=.

Next, open the Run/Debug Configurations panel in IntelliJ IDEA and create a new Application configuration. Follow these steps for the configuration:

  • Select hugegraph-dist as the Use classpath of module.
  • Set the Main class to org.apache.hugegraph.cmd.InitStore.
  • Set the program arguments to conf/rest-server.properties. Note that the path here is relative to the working directory, so make sure to set the working directory to path-to-your-directory.
  • (Optional, ToplingDB builds only) ToplingDB requires preloading dynamic libraries via the LD_PRELOAD mechanism. Developers need to set two environment variables: LD_LIBRARY_PATH should point to the library directory extracted by preload-topling.sh, and LD_PRELOAD should be set to libjemalloc.so:librocksdbjni-linux64.so to ensure the necessary libraries are correctly loaded at runtime.
    • LD_LIBRARY_PATH=/path/to/your/library:$LD_LIBRARY_PATH
    • LD_PRELOAD=libjemalloc.so:librocksdbjni-linux64.so

If user authentication (authenticator) is configured for HugeGraph-Server in the Java 11 environment, you need to refer to the script configuration in the binary package and add the following VM options:

--add-exports=java.base/jdk.internal.reflect=ALL-UNNAMED

Otherwise, an error will occur:

java.lang.reflect.InaccessibleObjectException: Unable to make public static synchronized void jdk.internal.reflect.Reflection.registerFieldsToFilter(java.lang.Class,java.lang.String[]) accessible: module java.base does not "exports jdk.internal.reflect" to unnamed module @xxx

Once the configuration is completed, run it. If the execution is successful, the following runtime logs will be displayed:

2023-06-05 00:43:37 [main] [INFO] o.a.h.u.ConfigUtil - Scanning option 'graphs' directory './conf/graphs'
2023-06-05 00:43:37 [main] [INFO] o.a.h.c.InitStore - Init graph with config file: ./conf/graphs/hugegraph.properties
......
2023-06-05 00:43:39 [main] [INFO] o.a.h.b.s.r.RocksDBStore - Write down the backend version: 1.11
2023-06-05 00:43:39 [main] [INFO] o.a.h.StandardHugeGraph - Graph 'hugegraph' has been initialized
2023-06-05 00:43:39 [main] [INFO] o.a.h.StandardHugeGraph - Close graph standardhugegraph[hugegraph]
2023-06-05 00:43:39 [db-open-1] [INFO] o.a.h.b.s.r.RocksDBStore - Opening RocksDB with data path: ./m
2023-06-05 00:43:39 [db-open-1] [INFO] o.a.h.b.s.r.RocksDBStore - Opening RocksDB with data path: ./s
2023-06-05 00:43:39 [db-open-1] [INFO] o.a.h.b.s.r.RocksDBStore - Opening RocksDB with data path: ./g
2023-06-05 00:43:39 [main] [INFO] o.a.h.HugeFactory - HugeFactory shutdown
2023-06-05 00:43:39 [hugegraph-shutdown] [INFO] o.a.h.HugeFactory - HugeGraph is shutting down

3. Running HugeGraphServer

Similarly, open the Run/Debug Configurations panel in IntelliJ IDEA and create a new Application configuration. Follow these steps for the configuration:

  • Select hugegraph-dist as the Use classpath of module.
  • Set the Main class to org.apache.hugegraph.dist.HugeGraphServer.
  • Set the program arguments to conf/gremlin-server.yaml conf/rest-server.properties. Similarly, note that the path here is relative to the working directory, so make sure to set the working directory to path-to-your-directory.

bin/hugegraph-server.sh in the binary package does not start this class directly. It starts org.apache.hugegraph.bootstrap.HugeGraphServerBootstrap, which takes a leading true/false security-check flag before the two configuration paths, installs HugeSecurityManager when that flag is true, and then hands over to HugeGraphServer. Running HugeGraphServer from IDEA skips that wrapper, so the security manager is not installed, which is normally what you want while debugging.

Similarly, if user authentication (authenticator) is configured for HugeGraph-Server in the Java 11 environment, you need to refer to the script configuration in the binary package and add the following VM options:

--add-exports=java.base/jdk.internal.reflect=ALL-UNNAMED --add-modules=jdk.unsupported --add-exports=java.base/sun.nio.ch=ALL-UNNAMED

Otherwise, an error will occur:

java.lang.reflect.InaccessibleObjectException: Unable to make public static synchronized void jdk.internal.reflect.Reflection.registerFieldsToFilter(java.lang.Class,java.lang.String[]) accessible: module java.base does not "exports jdk.internal.reflect" to unnamed module @xxx

Once the configuration is completed, run it. If you see the following logs, it means that HugeGraphServer has been successfully started:

......
2023-06-05 00:51:56 [gremlin-server-boss-1] [INFO] o.a.t.g.s.GremlinServer - Gremlin Server configured with worker thread pool of 1, gremlin pool of 8 and boss thread pool of 1.
2023-06-05 00:51:56 [gremlin-server-boss-1] [INFO] o.a.t.g.s.GremlinServer - Channel started at port 8182.

4. Debugging HugeGraphServer (optional)

After completing the above configuration, you can try debugging HugeGraphServer. Run HugeGraphServer in debug mode and set a breakpoint at the following location:

public String list(@Context GraphManager manager,
                   @PathParam("graph") String graph, @QueryParam("label") String label,
                   @QueryParam("properties") String properties, ......) {
    // ignore log
    Map<String, Object> props = parseProperties(properties);

Then use the RESTful API to request HugeGraphServer:

curl "http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices" | gunzip

At this point, you can view detailed variable information in the debugger.

5. Log4j2 Configuration

By default, when running InitStore and HugeGraphServer, the Log4j2 configuration file path read is hugegraph-server/hugegraph-dist/src/main/resources/log4j2.xml, not path-to-your-directory/conf/log4j2.xml. This configuration file is read when starting HugeGraph-Server using the script.

To avoid maintaining two separate configuration files, you can modify the Log4j2 configuration file path when running and debugging HugeGraph-Server in IntelliJ IDEA:

  1. Open the previously created Application configuration.
  2. Click on Modify options - Add VM options.
  3. Set the VM options to -Dlog4j.configurationFile=conf/log4j2.xml.

Possible Issues

1. java: package sun.misc does not exist

The reason may be that cross-compilation is triggered when using Java 11 to compile, causing the symbol of sun.misc.Unsafe used in the project to not be found. There are two possible solutions:

  1. In IntelliJ IDEA, go to Preferences/Settings and find the Java Compiler panel. Then, disable the --release option (recommended).
  2. Set the Project SDK to 8 (Deprecated soon).

2. java: *.store.raft.rpc.RaftRequests does not exist (RPC Generated Files)

The reason is that the source code didn’t include the RPC-generated files. You could try 2 ways to fix it:

  1. [CMD]mvn clean compile in the root directory (Recommend)
  2. [UI] right click on the hugegraph repo and select Maven->Generate Sources and Update Folders. This will rebuild the repo and correctly generate the required files.

3. Unable to Print Location Information (%l) in Log4j2

This is because Log4j2 uses asynchronous loggers. You can refer to the official documentation for configuration details.


References

  1. HugeGraph-Server Quick Start: Instructions for setting up HugeGraph-Server.
  2. Local Debugging Guide for HugeGraph Server (Win/Unix)
  3. “package sun.misc does not exist” compilation error
  4. Cannot compile: java: package sun.misc does not exist
  5. The code-style config for HugeGraph in IDEA

9.5 - Apache HugeGraph Committer Guide

This document outlines the requirements and process for becoming an Apache Committer. The corresponding ASF official document can be found at: https://community.apache.org/newcommitter.html

Candidate Requirements

  1. Candidates must adhere to the Apache Code of Conduct.
  2. PMC members will assess candidates’ interactions with others and contributions through mailing lists, issues, pull requests, and official documentation.
  3. Considerations for evaluating candidates as potential Committers include:
    1. Ability to collaborate with community members
    2. Mentorship capabilities
    3. Community involvement
    4. Level of contribution
    5. Personal skills/abilities

Nomination Process

Discussion → Vote → Invitation → Announcement

Initiate Community Discussion (DISCUSS)

Any PMC member of HugeGraph can initiate a voting discussion. After identifying valuable contributions from a community contributor and obtaining the candidate’s consent, a discussion can be initiated via private@hugegraph.apache.org. The initiator of the discussion should clearly state the candidate’s contributions in the discussion email and provide URLs or other information for confirming the contributions, facilitating discussion and analysis.

Below is a template for HugeGraph emails: (For reference only)

Note: The term xxx will be used to refer to the candidate. Typically, xxx represents an easily readable name (e.g., Simon Jay).

ASF-INFRA recommends avoiding the use of less readable ID directly as a reference to the person in emails (e.g., avoid simon321 or wh0isSim0n 😄).

In addition, it is best to choose the “pure text” mode, otherwise the typesetting may be chaotic in the ASF Mailing-list UI

To: private@hugegraph.apache.org
Subject: [DISCUSS] XXX as a HugeGraph Committer Candidate

Hi all:

I am pleased to nominate xxx for the role of HugeGraph Committer based on his/her contributions over the past few months.

[ Candidate's Contribution Summary ]

Here are the relevant PRs (issues) he/she has participated in:

**Core Features:**
- Feature 1: [ Reference Links ]
- ...

**Fix/Chore/Release:**

**Doc:**

[ Candidate's Current Notable Contributions ]

His/Her contributions bring the following benefits to the community, helping us in the following ways:

[ Candidate's Contributions and Benefits to the Community ]

In view of the above contributions, I elect xxx as Committer of the HugeGraph project.

[ Reference Links ]
1. PR1
2. PR2
3. ...

Welcome everyone to share opinions~

Thanks!

For contribution links in discussion emails, you can use the statistical feature of GitHub Search by entering corresponding keywords as needed. You can also adjust parameters and add new repositories such as repo:apache/hugegraph-computer. Pay special attention to adjusting the time range (below is a template reference, please adjust the parameters accordingly):

For participation in mailing lists, you can use https://lists.apache.org/list?dev@hugegraph.apache.org:lte=10M:xxx.

Initiate Community Voting Email (VOTE)

If there are no dissenting opinions within the specified time frame of the discussion email, the initiator of the discussion needs to initiate a voting email for the committer election at private@hugegraph.apache.org.

Below is the corresponding email template:

To: private@hugegraph.apache.org
Subject: [VOTE] xxx as a HugeGraph Committer

Hi all:

Through the discussion of last week:
[ Discussion Mailing List Link ]

We have discussed and listed what xxx participated in the HugeGraph community.
I believe making him/her a Committer will enhance the work for HugeGraph. 

So, I am happy to call VOTE to accept xxx as a HugeGraph Committer.
 
Voting will continue for at least 72 hours or until the required number of votes is reached.
 
Please vote accordingly:
[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove with the reason  

Thanks!

Then, PMC members reply to the email with +1 or -1 to express their opinions. Generally, at least 3 votes of +1 are needed to conclude the vote.

Announcement of Voting Results (RESULT)

After the voting email concludes, the initiator of the vote needs to remind the end of the voting in the email. Additionally, the initiator needs to announce the voting results via email to private@hugegraph.apache.org. The email template can be as follows:

To: private@hugegraph.apache.org
Subject: [RESULTS][VOTE] xxx as a HugeGraph Committer

Hi all: The vote for "xxx" as a HugeGraph Committer has PASSED and closed now.

The result is as follows: X PMC +1 Votes: 
- A (PMC ID)
- B
- C...

Vote thread:
put vote thread link here
 
Then I'm going to invite xxx to join us soon. Thanks for everyone's support!

Send Invitation Email to Candidate (INVITE)

After the announcement of the voting results email is sent, the initiator of the vote should send an invitation email to the candidate. The invitation email is addressed to the candidate and cc’d to private@hugegraph.apache.org. The invited candidate must reply to the specified email address to accept or reject the invitation.

Below is a template for reference:

To: [ Candidate's Email ]
Cc: private@hugegraph.apache.org
Subject: Invitation to become HugeGraph committer: xxx

Hello xxx,

The HugeGraph Project Management Committee (PMC)
hereby offers you committer privileges to the project.
These privileges are offered on the understanding that you'll use them
reasonably and with common sense. We like to work on trust
rather than unnecessary constraints.

Being a committer enables you to more easily make
changes without needing to go through the patch
submission process.

Being a committer does not require you to
participate any more than you already do. It does
tend to make one even more committed.  You will
probably find that you spend more time here.

Of course, you can decline and instead remain as a
contributor, participating as you do now.

A. This personal invitation is a chance for you to
accept or decline in private.  Either way, please
let us know in reply to the private@hugegraph.apache.org
address only.

B. If you accept, the next step is to register an iCLA:
    1. Details of the iCLA and the forms are found
    through this link: https://www.apache.org/licenses/#clas

    2. Instructions for its completion and return to
    the Secretary of the ASF are found at
    https://www.apache.org/licenses/#submitting

    3. When you transmit the completed iCLA, request
    to notify the Apache HugeGraph project and choose a
    unique Apache ID. Look to see if your preferred
    ID is already taken at
    https://people.apache.org/committer-index.html
    This will allow the Secretary to notify the PMC
    when your iCLA has been recorded.

When recording of your iCLA is noted, you will
receive a follow-up message with the next steps for
establishing you as a committer.

With the expectation of your acceptance, welcome!

The Apache HugeGraph PMC

Candidate Accepts Invitation (ACCEPT)

The candidate should reply to the aforementioned email (select reply all) to indicate acceptance of the invitation. Below is a template for the email:

To: [ Sender's Email ]
Cc: private@hugegraph.apache.org
Subject: Re: Invitation to become HugeGraph committer: xxx

Hello Apache HugeGraph PMC,

I accept the invitation.

Thanks to the Apache HugeGraph Community for recognizing my work, I
will continue to actively participate in the work of the Apache
HugeGraph.

Next, I will follow the instructions to complete the next steps:
Signing and submitting iCLA and registering Apache ID.

xxx

Of course, the candidate may also choose to decline the invitation, in which case there is no template:)

Once the invitation is accepted, the candidate needs to complete the following tasks:

ICLA Signing Process

  1. Download the ICLA
  2. Open the PDF and fill in the required information. All fields must be filled in English. It is recommended to use a PDF tool to edit and sign.
    1. Full name: First name followed by last name
    2. Public name: Optional, defaults to the same as Full name
    3. Check the box only if you entered names with your family name first
    4. Postal Address: English address, starting from small to large, including detailed street address
    5. Country: Country of residence in English
    6. E-mail: Email address, preferably the same as the one used in the invitation email
    7. (optional) preferred Apache id(s): Choose an SVN ID that is not listed on the Apache committer page
    8. (optional) notify project: Apache HugeGraph
    9. Signature: Must be handwritten using a PDF tool
    10. Date: Format as xxxx-xx-xx
  3. After signing, rename icla.pdf to name-pinyin-icla.pdf
  4. Send the following email and attach name-pinyin-icla.pdf as a reference.
To: secretary@apache.org
Subject: ICLA Information

Hello everyone:

I have accepted the Apache HugeGraph PMC invitation to
become a HugeGraph committer, the attachment is my ICLA information.

(Optional) My GitHub account is https://github.com/xxx. Thanks!

xxx

For more details, please refer to https://github.com/apache/hugegraph/issues/1732.

PMC members will await confirmation of the ICLA record from the Apache secretary team. Candidates and PMC members will receive the following email:

Dear xxx,

This message acknowledges receipt of your ICLA, which has been filed in the Apache Software Foundation records.

Your account (with id xxx) has been requested for you and you should receive email with next steps
within the next few days (this process can take up to a week).

Please refer to https://www.apache.org/foundation/how-it-works.html#developers
for more information about roles at Apache.

Setting Up Apache Account and Development Environment (CONFIG)

After the record is completed, the candidate will receive an email from root@apache.org with the subject Welcome to the Apache Software Foundation. At this point, the candidate needs to follow the steps in the email to set up the Apache account and development environment:

  1. Reset the password at https://id.apache.org/reset/enter.
  2. Configure personal information at https://whimsy.apache.org/roster/committer/xxx.
  3. Associate GitHub account at https://gitbox.apache.org/boxer.
    • This step requires configuring GitHub Two-Factor Authentication (2FA).
  4. The nominating PMC member must add the new Committer to the official list of committers via the Roster page. (Important, otherwise repository permissions will not take effect).
    • After this step, the candidate becomes a new Committer and gains write access to the GitHub HugeGraph repository.
  5. (Optional) The new Committer can apply for free use of JetBrains’ full range of products with their Apache account here.

Announcing via Email (ANNOUNCE)

After the candidate completes the above steps, they will officially become a Committer of HugeGraph. At this point, they need to send an announcement email to dev@hugegraph.apache.org. Below is a template for the email:

To: dev@hugegraph.apache.org
Subject: [ANNOUNCE] New Committer: xxx

Hi everyone, The PMC for Apache HugeGraph has invited xxx to
become a Committer and we are pleased to announce that he/she has accepted.

xxx is being active in the HugeGraph community & dedicated to ... modules, 
and we are glad to see his/her more interactions with the community in the future.

(Optional) His/Her GitHub account is https://github.com/xxx

Welcome xxx, and please enjoy your community journey~ 

Thanks! 

The Apache HugeGraph PMC

Update Governance Information

Since Apache HugeGraph graduated in January 2026, governance information is maintained in ASF committee/project data rather than Incubator clutch pages.

Please check:

If an update is required but does not appear automatically, coordinate with Apache Community Development or ASF Infra according to the official process.

References

  1. https://community.apache.org/newcommitter.html (ASF official documentation)
  2. https://infra.apache.org/new-committers-guide.html
  3. https://www.apache.org/dev/pmc.html#newcommitter
  4. https://linkis.apache.org/zh-CN/community/how-to-vote-a-committer-pmc
  5. https://www.apache.org/licenses/contributor-agreements.html#submitting
  6. https://www.apache.org/licenses/cla-faq.html#printer
  7. https://linkis.apache.org/zh-CN/community/how-to-sign-apache-icla
  8. https://github.com/apache/hugegraph/issues/1732 (HugeGraph ICLA related issue)

10 - CHANGELOGS

10.1 - HugeGraph 1.7.0 Release Notes

WIP: This doc is under construction, please wait for the final version (BETA)

Operating Environment / Version Description

For 1.7.0 version hugegraph, related components only support Java11.

hugegraph

API Changes

  • BREAKING CHANGE: Disable legacy backends include MySQL/PG/c*(.etc) #2746
  • BREAKING CHANGE: Release version 1.7.0 [server + pd + store] #2889

Feature Changes

  • Support MemoryManagement for graph query framework #2649
  • LoginAPI support token_expire field #2754
  • Add option for task role election #2843
  • Optimize perf by avoid boxing long #2861
  • StringId hold bytes to avoid decode/encode #2862
  • Add PerfExample5 and PerfExample6 #2860
  • RocksDBStore remove redundant checkOpened() call #2863
  • Add path filter #2898
  • Init serena memory system & add memories #2902

Bug Fixes

  • Filter dynamice path(PUT/GET/DELETE) with params cause OOM #2569
  • JRaft Histogram Metrics Value NaN #2631
  • Update server image desc #2702
  • Kneigbor-api has unmatched edge type with server #2699
  • Add license for swagger-ui & reset use stage to false in ci yml #2706
  • Fix build pd-store arm image #2744
  • Fix graph server cache notifier mechanism #2729
  • Tx leak when stopping the graph server #2791
  • Ensure backend is initialized in gremlin script #2824
  • Fix some potential lock & type cast issues #2895
  • Fix npe in getVersion #2897
  • Fix the support for graphsapi in rocksdb and add testing for graphsapi #2900
  • Remove graph path in auth api path #2899
  • Migrate to LTS jdk11 in all Dockerfile #2901
  • Remove the judgment for java8 compatibility in the init-store #2905
  • Add missing license and remove binary license.txt & fix tinkerpop ci & remove duplicate module #2910

Option Changes

  • Remove some outdated configuration #2678

Other Changes

  • Update outdated docs for release 1.5.0 #2690
  • Fix licenses and remove empty files #2692
  • Update repo artifacts references #2695
  • Adjust release fury version #2698
  • Fix the JSON license issue #2697
  • Add debug info for tp test #2688
  • Enhance words in README #2734
  • Add collaborators in asf config #2741
  • Adjust the related filters of sofa-bolt #2735
  • Reopen discussion in .asf.yml config #2751
  • Fix typo in README #2806
  • Centralize version management in project #2797
  • Update notice year #2826
  • Improve maven Reproducible Builds → upgrade plugins #2874
  • Enhance docker instruction with auth opened graph #2881
  • Remove the package existing in java8 #2792
  • Revise Docker usage instructions in README #2882
  • Add DeepWiki badge to README #2883
  • Update guidance for store module #2894
  • Update test commands and improve documentation clarity #2893
  • Bump rocksdb version from 7.2.2 to 8.10.2 #2896

hugegraph-toolchain

API Changes

  • Support graphspace #633

Feature Changes

  • Support jdbc date type & sync .editorconfig #648
  • Add a useSSL option for mysql #650
  • Patch for father sub edge #654
  • Improve user experience for user script #666
  • Support concurrent readers, short-id & Graphsrc #683
  • Init serena onboarding & project memory files #692

Bug Fixes

  • Typo word in display #655
  • Patch up missing classes and methods for hubble #657
  • Adjust Client to 1.7.0 server #689
  • Remove json license for release 1.7.0 #698

Other Changes

  • Update hugegraph source commit id #640
  • Add collaborators in asf config #656
  • Update pom for version-1.7.0 #681
  • Add DeepWiki badge to README #684
  • Adjust APIs to compatible with 1.7.0 server #685
  • Adjust LoadContext to 1.7.0 version #687
  • Migrate to LTS jdk11 in all Dockerfile #691
  • Update copyright year in NOTICE file #697

hugegraph-computer

Feature Changes

  • Migration Vermeer to hugegraph-computer #316
  • Make startChan’s size configurable #328
  • Assign WorkerGroup via worker configuration #332
  • Support task priority based scheduling #336
  • Avoid 800k #340

Bug Fixes

  • Fix docker file build #341

Other Changes

  • Update release version to 1.5.0 #318
  • Update go depends module & fix headers #321
  • Update go version to 1.23 #322
  • Add collaborator in .asf.yaml #323
  • Update the Go version in docker image #333
  • Add DeepWiki badge to README #337
  • Bump project version to 1.7.0 (RELEASE) #338
  • Update copyright year in NOTICE file #342

hugegraph-ai

API Changes

  • Support choose template in api #135
  • Add post method for paths-api #162
  • Support switch graph in api & add some query configs #184
  • Text2gremlin api #258
  • Support switching prompt EN/CN #269
  • BREAKING CHANGE: Update keyword extraction method #282

Feature Changes

  • Added the process of text2gql in graphrag V1.0 #105
  • Use pydantic-settings for config management #122
  • Timely execute vid embedding & enhance some HTTP logic #141
  • Use retry from tenacity #143
  • Modify the summary info and enhance the request logic #147
  • Automatic backup graph data timely #151
  • Add a button to backup data & count together #153
  • Extract topk_per_keyword & topk_return_results to .env #154
  • Modify clear buttons #156
  • Support intent recognition V1 #159
  • Change vid embedding x:yy to yy & use multi-thread #158
  • Support mathjax in rag query block V1 #157
  • Use poetry to manage the dependencies #149
  • Return schema.groovy first when backup graph data #161
  • Merge all logs into one file #171
  • Use uv for the CI action #175
  • Use EN prompt for keywords extraction #174
  • Support litellm LLM provider #178
  • Improve graph extraction default prompt #187
  • Replace vid by full vertexes info #189
  • Support asynchronous streaming generation in rag block by using async_generator and asyncio.wait #190
  • Generalize the regex extraction func #194
  • Create quick_start.md #196
  • Support Docker & K8s deployment way #195
  • Multi-stage building in Dockerfile #199
  • Support graph checking before updating vid embedding #205
  • Disable text2gql by default #216
  • Use 4.1-mini and 0.01 temperature by default #214
  • Enhance the multi configs for LLM #212
  • Textbox to Code #217
  • Replace the IP + Port with URL #209
  • Update gradio’s version #235
  • Use asyncio to get embeddings #215
  • Change QPS -> RPM for timer decorator #241
  • Support batch embedding #238
  • Using nuitka to provide a binary/perf way for the service #242
  • Use uv instead poetry #226
  • Basic compatible in text2gremlin generation #261
  • Enhance config path handling and add project root validation #262
  • Add vermeer python client for graph computing #263
  • Use uv in client & ml modules & adapter the CI #257
  • Use uv to manage pkgs & update README #272
  • Limit the deps version to handle critical init problems #279
  • Support semi-automated prompt generation #281
  • Support semi-automated generated graph schema #274
  • Unify all modules with uv #287
  • Add GitHub Actions for auto upstream sync and update SEALData subsample logic #289
  • Add a basic LLM/AI coding instruction file #290
  • Add rules for AI coding guideline - V1.0 #293
  • Replace QianFan by OpenAI-compatible format #285
  • Optimize vector index with asyncio embedding #264
  • Refactor embedding parallelization to preserve order #295
  • Support storing vector data for a graph instance by model type/name #265
  • Add AGENTS.md as new document standard #299
  • Add Fixed Workflow Execution Engine: Flow, Node, and Scheduler Architecture #302
  • Support vector db layer V1.0 #304

Bug Fixes

  • Limit the length of log & improve the format #121
  • Pylint in ml #125
  • Critical bug with pylint usage #131
  • Multi vid k-neighbor query only return the data of first vid #132
  • Replace getenv usage to settings #133
  • Correct header writing errors #140
  • Update prompt to fit prefix cache #137
  • Extract_graph_data use wrong method #145
  • Use empty str for llm config #155
  • Update gremlin generate prompt to apply fuzzy match #163
  • Enable fastapi auto reload function #164
  • Fix tiny bugs & optimize reranker layout #202
  • Enable tasks concurrency configs in Gradio #188
  • Align regex extraction of json to json format of prompt #211
  • Fix documentation sample code error #219
  • Failed to remove vectors when updating vid embedding #243
  • Skip empty chunk in LLM steaming mode #245
  • Ollama batch embedding bug #250
  • Fix Dockerfile to add pyproject.toml anchor file #266
  • Add missing ‘properties’ in gremlin prompt formatting #298
  • Fixed cgraph version #305
  • Ollama embedding API usage and config param #306

Option Changes

  • Remove enable_gql logic in api & rag block #148

Other Changes

  • Update README for python-client/SDK #150
  • Enable pip cache #142
  • Enable discussion & change merge way #201
  • Synchronization with official documentation #273
  • Fix grammar errors #275
  • Improve README clarity and deployment instructions #276
  • Add docker-compose deployment and improve container networking instructions #280
  • Update docker compose command #283
  • Reduce third-party library log output #244
  • Update README with improved setup instructions #294
  • Add collaborators in asf config #182

Release Details

Please check the release details/contributor in each repository:

10.2 - HugeGraph 1.5.0 Release Notes

WIP: This doc is under construction, please wait for the final version (BETA)

Operating Environment / Version Description

  1. From hugegraph version 1.5.0 and later, related components only support Java11.

PS: In the future, HugeGraph components will evolve through versions of Java 11 -> Java 17 -> Java 21.

hugegraph

This version introduces many new features and optimizations, particularly support for the new distributed backend HStore(Raft + RocksDB).

API Changes

  • BREAKING CHANGE: Support “parent & child” EdgeLabel type #2662

Feature Changes

  • Integrate pd-grpc, pd-common, and pd-client #2498
  • Integrate store-grpc, store-common, and store-client #2476
  • Integrate store-rocksdb submodule #2513
  • Integrate pd-core into HugeGraph #2478
  • Integrate pd-service into HugeGraph #2528
  • Integrate pd-dist into HugeGraph and add core tests, client tests, and REST tests for PD #2532
  • Integrate server-hstore into HugeGraph #2534
  • Integrate store-core submodule #2548
  • Integrate store-node submodule #2537
  • Support new backend Hstore #2560
  • Support Docker deployment for PD and Store #2573
  • Add a tool method encode #2647
  • Add basic MiniCluster module for distributed system testing #2615
  • Support disabling RocksDB auto-compaction via configuration #2586

Bug Fixes

  • Switch RocksDB backend to memory when executing Gremlin examples #2518
  • Avoid overriding backend config in Gremlin example scripts #2519
  • Update resource references #2522
  • Randomly generate default values #2568
  • Update build artifact path for Docker deployment #2590
  • Ensure thread safety for range attributes in PD #2641
  • Correct server Docker copy source path #2637
  • Fix JRaft Timer Metrics bug in Hstore #2602
  • Enable JRaft MaxBodySize configuration #2633

Option Changes

  • Mark old raft configs as deprecated #2661
  • Enlarge bytes write limit and remove big parameter when encoding/decoding string ID length #2622

Other Changes

  • Add Swagger-UI LICENSE files #2495
  • Translate CJK comments and punctuations to English across multiple modules #2536, #2623, #2645
  • Introduce install-dist module in root #2552
  • Enable up-to-date checks for UI (CI) #2609
  • Minor improvements for POM properties #2574
  • Migrate HugeGraph Commons #2628
  • Tar source and binary packages for HugeGraph with PD-Store #2594
  • Refactor: Enhance cache invalidation of the partition → leader shard in ClientCache #2588
  • Refactor: Remove redundant properties in LogMeta and PartitionMeta #2598

hugegraph-toolchain

API Changes

  • Support “parent & child” EdgeLabel type #624

Feature Changes

  • Support English interface & add a script/doc for it in Hubble #631

Bug Fixes

  • Serialize source and target label for non-father EdgeLabel #628
  • Encode/decode Chinese error after building Hubble package #627
  • Configure IPv4 to fix timeout of yarn install in Hubble #636
  • Remove debugging output to speed up the frontend construction in Hubble #638

Other Changes

  • Bump express from 4.18.2 to 4.19.2 in Hubble Frontend #598
  • Make IDEA support IssueNavigationLink #600
  • Update yarn.lock for Hubble #605
  • Introduce editorconfig-maven-plugin for verifying code style defined in .editorconfig #614
  • Upgrade distribution version to 1.5.0 #639

Documentation Changes

  • Clarify the contributing guidelines #604
  • Enhance the README file for Hubble #613
  • Update README style referring to the server’s style #615

hugegraph-ai

API Changes

  • Added local LLM API and version API. #41, #44
  • Implemented new API and optimized code structure. #63
  • Support for graphspace and refactored all APIs. #67

Feature Changes

  • Added openai’s apibase configuration and asynchronous methods in RAG web demo. #41, #58
  • Support for multi reranker and enhanced UI. #73
  • Node embedding, node classify, and graph classify with models based on DGL. #83
  • Graph learning algorithm implementation (10+). #102
  • Support for any openai-style API (standard). #95

Bug Fixes

  • Fixed fusiform_similarity test in traverser for server 1.3.0. #37
  • Avoid generating config twice and corrected e_cache type. #56, #117
  • Fixed null value detection on vid attributes. #115
  • Handled profile regenerate error. #98

Option Changes

  • Added auth for fastapi and gradio. #70
  • Support for multiple property types and importing graph from the entire doc. #84

Other Changes

  • Reformatted documentation and updated README. #36, #81
  • Introduced a black for code format in GitHub actions. #47
  • Updated dependencies and environment preparations. #45, #65
  • Enhanced user-friendly README. #82

hugegraph-computer

Feature Changes

  • Support Single Source Shortest Path Algorithm #285
  • Support Output Filter #303

Bug Fixes

  • Fix: base-ref/head-ref Missed in Dependency-Review on Schedule Push #304

Option Changes

  • Refactor(core): StringEncoding #300

Other Changes

  • Improve(algorithm): Random Walk Vertex Inactive #301
  • Upgrade Version to 1.3.0 #305
  • Doc(readme): Clarify the Contributing Guidelines #306
  • Doc(readme): Add Hyperlink to Apache 2.0 #308
  • Migrate Project to Computer Directory #310
  • Update for Release 1.5 #317
  • Fix Path When Exporting Source Package #319

Release Details

Please check the release details/contributor in each repository:

10.3 - HugeGraph 1.3.0 Release Notes

Operating Environment / Version Description

  1. consider using Java 11 in hugegraph/toolchain/commons, also compatible with Java 8 now.
  2. hugegraph-computer required to use Java 11, not compatible with Java 8!
  3. Using Java8 may loss some security ensured, we recommend using Java 11 in production env with AuthSystem enabled.

1.3.0 is the last major version compatible with Java 8, compatibility with Java 8 will end in next release(1.5.0) when PD/Store merged into master branch (Except for the java-client).

PS: In the future, we will gradually upgrade the java version from Java 11 -> Java 17 -> Java 21.

hugegraph

In this version, we have fixed some SEC-related issues. If used in an online service or exposed to the public, please upgrade to the latest version and enable authorization authentication

API Changes

  • feat(api): optimize adjacent-edges query (#2408)

Feature Changes

  • feat: support docker use the auth when starting (#2403)
  • feat: added the OpenTelemetry trace support (#2477)

Bug Fix

  • fix(core): task restore interrupt problem on restart server (#2401)
  • fix(server): reinitialize the progress to set up graph auth friendly (#2411)
  • fix(chore): remove zgc in dockerfile for ARM env (#2421)
  • fix(server): make CacheManager constructor private to satisfy the singleton pattern (#2432)
  • fix(server): unify the license headers (#2438)
  • fix: format and clean code in dist and example modules (#2441)
  • fix: format and clean code in core module (#2440)
  • fix: format and clean code in modules (#2439)
  • fix(server): clean up the code (#2456)
  • fix(server): remove extra blank lines (#2459)
  • fix(server): add tip for gremlin api NPE with an empty query (#2467)
  • fix(server): fix the metric name when promthus collects hugegraph metric, see issue (#2462)
  • fix(server): serverStarted error when execute gremlin example (#2473)
  • fix(auth): enhance the URL check (#2422)

Option Changes

  • refact(server): enhance the storage path in RocksDB & clean code (#2491)

Other Changes

  • chore: add a license link (#2398)
  • doc: enhance NOTICE info to keep it clear (#2409)
  • chore(server): update swagger info for default server profile (#2423)
  • fix(server): unify license header for protobuf file (#2448)
  • chore: improve license header checker confs and pre-check header when validating (#2445)
  • chore: unify to call SchemaLabel.getLabelId() (#2458)
  • chore: refine the hg-style.xml specification (#2457)
  • chore: Add a newline formatting configuration and a comment for warning (#2464)
  • chore(server): clear context after req done (#2470)

hugegraph-toolchain

API Changes

Feature Changes

  • fix(loader): update shade plugin for spark loader (#566)
  • fix(hubble): yarn install timeout in arm64 (#583)
  • fix(loader): support file name with prefix for hdfs source (#571)
  • feat(hubble): warp the exception info in HugeClientUtil (#589)

Bug Fix

  • fix: concurrency issue causing file overwrite due to identical filenames (#572)

Option Changes

  • feat(client): support user defined OKHTTPClient configs (#590)

Other Changes

  • doc: update copyright date(year) in NOTICE (#567)
  • chore(deps): bump ip from 1.1.5 to 1.1.9 in /hugegraph-hubble/hubble-fe (#580)
  • refactor(hubble): enhance maven front plugin (#568)
  • chore(deps): bump es5-ext from 0.10.53 to 0.10.63 in /hugegraph-hubble/hubble-fe (#582)
  • chore(hubble): Enhance code style in hubble (#592)
  • chore: upgrade version to 1.3.0 (#596)
  • chore(ci): update profile commit id for 1.3 (#597)

hugegraph-commons

Feature Changes

  • feat: support user defined RestClientConfig/HTTPClient params (#140)

Bug Fix

Other Changes

  • chore: disable clean flatten for deploy (#141)

hugegraph-ai

This is the first release version of hugegraph-ai, it contains a variety of features, including an initialized Python client, knowledge graph construction capabilities through LLM, and the integration of RAG based on HugeGraph.

It also adds significant functionalities on python-client such as variable APIs, auth, metric, traverser, and task APIs, as well as interactive and visual demo creation with Gradio. In addition to these features, the release addresses several bugs and issues, ensuring a more stable and error-free user experience. Maintenance tasks such as dependency updates, project structure improvements, and the addition of basic CI further enhance the project’s robustness and developer workflow.

This release encapsulates the collaborative efforts of the HugeGraph community, with contributions from various members, ensuring the project’s continuous growth and improvement.

Feature Changes

  • feat: initialize hugegraph python client (#5)
  • feat(llm): knowledge graph construction by llm (#7)
  • feat: initialize rag based on HugeGraph (#20)
  • feat(client): add variables api and test (#24)
  • feat: add llm wenxinyiyan & config util & spo_triple_extract (#27)
  • feat: add auth&metric&traverser&task api and ut (#28)
  • feat: refactor construct knowledge graph task (#29)
  • feat: Introduce gradio for creating interactive and visual demo (#30)

Bug Fix

  • fix: invalid GitHub label (#3)
  • fix: import error (#13)
  • fix: function getEdgeByPage(): the generated query url does not include the parameter page (#15)
  • fix: issue template (#23)
  • fix: base-ref/head-ref missed in dependency-check-ci on branch push (#25)

Other Changes

  • chore: add asf.yaml and ISSUE_TEMPLATE (#1)
  • Bump urllib3 from 2.0.3 to 2.0.7 in /hugegraph-python (#8)
  • chore: create .gitignore file for py (#9)
  • refact: improve project structure & add some basic CI (#17)
  • chore: Update LICENSE and NOTICE (#31)
  • chore: add release scripts (#33)
  • chore: change file chmod 755 (#34)

Release Details

Please check the release details/contributor in each repository:

10.4 - HugeGraph 1.2.0 Release Notes

Java version statement

In the future, we will gradually upgrade the java version, Java 11 -> Java 17 -> Java 21.

  1. hugegraph, hugegraph-toolchain, hugegraph-commons consider use Java 11, also compatible with Java 8 now.
  2. hugegraph-computer required to use Java 11, not compatible with Java 8 now!

v1.2.0 may be the last major version compatible with Java 8, compatibility with Java 8 will totally end in v1.5 when PD/Store merged into master branch (Except for the java-client).

hugegraph

API Changes

  • feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges (#2262)
  • feat(api): support embedded arthas agent in hugegraph-server (#2278,#2337)
  • feat(api): support metric API Prometheus format & add statistic metric api (#2286)
  • feat(api-core): support label & property filtering for both edge and vertex & support kout dfs mode (#2295)
  • feat(api): support recording slow query log (#2327)

Feature Changes

  • feat: support task auto manage by server role state machine (#2130)
  • feat: support parallel compress snapshot (#2136)
  • feat: use an enhanced CypherAPI to refactor it (#2143)
  • feat(perf): support JMH benchmark in HG-test module (#2238)
  • feat: optimising adjacency edge queries (#2242)
  • Feat: IP white list (#2299)
  • feat(cassandra): adapt cassandra from 3.11.12 to 4.0.10 (#2300)
  • feat: support Cassandra with docker-compose in server (#2307)
  • feat(core): support batch+parallel edges traverse (#2312)
  • feat: adapt Dockerfile for new project structur (#2344)
  • feat(server):swagger support auth for standardAuth mode by (#2360)
  • feat(core): add IntMapByDynamicHash V1 implement (#2377)

Bug Fix

  • fix: transfer add_peer/remove_peer command to leader (#2112)
  • fix query dirty edges of a vertex with cache (#2166)
  • fix exception of vertex-drop with index (#2181)
  • fix: remove dup ‘From’ in filterExpiredResultFromFromBackend (#2207)
  • fix: jdbc ssl mode parameter redundant (#2224)
  • fix: error when start gremlin-console with sample script (#2231)
  • fix(core): support order by id (#2233)
  • fix: update ssl_mode value (#2235)
  • fix: optimizing ClassNotFoundException error message for MYSQL (#2246)
  • fix: asf invalid notification scheme ‘discussions_status’ (#2247)
  • fix: asf invalid notification scheme ‘discussions_comment’ (#2250)
  • fix: incorrect use of ‘NO_LIMIT’ variable (#2253)
  • fix(core): close flat mapper iterator after usage (#2281)
  • fix(dist): avoid var PRELOAD cover environmnet vars (#2302)
  • fix: base-ref/head-ref missed in dependency-review on master (#2308)
  • fix(core): handle schema Cache expandCapacity concurrent problem (#2332)
  • fix: in wait-storage.sh, always wait for storage with default rocksdb (#2333)
  • fix(api): refactor/downgrade record logic for slow log (#2347)
  • fix(api): clean some code for release (#2348)
  • fix: remove redirect-to-master from synchronous Gremlin API (#2356)
  • fix HBase PrefixFilter bug (#2364)
  • chore: fix curl failed to request https urls (#2378)
  • fix(api): correct the vertex id in the edge-existence api (#2380)
  • fix: github action build docker image failed during the release 1.2 process (#2386)
  • fix: TinkerPop unit test lack some lables (#2387)

Option Changes

  • feat(dist): support pre-load test graph data in docker container (#2241)

Other Changes

  • refact: use standard UTF-8 charset & enhance CI configs (#2095)
  • move validate release to hugegraph-doc (#2109)
  • refact: use a slim way to build docker image on latest code & support zgc (#2118)
  • chore: remove stage-repo in pom due to release done & update mail rule (#2128)
  • doc: update issue template & README file (#2131)
  • chore: cmn algorithm optimization (#2134)
  • add github token for license check comment (#2139)
  • chore: disable PR up-to-date in branch (#2150)
  • refact(core): remove lock of globalMasterInfo to optimize perf (#2151)
  • chore: async remove left index shouldn’t effect query (#2199)
  • refact(rocksdb): clean & reformat some code (#2200)
  • refact(core): optimized batch removal of remaining indices consumed by a single consumer (#2203)
  • add com.janeluo.ikkanalyzer dependency to core model (#2206)
  • refact(core): early stop unnecessary loops in edge cache (#2211)
  • doc: update README & add QR code (#2218)
  • chore: update .asf.yaml for mail rule (#2221)
  • chore: improve the UI & content in README (#2227)
  • chore: add pr template (#2234)
  • doc: modify ASF and remove meaningless CLA (#2237)
  • chore(dist): replace wget to curl to download swagger-ui (#2277)
  • Update StandardStateMachineCallback.java (#2290)
  • doc: update README about start server with example graph (#2315)
  • README.md tiny improve (#2320)
  • doc: README.md tiny improve (#2331)
  • refact: adjust project structure for merge PD & Store[Breaking Change] (#2338)
  • chore: disable raft test in normal PR due to timeout problem (#2349)
  • chore(ci): add stage profile settings (#2361)
  • refact(api): update common 1.2 & fix jersey client code problem (#2365)
  • chore: move server info into GlobalMasterInfo (#2370)
  • chore: reset hugegraph version to 1.2.0 (#2382)

hugegraph-computer

Feature Changes

  • feat: implement fast-failover for MessageRecvManager and DataClientManager (#243)
  • feat: implement parallel send data in load graph step (#248)
  • feat(k8s): init operator project & add webhook (#259, #263)
  • feat(core): support load vertex/edge snapshot (#269)
  • feat(k8s): Add MinIO as internal(default) storage (#272)
  • feat(algorithm): support random walk in computer (#274, #280)
  • feat: use ‘foreground’ delete policy to cancel k8s job (#290)

Bug Fix

  • fix: superstep not take effect (#237)
  • fix(k8s): modify inconsistent apiGroups (#270)
  • fix(algorithm): record loop is not copied (#276)
  • refact(core): adaptor for common 1.2 & fix a string of possible CI problem (#286)
  • fix: remove okhttp1 due to conflicts risk (#294)
  • fix(core): io.grpc.grpc-core dependency conflic (#296)

Option Changes

  • feat(core): isolate namespace for different input data source (#252)
  • refact(core): support auth config for computer task (#265)

Other Changes

  • remove apache stage repo & update notification rule (#232)
  • chore: fix empty license file (#233)
  • chore: enhance mailbox settings & enable require ci (#235)
  • fix: typo errors in start-computer.sh (#238)
  • [Feature-241] Add PULL_REQUEST_TEMPLATE (#242, #257)
  • chore: change etcd url only for ci (#245)
  • doc: update readme & add QR code (#249)
  • doc(k8s): add building note for missing classes (#254)
  • chore: reduce mail to dev list (#255)
  • add: dependency-review (#266)
  • chore: correct incorrect comment (#268)
  • refactor(api): ListValue.getFirst() replaces ListValue.get(0) (#282)
  • Improve: Passing workerId to WorkerStat & Skip wait worker close if master executes failed (#292)
  • chore: add check dependencies (#293)
  • chore(license): update license for 1.2.0 (#299)

hugegraph-toolchain

API Changes

  • feat(client): support edgeExistence api (#544)
  • refact(client): update tests for new OLTP traverser APIs (#550)

Feature Changes

  • feat(spark): support spark-sink connector for loader (#497)
  • feat(loader): support kafka as datasource (#506)
  • feat(client): support go client for hugegraph (#514)
  • feat(loader): support docker for loader (#530)
  • feat: update common version and remove jersey code (#538)

Bug Fix

  • fix: convert numbers to strings (#465)
  • fix: hugegraph-spark-loader shell string length limit (#469)
  • fix: spark loader meet Exception: Class is not registered (#470)
  • fix: spark loader Task not serializable (#471)
  • fix: spark with loader has dependency conflicts (#480)
  • fix: spark-loader example schema and struct mismatch (#504)
  • fix(loader): error log (#499)
  • fix: checkstyle && add suppressions.xml (#500)
  • fix(loader): resolve error in loader script (#510)
  • fix: base-ref/head-ref missed in dependency-check-ci on branch push (#516, #551)
  • fix yarn network connection on linux/arm64 arch (#519)
  • fix(hubble): drop-down box could not display all options (#535)
  • fix(hubble): build with node and yarn (#543)
  • fix(loader): loader options (#548)
  • fix(hubble): parent override children dep version (#549)
  • fix: exclude okhttp1 which has different groupID with okhttp3 (#555)
  • fix: github action build docker image failed (#556, #557)
  • fix: build error with npm not exist & tiny improve (#558)

Option Changes

  • set default data when create graph (#447)

Other Changes

  • chore: remove apache stage repo & update mail rule (#433, #474, #479)
  • refact: clean extra store file in all modules (#434)
  • chore: use fixed node.js version 16 to avoid ci problem (#437, #441)
  • chore(hubble): use latest code in Dockerfile (#440)
  • chore: remove maven plugin for docker build (#443)
  • chore: improve spark parallel (#450)
  • doc: fix build status badge link (#455)
  • chore: keep hadoop-hdfs-client and hadoop-common version consistent (#457)
  • doc: add basic contact info & QR code in README (#462, #475)
  • chore: disable PR up-to-date in branch (#473)
  • chore: auto add pr auto label by path (#466, #528)
  • chore: unify the dependencies versions of the entire project (#478)
  • chore(deps): bump async, semver, word-wrap, browserify-sign in hubble-fe (#484, #491, #494, #529)
  • chore: add pr template (#498)
  • doc(hubble): add docker-compose to start with server (#522)
  • chore(ci): add stage profile settings (#536)
  • chore(client): increase the api num as the latest server commit + 10 (#546)
  • chore(spark): install hugegraph from source (#552)
  • doc: adjust docker related desc in readme (#559)
  • chore(license): update license for 1.2 (#560, #561)

hugegraph-commons

Feature Changes

  • feat(common): replace jersey dependencies with OkHttp (Breaking Change) (#133)

Bug Fix

  • fix(common): handle spring-boot2/jersey dependency conflicts (#131)
  • fix: Assert.assertThrows() should check result of exceptionConsumer (#135)
  • fix(common): json param convert (#137)

Other Changes

  • refact(common): add more construction methods for convenient (#132)
  • add: dependency-review (#134)
  • refact(common): rename jsonutil to avoid conflicts with server (#136)
  • doc: update README for release (#138)
  • update licence (#139)

Release Details

Please check the release details in each repository:

10.5 - HugeGraph 1.0.0 Release Notes

OLTP API & Client Changes

API Changes

  • feat(api): support hot set trace through /exception/trace API.
  • feat(api): support query by Cypher language.
  • feat(api): support swagger UI to viewing API.

Client Changes

  • feat(client) support Cypher query API.
  • refact(client): change ’limit’ type from long to int.
  • feat(client): server bypass for hbase writing (Beta).

Core & Server

Feature Changes

  • feat: support Java 11.
  • feat(core): support adamic-adar & resource-allocation algorithms.
  • feat(hbase): support hash rowkey & pre-init tables.
  • feat(core): support query by Cypher language.
  • feat(core): support automatic management and fail-over for cluster role.
  • feat(core): support 16 OLAP algorithms, like: LPA, Louvain, PageRank, BetweennessCentrality, RingsDetect.
  • feat: prepare for Apache release.

Bug Fix

  • fix(core): can’t query edges by multi labels + properties.
  • fix(core): occasionally NoSuchMethodError Relations().
  • fix(core): limit max depth for cycle detection.
  • fix(core): traversal contains Tree step has different result.
  • fix edge batch update error.
  • fix unexpected task status.
  • fix(core): edge cache not clear when update or delete associated vertex.
  • fix(mysql): run g.V() is error when it’s MySQL backend.
  • fix: close exception and server-info EXPIRED_INTERVAL.
  • fix: export ConditionP.
  • fix: query by within + Text.contains.
  • fix: schema label race condition of addIndexLabel/removeIndexLabel.
  • fix: limit admin role can drop graph.
  • fix: ProfileApi url check & add build package to ignore file.
  • fix: can’t shut down when starting with exception.
  • fix: Traversal.graph is empty in StepStrategy.apply() with count().is(0).
  • fix: possible extra comma before where statement in MySQL backend.
  • fix: JNA UnsatisfiedLinkError for Apple M1.
  • fix: start RpcServer NPE & args count of ACTION_CLEARED error & example error.
  • fix: rpc server not start.
  • fix: User-controlled data in numeric cast & remove word dependency.
  • fix: closing iterators on errors for Cassandra & Mysql.

Option Changes

  • move raft.endpoint option from graph scope to server scope.

Other Changes

  • refact(core): enhance schema job module.
  • refact(raft): improve raft module & test & install snapshot and add peer.
  • refact(core): remove early cycle detection & limit max depth.
  • cache: fix assert node.next==empty.
  • fix apache license conflicts: jnr-posix and jboss-logging.
  • chore: add logo in README & remove outdated log4j version.
  • refact(core): improve CachedGraphTransaction perf.
  • chore: update CI config & support ci robot & add codeQL SEC-check & graph option.
  • refact: ignore security check api & fix some bugs & clean code.
  • doc: enhance CONTRIBUTING.md & README.md.
  • refact: add checkstyle plugin & clean/format the code.
  • refact(core): improve decode string empty bytes & avoid array-construct columns in BackendEntry.
  • refact(cassandra): translate ipv4 to ipv6 metrics & update cassandra dependency version.
  • chore: use .asf.yaml for apache workflow & replace APPLICATION_JSON with TEXT_PLAIN.
  • feat: add system schema store.
  • refact(rocksdb): update rocksdb version to 6.22 & improve rocksdb code.
  • refact: update mysql scope to test & clean protobuf style/configs.
  • chore: upgrade Dockerfile server to 0.12.0 & add editorconfig & improve ci.
  • chore: upgrade grpc version.
  • feat: support updateIfPresent/updateIfAbsent operation.
  • chore: modify abnormal logs & upgrade netty-all to 4.1.44.
  • refact: upgrade dependencies & adopt new analyzer & clean code.
  • chore: improve .gitignore & update ci configs & add RAT/flatten plugin.
  • chore(license): add dependencies-check ci & 3rd-party dependency licenses.
  • refact: Shutdown log when shutdown process & fix tx leak & enhance the file path.
  • refact: rename package to apache & dependency in all modules (Breaking Change).
  • chore: add license checker & update antrun plugin & fix building problem in windows.
  • feat: support one-step script for apache release v1.0.0 release.

Computer (OLAP)

Algorithm Changes

  • feat: implement page-rank algorithm.
  • feat: implement wcc algorithm.
  • feat: implement degree centrality.
  • feat: implement triangle_count algorithm.
  • feat: implement rings-detection algorithm.
  • feat: implement LPA algorithm.
  • feat: implement kcore algorithm.
  • feat: implement closeness centrality algorithm.
  • feat: implement betweenness centrality algorithm.
  • feat: implement cluster coefficient algorithm.

Platform Changes

  • feat: init module computer-core & computer-algorithm & etcd dependency.
  • feat: add Id as base type of vertex id.
  • feat: init Vertex/Edge/Properties & JsonStructGraphOutput.
  • feat: load data from hugegraph server.
  • feat: init basic combiner, Bsp4Worker, Bsp4Master.
  • feat: init sort & transport interface & basic FileInput/Output Stream.
  • feat: init computation & ComputerOutput/Driver interface.
  • feat: init Partitioner and HashPartitioner
  • feat: init Master/WorkerService module.
  • feat: init Heap/LoserTree sorting.
  • feat: init rpc module.
  • feat: init transport server, client, en/decode, flowControl, heartbeat.
  • feat: init DataDirManager & PointerCombiner.
  • feat: init aggregator module & add copy() and assign() methods to Value class.
  • feat: add startAsync and finishAsync on client side, add onStarted and onFinished on server side.
  • feat: init store/sort module.
  • feat: link managers in worker sending end.
  • feat: implement data receiver of worker.
  • feat: implement StreamGraphInput and EntryInput.
  • feat: add Sender and Receiver to process compute message.
  • feat: add seqfile fromat.
  • feat: add ComputeManager.
  • feat: add computer-k8s and computer-k8s-operator.
  • feat: add startup and make docker image code.
  • feat: sort different type of message use different combiner.
  • feat: add HDFS output format.
  • feat: mount config-map and secret to container.
  • feat: support java11.
  • feat: support partition concurrent compute.
  • refact: abstract computer-api from computer-core.
  • refact: optimize data receiving.
  • fix: release file descriptor after input and compute.
  • doc: add operator deploy readme.
  • feat: prepare for Apache release.

Toolchain (loader, tools, hubble)

  • feat(loader): support use SQL to construct graph.
  • feat(loader): support Spark-Loader mode(include jdbc source).
  • feat(loader): support Flink-CDC mode.
  • fix(loader): fix NPE when loading ORC data.
  • fix(loader): fix schema is not cached with Spark/Flink mode.
  • fix(loader): fix json deserialize error.
  • fix(loader): fix jackson conflicts & missing dependencies.
  • feat(hubble): supplementary algorithms UI.
  • feat(hubble): support highlighting and hints for Gremlin text.
  • feat(hubble): add docker-file for hubble.
  • feat(hubble): display packaging log output progress while building.
  • fix(hubble): fix port-input placeholder UI.
  • feat: prepare for Apache release.

Commons (common,rpc)

  • feat: support assert-throws method returning Future.
  • feat: add Cnm and Anm to CollectionUtil.
  • feat: support custom content-type.
  • feat: prepare for Apache release.

Release Details

Please check the release details in each repository:

11 - Apache Contributor Agreements

Apache Contributor Agreements

HugeGraph uses the standard Apache Software Foundation (ASF) contributor agreements. It no longer uses a project-specific CLA or the GitHub CLA Assistant workflow.

Routine, small contributions are submitted under Section 5 of the Apache License 2.0. Before becoming a committer, or when making a substantial contribution, you must sign the ASF Individual Contributor License Agreement (ICLA). If a company owns the intellectual property in your contribution, a Corporate Contributor License Agreement (CCLA) may also be required; a CCLA does not replace your individual ICLA.

Refer to the official ASF pages for the agreement text, completion instructions, and submission process:

After reading and signing the agreement as instructed, send the completed file as a standalone attachment to secretary@apache.org. Do not send it to the HugeGraph developer mailing list or commit it to a GitHub repository.