HugeGraph Computing (OLAP)

• 1: HugeGraph-Vermeer Quick Start
• 2: HugeGraph-Computer Quick Start
• 3: HugeGraph-Computer Configuration Reference

flowchart TB
    Master["Master"] --> Workers["Workers"]
    Master -->|PD| PD["PD"]
    Workers -->|Scan| Store["Store"]
    Workers -.->|REST| Server["Server"]

flowchart TB
    Config["Job configuration"] --> Master["Master"] --> Workers["Workers"]
    Master -. BSP .-> Etcd["etcd"]
    Workers <-->|REST| Server["Server"]
    Workers <-->|HDFS| HDFS["HDFS (optional)"]

1 - HugeGraph-Vermeer Quick Start

1. Overview of Vermeer

1.1 Architecture

Vermeer is a high-performance, memory-first graph computing framework written in Go: start once and execute repeatedly. It supports fast execution of 15+ OLAP algorithms, often in seconds to minutes; actual time depends on graph size, algorithm parameters, and resources. A single master currently schedules multiple workers.

The master handles communication, forwarding, and aggregation, with modest computation and resource usage. Workers store graph data and execute tasks, consuming most memory and CPU. gRPC handles internal communication; REST provides external APIs.

At startup, built-in defaults are loaded first, followed by the [default] section of config/<env>.ini under the working directory, then explicit command-line overrides. For example, --env=master reads config/master.ini. The Docker image copies repository configuration to /go/bin/config/ and works from /go/bin/. A host bind mount hides the bundled files, so it must contain every ini file used. The configuration reader does not read environment variables.

Default ports:

The Docker examples publish only master HTTP on host loopback 127.0.0.1:6688:6688; master and worker gRPC ports stay within the container network.

flowchart LR
  Client["curl / Python client"] -->|"HTTP :6688"| Master["master"]
  Worker["worker"] <-->|"Bidirectional gRPC: master :6689, worker :6789"| Master
  Master -->|"gRPC partition lookup"| PD["HugeGraph PD"]
  Worker -->|"gRPC partition scan"| Store["HugeGraph Store"]

1.2 Running Vermeer

Both Docker options need a host configuration directory. From the Vermeer repository root, copy the supplied master.ini and worker.ini templates. The mount hides image configuration under /go/bin/config, so never mount an empty directory or your entire home directory there:

CONFIG_DIR="$HOME/vermeer-config"
mkdir -p "$CONFIG_DIR"
cp config/master.ini config/worker.ini "$CONFIG_DIR/"

Keep master HTTP/gRPC listeners at 0.0.0.0:6688 and 0.0.0.0:6689. The Compose example assigns 172.20.0.10 to master and 172.20.0.11 to worker; set these options in the copied worker.ini:

[default]
http_peer=0.0.0.0:6788
grpc_peer=172.20.0.11:6789
master_peer=172.20.0.10:6689
run_mode=worker
worker_group=$

This single-worker example uses literal worker_group=$, the general group used when no named group is bound. The repository template defaults to worker_group=default; if retaining that named group, bind it to the task space or graph before submitting tasks. For default space $DEFAULT, use:

curl --fail --show-error -X POST \
  'http://localhost:6688/admin/workers/alloc/default/%24DEFAULT'

An errcode=0 response confirms binding; with token authentication, also supply the authorization header. master_peer must reach master gRPC from the worker network. grpc_peer is both the listener and advertised worker address, so do not advertise 0.0.0.0, which other containers cannot reach. Update these IPs when changing the subnet or addresses; each additional worker needs a unique, mutually reachable grpc_peer.

1. Option 1: Docker Compose (Recommended)

Run from the Vermeer repository root. Modify existing docker-compose.yaml or use the following example. The repository file currently mounts all of ~/ as configuration and does not publish master HTTP; replace the mount and add the port mapping before starting:

services:
  vermeer-master:
    image: hugegraph/vermeer
    container_name: vermeer-master
    ports:
      - "127.0.0.1:6688:6688"
    volumes:
      - /home/user/vermeer-config:/go/bin/config:ro
    command: --env=master
    networks:
      vermeer_network:
        ipv4_address: 172.20.0.10 # Static master address

  vermeer-worker:
    image: hugegraph/vermeer
    container_name: vermeer-worker
    volumes:
      - /home/user/vermeer-config:/go/bin/config:ro
    command: --env=worker
    networks:
      vermeer_network:
        ipv4_address: 172.20.0.11 # Static worker address

networks:
  vermeer_network:
    driver: bridge
    ipam:
      config:
        - subnet: 172.20.0.0/24 # Change the subnet as needed

Replace /home/user/vermeer-config with your actual absolute configuration directory. Changing the subnet or static IPs of vermeer_network also requires updating grpc_peer and master_peer in worker.ini. Do not use the default grpc_peer=0.0.0.0:6789 as an advertised container address.

Build and start from the project directory:

# Build the image from the Vermeer root directory
docker build -t hugegraph/vermeer .

# Start from the Vermeer root directory
docker-compose up -d
# Or use the newer CLI:
# docker compose up -d

View logs / stop / remove:

docker-compose logs -f
docker-compose down

2. Option 2: Separate docker run Commands (Manual Network and Static IPs)

Set CONFIG_DIR to the prepared configuration directory, with grpc_peer and master_peer matching the static container addresses. Replace the example CONFIG_DIR with your actual absolute path.

Build the image:

docker build -t hugegraph/vermeer .

Create a custom bridge network (once):

docker network create --driver bridge \
  --subnet 172.20.0.0/24 \
  vermeer_network

Run master (use your absolute CONFIG_DIR and adjust IPs as needed):

CONFIG_DIR=/home/user/vermeer-config

docker run -d \
  --name vermeer-master \
  --network vermeer_network --ip 172.20.0.10 \
  -p 127.0.0.1:6688:6688 \
  -v ${CONFIG_DIR}:/go/bin/config:ro \
  hugegraph/vermeer \
  --env=master

Run worker:

docker run -d \
  --name vermeer-worker \
  --network vermeer_network --ip 172.20.0.11 \
  -v ${CONFIG_DIR}:/go/bin/config:ro \
  hugegraph/vermeer \
  --env=worker

View logs / stop / remove:

docker logs -f vermeer-master
docker logs -f vermeer-worker

docker stop vermeer-master vermeer-worker
docker rm vermeer-master vermeer-worker

# Delete the custom network if needed
docker network rm vermeer_network

3. Option 3: Build from Source

Build following the Vermeer README.

go build

Start from the Vermeer root directory with ./vermeer --env=master and ./vermeer --env=worker01. In worker01.ini, set grpc_peer to an address bindable locally and reachable by master and other workers; point master_peer to master gRPC.

After starting master, check its HTTP port from the host:

curl --fail --show-error http://localhost:6688/graphs

Expect HTTP 200 and JSON errcode=0.

2. Task Creation REST API

2.1 Introduction

Submit a load task, wait for loading to finish, then submit a compute task. A loaded graph can support repeated computations and is not deleted after completion. Asynchronous APIs return creation results and task information, including ID, before completion; synchronous APIs wait for success or failure, so client and proxy HTTP timeouts must be sufficiently long. Query task states: loaded for successful loading, complete for successful computation, error for failure, and canceled for cancellation; other states remain waiting or running. Graphs loading or in an error state cannot be computed. Deletion requires a deletable graph state and no current usage.

Available URLs:

• Asynchronous: POST http://master_ip:port/tasks/create; read the ID from response task.id.
• Synchronous: POST http://master_ip:port/tasks/create/sync; returns after the task ends.
• Query task: GET http://master_ip:port/task/{task_id}. errcode=0 indicates a successful query; inspect task.state for task status. Set a client polling deadline for asynchronous jobs; reaching it stops client waiting without canceling the server task.

2.2 Loading Graph Data

The examples list common load parameters. Each loader reads its own keys; unused keys do not change behavior.

Vermeer provides three loading methods:

1. Local files

load.vertex_files and load.edge_files map worker address host portions to file paths read by those workers. With containers, mount data into the worker and use container paths. In the Compose example, mount a host data directory at worker /data (such as - /host/data:/data:ro) and use 172.20.0.11 from grpc_peer as the mapping key. Other deployments use the host portion of their advertised worker addresses.

Obtain a dataset such as Twitter-2010; the first twitter-2010.txt.gz file is sufficient.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "load",
 "graph": "testdb",
 "params": {
  "load.parallel": "50",
  "load.type": "local",
  "load.vertex_files": "{\"172.20.0.11\":\"/data/twitter-2010.v_[0,99]\"}",
  "load.edge_files": "{\"172.20.0.11\":\"/data/twitter-2010.e_[0,99]\"}",
  "load.use_out_degree": "1",
  "load.use_outedge": "1"
 }
}

2. HugeGraph

Request example:

Replace request addresses, graph name, and credentials with your actual connection settings.

POST http://localhost:6688/tasks/create
{
  "task_type": "load",
  "graph": "testdb",
  "params": {
    "load.parallel": "50",
    "load.type": "hugegraph",
    "load.hg_pd_peers": "[\"<pd-address-reachable-from-vermeer>:8686\"]",
    "load.hugegraph_name": "DEFAULT/hugegraph2/g",
    "load.hugegraph_username": "admin",
    "load.hugegraph_password": "<your-password-here>",
    "load.use_out_degree": "1",
    "load.use_outedge": "1"
  }
}

Vermeer master connects to PD through load.hg_pd_peers to look up partitions; workers read data from the returned Store addresses. These services must be reachable from the corresponding Vermeer hosts or containers. Inside Docker, 127.0.0.1 identifies the container itself, not the host or another container.

3. HDFS

Request example:

POST http://localhost:6688/tasks/create
{
  "task_type": "load",
  "graph": "testdb",
  "params": {
    "load.parallel": "50",
    "load.type": "hdfs",
    "load.hdfs_namenode": "name_node1:9000",
    "load.hdfs_conf_path": "/path/to/conf",
    "load.krb_realm": "EXAMPLE.COM",
    "load.krb_name": "user@EXAMPLE.COM",
    "load.krb_keytab_path": "/path/to/keytab",
    "load.krb_conf_path": "/path/to/krb5.conf",
    "load.hdfs_use_krb": "1",
    "load.vertex_files": "/data/graph/vertices",
    "load.edge_files": "/data/graph/edges",
    "load.use_out_degree": "1",
    "load.use_outedge": "1"
  }
}

2.3 Outputting Computation Results

Current result writers support local, hdfs, and hugegraph through output.type; none disables output. Source retains an afs constant, but current master registers no AFS loader or writer. Set output.need_statistics=1 to put statistics into task information; supported statistics operators depend on each algorithm implementation.

The examples list common computation and output parameters; supported algorithm parameters depend on current Vermeer implementations.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "pagerank",
 "compute.parallel": "10",
 "compute.max_step": "10",
 "output.type": "local",
 "output.parallel": "1",
 "output.file_path": "result/pagerank"
  }
}

output.type=local writes to the executing worker’s local filesystem. For containers, mount the output directory if the host needs to read results.

3. Supported Algorithms

3.1 PageRank

The PageRank algorithm, also known as the web ranking algorithm, is a technique used by search engines to calculate the relevance and importance of web pages (nodes) based on their mutual hyperlinks.

• If a web page is linked to by many other web pages, it indicates that the web page is relatively important, and its PageRank value will be relatively high.
• If a web page with a high PageRank value links to other web pages, the PageRank value of the linked web pages will also increase accordingly.

The PageRank algorithm is suitable for scenarios such as web page ranking and identifying key figures in social networks.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "pagerank",
 "compute.parallel":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/pagerank",
 "compute.max_step":"10"
 }
}

3.2 WCC (Weakly Connected Components)

The weakly connected components algorithm calculates all connected subgraphs in an undirected graph and outputs the weakly connected subgraph ID to which each vertex belongs, indicating the connectivity between points and distinguishing different connected communities.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "wcc",
 "compute.parallel":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/wcc",
 "compute.max_step":"10"
 }
}

3.3 LPA (Label Propagation Algorithm)

The label propagation algorithm is a graph clustering algorithm commonly used in social networks to discover potential communities.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "lpa",
 "compute.parallel":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/lpa",
 "compute.max_step":"10"
 }
}

3.4 Degree Centrality

The degree centrality algorithm calculates the degree centrality value of each node in the graph, supporting both undirected and directed graphs. Degree centrality is an important indicator of node importance; the more edges a node has with other nodes, the higher its degree centrality value, and the more important the node is in the graph. In an undirected graph, degree centrality is calculated based on edge information to count the number of times a node appears, resulting in the degree centrality value of the node. In a directed graph, it is based on the direction of the edges, filtering based on input or output-edge information to count the number of times a node appears, resulting in the in-degree or out-degree value of the node. It indicates the importance of each point, with more important points having higher degrees.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "degree",
 "compute.parallel":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/degree",
 "degree.direction":"both"
 }
}

3.5 Closeness Centrality

Closeness centrality is used to calculate the inverse of the shortest distance from a node to all other reachable nodes, accumulating and normalizing the value. Closeness centrality can be used to measure the time it takes for information to be transmitted from the node to other nodes. The larger the closeness centrality of a node, the closer its position in the graph is to the center, suitable for scenarios such as identifying key nodes in social networks.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "closeness_centrality",
 "compute.parallel":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/closeness_centrality",
 "closeness_centrality.sample_rate":"0.01"
 }
}

3.6 Betweenness Centrality

The betweenness centrality algorithm determines the value of a node as a “bridge” node; the larger the value, the more likely it is to be a necessary path between two points in the graph. Typical examples include mutual followers in social networks. It is suitable for measuring the degree of aggregation around a node in a community.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "betweenness_centrality",
 "compute.parallel":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/betweenness_centrality",
 "betweenness_centrality.sample_rate":"0.01"
 }
}

3.7 Triangle Count

The triangle count algorithm calculates the number of triangles passing through each vertex, suitable for calculating the relationships between users and whether the associations form triangles. The more triangles, the higher the degree of association between nodes in the graph, and the tighter the organizational relationship. In social networks, triangles indicate cohesive communities, and identifying triangles helps understand clustering and interconnections among individuals or groups in the network. In financial or transaction networks, the presence of triangles may indicate suspicious or fraudulent activities, and triangle counting can help identify transaction patterns that may require further investigation.

The output result is the Triangle Count corresponding to each vertex, i.e., the number of triangles the vertex is part of.

Note: This algorithm is for undirected graphs and ignores edge directions.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "triangle_count",
 "compute.parallel":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/triangle_count"
 }
}

3.8 K-Core

The K-Core algorithm marks all vertices with a degree of K, suitable for graph pruning and finding the core part of the graph.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "kcore",
 "compute.parallel":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/kcore",
 "kcore.degree_k":"5"
 }
}

3.9 SSSP (Single Source Shortest Path)

The single source the shortest path algorithm calculates the shortest distance from one point to all other points.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "sssp",
 "compute.parallel":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/degree",
 "sssp.source":"tom"
 }
}

3.10 KOUT

Starting from a point, get the k-layer nodes of this point.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "kout",
 "compute.parallel":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/kout",
 "kout.source":"tom",
 "compute.max_step":"6"
 }
}

3.11 Louvain

The Louvain algorithm is a community detection algorithm based on modularity. The basic idea is that nodes in the network try to traverse all neighbor community labels and choose the community label that maximizes the modularity increment. After maximizing modularity, each community is regarded as a new node, and the process is repeated until the modularity no longer increases.

The distributed Louvain algorithm implemented on Vermeer is affected by factors such as node order and parallel computation. Due to the random traversal order of the Louvain algorithm, community compression also has a certain randomness, leading to different results in multiple executions. However, the overall trend will not change significantly.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "louvain",
 "compute.parallel":"10",
 "compute.max_step":"1000",
 "louvain.threshold":"0.0000001",
 "louvain.resolution":"1.0",
 "louvain.step":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/louvain"
  }
 }

3.12 Jaccard Similarity Coefficient

The Jaccard index, also known as the Jaccard similarity coefficient, is used to compare the similarity and diversity between finite sample sets. The larger the Jaccard coefficient value, the higher the similarity of the samples. It is used to calculate the Jaccard similarity coefficient between a given source point and all other points in the graph.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "jaccard",
 "compute.parallel":"10",
 "compute.max_step":"2",
 "jaccard.source":"123",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/jaccard"
 }
}

3.13 Personalized PageRank

The goal of personalized PageRank is to calculate the relevance of all nodes relative to user u. Starting from the node corresponding to user u, at each node, there is a probability of 1-d to stop walking and start again from u, or a probability of d to continue walking, randomly selecting a node from the nodes pointed to by the current node to walk down. It is used to calculate the personalized PageRank score starting from a given starting point, suitable for scenarios such as social recommendations.

Since the calculation requires using out-degree, load.use_out_degree needs to be set to 1 when reading the graph.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "ppr",
 "compute.parallel":"100",
 "compute.max_step":"10",
 "ppr.source":"123",
 "ppr.damping":"0.85",
 "ppr.diff_threshold":"0.00001",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/ppr"
 }
}

3.14 Global Kout

Calculate the k-degree neighbors of all nodes in the graph (excluding themselves and 1~k-1 degree neighbors). Due to the severe memory expansion of the global kout algorithm, k is currently limited to 1 and 2. Additionally, the global kout algorithm supports filtering functions (parameters such as “compute.filter”:“risk_level==1”), and the filtering condition is judged when calculating the k-degree. The final result set includes those that meet the filtering condition. The algorithm’s final output is the number of neighbors that meet the condition.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "kout_all",
 "compute.parallel":"10",
 "output.type":"local",
 "output.parallel":"10",
 "output.file_path":"result/kout",
 "compute.max_step":"2",
 "compute.filter":"risk_level==1"
 }
}

3.15 Clustering Coefficient

The clustering coefficient represents the coefficient of the clustering degree of nodes in a graph. In real networks, especially in specific networks, nodes tend to establish a tightly organized relationship due to relatively high-density connection points. The clustering coefficient algorithm (Cluster Coefficient) is used to calculate the clustering degree of nodes in the graph. This algorithm is for local clustering coefficients. The local clustering coefficient can measure the clustering degree around each node in the graph.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "clustering_coefficient",
 "compute.parallel":"100",
 "compute.max_step":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/cc"
 }
}

3.16 SCC (Strongly Connected Components)

In the mathematical theory of directed graphs, if every vertex of a graph can be reached from any other point in the graph, the graph is said to be strongly connected. The parts of any directed graph that can achieve strong connectivity are called strongly connected components. It indicates the connectivity between points and distinguishes different connected communities.

Request example:

POST http://localhost:6688/tasks/create
{
 "task_type": "compute",
 "graph": "testdb",
 "params": {
 "compute.algorithm": "scc",
 "compute.parallel":"10",
 "output.type":"local",
 "output.parallel":"1",
 "output.file_path":"result/scc",
 "compute.max_step":"200"
 }
}

🚧, further updates and improvements will be made at any time. Suggestions and feedback are welcome.

2 - HugeGraph-Computer Quick Start

1. Component Overview

HugeGraph-Computer is a Java distributed graph computing framework based on BSP (Bulk Synchronous Parallel), with algorithms running in iterative supersteps. Kubernetes Operator or YARN can schedule jobs; master and worker processes can also run on one machine for small trial jobs.

Computer and Vermeer, in the same repository, are separate implementations: Computer uses a Java/BSP runtime for distributed computing, while Vermeer is a Go in-memory graph computing platform with a master-worker architecture. They share HugeGraph data sources, but their deployment and job configurations are not interchangeable.

Computer reads graph data from HugeGraph or HDFS and writes results to either system. The runtime can spill some data to disk; whether a job completes still depends on input size, resources, and configuration. Disk spilling does not remove resource limits.

2. Prerequisites and Connections

Building and running require JDK 11 or later; source builds also require Maven 3.5 or later. The PageRank example needs a running HugeGraph-Server with graph data, plus etcd reachable by the master and workers.

For Kubernetes jobs, hugegraph.url must be reachable from all computing Pods; do not use a localhost address available only on your computer. If HugeGraph authentication is enabled, configure the username and password and supply matching credentials for REST queries.

See the Computer configuration reference for more options.

3. Obtain Source and Build a Distribution

The Apache HugeGraph download directory provides versioned Computer source archives, without separate precompiled Computer binaries. This example uses VERSION=1.7.0, whose source archive and build output names include -incubating-. For other versions, use the actual filenames in the download directory.

VERSION=1.7.0 # Replace with the release to download
ARCHIVE="apache-hugegraph-computer-incubating-${VERSION}-src.tar.gz"
DOWNLOAD_BASE="https://downloads.apache.org/hugegraph/${VERSION}"
curl -fLO "${DOWNLOAD_BASE}/${ARCHIVE}"
curl -fLO "${DOWNLOAD_BASE}/${ARCHIVE}.sha512"
curl -fLO "${DOWNLOAD_BASE}/${ARCHIVE}.asc"
curl -fLO https://downloads.apache.org/hugegraph/KEYS
shasum -a 512 -c "${ARCHIVE}.sha512"
gpg --import KEYS
gpg --verify "${ARCHIVE}.asc" "${ARCHIVE}"
tar -xzf "${ARCHIVE}"
cd "apache-hugegraph-computer-incubating-${VERSION}-src/computer"
mvn clean package -DskipTests
tar -xzf "target/apache-hugegraph-computer-incubating-${VERSION}.tar.gz"
cd "apache-hugegraph-computer-incubating-${VERSION}"

To build current master, clone and package the source:

git clone https://github.com/apache/hugegraph-computer.git
cd hugegraph-computer/computer
mvn clean package -DskipTests

The computer/computer-dist module assembles the distribution, including bin/start-computer.sh, runtime dependencies in lib/, built-in algorithms in algorithm/builtin-algorithm.jar, and default conf/computer.properties and conf/log4j2.xml. The source configuration is computer/computer-dist/src/assembly/static/conf/computer.properties. Release 1.7.0 build output includes -incubating-; current master omits it and takes its version from the POM. Use archive names matching the source you built.

4. Run PageRank Locally

Edit conf/computer.properties in the distribution directory with your HugeGraph URL, graph name, and credentials; point bsp.etcd_endpoints to reachable etcd. The default configuration selects built-in PageRankParams. Master and workers must use the same configuration and job.id; concurrent jobs need distinct job IDs.

Start processes from the distribution directory in two terminals. The script reads conf/computer.properties by default; use -c to select another file.

# Terminal 1: start master
bin/start-computer.sh -d local -r master

# Terminal 2: start worker
bin/start-computer.sh -d local -r worker

Master waits for the configured workers to register before running the job. Check both terminal outputs; the default logging configuration also writes master and worker logs under the distribution directory’s logs/. Process startup alone does not confirm completion: verify that input, superstep computation, and output all finish normally in the master log.

PageRank writes results to HugeGraph under page_rank. First ensure every target vertex label allows this property. Computer creates a DOUBLE, OLAP_COMMON property key but does not add it to vertex labels. If absent, create it:

curl --fail --request POST \
  --header 'Content-Type: application/json' \
  --data '{"name":"page_rank","data_type":"DOUBLE","cardinality":"SINGLE","write_type":"OLAP_COMMON"}' \
  'http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/propertykeys'

If the property key already exists, verify its type is DOUBLE and write_type is OLAP_COMMON. List vertex labels, then add nullable page_rank to every label being computed; replace person with the actual label:

curl --fail --compressed \
  'http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/vertexlabels'

curl --fail --request PUT \
  --header 'Content-Type: application/json' \
  --data '{"name":"person","properties":["page_rank"],"nullable_keys":["page_rank"]}' \
  'http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/vertexlabels/person?action=append'

If the current read mode hides OLAP writes, an administrator can set it to ALL:

curl --fail --request PUT \
  --header 'Content-Type: application/json' \
  --data '"ALL"' \
  'http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/graph_read_mode'

Query vertices to verify the result property:

curl --fail --compressed \
  'http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices?limit=3'

When authentication is required, add --user "$HG_USER:$HG_PASSWORD" to curl. See the graph read-mode REST API for the endpoint and permissions.

5. Run PageRank on Kubernetes

Ensure computing Pods can reach HugeGraph-Server, then install a cluster-compatible cert-manager using its official installation guide. The Computer Operator manifests deploy the Operator, etcd, and MinIO. Use the same Computer release for the CRD and Operator manifests; this example uses 1.7.0:

VERSION=1.7.0 # Use the same release for CRD and Operator manifests
kubectl apply -f "https://raw.githubusercontent.com/apache/hugegraph-computer/${VERSION}/computer/computer-k8s-operator/manifest/hugegraph-computer-crd.v1.yaml"
kubectl apply -f "https://raw.githubusercontent.com/apache/hugegraph-computer/${VERSION}/computer/computer-k8s-operator/manifest/hugegraph-computer-operator.yaml"
kubectl rollout status deployment/hugegraph-computer-operator-controller-manager \
  -n hugegraph-computer-operator-system --timeout=120s

kubectl patch service hugegraph-computer-operator-minio \
  -n hugegraph-computer-operator-system \
  --type=json \
  -p='[{"op":"replace","path":"/spec/ports/0/targetPort","value":9000}]'

Replace the HugeGraph URL with a service address reachable by computing Pods, then submit a HugeGraphComputerJob. This example uses the official Docker Hub image hugegraph/hugegraph-computer:latest and its built-in PageRank JAR. Custom algorithm JARs can be bundled in the image and selected with jarFile, or downloaded from HTTP(S) with remoteJarUri. The partition count must be at least the worker count.

Save the following YAML as pagerank-job.yaml, then apply it:

apiVersion: operator.hugegraph.apache.org/v1
kind: HugeGraphComputerJob
metadata:
  namespace: hugegraph-computer-operator-system
  name: pagerank-sample
spec:
  jobId: pagerank-sample
  algorithmName: page_rank
  image: hugegraph/hugegraph-computer:latest # Official Docker Hub image
  jarFile: /hugegraph/hugegraph-computer/algorithm/builtin-algorithm.jar
  pullPolicy: IfNotPresent
  workerInstances: 1
  masterCpu: 500m
  workerCpu: 500m
  masterMemory: 1Gi
  workerMemory: 2Gi
  computerConf:
    job.partitions_count: "1"
    algorithm.params_class: org.apache.hugegraph.computer.algorithm.centrality.pagerank.PageRankParams
    hugegraph.url: http://hugegraph-server:8080
    hugegraph.name: hugegraph

kubectl apply -f pagerank-job.yaml
kubectl get hcjob pagerank-sample -n hugegraph-computer-operator-system --watch

SUCCEEDED means the job completed. By default, the Operator deletes the CR and computing resources after completion. Expand the following instructions when you need to retain state or investigate a failed job.

kubectl set env deployment/hugegraph-computer-operator-controller-manager \
  -n hugegraph-computer-operator-system \
  --containers=controller AUTO_DESTROY_POD=false
kubectl rollout status deployment/hugegraph-computer-operator-controller-manager \
  -n hugegraph-computer-operator-system --timeout=120s

kubectl logs --follow <master-pod-name> -n hugegraph-computer-operator-system
kubectl logs --follow <worker-pod-name> -n hugegraph-computer-operator-system

kubectl delete hcjob pagerank-sample -n hugegraph-computer-operator-system
kubectl set env deployment/hugegraph-computer-operator-controller-manager \
  -n hugegraph-computer-operator-system \
  --containers=controller AUTO_DESTROY_POD=true

After PageRank writes to HugeGraph, set the read mode and query as in the previous section. For HDFS output, results are under output.hdfs_path_prefix/<job.id>/; filenames and partition layout depend on job configuration.

See the CRD configuration reference for all fields.

6. Built-in Algorithms and Development

Current built-in algorithms include:

• Centrality: PageRank, Betweenness Centrality, Closeness Centrality, Degree Centrality.
• Communities and structure: Clustering Coefficient, K-core, LPA, Triangle Count, WCC.
• Paths and sampling: ring detection, filtered ring detection, single-source shortest path, Random Walk.

Implementations are in computer/computer-algorithm. Custom algorithms must follow the Computer API and be packaged as loadable JARs; see the Computer README for modules and development entry points.

3 - HugeGraph-Computer Configuration Reference

Computer Config Options

The defaults in the tables come from ComputerOptions.java in computer-api; explicit conf/computer.properties overrides are shown as “code default (distribution: actual value)”. Common options such as rpc.* come from HugeGraph Commons; distribution values are listed separately below.

Configuration Sources

• Source template: computer/computer-dist/src/assembly/static/conf/computer.properties, with log4j2.xml in the same directory. Maven package copies these into distribution conf/, runtime dependencies into lib/, and built-in algorithm JARs into algorithm/. Templates are maintained in source; there is no separate configuration generator.
• Standalone and YARN: startup scripts read distribution conf/computer.properties by default. Override it with bin/start-computer.sh -c <configuration-path>; master and workers need the same job parameters.
• Kubernetes Operator: users supply spec.computerConf in the CRD; the Operator writes a ConfigMap mounted as computer.properties. It supplies job ID, worker count, Pod addresses, and etcd when unspecified. Unspecified or zero transfer/RPC ports become 8099/8190; transport.server_host and rpc.server_host become Pod IPs. Job startup scripts substitute only ${POD_IP}, ${HOSTNAME}, ${POD_NAME}, and ${POD_NAMESPACE}.
• Kubernetes job images must contain the Computer runtime. Bundle algorithm JARs and select them with jarFile, or use an HTTP(S) remoteJarUri for startup download. See the Computer Quick Start and CRD table below.

Apache downloads provide versioned Computer source archives without separate precompiled binaries. Release 1.7.0 source and distribution names include -incubating-; run mvn clean package -DskipTests in the tagged computer/ project to build them. Current master omits that marker and takes its version from the POM. Source archives cannot be started as built distributions.

1. Basic Configuration

Core job settings for HugeGraph-Computer.

2. Algorithm Configuration

Algorithm-specific configuration for computation logic.

3. Input Configuration

Configuration for loading input data from HugeGraph or other sources.

3.1 Input Source

3.2 Input Splits

3.3 Input Processing

3.4 Input Performance

4. Snapshot & Storage Configuration

HugeGraph-Computer supports snapshot functionality to save vertex/edge partitions to local storage or MinIO object storage, enabling checkpoint recovery or accelerating repeated computations.

4.1 Basic Snapshot Configuration

4.2 MinIO Integration (Optional)

MinIO can be used as a distributed object storage backend for snapshots in K8s deployments.

Usage Scenarios:

• Checkpoint Recovery: Resume from snapshots after job failures, avoiding data reloading
• Repeated Computations: Load data from snapshots when running the same algorithm multiple times
• A/B Testing: Save multiple snapshot versions of the same dataset to test different algorithm parameters

Example: Local Snapshot (in computer.properties):

snapshot.write=true
snapshot.name=pagerank-snapshot-20260201

Example: MinIO Snapshot (in K8s CRD computerConf):

computerConf:
  snapshot.write: "true"
  snapshot.name: "pagerank-snapshot-v1"
  snapshot.minio_endpoint: "http://minio:9000"
  snapshot.minio_access_key: "my-access-key"
  snapshot.minio_secret_key: "my-secret-key"
  snapshot.minio_bucket_name: "hugegraph-snapshots"

5. Worker & Master Configuration

Configuration for worker and master computation logic.

5.1 Master Configuration

5.2 Worker Computation

5.3 Worker Combiners

5.4 Worker Buffers

5.5 Worker Data & Timeouts

6. I/O & Output Configuration

Configuration for output computation results.

6.1 Output Class & Result

6.2 Output Behavior

6.3 Batch Output

6.4 HDFS Output

6.5 Retry & Timeout

7. Network & Transport Configuration

Configuration for network communication between workers and master.

7.1 Server Configuration

7.2 Client Configuration

7.3 Protocol Configuration

7.4 Buffer Configuration

7.5 Flow Control

7.6 Timeouts

7.7 Heartbeat

7.8 Advanced Network Settings

7.9 Master RPC Configuration

These options come from HugeGraph Commons RPC configuration; the distribution template explicitly sets host and port. Kubernetes Operator advertises the Pod IP and defaults an unspecified or zero port to 8190.

8. Storage & Persistence Configuration

Configuration for HGKV (HugeGraph Key-Value) storage engine and value files.

8.1 HGKV Configuration

8.2 Value File Configuration

9. BSP & Coordination Configuration

Configuration for Bulk Synchronous Parallel (BSP) protocol and etcd coordination.

10. Performance Tuning Configuration

Configuration for performance optimization.

11. System Administration Configuration

Kubernetes Operator fills or overrides these options. Avoid overriding them unless customizing the Operator or network. job.namespace is optional user configuration; see the basic configuration table.

Why these values must match:

• BSP/RPC: Must match deployed etcd/RPC services for coordination.
• Job settings: Must match the CRD to provide the correct worker count.
• Transport: Workers need mutually reachable Pod IPs and ports; the K8s default is 8099.

K8s Operator Config Options

NOTE: Option needs to be converted through environment variable settings, e.g. k8s.internal_etcd_url => INTERNAL_ETCD_URL

HugeGraph-Computer CRD

1.7.0 CRD: https://github.com/apache/hugegraph-computer/blob/1.7.0/computer/computer-k8s-operator/manifest/hugegraph-computer-crd.v1.yaml

KubeDriver Config Options