Skip to content

This is the multi-page printable view of this section. .

Return to the regular view of this page.

HugeGraph (OLTP)

1 - ToplingDB

Historical integration. The 2025 articles below describe an earlier interface and are retained for reference. For the proposed replacement, see the ToplingDB development guide. That guide describes unreleased changes; do not mix its settings with these historical examples.

1.1 - ToplingDB Quick Start

Historical integration. This 2025 article describes an earlier interface and is retained for reference. For the proposed replacement, see the ToplingDB development guide. That guide describes unreleased changes; do not mix its settings with these historical examples.

ToplingDB is a configurable and observable extension of RocksDB. It supports dynamic tuning via YAML files and enables real-time monitoring through a built-in Web Server.

Update hugegraph.properties

1backend=rocksdb
2serializer=binary
3rocksdb.data_path=.
4rocksdb.wal_path=.
5# Path to YAML configuration file
6#  For security reasons, HG only allows YAML files to be located under the `conf/graphs` directory
7rocksdb.option_path=./conf/graphs/rocksdb_plus.yaml
8# Enable Web Server
9rocksdb.open_http=true

Initialize the database (required on the first startup, or a new configuration was manually added under ‘conf/graphs/’)

1cd *hugegraph-${version}
2bin/init-store.sh

Start server

1bin/start-hugegraph.sh
2Starting HugeGraphServer...
3Connecting to HugeGraphServer (http://127.0.0.1:8080/graphs)....OK

Verify ToplingDB Is Working Properly

Prerequisite: Ensure Core Parameters Are Correctly Configured

Check that hugegraph.properties includes:

1# Configuration file path
2rocksdb.option_path=./conf/graphs/rocksdb_plus.yaml
3# Enable Web Server
4rocksdb.open_http=true

Method 1: Access the Web Monitoring Interface

Visualize the Web monitoring page. Example screenshot:

image

Verify via terminal:

1# Access http://localhost:2011 (port depends on listening_ports in YAML config)
2curl http://localhost:2011 | grep topling

The following output indicates the page is working:

1<p><a href="https://topling.cn">Topling Inc.</a>This is <strong>Engine Inspector</strong>, for metrics, see <a href='javascript:grafana()'>Grafana</a>!</p>

Method 2: Check ToplingDB Initialization Info in Logs

1tail -f logs/hugegraph-server.log | grep -i topling

Similar output indicates ToplingDB is the storage engine:

12025-10-14 08:56:25 [db-open-1] [INFO] o.a.h.b.s.r.RocksDBStdSessions - SidePluginRepo found. Will attempt to open multi CFs RocksDB using Topling plugin.
22025-10-14 08:56:25 [db-open-1] [INFO] o.a.h.b.s.r.RocksDBStdSessions - Topling HTTP Server has been started according to the listening_ports specified in ./conf/graphs/rocksdb_plus.yaml

Common Troubleshooting

Issue 1: Startup Failure Due to YAML Format Error

Sample log during startup:

12025-10-15 01:55:50 [db-open-1] [INFO] o.a.h.b.s.r.RocksDBStdSessions - SidePluginRepo found. Will attempt to open multi CFs RocksDB using Topling plugin.
221:1: (891B):ERROR: 
3sideplugin/rockside/3rdparty/rapidyaml/src/c4/yml/parse.cpp:3310: ERROR parsing yml: parse error: incorrect indentation?

Solutions:

  1. Check YAML indentation (must use spaces, not tabs)
  2. Validate YAML syntax: python -c "import yaml; yaml.safe_load(open('conf/graphs/rocksdb_plus.yaml'))"
  3. Review specific error messages in logs

Issue 2: Web Server Port Conflict

Sample log during startup:

12025-10-15 01:57:34 [db-open-1] [INFO] o.a.h.b.s.r.RocksDBStdSessions - SidePluginRepo found. Will attempt to open multi CFs RocksDB using Topling plugin.
22025-10-15 01:57:34 [db-open-1] [ERROR] o.a.h.b.s.r.RocksDBStore - Failed to open RocksDB 'rocksdb-data/data/g'
3org.rocksdb.RocksDBException: rocksdb::Status rocksdb::SidePluginRepo::StartHttpServer(): null context when constructing CivetServer. Possible problem binding to port.
4    at org.rocksdb.SidePluginRepo.startHttpServer(Native Method) ~[rocksdbjni-8.10.2-20250804.074027-4.jar:?]

Solutions:

  1. Check if the port is occupied: lsof -i :2011
  2. Modify listening_ports in the YAML file
  3. Restart HugeGraph Server

Issue 3: Database Initialization Failure

Sample output indicating lock acquisition failure, possibly due to write permission issues or another process locking the DB:

1Caused by: org.rocksdb.RocksDBException: While lock file: rocksdb-data/data/m/LOCK: Resource temporarily unavailable
2        at org.rocksdb.SidePluginRepo.nativeOpenDBMultiCF(Native Method)
3        at org.rocksdb.SidePluginRepo.openDB(SidePluginRepo.java:22)

Solutions:

  1. Confirm correct config path: rocksdb.option_path=./conf/graphs/rocksdb_plus.yaml
  2. Check data directory permissions: ensure read/write access for the running user
  3. Review detailed logs: bin/init-store.sh 2>&1 | tee init.log

Related Documentation

1.2 - ToplingDB YAML configuration file

Historical integration. This 2025 article describes an earlier interface and is retained for reference. For the proposed replacement, see the ToplingDB development guide. That guide describes unreleased changes; do not mix its settings with these historical examples.

RocksDB provides a rich set of configuration parameters, but most of them typically require hardcoded setup.

ToplingDB introduces a SidePlugin + YAML mechanism, making configuration more modular and composable.

This document focuses on the extended parameters of ToplingDB, helping readers understand their purpose and usage.

0. rocksdb_plus.yaml used in HugeGraph

The following includes only the configuration parameters used in HugeGraph. For the full configuration supported by ToplingDB, refer to the SidePlugin Wiki:

  1http: # Web Server related configuration
  2  # normally parent path of db path
  3  document_root: /dev/shm/rocksdb_resource # Static resource directory, extracted by `preload_topling.sh` in HugeGraph
  4  listening_ports: '127.0.0.1:2011' # Web Server listening port for management/monitoring
  5
  6setenv: # Environment variable settings
  7  StrSimpleEnvNameNotOverwrite: StringValue
  8  IntSimpleEnvNameNotOverwrite: 16384
  9  OverwriteThisEnv:
 10    #comment: overwrite is default to false
 11    overwrite: true
 12    value: force overwrite this env by overwrite true
 13
 14Cache: # Cache configuration
 15  lru_cache: # Define an LRU cache instance
 16    class: LRUCache
 17    params: 
 18      capacity: 8G # Cache capacity: 8GB
 19      num_shard_bits: -1 # Number of shards, -1 means auto
 20      strict_capacity_limit: false
 21      high_pri_pool_ratio: 0.5
 22      use_adaptive_mutex: false
 23      metadata_charge_policy: kFullChargeCacheMetadata # Metadata also counts toward cache capacity
 24
 25Statistics: # Sampling configuration
 26  stat:
 27    class: default
 28    params:
 29      discard_tickers: # Discarded tickers to reduce overhead
 30        - rocksdb.block.cache
 31        - rocksdb.block.cachecompressed
 32        - rocksdb.block
 33        - rocksdb.memtable.payload.bytes.at.flush
 34        - rocksdb.memtable.garbage.bytes.at.flush
 35        - rocksdb.txn
 36        - rocksdb.blobdb
 37        - rocksdb.row.cache
 38        - rocksdb.number.block
 39        - rocksdb.bloom.filter
 40        - rocksdb.persistent
 41        - rocksdb.sim.block.cache
 42      discard_histograms: # Discarded histogram metrics
 43        # comment: ....
 44        - rocksdb.blobdb
 45        - rocksdb.bytes.compressed
 46        - rocksdb.bytes.decompressed
 47        - rocksdb.num.index.and.filter.blocks.read.per.level
 48        - rocksdb.num.data.blocks.read.per.level
 49        - rocksdb.compression.times.nanos
 50        - rocksdb.decompression.times.nanos
 51        - rocksdb.read.block.get.micros
 52        - rocksdb.write.raw.block.micros
 53      # comment end of array
 54      #stats_level: kAll
 55      stats_level: kDisableAll  # Disable all statistics
 56
 57MemTableRepFactory: # MemTable implementation in memory
 58  cspp: # ToplingDB-specific high-concurrency memory structure
 59    class: cspp
 60    params:
 61      mem_cap: 16G # Preallocate sufficient virtual memory space; may reserve address space without actual allocation
 62      use_vm: false
 63      token_use_idle: true
 64      chunk_size: 16K # Internal allocation granularity
 65      convert_to_sst: kFileMmap # Convert MemTable directly to SST, skipping flush; options: {kDontConvert, kDumpMem, kFileMmap}
 66      sync_sst_file: false # Whether to fsync after SST conversion when using kFileMmap
 67
 68  skiplist: # Default skiplist structure in RocksDB
 69    class: SkipList
 70    params:
 71      lookahead: 0
 72
 73TableFactory:
 74  cspp_memtab_sst:
 75    class: CSPPMemTabTable # TableFactory paired with cspp
 76    params: # empty params
 77
 78  bb:
 79    class: BlockBasedTable # Default block-based table in RocksDB
 80    params:
 81      checksum: kCRC32c
 82      block_size: 4K
 83      block_restart_interval: 16
 84      index_block_restart_interval: 1
 85      metadata_block_size: 4K
 86      enable_index_compression: true
 87      block_cache: "${lru_cache}" # Use the LRU cache defined above
 88      block_cache_compressed:
 89      persistent_cache:
 90      filter_policy:
 91
 92  dispatch:
 93    class: DispatcherTable
 94    params:
 95      default: bb # Default to BlockBasedTable
 96      readers:
 97        BlockBasedTable: bb
 98        CSPPMemTabTable: cspp_memtab_sst
 99      level_writers: [ bb, bb, bb, bb, bb, bb ] # Custom write strategy per level
100
101CFOptions:
102  default:
103    max_write_buffer_number: 6
104    memtable_factory: "${cspp}" # Reference cspp defined above
105    write_buffer_size: 128M
106    # set target_file_size_base as small as 512K is to make many SST files,
107    # thus key prefix cache can present efficiency
108    target_file_size_base: 64M
109    target_file_size_multiplier: 1
110    table_factory: dispatch # Reference dispatch defined above
111    max_bytes_for_level_base: 512M
112    max_bytes_for_level_multiplier: 10
113    level_compaction_dynamic_level_bytes: false
114    level0_slowdown_writes_trigger: 20
115    level0_stop_writes_trigger: 36
116    level0_file_num_compaction_trigger: 2
117    merge_operator: uint64add # support merge
118    level_compaction_dynamic_file_size: true
119    optimize_filters_for_hits: true
120    allow_merge_memtables: true
121    min_write_buffer_number_to_merge: 2
122    compression_per_level:
123      - kNoCompression
124      - kNoCompression
125      - kSnappyCompression
126      - kSnappyCompression
127      - kSnappyCompression
128      - kSnappyCompression
129      - kSnappyCompression
130
131DBOptions:
132  dbo:
133    create_if_missing: true
134    create_missing_column_families: false # this is important, must be false to hugegraph
135    max_background_compactions: -1
136    max_subcompactions: 4
137    max_level1_subcompactions: 0
138    inplace_update_support: false
139    WAL_size_limit_MB: 0
140    statistics: "${stat}"  # Use the statistics config defined above
141    max_manifest_file_size: 100M
142    max_background_jobs: 8
143    compaction_readahead_size: 0
144    memtable_as_log_index: true # Combined with convert_to_sst: kFileMmap to enable [omit L0 Flush](https://github.com/topling/toplingdb/wiki/Omit-L0-Flush)

Key points:

  • listening_ports: '127.0.0.1:2011' sets the Web Server listening port to 2011 and restricts access to localhost.
  • memtable_as_log_index: true combined with convert_to_sst: kFileMmap enables omit L0 Flush.
  • memtable_factory: "${cspp}" specifies the memory structure as CSPP Memtable.
  • table_factory: dispatch sets the TableFactory to the custom DispatcherTable defined in YAML.

1. Plugin-based configuration and reference mechanism

  • YAML modularization: The configuration file is organized as objects; each object can be defined independently and referenced elsewhere.
  • Reference syntax: Objects can be reused across sections via ${lru_cache}, ${cspp}, etc.
  • DispatcherTable: Allows selecting different TableFactories at different levels or scenarios. RocksDB natively supports only a single TableFactory.

ToplingDB YAML Reference and Reuse Diagram:

ToplingDB YAML Reference Diagram

This mechanism makes configuration more flexible and easier to compose in complex scenarios.

2. New MemTable implementation: CSPP

ToplingDB provides a MemTable type that RocksDB does not natively have, configured with the following parameters:

mem_cap

mem_cap is the size of the virtual address space reserved for CSPP. This may be just reserved address space without actual physical allocation. The actual memory usage of mem_cap is approximately equal to write_buffer_size.

Background of mem_cap Design

The underlying algorithm of CSPP adopts a pre-allocation strategy to support high-concurrency writes. Once the pre-allocated memory is filled, no further writes can proceed. However, RocksDB itself lacks a mechanism that allows a memtable to actively report “the pre-allocated memory is full, a new memtable is required”. Due to the complexity of its call chain, it is impractical to refactor RocksDB to add this capability. Therefore, CSPP adapts to RocksDB’s behavior through parameter design.

Core Idea of mem_cap

ToplingDB sets mem_cap to be much larger than write_buffer_size, so that RocksDB will not prematurely trigger an “out of memory” error when writing to a memtable. During CSPP initialization (New), the system rechecks the setting. If mem_cap is found to be too small, it will be automatically adjusted to 2 * write_buffer_size to ensure stability during the write process.

The default value is 2G, and the effective maximum is 16G.

  • Small deployments (<16GB RAM): set to 20–30% of system memory
  • Medium deployments (16–64GB RAM): set to 8–16G
  • Large deployments (>64GB RAM): set to 16G

use_vm

When allocating memory via malloc/posix_memalign, the address space may already be physically allocated (heap space with mapped pages), while CSPP only needs reserved virtual address space.
When use_vm is true, allocation is forced to use mmap, ensuring reserved address space without occupying physical pages.
The default is true. If physical memory is sufficient, it is recommended to disable this option—establishing mappings from mmap’s virtual memory to physical pages can trigger many minor page faults and may affect performance.

convert_to_sst

convert_to_sst supports three enum values:

  • kDontConvert: Disables the feature (default). Uses the traditional Flush process, offering the best compatibility for stability-focused scenarios.
  • kDumpMem: During conversion, dumps the entire MemTable memory to an SST file, reducing CPU consumption but not memory usage.
  • kFileMmap: mmaps MemTable content into a file—the key feature—reduces both CPU and memory usage. You can also set DBOptions.memtable_as_log_index = true to essentially eliminate MemTable Flush.

These parameters offer more tunable options for the write path, allowing users to choose as needed.

For more design details, see: cspp-memtable, ToplingDB CSPP MemTable Design Essentials, CSPP Trie Design Analysis.

3. TableFactory extensions

  • CSPPMemTabTable: The TableFactory paired with the cspp MemTable.
  • DispatcherTable: Supports specifying different TableFactories per level, for example:
    • Use BlockBasedTable by default.
    • Use CSPPMemTabTable for specific levels.

This level of flexibility is not available in native RocksDB configuration.

4. Statistics and observability controls

  • discard_tickers / discard_histograms: Precisely specify which statistics to discard.
  • stats_level: kDisableAll: Combined with the above, flexibly control the overhead of statistics.

Compared to RocksDB’s coarse-grained controls, ToplingDB offers finer tuning.

5. New DBOptions parameter

  • memtable_as_log_index: true: Allows using the MemTable as a log index to speed up recovery.

6. Security considerations

Web Server security:

  • The Web Server pages are provided by ToplingDB and do not include authentication.
  • By default, listening_ports: '127.0.0.1:2011' restricts access to local requests.
  • In production, configure firewall rules to allow only intranet access.
  • To disable the Web Server, set rocksdb.open_http=false in hugegraph.properties.

Shared memory safety:

  • document_root: /dev/shm/rocksdb_resource uses a shared memory directory.
  • In multi-user environments, ensure proper file permissions to avoid unauthorized access.

7. Summary

ToplingDB adds the following capabilities on top of RocksDB:

  • Plugin-based configuration and object reuse
  • A new MemTable type (cspp) with a paired TableFactory
  • DispatcherTable for multi-factory scheduling
  • Built-in Web Server
  • More flexible statistics and observability controls
  • Special DBOptions (such as memtable_as_log_index)

These extensions give users more tuning space, particularly suited for scenarios requiring high write performance and flexible operations.

Related Documentation