HugeGraph Plugin mechanism and plug-in extension process

For plugins targeting the planned 1.8.0 shared-foundation API, follow the Java/SPI migration guide and recompile against matching artifacts. The examples below describe the 1.7.0 API.

Background

HugeGraph is not only open source and open, but also simple and easy to use. General users can easily add plug-in extension functions without changing the source code.
HugeGraph supports a variety of built-in storage backends, and also allows users to extend custom backends without changing the existing source code.
HugeGraph supports full-text search. The full-text search function involves word segmentation in various languages. Currently, there are 7 built-in word breakers (ansj, hanlp, smartcn, jieba, jcseg, mmseg4j, ikanalyzer), and it also allows users to expand custom word breakers without changing the existing source code.

Scalable dimension

Currently, the plug-in method provides extensions in the following dimensions:

backend storage
serializer
Custom configuration items
tokenizer

Plug-in implementation mechanism

Provide SPI interface: HugeGraph provides the HugeGraphPlugin interface, supporting pluggability via Java SPI.
Registration functions: HugeGraphPlugin exposes static registration methods: registerOptions(), registerBackend(), registerSerializer(), and registerAnalyzer().
Implement components: The developer implements the corresponding OptionHolder, BackendStoreProvider, AbstractSerializer, or Analyzer interfaces.
Register in plugin: The developer implements the register() method of HugeGraphPlugin to register implementation classes, declared in META-INF/services/.
Deploy and activate: Place the packaged JAR in the HugeGraph Server plugins/ directory, configure the custom options, and restart the service.

Plug-in implementation process example

1 Create a new maven project

1.1 Name the project name: hugegraph-plugin-demo

1.2 Add `hugegraph-core` Jar package dependencies

The Maven pom.xml is shown below. Match hugegraph.version to the target Server version; this example uses the current master version 1.7.0:

 1<?xml version="1.0" encoding="UTF-8"?>
 2
 3<project xmlns="http://maven.apache.org/POM/4.0.0"
 4         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 5         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
 6
 7    <modelVersion>4.0.0</modelVersion>
 8    <groupId>org.apache.hugegraph</groupId>
 9    <artifactId>hugegraph-plugin-demo</artifactId>
10    <version>1.0.0</version>
11    <packaging>jar</packaging>
12
13    <name>hugegraph-plugin-demo</name>
14
15    <properties>
16        <hugegraph.version>1.7.0</hugegraph.version>
17    </properties>
18
19    <dependencies>
20        <dependency>
21            <groupId>org.apache.hugegraph</groupId>
22            <artifactId>hugegraph-core</artifactId>
23            <version>${hugegraph.version}</version>
24        </dependency>
25    </dependencies>
26</project>

2 Realize extended functions

2.1 Extending a custom backend

2.1.1 Implement the interface BackendStoreProvider

Realizable interfaces: org.apache.hugegraph.backend.store.BackendStoreProvider
Or inherit an abstract class:org.apache.hugegraph.backend.store.AbstractBackendStoreProvider

Take the RocksDB backend RocksDBStoreProvider as an example:

 1public class RocksDBStoreProvider extends AbstractBackendStoreProvider {
 2
 3    protected String database() {
 4        return this.graph().toLowerCase();
 5    }
 6
 7    @Override
 8    protected BackendStore newSchemaStore(HugeConfig config, String store) {
 9        return new RocksDBStore.RocksDBSchemaStore(this, this.database(), store);
10    }
11
12    @Override
13    protected BackendStore newGraphStore(HugeConfig config, String store) {
14        return new RocksDBStore.RocksDBGraphStore(this, this.database(), store);
15    }
16
17    @Override
18    protected BackendStore newSystemStore(HugeConfig config, String store) {
19        return new RocksDBStore.RocksDBSystemStore(this, this.database(), store);
20    }
21
22    @Override
23    public String type() {
24        return "rocksdb";
25    }
26
27    @Override
28    public String driverVersion() {
29        return "1.11";
30    }
31}

2.1.2 Implement interface BackendStore

The main abstract methods of BackendStore are shown below; default helper methods are omitted:

 1public interface BackendStore {
 2    // Store name
 3    String store();
 4
 5    // Stored version
 6    String storedVersion();
 7
 8    // Database name
 9    String database();
10
11    // Get the parent provider
12    BackendStoreProvider provider();
13
14    // Get the system schema store
15    SystemSchemaStore systemSchemaStore();
16
17    // Whether it is the storage of schema
18    boolean isSchemaStore();
19
20    // Open/close database
21    void open(HugeConfig config);
22    void close();
23    boolean opened();
24
25    // Initialize/clear database
26    void init();
27    void clear(boolean clearSpace);
28    boolean initialized();
29
30    // Delete all data of database (keep table structure)
31    void truncate();
32
33    // Add/delete data
34    void mutate(BackendMutation mutation);
35
36    // Query data
37    Iterator<BackendEntry> query(Query query);
38    Iterator<Iterator<BackendEntry>> query(Iterator<Query> queries,
39                                          Function<Query, Query> queryWriter,
40                                          HugeGraph hugeGraph);
41    Number queryNumber(Query query);
42
43    // Transaction
44    void beginTx();
45    void commitTx();
46    void rollbackTx();
47
48    // Get metadata by key
49    <R> R metadata(HugeType type, String meta, Object[] args);
50
51    // Backend features
52    BackendFeatures features();
53
54    // Increase next id for specific type
55    void increaseCounter(HugeType type, long increment);
56
57    // Get current counter for a specific type
58    long getCounter(HugeType type);
59}

2.1.3 Extending custom serializers

The serializer must inherit the abstract class: org.apache.hugegraph.backend.serializer.AbstractSerializer ( implements GraphSerializer, SchemaSerializer) The main interface is defined as follows:

 1public interface GraphSerializer {
 2    BackendEntry writeVertex(HugeVertex vertex);
 3    BackendEntry writeOlapVertex(HugeVertex vertex);
 4    BackendEntry writeVertexProperty(HugeVertexProperty<?> prop);
 5    HugeVertex readVertex(HugeGraph graph, BackendEntry entry);
 6    BackendEntry writeEdge(HugeEdge edge);
 7    BackendEntry writeEdgeProperty(HugeEdgeProperty<?> prop);
 8    HugeEdge readEdge(HugeGraph graph, BackendEntry entry);
 9    CIter<Edge> readEdges(HugeGraph graph, BackendEntry bytesEntry);
10    BackendEntry writeIndex(HugeIndex index);
11    HugeIndex readIndex(HugeGraph graph, ConditionQuery query, BackendEntry entry);
12    BackendEntry writeId(HugeType type, Id id);
13    Query writeQuery(Query query);
14}
15
16public interface SchemaSerializer {
17    BackendEntry writeVertexLabel(VertexLabel vertexLabel);
18    VertexLabel readVertexLabel(HugeGraph graph, BackendEntry entry);
19    BackendEntry writeEdgeLabel(EdgeLabel edgeLabel);
20    EdgeLabel readEdgeLabel(HugeGraph graph, BackendEntry entry);
21    BackendEntry writePropertyKey(PropertyKey propertyKey);
22    PropertyKey readPropertyKey(HugeGraph graph, BackendEntry entry);
23    BackendEntry writeIndexLabel(IndexLabel indexLabel);
24    IndexLabel readIndexLabel(HugeGraph graph, BackendEntry entry);
25}

2.1.4 Extend custom configuration items

When adding a custom backend, it may be necessary to add new configuration items. The implementation process mainly includes:

Add a configuration item container class and implement the interface org.apache.hugegraph.config.OptionHolder
Provide a singleton method public static OptionHolder instance(), and call the method when the object is initialized OptionHolder.registerOptions()
Add configuration item declaration, single-value configuration item type is ConfigOption, multi-value configuration item type is ConfigListOption

Take the RocksDB configuration item definition as an example:

 1public class RocksDBOptions extends OptionHolder {
 2
 3    private RocksDBOptions() {
 4        super();
 5    }
 6
 7    private static volatile RocksDBOptions instance;
 8
 9    public static synchronized RocksDBOptions instance() {
10        if (instance == null) {
11            instance = new RocksDBOptions();
12            instance.registerOptions();
13        }
14        return instance;
15    }
16
17    public static final ConfigOption<String> DATA_PATH =
18            new ConfigOption<>(
19                    "rocksdb.data_path",
20                    "The path for storing data of RocksDB.",
21                    disallowEmpty(),
22                    "rocksdb-data/data"
23            );
24
25    public static final ConfigOption<String> WAL_PATH =
26            new ConfigOption<>(
27                    "rocksdb.wal_path",
28                    "The path for storing WAL of RocksDB.",
29                    disallowEmpty(),
30                    "rocksdb-data/wal"
31            );
32
33    public static final ConfigListOption<String> DATA_DISKS =
34            new ConfigListOption<>(
35                    "rocksdb.data_disks",
36                    false,
37                    "The optimized disks for storing data of RocksDB. " +
38                    "The format of each element: `STORE/TABLE: /path/disk`." +
39                    "Allowed keys are [g/vertex, g/edge_out, g/edge_in, " +
40                    "g/vertex_label_index, g/edge_label_index, " +
41                    "g/range_int_index, g/range_float_index, " +
42                    "g/range_long_index, g/range_double_index, " +
43                    "g/secondary_index, g/search_index, g/shard_index, " +
44                    "g/unique_index, g/olap]",
45                    null,
46                    String.class,
47                    ImmutableList.of()
48            );
49}

2.2 Extend custom tokenizer

The tokenizer needs to implement the interface org.apache.hugegraph.analyzer.Analyzer, take implementing a SpaceAnalyzer space tokenizer as an example.

 1package org.apache.hugegraph.plugin;
 2
 3import java.util.Arrays;
 4import java.util.HashSet;
 5import java.util.Set;
 6
 7import org.apache.hugegraph.analyzer.Analyzer;
 8
 9public class SpaceAnalyzer implements Analyzer {
10
11    @Override
12    public Set<String> segment(String text) {
13        return new HashSet<>(Arrays.asList(text.split(" ")));
14    }
15}

3. Implement the plug-in interface and register it

The plug-in registration entry is HugeGraphPlugin.register(), the custom plug-in must implement this interface method, and register the extension items defined above inside it. The interface org.apache.hugegraph.plugin.HugeGraphPlugin is defined as follows:

 1public interface HugeGraphPlugin {
 2
 3    String name();
 4
 5    void register();
 6
 7    String supportsMinVersion();
 8
 9    String supportsMaxVersion();
10}

And HugeGraphPlugin provides 4 static methods for registering extensions:

registerOptions(String name, String classPath): register configuration items
registerBackend(String name, String classPath): register backend (BackendStoreProvider)
registerSerializer(String name, String classPath): register serializer
registerAnalyzer(String name, String classPath): register tokenizer

The following is an example of registering the SpaceAnalyzer tokenizer:

 1package org.apache.hugegraph.plugin;
 2
 3public class DemoPlugin implements HugeGraphPlugin {
 4
 5    @Override
 6    public String name() {
 7        return "demo";
 8    }
 9
10    @Override
11    public void register() {
12        HugeGraphPlugin.registerAnalyzer("demo", SpaceAnalyzer.class.getName());
13    }
14
15    @Override
16    public String supportsMinVersion() {
17        return "1.7.0";
18    }
19
20    @Override
21    public String supportsMaxVersion() {
22        return "1.8.0";
23    }
24}

4. Configure SPI entry

Make sure the services directory exists: hugegraph-plugin-demo/resources/META-INF/services
Create a text file in the services directory: org.apache.hugegraph.plugin.HugeGraphPlugin
The content of the file is as follows: org.apache.hugegraph.plugin.DemoPlugin

5. Make Jar package

Through maven packaging, execute the command in the project directory mvn package, and a Jar package file will be generated in the target directory. Copy the Jar package to the plugins directory when using it, and restart the service to take effect.