跳转到主要内容

文档 1.5 版本已不再积极维护,你当前浏览的是一份归档快照。 最新文档请参阅 最新版本。

HugeGraph 配置

1 概述

配置文件的目录为 hugegraph-release/conf,所有关于服务和图本身的配置都在此目录下。

主要的配置文件包括:gremlin-server.yaml、rest-server.properties 和 hugegraph.properties

HugeGraphServer 内部集成了 GremlinServer 和 RestServer,而 gremlin-server.yaml 和 rest-server.properties 就是用来配置这两个 Server 的。

  • GremlinServer:GremlinServer 接受用户的 gremlin 语句,解析后转而调用 Core 的代码。
  • RestServer:提供 RESTful API,根据不同的 HTTP 请求,调用对应的 Core API,如果用户请求体是 gremlin 语句,则会转发给 GremlinServer,实现对图数据的操作。

下面对这三个配置文件逐一介绍。

2 gremlin-server.yaml

gremlin-server.yaml 文件默认的内容如下:

  1# host and port of gremlin server, need to be consistent with host and port in rest-server.properties
  2#host: 127.0.0.1
  3#port: 8182
  4
  5# Gremlin 查询中的超时时间(以毫秒为单位)
  6evaluationTimeout: 30000
  7
  8channelizer: org.apache.tinkerpop.gremlin.server.channel.WsAndHttpChannelizer
  9# 不要在此处设置图形,此功能将在支持动态添加图形后再进行处理
 10graphs: {
 11}
 12scriptEngines: {
 13  gremlin-groovy: {
 14    staticImports: [
 15      org.opencypher.gremlin.process.traversal.CustomPredicates.*',
 16      org.opencypher.gremlin.traversal.CustomFunctions.*
 17    ],
 18    plugins: {
 19      org.apache.hugegraph.plugin.HugeGraphGremlinPlugin: {},
 20      org.apache.tinkerpop.gremlin.server.jsr223.GremlinServerGremlinPlugin: {},
 21      org.apache.tinkerpop.gremlin.jsr223.ImportGremlinPlugin: {
 22        classImports: [
 23          java.lang.Math,
 24          org.apache.hugegraph.backend.id.IdGenerator,
 25          org.apache.hugegraph.type.define.Directions,
 26          org.apache.hugegraph.type.define.NodeRole,
 27          org.apache.hugegraph.traversal.algorithm.CollectionPathsTraverser,
 28          org.apache.hugegraph.traversal.algorithm.CountTraverser,
 29          org.apache.hugegraph.traversal.algorithm.CustomizedCrosspointsTraverser,
 30          org.apache.hugegraph.traversal.algorithm.CustomizePathsTraverser,
 31          org.apache.hugegraph.traversal.algorithm.FusiformSimilarityTraverser,
 32          org.apache.hugegraph.traversal.algorithm.HugeTraverser,
 33          org.apache.hugegraph.traversal.algorithm.JaccardSimilarTraverser,
 34          org.apache.hugegraph.traversal.algorithm.KneighborTraverser,
 35          org.apache.hugegraph.traversal.algorithm.KoutTraverser,
 36          org.apache.hugegraph.traversal.algorithm.MultiNodeShortestPathTraverser,
 37          org.apache.hugegraph.traversal.algorithm.NeighborRankTraverser,
 38          org.apache.hugegraph.traversal.algorithm.PathsTraverser,
 39          org.apache.hugegraph.traversal.algorithm.PersonalRankTraverser,
 40          org.apache.hugegraph.traversal.algorithm.SameNeighborTraverser,
 41          org.apache.hugegraph.traversal.algorithm.ShortestPathTraverser,
 42          org.apache.hugegraph.traversal.algorithm.SingleSourceShortestPathTraverser,
 43          org.apache.hugegraph.traversal.algorithm.SubGraphTraverser,
 44          org.apache.hugegraph.traversal.algorithm.TemplatePathsTraverser,
 45          org.apache.hugegraph.traversal.algorithm.steps.EdgeStep,
 46          org.apache.hugegraph.traversal.algorithm.steps.RepeatEdgeStep,
 47          org.apache.hugegraph.traversal.algorithm.steps.WeightedEdgeStep,
 48          org.apache.hugegraph.traversal.optimize.ConditionP,
 49          org.apache.hugegraph.traversal.optimize.Text,
 50          org.apache.hugegraph.traversal.optimize.TraversalUtil,
 51          org.apache.hugegraph.util.DateUtil,
 52          org.opencypher.gremlin.traversal.CustomFunctions,
 53          org.opencypher.gremlin.traversal.CustomPredicate
 54        ],
 55        methodImports: [
 56          java.lang.Math#*,
 57          org.opencypher.gremlin.traversal.CustomPredicate#*,
 58          org.opencypher.gremlin.traversal.CustomFunctions#*
 59        ]
 60      },
 61      org.apache.tinkerpop.gremlin.jsr223.ScriptFileGremlinPlugin: {
 62        files: [scripts/empty-sample.groovy]
 63      }
 64    }
 65  }
 66}
 67serializers:
 68  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1,
 69      config: {
 70        serializeResultToString: false,
 71        ioRegistries: [org.apache.hugegraph.io.HugeGraphIoRegistry]
 72      }
 73  }
 74  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0,
 75      config: {
 76        serializeResultToString: false,
 77        ioRegistries: [org.apache.hugegraph.io.HugeGraphIoRegistry]
 78      }
 79  }
 80  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV2d0,
 81      config: {
 82        serializeResultToString: false,
 83        ioRegistries: [org.apache.hugegraph.io.HugeGraphIoRegistry]
 84      }
 85  }
 86  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV3d0,
 87      config: {
 88        serializeResultToString: false,
 89        ioRegistries: [org.apache.hugegraph.io.HugeGraphIoRegistry]
 90      }
 91  }
 92metrics: {
 93  consoleReporter: {enabled: false, interval: 180000},
 94  csvReporter: {enabled: false, interval: 180000, fileName: ./metrics/gremlin-server-metrics.csv},
 95  jmxReporter: {enabled: false},
 96  slf4jReporter: {enabled: false, interval: 180000},
 97  gangliaReporter: {enabled: false, interval: 180000, addressingMode: MULTICAST},
 98  graphiteReporter: {enabled: false, interval: 180000}
 99}
100maxInitialLineLength: 4096
101maxHeaderSize: 8192
102maxChunkSize: 8192
103maxContentLength: 65536
104maxAccumulationBufferComponents: 1024
105resultIterationBatchSize: 64
106writeBufferLowWaterMark: 32768
107writeBufferHighWaterMark: 65536
108ssl: {
109  enabled: false
110}

上面的配置项很多,但目前只需要关注如下几个配置项:channelizer 和 graphs。

  • graphs:GremlinServer 启动时需要打开的图,该项是一个 map 结构,key 是图的名字,value 是该图的配置文件路径;
  • channelizer:GremlinServer 与客户端有两种通信方式,分别是 WebSocket 和 HTTP(默认)。如果选择 WebSocket, 用户可以通过 Gremlin-Console 快速体验 HugeGraph 的特性,但是不支持大规模数据导入, 推荐使用 HTTP 的通信方式,HugeGraph 的外围组件都是基于 HTTP 实现的;

默认 GremlinServer 是服务在 localhost:8182,如果需要修改,配置 host、port 即可

  • host:部署 GremlinServer 机器的机器名或 IP,目前 HugeGraphServer 不支持分布式部署,且 GremlinServer 不直接暴露给用户;
  • port:部署 GremlinServer 机器的端口;

同时需要在 rest-server.properties 中增加对应的配置项 gremlinserver.url=http://host:port

3 rest-server.properties

rest-server.properties 文件的默认内容如下:

 1# bind url
 2# could use '0.0.0.0' or specified (real)IP to expose external network access
 3restserver.url=http://127.0.0.1:8080
 4#restserver.enable_graphspaces_filter=false
 5# gremlin server url, need to be consistent with host and port in gremlin-server.yaml
 6#gremlinserver.url=http://127.0.0.1:8182
 7
 8graphs=./conf/graphs
 9
10# The maximum thread ratio for batch writing, only take effect if the batch.max_write_threads is 0
11batch.max_write_ratio=80
12batch.max_write_threads=0
13
14# configuration of arthas
15arthas.telnet_port=8562
16arthas.http_port=8561
17arthas.ip=127.0.0.1
18arthas.disabled_commands=jad
19
20# authentication configs
21# choose 'org.apache.hugegraph.auth.StandardAuthenticator' or
22# 'org.apache.hugegraph.auth.ConfigAuthenticator'
23#auth.authenticator=
24
25# for StandardAuthenticator mode
26#auth.graph_store=hugegraph
27# auth client config
28#auth.remote_url=127.0.0.1:8899,127.0.0.1:8898,127.0.0.1:8897
29
30# for ConfigAuthenticator mode
31#auth.admin_token=
32#auth.user_tokens=[]
33
34# TODO: Deprecated & removed later (useless from version 1.5.0)
35# rpc server configs for multi graph-servers or raft-servers
36#rpc.server_host=127.0.0.1
37#rpc.server_port=8091
38#rpc.server_timeout=30
39
40# rpc client configs (like enable to keep cache consistency)
41#rpc.remote_url=127.0.0.1:8091,127.0.0.1:8092,127.0.0.1:8093
42#rpc.client_connect_timeout=20
43#rpc.client_reconnect_period=10
44#rpc.client_read_timeout=40
45#rpc.client_retries=3
46#rpc.client_load_balancer=consistentHash
47
48# raft group initial peers
49#raft.group_peers=127.0.0.1:8091,127.0.0.1:8092,127.0.0.1:8093
50
51# lightweight load balancing (beta)
52server.id=server-1
53server.role=master
54
55# slow query log
56log.slow_query_threshold=1000
57
58# jvm(in-heap) memory usage monitor, set 1 to disable it
59memory_monitor.threshold=0.85
60memory_monitor.period=2000
  • restserver.url:RestServer 提供服务的 url,根据实际环境修改。如果其他 IP 地址无法访问,可以尝试修改为特定的地址;或修改为 http://0.0.0.0 来监听来自任何 IP 地址的请求,这种方案较为便捷,但需要留意服务可被访问的网络范围;
  • graphs:RestServer 启动时也需要打开图,该项为 map 结构,key 是图的名字,value 是该图的配置文件路径;

注意:gremlin-server.yaml 和 rest-server.properties 都包含 graphs 配置项,而 init-store 命令是根据 gremlin-server.yaml 的 graphs 下的图进行初始化的。

配置项 gremlinserver.url 是 GremlinServer 为 RestServer 提供服务的 url,该配置项默认为 http://localhost:8182,如需修改,需要和 gremlin-server.yaml 中的 host 和 port 相匹配;

4 hugegraph.properties

hugegraph.properties 是一类文件,因为如果系统存在多个图,则会有多个相似的文件。该文件用来配置与图存储和查询相关的参数,文件的默认内容如下:

 1# gremlin entrence to create graph
 2gremlin.graph=org.apache.hugegraph.HugeFactory
 3
 4# cache config
 5#schema.cache_capacity=100000
 6# vertex-cache default is 1000w, 10min expired
 7#vertex.cache_capacity=10000000
 8#vertex.cache_expire=600
 9# edge-cache default is 100w, 10min expired
10#edge.cache_capacity=1000000
11#edge.cache_expire=600
12
13# schema illegal name template
14#schema.illegal_name_regex=\s+|~.*
15
16#vertex.default_label=vertex
17
18backend=rocksdb
19serializer=binary
20
21store=hugegraph
22
23raft.mode=false
24raft.safe_read=false
25raft.use_snapshot=false
26raft.endpoint=127.0.0.1:8281
27raft.group_peers=127.0.0.1:8281,127.0.0.1:8282,127.0.0.1:8283
28raft.path=./raft-log
29raft.use_replicator_pipeline=true
30raft.election_timeout=10000
31raft.snapshot_interval=3600
32raft.backend_threads=48
33raft.read_index_threads=8
34raft.queue_size=16384
35raft.queue_publish_timeout=60
36raft.apply_batch=1
37raft.rpc_threads=80
38raft.rpc_connect_timeout=5000
39raft.rpc_timeout=60000
40
41# if use 'ikanalyzer', need download jar from 'https://github.com/apache/hugegraph-doc/raw/ik_binary/dist/server/ikanalyzer-2012_u6.jar' to lib directory
42search.text_analyzer=jieba
43search.text_analyzer_mode=INDEX
44
45# rocksdb backend config
46#rocksdb.data_path=/path/to/disk
47#rocksdb.wal_path=/path/to/disk
48
49# cassandra backend config
50cassandra.host=localhost
51cassandra.port=9042
52cassandra.username=
53cassandra.password=
54#cassandra.connect_timeout=5
55#cassandra.read_timeout=20
56#cassandra.keyspace.strategy=SimpleStrategy
57#cassandra.keyspace.replication=3
58
59# hbase backend config
60#hbase.hosts=localhost
61#hbase.port=2181
62#hbase.znode_parent=/hbase
63#hbase.threads_max=64
64
65# mysql backend config
66#jdbc.driver=com.mysql.jdbc.Driver
67#jdbc.url=jdbc:mysql://127.0.0.1:3306
68#jdbc.username=root
69#jdbc.password=
70#jdbc.reconnect_max_times=3
71#jdbc.reconnect_interval=3
72#jdbc.ssl_mode=false
73
74# postgresql & cockroachdb backend config
75#jdbc.driver=org.postgresql.Driver
76#jdbc.url=jdbc:postgresql://localhost:5432/
77#jdbc.username=postgres
78#jdbc.password=
79
80# palo backend config
81#palo.host=127.0.0.1
82#palo.poll_interval=10
83#palo.temp_dir=./palo-data
84#palo.file_limit_size=32

重点关注未注释的几项:

  • gremlin.graph:GremlinServer 的启动入口,用户不要修改此项;
  • backend:使用的后端存储,可选值有 memory、cassandra、scylladb、mysql、hbase、postgresql 和 rocksdb;
  • serializer:主要为内部使用,用于将 schema、vertex 和 edge 序列化到后端,对应的可选值为 text、cassandra、scylladb 和 binary;(注:rocksdb 后端值需是 binary,其他后端 backend 与 serializer 值需保持一致,如 hbase 后端该值为 hbase)
  • store:图存储到后端使用的数据库名,在 cassandra 和 scylladb 中就是 keyspace 名,此项的值与 GremlinServer 和 RestServer 中的图名并无关系,但是出于直观考虑,建议仍然使用相同的名字;
  • cassandra.host:backend 为 cassandra 或 scylladb 时此项才有意义,cassandra/scylladb 集群的 seeds;
  • cassandra.port:backend 为 cassandra 或 scylladb 时此项才有意义,cassandra/scylladb 集群的 native port;
  • rocksdb.data_path:backend 为 rocksdb 时此项才有意义,rocksdb 的数据目录
  • rocksdb.wal_path:backend 为 rocksdb 时此项才有意义,rocksdb 的日志目录
  • admin.token: 通过一个 token 来获取服务器的配置信息,例如:http://localhost:8080/graphs/hugegraph/conf?token=162f7848-0b6d-4faf-b557-3a0797869c55

5 多图配置

我们的系统是可以存在多个图的,并且各个图的后端可以不一样,比如图 hugegraph_rocksdb 和 hugegraph_mysql,其中 hugegraph_rocksdb 以 RocksDB 作为后端,hugegraph_mysql 以 MySQL 作为后端。

配置方法也很简单:

[可选]:修改 rest-server.properties

通过修改 rest-server.properties 中的 graphs 配置项来设置图的配置文件目录。默认配置为 graphs=./conf/graphs,如果想要修改为其它目录则调整 graphs 配置项,比如调整为 graphs=/etc/hugegraph/graphs,示例如下:

1graphs=./conf/graphs

在 conf/graphs 路径下基于 hugegraph.properties 修改得到 hugegraph_mysql_backend.properties 和 hugegraph_rocksdb_backend.properties

hugegraph_mysql_backend.properties 修改的部分如下:

 1backend=mysql
 2serializer=mysql
 3
 4store=hugegraph_mysql
 5
 6# mysql backend config
 7jdbc.driver=com.mysql.cj.jdbc.Driver
 8jdbc.url=jdbc:mysql://127.0.0.1:3306
 9jdbc.username=root
10jdbc.password=xxx
11jdbc.reconnect_max_times=3
12jdbc.reconnect_interval=3
13jdbc.ssl_mode=false

hugegraph_rocksdb_backend.properties 修改的部分如下:

1backend=rocksdb
2serializer=binary
3
4store=hugegraph_rocksdb

停止 Server,初始化执行 init-store.sh(为新的图创建数据库),重新启动 Server

1$ ./bin/stop-hugegraph.sh
 1$ ./bin/init-store.sh
 2
 3Initializing HugeGraph Store...
 42023-06-11 14:16:14 [main] [INFO] o.a.h.u.ConfigUtil - Scanning option 'graphs' directory './conf/graphs'
 52023-06-11 14:16:14 [main] [INFO] o.a.h.c.InitStore - Init graph with config file: ./conf/graphs/hugegraph_rocksdb_backend.properties
 6...
 72023-06-11 14:16:15 [main] [INFO] o.a.h.StandardHugeGraph - Graph 'hugegraph_rocksdb' has been initialized
 82023-06-11 14:16:15 [main] [INFO] o.a.h.c.InitStore - Init graph with config file: ./conf/graphs/hugegraph_mysql_backend.properties
 9...
102023-06-11 14:16:16 [main] [INFO] o.a.h.StandardHugeGraph - Graph 'hugegraph_mysql' has been initialized
112023-06-11 14:16:16 [main] [INFO] o.a.h.StandardHugeGraph - Close graph standardhugegraph[hugegraph_rocksdb]
12...
132023-06-11 14:16:16 [main] [INFO] o.a.h.HugeFactory - HugeFactory shutdown
142023-06-11 14:16:16 [hugegraph-shutdown] [INFO] o.a.h.HugeFactory - HugeGraph is shutting down
15Initialization finished.
1$ ./bin/start-hugegraph.sh
2
3Starting HugeGraphServer...
4Connecting to HugeGraphServer (http://127.0.0.1:8080/graphs)...OK
5Started [pid 21614]

查看创建的图:

1curl http://127.0.0.1:8080/graphs/
2
3{"graphs":["hugegraph_rocksdb","hugegraph_mysql"]}

查看某个图的信息:

1curl http://127.0.0.1:8080/graphs/hugegraph_mysql_backend
2
3{"name":"hugegraph_mysql","backend":"mysql"}
1curl http://127.0.0.1:8080/graphs/hugegraph_rocksdb_backend
2
3{"name":"hugegraph_rocksdb","backend":"rocksdb"}