HugeGraph 配置

Mon, 01 Jan 0001 00:00:00 +0000

1 概述

配置文件的目录为 hugegraph-release/conf，所有关于服务和图本身的配置都在此目录下。

HugeGraph 配置项

Mon, 01 Jan 0001 00:00:00 +0000

Gremlin Server 配置项

对应配置文件gremlin-server.yaml

config option	default value	description
host	127.0.0.1	The host or ip of Gremlin Server.
port	8182	The listening port of Gremlin Server.
graphs	hugegraph: conf/hugegraph.properties	The map of graphs with name and config file path.
scriptEvaluationTimeout	30000	The timeout for gremlin script execution(millisecond).
channelizer	org.apache.tinkerpop.gremlin.server.channel.HttpChannelizer	Indicates the protocol which the Gremlin Server provides service.
authentication	authenticator: org.apache.hugegraph.auth.StandardAuthenticator, config: {tokens: conf/rest-server.properties}	The authenticator and config(contains tokens path) of authentication mechanism.

Rest Server & API 配置项

对应配置文件rest-server.properties

HugeGraph 内置用户权限与扩展权限配置及使用

Mon, 01 Jan 0001 00:00:00 +0000

概述

HugeGraph 为了方便不同用户场景下的鉴权使用，目前内置了完备的StandardAuthenticator权限模式，支持多用户认证、以及细粒度的权限访问控制，采用基于“用户 - 用户组 - 操作 - 资源”的 4 层设计，灵活控制用户角色与权限 (支持多 GraphServer)

配置 HugeGraphServer 使用 https 协议

Mon, 01 Jan 0001 00:00:00 +0000

概述

HugeGraphServer 默认使用的是 http 协议，如果用户对请求的安全性有要求，可以配置成 https。

服务端配置

修改 conf/rest-server.properties 配置文件，将 restserver.url 的 schema 部分改为 https。

HugeGraph-Computer 配置

Mon, 01 Jan 0001 00:00:00 +0000

Computer Config Options

config option	default value	description
algorithm.message_class	org.apache.hugegraph.computer.core.config.Null	The class of message passed when compute vertex.
algorithm.params_class	org.apache.hugegraph.computer.core.config.Null	The class used to transfer algorithms’ parameters before algorithm been run.
algorithm.result_class	org.apache.hugegraph.computer.core.config.Null	The class of vertex’s value, the instance is used to store computation result for the vertex.
allocator.max_vertices_per_thread	10000	Maximum number of vertices per thread processed in each memory allocator
bsp.etcd_endpoints	http://localhost:2379	The end points to access etcd.
bsp.log_interval	30000	The log interval(in ms) to print the log while waiting bsp event.
bsp.max_super_step	10	The max super step of the algorithm.
bsp.register_timeout	300000	The max timeout to wait for master and works to register.
bsp.wait_master_timeout	86400000	The max timeout(in ms) to wait for master bsp event.
bsp.wait_workers_timeout	86400000	The max timeout to wait for workers bsp event.
hgkv.max_data_block_size	65536	The max byte size of hgkv-file data block.
hgkv.max_file_size	2147483648	The max number of bytes in each hgkv-file.
hgkv.max_merge_files	10	The max number of files to merge at one time.
hgkv.temp_file_dir	/tmp/hgkv	This folder is used to store temporary files, temporary files will be generated during the file merging process.
hugegraph.name	hugegraph	The graph name to load data and write results back.
hugegraph.url	http://127.0.0.1:8080	The hugegraph url to load data and write results back.
input.edge_direction	OUT	The data of the edge in which direction is loaded, when the value is BOTH, the edges in both OUT and IN direction will be loaded.
input.edge_freq	MULTIPLE	The frequency of edges can exist between a pair of vertices, allowed values: [SINGLE, SINGLE_PER_LABEL, MULTIPLE]. SINGLE means that only one edge can exist between a pair of vertices, use sourceId + targetId to identify it; SINGLE_PER_LABEL means that each edge label can exist one edge between a pair of vertices, use sourceId + edgelabel + targetId to identify it; MULTIPLE means that many edge can exist between a pair of vertices, use sourceId + edgelabel + sortValues + targetId to identify it.
input.filter_class	org.apache.hugegraph.computer.core.input.filter.DefaultInputFilter	The class to create input-filter object, input-filter is used to Filter vertex edges according to user needs.
input.loader_schema_path		The schema path of loader input, only takes effect when the input.source_type=loader is enabled
input.loader_struct_path		The struct path of loader input, only takes effect when the input.source_type=loader is enabled
input.max_edges_in_one_vertex	200	The maximum number of adjacent edges allowed to be attached to a vertex, the adjacent edges will be stored and transferred together as a batch unit.
input.source_type	hugegraph-server	The source type to load input data, allowed values: [‘hugegraph-server’, ‘hugegraph-loader’], the ‘hugegraph-loader’ means use hugegraph-loader load data from HDFS or file, if use ‘hugegraph-loader’ load data then please config ‘input.loader_struct_path’ and ‘input.loader_schema_path’.
input.split_fetch_timeout	300	The timeout in seconds to fetch input splits
input.split_max_splits	10000000	The maximum number of input splits
input.split_page_size	500	The page size for streamed load input split data
input.split_size	1048576	The input split size in bytes
job.id	local_0001	The job id on Yarn cluster or K8s cluster.
job.partitions_count	1	The partitions count for computing one graph algorithm job.
job.partitions_thread_nums	4	The number of threads for partition parallel compute.
job.workers_count	1	The workers count for computing one graph algorithm job.
master.computation_class	org.apache.hugegraph.computer.core.master.DefaultMasterComputation	Master-computation is computation that can determine whether to continue next superstep. It runs at the end of each superstep on master.
output.batch_size	500	The batch size of output
output.batch_threads	1	The threads number used to batch output
output.hdfs_core_site_path		The hdfs core site path.
output.hdfs_delimiter	,	The delimiter of hdfs output.
output.hdfs_kerberos_enable	false	Is Kerberos authentication enabled for Hdfs.
output.hdfs_kerberos_keytab		The Hdfs’s key tab file for kerberos authentication.
output.hdfs_kerberos_principal		The Hdfs’s principal for kerberos authentication.
output.hdfs_krb5_conf	/etc/krb5.conf	Kerberos configuration file.
output.hdfs_merge_partitions	true	Whether merge output files of multiple partitions.
output.hdfs_path_prefix	/hugegraph-computer/results	The directory of hdfs output result.
output.hdfs_replication	3	The replication number of hdfs.
output.hdfs_site_path		The hdfs site path.
output.hdfs_url	hdfs://127.0.0.1:9000	The hdfs url of output.
output.hdfs_user	hadoop	The hdfs user of output.
output.output_class	org.apache.hugegraph.computer.core.output.LogOutput	The class to output the computation result of each vertex. Be called after iteration computation.
output.result_name	value	The value is assigned dynamically by #name() of instance created by WORKER_COMPUTATION_CLASS.
output.result_write_type	OLAP_COMMON	The result write-type to output to hugegraph, allowed values are: [OLAP_COMMON, OLAP_SECONDARY, OLAP_RANGE].
output.retry_interval	10	The retry interval when output failed
output.retry_times	3	The retry times when output failed
output.single_threads	1	The threads number used to single output
output.thread_pool_shutdown_timeout	60	The timeout seconds of output threads pool shutdown
output.with_adjacent_edges	false	Output the adjacent edges of the vertex or not
output.with_edge_properties	false	Output the properties of the edge or not
output.with_vertex_properties	false	Output the properties of the vertex or not
sort.thread_nums	4	The number of threads performing internal sorting.
transport.client_connect_timeout	3000	The timeout(in ms) of client connect to server.
transport.client_threads	4	The number of transport threads for client.
transport.close_timeout	10000	The timeout(in ms) of close server or close client.
transport.finish_session_timeout	0	The timeout(in ms) to finish session, 0 means using (transport.sync_request_timeout * transport.max_pending_requests).
transport.heartbeat_interval	20000	The minimum interval(in ms) between heartbeats on client side.
transport.io_mode	AUTO	The network IO Mode, either ‘NIO’, ‘EPOLL’, ‘AUTO’, the ‘AUTO’ means selecting the property mode automatically.
transport.max_pending_requests	8	The max number of client unreceived ack, it will trigger the sending unavailable if the number of unreceived ack >= max_pending_requests.
transport.max_syn_backlog	511	The capacity of SYN queue on server side, 0 means using system default value.
transport.max_timeout_heartbeat_count	120	The maximum times of timeout heartbeat on client side, if the number of timeouts waiting for heartbeat response continuously > max_heartbeat_timeouts the channel will be closed from client side.
transport.min_ack_interval	200	The minimum interval(in ms) of server reply ack.
transport.min_pending_requests	6	The minimum number of client unreceived ack, it will trigger the sending available if the number of unreceived ack < min_pending_requests.
transport.network_retries	3	The number of retry attempts for network communication,if network unstable.
transport.provider_class	org.apache.hugegraph.computer.core.network.netty.NettyTransportProvider	The transport provider, currently only supports Netty.
transport.receive_buffer_size	0	The size of socket receive-buffer in bytes, 0 means using system default value.
transport.recv_file_mode	true	Whether enable receive buffer-file mode, it will receive buffer write file from socket by zero-copy if enable.
transport.send_buffer_size	0	The size of socket send-buffer in bytes, 0 means using system default value.
transport.server_host	127.0.0.1	The server hostname or ip to listen on to transfer data.
transport.server_idle_timeout	360000	The max timeout(in ms) of server idle.
transport.server_port	0	The server port to listen on to transfer data. The system will assign a random port if it’s set to 0.
transport.server_threads	4	The number of transport threads for server.
transport.sync_request_timeout	10000	The timeout(in ms) to wait response after sending sync-request.
transport.tcp_keep_alive	true	Whether enable TCP keep-alive.
transport.transport_epoll_lt	false	Whether enable EPOLL level-trigger.
transport.write_buffer_high_mark	67108864	The high water mark for write buffer in bytes, it will trigger the sending unavailable if the number of queued bytes > write_buffer_high_mark.
transport.write_buffer_low_mark	33554432	The low water mark for write buffer in bytes, it will trigger the sending available if the number of queued bytes < write_buffer_low_mark.org.apache.hugegraph.config.OptionChecker$$Lambda$97/0x00000008001c8440@776a6d9b
transport.write_socket_timeout	3000	The timeout(in ms) to write data to socket buffer.
valuefile.max_segment_size	1073741824	The max number of bytes in each segment of value-file.
worker.combiner_class	org.apache.hugegraph.computer.core.config.Null	Combiner can combine messages into one value for a vertex, for example page-rank algorithm can combine messages of a vertex to a sum value.
worker.computation_class	org.apache.hugegraph.computer.core.config.Null	The class to create worker-computation object, worker-computation is used to compute each vertex in each superstep.
worker.data_dirs	[jobs]	The directories separated by ‘,’ that received vertices and messages can persist into.
worker.edge_properties_combiner_class	org.apache.hugegraph.computer.core.combiner.OverwritePropertiesCombiner	The combiner can combine several properties of the same edge into one properties at inputstep.
worker.partitioner	org.apache.hugegraph.computer.core.graph.partition.HashPartitioner	The partitioner that decides which partition a vertex should be in, and which worker a partition should be in.
worker.received_buffers_bytes_limit	104857600	The limit bytes of buffers of received data, the total size of all buffers can’t excess this limit. If received buffers reach this limit, they will be merged into a file.
worker.vertex_properties_combiner_class	org.apache.hugegraph.computer.core.combiner.OverwritePropertiesCombiner	The combiner can combine several properties of the same vertex into one properties at inputstep.
worker.wait_finish_messages_timeout	86400000	The max timeout(in ms) message-handler wait for finish-message of all workers.
worker.wait_sort_timeout	600000	The max timeout(in ms) message-handler wait for sort-thread to sort one batch of buffers.
worker.write_buffer_capacity	52428800	The initial size of write buffer that used to store vertex or message.
worker.write_buffer_threshold	52428800	The threshold of write buffer, exceeding it will trigger sorting, the write buffer is used to store vertex or message.

K8s Operator Config Options

NOTE: Option needs to be converted through environment variable settings, e.g. k8s.internal_etcd_url => INTERNAL_ETCD_URL

Config on Apache HugeGraph

HugeGraph 配置

1 概述

HugeGraph 配置项

Gremlin Server 配置项

Rest Server & API 配置项

HugeGraph 内置用户权限与扩展权限配置及使用

概述

配置 HugeGraphServer 使用 https 协议

概述

服务端配置

HugeGraph-Computer 配置

Computer Config Options

K8s Operator Config Options