# HugeGraph-Computer Config LLMS index: [llms.txt](/versions/1.5/llms.txt) --- ### Computer Config Options | config option | default value | description | |-----------------------------------------|-------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | algorithm.message_class | org.apache.hugegraph.computer.core.config.Null | The class of message passed when compute vertex. | | algorithm.params_class | org.apache.hugegraph.computer.core.config.Null | The class used to transfer algorithms' parameters before algorithm been run. | | algorithm.result_class | org.apache.hugegraph.computer.core.config.Null | The class of vertex's value, the instance is used to store computation result for the vertex. | | allocator.max_vertices_per_thread | 10000 | Maximum number of vertices per thread processed in each memory allocator | | bsp.etcd_endpoints | http://localhost:2379 | The end points to access etcd. | | bsp.log_interval | 30000 | The log interval(in ms) to print the log while waiting bsp event. | | bsp.max_super_step | 10 | The max super step of the algorithm. | | bsp.register_timeout | 300000 | The max timeout to wait for master and works to register. | | bsp.wait_master_timeout | 86400000 | The max timeout(in ms) to wait for master bsp event. | | bsp.wait_workers_timeout | 86400000 | The max timeout to wait for workers bsp event. | | hgkv.max_data_block_size | 65536 | The max byte size of hgkv-file data block. | | hgkv.max_file_size | 2147483648 | The max number of bytes in each hgkv-file. | | hgkv.max_merge_files | 10 | The max number of files to merge at one time. | | hgkv.temp_file_dir | /tmp/hgkv | This folder is used to store temporary files, temporary files will be generated during the file merging process. | | hugegraph.name | hugegraph | The graph name to load data and write results back. | | hugegraph.url | http://127.0.0.1:8080 | The hugegraph url to load data and write results back. | | input.edge_direction | OUT | The data of the edge in which direction is loaded, when the value is BOTH, the edges in both OUT and IN direction will be loaded. | | input.edge_freq | MULTIPLE | The frequency of edges can exist between a pair of vertices, allowed values: [SINGLE, SINGLE_PER_LABEL, MULTIPLE]. SINGLE means that only one edge can exist between a pair of vertices, use sourceId + targetId to identify it; SINGLE_PER_LABEL means that each edge label can exist one edge between a pair of vertices, use sourceId + edgelabel + targetId to identify it; MULTIPLE means that many edge can exist between a pair of vertices, use sourceId + edgelabel + sortValues + targetId to identify it. | | input.filter_class | org.apache.hugegraph.computer.core.input.filter.DefaultInputFilter | The class to create input-filter object, input-filter is used to Filter vertex edges according to user needs. | | input.loader_schema_path || The schema path of loader input, only takes effect when the input.source_type=loader is enabled | | input.loader_struct_path || The struct path of loader input, only takes effect when the input.source_type=loader is enabled | | input.max_edges_in_one_vertex | 200 | The maximum number of adjacent edges allowed to be attached to a vertex, the adjacent edges will be stored and transferred together as a batch unit. | | input.source_type | hugegraph-server | The source type to load input data, allowed values: ['hugegraph-server', 'hugegraph-loader'], the 'hugegraph-loader' means use hugegraph-loader load data from HDFS or file, if use 'hugegraph-loader' load data then please config 'input.loader_struct_path' and 'input.loader_schema_path'. | | input.split_fetch_timeout | 300 | The timeout in seconds to fetch input splits | | input.split_max_splits | 10000000 | The maximum number of input splits | | input.split_page_size | 500 | The page size for streamed load input split data | | input.split_size | 1048576 | The input split size in bytes | | job.id | local_0001 | The job id on Yarn cluster or K8s cluster. | | job.partitions_count | 1 | The partitions count for computing one graph algorithm job. | | job.partitions_thread_nums | 4 | The number of threads for partition parallel compute. | | job.workers_count | 1 | The workers count for computing one graph algorithm job. | | master.computation_class | org.apache.hugegraph.computer.core.master.DefaultMasterComputation | Master-computation is computation that can determine whether to continue next superstep. It runs at the end of each superstep on master. | | output.batch_size | 500 | The batch size of output | | output.batch_threads | 1 | The threads number used to batch output | | output.hdfs_core_site_path || The hdfs core site path. | | output.hdfs_delimiter | , | The delimiter of hdfs output. | | output.hdfs_kerberos_enable | false | Is Kerberos authentication enabled for Hdfs. | | output.hdfs_kerberos_keytab || The Hdfs's key tab file for kerberos authentication. | | output.hdfs_kerberos_principal || The Hdfs's principal for kerberos authentication. | | output.hdfs_krb5_conf | /etc/krb5.conf | Kerberos configuration file. | | output.hdfs_merge_partitions | true | Whether merge output files of multiple partitions. | | output.hdfs_path_prefix | /hugegraph-computer/results | The directory of hdfs output result. | | output.hdfs_replication | 3 | The replication number of hdfs. | | output.hdfs_site_path || The hdfs site path. | | output.hdfs_url | hdfs://127.0.0.1:9000 | The hdfs url of output. | | output.hdfs_user | hadoop | The hdfs user of output. | | output.output_class | org.apache.hugegraph.computer.core.output.LogOutput | The class to output the computation result of each vertex. Be called after iteration computation. | | output.result_name | value | The value is assigned dynamically by #name() of instance created by WORKER_COMPUTATION_CLASS. | | output.result_write_type | OLAP_COMMON | The result write-type to output to hugegraph, allowed values are: [OLAP_COMMON, OLAP_SECONDARY, OLAP_RANGE]. | | output.retry_interval | 10 | The retry interval when output failed | | output.retry_times | 3 | The retry times when output failed | | output.single_threads | 1 | The threads number used to single output | | output.thread_pool_shutdown_timeout | 60 | The timeout seconds of output threads pool shutdown | | output.with_adjacent_edges | false | Output the adjacent edges of the vertex or not | | output.with_edge_properties | false | Output the properties of the edge or not | | output.with_vertex_properties | false | Output the properties of the vertex or not | | sort.thread_nums | 4 | The number of threads performing internal sorting. | | transport.client_connect_timeout | 3000 | The timeout(in ms) of client connect to server. | | transport.client_threads | 4 | The number of transport threads for client. | | transport.close_timeout | 10000 | The timeout(in ms) of close server or close client. | | transport.finish_session_timeout | 0 | The timeout(in ms) to finish session, 0 means using (transport.sync_request_timeout * transport.max_pending_requests). | | transport.heartbeat_interval | 20000 | The minimum interval(in ms) between heartbeats on client side. | | transport.io_mode | AUTO | The network IO Mode, either 'NIO', 'EPOLL', 'AUTO', the 'AUTO' means selecting the property mode automatically. | | transport.max_pending_requests | 8 | The max number of client unreceived ack, it will trigger the sending unavailable if the number of unreceived ack >= max_pending_requests. | | transport.max_syn_backlog | 511 | The capacity of SYN queue on server side, 0 means using system default value. | | transport.max_timeout_heartbeat_count | 120 | The maximum times of timeout heartbeat on client side, if the number of timeouts waiting for heartbeat response continuously > max_heartbeat_timeouts the channel will be closed from client side. | | transport.min_ack_interval | 200 | The minimum interval(in ms) of server reply ack. | | transport.min_pending_requests | 6 | The minimum number of client unreceived ack, it will trigger the sending available if the number of unreceived ack < min_pending_requests. | | transport.network_retries | 3 | The number of retry attempts for network communication,if network unstable. | | transport.provider_class | org.apache.hugegraph.computer.core.network.netty.NettyTransportProvider | The transport provider, currently only supports Netty. | | transport.receive_buffer_size | 0 | The size of socket receive-buffer in bytes, 0 means using system default value. | | transport.recv_file_mode | true | Whether enable receive buffer-file mode, it will receive buffer write file from socket by zero-copy if enable. | | transport.send_buffer_size | 0 | The size of socket send-buffer in bytes, 0 means using system default value. | | transport.server_host | 127.0.0.1 | The server hostname or ip to listen on to transfer data. | | transport.server_idle_timeout | 360000 | The max timeout(in ms) of server idle. | | transport.server_port | 0 | The server port to listen on to transfer data. The system will assign a random port if it's set to 0. | | transport.server_threads | 4 | The number of transport threads for server. | | transport.sync_request_timeout | 10000 | The timeout(in ms) to wait response after sending sync-request. | | transport.tcp_keep_alive | true | Whether enable TCP keep-alive. | | transport.transport_epoll_lt | false | Whether enable EPOLL level-trigger. | | transport.write_buffer_high_mark | 67108864 | The high water mark for write buffer in bytes, it will trigger the sending unavailable if the number of queued bytes > write_buffer_high_mark. | | transport.write_buffer_low_mark | 33554432 | The low water mark for write buffer in bytes, it will trigger the sending available if the number of queued bytes < write_buffer_low_mark.org.apache.hugegraph.config.OptionChecker$$Lambda$97/0x00000008001c8440@776a6d9b | | transport.write_socket_timeout | 3000 | The timeout(in ms) to write data to socket buffer. | | valuefile.max_segment_size | 1073741824 | The max number of bytes in each segment of value-file. | | worker.combiner_class | org.apache.hugegraph.computer.core.config.Null | Combiner can combine messages into one value for a vertex, for example page-rank algorithm can combine messages of a vertex to a sum value. | | worker.computation_class | org.apache.hugegraph.computer.core.config.Null | The class to create worker-computation object, worker-computation is used to compute each vertex in each superstep. | | worker.data_dirs | [jobs] | The directories separated by ',' that received vertices and messages can persist into. | | worker.edge_properties_combiner_class | org.apache.hugegraph.computer.core.combiner.OverwritePropertiesCombiner | The combiner can combine several properties of the same edge into one properties at inputstep. | | worker.partitioner | org.apache.hugegraph.computer.core.graph.partition.HashPartitioner | The partitioner that decides which partition a vertex should be in, and which worker a partition should be in. | | worker.received_buffers_bytes_limit | 104857600 | The limit bytes of buffers of received data, the total size of all buffers can't excess this limit. If received buffers reach this limit, they will be merged into a file. | | worker.vertex_properties_combiner_class | org.apache.hugegraph.computer.core.combiner.OverwritePropertiesCombiner | The combiner can combine several properties of the same vertex into one properties at inputstep. | | worker.wait_finish_messages_timeout | 86400000 | The max timeout(in ms) message-handler wait for finish-message of all workers. | | worker.wait_sort_timeout | 600000 | The max timeout(in ms) message-handler wait for sort-thread to sort one batch of buffers. | | worker.write_buffer_capacity | 52428800 | The initial size of write buffer that used to store vertex or message. | | worker.write_buffer_threshold | 52428800 | The threshold of write buffer, exceeding it will trigger sorting, the write buffer is used to store vertex or message. | ### K8s Operator Config Options > NOTE: Option needs to be converted through environment variable settings, e.g. k8s.internal_etcd_url => INTERNAL_ETCD_URL | config option | default value | description | |------------------------------|---------------------------|---------------------------------------------------------------------------------------------------------------------------------| | k8s.auto_destroy_pod | true | Whether to automatically destroy all pods when the job is completed or failed. | | k8s.close_reconciler_timeout | 120 | The max timeout(in ms) to close reconciler. | | k8s.internal_etcd_url | http://127.0.0.1:2379 | The internal etcd url for operator system. | | k8s.max_reconcile_retry | 3 | The max retry times of reconcile. | | k8s.probe_backlog | 50 | The maximum backlog for serving health probes. | | k8s.probe_port | 9892 | The value is the port that the controller bind to for serving health probes. | | k8s.ready_check_internal | 1000 | The time interval(ms) of check ready. | | k8s.ready_timeout | 30000 | The max timeout(in ms) of check ready. | | k8s.reconciler_count | 10 | The max number of reconciler thread. | | k8s.resync_period | 600000 | The minimum frequency at which watched resources are reconciled. | | k8s.timezone | Asia/Shanghai | The timezone of computer job and operator. | | k8s.watch_namespace | hugegraph-computer-system | The value is watch custom resources in the namespace, ignore other namespaces, the '*' means is all namespaces will be watched. | ### HugeGraph-Computer CRD > CRD: https://github.com/apache/hugegraph-computer/blob/master/computer-k8s-operator/manifest/hugegraph-computer-crd.v1.yaml | spec | default value | description | required | |-----------------|-------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------| | algorithmName | | The name of algorithm. | true | | jobId | | The job id. | true | | image | | The image of algorithm. | true | | computerConf | | The map of computer config options. | true | | workerInstances | | The number of worker instances, it will instead the 'job.workers_count' option. | true | | pullPolicy | Always | The pull-policy of image, detail please refer to: https://kubernetes.io/docs/concepts/containers/images/#image-pull-policy | false | | pullSecrets | | The pull-secrets of Image, detail please refer to: https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod | false | | masterCpu | | The cpu limit of master, the unit can be 'm' or without unit detail please refer to：[https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#meaning-of-cpu](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#meaning-of-cpu) | false | | workerCpu | | The cpu limit of worker, the unit can be 'm' or without unit detail please refer to：[https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#meaning-of-cpu](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#meaning-of-cpu) | false | | masterMemory | | The memory limit of master, the unit can be one of Ei、Pi、Ti、Gi、Mi、Ki detail please refer to：[https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#meaning-of-memory](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#meaning-of-memory) | false | | workerMemory | | The memory limit of worker, the unit can be one of Ei、Pi、Ti、Gi、Mi、Ki detail please refer to：[https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#meaning-of-memory](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#meaning-of-memory) | false | | log4jXml | | The content of log4j.xml for computer job. | false | | jarFile | | The jar path of computer algorithm. | false | | remoteJarUri | | The remote jar uri of computer algorithm, it will overlay algorithm image. | false | | jvmOptions | | The java startup parameters of computer job. | false | | envVars | | please refer to: https://kubernetes.io/docs/tasks/inject-data-application/define-interdependent-environment-variables/ | false | | envFrom | | please refer to: https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/ | false | | masterCommand | bin/start-computer.sh | The run command of master, equivalent to 'Entrypoint' field of Docker. | false | | masterArgs | ["-r master", "-d k8s"] | The run args of master, equivalent to 'Cmd' field of Docker. | false | | workerCommand | bin/start-computer.sh | The run command of worker, equivalent to 'Entrypoint' field of Docker. | false | | workerArgs | ["-r worker", "-d k8s"] | The run args of worker, equivalent to 'Cmd' field of Docker. | false | | volumes | | Please refer to: https://kubernetes.io/docs/concepts/storage/volumes/ | false | | volumeMounts | | Please refer to: https://kubernetes.io/docs/concepts/storage/volumes/ | false | | secretPaths | | The map of k8s-secret name and mount path. | false | | configMapPaths | | The map of k8s-configmap name and mount path. | false | | podTemplateSpec | | Please refer to: https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-template-v1/#PodTemplateSpec | false | | securityContext | | Please refer to: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/ | false | ### KubeDriver Config Options | config option | default value | description | |----------------------------------|------------------------------------------|-----------------------------------------------------------| | k8s.build_image_bash_path || The path of command used to build image. | | k8s.enable_internal_algorithm | true | Whether enable internal algorithm. | | k8s.framework_image_url | hugegraph/hugegraph-computer:latest | The image url of computer framework. | | k8s.image_repository_password || The password for login image repository. | | k8s.image_repository_registry || The address for login image repository. | | k8s.image_repository_url | hugegraph/hugegraph-computer | The url of image repository. | | k8s.image_repository_username || The username for login image repository. | | k8s.internal_algorithm | [pageRank] | The name list of all internal algorithm. | | k8s.internal_algorithm_image_url | hugegraph/hugegraph-computer:latest | The image url of internal algorithm. | | k8s.jar_file_dir | /cache/jars/ | The directory where the algorithm jar to upload location. | | k8s.kube_config | ~/.kube/config | The path of k8s config file. | | k8s.log4j_xml_path || The log4j.xml path for computer job. | | k8s.namespace | hugegraph-computer-system | The namespace of hugegraph-computer system. | | k8s.pull_secret_names | [] | The names of pull-secret for pulling image. |