Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Use nproc to get total number of cores
A next error occurred on ubuntu 14.04: ``` Exception in thread Thread-1: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner self.run() File "/usr/local/lib/python2.7/dist-packages/threading2/t2_base.py", line 331, in run self_run() File "/usr/lib/python2.7/threading.py", line 763, in run self.__target(*self.__args, **self.__kwargs) File "/usr/local/lib/python2.7/dist-packages/ferry/http/httpapi.py", line 48, in _stack_worker _allocate_new_worker(payload["_uuid"], payload) File "/usr/local/lib/python2.7/dist-packages/ferry/http/httpapi.py", line 476, in _allocate_new_worker new_stack=True) File "/usr/local/lib/python2.7/dist-packages/ferry/http/httpapi.py", line 215, in _allocate_backend replace = replace) File "/usr/local/lib/python2.7/dist-packages/ferry/docker/manager.py", line 1201, in allocate_storage args) File "/usr/local/lib/python2.7/dist-packages/ferry/docker/configfactory.py", line 129, in generate_storage_configuration return self._generate_configuration(uuid, container_info, service) File "/usr/local/lib/python2.7/dist-packages/ferry/docker/configfactory.py", line 76, in _generate_configuration return config_factory.apply(config, container_info) File "/usr/local/lib/python2.7/dist-packages/ferry/config/hadoop/hadoopconfig.py", line 513, in apply hadoop_dirs, hadoop_entry = self._apply_hadoop(config, hadoop_containers) File "/usr/local/lib/python2.7/dist-packages/ferry/config/hadoop/hadoopconfig.py", line 387, in _apply_hadoop self._generate_yarn_site(yarn_master, new_config_dir) File "/usr/local/lib/python2.7/dist-packages/ferry/config/hadoop/hadoopconfig.py", line 249, in _generate_yarn_site cores = self.system.get_num_cores() / 2 File "/usr/local/lib/python2.7/dist-packages/ferry/config/system/info.py", line 43, in get_num_cores return int(output.strip()) ValueError: invalid literal for int() with base 10: '2\n2' ``` On 2 core VM the output of the command is: ``` $ cat /proc/cpuinfo | grep cores | awk '{print $4}' 2 2 ``` nproc is a part of coreutil and is available on most unix systems.
- Loading branch information