permissions issues running `pelias compose up` #30

ajithcofficial · 2018-10-20T13:05:18Z

I was trying to setup a pelias geocoding server using docker and docker-compose, I followed the guidelines in https://github.com/pelias/docker/blob/master/README.md . after downloading all the docker images i used the command pelias compose up to start the services and it shows like

[root@192 portland-metro]# pelias compose up
Creating network "pelias_default" with driver "bridge"
Creating pelias_openstreetmap ... done
Creating pelias_openaddresses ...
Creating pelias_api ...
Creating pelias_pip-service ...
Creating pelias_polylines ...
Creating pelias_elasticsearch ...
Creating pelias_whosonfirst ...
Creating pelias_interpolation ...
Creating pelias_fuzzy_tester ...
Creating pelias_openstreetmap ...
Creating pelias_transit ...
Creating pelias_schema ...
Creating pelias_placeholder ...

Then I checked status of services using command pelias compose ps and it shows like

[root@192 portland-metro]# pelias compose ps
Name Command State Ports

pelias_api ./bin/start Restarting
pelias_elasticsearch /docker-entrypoint.sh elas ... Restarting
pelias_fuzzy_tester ./bin/fuzzy-tester --help Exit 1
pelias_interpolation ./interpolate server /data ... Restarting
pelias_libpostal ./bin/wof-libpostal-server ... Up 0.0.0.0:4400->4400/tcp
pelias_openaddresses /bin/bash Exit 0
pelias_openstreetmap /bin/bash Exit 0
pelias_pip-service ./bin/start Restarting
pelias_placeholder ./cmd/server.sh Restarting
pelias_polylines /bin/bash Exit 0
pelias_schema /bin/bash Exit 0
pelias_transit /bin/bash Exit 0
pelias_whosonfirst /bin/bash Exit 0

Then I checked the log by command pelias compose logs and it gives a list of errors as in the attachment.
logs_output.txt

I can't trace the problem. Somebody please help

The text was updated successfully, but these errors were encountered:

brucellino · 2018-10-21T05:28:45Z

You probably missed the creation of the two directories, including /code in the default workflow. see https://github.com/pelias/docker#generic-build-workflow

ajithcofficial · 2018-10-21T09:05:20Z

You probably missed the creation of the two directories, including /code in the default workflow. see https://github.com/pelias/docker#generic-build-workflow

No, I created both /code and /data directories as in the workflow.
when I am trying to start elastic search it shows

[root@barinlab portland-metro]# pelias compose logs
Attaching to pelias_elasticsearch
elasticsearch_1 | chown: cannot read directory '/usr/share/elasticsearch/data': Permission denied
elasticsearch_1 | chown: cannot read directory '/usr/share/elasticsearch/data': Permission denied
elasticsearch_1 | chown: cannot read directory '/usr/share/elasticsearch/data': Permission denied

I tried changing ownership of both /usr/share/elasticsearch/data and /data/elasticsearch/ to 1000:1000 but still i can't start elastic search.

I can give you remote access if you can help me with this

missinglink · 2018-10-23T13:10:31Z

Hi @ajithcofficial

Many users are running the docker setup without issue, so I suspect there might be something unusual about your setup.

What operating system are you using? are you using any non-standard OS configuration we should be aware of?

What changes have you made to docker-compose.yaml pelias.json or .env?

Please post the contents of your .env file.

missinglink · 2018-10-23T13:17:37Z

This issue #31 sounds related, your issues may be because you are running as root.

cc/ @orangejulius

ajithcofficial · 2018-10-23T16:06:34Z

This issue #31 sounds related, your issues may be because you are running as root.

cc/ @orangejulius

thank you for your response. I solved the permission issue by running docker images as root and elastic search service started successfully. but now when i type command

pelias download all

it shows like

[root@barinlab portland-metro]# pelias download all
module.js:549
    throw err;
    ^

Error: Cannot find module '/code/pelias.json'
    at Function.Module._resolveFilename (module.js:547:15)
    at Function.Module._load (module.js:474:25)
    at Module.require (module.js:596:17)
    at require (internal/module.js:11:18)
    at getConfig (/code/pelias/openstreetmap/node_modules/pelias-config/index.js:54:21)
    at Object.generate (/code/pelias/openstreetmap/node_modules/pelias-config/index.js:28:18)
    at Object.<anonymous> (/code/pelias/openstreetmap/util/download_data.js:7:43)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
[root@barinlab portland-metro]#

Why this happening?

mattwelke · 2018-10-23T16:36:19Z

@ajithcofficial Apparently we're supposed to run Pelias as a non-root user, according to instructions I received in the issue I reported. I don't think you should continue your troubleshooting effort while running it as root.

orangejulius · 2018-10-23T16:48:24Z

Hey folks,

I just opened up #32 which adds a line to the "quickstart" script in the readme to set proper permissions.

It should work out of the box for most people, whether the quickstart script is run as root or not (I suggest not). Please let me know if it helps.

mattwelke · 2018-10-23T17:41:56Z

@orangejulius Will test this soon. ty

orangejulius · 2018-10-23T19:26:07Z

I was able to reproduce the error above from running pelias download all. It happens in the case where the pelias/dockerfiles code itself is owned by a different user than the one configured in the .env file, and thus the non-root processes in the Docker containers cannot read pelias.json.

So @ajithcofficial please try the script again running as a non-root user, and let us know how it goes.

ajithcofficial · 2018-10-24T17:27:08Z

OS : CentOS Linux release 7.5.1804 (Core)
node -v : v8.12.0
npm -v : v8.12.0
docker -v : Docker version 1.13.1, build 8633870/1.13.1

Thank you for your responses, but still I am not able to fix this issue.
If the "quickstart" script is intended to run as a non root user then my operating system will not allow an unprivileged user to create a directory inside the root directory ( /code and /data). So I made some changes to the "quickstart" script as bellow.

#!/bin/bash
set -x

# create directories
sudo mkdir /code /data

# set proper permissions. make sure the user matches your `DOCKER_USER` setting in `.env`
sudo chown 1000:1000 /code /data

# clone repo
cd /code
git clone https://github.com/pelias/docker.git
cd docker

# install pelias script
sudo ln -sf "$(pwd)/pelias" /usr/local/bin/pelias

# cwd
cd projects/portland-metro

# configure environment
sed -i '/DATA_DIR/d' .env
echo 'DATA_DIR=/data' >> .env

# run build
pelias compose pull
pelias elastic start
pelias elastic wait
pelias elastic create
pelias download all
pelias prepare all
pelias import all
pelias compose up

# optionally run tests
pelias test run

Then I reconfigured my docker for running docker command as a non-root user as specified in https://docs.docker.com/install/linux/linux-postinstall/ .

After that I run the "quickstart" script and after executing few instructions it gone to a waiting state for starting elastic search. So I stopped the script, run

peliase compose ps

command and found that elastic search service is in restarting state.

Again I checked the log and it shows

elasticsearch_1  | chown: cannot read directory '/usr/share/elasticsearch/data': Permission denied
elasticsearch_1  | chown: cannot read directory '/usr/share/elasticsearch/data': Permission denied
elasticsearch_1  | chown: cannot read directory '/usr/share/elasticsearch/data': Permission denied
elasticsearch_1  | chown: cannot read directory '/usr/share/elasticsearch/data': Permission denied
elasticsearch_1  | chown: cannot read directory '/usr/share/elasticsearch/data': Permission denied
elasticsearch_1  | chown: cannot read directory '/usr/share/elasticsearch/data': Permission denied
elasticsearch_1  | chown: cannot read directory '/usr/share/elasticsearch/data': Permission denied

I was already familiar with this issue, To solve this issue i changed elastic search's host config file
/var/lib/docker/containers/01125f5ba7faef41200b7673068c9253a0f0a26c836be933fd38c15032ae8647/hostconfig.json to run elastic search service with root privileges.

Then I restarted docker service and successfully started elastic search service.

After that I sequentially executed the following commands and each of them taken a few second to complete and print nothing on screen.

[ajith@barinlab portland-metro]$ pelias download all
[ajith@barinlab portland-metro]$ pelias prepare all
[ajith@barinlab portland-metro]$ pelias prepare all
[ajith@barinlab portland-metro]$ pelias import all
[ajith@barinlab portland-metro]$ pelias compose up
Creating pelias_libpostal ... done
Creating pelias_openaddresses ... 
Creating pelias_openstreetmap ... 
Creating pelias_schema ... 
Creating pelias_interpolation ... 
Creating pelias_fuzzy_tester ... 
Creating pelias_whosonfirst ... 
Creating pelias_polylines ... 
Recreating pelias_elasticsearch ... 
Creating pelias_pip-service ... 
Creating pelias_api ... 
Creating pelias_transit ... 
Recreating pelias_elasticsearch ... done

Then I again checked status of all services and it shows,

[ajith@barinlab portland-metro]$ pelias compose ps
        Name                      Command                 State                          Ports                     
-------------------------------------------------------------------------------------------------------------------
pelias_api             ./bin/start                      Restarting                                                 
pelias_elasticsearch   /docker-entrypoint.sh elas ...   Up           0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp
pelias_fuzzy_tester    ./bin/fuzzy-tester --help        Exit 1                                                     
pelias_interpolation   ./interpolate server /data ...   Restarting                                                 
pelias_libpostal       ./bin/wof-libpostal-server ...   Up           0.0.0.0:4400->4400/tcp                        
pelias_openaddresses   /bin/bash                        Exit 0                                                     
pelias_openstreetmap   /bin/bash                        Exit 0                                                     
pelias_pip-service     ./bin/start                      Restarting                                                 
pelias_placeholder     ./cmd/server.sh                  Restarting                                                 
pelias_polylines       /bin/bash                        Exit 0                                                     
pelias_schema          /bin/bash                        Exit 0                                                     
pelias_transit         /bin/bash                        Exit 0                                                     
pelias_whosonfirst     /bin/bash                        Exit 0

Only elastic search and libpostal services were in up state. So I checked the log and it shows the following error messages repeatedly for all services.

module.js:549
    throw err;
    ^
Error: Cannot find module '/code/pelias.json'
    at Function.Module._resolveFilename (module.js:547:15)
    at Function.Module._load (module.js:474:25)
    at Module.require (module.js:596:17)
    at require (internal/module.js:11:18)
    at getConfig (/code/pelias/placeholder/node_modules/pelias-config/index.js:54:21)
    at Object.generate (/code/pelias/placeholder/node_modules/pelias-config/index.js:28:18)
    at Object.<anonymous> (/code/pelias/placeholder/node_modules/pelias-logger/index.js:10:30)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)

I have no idea what i had done wrong?
please help.

orangejulius · 2018-10-24T18:59:56Z

Hi @ajithcofficial,
Please don't make a habit of at-mentioning folks that aren't on the Pelias team :)

As I recall, CentOS is one of those few distros where the default user id is not 1000. Can you share the output of

id -u `whoami`

to confirm?

If my hunch is correct, try again with the correct user id and taking the documentation section on DOCKER_USER into account. I'd also recommend running sudo rm -rf /code /data to ensure you're starting completely from scratch with correct permissions.

ajithcofficial · 2018-10-25T02:54:10Z

Hi @ajithcofficial,
Please don't make a habit of at-mentioning folks that aren't on the Pelias team :)

As I recall, CentOS is one of those few distros where the default user id is not 1000. Can you share the output of
id -u `whoami`
to confirm?

If my hunch is correct, try again with the correct user id and taking the documentation section on DOCKER_USER into account. I'd also recommend running sudo rm -rf /code /data to ensure you're starting completely from scratch with correct permissions.

[ajith@barinlab portland-metro]$ whoami
ajith
[ajith@barinlab portland-metro]$ id -u ajith
1000
[ajith@barinlab portland-metro]$ id -g ajith
1000

userid is same as specified in .env file

[ajith@barinlab portland-metro]$ cat .env 
COMPOSE_PROJECT_NAME=pelias
DOCKER_USER=1000
DATA_DIR=/data
[ajith@barinlab portland-metro]$

dr0i mentioned this issue May 21, 2019

DATA_DIR ignored when importing data for openstreetmap #95

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

permissions issues running `pelias compose up` #30

permissions issues running `pelias compose up` #30

ajithcofficial commented Oct 20, 2018 •

edited

Loading

[root@192 portland-metro]# pelias compose ps
Name Command State Ports

brucellino commented Oct 21, 2018

ajithcofficial commented Oct 21, 2018 •

edited

Loading

missinglink commented Oct 23, 2018

missinglink commented Oct 23, 2018

ajithcofficial commented Oct 23, 2018

mattwelke commented Oct 23, 2018 •

edited

Loading

orangejulius commented Oct 23, 2018 •

edited

Loading

mattwelke commented Oct 23, 2018

orangejulius commented Oct 23, 2018 •

edited

Loading

ajithcofficial commented Oct 24, 2018 •

edited

Loading

orangejulius commented Oct 24, 2018

ajithcofficial commented Oct 25, 2018 •

edited

Loading

permissions issues running pelias compose up #30

permissions issues running pelias compose up #30

Comments

ajithcofficial commented Oct 20, 2018 • edited Loading

[root@192 portland-metro]# pelias compose ps Name Command State Ports

brucellino commented Oct 21, 2018

ajithcofficial commented Oct 21, 2018 • edited Loading

missinglink commented Oct 23, 2018

missinglink commented Oct 23, 2018

ajithcofficial commented Oct 23, 2018

mattwelke commented Oct 23, 2018 • edited Loading

orangejulius commented Oct 23, 2018 • edited Loading

mattwelke commented Oct 23, 2018

orangejulius commented Oct 23, 2018 • edited Loading

ajithcofficial commented Oct 24, 2018 • edited Loading

orangejulius commented Oct 24, 2018

ajithcofficial commented Oct 25, 2018 • edited Loading

permissions issues running `pelias compose up` #30

permissions issues running `pelias compose up` #30

ajithcofficial commented Oct 20, 2018 •

edited

Loading

[root@192 portland-metro]# pelias compose ps
Name Command State Ports

ajithcofficial commented Oct 21, 2018 •

edited

Loading

mattwelke commented Oct 23, 2018 •

edited

Loading

orangejulius commented Oct 23, 2018 •

edited

Loading

orangejulius commented Oct 23, 2018 •

edited

Loading

ajithcofficial commented Oct 24, 2018 •

edited

Loading

ajithcofficial commented Oct 25, 2018 •

edited

Loading