Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When deploying the application in Docker (locally, on cloud Linux, or in Render), both the worker and server containers fail to connect to Redis, resulting in an ECONNREFUSED error. #7768

Closed
eliasbiondo opened this issue Oct 16, 2024 · 18 comments

Comments

@eliasbiondo
Copy link

Bug Description

When deploying the application in Docker (locally, on cloud Linux, or in Render), both the worker and server containers fail to connect to Redis, resulting in an ECONNREFUSED error. The application logs show the following error:

[Nest] 7 - 10/16/2024, 10:27:38 PM ERROR [ExceptionHandler] connect ECONNREFUSED 127.0.0.1:6379
Error: connect ECONNREFUSED 127.0.0.1:6379
at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1495:16)

Technical inputs

  • The error suggests that the application is trying to connect to Redis on 127.0.0.1:6379, which might be incorrect in the context of Docker containers.
@Hitarthsheth07
Copy link
Contributor

Hitarthsheth07 commented Oct 17, 2024

This is because your redis is not running properly.

Follow this instructions: https://twenty.com/developers/local-setup#step-4:-set-up-a-redis-database-(cache)

Then try:
redis-cli

if you get an error message you're not connected to Redis server.
You can try restarting the redis server by the following command:

/etc/init.d/redis-server restart
OR
(In ubuntu): sudo service redis-server restart

Then try redis-cli again.

If you don't get error, type PING in cli. If Redis is up and running it will return PONG.

Also, join discord for asking questions: https://discord.gg/txbw7Uxe

@rosscdh-tpg
Copy link

rosscdh-tpg commented Oct 17, 2024

in docker latest
seems to be a hardcoded 127.0.0.1:6379 in the codebase as its not referring to REDIS_URL
in the compose file also dont need to expose 6379 as should all be in docker network

@rosscdh
Copy link

rosscdh commented Oct 17, 2024

I rebuilt docker image from source... same issue

please note the reference to 127.0.0.1 but the REDIS_URL is redis:6379

Screenshot 2024-10-17 at 18 56 31
Screenshot 2024-10-17 at 18 56 22

@FelixMalfait
Copy link
Member

Probably linked to https://github.com/twentyhq/twenty/pull/7736/files but I can't find what's wrong 🤔

@rosscdh
Copy link

rosscdh commented Oct 17, 2024

yep.. im looking too... it seems to ignore the ENV_VAR and i see no ref to 127.0.0.1 which imlpies perpas that the interface to the underlying library is wrong... but looks ok to me

@FelixMalfait
Copy link
Member

I guess the only different I see is REDIS_URL: ${REDIS_URL:-redis://localhost:6379} in the docker-compose which should have stayed REDIS_URL: ${REDIS_URL:-redis://redis:6379} to remain completely iso, but the variable you've set in .env should take precedence 🤔

@rosscdh
Copy link

rosscdh commented Oct 17, 2024

ive commented REDIS_URL out in the compose and in the .env and get the error its missing.. so... something.. WEIRD is going on.. maybe the underlying redis library doesnt like "redis://redis" which would be kinda....

@rosscdh
Copy link

rosscdh commented Oct 17, 2024

nope.. changed REDIS_URL=redis://redis.server:6379 and renamed redis service to the name.. .no joy..

twenty-server-1                | Error: connect ECONNREFUSED 127.0.0.1:6379
twenty-server-1                |     at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1495:16) {
twenty-server-1                |   errno: -111,
twenty-server-1                |   code: 'ECONNREFUSED',
twenty-server-1                |   syscall: 'connect',
twenty-server-1                |   address: '127.0.0.1',
twenty-server-1                |   port: 6379
twenty-server-1                | }

@rosscdh
Copy link

rosscdh commented Oct 17, 2024

@FelixMalfait perhaps redis/node-redis#2416 similar in the redis package you use?

@FelixMalfait
Copy link
Member

Aaaah I think I understood why!
You're pulling the 0.31 docker image which still uses the old env variable.
But if you use 1-click install or read the doc, it only talks about REDIS_URL which would only work on the unstable docker image tagged with "latest"

@stnby
Copy link

stnby commented Oct 17, 2024

Having same issue on fresh deployment. My theory is that this was merged without making a new release #7736
And broke deployment script.

Temporary fix is to add this to both server and worker containers:
REDIS_HOST: redis

@FelixMalfait
Copy link
Member

@stnby same comment at the same time haha :)

@FelixMalfait
Copy link
Member

We should update https://github.com/twentyhq/twenty/blob/main/install.sh#L48 to use the tagged release instead of the "main" branch. And we should probably update the docs on step 2 to replace curl -O https://raw.githubusercontent.com/twentyhq/twenty/main/packages/twenty-docker/docker-compose.yml by something https://github.com/twentyhq/twenty/tree/v{x.y.z} indicating {x.y.z} should be replaced by the version number

@rosscdh
Copy link

rosscdh commented Oct 17, 2024

TAG=v0.31.2

# POSTGRES_ADMIN_PASSWORD=replace_me_with_a_strong_password

PG_DATABASE_HOST=db:5432
REDIS_URL=redis://redis:6379/0
REDIS_HOST=redis

seems to work!

Good work I appreciate the input and well done, on what looks to be a fantastic CRM. I look forward to putting it through its paces

charlesBochet pushed a commit that referenced this issue Oct 19, 2024
## Description

- `REDIS_URL` is required 

<img width="1903" alt="Redis Required"
src="https://github.com/user-attachments/assets/5b53a4a6-29f4-4473-b7e4-52489f75204e">

---- 

- Closes #7849

- Might be related #7768

--- 

- Wasn't gracefully reseting database

``` 
   npx nx database:reset twenty-server
```
@Raphael-Quinones
Copy link

I'm curious as to why when I rebuilt the docker image from source, it doesn't work. But when pulling from the latest it does work when you apply all the fixes in this thread. Is the one on github not the same with the image on dockerhub?

@charlesBochet
Copy link
Member

there is a mismatch between our docker-container on main branch (pointing to REDIS_URL) and the latest docker image of twenty 0.31.3 (pointing to REDIS_HOST. This will be resolved by 0.32 end of the week
In the meantime, adding REDIS_HOST manually in the docker-compose is the way to go!

@FelixMalfait
Copy link
Member

Re-opening as I think we should solve the root cause in the 1-click install script (using main branch instead of version branch).

(Also fyi we realized REDIS_URL wasn't taken into account properly, a fix is being pushed today #7956)

@FelixMalfait
Copy link
Member

OK we can close I created this as a followup: #7968

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants