WebSocket connection closed: connection was closed uncleanly ("peer dropped the TCP connection without previous WebSocket closing handshake")

[0xshandy]

When I subscribe to userdata, since my account information doesn't change very often, the following warning appears and the TCP is dropped by the server. What can I do to avoid the subscription being interrupted by the server. By the way, the depth subscribe works fine

Issue subject

[2022-11-06 12:06:48 binance_client_protocol.py-WARNING]: WebSocket connection closed: connection was closed uncleanly ("peer dropped the TCP connection without previous WebSocket closing handshake"), code: 1006, clean: False, reason: connection was closed uncleanly ("peer dropped the TCP connection without previous WebSocket closing handshake")
[2022-11-06 12:06:48 binance_client_factory.py-ERROR]: Lost connection to Server. Reason: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionDone'>: Connection was closed cleanly.
].

Expected behaviour

subscribe UserData stream with no error

Actual behaviour

TCP is dropped by the server

Steps to reproduce

Receive userdata at low frequency

Environment

Provide any relevant information about your setup, such as:

• Version of binance-connector-python lastest
• Python version 3.10
• Operating system macOs

[0xshandy]

def sub_user_data(self):
    self.user_data(
        listen_key=self.listen_key,
        id=1,
        callback=self._user_data_handler,
    )

def _user_data_handler(self, msg):
    logging.info(msg)
    if msg.get('e') != 'executionReport':
        return
    if msg['X'] == 'FILLED':
        side = msg['S']
         ...

[Pet222]

Hi!

I am having issues also quite similar.....The recovery on my end is to restart the whole computer (ubuntu) since once the websocket is gone its gone forever.....python application stop->start does not solve the issue. And after restarting the application the websocket is still unseless....On my end I suspect that in my _user_data_handler there is a call aslo to binance via spot API which might take too long sometimes so maybe this could block some threads (not sure) and therefore the websocket is blocked too long....do you also have an SPOT API call in the code of the _user_data_handler after the "...." part?

[Pet222]

I debugged my code and let it run for quite a while. I am sure its not blocking any threads now and still the websocket stops sending messages and I get the same issue reported in your ticket....

"WebSocket connection closed: connection was closed uncleanly ("peer dropped the TCP connection without previous WebSocket closing handshake"), code: 1006, clean: False, reason: connection was closed uncleanly ("peer dropped the TCP connection without previous WebSocket closing handshake")" meanwhile the server does not block me: {'data': {'isLocked': False, 'plannedRecoverTime': 0, 'triggerCondition': {'UFR': 300, 'IFER': 150, 'GCR': 150}, 'updateTime': 0}}

[0xshandy]

I debugged my code and let it run for quite a while. I am sure its not blocking any threads now and still the websocket stops sending messages and I get the same issue reported in your ticket....

"WebSocket connection closed: connection was closed uncleanly ("peer dropped the TCP connection without previous WebSocket closing handshake"), code: 1006, clean: False, reason: connection was closed uncleanly ("peer dropped the TCP connection without previous WebSocket closing handshake")" meanwhile the server does not block me: {'data': {'isLocked': False, 'plannedRecoverTime': 0, 'triggerCondition': {'UFR': 300, 'IFER': 150, 'GCR': 150}, 'updateTime': 0}}

Hi sir:
After some observations, I found that the stream of userdata needs to remain active every 1 minute, which means that if your userdata does not change for more than 1 minute, the binance service will automatically disconnect the client. My solution is: in other active In the stream (for example: depth stream) send a new order (at a price that is unlikely to be traded) at intervals so that my userdata stream will not be disconnect, for your reference

[Pet222]

Hi! Thanks for you comment! Well I am querying the binance spot account every minute (which needs auth) meanwhile the websocket runs.....and after a while it stops sending messages. Do I understand it correctly from your comments that if I send any auth request to binance within 1 minute timeframes the websocket connection shall remain alive?

[0xshandy]

Hi! Thanks for you comment! Well I am querying the binance spot account every minute (which needs auth) meanwhile the websocket runs.....and after a while it stops sending messages. Do I understand it correctly from your comments that if I send any auth request to binance within 1 minute timeframes the websocket connection shall remain alive?

I think the userdata stream is only activated for requests involving account changes.

[Pet222]

Thx for the suggestion for the workaround. I'will try it....meanwhile I've checked the available pull requestes and it seems there is a fix for the websocket client...so I took it over (reactor.stop() was replaced) maybe this causes the socket issue with the closure...I have not check it so its a maybe...

Also I've checked the docu of binance and the websocket shall be kept alive for 24 hours without sending any userdata...

"A single connection to stream.binance.com is only valid for 24 hours; expect to be disconnected at the 24 hour mark
The websocket server will send a ping frame every 3 minutes. If the websocket server does not receive a pong frame back from the connection within a 10 minute period, the connection will be disconnected. Unsolicited pong frames are allowed."

The ping-pong is handled in the binance-connector ....

[Plotnychenko]

I debugged my code and let it run for quite a while. I am sure its not blocking any threads now and still the websocket stops sending messages and I get the same issue reported in your ticket....
"WebSocket connection closed: connection was closed uncleanly ("peer dropped the TCP connection without previous WebSocket closing handshake"), code: 1006, clean: False, reason: connection was closed uncleanly ("peer dropped the TCP connection without previous WebSocket closing handshake")" meanwhile the server does not block me: {'data': {'isLocked': False, 'plannedRecoverTime': 0, 'triggerCondition': {'UFR': 300, 'IFER': 150, 'GCR': 150}, 'updateTime': 0}}

Hi sir: After some observations, I found that the stream of userdata needs to remain active every 1 minute, which means that if your userdata does not change for more than 1 minute, the binance service will automatically disconnect the client. My solution is: in other active In the stream (for example: depth stream) send a new order (at a price that is unlikely to be traded) at intervals so that my userdata stream will not be disconnect, for your reference

Hi.
I cannot agree with the statement.
I have multiple connections running at the same time. But only one of them has this problem. The script is the same. The steps that are performed are the same. But on one connection there is this problem, on the others not.

[0xshandy]

I debugged my code and let it run for quite a while. I am sure its not blocking any threads now and still the websocket stops sending messages and I get the same issue reported in your ticket....
"WebSocket connection closed: connection was closed uncleanly ("peer dropped the TCP connection without previous WebSocket closing handshake"), code: 1006, clean: False, reason: connection was closed uncleanly ("peer dropped the TCP connection without previous WebSocket closing handshake")" meanwhile the server does not block me: {'data': {'isLocked': False, 'plannedRecoverTime': 0, 'triggerCondition': {'UFR': 300, 'IFER': 150, 'GCR': 150}, 'updateTime': 0}}

Hi sir: After some observations, I found that the stream of userdata needs to remain active every 1 minute, which means that if your userdata does not change for more than 1 minute, the binance service will automatically disconnect the client. My solution is: in other active In the stream (for example: depth stream) send a new order (at a price that is unlikely to be traded) at intervals so that my userdata stream will not be disconnect, for your reference

Hi. I cannot agree with the statement. I have multiple connections running at the same time. But only one of them has this problem. The script is the same. The steps that are performed are the same. But on one connection there is this problem, on the others not.

Hi sir, Do you mean subscribing to multiple streams by one ws client at the same time or multiple ws clients at the same time?

[Pet222]

hi gents! on my end it seems that the websocket ping-pong is fine. in case o pong from the server ping is sent. One time the socket died when after binance server sent a pong....(I did not find any description when it is done usually it sends a ping). But in an other case everything was fine (ping was sent and pong was replied etc) and shortly after the websocket died.

I am not sure if the same payload has to be sent back to the server (I've found no docu on this probably not....)
If so than binance-connect is not the reason....it might be the autobahn package.
Can you check the package version you have installed? It would be interesting which version you have...

Mine is:

import autobahn
autobahn.version
'22.7.1'

[0xshandy]

import autobahn
autobahn.version

22.7.1

[Plotnychenko]

I debugged my code and let it run for quite a while. I am sure its not blocking any threads now and still the websocket stops sending messages and I get the same issue reported in your ticket....
"WebSocket connection closed: connection was closed uncleanly ("peer dropped the TCP connection without previous WebSocket closing handshake"), code: 1006, clean: False, reason: connection was closed uncleanly ("peer dropped the TCP connection without previous WebSocket closing handshake")" meanwhile the server does not block me: {'data': {'isLocked': False, 'plannedRecoverTime': 0, 'triggerCondition': {'UFR': 300, 'IFER': 150, 'GCR': 150}, 'updateTime': 0}}

Hi sir: After some observations, I found that the stream of userdata needs to remain active every 1 minute, which means that if your userdata does not change for more than 1 minute, the binance service will automatically disconnect the client. My solution is: in other active In the stream (for example: depth stream) send a new order (at a price that is unlikely to be traded) at intervals so that my userdata stream will not be disconnect, for your reference

Hi. I cannot agree with the statement. I have multiple connections running at the same time. But only one of them has this problem. The script is the same. The steps that are performed are the same. But on one connection there is this problem, on the others not.

Hi sir, Do you mean subscribing to multiple streams by one ws client at the same time or multiple ws clients at the same time?

I have 5 clients running now. Each client is running a subscription to 100 streams. 4 clients work without problems, and the described problem occurs on one client

[Pet222]

"I have 5 clients running now. Each client is running a subscription to 100 streams. 4 clients work without problems, and the described problem occurs on one client"

When you mean 5 clients running: you mean the same SW python environment / docker container + same computer is used for every client or they are on a different PC or might use different package versions? (what could be the difference between the 4 client and the 1 on which the issue could be reproduced?)

[Plotnychenko]

Different computers. Everything else is the same

[Pet222]

Different computers. Everything else is the same

When you mean same:

• exactly the same HW for all 5 machine (Os, Wifi/Lan + drivers)?
• Have you compared the package lists and versions and after that you state that they are the same or you use the same docker container?

I think your setups are really valuable to find out what could be the root cause of this issue!
So it would be really helpful if you could share as much details as you could. Thx in advance!

update:
I've instrumented the code with some prints on many levels to see what is going on (printing the timestamps too). What I observe that during a day of cont. run this issue hit 5 times. ping-pongs were fine. 3 time when code 1006 was reported the websocket kept going on (or recovered really quick) and only 2 times the websocket of 1m ticker was not able to recover within 30s.
Fri, 18 Nov 2022 00:34:37 +0000
['elapsed time since last update', 34, 'websocket update received:', {'MTS': 1668731699999, 'OPEN': 16889.17, 'CLOSE': 16894.9, 'HIGH': 16897.57, 'LOW': 16887.17, 'VOLUME': 140.53014, 'symbol': 'BTCUSDT', 'interval': '1m'}]
Fri, 18 Nov 2022 00:34:39 +0000
Fri, 18 Nov 2022 00:34:39 +0000
['elapsed time since last update', 36, 'websocket update received:', {'MTS': 1668731699999, 'OPEN': 16889.17, 'CLOSE': 16896.23, 'HIGH': 16897.57, 'LOW': 16887.17, 'VOLUME': 149.72488, 'symbol': 'BTCUSDT', 'interval': '1m'}]
Fri, 18 Nov 2022 00:34:39 +0000
WebSocket connection closed: connection was closed uncleanly ("peer dropped the TCP connection without previous WebSocket closing handshake"), code: 1006, clean: False, reason: connection was closed uncleanly ("peer dropped the TCP connection without previous WebSocket closing handshake")
Received Ping from server..payload
b'1668731678977'
Fri, 18 Nov 2022 00:34:39 +0000
Responded Pong to server
Fri, 18 Nov 2022 00:34:41 +0000
Fri, 18 Nov 2022 00:34:41 +0000
['elapsed time since last update', 38, 'websocket update received:', {'MTS': 1668731699999, 'OPEN': 16889.17, 'CLOSE': 16897.32, 'HIGH': 16899.51, 'LOW': 16887.17, 'VOLUME': 162.56665, 'symbol': 'BTCUSDT', 'interval': '1m'}]

[2pd]

Please check the v3 websocket which has the new design. We are happy to collect more feedbacks to improve it.

[hgghyxo]

If I'm running the code on my comp it fails, with the above mentioned error code, but if I run it in a docker container, complied with the same requriements.txt file, same python version, it works... so not sure why, but I have to use docker for local development as well.