Rate limiting on files.pythonhosted.org ? #2239
Description
My Platform
We're seeing connection issues to files.pythonhosted.org. After a few attempts to connect to either files.pythonhosted.org or pypi.org we are getting a time out. With strace and tcpdump we could see all downloads stalling with TCP SYN packets being sent, but not SYN/ACK’ed by the webservers. Our best guess is that we’re rate limited at remote side.
We're using this one-liner to test the connection:
for i in $(seq 1 20) ; do echo $i ; wget -O - https://files.pythonhosted.org/ > /dev/null ; done
After 5 attempts it starts blocking. On another machine on the same network, we get the same results. Moving to another network (with another external IP), it works.
Are these hosts employing rate limiting, if so, what is the frequency?
Fastly Debug
No response
DNS Resolution
We only do IPV4
$ dig files.pythonhosted.org A
; <<>> DiG 9.16.1-Ubuntu <<>> files.pythonhosted.org A
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 13755
;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;files.pythonhosted.org. IN A
;; ANSWER SECTION:
files.pythonhosted.org. 6846 IN CNAME dualstack.r.ssl.global.fastly.net.
dualstack.r.ssl.global.fastly.net. 8 IN A 151.101.129.63
dualstack.r.ssl.global.fastly.net. 8 IN A 151.101.1.63
dualstack.r.ssl.global.fastly.net. 8 IN A 151.101.65.63
dualstack.r.ssl.global.fastly.net. 8 IN A 151.101.193.63
;; Query time: 0 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Mon Sep 19 10:42:58 CEST 2022
;; MSG SIZE rcvd: 162Traceroutes / IPv4
traceroute files.pythonhosted.org
traceroute to files.pythonhosted.org (151.101.65.63), 30 hops max, 60 byte packets
1 _gateway (10.31.1.254) 0.120 ms 0.080 ms 0.066 ms
2 10.34.2.254 (10.34.2.254) 0.344 ms 0.329 ms 0.270 ms
3 * * *
4 * * *
5 149.29.4.49 (149.29.4.49) 4.826 ms 4.942 ms 4.851 ms
6 be3718.rcr11.gne01.atlas.cogentco.com (154.25.2.97) 4.973 ms 5.110 ms 5.278 ms
7 be3075.ccr41.lon13.atlas.cogentco.com (130.117.1.249) 9.603 ms 9.630 ms 9.801 ms
8 be3384.ccr41.ams03.atlas.cogentco.com (154.54.58.165) 8.061 ms be2868.ccr21.lon01.atlas.cogentco.com (154.54.57.154) 9.416 ms be3384.ccr41.ams03.atlas.cogentco.com (154.54.58.165) 8.221 ms
9 * * *
10 * * *
11 * * *
12 * * *
13 * * *
14 * * *
15 * * *
16 * * *
17 * * *
18 * * *
19 * * *
20 * * *
21 * * *
22 * * *
23 * * *
24 * * *
25 * * *
26 * * *
27 * * *
28 * * *
29 * * *
30 * * *
Traceroutes / IPv6 (If available)
No response
HTTPS Requests / IPv4
curl -vvv -I --ipv4 https://pypi.org/pypi/pip/json
* Trying 151.101.192.223:443...
* TCP_NODELAY setcurl -vvv -I --ipv4 https://files.pythonhosted.org/packages/ae/e8/2340d46ecadb1692a1e455f13f75e596d4eab3d11a57446f08259dee8f02/pip-10.0.1.tar.gz
* Trying 151.101.1.63:443...
* TCP_NODELAY setHTTPS Requests / IPv6 (If available)
No response
TLS Debug / IPv4
echo -n | openssl s_client -4 -connect pypi.org:443
<no response>echo -n | openssl s_client -4 -connect files.pythonhosted.org:443
<no response>TLS Debug / IPv6 (If available)
No response
Code of Conduct
- I agree to follow the PSF Code of Conduct