Default configuration adds too many trackers to magnets

[Nodens-]

The list of added trackers is too long and this can cause a lot of problems as I found out. The hard way.
I do not know if this just affects rtorrent users but rtorrent is the default headless client for seedboxes and linux.

When Medusa adds a few magnets to rtorrent that has a decent number of seeding torrents, rtorrent is adding all the trackers to a list and tries to contact them in intervals. The problem starts to become serious when some of these trackers do not respond or have their dns records pointing to whatever instead of valid listening ips. This failed connection and/or dns resolve spam causes rtorrent to slow down
and eventually freeze entirely after a threshold, as I found out. It took me a few hours debugging this on one of my users' rtorrent client with 2K seeding torrents . Perhaps other torrent clients handle this differently in code but with the userbase of rtorrent on seedboxes, this is certainly an issue.

Considering that, for public trackers, with DHT/PEX you don't even need a tracker, adding 3-4 trackers tops, the most common ones, is the way to go.

[duramato]

This just seems like a bug with rTorrent, which is bad, because this just means if you add any torrent with more than a few trackers it just becomes unresponsive!
Trackers are usefull for people that do not have DHT/PEX enabled, and as you noticed they tend to die off, but don't worry, you can edit the default trackers, from within the Medusa config in the "search settings" tab.

[Nodens-]

It is not really a bug but a lacking feature. After examining the code and relevant issue on rtorrent git, it's just that rtorrent does not do async dns resolving for udp trackers which causes the blocking behavior. (It does for tcp trackers if libcurl is built with c-ares support). There is a PR waiting in queue for about a year now that implements udns support.

My original point was not to add all these trackers by default as the most common usecase of Medusa+seedbox uses rtorrent. Not having DHT/PEX is usually the edge case when public trackers are involved and I do not know if the other torrent clients that are commonly used (deluge/transmission) implement async dns at all (not just for udp). Besides, adding a lot of trackers is frowned upon by other client developers as well (uTorrent developers come to mind).

Not many users have the technical expertise to debug and troubleshoot this issue so in favor of less problems I believe the best solution is to reduce the trackers added by default and link this list below the setting, which is actively maintained, https://raw.githubusercontent.com/ngosang/trackerslist/master/trackers_all_udp.txt for anyone wishing to change/add more as a conscious choice.

My concern is default configuration being problematic for the majority of the torrenting userbase (obviously this does not affect Usenet/NZB users).

EDIT: Going on step further, you can even automatically update the list of added trackers with the first 4 trackers from the maintained list I linked and provide redundancy/always working trackers.

[p0psicles]

Sounds like a good approach @duramato @medariox?

[medariox]

@Nodens-
How do you come to the conclusion that 4 trackers is the correct amount of trackers to add? Why not 6, why not 2? However, I agree that adding 20 trackers is probably excessive.

[p0psicles]

Yeah but its good to have a list of working trackers that we dont have to maintain right? I know @duramato always keeps track, but still.

[Nodens-]

@medariox
Normally, there should be just a single tracker (eg like what private trackers use). More trackers are added only for redundancy as public trackers tend to come and go which is also the reason why DHT and PEX were created (which render the need for redundancy obsolete). I would say 4 is more than enough (it is a conservative setting with DHT/PEX disabled in mind).
Do notice that connections to trackers count towards your connections limits as well and that trackers may temporarily autoban your ip if you're connecting to them too fast which can easily happen when you keep adding magnets with the same trackers if the tracker is just sending interval setting and not also sending the minimum interval setting (although that falls into tracker misconfiguration and it's a rare sight these days).
You may also want to read this thread: https://www.reddit.com/r/torrents/comments/3ysqbt/psa_adding_a_lot_of_trackers/

[Nodens-]

For the sake of completion and in order to help people troubleshooting that may land on this issue via google:
rTorrent related issue: rakshasa/rtorrent#180
PR with udns support: rakshasa/libtorrent#134
libtorrent with udns support (if someone wants to build it as is currently). Needs udns lib and headers available: https://github.com/slingamn/libtorrent
Script to detect started torrents with broken trackers to help clean existing torrents (needs pyrocore): https://pyrocore.readthedocs.io/en/latest/custom.html#list-stuck-tracker-announces

[rafi-d]

I don't think Medusa should intervene/modify any magnet link, unless it is required by the client.
So, for utorrent, the limitation is - 1024 bytes magnet link length, and this is implemented in Medusa (tho, can be improved, since one/last tracker is always illegal url).

Maybe you can add an option global/per-client for max # of trackers, or magnet length, but do not do this unconditionally.
Regardless, trackers' URLs should be left valid.

[rafi-d]

Since this seems to be postponed for a long time, I'd like to add that at least with utorrent, there is a bug in how Medusa is adding/sending trackers from the magnet link itself via WebAPI. The limitation for uTorrent is a 1024 byte long magnet, and if the list is longer, the last tracker is 99% of the times - truncated badly. I'm attaching my fix for that, so you can compare and change/fix too.
utorrent.py.txt

[medariox]

@rafi-d
Please open a PR if you want to submit a code change suggestion.

[rafi-d]

It's meant just as a related bug report (related to the too many trackers topic here) , I didn't intend for you to use my code (which I guess is not the best/optimal solution for a "smart truncate"...) , just as a way to point to the location the bug is...