New anti-bot blocker on Youtube prevents to get video manifests (Sign in to confirm you’re not a bot)

[Darki93]

Version

6.3.16

Platform

.NET 8 / Debian

Steps to reproduce

Query and download many videos for a longer time on the same system with static IP address.

Details

I am executing YoutubeExplode on a Debian server with static IP address.
Since a few days I am encountering the following exception on the GetManifestAsync method:

YoutubeExplode.Exceptions.VideoUnplayableException: Video '' is unplayable. Reason: 'Sign in to confirm you’re not a bot'.
at YoutubeExplode.Videos.Streams.StreamClient.GetStreamInfosAsync(VideoId videoId, PlayerResponse playerResponse, CancellationToken cancellationToken) in //YoutubeExplode/Videos/Streams/StreamClient.cs:line 212
at YoutubeExplode.Videos.Streams.StreamClient.GetStreamInfosAsync(VideoId videoId, CancellationToken cancellationToken) in //YoutubeExplode/Videos/Streams/StreamClient.cs:line 276
at YoutubeExplode.Videos.Streams.StreamClient.GetManifestAsync(VideoId videoId, CancellationToken cancellationToken) in /_/YoutubeExplode/Videos/Streams/StreamClient.cs:line 292

Seems like Youtube added a new mechanism to stop multiple videos to be queried/downloaded from the same machine/IP without logging in.

Checklist

• I have looked through existing issues to make sure that this bug has not been reported before
• I have provided a descriptive title for this issue
• I have made sure that this bug is reproducible on the latest version of the package
• I have provided all the information needed to reproduce this bug as efficiently as possible
• I have sponsored this project
• I have not read any of the above and just checked all the boxes to submit the issue

[Tyrrrz]

Seems like Youtube added a new mechanism to stop multiple videos to be queried/downloaded from the same machine/IP without logging in.

I don't think there's much that you can do if that's the case. Would have to sign in.

[ReenigneArcher]

This was kind of random before... it's extremely repeatable now.

[rpm61]

I am also getting "Video 'AfWnMPDM0G4' is not available." from our Azure servers, noit getting it locally. Is it possible to use proxies with Explode?

[Darki93]

There is a YoutubeClient constructor overload taking a HttpClient as argument, which can be configured to use a proxy.
However, it is probably just a matter of time until the proxy's IP will also be flagged as bot.

[rpm61]

Thank you, should have seen that. Using proxies have solved the issue for me.

[Aimeast]

Maybe you could try bringing your authentication cookies or anti-bot cookies

[ReenigneArcher]

According to youtube-dl, cookies probably won't work for long if they even work at all. ytdl-org/youtube-dl#32905 (comment)

[qtlin]

Is there any way to make an authenticated request to satisfy anti-bot protection?
For the record, the problem is easy to reproduce using any browser private/incognito window to trigger "login to view" message from an IP address blocked by YouTube for any reason. At the same time, regular browser window is allowed by YouTube to view videos from the same IP address. Seems like YouTube views YoutubeExplode request like private/incognito window, at least based on YouTube response. May be above will help to find a remedy.

[Tyrrrz]

Somewhat useful observations:

• CI runs have been consistently failing for over a month now
• Tests that are failing are always the same:
  • All optimistic tests in StreamSpecs fail
  • Most optimistic tests in VideoSpecs fail
    • For some reason, tests for videos qs3NZHVM_Ik and rXMX4YJ7Lks do not fail
• Tests run fine locally most of the time, although I have observed rare failures
• Failure doesn't seem to be dependent on request rate/density, rather on origin IP ranges

[Tyrrrz]

Is there any way to make an authenticated request to satisfy anti-bot protection?

99% answers can usually be found in the readme: https://github.com/Tyrrrz/YoutubeExplode#authentication

[Tyrrrz]

When testing locally, it seems that instrumenting a short delay with no parallelism helped avoid the bot flagging mechanism (see #813). Also, when the client does get flagged, the issue automatically goes away after some time (not sure on the exact duration, but definitely less than an hour).

On CI, however, this did not yield any useful results. It seems that the public GitHub Actions runners are pre-identified as suspicious by YouTube and the corresponding requests are pre-emptively banned. I imagine the same would apply to apps running on other public infrastructure as well.

[qtlin]

Is there any way to make an authenticated request to satisfy anti-bot protection?

99% answers can usually be found in the readme: https://github.com/Tyrrrz/YoutubeExplode#authentication

Thank you very much, I am using an application that calls YoutubeExplode, I'll report your answer there.

[pagdot]

I've switched for a personal project to YoutubeDLSharp with yt-dlp and a youtube oauth login plugin

Hadn't had any issues with it yet

[3052]

original post is vague as hell, to the point of being not actionable - can someone please describe an actual test case? I might have a fix but its impossible to say give the current (lack of) information

[ReenigneArcher]

@3052 personally, I have a jellyfin plugin using this library. This unit tests is failing because of YouTube's anti bot measures. My guess is YouTube is blocking the IPs of GitHub runners.

https://github.com/LizardByte/Themerr-jellyfin/blob/1b5d4d71dd7216655ef9aa81991d0a75d908df4c/Jellyfin.Plugin.Themerr.Tests/TestThemerrManager.cs#L103

Edit: Same issue with yt-dlp (saw your comment there) and youtube-dl with other repos.

[Tyrrrz]

original post is vague as hell, to the point of being not actionable - can someone please describe an actual test case? I might have a fix but its impossible to say give the current (lack of) information

I was able to reproduce the issue by running YoutubeExplode's test suite 2-3 times.

It's also consistently reproducible on CI because the agent's IP/range is presumably banned: https://github.com/Tyrrrz/YoutubeExplode/actions/runs/11024028945

[3052]

OK I will continue my work in private then, good luck

[qtlin]

@Tyrrrz you can consistently reproduce at home/office if you connect to almost any VPN.

[rogerfar]

I tried 6.4.2 and it I'm getting much less 403 errors.

[noammaoz]

Any idea how to fix this issue ("Please sign in") or do I need to use another library?

[qtlin]

FWIW: Open source https://freetubeapp.io/ plays and downloads videos without any login on the same blacklisted IP where Firefox and other download tools are required to login in order to view.
UPD: spoke too soon. Found some addresses where even freetube does not play but iPhone iOS has no problems without login. Possibly uses previous cookies, needs to be tested with clean client.

[DotNet-Fan]

@Tyrrrz i was tryng passing the cookies from the webview and still get the below error when accessing stream manifest.

Response status code does not indicate success: 400 ().

Reference (#861 (comment))

[Anjoland]

I just started getting the ("Please sign in") issue yesterday. Seeing how old this thread is without a solution I am assuming this project is now dead?

[Tyrrrz]

I just started getting the ("Please sign in") issue yesterday. Seeing how old this thread is without a solution I am assuming this project is now dead?

Have you considered for a moment that a solution may simply not exist? There's nothing a library can do if YouTube chooses to enforce strict rate limits on their end.

[Anjoland]

I said this was an old thread with no solution so yes its been considered that a solution may not exist, but after 6 months I think its more a solution does not exist. I simply asked based on that is this now a dead library, I think others are curious as well.

[Tyrrrz]

That was a stupid question to ask. If you have something to contribute, make a PR or provide useful information. An open-source project is only as dead as the number of freeloaders waiting for someone else to fix their problems for them.

[cata76]

I'm experiencing this issue as well. Unfortunately, in recent days, it seems that these restrictions from YouTube have become significantly stricter. I've already accumulated quite a few banned addresses. It only takes a few downloads to be blocked. I doubt a solution can be found.

[sskodje]

The behavior of the anti bot system is quite strange. I seemingly randomly get "Please sign in" error on GetManifestAsync, which does not seem to be related to frequency or timeout between attempts, nor any specific videos, but the odd thing is that if i call GetManifestAsync inside a loop, it will go through within 2 or 3 attempts. I'll test some more, but it seems that for my use case, this very simple fix works for now.

[smgamermat77]

Well I'm just glad to see people posting here today. Seems like youtube is finally catching up to third party downloaders. First they shut down multiple websites, and now this tool doesn't work.

THE END IS NIGH!!!!11!1!!1!

[YumYumfisch]

Ill throw my experiences in here as well.
For a couple of months i sometimes got the "Please sign in" error, but it was very rarely, so i just retry failed downloads three times to catch these random failures.
The only videos that get this error consistently are the ones which are marked as 18+ by YouTube, which makes sense.

Throughout the last week, the "Please sign in" error was thrown much more often, but i could still download videos and with my "retry three times"-strategy most videos could be downloaded eventually.

However today, downloading any video results in this error.
I updatet from version 6.4.4 to 6.5.1, but this did not change anything.
Switching to a different ISP did not work either. (not using a VPN, i am talking about regular consumer internet access)

I guess either YouTube misconfigured something on their servers accidentally, but more likely their bot detection is at a point where it can detect YouTube Explode consistently.

[Nowammm]

I've also been getting this recently infamous "Please sign in" error. I've tried everything but to no avail.

I've found that the error disappears when you supply the YoutubeClient with an HttpClient that uses a proxy. But instead, it just doesn't complete the async method and times out.

var proxy = new WebProxy
{
    Address = new Uri($"http://122.152.4.133:6000"),
    BypassProxyOnLocal = false,
    UseDefaultCredentials = false,
};

// Create a client handler that uses the proxy
var httpClientHandler = new HttpClientHandler
{
    Proxy = proxy,
};

// Disable SSL verification
httpClientHandler.ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator;

// Finally, create the HTTP client object
var client = new HttpClient(handler: httpClientHandler, disposeHandler: true);

client.Timeout = TimeSpan.FromSeconds(10);

YoutubeClient youtube = new YoutubeClient(client);

StreamManifest streamManifest = await youtube.Videos.Streams.GetManifestAsync(id);

Running the code gives this error after the specified 10 second timeout:

System.Threading.Tasks.TaskCanceledException: 'The operation was canceled.'

I am completely bamboozled.

[DotNet-Fan]

@Tyrrrz Is there a data api quota involved here - https://developers.google.com/youtube/v3/docs/videos/list

https://developers.google.com/youtube/v3/getting-started#quota

[Walter5923]

Hi all,

First – thank you very much for this great library. Here are my two cents and observations… My MAUI app stopped working with the infamous

Video 'Flwh7__Rb_Q' is unplayable. Reason: 'Please sign in'.

message two days ago. At the same time New Pipe v0.27.4 stopped working as well. The New Pipe team released a new version yesterday. This version works fine again – in the same networks (mobile / WLAN) with probably the same IP address where my app still fails. Maybe it isn’t IP address blocking?

I tried to implement authentication in my app yesterday. But the CookieContainer in MAUI android WebView is still buggy. After login I get a page in WebView saying something was wrong with cookie management in my browser. Downloading videos with those cookies then fails with Bad Request.

Thanks again.

[sskodje]

It's not IP blocking i think. When i was testing, i could sporadically get it to work on one pc but not the other, on the same network.

[9903286]

Hi all,

First – thank you very much for this great library. Here are my two cents and observations… My MAUI app stopped working with the infamous

Video 'Flwh7__Rb_Q' is unplayable. Reason: 'Please sign in'.

message two days ago. At the same time New Pipe v0.27.4 stopped working as well. The New Pipe team released a new version yesterday. This version works fine again – in the same networks (mobile / WLAN) with probably the same IP address where my app still fails. Maybe it isn’t IP address blocking?

I tried to implement authentication in my app yesterday. But the CookieContainer in MAUI android WebView is still buggy. After login I get a page in WebView saying something was wrong with cookie management in my browser. Downloading videos with those cookies then fails with Bad Request.

Thanks again.

So NewPipe had this issue but fixed it? The main thing I can see they added in their repo is changing client version and adding visitor data:
4c72032
Is that something YoutubeExplode does?

[Col-V]

FYI...
4K Video Downloader (https://www.4kdownload.com/) is able to download videos from YouTube without requiring any kind of registration.

The current issue should therefore also be solvable in Tyrrrz/YoutubeDownloader. I would be happy if Oleksii could get his YoutubeDownloader working again.

[ReenigneArcher]

Does this now work from GitHub hosted runners in ci/cd pipelines, or are we still IP blocked on that front?

[Tyrrrz]

Huge thanks to @Ales315!

[Tyrrrz]

Does this now work from GitHub hosted runners in ci/cd pipelines, or are we still IP blocked on that front?

Doesn't appear so #867 (https://github.com/Tyrrrz/YoutubeExplode/actions/runs/12968580523?pr=867)

It did help with my local test runs though, so that's something at least

[Darki93]

I can confirm that I also still get the 'please sign in' exception on my server with static IP address.
However, I am currently working around that by hosting a socks5-proxy docker container in my home network which gets a new IP address from my ISP every day and is resolved via DynDNS.
This approach also failed for a few days, but now with the latest changes it works again. Thanks to everyone who supported here 😊

[Col-V]

I tested YoutubeExplode 6.5.2 this evening. It looks very good, so far there are no issues.

Thank you for the great work.