Playback mode that automatically skips silent parts.

[LichenSymbiont]

Like this ffmpeg-based script that processes a file to remove silent parts with certain inputs for how silent it should be when cutting and such: github.com/carykh/jumpcutter

Another playback mode that would be useful is to detect silence from one of 2 audio channels, and playing mono.

There's probably a lot of things you can do with a "playback mode" feature.

[ghost]

It'd be pretty trivial to write a filter which does this for audio only. But if video is involved, it would be much more complex.

[afcady]

The audio filter should be disabled if there is any video stream. Is it possible to make an audio filter know whether there is video playing?

[Zocker1999NET]

I would want to use this filter in a video file too, so there should be an option or something like that, if that is even possible.

[cankaratepe23]

I would like to see this implemented as well.

Would it be possible to have mpv run silencedetect on the file and store the output (internally or in a temp directory) to skip the detected sections maybe?

[barolo]

I've recently noticed that NewPipe has such option for videos, but it's used for fast forwarding [auto fast-forwards when silence is detected ] and I'm completely addicted to it

[avih]

What kinds of videos have silence which should be skipped? At all the clips I've ever watched, usually there's never real silence, and even if there was - I'd still like to watch the video...

[barolo]

I don't know how it works but it's not a total silence, and surprisingly it's useful with many that I watch, game streams, any educational content, longform talks, reviews, obviously not with music.
I'd love to have it for podcasts though.
It doesn't skip/cut the content, just speeds it up

[cankaratepe23]

What kinds of videos have silence which should be skipped? At all the clips I've ever watched, usually there's never real silence, and even if there was - I'd still like to watch the video...

For me personally, it's recordings of online lectures. I'm sure this would be useful to many people as well, and it can also be used for similar events such as webinars etc.

[barolo]

Yes, of course. I'm using it for lectures too, together with a small speed up. It's a real godsend, wish I knew about it earlier. It's a huge timesaver
Hopefully someone will be able to jank it out of NewPipe, together with scaletempo2 it would be a killer

[avih]

Can someone upload a sample video someplace, and list the first few timestamps+duration of silence which should be removed?

I wonder if ffmpeg's silencedetect can be used in a script in a similar way which cropdetect is used.

[barolo]

silencededetect been used that way for audio only things as per reddit, (outdated)
The NewPipe one is pretty aggressive and it skips between words, you can see it happening in video but it's unnoticeable in audio and it's hard to timestamp, I'll look for something with longer pauses

[barolo]

For example, it turns this https://youtu.be/T7SWETadMn0 16 min video into a 9 min video and from listening just to the audio I wouldn't be able to tell, it just sounds like she's speaking without ever catching breath

AFAIK they use this
from ExoPlayer

[avih]

Please don't refer to other players and what they do or don't do right. Just give a sample clip and describe the first few timestamps (seconds resolution would be enough) and rough duration where such silence should be removed.

[barolo]

That wasn't my intention, since they're open source I've linked directly to the code bit so it might be of use to someone.
As I said it's hard to do because the implementation that I care about removes silence between words, so even shorter than a second, turning 2 hour podcasts into an hour one, without becoming exhausting to listen

[afcady]

Personally I want it for music. Because I would like to skip the 10 to 15 seconds of silence that for some reason initiates many recordings of classical music. I just want the music to start instantly when I start the player.

[barolo]

@afcady that one is achievable with lavfi skipsilence filter, it's pretty much what it's meant for.

[pascal-mueller]

I would love to see this feature. I think a lot of people would. I'd love to be able to do something like:

mpv -detectSilence video.mp4

which results in mpv searching the audio stream for silence (for some given threshold) and opening it. It then set's like a"breakpoints" where the given silent part begins and ends and automatically skips it. Now people say that's a bad solution because you might skip content that's important but most people use it for lectures and how many lectures did you watch where the lecturer showed something without saying anything about it? Rarely, if ever.

To accommodate that, one would need motion detection - but that is resource heavy, so for the sake of efficiency I'd just stick with the above approach.

To give it more user friendlieness, one could enable the auto skip only on the first run through. So you watch the video, you come to a part where the video is silent for let's say 2 seconds and mpv automatically skips it. You now see that you somehow got lost and skip back. Since you skipped back manually, mpv won't skip forward again but let's you watch the previously skipped part.

Would make the life of educational content consumer a lot easier.

[rwmpelstilzchen]

I really hope this feature will be implemented.

IMHO, the user should have two options: to skip silent segments whenever they occur or to skip them only at the beginning and the end of the file.

• The first option is useful for speeding up the silent parts while listening/watching (e.g., when listening to a talk or a conversation with a lot of silent moments).
• The second one is useful for handling files in which the content actually begin a few seconds after the beginning of the file and ends a few seconds before its end (e.g., recorded example sentences, where there is a short delay between hitting ⏺ and speaking, and then hitting ⏹).

[MinmoTech]

I would like something similar to this browser extension: https://github.com/vantezzen/skip-silence

[barolo]

This script solves the issue for SOME, videos, some meaning ones with subs/captions as it fast-forwards when no text is visible.
[text can be made invisible easily]
In practice it should mean most of the lectures and YT videos, works surprisingly well.

Edit. just found that there's one for audio too

[pironside44]

I've recently noticed that NewPipe has such option for videos, but it's used for fast forwarding [auto fast-forwards when silence is detected ] and I'm completely addicted to it

RIGHT? I need this for MPV and as a chrome extension!

[barolo]

T

I've recently noticed that NewPipe has such option for videos, but it's used for fast forwarding [auto fast-forwards when silence is detected ] and I'm completely addicted to it

RIGHT? I need this for MPV and as a chrome extension!

There is a chrome/firefox extension already called skip-silence mentioned above, it works fairly well
If you want for it to work with your local videos just use something like Jellyfin as media provider, it will work with it too.

[ishitatsuyuki]

Hi,
I made a mpv script that uses the analysis data from https://github.com/WyattBlue/auto-editor to perform skipping.

https://gist.github.com/6ed6308be78dd8a2f58c9a9109fe050b

See the comments on the top for usage.
If you have any question, put it in the gist comments (and not this issue thread).

Enjoy!

[barolo]

Hi, I made a mpv script that uses the analysis data from https://github.com/WyattBlue/auto-editor to perform skipping.

https://gist.github.com/6ed6308be78dd8a2f58c9a9109fe050b

See the comments on the top for usage. If you have any question, put it in the gist comments (and not this issue thread).

Enjoy!

Thank you so much for this!. It works well. Finally, I can get back to watching lectures on the desktop.

[notevenaperson]

speed-transition.lua script skips to the next subtitle rather than audio. No one has mentioned it here, it achieves quite a similar purpose if your video is subtitled. You can also use Ctrl+RIGHT/LEFT to skip to the next/previous subtitle, this keybind comes out of the box in mpv.
Edit: Found another: sub-skip.lua

[barolo]

speed-transition.lua script skips to the next subtitle rather than audio. No one has mentioned it here, it achieves quite a similar purpose if your video is subtitled. You can also use Ctrl+RIGHT/LEFT to skip to the next/previous subtitle, this keybind comes out of the box in mpv. Edit: Found another: sub-skip.lua

Unfortunately most of my classes/lectures hold subs for pauses, rendering these useless for me. They kick in with a delay too.
Auto-editor based script works best for now, just needs speedup instead of cutting for more comfortable viewing.

[notevenaperson]

Unfortunately most of my classes/lectures hold subs for pauses, rendering these useless for me. They kick in with a delay too.

You might just re-do the timings with an automatic retimer like alass.

I agree that speedup is preferable to cutting.

[PriyanshuPriyadarshi]

There is similar feature in NewPipe and it is very useful while watching lecature.

[Topping1]

I have tweaked the script https://gist.github.com/bitingsock/e8a56446ad9c1ed92d872aeb38edf124 (speed-transition-Audio.lua) and it works better now. Mainly the speed up to 5x during silences was too much and sometimes the upcoming audio would cut out. Now there is a gradual transition to 3x and works better.

[ghost]

Have you found a satisfying way to use ffmpeg's silencedetect / silenceremove in mpv?

I'm interested in experimenting with skipping over any silence in audio lasting more than a second or two.

[AN3223]

I wrote a script here and there is also another script here. Mine requires hwdec to be off and it works by fast forwarding, so it may or may not be practical for your use case @07416.

EDIT: I'm now seeing bitingsock's script in this thread, it seems to work in a similar fashion.

[ghost]

I wrote a script here and there is also another script here. Mine requires hwdec to be off and it works by fast forwarding

@AN3223 Good recommendations. Do you happen to know an entirely automated script (all three set keybinds)? I plan creating a dedicated profile for speech-only playback and perhaps another for music, skipping silence between songs and discovering hidden tracks. For this workflow manual actions don't quite fit.

[AN3223]

Hm, I'm not sure. Maybe someone can adapt one of the scripts for that purpose. I don't really use mine and I find it to be unreliable.

[DonRichie]

Hello, I have a proof-of-concept to show. The following mpv lua script:
https://gist.github.com/DonRichie/23d8442e91ef8dc4917f9e6a9d6fe5c0 <- outdated, see next post

The idea:

1. execute ffmpeg in a subprocess to determine silent parts beforehand
2. Use the information to execute mpv's "seek" command and skip the silent parts
   There is also a variable edge_delta = 0.06 which can be adjusted to leave a bit of the silence for softening the jumpcuts.

Usage:

1. save script in ~/.config/mpv/scripts
2. play local video file (web urls do not work)
3. Press F2 to toggle on/off , it will need around 1/60 of the video duration to execute ffmpeg,
   then the osd will show a message and the effect starts to happen.
4. Use silence_threshold variable in the script to adjust the silence threshold. There is also the possibility to dynamically decrease it with F3 and increase it with F4, but since I only allow one ffmpeg process at once this feature is currently very annoying.

Problems:

• It can't use web urls, only local files, ffmpeg needs a usable path
• The seeking feels a bit slow, but it is usable
• ffmpeg needs around 1 minute for a 1 hour videofile. This is sufficient for many use cases like lectures, but in general very long.

Possible improvements:

• It could be possible to stream the output of ffmpeg so that the skipping can start immediately.
• Support for web url's
• The function recalc_next_skip() is executed very often. Reason is that it has to be tolerant against skipping forth and back. But I already have an idea how I could improve that.

[DonRichie]

I rewrote my previous proof-of-concept and created a new script based on the same idea.
It is now able to stream the ffmpeg output line by line. This means huge video files can now start skipping silence, shortly after the functionality is activated.

See my comments in the file for further information:
https://gist.github.com/DonRichie/1628527b41d5076712c97f8d6e76d4db

Have fun.
If you find bugs, let me know. I would be happy to hear some opinions.

Main problems:
- I can't kill running ffmpeg processes, if you use F3 and F4 or F2 multiple times it will slow down your system.
- Web URLs do not work.

[DonRichie]

There is a new revision available under the link in my last post.
The minimum duration of silence which ffmpeg detects is now configurable.
Lower values heavily impact the time ffmpeg needs to scan the file.

Also if it is set too low, it can be that mpv can't skip fast enough
This for example will effectively skip back a split second instead of forth:
[my] skipping to 17.62 , saved_time=0.02

[ferreum]

Based on https://gist.github.com/bitingsock/e8a56446ad9c1ed92d872aeb38edf124 I created a highly configurable script that works without external programs. I've published it here: https://github.com/ferreum/mpv-skipsilence
I'm surprised that it was possible to make it work so well, very similar to the NewPipe feature when configured with aggressive speedup ramps.
Unfortunately the speed changes very easily cause audio-video desynchronization, so I suppose it would still be nice to have something proper implemented in mpv itself (unless someone finds a solution to the desyncs).

[alopatindev]

speed-transition.lua script skips to the next subtitle rather than audio
...
Edit: Found another: sub-skip.lua

Unfortunately most of my classes/lectures hold subs for pauses, rendering these useless for me. They kick in with a delay too.

If you combine it with OpenAI's whisper-based subtitles generator (specifically this yt-whisper fork) — you will likely get desired result. I use whisper specifically to cut pauses in vlog-recorder with very high precision, the same way yt-whisper generates subtitles.

Upd: something based on whisper.cpp, like whisper-subs, would be even better.

[ferreum]

Short update: With mpv 0.37, the audio/video desynchronization I mentioned above has been fixed. As a result, the skipsilence script works very well now.

[barolo]

speed-transition.lua script skips to the next subtitle rather than audio
...
Edit: Found another: sub-skip.lua

Unfortunately most of my classes/lectures hold subs for pauses, rendering these useless for me. They kick in with a delay too.

If you combine it with OpenAI's whisper-based subtitles generator (specifically this yt-whisper fork) — you will likely get desired result. I use whisper specifically to cut pauses in vlog-recorder with very high precision, the same way yt-whisper generates subtitles.

Upd: something based on whisper.cpp, like whisper-subs, would be even better.

That's cool and all, there's one issue though, English isn't the only spoken language existing.

Short update: With mpv 0.37, the audio/video desynchronization I mentioned above has been fixed. As a result, the skipsilence script works very well now.

Amazing work, does everything that I wanted from such feature. It's a literal life-saver.

[alopatindev]

That's cool and all, there's one issue though, English isn't the only spoken language existing

There are multilingual models for whisper and whisper.cpp as well (for limited number of languages).