Not working on any other video other than Kennedy or Mona

[Tauseefahmed1451]

I have followed all the instructions and downloaded every model and placed it at defined places. Now when I run the sh file, it works if the name is kennedy and the audio is also the given one. Although if I only change the audio, it seems to process everything but does not store the output video. Lastly, if I change the video (my videos are also real humans with clear faces and no obstructions, looking into the camera) it fails at extracting frames. Also, torchvision gives a warning about deprecated parameter
This is the entire output
(lipsync) (base) root@07cb68bb-cd8a-4674-8c71-08d02fdf1aaf:/opt/trainml/Wav2Lip-HD(main)# bash run_final.shhey... /opt/trainml/Wav2Lip-HD hey... /opt/trainml/Wav2Lip-HD hey... /opt/trainml/Wav2Lip-HD Using cuda for inference. Traceback (most recent call last): File "/opt/trainml/Wav2Lip-HD/inference.py", line 359, in <module> main() File "/opt/trainml/Wav2Lip-HD/inference.py", line 263, in main mel = audio.melspectrogram(wav) File "/opt/trainml/Wav2Lip-HD/audio.py", line 47, in melspectrogram S = _amp_to_db(_linear_to_mel(np.abs(D))) - hp.ref_level_db File "/opt/trainml/Wav2Lip-HD/audio.py", line 95, in _linear_to_mel _mel_basis = _build_mel_basis() File "/opt/trainml/Wav2Lip-HD/audio.py", line 100, in _build_mel_basis return librosa.filters.mel(hp.sample_rate, hp.n_fft, n_mels=hp.num_mels, TypeError: mel() takes 0 positional arguments but 2 positional arguments (and 3 keyword-only arguments) were given Frames extracted and stored at frames_wav2lip/cot /opt/trainml/lipsync/lib/python3.10/site-packages/torchvision/transforms/functional_tensor.py:5: UserWarning: The torchvision.transforms.functional_tensor module is deprecated in 0.15 and will be **removed in 0.17**. Please don't rely on it. You probably just need to use APIs in torchvision.transforms.functional or in torchvision.transforms.v2.functional. warnings.warn( /opt/trainml/lipsync/lib/python3.10/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead. warnings.warn( /opt/trainml/lipsync/lib/python3.10/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or Nonefor 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passingweights=None. warnings.warn(msg)

[oykuparlakk]

I'm having the same problem as you. Did you find a solution? @Tauseefahmed1451 @saifhassan

[AIhasArrived]

anyone found ? @oykuparlakk

[gutscdav000]

same here @saifhassan any recommendations?

[start-life]

I have this problem too

[oykuparlakk]

Unfortunately I couldn't find it @gutscdav000 @AIhasArrived @start-life

[lbdave94]

it's sufficient uncomment (and slightly modify) the last line of run_final.sh.

the command that creates the hd video is the commented

#ffmpeg -r 20 -i ${back_dir}/${frames_hd}/${filename}/frame_%05d_out.jpg -i ${back_dir}/${input_audios} -vcodec libx264 -crf 25 -preset veryslow -acodec copy ${back_dir}/${output_videos_hd}/${filename}.mkv

I'm not able yet to fix passing global variable, but if you uncomment and modify with values it should work properly. E.g. with

ffmpeg -framerate 20 -pattern_type glob -i 'frames_hd/mona/*.jpg' -i 'input_audios/ai.wav' -c:v libx264 -preset veryslow -crf 25 -c:a copy 'output_videos_hd/mona.mkv'

[tranthuhoai3786]

@lbdave94 I used your command:
ffmpeg -framerate 20 -pattern_type glob -i 'frames_hd/mona/*.jpg' -i 'input_audios/ai.wav' -c:v libx264 -preset veryslow -crf 25 -c:a copy 'output_videos_hd/mona.mkv'
But the frame is slower than the audio, do you have any way to solve it?

[lbdave94]

@tranthuhoai3786 I think you just have to change the -framerate value, you can try with 25. But it's just a supposition, didn't try it.