-stream_loop option works neither with QSV, nor VA-API

[eero-t]

Summary of the bug:

"-stream_loop" option works with libx264, but neither with with QSV or VA-API.

Test-case:

1. Get e.g. quicktime/AVC "trailer_iphone.m4v" trailer from here: ​​https://peach.blender.org/trailer-page/
2. Loop that to output with libx264:

   $ ffmpeg -stream_loop 2 -y -i trailer_iphone.m4v -c:v libx264  output.h264
   ...
   frame= 3684 fps=711 q=-1.0 Lsize=    2941kB time=00:02:27.28 bitrate= 163.6kbits/s dup=1254 drop=0 speed=28.4x    
   video:2941kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
   

3. Loop that to output with VA-API:

   $ ffmpeg -stream_loop 2 -y -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i trailer_iphone.m4v -c:v h264_vaapi output.h264
   ffmpeg version 3.4.6-0ubuntu0.18.04.1 Copyright (c) 2000-2019 the FFmpeg developers
     built with gcc 7 (Ubuntu 7.3.0-16ubuntu3)
   <... after ~800 frames, when video should loop ...>
   Impossible to convert between the formats supported by the filter 'Parsed_null_0' and the filter 'auto_scaler_0'
   Error reinitializing filters!
   Failed to inject frame into filter network: Function not implemented
   Error while processing the decoded data for stream #0:1
   Conversion failed!
   

4. Loop that to output with QSV:

   $ ffmpeg -stream_loop 2 -y -hwaccel qsv -qsv_device /dev/dri/renderD128 -c:v h264_qsv -i trailer_iphone.m4v -c:v h264_qsv output.h264
   <... after ~800 frames, when video should loop ...>
   Impossible to convert between the formats supported by the filter 'Parsed_null_0' and the filter 'auto_scaler_0'
   Error reinitializing filters!
   Failed to inject frame into filter network: Function not implemented
   Error while processing the decoded data for stream #0:1
   Conversion failed!
   

Same issue regardless of whether input video is in container or raw video like .h264.

[Carl Eugen Hoyos]

Please test current FFmpeg git head and provide the command line you tested together with the complete, uncut console output to make this a valid ticket.

[eero-t]

Replying to cehoyos:

Please test current FFmpeg git head

I was testing HW acceleration both with Git head (iHD) and old Ubuntu LTS version (i965). Libx264 testing I did only with Ubuntu version, as I'm not building my own FFmpeg versions with CPU encoding.

Specifying looping with output filtering works with libx264, but also fails with HW acceleration:
-filter_complex loop=loop=<count>:start=<frame>:size=<frames>

As does using: -lavfi "movie=...."

Using this as input works with HW accelerated transcoding though:

$ cat list.txt
ffconcat version 1.0
file trailer_iphone.m4v
file trailer_iphone.m4v
file trailer_iphone.m4v

As does doing the decoding with CPU and uploading it to GPU for encoding:

ffmpeg -stream_loop 8 -y -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -i trailer_iphone.m4v -vf hwupload -c:v h264_vaapi  output.h264

=> Looping of HW decoding is broken

provide the command line you tested together with the complete, uncut console output to make this a valid ticket.

Here's full FFmpeg Git version output from the failure:

$ ffmpeg -stream_loop 2 -y -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i trailer_iphone.m4v -c:v h264_vaapi  output.h264 
ffmpeg version N-95843-g55ca21d54e Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.4.0-1ubuntu1~18.04.1)
  configuration: --enable-libmfx --enable-vaapi --enable-sdl2 --disable-libx265 --disable-libx264 --disable-libvpx --enable-libvorbis --enable-libopus --disable-libmp3lame --disable-libass --disable-sndio --enable-libfreetype --enable-gpl --disable-doc
  libavutil      56. 36.100 / 56. 36.100
  libavcodec     58. 62.100 / 58. 62.100
  libavformat    58. 35.100 / 58. 35.100
  libavdevice    58.  9.101 / 58.  9.101
  libavfilter     7. 66.100 /  7. 66.100
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'trailer_iphone.m4v':
  Metadata:
    major_brand     : M4VP
    minor_version   : 1
    compatible_brands: M4VPM4A mp42isom
    creation_time   : 2008-03-18T09:48:34.000000Z
  Duration: 00:00:33.02, start: 0.000000, bitrate: 942 kb/s
    Stream #0:0(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 127 kb/s (default)
    Metadata:
      creation_time   : 2008-03-18T09:48:34.000000Z
      handler_name    : Apple Sound Media Handler
    Stream #0:1(eng): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p(tv, smpte170m/smpte170m/bt709), 480x270, 824 kb/s, 25 fps, 25 tbr, 600 tbn, 1200 tbc (default)
    Metadata:
      creation_time   : 2008-03-18T09:48:34.000000Z
      handler_name    : Apple Video Media Handler
Stream mapping:
  Stream #0:1 -> #0:0 (h264 (native) -> h264 (h264_vaapi))
Press [q] to stop, [?] for help
[h264_vaapi @ 0x5596c3804440] No quality level set; using default (20).
Output #0, h264, to 'output.h264':
  Metadata:
    major_brand     : M4VP
    minor_version   : 1
    compatible_brands: M4VPM4A mp42isom
    encoder         : Lavf58.35.100
    Stream #0:0(eng): Video: h264 (h264_vaapi) (High), vaapi_vld, 480x270, q=-1--1, 25 fps, 25 tbn, 25 tbc (default)
    Metadata:
      creation_time   : 2008-03-18T09:48:34.000000Z
      handler_name    : Apple Video Media Handler
      encoder         : Lavc58.62.100 h264_vaapi
Impossible to convert between the formats supported by the filter 'Parsed_null_0' and the filter 'auto_scaler_0'
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
Error while processing the decoded data for stream #0:1
Conversion failed!

[Zhong,Li]

Please try this patch: ​https://patchwork.ffmpeg.org/patch/14160/ to disable auto_scale.

[eero-t]

With that patch, if I add "-noautoscale" before the input file, looping does work!

Why libx264 doesn't need such option to work?

Btw. if I add that same option after the input file, it seems to break option parsing:

$ ffmpeg -stream_loop 2  -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i trailer_iphone.m4v -noautoscale -c:v h264_vaapi output.h264
...
Option noautoscale (automatically insert a scale filter at the end of the filter graph if a resolutionchange is detected. This ensures all frames are the same resolution as the first framewhen they leave the filter chain (this option is enabled by default).If disabled, some encoders/muxers may not support this mode.) cannot be applied to output url output.h264 -- you are trying to apply an input option to an output file or vice versa. Move this option before the file it belongs to.
Error parsing options for output file output.h264.
Error opening output files: Invalid argument

As to the patch itself, as you can see from above output, you need to add few spaces to the error text, now words at end & start of the text lines run together.

[eero-t]

Correction, with the patch looping works for VA-API.

With QSV, it still fails. I just get a different error:

$ ffmpeg -stream_loop 2 -noautoscale -y -hwaccel qsv -qsv_device /dev/dri/renderD128 -c:v h264_qsv -i trailer_iphone.m4v -c:v h264_qsv output.h264
ffmpeg version N-95880-gd07d1c1dd0 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.4.0-1ubuntu1~18.04.1)
...
[h264_qsv @ 0x557b4fbe7f80] A decode call did not consume any data: expect more data at input (-10)
    Last message repeated 28 times
[h264_qsv @ 0x557b4fbe7540] Error submitting the frame for encoding.
Video encoding failed
Conversion failed!

[eero-t]

Is "-noautoscale" support getting upstream, or is "-stream_loop" issue going to be fixed in some other way?

[wenbin,chen]

I tried the following cmdline and it works fine.

ffmpeg -stream_loop 2 -y -i input.m4v -c:v h264_qsv output.h264

ffmpeg -stream_loop 2 -y -vaapi_device /dev/dri/renderD128 -i input.m4v -vf "format=nv12,hwupload" -c:v h264_vaapi output.h264