Opened 7 months ago

Closed 6 months ago

#6214 closed defect (fixed)

raw dnxhd corrupted

Reported by: ronag Owned by:
Priority: important Component: avcodec
Version: git-master Keywords: dnxhd regression
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

When muxing -> pipe -> demuxing raw dnxhd using ffmpeg I get warnings e.g. "Invalid data found when processing input" but also other header parsing related errors.

Is this a bug in dnxhd_parser?

ffmpeg -i ST01489A.nut -c:v dnxhd -flags +ilme+ildct -b:v 120M -f dnxhd - | ffmpeg -i - -f nut - > /dev/null

ffmpeg -i ST01489A.nut -c:v dnxhd -flags +ilme+ildct -b:v 120M -f dnxhd - | ffmpeg -i - -f nut - > /dev/null
ffmpeg version 3.2.4ffmpeg version 3.2.4 Copyright (c) 2000-2017 the FFmpeg developers Copyright (c) 2000-2017 the FFmpeg developers
  built with Apple LLVM version 8.0.0 (clang-800.0.42.1)

  configuration: --prefix=/usr/local/Cellar/ffmpeg/3.2.4 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-libmp3lame --enable-libwebp --enable-libx264 --enable-libxvid --enable-opencl --disable-lzma --enable-vda
  built with Apple LLVM version 8.0.0 (clang-800.0.42.1)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/3.2.4 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-libmp3lame --enable-libwebp --enable-libx264 --enable-libxvid --enable-opencl --disable-lzma --enable-vda
  libavutil      55. 34.101 / 55. 34.101
  libavcodec     57. 64.101 / 57. 64.101
  libavutil      55. 34.101 / 55. 34.101
  libavformat    57. 56.101 / 57. 56.101
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.101 / 57. 56.101
  libavdevice    57.  1.100 / 57.  1.100
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
Input #0, nut, from 'ST01489A.nut':
  Metadata:
    uid             : cf8d6ac2-df42-f44c-8b5e-1ac71f424110
    generation_uid  : 3468e120-b15f-0c44-adda-39ba7e0bcf4b
    company_name    : AVID
    product_name    : TRMG
    product_version : 3.01
    product_uid     : 00000000-0000-0000-0000-000000000000
    modification_date: 2016-11-10 14:11:09
    material_package_umid: 0x060A2B340101010501010D1313000000A2A55C5A1B72414A8842BD6E54E3881C
    timecode        : 00:00:00:00
    encoder         : Lavf57.41.100
  Duration: 00:00:31.59, start: 0.058667, bitrate: 18388 kb/s
    Stream #0:0: Audio: aac (LC) ([255][0][0][0] / 0x00FF), 48000 Hz, stereo, fltp (default)
    Metadata:
      encoder         : Lavc57.48.101 aac
    Stream #0:1: Video: h264 (High 4:2:2) (H264 / 0x34363248), yuv422p, 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 51200 tbn, 50 tbc
    Metadata:
      file_package_umid: 0x060A2B340101010501010D131300000017706BA67365EB40993508EA2E943181
      encoder         : Lavc57.48.101 libx264
Output #0, dnxhd, to 'pipe:':
  Metadata:
    uid             : cf8d6ac2-df42-f44c-8b5e-1ac71f424110
    generation_uid  : 3468e120-b15f-0c44-adda-39ba7e0bcf4b
    company_name    : AVID
    product_name    : TRMG
    product_version : 3.01
    product_uid     : 00000000-0000-0000-0000-000000000000
    modification_date: 2016-11-10 14:11:09
    material_package_umid: 0x060A2B340101010501010D1313000000A2A55C5A1B72414A8842BD6E54E3881C
    timecode        : 00:00:00:00
    encoder         : Lavf57.56.101
    Stream #0:0: Video: dnxhd (DNXHD), yuv422p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-1024, 120000 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      file_package_umid: 0x060A2B340101010501010D131300000017706BA67365EB40993508EA2E943181
      encoder         : Lavc57.64.101 dnxhd
Stream mapping:
  Stream #0:1 -> #0:0 (h264 (native) -> dnxhd (native))
Press [q] to stop, [?] for help
Input #0, dnxhd, from 'pipe:':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: dnxhd (DNXHD), yuv422p(bt709/unknown/unknown), 1920x1080, 25 fps, 25 tbr, 1200k tbn, 25 tbc
Output #0, nut, to 'pipe:':
  Metadata:
    encoder         : Lavf57.56.101
    Stream #0:0: Video: mpeg4 (FMP4 / 0x34504D46), yuv420p, 1920x1080, q=2-31, 200 kb/s, 25 fps, 51200 tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.64.101 mpeg4
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
Stream mapping:
  Stream #0:0 -> #0:0 (dnxhd (native) -> mpeg4 (native))
[dnxhd @ 0x7fb7ea804200] buffer too small (0 < 640).:18.60 bitrate=121241.6kbits/s dup=1 drop=0 speed=1.91x    
    Last message repeated 1 times
Error while decoding stream #0:0: Invalid data found when processing inputkbits/s speed=1.91x    
    Last message repeated 1 times288896kB time=00:00:19.52 bitrate=121241.6kbits/s dup=1 drop=0 speed=1.91x    
frame=  789 fps= 48 q=3.0 Lsize=  467088kB time=00:00:31.56 bitrate=121241.6kbits/s dup=1 drop=0 speed=1.92x    
video:467088kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
frame=  788 fps= 48 q=24.8 Lsize=    9121kB time=00:00:31.56 bitrate=2367.5kbits/s speed=1.94x    
video:9112kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.092401%

Attachments (2)

ST01489A_cut.dnxhd (2.3 MB) - added by cehoyos 7 months ago.
debug2_cut.dnxhd (2.3 MB) - added by cehoyos 6 months ago.

Change History (10)

comment:1 Changed 7 months ago by cehoyos

Is the issue you see only reproducible with pipes or also using files?
Is it reproducible using current FFmpeg git head?
Is a specific input necessary or is it also reproducible with -f lavfi -i testsrc?

comment:2 follow-up: Changed 7 months ago by ronag

Is the issue you see only reproducible with pipes or also using files?

Same issue with files

Is it reproducible using current FFmpeg git head?

Yes.

Is a specific input necessary or is it also reproducible with -f lavfi -i testsrc?

No errors with lavfi. But I've tried several different input files which cause the error.

comment:3 in reply to: ↑ 2 Changed 7 months ago by cehoyos

Replying to ronag:

Is the issue you see only reproducible with pipes or also using files?

Same issue with files

Is it reproducible using current FFmpeg git head?

Yes.

Please provide command line and complete, uncut console output for current FFmpeg git head without using pipes so I can try to reproduce.

comment:4 Changed 7 months ago by ronag

https://www.dropbox.com/s/lrxjhvfsiy1ap65/ST01489A.nut?dl=0

Roberts-MacBook-Pro-2:EFT ronagy$ ffmpeg -i ST01489A.nut -c:v dnxhd -flags +ilme+ildct -b:v 120M -f dnxhd asd.dnxhd -y
ffmpeg version git-2017-03-02-46082c2 Copyright (c) 2000-2017 the FFmpeg developers
  built with Apple LLVM version 8.0.0 (clang-800.0.42.1)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD-46082c2 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-libmp3lame --enable-libwebp --enable-libx264 --enable-libxvid --enable-opencl --disable-lzma --enable-vda
  libavutil      55. 47.101 / 55. 47.101
  libavcodec     57. 82.100 / 57. 82.100
  libavformat    57. 66.103 / 57. 66.103
  libavdevice    57.  3.100 / 57.  3.100
  libavfilter     6. 74.100 /  6. 74.100
  libavresample   3.  2.  0 /  3.  2.  0
  libswscale      4.  3.101 /  4.  3.101
  libswresample   2.  4.100 /  2.  4.100
  libpostproc    54.  2.100 / 54.  2.100
Input #0, nut, from 'ST01489A.nut':
  Metadata:
    uid             : cf8d6ac2-df42-f44c-8b5e-1ac71f424110
    generation_uid  : 3468e120-b15f-0c44-adda-39ba7e0bcf4b
    company_name    : AVID
    product_name    : TRMG
    product_version : 3.01
    product_uid     : 00000000-0000-0000-0000-000000000000
    modification_date: 2016-11-10 14:11:09
    material_package_umid: 0x060A2B340101010501010D1313000000A2A55C5A1B72414A8842BD6E54E3881C
    timecode        : 00:00:00:00
    encoder         : Lavf57.41.100
  Duration: 00:00:31.59, start: 0.058667, bitrate: 18388 kb/s
    Stream #0:0: Audio: aac (LC) ([255][0][0][0] / 0x00FF), 48000 Hz, stereo, fltp (default)
    Metadata:
      encoder         : Lavc57.48.101 aac
    Stream #0:1: Video: h264 (High 4:2:2) (H264 / 0x34363248), yuv422p, 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 51200 tbn, 50 tbc
    Metadata:
      file_package_umid: 0x060A2B340101010501010D131300000017706BA67365EB40993508EA2E943181
      encoder         : Lavc57.48.101 libx264
Output #0, dnxhd, to 'asd.dnxhd':
  Metadata:
    uid             : cf8d6ac2-df42-f44c-8b5e-1ac71f424110
    generation_uid  : 3468e120-b15f-0c44-adda-39ba7e0bcf4b
    company_name    : AVID
    product_name    : TRMG
    product_version : 3.01
    product_uid     : 00000000-0000-0000-0000-000000000000
    modification_date: 2016-11-10 14:11:09
    material_package_umid: 0x060A2B340101010501010D1313000000A2A55C5A1B72414A8842BD6E54E3881C
    timecode        : 00:00:00:00
    encoder         : Lavf57.66.103
    Stream #0:0: Video: dnxhd (DNXHD), yuv422p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-1024, 120000 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      file_package_umid: 0x060A2B340101010501010D131300000017706BA67365EB40993508EA2E943181
      encoder         : Lavc57.82.100 dnxhd
Stream mapping:
  Stream #0:1 -> #0:0 (h264 (native) -> dnxhd (native))
Press [q] to stop, [?] for help
frame=  789 fps= 77 q=3.0 Lsize=  467088kB time=00:00:31.56 bitrate=121241.6kbits/s dup=1 drop=0 speed=3.09x    
video:467088kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
Roberts-MacBook-Pro-2:EFT ronagy$ ffmpeg -i asd.dnxhd -f null - > /dev/null
ffmpeg version git-2017-03-02-46082c2 Copyright (c) 2000-2017 the FFmpeg developers
  built with Apple LLVM version 8.0.0 (clang-800.0.42.1)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD-46082c2 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-libmp3lame --enable-libwebp --enable-libx264 --enable-libxvid --enable-opencl --disable-lzma --enable-vda
  libavutil      55. 47.101 / 55. 47.101
  libavcodec     57. 82.100 / 57. 82.100
  libavformat    57. 66.103 / 57. 66.103
  libavdevice    57.  3.100 / 57.  3.100
  libavfilter     6. 74.100 /  6. 74.100
  libavresample   3.  2.  0 /  3.  2.  0
  libswscale      4.  3.101 /  4.  3.101
  libswresample   2.  4.100 /  2.  4.100
  libpostproc    54.  2.100 / 54.  2.100
Input #0, dnxhd, from 'asd.dnxhd':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: dnxhd (DNXHD), yuv422p(bt709/unknown/unknown), 1920x1080, 25 fps, 25 tbr, 1200k tbn, 25 tbc
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf57.66.103
    Stream #0:0: Video: wrapped_avframe, yuv422p, 1920x1080, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.82.100 wrapped_avframe
Stream mapping:
  Stream #0:0 -> #0:0 (dnxhd (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
[dnxhd @ 0x7fcc6780bc00] buffer too small (0 < 640). bitrate=N/A speed=  10x    
    Last message repeated 1 times
Error while decoding stream #0:0: Invalid data found when processing input
    Last message repeated 1 times
frame=  788 fps=255 q=-0.0 Lsize=N/A time=00:00:31.60 bitrate=N/A speed=10.2x    
video:382kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Roberts-MacBook-Pro-2:EFT ronagy$ 

comment:5 Changed 7 months ago by ronag

Can't say I fully understand how the dnxhd_parser works. But it looks to me that if you are unlucky you could hit a data byte that matches the prefix. https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/dnxhd_parser.c#L62

Also once pic_found == true don't we already know the offset to the next header? Why look for ff_dnxhd_check_header_prefix for every frame?

Last edited 7 months ago by ronag (previous) (diff)

comment:6 Changed 7 months ago by cehoyos

  • Component changed from undetermined to avcodec
  • Keywords regression added
  • Priority changed from normal to important
  • Reproduced by developer set
  • Status changed from new to open
  • Version changed from unspecified to git-master

Regression since ed0dc14ebba1d2e65fe39b7e8c10359adbfba36f
The third of the four attached frames is not decoded / parsed.

$ ffmpeg -i ST01489A_cut.dnxhd -f framecrc -
ffmpeg version N-83721-g55196e5 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl
  libavutil      55. 47.101 / 55. 47.101
  libavcodec     57. 82.100 / 57. 82.100
  libavformat    57. 66.103 / 57. 66.103
  libavdevice    57.  3.100 / 57.  3.100
  libavfilter     6. 74.100 /  6. 74.100
  libswscale      4.  3.101 /  4.  3.101
  libswresample   2.  4.100 /  2.  4.100
  libpostproc    54.  2.100 / 54.  2.100
Input #0, dnxhd, from 'ST01489A_cut.dnxhd':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: dnxhd (DNXHD), yuv422p(bt709/unknown/unknown), 1920x1080, 25 fps, 25 tbr, 1200k tbn, 25 tbc
#software: Lavf57.66.103
#tb 0: 1/25
#media_type 0: video
#codec_id 0: rawvideo
#dimensions 0: 1920x1080
#sar 0: 0/1
Output #0, framecrc, to 'pipe:':
  Metadata:
    encoder         : Lavf57.66.103
    Stream #0:0: Video: rawvideo (Y42B / 0x42323459), yuv422p, 1920x1080, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.82.100 rawvideo
Stream mapping:
  Stream #0:0 -> #0:0 (dnxhd (native) -> rawvideo (native))
Press [q] to stop, [?] for help
[dnxhd @ 0x29122a0] buffer too small (0 < 640).
0,          0,          0,        1,  4147200, 0x2b31c250
0,          1,          1,        1,  4147200, 0x718d54bd
0,          4,          4,        1,  4147200, 0x79750101
    Last message repeated 1 times
Error while decoding stream #0:0: Invalid data found when processing input
frame=    3 fps=0.0 q=-0.0 Lsize=       0kB time=00:00:00.20 bitrate=  11.6kbits/s speed=7.17x
video:12150kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

Changed 7 months ago by cehoyos

Changed 6 months ago by cehoyos

comment:8 Changed 6 months ago by richardpl

  • Resolution set to fixed
  • Status changed from open to closed
Note: See TracTickets for help on using tickets.