Opened 7 years ago

Closed 7 years ago

#6214 closed defect (fixed)

raw dnxhd corrupted

Reported by: ronag Owned by:
Priority: important Component: avcodec
Version: git-master Keywords: dnxhd regression
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

When muxing -> pipe -> demuxing raw dnxhd using ffmpeg I get warnings e.g. "Invalid data found when processing input" but also other header parsing related errors.

Is this a bug in dnxhd_parser?

ffmpeg -i ST01489A.nut -c:v dnxhd -flags +ilme+ildct -b:v 120M -f dnxhd - | ffmpeg -i - -f nut - > /dev/null

ffmpeg -i ST01489A.nut -c:v dnxhd -flags +ilme+ildct -b:v 120M -f dnxhd - | ffmpeg -i - -f nut - > /dev/null
ffmpeg version 3.2.4ffmpeg version 3.2.4 Copyright (c) 2000-2017 the FFmpeg developers Copyright (c) 2000-2017 the FFmpeg developers
  built with Apple LLVM version 8.0.0 (clang-800.0.42.1)

  configuration: --prefix=/usr/local/Cellar/ffmpeg/3.2.4 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-libmp3lame --enable-libwebp --enable-libx264 --enable-libxvid --enable-opencl --disable-lzma --enable-vda
  built with Apple LLVM version 8.0.0 (clang-800.0.42.1)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/3.2.4 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-libmp3lame --enable-libwebp --enable-libx264 --enable-libxvid --enable-opencl --disable-lzma --enable-vda
  libavutil      55. 34.101 / 55. 34.101
  libavcodec     57. 64.101 / 57. 64.101
  libavutil      55. 34.101 / 55. 34.101
  libavformat    57. 56.101 / 57. 56.101
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.101 / 57. 56.101
  libavdevice    57.  1.100 / 57.  1.100
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
Input #0, nut, from 'ST01489A.nut':
  Metadata:
    uid             : cf8d6ac2-df42-f44c-8b5e-1ac71f424110
    generation_uid  : 3468e120-b15f-0c44-adda-39ba7e0bcf4b
    company_name    : AVID
    product_name    : TRMG
    product_version : 3.01
    product_uid     : 00000000-0000-0000-0000-000000000000
    modification_date: 2016-11-10 14:11:09
    material_package_umid: 0x060A2B340101010501010D1313000000A2A55C5A1B72414A8842BD6E54E3881C
    timecode        : 00:00:00:00
    encoder         : Lavf57.41.100
  Duration: 00:00:31.59, start: 0.058667, bitrate: 18388 kb/s
    Stream #0:0: Audio: aac (LC) ([255][0][0][0] / 0x00FF), 48000 Hz, stereo, fltp (default)
    Metadata:
      encoder         : Lavc57.48.101 aac
    Stream #0:1: Video: h264 (High 4:2:2) (H264 / 0x34363248), yuv422p, 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 51200 tbn, 50 tbc
    Metadata:
      file_package_umid: 0x060A2B340101010501010D131300000017706BA67365EB40993508EA2E943181
      encoder         : Lavc57.48.101 libx264
Output #0, dnxhd, to 'pipe:':
  Metadata:
    uid             : cf8d6ac2-df42-f44c-8b5e-1ac71f424110
    generation_uid  : 3468e120-b15f-0c44-adda-39ba7e0bcf4b
    company_name    : AVID
    product_name    : TRMG
    product_version : 3.01
    product_uid     : 00000000-0000-0000-0000-000000000000
    modification_date: 2016-11-10 14:11:09
    material_package_umid: 0x060A2B340101010501010D1313000000A2A55C5A1B72414A8842BD6E54E3881C
    timecode        : 00:00:00:00
    encoder         : Lavf57.56.101
    Stream #0:0: Video: dnxhd (DNXHD), yuv422p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-1024, 120000 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      file_package_umid: 0x060A2B340101010501010D131300000017706BA67365EB40993508EA2E943181
      encoder         : Lavc57.64.101 dnxhd
Stream mapping:
  Stream #0:1 -> #0:0 (h264 (native) -> dnxhd (native))
Press [q] to stop, [?] for help
Input #0, dnxhd, from 'pipe:':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: dnxhd (DNXHD), yuv422p(bt709/unknown/unknown), 1920x1080, 25 fps, 25 tbr, 1200k tbn, 25 tbc
Output #0, nut, to 'pipe:':
  Metadata:
    encoder         : Lavf57.56.101
    Stream #0:0: Video: mpeg4 (FMP4 / 0x34504D46), yuv420p, 1920x1080, q=2-31, 200 kb/s, 25 fps, 51200 tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.64.101 mpeg4
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
Stream mapping:
  Stream #0:0 -> #0:0 (dnxhd (native) -> mpeg4 (native))
[dnxhd @ 0x7fb7ea804200] buffer too small (0 < 640).:18.60 bitrate=121241.6kbits/s dup=1 drop=0 speed=1.91x    
    Last message repeated 1 times
Error while decoding stream #0:0: Invalid data found when processing inputkbits/s speed=1.91x    
    Last message repeated 1 times288896kB time=00:00:19.52 bitrate=121241.6kbits/s dup=1 drop=0 speed=1.91x    
frame=  789 fps= 48 q=3.0 Lsize=  467088kB time=00:00:31.56 bitrate=121241.6kbits/s dup=1 drop=0 speed=1.92x    
video:467088kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
frame=  788 fps= 48 q=24.8 Lsize=    9121kB time=00:00:31.56 bitrate=2367.5kbits/s speed=1.94x    
video:9112kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.092401%

Attachments (2)

ST01489A_cut.dnxhd (2.3 MB ) - added by Carl Eugen Hoyos 7 years ago.
debug2_cut.dnxhd (2.3 MB ) - added by Carl Eugen Hoyos 7 years ago.

Change History (10)

comment:1 by Carl Eugen Hoyos, 7 years ago

Is the issue you see only reproducible with pipes or also using files?
Is it reproducible using current FFmpeg git head?
Is a specific input necessary or is it also reproducible with -f lavfi -i testsrc?

comment:2 by ronag, 7 years ago

Is the issue you see only reproducible with pipes or also using files?

Same issue with files

Is it reproducible using current FFmpeg git head?

Yes.

Is a specific input necessary or is it also reproducible with -f lavfi -i testsrc?

No errors with lavfi. But I've tried several different input files which cause the error.

in reply to:  2 comment:3 by Carl Eugen Hoyos, 7 years ago

Replying to ronag:

Is the issue you see only reproducible with pipes or also using files?

Same issue with files

Is it reproducible using current FFmpeg git head?

Yes.

Please provide command line and complete, uncut console output for current FFmpeg git head without using pipes so I can try to reproduce.

comment:4 by ronag, 7 years ago

https://www.dropbox.com/s/lrxjhvfsiy1ap65/ST01489A.nut?dl=0

Roberts-MacBook-Pro-2:EFT ronagy$ ffmpeg -i ST01489A.nut -c:v dnxhd -flags +ilme+ildct -b:v 120M -f dnxhd asd.dnxhd -y
ffmpeg version git-2017-03-02-46082c2 Copyright (c) 2000-2017 the FFmpeg developers
  built with Apple LLVM version 8.0.0 (clang-800.0.42.1)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD-46082c2 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-libmp3lame --enable-libwebp --enable-libx264 --enable-libxvid --enable-opencl --disable-lzma --enable-vda
  libavutil      55. 47.101 / 55. 47.101
  libavcodec     57. 82.100 / 57. 82.100
  libavformat    57. 66.103 / 57. 66.103
  libavdevice    57.  3.100 / 57.  3.100
  libavfilter     6. 74.100 /  6. 74.100
  libavresample   3.  2.  0 /  3.  2.  0
  libswscale      4.  3.101 /  4.  3.101
  libswresample   2.  4.100 /  2.  4.100
  libpostproc    54.  2.100 / 54.  2.100
Input #0, nut, from 'ST01489A.nut':
  Metadata:
    uid             : cf8d6ac2-df42-f44c-8b5e-1ac71f424110
    generation_uid  : 3468e120-b15f-0c44-adda-39ba7e0bcf4b
    company_name    : AVID
    product_name    : TRMG
    product_version : 3.01
    product_uid     : 00000000-0000-0000-0000-000000000000
    modification_date: 2016-11-10 14:11:09
    material_package_umid: 0x060A2B340101010501010D1313000000A2A55C5A1B72414A8842BD6E54E3881C
    timecode        : 00:00:00:00
    encoder         : Lavf57.41.100
  Duration: 00:00:31.59, start: 0.058667, bitrate: 18388 kb/s
    Stream #0:0: Audio: aac (LC) ([255][0][0][0] / 0x00FF), 48000 Hz, stereo, fltp (default)
    Metadata:
      encoder         : Lavc57.48.101 aac
    Stream #0:1: Video: h264 (High 4:2:2) (H264 / 0x34363248), yuv422p, 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 51200 tbn, 50 tbc
    Metadata:
      file_package_umid: 0x060A2B340101010501010D131300000017706BA67365EB40993508EA2E943181
      encoder         : Lavc57.48.101 libx264
Output #0, dnxhd, to 'asd.dnxhd':
  Metadata:
    uid             : cf8d6ac2-df42-f44c-8b5e-1ac71f424110
    generation_uid  : 3468e120-b15f-0c44-adda-39ba7e0bcf4b
    company_name    : AVID
    product_name    : TRMG
    product_version : 3.01
    product_uid     : 00000000-0000-0000-0000-000000000000
    modification_date: 2016-11-10 14:11:09
    material_package_umid: 0x060A2B340101010501010D1313000000A2A55C5A1B72414A8842BD6E54E3881C
    timecode        : 00:00:00:00
    encoder         : Lavf57.66.103
    Stream #0:0: Video: dnxhd (DNXHD), yuv422p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-1024, 120000 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      file_package_umid: 0x060A2B340101010501010D131300000017706BA67365EB40993508EA2E943181
      encoder         : Lavc57.82.100 dnxhd
Stream mapping:
  Stream #0:1 -> #0:0 (h264 (native) -> dnxhd (native))
Press [q] to stop, [?] for help
frame=  789 fps= 77 q=3.0 Lsize=  467088kB time=00:00:31.56 bitrate=121241.6kbits/s dup=1 drop=0 speed=3.09x    
video:467088kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
Roberts-MacBook-Pro-2:EFT ronagy$ ffmpeg -i asd.dnxhd -f null - > /dev/null
ffmpeg version git-2017-03-02-46082c2 Copyright (c) 2000-2017 the FFmpeg developers
  built with Apple LLVM version 8.0.0 (clang-800.0.42.1)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD-46082c2 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-libmp3lame --enable-libwebp --enable-libx264 --enable-libxvid --enable-opencl --disable-lzma --enable-vda
  libavutil      55. 47.101 / 55. 47.101
  libavcodec     57. 82.100 / 57. 82.100
  libavformat    57. 66.103 / 57. 66.103
  libavdevice    57.  3.100 / 57.  3.100
  libavfilter     6. 74.100 /  6. 74.100
  libavresample   3.  2.  0 /  3.  2.  0
  libswscale      4.  3.101 /  4.  3.101
  libswresample   2.  4.100 /  2.  4.100
  libpostproc    54.  2.100 / 54.  2.100
Input #0, dnxhd, from 'asd.dnxhd':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: dnxhd (DNXHD), yuv422p(bt709/unknown/unknown), 1920x1080, 25 fps, 25 tbr, 1200k tbn, 25 tbc
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf57.66.103
    Stream #0:0: Video: wrapped_avframe, yuv422p, 1920x1080, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.82.100 wrapped_avframe
Stream mapping:
  Stream #0:0 -> #0:0 (dnxhd (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
[dnxhd @ 0x7fcc6780bc00] buffer too small (0 < 640). bitrate=N/A speed=  10x    
    Last message repeated 1 times
Error while decoding stream #0:0: Invalid data found when processing input
    Last message repeated 1 times
frame=  788 fps=255 q=-0.0 Lsize=N/A time=00:00:31.60 bitrate=N/A speed=10.2x    
video:382kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Roberts-MacBook-Pro-2:EFT ronagy$ 

comment:5 by ronag, 7 years ago

Can't say I fully understand how the dnxhd_parser works. But it looks to me that if you are unlucky you could hit a data byte that matches the prefix. https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/dnxhd_parser.c#L62

Also once pic_found == true don't we already know the offset to the next header? Why look for ff_dnxhd_check_header_prefix for every frame?

Last edited 7 years ago by ronag (previous) (diff)

comment:6 by Carl Eugen Hoyos, 7 years ago

Component: undeterminedavcodec
Keywords: regression added
Priority: normalimportant
Reproduced by developer: set
Status: newopen
Version: unspecifiedgit-master

Regression since ed0dc14ebba1d2e65fe39b7e8c10359adbfba36f
The third of the four attached frames is not decoded / parsed.

$ ffmpeg -i ST01489A_cut.dnxhd -f framecrc -
ffmpeg version N-83721-g55196e5 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl
  libavutil      55. 47.101 / 55. 47.101
  libavcodec     57. 82.100 / 57. 82.100
  libavformat    57. 66.103 / 57. 66.103
  libavdevice    57.  3.100 / 57.  3.100
  libavfilter     6. 74.100 /  6. 74.100
  libswscale      4.  3.101 /  4.  3.101
  libswresample   2.  4.100 /  2.  4.100
  libpostproc    54.  2.100 / 54.  2.100
Input #0, dnxhd, from 'ST01489A_cut.dnxhd':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: dnxhd (DNXHD), yuv422p(bt709/unknown/unknown), 1920x1080, 25 fps, 25 tbr, 1200k tbn, 25 tbc
#software: Lavf57.66.103
#tb 0: 1/25
#media_type 0: video
#codec_id 0: rawvideo
#dimensions 0: 1920x1080
#sar 0: 0/1
Output #0, framecrc, to 'pipe:':
  Metadata:
    encoder         : Lavf57.66.103
    Stream #0:0: Video: rawvideo (Y42B / 0x42323459), yuv422p, 1920x1080, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.82.100 rawvideo
Stream mapping:
  Stream #0:0 -> #0:0 (dnxhd (native) -> rawvideo (native))
Press [q] to stop, [?] for help
[dnxhd @ 0x29122a0] buffer too small (0 < 640).
0,          0,          0,        1,  4147200, 0x2b31c250
0,          1,          1,        1,  4147200, 0x718d54bd
0,          4,          4,        1,  4147200, 0x79750101
    Last message repeated 1 times
Error while decoding stream #0:0: Invalid data found when processing input
frame=    3 fps=0.0 q=-0.0 Lsize=       0kB time=00:00:00.20 bitrate=  11.6kbits/s speed=7.17x
video:12150kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

by Carl Eugen Hoyos, 7 years ago

Attachment: ST01489A_cut.dnxhd added

by Carl Eugen Hoyos, 7 years ago

Attachment: debug2_cut.dnxhd added

comment:8 by Elon Musk, 7 years ago

Resolution: fixed
Status: openclosed
Note: See TracTickets for help on using tickets.