wtv muxer does not check H.264 streams for startcode

[ramitb]

Summary of the bug: If ffmpeg is used to remux a mp4 file into a WTV file the resulting WTV file has no video in WMP or Media Center when played back.
The original MP4 file is created by handbrake (the purpose being to use QuickSync hardware encoding). This happens with any file.
I'm showing an example where I'm converting my MPEG2 WTV collection to a MPEG4 WTV collection and I'm using handbrake to convert from MPEG2 to MPEG4 and FFMPEG to remux from MP4 to WTV.
When the player open it detects teh video size (ie. the window resizes) but the screen is blank (audio is fine).

How to reproduce:

HandBrakeCLI.exe -i Wheel.wtv -o Wheel.mp4 -e
 qsv_h264 -E copy
[10:24:06] hb_init: starting libhb thread
HandBrake svn6013 (2014020301) - MinGW x86_64 - http://handbrake.fr
4 CPUs detected
Opening Wheel.wtv...
[10:24:06] CPU: Intel(R) Core(TM) i7-3540M CPU @ 3.00GHz
[10:24:06]  - Intel microarchitecture Ivy Bridge
[10:24:06]  - logical processor count: 4
[10:24:06] OpenCL device #1: Intel(R) Corporation Intel(R) HD Graphics 4000
[10:24:06]  - OpenCL version: 1.1
[10:24:06]  - driver version: 9.17.10.2849
[10:24:06]  - device type:    GPU
[10:24:06]  - supported:      no
[10:24:06] Intel Quick Sync Video support: yes
[10:24:06]  - Intel Media SDK hardware: API 1.4 (minimum: 1.3)
[10:24:06]  - Preferred implementation: hardware (any)
[10:24:06] hb_scan: path=Wheel.wtv, title_index=1
libbluray/bdnav/index_parse.c:162: indx_parse(): error opening Wheel.wtv/BDMV/in
dex.bdmv
libbluray/bdnav/index_parse.c:162: indx_parse(): error opening Wheel.wtv/BDMV/BA
CKUP/index.bdmv
libbluray/bluray.c:2182: nav_get_title_list(Wheel.wtv) failed
[10:24:06] bd: not a bd - trying as a stream/file instead
libdvdnav: Using dvdnav version 5.0.0
libdvdread: Encrypted DVD support unavailable.
libdvdread:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.IFO failed
libdvdread:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.BUP failed
libdvdread: Can't open file VIDEO_TS.IFO.
libdvdnav: vm: failed to read VIDEO_TS.IFO
[10:24:07] dvd: not a dvd - trying as a stream/file instead
[mpeg2video @ 0000000003fcba60] Invalid frame dimensions 0x0.
Input #0, wtv, from 'Wheel.wtv':
  Metadata:
    WM/MediaClassPrimaryID: db9830bd-3ab3-4fab-8a371a995f7ff74
    WM/MediaClassSecondaryID: ba7f258a-62f7-47a9-b21f4651c42a000
    Title           : Wheel of Fortune
    WM/SubTitle     : Sandals Resorts Romance Week
    genre           : Series;Game Show
    WM/OriginalReleaseTime: 0
    WM/MediaCredits : ;;Pat Sajak/Vanna White;
    service_provider: WBZDT
    service_name    : WBZDT (WBZ-DT)
    WM/MediaNetworkAffiliation: CBS Affiliate
    WM/MediaOriginalChannel: 4
    WM/MediaOriginalChannelSubNumber: 1
    WM/MediaOriginalBroadcastDateTime: 2012-02-15T05:00:00Z
    WM/MediaOriginalRunTime: 5219881080
    WM/MediaIsStereo: false
    WM/MediaIsRepeat: false
    WM/MediaIsLive  : false
    WM/MediaIsTape  : false
    WM/MediaIsDelay : false
    WM/MediaIsSubtitled: false
    WM/MediaIsMovie : false
    WM/MediaIsPremiere: false
    WM/MediaIsFinale: false
    WM/MediaIsSAP   : false
    WM/MediaIsSport : false
    WM/ParentalRating: TV-G
    WM/Provider     : MediaCenterDefault
    WM/VideoClosedCaptioning: true
    WM/WMRVEncodeTime: 2012-02-16 00:23:18
    WM/WMRVSeriesUID: !MCSeries!00004762
    WM/WMRVServiceID: !MCService!28457676
    WM/WMRVProgramID: !MCProgram!268783610
    WM/WMRVRequestID: 0
    WM/WMRVScheduleItemID: 0
    WM/WMRVQuality  : 0
    WM/WMRVOriginalSoftPrePadding: 1394
    WM/WMRVOriginalSoftPostPadding: 120
    WM/WMRVHardPrePadding: -1394
    WM/WMRVHardPostPadding: 0
    WM/WMRVATSCContent: true
    WM/WMRVDTVContent: true
    WM/WMRVHDContent: true
    WM/MediaThumbWidth: 352
    WM/MediaThumbHeight: 198
    WM/MediaThumbStride: 1056
    WM/MediaThumbRet: 0
    WM/MediaThumbRatingSystem: 9
    WM/MediaThumbRatingLevel: 8
    WM/MediaThumbRatingAttributes: 0
    WM/MediaThumbAspectRatioX: 16
    WM/MediaThumbAspectRatioY: 9
    WM/MediaThumbTimeStamp: -4616189618054758400
    WM/MediaThumbType: 2
    Duration        : 318592637
    WM/WMRVEndTime  : 2012-02-16 00:23:49
    WM/WMRVBitrate  : 3.014656
    WM/WMRVKeepUntil: -1
    WM/WMRVActualSoftPrePadding: 1420
    WM/WMRVActualSoftPostPadding: -370
    WM/WMRVContentProtected: false
    WM/WMRVContentProtectedPercent: 0
    WM/WMRVExpirationDate:
    WM/WMRVExpirationSpan: 9223372036854775807
    WM/WMRVInBandRatingSystem: 255
    WM/WMRVInBandRatingLevel: 255
    WM/WMRVInBandRatingAttributes: 0
    WM/WMRVWatched  : true
  Duration: 00:00:31.26, start: 8.654056, bitrate: 14690 kb/s
    Stream #0.0[0x19](eng): Audio: ac3, 48000 Hz, 5.1, fltp, 384 kb/s
    Stream #0.1[0x1a](spa): Audio: ac3, 48000 Hz, stereo, fltp, 192 kb/s
    Stream #0.2[0x1b]: Video: mpeg2video (Main), yuv420p, 1920x1080 [PAR 1:1 DAR
 16:9], 17098 kb/s, 28.33 fps, 10000k tbn, 59.94 tbc
    Stream #0.3[0x1c]: Subtitle: [0][0][0][0] / 0x0000
    Stream #0.4[0xffffffff]: Attachment: mjpeg
    Metadata:
      title           : TV Thumbnail
[10:24:07] add_ffmpeg_subtitle: unknown subtitle stream type: 0x17007
[10:24:07] scan: decoding previews for title 1
[10:24:07] scan: audio 0x1: AC-3, rate=48000Hz, bitrate=192000 Espanol (AC3) (2.
0 ch)
[10:24:07] scan: audio 0x0: AC-3, rate=48000Hz, bitrate=384000 English (AC3) (5.
1 ch)
Scanning title 1 of 1, preview 9, 90.00 %[10:24:08] scan: 10 previews, 1920x1080
, 28.333 fps, autocrop = 0/2/240/240, aspect 16:9, PAR 1:1
[10:24:08] libhb: scan thread found 1 valid title(s)
+ title 1:
  + stream: Wheel.wtv
  + duration: 00:00:31
  + size: 1920x1080, pixel aspect: 1/1, display aspect: 1.78, 28.333 fps
  + autocrop: 0/2/240/240
  + support opencl: no
  + support hwd: yes
  + chapters:
    + 1: cells 0->0, 0 blocks, duration 00:00:31
  + audio tracks:
    + 1, English (AC3) (5.1 ch) (iso639-2: eng), 48000Hz, 384000bps
    + 2, Espanol (AC3) (2.0 ch) (iso639-2: spa), 48000Hz, 192000bps
  + subtitle tracks:
    + 1, Closed Captions (iso639-2: eng) (Text)(CC)
[10:24:08] 1 job(s) to process
[10:24:08] starting job
[10:24:08] Auto Passthru: allowed codecs are AAC, AC3, DTS, DTS-HD, MP3
[10:24:08] Auto Passthru: no valid fallback specified
[10:24:08] Auto Passthru: using AC3 Passthru for track 1
[10:24:08] sync: expecting 885 video frames
[10:24:08] job configuration:
[10:24:08]  * source
[10:24:08]    + Wheel.wtv
[10:24:08]    + title 1, chapter(s) 1 to 1
[10:24:08]    + container: wtv
[10:24:08]    + data rate: 14690 kbps
[10:24:08]  * destination
[10:24:08]    + Wheel.mp4
[10:24:08]    + container: MPEG-4 (avformat)
[10:24:08]  * video track
[10:24:08]    + decoder: mpeg2video
[10:24:08]      + bitrate 17098 kbps
[10:24:08]    + filters
[10:24:08]      + Framerate Shaper (0:27000000:952941)
[10:24:08]        + frame rate: same as source (around 28.333 fps)
[10:24:08]      + Crop and Scale (1440:1072:0:2:240:240)
[10:24:08]        + source: 1920 * 1080, crop (0/2/240/240): 1440 * 1078, scale:
 1440 * 1072
[10:24:08]    + dimensions: 1440 * 1072, mod 0
[10:24:08]    + encoder: H.264 (Intel QSV)
[10:24:08]      + bitrate: 1000 kbps, pass: 0
[10:24:08]  * audio track 1
[10:24:08]    + decoder: English (AC3) (5.1 ch) (track 1, id 0x0)
[10:24:08]      + bitrate: 384 kbps, samplerate: 48000 Hz
[10:24:08]    + AC3 Passthru
[mpeg2video @ 0000000003fd1920] Invalid frame dimensions 0x0.
[10:24:08] reader: first SCR 813160 id 0x2 DTS 813160
Encoding: task 1 of 1, 0.00 %[10:24:08] encqsvInit: using encode-only path
[10:24:08] encqsvInit: TargetUsage 2 AsyncDepth 4
[10:24:08] encqsvInit: GopRefDist 3 GopPicSize 28 NumRefFrame 2
[10:24:08] encqsvInit: BFrames on
[10:24:08] encqsvInit: RateControlMethod VBR TargetKbps 1000 MaxKbps 1500 Buffer
SizeInKB 375 InitialDelayInKB 187
[10:24:08] encqsvInit: CAVLC off
[10:24:08] encqsvInit: H.264 profile High @ level 4.0
[10:24:08] sync: first pts is 15015
Encoding: task 1 of 1, 2.71 %[10:24:08] qsv_enc_init: using 'hardware (1)' imple
mentation, API: 1.4
Encoding: task 1 of 1, 98.76 % (93.67 fps, avg 90.09 fps, ETA 00h00m01s)[10:24:1
8] reader: done. 1 scr changes
Encoding: task 1 of 1, 100.00 % (94.74 fps, avg 90.80 fps, ETA 00h00m00s)[10:24:
18] work: average encoding speed for job is 90.795891 fps
Encoding: task 1 of 1, 100.00 % (94.74 fps, avg 90.80 fps, ETA 00h00m00s)[10:24:
19] sync: got 932 frames, 885 expected
[10:24:19] render: lost time: 0 (0 frames)
[10:24:19] render: gained time: 0 (0 frames) (0 not accounted for)
[10:24:19] mpeg2video-decoder done: 932 frames, 1 decoder errors, 0 drops
[10:24:19] enc_qsv done: frames: 932 in, 932 out
[10:24:19] mux: track 0, 932 frames, 3899361 bytes, 996.45 kbps, fifo 1024
[10:24:19] mux: track 1, 965 frames, 1482240 bytes, 378.78 kbps, fifo 1024
Muxing: this may take awhile...[10:24:19] libhb: work result = 0

Encode done!

HandBrake has exited.

Remux MP4 to WTV using ffmpeg

ffmpeg -i Wheel.mp4 -vcodec copy -acodec copy test.wtv
ffmpeg version N-61478-g06a3185 Copyright (c) 2000-2014 the FFmpeg developers
  built on Mar 15 2014 13:06:50 with gcc 4.8.0 (GCC)
  configuration: --arch=x86_64 --target-os=mingw32 --cross-prefix=/Software/ffmp
eg/sandbox/mingw-w64-x86_64/bin/x86_64-w64-mingw32- --pkg-config=pkg-config --en
able-gpl --enable-libx264 --enable-avisynth --enable-libxvid --enable-libmp3lame
 --enable-version3 --enable-zlib --enable-librtmp --enable-libvorbis --enable-li
btheora --enable-libspeex --enable-libopenjpeg --enable-gnutls --enable-libgsm -
-enable-libfreetype --enable-libopus --disable-w32threads --enable-frei0r --enab
le-filter=frei0r --enable-libvo-aacenc --enable-bzlib --enable-libxavs --extra-c
flags=-DPTW32_STATIC_LIB --enable-libopencore-amrnb --enable-libopencore-amrwb -
-enable-libvo-amrwbenc --enable-libschroedinger --enable-libvpx --enable-libilbc
 --prefix=/Software/ffmpeg/sandbox/mingw-w64-x86_64/x86_64-w64-mingw32 --enable-
static --disable-shared --enable-libsoxr --enable-fontconfig --enable-libass --e
nable-libutvideo --enable-libbluray --enable-iconv --enable-libtwolame --extra-c
flags=-DLIBTWOLAME_STATIC --enable-libzvbi --enable-libcaca --enable-libmodplug
--extra-libs=-lstdc++ --extra-libs=-lpng --enable-libvidstab --extra-cflags= --e
xtra-cflags= --enable-nonfree --enable-libfdk-aac --enable-libfaac --enable-runt
ime-cpudetect
  libavutil      52. 66.101 / 52. 66.101
  libavcodec     55. 52.102 / 55. 52.102
  libavformat    55. 34.101 / 55. 34.101
  libavdevice    55. 11.100 / 55. 11.100
  libavfilter     4.  3.100 /  4.  3.100
  libswscale      2.  5.101 /  2.  5.101
  libswresample   0. 18.100 /  0. 18.100
  libpostproc    52.  3.100 / 52.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Wheel.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    title           : Wheel of Fortune
    encoder         : Lavf55.10.3
    genre           : Series;Game Show
  Duration: 00:00:31.43, start: 0.197000, bitrate: 1374 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709)
, 1440x1072 [SAR 1:1 DAR 90:67], 992 kb/s, 29.66 fps, 29.97 tbr, 90k tbn, 56.67
tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(eng): Audio: ac3 (ac-3 / 0x332D6361), 48000 Hz, 5.1(side), fltp,
 384 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Output #0, wtv, to 'test.wtv':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    title           : Wheel of Fortune
    genre           : Series;Game Show
    encoder         : Lavf55.34.101
    Stream #0:0(und): Video: h264 (avc1 / 0x31637661), yuv420p, 1440x1072 [SAR 1
:1 DAR 90:67], q=2-31, 992 kb/s, 29.66 fps, 10000k tbn, 28.33 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(eng): Audio: ac3 ([0] [0][0] / 0x2000), 48000 Hz, 5.1(side), 384
 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame=  932 fps=0.0 q=-1.0 Lsize=    5660kB time=00:00:31.23 bitrate=1484.7kbits
/s
video:3808kB audio:1448kB subtitle:0 data:0 global headers:0kB muxing overhead 7
.697319%

[Carl Eugen Hoyos]

Please reopen this ticket if you can provide an input sample.

[ramitb]

i've uploaded a sample #3473.mp4 which was encoded using handbrake and the settings used on the initial comment above.

[Carl Eugen Hoyos]

$ ffmpeg -i \#3473.mp4 -vcodec copy -acodec copy -vbsf h264_mp4toannexb test.wtv

[ramitb]

Shouldn't ffmpeg take care of that automatically, if wtv requires the h264_mp4toannexb bsf to be applied for it work, it should be a default option or ATLEAST a pop up warning message about setting the BSF

[Carl Eugen Hoyos]

This is not what you reported (you do remember that this is a bug tracker, not a support forum?) and it would be a duplicate of ticket #2680.

There is apparently a bug because the wtv muxer does not check if the H.264 stream is in the correct format.

[pross]

(Perhaps a better fault description would be "wtv muxer discards extradata, resulting in unplayable files")

Analysis:
The test file linked here (#3474.mp4) has H264 extradata containing startcode information, but the WTV muxer discards codec extradata when using -vcodec copy. The resulting file will not play back in FFmpeg or WMP.

Solution:
Will need to investigate whether/how the WTV format actually supports the extradata concept. And if not, insert the startcode information as a packet _or_ auto apply bitstream filter.

[Carl Eugen Hoyos]

Fixed by Peter Ross in 92d657b5