Opened 4 years ago

Closed 4 years ago

#2579 closed defect (fixed)

Data stream from UAV video reported as "Unknown" type and without codec_id set

Reported by: mleotta Owned by:
Priority: normal Component: avformat
Version: git-master Keywords: mpegts regression
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:

In some older videos collected from a UAV (e.g. Predator drone), the KLV metadata stream is not recognized as a "Data" stream and is parsed as "Unknown". I would like to be able to ready this data stream as "Data".

In much older versions of FFmpeg (e.g. version 0.8 and before) the data stream was interpreted as "Data" and I could read the data correctly. I used git bisect to determine that metadata stream is interpreted as "Unknown" after commit e78d1a59.

I am actually using the APIs for libavformat, libavcodec, etc. to work with the video and metadata, but the bug can be determined by simply running ffmpeg and looking at the stream types.

How to reproduce:

% ffmpeg -i old_klv_data_stream.mpg
ffmpeg version N-53159-gd94c907 Copyright (c) 2000-2013 the FFmpeg developers
  built on May 16 2013 14:15:13 with gcc 4.6 (Ubuntu/Linaro 4.6.3-1ubuntu5)
  configuration: 
  libavutil      52. 32.100 / 52. 32.100
  libavcodec     55.  9.100 / 55.  9.100
  libavformat    55.  7.100 / 55.  7.100
  libavdevice    55.  0.100 / 55.  0.100
  libavfilter     3. 68.101 /  3. 68.101
  libswscale      2.  3.100 /  2.  3.100
  libswresample   0. 17.102 /  0. 17.102
[mpegts @ 0x3371020] probed stream 1 failed
[mpegts @ 0x3371020] max_analyze_duration 5000000 reached at 5003333 microseconds
[mpegts @ 0x3371020] Could not find codec parameters for stream 1 (Unknown: none ([21][0][0][0] / 0x0015)): unknown codec
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[mpegts @ 0x3371020] PES packet size mismatch
Input #0, mpegts, from 'old_klv_data_stream.mpg':
  Duration: 00:00:16.15, start: 0.909400, bitrate: 5072 kb/s
  Program 1 
    Stream #0:0[0x1e1]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 720x480 [SAR 1:1 DAR 3:2], 29.97 fps, 59.94 tbr, 90k tbn, 59.94 tbc
    Stream #0:1[0x1f1]: Unknown: none ([21][0][0][0] / 0x0015)
At least one output file must be specified

I will be uploading the sample file specified above, and attaching a report generated by ffmpeg -v 9 -loglevel 99 -report -i old_klv_data_stream.mpg.

I have hundreds video files that have these issues. Video from newer UAVs seem to use a different stream type and the metadata is correctly reported as

Stream #0:1[0x90]: Data: klv (KLVA / 0x41564C4B)

Attachments (3)

ffmpeg-20130516-153452.log (10.5 KB) - added by mleotta 4 years ago.
log of ffmpeg -v 9 -loglevel 99 -i old_klv_data_stream.mpg
old_klv_data_stream.patch (568 bytes) - added by mleotta 4 years ago.
This patch provides one possible solution
0001-mpegts-demux-synchronous-SMPTE-336M-Key-Length-Value.patch (2.1 KB) - added by pross 4 years ago.

Download all attachments as: .zip

Change History (21)

Changed 4 years ago by mleotta

log of ffmpeg -v 9 -loglevel 99 -i old_klv_data_stream.mpg

comment:1 follow-up: Changed 4 years ago by compn

are your UAV / drones used in a war or combat area?
maybe some developers dont want to support thier software used to shoot at people :P

if the video is too big you can upload it to our ftp, ftp://upload.ffmpeg.org/incoming/

comment:2 Changed 4 years ago by cehoyos

  • Keywords mpegts regression added; KLV Data Stream removed
  • Status changed from new to open

comment:3 Changed 4 years ago by mleotta

I believe the sample data file old_klv_data_stream.mpg has been uploaded to the ftp site. Also uploaded is old_klv_data_stream.txt, which is a short text file referring back to this Trac ticket and summarizing the problem. I received upload success messages, but have no way to verify that the files are actually on the server.

comment:4 Changed 4 years ago by cehoyos

Status "open" indicates that all necessary information is available (including a sample).

comment:5 in reply to: ↑ 1 Changed 4 years ago by mleotta

Replying to compn:

are your UAV / drones used in a war or combat area?

I do not actually have any UAVs, only data from them. The data in question is from a public data set (http://midas.kitware.com/community/view/28) used in the academic computer vision research community to develop algorithms for automatically detecting activities in video. The data set was proposed in the following paper:

"A Large-scale Benchmark Dataset for Event Recognition in Surveillance Video" by Sangmin Oh, et. al, in Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR), 2011. http://sangminoh.org/Publications_files/Oh2011cvpr.pdf

comment:6 follow-up: Changed 4 years ago by cehoyos

It should probably be added here that FFmpeg's software license allows you to use FFmpeg in a war or combat area or to shoot at people.

comment:7 in reply to: ↑ 6 Changed 4 years ago by mleotta

Replying to cehoyos:

It should probably be added here that FFmpeg's software license allows you to use FFmpeg in a war or combat area or to shoot at people.

That is true, and I would actually be surprised if it is not already being used by someone for that purpose. However, that is not the application or intent in this case.

Changed 4 years ago by mleotta

This patch provides one possible solution

comment:8 follow-up: Changed 4 years ago by cehoyos

Please send patches to ffmpeg-devel, they generally receive more attention there.

comment:9 follow-up: Changed 4 years ago by mleotta

The patch I have attached provides one possible solution. It partially reverts the change from e78d1a59 for the case where the media type was previously unknown.

The resulting output becomes

    Stream #0:1[0x1f1]: Data: none ([21][0][0][0] / 0x0015)

I think this comes down to a design decision. If a stream type is unknown do you treat it as "Unknown" and allow no access to the data, or do you treat it as "Data" of an unknown format. I prefer the latter because it allows the downstream developer to have access to the data and make use of it if they know how.

Another solution to this problem is to "register" this particular type of data stream so that it is not unknown. I don't know enough about the stream type or how the code works to suggest such a patch. However, I can say that this encoding appears to be common for KLV data streams prior to the more recent standards.

comment:10 in reply to: ↑ 8 Changed 4 years ago by mleotta

Replying to cehoyos:

Please send patches to ffmpeg-devel, they generally receive more attention there.

Okay, will do!

comment:11 Changed 4 years ago by cehoyos

A question from Michael waits for clarification:
http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/164033/focus=164039

comment:12 Changed 4 years ago by mleotta

Thank you for the update, was not subscribed to ffmpeg-devel and did not see it. I am now subscribed and have responded.

comment:13 in reply to: ↑ 9 Changed 4 years ago by michael

Replying to mleotta:

Another solution to this problem is to "register" this particular type of data stream so that it is not unknown. I don't know enough about the stream type or how the code works to suggest such a

Detecting the stream and setting codec id and type seems most sane to me. Simply turning all unknown into data is not correct

comment:14 follow-up: Changed 4 years ago by michael

How can these old streams be identified ?

comment:15 Changed 4 years ago by michael

  • Priority changed from important to normal
  • Summary changed from Data stream from UAV video reported as "Unknown" type to Data stream from UAV video reported as "Unknown" type and without codec_id set

Its a bit boarderline if this is a regression or not, considering that codec_id was never set correctly and the stream type was data more by bug than feature. Technically its a regression though but IMO its normal not important considering it never really worked

comment:16 in reply to: ↑ 14 Changed 4 years ago by mleotta

Replying to michael:

How can these old streams be identified ?

I uploaded a sample video with the hope that someone could help me figure that out. Unfortunately I know little about how this normally works. How are streams normally identified? I'm happy to help dig into this if someone can point me in the right direction.

comment:17 Changed 4 years ago by pross

Patch enclosed.

comment:18 Changed 4 years ago by michael

  • Resolution set to fixed
  • Status changed from open to closed

patch applied

thanks

Note: See TracTickets for help on using tickets.