Opened 9 years ago

Closed 9 years ago

#614 closed defect (invalid)

mpegts issue on localhost

Reported by: burek Owned by: michael
Priority: normal Component: undetermined
Version: git-master Keywords:
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Hi,

In response to https://ffmpeg.org/trac/ffmpeg/ticket/602 (cehoyos), I'll try to provide my experience with mpegts format.

Here is the first issue. I have 2 ffmpeg processes, the first one should send the a/v (aac+h264) to the localhost:port using udp, and the 2nd ffmpeg process should pick it up and just save to the .ts file.

This is the command line of the first ffmpeg:

ffmpeg -y -re \
        -f video4linux2 -r 30 -s qvga -i /dev/v4l/by-id/usb-046d_0809_33A881A3-video-index0 \
        -f alsa -ar 44100 -ac 2 -i default:CARD=Intel \
        -vcodec libx264 -crf 20 -threads 0 \
        -acodec aac -ab 32k -ar 44100 -ac 2 \
        -strict experimental \
        -vbsf h264_mp4toannexb \
        -absf aac_adtstoasc \
        -f mpegts udp://127.0.0.1:10001

And this is the command of the 2nd ffmpeg:

ffmpeg -muxdelay 15 -f mpegts -i udp://localhost:10001 -acodec copy -vcodec copy out.ts

Now, when I try to play the .ts file in VLC, I can watch the video, but in the message log I can see a lot of those:

main warning: late picture skipped (35805 > -268)
main warning: late picture skipped (2472 > -268)
main warning: late picture skipped (7218 > -262)
main warning: late picture skipped (1171 > -334)
main warning: late picture skipped (24818 > -266)
main warning: late picture skipped (19257 > -277)
main warning: late picture skipped (14977 > -303)
main warning: late picture skipped (57074 > -266)
main warning: late picture skipped (23740 > -266)
main warning: late picture skipped (5374 > -265)
main warning: late picture skipped (1929 > -15969)
main warning: late picture skipped (1376 > -264)

I'm sure the network is not an issue here :) and also, I've monitored cpu usage (on a dual core) and it didn't go over 40% at any time.

Also, when I record a live stream this way for a very long time (like 12 hours or more) the audio and video get totally async, meaning audio is progressively advancing before the video and after like 3-4 hours of recording, the drift is around 10 seconds and will get even bigger for the longer period of the recording.

I can send you samples of .ts files, only I'll need to record for that long, so you can see that it really happens.

Other issues are related to remuxing from .ts to .flv (also with h264+aac), but I'll see to create new tickets for that.

Change History (13)

comment:1 Changed 9 years ago by cehoyos

  • Component changed from FFmpeg to undetermined

Assuming this is not reproducible with

ffmpeg -re -f video4linux2 -r 30 -s qvga -i /dev/v4l/by-id/usb-046d_0809_33A881A3-video-index0 -vcodec libx264 -crf 20 -vbsf h264_mp4toannexb out.ts

Isn't this more an udp issue?

And why are you using the bsf?

comment:2 Changed 9 years ago by burek

Well, I didn't try just recording to .ts file for that long, more rational way was to stream to an udp port because that way I could duplicate the data and watch it, while it is being saved to a file.

Anyway, for the absf, I don't remember now where exactly did I pick up that option (was it in ffmpeg channel or google), I just know that without it, when I play the udp stream in VLC, then the audio is a lot more quirky, skipping up/down-sampling and stuff... And since there is no official documentation on that filter ( http://ffmpeg.org/ffmpeg.html#aac_005fadtstoasc ), I didn't have chance to read about its real intended usage, so it might be the reason of the async behavior.

Can You give me one working example (default or something) of using mpegts (h264+aac) over udp, just to test it, because I've tried so many different combinations and variations I can't even remember now and none of them gave the smooth and clean playback without async (I'm talking about 100/100 LAN network). What could be the simplest way to start such a stream?

comment:3 Changed 9 years ago by cehoyos

I meant the video bitstream filter that should only make sense when stream copying from mp4.

To clarify:
Does creating the transport stream without using udp (and without audio and without the video bitstream filter) produce a file that does not show the warnings in vlc?

comment:4 Changed 9 years ago by burek

why without audio?

Version 0, edited 9 years ago by burek (next)

comment:5 Changed 9 years ago by burek

encoding audio only works just fine:

ffmpeg -y -re \
        -f alsa -ar 44100 -ac 2 -i default:CARD=Intel \
        -acodec aac -ab 32k -ar 44100 -ac 2 \
        -strict experimental \
        out.ts

No warnings, errors, nothin, just clear audio :)

comment:6 Changed 9 years ago by cehoyos

Is this only reproducible with an external library (-vcodec libx264) or also with an internal encoder?

Please post complete, uncut console output (including command line) without -threads 0.

And do you have any idea what the vlc warnings mean? Are there any visible problems with the resulting file? Do other players (ffplay, MPlayer, WMP 12) show any problems?

comment:7 Changed 9 years ago by burek

Same scenario with this example (mpeg2video):

ffmpeg -y -re \
        -f video4linux2 -r 30 -s qvga -i /dev/v4l/by-id/usb-046d_0809_33A881A3-video-index0 \
        out.ts

ffmpeg version N-34519-g54a09f1, Copyright (c) 2000-2011 the FFmpeg developers
  built on Nov  6 2011 12:46:05 with gcc 4.6.1
  configuration: --enable-static --enable-shared --enable-gpl --enable-nonfree --enable-postproc --enable-libx264 --enable-libaacplus --enable-libmp3lame --enable-libv4l2 --enable-libvo-aacenc --enable-version3
  libavutil    51. 23. 0 / 51. 23. 0
  libavcodec   53. 28. 0 / 53. 28. 0
  libavformat  53. 19. 0 / 53. 19. 0
  libavdevice  53.  4. 0 / 53.  4. 0
  libavfilter   2. 47. 0 /  2. 47. 0
  libswscale    2.  1. 0 /  2.  1. 0
  libpostproc  51.  2. 0 / 51.  2. 0
[video4linux2,v4l2 @ 0x1f9c980] Estimating duration from bitrate, this may be inaccurate
Input #0, video4linux2,v4l2, from '/dev/v4l/by-id/usb-046d_0809_33A881A3-video-index0':
  Duration: N/A, start: 27501.143731, bitrate: 27648 kb/s
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 320x240, 27648 kb/s, 30 tbr, 1000k tbn, 30 tbc
[buffer @ 0x1f9c800] w:320 h:240 pixfmt:yuv420p tb:1/1000000 sar:0/1 sws_param:
[mpegts @ 0x1f97040] muxrate VBR, pcr every 3 pkts, sdt every 200, pat/pmt every 40 pkts
Output #0, mpegts, to 'out.ts':
  Metadata:
    encoder         : Lavf53.19.0
    Stream #0:0: Video: mpeg2video, yuv420p, 320x240, q=2-31, 200 kb/s, 90k tbn, 30 tbc
Stream mapping:
  Stream #0.0 -> #0.0 (rawvideo -> mpeg2video)
Press [q] to stop, [?] for help
frame=  976 fps= 30 q=5.7 Lsize=    1257kB time=00:00:32.50 bitrate= 317.0kbits/s dup=638 drop=0
video:1070kB audio:0kB global headers:0kB muxing overhead 17.558359%

And here is libx264 without -threads:

# ffmpeg -y -re \
        -f video4linux2 -r 30 -s qvga -i /dev/v4l/by-id/usb-046d_0809_33A881A3-video-index0 \
        -vcodec libx264 -crf 20 \
        out.ts

ffmpeg version N-34519-g54a09f1, Copyright (c) 2000-2011 the FFmpeg developers
  built on Nov  6 2011 12:46:05 with gcc 4.6.1
  configuration: --enable-static --enable-shared --enable-gpl --enable-nonfree --enable-postproc --enable-libx264 --enable-libaacplus --enable-libmp3lame --enable-libv4l2 --enable-libvo-aacenc --enable-version3
  libavutil    51. 23. 0 / 51. 23. 0
  libavcodec   53. 28. 0 / 53. 28. 0
  libavformat  53. 19. 0 / 53. 19. 0
  libavdevice  53.  4. 0 / 53.  4. 0
  libavfilter   2. 47. 0 /  2. 47. 0
  libswscale    2.  1. 0 /  2.  1. 0
  libpostproc  51.  2. 0 / 51.  2. 0
[video4linux2,v4l2 @ 0x707980] Estimating duration from bitrate, this may be inaccurate
Input #0, video4linux2,v4l2, from '/dev/v4l/by-id/usb-046d_0809_33A881A3-video-index0':
  Duration: N/A, start: 27685.969253, bitrate: 27648 kb/s
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 320x240, 27648 kb/s, 30 tbr, 1000k tbn, 30 tbc
[buffer @ 0x708440] w:320 h:240 pixfmt:yuv420p tb:1/1000000 sar:0/1 sws_param:
[libx264 @ 0x7025a0] using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
[libx264 @ 0x7025a0] profile High, level 1.3
[mpegts @ 0x702040] muxrate VBR, pcr every 3 pkts, sdt every 200, pat/pmt every 40 pkts
Output #0, mpegts, to 'out.ts':
  Metadata:
    encoder         : Lavf53.19.0
    Stream #0:0: Video: h264, yuv420p, 320x240, q=-1--1, 90k tbn, 30 tbc
Stream mapping:
  Stream #0.0 -> #0.0 (rawvideo -> libx264)
Press [q] to stop, [?] for help
frame=  636 fps= 29 q=-1.0 Lsize=     626kB time=00:00:21.13 bitrate= 242.5kbits/s dup=442 drop=0
video:497kB audio:0kB global headers:0kB muxing overhead 25.792506%
[libx264 @ 0x7025a0] frame I:3     Avg QP:14.98  size: 15692
[libx264 @ 0x7025a0] frame P:424   Avg QP:19.03  size:  1074
[libx264 @ 0x7025a0] frame B:209   Avg QP:25.71  size:    33
[libx264 @ 0x7025a0] consecutive B-frames: 56.1%  0.0%  0.5% 43.4%
[libx264 @ 0x7025a0] mb I  I16..4:  2.4% 62.2% 35.3%
[libx264 @ 0x7025a0] mb P  I16..4:  0.0%  0.1%  0.0%  P16..4: 33.9%  9.4%  7.7%  0.0%  0.0%    skip:48.9%
[libx264 @ 0x7025a0] mb B  I16..4:  0.0%  0.0%  0.0%  B16..8:  3.4%  0.0%  0.0%  direct: 0.2%  skip:96.3%  L0:54.6% L1:44.0% BI: 1.4%
[libx264 @ 0x7025a0] 8x8 transform intra:64.4% inter:74.7%
[libx264 @ 0x7025a0] coded y,uvDC,uvAC intra: 97.3% 99.6% 98.7% inter: 9.6% 30.5% 18.6%
[libx264 @ 0x7025a0] i16 v,h,dc,p:  3%  3% 29% 65%
[libx264 @ 0x7025a0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 18% 20% 13%  3%  9%  5% 17%  5%  9%
[libx264 @ 0x7025a0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 18% 20%  8%  5% 10%  6% 15%  9%  9%
[libx264 @ 0x7025a0] i8c dc,h,v,p: 62% 19% 10%  9%
[libx264 @ 0x7025a0] Weighted P-Frames: Y:0.7% UV:0.2%
[libx264 @ 0x7025a0] ref P L0: 63.4%  6.8%  7.9% 21.7%  0.1%
[libx264 @ 0x7025a0] ref B L0: 85.1% 13.1%  1.8%
[libx264 @ 0x7025a0] ref B L1: 91.4%  8.6%
[libx264 @ 0x7025a0] kb/s:192.20

The problem still exists.
That message tells that the timestamps are somehow wrong, I guess. Sometimes I can see the image freezes for a fraction of a second and the image goes distorted just like a wrong keyframe was used for all the subsequent delta frames, but it gets back to normal after a couple of seconds.

comment:8 Changed 9 years ago by cehoyos

Are the problems only reproducible with -vcodec mpeg2video -f mpegts or also for -vcodec mpeg2video -f avi/mpg/mkv ?

comment:9 Changed 9 years ago by burek

-vcodec mpeg2video -f avi: no warnings at all
-vcodec mpeg2video -f mpeg: warnings are there constantly, just like with mpegts
-vcodec mpeg2video -f matroska: no warnings at all

comment:10 Changed 9 years ago by cehoyos

Is the problem also reproducible with

ffmpeg -f video4linux2 -i /dev/v4l/by-id/usb-046d_0809_33A881A3-video-index0 out.ts

?
Or is -r 30 needed to make the conversion work at all?

comment:11 Changed 9 years ago by burek

-r 30 just tells the driver to correctly sample the video input, so we don't have to resample it afterwards

here is the log:

# ffmpeg -f video4linux2 -i /dev/v4l/by-id/usb-046d_0809_33A881A3-video-index0 out.ts
ffmpeg version N-34519-g54a09f1, Copyright (c) 2000-2011 the FFmpeg developers
  built on Nov  6 2011 12:46:05 with gcc 4.6.1
  configuration: --enable-static --enable-shared --enable-gpl --enable-nonfree --enable-postproc --enable-libx264 --enab                                     le-libaacplus --enable-libmp3lame --enable-libv4l2 --enable-libvo-aacenc --enable-version3
  libavutil    51. 23. 0 / 51. 23. 0
  libavcodec   53. 28. 0 / 53. 28. 0
  libavformat  53. 19. 0 / 53. 19. 0
  libavdevice  53.  4. 0 / 53.  4. 0
  libavfilter   2. 47. 0 /  2. 47. 0
  libswscale    2.  1. 0 /  2.  1. 0
  libpostproc  51.  2. 0 / 51.  2. 0
[video4linux2,v4l2 @ 0x1daf780] The driver changed the time per frame from 32767/1996117544 to 1/30
[video4linux2,v4l2 @ 0x1daf780] Estimating duration from bitrate, this may be inaccurate
Input #0, video4linux2,v4l2, from '/dev/v4l/by-id/usb-046d_0809_33A881A3-video-index0':
  Duration: N/A, start: 25528.008073, bitrate: 27648 kb/s
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 320x240, 27648 kb/s, 30 tbr, 1000k tbn, 30 tbc
[buffer @ 0x1dbdf40] w:320 h:240 pixfmt:yuv420p tb:1/1000000 sar:0/1 sws_param:
[mpegts @ 0x1dbd0a0] muxrate VBR, pcr every 3 pkts, sdt every 200, pat/pmt every 40 pkts
Output #0, mpegts, to 'out.ts':
  Metadata:
    encoder         : Lavf53.19.0
    Stream #0:0: Video: mpeg2video, yuv420p, 320x240, q=2-31, 200 kb/s, 90k tbn, 30 tbc
Stream mapping:
  Stream #0.0 -> #0.0 (rawvideo -> mpeg2video)
Press [q] to stop, [?] for help
frame=  480 fps= 37 q=10.3 Lsize=     754kB time=00:00:15.96 bitrate= 386.9kbits/s dup=287 drop=0
video:658kB audio:0kB global headers:0kB muxing overhead 14.622744%

and warnings are also there again :/

comment:12 Changed 9 years ago by burek

I apologize, it seems that this error is related to a slow hardware (??) although I'm not sure why, because the cpu usage never gets over 30-40%, but they say its related to a slow decoder/cpu or something like that, so this bug report can be closed without further examination, I guess.

comment:13 Changed 9 years ago by cehoyos

  • Resolution set to invalid
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.