mpegtsraw open of multicast stream may hang

[Andrey Utkin]

Summary of the bug:
When opening many multicast UDP streams using avformat_open_input(), some instances may loop infinitely in mpegts_read_header() in lines 2016-2030.
Inspecting with gdb shows that nb_packets number is big, so it does not hang inside avio-stuff.
How to reproduce:

1. Have a lot of multicast MPEGTS streams. I had my on udp://239.195.0.1-50:1234
2. Code snippet:

#include <assert.h>
#include <stdio.h>
#include <pthread.h>
#include <libavformat/avformat.h>

#define THREADS_NUM 50

void *thread_work(void* dummy) {
  int ret;
  int i;
  char *url = dummy;
  printf("'%s' thread starts\n", url);
  AVFormatContext *ctx = avformat_alloc_context();
  assert(ctx);
  AVDictionary *avio_opts = NULL;
  av_dict_set(&avio_opts, "timeout", "2000000", 0);
  av_dict_set(&avio_opts, "fifo_size", "0", 0);  // bug happens both with and without separate UDP-fetching thread
  ret = avio_open2(&ctx->pb, url, AVIO_FLAG_READ, NULL, &avio_opts);
  assert(!ret);
  ret = avformat_open_input(&ctx, url, av_find_input_format("mpegtsraw"), NULL);
  if (!ret) {
    // read a bit, to add networking load
    for (i = 0; i < 1000; i++) {
      AVPacket pkt;
      ret = av_read_frame(ctx, &pkt);
    }
  }
  printf("'%s' thread exits\n", url);
  return NULL;
}

int main() {
  int ret;
  int i;
  pthread_t tids[THREADS_NUM];

  av_register_all();
  avformat_network_init();

  for (i = 0; i < THREADS_NUM; i++) {
    char *url = NULL;
    ret = asprintf(&url, "udp://239.195.0.%d:1234", i+1);
    assert(ret != -1);
    ret = pthread_create(&tids[i], NULL, thread_work, url);
    printf("created %d-th thread\n", i);
    assert(!ret);
  }
  for (i = 0; i < THREADS_NUM; i++) {
    pthread_join(tids[i], NULL);
    printf("joined %d-th thread\n", i);
  }
  return 0;
}

The defective result is that the application does not terminate. This happens not in 100% of launch cases, but happens most of times.
I could not replicate this bug on local UDP streams, sorry.
The same problem affects libav.org fork.

[Carl Eugen Hoyos]

Please provide a backtrace.

[Carl Eugen Hoyos]

Please reopen if you can provide a backtrace.

[jhowe321]

Replying to cehoyos:

Please reopen if you can provide a backtrace.

Here's one on Windows, multiple branches. Occurs < 5% of the time. Cannot reproduce on Linux.

	avformat-57.dll!ff_poll(pollfd * fds, unsigned long numfds, int timeout) Line 314	C	Symbols loaded.
 	avformat-57.dll!ff_network_wait_fd(int fd, int write) Line 79	C	Symbols loaded.
 	avformat-57.dll!udp_read(URLContext * h, unsigned char * buf, int size) Line 1069	C	Symbols loaded.
 	avformat-57.dll!retry_transfer_wrapper(URLContext * h, unsigned char * buf, int size, int size_min, int(*)(URLContext *, unsigned char *, int) transfer_func) Line 376	C	Symbols loaded.
 	avformat-57.dll!ffurl_read(URLContext * h, unsigned char * buf, int size) Line 410	C	Symbols loaded.
 	avformat-57.dll!fill_buffer(AVIOContext * s) Line 573	C	Symbols loaded.
 	avformat-57.dll!avio_read(AVIOContext * s, unsigned char * buf, int size) Line 668	C	Symbols loaded.
 	avformat-57.dll!read_packet(AVFormatContext * s, unsigned char * buf, int raw_packet_size, const unsigned char * * data) Line 2454	C	Symbols loaded.
 	avformat-57.dll!mpegts_read_header(AVFormatContext * s) Line 2700	C	Symbols loaded.
 	avformat-57.dll!avformat_open_input(AVFormatContext * * ps, const char * filename, AVInputFormat * fmt, AVDictionary * * options) Line 595	C	Symbols loaded.

libavformat/os_support.c

    if (timeout < 0) {
        rc = select(n, &read_set, &write_set, &exception_set, NULL);
    } else {
        struct timeval tv;
        tv.tv_sec  = timeout / 1000;
        tv.tv_usec = 1000 * (timeout % 1000);
    ---> rc         = select(n, &read_set, &write_set, &exception_set, &tv);
    }

[Carl Eugen Hoyos]

Replying to jhowe321:

Replying to cehoyos:

Please reopen if you can provide a backtrace.

Here's one on Windows, multiple branches. Occurs < 5% of the time. Cannot reproduce on Linux.

	avformat-57.dll!ff_poll(pollfd * fds, unsigned long numfds, int timeout) Line 314	C	Symbols loaded.

This looks outdated, please test with current FFmpeg git head and explain how I can reproduce (on Windows).

[jhowe321]

I will try. In the mean time, for whatever it's worth, I did observe that the issue is not the select per se. When the issue does occur, I can step through mpegts_read_header for the mpegtsraw case and observe that nb_packets grows without bound in the for (;;) loop. So packets are being read but the break for the PCR delta is never taken.