ffmpeg crash when converting subtitles from ASS to MOV_TEXT

[kofolamaster]

Summary of the bug: crashes when converting ASS to MOV_TEXT
How to reproduce:

ffmpeg.exe -y -i ass_to_mov_text_crash.ass -scodec mov_text crashes.mkv
ffmpeg version N-85266-g1229007 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 6.3.0 (GCC)

[Carl Eugen Hoyos]

For future crash reports: Please remember to provide the information requested at https://ffmpeg.org/bugreports.html

Regression since 6433618d

(gdb) r -i ass_to_mov_text_crash.ass -scodec mov_text -f null -
Starting program: ffmpeg_g -i ass_to_mov_text_crash.ass -scodec mov_text -f null -
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
ffmpeg version N-85455-ga44b3ab Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 6.3.0 (GCC)
  configuration: --enable-gpl
  libavutil      55. 60.101 / 55. 60.101
  libavcodec     57. 92.100 / 57. 92.100
  libavformat    57. 72.100 / 57. 72.100
  libavdevice    57.  7.100 / 57.  7.100
  libavfilter     6. 84.101 /  6. 84.101
  libswscale      4.  7.101 /  4.  7.101
  libswresample   2.  8.100 /  2.  8.100
  libpostproc    54.  6.100 / 54.  6.100
Input #0, ass, from 'ass_to_mov_text_crash.ass':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Subtitle: ass
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf57.72.100
    Stream #0:0: Subtitle: mov_text
    Metadata:
      encoder         : Lavc57.92.100 mov_text
Stream mapping:
  Stream #0:0 -> #0:0 (ass (ssa) -> mov_text (native))
Press [q] to stop, [?] for help

Program received signal SIGSEGV, Segmentation fault.
0x000000000094ac63 in av_bswap16 (x=<optimized out>) at libavutil/bswap.h:60
60          x= (x>>8) | (x<<8);
(gdb) bt
#0  0x000000000094ac63 in av_bswap16 (x=<optimized out>) at libavutil/bswap.h:60
#1  mov_text_style_cb (priv=0x2009dc0, style=<optimized out>, close=<optimized out>)
    at libavcodec/movtextenc.c:251
#2  0x0000000000d8950c in ff_ass_split_override_codes (callbacks=callbacks@entry=0x1202fe0 <mov_text_callbacks>,
    priv=priv@entry=0x2009dc0, buf=0x200ad31 "\\b0\\c&H00CEFF&\\3c&H000000&\\blur2}Gloria")
    at libavcodec/ass_split.c:521
#3  0x000000000094a7a6 in mov_text_encode_frame (avctx=0x20098e0, buf=0x7ffff7eb8040 "", bufsize=1048576,
    sub=0x7fffffffd230) at libavcodec/movtextenc.c:354
#4  0x00000000007d3d05 in avcodec_encode_subtitle (avctx=avctx@entry=0x20098e0, buf=<optimized out>,
    buf_size=buf_size@entry=1048576, sub=sub@entry=0x7fffffffd230) at libavcodec/encode.c:358
#5  0x000000000049d9c5 in do_subtitle_out (sub=0x7fffffffd230, ost=0x2009660, of=<optimized out>)
    at ffmpeg.c:1007
#6  transcode_subtitles (ist=ist@entry=0x2007680, pkt=pkt@entry=0x7fffffffd3a0,
    got_output=got_output@entry=0x7fffffffd360, decode_failed=decode_failed@entry=0x7fffffffd460)
    at ffmpeg.c:2560
#7  0x000000000049e2a8 in process_input_packet (ist=0x2007680, pkt=0x7fffffffd7e0, no_eof=0) at ffmpeg.c:2657
#8  0x000000000047e43a in process_input (file_index=<optimized out>) at ffmpeg.c:4390
#9  transcode_step () at ffmpeg.c:4501
#10 transcode () at ffmpeg.c:4555
#11 main (argc=<optimized out>, argv=<optimized out>) at ffmpeg.c:4760
(gdb) disass $pc-32,$pc+32
Dump of assembler code from 0x94ac43 to 0x94ac83:
   0x000000000094ac43 <mov_text_style_cb+131>:  xor    %al,(%rax,%rax,1)
   0x000000000094ac46 <mov_text_style_cb+134>:  add    %cl,-0x75(%rax)
   0x000000000094ac49 <mov_text_style_cb+137>:  xchg   %eax,%edi
   0x000000000094ac4a <mov_text_style_cb+138>:  adc    %al,(%rax,%rax,1)
   0x000000000094ac4d <mov_text_style_cb+141>:  add    %cl,-0x59(%rbp,%rcx,4)
   0x000000000094ac51 <mov_text_style_cb+145>:  or     %al,(%rax,%rax,1)
   0x000000000094ac54 <mov_text_style_cb+148>:  add    %cl,-0x73(%rax)
   0x000000000094ac57 <mov_text_style_cb+151>:  mov    $0x20,%bh
   0x000000000094ac59 <mov_text_style_cb+153>:  add    $0x0,%al
   0x000000000094ac5b <mov_text_style_cb+155>:  add    %cl,-0x19(%rcx,%rcx,4)
   0x000000000094ac5f <mov_text_style_cb+159>:  rol    $0x8,%ax
=> 0x000000000094ac63 <mov_text_style_cb+163>:  mov    %ax,0x2(%rdx)
   0x000000000094ac67 <mov_text_style_cb+167>:  callq  0x1024760 <av_dynarray_add>
   0x000000000094ac6c <mov_text_style_cb+172>:  mov    $0x6,%edi
   0x000000000094ac71 <mov_text_style_cb+177>:  callq  0x10240b0 <av_malloc>
   0x000000000094ac76 <mov_text_style_cb+182>:  test   %rax,%rax
   0x000000000094ac79 <mov_text_style_cb+185>:  mov    %rax,0x410(%rbx)
   0x000000000094ac80 <mov_text_style_cb+192>:  je     0x94ae20 <mov_text_style_cb+608>
End of assembler dump.
(gdb) info register
rax            0xe00    3584
rbx            0x2009dc0        33594816
rcx            0x94abc0 9743296
rdx            0x0      0
rsi            0x200a1e0        33595872
rdi            0x200a1c8        33595848
rbp            0x62     0x62
rsp            0x7fffffffd010   0x7fffffffd010
r8             0x3      3
r9             0x0      0
r10            0x30     48
r11            0x0      0
r12            0x200a1c8        33595848
r13            0x0      0
r14            0x200ad31        33598769
r15            0xffffffff       4294967295
rip            0x94ac63 0x94ac63 <mov_text_style_cb+163>
eflags         0x10202  [ IF RF ]
cs             0x33     51
ss             0x2b     43
ds             0x0      0
es             0x0      0
fs             0x0      0
gs             0x0      0

[Philip Langdale]

commit f95c81ce104554b6860d94724a681a1bac0c4fbd
Author: Philip Langdale <​philipl@overt.org>
Date: Sun Apr 23 10:42:25 2017 -0700

avcodec/movtextenc: Ignore unmatched closing style tags

The existing code will segfault if a closing tag shows up when there
was never an opening tag. This isn't a well formed style, but it's also
not a reason to crash.

Fixes: ​https://trac.ffmpeg.org/ticket/6303