Opened 6 years ago

Last modified 9 months ago

#7395 open defect

DASH demuxing error "unterminated entity reference" due to ampersand in URL

Reported by: Starchild Owned by:
Priority: normal Component: avformat
Version: git-master Keywords: dash regression
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Commit 7a63da8 broke the DASH demuxer if the manifest contains a BaseURL with an ampersand (&) or other characters that need to be escaped in XML.

This is because the method resolve_content_path calls xmlNodeGetContent, which unescapes the string, and then passes it back to xmlNodeSetContent, which apparently expects an escaped string.

Unfortunately I cannot provide an example MPD, but it should be easy enough to manufacture one by simply adding &v=3 or so to the BaseURL in a simple manifest.

Attachments (1)

dash.mpd​ (6.5 KB ) - added by gorilla.maguila 22 months ago.
MPD file with & in BaseURL

Download all attachments as: .zip

Change History (7)

comment:1 by Carl Eugen Hoyos, 6 years ago

Keywords: regression added; dashdec removed

comment:2 by cyclone200, 3 years ago

Status: new → open

This error is still present in the latest build (11-2021).

by gorilla.maguila, 22 months ago

Attachment: dash.mpd​ added

MPD file with & in BaseURL

comment:3 by gorilla.maguila, 22 months ago

The bug is still present in latest git:

$ ffplay -allowed_extensions ALL -loglevel debug dash.mpd
ffplay version N-109662-g2c3107c3e9 Copyright (c) 2003-2023 the FFmpeg developers
  built with gcc 12.2.1 (GCC) 20230111
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-shared --enable-version3
  libavutil      57. 44.100 / 57. 44.100
  libavcodec     59. 57.100 / 59. 57.100
  libavformat    59. 36.100 / 59. 36.100
  libavdevice    59.  8.101 / 59.  8.101
  libavfilter     8. 54.100 /  8. 54.100
  libswscale      6.  8.112 /  6.  8.112
  libswresample   4.  9.100 /  4.  9.100
  libpostproc    56.  7.100 / 56.  7.100
Initialized opengl renderer.
[NULL @ 0x7fa2b8000c80] Opening 'dash.mpd' for reading
[file @ 0x7fa2b8001580] Setting default whitelist 'file,crypto,data'
[dash @ 0x7fa2b8000c80] Format dash probed with size=2048 and score=100
dash.mpd:-1: error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfcjJfYXZjX2dlbjFhdmNfbGNfcTQwX2ZyYWdfMl92aWRlbyJ9&_nc_ohc=gXFNqgRhAcwAX8Yb-D0&_nc_ht=video.fleu1-1.fna&edm=AGo2L-IEAAAA&oh=00_AfDMoCt5QVQN33LvqnQlS71aelYjHdt_PPdK5BziM80XIw&oe=63D6955E
error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfcjJfYXZjX2dlbjFhdmNfbGNfcTQwX2ZyYWdfMl92aWRlbyJ9&_nc_ohc=gXFNqgRhAcwAX8Yb-D0&_nc_ht=video.fleu1-1.fna&edm=AGo2L-IEAAAA&oh=00_AfDMoCt5QVQN33LvqnQlS71aelYjHdt_PPdK5BziM80XIw&oe=63D6955E/
dash.mpd:-1: error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfcjJfYXZjX2dlbjFhdmNfbGNfcTUwX2ZyYWdfMl92aWRlbyJ9&_nc_ohc=6f9UIoVRzpMAX9M_6S_&_nc_ht=video.fleu1-2.fna&edm=AGo2L-IEAAAA&oh=00_AfAVIx_3CNuMDq0Q6O_jQ1gU7rZGD1qjkS9Sn4f3I6ekNg&oe=63D6DE1E
error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfcjJfYXZjX2dlbjFhdmNfbGNfcTUwX2ZyYWdfMl92aWRlbyJ9&_nc_ohc=6f9UIoVRzpMAX9M_6S_&_nc_ht=video.fleu1-2.fna&edm=AGo2L-IEAAAA&oh=00_AfAVIx_3CNuMDq0Q6O_jQ1gU7rZGD1qjkS9Sn4f3I6ekNg&oe=63D6DE1E/
dash.mpd:-1: error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfcjJfYXZjX2dlbjFhdmNfbGNfcTYwX2ZyYWdfMl92aWRlbyJ9&_nc_ohc=SOsmeiEVSywAX-p9YlN&_nc_ht=video.fleu1-1.fna&edm=AGo2L-IEAAAA&oh=00_AfANHbgTW4WLZubbDDBk5afYfMjNLWu9jSpsSRjzKv6iVQ&oe=63D59A7E
error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfcjJfYXZjX2dlbjFhdmNfbGNfcTYwX2ZyYWdfMl92aWRlbyJ9&_nc_ohc=SOsmeiEVSywAX-p9YlN&_nc_ht=video.fleu1-1.fna&edm=AGo2L-IEAAAA&oh=00_AfANHbgTW4WLZubbDDBk5afYfMjNLWu9jSpsSRjzKv6iVQ&oe=63D59A7E/
dash.mpd:-1: error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfcjJfYXZjX2dlbjFhdmNfbGNfcTcwX2ZyYWdfMl92aWRlbyJ9&_nc_ohc=32rdvxB-5cUAX8YK0s3&_nc_ht=video.fleu1-1.fna&edm=AGo2L-IEAAAA&oh=00_AfDthNfPzYihPAVrPXm9Xy_hds7G82_bB8g8bVS2cFAoZg&oe=63D72EFC
error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfcjJfYXZjX2dlbjFhdmNfbGNfcTcwX2ZyYWdfMl92aWRlbyJ9&_nc_ohc=32rdvxB-5cUAX8YK0s3&_nc_ht=video.fleu1-1.fna&edm=AGo2L-IEAAAA&oh=00_AfDthNfPzYihPAVrPXm9Xy_hds7G82_bB8g8bVS2cFAoZg&oe=63D72EFC/
dash.mpd:-1: error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfcjJfYXZjX2dlbjFhdmNfbGNfcTgwX2ZyYWdfMl92aWRlbyJ9&_nc_ohc=2wtbU20szVsAX9nMvdW&_nc_ht=video.fleu1-2.fna&edm=AGo2L-IEAAAA&oh=00_AfD5aHEagrX72G6pzebuQv2sGa1PDt7pzZ0R3pOZes2ljQ&oe=63D6823D
error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfcjJfYXZjX2dlbjFhdmNfbGNfcTgwX2ZyYWdfMl92aWRlbyJ9&_nc_ohc=2wtbU20szVsAX9nMvdW&_nc_ht=video.fleu1-2.fna&edm=AGo2L-IEAAAA&oh=00_AfD5aHEagrX72G6pzebuQv2sGa1PDt7pzZ0R3pOZes2ljQ&oe=63D6823D/
dash.mpd:-1: error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfcjJfYXZjX2dlbjFhdmNfbGNfcTkwX2ZyYWdfMl92aWRlbyJ9&_nc_ohc=a9sD6YEzYzkAX9vy4Q9&_nc_ht=video.fleu1-2.fna&edm=AGo2L-IEAAAA&oh=00_AfAEJ8jltp7lMQcF2FdPP1ezL60Ak5pUi4u2YngiKq_hQw&oe=63D6A263
error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfcjJfYXZjX2dlbjFhdmNfbGNfcTkwX2ZyYWdfMl92aWRlbyJ9&_nc_ohc=a9sD6YEzYzkAX9vy4Q9&_nc_ht=video.fleu1-2.fna&edm=AGo2L-IEAAAA&oh=00_AfAEJ8jltp7lMQcF2FdPP1ezL60Ak5pUi4u2YngiKq_hQw&oe=63D6A263/
dash.mpd:-1: error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfYXVkaW9fYWFjcF82NF9mbm9ybTE0X2ZyYWdfMl9hdWRpbyJ9&_nc_ohc=baVxU1WspCUAX9iX-Lh&_nc_ht=video.fleu1-1.fna&edm=AGo2L-IEAAAA&oh=00_AfBUl1r3xv3JppBe1hPALfyVYde_1mDYnsHgWB_IrrM5vw&oe=63D70764
error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfYXVkaW9fYWFjcF82NF9mbm9ybTE0X2ZyYWdfMl9hdWRpbyJ9&_nc_ohc=baVxU1WspCUAX9iX-Lh&_nc_ht=video.fleu1-1.fna&edm=AGo2L-IEAAAA&oh=00_AfBUl1r3xv3JppBe1hPALfyVYde_1mDYnsHgWB_IrrM5vw&oe=63D70764/
[dash @ 0x7fa2b8000c80] DASH request for url 'dash.mpd', offset 0
[file @ 0x7fa2b801d7c0] Setting default whitelist 'file,crypto,data'
Format dash probed with size=2048 and score=100
:-1: error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfcjJfYXZjX2dlbjFhdmNfbGNfcTQwX2ZyYWdfMl92aWRlbyJ9&_nc_ohc=gXFNqgRhAcwAX8Yb-D0&_nc_ht=video.fleu1-1.fna&edm=AGo2L-IEAAAA&oh=00_AfDMoCt5QVQN33LvqnQlS71aelYjHdt_PPdK5BziM80XIw&oe=63D6955E
error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfcjJfYXZjX2dlbjFhdmNfbGNfcTQwX2ZyYWdfMl92aWRlbyJ9&_nc_ohc=gXFNqgRhAcwAX8Yb-D0&_nc_ht=video.fleu1-1.fna&edm=AGo2L-IEAAAA&oh=00_AfDMoCt5QVQN33LvqnQlS71aelYjHdt_PPdK5BziM80XIw&oe=63D6955E/
:-1: error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfcjJfYXZjX2dlbjFhdmNfbGNfcTUwX2ZyYWdfMl92aWRlbyJ9&_nc_ohc=6f9UIoVRzpMAX9M_6S_&_nc_ht=video.fleu1-2.fna&edm=AGo2L-IEAAAA&oh=00_AfAVIx_3CNuMDq0Q6O_jQ1gU7rZGD1qjkS9Sn4f3I6ekNg&oe=63D6DE1E
error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfcjJfYXZjX2dlbjFhdmNfbGNfcTUwX2ZyYWdfMl92aWRlbyJ9&_nc_ohc=6f9UIoVRzpMAX9M_6S_&_nc_ht=video.fleu1-2.fna&edm=AGo2L-IEAAAA&oh=00_AfAVIx_3CNuMDq0Q6O_jQ1gU7rZGD1qjkS9Sn4f3I6ekNg&oe=63D6DE1E/
:-1: error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfcjJfYXZjX2dlbjFhdmNfbGNfcTYwX2ZyYWdfMl92aWRlbyJ9&_nc_ohc=SOsmeiEVSywAX-p9YlN&_nc_ht=video.fleu1-1.fna&edm=AGo2L-IEAAAA&oh=00_AfANHbgTW4WLZubbDDBk5afYfMjNLWu9jSpsSRjzKv6iVQ&oe=63D59A7E
error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfcjJfYXZjX2dlbjFhdmNfbGNfcTYwX2ZyYWdfMl92aWRlbyJ9&_nc_ohc=SOsmeiEVSywAX-p9YlN&_nc_ht=video.fleu1-1.fna&edm=AGo2L-IEAAAA&oh=00_AfANHbgTW4WLZubbDDBk5afYfMjNLWu9jSpsSRjzKv6iVQ&oe=63D59A7E/
:-1: error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfcjJfYXZjX2dlbjFhdmNfbGNfcTcwX2ZyYWdfMl92aWRlbyJ9&_nc_ohc=32rdvxB-5cUAX8YK0s3&_nc_ht=video.fleu1-1.fna&edm=AGo2L-IEAAAA&oh=00_AfDthNfPzYihPAVrPXm9Xy_hds7G82_bB8g8bVS2cFAoZg&oe=63D72EFC
error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfcjJfYXZjX2dlbjFhdmNfbGNfcTcwX2ZyYWdfMl92aWRlbyJ9&_nc_ohc=32rdvxB-5cUAX8YK0s3&_nc_ht=video.fleu1-1.fna&edm=AGo2L-IEAAAA&oh=00_AfDthNfPzYihPAVrPXm9Xy_hds7G82_bB8g8bVS2cFAoZg&oe=63D72EFC/
:-1: error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfcjJfYXZjX2dlbjFhdmNfbGNfcTgwX2ZyYWdfMl92aWRlbyJ9&_nc_ohc=2wtbU20szVsAX9nMvdW&_nc_ht=video.fleu1-2.fna&edm=AGo2L-IEAAAA&oh=00_AfD5aHEagrX72G6pzebuQv2sGa1PDt7pzZ0R3pOZes2ljQ&oe=63D6823D
error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfcjJfYXZjX2dlbjFhdmNfbGNfcTgwX2ZyYWdfMl92aWRlbyJ9&_nc_ohc=2wtbU20szVsAX9nMvdW&_nc_ht=video.fleu1-2.fna&edm=AGo2L-IEAAAA&oh=00_AfD5aHEagrX72G6pzebuQv2sGa1PDt7pzZ0R3pOZes2ljQ&oe=63D6823D/
:-1: error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfcjJfYXZjX2dlbjFhdmNfbGNfcTkwX2ZyYWdfMl92aWRlbyJ9&_nc_ohc=a9sD6YEzYzkAX9vy4Q9&_nc_ht=video.fleu1-2.fna&edm=AGo2L-IEAAAA&oh=00_AfAEJ8jltp7lMQcF2FdPP1ezL60Ak5pUi4u2YngiKq_hQw&oe=63D6A263
error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfcjJfYXZjX2dlbjFhdmNfbGNfcTkwX2ZyYWdfMl92aWRlbyJ9&_nc_ohc=a9sD6YEzYzkAX9vy4Q9&_nc_ht=video.fleu1-2.fna&edm=AGo2L-IEAAAA&oh=00_AfAEJ8jltp7lMQcF2FdPP1ezL60Ak5pUi4u2YngiKq_hQw&oe=63D6A263/
:-1: error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfYXVkaW9fYWFjcF82NF9mbm9ybTE0X2ZyYWdfMl9hdWRpbyJ9&_nc_ohc=baVxU1WspCUAX9iX-Lh&_nc_ht=video.fleu1-1.fna&edm=AGo2L-IEAAAA&oh=00_AfBUl1r3xv3JppBe1hPALfyVYde_1mDYnsHgWB_IrrM5vw&oe=63D70764
error : unterminated entity reference ccb=1-7&_nc_sid=5aebc0&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfYXVkaW9fYWFjcF82NF9mbm9ybTE0X2ZyYWdfMl9hdWRpbyJ9&_nc_ohc=baVxU1WspCUAX9iX-Lh&_nc_ht=video.fleu1-1.fna&edm=AGo2L-IEAAAA&oh=00_AfBUl1r3xv3JppBe1hPALfyVYde_1mDYnsHgWB_IrrM5vw&oe=63D70764/
[dash @ 0x7fa2b8007240] DASH request for url '', offset 0
[dash @ 0x7fa2b8007240] Filename extension of '' is not a common multimedia extension, blocked for security reasons.
If you wish to override this adjust allowed_extensions, you can set it to 'ALL' to allow all
[dash @ 0x7fa2b8007240] Failed to open fragment of playlist
[dash @ 0x7fa2b8007240] Error when loading first fragment of playlist
[AVIOContext @ 0x7fa2b801e500] Statistics: 6606 bytes read, 0 seeks
[AVIOContext @ 0x7fa2b8009a00] Statistics: 6606 bytes read, 0 seeks
dash.mpd: Invalid data found when processing input

comment:4 by gorilla.maguila, 22 months ago

Probably xmlEncodeEntitiesReentrant() or xmlEncodeSpecialChars() must be used before xmlNodeSetContent()

​https://cpp.hotexamples.com/examples/-/-/xmlEncodeSpecialChars/cpp-xmlencodespecialchars-function-examples.html

comment:5 by paul31415, 10 months ago

For anyone interested, this fixes the issue:

diff --git a/libavformat/dashdec.c b/libavformat/dashdec.c
index 9ae28491e1ecd..41311d6e8f931 100644
--- a/libavformat/dashdec.c
+++ b/libavformat/dashdec.c
@@ -805,8 +805,10 @@ static int resolve_content_path(AVFormatContext *s, const char *url, int *max_ur
                 memset(p + 1, 0, strlen(p));
             }
             av_strlcat(tmp_str, text + start, tmp_max_url_size);
-            xmlNodeSetContent(baseurl_nodes[i], tmp_str);
+            xmlChar *escaped = xmlEncodeSpecialChars(NULL, tmp_str);
+            xmlNodeSetContent(baseurl_nodes[i], escaped);
             updated = 1;
+            xmlFree(escaped);
             xmlFree(text);
         }
     }
Last edited 9 months ago by paul31415 (previous) (diff)

comment:6 by paul31415, 10 months ago

<snip>

Last edited 9 months ago by paul31415 (previous) (diff)
Note: See TracTickets for help on using tickets.