Opened 6 years ago

Closed 5 years ago

#7310 closed defect (fixed)

configure broken with GNU sed version 4.1.5

Reported by: Carl Eugen Hoyos Owned by:
Priority: important Component: build system
Version: git-master Keywords: regression
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Current FFmpeg fails to configure libavfilter with older GNU sed.

$ ./configure
sed: invalid option -- 'E'
Usage: sed [OPTION]... {script-only-if-no-other-script} [input-file]...

  -n, --quiet, --silent
                 suppress automatic printing of pattern space
  -e script, --expression=script
                 add the script to the commands to be executed
  -f script-file, --file=script-file
                 add the contents of script-file to the commands to be executed
  --follow-symlinks
                 follow symlinks when processing in place
  -i[SUFFIX], --in-place[=SUFFIX]
                 edit files in place (makes backup if extension supplied)
  -l N, --line-length=N
                 specify the desired line-wrap length for the `l' command
  --posix
                 disable all GNU extensions.
  -r, --regexp-extended
                 use extended regular expressions in the script.
  -s, --separate
                 consider files as separate rather than as a single continuous
                 long stream.
  -u, --unbuffered
                 load minimal amounts of data from the input files and flush
                 the output buffers more often
      --help     display this help and exit
      --version  output version information and exit

If no -e, --expression, -f, or --file option is given, then the first
non-option argument is taken as the sed script to interpret.  All
remaining arguments are names of input files; if no input files are
specified, then the standard input is read.

E-mail bug reports to: bonzini@gnu.org .
Be sure to include the word ``sed'' somewhere in the ``Subject:'' field.
install prefix            /usr/local
source path               .
C compiler                gcc
C library                 glibc
ARCH                      x86 (generic)
big-endian                no
runtime cpu detection     yes
standalone assembly       yes
x86 assembler             nasm
MMX enabled               yes
MMXEXT enabled            yes
3DNow! enabled            yes
3DNow! extended enabled   yes
SSE enabled               yes
SSSE3 enabled             yes
AESNI enabled             yes
AVX enabled               yes
AVX2 enabled              yes
AVX-512 enabled           yes
XOP enabled               yes
FMA3 enabled              yes
FMA4 enabled              yes
i686 features enabled     yes
CMOV is fast              no
EBX available             yes
EBP available             yes
debug symbols             yes
strip symbols             yes
optimize for size         no
optimizations             yes
static                    yes
shared                    no
postprocessing support    no
network support           yes
threading support         pthreads
safe bitstream reader     yes
texi2html enabled         no
perl enabled              yes
pod2man enabled           yes
makeinfo enabled          no
makeinfo supports HTML    no

External libraries:
alsa                       iconv                      zlib
bzlib                      xlib

External libraries providing hardware acceleration:
xvmc

Libraries:
avcodec                    avformat                   swresample
avdevice                   avutil                     swscale
avfilter

Programs:
ffmpeg                     ffprobe

Enabled decoders:
aac                        atrac3p                    eightsvx_fib
aac_fixed                  atrac3pal                  escape124
aac_latm                   atrac9                     escape130
aasc                       aura                       evrc
ac3                        aura2                      exr
ac3_fixed                  avrn                       ffv1
adpcm_4xm                  avrp                       ffvhuff
adpcm_adx                  avs                        ffwavesynth
adpcm_afc                  avui                       fic
adpcm_aica                 ayuv                       fits
adpcm_ct                   bethsoftvid                flac
adpcm_dtk                  bfi                        flashsv
adpcm_ea                   bink                       flashsv2
adpcm_ea_maxis_xa          binkaudio_dct              flic
adpcm_ea_r1                binkaudio_rdft             flv
adpcm_ea_r2                bintext                    fmvc
adpcm_ea_r3                bitpacked                  fourxm
adpcm_ea_xas               bmp                        fraps
adpcm_g722                 bmv_audio                  frwu
adpcm_g726                 bmv_video                  g2m
adpcm_g726le               brender_pix                g723_1
adpcm_ima_amv              c93                        g729
adpcm_ima_apc              cavs                       gdv
adpcm_ima_dat4             ccaption                   gif
adpcm_ima_dk3              cdgraphics                 gremlin_dpcm
adpcm_ima_dk4              cdxl                       gsm
adpcm_ima_ea_eacs          cfhd                       gsm_ms
adpcm_ima_ea_sead          cinepak                    h261
adpcm_ima_iss              clearvideo                 h263
adpcm_ima_oki              cljr                       h263i
adpcm_ima_qt               cllc                       h263p
adpcm_ima_rad              comfortnoise               h264
adpcm_ima_smjpeg           cook                       hap
adpcm_ima_wav              cpia                       hevc
adpcm_ima_ws               cscd                       hnm4_video
adpcm_ms                   cyuv                       hq_hqa
adpcm_mtaf                 dca                        hqx
adpcm_psx                  dds                        huffyuv
adpcm_sbpro_2              dfa                        iac
adpcm_sbpro_3              dirac                      idcin
adpcm_sbpro_4              dnxhd                      idf
adpcm_swf                  dolby_e                    iff_ilbm
adpcm_thp                  dpx                        imc
adpcm_thp_le               dsd_lsbf                   indeo2
adpcm_vima                 dsd_lsbf_planar            indeo3
adpcm_xa                   dsd_msbf                   indeo4
adpcm_yamaha               dsd_msbf_planar            indeo5
aic                        dsicinaudio                interplay_acm
alac                       dsicinvideo                interplay_dpcm
alias_pix                  dss_sp                     interplay_video
als                        dst                        jacosub
amrnb                      dvaudio                    jpeg2000
amrwb                      dvbsub                     jpegls
amv                        dvdsub                     jv
anm                        dvvideo                    kgv1
ansi                       dxa                        kmvc
ape                        dxtory                     lagarith
apng                       dxv                        loco
aptx                       eac3                       m101
aptx_hd                    eacmv                      mace3
ass                        eamad                      mace6
asv1                       eatgq                      magicyuv
asv2                       eatgv                      mdec
atrac1                     eatqi                      metasound
atrac3                     eightbps                   microdvd
atrac3al                   eightsvx_exp               mimic
mjpeg                      pcm_s32le_planar           sp5x
mjpegb                     pcm_s64be                  speedhq
mlp                        pcm_s64le                  srgc
mmvideo                    pcm_s8                     srt
motionpixels               pcm_s8_planar              ssa
movtext                    pcm_u16be                  stl
mp1                        pcm_u16le                  subrip
mp1float                   pcm_u24be                  subviewer
mp2                        pcm_u24le                  subviewer1
mp2float                   pcm_u32be                  sunrast
mp3                        pcm_u32le                  svq1
mp3adu                     pcm_u8                     svq3
mp3adufloat                pcm_zork                   tak
mp3float                   pcx                        targa
mp3on4                     pgm                        targa_y216
mp3on4float                pgmyuv                     tdsc
mpc7                       pgssub                     text
mpc8                       pictor                     theora
mpeg1video                 pixlet                     thp
mpeg2video                 pjs                        tiertexseqvideo
mpeg4                      png                        tiff
mpegvideo                  ppm                        tmv
mpl2                       prores                     truehd
msa1                       psd                        truemotion1
mscc                       ptx                        truemotion2
msmpeg4v1                  qcelp                      truemotion2rt
msmpeg4v2                  qdm2                       truespeech
msmpeg4v3                  qdmc                       tscc
msrle                      qdraw                      tscc2
mss1                       qpeg                       tta
mss2                       qtrle                      twinvq
msvideo1                   r10k                       txd
mszh                       r210                       ulti
mts2                       ra_144                     utvideo
mvc1                       ra_288                     v210
mvc2                       ralf                       v210x
mxpeg                      rawvideo                   v308
nellymoser                 realtext                   v408
nuv                        rl2                        v410
on2avc                     roq                        vb
opus                       roq_dpcm                   vble
paf_audio                  rpza                       vc1
paf_video                  rscc                       vc1image
pam                        rv10                       vcr1
pbm                        rv20                       vmdaudio
pcm_alaw                   rv30                       vmdvideo
pcm_bluray                 rv40                       vmnc
pcm_dvd                    s302m                      vorbis
pcm_f16le                  sami                       vp3
pcm_f24le                  sanm                       vp5
pcm_f32be                  sbc                        vp6
pcm_f32le                  scpr                       vp6a
pcm_f64be                  screenpresso               vp6f
pcm_f64le                  sdx2_dpcm                  vp7
pcm_lxf                    sgi                        vp8
pcm_mulaw                  sgirle                     vp9
pcm_s16be                  sheervideo                 vplayer
pcm_s16be_planar           shorten                    vqa
pcm_s16le                  sipr                       wavpack
pcm_s16le_planar           smackaud                   webp
pcm_s24be                  smacker                    webvtt
pcm_s24daud                smc                        wmalossless
pcm_s24le                  smvjpeg                    wmapro
pcm_s24le_planar           snow                       wmav1
pcm_s32be                  sol_dpcm                   wmav2
pcm_s32le                  sonic                      wmavoice
wmv1                       xan_wc4                    xwd
wmv2                       xbin                       y41p
wmv3                       xbm                        ylc
wmv3image                  xface                      yop
wnv1                       xl                         yuv4
wrapped_avframe            xma1                       zero12v
ws_snd1                    xma2                       zerocodec
xan_dpcm                   xpm                        zlib
xan_wc3                    xsub                       zmbv

Enabled encoders:
a64multi                   jpegls                     prores
a64multi5                  ljpeg                      prores_aw
aac                        magicyuv                   prores_ks
ac3                        mjpeg                      qtrle
ac3_fixed                  mlp                        r10k
adpcm_adx                  movtext                    r210
adpcm_g722                 mp2                        ra_144
adpcm_g726                 mp2fixed                   rawvideo
adpcm_g726le               mpeg1video                 roq
adpcm_ima_qt               mpeg2video                 roq_dpcm
adpcm_ima_wav              mpeg4                      rv10
adpcm_ms                   msmpeg4v2                  rv20
adpcm_swf                  msmpeg4v3                  s302m
adpcm_yamaha               msvideo1                   sbc
alac                       nellymoser                 sgi
alias_pix                  opus                       snow
amv                        pam                        sonic
apng                       pbm                        sonic_ls
aptx                       pcm_alaw                   srt
aptx_hd                    pcm_f32be                  ssa
ass                        pcm_f32le                  subrip
asv1                       pcm_f64be                  sunrast
asv2                       pcm_f64le                  svq1
avrp                       pcm_mulaw                  targa
avui                       pcm_s16be                  text
ayuv                       pcm_s16be_planar           tiff
bmp                        pcm_s16le                  truehd
cinepak                    pcm_s16le_planar           tta
cljr                       pcm_s24be                  utvideo
comfortnoise               pcm_s24daud                v210
dca                        pcm_s24le                  v308
dnxhd                      pcm_s24le_planar           v408
dpx                        pcm_s32be                  v410
dvbsub                     pcm_s32le                  vc2
dvdsub                     pcm_s32le_planar           vorbis
dvvideo                    pcm_s64be                  wavpack
eac3                       pcm_s64le                  webvtt
ffv1                       pcm_s8                     wmav1
ffvhuff                    pcm_s8_planar              wmav2
fits                       pcm_u16be                  wmv1
flac                       pcm_u16le                  wmv2
flashsv                    pcm_u24be                  wrapped_avframe
flashsv2                   pcm_u24le                  xbm
flv                        pcm_u32be                  xface
g723_1                     pcm_u32le                  xsub
gif                        pcm_u8                     xwd
h261                       pcx                        y41p
h263                       pgm                        yuv4
h263p                      pgmyuv                     zlib
huffyuv                    png                        zmbv
jpeg2000                   ppm

Enabled hwaccels:
mpeg1_xvmc                 mpeg2_xvmc

Enabled parsers:
aac                        dvdsub                     png
aac_latm                   flac                       pnm
ac3                        g729                       rv30
adx                        gsm                        rv40
bmp                        h261                       sbc
cavsvideo                  h263                       sipr
cook                       h264                       tak
dca                        hevc                       vc1
dirac                      mjpeg                      vorbis
dnxhd                      mlp                        vp3
dpx                        mpeg4video                 vp8
dvaudio                    mpegaudio                  vp9
dvbsub                     mpegvideo                  xma
dvd_nav                    opus

Enabled demuxers:
aa                         eac3                       ivr
aac                        epaf                       jacosub
ac3                        ffmetadata                 jv
acm                        filmstrip                  live_flv
act                        fits                       lmlm4
adf                        flac                       loas
adp                        flic                       lrc
ads                        flv                        lvf
adx                        fourxm                     lxf
aea                        frm                        m4v
afc                        fsb                        matroska
aiff                       g722                       mgsts
aix                        g723_1                     microdvd
amr                        g726                       mjpeg
amrnb                      g726le                     mjpeg_2000
amrwb                      g729                       mlp
anm                        gdv                        mlv
apc                        genh                       mm
ape                        gif                        mmf
apng                       gsm                        mov
aptx                       gxf                        mp3
aptx_hd                    h261                       mpc
aqtitle                    h263                       mpc8
asf                        h264                       mpegps
asf_o                      hevc                       mpegts
ass                        hls                        mpegtsraw
ast                        hnm                        mpegvideo
au                         ico                        mpjpeg
avi                        idcin                      mpl2
avr                        idf                        mpsub
avs                        iff                        msf
bethsoftvid                ilbc                       msnwc_tcp
bfi                        image2                     mtaf
bfstm                      image2_alias_pix           mtv
bink                       image2_brender_pix         musx
bintext                    image2pipe                 mv
bit                        image_bmp_pipe             mvi
bmv                        image_dds_pipe             mxf
boa                        image_dpx_pipe             mxg
brstm                      image_exr_pipe             nc
c93                        image_j2k_pipe             nistsphere
caf                        image_jpeg_pipe            nsp
cavsvideo                  image_jpegls_pipe          nsv
cdg                        image_pam_pipe             nut
cdxl                       image_pbm_pipe             nuv
cine                       image_pcx_pipe             ogg
codec2                     image_pgm_pipe             oma
codec2raw                  image_pgmyuv_pipe          paf
concat                     image_pictor_pipe          pcm_alaw
data                       image_png_pipe             pcm_f32be
daud                       image_ppm_pipe             pcm_f32le
dcstr                      image_psd_pipe             pcm_f64be
dfa                        image_qdraw_pipe           pcm_f64le
dirac                      image_sgi_pipe             pcm_mulaw
dnxhd                      image_sunrast_pipe         pcm_s16be
dsf                        image_svg_pipe             pcm_s16le
dsicin                     image_tiff_pipe            pcm_s24be
dss                        image_webp_pipe            pcm_s24le
dts                        image_xpm_pipe             pcm_s32be
dtshd                      image_xwd_pipe             pcm_s32le
dv                         ingenient                  pcm_s8
dvbsub                     ipmovie                    pcm_u16be
dvbtxt                     ircam                      pcm_u16le
dxa                        iss                        pcm_u24be
ea                         iv8                        pcm_u24le
ea_cdata                   ivf                        pcm_u32be
pcm_u32le                  shorten                    v210x
pcm_u8                     siff                       vag
pjs                        sln                        vc1
pmp                        smacker                    vc1t
pva                        smjpeg                     vivo
pvf                        smush                      vmd
qcp                        sol                        vobsub
r3d                        sox                        voc
rawvideo                   spdif                      vpk
realtext                   srt                        vplayer
redspark                   stl                        vqf
rl2                        str                        w64
rm                         subviewer                  wav
roq                        subviewer1                 wc3
rpl                        sup                        webm_dash_manifest
rsd                        svag                       webvtt
rso                        swf                        wsaud
rtp                        tak                        wsd
rtsp                       tedcaptions                wsvqa
s337m                      thp                        wtv
sami                       threedostr                 wv
sap                        tiertexseq                 wve
sbc                        tmv                        xa
sbg                        truehd                     xbin
scc                        tta                        xmv
sdp                        tty                        xvag
sdr2                       txd                        xwma
sds                        ty                         yop
sdx                        v210                       yuv4mpegpipe
segafilm

Enabled muxers:
a64                        hls                        pcm_s24le
ac3                        ico                        pcm_s32be
adts                       ilbc                       pcm_s32le
adx                        image2                     pcm_s8
aiff                       image2pipe                 pcm_u16be
amr                        ipod                       pcm_u16le
apng                       ircam                      pcm_u24be
aptx                       ismv                       pcm_u24le
aptx_hd                    ivf                        pcm_u32be
asf                        jacosub                    pcm_u32le
asf_stream                 latm                       pcm_u8
ass                        lrc                        psp
ast                        m4v                        rawvideo
au                         matroska                   rm
avi                        matroska_audio             roq
avm2                       md5                        rso
bit                        microdvd                   rtp
caf                        mjpeg                      rtp_mpegts
cavsvideo                  mkvtimestamp_v2            rtsp
codec2                     mlp                        sap
codec2raw                  mmf                        sbc
crc                        mov                        scc
dash                       mp2                        segafilm
data                       mp3                        segment
daud                       mp4                        singlejpeg
dirac                      mpeg1system                smjpeg
dnxhd                      mpeg1vcd                   smoothstreaming
dts                        mpeg1video                 sox
dv                         mpeg2dvd                   spdif
eac3                       mpeg2svcd                  spx
f4v                        mpeg2video                 srt
ffmetadata                 mpeg2vob                   stream_segment
fifo                       mpegts                     sup
fifo_test                  mpjpeg                     swf
filmstrip                  mxf                        tee
fits                       mxf_d10                    tg2
flac                       mxf_opatom                 tgp
flv                        null                       truehd
framecrc                   nut                        tta
framehash                  oga                        uncodedframecrc
framemd5                   ogg                        vc1
g722                       ogv                        vc1t
g723_1                     oma                        voc
g726                       opus                       w64
g726le                     pcm_alaw                   wav
gif                        pcm_f32be                  webm
gsm                        pcm_f32le                  webm_chunk
gxf                        pcm_f64be                  webm_dash_manifest
h261                       pcm_f64le                  webp
h263                       pcm_mulaw                  webvtt
h264                       pcm_s16be                  wtv
hash                       pcm_s16le                  wv
hds                        pcm_s24be                  yuv4mpegpipe
hevc

Enabled protocols:
async                      http                       rtmpt
cache                      httpproxy                  rtp
concat                     icecast                    srtp
crypto                     md5                        subfile
data                       mmsh                       tcp
ffrtmphttp                 mmst                       tee
file                       pipe                       udp
ftp                        prompeg                    udplite
gopher                     rtmp                       unix
hls

Enabled filters:

Enabled bsfs:
aac_adtstoasc              hapqa_extract              noise
chomp                      hevc_metadata              null
dca_core                   hevc_mp4toannexb           remove_extradata
dump_extradata             imx_dump_header            text2movsub
eac3_core                  mjpeg2jpeg                 trace_headers
extract_extradata          mjpega_dump_header         vp9_metadata
filter_units               mov2textsub                vp9_raw_reorder
h264_metadata              mp3_header_decompress      vp9_superframe
h264_mp4toannexb           mpeg2_metadata             vp9_superframe_split
h264_redundant_pps         mpeg4_unpack_bframes

Enabled indevs:
alsa                       lavfi                      v4l2
fbdev                      oss

Enabled outdevs:
alsa                       oss                        xv
fbdev                      v4l2

License: LGPL version 2.1 or later
libavutil/avconfig.h is unchanged
libavcodec/codec_list.c is unchanged
libavcodec/parser_list.c is unchanged
libavcodec/bsf_list.c is unchanged
libavformat/demuxer_list.c is unchanged
libavformat/muxer_list.c is unchanged
libavdevice/indev_list.c is unchanged
libavdevice/outdev_list.c is unchanged
libavformat/protocol_list.c is unchanged
$ grep CONFIG_FILTERS config.h
#define CONFIG_FILTERS 0
$ cat libavfilter/filter_list.c
static const AVFilter * const filter_list[] = {
    &ff_asrc_abuffer,
    &ff_vsrc_buffer,
    &ff_asink_abuffer,
    &ff_vsink_buffer,
    NULL };

Change History (9)

comment:1 by Carl Eugen Hoyos, 6 years ago

Priority: normalimportant

Regression since 8f1382f80e0d4184c54c14afdda6482f050fbba7, fails at least with AIX sed and Solaris sed.

comment:2 by James, 6 years ago

Newer sed shows

$ sed --help
Usage: sed [OPTION]... {script-only-if-no-other-script} [input-file]...

  -n, --quiet, --silent
                 suppress automatic printing of pattern space
  -e script, --expression=script
                 add the script to the commands to be executed
  -f script-file, --file=script-file
                 add the contents of script-file to the commands to be executed
  --follow-symlinks
                 follow symlinks when processing in place
  -i[SUFFIX], --in-place[=SUFFIX]
                 edit files in place (makes backup if SUFFIX supplied)
  -b, --binary
                 open files in binary mode (CR+LFs are not processed specially)
  -l N, --line-length=N
                 specify the desired line-wrap length for the `l' command
  --posix
                 disable all GNU extensions.
  -E, -r, --regexp-extended
                 use extended regular expressions in the script
                 (for portability use POSIX -E).
  -s, --separate
                 consider files as separate rather than as a single,
                 continuous long stream.
      --sandbox
                 operate in sandbox mode (disable e/r/w commands).
  -u, --unbuffered
                 load minimal amounts of data from the input files and flush
                 the output buffers more often
  -z, --null-data
                 separate lines by NUL characters
      --help     display this help and exit
      --version  output version information and exit

So looks like it should be a matter of replacing that -E with -r, assuming it doesn't break any other target.

comment:3 by Carl Eugen Hoyos, 6 years ago

That wouldn't help on Solaris or aix.

comment:4 by James, 6 years ago

The output you pasted reports -r as a valid option. Is it not the sed available in Solaris and/or AIX? Am i missing something?

comment:5 by Carl Eugen Hoyos, 6 years ago

No, extended regular expressions are neither available on Solaris nor aix, they are a gnu extension.

Last edited 6 years ago by Carl Eugen Hoyos (previous) (diff)

comment:6 by Carl Eugen Hoyos, 6 years ago

This is made worse by the fact that our configure script on Solaris insists on using /usr/xpg4/bin/sed and the user cannot change this.

comment:7 by michael.kostylev, 5 years ago

If a user cannot replace sed or does not want to patch configure, then one can override stat("usr/xpg4/bin/sed",...)" via LD_PRELOAD. Obviously, this is overkill, but it works. Probably, the LD_PRELOAD trick would be more useful when a user cannot replace (broken) /bin/sh to run the FATE scripts.
Going back to sed. Can someone test the following:

% for sed in $(which -a sed) ; do echo "$sed:" ; for expr in av_AV av_AZ1 avx_z2 avfrc_b34 aaaaaa_X5 avsin_Abc_09 ; do echo 'extern AVFilter ff_'$expr';' | $sed -n 's/^extern AVFilter ff_[avfsinkrc]\{2,5\}_\([a-zA-Z0-9_]\{1,\}\);/\1_filter/p' ; done ; echo ; done

The output should be like this:

/usr/xpg4/bin/sed:
AV_filter
AZ1_filter
b34_filter
Abc_09_filter

/opt/csw/gnu/sed:
AV_filter
AZ1_filter
b34_filter
Abc_09_filter

/usr/bin/sed:
AV_filter
AZ1_filter
b34_filter
Abc_09_filter

In that case we can replace the first ERE with the BRE:

-    sed -E -n "s/^extern AVFilter ff_([avfsinkrc]{2,5})_([a-zA-Z0-9_]+);/\2_filter/p" $file
+    sed -n "s/^extern AVFilter ff_[avfsinkrc]\{2,5\}_\([a-zA-Z0-9_]\{1,\}\);/\1_filter/p" $file

comment:8 by michael.kostylev, 5 years ago

The above quick test passed on FreeBSD 11.2, OpenBSD 6.2, Solaris 10,11, OpenIndiana 2017.10, AIX 7.1, numerous ancient Linuxes.
And FATE passed on Sol 11.

comment:9 by Carl Eugen Hoyos, 5 years ago

Resolution: fixed
Status: newclosed
Note: See TracTickets for help on using tickets.