git shallow clone is not working

[milaxnuts]

Summary of the bug:

I want to do a 'git shallow clone' of the ffmpeg repo,
to compile ffmpeg with libfdk_aac. [boo GPL.]

While the default 'git clone' works fine with ...

Receiving objects: 100% (560754/560754), 132.69 MiB | 2.79 MiB/s, done.

... the 'shallow clone' fails:

How to reproduce:

$ git clone --depth=1 https://git.ffmpeg.org/ffmpeg.git
Cloning into 'ffmpeg'...
fatal: the remote end hung up unexpectedly
fatal: protocol error: bad pack header

Probably related:
​https://bugs.launchpad.net/linaro-infrastructure-misc/+bug/1253139

From Apache log, when trying to clone a repository, we can find this:

fatal: Unable to create temporary file '/srv/repositories/lava/lava-server.git/shallow_zjbO0B': Permission denied

The repository directories under /srv/repositories on the server are set with the following access: rwxr-----, and are owned by git:git.

The process that runs git during a clone operation via smart HTTP is run under www-data, and www-data is part of the git group on the server, but does not have write access to the directories.

Just changing the permission on the directory solved the problem, and I was able to perform a clone operation.

.

Why shallow clone:

cos it saves bandwidth.

when i do a local shallow clone ...

$ git clone --depth=1 ffmpeg-full ffmpeg-shallow
Receiving objects: 100% (6934/6934), 14.57 MiB | 8.15 MiB/s, done.

... then i need almost 10 times less traffic.

Why not 'greedy' shallow clone?

greedy as in depth=1 compared to like depth=50

​https://bugs.eclipse.org/bugs/show_bug.cgi?id=436543

There are edge cases which could cause build issues when you have a really low depth. Such as if you are using polling based build triggers and multiple changes are happening rapidly, your build may miss a few commits.

.

with depth=50 i get

$ git clone --depth=50 ffmpeg-full ffmpeg-shallow-50
Receiving objects: 100% (7150/7150), 14.61 MiB | 8.14 MiB/s, done.

which is still better than 132.69 MiB

[Carl Eugen Hoyos]

What's wrong with git clone --depth=1 git://source.ffmpeg.org/ffmpeg.git?

[milaxnuts]

yay, that is working ... but, whats wrong?

1. that server is slow -- 107.00 KiB/s versus 2.79 MiB/s -- 2 minutes shallow vs 1 minute full clone

2. https makes me feel more secure -- is there a public login for ssh access?

3. http is more common

one workaround to get a 'shallow clone' via https ...

$ wget https://ffmpeg.org/releases/ffmpeg-snapshot.tar.bz2

which is only 10 megabyte
done after 5 seconds at 1.95 MiB/s
con: i cant choose the clone depth
fix: use the latest stable release -- i am missing a ffmpeg-latest.tar.bz2 that redirects to ffmpeg-4.1.tar.bz2, for example

another workaround is to use microhub

$ git clone --depth=50 https://github.com/FFmpeg/FFmpeg.git ffmpeg

done after 20 seconds at 1.95 MiB/s

[Carl Eugen Hoyos]

Replying to milaxnuts:

i am missing a ffmpeg-latest.tar.bz2 that redirects to ffmpeg-4.1.tar.bz2, for example

Since we offer no release support, this would be counter-productive.

[Carl Eugen Hoyos]

I believe this issue was fixed.