'file' maintainer? (or fun with PIE and magic)
Anyone know how to get in touch with the maintainers of 'file'? Seems the links in the man pages and Ian Darwin's site - http://www.darwinsys.com/file/ - don't work. The file magic database needs an update to correctly recognize PIE (Position Independent Executable) x86 ELF binaries as application/x-executable. This might seem an incredibly trivial thing, but it effectively stops graphical file managers from executing binaries, as they use magic (5) to identify files. Debian switched to making PIE a default for gcc for security reasons, but probably didn't expect it to break graphical UIs. Here's what I'm seeing: Expected behaviour: $ echo "int main() { return 0; }" > foo.c $ gcc -o foo foo.c $ file foo foo: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=6e7749f995a89a53f74ec29d3c16fcf3f56be90f, not stripped $ file --mime-type foo foo: application/x-executable Actual behaviour: $ echo "int main() { return 0; }" > foo.c $ gcc -o foo foo.c $ file foo foo: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=6e7749f995a89a53f74ec29d3c16fcf3f56be90f, not stripped $ file --mime-type foo foo: application/x-sharedlib Workaround: $ echo "int main() { return 0; }" > foo.c $ gcc -o foo-nopie foo.c -no-pie $ file foo-nopie foo-nopie: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=3eb8c581f43c19997e3c828f5a9730dbdc794470, not stripped $ file --mime-type foo-nopie foo-nopie: application/x-executable I'm a bit worried that the workaround allows less safe binaries to be double-clicked and run. I'm not sure how much of a security issue PIE vs non-PIE is, though. cheers, Stewart
I just sent Ian a note; hopefully his email isn't also down, as his web site seems to be... If he contacts you asking more, that's great!
Hey, Stewart, On Wednesday, February 07 2018, Stewart C. Russell via talk wrote:
Anyone know how to get in touch with the maintainers of 'file'? Seems the links in the man pages and Ian Darwin's site - http://www.darwinsys.com/file/ - don't work. The file magic database needs an update to correctly recognize PIE (Position Independent Executable) x86 ELF binaries as application/x-executable.
Unfortunately it seems that "file" is not "properly maintained" in the sense that the project doesn't have a trivial way to receive contributions. In this scenario, what I recommend is to file a bug downstream (against Debian's "file", for example), and ask the maintainer to forward the fix upstream.
This might seem an incredibly trivial thing, but it effectively stops graphical file managers from executing binaries, as they use magic (5) to identify files. Debian switched to making PIE a default for gcc for security reasons, but probably didn't expect it to break graphical UIs.
Here's what I'm seeing:
Expected behaviour:
$ echo "int main() { return 0; }" > foo.c $ gcc -o foo foo.c $ file foo foo: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=6e7749f995a89a53f74ec29d3c16fcf3f56be90f, not stripped $ file --mime-type foo foo: application/x-executable
Actual behaviour:
$ echo "int main() { return 0; }" > foo.c $ gcc -o foo foo.c $ file foo foo: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=6e7749f995a89a53f74ec29d3c16fcf3f56be90f, not stripped $ file --mime-type foo foo: application/x-sharedlib
This is expected, as you noted. The problem here is not only the magic number, but the fact that the binary has its ELF e_type marked as ET_DYN, and not ET_EXEC. This is proposital; GCC does that because it is possible to create shared libraries that are also executables when you use -pie. It seems to me that perhaps the graphical UI could rely not only on the MIME type of a file, but also if it is marked as executable or not. Debian explicitly advises (in the form of a lintian error) against having the executable bit set for libraries, so only executable files will have +x.
Workaround:
$ echo "int main() { return 0; }" > foo.c $ gcc -o foo-nopie foo.c -no-pie $ file foo-nopie foo-nopie: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=3eb8c581f43c19997e3c828f5a9730dbdc794470, not stripped $ file --mime-type foo-nopie foo-nopie: application/x-executable
I'm a bit worried that the workaround allows less safe binaries to be double-clicked and run. I'm not sure how much of a security issue PIE vs non-PIE is, though.
Yeah, this is not really a workaround per se, because it disables PIE when compiling the binary. Having PIE enabled is a nice security feature, so I would recommend against doing that. -- Sergio GPG key ID: 237A 54B1 0287 28BF 00EF 31F4 D0EB 7628 65FC 5E36 Please send encrypted e-mail if possible http://sergiodj.net/
| From: Sergio Durigan Junior via talk <talk@gtalug.org> | It seems to me that perhaps the graphical UI could rely not only on the | MIME type of a file, but also if it is marked as executable or not. Some filesystems don't have execute permission bits (FAT etc.) and Linux pastes the same setting on all the files, based on mount flags (if I remember correctly). So execute permission is not proof that the thing is executable. This does not invalidate your suggestion.
On Wednesday, February 07 2018, D. Hugh Redelmeier via talk wrote:
| From: Sergio Durigan Junior via talk <talk@gtalug.org>
| It seems to me that perhaps the graphical UI could rely not only on the | MIME type of a file, but also if it is marked as executable or not.
Some filesystems don't have execute permission bits (FAT etc.) and Linux pastes the same setting on all the files, based on mount flags (if I remember correctly). So execute permission is not proof that the thing is executable.
Ah, right, thanks for complementing I guess purposedly forgot about FAT ;-). But yeah, my suggestion is a really blind shot coming from someone who doesn't use graphic file managers, so definitely take it with a grain of salt. Cheers, -- Sergio GPG key ID: 237A 54B1 0287 28BF 00EF 31F4 D0EB 7628 65FC 5E36 Please send encrypted e-mail if possible http://sergiodj.net/
On 2018-02-07 02:54 PM, Sergio Durigan Junior wrote:
Unfortunately it seems that "file" is not "properly maintained" in the sense that the project doesn't have a trivial way to receive contributions.
But it does help when list members like Chris know the original developer and can get you the contact details of the most-upstream-of-all maintainer! Unfortunately, the maintainer doesn't consider it a bug:
Thanks for the bug report; only this is not a bug. PIE executables are shared objects. There is no way to tell them apart from shared objects.
So we're kind of stuck.
In this scenario, what I recommend is to file a bug downstream (against Debian's "file", for example), and ask the maintainer to forward the fix upstream.
I did, with Ubuntu: https://bugs.launchpad.net/ubuntu/+source/file/+bug/1747711 Unfortunately, if this really isn't something that the file (1) maintainers can fix, I'm going to have to file bugs with the Gnome Files/Nautilus (Gnome), PCManFM (LXDE), Konqueror (KDE) and Thunar (XFCE) folks to advise them not to rely on magic (5) alone now.
It seems to me that perhaps the graphical UI could rely not only on the MIME type of a file, but also if it is marked as executable or not.
That would be sensible, yes.
Debian explicitly advises (in the form of a lintian error) against having the executable bit set for libraries, so only executable files will have +x.
But now we'll need to fix lintian, as Debian binaries are being supplied as shared objects. If you've got a recently-updated Debian Stretch distro running, binaries such as /usr/bin/curl are PIE shared objects …
Yeah, this is not really a workaround per se, because it disables PIE when compiling the binary. Having PIE enabled is a nice security feature, so I would recommend against doing that.
For sure. But since naïve users (and me ☺) use graphical file managers, and the only way to get binaries to run these days is to make them non-PIE, we have a security problem. cheers, Stewart
On Thursday, February 08 2018, Stewart C. Russell via talk wrote:
On 2018-02-07 02:54 PM, Sergio Durigan Junior wrote:
Thanks for the bug report; only this is not a bug. PIE executables are shared objects. There is no way to tell them apart from shared objects.
So we're kind of stuck.
Yeah, I tend to agree with the maintainer here, although I find the term "shared object" a bit overloaded. There are executable and non-executable shared objects (see below), so maybe "file" could be more explicit.
In this scenario, what I recommend is to file a bug downstream (against Debian's "file", for example), and ask the maintainer to forward the fix upstream.
I did, with Ubuntu: https://bugs.launchpad.net/ubuntu/+source/file/+bug/1747711
Unfortunately, if this really isn't something that the file (1) maintainers can fix, I'm going to have to file bugs with the Gnome Files/Nautilus (Gnome), PCManFM (LXDE), Konqueror (KDE) and Thunar (XFCE) folks to advise them not to rely on magic (5) alone now.
While it is questionable whose bug this is, I think it's really with these tools that the discussion should begin. However, it's interesting to see the first reply of <https://bugzilla.gnome.org/show_bug.cgi?id=737849> (a bug that is linked in your bug report): nautilus is not an application launcher, really. But using a desktop file to launch your app works just fine here; using both gtk-launch or nautilus. What problem are you seeing with that ? So I guess there may be a different interpretation of what "executing a file using a file manager" means. There's also a very interesting comment later on (comment #10), saying that Nautilus doesn't rely on "file", but rather on other libraries which provide the "MIME guessing" for it.
It seems to me that perhaps the graphical UI could rely not only on the MIME type of a file, but also if it is marked as executable or not.
That would be sensible, yes.
Debian explicitly advises (in the form of a lintian error) against having the executable bit set for libraries, so only executable files will have +x.
But now we'll need to fix lintian, as Debian binaries are being supplied as shared objects. If you've got a recently-updated Debian Stretch distro running, binaries such as /usr/bin/curl are PIE shared objects …
No fix is needed for lintian, because a shared library is not the same thing as an executable shared object. The latter has a PT_INTERP entry which specifies that it needs an interpreter to run (in the case of an executable, it's ld.so). So the rule still applies: shared libraries can't have their execution bit set, but executable shared objects can.
Yeah, this is not really a workaround per se, because it disables PIE when compiling the binary. Having PIE enabled is a nice security feature, so I would recommend against doing that.
For sure. But since naïve users (and me ☺) use graphical file managers, and the only way to get binaries to run these days is to make them non-PIE, we have a security problem.
Agreed. Thanks, -- Sergio GPG key ID: 237A 54B1 0287 28BF 00EF 31F4 D0EB 7628 65FC 5E36 Please send encrypted e-mail if possible http://sergiodj.net/
On 2018-02-08 02:47 PM, Sergio Durigan Junior wrote:
Yeah, I tend to agree with the maintainer here, although I find the term "shared object" a bit overloaded.
Good news is that he *has* decided to make the change, since so many packages depend on what file/magic does.
So I guess there may be a different interpretation of what "executing a file using a file manager" means.
I'm not sure Nautilus is a shining example. First off, they seem hell-bent on removing useful features: current releases make it deliberately difficult to use more than one tool to open files, and the next release will have *no* desktop integration. No icons on the desktop for us pesky users who have expected it for the last ~30 years! Secondly, the attitude of "just make a desktop file" is utterly infuriating. I can probably successfully make these 1 out of 3, and unless it's your job to do it, it's really not obvious how it's done. (The 1 out of the 3 I can get working is because I have a working application launcher desktop file, and I just sed out the executable name …) I was really hopeful that 2018 would be the year of Linux on the desktop, too.
There's also a very interesting comment later on (comment #10), saying that Nautilus doesn't rely on "file", but rather on other libraries which provide the "MIME guessing" for it.
oh great — another point of failure! So if even the original file/magic system is updated, Gnome's will lag behind. Stewart
participants (4)
-
Christopher Browne -
D. Hugh Redelmeier -
Sergio Durigan Junior -
Stewart C. Russell