On Wed, Jun 12, 2019 at 4:40 PM D. Hugh Redelmeier via talk <talk@gtalug.org> wrote:

| From: Christopher Browne via talk <talk@gtalug.org>

| $ fselect path, size from ~/Downloads where size gt 30mb order by size desc limit 25

That looks pretty ugly too.

| FYI, fselect is written in Rust, and has a goodly number of ("painfully | large number of") dependencies, quasi-self-managed via the Rust-oriented | dependency tool, Cargo.

Rust culture and practice doesn't seem to like shared libraries.. I do. (Just like I find the update model for persistent containers unfortunate.)

Sorry - - - Mr Hugh - - - would you please explain the above last statement? Regards

On Thu, Jun 13, 2019 at 1:12 AM D. Hugh Redelmeier via talk <talk@gtalug.org> wrote:

| From: o1bigtenor via talk <talk@gtalug.org>

| > Rust culture and practice doesn't seem to like shared libraries.. I do. | > (Just like I find the update model for persistent containers unfortunate.)

| would you please explain the above last statement?

As I understand it, Rust programs are statically linked. Furthermore, a Rust program tree has copies of "crates" that only get updated when you ask to have them upgraded (re-imported from crates.io).

Contrast that with C source. System header files, referenced by #include in the program, are updated whenever updates are supplied by the distro and accepted when the sysadmin applies updates to the host system. Static and dynamic libraries are updated the same way.

So: if there is an update that fixes a bug, including a security bug:

- for a dynamic library, a system update will fix the bug in every binary program that uses it. No re-compilation is needed.

- for a static library, a system update plus a re-link of each program that uses it will fix each of them. Recompilation isn't needed but that isn't often an important advantage.

- for a header file, a system update plus a recompilation of each

Gord

program that uses it will fix the each of them.

- For Rust: each Rust project must decide to update the version that is incorporated the crate. I don't know that this is nearly as automated as Linux distro system updates are. I don't get the feeling that performing such updates are part of the Rust culture. Then a project recompile/rebuild is required.

- for containers: for updates to libraries (or other components) the container must be rebuilt. This must be done on a system that itself has been updated. Since there are many container technologies, I cannot be more explicit.

Summary: one system-wide update that fixes a shared library, all C programs on that system are fixed too. This isn't true of Rust programs. This isn't true of things inside containers.

On the other hand, bug fixes might make a calling program behave differently. That change might actually destabilize some system. Some folks thing stability is more important than bug fixes.

Also: Rust libraries are young and thus change more frequently than C libraries. Often for reasons other than bug fixes. This may amplify the previous argument for stability.

Windows has suffered from "dll hell" with undisciplined use of shared libraries. So much so that they often end up not being shared or updated. But the Unix world, generally, has not. This Windows experience has given shared libraries a bad name.

To a certain extent, if you link your C program statically, you can execute its binary on any Linux distro. This does not work with shared libraries.

Shared libraries significantly reduce the size of programs. Rust binaries are probably quite a bit larger than C binaries.

Thank you for your explanation!!! Given what you have described I'm wondering if 'Go' functions similarly - - - and then maybe that is why LXD is currently only supported when its installed using Snapd and the miasma forces updating - - - no matter what (max 2 months). I don't check weekly for system updates here but I do try to keep my systems updated for security reasons but I am very very uncomfortable when the right to control the timing of the updating is taken from me. Thanking you for your assistance!! Regards

On 6/13/19 10:40 AM, David Mason via talk wrote:

My preface: 40 years of C programming is enough! I don’t plan to write any more C; where I need that low-level efficiency, I’ll use Rust. Ditto no C++; if I want objects, I’ll do it right with Smalltalk.

With that said, Hugh makes a legitimate point (if we were talking apples and apples).

...

On Thu, Jun 13, 2019 at 1:12 AM (actually, earlier) D. Hugh Redelmeier wrote:  |   Rust culture and practice doesn’t seem to like shared libraries.. I do.

Rust does tend to create static code from crates (its packaging system). But C does too, for source code that you compile into your code. The difference is that, because crates are understood by the compiler it is *much* easier to make decisions about what version you want in Rust.

Rust also *does* use dynamic libraries for C/system code. I have a web server that uses sqlite3 and the generated executable includes the sqlite3, resolve, and system dynamic libraries.

So the static part is the crates, and it is trivial to discard the previously frozen versions and compile with the latest versions.

HOWEVER, since Rust code is intrinsically much, much safer than C code, stability of API is much more legitimate a characterizer of the version that you want than bug-fixes (and bug-fixes are almost never security/safety related). Rust uses semantic versioning of crates, so you can specify, e.g. version 2.3.* of a crate, to get any bug-fixes without any breaking changes.

Your mixing issues here. C defiantly has a number of shortcomings and features that can shoot you in the foot but that has nothing to do with shared libraries. Shared libraries are not a "feature" of C and any programming environment can be designed to have shared runtime linkable modules. The first reason to not have runtime linkable modules is because doing it right is complex and it has take the unix world decades to get the current reliable and stable environment working. As for versioning most times versions are to keep track of bug fixes and no matter how safe and rigorous  a programming language is programmers are much smarter and find ways to introduce bugs into their code. Having to recompile your programs whenever there is a version change to a sub sub sub module is a major pain in the ass and just imagine you had to rebuild your whole linux system any time there was a software change. There are lots of reasons to not use runtime linkable libraries but none of those reasons is the perceived "safety" of the language.

All in all, I think Rust is making the right choice here. (And C is making the right choice for C code, too - because it’s so unsafe!)

../Dave

-- Alvin Starr || land: (647)478-6285 Netvel Inc. || Cell: (416)806-0133 alvin@netvel.net ||

HOWEVER, since Rust code is intrinsically much, much safer than C code, stability of API is much more legitimate a characterizer of the version that you want than bug-fixes (and bug-fixes are almost never security/safety related).

Please please please. Bug fixes, specifically the ones that get shipped fairly quickly, are almost always security related. Security bugs are just a class of bugs. Also, till I see some way of formally verifying, that rust code is safe enough that security issues are not possible, remind me to be quite sceptical of the claim that Rust code is secure. In the last two years, we have seen classes of bugs believed to be impossible. I will accept your claim that in the hands of an average programmer, Rust is probably safer, but we have enough experience with C/Assembly, that I am gonig to claim that with someone experienced, they can create as secure/safe programs in C, as experienced folks in Rust.

| From: David Mason via talk <talk@gtalug.org> | I was unclear. I completely agree that in the C/Assembly world, | bug-fixes are almost always security related. In the Rust world, | security bugs are very hard to create (not impossible, but hard), so | most *Rust* bug-fixes are bugs for sure, but not security bugs. Rust does a few things that are relevant (I think -- I've not actually used Rust): - when programming language abstractions are broken (eg. array bound violations), the program aborts rather than continue doing (exploitable) nonsense. - unfortunately, I think that Rust only catches integer overflow in debug mode. That's a mistake, but it's probably because checking is considered too expensive. <http://huonw.github.io/blog/2016/04/myths-and-legends-about-integer-overflow-in-rust/> - it takes real work to get dangling pointers (pointers that point to freed memory). You cannot do it in Rust, except at a higher level (i.e. by creating your own simulation of pointers) - it takes work to create leaks (but leaks are not usually exploitable except for Denial of Service) Any of these bugs in C code can, and regularly do, lead to security problems. Furthermore, since they are not checked by the system, these bugs can apparently debugged code. But there is a lot of scope for security bugs that isn't covered by these improvements. Dave: is this a summary of what you are saying? It is often possible to exploit a C program bug to inject arbitrary machine code into the execution. Rust program bugs cannot be exploited this way. (Assuming you don't use "unsafe".) ================ Anecdote: overflow as a security hole Here's the first security bug that I remember exploiting. This was about 1970, in WATFIV, a Fortran compiler that, among other things, was supposed to keep the program safely bottled up. character*1 m(256,256,256,256) This ostensibly gave me an array "m" of 4GiB (the machine had 1MiB of real memory, no virtual memory, and a number of other jobs) -- impossible. But the compiler got an overflow doing the size calculation (in 32 bits) and actually allocated 0 bytes. I could then subscript this array, passing bounds checks, and access all memory available to the process running the program. Havoc ensued. (I was a Good Guy. I told the compiler maintainer and this bug was fixed.) Years later I went to a talk by one of the people who had created PL/C, the Cornell PL/1 compiler. He claimed that their compiler was 100% safe. I said that I thought that with an interface as wide as the PL/1 language, that this was unlikely but the speaker was confident (it had been used for safely running student programs for several years in several universities). Within about 20 minutes of the talk, I had broken it (slowing me down: I had to use punch-cards). - same trick - but their compiler did check for overflows - so I moved the array declaration to run-time (a PL/I feature) - the runtime system caught the overflow - but the PL/1 language allows overflow checking to be suppressed in your code. When I did this, I could use this trick. Like a combination lock, it took the interaction of several features to get in. That's one reason security problems are so hard to find. Cornell subsequently fixed their compiler so that runtime overflow checks were only suppressed in user's code, not in the internally generated array-size calculation. According to <https://www.ibm.com/support/knowledgecenter/SSQ2R2_9.5.1/com.ibm.ent.pl1.zos.doc/topics/xf5640.htm> PL/I no longer checks for overflow on FIXED BINARY operations, which include normal integer operations. Too bad. ================ On the IBM/360, it was possible to get an interrupt on overflow but it isn't on the PDP-11 and its successors (eg. the x86 family, ARM, etc.). So it is no longer possible to get overflow checking "for free". That's probably why Rust and PL/I no longer check for integer overflow as a matter of course. With current hardware, code that calculates the correct result may be more efficient than code that catches overflow in each possible place. By that I mean faithfully doing the calculation in a wide-enough type and only checking when the result must be stored in a narrower type. If overflow is checked on each operation, +, -, and * are no longer commutative and associative. This confuses programmer and hamstrings compilers. ================ Pascal lets you define variables and formal parameters as belonging to a subrange of integers. For a few years I programmed in a Pascal variant that forced all integer declarations to be of this form. I found it very a powerful tool for capturing and enforcing these invariants and I wish Rust allowed this. It makes overflow much more evident. It also gets rid of defining integer widths by modifiers such as "char", "short", "long", "long long" "signed", "unsigned" or by the number of bits. A subrange is a more natural way of describing the desired capactiy of a variable.

Good point. You can turn the checks on, but see this discussion about the default: https://github.com/rust-lang/rust/issues/47739 ../Dave On Jun 20, 2019, 12:26 AM -0400, D. Hugh Redelmeier via talk <talk@gtalug.org>, wrote:

| From: D. Hugh Redelmeier via talk <talk@gtalug.org>

| Rust does a few things that are relevant (I think -- I've not actually | used Rust):

| - unfortunately, I think that Rust only catches integer overflow in | debug mode. That's a mistake, but it's probably because checking is | considered too expensive. | <http://huonw.github.io/blog/2016/04/myths-and-legends-about-integer-overflow-in-rust/>

A very recent CVE against the Linux kernel exploits integer overflow CVE-2019-11477: SACK Panic (Linux >= 2.6.29) <https://github.com/Netflix/security-bulletins/blob/master/advisories/third-party/2019-001.md> --- Talk Mailing List talk@gtalug.org https://gtalug.org/mailman/listinfo/talk

Lots of checks are being dropped in hardware to allow uniprocessors running sequential programs to run fast. As it happens, we run multiprocessors, mostly running sequential programs. Premature optimization is the root of all evil -- Knuth Do modern languages like Go and Rust need that particular set of optimizations? --dave On 2019-06-20 11:41 a.m., D. Hugh Redelmeier via talk wrote: | From: David Mason via talk <talk@gtalug.org><mailto:talk@gtalug.org> | You can turn the checks on, but see this discussion about the default: https://github.com/rust-lang/rust/issues/47739 Thanks. The issue isn't "resolved" so I guess that the default has not been changed. One comment pointed at this which I found fascinating: <https://capnproto.org/news/2015-03-02-security-advisory-and-integer-overflow-protection.html><https://capnproto.org/news/2015-03-02-security-advisory-and-integer-overflow-protection.html> Again, I found that being able to declare variables as being within a range as being very powerful. I admit that the edges get messy: just as requiring arrays to have compile-time bounds is constraining, tight compile-time bounds on integer types is sometimes overly constraining. Of course what we have in most languages is compile-time bounds, but with the hope that they are generous. And usually they are. One could argue that if overflow is rare, we should be able to handle it by falling over to bignums. Unfortunately that adds overhead to every operation -- perhaps JIT compiling techniques could fix that. Maybe a single solution doesn't fit all cases. But silently yielding a wrong answer on overflow is just dangerous. And defining the wrong answer to be correct is not really better. My intuition is that range declaration and checking might actually might be more efficient that simple overflow checking. The compiler would have more actionable information to optimize out checks. After all, most functions are not "partial": the result is well-defined if the parameters conform to their constraints. It's sometimes hard to express those constraints in the type system. --- Talk Mailing List talk@gtalug.org<mailto:talk@gtalug.org> https://gtalug.org/mailman/listinfo/talk -- David Collier-Brown, | Always do right. This will gratify System Programmer and Author | some people and astonish the rest dave.collier-brown@indexexchange.com<mailto:dave.collier-brown@indexexchange.com> | -- Mark Twain CONFIDENTIALITY NOTICE AND DISCLAIMER : This telecommunication, including any and all attachments, contains confidential information intended only for the person(s) to whom it is addressed. Any dissemination, distribution, copying or disclosure is strictly prohibited and is not a waiver of confidentiality. If you have received this telecommunication in error, please notify the sender immediately by return electronic mail and delete the message from your inbox and deleted items folders. This telecommunication does not constitute an express or implied agreement to conduct transactions by electronic means, nor does it constitute a contract offer, a contract amendment or an acceptance of a contract offer. Contract terms contained in this telecommunication are subject to legal review and the completion of formal documentation and are not binding until same is confirmed in writing and has been signed by an authorized signatory.

On Thu, Jun 13, 2019 at 02:12:22AM -0400, D. Hugh Redelmeier via talk wrote:

As I understand it, Rust programs are statically linked. Furthermore, a Rust program tree has copies of "crates" that only get updated when you ask to have them upgraded (re-imported from crates.io).

Contrast that with C source. System header files, referenced by #include in the program, are updated whenever updates are supplied by the distro and accepted when the sysadmin applies updates to the host system. Static and dynamic libraries are updated the same way.

So: if there is an update that fixes a bug, including a security bug:

- for a dynamic library, a system update will fix the bug in every binary program that uses it. No re-compilation is needed.

- for a static library, a system update plus a re-link of each program that uses it will fix each of them. Recompilation isn't needed but that isn't often an important advantage.

- for a header file, a system update plus a recompilation of each program that uses it will fix the each of them.

- For Rust: each Rust project must decide to update the version that is incorporated the crate. I don't know that this is nearly as automated as Linux distro system updates are. I don't get the feeling that performing such updates are part of the Rust culture. Then a project recompile/rebuild is required.

- for containers: for updates to libraries (or other components) the container must be rebuilt. This must be done on a system that itself has been updated. Since there are many container technologies, I cannot be more explicit.

Summary: one system-wide update that fixes a shared library, all C programs on that system are fixed too. This isn't true of Rust programs. This isn't true of things inside containers.

On the other hand, bug fixes might make a calling program behave differently. That change might actually destabilize some system. Some folks thing stability is more important than bug fixes.

Also: Rust libraries are young and thus change more frequently than C libraries. Often for reasons other than bug fixes. This may amplify the previous argument for stability.

Windows has suffered from "dll hell" with undisciplined use of shared libraries. So much so that they often end up not being shared or updated. But the Unix world, generally, has not. This Windows experience has given shared libraries a bad name.

To a certain extent, if you link your C program statically, you can execute its binary on any Linux distro. This does not work with shared libraries.

Shared libraries significantly reduce the size of programs. Rust binaries are probably quite a bit larger than C binaries.

Go has the same design flaw. Giant static binaries. Amazing how many new programing languages don't want to pay attention to history and the security lessons already learned. -- Len Sorensen

...

Shared libraries significantly reduce the size of programs. Rust binaries are probably quite a bit larger than C binaries.

Go has the same design flaw. Giant static binaries.

Amazing how many new programing languages don't want to pay attention to history and the security lessons already learned.

I agree with you on this, but also seeing how some libraries get developed/updated (I am looking you, npm), I can see why some programmers prefer static libraries. Dhaval

| From: Lennart Sorensen via talk <talk@gtalug.org> | On Thu, Jun 13, 2019 at 10:06:37PM -0700, Dhaval Giani via talk wrote: | > I agree with you on this, but also seeing how some libraries get | > developed/updated (I am looking you, npm), I can see why some | > programmers prefer static libraries. | | But npm is one of those modern eco systems that believes every project | should pick its own version of everything. It is effectively static | linking. | | So npm is the static linking problem. I'm thinking about npm (a JavaScript repository). Not that I know anything about it. The word "curation" has been used to excuse a bunch of things, good and bad. But it's not a bad description of the role I'm thinking of. I admit that I go on and on about these issues. See for example updates and shared libraries [was Re: A find alternative: fselect] from last week. Or my last lightning talk "What is a distro?". But I think that they are really important. There is too much software that we want and need for each of us to do quality control: - is well designed for the problem it addresses? - is it better than the alternatives? - does it fit into existing environments? - is it sufficiently stable? Bug free? - does it have a reasonable liklihood of not being a security risk? - is it stable? - does it have a liklihood of ongoing support and development? - is there a reasonable way of feeding back bug reports to the developers? I mostly depend on Fedora to do these things for me. Perhaps not perfectly, but a lot better than I can do on my own. I chose my distro partly based on how well I think that they do this curation. Non-Linux folks are often uncomfortable with the level of curation by Linux distros. They feel more comfortable with Microsoft or Apple. A bunch of systems that exist as part of Linux also bypass the distro: - Python Package Index (python) - PEAR (PHP) - CTAN (TeX) - CPAN (Perl) - npm (JavaScript) - crate.io (Rust) Who then does the curation? Are they any good at it? It's easy to say "the user", but for big things like these that is impractical. I'm not particularly comfortable with any of these repositories, possibly due to ignorance. If you need things from these repos, it is pretty easy to rationalize trusting them.

There are a couple of really odd CLIs in Unix. find(1) is one, and dd(7) is another. dd is pure multics, and find is just weird, with the options and parameters reversed from common practice. Hysterical accident! --dave On 2019-06-12 5:39 p.m., D. Hugh Redelmeier via talk wrote:

| From: Christopher Browne via talk <talk@gtalug.org>

| $ fselect path, size from ~/Downloads where size gt 30mb order by size desc limit 25

That looks pretty ugly too.

| FYI, fselect is written in Rust, and has a goodly number of ("painfully | large number of") dependencies, quasi-self-managed via the Rust-oriented | dependency tool, Cargo.

Rust culture and practice doesn't seem to like shared libraries.. I do. (Just like I find the update model for persistent containers unfortunate.)

| The output is intended to be human readable moreso than machine readable. | (That said, how many of us still use /usr/bin/find to generate input for | cpio???)

xargs(1)

It too is useful but ugly. --- Talk Mailing List talk@gtalug.org https://gtalug.org/mailman/listinfo/talk

-- David Collier-Brown, | Always do right. This will gratify System Programmer and Author | some people and astonish the rest dave.collier-brown@indexexchange.com | -- Mark Twain CONFIDENTIALITY NOTICE AND DISCLAIMER : This telecommunication, including any and all attachments, contains confidential information intended only for the person(s) to whom it is addressed. Any dissemination, distribution, copying or disclosure is strictly prohibited and is not a waiver of confidentiality. If you have received this telecommunication in error, please notify the sender immediately by return electronic mail and delete the message from your inbox and deleted items folders. This telecommunication does not constitute an express or implied agreement to conduct transactions by electronic means, nor does it constitute a contract offer, a contract amendment or an acceptance of a contract offer. Contract terms contained in this telecommunication are subject to legal review and the completion of formal documentation and are not binding until same is confirmed in writing and has been signed by an authorized signatory.