- Talk - lists.gtalug.org

[GTALUG-Announce] GTALUG Meeting on Tuesday 13 February at 7:30pm
by hi＠gtalug.org 05 Feb '18

05 Feb '18

<http://gtalug.org/meeting/2018-02/> # Postgres, MVCC, and You (or, why COUNT(*) is slow) with David Wolever It's hard to be a developer today without using a database… but they're often surrounded by an air of reverent mystery. One of those mysteries is why it's so slow to count all the rows in a table using COUNT(*). After all, it's just a matter of walking a b-tree and counting leaves… and that should be trivial to optimize! In this talk I'll answer the question of "why COUNT(*) is slow" by taking a deep dive into the the internals of Postgres' MVCC implementation, looking at: The question of "why COUNT(*) is slow" The on-disk storage layout and why, under the hood, it's not a b-tree What Postgres means by MVCC, with examples of the utility of transactions Introduce xid, xmin, xmax Discuss tuple visibility Explain VACUUM and xid wraparound Show off transaction isolation levels This talk is accessible to anyone who's used an SQL database, with enough depth that experienced developers will find some interesting tidbits. About me: David is a Pythonista from Toronto, Canada. He's co-founder of both PyCon Canada – Canada's regional Python conference – and Akindi.com – a small company that's making multiple-choice bubble sheet tests a little bit less terrible. He's also the author of nose-parameterized, a parameterized testing for every Python testrunner, and pprint++, a Python pretty-printer that's actually pretty. Say hi on Twitter: @wolever! ## Location George Vari Engineering and Computing Centre 245 Church Street, Room 203 Ryerson University <http://goo.gl/maps/16oJ2> <http://www.openstreetmap.org/way/23447525> ## Schedule * 6:00 pm - Please discuss on the general mailing list (i.e. <talk(a)gtalug.org>) where you want to go for dinner. * 7:30 pm - Meeting and presentation. * 9:00 pm - After each meeting, a group of GTALUGers move to The Imperial Pub (54 Dundas St East) for refreshments and more socialising. # Code of Conduct We want a productive happy community that can welcome new ideas, improve every process every year, and foster collaboration between individuals with differing needs, interests and skills. We gain strength from diversity, and actively seek participation from those who enhance it. This code of conduct exists to ensure that diverse groups collaborate to mutual advantage and enjoyment. We will challenge prejudice that could jeopardise the participation of any person in the community. The Code of Conduct governs how we behave in public or in private whenever the Linux community will be judged by our actions. We expect it to be honoured by everyone who represents the community officially or informally, claims affiliation or participates directly. It applies to activities online or offline. We invite anybody to participate. Our community is open. Please read more about the GTALUG Code of Conduct here: <http://gtalug.org/about/code-of-conduct/>. If you have any questions, comments, or concerns about the GTALUG Code of Conduct please contact the GTALUG Board @ <board(a)gtalug.org>. --- GTALUG Announce mailing list announce(a)gtalug.org https://gtalug.org/mailman/listinfo/announce

1 0

more on Spectre v2
by D. Hugh Redelmeier 05 Feb '18

05 Feb '18

Spectre v2 is really complicated to deal with. Just read this recent thread in the LKML <https://lkml.org/lkml/2018/1/20/158> I'm impressed how well written those messages are. Intel Skylake CPUs are particularly problematic. Those are most of the "6th generation Core" processors and some of the "7th generation Core". <https://en.wikipedia.org/wiki/Skylake_(microarchitecture)> The indirect branch predictor is a big problem. The retpoline deals with most cases. On Skylake, this predictor is used in another case: for return instructions that cause underflow in the call/return predictor stack. That means that EVERY return instruction is suspect, and the cost of being suspect is high. OK, if you can prove that the call/return predictor stack has not underflowed, you can just do a return. But how? The best fix is not on the table: it would be great if Intel could patch microcode so that the return predictor did not fall back to the indirect branch predictor. It sounds easy, but I infer that it is not technically possible.

2 3

Removing write protection from MicroSD card?
by Evan Leibovitch 05 Feb '18

05 Feb '18

Hey there. Subject says it. Regular SD cards have a physical switch but MicroSD don't, yet I can't use this because it's reported as write protected. Any suggestions? Is it a micro-brick? Thanks!

4 5

Seen in passing: PDF to epub/docx converter for windows
by David Collier-Brown 04 Feb '18

04 Feb '18

https://www.lightenpdf.com/pdf-converter.html $29.99, with a free trial if you have a windows emulator handy --dave -- David Collier-Brown, | Always do right. This will gratify System Programmer and Author | some people and astonish the rest davecb(a)spamcop.net | -- Mark Twain

3 2

Increasing interest in the Go language
by David Collier-Brown 04 Feb '18

04 Feb '18

Later this month I'm joining a company that is fairly Go-intensive. They originally prototyped in Perl, but over time needed more performance but not to the level that would require assembler or even C. What else have folks observed? --dave -- David Collier-Brown, | Always do right. This will gratify System Programmer and Author | some people and astonish the rest davecb(a)spamcop.net | -- Mark Twain

10 11

Linux Journal, RIP
by Stewart C. Russell 01 Feb '18

01 Feb '18

Looks like last month was their last month of even digital publication: https://www.linuxjournal.com/content/linux-journal-ceases-publication Jason Scott has saved everything he can on archive.org: https://archive.org/details/linuxjournalmagazine Looks like he could use some help creating tables of contents for all the issues, though: So, next bit. I took the table of contents from a page on the site to generate the table of contents here: https://archive.org/details/Linux-Journal-2011-06 - so, unfortunately, this has to be done for each issue. Any brave soul want to do this task? — @jscott : https://twitter.com/textfiles/status/938198975982850048 I'm sure it would be an amusing SMOP for someone on the list, and Jason's good people to work with. archive.org's Python interface is pretty powerful, too — https://internetarchive.readthedocs.io/en/latest/ cheers, Stewart

10 10

How to go fast without speculating... maybe
by David Collier-Brown 31 Jan '18

31 Jan '18

Kunle Olukotun didn't like systems that wasted their time stalled on loads and branches. He and his team at Afara Websystems therefor designed a non-speculating processor that did work without waits. It became the Sun T1. Speed without speculating The basic idea is to have more decoders than ALUs, so you can have lots of threads competing for an ALU. If, for example, thread 0 comes to a load, it will stall, so on the next instruction thread 1 gets the ALU, and runs... until it stalls and thread 2 get the ALU. Ditto for thread 3, and control goes back to thread 0, which has completed a multi-cycle fetch from cache and is ready to proceed once more. That is the basic idea of the Sun T-series processors. The strength is that the ALUs are never waiting for work. The weakness is that individual threads still have to wait for data to come from cache. You can improve on that Now imagine it isn't entire ALUs that are the available resources, its individual ALU component, like adders. Now the scenario becomes * thread 0 stalls * thread 1 get an adder * thread 2 gets a compare (really a subtracter) * thread 3 gets a branch unit, and will probably need to wait in the next cycle * thread 4 gets an adder * thread 5 gets an FPU ... and so on. Each cycle, the hardware assigns as many ALU components as it has available to threads, all of which can run. Only the stalled threads are waiting, and they don't need ALU bits to do that. Now more threads can run at the same time, the ALU components are (probabilistically) all busy, and we have increased capacity. But individual threads are still waiting for cache... Do I feel lucky? In principle, we could allocate two adders to thread 5, one doing the current instruction and another doing a subsequent, non-dependent instruction. It's not speculative, but it is out-of-order. That makes some threads twice as fast when doing non-interacting calculations. Allocate it three adders and it's three times as fast. If we're prepared to have more ALU components than decoders, decode deeply and we have enough of each to be likely to be able to find lots of non-dependent instructions, then we can be executing multiple instructions at once in multiple streams, and probabilistically get /startlingly/ better performance. I can see a new kind of optimizing compiler, too: one which tries to group non-dependent instructions together. Conclusion Is this what happens in a T5? That's a question for a hardware developer: I have no idea... yet Links: https://en.wikipedia.org/wiki/Kunle_Olukotun https://en.wikipedia.org/wiki/Afara_Websystems https://web.archive.org/web/20110720050850/http://www-hydra.stanford.edu/~k… -- David Collier-Brown, | Always do right. This will gratify System Programmer and Author | some people and astonish the rest davecb(a)spamcop.net | -- Mark Twain

8 20

Re: [GTALUG] January TASK Reminder
by David Collier-Brown 30 Jan '18

30 Jan '18

Task seems to have a permanent location now, at Cisco > <https://task.us12.list-manage.com/track/click?u=55cba2f29693cfd9f99a07a20&i…> > > > *January TASK Reminder* > > > A quick reminder this month's TASK will be held tomorrow night > (Wednesday, January 31) at our *new location* on the 29th Floor at 88 > Queens Quay West, courtesy of Cisco. > > This month's sessions will see Apurva Kumar and Jeremy Richards > discuss the newly published Dark Caracal investigation, and Lee Kagan > talk C2 tradecraft and design. See below for more details. > > As always, TASK is free to attend, no registration is required, and we > welcome you to invite friends and colleagues. > > *January TASK* > *Date: *January 31, 2018 > *Time: *6:00 pm > *Location:* > Floor 29, 88 Queens Quay West, Toronto > <https://task.us12.list-manage.com/track/click?u=55cba2f29693cfd9f99a07a20&i…> > > We look forward to seeing you there, > /TASK Steering Committee/ > > > >

1 0

Getting external IP
by William Park 29 Jan '18

29 Jan '18

Hi, I recently switched to CarryTel, and I'm getting disconnected quite often. So, I want to monitor my external IP, every minute. For this, I usually use <http://myexternalip.com/raw>, but it has rate limit. What site do you use to get your external IP? I want to round-robin through these. Google gives you this when you search "what is my ip", but it's difficult to dig it out. Simpler, the better. -- William Park <opengeometry(a)yahoo.ca>

9 15

Swap and Hibernate on SSDs
by Giles Orr 28 Jan '18

28 Jan '18

In the early days of SSDs, there was a lot of talk in the Linux community that you shouldn't have a swap partition on an SSD. An indirect side effect of that is that it becomes impossible to suspend-to-disk (aka "hibernate") on a SSD-only system. Although I suppose that you could have the swap partition and set swappiness to 0. So I have a new SSD system, and was wondering the current state of these things: - is it considered okay to have a swap partition on an SSD drive? - how should I set the swappiness? - is it considered okay to hibernate to an SSD? -- Giles https://www.gilesorr.com/ gilesorr(a)gmail.com

4 3