python sweetness — The mysterious case of the Linux Page Table
Here's an interesting read; as the author says, it looks like it's an intriguing start to the new year. http://pythonsweetness.tumblr.com/post/169166980422/the-mysterious-case-of-t... -- Russell
Trying to write a haiku about Python on Linux - something along the line of drawing analogy from the relationship of Cleopatra and Mark Anthony. Anyone has too much time on their hands? r360design.ca On Wed, Jan 3, 2018 at 8:06 AM, Russell via talk <talk@gtalug.org> wrote:
Here's an interesting read; as the author says, it looks like it's an intriguing start to the new year.
http://pythonsweetness.tumblr.com/post/169166980422/the-mysterious-case-of-t... -- Russell --- Talk Mailing List talk@gtalug.org https://gtalug.org/mailman/listinfo/talk
| From: R360 Design INC via talk <talk@gtalug.org> | Trying to write a haiku about Python on Linux - something along the | line of drawing analogy from the relationship of Cleopatra and Mark | Anthony. Anyone has too much time on their hands? Cleopatra was killed by asp, not python. Active Server Pages are part of .net, not Linux.
As more proprietary information emerges, it appears to look more like I. Flemming than W. Shakespeare. Typically an out of order execution a la SPECTRE. https://meltdownattack.com On January 3, 2018 9:50:38 PM EST, "D. Hugh Redelmeier via talk" <talk@gtalug.org> wrote:
| From: R360 Design INC via talk <talk@gtalug.org>
| Trying to write a haiku about Python on Linux - something along the | line of drawing analogy from the relationship of Cleopatra and Mark | Anthony. Anyone has too much time on their hands?
Cleopatra was killed by asp, not python. Active Server Pages are part of .net, not Linux. --- Talk Mailing List talk@gtalug.org https://gtalug.org/mailman/listinfo/talk
-- Russell
https://googleprojectzero.blogspot.ca/2018/01/reading-privileged-memory-with... gives the gory details At this point, I cannot stress on how important it is to update your systems as soon as your distribution ships them. I am hoping this remains to be a once in a lifetime event. Dhaval
One could assert that the days of time sharing systems are largely over, at least on production systems that people care about. And I think it's fair to say that it has been good practice for quite some time to not allow random binaries to run on systems you care about. I have no idea whether hypervisors (like xen or esxi) are vulnerable. But the same guidelines can be applied to VMs running on hypervisors. I wonder how exploitable this problem really is? Cheers, happy new year John On Wed, 2018/01/03 10:56:30PM -0500, Dhaval Giani via talk <talk@gtalug.org> wrote: | https://googleprojectzero.blogspot.ca/2018/01/reading-privileged-memory-with... | gives the gory details | | At this point, I cannot stress on how important it is to update your | systems as soon as your distribution ships them. I am hoping this | remains to be a once in a lifetime event. | | Dhaval | --- | Talk Mailing List | talk@gtalug.org | https://gtalug.org/mailman/listinfo/talk
On Wed, Jan 3, 2018 at 11:53 PM John Sellens via talk <talk@gtalug.org> wrote:
One could assert that the days of time sharing systems are largely over, at least on production systems that people care about.
And I think it's fair to say that it has been good practice for quite some time to not allow random binaries to run on systems you care about.
I have no idea whether hypervisors (like xen or esxi) are vulnerable. But the same guidelines can be applied to VMs running on hypervisors.
Xen and kvm are both affected.
I wonder how exploitable this problem really is?
Meltdown already has some exploits around that I am seeing. I also believe there is some poc code out there to exploit it. One of which I believe is executing JavaScript in your web browser to get kernel space data. Dhaval
On January 3, 2018 10:56:30 PM EST, Dhaval Giani <dhaval.giani@gmail.com> wrote:
https://googleprojectzero.blogspot.ca/2018/01/reading-privileged-memory-with...
gives the gory details
At this point, I cannot stress on how important it is to update your systems as soon as your distribution ships them. I am hoping this remains to be a once in a lifetime event.
I admire your optimism. To me it looks like this is a kind of example of feeping creaturisim in hypervisor's; not necessarily an easy patch. The idea of the necessity of some sort of kernel isolation has been around for quite a while. In part as a response to the ease with which userland interpreters can polute kernelspace. https://lwn.net/Articles/39283/ I've read that some of the proposed solutions could add as much as a 30% operational overhead. Not much of an issue for average home users but for enterprise this could be a real game changer. Like finding out the real cost, in energy consumption, of a Bitcoin transaction, is converging with the real costs of maintaining a paper currency.
Dhaval
Russell On Wed, Jan 3, 2018 at 11:59 PM Russell Reiter <rreiter91@gmail.com> wrote:
On January 3, 2018 10:56:30 PM EST, Dhaval Giani <dhaval.giani@gmail.com> wrote:
https://googleprojectzero.blogspot.ca/2018/01/reading-privileged-memory-with...
gives the gory details
At this point, I cannot stress on how important it is to update your systems as soon as your distribution ships them. I am hoping this remains to be a once in a lifetime event.
I admire your optimism. To me it looks like this is a kind of example of feeping creaturisim in hypervisor's; not necessarily an easy patch.
I am unsure what you are implying. This is a hardware issue which has been fixed in software. There are exploits out already that I am seeing able to run through your web browser. This is serious stuff. Also unsure what this has to do with hypervisors apart from them also needing to mitigate this exploit.
The idea of the necessity of some sort of kernel isolation has been around for quite a while. In part as a response to the ease with which userland interpreters can polute kernelspace.
https://lwn.net/Articles/39283/
I've read that some of the proposed solutions could add as much as a 30% operational overhead. Not much of an issue for average home users but for enterprise this could be a real game changer.
The 30% overhead is for a pathological case. A 5-10% overhead is more likely. And do you honestly think that upstream is not going to work on getting that overhead down? Dhaval
On January 4, 2018 12:08:16 AM EST, Dhaval Giani <dhaval.giani@gmail.com> wrote:
Russell
On Wed, Jan 3, 2018 at 11:59 PM Russell Reiter <rreiter91@gmail.com> wrote:
On January 3, 2018 10:56:30 PM EST, Dhaval Giani
<dhaval.giani@gmail.com>
wrote:
https://googleprojectzero.blogspot.ca/2018/01/reading-privileged-memory-with...
gives the gory details
At this point, I cannot stress on how important it is to update your systems as soon as your distribution ships them. I am hoping this remains to be a once in a lifetime event.
I admire your optimism. To me it looks like this is a kind of example of feeping creaturisim in hypervisor's; not necessarily an easy patch.
I am unsure what you are implying. This is a hardware issue which has been fixed in software. There are
There currently are patches which address a number of security issues. No software fixes yet tho. A fix implies a final solution has been implemented and the problem is no longer an issue. Clearly not the case here. exploits out already that I am seeing able
to run through your web browser. This is serious stuff. Also unsure what this has to do with hypervisors apart from them also needing to mitigate this exploit.
Semantically, a hypervisor is the kernel supervisor of the OS. So it is a core computational element in need of basic patching. (in this case) I think you are confusing a "virtual" hypervisor with a native "bare metal" hypervisor. They have the appearance of being the same thing ... only they are different.
The idea of the necessity of some sort of kernel isolation has been
around
for quite a while. In part as a response to the ease with which userland interpreters can polute kernelspace.
https://lwn.net/Articles/39283/
I've read that some of the proposed solutions could add as much as a 30% operational overhead. Not much of an issue for average home users but for enterprise this could be a real game changer.
The 30% overhead is for a pathological case. A 5-10%
overhead is more
likely. And do you honestly think that upstream is not going to work on getting that overhead down?
For a problem like this one and given it's scope and complexity, it is premature to downplay the core and it's overhead issue. This is not like in the movies where the producer says, it's not a problem, we can fix it in POST. This is a preproduction issue with the actors. If you want to get all biological about pathology. The pathology of this problem is far from well understood. Finding the proper namespace is important. At Linus's request KAISER has been dropped. However fuckwit (Forcefully Unmap Complete Kernel With Interrupt Trampolines, ) has not been adopted, by most people anyway. https://lkml.org/lkml/2017/12/4/709
Dhaval
-- Russell
On Thu, Jan 04, 2018 at 12:54:50AM -0500, Russell via talk wrote:
For a problem like this one and given it's scope and complexity, it is premature to downplay the core and it's overhead issue. This is not like in the movies where the producer says, it's not a problem, we can fix it in POST. This is a preproduction issue with the actors.
If you want to get all biological about pathology. The pathology of this problem is far from well understood. Finding the proper namespace is important.
At Linus's request KAISER has been dropped. However fuckwit (Forcefully Unmap Complete Kernel With Interrupt Trampolines, ) has not been adopted, by most people anyway.
Well code I see committed in the kernel uses pti_ for the functions, so seems they settled on page table isolation. It was previously suggesting kpti for kernel page table isolation but I guess the kernel bit was deemed redundant. -- Len Sorensen
On 04/01/18 11:22 AM, Lennart Sorensen via talk wrote:
On Thu, Jan 04, 2018 at 12:54:50AM -0500, Russell via talk wrote:
For a problem like this one and given it's scope and complexity, it is premature to downplay the core and it's overhead issue. This is not like in the movies where the producer says, it's not a problem, we can fix it in POST. This is a preproduction issue with the actors.
If you want to get all biological about pathology. The pathology of this problem is far from well understood. Finding the proper namespace is important.
At Linus's request KAISER has been dropped. However fuckwit (Forcefully Unmap Complete Kernel With Interrupt Trampolines, ) has not been adopted, by most people anyway.
https://lkml.org/lkml/2017/12/4/709 Well code I see committed in the kernel uses pti_ for the functions, so seems they settled on page table isolation. It was previously suggesting kpti for kernel page table isolation but I guess the kernel bit was deemed redundant.
From what I read, it was the time cost of going between subset and superset of page tables on every systems call or interrupt that caused push-back. There was also some discussion of deferring the change until there was a better algorithm, but it looks like the Intel issue came along and "motivated" more rapid action. /When a man knows he is to be hanged in a fortnight, it concentrates his mind wonderfully. /-- Samuel Johnson --dave -- David Collier-Brown, | Always do right. This will gratify System Programmer and Author | some people and astonish the rest davecb@spamcop.net | -- Mark Twain
On January 4, 2018 11:22:05 AM EST, lsorense@csclub.uwaterloo.ca wrote:
For a problem like this one and given it's scope and complexity, it is premature to downplay the core and it's overhead issue. This is not
On Thu, Jan 04, 2018 at 12:54:50AM -0500, Russell via talk wrote: like in the movies where the producer says, it's not a problem, we can fix it in POST. This is a preproduction issue with the actors.
If you want to get all biological about pathology. The pathology of
this problem is far from well understood. Finding the proper namespace is important.
At Linus's request KAISER has been dropped. However fuckwit
(Forcefully Unmap Complete Kernel With Interrupt Trampolines, ) has not been adopted, by most people anyway.
Well code I see committed in the kernel uses pti_ for the functions, so seems they settled on page table isolation. It was previously suggesting kpti for kernel page table isolation but I guess the kernel bit was deemed redundant.
Linus had strong words for Intel yesterday. https://lkml.org/lkml/2018/1/3/797 Takes a lot of muscle to strongARM lassez fair manufacturing into opening up their proprietary process's. On the other hand it is valid to say; at least I should be able to configure this process and engage mitigation in setting boundaries. It's going to be an interesting new year, no doubt about that.
-- Len Sorensen
-- Russell
Hi, Was wondering if we could squeeze in a question about Python installation on OS X on this thread: To set up Python on Mac, we were advised to install the latest version of Python 3.6.4 and PIP3 (According to the Hitchhiker's Guide to Python http://docs.python-guide.org/en/latest/starting/install3/osx/) . The installation folder is: MacBook-Pro: which python3 /usr/local/bin/python3 However, when we checked the package list, the packages are still pointed at the Python 2.7 packages folder (the native version of Python that ships with Mac). Is this correct or we need to re-install the packages for Python 3.6.4 using PIP3? MacBook-Pro:bin owner$ pip3 show pandas Name: pandas Version: 0.22.0 Summary: Powerful data structures for data analysis, time series,and statistics Home-page: http://pandas.pydata.org Author: The PyData Development Team Author-email: pydata@googlegroups.com License: BSD Location: /Library/Python/2.7/site-packages/pandas-0.22.0-py2.7-macosx-10.12-intel.egg Requires: numpy, python-dateutil, pytz MacBook-Pro:bin owner$ pip3 show six Name: six Version: 1.11.0 Summary: Python 2 and 3 compatibility utilities Home-page: http://pypi.python.org/pypi/six/ Author: Benjamin Peterson Author-email: benjamin@python.org License: MIT Location: /Library/Python/2.7/site-packages Thanks! r360design.ca On Thu, Jan 4, 2018 at 11:51 AM, Russell via talk <talk@gtalug.org> wrote:
On January 4, 2018 11:22:05 AM EST, lsorense@csclub.uwaterloo.ca wrote:
For a problem like this one and given it's scope and complexity, it is premature to downplay the core and it's overhead issue. This is not
On Thu, Jan 04, 2018 at 12:54:50AM -0500, Russell via talk wrote: like in the movies where the producer says, it's not a problem, we can fix it in POST. This is a preproduction issue with the actors.
If you want to get all biological about pathology. The pathology of
this problem is far from well understood. Finding the proper namespace is important.
At Linus's request KAISER has been dropped. However fuckwit
(Forcefully Unmap Complete Kernel With Interrupt Trampolines, ) has not been adopted, by most people anyway.
Well code I see committed in the kernel uses pti_ for the functions, so seems they settled on page table isolation. It was previously suggesting kpti for kernel page table isolation but I guess the kernel bit was deemed redundant.
Linus had strong words for Intel yesterday.
https://lkml.org/lkml/2018/1/3/797
Takes a lot of muscle to strongARM lassez fair manufacturing into opening up their proprietary process's. On the other hand it is valid to say; at least I should be able to configure this process and engage mitigation in setting boundaries.
It's going to be an interesting new year, no doubt about that.
-- Len Sorensen
-- Russell --- Talk Mailing List talk@gtalug.org https://gtalug.org/mailman/listinfo/talk
R360 Design INC via talk wrote:
Hi, Was wondering if we could squeeze in a question about Python installation on OS X on this thread:
To set up Python on Mac, we were advised to install the latest version of Python 3.6.4 and PIP3 (According to the Hitchhiker's Guide to Python http://docs.python-guide.org/en/latest/starting/install3/osx/) . The installation folder is:
MacBook-Pro: which python3 /usr/local/bin/python3
However, when we checked the package list, the packages are still pointed at the Python 2.7 packages folder (the native version of Python that ships with Mac). Is this correct or we need to re-install the packages for Python 3.6.4 using PIP3?
MacBook-Pro:bin owner$ pip3 show pandas Name: pandas Version: 0.22.0 Summary: Powerful data structures for data analysis, time series,and statistics Home-page: http://pandas.pydata.org Author: The PyData Development Team Author-email: pydata@googlegroups.com License: BSD Location: /Library/Python/2.7/site-packages/pandas-0.22.0-py2.7-macosx-10.12-intel.egg Requires: numpy, python-dateutil, pytz MacBook-Pro:bin owner$ pip3 show six Name: six Version: 1.11.0 Summary: Python 2 and 3 compatibility utilities Home-page: http://pypi.python.org/pypi/six/ Author: Benjamin Peterson Author-email: benjamin@python.org License: MIT Location: /Library/Python/2.7/site-packages
What does `pip3 --version` and `python3 -m pip show pip` show you? As an example mine shows: ``` $ pip3 --version pip 9.0.1 from /usr/local/lib/python3.6/site-packages (python 3.6) $ python3 -m pip show pip Name: pip Version: 9.0.1 Summary: The PyPA recommended tool for installing Python packages. Home-page: https://pip.pypa.io/ Author: The pip developers Author-email: python-virtualenv@groups.google.com License: MIT Location: /usr/local/lib/python3.6/site-packages Requires: ``` I think pip for Python3 wasn't installed correctly. You might want to try reinstalled pip, <https://pip.pypa.io/en/stable/installing/> with python3. Also it's generally best practice when replying to a thread in a mailing list with a new topic to change the subject like I did above. It helps the archiving of the mailing list so people can better browse.
On January 5, 2018 8:59:13 AM EST, "Myles Braithwaite 👾 via talk" <talk@gtalug.org> wrote:
R360 Design INC via talk wrote:
Hi, Was wondering if we could squeeze in a question about Python installation on OS X on this thread:
To set up Python on Mac, we were advised to install the latest version of Python 3.6.4 and PIP3 (According to the Hitchhiker's Guide to Python http://docs.python-guide.org/en/latest/starting/install3/osx/) . The installation folder is:
MacBook-Pro: which python3 /usr/local/bin/python3
However, when we checked the package list, the packages are still pointed at the Python 2.7 packages folder (the native version of Python that ships with Mac). Is this correct or we need to re-install the packages for Python 3.6.4 using PIP3?
MacBook-Pro:bin owner$ pip3 show pandas Name: pandas Version: 0.22.0 Summary: Powerful data structures for data analysis, time series,and statistics Home-page: http://pandas.pydata.org Author: The PyData Development Team Author-email: pydata@googlegroups.com License: BSD Location: /Library/Python/2.7/site-packages/pandas-0.22.0-py2.7-macosx-10.12-intel.egg Requires: numpy, python-dateutil, pytz MacBook-Pro:bin owner$ pip3 show six Name: six Version: 1.11.0 Summary: Python 2 and 3 compatibility utilities Home-page: http://pypi.python.org/pypi/six/ Author: Benjamin Peterson Author-email: benjamin@python.org License: MIT Location: /Library/Python/2.7/site-packages
What does `pip3 --version` and `python3 -m pip show pip` show you?
As an example mine shows:
``` $ pip3 --version pip 9.0.1 from /usr/local/lib/python3.6/site-packages (python 3.6)
$ python3 -m pip show pip Name: pip Version: 9.0.1 Summary: The PyPA recommended tool for installing Python packages. Home-page: https://pip.pypa.io/ Author: The pip developers Author-email: python-virtualenv@groups.google.com License: MIT Location: /usr/local/lib/python3.6/site-packages Requires: ```
I think pip for Python3 wasn't installed correctly. You might want to try reinstalled pip, <https://pip.pypa.io/en/stable/installing/> with python3.
Also it's generally best practice when replying to a thread in a mailing list with a new topic to change the subject like I did above. It helps the archiving of the mailing list so people can better browse.
I'm not particularly familiar with OS-X but you also could check out these links. Homebrew apparently handles package management for easy upgrades. https://stackoverflow.com/questions/17271319/how-do-i-install-pip-on-macos-o... http://docs.python-guide.org/en/latest/starting/install3/osx/#install3-osx
--- Talk Mailing List talk@gtalug.org https://gtalug.org/mailman/listinfo/talk
-- Russell
participants (9)
-
D. Hugh Redelmeier -
David Collier-Brown -
Dhaval Giani -
John Sellens -
lsorense＠csclub.uwaterloo.ca -
Myles Braithwaite 👾 -
R360 Design INC -
Russell -
Russell Reiter