Alvin Starr via talk wrote on 2025-02-03 10:20:
Think of it like this:
$ python prog1.py &
$ python prog2.py &
If my memory serves me correctly all the code is shared between the processes Its just the data which is private to each process. Now in the case of an interpreter, the xxx.py programs are just data from the OS point of view so that is duplicated.
I'd like to read more about that if anyone has links that indicate launching multiple programs (i.e. Python scripts) only load python once. The systemctl output shows multiple pythons running, each with their own PID (lines chopped to avoid wrapping): Tasks: 9 (limit: 4915) Memory: 95.2M CGroup: /system.slice/mailman.service ├─25091 /usr/bin/python /usr/lib/mailman/bin/mailmanctl ├─25092 /usr/bin/python /var/lib/mailman/bin/qrunner ├─25093 /usr/bin/python /var/lib/mailman/bin/qrunner ├─25094 /usr/bin/python /var/lib/mailman/bin/qrunner ├─25095 /usr/bin/python /var/lib/mailman/bin/qrunner ├─25096 /usr/bin/python /var/lib/mailman/bin/qrunner ├─25097 /usr/bin/python /var/lib/mailman/bin/qrunner ├─25098 /usr/bin/python /var/lib/mailman/bin/qrunner └─25099 /usr/bin/python /var/lib/mailman/bin/qrunner Surely that means python is loaded and re-loaded? I'm going to dig around a bit and see if I can account for that 95.2M RAM usage... Python2.7 interpreter size: # ls -l $(which python2.7) -h ... 3.7M 2022-02-06 15:16 /usr/bin/python2.7* The qrunner which is a python script that is invoked repeatedly with different options (which are, in turn other scripts): # ls -l /usr/lib/mailman/bin/qrunner ... 9617 2022-06-08 14:24 /usr/lib/mailman/bin/qrunner* And these are the scripts loaded by qrunner: ll /usr/lib/mailman/Mailman/Queue/ArchRunner.py* ... 3099 2022-06-08 14:24 /usr/.../Mailman/Queue/ArchRunner.py ... 1831 2022-06-22 17:55 /usr/.../Mailman/Queue/ArchRunner.pyc # ll /usr/lib/mailman/Mailman/Queue/BounceRunner.py* ... 15901 2022-06-08 14:24 /usr/.../Mailman/Queue/BounceRunner.py ... 8830 2022-06-22 17:55 /usr/.../Mailman/Queue/BounceRunner.pyc # ll /usr/lib/mailman/Mailman/Queue/CommandRunner.py* ... 11882 2022-06-08 14:24 /usr/.../Mailman/Queue/CommandRunner.py ... 7639 2022-06-22 17:55 /usr/.../Mailman/Queue/CommandRunner.pyc # ll /usr/lib/mailman/Mailman/Queue/IncomingRunner.py* ... 9080 2022-06-08 14:24 /usr/.../Mailman/Queue/IncomingRunner.py ... 2632 2022-06-22 17:55 /usr/.../Mailman/Queue/IncomingRunner.pyc # ll /usr/lib/mailman/Mailman/Queue/NewsRunner.py* ... 7365 2022-06-08 14:24 /usr/.../Mailman/Queue/NewsRunner.py ... 4303 2022-06-22 17:55 /usr/.../Mailman/Queue/NewsRunner.pyc # ll /usr/lib/mailman/Mailman/Queue/OutgoingRunner.py* ... 5707 2022-06-08 14:24 /usr/.../Mailman/Queue/OutgoingRunner.py ... 3282 2022-06-22 17:55 /usr/.../Mailman/Queue/OutgoingRunner.pyc # ll /usr/lib/mailman/Mailman/Queue/VirginRunner.py* ... 1732 2022-06-08 14:24 /usr/.../Mailman/Queue/VirginRunner.py ... 1311 2022-06-22 17:55 /usr/.../Mailman/Queue/VirginRunner.pyc # ll /usr/lib/mailman/Mailman/Queue/RetryRunner.py* ... 1481 2022-06-08 14:24 /usr/.../Mailman/Queue/RetryRunner.py ... 1439 2022-06-22 17:55 /usr/.../Mailman/Queue/RetryRunner.pyc Adding the *.pyc file sizes: 1831 + 8830 + 7639 + 2632 + 4303 + 3282 + 1311 + 1439 = 31267 bytes. Each of those is invoked via qrunner (8 instances at 9617 bytes): 9617 * 8 = 76936 bytes So far, I can account for 76936 + 31267 bytes = 108203 bytes. Need this process too: # ls -l /usr/lib/mailman/bin/mailmanctl ... 21431 2022-06-08 14:24 /usr/lib/mailman/bin/mailmanctl* Up to 21431 + 108203 = 129634 bytes. # systemctl status mailman Tasks: 9 (limit: 4915) Memory: 95.2M My numbers are way, *way* short of 96MB so let's add in some Python2.7 interpreters (one for each qrunner) as qrunner is a python script, and one for mailmanctl: # file /var/lib/mailman/bin/qrunner /var/lib/mailman/bin/qrunner: a /usr/bin/python script, ASCII text executable 9 * 3775416 = 33,978,744 MB So, accounting for all the compiled python code plus the interpreters gets to about ⅓ the RAM usage of Mailman v2. That seems... reasonable? Have I just wasted a bunch of time and entirely gotten the part about multiple Python interpreters loaded in memory?