War Story: Fedora 33 update silently broke Dovecot (IMAP / POP3 server)
Only one of us uses IMAP / POP3. She stopped being able to pick up mail. The message was obscure: Thunderbird reported getting resets from the server. Detective work (leaving out the blind alleys): Look for funny messages in the server log. The right command is arcane: $ journalctl -b _SYSTEMD_UNIT=dovecot.service - -b means "since last boot". Very handy because the log can go back a long time. - _SYSTEMD_UNIT=dovecot.service: only messages about dovecot. Such an intuitive form. It won't work without the .service. It turns out that there is a shortform, -u or --unit=. May 04 14:07:12 redhop-mimosa-com dovecot[977]: config: Warning: please set ssl_dh=</etc/dovecot/dh.pem May 04 14:07:12 redhop-mimosa-com dovecot[977]: config: Warning: You can generate it with: dd if=/var/lib/dovecot/ssl-parameters.dat bs=1 skip=88 | openssl dhparam -inform der > /etc/dovecot/dh.pem May 04 14:07:12 redhop-mimosa-com dovecot[977]: pop3-login: Error: Failed to initialize SSL server context: Can't load DH parameters (ssl_dh setting): error:1408518A:SSL routines:ssl3_ctx_ctrl:dh key too small: user=<>, rip=192.139.70.95, lip=192.139.70.82, session=<jkGQ8oTBnIDAi0Zf> May 04 15:30:28 redhop-mimosa-com dovecot[977]: pop3-login: Error: Failed to initialize SSL server context: Can't load DH parameters (ssl_dh setting): error:1408518A:SSL routines:ssl3_ctx_ctrl:dh key too small: user=<>, rip=192.139.70.95, lip=192.139.70.82, session=<SWRMHIbBmILAi0Zf> The first two messages were almost impossible to read because the were in yellow. It turns out that the first means that you have to edit /etc/dovecot/conf.d/10-ssl.conf and change #ssl_dh = </etc/dovecot/dh.pem to ssl_dh = </etc/dovecot/dh.pem /etc/dovecot/dh.pem was already there. OPTIONAL: /etc/dovecot/dh.pem specified a really weak Diffie-Hellman group. Your should probably strengthen it. I chose ffdhe2048, a weak one (2048 bits) that is still "OK". See: <https://wiki.mozilla.org/Security/Server_Side_TLS#Pre-defined_DHE_groups> You need it in PEM format. First save the old one. Then grab a new one: curl https://ssl-config.mozilla.org/ffdhe2048.txt > /etc/dovecot/dh.pem Then adjust ownership and permissions.
On Fri, 7 May 2021 at 22:43, D. Hugh Redelmeier via talk <talk@gtalug.org> wrote:
Only one of us uses IMAP / POP3. She stopped being able to pick up mail. The message was obscure: Thunderbird reported getting resets from the server.
Detective work (leaving out the blind alleys):
Look for funny messages in the server log. The right command is arcane: $ journalctl -b _SYSTEMD_UNIT=dovecot.service
- -b means "since last boot". Very handy because the log can go back a long time.
- _SYSTEMD_UNIT=dovecot.service: only messages about dovecot. Such an intuitive form. It won't work without the .service. It turns out that there is a shortform, -u or --unit=.
May 04 14:07:12 redhop-mimosa-com dovecot[977]: config: Warning: please set ssl_dh=</etc/dovecot/dh.pem May 04 14:07:12 redhop-mimosa-com dovecot[977]: config: Warning: You can generate it with: dd if=/var/lib/dovecot/ssl-parameters.dat bs=1 skip=88 | openssl dhparam -inform der > /etc/dovecot/dh.pem May 04 14:07:12 redhop-mimosa-com dovecot[977]: pop3-login: Error: Failed to initialize SSL server context: Can't load DH parameters (ssl_dh setting): error:1408518A:SSL routines:ssl3_ctx_ctrl:dh key too small: user=<>, rip=192.139.70.95, lip=192.139.70.82, session=<jkGQ8oTBnIDAi0Zf> May 04 15:30:28 redhop-mimosa-com dovecot[977]: pop3-login: Error: Failed to initialize SSL server context: Can't load DH parameters (ssl_dh setting): error:1408518A:SSL routines:ssl3_ctx_ctrl:dh key too small: user=<>, rip=192.139.70.95, lip=192.139.70.82, session=<SWRMHIbBmILAi0Zf>
The first two messages were almost impossible to read because the were in yellow.
It turns out that the first means that you have to edit /etc/dovecot/conf.d/10-ssl.conf and change #ssl_dh = </etc/dovecot/dh.pem to ssl_dh = </etc/dovecot/dh.pem
/etc/dovecot/dh.pem was already there.
OPTIONAL:
/etc/dovecot/dh.pem specified a really weak Diffie-Hellman group. Your should probably strengthen it. I chose ffdhe2048, a weak one (2048 bits) that is still "OK". See: <https://wiki.mozilla.org/Security/Server_Side_TLS#Pre-defined_DHE_groups>
You need it in PEM format. First save the old one. Then grab a new one: curl https://ssl-config.mozilla.org/ffdhe2048.txt > /etc/dovecot/dh.pem
Then adjust ownership and permissions.
This sent me on a bit of a chase. Nginx uses a DH parameters file that's used in the same way. And it's considered a bad idea to use a widely known DH parameter (like the one that ships with the software, or that sits on a Mozilla server). This is a semi-useful read: https://security.stackexchange.com/questions/94390/whats-the-purpose-of-dh-p.... I say "semi-useful" because honestly some of it was over my head. And that leads to reading about Logjam ( https://weakdh.org/ ). With Ansible I've automated the generation of a new DH parameter file on each new server: openssl dhparam -out <filename> <size> Generating this file takes a significant amount of time (minutes) if "size" is reasonably large (2096, although I would recommend 4192) even on a modern machine. But it's a one-off for any given machine, so it's a small imposition that I'd recommend. If this breaks the mail setup, you can obviously return to the known value from Mozilla. -- Giles https://www.gilesorr.com/ gilesorr@gmail.com
On Sun, 9 May 2021 at 08:30, Giles Orr <gilesorr@gmail.com> wrote:
On Fri, 7 May 2021 at 22:43, D. Hugh Redelmeier via talk <talk@gtalug.org> wrote:
Only one of us uses IMAP / POP3. She stopped being able to pick up mail. The message was obscure: Thunderbird reported getting resets from the server.
Detective work (leaving out the blind alleys):
Look for funny messages in the server log. The right command is arcane: $ journalctl -b _SYSTEMD_UNIT=dovecot.service
- -b means "since last boot". Very handy because the log can go back a long time.
- _SYSTEMD_UNIT=dovecot.service: only messages about dovecot. Such an intuitive form. It won't work without the .service. It turns out that there is a shortform, -u or --unit=.
May 04 14:07:12 redhop-mimosa-com dovecot[977]: config: Warning: please set ssl_dh=</etc/dovecot/dh.pem May 04 14:07:12 redhop-mimosa-com dovecot[977]: config: Warning: You can generate it with: dd if=/var/lib/dovecot/ssl-parameters.dat bs=1 skip=88 | openssl dhparam -inform der > /etc/dovecot/dh.pem May 04 14:07:12 redhop-mimosa-com dovecot[977]: pop3-login: Error: Failed to initialize SSL server context: Can't load DH parameters (ssl_dh setting): error:1408518A:SSL routines:ssl3_ctx_ctrl:dh key too small: user=<>, rip=192.139.70.95, lip=192.139.70.82, session=<jkGQ8oTBnIDAi0Zf> May 04 15:30:28 redhop-mimosa-com dovecot[977]: pop3-login: Error: Failed to initialize SSL server context: Can't load DH parameters (ssl_dh setting): error:1408518A:SSL routines:ssl3_ctx_ctrl:dh key too small: user=<>, rip=192.139.70.95, lip=192.139.70.82, session=<SWRMHIbBmILAi0Zf>
The first two messages were almost impossible to read because the were in yellow.
It turns out that the first means that you have to edit /etc/dovecot/conf.d/10-ssl.conf and change #ssl_dh = </etc/dovecot/dh.pem to ssl_dh = </etc/dovecot/dh.pem
/etc/dovecot/dh.pem was already there.
OPTIONAL:
/etc/dovecot/dh.pem specified a really weak Diffie-Hellman group. Your should probably strengthen it. I chose ffdhe2048, a weak one (2048 bits) that is still "OK". See: <https://wiki.mozilla.org/Security/Server_Side_TLS#Pre-defined_DHE_groups>
You need it in PEM format. First save the old one. Then grab a new one: curl https://ssl-config.mozilla.org/ffdhe2048.txt > /etc/dovecot/dh.pem
Then adjust ownership and permissions.
This sent me on a bit of a chase. Nginx uses a DH parameters file that's used in the same way. And it's considered a bad idea to use a widely known DH parameter (like the one that ships with the software, or that sits on a Mozilla server). This is a semi-useful read: https://security.stackexchange.com/questions/94390/whats-the-purpose-of-dh-p.... I say "semi-useful" because honestly some of it was over my head. And that leads to reading about Logjam ( https://weakdh.org/ ).
With Ansible I've automated the generation of a new DH parameter file on each new server:
openssl dhparam -out <filename> <size>
Generating this file takes a significant amount of time (minutes) if "size" is reasonably large (2096, although I would recommend 4192) even on a modern machine. But it's a one-off for any given machine, so it's a small imposition that I'd recommend. If this breaks the mail setup, you can obviously return to the known value from Mozilla.
"2096" "4192" ... <facepalm> They're just numbers, but I did mean to say "2048" and "4096" as regular power-of-2 numbers ... -- Giles https://www.gilesorr.com/ gilesorr@gmail.com
| From: Giles Orr via talk <talk@gtalug.org> | And it's considered a bad idea to use a | widely known DH parameter (like the one that ships with the software, | or that sits on a Mozilla server). I'm not sure that's correct. The counter-argument is that some are weak and some are strong and it would be better to use ones attested to in a process you consider trustable. That seems to be what is recommended the Mozilla link I included and by RFC 7919 that it links to (I haven't read it or the updates and errata). | This is a semi-useful read: | https://security.stackexchange.com/questions/94390/whats-the-purpose-of-dh-p.... I don't trust stack exchange but I do find it useful. That isn't actually a contradiction. | I say "semi-useful" because honestly some of it was over my head. And | that leads to reading about Logjam ( https://weakdh.org/ ). The DH (Diffie-Hellman) exchange is the magic that makes privacy on the internet work. It's really cute and simple. Weakness: it cannot defend against an active man-in-the-middle attack. It's actually simple. The SE first answer, second paragraph, explains how it works. Let me fill in some missing bits: The integers, modulo p [a prime], form a mathematical group. g, p are the DH parameters and are publicly known. a is Alice's secret, never to go on the wire. b is Bob's secret, never to go on the wire "^" denotes exponentiation (think repeated multiplication). All arithmetic is modulo p Alice sends Bob: g^a Bob sends Alice g^b Alice can compute (g^b) ^ a Bob Can compute (g^a) ^ b And those are both the same! Now Alice and Bob share a secret that nobody else has. That's all that's needed for bootstrapping privacy. Avoiding an active man-in-the-middle attack is much harder logistically. For that you need some kind of authentication (what one chooses to mean by authentication is a very interesting decision). That's part of why we have the horror show of certificates. They are not necessary but they are sufficient (as long as the certificate system isn't broken). | With Ansible I've automated the generation of a new DH parameter file | on each new server: | | openssl dhparam -out <filename> <size> | | Generating this file takes a significant amount of time (minutes) if | "size" is reasonably large (2096, although I would recommend 4192) | even on a modern machine. You consider the numbers you gave as a typo (according to later mail). Actually, there is an argument to be made that if you are rolling your own, don't use popular powers of two. A bad guy might have hardware optimized for powers of two (they are used in the vast majority of cases). Or precomputed tables. If you don't use a power of two, you through him off his book. The core of this is that we think that g^a is much much much cheaper to compute than the corresponding discrete log (i.e. compute a, given g and g^a).
On Sun, 9 May 2021 13:09:39 -0400 (EDT) "D. Hugh Redelmeier via talk" <talk@gtalug.org> wrote:
| From: Giles Orr via talk <talk@gtalug.org> | And it's considered a bad idea to use a | widely known DH parameter (like the one that ships with the software, | or that sits on a Mozilla server). I'm not sure that's correct. The counter-argument is that some are weak and some are strong and it would be better to use ones attested to in a process you consider trustable. That seems to be what is recommended the Mozilla link I included and by RFC 7919 that it links to (I haven't read it or the updates and errata). | This is a semi-useful read: https://security.stackexchange.com/questions/94390/whats-the-purpose-of-dh-p.... I don't trust stack exchange but I do find it useful. That isn't actually a contradiction. | I say "semi-useful" because honestly some of it was over my head. And | that leads to reading about Logjam ( https://weakdh.org/ ). The DH (Diffie-Hellman) exchange is the magic that makes privacy on the internet work. It's really cute and simple. Weakness: it cannot defend against an active man-in-the-middle attack.
it can if it is used in combination with an additional protocol? as you mentioned further below, the certificate system and the costs of calculating g^a and sizes etc etc. but, adding a set and/or 'unknown' protocol for example: vanilla/degfault ssh adds ecdsa-sha2 host keys, which is compared with each login. So, active mim may work the first original login but the second time will be problematic and when storing cascaded host keys, which themselves are hard copied, mim becomes close to impossible as active mim has to be on all routes, of course I am aware of the workaround ideas for this also, am just saying that additional set protocols and hard copied keys, etc all make DH much more 'secure' :)
It's actually simple. The SE first answer, second paragraph, explains how it works. Let me fill in some missing bits: The integers, modulo p [a prime], form a mathematical group. g, p are the DH parameters and are publicly known. a is Alice's secret, never to go on the wire. b is Bob's secret, never to go on the wire "^" denotes exponentiation (think repeated multiplication). All arithmetic is modulo p Alice sends Bob: g^a Bob sends Alice g^b Alice can compute (g^b) ^ a Bob Can compute (g^a) ^ b And those are both the same! Now Alice and Bob share a secret that nobody else has. That's all that's needed for bootstrapping privacy. Avoiding an active man-in-the-middle attack is much harder logistically. For that you need some kind of authentication (what one chooses to mean by authentication is a very interesting decision). That's part of why we have the horror show of certificates. They are not necessary but they are sufficient (as long as the certificate system isn't broken). | With Ansible I've automated the generation of a new DH parameter file | on each new server: | | openssl dhparam -out <filename> <size> | | Generating this file takes a significant amount of time (minutes) if | "size" is reasonably large (2096, although I would recommend 4192) | even on a modern machine.
You consider the numbers you gave as a typo (according to later mail). Actually, there is an argument to be made that if you are rolling your own, don't use popular powers of two.
A bad guy might have hardware optimized for powers of two (they are used in the vast majority of cases). Or precomputed tables. If you don't use a power of two, you through him off his book.
The core of this is that we think that g^a is much much much cheaper to compute than the corresponding discrete log (i.e. compute a, given g and g^a). --- Post to this mailing list talk@gtalug.org Unsubscribe from this mailing list https://gtalug.org/mailman/listinfo/talk
participants (3)
-
ac -
D. Hugh Redelmeier -
Giles Orr