user command, but after I entered the command
the connection was terminated before I could enter the pass
command with the password for the account.# telnet 127.0.0.1 110 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. user nell Connection closed by foreign host. #
So I then checked Dovecot's log file. I saw many entries similar to the following ones in that file:
# grep dovecot /var/log/maillog | tail -5 Jul 4 09:15:44 moonpoint dovecot: master: Warning: service(pop3-login): process _limit (100) reached, client connections are being dropped Jul 4 09:18:55 moonpoint dovecot: master: Warning: service(pop3-login): process _limit (100) reached, client connections are being dropped Jul 4 09:19:57 moonpoint dovecot: master: Warning: service(pop3-login): process _limit (100) reached, client connections are being dropped Jul 4 09:21:01 moonpoint dovecot: master: Warning: service(pop3-login): process _limit (100) reached, client connections are being dropped Jul 4 09:26:13 moonpoint dovecot: master: Warning: service(pop3-login): process _limit (100) reached, client connections are being dropped #
The Dovecot wiki explains at Login processes notes:
Since one login process can handle only one connection, the service's process_limit setting limits the number of users that can be logging in at the same time (defaults to default_process_limit=100). SSL/TLS proxying processes are also counted here, so if you're using SSL/TLS you'll need to make sure this count is higher than the maximum number of users that can be logged in simultaneously.
I checked the default settings for Dovecot using doveconf -d and saw the following for the default_process_limit setting:
# doveconf -d | grep "default_process_limit"
default_process_limit = 100You can use doveconf -a to check on all settings with their
currently configured values; a setting may be different than the default
setting if you have changed it in a Dovecot configuration file.
# doveconf -a | grep "default_process_limit"
default_process_limit = 100The server haas far less than 100 users, so the process limit of 100 seemed adequate. So I didn't want to just raise the limit and restart Dovecot without understanding why the problem occurred. When I checked on the number of Dovecot processes on the system, I saw a count of 245.
# ps -aux | grep dovecot | grep -v grep | wc -l 245 #
When I looked just for "dovecot/pop3-login" processes, though, I saw the following:
# ps -aux | grep "dovecot/pop3-login" | wc -l 101 #
The command doveadm who can be used to check the number of connections per user and the associated source IPs. But I didn't see any listed.
# doveadm who username # proto (pids) (ips) #
I asked one of the users who was experiencing the problem to check her email and issued the command at the same time, but got the same output. She uses Outlook 2013 as her email client and saw the following "0x800CCC0F" error message from it when she attempted to check her email:
Task 'ann@example.com - Receiving' reported error (0x800CCC0F): 'The connection to the server was interrupted. If this problem continues, contact your server administrator or Internet srice provider (ISP).'
Since the system has far less than 100 user accounts that would check email on the server, I wondered if the issue might have been caused by someone attempting to gain access to an account by password guessing, e.g., perhaps by a dictionary attack. I used the doveadm command to determine the location of the files Dovecot used for logging.
# doveadm log find Looking for log files from /var/log Debug: /var/log/maillog Info: /var/log/maillog Warning: /var/log/maillog Error: /var/log/maillog Fatal: /var/log/maillog #
But when I checked /var/log/maillog, I didn't see any
evidence of failed POP3 or POP3S logins. However, when I checked the
Dovecot default and current settings for logging failed password attempts,
I saw that Dovecot was not configured to log information on failed login
attempts.
# doveconf -d | grep "auth_verbose" auth_verbose = no auth_verbose_passwords = no [root@moonpoint ~]# doveconf -a | grep "auth_verbose" auth_verbose = no auth_verbose_passwords = no #
You can find information on the Dovecot configuration settings for logging information on failed logins at Dovecot Logging:
There are several settings that control logging verbosity. By default they're all disabled, but they may be useful for debugging.
I edited the Dovecot configuration file, /etc/dovecot/dovecot.conf adding auth_verbose = yes to the end of the file to log failed authentication attempts. I then restarted Dovecot.
# service dovecot restart Redirecting to /bin/systemctl restart dovecot.service #
I then verified that the setting for "auth_verbose" was changed.
# doveconf -d | grep "auth_verbose" auth_verbose = no auth_verbose_passwords = no # doveconf -a | grep "auth_verbose" auth_verbose = yes auth_verbose_passwords = no #
The -d option shows that the default setting is "no" for
"auth_verbose", but the active setting was now "yes" as shown by
doveconf -a.
When I then checked on "dovecot/pop3-login", I saw only 1 process listed.
# ps -aux | grep "dovecot/pop3-login" | wc -l 1 #
I then checked to ensure that I could check email for a user's account by using Telnet to connect to port 110 and issue the commands to log in to a user's account and check for email.
# telnet 127.0.0.1 110 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. +OK Dovecot ready. user nell +OK pass SecretPassword +OK Logged in. stat +OK 0 0 quit +OK Logging out. Connection closed by foreign host. #
I also tried an incorrect password to verify that a log entry would be created in /var/log/maillog for the failed password.
# telnet 127.0.0.1 110 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. +OK Dovecot ready. user nell +OK pass testing -ERR [AUTH] Authentication failed. quit +OK Logging out Connection closed by foreign host. #
I then checked /var/log/maillog again.
# grep "dovecot" /var/log/maillog | tail -n 1 Jul 4 13:34:20 morpheus1 dovecot: pop3-login: Aborted login (auth failed, 1 att empts in 6 secs): user=<nell>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secur ed, session=<yRW9vtI2ywB/AAAB> #
The users were also now able to download email for their accounts.
References