SparkPost smart host issue with Sendmail

I recently created a SparkPost account to use SparkPost for smart host email delivery service for an organization's monthly newsletter distribution by email. SparkPost provides a free level of service that will allow one to send up to 100,000 messages per month. I had switched back to another service, but when I discovered a problem with deliveries through the other service yesterday, I reconfigured Sendmail to use the SparkPost SMTP server as the smart host. To use the SparkPost SMTP server, smtp.sparkpostmail.com, as the smart host, I had the following lines in /etc/mail/sendmail.mc, but email was not reaching recipients.

dnl # Uncomment and edit the following line if your outgoing mail needs to
dnl # be sent out through an external mail server:
dnl #
define(`RELAY_MAILER_ARGS', `TCP $h 587')
define(`ESMTP_MAILER_ARGS', `TCP $h 587')
define(`SMART_HOST', `smtp.sparkpostmail.com')dnl

The RELAY_MAILER_ARGS and ESMTP_MAILER_ARGS lines reference the well-known port 587 for email submission, since SparkPost's SparkPost | Sending Your First Email article includes the following:

Note that you should connect to smtp.sparkpostmail.com on port 587 using SSL/TLS and use AUTH LOGIN. Remember, you must issue your STARTTLS command with your application prior to issuing your AUTH LOGIN commands.

I had the following line in /etc/mail/access - the authentication credentials have, of course, been altered - to authenticate with the SparkPost SMTP server.

Authinfo:smtp.sparkpostmail.com "U:SMTP_Injection" "P:d12cc66a1a29755ef10fc1a1450d96dc23495970" "M:Auth Login"

I had previously created the /etc/mail/access.db file from the /etc/mail/access file with the makemap utility by running makemap hash /etc/mail/access.db < /etc/mail/access - see Configuring Sendmail to Use a Smart Host.

After changing the smart host in /etc/sendmail.mc, I restarted sendmail. Note: since the server runs CentOS 7, I no longer need to rebuild sendmail.cf from sendmail.mc with the m4 command, since it will automatically be recreated when I restart Sendmail.

# service sendmail restart
Redirecting to /bin/systemctl restart  sendmail.service
#

Though I thought I had previously been able to successfully send email through the SparkPost smart host, I found that test messages I sent weren't reaching their destinations and when I checked the mail queue, I saw the following:

# service sendmail restart
Redirecting to /bin/systemctl restart  sendmail.service
# mailq
                /var/spool/mqueue (2 requests)
-----Q-ID----- --Size-- -----Q-Time----- ------------Sender/Recipient-----------
u830l5aX024910       54 Fri Sep  2 20:47 <arachnidtrap@moonpoint.com>
                 (Deferred: Connection refused by mail.user-mail.net.)
                                         <test1a2b3c@aol.com>
                                         <abcde12ab@hotmail.com>
u830coTD024306      103 Fri Sep  2 20:38 <arachnidtrap@moonpoint.com>
                 (Deferred: Connection refused by mail.user-mail.net.)
                                         <moon1112223@gmail.com>
                                         <john.smith12@example.com>
                Total requests: 2
#

Since I was using smtp.sparkpostmail.com as the smart host server in /etc/mail/sendmail.mc, I wondered why I was seeing "Connection refused by mail.user-mail.net". An nslookup on smtp.sparkpostmail.com showed the following:

# nslookup smtp.sparkpostmail.com 8.8.8.8
Server:         8.8.8.8
Address:        8.8.8.8#53

Non-authoritative answer:
Name:   smtp.sparkpostmail.com
Address: 52.43.95.63
Name:   smtp.sparkpostmail.com
Address: 52.43.173.238
Name:   smtp.sparkpostmail.com
Address: 52.25.20.60
#

Whiile an nslookup on mail.user-mail.net showed quite different IP addresses:

# nslookup mail.user-mail.net 8.8.8.8
Server:         8.8.8.8
Address:        8.8.8.8#53

Non-authoritative answer:
Name:   mail.user-mail.net
Address: 198.133.159.138
Name:   mail.user-mail.net
Address: 198.133.159.121
Name:   mail.user-mail.net
Address: 198.133.159.122
Name:   mail.user-mail.net
Address: 198.133.159.133
Name:   mail.user-mail.net
Address: 198.133.159.137
Name:   mail.user-mail.net
Address: 198.133.159.139
Name:   mail.user-mail.net
Address: 198.133.159.135
Name:   mail.user-mail.net
Address: 198.133.159.132
Name:   mail.user-mail.net
Address: 198.133.159.125
Name:   mail.user-mail.net
Address: 198.133.159.120
Name:   mail.user-mail.net
Address: 198.133.159.134
Name:   mail.user-mail.net
Address: 198.133.159.126
Name:   mail.user-mail.net
Address: 198.133.159.136
Name:   mail.user-mail.net
Address: 198.133.159.123
Name:   mail.user-mail.net
Address: 198.133.159.119
Name:   mail.user-mail.net
Address: 198.133.159.124

#

I wasn't surprised by the number of IP addresses returned, since service providers often use a round-robin DNS technique for distributing the load between servers, but the IP addresses returned for mail.user-mail.net were in a different address block than the addresses for smtp.sparkpostmail.com. So I used the sendmail command sendmail -bt to see the mail exchanger (MX) system for smtp.sparkpostmail.com, i.e., the system designated to receive email addressed to that fully qualified domain name (FQDN). That showed that mail.user-mail.net was the mail exchanger system for smtp.sparkpostmail.com.

# sendmail -bt
ADDRESS TEST MODE (ruleset 3 NOT automatically invoked)
Enter <ruleset> <address>
> /mx smtp.sparkpostmail.com
getmxrr(smtp.sparkpostmail.com) returns 1 value(s):
        mail.user-mail.net.
>

I tried sendmail -q -v to force sendmail to immediately try sending the email messages again, but that resulted in the "Connection refused by mail.user-mail.net" message:

# sendmail -q -v

Running /var/spool/mqueue/u830l5aX024910 (sequence 1 of 2)
<abcde12ab@hotmail.com>,<test1a2b3c@aol.com>... Connecting to mail.user-mail.net. port 587 via relay...
<abcde12ab@hotmail.com>,<test1a2b3c@aol.com>... Deferred: Connection refused by mail.user-mail.net.

Running /var/spool/mqueue/u830coTD024306 (sequence 2 of 2)
<john.smith12@example.com>,<moon1112223@gmail.com>... Deferred: Connection refused by mail.user-mail.net.

I then tested to see if I could even conect to port 587 using the ncat utility. I received a "connection refused" message when I tried to access port 587 on mail.user-mail.net, though I was able to access SMTP port 25.

# ncat -vt mail.user-mail.net 587
Ncat: Version 6.40 ( http://nmap.org/ncat )
Ncat: Connection refused.
# ncat -vt mail.user-mail.net 25
Ncat: Version 6.40 ( http://nmap.org/ncat )
Ncat: Connected to 198.133.159.125:25.
220 SMMail 1.48.1 ESMTP
quit
221 mc117 closing connection. Have a wonderful day.
^C
#

I verified that the Internet Service Provider (ISP) that provides connectivity to the system sending email wasn't blocking that port by connecting to the portquiz.net site which allows you to test outgoing ports because the site listens on all ports. I was able to successfully connect to port 587 using both telnet and nc (nc is the same as ncat, which you can see by issuing the command nc -v).

# telnet portquiz.net 587
Trying 178.33.250.62...
Connected to portquiz.net.
Escape character is '^]'.
^C
Connection closed by foreign host.
# nc -v portquiz.net 587
Ncat: Version 6.40 ( http://nmap.org/ncat )
Ncat: Connected to 178.33.250.62:587.
^C
#

But I was able to connect to port 587 on smtp.sparkpostmail.com, which is the system SparkPost states should be used. E.g. SMTP Connection Problems states "The SMTP server name should be smtp.sparkpostmail.com or on some systems tls://smtp.sparkpostmail.com". I was able to connect to port 587 on that system with Telnet and ncat.

# ncat -vt smtp.sparkpostmail.com 587
Ncat: Version 6.40 ( http://nmap.org/ncat )
Ncat: Connected to 52.43.173.238:587.
220 2.0.0 smtp.sparkpostmail.com ESMTP ecelerity 4.2.24.56884 r(Core:4.2.24.8) Sat, 03 Sep 2016 21:15:12 +0000
^C
# telnet smtp.sparkpostmail.com 587
Trying 52.25.20.60...
Connected to smtp.sparkpostmail.com.
Escape character is '^]'.
220 2.0.0 smtp.sparkpostmail.com ESMTP ecelerity 4.2.24.56884 r(Core:4.2.24.8) Sat, 03 Sep 2016 21:16:04 +0000
ehlo moonpoint.com
250-momentum2.platform1.us-west-2.aws.cl.messagesystems.com says EHLO to 207.255.181.210:2814
250-ENHANCEDSTATUSCODES
250-8BITMIME
250-AUTH=LOGIN PLAIN
250-AUTH LOGIN PLAIN
250-PIPELINING
250 STARTTLS
quit
221 2.3.0 momentum2.platform1.us-west-2.aws.cl.messagesystems.com closing connection
Connection closed by foreign host.
#

The problem with email not being transmitted via smtp.sparkpostmail.com was that Sendmail was performing a Domain Name System (DNS) check for an mail exchanger record for smtp.sparkpostmail.com. Since it found such a record pointing to mail.user-mail.net, it would then attempt to send the email via that email server, instead. Normally, if you attempt to send email to an email address such as jsmith@example.com, the sending server will check to see what system is designated to receive email for the domain name, e.g., example.com. An MX record pointing to another system, e.g. mail.example2.com, indicates that the email should be sent to the system designated to receive email for that domain name by the MX record. But I didn't know how to keep Sendmail from performing the MX lookup until I came across SparkPost as a SendMail relay. I noticed in the example posted there that smtp.sparkpostmail.com was enclosed within brackets in the smart host line in /etc/mail/sendmail.mc. I.e., in the sendmail.mc configuration posted there, I saw the following three lines:

oreilly.com - Your tech ebook super store
define(`SMART_HOST',`[smtp.sparkpostmail.com]')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
define(`ESMTP_MAILER_ARGS', `TCP $h 587')dnl

Whereas I had the following comparable lines:

define(`RELAY_MAILER_ARGS', `TCP $h 587')
define(`ESMTP_MAILER_ARGS', `TCP $h 587')
define(`SMART_HOST', `smtp.sparkpostmail.com')dnl

So I placed brackets around smtp.sparkpostmail.com and restarted sendmail. I then issued the command sendmail -v -q to force Sendmail to process the mail queue. I saw quite different results this time and when I checked the mail queue aftewards, all of the queued email had been sent and when I checked one of the recipient addresses, I found a test message that had been queued was delivered.

# sendmail -v -q

Running /var/spool/mqueue/u834WrN4014574 (sequence 1 of 3)
u834WrN4014574: locked

Running /var/spool/mqueue/u830l5aX024910 (sequence 2 of 3)
<abcde12ab@hotmail.com>,<test1a2b3c@aol.com>... Connecting to smtp.sparkpostmail.com. port 587 via relay...
220 2.0.0 smtp.sparkpostmail.com ESMTP ecelerity 4.2.24.56884 r(Core:4.2.24.8) Sat, 03 Sep 2016 21:23:15 +0000
>>> EHLO moonpoint.com
250-momentum2.platform1.us-west-2.aws.cl.messagesystems.com says EHLO to 207.255.181.210:2485
250-8BITMIME
250-AUTH=LOGIN PLAIN
250-AUTH LOGIN PLAIN
250-STARTTLS
250-PIPELINING
250 ENHANCEDSTATUSCODES
>>> STARTTLS
220 2.0.0 continue
>>> EHLO moonpoint.com
250-momentum2.platform1.us-west-2.aws.cl.messagesystems.com says EHLO to 207.255.181.210:2485
250-8BITMIME
250-ENHANCEDSTATUSCODES
250-AUTH=LOGIN PLAIN
250-AUTH LOGIN PLAIN
250 PIPELINING
>>> AUTH LOGIN
334 VXNlcm5hbWU6
>>> U01UUF9JbmplY3Rpb24=
334 UGFzc3dvcmQ6
>>> ZDEyY2M2NmExYTI5NzU1ZWYxMGZjMWExNDUwZDk2ZGMyMzQ5NTk3MA==
235 2.0.0 Authed. Go on.
>>> MAIL From:<arachnidtrap@moonpoint.com>
250 2.0.0 MAIL FROM accepted
>>> RCPT To:<abcde12ab@hotmail.com>
>>> RCPT To:<test1a2b3c@aol.com>
>>> DATA
250 2.0.0 RCPT TO accepted
250 2.0.0 RCPT TO accepted
354 3.0.0 continue.  finished with "\r\n.\r\n"
>>> .
250 2.0.0 OK F2/94-14584-44F3BC75
<abcde12ab@hotmail.com>,<test1a2b3c@aol.com>... Sent (OK F2/94-14584-44F3BC75)

Running /var/spool/mqueue/u830coTD024306 (sequence 3 of 3)
Closing connection to smtp.sparkpostmail.com.
>>> QUIT
221 2.3.0 momentum2.platform1.us-west-2.aws.cl.messagesystems.com closing connection
# mailq
/var/spool/mqueue is empty
                Total requests: 0
#

Note: for the lines below, the meaning of the lines is explained below them:

>>> AUTH LOGIN
334 VXNlcm5hbWU6
>>> U01UUF9JbmplY3Rpb24=
334 UGFzc3dvcmQ6
>>> ZDEyY2M2NmExYTI5NzU1ZWYxMGZjMWExNDUwZDk2ZGMyMzQ5NTk3MA==
235 2.0.0 Authed. Go on.

The first line is sent by the sending system indicating that the "AUTH LOGIN" method will be used to authenticate with the SMTP server. The AUTH command is an extension to the Simple Mail Transfer Protocol (SMTP) that allows an SMTP client to specify an authentication, method, in this case the LOGIN method, which indicates Base64-encoding will be used for the authentication credentials, i.e., the user name and password - see SMTP Authentication - Details for other types of authentication that can be used. The SMTP server responds with 334 VXNlcm5hbWU6. The VXNlcm5hbWU6 is Username: encoded by Base64 as you can see if you paste it into the decoder at Base64 Decode and Encode. U01UUF9JbmplY3Rpb24= is then the base64-encoded userid. SparkPost uses SMTP_Injection for the userid and U01UUF9JbmplY3Rpb24= is that userid encoded by Base64. The SMTP server then resonds with another 334 response, 334 UGFzc3dvcmQ6. That is the Password: challenge by the server. The response above, of course, is not the actual password that is used for authentication.

Note: you can find the Base64 encoded value for text by using an openssl command as shown below:

$ echo -e -n "SMTP_Injection" | openssl enc -base64
U01UUF9JbmplY3Rpb24=
$

After resolving the email delivery problem by adding the brackets around the fully qualified domain name (FQDN) for the smart host in /etc/mail/sendmail.mc and restarting Sendmail (since the system is a CentOS 7 system, I don't need to use the m4 command to rebuild sendmail.cf, it is rebuilt when Sendmail restarts) with service sendmail restart, I then also found the following at sendmail to a smarthost?:

Learn on Udemy Today!
In order to suppress sendmail doing DNS lookups for MX records, you
should be able to use the standard sendmail feature of putting the
hostname in [square brackets].  This may or may not work -- you'll
have to do some experiments.  First of all try putting the square
brackets into the SMART_HOST define above:

   define(`SMART_HOST', `[smtp.example.com]') 

If that doesn't work, you can try using the mailertable functionality.
Create a file /etc/mail/mailertable containing the line:

    .        relay:[smtp.example.com]

and process that into a .db format hashed file by:

    # make

Note: you can produce a .db file from /etc/mail/mailertable by using the command makemap hash /etc/mail/mailerable < /etc/mail/mailertable, but putting the smart host within brackets in /etc/mail/sendmail.mc resolved the problem I was encountering.

References:

  1. Sending Your First Email
    Last Updated: Jun 23, 2016 10:41AM PDT
    SparkPost Support Center
  2. SMTP Connection Problems
    Last Updated: Jun 07, 2016 05:57AM PDT
    SparkPost Support Center
  3. TUTORIAL - SparkPost as a SendMail relay
    By: jpstaub
    April 22, 2016
    PBXinaflash
  4. sendmail to a smarthost?
    By: Dr Matthew J Seaman MA, D.Phil.
    Date: February 8, 2006
    lists.freebsd.org Mailing Lists

Related articles:

  1. SparkPost - Addresses Associated With Bounced Email
    Created: Sunday September 4, 2016
    MoonPoint Support

 

TechRabbit ad 300x250 newegg.com

Justdeals Daily Electronics Deals1x1 px