When setting up secure SMTP connections, also known as SMTPS or SMTP over TLS (Transport Layer Security), you encounter issues with SMTP obfuscating appliances, like Cisco ASA or PIX.
These appliances contain a feature called fixup protocol smtp, SMTP fixup, (E)SMTP inspect(ion) or Mailguard (Cisco), which – when enabled – limit the number of allowed SMTP verbs or obfuscate parts of SMTP dialogs. This tampering will cause problems when you try to configure SMTPS connections between e-mail servers and will prevent you from configuring a working Exchange Hybrid deployment, where TLS secured communications between the on-premise environment and Office 365 is enforced.
Note: By enforced, I mean it requires TLS which is more strict than opportunistic TLS, which will attempt to set up TLS before continuing with regular SMTP communications.
You’ll end up with Office 365 specific Receive and Send connectors after setting up your Hybrid configuration using the Hybrid Configuration Wizard (or manually). Then, when sending e-mail between on-premise and Office 365, you notice e-mail doesn’t arrive and will remain queued.
After verifying your ISA or TMG isn’t blocking things (ports are open, SMTP filter not blocking STARTTLS etc), you start troubleshooting the issue by enabling Verbose logging for the Office 365 send connector (of course, this can also be achieved using the Exchange Management Console):
Set-SendConnector “Outbound to Office 365” –ProtocolLoggingLevel Verbose
The logs will by default be generated in the $exinstall \V14\TransportRoles\Logs\ProtocolLog\SmtpSend ($exinstall will contain the Exchange installation path).
If you look at the current log file, you’ll see the following pattern for each possible destination – in the example below the TX2EHSMHS017.bigfish.com – as the local Exchange 2010 hybrid server will try to get the message(s) delivered:
Outbound to Office 365,08CEBBEB07152FBC,0,,184.108.40.206:25,*,,attempting to connect Outbound to Office 365,08CEBBEB07152FBC,1,192.168.1.11:51841,220.127.116.11:25,+,, Outbound to Office 365,08CEBBEB07152FBC,2,192.168.1.11:51841,18.104.22.168:25,<,220 **********************************************************************************************, Outbound to Office 365,08CEBBEB07152FBC,3,192.168.1.11:51841,22.214.171.124:25,>,EHLO mail.contoso.com, Outbound to Office 365,08CEBBEB07152FBC,4,192.168.1.11:51841,126.96.36.199:25,<,250-TX2EHSMHS017.bigfish.com Hello [188.8.131.52], Outbound to Office 365,08CEBBEB07152FBC,5,192.168.1.11:51841,184.108.40.206:25,<,250-SIZE 157286400, Outbound to Office 365,08CEBBEB07152FBC,6,192.168.1.11:51841,220.127.116.11:25,<,250-PIPELINING, Outbound to Office 365,08CEBBEB07152FBC,7,192.168.1.11:51841,18.104.22.168:25,<,250-ENHANCEDSTATUSCODES, Outbound to Office 365,08CEBBEB07152FBC,8,192.168.1.11:51841,22.214.171.124:25,<,250-XXXXXXXA, Outbound to Office 365,08CEBBEB07152FBC,9,192.168.1.11:51841,126.96.36.199:25,<,250-AUTH, Outbound to Office 365,08CEBBEB07152FBC,10,192.168.1.11:51841,188.8.131.52:25,<,250-8BITMIME, Outbound to Office 365,08CEBBEB07152FBC,11,192.168.1.11:51841,184.108.40.206:25,<,250-BINARYMIME, Outbound to Office 365,08CEBBEB07152FBC,12,192.168.1.11:51841,220.127.116.11:25,<,250 XXXXXXXB, Outbound to Office 365,08CEBBEB07152FBC,13,192.168.1.11:51841,18.104.22.168:25,*,,Connector is configured to send mail only over TLS connections and remote doesn't support TLS Outbound to Office 365,08CEBBEB07152FBC,14,192.168.1.11:51841,22.214.171.124:25,>,QUIT,
As you can notice in this example, the SMTP banner has been turned into stars and AUTH and STARTTLS have been changed by the appliance into XXXXXXXA and XXXXXXXB. This behavior is typical of Cisco’s Mailguard feature; other smtp obfuscating appliances might result in different output. Key indicator is, expected elements are missing or garbled.
In such cases, you need to make sure SMTP traffic between on-premise and Office 365 goes through unfiltered. Depending on the capabilities of the device, you could allow the FOPE addresses to go through unfiltered. If that’s not an option, or you don’t like managing that address set, you need to disable the feature completely.
Note that Cisco is not the only vendor offering SMTP obfuscating products; companies like Checkpoint, Barracuda, Sonicwall or Symantec offer products with the feature mentioned above.
you can turn off this feature, it causes troubles not only between FOPE and Exchange it causes problems in mail traffic between internal Exchange servers including Exchange 2003, 2007 and 2010.
to turn it off use : no fixup protocol smtp
Thanks for the addition, I’ve added them to the post. As with Hybrid deployments TLS becomes a requirement, people might encounter this issue earlier. However, there are still people out there who think obfuscating the e-mail product/version in the SMTP banner is a good security measure.
Pingback: Exchange Hybrid deployment and SMTP inspection | EighTwOne (821) « JC’s Blog-O-Gibberish
We experienced this same behavior, but instead of disabling inspection completely, it is possible to make a custom inspection policy that allows TLS connections. Non-TLS conversations are still inspected. This worked for us for connecting from our hybrid servers to Office 365.
This policy below is a copy of the default policy, but allows and logs TLS. This inspect map then has to be specified on the inspection_default policy.
policy-map type inspect esmtp ESMTP_TLS-exempt_Inspection
description Custom ESMTP policy to allow TLS connections
allow-tls action log
match cmd line length gt 512
match cmd RCPT count gt 100
match body line length gt 998
match header line length gt 998
match sender-address length gt 320
match MIME filename length gt 255
match ehlo-reply-parameter others