Back in October 2006, I began blogging problems we were having in getting emails out to certain domain addresses - notably, any within the Microsoft group of companies, plus AOL and a growing list of others.  As we dug deeper into the issue, I added more posts regarding Microsoft proprietorising the Open SPF sender verification system and naming it as SenderID. 

By December, thoroughly exasperated, I was searching anywhere on the internet for a resolution and came across some useful blogs by other writers, that helped to move the topic forward, and for most of January, we had a functioning email server on our internal network, but still could not get email out to the domains mentioned above.

Now into February, the problem has escalated to one where no emails are leaving our network at all.  They just sit in the server queue until they time-out and are then returned to the internal user with an SMTP 4.7.0 error (message timed out and could not be delivered).  Suspicious that a new install of McAfee may have been at the heart of this new problem, and before attempting any resolution by fiddling with the servers, I went a-searching again.

One useful article I tripped over, harkened back to one of the error messages we were getting back from Hotmail and MSN, tagged onto an SMTP 5.x.x series error - it related to all Microsoft domains refusing email from unauthenticated senders.  I knew that our Exchange 2000 server had no method of authenticating outbound SMTP (Microsoft freely admits that they did not provide one in Exchange 2000) and immediately homed in on this article on Microsoft Technet.

The article title “How to Prevent Exchange 2000 From Resolving Anonymous E-mail Messages” was last modified in May 2005, and its introduction states -

Unlike Exchange Server 2003, Exchange 2000 Server resolves messages to the Global Address Lists (GAL) that are submitted anonymously. Exchange Server 2003 authenticates the message at the time of submission; if it does not authenticate, then the message is marked as such. For this reason, if you are upgrading from Exchange 2000, it is recommended that you upgrade gateway servers to Exchange 2003 before upgrading mailbox and other Exchange servers. Alternatively, to prevent your Exchange 2000 servers from resolving anonymous mail, you can perform the following procedure…..

“Oooh!” thought I, “maybe this will fix our SMTP error about unauthenticated email senders being rejected by Hotmail et al”.  I’ve just implemented the fix, which involves doing rather scary things with the Windows registry, but at least I didn’t have to worry about wide area network issues and cascading properties from gateway servers etc, that are mentioned in the article.

It isn’t a fix for no email at all going out, but it may help with undeliverable email issues hanging over from last year.  I’ll return and let you know, just as soon as we can get email to leave the local network and go out into the big world wide web again.

During the same hunt-the-needle game in microsoft.com’s haystack, I also came across this new knowledge base article, published 3 December 2007 (could it have been published due to the haranguing I gave them by email?).  Titled, “You are unable to send or receive SMTP messages from certain Internet domains in Exchange 2000 Server, in Exchange Server 2003, and in Small Business Server 2003“,  it addresses the SMTP 5.5.0 and SMTP 5.5.4 error messages. 

Unfortunately, when reading that article, Microsoft seem to have fudged the problem.  The article carefully explains why you cannot deliver SMTP mail to Hotmail, AOL etc, and describes the tech issues behind it, but they fail to give a definitive solution for the issue, and conclude the article with the raised-middle-digit statement of “This behaviour is by design”.  My current interpretation of this is that Microsoft is thinly disguising that they no longer want micro and SME businesses to use their server products, and only have tolerance (and support intent) for large enterprises with large budgets.

Again, I will dig into this further and try to return with an answer, though I’m not holding much hope - the last communication I got from Microsoft support (paraphrased) basically said, “Get a fixed IP from your ISP and bugger off and stop pestering us”.  Like all businesses who’ve got too big, I guess they’ve forgotten the little people who put them where they are today?

Ed

I’ve been doing some research (and receiving some from our forum users) regarding the ongoing saga of Hotmail (and others) blocking delivery of our emails to their users.

I’ve bumped into lots of other blogs which cover the topic including John Ward’s Thin End of the Wedge blog in Ireland, which gives some very good advice in terms of prioritising which parts of the problem to tackle first (thanks for the nudge Gaz), and Jim McBee’s Mostly Exchange blog from Honolulu covering Microsoft Exchange Server connectivity and other issues.

Another good primer article Gaz discovered is on the ListServ site, and it covers the technologies and authentication types used by the big internet mail providers.  Quote - “Yahoo! is a primary user of DomainKeys; GMail and AOL use SPF; and Microsoft’s Hotmail uses Sender ID.”  Unfortunately, it doesn’t give links through to the various systems or how to get enrolled with them, so that your emails can start getting through.  (A job for me to put right later ).

 It seems the whole issue is a lot bigger than I first though, and a lot more complicated to fix than simply adding a text record to your DNS records.

More on this will no doubt follow over the next few months - until then, you can catch previous posts of the problems in the Email Systems part of the blog, and the same-named section in the forums.

Ed 

Last month I blogged about Microsoft protecting Hotmail from Spam and blocking the world.  This month we’ve got some updates, and useful links. 

The following is a slightly edited version of a post by BuildaSkill member GazLanNaThai, originally posted on the community forums at www.eBid.tv - many thanks to Gaz for the heads-up, and permission to post it here, it saves me writing a post.

We’ve been having a major issue at this end with our mail servers not getting emails through to specific mail hosts - in particular Hotmail, MSN, gmail, AOL, and more recently several others. It came to a head in October 2007 when we noticed our mails were being returned as blocked by several private domain names. The Non-Delivery Record (NDR) gave an SMTP:5.0.0 error message, though the text note varied.

(Ed - same problem as we’ve been having, and the reason for the original blog post)

The prime problem appears if you have the following email setup -
1 - You use a broadband, ISDN, or fixed line connection to the internet, and
2 - you have your own internet domain name, and
3 - you host an internal mail server (e.g. MS Exchange or similar) at home or office

If you use only “Internet mail” through XP / Vista, or similar, on a single PC, you are very unlikely to be affected.

If you have a network of several PCs and use only direct “Internet email” (i.e. each PC uses your ISP mail server settings in Outlook etc) then you are unlikely to be affected.

Problem -

When sending email out to specific domains you get the problem above, but you are able to send email to domains such as eBid, eBay, or to Yahoo or other popular ISP mail users such as BTinternet, BlueYonder etc. But all email to Hotmail, gmail, AOL etc is getting bounced back.

This might be because the recipient has you in their spam list, but it is more likely to be because of a new anti-spam initiative called “SenderID” (by Microsoft and friends), or “Sender Policy File” (SPF) by the wider internet community.

SenderID and SPF are slightly different, but essentially do the same thing in the same way using the same protocols - unless you are a deep level techie, don’t worry about the difference - it only really affects massive corporate networks.

INFO SOURCES

The first thing you need to do is to identify if you’re likely to be directly affected.

The easiest way to do this is to go to www.spamhaus.org and click on the PBL link in the top middle of the header.
Then get your current IP address from your ADSL router or similar - this is the IP address your ISP gives to you each time your router connects to them (it changes each time).
On the spamhaus PBL page, at the top of the left margin is a box to check your IP address - copy and paste, or type, your IP address into it and click the button to check your IP.
If it returns a page with three green links, Great! It doesn’t affect you (today).
If one or more of them are red - then BOO!!!! you’re on the block list - and it’s probably not because of anything you’ve done, but because of what someone who previously had that IP address, has done.

Now you have to start studying - SpamHaus contains lots of reading material, and if you’re in anyway PC-techie, you need to read it all.  It’s a little heavy, but fairly straightforward if you have any IT Support experience.

SOLUTIONS

You’ll find that SpamHaus refers you to a Microsoft page www.microsoft.com/senderid/wizard - be very, very, VERY, careful if you use it. It’s unbelievably easy to make a complete and utter mess of your ability to send or receive emails through your internal server (I know! Been there, done it, got the t-shirt, broke my server ).

If you create a SenderID-SPF file using that wizard, BE WARNED - you DO NOT add it to your internal server’s DNS zones - it goes to your ISP for them to add to THEIR DNS server.

OPENSPF - OpenSPF are the original team who began developing the SPF anti-spam global system (typically Microsoft hijacked their idea and then began customising it to make it proprietary to Microsoft technologies, just like they did with SQL and other technologies).

www.openspf.org contains lots of info about the technology, from the originators, in both plain language and tech-speak - you can choose how deep you want to learn about it.  They also have a wizard ( www.openspf.org/wizard.html ) that does the same as the microsoft one, but it creates a truly universal SPF file - one that can be used on UNIX / LINUX as well as Microsoft servers and other less common server systems.  Their “SenderID vs SPF” page explains the differences in full, and makes a complling case for using the OpenSPF version.

READ the info pages before implementing - again, the SPF file created gets sent to your ISP - do not add it to your own server unless you’re on a permanent IP with fixed line connection.

Once your SPF file is in place at your ISP, and their DNS server has promulgated it around the world (allow 12-24 hours) you should get no more issues with Hotmail & co bouncing your emails due to “anonymous access” issues (read the sites mentioned to understand that).

Any tech questions - please refer to the sites stated, or your ISP support division.  If you ask me for any support, you’ll have to pay for it.  Your ISP’s will likely give you support for free.

Original content by Gaz, slightly edited by Ed.

Discuss this in our (members-only) forums or add a reply here.

About a month ago, I discovered that Hotmail are actively banning some email domains from sending email into the Hotmail system.  Fair enough you might think until you realise how they’re doing it, and that it has already been affecting you.

The problem shows itself as lots of problems sending email to Hotmail addresses via a web site, or a desktop through an ISP when you have your own domain name.   (more…)