Snappy, The HostGator Mascot

Gator Crossing

The Official HostGator Company Blog!

Dragonfly
AirPlane

Global Blacklisting Results in Undeliverable Email

Written by Sean Valant

Friday, August 23rd, 2013

Yesterday (August 22nd, 2013) a massive number of IP addresses used for email gateways on virtually every webhost in the world became blacklisted on multiple networks.  This resulted in a global inability for email to be received (any time the email originated from one of the blacklisted IPs and was “received” on one of the blacklisting networks).

The issue is on-going at the time of this writing, and some customers are still being affected at this moment, however HostGator was one of the first companies to successfully mitigate the situation and we have since been assisting other companies with this issue.  As it stands, we are presently working to now get our IP’s removed from the blacklists and restore full worldwide email deliverability from our network.

This situation resulted from a combination of multiple factors stretching back a few months.  Before we explain the circumstances, we want to once again stress the importance of keeping all scripts on all hosting accounts updated.  Failure to update scripts, as well as  not exercising basic security practices, is what allows situations like this to continue to occur.  An out-dated script on a hosting account is akin to an unlocked car left in a parking lot… it’s an invitation for maliciousness by unscrupulous individuals.

Unlike the situation back in April that affected WordPress, this time the target was Joomla.  Back in May, there was a string of exploits against known vulnerabilities in Joomla.  These vulnerabilities, related to a component called JCE, had been previously addressed via certain mod_sec rules.  However, a workaround was discovered that allowed malware to be installed, and later activated, to allow the uploading and execution of mailing scripts.

These mailing scripts were activated en masse yesterday, beginning a massive spamming campaign resulting in the blacklisting of email gateway IPs worldwide.  One of the largest networks with users reporting issues initially was AOL, resulting in us creating this forum post.

As with all issues of this nature, there are lessons to be learned.  The most important lesson here is to (again) keep all scripts on your hosting account up-to-date.  Most scripts have a one-click feature to update them anytime a new version is released.  Keeping scripts up-to-date is paramount in ensuring a secure hosting account.

HostGator has now added additional monitoring capability to our systems which will alert us to situations like this even faster than yesterday.  Our work is on-going, though we should have the majority of the blocks resolved by tomorrow (spam lists move slow, with good reason).  But remember, there is no better way to keep your car safe than to lock it.  Please take this moment to log into your hosting script back-ends and ensure they are up-to-date.  Don’t give the bad guys an open door to walk through.

Posted in

News Bites, Web Hosting News
Comments
  • http://blog.hostgator.com HostGator

    Each of the above have their benefits and drawbacks. But it is a shame that individuals with bad intentions can negatively affect such a large group of people.

  • รักหมดจัย นายตัวแสบ

    ความรัก

  • Daniel

    I still haven’t received my e-mail…:(

  • Leave Comments

    This is not true, all email with anything to do with AOL is being blocked for the 3rd day. My clients cannot conduct their businesses like this.
    This is a disaster. First Provo and now this. What a mess.

    • http://blog.hostgator.com HostGator

      Yes, AOL has been the big-name is the blacklisting as a result of this situation. We are still working directly with them to get all of our IPs whitelisted.

    • KittyDigitize

      AOL Blocking has been going on for the past 12 days. Not just 3 days. I’ve got well over 3,000 undelivered emails sitting inside my INBOX and folders.

  • Çiçekçi Siparişi

    izmir hali yikama izmirde hali yikama http://www.izmirhaliyikama.com.tr hali yikama izmir web sayfamizi sizlere sunmaktan gurur duyuyoruz. Sizlerinde katkısını bekliyoruz. Teşekkürler iyi çalışmalar dileriz.

  • Çiçekçi Siparişi

    izmir hali yikama hali yikama izmir http://www.ilaydahaliyikama.com web sitemizde sizlerinde katkilarini bekliyoruz izmir hali yikama fiyatlari

  • Buta Medya

    Emails do not work. I am going to loose my client. (gurkartekstil.com) :(

  • Mike_Janick

    We STILL CAN NOT SEND EMAIL.. How long does it take??!!

    • http://blog.hostgator.com HostGator

      The answer to that wholly depends on the receiving network and how long it takes them to properly clear out their incorrect blacklist entries. We do wish we had a better answer, but the final solution truly is beyond us and in the hands of the other networks.

  • KittyDigitize

    LIVE is blocking emails from my server and doesn’t even let them reach the JUNK folder when I test…. so the free services can also become and pain in the BEEHIND.

    • KittyDigitize

      I use HostGators email system… but when I’m testing and send from my server email addy to my LIVE account, LIVE blocks the test emails. They don’t even show up in SPAM or JUNK folders.

    • KittyDigitize

      Right now, I’ve set up “Postmark” and plan to use it to get around this mess….

  • BNB

    This is why you need to use Google Apps. You can setup your DOMAIN to use google/gmail very easily and still use Outlook or any other email program you are used to. Having email on your own server nowadays is foolish.

    • KittyDigitize

      But what prevents, say AOL from blocking GMail? :)

      • BNB

        It’s unlikely, as I don’t think AOL will be blocking Google anytime soon. And if there is a block, I trust that the biggest email provider and internet monster that is Google will have no problem removing the black list.

    • KittyDigitize

      Is there a place to read how to set this up? I’m sure many would follow suit….

      • BNB

        http://www.google.com/enterprise/apps/business/

        It’s actually quite simple. On your CPanel, you will just change the value of the “MX Entry” to the Google server. When you setup your account with Google, the instructions are very clear.

        You also get access to all of the Google apps and Gmail webmail. All email still shows just like your normal email/domain.

  • ssalari

    NIXSPAM, SORBS, and now SPAMCOP blacklisted HG!!??!
    AOL, LIVE, and thousands of other sites use above RBL listings. We and our clients can’t run business this way…Not acceptable!

    Use a new set block of IP addresses and switch over until you clean up the old set. There must be much higher resolution than simply email AOL and to ask to be delisted. We can’t just sit around and wait…too many customers and revenue at stake here.
    Are you saying that every time HG gets blacklisted, all your clients have to wait for weeks? Really?
    Is HG planning to give us credit for using a third party SMTP service provider while this mess is sorted out with solid backup plans in place?

    • Mike_Janick

      Can not agree MORE. This is truly ridiculous! After losing our server last month for days, now this??!! No amount of free months from HostGator is worth this..

    • ssalari

      1. HG has multiple gateways. Why the gateways with delisted IPs are not being decommissioned while you are working on fixing the problem?
      It’s better to send a delayed email instead of none at all!
      2. Why HG is not bringing online new IP addresses?
      What has HG done in the last 7 days? That’s a loooong time in technology terms in 2013!

      • disqus_JdOfEMCrTe

        There is no magical “Remove all blacklists” option though.. ANY webhost, not just HG wouldn’t be able to call the CEO of AOL for example and resolve the whole situation in an hour. That is why you’ve been waiting a week.

        I think you should educate yourself a bit more before making claims such as “Simply change the IP address blocks” … or “decommission problematic IP’s”… That would cause more havoc on the server than you can even fathom.

        Of course this isn’t acceptable. Nobody likes these issues. HostGator didn’t ask to be blacklisted, nor was it planned, so why should they compensate you..? Perhaps because you fail to have a backup plain?

        The servers are not offline. Your host doesn’t need to compensate you for something they didn’t cause. Use Gmail or Hotmail or something. Takes less than 5mins to sign up.

  • Mike_Janick

    And, hey, HostGator!! You guys took 21+ Hours to respond to a Ticket.. In fact, that is now the norm.. Open a Ticket on a Monday and don’t here crap until Tuesday??!! Come on..

  • Chris

    Why can I not find anything else that is related to this BL of IP addresses aside form this article on Hostgator and a post on your facebook wall? This leads me to think that the problem is only with hostgator.

    Its been 7 days now.

  • Markus Nelson

    Is there any update on this? I just lost another client. Spamcop should be contacted directly. .

  • Jason Haputudala

    I honestly dont see how this is hostgators fault. This affects many different hosts apparently as HG wasnt the only hosting company targeted. IF any of you actually read this, its apparent that the user level is at fault by not keeping your CMS up to date. Now HG has to fix all of this and if anyone has ever been blacklisted or infected with mail sending malware, you know it takes forever to get resolved. Now HG has to work with many different providers, at once, to get everything back in order which is going to take time since the blacklists are in place for a reason, simply “undoing” them all is not an option as many still need to be in place. HG has always treated me well as a customer, righting the wrongs that may or may not be their fault, heck, I even got a free month on them for the provo outage. Moral of the story, keep your CMS up today so you are not a contributing factor to something that someone else has to fix and inform yourself of a situation before placing blame.

  • vicki cramer

    Just wondering….how are we, HostGator customers/clients, going to know when this issue is fixed? I logged a ticket about a day ago and it is sitting in a queue w/out HG Tech Support letting me know what the issue is.

    Also, I am using Gmail with my HostGator email addy but I’m am having the issue of mail blocked/not getting through..so I guess I don’t have things set up correctly and/or gmail is also blacklisting HG? Sorry…I’m trying to understand the big picture of the issue and might have it a bit confused and I’m still reading through all the comments.

  • Schatje

    And now happen on my IP on TREND MICRO, LOL

  • James

    Can anyone comment on what will happen to the emails currently in the ‘holding pattern’ that this interruption is causing? Will these emails bounce back to senders? Will HG have the ability to attempt redelivery once the dust settles? I need to tell my clients something other than, ‘Just wait and we’ll see what happens’.

  • Charlie

    WHEN WILL THIS BE RESOLVED?????????

  • http://brandmanager.com.hr/ Jurica Dujmović

    Hostgator – we still have issues across all of our domains – all our mail got delayed and yesterday I was told we’re also getting blocked, so you haven’t yet resolved the problem!

  • Warren Daly

    Not completely true. Today (21 October 2013) you are still listed in several RBL lists.

qwaszxerdfcv3.14 | 1776zxasqw!!