The Email Server is Up
Monday, December 21st, 2009The issue has been resolved, but it may take a few hours to work through the 10,000+ emails that were jammed up.
Our apologies for any inconvenience this may have caused!
The issue has been resolved, but it may take a few hours to work through the 10,000+ emails that were jammed up.
Our apologies for any inconvenience this may have caused!
We are currently experiencing an issue with the mail server. We hope to have it resolved sometime this afternoon.
Our service was down today from 11:23 AM to 12:50 PM EDT. There was an issue with a power tap at the NOC where our server is located, which in turn fried a network switch.
MailScanner was installed this morning. You should see an icon for it in your hosting control panel. MailScanner incorporates SpamAssassin, so you will no longer see a link for that in the Mail section.
Those of you who have been using SpamAssassin will need to configure MailScanner to tell it how to handle your spam. You can now specify separate actions for each domain (if you have multiple domains) as well as two levels of SpamAssassin scores (and you can configure what those levels are). You can also whitelist (or blacklist) specific addresses or domains.
Recently, we’ve had a number of customers unknowingly use up all their allocated disk space. Usually what tips folks off to this is that they stop receiving e-mail. More times than not, the disk space is used up by spam e-mail that has accumulated in one or more mailboxes.
Each hosting account has what is known as a Default Address. This is where all e-mail ends up that is sent to an address on your domain that is not defined by a POP3 mailbox or an alias (forwarder). By default, this is set to your main account mailbox (defined by your username). Over time, this mailbox can fill up and potentially use all of your allocated disk space, which can prevent ANY e-mail from reaching your account.
How does this happen? One way is through what are known as ‘dictionary attacks.’ Spammers will often send messages to many names at your domain hoping that some will be legitimate e-mail addresses. All the ones that fail will end up in your Default Address mailbox. This mailbox may also contain ‘bounce’ messages, which are e-mails sent back to you when an e-mail from your domain can’t reach it’s intended recipient. Often times, this is also the result of spammers forging the ‘From’ address of their messages using an address at your domain.
Our recommendation is that you change your Default Address. To do this, click on the Mail applet in your hosting control panel, then click on Default Address. The recommendation given by CPanel here is incorrect. We recommend that you use ‘:fail:’ and not ‘:blackhole:’ (include the colons but not the quotes). When you set the Default Address to ‘:fail:’, mail sent to a non-existent address is rejected when another mail server tries to deliver it to our mail server. A bounce message is no longer sent to the sender, which in the past could have been used to indicate what addresses were legitimate.
By the way, some of you are using your account mailbox as a regular e-mail account. We do not recommend that you do this, and in fact, the above recommendation will prevent you from doing this. If you need to use your username as an e-mail address, create a POP3 mailbox with the same name.
As usual, if you have any questions or problems with this, please submit a ticket on our help desk.
The mailbox storage upgrade has been completed, but the installation of MailScanner has not. In order to ensure that things go smoothly, this has been contracted out to a third-party installer, and we are still waiting for them to reach us in their queue.
Tonight we will be upgrading our hosting service to a new e-mail storage format. The new format will give us better performance and will allow us to put some improved virus and spam checking tools in place.
The impact on most of you will be minimal. The e-mail server will be shut down for a short time, but no e-mail will be lost.
Those of you using NeoMail, or those of you using another IMAP client, may feel the effects of this a bit more. After the conversion, NeoMail will no longer be available. If you are using another IMAP client, you will need to resubscribe to any user-created folders you may be using.
Those of you using SpamAssassin will also be affected. After the conversion, you will find this tool replaced in your hosting control panel with one called Mailscanner. Mailscanner still uses SpamAssassin “under the hood” to manage spam, but offers some additional options not available with the old package. Unfortunately, if you are currently using SpamAssassin, you will need to reconfigure this tool after the conversion takes place.
If you have any problems with your e-mail after the conversion, or if you have any other questions or concerns, please open up a support ticket on our help desk.
Here are the details:
We are moving from UW-IMAP, which uses the mbox convention, to Courier-IMAP, which uses the maildir convention. Under the current (mbox) system, all mail you receive is stored in a single file which you can then download via a POP client such as Outlook or view via an IMAP client such as Outlook or one of the webmail programs offered in the control panel.
From a server administrator as well as a user viewpoint, there are some issues with this type of implementation. When mail is stored in a single file, for instance, interrupted POP sessions can leave behind lockfiles that must be manually removed, can result in duplicate download of mail when the session is resumed, and is difficult to manage when corruption of a message is suspected. In addition, the resource usage associated with some of the processes, such as the current POP daemon, can be quite high. Mail delivery can also be impacted by the single-file convention: when the inbox is locked while mail is being retrieved via a POP session or when a lockfile remains after an interrupted session, new mail cannot be delivered until that session is completed and the lock ends when the process is completed or until the lockfile is manually removed.
The new system is a non-locking system. Mail is stored in individual files rather than in a single large mailbox. This eliminates both the lockfiles that are required under the current system and the problems associated with those locks if sessions are interrupted - an interruption under the new system will simply allow the user to resume retrieval when they connect again. Due to the manner in which mail is moved to the user’s space, mail can also be delivered even if retrieval is currently underway. The resource usage under the new system is also significantly lower than under the current system, and will solve solve administrative-related issues we can see on servers that handle the largest volume of mail within the network.
One of the largest impacts of the conversion will be in the area of webmail. Under the new system, Neomail will no longer be offered as a webmail client from the control panel. Although we understand many people use and like Neomail, it has issues of its own. Of the three webmail clients available, Neomail is the most resource-intensive and the most unstable. Neomail has a bad habit of corrupting the files it uses and has, on occasion, and completely randomly, lost all mail for a user if they tried to perform an operation within Neomail. In addition, Neomail has not been in active development since 2002. Both Horde and SquirrelMail will continue to appear in the control panel. Of the two, we recommend the use of Horde over SquirrelMail. Part of the conversion process will take the Neomail addressbook (if it exists) and convert it to the addressbook for Horde.
One other impact at the user level is related to the use of folders, and requires a post-conversion operation at the user level. This is discussed below in “after the conversion”.
During the conversion:
Various processses that run the current mail system will be shut down. This is to protect the integrity of the files being converted, and means mail-related activities will not be available while the conversion completes. Mail sent to the server during this time will be stored while the conversion process takes place, and will not be lost. The new system will be installed and all mail directories will be converted to the new format required by maildir. Existing mail will be processed and converted. Neomail address books will be converted to Horde.
After the conversion:
Once the conversion is complete, the mail processes will be restarted and will be immediately available. Any mail that was stored during the conversion will be delivered at this time. Neomail will no longer function and will not be available for login. Clients will need to take action if utilizing an IMAP client (any standalone IMAP client (such as Outlook), Horde, Squirrelmail, and so on) or if previously using Neomail. Due to the conversion of mail folders, those people will need to resubscribe to their folders before those folders will be visible. Note that the mail within these folders has not been lost - the folders simply are not immediately visible because of the conversion process, which prepends the folder name with a leading dot.
The simplest way to resubscribe is to log in to SquirrelMail for the user who has folders that require resubscription. Select Folders from the top menu. On that page, you will be presented with a list of folders available for subscription. Select the folders to which you wish to subscribe and submit the form. To select multiple folders, hold the ctrl key and click on the folders to which you wish to subscribe. Refresh your folder list by clicking the “check for new mail” option in the upper left navigation bar. The folders will now be visible in the folder listing and the mail within those folders available for viewing and other operations. This is a one time operation. Folders created after the conversion process will be named in the new format and will be immediately available. Users of Horde will need to follow this procedure as well, but again, this procedure must only be done one time following conversion.
There are some changes to the mail directory structure after the conversion has completed. You will find several new directories in /home/users/mail/domain.com/username: tmp, new, cur, and (depending on how you get your mail), several folders beginning with “courier”. The tmp, new, and cur folders are where incoming messages are processed by the system. The “courier” directories hold information specific to your mail account(s). These new directories should not be removed.