Just How Do Email Servers Detect Spam?

January 11th, 2016

An email server detects spam by using spam filter software which evaluates incoming emails on a number of criteria. (Yes, you can run an email server without having spam filter software enabled - you’d just see any and all spam email.)

Now how do anti-spam software/services detect spam? They primarily utilize different methods including content-based message encoding and keyword filtering rules, email authentication rules (if the sending server has SPF/DomainKeys/DKIM/Sender ID enabled), mail server IP blacklists, and domain blacklists for sending domain and email content.

When a new email arrives on your mail server, it is initially evaluated against the IP and domain blacklists and for email server authentication status, and then for the email content rules. Usually, if the email scores higher than a preset criteria (for most services, an administrator can set scoring criteria) the email is marked as spam and dealt with appropriately.

Note: To reduce server load, spam filtering services will outright reject email that arrives from IPs or domains on blacklists (this is why ensuring the sending IP and all domains are not on blacklists is so important). They also might reject or delay email if you are an unknown email sender violating email sending limits (this is called greylisting). These tactics are designed to relieve email servers from abusive email spammers who try to send as many spam emails as they can in a short amount of time.

The most common spam filtering software is SpamAssassin, and many other popular spam filtering software/services use SA as a primary source. Other popular spam filtering services include Barracuda (hardware device based), Cloudmark, and a variety of MS Exchange based products.

Post inspired by this Quora thread.

Can Web Analytics Track Company Web Site Visits?

January 7th, 2016

Services like Hubspot, Leadlander and even the Google Analytics network domain function reverse lookup the WHOIS information for the visitor’s IP address to sometimes find the company domain of the visitor.

In some cases, the visitor’s organization may be acting as their own Internet Service Provider, and that information is reflected in the WHOIS lookup of the IP address. At other times, the organization may use a third party ISP like Comcast Business, and Comcast Business provides the organization’s name on the WHOIS record.

(Remember: you can always perform a manual whois lookup from an online tool like network-tools.com or via the command line.)

However, in my 20+ years of experience on the Internet, IP WHOIS lookup for an originating organization is maybe 70% accurate if you are very lucky. As of right now, there is no convention compelling ISPs to publish customer information - business or individual - on IP WHOIS records. Back in the day, Pacific Bell used to provide the customer name on the WHOIS for static IPs, but then there were privacy concerns for the customer.

In addition, I’ve heard of some marketing automation tools being able to identify a current visitor from an organization if a previous visitor with that same IP address (or IP address block) had given information to someone else using the marketing automation tool. You should consult your software provider on if/how they implement this “secondary lookup” method.

Post inspired by this Quora thread.

What Is An Email Suppression List?

January 6th, 2016

An email suppression list list — aka a suppression mail list — is a list of email addresses of people who have either opted-out of or have unsubscribed from your organization’s email marketing programs. Basically, you use a suppression list in conjunction with email marketing programs so you DO NOT contact these recipients. Most email marketing software has the functionality to allow you to maintain this list separate from your recipient lists. (A marketing pro-tip: Up-to-date unsubscribe and opt-out information should also be recorded in your organization’s CRM system so other parts of the organization always has access to this information.)

In addition, your organization’s email suppression list should be shared in the following situations: if you use a third-party provider for email marketing or if a partner executes a marketing campaign on your behalf. In these cases, you would provide your email suppression list prior to the launch of any marketing campaign so those suppressed email addresses can be scrubbed.

In the US, the CAN-SPAM Act allows organizations to take up to 10 business days to add unsubscribes and opt-outs to the suppression list, so it’s in your best interest to keep the list updated as soon as possible, if not in real time.

2016 Easy SEO: Update Your Web Site Copyright Date

December 30th, 2015

Here’s a quick and easy SEO trick for you.

We all know that Google indexes freshly updated content on your Web site. Since it’s the New Year, take a moment and check if your Web site copyright notices are updated. Not only do you get that piece of mind from an updated copyright date, but you’ve just updated each page of your Web site since each page should have the footer.

For those of you using PHP, here’s the code to insert into your footer, if it isn’t already there.

© Copyright < ?php echo date("Y") ?>

Easy, huh? (Yes, I have blogged this SEO tip before.)

Monitoring Email Deliverability With Email Seeding

December 20th, 2015

Most email marketers know they can track email opens, clicks and bounces. However, there’s a second common email marketing question: Can I see if my email lands in my recipient’s inbox or junk folder?

You can try what’s called “email seeding” or putting email addresses from various Email Service Providers onto your email lists so you can monitor where your email lands on each service.

A quick do it yourself way would be to have accounts on Gmail, Yahoo, Hotmail/Outlook, etc. and add those to your lists. I would also make sure to have seed accounts running Sendmail with a default SpamAssassin setup and a Microsoft Exchange with Outlook client setup, as these are the two most common private email server configurations. Many third party spam filtering solutions are based off of SpamAssassin filtering rules.

The disadvantage to the DIY approach is that you wouldn’t have direct access to private or corporate email systems. Professional email seeding services like Return Path, have a farther reach and claim to have access to a large number of ESP seed accounts.

Topic inspired by this Quora thread.

HubSpot Webcrawler

November 23rd, 2015

Apparently, SMB Internet Marketing services firm HubSpot has a webcrawler. This bot hit cleverhack over the weekend, crawling multiple pages of the site from 6 different IP addresses within the 54.174.#.# IP block (HubSpot AWS-HUBSPOT (NET-54-174-56-0-1) -, which is HubSpot using Amazon Web Services.

HubSpot Webcrawler

Yes we can bounce email from Mail.app

November 20th, 2015

Up until somewhat recently, Mac OS X mail.app had the handy feature of being able to manually bounce email. Technically, the mail bounces with a new header created, but still.

There are the Mail.app email bounce instructions from Lechnyr.

First, we need to add the ability to bounce email back into OS X’s Mail.app program. To accomplish this:

  1. Run the Automator program, located in your /Applications folder.
  2. When prompted to choose a type for your document, select Service and click the Choose button. You’ll now have a window that you can drag and drop various actions in to.
  3. Using the drop down menus at the top, make certain to indicate that the service receives no input in the Mail application.
  4. Drag Get Selected Mail Items into the workflow window.
  5. Next, drag Run AppleScript into the workflow window.
  6. Enter in some code (below) and save the workflow with a meaningful name such as, Bounce Message.

Here’s the AppleScript to copy and paste into the Automator workflow.

on run {input, parameters}
tell application "Mail"
repeat with eachMessage in input
bounce eachMessage
delete eachMessage
end repeat
end tell
end run

After you’ve saved the workflow, you’ll see this in your Mail.app menu. And yes, it does work in OS X Yosemite.

Bounce Workflow in Mail.app menu

Are 2015 E-Commerce Sales Slowing?

November 19th, 2015

There’s been some rumblings in the news lately that e-commerce - especially for traditional retailers - is in a slowdown. Witness the following headlines…

Target’s Digital Sales Slowdown Disappoints Investors - Forbes

Target, Wal-Mart See Online Sales Growth Ebb in Ominous Sign - Bloomberg

Macy’s and Nordstrom Struggle to Adapt to Changing Retail Trends - 24/7 Wall Street

Weak sales trend hits Dick’s Sporting Goods - Chain Store Age

So why are these traditional retailers having problems? The economy going soft? Buyers willing to use no-name or smaller e-commerce sites? The ubiquity of Amazon? Google traffic directed elsewhere?

Schrödinger’s Cat Email Marketing

November 10th, 2015

If you’re a Web-based service which requires your users to log in to unsubscribe or to change email preferences, I think you deserve your place in the spam folder.

Even moreso when your service fails to send password reset emails due to email deliverability issues and you require users to log in to use your Web site’s help functionality.


Adbeat bot

September 7th, 2015

Some Web bots are cool, others not so much.

If you take a look at the screenshot below, this adbeat bot (a competitive research tool for online advertisers) hit 108 pages on cleverhack in nearly 40 minutes. Hrm…

Not cool bot, not cool

Google PageSpeed Insights

August 30th, 2015

As we all — or at least SEOs and Web Developers — know by now, Google uses site speed as a factor in their ranking algorithm because in theory, a faster loading site is more useful.

This is a user agent for Google PageSpeed Insights (at least Google announces the intention in the description). Also, of note, this instance of the crawler was not part of a manual request but instead part of a quasi-regular crawl.

Http Code: 200 Date: Aug 30 09:47:41
Http Version: HTTP/1.1
Size in Bytes: 14300
Referer: -
Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; Google Page Speed Insights) Chrome/27.0.1453 Safari/537.36

Time goes by

August 24th, 2015

Upon the news that Rutgers University is spending millions on cyber security.

I’ll just sit here on my (virtual) rocking chair and muse about the fact that about 20 years ago, abuse of the the Unix ‘wall’ command caused minor havoc on eden.rutgers.edu, especially during heavy use in the evening.

And yes, the ability to use ‘wall’ was quickly removed by the admins.

Carry on.