What drives open rates? Study #1: A B2C company with send times, emojis, and more!

September 27, 2018 | Daryl Lu

Back when email broke out onto the world wide web, open rates were reaching 90% open rates. (This wasn’t an anomaly!) Of course, fast forward 20 years, and the average across industries is just over 20% (MailChimp). However, once you break these open rates by industries and markets, you have a much wider spectrum of email open rates. This is why a marketer’s job never ends — it’s about launching, measuring, iterating — for improvement.

We have recently started helping customers understand what their data means from their email campaigns. In some cases, this includes gathering the data via testing strategies to maximize open rates.

Case Study of a B2C eCommerce Store

Our current case study looks at a B2C online store. Our testing methodology included launching campaigns in four groups of 5,000 existing customers, running A/B tests (ensuring statistically relevant sample sizes), and with a 3-email series — first email, follow-up, and final follow-up emails spaced every 7 days. The existing customer was targeting a win-back strategy for those who engage on email (our “email lovers” segment) but have not recently visited the store. Average open rate of email campaigns over the previous 7 months was 12.05% — our baseline.

Our primary levers for testing open rates included:

  • Include/ exclude various emojis
  • Vary send days during the week (Monday, Wednesday, and Friday)
  • Vary send times (9AM vs. 3PM local times)
  • Other subject line content
    • Including a Play () symbol vs. a dash (-) following the insertion of the customer’s first name in the subject line, i.e. “Don ▷ Special Offer” or “Don – Special Offer”
    • Including one of the following phrases of: “You Deserve This” or “Reclaim Your Time” (Variant A: “Don ▷ You Deserve This” vs. Variant B: “Don ▷ Reclaim Your Time”)

Results

  • We found our first emails varied greatly between 16.4% to a high of 40.8% with an average of 25.12% (2.08x improvement over the baseline).
  • The second email in the 3-email series (follow-up) garnered open rates between 16.4% and 21.3% and average of 18.5% (1.54x improvement).
  • Our final follow-up email to the email series earned open rates from 21.1% to 25.8% and average of 22.78% (1.89x improvement).

What does this mean? Well, for our customers, we found including an emoji had a negative impact on open rates. Sending with an emoji resulted in a 20% open rate compared to 25% open rate without the emoji (1.66x and 2.07x improvements against baseline, respectively).

Sample A/B Tests Open Rates Outcomes
Sending on a Wednesday (average open rate of 30.28%, 2.51x improvement over baseline) drew 75% improvement on open rates compared to Monday (average open rate of 17.25%, 1.43x improvement over baseline).

Meanwhile, sending at 3PM had an average 28.6% open rate (2.37x improvement over baseline) compared to the morning at 9AM which saw a 23.5% open rate (1.95x improvement over baseline).

Average Open Rates by the Day

Of course, there are more tests to run. The most important aspect to running tests like we did is having enough data points. Though our tests have shown good results with at least 1.43x improvement over baseline, most are directional — not optimal.

Marketing today requires quick, thoughtful execution. Withholding campaigns for the sake of creating the “perfect campaign” would be a fool’s errand. Customers can quickly go to a competitor. Instead, it’s important to continually launch and test. Then, fine-tune your campaign to optimize. And even then, strategies will likely have to adapt to the needs of that time. Let continuous improvement and marketing innovation reign!

Other considerations

There are a number of other levers that not only can be tested, but have significant influence on the outcomes of things like open rate, including:

  • Email content (including structure). ISPs and email providers assess email content regularly which can heavily influence deliverability.
  • Considering other optimal times beyond the days and times tested. But also, consider shifting send times to be more in line with open times to drive higher open times by sitting towards to top of the email stack.
  • Emoji use has dramatically increased in all forms of communication, not just SMS. It’s no wonder companies like Apple continue to add new emojis to the selection. The consideration, then, must include not just the placement of the emoji, but which emoji? This becomes a study much like semantics.
  • The above results and campaigns were performed on a single segment of the customer base. It’s imperative to weigh open rates (and strategies) against other segments and their characteristics.
  • There’s a fine balance to be struck for marketing to customers on a consistent basis to stay top-of-mind with enough time between each email so that the customer does not feel inundated by your brand.
  • Click-through and checkout. It’s all about the full funnel/ sales process. Consider which you prefer: campaign 1 with high open rates but low checkout rates; or campaign 2 with low open rates but high checkout rates. Each has important factors to customer lifetime value beyond campaign revenue.
  • Though the average open rate hovers around 20%, this is highly variable depending on industries and markets. This is why testing is so imperative against a baseline of the business and competitors.