EXM scheduling of A/B Test in non-UTC timezone

A/B Testing in EXM is a great way to learn from and optimise email campaigns to improve engagement results.  However I came across an an edge case that had some unintended consequences (at least in Sitecore 9.0 Update 1, perhaps in later releases?).


Very specifically, this issue is only apparent when you set up an EXM campaign to dispatch with the following:

  • A multivariate (A/B) test with the winner selected Automatically
  • It is scheduled for the future
  • It is scheduled in a Timezone that is not UTC
SendTest2
Setup the A/B test to be scheduled in a non-UTC timezone

We were seeing the test begin as usual, however once the test was complete the winner was not selected and the remainder of the campaign was not sent after the designated delay (stuck in the “A/B Testing” status).  This was odd, but I slept on it.

Investigating again in the morning, the campaign had completed in the middle of the night. 🤔  Odd.  So I set up another test with the winner to be selected just 1 hour after dispatch.  Same result.  Stuck in the “A/B Testing” after the winner should’ve been selected.

After some digging I got a better understanding of how the scheduling of EXM mail works.  At the time of scheduling the dispatch, a “Dispatch” item is created in the /sitecore/system/Settings/Email/Tasks node of the Sitecore tree.  This item has the scheduled time (stored as UTC with appropriate timezone info) and various bits of info to associate the task with a campaign.

Dispatch task
Dispatch task item created when scheduling an EXM campaign

When scheduling an A/B test, there is an additional task item created called “Select Winner” that is supposed to have a scheduled time (in this case 1 hour after the dispatch task starts) in line with the settings you created.

select winner
Select Winner task with incorrect scheduled time

A close look revealed that the “Dispatch” Task got created as expected, with the appropriate scheduled time.  However the Select Winner task was created at the expected time, but in the UTC timezone.  In our case, being UTC +8, this meant the Winner was selected 8 hours after we would have expected. Not a HUGE issue, but undesirable.  What happens to a test scheduled in a UTC -6 timezone? Well…I would expect the Select winner task gets triggered almost immediately.

The workaround is to always schedule your A/B tests in UTC.  This will ensure everything is triggered at the correct time.  However, this can be error prone due to PEBKAC errors and our content editors have enough on their plate without always having to do time conversions.

After reporting to Support, there is now also a patch available, which Sitecore support should be able to provide guidance on whether it suits your solution.  Worked a treat for us.

 

 

 

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s