A/B Testing in EXM is a great way to learn from and optimise email campaigns to improve engagement results. However I came across an an edge case that had some unintended consequences (at least in Sitecore 9.0 Update 1, perhaps in later releases?).
Author: George (@gwdt)
List Manager and EXM contact search not returning contacts
While testing EXM on a Sitecore 9.0.1 instance we noticed some strange happenings when attempting to view previews for a specific recipient. While the lists were being populated ok, it was not searchable by email. This was also the case in List Manager.
Read More »
List binary details with powershell
When troubleshooting Sitecore issues and interacting with support there is often a need to confirm the versions of binaries in your solution. While there are tools to help provide all the information, sometimes its just not possible in some environments.
One particular issue we had required a detailed list of the bin folder. Handily the sitecore does write details of all the binaries at startup to the log. Un-handily, if you’re using Application Insights, you’ll likely have the log message truncated as it tends to exceed the 10,000 character trace message size limit.
Unfortunately it’s not as easy as just dir *.dll, as the details required include assembly names and version. But, powershell to the rescue.
get-childitem .\bin\* -include *.dll
| foreach-object { "{0}`t({1},{2})" -f $_, [System.Diagnostics.FileVersionInfo]::GetVersionInfo($_).Comments, [System.Diagnostics.FileVersionInfo]::GetVersionInfo($_).ProductVersion }
| Out-File -filepath assemblies.txt
This will dump the binary details out to a file in the same format as the Sitecore log would’ve done. If you’re on Azure Web apps, you can run this via the powershell console in KUDU. This file can then easily be downloaded then analysed, compared or submitted to support.
Sitecore Community roundup
TL;DR
All the links: https://www.one-tab.com/page/hhG8MeijS92Pk8xlCq9eFw
At the most recent meetup of the Perth Sitecore User Group, we tried a bit of a new format. Given we are physically isolated (surrounded by desert and sharks) and in a timezone that bends the space/time continuum, we have come up with some plans to further engage with the community around the world and start some discussions locally.
At the last meetup we kicked it off with Andy Parry giving a knowledgeable presentation on Universal tracker usage and installation. Then following a short pizza & beer break we moved into what we are terming the “Sitecore community roundup”. This is loosely based on the format of the “ASP.Net live community standup” (https://live.asp.net/), which if you haven’t checked out already….do. I presented a curated a list of blog posts/videos, community news & events that had been shared amongst the wider Sitecore community over the past month or so. We stepped through each link and gave a quick summary and kudos mentions to the authors before handing over to the floor for discussion. This worked really well, with participants engaging with related issues/solutions and starting some ad hoc discussion around each topic. It was a great conversation starter and attendees now have the chance to grab the links list and follow up on the topics that particularly interest them.
In future, we hope to try and engage with authors/contributors in a short, live Q&A or recorded interview typed format, to help foster further interaction with “the outside world”. If you find yourself reading this and have written a cool post that you’re keen to share, get in touch on twitter of slack. We’ll be doing this again mid-April.
Last, but not least. Below is a onetab link to all of the blog posts, videos, community news & events we talked about. If you wrote/recorded any of those…Thank you for your contribution! We’d love to hear from you to chat about it at a future meetup.
All the links: https://www.one-tab.com/page/hhG8MeijS92Pk8xlCq9eFw
Sitecore 9 slow login – clean up your tickets
We noticed that the login on our CM was degrading, but with no real changes that we thought may affect this. At it’s worst it was taking up to 45 seconds to login successfully. I did a quick profile to see what was going on. Lo and behold a lot of time was being spent hitting the core db looking for existing login tickets, which allows for the remember me functionality to work.
Upon investigating there were LOTS of tickets in the properties table of the core db. A quick google led me to some older but helpful posts including: https://blog.coates.dk/2016/05/11/sitecore-client-and-logon-is-very-slow-properties-table-again/
As mentioned in this post, there is an agent that is supposed to clean up the tickets once expired, but clearly something was going amiss. I was able to confirm the CleanupAuthenticationTicketsAgent was running, however it was never picking up any tickets for cleanup.
ManagedPoolThread #2 12:04:16 INFO Job started: Sitecore.Tasks.CleanupAuthenticationTicketsAgent
ManagedPoolThread #2 12:04:16 INFO CleanupAuthenticationTicketsAgent: Number of expired authentication tickets that have been removed: 0
ManagedPoolThread #2 12:04:16 INFO CleanupAuthenticationTicketsAgent: Total number of authentication tickets to process: 0
ManagedPoolThread #2 12:04:16 INFO Job ended: Sitecore.Tasks.CleanupAuthenticationTicketsAgent (units processed: )There were definitely tickets that should have expired, so I hit up Sitecore support for some insight. As it turns out, this is a known bug in 9.0.1 and 9.0.2 as outlined at https://github.com/SitecoreSupport/Sitecore.Support.223702 . Before installing the patch though, we manually cleared out really old tokens in SQL.
-- EG. This will delete all tickets generated in Dec 2018.
-- Adjust the date string to suit your needs. Be careful :D
USE {core_database_name};
DELETE
FROM [dbo].[Properties]
WHERE [dbo].[Properties].[Key] like '%SC_TICKET%'
AND [dbo].[Properties].[Value] like '%^201812%';
This had an immediate positive impact to login performance, jumping down to a second or two.
The patch has since been installed and tickets are getting cleaned up happily as reported in the logs. Happy days. As always check with support first before installing any patches that may not be appropriate for your specific environment.
Sitecore MVP Awards 2019
Well. Just. Wow. I have been recognised as a Sitecore MVP for the first time. It’s still really sinking in and means a great deal to me. Being named alongside this incredibly talented list of elite professionals that show dedication contributing to the Sitecore community is truly humbling (and exciting). Please check out the full MVP list for 2019 and reach out to anyone that has helped you and say thanks…they deserve it! Thanks also to all past MVPs….we stand on the shoulders of giants.
Yeah, cool…..what is it though? I defer to the Sitecore MVP website for a great definition
The Sitecore MVP Award celebrates the most active Sitecore community members from around the world who provide valuable online and offline expertise that enriches the community experience and makes a difference.
Each year, the Sitecore Most Valuable Professional (MVP) Awards recognize individuals with a passion for sharing their knowledge and expertise through active participation in online and offline Sitecore communities.
This year there were 315 MVPs recognized (186 Technology MVPs, 37 Strategy MVPs, 18 Commerce MVPs, and 74 Ambassador MVPs) in 30 countries (13 in Australia).
I still consider myself to be very early into my Sitecore journey and I would not have received this award without having a really supportive organization, team, partner network and Sitecore reps. Thank you.
I’m glad to be a part of the community and looking forward to helping it grow, particularly locally in the most isolated city in the world! <plug>Get in touch if you’re in Perth to attend/present or would like to present remotely at #SUGPERTH </plug>. I’m also keen to present at other SUGs, so get in touch!
I’m looking forward to getting in contact and meeting (SUGCON EU? Symposium?!) more of you in the community this year. On the line, say Hi on Twitter, Slack or LinkedIn.
Really excited to see what we can do in 2019.
EXM previews not sending
Problem
When doing a “send quick test” from the EXM review tab, we were seeing everything working as expected for the first few sends. However on subsequent sends (usually the 4th – 5th) the email was not sent and the spinner next to the send button took a long time. The XHR request to /sitecore/api/ssc/EXM/ExecuteSendQuickTest eventually timed out and showed the error “We are very sorry, but there has been a problem, please contact your system administrator.”. I thought about doing that….but that would probably cause some sideways looks given I’d be talking to myself.
This only appeared to be an issue in a Paas environment (we were unable to reproduce locally) on Sitecore 9.0.1 (with the EXM cumulative hotfix). This did not appear to be an issue with any dispatch tasks (just the “send quick test”).
Investigation
Investigating the CM logs revealed that the latter sends that were failing had the following entry, followed by not much at all:
ManagedPoolThread #6 07:36:59 INFO MessageTaskRunner is starting 0 e-mail dispatch worker threads.
Previous successful sends would see something more like :
INFO MessageTaskRunner is starting 10 e-mail dispatch worker threads.
Then a series of entries like the following indicating the thread spinning up, then exiting once no more work is required:
INFO E-mail dispatch worker thread ‘MessageTaskRunner worker thread 3’ is starting.
………
INFO E-mail dispatch worker thread ‘MessageTaskRunner worker thread 3’ did not find any active tasks and exits.
Solution
As this appeared to be a threading issue we experimented with various settings such as Numthreads & MaxGenerationThreads, but see similar issues for each. Just with varying number of threads spinning up/exiting on the successful attempts.
After discussing with support, they were able to identify a bug and provide a patch (ref 214025). This issue applies to Sitecore 9.01 and 9.02.
This patch replaces the implementation of the SendEmail processor in the SendEmail pipeline, which ensures that the thread semaphore is fully released for test sends. Without the patch EXM would only be able to send up to the amount of test messages equal to the MaxGenerationThreads setting as the threads are not properly released on previous attempts.
Symposium 2018 roundup links
In preparation for a Symposium roundup presentation for the Perth Sitecore User Group, I collated a bunch of links to blog post roundups, slide decks, session videos and even some Github repos of demo code. There was way too much content to convey at a SUG presentation, but well worth sharing. So….here they are in no particular order! Of course, let me know if you’d like any links added.
Roundups:
- http://www.flux-digital.com/blog/sitecore-symposium-2018-thoughts-takeaways/
- https://bramstoop.com/2018/10/16/sitecore-symposium-must-read-takeaways/
- https://sitecorerunner.com/2018/10/08/sitecore-symposium-2018-day-1-sitecoresym-paragondev/
- https://soyouwannasitecore.wordpress.com/2018/10/17/symposium-2018-takeaways-and-note-taking
- https://tothecore.sk/2018/10/18/sitecore-symposium-2018-reflection-summary-series/
- https://herodigital.com/perspectives/sitecore-symposium-wrap-up/
- https://www.robhabraken.nl/index.php/3032/symposium-2018/
- https://sitecorerican.wordpress.com/2018/10/18/my-first-sitecore-symposium/
Session blog posts, slides and repos:
- https://github.com/cassidydotdk/Sym2018
- https://github.com/millsb/symp-gatsby
- http://www.pieterbrinkman.com/2018/10/18/whats-new-in-sitecore-9-1-a-technical-overview/
- https://www.robhabraken.nl/index.php/3000/integrated-devops/
- https://www.konabos.com/blog/video-of-our-sitecore-symposium-2018-talk-how-to-create-customer-loyalty-programs-using-sitecore-experience-commerce-and-email-experience-manager/
- https://www.jflh.ca/2018-10-13-helix-and-sitecore-javascript-services
- http://blog.baslijten.com/automate-your-pipeline-part-1-worlds-fastest-delivery-pipeline-for-sitecore-on-azure/
- https://jermdavis.wordpress.com/2018/10/11/get-your-sitecore-simposium-slides/
- https://www.slideshare.net/MarkStiles6/sitecore-symposium-2018-supercharge-your-author-experience-with-machine-learning
- https://blogs.perficientdigital.com/2018/10/10/sitecore-symposium-2018-measure-if-you-want-to-go-faster/
- https://www.dropbox.com/s/96lqlrksi2ayi3z/When_sitecore_squadrons_feel_the_need_for_speed-adam_simmonds.pdf?dl=0
9.1 (Not specifically Symposium related):
Active tests not showing
A/B Testing is great
Sitecore’s Experience Optimization content testing functionality is a powerful tool in the Sitecore XP arsenal. When coupled with a solid engagement value strategy, marketers can get deep insights into what actually works (rather than what we think may work).
….when it works
Recently we hit an issue in a recently upgraded (8.2 -> 9.0.1) installation that would not display newly created tests correctly. The process of creating the test all went smoothly, the test would actually be running…but it would not display in the “Active Tests” tab of Experience Optimization. Drafts and Historical tests all displayed as expected.
Testing through our environments identified that things were working locally, but any upstream deployments all reproduced the issue.
All tests get indexed by your search provider upon creation. As part of the upgrade we’d also switched search providers to Azure Search (locally we use SOLR) so thought I’d investigate the [sitecore-testing-index]. All looked ok ¯\_(ツ)_/¯ . Documents were being added for each test we created with what seemed appropriate data. There was also a fairly cryptic INFO message in the logs when attempting to load the Active tests:
9300 02:31:53 INFO AzureSearch Query [sitecore_testing_index]: &search=This_Is_Equal_ConstNode_Return_Nothing
After hitting up colleagues and Sitecore community slack we were still at a loss, so raised a Sitecore support ticket. After an open dialogue and many configs later, we had a solution!
The solution
This was a config issue with Azure search. One of the Sitecore fields to be added to the [sitecore-testing-index] is a flag to whether the test is running (“__is running“). The default configs fail to add this field to the index correctly. Easily fixed…..it’s a one line change (isn’t it always?).
The Content Testing configs for 9.0.1 (and 9.0.2) provided in the Sitecore package downloads (for Azure AppService) you’ll see the following:
<field fieldName="__is_running" cloudFieldName="is_running____" boost="1f" type="System.Boolean" settingType="Sitecore.ContentSearch.Azure.CloudSearchFieldConfiguration, Sitecore.ContentSearch.Azure" />
This code needs to change to the following (I recommend patching it to avoid any future deployment/upgrade issues):
<field fieldName="__is running" cloudFieldName="is_running_____b" boost="1f" type="System.Boolean" settingType="Sitecore.ContentSearch.Azure.CloudSearchFieldConfiguration, Sitecore.ContentSearch.Azure" />
The changes are subtle, but important! Firstly, it’s referencing the field name correctly (with a space). Secondly it is giving the index field name a _b suffix to further indicate it is a boolean.
After making this change and then rebuilding the [sitecore-testing-index] in the control panel, marketers and analytics teams rejoiced as they were able to view all of their glorious active tests.
Marketing Automation stored procedures and tables missing
Seen this message in the exception logs of your shiny new Azure app service instance of Sitecore 9?
Could not find stored procedure ‘xdb_ma_pool.AutomationPool_Stall’
ARRRRRGH. Missing Stored Procedures?!?! Who’s been dropping stuff in my DB! It’s newly deployed…how can this be! It worked on my dev machine! Never fear, there’s an explanation and easy fix.
No…you haven’t been dropping DB objects in your sleep. This just appears to be an oversight in the Marketing Automation SQL dacpac in the Sitecore 9 Azure App Service web deploy packages. In fact there are a few other objects missing too:
Stored Procedures
AutomationPool_Stall
ContactWorkerAffinity_ReleaseLease
ContactWorkerAffinity_TakeLease
Table
StalledAutomationPool
This has been confirmed by Sitecore support and they’ve published a KB https://kb.sitecore.net/articles/065636 . Also the KB now contains a SQL script to create the missing objects. Running this script on the Marketing Automation DB in your Azure environment should create the missing objects and resolve the issue. This is confirmed as an issue on 9.0.1 and 9.0.2.
We’ll be adding this as a post-install script to all our ARM template deployments to avoid any manual steps in future deployments.
