Sitecore Symposium 2019 is coming up at the Swan and Dolphin* Resort in Orlando, FL. I live on the other side of the world from Orlando. Almost literally. It’ll take me just short of 24 hours flight time to get there. Add in transit and the inevitable delays, it’s over 30+ hours door to door…..each way. I’ll also be leaving a young family and (very understanding) wife at home. This bit, in particular I am not looking forward to.
When troubleshooting Sitecore issues and interacting with support there is often a need to confirm the versions of binaries in your solution. While there are tools to help provide all the information, sometimes its just not possible in some environments.
One particular issue we had required a detailed list of the bin folder. Handily the sitecore does write details of all the binaries at startup to the log. Un-handily, if you’re using Application Insights, you’ll likely have the log message truncated as it tends to exceed the 10,000 character trace message size limit.
Unfortunately it’s not as easy as just dir *.dll, as the details required include assembly names and version. But, powershell to the rescue.
This will dump the binary details out to a file in the same format as the Sitecore log would’ve done. If you’re on Azure Web apps, you can run this via the powershell console in KUDU. This file can then easily be downloaded then analysed, compared or submitted to support.
At the most recent meetup of the Perth Sitecore User Group, we tried a bit of a new format. Given we are physically isolated (surrounded by desert and sharks) and in a timezone that bends the space/time continuum, we have come up with some plans to further engage with the community around the world and start some discussions locally.
At the last meetup we kicked it off with Andy Parry giving a knowledgeable presentation on Universal tracker usage and installation. Then following a short pizza & beer break we moved into what we are terming the “Sitecore community roundup”. This is loosely based on the format of the “ASP.Net live community standup” (https://live.asp.net/), which if you haven’t checked out already….do. I presented a curated a list of blog posts/videos, community news & events that had been shared amongst the wider Sitecore community over the past month or so. We stepped through each link and gave a quick summary and kudos mentions to the authors before handing over to the floor for discussion. This worked really well, with participants engaging with related issues/solutions and starting some ad hoc discussion around each topic. It was a great conversation starter and attendees now have the chance to grab the links list and follow up on the topics that particularly interest them.
In future, we hope to try and engage with authors/contributors in a short, live Q&A or recorded interview typed format, to help foster further interaction with “the outside world”. If you find yourself reading this and have written a cool post that you’re keen to share, get in touch on twitter of slack. We’ll be doing this again mid-April.
Last, but not least. Below is a onetab link to all of the blog posts, videos, community news & events we talked about. If you wrote/recorded any of those…Thank you for your contribution! We’d love to hear from you to chat about it at a future meetup.
We noticed that the login on our CM was degrading, but with no real changes that we thought may affect this. At it’s worst it was taking up to 45 seconds to login successfully. I did a quick profile to see what was going on. Lo and behold a lot of time was being spent hitting the core db looking for existing login tickets, which allows for the remember me functionality to work.
As mentioned in this post, there is an agent that is supposed to clean up the tickets once expired, but clearly something was going amiss. I was able to confirm the CleanupAuthenticationTicketsAgent was running, however it was never picking up any tickets for cleanup.
ManagedPoolThread #2 12:04:16 INFO Job started: Sitecore.Tasks.CleanupAuthenticationTicketsAgent
ManagedPoolThread #2 12:04:16 INFO CleanupAuthenticationTicketsAgent: Number of expired authentication tickets that have been removed: 0
ManagedPoolThread #2 12:04:16 INFO CleanupAuthenticationTicketsAgent: Total number of authentication tickets to process: 0
ManagedPoolThread #2 12:04:16 INFO Job ended: Sitecore.Tasks.CleanupAuthenticationTicketsAgent (units processed: )
There were definitely tickets that should have expired, so I hit up Sitecore support for some insight. As it turns out, this is a known bug in 9.0.1 and 9.0.2 as outlined at https://github.com/SitecoreSupport/Sitecore.Support.223702 . Before installing the patch though, we manually cleared out really old tokens in SQL.
-- EG. This will delete all tickets generated in Dec 2018.
-- Adjust the date string to suit your needs. Be careful :D
WHERE [dbo].[Properties].[Key] like '%SC_TICKET%'
AND [dbo].[Properties].[Value] like '%^201812%';
This had an immediate positive impact to login performance, jumping down to a second or two.
The patch has since been installed and tickets are getting cleaned up happily as reported in the logs. Happy days. As always check with support first before installing any patches that may not be appropriate for your specific environment.
When doing a “send quick test” from the EXM review tab, we were seeing everything working as expected for the first few sends. However on subsequent sends (usually the 4th – 5th) the email was not sent and the spinner next to the send button took a long time. The XHR request to /sitecore/api/ssc/EXM/ExecuteSendQuickTest eventually timed out and showed the error “We are very sorry, but there has been a problem, please contact your system administrator.”. I thought about doing that….but that would probably cause some sideways looks given I’d be talking to myself.
This only appeared to be an issue in a Paas environment (we were unable to reproduce locally) on Sitecore 9.0.1 (with the EXM cumulative hotfix). This did not appear to be an issue with any dispatch tasks (just the “send quick test”).
Investigating the CM logs revealed that the latter sends that were failing had the following entry, followed by not much at all:
ManagedPoolThread #6 07:36:59 INFO MessageTaskRunner is starting 0 e-mail dispatch worker threads.
Previous successful sends would see something more like :
INFO MessageTaskRunner is starting 10 e-mail dispatch worker threads.
Then a series of entries like the following indicating the thread spinning up, then exiting once no more work is required:
INFO E-mail dispatch worker thread ‘MessageTaskRunner worker thread 3’ is starting.
INFO E-mail dispatch worker thread ‘MessageTaskRunner worker thread 3’ did not find any active tasks and exits.
As this appeared to be a threading issue we experimented with various settings such as Numthreads & MaxGenerationThreads, but see similar issues for each. Just with varying number of threads spinning up/exiting on the successful attempts.
After discussing with support, they were able to identify a bug and provide a patch (ref 214025). This issue applies to Sitecore 9.01 and 9.02.
This patch replaces the implementation of the SendEmail processor in the SendEmail pipeline, which ensures that the thread semaphore is fully released for test sends. Without the patch EXM would only be able to send up to the amount of test messages equal to the MaxGenerationThreads setting as the threads are not properly released on previous attempts.
In preparation for a Symposium roundup presentation for the Perth Sitecore User Group, I collated a bunch of links to blog post roundups, slide decks, session videos and even some Github repos of demo code. There was way too much content to convey at a SUG presentation, but well worth sharing. So….here they are in no particular order! Of course, let me know if you’d like any links added.
Sitecore’s Experience Optimization content testing functionality is a powerful tool in the Sitecore XP arsenal. When coupled with a solid engagement value strategy, marketers can get deep insights into what actually works (rather than what we think may work).
….when it works
Recently we hit an issue in a recently upgraded (8.2 -> 9.0.1) installation that would not display newly created tests correctly. The process of creating the test all went smoothly, the test would actually be running…but it would not display in the “Active Tests” tab of Experience Optimization. Drafts and Historical tests all displayed as expected.
Testing through our environments identified that things were working locally, but any upstream deployments all reproduced the issue.
All tests get indexed by your search provider upon creation. As part of the upgrade we’d also switched search providers to Azure Search (locally we use SOLR) so thought I’d investigate the [sitecore-testing-index]. All looked ok ¯\_(ツ)_/¯ . Documents were being added for each test we created with what seemed appropriate data. There was also a fairly cryptic INFO message in the logs when attempting to load the Active tests:
9300 02:31:53 INFO AzureSearch Query [sitecore_testing_index]: &search=This_Is_Equal_ConstNode_Return_Nothing
After hitting up colleagues and Sitecore community slack we were still at a loss, so raised a Sitecore support ticket. After an open dialogue and many configs later, we had a solution!
This was a config issue with Azure search. One of the Sitecore fields to be added to the [sitecore-testing-index] is a flag to whether the test is running (“__is running“). The default configs fail to add this field to the index correctly. Easily fixed…..it’s a one line change (isn’t it always?).
The Content Testing configs for 9.0.1 (and 9.0.2) provided in the Sitecore package downloads (for Azure AppService) you’ll see the following: