Effective Stack Planning for Sitecore and SOLR

Effective Stack Planning for Sitecore and SOLR

  1. Overview
  2. Understanding Your Stack
  3. Assessing Your Needs
  4. Planning and Design
  5. Conclusion

Overview

The goal of this article is to document the lessons learned around maintaining Sitecore 9.3 and SOLR 8.1 and filling in some of the gaps that are left at times. I want to break this down into many parts. This will be more of a checklist of things I would have done differently and concepts I found useful. It will not be step-by-step. I have other articles that dive deeper into a few things.


Understanding Your Stack

While this may seem obvious to some, it is often times not focused enough on or poorly done. It’s important to understand the technology you are using, and the limitations it has.

I firmly believe that documentation by the creators of the technology is among the best ways to learn how it works and what it was built to do. Who else would put together a good baseline?

The best way to discover the limitations is by giving it to a skilled team. This allows them to see if its intended functions meet the needs of the business, and envision the potential.

In the context of the work I do, I applied these thoughts to both SOLR and Sitecore. Apache, the creators of SOLR, has put together some very in-depth documentation. It covers functionality, setup, administration, and maintenance for all of their SOLR versions. By reading, and even admittedly skimming these documents, I was able to build a really solid baselines of how it works and what it can do. It was up to me to see if it could be fit to meet our needs as a business. The same goes for a lot of the Sitecore documentation, the wiki can give you a good base understanding of anything. But again, it is up to the individual to test the limitations.

Without knowing this or simply reading and understanding the instructions, you end up with a sub-optimal setup right out of the gate. This leads to deploying what equates to a dev environment as production. Maybe it works for a few months, maybe it fails right away, but without an understanding of what you were setting up, you won’t be able to fix it.

It is also important to understand and know beyond just being able to fix or setup a stack. It allows to speak intelligently on the subjects, and assures that you are not being led down the wrong path by blindly trusting someone because you are not even sure what your needs are. This is often how shelfware is purchased.

This could also apply to your business needs, in that if you don’t have some understanding of them, you end up with something bloated or just outright wrong.

I don’t think you need to know every possible thing about your stack, but it is important to have that baseline and knowing where to find the resources to understand topics. (ex: Official wikis, documentation from the company)


Assessing Your Needs

This one has some overlap with planning and design, but I feel it also deservers its own section.

Often times we are not building something new, but rather maintaining something old. Consider the idea of moving your website between CMS or even from standalone to a CMS. You’re more than likely forced to port over something from the old solution. Whether it is data structures that have not seen a change since Windows Millennium Edition, or a UI that is essentially a graveyard of dead NPM packages, long abandoned by the passage of time, we are carrying over something from the past that we might not need to be.

I implore you to push and advocate for trimming some of the fat. If there is any chance for you as the engineer to push back even a little on the current setup, I advise you do so. Assuming you have either been on the same project long enough to have seen these pitfalls bite you over and over, or are just coming in and have some fresh eyes to give, you have to at least try. The business and processes have probably changed over the years, and you are more than likely fighting these old design patterns that no longer fit in with the business needs.

Re-evaluate technology in use. While big companies can float this shelfware year after year, when you have the opportunity to trim things, why keep another library you will have to maintain, update, or keep track of that the solution is not even using? I am on board with having a nice set of tools to use, and integrations, but what if it sits for years and you are not even using it, but having to code around it or maintain it? Perhaps even maintain infrastructure for it. This is why I advocate for challenging getting rid of it, or at the very least, re-evaluate and determine its usage offer a chart of pros and cons.


Planning and Design

This might be more specific to my experiences, but perhaps someone can find comfort knowing they are not the only one experiencing the same.

Planning relies on understanding, because a good plan cannot be executed without proper understanding. Sure you can toss something together and just jump right in, and this could work, but the risk of failure is much higher.

My preferred way of planning a stack has been a visual layout annotated with information. I also ask a few questions I may have as I go and get feedback on these items either during the reveal or even during the initial drawing stage. Having others answer questions also helps you perform a knowledge check.

in the context of Sitecore, it is pretty easy. I usually grab the things that are considered constants first. Based on the baseline understanding of Sitecore’s requirements laid out on their documentation, I know that I will need at least one CMS server, one SQL server, SOLR Index server. This would be the bare minimum to run Sitecore. Assuming you are not needing to scale of course. Sitecore provides a good separation breakdown in their documentation for scaled, so I recommend reading that for a true representation.

To continue the example, I would then determine the business constants. I ask myself high level questions with the intention of diving a little deeper into each one. For example, in no particular order:

  • What is our traffic like?
    • This is a ‘constant’ if you have it, if not, some assumptions can be made by evaluating similar services/sites via public trackers
    • This will also determine if you need to consider scaling out, and how far your static scaling needs to go
    • Helps to know numbers when load testing your site
    • Do we have or need a CDN? This can also be determined if you need one if you are heavy on the imagery and/or scripts. A CDN is never going to harm you if you can get your hands on it
  • Are we hosting internally or on a cloud platform like AWS?
    • This determines what sort of network topology you may have, server architecture, and what features you may or may not have access to
    • Determines what the team does/should know architecturally
  • Resources at our disposal?
    • We do not want to bottle neck ourselves with assumptions when we are allocated only so much RAM/CPU/Servers – some companies have endless resources
    • This will also let you know if you can build out a full SDLC (ex: Dev env, Test env, QA env, prod env)
    • We also want to at least try to be budget conscious of course too
  • Separation of duties?
    • This is important, because you again cannot make assumptions about having access to everything. It makes sense that a server team would not want anyone and everyone in their production environment. A frustrating, yet understandable hurdle. One important to understand early on when planning
    • Who makes the calls in the project? Product owner if any?
  • Approved software and licensing?
    • Sometimes you are locked into certain software, and license restrictions, this should be taken into account
    • This could impact build tools, deployment tools, and how this solution will be published
    • This is a constant, because most of the time a company wouldn’t or shouldn’t shift constantly to different deployment tools.
  • Current technology that must be integrated?
    • Some companies have antiquated technology that you must work alongside, and knowing this can help you add this to your planning.

You could modify these same questions to fit a correction of a bad setup as well, its not just exclusive to new setups, just something I wish I had the opportunity to do, and hoping to pass along these thoughts.

While the topology is very important, you cannot forget about the code side of things. Solution planning is very important. The base Sitecore solution is a great starting point, but if you plan to have various customizations, it’s best to set it up in a way that best suits your needs.

Sitecore’s documentation for setting up a solution in Visual Studio can be found here:

So assuming you are going to be following the above guide, or doing your own thing, this is more of a guide of what I prefer personally.

Realistically, you know what you need best, and this might not be a good fit for you and your team. This can start the discussion at the very least!

.
└── 1024KB.Solution/
    ├── 1024KB.Documentation/
    │   ├── Setup/
    │   │   ├── setup.md
    │   │   └── files/
    │   │       └── setupFiles.zip
    │   ├── readme.md
    │   ├── tdsglobal.config
    │   ├── buildfile.any
    │   ├── .gitignore
    │   └── .gitattributes
    ├── 1024KB.Foundation/
    │   ├── Kernal/
    │   │   ├── App_Config/
    │   │   │   └── Include/
    │   │   │       └── Configs/
    │   │   │           └── Mycustom.config
    │   │   ├── Constants/
    │   │   │   ├── SiteConstants.cs
    │   │   │   ├── ContentItemConstants.cs
    │   │   │   └── TemplateConstants.cs
    │   │   └── Models/
    │   │       ├── KernalObjectModel.cs
    │   │       └── Generated.cs
    │   └── Kernal.Items/
    │       ├── Master/
    │       │   └── ...
    │       ├── Web/
    │       │   └── ...
    │       └── Core/
    │           └── ...         
    ├── 1024KB.Frontend/
    │   ├── Areas/
    │   │   └── Web/
    │   │       └── Customview.cshtml
    │   ├── Models/
    │   │   ├── Base/
    │   │   │   └── BaseObjectModel.cs
    │   │   ├── FrontendObject.cs
    │   │   └── Generated.cs
    │   ├── Controllers/
    │   │   └── FrontendController.cs
    │   ├── Assets/
    │   │   └── src/
    │   │       ├── imgs
    │   │       ├── css
    │   │       └── js
    │   └── Frontend.Items/
    │       ├── Master/
    │       │   └── ...
    │       ├── Web/
    │       │   └── ...
    │       └── Core/
    │           └── ...    
    └── 1024KB.Services/
        ├── Models/
        │   └── Api/
        │       └── CustomApi/
        │           ├── Base/
        │           │   └── ApiObjectBase.cs
        │           └── ApiObjectExt.cs
        ├── Controllers/
        │   └── Api/
        │       └── CustomApiController.cs
        ├── Services/
        │   ├── Interfaces/
        │   │   └── IMyService.cs
        │   └── MyService.cs
        └── Services.Items/
            ├── Master/
            │   └── ...
            ├── Web/
            │   └── ...
            └── Core/
                └── ...    

My assumptions here is that you are using TDS, I also did not include my Unit Test folders for each sub folder, simply to save space. This would just be the starting point, the whole thing would change to suit my needs.

Something else to keep in mind that I won’t get too deep into, is the technology dependencies, such as nuget packages, NPM packages, build technologies, etc – I have run into issues where something just gets dropped, and we are so engrained with the technology – be it build tools or any third party tool – we cannot easily decouple from it. Something that should be considered as you plan this stage, but sometimes there is no way to know.

These are just a few things I recommend taking into consideration during your planning phase that, not only I wish I always considered, but now currently do. Maybe it was things you already knew, maybe not, I hope there was something you got out of this section, though!

Conclusion

To conclude for now, these are simply a few takeaways from what I have learned when planning, developing, and implementing a Sitecore solution. I would even say in general, it’s useful for overall setup of any project, if you take these as concepts.

Leave a comment

Damien Rincon

From Debug to Deploy: Lessons from the Fullstack Trenches.