Some of you may have read my blogs on SharePoint testing, I wrote several blogs on the topic such as SharePoint test and Quality Assurance – Testing SharePoint out of the Box Part 3. Often overlooked or under funded, testing is very important to minimizing Business and IT risk because of the user interface differences, performance/capacity, user training impacts, custom code/templates, data and security policy impacts.
The blog will focus on the creating your test plan, test cases, report and presenting the results to manage expectations and correct any faults. Where possible I will provide documented example templates you can use to kick start you’re testing.
So where do you start? Create a test plan as follows:
- Define the purpose of the testing – generally it’s to manage the risk associated with migration to understand the following:
- Impacts of migration to end user from a usability perspective
- New training required for end users
- Determine initial impact help desk and support resources
- Developing/Refining the communication plan
- Verifying connectivity required between the environments
- Verifying ability to enforce data and security policy
- Testing migration tool sets to understand what can be automated
- Identifying manual process tools wont address
- Where additional scripting is required
- Creating migration guides for the migration team (forms the basis)
- Determining impact to servers, storage and network
- Document operational jobs (e.g. backup, indexing, virus scans etc.) start and completion times as they will impact migration performance
- DLP settings / upload non compliant data
- Multifactor authentication settings / test variety of scenarios
- Intrusion detection
- Determine the staff and skills required.
- Funding required for the staffing, tools and thirdparty.
- Defining the test environment from a technology perspective.
- Source and destination environments (Servers, network, software etc.)
- Network requirements (e.g. bandwidth, firewall rules etc.)
- Monitoring and reporting – capacity, performance and progress
- Defining the dataset that will be used (e.g. what datasets are available that mimic production? Must a dataset be assembled from scratch?)
- Create manually (Takes a lot of time or use scripts)
- I’ve implemented a mix of both to best try to mimic production
- Defining key outcomes / success factors.
- Link to test plan template
Once you have completed writing your test plan, circulate the document for review and sign off with the key stakeholders such as the following:
- IT Director – Executive responsible for environment. Depending on signing authority I suggest the CIO be looped in as well.
- Product manager – owner of SharePoint / Collaboration service.
- Service management – operation teams such as support, server, network, storage and directory teams
- Quality Assurance Manager – manages QA team.
- Engineering – engineering / architects responsible for the environment
- Microsoft – your friendly Microsoft support person / team.
- Third party – any third parties that support the environment such as staff augmentation our outsourcing.
Note it’s important to collect everyone’s feedback, answer their questions, educate each other and obtain sign off. By completing this stage correctly you will avoid organizational risk that could create roadblocks due to misunderstanding objectives and outcomes.
With the Test Plan signed off, you can now create your test cases and report documents. Your test cases document will contain a list of test cases which must follow the Rational Unified Process (RUP) format. Keeping its simple, each test case should contain:
- Name – A descriptive name.
- Description – Short description of the test case.
- Data set – A description of the data set being used using the test.
- Preconditions – What must be in place or is expected to occur be for test.
- Post conditions – expected post conditions as a result of completing the test.
- Steps – steps for tester to carry out test.
- Outcomes – The results of the testing
- Screenshots – Screenshots that show the outcomes where it makes sense (can be very effective in communicating).
- Link to test cases template
With the above format in place, the following test cases are recommended:
- Site Migration to destination environment(s) SP201x and or O365
- Home page migration (especially from 2007 environments)
- Permissions groups
- User IDs
- Lists especially large
- List views
- Complex list / library / View
- Custom site template
- Custom list / library template
- Email enable libraries
- Custom workflows
- Custom forms
- Drive mappings
- Sub Subsites
- Custom .Net code
- Unique permissions on child objects
- Custom embedded HTML/Java code
- Third Party applications
- Site migration duration 500 MB, 1 GB, 5 GB, 25 GB and 100GB (Do for both SP201x and O365)
- Broken links identification and correction (Update)
- Impact on server work load
- Impact on storage work load
- Impact on network work load (Especially network link to O365)
- Impact on capacity and reporting
- DLP detection event capture, logging and alerting
- Multifactor authentication
- Intrusion detection event capture, logging and alerting
The above items are some basic tests than you can start with and build from there depending on your environment. For example, you might want to test Content Editor WebPart migration that contain custom CSS and Java, third party application migration, test impact/implications of settings such as versions and recycle bin. From a manual process perspective you might want to test Large List cleanup processes (Views, Indexing etc.). Some common problems you will find through testing (Each migration tool and environment will have its own quirks) are as follows:
- UI changes – this is huge, users will be lost unless you test and providing quick training in easily consumable chunks.
- Corrupted permissions – especially where there are unique permissions.
- Large lists – 5000 item limit reached especially if you have legacy 2007 environments.
- Corrupted lists/libraries – views don’t work properly, filtering errors and form issues.
- Excel Spreadsheets – use lists in other sites as data sources and break during migrations.
- Custom Code – code that wont migrate, must be rewritten.
- InfoPath forms – especially those that use the old environment for lookups.
- Wide open sites – permissions configured to leave site wide open.
- Broken links – requires update tool in most cases.
- Project 11 sites – these are sites the business builds on their own because their project wasn’t funded – often require time consuming reverse engineering.
- FAB40 templates – if you have legacy 2007 environment(s).
For monitoring during load tests you should monitor network, VM Host (If you are virtualized) servers, network and storage devices. For Servers you can use PAL, for network speak with the network team and monitor traffic between new and old farms and O365. Don’t underestimate the impact on your capacity need for the link between Microsoft Office 365 and your office.
Some lessons learned include the team must have experience, management will oversimplify the work and bypass important tasks, document everything (Nothing in verbal format all in writing), capacity planning must be done in advance not as an afterthought, circulating and communicating results, dealing with problem managers/employees quickly and proactively/aggressively managing expectations. Involve all stakeholders and make sure they sign off in writing to avoid roadblocks and politics.
Found this helpful or have feedback? Contact me firstname.lastname@example.org