SharePoint Test and Quality Assurance – Part 5 Office 365 Migration Testing


Some of you may have read my blogs on SharePoint testing, I wrote several blogs on the topic such as SharePoint test and Quality Assurance – Testing SharePoint out of the Box Part 3. Often overlooked or under funded, testing is very important to minimizing Business and IT risk because of the user interface differences, performance/capacity, user training impacts, custom code/templates, data and security policy impacts.

The blog will focus on the creating your test plan, test cases, report and presenting the results to manage expectations and correct any faults. Where possible I will provide documented example templates you can use to kick start you’re testing.

So where do you start? Create a test plan as follows:

  • Define the purpose of the testing – generally it’s to manage the risk associated with migration to understand the following:
    • Impacts of migration to end user from a usability perspective
    • New training required for end users
    • Determine initial impact help desk and support resources
    • Developing/Refining the communication plan
    • Verifying connectivity required between the environments
    • Verifying ability to enforce data and security policy
    • Testing migration tool sets to understand what can be automated
    • Identifying manual process tools wont address
    • Where additional scripting is required
    • Creating migration guides for the migration team (forms the basis)
    • Determining impact to servers, storage and network
    • Document operational jobs (e.g. backup, indexing, virus scans etc.) start and completion times as they will impact migration performance
    • DLP settings / upload non compliant data
    • Multifactor authentication settings / test variety of scenarios
    • Intrusion detection
  • Determine the staff and skills required.
  • Funding required for the staffing, tools and thirdparty.
  • Defining the test environment from a technology perspective.
    • Source and destination environments (Servers, network, software etc.)
    • Network requirements (e.g. bandwidth, firewall rules etc.)
    • Monitoring and reporting – capacity, performance and progress
  • Defining the dataset that will be used (e.g. what datasets are available that mimic production? Must a dataset be assembled from scratch?)
    • Create manually (Takes a lot of time or use scripts)
    • I’ve implemented a mix of both to best try to mimic production
  • Defining key outcomes / success factors.
  • Link to test plan template

Once you have completed writing your test plan, circulate the document for review and sign off with the key stakeholders such as the following:

  • IT Director – Executive responsible for environment. Depending on signing authority I suggest the CIO be looped in as well.
  • Product manager – owner of SharePoint / Collaboration service.
  • Service management – operation teams such as support, server, network, storage and directory teams
  • Quality Assurance Manager – manages QA team.
  • Engineering – engineering / architects responsible for the environment
  • Microsoft – your friendly Microsoft support person / team.
  • Third party – any third parties that support the environment such as staff augmentation our outsourcing.

Note it’s important to collect everyone’s feedback, answer their questions, educate each other and obtain sign off. By completing this stage correctly you will avoid organizational risk that could create roadblocks due to misunderstanding objectives and outcomes.

With the Test Plan signed off, you can now create your test cases and report documents. Your test cases document will contain a list of test cases which must follow the Rational Unified Process (RUP) format. Keeping its simple, each test case should contain:

  • Name – A descriptive name.
  • Description – Short description of the test case.
  • Data set – A description of the data set being used using the test.
  • Preconditions – What must be in place or is expected to occur be for test.
  • Post conditions – expected post conditions as a result of completing the test.
  • Steps – steps for tester to carry out test.
  • Outcomes – The results of the testing
  • Screenshots – Screenshots that show the outcomes where it makes sense (can be very effective in communicating).
  • Link to test cases template

With the above format in place, the following test cases are recommended:

  • Site Migration to destination environment(s) SP201x and or O365
  • Home page migration (especially from 2007 environments)
  • Permissions groups
  • User IDs
  • Lists especially large
  • List views
  • Workflow
  • Complex list / library / View
  • Custom site template
  • Custom list / library template
  • Email enable libraries
  • Custom workflows
  • Custom forms
  • Drive mappings
  • Subsites
  • Sub Subsites
  • Custom .Net code
  • Unique permissions on child objects
  • Custom embedded HTML/Java code
  • Third Party applications
  • Site migration duration 500 MB, 1 GB, 5 GB, 25 GB and 100GB (Do for both SP201x and O365)
  • Broken links identification and correction (Update)
  • Impact on server work load
  • Impact on storage work load
  • Impact on network work load (Especially network link to O365)
  • Impact on capacity and reporting
  • DLP detection event capture, logging and alerting
  • Multifactor authentication
  • Intrusion detection event capture, logging and alerting

The above items are some basic tests than you can start with and build from there depending on your environment. For example, you might want to test Content Editor WebPart migration that contain custom CSS and Java, third party application migration, test impact/implications of settings such as versions and recycle bin. From a manual process perspective you might want to test Large List cleanup processes (Views, Indexing etc.). Some common problems you will find through testing (Each migration tool and environment will have its own quirks) are as follows:

  • UI changes – this is huge, users will be lost unless you test and providing quick training in easily consumable chunks.
  • Corrupted permissions – especially where there are unique permissions.
  • Large lists – 5000 item limit reached especially if you have legacy 2007 environments.
  • Corrupted lists/libraries – views don’t work properly, filtering errors and form issues.
  • Excel Spreadsheets – use lists in other sites as data sources and break during migrations.
  • Custom Code – code that wont migrate, must be rewritten.
  • UI quirks – content editor webpart, navigation, CSS and JavaScript.
  • InfoPath forms – especially those that use the old environment for lookups.
  • Wide open sites – permissions configured to leave site wide open.
  • Broken links – requires update tool in most cases.
  • Project 11 sites – these are sites the business builds on their own because their project wasn’t funded – often require time consuming reverse engineering.
  • FAB40 templates – if you have legacy 2007 environment(s).

For monitoring during load tests you should monitor network, VM Host (If you are virtualized) servers, network and storage devices. For Servers you can use PAL, for network speak with the network team and monitor traffic between new and old farms and O365. Don’t underestimate the impact on your capacity need for the link between Microsoft Office 365 and your office.

Some lessons learned include the team must have experience, management will oversimplify the work and bypass important tasks, document everything (Nothing in verbal format all in writing), capacity planning must be done in advance not as an afterthought, circulating and communicating results, dealing with problem managers/employees quickly and proactively/aggressively managing expectations. Involve all stakeholders and make sure they sign off in writing to avoid roadblocks and politics.

Found this helpful or have feedback? Contact me

How to onboard a new employee

1207671864ZIz0d4Chatting about this topic with some colleagues last week there were some interesting comments such as “You should know what to do…” or “We haven’t hired a new person in 12 years and don’t know what to do”.

Onboarding is a process of welcoming, educating, connecting, and acculturating new employees. It helps assimilate them into work and team processes and into an organizational culture. It provides new employees with the necessary tools and resources to carry out their jobs and clear channels for ongoing knowledge acquisition and collaboration. It instills in them a sense of connection to individual, group, and organization goals and a drive to contribute.

Keep in mind the onboarding experience sets the employees perception of the organization they joined. Therefore it’s in the employer’s best interest to make sure the experience is positive.

The best examples I’ve seen of onboarding are as follows:

  • Manager introduces employee to the team and persons they will work with directly.
  • The usual tour of the office and its amenities.
  • Provide laptop, IDs and access tokens as required.
  • Office dress code – give examples. I mention this because the policy is so varied now in organizations. Older workers might prefer suite and tie while younger workers jeans and shirt.
  • Mobile office policy/home office – provide employee with direction regarding policy surrounding working from home or mobile. Can they yes or no? If yes under what circumstances. Again this varies by company, some companies allow employee to work from where they need to as long as they meet their goals, some want butt in chair no questions asked.
  • Connect employee with HR for payroll and benefits information and enrolment.
  • Explain how their performance will be measured – the specifics prioritized. Such as utilization on billable projects, sales quota for services and products, winning proposals, project completion on budget and time.
  • Establish the communication rules between the employee and new hire – for example, use email? meetings? face to face? Explain how you prefer to communicate.
  • Highlights what’s in scope of the job and what isn’t giving real examples. For example, providing scope of services for the department, deliverables it provides and examples of the work the team does and doesn’t do.
  • Outlines required reporting, tools he/she will use and training required to achieve performance levels. Such as time management and billing capture.
  • Clearly outline demarcation points in the case of task handoffs, complex multi-owner scenarios and in writing notifies parties involved.
  • Quickly address team disconnects regarding new employees role – should they occur. Provide specific examples, steps and behaviour employee should have taken instead.
  • Help employee understand company culture and politics – makes employee mindful of landmines.
  • Assigns a great mentor (Has people, technical skills, knowledge of environment and how to get things done) to help the new employee be successful.
  • Establishes regular updates to provide feedback and coaching – use specific examples with background, not “I heard this…or someone said…”. Use actual job activities and peoples names. Provide both positive feedback on work and areas requiring improvement and how to improve.

When it comes to people, they can be your greatest asset and helping them onboard sets the stage for their performance. Some great examples I remember are those that helped the employee focus their strengths and gave them work that enabled them to achieve. Bad examples were micro management, not helping employee navigate land mines or directing them to enter a minefield for political reasons – this only creates distrust and demotivates the employee. The following is an example of a general onboarding checklist you can use as a basis.

Have feedback or stories to share? You can reach me at

Migrating to Office 365 or SharePoint Online? Part 6: Document and test your processes

iStock-Unfinished-Business-2Early in my career I lucked out being able to conduct process related consulting for business processes, receive Capability Maturity Model Integration (CMMI) and Rational Unified Process (RUP) training at HP and later work with some great quality assurance professionals. They say if you ask a manager for a solution, they will recommend rigorous process and policy to enforce. If you speak to a technologist, they will recommend tools – they are both correct. Not taking a holistic approach to migration projects will lead to delays, cost overruns and quality issues – a lot of frustration as well for all involved.

To proactively manage expectations, quality and consistency, document the processes for conducting inventories, communications, site cleanup, site migrations, customer support, training and retirement/archival of old sites and content. For a summary of my approach read my Migrating to Office 365 – 12 steps that will help you get there blog which is a summary of my migration framework scars and all.

Where do you start? First off you will require a team consisting of the following roles/persons and note the list varies in accordance with the size of your organization and staffing:

  • Technical writer – This person is a writer by profession and is tasked with writing the documentation (Very involved).
  • Communications person –This person is responsible for communications being distributed to the end user community such as emails, website communications, lunch and learns and posters.
  • Migration tool vender technical person – This is a contact from the vender of choice that will help you with inventory and migration testing and documentation (Very involved).
  • Project manager – A friendly PM that will help coordinate the efforts and herd cats as required – such as procedural ambiguity, anti-supporters that want you to fail – have seen a lot of these.
  • Quality assurance person – This person will create the test plan, test cases and write the test report (Very involved).
  • Operations person – This person will provide production insight and support.
  • Product management – The product manager for SharePoint/O365 must be involved to help with project activities and decisions related to the service offering and roadmap.
  • Engineering/architect – This is the technical lead with SharePoint, O365, Migration and procedural experience.
  • Help Desk – this is the lead for documenting help desk processes for calls, simple resolution, routing and escalation. If you have multiple help desks this person engages them and shares information. Note your help desk volume could easily increase 300-400%-Be prepared for this.
  • Security person – The security person provides security insight specific to SharePoint site security and data protection.
  • Legal counsel/records manager – provides guidance and direction regarding audit and records management such as site and data disposition.

The persons with “Very involved” are tasked with most the hands-on work while the others play and very important role as well but mostly provide guidance, reviews and other required insight specific to their area of practice.

Once you have the team assembled, they will be tasked with the following:

  • Create a Migration Central Site – this site will act as the hub for site owners and the migration project team for communicating project goals, site migration status (dashboard of sorts) and actions required, provide training materials, FAQs, host discussions and related project materials. By utilizing this site, you will reduce confusion significantly as people will know where to go for information and not chase down email and spreadsheets. Read more
  • Document your inventory process – working with either your toolset vender or developer, document the process for running inventories, assessing and categorizing sites by complexity (work required to migrate). I use the traffic light analogy, Green = out of the box, yellow = out of the box but with large lists, InfoPath and workflows and Red = Visual Studio customizations and or third party add-ons or those that have seriously exceed software boundaries (e.g. 250k item lists and 400GB site collections.
  • Document your quality assurance materials – this includes test plan, test cases and test report. You will be very surprised by the discussions this work initiates and it’s all good. The plan will include how testing will be conducted, why, the data set to be used to name a few. Test cases will include detailed test cases for migrating and testing sites. The test report is the outcomes of the testing and recommended next steps. I use the rational unified process, if you’re not sure what to test contact me or read more
  • Document your migration process – this is a document that steps the reader through the end to end process for migration sites whether they be simple (Out of the box) or complex (customized with custom code, add-ons etc.) sites. These steps will be tool specific and in the case of heavily customized sites, specific to your environment. To get started utilize the toolset vender’s manuals as they will help jump start the process significantly. For migration tools I recommend DB Attach as its supported by Microsoft (so no post migration support headaches) and is fast as long as you do your cleanup and planning. If you chose tools, there are several out there, some are difficult to work with, have poor support, have quirks, I recommend ShareGate for ease of use, price and company stability. if you’re not sure what to test contact me or read my O365 Migration testing blog.
  • Document your communications – all processes related to user awareness, preparation, status and follow-up. For example, what emails (Prep, during and post communications) will be sent to users, what it will contain and how replies and no-replies will be handled. Also how the help desk will be integrated into the process to screen and escalate calls accordingly. For example directing users to site migration status, FAQs, escalation process, VIP handling process, migration back out process, Training and Discussions on Migration Central Site. Or engaging support for port migration issues such as permissions or other functionality that not working as expected.
  • Document your Help Desk process and policy – the migration will generate additional help Desk traffic so be prepared, I’ve seen as much as 400-500% increase. Document process for taking migration related calls, what to collect, how to route and escalate, where to obtain additional support and training. Basic help desk scripts will help and a flow chart.
  • Capacity planning – projecting capacity needs (Servers, network, storage) and accounting for provisioning timelines (Provisioning lead times can be weeks or months depending on complexity of operating environment) for your environment. Weekly reporting is highly recommended to stay on top of projects versus reality. Some items to keep an eye on are growth in areas of sites, quotas, number of databases and size, search index size, SQL server drive space and server CPU, Disk, Memory and Network. If your virtualized make sure your farm is spread across multiple hosts and you mentor them as well.
  • Sign offs – Signs offs on all documentation by the major stakeholders such as IT, Records Management, Legal Counsel, Security, Operations, Engineering, Architecture and everyone else that is a stakeholder such as third party service providers.

But this is a summary only! Correct, it is. I’m under NDA in most cases and these materials are customer specific and cannot be distributed. Do I reuse them to help my customers fast track? Yes, but they require scrubbing and refactoring.

Some advice, work to involve all the stakeholders, remove negative people early in process as they will only work against you, raise awareness at senior levels (2-3 layers above frontline management), be prepared to deal with anti-supporters and noise makers – make sure everyone is actively involved as best makes sense and finally make sure your toolset provider has a local presence and experienced staff.

In summary, the aforementioned list of documents and advice will help accelerate your efforts. If you’re in a regulated industry such as Pharmaceuticals you might require more sign offs and documents such as IQ/OQs – your executive sponsor and PMO can guide you in this area. As I’ve mentioned in prior blogs I’ve worked in many industries and countries which helps me guide clients – make sure your PM and Architect have such experience.

Found this blog helpful or have suggestions? Contact me

Migrating to Office 365 or SharePoint Online? Part 5: Develop a technical architecture

Information-OverloadThere is a wealth of information available regarding how to design and build technical architectures but there isn’t a clear view of your organizations infrastructure, risks and operational details and politics. We can talk about the number of servers, network connection to Microsoft if your deploying O365 but at the end of the day the design will depend greatly on your environment from a technical, operating and financial perspective. For a summary of my approach read my Migrating to Office 365 – 12 steps that will help you get there blog which is a summary of my migration framework scars and all.

Let’s dive into these topics further and I’ll recommend some reading for those wanting more information.

  • Financial – First off, what is the operating budget for your environment (e.g. funding for staff, infrastructure and contingency)? Do you own the infrastructure or is it leased (e.g. is it new? Fully depreciated? Lease ending soon)? What funding do you have remaining currently? What have you budgeted for next year? If you can answer the questions, then great and if not I suggest you seek out someone with financial management insight and experience. With financial insight, you can  assemble a meaningful business case for your new environment based current state as a baseline. When selling to the business you can speak to new services, agility, consistency, simplification and mobility (Speaking to enablement, alignment with business roadmaps and ease of use are best). When selling to IT management its generally about cost optimization and risk mitigation (Value add will create road blocks as IT people especially Operations hate change and are usually oblivious to business roadmaps and risk).
  • Operating – What staff and skills do you have? Current workload? Full time vs contractors? Service agreements with venders and service providers? For example, depending on skills and workload you might have to lean on service providers more than usual. If you have outsourcing agreements in place, contractual changes might be required for new infrastructure and staffing for the proposed environment. Another example focuses on your organizations support model and the alignment between the groups such as SharePoint, SQL, Windows, Network,  Storage and backups. As organizations grow larger, outsource and pepper in company culture you quickly end up in a politically charged environment full of poorly designed handoff points, nebulous process and policy, finger pointing and management that doesn’t know how to execute or has lost control. For example, in 2010 I managed a large SharePoint environment and most the issues were organizational – people did not work together due to trust issues, IT did not have a solid grasp of their infrastructure – IT was powerless as they didn’t have executive support. Add poorly documented and negotiated outsourcing agreements which ended up paralyzing execution.
  • Technical – There are many variables that will steer your technical architecture such as funding, functional requirements, compliance and audit obligations to name a few. To simplify the discussion let’s assume you have decided on a Hybrid model with SharePoint 201x and O365 (50/50 workload distribution. There are a few key areas I’d suggest focusing on:
    • Data management – using you control plan as a reference, what tools and settings are required to ensure your data will reside in the correct environment and be enforced ongoing. For example, how will O365 Compliance Center be configured, reports required and staffing to operate? How will classification be optimized for user experience? What have past audits recommended? Yes, this is a huge topic, search my blogs for more information as this blog is merely a summary. Read more here
    • Network – what changes are required if any to support the network bandwidth required between your network and the Microsoft O365 data center(s)? Has your network team conducted an analysis? Your network team can run reports on your current SharePoint workloads from a network perspective. If additional bandwidth is required, how will it be funding? Key message, involve your network team and vender in the study and be aware of global implications as some regions have limited bandwidth and latency capabilities. Read more here
    • O365 tenancy – what applications do you require? Features and functionality? Capacity? Microsoft has some excellent documentation on the topics that will help you through the process of answering the questions. Read more here
    • On premise – what do you require on premise? Virtualization options? Server standards? How flexible is the virtualization team? How stable is their environment? How quickly can they respond to capacity and feature add requests? For example, I’ve worked with organizations where the SharePoint team owned the virtualized environment and optimized it for SharePoint and SQL Servers – ran smooth and was optimized. I’ve also worked with organizations where the Virtualized environment was not optimized for SharePoint and SQL workloads, it took 3-4 months to adjust for capacity and feature requirements and the team track record was not so good. Key message, as you look at options know who you are working with, their ability to deliver and reputation for team work. Read more here
    • Tools – You’ll require tools for migration, moving content, enforcing security and data compliance. For example, how will you migrate content to O365? How will you copy content from your on premise 201x environment to O365. How will you meet data backup and restore requirements? Assuming your migrating from 2007 or 2010 you will require a migration tool that that will migrate you to O365 and or 2016. Or maybe you have 2013 and can use DB Attach and then copy to O365 using tools. This all assumes your source environment can be migrated as is and doesn’t require significant remediation due to data policy violation and or technical issues such as exceeding software boundaries (e.g. large lists, site collection size, custom code or third party add-ons). Read more here

Many topics covered in a summary manner, yes it would be nice to have an end to end checklist or best practices but nothing replaces experience. The worst scenarios I’ve witnessed are oblivious management, an inexperienced technical team and project management. It’s not their fault, the executive team has not supported them sufficiently – SharePoint is generally a nice to have and not mission critical so doesn’t get much of their attention until an “Incident occurs”.

In summary, your end state architecture will depend on your information architecture, security and data policy, service levels, services available to you such as virtualization, capacity plan, operational model and deployment scenario. Specifically, whether you plan deploy hybrid or Office 365 only – Read my how to choose blog and SharePoint pro articles on Architecture. Microsoft has much information on this topic so I won’t go into much detail but will say that if you plan to go hybrid, consider SharePoint 2016 as hybrid is greatly improved and user experience more consistent with O365.

Found this blog helpful or have suggestions? Contact me

Migrating to Office 365 or SharePoint Online? Part 4: Enforcing your data and security policy

data-breachesMost SharePoint environments have grown organically and as a result SharePoint sites have become digital landfills with no monitoring, reporting and or enforcement. Its not that IT departments don’t want to do the right thing, they simply can’t in most cases due to lack of staff and tools. In addition, no control plan and executive support to execute and manage the control plan ongoing. Unfortunately, until there is a serious breach or lawsuit most executives are oblivious regarding the risks and the steps required to correct the situation. For my 12 step plan for moving to the cloud, read Migrating to Office 365 – 12 steps that will help you get there.

In Part 3, you were tasked with updating and reporting on all the site collection owners. Completing Part 3 the site collections ownership has been updated to include the business area executive, primary and secondary owners to reestablish security and data policy compliance, billing and audit requirements.

In Part 4, you will focus on a detailed data classification and mapping exercise that will enable your organization to enforce its data and security policy. For example, migrating data to the appropriate platform (Cloud / On premise) based on your policy. Transitory data that isn’t classified confidential or non-public can be moved to the cloud. Data that is confidential will be kept on premise. In addition, cleaning up permissions before the migration is recommended as most environments have not been setup and managed by untrained site owners. It’s common to find permissions issues ranging from numerous sites owners, lack of group usage and broken permissions at all levels of the sites and sub sites.

Sounds like a lot of work doesn’t it? It is and many skip it and risk data loss and law suits from exposing client information and or other non-public information. As one of my colleagues said to me once “Pain me know or pain me later”.

The following are the tasks that must be carried out:

  • Data Mapping – This exercise focuses on mapping the data in SharePoint to the new information architecture. It’s critical that this step occurs and that a high degree of communication and sign off occurs with the business users. To carry out this activity you will require a tool (e.g. NextLabs Enforcer) that scans your site collections and tags data based on your data policy. Work closely with your vender of choice as they will be able to provide you with the support and guidance required. Most scanners have options for scanning based on PCI for example. To determine what you must scan for speak with your Security Manager – request the Control Plan and Data policy documents. These documents will provide you with the information required to run scans and classify data. Keep in mind you’re not conducting an exhaustive classification exercise, that would take much to long in many cases due to the sheer amount of data that exists in many organizations and lack of resources. What your aiming for is traceability back to your Data policy so you can demonstrate compliance to an Auditor. Note that some may choose to simply migrate to Office 365 and then utilize Microsoft Compliance Center in Office 365. The key outcome of this exercise is an itemized listing of sites that must remain on premise and or data removed from SharePoint sites to remain compliant and a updated control plan (e.g. how to handle non-compliance such as notifying site owners and removing PCI data from cloud).
  • Security Model – Your security will provide guidance to SharePoint Admins and Site Owners for applying and enforcing security for site collections and the data that resides within them. Microsoft offer some guidance with a series of documents. I highly recommend keeping this model very simple (and building in measures, enforcement and reporting) as most likely your organization won’t invest much in site security. Here are some simple guideline and feel free to adjust them to your needs:
    • Utilize SharePoint Site Groups as the standard.
    • For data requiring different permissions (e.g. confidential vs public) utilize another site or create a library and break permission inheritance.
    • Allow site owners to manage all groups except Site Collection Admins and Site Owners.
    • Provide online how to videos – quick 1-2 minute how to.
    • Provide your help desk with support scripts.
    • Make site owner SharePoint training a yearly mandatory exercise.
    • Automate the deletion of users that have left the company.
    • Create procedures for audits and the work involved. Provide links to the site owners for the procedures.
    • Updated control plan that details how security will be configured, training required, reported on and enforced by tools, policy and staff.
  • Security cleanup – Here you will report on the current state of security for each site collection. This involves running reports on the site collection security and itemizing each entry and correcting if required. Sounds exhaustive? Yes, it is but in regulated industries you must provide such reports to Auditors to demonstrate compliance. There are many scripts available that can help you with this step such as As with any such resource and time intensive process utilize common sense. You have site collections that your SharePoint admin knows are a mess, focus on those first – think 80/20. The outcome of this exercise is a plan that addresses the highest risk site collections, how to address audit exercises and a plan for enforcing ongoing that is endorsed by the executive sponsor.
  • Multi factor authentication – create user training, policy and procedures for applying, monitoring and enforcing.
  • Ongoing scanning – since you have deployed tools to scan you should (MUST) consider implementing scanning on your destination environments so that you can enforce data policy. For office 365, utilize Microsoft Compliance Center and for on premise utilize Enforcer.
  • Vulnerability Assessment – conduct ongoing at least once every 1-2 years.
  • Control plan – detailing the upkeep roles (e.g. data custodians, provisioning, exception handling etc., monitoring (e.g. site security and data scanning), and enforcement so organization meets audit requirements. Finally, document any risks that have not been addressed and assign ownership to an executive sponsor whose mandate is complying with data and security policies.

Note that as you address your data and security needs, auditors do not like SharePoint (My Audit training in NJ 2008 was valuable, it gave me insight into how auditors think and how companies handle them) Why? Its distributed security model (e.g. Think AD which is centrally managed with rigor vs SharePoint sites which is in many cases a free for all) and lack of enforcement and structure. If you can demonstrate that your enforcing security and data policy to the best of your ability (e.g. executive support and funding) your ahead of the game.

Found this blog helpful or have suggestions? Contact me

Product Review for ITUnity – QIPoint – A Feature Rich Broken Link Reporting Solution for SharePoint

linksMany organizations have made a substantial investment in technologies such as SharePoint and the business users have embraced the technology for their everyday work. Generating thousands of sites and documents, organizations now have a user experience and maintenance issues managing the links contained with sites and embedded within documents. In many cases, the sites and links have become outdated and their value reduced because the creators of the sites and documents have left the company or moved on within the organization. Additionally, the sheer volume of sites and documents that contain links has grown to the point where it has become unmanageable using manual day-to-day processes. The impact to the user is a confusing experience where browsing results in errors and, as a result, users sometimes cannot carry out job tasks and lose confidence in the content. What’s required to prevent this is an automated process for scanning sites and documents on a regular basis and providing a report to site and document owners so they can best determine how to update the links.

Read full article here

Product Review for ITUnity – FlowForma – A Powerful Workflow Solution for SharePoint

workflowThere are several reasons to consider using workflow tools in your organization. For example, they can simplify, standardize, automate, reduce errors and accelerate business processes. Many organizations have business processes that are cumbersome and rely on a mix of tacit knowledge, email and printed forms that are slow, error prone and unstructured. Some organizations had adopted InfoPath but in January 2014, Microsoft announced that their investment in InfoPath had come to an end. With no recommended migration path in the SharePoint ecosystem, this is where FlowForma shines, with its usability and rich feature set. While some organizations had thought that their processes were too complex to be brought online, FlowForma is proving them wrong. The solution has already been highly successful in automating some of the most complex and fundamental business processes within large organizations.

Read full article here

Migrating to Office 365 or SharePoint Online? Part 3: Update your site and site collection ownership

ownershioMost SharePoint environments have grown organically and as a result the ownership hasn’t been kept up to date with employees, moving, leaving and changing roles. Why be concerned? Keeping ownership up to date is critical to enforcing data and security  policy, communicating migration plans to the business areas and site users. Expect to find many out of date and requiring time and attention to update.

As discovered in Part 2, your Information Architecture for SharePoint whether it be SharePoint 201x and Office 365 or simply Office 365 is critical for several factors such as usability, scalability, compliance and begin touching on your data custodian policy (Site ownership). Focusing on this exercise to document your organizations taxonomy (organizational lingo) and incorporate data and security policy is best practice.

For Part 3, you’re tasked with updating and reporting on all the site collection owners and in some cases sub sites as well depending on how your organization has managed site ownership. In many cases the site collections are owned by a business area with two owners and an executive sponsor to satisfy security, billing and audit requirements. In some cases this ownership is fully up to date and passed audits (have reported on ownership for audits and its 100% up to date). In other cases the ownership has not been kept up to date and audits failed.

Where do you start? The steps for Part 3 include the following:

  • Generate a report on site collection ownership – this script can help if you don’t have tools in place .
  • Have site archival process and policy ready as some site owners my ask you to delete the site collections rather than deal with the cleanup and preparation. Work with your records Management and or Compliance team.
  • Create and deploy a tool / process for updating site collection ownership. Some software venders offer tools for managing site lifecycle as opposed to developing your own which I have worked with. Here is a good blog on the topic Here is an example of a toolset .
  • Create and deploy a communication plan to reach out to each site collection owners to 1) confirm ownership and 2) update ownership if it’s outdated. Corporate communication policy and process aside, here is an interesting script that might help . The communication must consist of a multifaceted approach for communicating as no one approach will reach the masses. Such as mangers speaking with staff, email, messages posted on corporate Intranet, lunch and learns and coffee talks.
  • Create and deploy an escalation process for situations where ownership cannot be confirmed or reassigned. Work with corporate communications and your SharePoint user group to carry out this step. It must include an executive sponsored process and policy for contacting business owners and or management to request and obtain updated owner information.
  • For those with corporate provisioning systems that are currently being used to provision sites and manage lifecycle, loop in the team that manages the system to provide you with support for your project. This support could be reporting on site ownership, archival status (Retire as opposed to migrate), updates to ownership to name a few.
  • Leverage your SharePoint user (consists of business users) group. It will help communicate site ownership updates, help you get direct business user feedback which is very important. Its surprising how many companies have not yet established an open line of communication with the business.

The team required to carry out this work consists of an executive sponsor, corporate communications officer, Corporate provisioning team, SharePoint Admin, Developer, coordinator and SharePoint Product Manager.

Simple to carry out? Well it depends on the size of your organization (1000 vs 100,000), its culture (traditional hierarchical vs. team and goal centric) and how out of date (10% vs 50%) the ownership is for site collections.

Migrating to Office 365 or SharePoint Online? Part 2: Information Architecture and Control Plan

ExplainIA-Poster1-1024x791In Part 1, we focused on the discovery which focused on six critical reports that detailed your SharePoint farms and business data. Part 1 can be found here Migrating to Office 365 to SharePoint Online? Part 1: How to get started

Part 2 will focus on your Information Architecture for SharePoint whether it be SharePoint 201x and Office 365 or simply Office 365. The focus of this exercise is to document your organizations taxonomy (organizational lingo) and incorporate data and security policy as well. The desired outcome is a series of documents that will help you make decisions, set policy and its consistent enforcement and to configure your SharePoint 201x and Office 365 environment.

Why do organizations struggle with this topic? Information Architecture is a unique skill set very different from typical IT skills of technical infrastructure and applications. Information Architecture requires library science, facilitation and domain (e.g. Pharmaceuticals, health) knowledge because you’re dealing with information, its value in relation to job activities and its classification so it’s surfaced, protected and can be leveraged providing value to the organization. The value of such a role depends on how your organization views the value of its information, complying regulatory guidelines, passing audits successfully and your ability to leverage information assets. The value is compounded when you introduce cloud technologies because now you have a vehicle for storing data on a service residing within a vender’s data center – which introduces data protection risks.

So how do you get started?

  1. Assemble a team that consists of the required skill sets (Information architect/tech architect/facilitator, scribe…) and business representation. In a small organization this team might consist of a SharePoint Service Manager, Records Manager, Cloud vender and representatives from each line of business. In a large organization team might consist of a SharePoint Service/Product Manager, Records Manager, Compliance Office, Security Officer, and Representatives from each line of business and service management team (Operation team and or service providers).
  2. Review your organizations security policy and understand how that policy applies to protecting information – location, tagging, permissions and ownership chain. This information can be collected from a couple sources such as the Security team who should be able to provide data security guidelines and or policy documents and control plans. Also, auditor reports may also be made available which provides your insight regarding how your environment scored from an Auditors perspective. In general, auditors dislike the distributed permission control within SharePoint and the lack of ongoing data scanning and policy environment – view SharePoint as a black hole of sorts.
  3. Review your organizations data policy and understand how that policy applies to retaining information, tagging and classification. This information can be collected from a couple sources such as the Records Management team who should be able to provide data protection and retention guidelines and or policy documents and control plans. Also, auditor reports may also be made available which provides your insight regarding how your environment scored from an Auditors perspective. In general, auditors dislike the distributed permission control within SharePoint and the lack of ongoing data scanning and policy enforcement – think control plan, more on that later.
  4. Work with each line of business to itemize the documents, applications, people etc. they work with to get their jobs done. This exercise would be conducted for the major lines of business and departments and refined ongoing by working with department SMEs. For example, when I think about corporate information, the following questions come to mind. What information does the business collect? How is it used? How much of it is stored and where? Why is it kept? For how long? Here are some focused questions that will help but keep in mind you require someone that has done this before successfully:
    • What are all the different types of data and how are they classified? Do data owners exist for each data type or aggregate data collections?
    • How is data obtained? From whom? Why? Associated business process and or task?
    • What format is the data in? Application? Documents? Persons contact details?
    • How is data shared? With whom? Why? Associated business process and or task?
    • What are the business information availability requirements? Why?
    • What confidentiality, integrity, and availability requirements apply?
    • What is the legal environment surrounding the organization’s industry and the data it uses?
    • What issues have they experienced in the past?
  5. Once you have completed steps one through four the information architecture for the SharePoint environment(s) is documented to include the environments purpose (e.g. Office for external team collaboration or business partner collaboration), web applications, site collections, site templates (with branding), content types and related meta data (e.g. Dublin Core) and settings such as quotas, permissions settings and information management policy settings. Additionally, a control plan detailing the Information Architecture upkeep roles (e.g. data custodians, provisioning, exception handling etc., monitoring (e.g. site security and data scanning), and enforcement and reporting must be created so that Governance knows how to steer and enforce the Information Architecture. Finally, document any risks that have not been addressed and assign ownership to an executive sponsor whose mandate is complying with data policies and maintaining end user experience.

It’s important to note that your Information Architecture will evolve over time, refining it is very much an iterative process. As they say you cant boil the ocean and if you try, it will be a frustrating and unsuccessful experience. Hence why a diverse team and executive support is so important. Most organizations I’ve been asked to help focus on plumbing and not user experience from an information and content perspective. What’s the use of a highly available environment if the content is hard to find and of no relevance to the business?

Want to read up on the topic?


Migrating to Office 365 or SharePoint Online? Part 1: How to get started

cloud-whiteboardIn a prior blog I outlined a best practice approach for migrating to Office 365. Whether your just getting started or have had some false starts this blog and the series that follow will help you kick off your project or get you back on track. These blogs are written for the technical PM or the technical sarchitect looking for guidance on the subject.

To get started I will assume you have gone through your initial feasibility assessment and are a go. You are now at the stage of building your plan so you can execute your migration. If have not read my blog 12 steps to get to the cloud please read it now so we are on the same page regarding the work required to migrate successfully. For now i will assume steps 1 and 2 are complete, note that step 2 is critical as if you have issues with your farm such as errors and or capacity issues those will impact steps 3 onward – please correct those.

The discovery will provide critical reports that detail your SharePoint farms and business data. For the SharePoint farms, site owners, a detailed inventory of their configuration, customizations, data policy violations, third party tools to name a few. This inventory is very important because there could be elements of your configuration that cannot be migrated to the cloud (e.g. customizations, third party tools not available for the cloud).  For the business data, a detailed inventory of the site ownership, data contained within the sites, meta data, classification and security information. This report is critical to understand SharePoints current state regarding data and security policy compliance. Your records manager and security manager will be able to provide significant guidance in this area. For discovery tools, most venders provide tools for conducting the discovery exercises.

SharePoint farm data includes:

  • Provisioning, monitoring, correction and retirement systems, process and policy review. Documenting current strengths, gaps and risks with current solution and create plan to integrate Office 365. For example, data and security policy controls must be integrated into solution and be monitored, reported and enforced. user training and referenced material required so that end users are aware and informed.
  • End user training and support solutions must be reviewed, gaps/risks identified and a plan created. This include a central repository for training materials such as quick reference, how to videos, FAQ and QA forums to name a few. Also, training and compliance sign offs enforced by HR.
  • Server hardware and operating system information, location, capacity details, SharePoint version and patching details and third party tool/in-house developed add ons/customizations. It also includes a list of web applications, sites collections, sites and detail regarding each. For example, site collection type, features enabled, managed paths, site ownership, customizations consumed and inventory of sites, data, workflows, content types, templates and lists. Some useful scripts if you don’t have tools.
  • Network and security systems must also be assessed to make sure there is sufficient capacity, data policy enforcement tools, process and people in place to ensure compliance – control plan enforcement. This will help come audits to demonstrate compliance. If your not familiar with the topics visit SharePoint Pro for some of my articles and webinar recordings.
  • Security and data policy scans and reports must be completed of production environment to confirm sites are secure. If your organization doesn’t scan again open sites, run scans for NT\Authenticated user and Everyone as this is a security whole in most cases. Also run Credit Card and Personal Information scans as well to identify non compliant sites.
  • Application inventory is a detail view of each application that will be used to access its viability in the cloud or as a hybrid. Working with the business and IT owners to review requirements documentation (Functionality, architecture, SLAs etc.) and assessing workload requirements. For example the application may be an ideal candidate for the cloud running on Azure or it may be so old that it requires updating to run on the latest OS. There might also be interdependencies and latency might cause code to break. Worst case the developers are gone and there is no source control or documentation.
  • Business data  inventory is a detailed review of data residing on the farm with emphasis on data and security management policy compliance. The data on the farm is assessed for compliance to prevent customer data and or company confidential data from being placed in the cloud – security and risk teams nightmare. The data assessment requires a detailed scan of all site collections and sites using a tool (Nextlabs, Symantec, Quest and ShareGate are good companies to work with on this.) that can scan the data and look for patterns such as credit cards, social insurance numbers and the like. The tools also enable you to create your own patterns that enable you to scan on information related to your custom business processes such as account codes.

Key outcomes from this step are as follows:

  • You have an up to date list of site owners.
  • The site collections and sites are documented.
  • Security settings on SharePoint that’s not within policy are documented and recommendation documented.
  • Customizations and documentation uploaded to source control library.
  • The farm configuration (SharePoint, Sql Server, Windows, Servers, Storage and Network) is documented.
  • Data residing on SharePoint that’s not within policy is listed in a document along with the violation and recommendations.
  • Network impact assessment, capacity projections and recommendations.
  • Capacity projection for number of users and data along with associated risks (e.g. will more servers and storage be required? How much? When?).
  • The aforementioned documentation/reports are living documents, assigned owners (can be updated by running tools/scripts). Note the owners should not be contractors for audit and continuity purposes longer term.

Once you have the data its time for step 4, Develop an Information Architecture. That will be the next blog.

Have feedback? Would like to hear from you and your experience