The “Soft” part of SharePoint - Part 2, Impact of design choices

During every SharePoint design phase, choices have to be made on exactly how to implement SharePoint and its components. These choice can have serious implications later on during the administration phase.

To be able to create a good design, it is imperative to have good requirements for the environment. These requirements must be gathered both at the business and at IT end, as they both have to “use” the platform in the future, although their use will be completely different.

For example:

1. Where the business has certain availability requirements (e.g. 99.9% 24x7), IT has the requirement that these availability requirements must be achieved using redundancy. The final design has to be an environment design with which all parties can live with.
2. In order to deliver a good service, IT will have to be able to test each change to the environment. In order to do this, they will require testing facilities in the form of DTAP environments. In most cases, the business will experience the DTAP strategy as annoying and time consuming.

It is very important that all choices are documented in a design document, especially when the persons performing the implementation are different than the actual administrators of the environment. The design document has to be reviewed and approved by the future administrators before implementation starts. If during implementation a deviation from the original design has to be implemented, this deviation has to be agreed between both parties.

Basically: If the implementation project messes up (for whatever reason), the future administrators will suffer the consequences!!

Very good examples of critical design choices with high impact are:

• Use of a DTAP environment* - During a project, implementing a DTAP strategy requires time and money. Two things a project very often doesn’t have a lot of. Skipping the implementation of DTA environments saves time and money for the project. However this choice will seriously impact administrators in their ability to test changes before implementation (patches/service packs, new solutions or configuration changes).
• Redundancy in the environment - Implementing redundancy requires extra hardware and therefore extra costs for hardware, software licenses and installation. If during implementation the requirements do not include redundancy, this will not be designed and implemented. Unfortunately, adding redundancy later will require a lot of work and has a high impact, especially for SQL Server where adding redundancy (clustering) will mean a complete reinstallation of the entire SQL environment.
• Sizing - I have been part of a project where the available storage was limited. We only had a certain amount of storage available, that was it. The disk space for all servers was just enough to contain all data. Once the environment was transferred to the administrators, one of the first things they had to do was adding extra disk space and moving database across these extra disks.

The message of this story: Most items mentioned might sound obvious, but unfortunately I have seen a lot of situations where this turned out to be harder than you would think/like.

If you are part of the implementation project, make sure that you involve the future administrators as soon as possible. If you are the future administrator, make sure you get involved as soon as possible. Each design decision has to be approved by both parties! So check and communicate between both parties early and continuously.

A very good input for IT requirements are acceptation criteria. This is a list of criteria to which the project has to comply to. That way you can save yourself a huge amount of time, effort and stress! So make sure you create these!

*More on DTAP will follow in a later post

The “Soft” part of SharePoint - Part 1: Skills

With products like Exchange, only IT Pros are involved, the people who are implementing or administering the environment are often the same people that are managing the Windows operating system. At least they have sufficient (infrastructure) knowledge to cover all bases.

With SharePoint, this is a whole different ball-game. As SharePoint is build on top of a lot of Microsoft infrastructure products, good infrastructure knowledge is very important (Windows, DNS, Active Directory, ISA Server/TMG, etc). Unfortunately more than often, SharePoint knowledge is very limited with people who are cracks in those technologies. SharePoint developers however have very limited infrastructure knowledge, so they have a hard time doing a very good job from a performance/security perspective. In other words implementing or managing SharePoint properly can be quite a challenge.

Then what do you need?
When you are implementing or maintaining SharePoint in a proper way, you will need various kinds of skills. Most of the times these skills are not found in one person and therefore a team needs to be created.

The required skills are:

1. Server - Hardware and operating system
   This is the most common discipline and the knowledge that is the most commonly available. Just like with other products, basic server management is required. This includes server patching, monitoring, backup/restore, antivirus management, troubleshooting, etc.


2. Database - SQL server
   SharePoint is build on SQL databases. Without proper administration, these databases potentially can cause issues. However SharePoint databases cannot be compared to regular SQL database. They are very sensitive on actions you can and cannot do with them. Microsoft has released guidance (whitepaper and KB articles) for database administrators on how to manage SharePoint SQL databases.


3. Technical SharePoint
   SharePoint Technical Application Management (TAM) is implementation and management of the infrastructure side of SharePoint. The full SharePoint configuration is the responsibility of TAM, therefore they are the only ones that are allowed to change that configuration. All TAM activities are done by logging onto the server or via the Central Administration. Most common TAM activities are: Application monitoring, troubleshooting application issues, application backup/restore, deploying SharePoint solutions, configuring SharePoint, creating web applications, SharePoint antivirus, etc.


4. Functional SharePoint
   SharePoint Functional Application Management (FAM) is implementation and management of SharePoint from an end-user perspective. Determine which functionalities the user needs, how the logical structure is going to look like and how certain SharePoint components need to be configured. Tasks of FAM would be: Assisting end-users, setting up sites, site structure, content types, determining the desired configuration of SharePoint components like search/user profiles, etc. In an ideal world do these activities not require any access to the SharePoint servers directly. Everything can be done via a web browser.


5. SharePoint Custom Development
   SharePoint is a very extendible platform. If the default out-of-the-box functionalities are not sufficient for end-users, extra functionality can be custom developed. All customizations are build using the .NET Framework, however pure .NET developers are not automatically SharePoint developers. SharePoint has its own set of development rules and best practices. SharePoint developers use Visual Studio to develop their custom code and package that in a SharePoint solution package. This package is then delivered to TAM, who are going to deploy this to the environment.

The technical and functional application management skills can be a grey area, as they can have some overlap. An example is the creation of Managed Properties:

• Specifying managed properties is done by FAM, creating and configuring the managed properties is done by TAM.
• Search management is the full responsibility of FAM

Each of the above skills can be implemented as a dedicated person(s), but also a combination of an SharePoint Technical guy (Skills: TAM/FAM) and a SharePoint Developer (Skills: FAM/Development). This totally depends on your organization. In a large organization, a split might be a good option to have a clear separation of duties.

The team that implements or administers SharePoint must have at least the first four skills available. The skill “SharePoint Development” is very useful (creating visual design, creating tools, etc), but not required.

The “Soft” part of SharePoint - Introduction

Over the past years I have done many SharePoint (2003 and 2007) implementations or supported with existing implementations. With each of these environments I have ran into the same types of issues. I have decided to capture all of this knowledge into some blog posts. In the next few weeks, I will post these here, starting with “Skills”. So keep posted!!

• Part 1: Skills
• Part 2: Impact of design choices
• Part 3: The solution intake process
• Part 4: DTAP

[MOSS2007/WSSv3] User profiles - Updates and deletions

The how, what and why with user profiles
SharePoint is made up of two separate products, which are very tightly integrated: Windows SharePoint Services v3 (WSSv3) and Microsoft Office SharePoint Server 2007 (MOSS2007). Because WSSv3 also can be implemented by itself, Microsoft has implemented WSS user profiles in order to keep track of user information in a WSSv3 environment. Each site collection has its own User Information List with profile information for each user that has ever logged onto the site collection.

When implementing MOSS2007, extra user profile functionality is implemented on top of WSSv3, but the WSSv3 profiles still exist in the site collections. To keep all user profile information synchronized across both the MOSS profiles and the WSS profiles, Microsoft has implemented a User Profile Synchronization mechanism which synchronizes all MOSS profile information to all WSS user profiles. Because MOSS profiles can be synchronized with AD, changes in AD are synchronized to all user profiles using this mechanism.

More information on user profiles can be found here (in Dutch).

How are deletions of AD accounts handled
When a full import from AD is done and a user which has a MOSS profile is not present in the AD import, this account is marked “Missing from Import”. When the user is missing from import during three imports, SharePoint considers this account as deleted and deletes the account from the MOSS profiles. Unfortunately these deletions are NOT replicated to the WSS profiles, so the WSS profiles of those users remain in the system.

Ok, what does that mean
I found out that when you are trying to grant a user permissions somewhere in a site collection, the SharePoint People Picker both checks the User Information List and Active Directory. If a user has been deleted from AD, but still exists in the User Information List (still has a WSS profile), this user is returned in the results. This means that you can still see users that have left the company ages ago, which is very confusing for site collection administrators:

Actual real life situation:
E.g. John Doe has left the company, I can’t see him anymore in the Outlook GAL, but I can still grant him permissions in my site collection.

Crap, how do I solve this
This can be solved by cleaning your WSS profiles regularly. Unfortunately by default this is a manual process. If you only have a few site collections and a few users, this is quite easy. But if you have many site collections and many users, this is a major challenge!

In my environment we created a custom tool for this issue:

• The tool checks the MOSS profiles against the WSS profiles and creates a delta file, “which WSS profiles do not have a MOSS profile”
• The output we compare (automated) with AD. This because we do not import all accounts like admin accounts
• The ouput is then feeded into another tool, which deletes these accounts

And are there any things I need to pay attention to
Of course! Nothing comes without a price! Because the WSS profile of users are deleted all items or documents that have been by those users become “orphaned”. SharePoint is not able to display who created or changed the item or document. So be careful!

Another issue we encountered is with search. If permissions have been granted to individual users and these users are deleted, the crawl has to change these permissions in the index file. If your index is small and have a limited amount of content, reset the index and start a new few crawl is the quickest way. In our case we ran an incremental crawl, which took three times the time a full crawl usually needs. During the crawl it looked like the crawl process was stuck. This because it was updating the index file, which just takes a very long time!

3rd edition of the Free DIWUG SharePoint magazine

The Dutch Information Worker User Group has released their 3rd edition of the Free DIWUG SharePoint Magazine. More information can be found at: http://www.diwug.nl/Pages/downloads.aspx

Available for download at:
http://www.diwug.nl/Downloads/DIWUG_SharePoint_eMagazine3.pdf

[MOSS2007] Cross domain Manager property

In an Active Directory, it is possible to configure a manager for a specific user. SharePoint is able to use this information and show you a hierarchy on a users MySite. Unfortunately this functionality has a limitation caused by Active Directory.

Active Directory issue
Within Active Directory It is only possible to select users (or contacts) that are in the same forest or domain as manager. So if you have (like at a customer of mine) a multi forest, multi domain environment, with Forest Trusts between all forests, you cannot select a user from domain 1 in forest 1 as the manager of a user in domain 2 in forest 2.

Customer situation
At the customer, they are using two tools to manage the data in AD:
1.) A replication engine to replicate users in one domain as a contact in the other domain. These contact are used for Exchange in that domain.
2.) A self service portal where users can configure their own data, including manager. When a user changes his/her manager, this is changed in the domain that has been marked as master domain. If the manager does not exist in that domain as a user, the contact in that domain is selected.

SharePoint behavior
This situation causes SharePoint to "generate" two different kind of hierarchy trees. One for domain1 and the other for domain2. Because contacts are not treated as users in SharePoint (as it shouldn't), when browsing through the hierarchy you never end up with the real manager account.

Then how to solve this issue
In order to work around the issue, we have modified the replication engine in such a way that the replication engine performs the following steps:

1. Check the manager property and retrieve the master account in the other domain
2. Configure the master account name in a separate AD property, which is not used
3. Configure SharePoint to import that AD property as the manager, with the format <domain>\<userid> (configurable in property mapping settings)

This mechanism now works like a charm: Managers are imported correctly and more important, the hierarchy tree is displayed correctly!!