[SP2010/SPF] Required permissions to run command line tools or other tooling that uses the OM

[ISSUE]
With SharePoint in many cases the SharePoint administrators also are the Windows of the SQL server and/or SQL administrator. This means that those persons also having loads of permissions in SQL Server, which means that everything will work without issues. They can do anything they want, using any tool they want.

But what about a situation where you don't have permissions on SQL. You will see actions via the Central Administration site work fine, however certain things like console applications (custom like SharePoint Manage or default applications like stsadm), scripts via PowerShell and other tooling (basically everything that uses the object model) won't work.

This is caused by the fact that SharePoint requires the account to have certain permissions on the databases in order for it to work. When using the Central Administration, this is done via the application pool account which has sufficient permissions. When using the object model outside of the Central Administration, this is done under the account the command is executed with.

[SOLUTION]
Then which permissions are required? Here is where it becomes a little tricky!

PowerShell scripts:

• When you are running SharePoint 2010 and you are trying to use PowerShell, Microsoft has created a SharePoint internal group called ShellAdmins. By adding your account to this group (Add-SPShellAdmins) your account has sufficient permissions to run PowerShell scripts. Other activities via the object model still don't work.

Other tools:

• To enable all object model activities, grant the account the following permissions:
• SharePoint (Depending on the activities you are going to do)
• Farm Administrator
• Permissions in the site collection(s), for example site collection administrator or contributor to a list
• Database:
• Configuration database: WSS_Content_Application_Pools
• Content database:
• Db_datareader
• Db_datawriter
• GRANT EXECUTE
• NOTE: An alternative would be db_owner permissions on the content database, however from SQL administration perspective this might be unwanted/against policy

NOTE: I am assuming that the account already has local administrator permissions on the SharePoint server, how else will you be able to run tools or scripts on the SharePoint server :-)

More info: SharePoint 2010 PowerShell Permissions Explainedhttp://sharepoint.microsoft.com/Blogs/zach/Lists/Posts/Post.aspx?ID=56

[SP ALL] Opening a web service is returning a 401.1 "Access Denied" error

[ISSUE]
Yesterday I was asked to assist in troubleshooting an issue with a SharePoint web service. The SharePoint indexing process failed to work properly for just one web application. Some investigation revealed that the indexer was unable to open the sitedata.asmx web service. When trying to open the same web service via IE, I was prompted for credentials however whatever credentials were entered, after three attempts an "Access Denied" page (401.1 error) was shown.

[SOLUTION]
Unfortunately Process Monitor didn't reveal anything and I noticed that the sitedata web service wasn't the only web service that failed. After some troubleshooting I found out that the cause was in the web config:

The "remove verb *.asmx" line was placed after the "add verb *.asmx" line in the httphandler setting, essentially removing the configuration after adding it. For example:

<add verb="*" path="*.asmx" validate="false" type="System.Web.Script.Services.ScriptHandlerFactory, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
....
<remove verb="*" path="*.asmx" />

After correcting this by placing the remove line in front of the add line, all web services started working just fine!

<remove verb="*" path="*.asmx" />
....
<add verb="*" path="*.asmx" validate="false" type="System.Web.Script.Services.ScriptHandlerFactory, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />

[EXPLANATION]
Why is this happening? By first placing the remove line, you make sure any declarations that are done in other (global) configuration files are made void. That way you know for sure that no conflicts will occur between two configurations. However after removing the asmx httphandler, you have to declare it again, else SharePoint (or better yet IIS) does not know how to handle the asmx file. The confusing part for this issue is that it will display an authentication prompt to the user, without it actually being an authentication issue.

[SP2010/SPF] SharePoint 2010 Build numbers

Here a list of build numbers for SharePoint 2010. Based on the build number you can determine which patchlevel your SharePoint environment is on:

14.0.6114.5000 - December '11 CU
14.0.6112.5000 - October '11 CU
14.0.6109.5002 - August '11 CU
14.0.6106.5000 - June '11 CU
14.0.6029.1000 - Service Pack 1
14.0.5138.5001 - April '11 CU
14.0.5136.5002 - February '11 CU
14.0.5130.5002 - December '10 CU
14.0.5128.5003 - October '10 CU
14.0.5128.5000 - October '10 CU (Incorrect package)
14.0.5123.5000 - August '10 CU
14.0.5114.5003 - June '10 CU
14.0.4763.1000 - RTM

[SP2010] Upgrade to SP2010 issue with missing content types

[ISSUE]
During a database upgrade from SP2007 to SP2010 I encountered an issue, which blocked a succesful upgrade. Several features could not be upgraded because of an issue with content types.

[ERROR]
[powershell] [SPSiteWssSequence2] [ERROR] [7/15/2011 1:14:56 PM]: Feature upgrade incomplete for Feature 'PublishingSite' (Id: 'f6924d36-2fa8-4f0b-b16d-06b7250180fa') in Site 'http://www.domain.com/sites/sitecollectionname'. Exception: The parent content type specified by content type identifier 0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF39 does not exist.

[CAUSE]
This issue is caused by the fact that:
- The publishing features (“SharePoint Server Publishing Infrastructure” and the features it is depending on) have been enabled and disabled in the past
- Content types have been deleted the incorrect way: Deleted while other components were still using them.

[SOLUTION]
The mentioned content type ID is the ID of the content type “Page”, which does not exist anymore in the Site Content Types. To recreate these, run the following stsadm commands for each site collection that is experiencing the issue (retrieve the list from the upgrade log file):

stsadm -o activatefeature -name PublishingSite -url <url> -force
stsadm -o activatefeature -name PublishingResources -url <url> -force

[SP2010] Upgrade SharePoint 2007 content to SharePoint 2010 via the Database Attach method

Last week I performed an upgrade of SharePoint 2007 to SharePoint 2010 using the Database Attach method. Unfortunately the database upgrade "Completed with errors". However, the site collections were available in SP2010 without errors. Also changing the visual style to SP2010 worked just fine. Detaching the database and reattaching did not give me any error, but it didn't restart or continue the upgrade process. Then how to fix this:

Troubleshoot upgrade errors
Each upgrade process produces an upgrade log file, which displays each error and warning that is found during the upgrade. To successfully complete the upgrade process you will have to fix each error and restart the upgrade process to upgrade the remaining, not yet upgraded, site collections.

Restart/resume the database upgrade
Once you have fixed all upgrade issues, you have to restart the upgrade process. This can be done using the following PowerShell script:
Source: http://technet.microsoft.com/en-us/library/ff382638.aspx

$guid = Get-SPContentDatabase -Identity <dbname>
upgrade-spcontentdatabase -id $guid

However if the database still contains issues, the upgrade will fail again. Review the generated log file to check what the errors are and retrieve more info for troubleshooting.

Verify upgrade status
To verify if all components in the environment are upgraded successfully, run the following command:

stsadm -o localupgradestatus

This command generates a report that contains a summary at the bottom. The part marked bold is the important part and specifies that in all databases there are 16 site collections not upgraded yet, this number should be zero.

[9] content database(s) encountered.
[0] content database(s) still need upgrade or cannot be upgraded.
[43] site collection(s) are contained in the content databases.
[16] site collection(s) still need upgrade.
[82] other objects encountered, [0] of them still need upgrade or cannot be upgraded.

If you send the output of the command to file and search for the text "Needs Upgrade", you will find the site collections that aren’t upgraded yet:

  <object>
    <name>https://www.domain.com/sites/sitecollectionname</name>
    <type>Microsoft.SharePoint.SPSite</type>
    <level>6</level>
    <status>Needs Upgrade</status>
  </object>

Two very good SharePoint articles

Yesterday I ran into two very good SharePoint articles, that describe an often forgotten part of a SharePoint implementation: Governance!

• We've installed SharePoint, we have a successful platform right? Wrong!
• SharePoint is a car

[SP2007/SP2010] Migrate SharePoint across domains

A while ago I worked on a project where we had to migrate a customer’s SharePoint 2007 environment from another service provider to a newly created environment in our own datacenter. The challenge we had during this project was that the new environment was built from scratch, meaning that the Active Directory would be a different one than the original environment was located in. Unfortunately there were no possibilities to create a trust between the two domains.

The above would mean that since the Active Directory changed, , the domain name would change as well as all user accounts (or SIDs). This meant that all security permissions, alerts and ownerships would become unusable. These had to be migrated to the new accounts in the new AD.

For migrating users, SharePoint offers a stsadm operation called “migrateuser”. However, at the time of the project there was no operation for groups migration, so we needed a solution for that as well.

[PROJECT INFO]

• The web application URL's would not change
• The user account format would not change in the new Active Directory. User1 in the old AD, would be User1 in the new AD.
• MIIS was used to create the users in the source environment. ILM2007 would be used in the new environment. Any custom code used in MIIS could be migrated to the ILM2007 environment, however some changes and updates would be made in the process.
• The old environment was based on 32 bit SharePoint 2007 on Windows Server 2003. The new environment would be based on 64 bit SharePoint 2007 on Windows Server 2008.

[ISSUES]

• The source SharePoint environment contained a SSP. Unfortunately there is no way to copy the SSP or its settings to the new environment automatically. The SSP had to be recreated manually.
• The user profiles in the SSP had to be migrated as well. There was no tool available that was able to export the user profiles and import in our new environment. We had to create a tool for this. On Codeplex we found a Profile Import tool (MOSS Profile Importer), but that was unable to export the information from an existing SharePoint farm. We used this code as a starting point for our own tool.
• The migrategroup command did not exist yet, fortunately only seven different AD groups were used. These needed to be migrated manually.
• The stsadm operation migrateuser has to be run for each user id. A custom solution is required to generate a script for all users. Running this script consumes much time and needs to be shortened as much as possible.

[MIGRATION]
The preparation steps we took were:

• Create the custom tooling require to perform the migration (profile export/import, migrateuser script)
  
• Perform a test migration in order to validate the migration steps and target environments.
  

The steps we took to migrate the content were:
Environment setup

• Setup the new SharePoint 2007 environment and use same patch level as the original farm
• Install all custom solutions on the target environment
• Create all users in the new Active Directory
• Setup the SSP in the target environment and configure it according to the settings of the old environment (user profile properties, profile import, audiences, search, etc)

User profile information

• Import all users from AD into the SSP
• Backup all user profile information to file
• The import tool is using the user id to import the data to the correct profile, so we had to replace the old domain name with the new domain name in the export file
• Restore all user profile information into the new SSP

Content

• Create SQL backup of the source content databases (web applications and MySites) to a USB disk
• Ship the disk to the other datacenter and connect it to the server
• Restore the SQL backups on the target SQL server from USB disk
• Connect the content databases to the correct web applications
• Test the site collections for correct operation of the databases
• Run the migrateuser script generation tool. This tool created three script files, which we could run on three different servers to speed up the migration process.
• Run the migration scripts
• Manually change group membership for each used group (add new group, grant permissions and remove old group) in the entire site structure

Testing

• Test, test, test

[POST INFO]

• Since the August 2009 Cumulative Update, SharePoint 2007 stsadm includes the migrategroup operation, which is able to migrate groups the same way migrateuser does for users.

The “Soft” part of SharePoint - Part 6, Challenge the Business

This is a tricky one! I know the saying "the customer is always right" or "the customer is king". In IT the business is your customer, IT by itself is useless. IT is there to support the business to work as efficient as possible, but does this mean that the customer is indeed always right.......

I have seen many cases where IT puts the business before everything. They do pretty much everything to do what the business asks them to do, often without questions. Very often this resulted in projects turning into disasters:

• The business asks for more that they need
• They radically change their requirements when you just started the implementation

And of course, afterwards the business starts complaining that it took too long and the costs were too high.

Then what is causing these kind of issues. Two reasons behind these drama’s are:

1. The business doesn’t know what exactly they want.
• Result 1: They ask something else every day
• Result 2: They ask what they think they want
2. The business is asking for IT solutions instead of specifying the business need
• Result: They ask for a technical solution which is not fitting their business need

That is why IT should challenge everything the business is asking for: Do they have a clear picture what they want? Do they really want what they ask? Don't they mean something else and therefore ask the wrong question? Isn’t there an easier way to meet their business need?

This is not to question everything they do, but to get to know the true question behind the question. When all requirements are clear on all ends, projects can be executed easier and cheaper.

[SP2010/SPF] SharePoint 2010 Build numbers

Here a list of build numbers for SharePoint 2010. Based on the build number you can determine which patchlevel your SharePoint environment is on:

14.0.5130.5002 - December 10 CU
14.0.5128.5003 - October 10 CU
14.0.5128.5000 - October 10 CU (Incorrect package)
14.0.5123.5000 - August 10 CU
14.0.5114.5003 - June 10 CU
14.0.4763.1000 - RTM