Monday, December 01, 2008

[MOSS2007] Error while opening Site Columns page (/_layouts/mngfields.aspx)

[ISSUE]
When you open the Site Columns page (/_layouts/mngfields.aspx, Site Settings > Site Columns) SharePoint throws an "Unknown Error" at you.

When you change the CustomErrors=Off and CallStack=true values in the web config, SharePoint shows you the following error:
Object reference not set to an instance of an object.
at Microsoft.SharePoint.ApplicationPages.FieldListRenderer.Render(HtmlTextWriter output)
at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)
at System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter)
at System.Web.UI.Control.RenderControl(HtmlTextWriter writer)
at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)
at System.Web.UI.Control.RenderChildren(HtmlTextWriter writer)

[CAUSE]
This issue is caused by the fact that a feature creates site columns using the incorrect casing in the url. For example:
http://<site url>/sites/Test instead of http://<site url>/sites/test
http://<site url>/Sites/test instead of http://<site url>/sites/test
source

[SOLUTION]
To determine which site columns are causing this issue:
  • Open a sub site and open Site Columns of that site
  • To the left of each column, the location where they have been created is shown
  • Hover over links pointing to the parent site and check URL to which the link is pointing
  • One or more URLs will have incorrect casing.

To solve this issue, you either have to change your feature so that it uses the internal URL which SharePoint is using. In my case, the feature that caused this issue was the "Office SharePoint Server 2007 Enterprise Site Collection Feature". In other words, some code I could not change.

I found an article on the Internet where they mentioned changing the site url in the database. I know database changes are not supported by Microsoft, but there was no other way to change this.

  • Open the Content Database
  • Open the table ContentTypes
  • Check the column Scopes, you will find some URLs with the correct casing and some with incorrect casing. Change the casing of the incorrect records.

Wednesday, November 26, 2008

[MOSS2007] Profile synchronization does not work for databases which are off-line/stopped

Last week I got some complaints about user profile information not being synchronized to other sites in a SharePoint 2007 environment. After some investigation I found the following error in the ULS log:
<date> <time> OWSTIMER.EXE (0x0864) 0x08A4 SharePoint Portal Server User Profiles 8zzy Medium Aborting sweepsynch for guid instance <GUID> due to null or non-online content database

[Solution]
After configuring the database as Ready again, the profile information synchronization resumed again.

[Background information]
If you put you content database off-line, SharePoint is still able to access it but it will not create new site collections. Because were creating new site collections, which we wanted created in a specific database we put all databases (except for that specific one) on off-line. As it turned out, the downside of this is that SharePoint is unable to synchronized the user profile information to databases that are off-line.

Monday, November 24, 2008

[MOSS2007/WSSv3] Messages about missing resource files in SharePoint ULS logs

[ISSUE]
The following messages appear a lot in the ULS logs of SharePoint:
11/24/2008 10:06:58.66 w3wp.exe (0x07F4) 0x1550 Windows SharePoint Services General 72kg High #20015: Cannot open "Proj.en-US.resx": no such file or folder.
11/24/2008 10:06:58.66 w3wp.exe (0x07F4) 0x1550 Windows SharePoint Services General 72kg High (#2: Cannot open "Proj.en-US.resx": no such file or folder.

[SOLUTION]
This is caused by the fact that the resource file is not present with a specific English tag in its name. To solve this issue do the following
  • Open the directory C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\Resources
  • The mentioned files are not in there, but the proj.resx is
  • Open a command prompt and browse to the same directory
  • Run the following commands:
    copy avm.resx avm.en-US.resx
    copy compproc.resx compproc.en-US.resx
    copy ep.resx ep.en-US.resx
    copy er.resx er.en-US.resx
    copy hlpdsk.resx hlpdsk.en-US.resx
    copy itteam.resx itteam.en-US.resx
    copy proj.resx proj.en-US.resx
    copy pws.resx pws.en-US.resx
    copy rer.resx rer.en-US.resx
    copy tsa.resx tsa.en-US.resx
  • This will copy the default resource files to the English version resource files and the messages will disappear.

Monday, November 17, 2008

[MOSS2007/WSSv3] Kerberos, CNAMEs and SharePoint

When trying to configure Kerberos on the environment, I ran into the following issue:

[ISSUE]
When opening the SharePoint site from another machine, Internet Explorer logs on with NTLM anyway. When opening the site from the server, a logon prompt appears which won't accept any credentials. After three times, an Unauthorized message is shown.

[TROUBLESHOOTING]
I had all Service Principal Names configured as described in Configure Kerberos authentication (Office SharePoint Server). To get more information I enabled Kerberos logging (How to enable Kerberos event logging), which resulted in a lot of events in the System log:


Source: Kerberos
Type: Error
Event: 3
Message: "A Kerberos Error Message was received:
on logon session <user>
Client Time:
Server Time: 12:5:19.0000 10/30/2008 Z
Error Code: 0x34 KRB_ERR_RESPONSE_TOO_BIG
Extended Error:
Client Realm:
Client Name:
Server Realm: <domain>
Server Name: krbtgt/<domain>
Target Name: krbtgt/<domain>@<domain>
Error Text:
File: e
Line: 6c0
Error Data is in record data.



[SOLUTION]
After using Network Monitor v3.2, I found that the Kerberos traffic from the server was requesting a different Service Principal Name than the url of the SharePoint site. The error which was also listed in the eventlog was returned by AD. The SPN that was send, contained the CNAME value instead of the original URL. Using Google, I found the following article:
The Problem with Kerberos Delegation (Section: "Confused Yet?")

This article states the following:
  • Computer running IIS: ws.identitychaos.com (WS)
  • Service Account hosting the Central Admin app pool: idchaos\svc.wss.farm
  • Host Header for Central Admin: spca.identitychaos.com
  • SPN's registered to idchaos\svc.wss.farm
    • HTTP/spca
    • HTTP/spca.identitychaos.com

Using a CNAME, when IE requests the SPN, spca.identitychaos.com resolves to ws.identitychaos.com (the alias resolves to the host) so IE asks AD, "tell me which account has an SPN registration of HTTP/ws.identitychaos.com" and it gets the WS computer account, not the service account. How do you fix it? You use A records instead of CNAME's.


After changing our CNAME records to A records, all Kerberos issues were resolved.

In other words:
Do not use CNAMEs when using Kerberos authentication!!!

Wednesday, November 12, 2008

[MOSS2007] Search issues due to corrupt SSL certificate

After installing MOSS2007 which has Service Pack 1, the Infrastructure Update and the August Cumulative Update slipstreamed I ran into the following issue:

My environment consists of one Index server and two Search servers. When opening the new Search Administration page, it is supposed to list the status of all of the index and search servers. Unfortunately the Search servers were listed as Error. After doing some more investigation I found the following error appear in the application log of both search servers each minute:

Event: 6482
Type: Error
Source: Office SharePoint Server
Category:Office Server Shared Services

Application Server Administration job failed for service instance Microsoft.Office.Server.Search.Administration.SearchServiceInstance (aa8a83f6-ff42-4be3-ae6c-c1cb2205cc8f).

Reason: The underlying connection was closed: An unexpected error occurred on a send.

Techinal Support Details:
System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. ---> System.IO.IOException: Authentication failed because the remote party has closed the transport stream.
at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Net.TlsStream.ProcessAuthentication(LazyAsyncResult result)
at System.Net.TlsStream.Write(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.PooledStream.Write(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.ConnectStream.WriteHeaders(Boolean async)
--- End of inner exception stack trace ---
at Microsoft.Office.Server.Search.Administration.SearchApi.RunOnServer[T](CodeToRun`1 remoteCode, CodeToRun`1 localCode, Boolean useCurrentSecurityContext, Int32 versionIn)
at Microsoft.Office.Server.Search.Administration.SearchApi..ctor(WellKnownSearchCatalogs catalog, SearchSharedApplication application)
at Microsoft.Office.Server.Search.Administration.SearchServiceInstance.Synchronize()
at Microsoft.Office.Server.Administration.ApplicationServerJob.ProvisionLocalSharedServiceInstances(Boolean isAdministrationServiceJob)

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.


When I tried to open the SearchAdmin.asmx page (http://<index or search server name>/SearchAdmin.asmx), I get a page cannot be displayed.

This issue is caused by a corrupt SSL certificate on the "Office Web Services" web site. To solve this issue, you have to create a new certificate. You can use the SelfSSL tool, included in the IIS resource kit.

Run the following command on all search and index servers:
selfssl.exe /N:CN=<server> /K:1024 /V:9999 /S:951338967 /P:56738

Thursday, November 06, 2008

[MOSS2007] Crawl issue after installation .NET Framework 3.5 SP1

[ISSUE]
When the .NET Framework 3.5 SP1 has been installed, it is possible that you get some Access Denied error messages when crawling your MOSS sites. If you're indexer is set to crawl itself, via the Central Admin setting, or manual hosts modification, you might be running into this issue. You will see a combination of errors in your event logs, ULS logs and crawl logs that match the following:

Crawl Log:Event Type: Warning
Event Source: Office Server Search
Event Category: Gatherer
Event ID: 2436
Date: 10/22/2008
Time: 10:02:55 AM
User: N/A
Computer: MOSS2007
Description:The start address <http://portal/> cannot be crawled.

Context: Application 'SharedServices1', Catalog 'Portal_Content'
Details:Access is denied. Check that the Default Content Access Account has access to this content, or add a crawl rule to crawl this content. (0x80041205)
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

ULS Logs: (verbose)10/22/2008 10:59:04.23 mssdmn.exe (0x1420) 0x151C Search Server Common MS Search Indexing 7hoo Verbose GetVirtualServerPolicy is about to call WS for virtual server, strVirtualServerUrl http://portal/10/22/2008 10:59:04.23 mssdmn.exe (0x1420) 0x151C Search Server Common PHSts 0 Monitorable ***** Couldn't retrieve server http://portal/ policy, hr = 80041205 - File:d:\office\source\search\search\gather\protocols\sts3\sts3util.cxx Line:548
10/22/2008 10:59:04.23 mssdmn.exe (0x1420) 0x151C Search Server Common PHSts 0 Monitorable ****** Server portal security initialization failed, hr = 80041205 - File:d:\office\source\search\search\gather\protocols\sts3\sts3util.cxx Line:317
10/22/2008 10:59:04.23 mssdmn.exe (0x1420) 0x151C Search Server Common PHSts 0 Monitorable CSTS3Accessor::InitURLType: Return error to caller, hr=80041205 - File:d:\office\source\search\search\gather\protocols\sts3\sts3acc.cxx Line:1542
10/22/2008 10:59:04.23 mssdmn.exe (0x1420) 0x151C Search Server Common PHSts 0 Monitorable CSTS3Accessor::InitURLType fails, Url http://portal/, hr=80041205 - File:d:\office\source\search\search\gather\protocols\sts3\sts3acc.cxx Line:184
10/22/2008 10:59:04.23 mssdmn.exe (0x1420) 0x151C Search Server Common PHSts 0 Monitorable CSTS3Accessor::Init fails, Url http://portal/, hr=80041205 - File:d:\office\source\search\search\gather\protocols\sts3\sts3handler.cxx Line:243
10/22/2008 10:59:04.23 mssdmn.exe (0x1420) 0x151C Search Server Common PHSts 0 Monitorable CSTS3Handler::CreateAccessorExB: Return error to caller, hr=80041205 - File:d:\office\source\search\search\gather\protocols\sts3\sts3handler.cxx Line:261

Security Logs:Logon Failure:
Reason: An error occurred during logon
User Name: mosscrawl
Domain: SPI
Logon Type: 3
Logon Process: Èù8
Authentication Package: NTLM
Workstation Name: MOSS2007
Status code: 0xC000006D
Substatus code: 0x0
Caller User Name: -
Caller Domain: -
Caller Logon ID: -
Caller Process ID: -
Transited Services: -
Source Network Address: XX.XX.XX.XX
Source Port: 1417

[SOLUTION]
Perform the steps mentioned in the KB article below to fix this issue:
http://support.microsoft.com/kb/896861
Source: Ron Grzywacz's Blog

Wednesday, November 05, 2008

[MOSS2007/WSSv3] Login box when a normal user logs on, but not with admin

[ISSUE]
A logon box constantly appears when a users is trying to log onto a SharePoint site. Whatever the user enters, he cannot get onto SharePoint.

[CAUSE]
The environment on which we experienced this behaviour was locked down according to security guidelines. One of these guidelines was to strip the "Read & Execute" permissions on the C:\Windows folder from the local users group. The .NET Framework requires Execute permissions on the C:\Windows directory to operate properly and because SharePoint relies on .NET, users were prompted for credentials.

[SOLUTION]
Add the Execute permissions for the Users group on the C:\Windows folder or restore the permissions on the C:\Windows directory to default:
- Open a command prompt.
- On a domain controller, run
secedit /configure /db c:\windows\temp\seceditsv.sdb /cfg "c:\windows\security\templates\DC security.inf" /log c:\windows\temp\seceditsv.log
- On a non-domain controller, run
secedit /configure /db c:\windows\temp\seceditsv.sdb /cfg "c:\windows\security\templates\setup security.inf" /log c:\windows\temp\seceditsv.log

Friday, October 31, 2008

[MOSS2007/WSSv3] Using a clustered SQL named instance behind a firewall in SharePoint

[Issue]
For a customer I have been working on a new installation of SharePoint 2007. The environment has high availability requirements, so all components need to be redundant. The customer environment already contained a clustered SQL environment which we could use for SharePoint. We just needed to install a new SQL named instance, dedicated for the SharePoint 2007 installation.

For extra security, the SQL cluster was placed in a separate database VLAN, which was secured by a firewall. To enable communication between the SharePoint 2007 servers and the SQL named instance we opened UDP port 1434 and TCP port 4000 (port used by the instance).

After installing SharePoint 2007 on one server, I ran the SharePoint Products and Technologies Configuration Wizard to create a new farm/Configuration Database. When I used the <server>\<instance> format, SharePoint reported that it could not connect to the instance. It reported that the instance was not running, the server wasn’t a SQL server or a firewall was blocking the traffic.

[Reason]
After troubleshooting on the SharePoint and SQL servers, I turned to the firewall guys. With their troubleshooting tools we found out that:


  • SharePoint connected to virtual SQL cluster name (ip 10.10.10.3) using the UDP protocol over port 1434 to determine the port of the SQL instance (see “Background Information”)

  • The firewall allowed the traffic to the virtual SQL cluster name

  • The server replied on the physical node ip address (ip 10.10.10.1) back to the server

  • Because the server registered an outbound connection on the virtual SQL cluster name, it does not allow a reply on a different ip address and blocked the reply

Due to this behavior the SharePoint Config Wizard could not determine the port used by the instance and therefore could not connect to the instance.

After using Google we ran into the following article:
Unable to connect to a SQL Server named instance on a cluster


This issue is caused by the fact that the SQL Browser is not “cluster-aware”. It listens to all ip addresses of the server and replies always on the ip addresses of the physical node instead of on the ip address on which it received the request. The solution according to the blog post: Use SQL Server 2008 in which it is solved…….no real option at the moment.

[Solution]
To solve this issue, we created a ODBC System DSN which is configured to connect to the correct port directly and preventing the discovery process from happening:



  • Start > Settings > Control Panel > Administrative Tools > Data Sources (ODBC)

  • Tab System DSN

  • Click Add

  • Select SQL Server

  • Enter the name you would like the DSN to have

  • Enter a description

  • Enter the name of the server

  • Click Next

  • Click “Client Configuration”
    Clear the “Dynamically determine port” checkbox and enter the port number of the SQL instance

  • Click Next

  • The connection is now checked. Click Finish to save the DSN

Now use the name of the DSN as the SQL server name.

[Background information]
When a named instance is used, in most cases that instance is running on a port which is chosen randomly at installation time (customizable afterwards). Any processes that would like to use that specific instance first connect to the SQL Browser (using UDP over port 1434), which supplies the port that the instance is running on. The process then connects to the specific port.

Wednesday, October 29, 2008

[MOSS2007] Error in search configuration after installing Infrastructure Update

The infrastructure update introduces a Search Administration page on the Shared Services. When creating a slipstream installation, this page is automatically created when creating the Shared Services. But when you install the Infrastructure Update on an existing environment, this page is not automatically created. It does change the breadcrumb trail on the Search pages of the Shared Services site, which cause some error messages. You cannot open the “Search Administration” page (http://<ssp_url>/ssp/admin/searchadministration.aspx).

To solve this, run the following commands:
stsadm.exe -o installfeature -name S2SearchAdmin
stsadm.exe -o activatefeature -name S2SearchAdmin -url http://<ssp_url>/ssp/admin -force

Monday, October 13, 2008

Usefull and Free SharePoint Tools (Part 2)

A while ago I posted a list of tools I ran into and found very usefull. Here is part 2: More usefull tools :-)

SharePoint Inspector
Using the SharePoint Inspector you can browse through the SharePoint object model, can see objects composing its structure, get their properties by reflection. This cool can be very useful when you want to check if your code does what it should do. You can use also some advanced features like activate/deactivate SharePoint features, add/remove event receivers, manage your recycle bin.

Link: SharePoint Inspector

SharePoint Manager
The SharePoint Manager 2007 is a SharePoint object model explorer. It enables you to browse every site on the local farm and view every property. It also enables you to change the properties (at your own risk). It is a very powerfull tool for developers that like to know what the SharePoint holds of secrets.

Link: SharePoint Manager

MS IT Site Delete Capture
Every had the unpleasant question of users if you could restore a site or site collection they accidentally deleted? The only way to do this by default is to perform a restore to a separate environment and use stsadm to extract the site and restore into your environment again. Even though SharePoint 2007 does have a recycle bin, this does not capture site or site collection deletions.

After you installed the Microsoft IT Site Delete Capture Feature 1.0 SharePoint administrators can intercept both site/web delete requests and archive the site/web to a resource local to the web front-end computer or UNC path before the site/web is removed from the configuration and content databases.

The Microsoft IT Site Delete Capture Feature 1.0 also exposes functionality allowing SharePoint administrators to send e-mail messages to the end-user indicating the site has been archived and deleted, any failure in the event receiver will generate an e-mail message to the end-user indicating that the site/web has not been deleted. The message format, text, and language are stored in a flexible, culture-independent extensible markup language configuration file.

Link: MS IT Site Delete Capture

SharePoint 2007 Test Data Population Tool
The SharePoint 2007 Test Data Population Tool is a capacity planning and performance testing tool that populates data for testing SharePoint deployments. The SharePoint 2007 Test Data Population Tool is available as a command-line executable program that extracts information about how to populate the server from an XML configuration file.

Use this tool to populate test data for SharePoint capacity planning. Also included are some sample test scripts that can be used for performance and load testing SharePoint and Excel Services with the test data. These tools are being provided as is and there is no ongoing support for these tools.

Link: SharePoint 2007 Test Data Population Tool

SharePoint Logging Spy
A lot of SharePoint 2007 environments consist of multiple servers. Each server is producing logs and in case of issues you always have to dig through multiple server logs. The SharePoint Logging Spy allows you real time diagnostics of multiple servers in a SharePoint farm using a single console view.

SharePoint Logging Spy is a real time diagnostic application for MOSS 2007 which allows a SharePoint administrator to view (and save to disk) the ULS Log & Event log entries from multiple machines in a SharePoint farm through a single console. NOTE - There is a known issue with Event Log Entries not being displayed from remote machines. SharePoint Logging Spy also allows the administrator to view the historic ULS logs on the farm using the same interface.
SharePoint Logging Spy also has a treeview which allows a developer or administrator to view the state of the current SharePoint enviroment. The Plug-in architechture allows for extensibility of the core application.

The application also provides drill downs for information regarding the thread or process which generated the log entries which are reported in the main screen. The thread and process information is collected on demand (when you click the menu) and is NOT historical therefore it is feasable that the process/thread may no longer exist when you ask for the information and you will get an empty result set.
To easy configuration there is a discovery wizard which will discover on-line servers in the farm and allow selective monitoring to begin.

Link: SharePoint Logging Spy

SharePoint SUSHI
SUSHI is a powerful, user-friendly SharePoint application enabling you to accomplish common SharePoint administrative and development tasks. You can think of SUSHI as a Swiss army knife for SharePoint.

SUSHI = SharePoint Utility with a Smart, Helpful Interface

SUSHI Tools
Administration
Security Reports: View all sites and lists a user has access to (VERY USEFULL!!!)
Profile Images Import: Upload Profile Images and Apply them to User Profiles.
Backup: Backup a site. Restore: Restore site from backup.
Email Test: Send a test email to make sure outbound email is set up correctly.

Lists
Copy View: Copy a view from one SharePoint list to another. Meta Data: Get detailed information about metadata in a list. Move metadata from one column to another for all items in a list. And more.
Bulk List: Creation Create a large number of subsites based on a site template.
Bulk Site Columns: Create Site Columns. Import Documents: Upload entire file structures from a file share into a document library.

Sites
Bulk Site Creation: Create a large number of subsites based on a site template.
Themes: Apply a Theme to many sites at once.

Link: SharePoint SUSHI

Wednesday, September 17, 2008

[MOSS2007/WSSv3] Technical Library content available as downloadable CHM file

Just recently Microsoft released the entire Technical Library for Windows SharePoint Services v3 and Office SharePoint Server 2007 in Compiled Help format (CHM) to be downloaded from their site:

Available here:
- Windows SharePoint Services 3.0 Technical Library in Compiled Help format
- Office SharePoint Server 2007 Technical Library in Compiled Help format

Monday, September 08, 2008

[MOSS2007] Error during editing navigation items

[ISSUE]
When trying to edit the navigation items in a site, I get the following error message:
"The page has been modified by another author on <time>"
.....even when I am 100% nobody is changing the navigation of this site.

The reason why I was editing the navigation was because some items were having some strange URL's:
/_layouts/viewlsts.aspx?BaseType=0?BaseType=0?BaseType=0?BaseType=0

[CAUSE]
After browsing on the Internet I found that this had been caused because I used the Content and Structure Manager to move a site. SharePoint has an issue, which is causing the behaviour above.

[SOLUTION]
To solve this issue, do the following:


  • Remove all of the Document Libraries and lists from the navigation by:
    • Open the list/library you want to remove from the quick launch
    • Click Settings > List Settings (or Library Settings)
    • Click on Title, description and navigation
    • Choose No for Display this list on the Quick Launch?
  • Remove all corrupted navigation items
  • Readd the Lists/Libraries

[MORE INFO]
Technet Forum
Windows Tech
Blog Paul Gavin


Thursday, August 28, 2008

[MOSS2007/WSSv3] SharePoint 2007 and Console Applications [UPDATED]

During a project, the developers created a console application which they wanted to schedule in Task Scheduler. This turned out to be a challenge because using console applications with SharePoint 2007 requires permissions on three locations. The service account will need to have the following permissions:

1.) Permissions on the SharePoint databases

  • Open SQL Management Studio
  • Go to Security > Logins
  • Right click Logins and select New Login
  • Enter account name and open User Mappings tab
  • Select the master database, permissions public
  • Select the config database (default SharePoint_Config), permissions public and WSS_Content_Application_Pools
  • Select the content database (default format WSS_Content_), permissions public and db_owner
2.) Permissions on the SharePoint server
  • Add the service account to the local administrator group (See update below)
3.) Permissions on the SharePoint site
  • Grant the service account the required permissions in the SharePoint site
  • For example: add to Site Owners group
[UPDATE] The service account doesn't need to be local admin on the server. You can use the following permissions as well:
  • "Logon as batch job" in Local Security Policy
  • Member of the local WSS_ADMIN_WPG group
  • Grant the local group BATCH read permissions to C:\Windows\System32\cmd.exe to make sure the scheduled task can run batch scripts.

Friday, August 01, 2008

[MOSS2007] Error "Cannot complete this action. Please try again."

Over the past month I have been working on a major incident on our production SharePoint 2007 environment.

ISSUE:

When opening a sub site in a site collection by entering it url without default.aspx file, SharePoint would show an HTTP 500 error (Internal Server error). When you try to open the default.aspx file, SharePoint would display the error "Cannot complete this action. Please try again.". The symptoms were similar to:
http://support.microsoft.com/default.aspx?scid=kb;EN-US;937038


CAUSE:
About a month ago a site administrator of a sub site noticed that the NT AUTHORITY\AUTHENTICATED USERS group was listed in the site members group. In other words, everybody who was authenticated in the domain or trusted domains had Contribute permissions. As you can imagine this was not supposed to happen, so he tried to delete the AUTHENTICATED USERS group from the members group. What the user exactly did and what happened is not really clear, but the the result was that the sub site was no longer reachable.

STEPS TAKEN:
After searching for a solution for about a day, we could not figure it out and called in the support of Microsoft. Unfortunately they also weren't very sure how to solve it, but did mentioned the possibility of installing SP1 which might solve the issue. On the other hand, the also feared that installing SP1 would fail because of the broken site. To be sure we prefered to test the installation of SP1 first. In the meantime, the users couldn't access their data and we had to fix that first.

The suggestion of Microsoft was to use stsadm -o export/import. We would restore a known good copy of the database to a restore environment, export the sub site tree, copy it to the production environment and import on top of the broken sub site. Unfortunately this did not work, we received the same error. The alternative was to restore to a new sub site. The import started fine, but after about seven hours stsadm crashed! Also a second import crashed......now what.

We had bought MetaLogix Migration Manager to migrate content from file shares, ASP/HTML websites and other SharePoint environments into our new central MOSS2007 environment. So we gave that tool a go. Fortunately that worked. After several hours, the data was available again for the users.

FIX THE PRODUCTION ENVIRONMENT:
I wanted to test the installation of SP1 on a copy of that site collection. I tried to use stsadm -o backup/restore to copy the production site collection and to a restore environment. The backup went fine, but restore failed after 8 to 10 hours :-(

I then tried to create a backup of the entire database, restore that to the restore database server and attach to an empty web application. Fortunately that worked and I had an environment to test SP1 on. The installation of SP1 completed successfully and we now know that we can install SP1 without any issues on the environment.

KEY INFO:
The entire site collection was about 48GB of size. The sub site tree was about 25GB. Moveing the data using the default stsadm tools, did not work. It looks like stsadm is having some issues with large sites.

According to Microsoft Product Support, SP1 contains stored procedures which check and fix security issues. They weren't sure if those stored procedures would fix our issue. Obviously it did :-)

Wednesday, June 04, 2008

[MOSS2007] Version 2.0 of my Installation Guide has just been released

I finally had the time to complete a new version of my MOSS2007 Installation Guide and have just uploaded it. This new version includes installing Service Pack 1 for the .NET 2.0 and 3.0 Frameworks and MOSS2007. I also added the SQL Server 2005 SP2 Cumulative Update Pack and an IIS fix, which solves an issue which caused the error message "Attempted to read or write protected memory" in the event log.

The new version can be downloaded here: HOWTO - Install MOSS2007 v2.0

And again: if you have any comments or additions, please let me know!

Have fun!

Tuesday, May 06, 2008

[MOSS2007] Hotfixes to clear up 95% off the MOSS2007 event logs errors

Last week a colleague of mine send me an e-mail about the patches and hotfixes that are currently available and fix about 95% of all event log error messages:

---------------------------------------------------------------------
Hi Folks,

As promised, here’s a list of Microsoft hotfixes and patches that cleared up 95% of the MOSS 2007 event log errors:-

It’s assumed that you’ve already installed WSS v3.0 Service Pack 1 and MOSS 2007 Service Pack 1

As always, read the release notes and any associated guidance before applying.

KB923028 - Attempted to read or write protected memory
http://support.microsoft.com/kb/923028
Also fixed in .NET Framework Service Pack 1: http://support.microsoft.com/kb/945757

KB933061 - WMI Hotfix
http://support.microsoft.com/kb/933061
Note: This one was geared more towards fixing issues with SQL Server Reporting Services (SSRS) 2005. Might want to install this if you’re using the SSRS 2005 SharePoint Add-In.

KB946608 - SQL Server Cumulative Update 6
http://support.microsoft.com/kb/946608
Note: Once again, geared towards the SSRS 2005 Add-In. This MUST be installed if you are using the SSRS 2005 Add-In for SharePoint

KB941274 - MOSS 2007 Post SP1 Hotfix
http://support.microsoft.com/kb/941274
Note: Several WSS v3.0/MOSS 2007 fixes included in this Hotfix package

KB946517 - IIS6 Multiple Thread Fix
http://support.microsoft.com/kb/946517

Happy Hotfixing!
---------------------------------------------------------------------

Thursday, April 03, 2008

[MOSS2007] License has expired error

ISSUE:
Users got the error message "Your licence for Office Server Search has expired" when performing a search or when opening a page which contained a web part that used the search.

CAUSE:
The other day I had to install some language pack on a MOSS2007 environment. After installing a language pack, you need to run the Configuration Wizard. But because people were working on the environment and the Configuration Wizard stops some services, I waited until a more suitable time to run the wizard.

While troubleshooting the issue I noticed that permissions on the registry were gone. So MOSS2007 wasn't able to read the license information and therefore reverts to expired mode, presenting the users with the error mentioned above.

Putting two and two together: Installing a language pack resets some permissions on the file system and/or registry. This is corrected when you run the Configuration Wizard.

RESOLUTION:
After running the Configuration Wizard, all problems were solved.

LESSON LEARNED:
Do not install language packs when you cannot run the Configuration Wizard immediately afterwards.

Wednesday, April 02, 2008

[MOSS2007] Fix for Event ID 6398 - Corrupted Memory

ISSUE:
On several environments I ran into an issue which logged an event in the event log (event id 6398). The following event occures regularly:
Event Type: Error
Event Source: Windows SharePoint Services 3
Event Category: Timer
Event ID: 6398
Date: 17-9-2007
Time: 9:38:18
User: N/A
Computer: Webserver
Description:The Execute method of job definition Microsoft.Office.Server.Administration.ApplicationServerAdministrationServiceJob (ID 3dc44d7a-008a-4c71-8e96-8c19da3d2ce2) threw an exception. More information is included below.

Attempted to read or write protected memory. This is often an indication that other memory is corrupt.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.


RESOLUTION:
At first I found some solutions which directed me to a .NET patch (KB923028, also included in .NET v2.0 SP1). This patch solved the issue for a short time, but after a while the messages reappeared.

Last week I found KB article 946517, which provides a fix for IIS. This patches fixes the issue mentioned above. The message never appeared again.

Tuesday, March 25, 2008

[MOSS2007] "Database Default Locations" ignored by SharePoint 2007

PROBLEM:
While installing a new environment today, I changed the "Database Default Locations" setting in SQL Server 2005 to make sure the database files and log files were created on a certain location. Testing by creating a database manually in the SQL Management Studio worked, but when I ran the SharePoint Configuration Wizard the databases were created in the old location.

CAUSE
After some troubleshooting I found out that SharePoint somehow retrieves the old setting until SQL Server 2005 has been restarted (reboot of the server or restart of the services). Why this is I have no clue.

RESOLUTION
Just restart SQL Server 2005 and try again. The new databases are now created in the correct place.

Wednesday, March 19, 2008

Usefull and Free SharePoint Tools (Part 1)

Over the past few months I have been working with several tools for SharePoint. Some of these tools I find very usefull:

DAVCopy
When you need to copy a lot of data to SharePoint (for example from a file share), doing this manually is:


  1. A lot of work
  2. Not easy to do, especially when you need to upload a directory structure. You have to use the explorer view
  3. Not really IT like (script what you can script, better lazy than tired :-)

Browsing through the Internet I ran into a tool called DAVCopy. This tool can upload files and directories to SharePoint, with the same functionality as XCopy. For example /E for subdirectories including empty ones.

The tool has some shortcomings though:

  1. As most of you probably already know, SharePoint doesn't accept certain characters in filenames (http://blogs.msdn.com/joelo/archive/2007/06/27/file-name-length-size-and-invalid-character-restrictions-and-recommendations.aspx). Just like copying with the Explorer view, DAVCopy just tries to upload the files without checking any illegal characters. A good thing is that DAVCopy logs the error and skips the illegal file.

    To get around this issue I created my own filename checker script, which corrects all illegal characters before running DAVCopy.
  2. The logging of the tool is very limited. It can only write logging to the console. A nice logging to a text file and especially a good summary will be a great improvement!

Link: DAVCopy


SharePoint 2007 Shared Services Provider User Profile Property Replicator
On a lot of projects we are using a development/test/acceptance/production strategy. Each environment must have its own settings. When migrating a solution to a new environment, all required settings need to be migrated as well. One of these settings are the custom User Profile Properties. A solution I recently deployed required these properties to be present before it started to work.

Creating these properties manually is not only time consuming, but also susceptable to errors. The User Profile Property Replicator tool is able to export and import these properties using XML files. I created an export of all properties in my development environment, changed the XML file to just include my custom properties and used this file to create the properties on all other environments. Quick, easy and without errors!

The only thing that can be improved is changing properties. When importing a property that already exists, the tool skips this property. I would like to see that the tool would update the property instead of skipping it. This is especially usefull when using big choice lists!

Link: MOSS Profile Replicate

Sharepoint Shared Services Search Provider Property Creation
Just like the tool above, this tool is able to export and import properties to/from XML files. Only this tool can export and import crawled properties, managed properties, content sources and search scopes.

A solution I had to deploy was relying on crawled and managed properties. To be able to map crawled properties to managed properties, the indexer first needed to discover these crawled properties in documents or content types. Using this tool I was able to export all managed and crawled properties from my development environment, change the exported XML files and import these properties into other environment. Again quick, easy and without errors!

Link: MOSS Provider Property Creation

SharePoint 2007 Shared Services Provider User Profile Importer
SharePoint is able to import settings from a external source (Active Directory, BDC source, etc) and map these values to certain profile fields. The external source is then leading for that property. But for some fields, you might not have an external source available but do want the field populated. Using the Profile Importer you can. Just create a XML input file and a XML mapping file (map the used XML tags in the input file to User Profile Property field) and off you go.

After creating my own User Profile Property, I would like to have certain values populated for certain users. Unfortunately these values were not in the Active Directory, so I could not sync them from AD. Using the Profile Importer tool I am able to populate the values I want into the fields I want for the users I want. All and all a great tool!

Link: MOSS Profile Import

SharePoint Solution Installer
As of SharePoint 2007 all custom components can be deployed using the Solution Deployment framework. Developer just have to place their solutions into a wsp package, which can then be imported into SharePoint 2007 and deployed to all servers.

Importing and removing wsp files into SharePoint 2007 must be done using the stsadm command, after which the solutions can be deployed from the Central Admin. The steps become difficult when you try to deploy a new version of a wsp file:

  1. Retract the old version from the Central Admin
  2. Use stsadm to remove the old version from solution deployment
  3. Use stsadm to add the new version to solution deployment
  4. Deploy the new version from the Central Admin

In other words switching between tools a lot. There must be an easier way!

And with the SharePoint Solution Installer there is! The tool first checks if SharePoint is installed, then if you have sufficient permissions to perform a deployment, if the Administration and Timer services are running and if the solution is already installed. If this is not the case, you can choose the web application to which the solution must be deployed. If it is installed, you can select to remove or repair (!!!!) the solution. In case of a repair, the tool automatically performs the four steps above automatically.

Installing wsp files made easy!!

Link: Sharepoint Solution Installer

Friday, January 11, 2008

SSL cannot find private key

ISSUE
Last week I generated a certificate request in IIS and requested an SSL certificate. Today I tried to install the recieved certificate, but after "Processing the request" the certificate was damaged. I was able to locate it in the Certificates MMC, but while trying to export the certificate the wizard came with the message "A private key cannot be found".
When I tried to apply the certificate to a IIS web site, the web site couldn't be reached using https. No error messages were reported.

TROUBLESHOOTING STEPS
I found the Microsoft SSL Diag tool on the Microsoft site and used this tool to troubleshoot. It came back with the error ""You have a private key that corresponds to this certificate but CryptAcquireCertificatePrivateKey failed"".

CAUSE
The above issue was caused by the fact that permissions on files in the following directory were incorrect:C:\Documents and Settings\All Users\Application Data\Microsoft\Crypto\RSA\MachineKeysThis directory contains files with the server's private keys. Due to the incorrect permissions, was the server unable to read the certificate private keys.

RESOLUTION
I reapplied the security settings of the directory to all files by opening the properties of the directory, select the security tab, click the Advanced button, select the "Replace permission entries on all child objects with entries show here that apply to child objects" and click OK.
The server was now able to read the private key files and therefore able to use the certificate.