trentacular

SharePoint 2013 Enterprise Search has the built-in ability to OCR and index the content of your scanned tiff images during a crawl (whether they are are stored in SharePoint or not). This is a very powerful feature, yet a bit mysterious to configure as the configuration steps have changed since the 2010 version. I’ll outline the steps below:

1. Using Server Manager, ensure the Windows TIFF iFilter feature is enabled on each crawl server
   
2. Open the Local Group Policy Editor and locate the OCR folder beneath Computer Configuration > Administrative Templates.
   
3. Edit the policy setting for “Select OCR languages from a code page”.  Choose Enabled and select the appropriate languages.
   
4. Open the SharePoint Management Shell (using Run as Administrator) and run the following commands to configure content parsing for TIFF images.

   $ssa = Get-SPEnterpriseSearchServiceApplication
   New-SPEnterpriseSearchFileFormat -SearchApplication $ssa tif "TIFF Image File" "image/tiff"
   New-SPEnterpriseSearchFileFormat -SearchApplication $ssa tiff "TIFF Image File" "image/tiff"

5. Restart the SharePoint Search Host Controller service.
   
6. Open the Search Service Application administration.  Under the Crawling navigation item, navigate to File Types.  Add two new File Types for tif and tiff.
7. Perform a Full Crawl of your content.

Depending on how many TIFF images are crawled, this may be a considerably longer amount of time than your previous crawl time.  Additional planning may be necessary, such as potentially scoping a Content Source to only content that should be OCR’d, or adjusting crawl schedules.

Today I attempted to deploy a BDC model to my client’s SharePoint farm. I used the Visual Studio 2010 SharePoint project template and the Business Data Catalog Model item template to create the model, and I had no issues using the convenient Deploy command from within Visual Studio to deploy the model and make use of it in my virtual development environment. But when I deployed my solution to my client’s farm and attempted to activate the BDC model’s Feature, the activation would fail with the following error:

The default web application could not be determined. Set the SiteUrl property in feature “My BDC Model Feature” to the URL of the desired site and retry activation.
Parameter name: properties

I soon found Frederik Prijck’s blog post that explained that I am supposed to specify a SiteUrl as a property of the model that can be used to resolve a Web application and mapped BCS service application to deploy the model to. So why was it I was able to deploy the model in my development environment?

The answer is that in my development environment, I have one and only one Web application that is hosted on port 80. As Frederik explained:

Strange enough this BCS entity will automatically attach an event receiver (Microsoft.Office.SharePoint.ClientExtensions.Deployment.ImportModelReceiver) to the feature. This event receiver has a method named GetDefaultWebApp which will be called when there is no SiteUrl property set in the manifest file. GetDefaultWebApp (as you can see using reflector) will search for a webapplication on port 80 or port 443 (not 100% sure about port 443, but you will understand what i mean i hope).

Frederik then goes on to talk of a dirty solution involving creating an empty Web application on port 80 just to support deploying BDC models without having to hard code a SiteUrl in the model.

The problem for me though is my client’s farm already has a Web application on port 80. Actually they have 8 Web applications using port 80, all with different host headers.

Opening up ILSpy to see exactly what was going on in the GetDefaultWebAppUrl method reveals that there is a little more to the logic. First of all, Web applications on port 80 will take precedence over Web applications on port 443. If there is only a single Web application on port 80 or 443, the method returns that Web application’s default zone URL. If there are multiple Web applications, the method then attempts to choose the Web application whose default zone’s host name has the least number of parts when split by the period character. For example, intranet.contoso.com would take precedence over test.intranet.contoso.com. If there is still ambiguity, the method makes one last check to see if one of the default zone’s host name begins with ‘www’. If ambiguity remains, the method throws the above quoted exception.

I can’t explain why Microsoft has chosen to implement this method in such a way, nor why they choose not to document it, but knowing this behavior provides another workaround. Taking a similar approach as Frederik, I created a dummy Web application having a host header beginning with ‘www’, created the root Site Collection, and my BDC model now deploys like a charm without a hardcoded SiteUrl.

SharePoint 2010′s Secure Store Service provides a way to map credentials and delegate access to remote resources. You may already be familiar with the MOSS 2007 Single Sign-on Shared Service, which was the former equivalent. The Secure Store Service integrates seemlessly with Business Connectivity Services (BCS), but it also features an API that can be taken advantage of within custom development projects. This makes the service an attractive option for storing sensitive configuration data such as connection strings, Web service credentials, etc.

The Secure Store Service allows us to create Target Applications which house sets of credentials. The two main types are Individual and Group applications, Individual meaning credentials are mapped to individual users, and Group meaning all users share the same set of credentials.

While the raw API isn’t very intuitive, its design was likely intentional (additional security by obfuscation). With a little marshalling help from our interop library friends, we are able to retrieve credentials (provided the appropriate permissions to the target application).

To begin, we need to reference a couple of assemblies.

Microsoft.BusinessData.dll

C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\ISAPI\Microsoft.BusinessData.dll

Microsoft.Office.SecureStoreService.dll

C:\Windows\assembly\GAC_MSIL\Microsoft.Office.SecureStoreService\14.0.0.0__71e9bce111e9429c\Microsoft.Office.SecureStoreService.dll

And now for the reason you came to this post … the code

using System.Collections.Generic;
using System.Linq;
using System.Runtime.InteropServices;
using System.Security;
using Microsoft.BusinessData.Infrastructure.SecureStore;
using Microsoft.Office.SecureStoreService.Server;
using Microsoft.SharePoint;
 
namespace Trentacular.SharePoint.Util
{
    public static class SecureStoreUtils
    {
        public static Dictionary<string, string> GetCredentials(string applicationID)
        {
            var serviceContext = SPServiceContext.Current;
            var secureStoreProvider = new SecureStoreProvider { Context = serviceContext };
            var credentialMap = new Dictionary<string, string>();
 
            using (var credentials = secureStoreProvider.GetCredentials(applicationID))
            {
                var fields = secureStoreProvider.GetTargetApplicationFields(applicationID);
                for (var i = 0; i < fields.Count; i++)
                {
                    var field = fields[i];
                    var credential = credentials[i];
                    var decryptedCredential = ToClrString(credential.Credential);
 
                    credentialMap.Add(field.Name, decryptedCredential);
                }
            }
 
            return credentialMap;
        }
 
        public static string ToClrString(this SecureString secureString)
        {
            var ptr = Marshal.SecureStringToBSTR(secureString);
 
            try
            {
                return Marshal.PtrToStringBSTR(ptr);
            }
            finally
            {
                Marshal.FreeBSTR(ptr);
            }
        }
    }
}

I just wrapped up a custom implementation of a faceted search web part that uses the MOSS Full Text Sql Query API. During the development, I ran into issues querying certain multi-valued Managed Properties, specifically Skills and Interests. In my query, I was using the CONTAINS predicate like such:

SELECT
UserProfile_GUID, PreferredName, JobTitle, Department, WorkPhone, OfficeNumber,
AboutMe, PictureURL, WorkEmail, WebSite, Path, HitHighlightedSummary,
HitHighlightedProperties, Responsibility, Skills, SipAddress
FROM scope()
WHERE freetext(defaultproperties,'+trent')
AND ("scope" = 'People')
AND CONTAINS(Skills, '"numchucks"')
AND CONTAINS(Skills, '"rockstar"')

Use of the CONTAINS predicate was purely due to my need to query multi-valued properties. If you are not querying a multi-valued property, you may simply use the ‘=’ or ‘LIKE’ predicates.

It turns out that the CONTAINS predicate will only work against managed properties that have been enabled as FullTextQueriable. I am not aware of a way to enable a property in the SSP Search Settings, so this has to be done using the API. I ended up including the following method as part of a Web Application scoped Feature Receiver to ensure certain Managed Properties were ‘FullTextQueriable’.

    using Microsoft.Office.Server;
    using Microsoft.Office.Server.Search.Administration;
 
    private void EnsureFullTextQueriableManagedProperties(ServerContext serverContext, params string[] managedPropertyNames)
    {
        var schema = new Schema(SearchContext.GetContext(serverContext));
        foreach (ManagedProperty managedProperty in schema.AllManagedProperties)
        {
            if (!managedPropertyNames.Contains(managedProperty.Name))
                continue;
 
            if (managedProperty.FullTextQueriable)
                continue;
 
            try
            {
                managedProperty.FullTextQueriable = true;
                managedProperty.Update();
                Log.Info(m => m("Successfully set managed property {0} to be FullTextQueriable", managedProperty.Name));
            }
            catch (Exception e)
            {
                Log.Error(m => m("Error updating managed property {0}", managedProperty.Name), e);
            }
        }
    }

So on the same note of Managed Property flags not visible in the SSP Search Settings, you may also want to know about the Retrievable flag. This flag prevents a Managed Property’s value from being returned if specified as a column in the SELECT statement.

The following table lists most of the out of the box Managed Properties and their default FullTextQueriable, HasMultipleValues, and Retrievable flag values.

Name	FullTextQueriable	HasMultipleValues	Retrievable
AboutMe
Account
AccountName
AssignedTo
Assistant
Author
Authority
BestBetKeywords
Birthday
CachedPath
CategoryNavigationUrl
CollapsingStatus
Company
contentclass
ContentSource
ContentType
Created
CreatedBy
DataSource
DatePictureTaken
Department
Description
DisplayTitle
DocComments
DocKeywords
DocSignature
DocSubject
DottedLine
EMail
EndDate
Fax
FileExtension
Filename
FirstName
FollowAllAnchor
HighConfidenceDisplayProperty1
HighConfidenceDisplayProperty10
HighConfidenceDisplayProperty11
HighConfidenceDisplayProperty12
HighConfidenceDisplayProperty13
HighConfidenceDisplayProperty14
HighConfidenceDisplayProperty15
HighConfidenceDisplayProperty2
HighConfidenceDisplayProperty3
HighConfidenceDisplayProperty4
HighConfidenceDisplayProperty5
HighConfidenceDisplayProperty6
HighConfidenceDisplayProperty7
HighConfidenceDisplayProperty8
HighConfidenceDisplayProperty9
HighConfidenceImageURL
HighConfidenceMatching
HighConfidenceResultType
HireDate
HitHighlightedProperties
HitHighlightedSummary
HomePhone
Interests
IsDocument
JobTitle
Keywords
LastModifiedTime
LastName
Location
Manager
MemberOf
Memberships
MobilePhone
ModifiedBy
MySiteWizard
NLCodePage
Notes
objectid
OfficeNumber
OWS_URL
PastProjects
Path
Peers
PersonalSpace
PictureHeight
PictureSize
PictureThumbnailURL
PictureURL
PictureWidth
PreferredName
Priority
ProxyAddresses
PublicSiteRedirect
Purpose
Rank
RankDetail
Responsibilities
Schools
SID
SipAddress
Site
SiteName
SiteTitle
Size
Skills
StartDate
Status
Title
UrlDepth
UserName
UserProfile_GUID
WebId
WebSite
WorkAddress
WorkCity
WorkCountry
WorkEmail
WorkId
WorkPhone
WorkState
WorkZip

As I am waiting for Reflector to load and my browser to unfreeze, I thought maybe if I write down all the things that cause me friction when attempting to be productive, I may just feel a little less frustrated by it all.

So here it goes:

1. Lotus Notes takes up to 20 seconds to redraw itself when returning from my VM
2. SameTime can also take this long
3. SharePoint Development Requirements – Lots of RAM and Windows Server OS
4. SharePoint GAC Deployment – Recycling of App Pools gets old
5. Getting distracted writing this blog post after Reflector finished loading 5 minutes ago
6. Visual Studio Add Reference Dialog (can’t wait to get my whole team over to VS 2010 to cross this one off the list)
7. Misleading WCF Add Service Reference Error Messages
8. Did I mention Lotus Notes
9. Web Meetings and Teleconferencing
10. Two-factor authentication with a 5 minute timeout (Explanation: I am working for a defense contractor)

On the other hand, there are certain things that just plain allow me to be productive:

1. My Wife Sarah
2. GMail
3. Google Reader
4. Google Chat
5. Visual Studio Intellisense
6. FireBug
7. Notepad++
8. My Colleague Winston’s MindTree application
9. Pencil and Paper (not the name of some app, I mean physical pencil and paper)
10. Whiteboards

So what is causing you friction?

I am currently customizing the display of SharePoint search results for my client and wanted a quick way to do iterative development of the XSLT style sheet used to render the results.  I decided to upload the style sheet to a Document Library in the Search Center site, configure the Core Results Web Part’s XSL Link to point to the url of the style sheet, and use WebDAV to iteratively edit/save the file, having my changes appear instantly in the Core Results Web Part.

What I didn’t realize is there are some additional steps to take to enable WebDAV for use on my development server (Windows Server 2003 VM).  Here they are in case you are trying to do something similar:

1. Enable WebDAV in IIS (found in the Web Service Extensions section)
2. Enable and start the WebClient service

Apparently in Windows Server 2003, the WebClient service is disabled by default.  In case you are wondering what this service does, here is its description:

Enables Windows-based programs to create, access, and modify Internet-based files.  If this service is stopped, these functions will not be available.  If this service is disabled, any services that explicitly depend on it will fail to start.

Wahlah, I am now able to access the document library using Windows Explorer with a path like the following:

\\mossdev\sites\SearchCenter\Documents

Stealing the name from Jan Tielens’ SmartPart for SharePoint, I’ve put together a framework for building “Smart” Editor Parts — “Smart” Editor Parts being User Controls that serve as Editor Parts.  This blog post is an excerpt from the complete writeup of the framework.

Downloads

SharePoint Smart Editor Parts.docx (~159 KB)
A complete writeup of the SharePoint “Smart” Editor Part framework

Trentacular.SharePoint.SmartEditorPart.zip (~ 320 KB)
The Visual Studio solution of the SharePoint “Smart” Editor Part framework
(Requires Visual Studio 2010 and WSPBuilder Extensions 2010)

Introduction

About the SharePoint Web Part Platform

Windows SharePoint Services 3.0 (WSS) offers a robust platform for hosting Web Parts, providing a Web Part Manager implementation that builds a rich tool pane for configuring the Web Parts.  Out of the box, WSS provides tool panes for customizing the appearance and layout, and will even generate basic fields for custom Web Part properties that are marked with the appropriate attributes.   The following table describes how custom property types are displayed in the property pane:

Building Custom Editor Parts

When a custom property requires a user interface element other than the ones listed above or needs custom validation, you can write a custom Editor Part that inherits from System.Web.UI.WebControls.WebParts.EditorPart and have your Web Part implement System.Web.UI.WebControls.WebParts.IWebEditable to create your custom Editor Part.

If you would like to build your custom Editor Part using a User Control, there are a few more steps you will have to take, because the abstract System.Web.UI.WebControls.WebParts.EditorPart base class does not inherit from the System.Web.UI.UserControl class.  The rest of this guide will walk you through the steps to create a framework for building Editor Parts with User Controls.

Inheriting from a Web Part Base Class

Web Parts inheriting from Microsoft.SharePoint.WebPartPages.WebPart can also take advantage of this framework with slight modifications to this guide.  You will be creating a base class that inherits from System.Web.UI.WebControls.WebParts.WebPart that will encapsulate the majority of the Editor Part framework and will be used by the example Site Members Web Part.  If you prefer to inherit from Microsoft.SharePoint.WebPartPages.WebPart, you may change the base class you will be creating to inherit from this class instead; however, Microsoft recommends that you inherit from System.Web.UI.WebControls.WebParts.WebPart whenever possible.

Overview of the User Control Editor Part Framework

Overview of Building an Editor Part

In order for a Web Part to have a custom Editor Part, the Web Part can implement the System.Web.UI.WebControls.WebParts.IWebEditable interface.  The abstract WebPart class that is typically used as the base class for custom Web Part implementations already implements the IWebEditable interface, which includes two exposed members.  The WebBrowsableObject property provides a way for EditorPart controls to get a reference to the associated Web Part.  The CreateEditorParts method is used to create an instance of each custom EditorPart control associated with the Web Part, and return them as a collection.  Custom Web Parts that inherit from the abstract WebPart class need only to override the CreateEditorParts method in order to add additional Editor Parts.

The EditorPart class exposes a property named WebPartToEdit of type WebPart that the Editor Part can use to read and set the configurable properties of the Web Part.  Additionally, two abstract methods SyncChanges and ApplyChanges are exposed that are called by the Web Part framework at the appropriate times when the Editor Part should read or set these properties on the associated Web Part.

Extending the Editor Part Framework to Support User Controls

In order to use a User Control as an Editor Part, a generic Editor Part called UserControlEditorPart<T> is needed that will wrap the appropriate User Control.  The UserControlEditorPart<T> will also need to pass the SyncChanges and ApplyChanges method calls along to the User Control.  This can be accomplished by defining an interface to be implemented by the User Control.  This interface will be called IWebPartEditor<T>.

The UserControlEditorPart<T> class is generic because it will inherit from another base class that is also generic called BaseEditorPart<T>.  This base class is extracted so that it can be inherited by both the UserControlEditorPart <T> and any other custom non-User Control Editor Parts.  Its sole purpose is to provide the convenience of casting the WebPartToEdit property of the EditorPart abstract class to the specific Web Part type associated to the EditorPart.

Enabling Attribute Decorations for Editor Part Associations

While it is relatively straight-forward to implement the IWebEditable interface’s CreateEditorParts method in a custom Web Part class, the framework additionally abstracts this step away, enabling you to associate Editor Parts to your Web Part using attributes.  This is accomplished by supplying two additional classes: WebEditableWebPart and EditorPartAttribute.

The WebEditableWebPart class is a new base class to be used by your custom Web Parts.  The class inherits from WebPart and overrides the CreateEditorParts method to add any Editor Parts to the Web Part that are specified by the EditorPartAttribute class.

The EditorPartAttribute class is to be applied as a class attribute to a Web Part requiring a custom Editor Part.  The attribute allows for an Editor Part to be specified by either its type or by a virtual path to a User Control that implements IWebPartEditor<T>.

Using the User Control Editor Part Framework

With the User Control Editor Part framework in place, there are now just three steps needed in order to create an Editor Part as a User Control:

1. Ensure the Web Part inherits from WebEditableWebPart.  In the case where you are already using a custom base class for your Web Parts, you may simply change your custom base class to inherit from WebEditableWebPart.
2. Create a User Control that implements IWebPartEditor<T>.
3. Decorate your Web Part using the EditorPartAttribute class specifying the virtual path to the User Control.

When managing users and groups within a SharePoint Web application configured to use Windows Integrated Authentication,  there is a convenient “Add all authenticated users” link that adds a special Active Directory group – NT AUTHORITY\authenticated users – to the Users/Groups People Editor.  This group refers to any non-anonymous user, which if you ask me, seems like a pretty common group to have around.  However, when working within a SharePoint Web application configured to use Forms Based Authentication (FBA), this convenient group is no longer available.

When using FBA, the only “non-SharePoint” groups available to us are the roles exposed by an ASP.Net Role Provider.  If you are already using a custom Role Provider and are not able to make changes to it, then you can stop here.  This post is not for you.  If you are like me though, and are using FBA merely for authentication and are leveraging SharePoint for all authorization, then the single “All Authenticated Users” role is all I need from my Role Provider.  As a result, there is no need to use a heavy weight Role Provider (i.e., the SQL Role Provider) to accomplish this, but rather roll your own very dumb role provider.  There is only a single method that you will need to implement – GetRolesForUser – in which you can assume the user is already authenticated and always return the “All Authenticated Users” role for the user. Here is the Role Provider I am currently using:

using System;
using System.Web.Security;
 
namespace Trentacular.Web.Security
{
    public class SimpleAllAuthenticatedUsersRoleProvider : RoleProvider
    {
        public const string AllAuthenticatedUsersRoleName = "All Authenticated Users";
 
        public override string ApplicationName { get; set; }
 
        public override string[] GetRolesForUser(string username)
        {
            return new[] { AllAuthenticatedUsersRoleName };
        }
 
        #region Methods Not Implemented
 
        public override string[] GetAllRoles() { throw new NotImplementedException(); }
        public override bool IsUserInRole(string username, string roleName) { throw new NotImplementedException(); }
        public override bool RoleExists(string roleName) { throw new NotImplementedException(); }
        public override void AddUsersToRoles(string[] usernames, string[] roleNames) { throw new NotImplementedException(); }
        public override void CreateRole(string roleName) { throw new NotImplementedException(); }
        public override bool DeleteRole(string roleName, bool throwOnPopulatedRole) { throw new NotImplementedException(); }
        public override string[] FindUsersInRole(string roleName, string usernameToMatch) { throw new NotImplementedException(); }
        public override string[] GetUsersInRole(string roleName) { throw new NotImplementedException(); }
        public override void RemoveUsersFromRoles(string[] usernames, string[] roleNames) { throw new NotImplementedException(); }
 
        #endregion
    }
}

After rolling your own role provider, you will need to register it in the web.config inside the <system.web> section as such:

<roleManager enabled="true" defaultProvider="SimpleAllAuthenticatedUsersRoleProvider">
    <providers>
        <add name="SimpleAllAuthenticatedUsersRoleProvider" type="Trentacular.Web.Security.SimpleAllAuthenticatedUsersRoleProvider, Trentacular.Web, Version=1.0.0.0, Culture=neutral, PublicKeyToken=aaaaaaaaaaaaaaaa" />
    </providers>
</roleManager>

I just released a new stsadm command in the Trentacular SharePoint 2007 Features CodePlex project called smartsolutionupgrade.  The purpose of this command is to perform Solution upgrades in a smart manner by performing the following actions:

1. Accepts as input either a single Solution filename or a filename of a text file containing a list of Solutions to be upgraded
2. Extracts and parses the Solution manifest file from each existing Solution to be upgraded in order to determine the Features that will be affected by the upgrade
3. Inventories the deployment states of the existing Solutions
4. Inventories the activation states of the affected Features at all scopes within the SharePoint Farm
5. Deactivates all affected Features
6. Retracts each of the existing Solutions and deletes them from the Solution store
7. Adds the updated Solution to the Solution store
8. Deploys each of the upgraded Solutions according to their previous deployment state
9. Activates all affected Features according to their previous activation state

Usage

stsadm -o smartsolutionupgrade [-filename <Solution filename>] [-filenamelist <Path to text file containing each of the solution filenames on separate lines>]

Job Definition Execution

Included in the project is a Job Definition Executor that takes care of running one-time scheduled service jobs and waiting for their completion before releasing control.  One particular issue addressed by the Job Definition Executor is the “A web configuration modification operation is already running” error caused by successive execution of the stsadm -o activatefeature command on Features that perform web configuration modifications.

In a multi-server farm, web configuration modifications are delegated to Timer Jobs; therefore, the return from execution of the activatefeature command does not mean the Feature activation (and thus the web configuration modification) has completed.  So you might say, well duh, you need to also execute the execadmsvcjobs command, but this command simply kicks of the job execution and returns from execution while still not waiting for the jobs to actually complete.

The Job Definition Executor solves this problem by kicking of the jobs and monitoring for their completion.  We can now safely execute multiple web configuration modification Features in sequence.

Update

I just released a second stsadm command called smartexecjobdefs that simply wraps the Job Definition Executor and can be used as a substitute for the execadmsvcjobs command.

I just published an enhanced List View Web Part in the Trentacular SharePoint 2007 Features CodePlex project. The Web Part is a substitute for the out of the box List View Web Part with two additional capabilities:

• Toggling of views in place (without leaving the web part page)
• Ability to display lists located in different sites of the same SharePoint Farm

Development of this Web Part has proved to be a very difficult undertaking, and I would love for any assistance or ideas with how to go about accomplishing this differently.

Here is where I’ve currently landed in the Web Part’s development:

• I am accessing the SPLimitedWebPartManager at the selected view’s url to load the SharePoint List View Web Part that was provisioned for the view when created
• I then use reflection to invoke the private RenderView method of the Web Part

Yes, this sounds extremely hacky, and it is. Using the avaliable SharePoint ListView control didn’t play nice with many custom views (especially when the list used managed content types).  I’ve also tried the SPView.RenderAsHtml method, but this also presented problems.

Another possible option would be to use the ListViewByQuery WebControl, but this would require an intense amount of code to duplicate the sorting and filtering that is available through the ListView control. So if you think this Web Part would be useful and would like to contribute to furthering it along, please get in touch.

Here are some screenshots: