Extending CoreResultsWebPart to Handle Search Queries Written in FAST Query Language

This was one of unanswered items in my recent two presentations on Search at TSPUG and MSPUG, so I was driven to figure it out and eventually did get it to work although not without some controversial steps. In this post I chose to also describe other approaches I tried and things I learned along the way, which didn’t necessarily get me to the end goal but may still be useful in your specific scenario. If you are looking for just extending CoreResultsWebPart so that it can “understand” FQL then you may want to scroll down a bit. You can download the complete source code here. I was testing my code on a SharePoint Server 2010 with December 2010 CU installed.

As you may know search web parts in SharePoint 2010 are no longer sealed, which gives you lots of flexibility via extending them. CoreResultsWebPart is probably the most important of all and therefore it is a great candidate for being extended. I wanted to take a search phrase passed as a query string argument to a search results page and write my own FQL query using this phrase as a parameter. My FQL query would do something interesting with it, for example use XRANK to boost documents created by a particular author. I certainly wanted to leverage all the goodness of CoreResultsWebPart, just use my FQL query instead of a plain search phrase. Contrary to my expectations it turned out to be not trivial to accomplish. So let’s dive into the details.

The story started with it completely not working, so I was forced to write a custom web part that used query object model and in particular the KeywordQuery class to deal with queries written in FAST Query Language (FQL). This is the one I have demonstrated at MSPUG and (with little success) at TSPUG.  Below is a fragment showing how the query is submitted and results are rendered (BTW here is related MSDN reference with example).

using (KeywordQuery q = new KeywordQuery(ssaProxy))
 {
     q.QueryText = fql;
     q.EnableFQL = true;
     q.ResultTypes = ResultType.RelevantResults;
     ResultTableCollection tables = q.Execute();
     
     if (tables.Count == 0)
     {
         return;
     }

     using (ResultTable table = tables[ResultType.RelevantResults])
     {
         while (table.Read())
         {
             int ordinal = table.GetOrdinal("Title");
             string title = table.GetString(ordinal);
             TitleLabel.Controls.Add(
                 new LiteralControl(
                     String.Format("<br>{0}</br>\r\n", title)));
         }

         table.Close();
     }
 }

As you can see, rendering of results is very basic. Of course it can be made whatever it needs to be but why bother if there is CoreResultsWebPart around, which does it great already and is also highly customizable? So at this point I rolled up my sleeves. I have a book titled “Professional SharePoint 2010 Development” with a red Boeing 767 on the cover, and in Chapter 6 there is a great discussion of how to extend CoreResultsWebPart. Also Todd Carter speaks about it and shows a demo in his excellent video on Search. If you haven’t seen it I highly recommend spending an hour and 16 minutes watching it (Module 7).

Empowered with all this I wrote a web part that extended the CoreResultsWebPart and used a FixedQuery property to set a custom query, being almost one-to-one copy of Todd’s demo example. Here is the listing of this web part class (note how ConfigureDataSourceProperties() method override is used to set the FixedQuery property):

public class ExtendedCoreResultsWebPartWithKeywordSyntax : CoreResultsWebPart
{
    protected override void ConfigureDataSourceProperties()
    {
        const string LongQueryFormat = "(FileExtension=\"doc\" OR FileExtension=\"docx\") AND (Author:\"{0}\")";
        const string ShortQuery = "(FileExtension=\"doc\" OR FileExtension=\"docx\")";
        string query = null;

        if (String.IsNullOrEmpty(AuthorToFilterBy))
        {
            query = ShortQuery;
        }
        else
        {
            query = String.Format(LongQueryFormat, AuthorToFilterBy);
        }

        this.FixedQuery = query;
        base.ConfigureDataSourceProperties();
    }

    [SPWebCategoryName("Custom"),
    Personalizable(PersonalizationScope.Shared),
    WebPartStorage(Storage.Shared),
    WebBrowsable(true),
    WebDisplayName("Author to filter by"),
    Description("First and last name of the author to filter results by.")]
    public string AuthorToFilterBy { get; set; }
} 

The web part actually filters results by a given author. It uses keyword query syntax and not the FQL so we are far from being done yet. Remember how in the previous code fragment there was a line q.EnableFQL = true;? If just we could set it somewhere we would be essentially done! Well right but the KeywordQuery object is not directly accessible from the CoreResultsWebPart because it uses federation object model on top of the query object model (as do other search web parts). Purpose of the federation object model is sending same query to multiple search results providers and aggregating results later either in different spots on results page or in the same list. This is done by abstracting each search results provider by means of a Location class. Important classes in federation object model are shown on the diagram below.

 

As you can see, CoreResultsWebPart is connected to the federation OM through CoreResultsDatasource and CoreResultsDatasourceView types with the latter actually doing all the hard work interacting with the model. And also our objective, the EnableFQL property, exists in the FASTSearchRuntime class, which in turn sets this property on the KeywordQuery class, and as I showed at the beginning, this is what’s required to get FQL query syntax accepted.

As Todd Carter and the authors of Professional SharePoint 2010 Development point out, we need to extend the CoreResultsDatasourceView class in order to be able to control how our query is handled, and wire up our own class by also extending CoreResultsDatasource class. CoreResultsDatasourceView class creates a LocationList with a single Location object and correctly determines which concrete implementation type of ILocationRuntime to wire up based on search service application configuration. In other words, federation by default is not happening for the CoreResultsWebPart. There is another web part, FederatedResultsWebPart, and another view class, FederatedResultsDatasourceView, whose purpose is to do exactly that. With that, let us get back to our objective.

If we were using SharePoint Enterprise Search then we would be almost done, because public virtual method AddSortOrder(SharePointSearchRuntime runtime) defined in the SearchResultsBaseDatasourceView class would let us get our hands on the instance of ILocationRuntime. But since we deal with FASTSearchRuntime we are sort of out of luck. Yes there exists a method overload AddSortOrder(FASTSearchRuntime runtime) but it is internal! This is where the controversy that I have mentioned at the beginning comes to play: I was not able to find a better way than to invoke an internal member via Reflection. My way works for me, but keep in mind that usually methods are made private or internal for a reason. I used Reflection to access internal property LocationRuntime of the Location object. I don’t know why this property is internal. If someone knows or has a better way to get at the FASTSearchRuntime instance or the KeywordQuery instance – please leave a comment! Here is a code fragment showing extension of the CoreResultsDatasourceView and getting an instance of FASTSearchRuntime from there.

class CoreFqlResultsDataSourceView : CoreResultsDatasourceView
{
    public CoreFqlResultsDataSourceView(SearchResultsBaseDatasource dataSourceOwner, string viewName)
        : base(dataSourceOwner, viewName)
    {
        CoreFqlResultsDataSource fqlDataSourceOwner = base.DataSourceOwner as CoreFqlResultsDataSource;

        if (fqlDataSourceOwner == null)
        {
            throw new ArgumentOutOfRangeException();
        }
    }

    public override void SetPropertiesOnQdra()
    {
        base.SetPropertiesOnQdra();
        // At this point the query has not yet been dispatched to a search 
        // location and we can set properties on that location, which will 
        // let it understand the FQL syntax.
        UpdateFastSearchLocation();
    }

    private void UpdateFastSearchLocation()
    {
        if (base.LocationList == null || 0 == base.LocationList.Count)
        {
            return;
        }

        foreach (Location location in base.LocationList)
        {
            // We examine the contents of an internal 
            // location.LocationRuntime property using Reflection. This is 
            // the key step, which is also controversial since there is 
            // probably a reason for not exposing the runtime publically.
            Type locationType = location.GetType();
            PropertyInfo info = locationType.GetProperty(
                "LocationRuntime",
                BindingFlags.NonPublic | BindingFlags.Instance,
                null,
                typeof(ILocationRuntime),
                new Type[0],
                null);
            object value = info.GetValue(location, null);
            FASTSearchRuntime runtime = value as FASTSearchRuntime;

            if (null != runtime)
            {
                // This is a FAST Search runtime. We can now enable FQL.
                runtime.EnableFQL = true;
                break;
            }
        }
    }
}

By the way, another limitation of my approach is that using Reflection requires full trust CAS policy level. That said, finally we have arrived at our objective – we can set the flag on the FASTSearchRuntime, and it will understand our FQL queries. Our extended search results web part will show results as directed by the query (in the attached source code it uses XRANK) and leverage presentation richness of the CoreResultsWebPart.