Automate Internet Proxy Management with PowerShell

The solution is very simple, yet the issue was so annoying to me that I thought to share this. I have a laptop and do work on it on site at my client and when I am at home. When I am in the office connected to my client’s network I need to use a proxy to access the Internet, while usually I do not need a proxy when I am located elsewhere. You normally set up a proxy using browser settings. With Internet Explorer 8 you open a browser, then go to Tools >> Internet Options >> Connections >> LAN Settings. There you configure the proxy parameters. As a part of configuration you can turn the proxy on or off by setting or clearing a Use a proxy server for your LAN check box, and as you switch the proxy the rest of your proxy configuration settings such as exceptions is preserved for you. Great!

My problem was that on almost daily basis I had to open the browser, navigate to settings then check or uncheck that box. I would come to work and forget to go through the steps so my browser would get stuck, and then I would go “oh yeah, I turned the proxy off last night”. Of course the reverse would happen at home… I tolerated this too long because I didn’t expect that a simple solution exists, but it does and here it is:

 HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ProxyEnable key stores a DWORD value, either 1 or 0 to indicate if the proxy is enabled.

So I wrote two 1-line PowerShell scripts and added two shortcuts to them to my Windows taskbar: Enable Proxy, Disable Proxy! Here is how the script to enable proxy looks like:

Set-ItemProperty "HKCU:\Software\Microsoft\Windows\CurrentVersion\Internet Settings" -Name ProxyEnable -Value 1

Then I’ve got excited, and wrote yet another rock star PowerShell script. Here it is:

$matches = @(ipconfig /all | findstr "myclientdomain")
if($matches.Count -gt 0){
    ./EnableProxy.ps1
}else{
    ./DisableProxy.ps1
}

That’s right I am using output of the ipconfig command to determine if I am on the client’s network and if yes then I turn on my proxy. After I have added a scheduled task running at user logon to invoke this script, the quality of my professional life has improved.

Writing Trace Output to ULS Log in SharePoint 2010

SharePoint 2010 supports writing custom messages to ULS log. A blog post by Waldek Mastykarz of Mavention provides a good example of how to do this. UPDATE 02/24/2011: Andrew Connell's article on MSDN from December 2010 provides essential information on diagnostics logging.

Ability to send messages from System.Diagnostics.Trace.Write() and Trace.Fail() method overloads to ULS is yet another “logging novelty” I will be focusing on in this post.

There is a new type in Microsoft.SharePoint assembly, which was not there in version 12 – Microsoft.SharePoint.SPULSTraceListener. As its name suggests it allows sending trace messages to ULS.

If you are instrumenting your custom SharePoint code and need to write diagnostic traces to ULS then you can wire up SPULSTraceListener type in your web.config and use tracing infrastructure available in System.Diagnostics.Trace.  Here are the steps:

1. Add a trace listener element to web.config. For more information see description of system.diagnostics element on MSDN.

<system.diagnostics>
    <trace>
        <listeners>
            <remove name="Default"/>
            <add 
                name="ULSTraceListener" 
                type="Microsoft.SharePoint.SPULSTraceListener, Microsoft.SharePoint, version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
        </listeners>
    </trace>
</system.diagnostics>

In this XML snippet note the <remove name=”Default”>. This is done to remove default trace listener, which is wired up by default and displays a pop-up window with the message when Trace.Fail() is called.

2. In central administration site configure diagnostic logging (Central Administration >> Diagnostic Logging). If you are using Trace.Write() method calls then they are logged as Verbose and so in order to actually see them in ULS logs you need to make sure that throttling level is set to Verbose for SharePoint Foundation > Service Connections category. If you use Trace.Fail() overloaded methods then the message will be written as a High level, which by default is on, so the message will normally end up being written to a log file. And yes, all messages will be written to the log under Service Connections category.

What About TraceContext.Write()?

UserControl, Page and HttpContext classes have a property named Trace of type System.Web.TraceContext. It allows tracing messages using ASP.NET trace infrastructure, for example: HttpContext.Current.Trace.Write(string message);. You can send ASP.NET trace messages to ULS using standard technique but with a caveat:

Add trace configuration element under system.web element in your web.config as follows:

<system.web>
  <trace enabled="true" requestLimit="1000" mostRecent="true" writeToDiagnosticsTrace="true"/>
    ...

Here the key attribute is writeToDiagnosticsTrace instructing ASP.NET trace to use tracing infrastructure from System.Diagnostics. Now trace messages written using TraceContext.Write() will be sent to ULS.

Now the caveat: adding system.web/trace element appears to destabilize SharePoint. I have tried this on SharePoint Foundation and SharePoint Server 2010 with October 2010 CU, and although tracing does work as expected I have got a consistent error when trying to create a new web part page:

Looking at the ULS log, I found these messages logged under SharePoint Foundation category:

SharePoint Foundation    General    b9y3    High    Failed to open the file 'C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\Resources\wss.resx'.

SharePoint Foundation    General    b9y4    High    #20015: Cannot open "": no such file or folder.   
SharePoint Foundation    General    b9y4    High    (#2: Cannot open "": no such file or folder.)   
SharePoint Foundation    General    b9y9    High    Failed to read resource file "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\Resources\wss.resx" from feature id "(null)".

 

As soon as I disable the system.web/tracing the error disappears. I am not sure if this is a bug or adding <trace> element under <system.web> is not supported. In any case there is enough of rich diagnostic features available, and if you are using tracing in your application then just avoid System.Web.TraceContext.Write() in favor of System.Diagnostics.Trace.Write() or at least use it temporarily if you need to. By the way System.Diagnostics.Debug.Write() also sends messages to trace log.

UPDATE (01-16-2011): There is yet another issue caused by turning on trace that I ran into today: SharePoint designer cannot open a site in a web application for which tracing is enabled, and keeps prompting for credentials. If you check out traffic with Fiddler, you will find that SPD makes requests to http://devserver2010/_vti_bin/shtml.dll/_vti_rpc, which return a 401 UNAUTHORIZED. Not even verbose logging reveals anything useful. Well, commenting out <trace> element in web.config restores normal functionality.

Configure Links Web Part to Open URLs In a New Window With SharePoint 2010

With SharePoint 2010 (or SharePoint Foundation 2010) you can customize list views using XSLT, which is very powerful and for simple changes also quite simple. Here is an example of how to make URLs in Links list web part open up pages in a new browser window.

Links web part as well as other list-based web parts uses a predefined list view to render its content. Since views are now defined using XSLT style sheets, all that is needed to control the HTML is to override appropriate XSLT template with a custom one that has HTML markup that we need, for example <a href=”….” target=”_blank”>. List web part’s edit mode panel contains a text field titled XSL Link under Miscellaneous category where you can provide a URL of an XSLT file, which will override default rendering behavior.

Let’s say we have a Links list with a few URLs in it and we have added a Links web part to a page. We want to have all links shown by the web part open up in a new window. We need to find an XSLT template which describes rendering of the field with a link. One easy way to determine this is to sort by the URL field then take a note of the value of SortField query string value, which tells us that the field name we need is URLNoMenu. URL field is not a part of default view, default view has a URL (URL with edit menu) column, so you would need to add a column titled URL. As shown in the picture below, I’ve added my own view “Only Url” to Links list with this field only and set the web part to use this view.

Now we need to find an XSLT template for the URLNoMenu field.  Out-Of-The-Box templates are stored under the 14 hive in TEMPLATE\LAYOUTS\XSL on server file system. For what we need there are 2 important files there: main.xsl and fldtypes.xsl. The former file is a default file used for formatting, which imports other XSLT files including fldtypes.xsl. The latter file is the one which contains many templates for rendering different fields. Doing search inside this file for URLNoMenu yields:

<xsl:template name="FieldRef_URLNoMenu_body" ddwrt:dvt_mode="body" match ="FieldRef[@Name='URLNoMenu']" mode="Computed_body">

Next we copy this template XML element and put it in a new XSLT file. I named mine TargetBlank.xsl. Inside the new file we change HTML anchor tag to include target=”_blank” (or anything else we need) and save the file. Once we specify URL to this new file in the XSL Link field of a web part being edited, it will override all default formatting, therefore we also need to make sure that other templates still apply. Thanks to XSL import and overriding mechanism this is easy to do, we just need to import main.xsl file, and our version of the FieldRef_URLNoMenu_body template will override the one located inside fldtypes.xsl. Here is the complete listing of the TargetBlank.xsl.

<xsl:stylesheet xmlns:x="http://www.w3.org/2001/XMLSchema" 
                xmlns:d="http://schemas.microsoft.com/sharepoint/dsp" 
                version="1.0" 
                exclude-result-prefixes="xsl msxsl ddwrt" 
                xmlns:ddwrt="http://schemas.microsoft.com/WebParts/v2/DataView/runtime" 
                xmlns:asp="http://schemas.microsoft.com/ASPNET/20" 
                xmlns:__designer="http://schemas.microsoft.com/WebParts/v2/DataView/designer" 
                xmlns:xsl="http://www.w3.org/1999/XSL/Transform" 
                xmlns:msxsl="urn:schemas-microsoft-com:xslt" 
                xmlns:SharePoint="Microsoft.SharePoint.WebControls" 
                xmlns:ddwrt2="urn:frontpage:internal">
  <xsl:import href="/_layouts/xsl/main.xsl"/> 
    <xsl:output method="html" indent="no"/>
    <xsl:template name="FieldRef_URLNoMenu_body" 
                  ddwrt:dvt_mode="body" 
                  match ="FieldRef[@Name='URLNoMenu']" 
                  mode="Computed_body">
        <xsl:param name="thisNode" select="."/>
        <xsl:variable name="url" select="$thisNode/@URL" />
        <xsl:variable name="desc" select="$thisNode/@URL.desc" />
        <xsl:choose>
            <xsl:when test="$url=''">
                <xsl:value-of select="$desc" />
            </xsl:when>
            <xsl:otherwise>
                <xsl:choose>
                    <xsl:when test="@Format='Image'">
                        <img onfocus="OnLink(this)" src="{$url}" alt="{$desc}" />
                    </xsl:when>
                    <xsl:otherwise>
                        <a onfocus="OnLink(this)" href="{$url}" target="_blank">
                            <xsl:choose>
                                <xsl:when test="$desc=''">
                                    <xsl:value-of select="$url" />
                                </xsl:when>
                                <xsl:otherwise>
                                    <xsl:value-of select="$desc" />
                                </xsl:otherwise>
                            </xsl:choose>
                        </a>
                    </xsl:otherwise>
                </xsl:choose>
            </xsl:otherwise>
        </xsl:choose>
    </xsl:template>
</xsl:stylesheet>

In this file the xsl:stylesheet element is copied from the main.xsl file. Also note the xsl:import and the xsl:output statements. Lastly we need to upload the XSLT file and reference its URL in the XSL Link field of our web part. That’s all there is to it. Our links should now open in a new browser window.

Decks, Source Code and Resources from October 19 MSPUG Meeting

Thanks a lot to the folks who showed up yesterday at the Mississauga User Group meeting. Also I’d like to thank Ray Outair for his efforts in running the group in celebration of its first year anniversary. I have posted the deck for my presentation “Leveraging SharePoint 2010 Search Technologies” and the source code of the demo web part demonstrating FQL capabilities on SoftForte site here.

If you are planning search deployment I highly recommend checking out technical diagrams on TechNet: http://technet.microsoft.com/en-ca/library/cc263199.aspx. There are 4 diagrams there dedicated specifically to Search. Also check out downloadable content here http://technet.microsoft.com/en-ca/library/cc262788.aspx – it has been recently updated.

As I mentioned at the meeting, I also recommend a book titled Professional Microsoft Search: FAST Search, SharePoint Search, and Search Server by Mark Bennett, Jeff Fried, Miles Kehoe and Natalya Voskresenskaya. It is a great resource on planning and architecture of SharePoint Enterprise Search, and specifically FAST Search products.

Last but not least is a reference to Steve Peschka’s blog post where Steve introduces a Search Explorer tool that he wrote. Steve’s blog is an essential resource on developing Search and other SharePoint development topics.

So I’d like to re-cap on a few questions we left unanswered yesterday:

1. FAST visual best bets were displayed, but did not get filtered based on User Context’s managed properties. The question is how to create/promote properties into user context for such filtering to happen correctly. UPDATE: I figured this one out

2. I was not lucky extending CoreResultsWebPart and make it process custom FQL queries, and was forced instead to write a web part from scratch. The question is whether there is a supported way to extend CoreResultsWebPart for FAST search queries.
UPDATE: this one is now also addressed. Check it out here.

3. Somebody has mentioned that they could not get PowerPoint visual preview to work with FAST search results, suspecting that SSL encryption between FAST Search Server and query SSA was to blame. Looks like some research is needed here.

I plan to look into these and post my findings. Please feel free to comment on if you get answers sooner.

Enumerate SharePoint 2010 Databases

If you are working with SharePoint in a lab environment and use one shared database server to store databases from multiple farms then with SharePoint 2010 you can quickly lose control of your databases since there are much more of them now. When you set up a lab farm you typically run Products Configuration Wizard which leaves you with several databases created whose names contain GUIDs, which makes it hard to distinguish between them and relate them to corresponding farms.

Let’s say you need to get rid of a farm and want to delete corresponding databases after. What do you do? If you haven’t yet disconnected from the farm and uninstalled SharePoint then PowerShell comes at help. Log in to one of the farm servers, start up SharePoint 2010 Management Shell and enter this command:

Get-SPDatabase | % {$_.Name}

It will list names of all databases utilized in the current farm. You can now create a script to drop these databases or do whatever you needed to do with them. If however it is too late and you have uninstalled your servers, you can still get the list of database names using a SQL query:

USE SharePoint_Config
select distinct name from Objects WITH(NOLOCK) where Name in (
select Name COLLATE Latin1_General_CI_AS_KS_WS from sys.databases WITH(NOLOCK))
GO

Replace SharePoint_Config with the name of configuration database for your farm. SharePoint products use Latin1_General_CI_AS_KS_WS collation so you need to do a cast. It is of course not supported to issue queries directly against SQL Server so I would use this approach for development environment only, and only when you are really cleaning it up and PowerShell isn’t an option.

Alternatively to avoid the mess in your database names from the start you can also pre-create them. Here is the guidance. Lastly, although the guidance does not suggest sharing database role between farms in production environment, it is quite practical to share a SQL server in development environment if you are not using your farm for performance and capacity testing.

UPDATE: on technet there is a list of databases used by SharePoint (a section in storage and SQL Server capacity planning article): http://technet.microsoft.com/en-us/library/cc298801.aspx#section1a

Slipstreamed Installation of SharePoint Server 2010 with June 2010 CU

I wanted to create a slipstreamed installation point for SharePoint Server 2010, which would include June 2010 CU. I could not find guidance on how to do this for the current version. MOSS 2007 guidance recommends extracting patches into \Update folder of the installation point. This works as well for the SharePoint 2010. Would the installer also pick up and apply executable packages, or  their extracts in subfolders under the \Update? I tried both variants – it does not.

The reason I went that path was because June 2010 CU consists of 6 packages, which is unusual given regular MOSS CUs single file format. Also when extracting each of the patches manually to a common folder the following two files are duplicated between KB983319 and KB983497 patches: osrchwfe-x-none.msp, osrchwfe-x-none.xml; and the following two - between KB2281364 and KB983497: pplwfe-x-none.msp, pplwfe-x-none.xml. By reviewing KB articles it appears that the files are identical so it is ok to overwrite them, which is what I’ve done using a script:

office-kb2124512-fullfile-x86-glb.exe /extract:C:\temp\sharepoint\Updates
office-kb2204024-fullfile-x64-glb.exe /extract:C:\temp\sharepoint\Updates
office-kb2281364-fullfile-x64-glb.exe /extract:C:\temp\sharepoint\Updates
office-kb983319-fullfile-x64-glb.exe /extract:C:\temp\sharepoint\Updates
office-kb983497-fullfile-x64-glb.exe /extract:C:\temp\sharepoint\Updates
spf-kb2028568-fullfile-x64-glb.exe /extract:C:\temp\sharepoint\Updates

Next I copied extracted files to the \Updates folder in my installation point.

After installing SharePoint and running configuration wizard most of the patches were applied, except for KB2124512 and KB2204024 according to DLL version check. I am running Windows 2008 R2, and SQL Server 2008 R2, and made a farm installation of SharePoint. It could be that either these patches do not apply in my environment, or something is not working in my installation. I will post an update if I find out more. For now I consider that the slipstream approach is generally the same as before even for multi-file CUs such as this one.

Modified Date Field is not Visible after Content Deployment

It is the simple things, or at least ones you assume must be simple that often create some extra anxiety with SharePoint – at least from what I’ve seen. Here is one I ran into a couple of days ago: my customer runs a SharePoint 2007 publishing portal with February 2010 CU, where content is created in authoring environment, then content deployment job transfers it to production farm located in perimeter zone and accessible to anonymous Internet users. The application is using a significant amount of custom code. In many cases SharePoint API is used as a data provider, and is accessed through an adapter layer, which helps decoupling custom code from the platform. Quite logical.

An example of how the adapter accesses SharePoint API is retrieving SharePoint document library information through SPList.GetItems(SPQuery).GetDataTable(). The DataTable instance is then returned back to the consuming custom code. The problem is that Modified column is missing from the returned DataTable in production farm, while it is there in authoring farm, which certainly breaks a lot of application logic.

It turns out that a Content Deployment Job Path setting (Central Administration > Operations > Manage Content Deployment Paths and Jobs > Content Deployment Path) is to blame: Security Information must be set to All in order for the issue to go away, and you would need to re-create the target site collection. Now this will not be a completely painless move – you will start getting a content deployment warning: User security information cannot be properly imported without setting UserInfoDateTime option to ImportAll. This is because you have a Deploy User Names checkbox cleared, which you should according to the guidance. Here is a related post about this warning.

What actually happens is when Security Information is not set to All, this results in hiding certain fields, including Modified date field. So the GetItems(SPQuery) will still return correct items satisfying the SPQuery parameter, but if you inspect properties of an SPField for the returned items you will see that:

 item.Fields["Modified"].CanToggleHidden == false;
 item.Fields["Modified"].Hidden == true;

and the SPListItemCollection.GetDataTable() method omits hidden fields when it constructs the DataTable. Knowing the above you can either edit content deployment path and do a clean content deployment, or if it is problematic for some reason, then you can change your custom code to manually construct a DataTable rather than using SPListItemCollection.GetDataTable().

UPDATE: The issue was fixed in April Cumulative Update (see http://support.microsoft.com/kb/981040/ for more details). Thanks to Bill Brockbank for noting this. A good moment to emphasize importance of running up-to-date software.