MC+A

The Problem

Header requests is the default method for how the Google Search Appliance (GSA) performs authorization on a document level (also known as late binding) for web based content (See The Header Requester).  There are numerous advantages and disadvantages.  One of the minuses, is that it relies on the content source to adhere to HTTP protocols.

We’ve experienced numerous content systems that don’t fully support the correct HTTP response for this to work.  In many cases of Lotus Domino or Microsoft SharePoint, a friendly message is return or their is an embedded header.  This causes the to misinterpret the response from the server and think the user has access to the document.

The common method pre 6.4 was to implement a SAML interface and develop custom code to handle the logic for the variety of content sources.  Google released several Open Source projects to jump start your efforts.  Most notably they are:

• Google Value Security Framework
• Google SAML Bridge for Windows

The Solution: Header Request Deny Rules

Those tended to be difficult for our clients to implement and another piece of infrastructure to deploy and manage.  In version 6.4, Google has added additional rule validation on the appliance.  You now can check the most common sets on the appliance with simple configuration:

Screen shot of the Header Request Deny Rule Form

This virtually eliminates many of the customizations that we’ve made for the wrong response.  How Neat!!!

New Version Released For 6.4

Google released a patch for the latest search appliance release.

Release notes can be found here.

MC+A support customers will be contacted to schedule the update.

Google has released a patch update to the 6.2 software version. The following list contains a list of the fixes. Compare this list to any issues you having outstanding.

2399366

Sometimes documents with the same date are not sorted by relevance when sorting by date.

2386544

Sometimes sort by date does not sort documents that are chronologically close in the correct order.

2371696

Documents with dates that are in the future are not sorted properly when sorting by ascending dates.

1894928

Error on GSA when configuring Google Apps: “Cannot enable Google Apps Integration.”

2311340

GSA Admin API fails to authenticate requests if timezone is not set to US/Pacific.

1837591

Some Word documents can not be converted resulting in a crawling error.

2300088

GSA cannot convert some documents that contain embedded docs.

2327144

Some internal indexing processes are not properly terminated.

2435021

Stalled connections are not properly detected in replication setup.

2551148

System logs are not properly rotated which can fill disks.

1971858

Search Latency graph is broken under ‘Status and Reports>Serving Status’.

1408031

Encoded non-ascii filenames become garbled if saved with Internet Explorer.

2287126

SupportCall via proxy server fails with Error establishing TCP connection to supportcall.google.com:443: (111, ‘Connection refused’).

2534465

Permanent Redirect URLs with extensions that require Conversion(.doc, .pdf, .xls, etc) are not followed if at redirect time the content-length response is different than 0.

2517840

The Crawl Queue URLs in 6.2 are shown as encoded.

2633250

In federation the primary node tries to authorize urls that have already been authorized by secondary nodes.

2335370

Results with same snippet and title do not get filtered in the search results when using the ‘filter=p’ search parameter.

2487684

The lang_zh-tw language filter fails to show search results beyond the first page.

2358978

RK parameter is not working properly.

2728200

In 6.2.0.G.44-P6 Admin Console is only available in English.

I have had some issue utilizing the Google Admin Toolkit’s python scripts.  Looks like there is some changes to the login and cookie exchange on the new version of the appliance.  There was an old version of a script called db_sync that is no longer posted on there. 

Here it is again and an example of the syntax.

c:\python26\python db_sync.py –hostname 10.10.10.29 –username backup –password Replacewithyourpassword –source db_product

where:

• 10.10.10.29 is the host name or the ip of the server
• backup is the name of the user that will be logging in.
• Replacewithyourpassword is the password (in this case, backup’s password)
• db_product is a db_connector

You can download the zip file here: db_sync

Google has just released 6.4 on the enterprise support site and has updated code.google.com’s documentation.  We have yet to install and verify.  But some of the big changes so far:

JavaScript Crawling

The search appliance now can detect links and content generated dynamically.  I wonder if this will help the forms authentication wizard?

Serve Time Authentication

You will no longer need to enable or disable security manager and legacy auth.

Administrative Improvements

There are various administrative improvements and organization.

The documentation is here.

Previously I had authored a series of new features that came as part of the 6.0 software release that was not noted in the software release. For many of our customers, these feature releases are more significant than say GSA unification since they only have a single GSA.

Metadata Base64 Encoding

In software release 6.2, you can now base64 encode both metadata names and metadata values.

 <record url=<a href="http://www.mcplusa.com">http://www.mcplusa.com</a> action="add" mimetype="text/html">
<metadata>
<meta encoding="base64binary" name="acdJvamFXQWjdF9uWEXl" content="asdfLKZUoiuoiasdfmoaioeit">
</metadata>
</record>

Note: the correct encoding attribute is base64binary and not base64 as Google’s documentation states.

A month ago I reported a screenshot of Google’s new layout in a previous post.   During my last few searches I’ve been given this layout.  The icons have been updated and take up less of the screen.  As well, the filtering has been compacted.

Google announced  new details about improvements to Google Docs.  The main features of the release are:

• A new margin ruler
• Better numbering and bullets
• Better placement of images
• “Higher Fidelity” document import
• Faster JavaScript process
• Faster Collaboration

Check it out below

Google released the latest patch to software version 6.2.  Please contact your MC+A customer support contact for information regarding this or to schedule a time for an upgrade to be performed.

The Steps To Implement Search History On A Google Search Appliance or Google Mini

A few weeks ago there was a thread on the Google Groups regarding how to display search history on a Google Search Appliance query results page.  Our approach typically at MC+A is to utilize the xslt as much as possible as it provides less complicated infrastructure.  Because of this, we tend to utilize JQuery to provide some dynamics features to the interface.  In this case, it’s search history.  This article assumes you have some idea about jQuery, specifically the Ready function.

Step 1 – Create A DIV

Create a HTML DIV in your results page that you would like the history to be displayed.  For example:

<div id=”recentHistory”>No Recent Searches</div>

Note the id that you use for later in this process. You can also place text with the div that will be displayed until the jQuery finishes loading.

Step 2 – Add A Function Call to jQuery.Ready

$(document).ready(function(){
    setHistory(); //Function to set the Div
});

Step 3 – Add the Function Call

The following function relies on 5 other functions:

1. GetCookieVal – Get’s the Cookie Value
2. GetCookie – Get’s a raw Cookie
3. DeleteCookie – Delet’e’s a Cookie
4. SetCookie – Set’s a Cookie
5. GetParameter – Get’s a Query String Value

function setHistory(){
    var gsaCollection = getParameter(window.top.location.search.substring(1), "site");
    var gsaClient = getParameter(window.top.location.search.substring(1), "client");
    var gsaStylesheet = getParameter(window.top.location.search.substring(1), "proxystylesheet");
    //var gsaAccess =getParameter(window.top.location.search.substring(1),"a");
    var recentHistory = "<ul>";
    var historyTemp = GetCookie("searchhistory");
    var history;
    if (historyTemp == null) {
        var q = getParameter(window.top.location.search.substring(1), "q")
        if ((q == null) || (q == "")) {
            //no history
            recentHistory = recentHistory + "<li>None</li>";
        }
        else {
            recentHistory = recentHistory + "<li><a href=\"search?site=" + gsaCollection + "&client=" +
            gsaClient +
            "&proxystylesheet=" +
            gsaStylesheet +
            "&access=a" +
            "&q=" +
            q +
            "&output=xml_no_dtd" +
            "\" > " +
            q +
            "</a></li>";
            SetCookie("searchhistory", q, expiry);
        }
    }
    else {
        historyString = new String(historyTemp);
        history = historyString.split("|");
        var newHistory = new Array(5);
        var newHistoryString = "";
        newHistory[0] = getParameter(window.top.location.search.substring(1), "q");
        newHistoryString = newHistory[0];
        recentHistory = recentHistory + "<li><a href=\"search?site=" + gsaCollection + "&client=" +
        gsaClient +
        "&proxystylesheet=" +
        gsaStylesheet +
        "&access=a" +
        "&output=xml_no_dtd" +
        "&q=" +
        newHistory[0] +
        "\" > " +
        newHistory[0].replace(‘+’, ‘ ‘) +
        "</a></li>";
        for (var i = 0; i < history.length & i < 4; i++) {
            newHistory[i + 1] = history[i];
            newHistoryString = newHistoryString + "|" + history[i];
            recentHistory = recentHistory + "<li><a href=\"search?site=" + gsaCollection + "&client=" +
            gsaClient +
            "&proxystylesheet=" +
            gsaStylesheet +
            "&access=a" +
            "&output=xml_no_dtd" +
            "&q=" +
            history[i] +
            "\" > " +
            history[i].replace(‘+’, ‘ ‘) +
            "</a></li>";
        }
        SetCookie("searchhistory", newHistoryString, expiry);
    }
    recentHistory = recentHistory + "</ul>";
    document.getElementById("recentSearches").innerHTML = recentHistory;
}

I am planning on cleaning up the function and posting it here it its entirety in the next couple of days.;

Reply with a comment to nudge me along if I forget.