MC+A

The Steps To Implement Search History On A Google Search Appliance or Google Mini

A few weeks ago there was a thread on the Google Groups regarding how to display search history on a Google Search Appliance query results page.  Our approach typically at MC+A is to utilize the xslt as much as possible as it provides less complicated infrastructure.  Because of this, we tend to utilize JQuery to provide some dynamics features to the interface.  In this case, it’s search history.  This article assumes you have some idea about jQuery, specifically the Ready function.

Step 1 – Create A DIV

Create a HTML DIV in your results page that you would like the history to be displayed.  For example:

<div id=”recentHistory”>No Recent Searches</div>

Note the id that you use for later in this process. You can also place text with the div that will be displayed until the jQuery finishes loading.

Step 2 – Add A Function Call to jQuery.Ready

$(document).ready(function(){
    setHistory(); //Function to set the Div
});

Step 3 – Add the Function Call

The following function relies on 5 other functions:

1. GetCookieVal – Get’s the Cookie Value
2. GetCookie – Get’s a raw Cookie
3. DeleteCookie – Delet’e’s a Cookie
4. SetCookie – Set’s a Cookie
5. GetParameter – Get’s a Query String Value

function setHistory(){
    var gsaCollection = getParameter(window.top.location.search.substring(1), "site");
    var gsaClient = getParameter(window.top.location.search.substring(1), "client");
    var gsaStylesheet = getParameter(window.top.location.search.substring(1), "proxystylesheet");
    //var gsaAccess =getParameter(window.top.location.search.substring(1),"a");
    var recentHistory = "<ul>";
    var historyTemp = GetCookie("searchhistory");
    var history;
    if (historyTemp == null) {
        var q = getParameter(window.top.location.search.substring(1), "q")
        if ((q == null) || (q == "")) {
            //no history
            recentHistory = recentHistory + "<li>None</li>";
        }
        else {
            recentHistory = recentHistory + "<li><a href=\"search?site=" + gsaCollection + "&client=" +
            gsaClient +
            "&proxystylesheet=" +
            gsaStylesheet +
            "&access=a" +
            "&q=" +
            q +
            "&output=xml_no_dtd" +
            "\" > " +
            q +
            "</a></li>";
            SetCookie("searchhistory", q, expiry);
        }
    }
    else {
        historyString = new String(historyTemp);
        history = historyString.split("|");
        var newHistory = new Array(5);
        var newHistoryString = "";
        newHistory[0] = getParameter(window.top.location.search.substring(1), "q");
        newHistoryString = newHistory[0];
        recentHistory = recentHistory + "<li><a href=\"search?site=" + gsaCollection + "&client=" +
        gsaClient +
        "&proxystylesheet=" +
        gsaStylesheet +
        "&access=a" +
        "&output=xml_no_dtd" +
        "&q=" +
        newHistory[0] +
        "\" > " +
        newHistory[0].replace(‘+’, ‘ ‘) +
        "</a></li>";
        for (var i = 0; i < history.length & i < 4; i++) {
            newHistory[i + 1] = history[i];
            newHistoryString = newHistoryString + "|" + history[i];
            recentHistory = recentHistory + "<li><a href=\"search?site=" + gsaCollection + "&client=" +
            gsaClient +
            "&proxystylesheet=" +
            gsaStylesheet +
            "&access=a" +
            "&output=xml_no_dtd" +
            "&q=" +
            history[i] +
            "\" > " +
            history[i].replace(‘+’, ‘ ‘) +
            "</a></li>";
        }
        SetCookie("searchhistory", newHistoryString, expiry);
    }
    recentHistory = recentHistory + "</ul>";
    document.getElementById("recentSearches").innerHTML = recentHistory;
}

I am planning on cleaning up the function and posting it here it its entirety in the next couple of days.;

Reply with a comment to nudge me along if I forget.

Answer: Yes – The Google Mini Can Index documents stored in Micrsoft SharePoint!

The Google Mini along with the MC+A Google SharePoint Sitemap can effectively be configured to crawl Microsoft SharePoint sites.  The MC+A Google SharePoint Sitemap produces a dyanmic site map which the Mini can use to crawl deep within SharePoint.   You can further enhance this by adding a Content Web Part onto the search page which will allow you to have the Google Mini produce search results within SharePoint.

Here’s it working in our environment.

and in edit mode:

How is the integration achieved?

Step 1 – Navigate to the search results page
Step 2 – Switch the search results page to edit mode
Step 3 – Delete the SharePoint search web parts
Step 4 – Add content editor web part
Step 5 Add the following code(modify for your environment)
Content for web part

How does this work?
1) SharePoint search web part reads the k parameter query string and makes an ‘ajax’ type of request to the appliance.
2) The end use broswer communicates directly with the Mini.

If you purchase a Google Mini from MC+A, we offer a free GSS with support matching for the term of the Google Mini. Otherwise, the product lists for $500 and includes a 30 day trial support during the trial. MC+A professional services can be engaged to assist with the integration within SharePoint.

Last month, I wrote about best practices for and some business impact of search technology. This month, I am delving into the Google Search Appliance’s ability to protect corporate information. To stop the loss of data, corporations spend thousands, nay, millions of dollars per year, either preventing the loss or cleaning up after it has been leaked (see my post on the $20 million laptop). It would be safe to say that protecting confidential information is at the top of the executive priority list.

Understanding how the Google Search Appliance works is part of what my job entails. Every time I hear something in relation to the Google Search Appliance or the Google Mini its usually their ability to keep secure information secure. From the Google Enterprise Search site, they provide a list of ways in which the Google Appliances can protect important and confidential information.

Here is a list of ways that Google Enterprise Search will protect sensitive information:

1. The Google Appliances work already existing security policies within the organization.
2. Kerberos Support: Provides native support for Kerberos, enabling a silent authentication process for end-users
3. Metadata Biasing: Administrators can bias results based on metadata (in addition to biasing on source, URL or date)
4. Advanced Reporting: View and export daily and hourly result sets, top queries, special feature usage and more. Report for every query, including reports on which queries receive no clicks by a user and how often the user is clicking on sponsored links in comparison to organic search results or OneBox modules.
5. Secure Web Content: Enable secure searching of information protected by basic HTTP authentication, NTLM versions 1 and 2, as well as Windows Integrated Authentication
6. Enterprise Single Sign-on: Integrate with forms-based single sign-on security systems, included Oblix and Netegrity, to enable seamless searching across secure content
7. Secure Content API: Search across secure content using Google’s SAML Authorization SPI to integrate into existing access control systems
8. X509 Client Certificates: Serve secure results to users using mutually authenticated x509 client certificates
9. Lotus Domino Security: Integrate with Lotus Notes environments using fast, efficient crawling of Lotus Domino servers

Here are nine specific examples of how the Google Appliances can protect sensitive information from reaching the end-users. Coming from a non-IT background, I understand that some of these reasons seem a bit difficult to understand. That is why I added the number one protection item (The Google Appliances work with existing security policing all ready existing within the organization). This means that no matter what environment the Google Appliance is placed in, it will not release information to users who do not identify themselves as an individual who has access to see that information. All this right out of the box!

If more proof is needed you can take a look at Windows Networking’s “10 million documents in a single box” post. It is also understood that security is constantly on Google’s mind for their Enterprise Search as well . There are other high marks that are touted for the GSA such as:

“The Google Search Appliance’s unrivalled security functions were the perfect answer to our requirements and allowed for optimal information sharing”- Administrative and Financial Manager, Societe Generale.

For searching secure content, Google enterprise search seems to be the choice to make. It’s plug-and-play simplicity for security is unmatched in the field of enterprise search. I have seen first-hand the impact that Google Enterprise Search has on an organization and its power and speed are only rivaled by its security.

Next month, I will be analyzing workers and how the Google Search Appliance has benefited them. Happy searching!

It is interesting to see another year come and go. In the waning months of 2008, it was great to get involved in such a versatile and needed industry. Over the past couple months the importance of the Google Search Appliance and what it can do for companies, has really come into focus for me.

With each passing day, more and more documents are created, modified and deleted. When end users are going through the documents that make their workday easier, it is important for them to know whether documents have been altered in anyway. Corporate information security is particularly analyzed and scrutinized by both legal and IT departments. So would it not be beneficial for employees to know if documents are the most relevant or now out-of-date? The Google Search Appliance and the Google Mini assist end users much the way Google helps internet users find the sites they are looking for online.

For the new year, I have set out to do a best practices analysis of companies that utilize the Google Search Appliance (aka GSA) or the Google Mini. My blog post will be what I will call a “secular” (or non-IT) analysis of the Google Search Appliance and which companies would benefit most from this outstanding and adaptable technology.

What are your company’s most important needs? Most managers and executives would agree with a 5 simple answers: information security (also known as data leak prevention), knowledgeable workers, timeliness, effective and efficient task completion, and great customer service. The amazing aspect of the Google Search Appliance (GSA) and the Google Mini (as well as other products offered by MC+A) is that it can assist, if not alter, all of these important business facets.

In upcoming blog posts, I will go through each of these business needs and analyze in depth how the Google Search Appliance and/or the Google Mini can help with each of these areas. I hope that you have a great New Year and that 2009 brings market growth and with it a healthy bottom line.

Two weeks ago MC+A did a webinar on using a Google Search Appliance (GSA) to index product catalog data.  The demonstration was developed to showcase how effective Google search technology can be added to a public facing website search.  Jason Wasserman from boats.net joined us on the call.  I recommend visiting their site to see great example of the technology in production use ( and for all of your OEM Yamaha and Honda marine parts).  This post is a detailing of how we built the example site for the demonstration.

Content Acquisition

Product catalog information provides some challenges because skus vary greatly from product line to product line.  Many times there are white space characters such as ‘-’ or ‘_’ in the sku name.  By default, when the search engine indexes these documents, the skus will be tokenized so that the single sku will become multiple keywords.  Additional, meta data values become significant when a user would like to filter based on structured information.

The GSA has two effective means for this.  First there is the Google web crawler and navigator your site hierarchy and find all of the documents very much like the Google.com crawlers do.  Second, the GSA comes with a database crawler that queries the database directly, indexes the result set and provide a direct link to the document.  There are some benefits for each approach.    Specifically,

Web Crawler

• The Web Crawler acts similarly to the public crawlers.  Improvements to it will positively effect your public search ranking.
• Utilizing the Web Crawler requires no direct access to the database by the GSA.
• Search Engine Optimizations performed for improving the GSA search quality will also positively effectively your public search results.
• Meta data can be added via meta data tag anywhere in the content (i.e. <meta name=”price” content=”12.95″)

Database Crawler

• Very fast reindexing.  (Jason mentioned his 1.5 million documents get indexed in 20 minutes.)
• The content that is indexed is limited to the query result and very specific to the product content.   We say that it has limited ‘noise’ or interference.
• You can add meta data per column of the result set.  This does not have to be included in your public content.
• Customization on how the content is index via the dbstylesheet.

For the purpose of the demonstration we choose the db crawler.  We set up a commerce website at http://catalog.mcplusa-dev.com.  It is a fresh install of OS Commerce.  OS Commerce by default uses product keys as the url.  This is a less effective way to reference a document.  For example, http://catalog.mcplusa-dev.com/product_info.php?products_id=1 would be better if it were http://catalog.mcplusa-dev.com/graphics/Matrox_G200_MMS.  There are url rewriters for most standard commerce products that help you achieve this.

Here's the out of the box installation of OS Commerce

Step 1: Setting up the Database Crawler

For the purposes of the webinar we set up a simple query based on the schema of  OS Commerce.  We create a Google Search Appliance data source with the following settings:

1. Database Type: MySQL
2. Host name: the Ip address of the server
3. Port: 3306
4. DatabaseName/UserName/Password: Specific to the db
5. Crawl Query:
   

   select products_description.products_name, products.*, manufacturers.manufacturers_name, products_description.products_description, concat(‘http://catalog.mcplusa-dev.com/product_info.php?products_id=’, cast(products.products_id as char)) as url from
   products inner join products_description
   inner join manufacturers on products_description.products_id = products.products_id and
   manufacturers.manufacturers_id = products.manufacturers_id
   where products_description.language_id =1

6. DB StyleSheet: dbstylesheet_catalog_testxsl This xsl was written by John Gregory a couple of years ago.  Its better than the on board stylesheet because it changes the title to the first column returned from your query and it makes each column a meta data field.
7. Serve URL Field (Radio Checked): “Url”  Url is a computed column in the result set and links directly to the product page.

Basic configuration of the Database Crawler

Step 2: Setting Up the Collection

The set up of the collection was pretty straight forward.  We set the url pattern to be: http://catalog.mcplusa-dev.com
Content Serving
We create a separate GSA frontend for this demonstration and added several tool-kits to it.  The first one we added was parametric navigation.  This toolkit allows you to filter your result set based on meta data.  Secondly, we added Search As you Type open source toolkit.  Following the instructions in each toolkit we were able to build the front end in about 20 minutes (and it probably shows ).

Search Results with Parametric and Search As You Type

Conclusion

We’ve demonstrated the steps necessary to effectively implement the Google Search Appliance for a product catalog.  Customers of ours who have used these steps have seen conversion rates increase 3 fold using the search compared to those who have not.   Feel free to contact us should you have any additional questions.

Many of our customers find it difficult to figure out which LDAP query to run in their Google Search Appliance (GSA) or Google Mini (Mini) LDAP setup wizard.  When configuring LDAP on the GSA, a common issue that the user may run into is finding the correct information for the “Distinguished Name” (DN) field.

Our solution to this problem is a handy command that can be executed on your server providing Active Directory.

In order to quickly find the DN, open a command prompt and enter the command “adsiedit.msc”.

After you execute this command navigate to the account in which you are seeking the distinguished name (This is typically your account).

Right-click on that account and select “properties”.

Then find the “Distinguished Name” category, double-click, and you will have the answer you seek.

In a recent Tech Target interview, MC+A’s president, Michael Cizmar, outlined his opinions on the growing popularity of cloud computing and how it will affect both IT solutions companies and enterprise end-users. 

 While a strong advocate for innovation, Michael believes that even in a “cloudy” world, there will still be a great need for solution companies who can add value by efficiently implementing new technologies across the enterprise.

According to Michael, those who prove their expertise by developing applications that make the cloud more efficient and productive, will continue to play a vital role in the technology world.  In addition, enterprise users win as talented developers are incented to showcase their abilities to an otherwise difficult to reach audience.   

As Michael notes, “If you have an application, and put it on the Google marketplace, you’re immediately exposed to 500,000 customers.”  With this type of opportunity for developers, and the resulting benefits to enterprise users, cloud computing – and innovative companies like MC+A – may be in the process of creating the perfect storm. 

Read the full interview here: 

http://channelmarker.blogs.techtarget.com/2008/06/12/the-impending-cloud/

Last year, Google Enterprise published a custom Keymatch OneBox project written in Java. We ported it to C# for those .NET inclined.

Project Description

This OneBox module mimics the Google Search Appliance’s built-in KeyMatch functionality. By using this instead of the built-in KeyMatch feature, you can customize the C# Service implementation to provide alternative behavior that better suits your needs.

Check it out.

We’ve received numerous requests over the past couple of months on using OneBox technology to pull data from SAP systems. We’ve created a solution offering for customers that employs our SAP OneBox Framework. We can use this framework to pull a variety of SAP data out of Netweaver. We’ve developed a sample using the classic flight data scenario.

Check out our PowerPoint presentation on our offering through this link:

Get Presentation Here!

Thanks to Adam Jenkin for reporting an issue with the GSALib. We now have released an update that moves the configuration from a seperate file to the appSettings of the configuration file.

There is also the simplest of demonstrations of how to use the library in an ASP.Net VB application.

Please goto: http://www.codeplex.com/GSALib to download the updates.