Saturday, February 19, 2011

Tredix IFS - Information retrieval done right

The Tredix Internet Find Store (IFS) is a custom framework developed for just one purpose: Search. Or to be more specific: Find.

IFS results are all about quality, relevance, and speed. Let's take a closer look at some of the IFS search services:

Full text search - The royal league of search

So the user enters some free text - but what does it mean? The IFS knows, because it understands linguistics: Language form, language meaning, and language in context. It works multi-lingual with language-specific stemming, lemmatisation, stop words, and normalisation. Distributed words as well as phrases are recognized and contribute to the quality rank. Synonyms and decomposed compounds provide additional hits. In case of literal errors, the IFS suggests language-independent phonetic similarities aka "Did You Means". Custom filters allow for flexible result manipulation.

If this is technobabble to you, here's a real world example of the IFS result quality. We recently built a demo for German fancy foods distributor "Bos Food". They have tens of thousands of special delicacies on stock. Since the goods originate from all seven continents, of course the product names are mostly foreign-language - and sometimes even made-up. French and spanish names might still sound familiar to the mainly German Bos customers, but what about those African or ethno food names? Clients simply couldn't spell the products, so Bos' classic SQL-based search failed continuously. People couldn't find the products and Bos lost massive business on his web shop.

When we replaced their SQL with the Tredix IFS (which by the way never got rolled out on their web site), search worked perfectly: Salsa Fumy would find Sansai Fumi, general queries like poultry or pastry would deliver both duck and chicken or cookie and pie; fishing would also find fish; and a solid German compound like Holzboot would even find Schiffchen aus Holz. Finally, just for the records, users of keyboards without diacritics would still be able to find Öl or Rougié, of course.

Selling is all about finding in the first place, isn't it?

Suggestions - Instant dynamic feedback

Unlike with a full text search box where you have to type a full search term and hit return, results in a suggest box will appear instantly as you type, helping you see where you're headed, every step of the way. The IFS Suggester supports pretty much all the search logic like its full text sibling, however quality and weight of the suggestions need to be calculated in a more "predictive" way. Since suggestions appear while you still type, we prefer to call them predictions as they actually help guide your search. So result ranking is vitally important.

Now apart from the different levels of match quality, usually popularity makes one of the most important ranking factors. However, under certain circumstances popularity can be misleading. Think about our casa.keru map application where you can jump to a geographic location by entering a location's name into a suggest box. In Portugal, many towns have the same name; for example, there are more than a dozen places called "Luz" all over the country. Imagine you are currently navigating around Faro on the southern coast of Portugal and want to zoom into the Luz area in the Eastern Algarve. If the IFS Suggester would simply rank the most popular (most searched for) city of Luz best, you would be taken to the city of Luz north of Lisbon.

Fortunately, the IFS framework exposes plugin interfaces to further customize each service. Since we are not searching for the best text match but the best location match, we added geographic distance calculation to the ranking algorithm in casa.keru. You guessed it: The IFS would now suggest to take you to the city of Luz in the Faro neighbourhood, which is usually the desired effect for people hunting for homes on casa.keru. This is a good example on how to transform a simple text suggestion into a smart location prediction.

Geologic - More intelligence for maps

Talking about location, the IFS provides native search on maps, too. The IFS Geofinder usually returns a set of coordinates for a given filter set and map section. In this context, result representation poses the biggest challenge. Geo point density must be adjusted in relationship to the map dimensions and zoom level. So the IFS Geofinder combines stacked coordinate pairs into clusters where necessary. This way, a perfect representation on the map is always guaranteed. For more non-technical information on the advantages of clustering, please see my post "Clustering is key to map application success".

On top, the IFS Geofinder also provides all the necessary geopolygon logic to identify area relationships, distances, proportions etc. on the server-side. Geo points can be allocated within areas like city limits or administrative divisions. Apart from providing the parent subnational entities like parish, district, etc. for any given coordinate, geopolygon logic is also extremely useful for tagging the Geo points. If, for example, you got a polygon of a nature reserve or a polyline of the coastline, the coordinate could automatically get tagged with "Ria Formosa" and "beach-front". Tags are supported in any IFS search service.

Today we have learned about some key features of the IFS search services. But how about reusability, integration, scalability, and customization of these services? I'll talk about the design of the IFS framework in one of my next posts. So keep following - further blog updates are on their way.

No comments:

Post a Comment