Every organization’s search landscape is unique, so comparing enterprise search solutions is only possible where two or more solutions can be tested against the company’s dataset. What is more useful in comparing search solutions is measuring the accuracy of the Word Sense Disambiguation (WSD) technology underpinning the search solution. Not only does SpringSense’s WSD outperform all known fine-sense disambiguators, it also does it faster.
SpringSense achieves 82.4% precision in the meaning recognition of 12,744 nouns
In order to gauge the effectiveness of a WSD technology, the precision of the technology is measured and compared to a baseline over a given text, the larger the better. The precision, which is the total number of correct answers divided by the number of answers attempted, shows how good those answers are.
SpringSense was evaluated by performing an all noun disambiguation of 5000 randomly selected lines of text from the Semantically Tagged glosses of WordNeti.
SpringSense achieves 82.4% precision in the meaning recognition of 12,744 nouns in 5000 randomly selected sentences from the WordNet annotated glosses. This is 16% above the most frequent meaning baseline and 8 times better than any other unsupervised, fine sense, meaning recognition technology with unlimited vocabulary.
To put these results in perspective, prior to SpringSense state of the art precision for all word tasks was established to be between 65% and 70% when WordNet is adopted.
SpringSense In The Field
Jose Herrera-Perea, Director, Digital at OUA said “It is clear that SpringSense has proved a significant improvement in quality of searches in our in-site search for course/unit”
The importance of SpringSense’s WSD can be seen when search relevancy is measured. In a recent trial at Open Universities Australia (OUA), data from the 200 most popular searches used on the OUA website was compared before and after the SpringSense integration. The results were impressive in both precision and recall, Jose Herrera-Perea, Director, Digital at OUA said “It is clear that SpringSense has proved a significant improvement in quality of searches in our in-site search for course/unit”.
To read more about the Open Universities Trial, you can download the OUA Case Study here.
i http://wordnet.princeton.edu/glosstag.shtml
