November 25, 2015

904Labs supports LivingLabs

Research in the field of search engine technology has been dependent on manually constructed test collections for many years. These collections are used to measure the search quality of systems and to compare various algorithms. Although these collection have helped the field enormously, we have reached the limits of what can be investigated using these static collections.

Evaluation in the wild

Current research focuses mainly on dynamic collections and the use of real user feedback to create ever smarter search algorithms. The problem is that most universities and research institutes do not have access to (large) live search systems. Living Labs has jumped on the challenge and aims to provide a one-shop access point to various live search systems. Participants to Living Labs (e.g., universities) can sign up and run their search algorithms on live systems of, for example, Microsoft Academic Search, CiteSeerX, and SSOAR. This way, research in search engine technology gets another boost, one that we follow closely of course.