InfoSpiders - Complementing search engines with online browsing agents

The scalability limitations of the current state of the art in Web search technology lead us to explore alternative approaches to centralized indexing, such as agent-based online search. The possible advantages of searching online, or agent-based browsing, are often down-played in the face of the apparently unquestionable loss of efficiency. This talk is an attempt to reach a more balanced view in which both the obvious and hidden costs of the two approaches are considered. The two approaches can then be compared more fairly, and possible complementarities and synergies are explored.

Click here to start

Table of Contents

InfoSpiders - Complementing search engines with online browsing agents

Outline

Problems with search engines

Problems with ìtraditionalî IR Systems on the Web

Can my computer browse for me?

Multi-agent approach

Evolutionary algorithm

PPT Slide

How to pick the next link?

Adaptation in InfoSpiders

Main contribution

Multithreaded InfoSpiders algorithm

InfoSpidersí parameters

InfoSpiders starting points

InfoSpiders interface (after 100 pages)

Result Analysis

Finding new relevant pages

Performance for multithreaded implementation

Multithreaded versus sequential implementation - speedup

Does better efficiency imply worse quality?

Conclusions

Future directions

Try it!

Authors:

Filippo Menczer
Department of Management Sciences
University of Iowa

Filippo Menczer, 1999 Fellow-At-Large of the Santa Fe Institute, teaches Management Information Systems at the University of Iowa. In 1991, after receiving a Laurea degree in Physics from the University of Rome, he was affiliated with the Institute of Psychology of the Italian National Research Council. In 1998 he graduated from the University of California at San Diego, where he received a M.S. in Computer Science and a Ph.D. in Computer Science and Cognitive Science. Filippo has published many refereed papers, book chapters, and serves as a reviewer for several journals and conferences. He has developed the LEE artificial life simulation tool, which is distributed with Linux and widely used in experimental and instructional settings. He has been the recipient of Fulbright, Rotary Foundation, CNR, Apple, and NATO fellowships, among others. His interdisciplinary research interests span from ecological theory to distributed information systems; they include artificial life, evolutionary computation, neural networks, machine learning, information retrieval, and adaptive intelligent agents.

Melania Degeratu
Program in Applied Mathematical and Computational Sciences
University of Iowa

Melania Degeratu is currently a PhD student in the Computer Science Department at Columbia University, New York.