Web IR, topical crawlers, and InfoSpiders
We study Web crawling algorithms.
We are particularly interested in adaptive, intelligent agent techniques inspired
by ecological and artificial life systems.
InfoSpiders is a class of adaptive multiagent algorithms for autonomous,
scalable topical Web crawling.
Adaptive Information Retrieval (ch. 7 of
Finding Out About by
Richard K. Belew) discusses InfoSpiders in the context of adaptive
systems for Web IR.
InfoSpiders/MySpiders are also featured Web-Searching Agents in AAAI's AI Topics and in
Movies of three InfoSpiders runs on the
Encyclopaedia Britannica (the EB5 corpus is described in my
and available on the
included with the FOA book)
This material is based upon work supported by the National Science
Foundation under CAREER grant No. IIS-0133124/0348940.
Any opinions, findings, and conclusions or recommendations expressed
in this material are those of the author(s) and do not necessarily
reflect the views of the National Science Foundation.