A B C D E F G H I J L M N P R S T U X

A

ActiveThreads - class spider.crawl.ActiveThreads.

keeping track of crawler threads so that they can be stopped when none of them have further URLs to crawl

ActiveThreads() - Constructor for class spider.crawl.ActiveThreads

add() - Method in class spider.crawl.ActiveThreads

add to active thread count

add(char) - Method in class spider.util.Stemmer

Add a character to the word being stemmed.

add(char[], int) - Method in class spider.util.Stemmer

Adds wLen characters to the word being stemmed contained in a portion of a char[] array.

add(String, String, double) - Method in class spider.crawl.History

add into history

add(String, Vector) - Method in class spider.util.RobotExclusion

add an entry to the robot exclusion hash (one entry per server)

addElement(String) - Static method in class spider.util.BadURLList

addElement(String, String) - Static method in class spider.util.Redirections

add a redirection entry to the Hashtable

addElements(Vector) - Method in class spider.crawl.Frontier

Adds a set of elements to the frontier given a sorted list of urls (sorted by score) The function allows optimization of code when many URLs need to be added returns true unless Exception is caught - omitted for now

addOkay(int) - Method in class spider.crawl.Statistics

add to noOkay

addTimeouts(int) - Method in class spider.crawl.Statistics

add to noTimeouts

addToCache(String, String, String, long) - Method in class spider.crawl.Cache

To add the content of a url to cache

attribs - Variable in class spider.util.StackElement