Package spider.util

Class Summary
BadURLList a list of URLs that do not repond properly to HTTP (text/html) request
Fetcher Fetches HTTP pages and various details from its response header
Hashing uses MD5 hashing to convert arbitrary strings into a 128 bit hexadecimal (String).
Helper A bunch of static helper functions
HTMLParser Description: The class provides methods to parse an html page and convert it into an XML format
Page Class defines the web page data structure
Redirections maintains a static datastructure of redirections
RobotExclusion stores the robot permissions for various Web sites
StackElement  
Stemmer Stemmer, implementing the Porter Stemming Algorithm The Stemmer class transforms a word into its root form.
Stopper The class Stopper is used to remove stop words
Tester To change this generated comment edit the template variable "typecomment": Window>Preferences>Java>Templates.
Tester2  
ThreadTimer Stops a thread after specified time
XMLParser To be used in conjunction with HTMLParser.