spider.crawl
Class Statistics

java.lang.Object
  |
  +--java.lang.Thread
        |
        +--spider.crawl.Statistics

All Implemented Interfaces:

java.lang.Runnable

public class Statistics

extends java.lang.Thread

maintain statistics related with crawl 1. no of timeouts 2. no of redirections 3. no of parsing errors 4. time since start 5. number of threads 6. size of frontier 7. Memory in use 8. Available memory 9. no of pages fetched with 200 OK 10. no of pages crawled the statistics can be started as a thread and it prints out a file every t minutes

Author:

Gautam Pant

Field Summary

Fields inherited from class java.lang.Thread

MAX_PRIORITY, MIN_PRIORITY, NORM_PRIORITY

Constructor Summary

Statistics()

Statistics(long start, History h, Frontier front)

Method Summary

void	addOkay(int k) add to noOkay
void	addTimeouts(int k) add to noTimeouts
void	foundInCache(int k) add to noFoundInCache
void	notFound(int k) add to noNotFound
void	pagesCrawled(long p) pages crawled
void	parseErrors(int k) add to noParseErrors
void	run()
void	setFile(java.lang.String file) Sets the file.
void	setStop(boolean stop) Sets the stop.
void	toFile()

Methods inherited from class java.lang.Thread

activeCount, checkAccess, countStackFrames, currentThread, destroy, dumpStack, enumerate, getContextClassLoader, getName, getPriority, getThreadGroup, holdsLock, interrupt, interrupted, isAlive, isDaemon, isInterrupted, join, join, join, resume, setContextClassLoader, setDaemon, setName, setPriority, sleep, sleep, start, stop, stop, suspend, toString, yield

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Statistics

public Statistics(long start,
                  History h,
                  Frontier front)

Statistics

public Statistics()

Method Detail

run

public void run()

Specified by:

run in interface java.lang.Runnable

Overrides:

run in class java.lang.Thread

toFile

public void toFile()

setFile

public void setFile(java.lang.String file)

Sets the file.

Parameters:

file - The file to set

setStop

public void setStop(boolean stop)

Sets the stop.

Parameters:

stop - The stop to set

addTimeouts

public void addTimeouts(int k)

add to noTimeouts

addOkay

public void addOkay(int k)

add to noOkay

notFound

public void notFound(int k)

add to noNotFound

parseErrors

public void parseErrors(int k)

add to noParseErrors

foundInCache

public void foundInCache(int k)

add to noFoundInCache

pagesCrawled

public void pagesCrawled(long p)

pages crawled