All Classes
-
All Classes Interface Summary Class Summary Enum Summary Exception Summary Class Description AbstractJobQueue AbstractPriorityJobQueue AsyncFetcher This class holds the implementation to provide how items are fetched from the web, to validate the item and to store it if specified.AsyncFetcher.Builder A builder for async fetcher class.AsyncResponseConsumer On top of the abstract class, this class handles the parsing of a response from the web service.BaseResponse Callback Crawler This class handles the coordination between classes during the pre and post fetching of a page such as executing threads, calling to fetcher and manipulating the priority of a scheduled request.Crawler.Builder A builder for crawler class.CrawlerRequest This class allows the removal of proxy from request.DefaultUserAgent DummyFileManager This class implements a FileManager that writes response content to a file on the file system.EmptyContentValidator This class determines the validity of a response by its content length.FatalHandlerException This class defines fatal runtime exception forHandler
.Fetcher This interface represents only the most basic of fetching a request.FIFOJobQueue This class provides and implementation of scheduler with a first in first out queue.FIFOQueueScheduler Deprecated. FileManager<T> This interface represents the basic functions a FileManager should have.FileManagerCallback This class implements the default callback for file managers.Handler This interface represents the method call when the response is makeValidResponse the parsing logic will be up to individual implementations.HandlerRouter This interface allows the user to map request to handler.HttpFetcherRequest HttpFetcherRequest.Diagnostics This class contains the diagnostic information for this request.InlineExecutorService Interruptible Job This class will be placed in a scheduler for queuing requests.JobAttribute This interface represents attributes that can be added to jobs to manipulate the crawling process.LazyPriorityJobQueue This class provides and implementation of scheduler with a priority sensitive queue and polls from iterator when queue is empty.LazyScheduler Deprecated. MimeTypeValidator This class determines the validity of a response by its mime type.MysqlFileManager This class implements a FileManager that writes response content to a file on the file system and a record in MySQL database pointing to the record and allows retrieving the file using an id or request.MysqlFileManager.CompletedThreadedCallback A callback wrapper for to run complete multithreaded.PipelineValidator Priority Job priorities, list in descending order of priority: HIGHEST, HIGH, NORMAL, LOW, LOWEST.PriorityJobAttribute This class provides an implementation of job attribute with comparable priority.PriorityJobQueue This class provides an implementation of scheduler with a priority sensitive queue.PriorityJobScheduler Deprecated. ProxyProvider This interface allows the user to define proxies to be used for requests.Record<T> This interface represents only the most basic of a record and the fields that should be retrievable from database.Request Request.Method The method of the request to be made.Response ResponseDecompressor Modified from: org.apache.http.client.protocol.ResponseContentEncoding.Retrievable This interface represents that the response can be/ has been stored.Scheduler This interface represents only the adding part a scheduler.Session Session.Builder Builder for Session.Session.Key<T> A class representing the key for a session.SleepScheduler SocksConnectingIOReactor This IOReactor makes sure that the suppliedIOEventDispatch
is decorated withSocksIOEventDispatch
.SocksHttpRoutePlanner This route planners ensures that the connection to https server via socks proxy works.SocksIOEventDispatch This class wraps and handles IO dispatch related toSocksIOSession
.SocksIOSession The class establishes Socks4a connection and delegates the interface calls to a decoratedIOSession
.SocksIOSessionStrategy Socks + TSL/SSL layering strategy.StatusOkValidator This class determines the validity of a response by its status code.StopCodeException StorageException StorageFetcher This class holds the implementation to provide how items are fetched from a database, to validate the item and to store it if specified.StorageFetcher.Builder A builder for StorageFetcher class.StorageFetcherRequest StorageRecord<T> This class implements a default storage record.StorageRecord.Builder<T> A builder for StorageRecord class.StorageResponse StorageUtil A utility for storage needs.ThreadedWorkerManager ThreadedWorkerManager.AbstractManagedBlockingWorker This abstract class exposes the methods to allow submitting tasks for multithreading and implements inline blocking method.Unwrappable This interface represents that the request can be unwrapped.Unwrappable This interface represents that the response can be unwrapped.UrlRouter This class provides an implementation to select a handler based on the url from which they were fetched.UrlUtil A utility for managing URLs.UserAgent This interface allows the user to define a user agent string.ValidationException Validator Validator.Status The allowed return status of validation.ValidatorRouter This interface allows the user to map request to handler.VRequest An implementation of HTTP request.VRequest.Builder<T extends VRequest.Builder<T>> A builder for VRequest class.VResponse Worker WorkerManager