A B C D E F G H I J K L M N O P R S T U V W 
All Classes All Packages

A

AbstractJobQueue - Class in ai.preferred.venom.job
 
AbstractJobQueue(BlockingQueue<Job>) - Constructor for class ai.preferred.venom.job.AbstractJobQueue
Constructs an instance of AbstractJobQueue.
AbstractManagedBlockingWorker() - Constructor for class ai.preferred.venom.ThreadedWorkerManager.AbstractManagedBlockingWorker
 
AbstractPriorityJobQueue - Class in ai.preferred.venom.job
 
AbstractPriorityJobQueue() - Constructor for class ai.preferred.venom.job.AbstractPriorityJobQueue
Constructs an instance of AbstractJobQueue.
add(Request) - Method in class ai.preferred.venom.job.Scheduler
Adds a request to the queue.
add(Request, JobAttribute...) - Method in class ai.preferred.venom.job.Scheduler
Adds a request to the queue.
add(Request, Handler, JobAttribute...) - Method in class ai.preferred.venom.job.Scheduler
Adds a request to the queue.
add(Request, Handler, Priority) - Method in class ai.preferred.venom.job.Scheduler
Deprecated.
add(Request, Handler, Priority, Priority) - Method in class ai.preferred.venom.job.Scheduler
Deprecated.
add(Request, Priority) - Method in class ai.preferred.venom.job.Scheduler
Deprecated.
add(Request, Priority, Priority) - Method in class ai.preferred.venom.job.Scheduler
Deprecated.
add(Request, Handler) - Method in class ai.preferred.venom.job.Scheduler
Adds a request to the queue.
addHeader(String, String) - Method in class ai.preferred.venom.request.VRequest.Builder
Adds header to be used
addHeaders(Map<String, String>) - Method in class ai.preferred.venom.request.VRequest.Builder
Add headers to be used
ai.preferred.venom - package ai.preferred.venom
 
ai.preferred.venom.fetcher - package ai.preferred.venom.fetcher
 
ai.preferred.venom.job - package ai.preferred.venom.job
 
ai.preferred.venom.request - package ai.preferred.venom.request
 
ai.preferred.venom.response - package ai.preferred.venom.response
 
ai.preferred.venom.socks - package ai.preferred.venom.socks
 
ai.preferred.venom.storage - package ai.preferred.venom.storage
 
ai.preferred.venom.uagent - package ai.preferred.venom.uagent
 
ai.preferred.venom.utils - package ai.preferred.venom.utils
 
ai.preferred.venom.validator - package ai.preferred.venom.validator
 
ALWAYS_VALID - Static variable in interface ai.preferred.venom.validator.Validator
A validator that always return valid.
AsyncFetcher - Class in ai.preferred.venom.fetcher
This class holds the implementation to provide how items are fetched from the web, to validate the item and to store it if specified.
AsyncFetcher.Builder - Class in ai.preferred.venom.fetcher
A builder for async fetcher class.
AsyncResponseConsumer - Class in ai.preferred.venom.fetcher
On top of the abstract class, this class handles the parsing of a response from the web service.
awaitTermination(long, TimeUnit) - Method in class ai.preferred.venom.utils.InlineExecutorService
 

B

BaseResponse - Class in ai.preferred.venom.response
 
BaseResponse(int, String, byte[], ContentType, Header[], HttpHost) - Constructor for class ai.preferred.venom.response.BaseResponse
Constructs a base response.
build() - Method in class ai.preferred.venom.Crawler.Builder
Builds the crawler with the options specified.
build() - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
Builds the fetcher with the options specified.
build() - Method in class ai.preferred.venom.fetcher.StorageFetcher.Builder
Builds the fetcher with the options specified.
build() - Method in class ai.preferred.venom.request.VRequest.Builder
Builds the request with the options specified.
build() - Method in class ai.preferred.venom.Session.Builder
Create a new instance of session.
build() - Method in class ai.preferred.venom.storage.StorageRecord.Builder
Builds the storage record with the details specified.
build(Request.Method, String) - Static method in class ai.preferred.venom.request.VRequest
Create a new instance of builder with a method and url.
buildDefault() - Static method in class ai.preferred.venom.Crawler
Builds a new default instance of Crawler.
buildDefault() - Static method in class ai.preferred.venom.fetcher.AsyncFetcher
Create an instance of AsyncFetcher with default options.
builder() - Static method in class ai.preferred.venom.Crawler
Creates a new instance of Builder.
builder() - Static method in class ai.preferred.venom.fetcher.AsyncFetcher
Create an instance of builder.
builder() - Static method in class ai.preferred.venom.Session
Create a new instance of builder.
builder(FileManager<?>) - Static method in class ai.preferred.venom.fetcher.StorageFetcher
Create an instance of builder.
builder(T) - Static method in class ai.preferred.venom.storage.StorageRecord
Create an instance of builder.
Builder() - Constructor for class ai.preferred.venom.Session.Builder
 
Builder(Request.Method, String) - Constructor for class ai.preferred.venom.request.VRequest.Builder
Constructs an instance of builder.
buildResult(HttpContext) - Method in class ai.preferred.venom.fetcher.AsyncResponseConsumer
 

C

Callback - Interface in ai.preferred.venom.fetcher
 
cancelled(Request) - Method in interface ai.preferred.venom.fetcher.Callback
Method to be call upon cancellation of request.
cancelled(Request) - Method in class ai.preferred.venom.storage.MysqlFileManager.CompletedThreadedCallback
 
cancelled(Request) - Method in class ai.preferred.venom.storage.FileManagerCallback
 
channel() - Method in class ai.preferred.venom.socks.SocksIOSession
 
clearEvent(int) - Method in class ai.preferred.venom.socks.SocksIOSession
 
close() - Method in class ai.preferred.venom.Crawler
 
close() - Method in class ai.preferred.venom.fetcher.AsyncFetcher
 
close() - Method in class ai.preferred.venom.fetcher.StorageFetcher
 
close() - Method in class ai.preferred.venom.socks.SocksIOSession
 
close() - Method in class ai.preferred.venom.storage.DummyFileManager
 
close() - Method in class ai.preferred.venom.storage.MysqlFileManager
 
close() - Method in class ai.preferred.venom.ThreadedWorkerManager
 
compareTo(PriorityJobAttribute) - Method in class ai.preferred.venom.job.PriorityJobAttribute
 
completed(Request, Response) - Method in interface ai.preferred.venom.fetcher.Callback
Method to be call upon completion of request.
completed(Request, Response) - Method in class ai.preferred.venom.storage.MysqlFileManager.CompletedThreadedCallback
 
completed(Request, Response) - Method in class ai.preferred.venom.storage.FileManagerCallback
 
connected(IOSession) - Method in class ai.preferred.venom.socks.SocksIOEventDispatch
 
Crawler - Class in ai.preferred.venom
This class handles the coordination between classes during the pre and post fetching of a page such as executing threads, calling to fetcher and manipulating the priority of a scheduled request.
Crawler.Builder - Class in ai.preferred.venom
A builder for crawler class.
CrawlerRequest - Class in ai.preferred.venom.request
This class allows the removal of proxy from request.
CrawlerRequest(Request) - Constructor for class ai.preferred.venom.request.CrawlerRequest
Constructs an instance of crawler request with an underlying request.

D

decompress(HttpResponse) - Method in class ai.preferred.venom.utils.ResponseDecompressor
Decompress http response.
DEFAULT - Static variable in enum ai.preferred.venom.job.Priority
The default starting priority for a job.
DEFAULT_CHARSET - Static variable in class ai.preferred.venom.response.VResponse
The default charset to be used to decode response.
DefaultUserAgent - Class in ai.preferred.venom.uagent
 
DefaultUserAgent() - Constructor for class ai.preferred.venom.uagent.DefaultUserAgent
 
delete(String) - Static method in class ai.preferred.venom.request.VRequest.Builder
Creates a new instance of builder with method type delete.
DELETE - ai.preferred.venom.request.Request.Method
DELETE method.
determineRoute(HttpHost, HttpRequest, HttpContext) - Method in class ai.preferred.venom.socks.SocksHttpRoutePlanner
 
disableCompression() - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
Disables request for compress pages and to decompress pages after it is fetched.
disableCookies() - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
Disables cookie storage.
disconnected(IOSession) - Method in class ai.preferred.venom.socks.SocksIOEventDispatch
 
downgrade() - Method in enum ai.preferred.venom.job.Priority
Returns the priority one level below the current priority if priority is higher than the default floor or the lowest available priority.
downgrade(Priority) - Method in enum ai.preferred.venom.job.Priority
Returns the priority one level below the current priority if priority is higher than the specified floor or the lowest available priority.
drainTo(Collection<? super Job>) - Method in class ai.preferred.venom.job.AbstractJobQueue
 
drainTo(Collection<? super Job>, int) - Method in class ai.preferred.venom.job.AbstractJobQueue
 
DummyFileManager - Class in ai.preferred.venom.storage
This class implements a FileManager that writes response content to a file on the file system.
DummyFileManager(File) - Constructor for class ai.preferred.venom.storage.DummyFileManager
Constructs an instance of DummyFileManager.
DummyFileManager(String) - Constructor for class ai.preferred.venom.storage.DummyFileManager
Constructs an instance of DummyFileManager.

E

EMPTY_CALLBACK - Static variable in interface ai.preferred.venom.fetcher.Callback
An instance of empty callback.
EMPTY_SESSION - Static variable in class ai.preferred.venom.Session
An instance of an empty session.
EmptyContentValidator - Class in ai.preferred.venom.validator
This class determines the validity of a response by its content length.
EmptyContentValidator() - Constructor for class ai.preferred.venom.validator.EmptyContentValidator
 
enableSocksProxy() - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
Enables SOCKS protocol for proxies (socks://).
execute(Runnable) - Method in class ai.preferred.venom.utils.InlineExecutorService
 
execute(IOEventDispatch) - Method in class ai.preferred.venom.socks.SocksConnectingIOReactor
 
executeBlockingIO(Runnable) - Method in class ai.preferred.venom.ThreadedWorkerManager.AbstractManagedBlockingWorker
 
executeBlockingIO(Runnable) - Method in interface ai.preferred.venom.Worker
Performs the given task inline, and increase available threads in the pool by one for the execution of other tasks.

F

failed(Request, Exception) - Method in interface ai.preferred.venom.fetcher.Callback
Method to be call upon failure of request.
failed(Request, Exception) - Method in class ai.preferred.venom.storage.MysqlFileManager.CompletedThreadedCallback
 
failed(Request, Exception) - Method in class ai.preferred.venom.storage.FileManagerCallback
 
FatalHandlerException - Exception in ai.preferred.venom
This class defines fatal runtime exception for Handler.
FatalHandlerException() - Constructor for exception ai.preferred.venom.FatalHandlerException
Constructs a new fatal handler exception with null as its detail message.
FatalHandlerException(String) - Constructor for exception ai.preferred.venom.FatalHandlerException
Constructs a new fatal handler exception with the specified detail message.
FatalHandlerException(String, Throwable) - Constructor for exception ai.preferred.venom.FatalHandlerException
Constructs a new fatal handler exception with the specified detail message and cause.
FatalHandlerException(Throwable) - Constructor for exception ai.preferred.venom.FatalHandlerException
Constructs a new fatal handler exception with the specified cause and a detail message of (cause==null ? null : cause.toString()) (which typically contains the class and detail message of cause).
fetch(Request) - Method in interface ai.preferred.venom.fetcher.Fetcher
Fetch the desired HTTP page given in Request.
fetch(Request, Callback) - Method in interface ai.preferred.venom.fetcher.Fetcher
Fetch the desired HTTP page given in Request.
fetch(Request) - Method in class ai.preferred.venom.fetcher.AsyncFetcher
 
fetch(Request) - Method in class ai.preferred.venom.fetcher.StorageFetcher
 
fetch(Request, Callback) - Method in class ai.preferred.venom.fetcher.AsyncFetcher
 
fetch(Request, Callback) - Method in class ai.preferred.venom.fetcher.StorageFetcher
 
Fetcher - Interface in ai.preferred.venom.fetcher
This interface represents only the most basic of fetching a request.
FIFOJobQueue - Class in ai.preferred.venom.job
This class provides and implementation of scheduler with a first in first out queue.
FIFOJobQueue() - Constructor for class ai.preferred.venom.job.FIFOJobQueue
Constructs an instance of FIFOJobQueue.
FIFOQueueScheduler - Class in ai.preferred.venom.job
Deprecated.
FIFOQueueScheduler() - Constructor for class ai.preferred.venom.job.FIFOQueueScheduler
Deprecated.
 
FileManager<T> - Interface in ai.preferred.venom.storage
This interface represents the basic functions a FileManager should have.
FileManagerCallback - Class in ai.preferred.venom.storage
This class implements the default callback for file managers.
FileManagerCallback(FileManager<?>) - Constructor for class ai.preferred.venom.storage.FileManagerCallback
Constructs an instance of file manager callback.
fillInStackTrace() - Method in exception ai.preferred.venom.fetcher.StopCodeException
 
fillInStackTrace() - Method in exception ai.preferred.venom.fetcher.ValidationException
 
FLOOR - Static variable in enum ai.preferred.venom.job.Priority
The default lowest priority for a job.

G

get() - Method in class ai.preferred.venom.uagent.DefaultUserAgent
 
get() - Method in interface ai.preferred.venom.uagent.UserAgent
Gets the user agent for a request.
get(Request) - Method in interface ai.preferred.venom.ProxyProvider
Returns the get proxy from the list.
get(Request) - Method in interface ai.preferred.venom.storage.FileManager
Returns latest record matching request.
get(Session.Key<T>) - Method in class ai.preferred.venom.Session
Returns the session variable from the store.
get(Request) - Method in class ai.preferred.venom.storage.DummyFileManager
 
get(Request) - Method in class ai.preferred.venom.storage.MysqlFileManager
 
get(Integer) - Method in class ai.preferred.venom.storage.MysqlFileManager
 
get(Object) - Method in class ai.preferred.venom.storage.DummyFileManager
 
get(String) - Static method in class ai.preferred.venom.request.VRequest.Builder
Creates a new instance of builder with method type get.
get(T) - Method in interface ai.preferred.venom.storage.FileManager
Returns record by the internal record id.
GET - ai.preferred.venom.request.Request.Method
GET method.
getAcknowledge() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
Get the acknowledge time of the request.
getAttribute(String) - Method in class ai.preferred.venom.socks.SocksIOSession
 
getBaseUrl() - Method in class ai.preferred.venom.response.BaseResponse
 
getBaseUrl() - Method in interface ai.preferred.venom.response.Response
Deprecated.
getBaseUrl() - Method in class ai.preferred.venom.response.StorageResponse
 
getBaseUrl() - Method in class ai.preferred.venom.response.VResponse
 
getBaseUrl(Request) - Static method in class ai.preferred.venom.utils.UrlUtil
Deprecated.
getBody() - Method in class ai.preferred.venom.request.CrawlerRequest
 
getBody() - Method in class ai.preferred.venom.request.HttpFetcherRequest
 
getBody() - Method in interface ai.preferred.venom.request.Request
Returns the request body of the request or null if none specified.
getBody() - Method in class ai.preferred.venom.request.StorageFetcherRequest
 
getBody() - Method in class ai.preferred.venom.request.VRequest
 
getCallback() - Method in class ai.preferred.venom.storage.DummyFileManager
 
getCallback() - Method in interface ai.preferred.venom.storage.FileManager
Get callback upon completion of request.
getCallback() - Method in class ai.preferred.venom.storage.MysqlFileManager
 
getComplete() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
Get the complete time of the request.
getContent() - Method in class ai.preferred.venom.response.BaseResponse
 
getContent() - Method in interface ai.preferred.venom.response.Response
Returns raw content of the response.
getContent() - Method in class ai.preferred.venom.response.StorageResponse
 
getContent() - Method in class ai.preferred.venom.response.VResponse
 
getContentType() - Method in class ai.preferred.venom.response.BaseResponse
 
getContentType() - Method in interface ai.preferred.venom.response.Response
Returns the content type of the content fetched.
getContentType() - Method in class ai.preferred.venom.response.StorageResponse
 
getContentType() - Method in class ai.preferred.venom.response.VResponse
 
getContentType() - Method in interface ai.preferred.venom.storage.Record
 
getContentType() - Method in class ai.preferred.venom.storage.StorageRecord
 
getContentType(HttpEntity) - Method in class ai.preferred.venom.fetcher.AsyncResponseConsumer
 
getDateCreated() - Method in interface ai.preferred.venom.storage.Record
 
getDateCreated() - Method in class ai.preferred.venom.storage.StorageRecord
 
getDiagnostics() - Method in class ai.preferred.venom.request.HttpFetcherRequest
Get diagnostic information for this request.
getEventMask() - Method in class ai.preferred.venom.socks.SocksIOSession
 
getFileExtension(Response) - Static method in class ai.preferred.venom.storage.StorageUtil
Get file extension from a response.
getFileExtension(String) - Static method in class ai.preferred.venom.storage.StorageUtil
Get file extension from mime type name.
getHandler() - Method in class ai.preferred.venom.job.Job
Get the handler to handle the response of the job.
getHandler(Request) - Method in interface ai.preferred.venom.HandlerRouter
Returns the handler to be used for a specified request.
getHandler(Request) - Method in class ai.preferred.venom.UrlRouter
 
getHeaders() - Method in class ai.preferred.venom.request.CrawlerRequest
 
getHeaders() - Method in class ai.preferred.venom.request.HttpFetcherRequest
 
getHeaders() - Method in interface ai.preferred.venom.request.Request
Returns the headers set for the request.
getHeaders() - Method in class ai.preferred.venom.request.StorageFetcherRequest
 
getHeaders() - Method in class ai.preferred.venom.request.VRequest
 
getHeaders() - Method in class ai.preferred.venom.response.BaseResponse
 
getHeaders() - Method in interface ai.preferred.venom.response.Response
Returns the headers that were used to trigger this response.
getHeaders() - Method in class ai.preferred.venom.response.StorageResponse
 
getHeaders() - Method in class ai.preferred.venom.response.VResponse
 
getHtml() - Method in class ai.preferred.venom.response.VResponse
Returns the html in string format.
getHtml(Charset) - Method in class ai.preferred.venom.response.VResponse
Returns the html in string format.
getId() - Method in interface ai.preferred.venom.storage.Record
 
getId() - Method in class ai.preferred.venom.storage.StorageRecord
 
getInner() - Method in class ai.preferred.venom.request.CrawlerRequest
 
getInner() - Method in class ai.preferred.venom.request.HttpFetcherRequest
 
getInner() - Method in class ai.preferred.venom.request.StorageFetcherRequest
 
getInner() - Method in interface ai.preferred.venom.request.Unwrappable
Returns the unwrapped version of this request.
getInner() - Method in interface ai.preferred.venom.response.Unwrappable
Returns the unwrapped version of this request.
getInner() - Method in class ai.preferred.venom.response.VResponse
 
getJobAttribute(Class<T>) - Method in class ai.preferred.venom.job.Job
Get the job attribute for a specific attribute class or return null if not found.
getJsoup() - Method in class ai.preferred.venom.response.VResponse
Returns a jsoup document of this response.
getJsoup(Charset) - Method in class ai.preferred.venom.response.VResponse
Returns a jsoup document of this response.
getLatency() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
Get the latency between sending of the request and the first response.
getLocalAddress() - Method in class ai.preferred.venom.socks.SocksIOSession
 
getMD5() - Method in class ai.preferred.venom.storage.StorageRecord
Get md5 hash of the response content.
getMethod() - Method in class ai.preferred.venom.request.CrawlerRequest
 
getMethod() - Method in class ai.preferred.venom.request.HttpFetcherRequest
 
getMethod() - Method in interface ai.preferred.venom.request.Request
Returns the method type of the request.
getMethod() - Method in class ai.preferred.venom.request.StorageFetcherRequest
 
getMethod() - Method in class ai.preferred.venom.request.VRequest
 
getPriority() - Method in class ai.preferred.venom.job.PriorityJobAttribute
Get the priority in this attribute.
getProxy() - Method in class ai.preferred.venom.request.CrawlerRequest
 
getProxy() - Method in class ai.preferred.venom.request.HttpFetcherRequest
 
getProxy() - Method in interface ai.preferred.venom.request.Request
Returns the proxy set to be used for the request or default to fetcher if none specified.
getProxy() - Method in class ai.preferred.venom.request.StorageFetcherRequest
 
getProxy() - Method in class ai.preferred.venom.request.VRequest
 
getProxy() - Method in class ai.preferred.venom.response.BaseResponse
 
getProxy() - Method in interface ai.preferred.venom.response.Response
Returns the proxy that was used to trigger this response.
getProxy() - Method in class ai.preferred.venom.response.StorageResponse
 
getProxy() - Method in class ai.preferred.venom.response.VResponse
 
getQueue() - Method in class ai.preferred.venom.job.AbstractJobQueue
Get the BlockingQueue backing this scheduler.
getRecord() - Method in interface ai.preferred.venom.response.Retrievable
Returns record archive of this response that has been insert into a persistent storage.
getRecord() - Method in class ai.preferred.venom.response.StorageResponse
 
getRemoteAddress() - Method in class ai.preferred.venom.socks.SocksIOSession
 
getRequest() - Method in class ai.preferred.venom.job.Job
Get the request of this job.
getRequestBody() - Method in interface ai.preferred.venom.storage.Record
 
getRequestBody() - Method in class ai.preferred.venom.storage.StorageRecord
 
getRequestHeaders() - Method in interface ai.preferred.venom.storage.Record
 
getRequestHeaders() - Method in class ai.preferred.venom.storage.StorageRecord
 
getRequestMethod() - Method in interface ai.preferred.venom.storage.Record
 
getRequestMethod() - Method in class ai.preferred.venom.storage.StorageRecord
 
getResponse() - Method in exception ai.preferred.venom.fetcher.ValidationException
Get the response validated.
getResponseContent() - Method in interface ai.preferred.venom.storage.Record
 
getResponseContent() - Method in class ai.preferred.venom.storage.StorageRecord
 
getResponseHeaders() - Method in interface ai.preferred.venom.storage.Record
 
getResponseHeaders() - Method in class ai.preferred.venom.storage.StorageRecord
 
getScheduler() - Method in class ai.preferred.venom.Crawler
Get the instance of scheduler used.
getSize() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
Get the size the response.
getSleepScheduler() - Method in class ai.preferred.venom.request.CrawlerRequest
 
getSleepScheduler() - Method in class ai.preferred.venom.request.HttpFetcherRequest
 
getSleepScheduler() - Method in interface ai.preferred.venom.request.Request
Returns information about the amount of sleep before this request is made.
getSleepScheduler() - Method in class ai.preferred.venom.request.StorageFetcherRequest
 
getSleepScheduler() - Method in class ai.preferred.venom.request.VRequest
 
getSleepTime() - Method in class ai.preferred.venom.SleepScheduler
Get the amount of time to wait specified in this class.
getSocketTimeout() - Method in class ai.preferred.venom.socks.SocksIOSession
 
getSpeed() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
Get download speed in bytes per second.
getStart() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
Get the start time of the request.
getStatus() - Method in exception ai.preferred.venom.fetcher.ValidationException
Get the validation status of the response.
getStatus() - Method in class ai.preferred.venom.socks.SocksIOSession
 
getStatusCode() - Method in class ai.preferred.venom.response.BaseResponse
 
getStatusCode() - Method in interface ai.preferred.venom.response.Response
Returns status code of the response.
getStatusCode() - Method in class ai.preferred.venom.response.StorageResponse
 
getStatusCode() - Method in class ai.preferred.venom.response.VResponse
 
getStatusCode() - Method in interface ai.preferred.venom.storage.Record
 
getStatusCode() - Method in class ai.preferred.venom.storage.StorageRecord
 
getTryCount() - Method in class ai.preferred.venom.job.Job
Get attempt number of this job.
getUrl() - Method in class ai.preferred.venom.request.CrawlerRequest
 
getUrl() - Method in class ai.preferred.venom.request.HttpFetcherRequest
 
getUrl() - Method in interface ai.preferred.venom.request.Request
Returns the url of the request.
getUrl() - Method in class ai.preferred.venom.request.StorageFetcherRequest
 
getUrl() - Method in class ai.preferred.venom.request.VRequest
 
getUrl() - Method in class ai.preferred.venom.response.BaseResponse
 
getUrl() - Method in interface ai.preferred.venom.response.Response
Returns the url used to fetch the response, if the request is redirected, this will be the final requested url.
getUrl() - Method in class ai.preferred.venom.response.StorageResponse
 
getUrl() - Method in class ai.preferred.venom.response.VResponse
 
getURL() - Method in interface ai.preferred.venom.storage.Record
 
getURL() - Method in class ai.preferred.venom.storage.StorageRecord
 
getValidator(Request) - Method in interface ai.preferred.venom.ValidatorRouter
Returns the handler to be used for a specified request.
getValidator(Request) - Method in class ai.preferred.venom.UrlRouter
 
getWorker() - Method in class ai.preferred.venom.ThreadedWorkerManager
 
getWorker() - Method in interface ai.preferred.venom.WorkerManager
Get the result collector in use.

H

handle(Request, VResponse, Scheduler, Session, Worker) - Method in interface ai.preferred.venom.Handler
This function is called when the request is fetched successfully.
Handler - Interface in ai.preferred.venom
This interface represents the method call when the response is makeValidResponse the parsing logic will be up to individual implementations.
HandlerRouter - Interface in ai.preferred.venom
This interface allows the user to map request to handler.
hasBufferedInput() - Method in class ai.preferred.venom.socks.SocksIOSession
 
hasBufferedOutput() - Method in class ai.preferred.venom.socks.SocksIOSession
 
head(String) - Static method in class ai.preferred.venom.request.VRequest.Builder
Creates a new instance of builder with method type head.
HEAD - ai.preferred.venom.request.Request.Method
HEAD method.
HIGH - ai.preferred.venom.job.Priority
High priority.
HIGHEST - ai.preferred.venom.job.Priority
Highest priority.
HttpFetcherRequest - Class in ai.preferred.venom.request
 
HttpFetcherRequest(Request) - Constructor for class ai.preferred.venom.request.HttpFetcherRequest
Constructs an instance of http fetcher request.
HttpFetcherRequest.Diagnostics - Class in ai.preferred.venom.request
This class contains the diagnostic information for this request.

I

InlineExecutorService - Class in ai.preferred.venom.utils
 
InlineExecutorService() - Constructor for class ai.preferred.venom.utils.InlineExecutorService
 
inputReady(IOSession) - Method in class ai.preferred.venom.socks.SocksIOEventDispatch
 
INSTANCE - Static variable in class ai.preferred.venom.validator.EmptyContentValidator
An instance of this validator.
INSTANCE - Static variable in class ai.preferred.venom.validator.StatusOkValidator
An instance of this validator.
interrupt() - Method in class ai.preferred.venom.Crawler
Interrupts crawler, fetcher and worker threads.
interrupt() - Method in interface ai.preferred.venom.Interruptible
Interrupt the underlying mechanisms of the class.
interrupt() - Method in class ai.preferred.venom.ThreadedWorkerManager
 
interruptAndClose() - Method in class ai.preferred.venom.Crawler
Interrupts then close this object.
Interruptible - Interface in ai.preferred.venom
 
INVALID_BLOCKED - ai.preferred.venom.validator.Validator.Status
The response is invalid as it is blocked.
INVALID_CONTENT - ai.preferred.venom.validator.Validator.Status
The response has invalid content.
INVALID_STATUS_CODE - ai.preferred.venom.validator.Validator.Status
The response is invalid due to the status code.
isAcknowledged() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
Check if request has been started and acknowledged.
isClosed() - Method in class ai.preferred.venom.socks.SocksIOSession
 
isCompleted() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
Check if request has been started, acknowledged and completed.
isEmpty() - Method in class ai.preferred.venom.job.LazyPriorityJobQueue
 
isLayeringRequired() - Method in class ai.preferred.venom.socks.SocksIOSessionStrategy
 
isShutdown() - Method in class ai.preferred.venom.utils.InlineExecutorService
 
isStarted() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
Check if request has been started.
isTerminated() - Method in class ai.preferred.venom.utils.InlineExecutorService
 
isValid(Request, Response) - Method in interface ai.preferred.venom.validator.Validator
Method will be called when a response need validation.
isValid(Request, Response) - Method in class ai.preferred.venom.validator.EmptyContentValidator
 
isValid(Request, Response) - Method in class ai.preferred.venom.validator.MimeTypeValidator
 
isValid(Request, Response) - Method in class ai.preferred.venom.validator.PipelineValidator
 
isValid(Request, Response) - Method in class ai.preferred.venom.validator.StatusOkValidator
 
iterator() - Method in class ai.preferred.venom.job.AbstractJobQueue
 

J

Job - Class in ai.preferred.venom.job
This class will be placed in a scheduler for queuing requests.
Job(Request) - Constructor for class ai.preferred.venom.job.Job
Constructs a basic job.
Job(Request, Handler) - Constructor for class ai.preferred.venom.job.Job
Constructs a basic job.
Job(Request, Handler, JobAttribute...) - Constructor for class ai.preferred.venom.job.Job
Constructs a basic job.
JobAttribute - Interface in ai.preferred.venom.job
This interface represents attributes that can be added to jobs to manipulate the crawling process.

K

Key() - Constructor for class ai.preferred.venom.Session.Key
 

L

LazyPriorityJobQueue - Class in ai.preferred.venom.job
This class provides and implementation of scheduler with a priority sensitive queue and polls from iterator when queue is empty.
LazyPriorityJobQueue(Iterator<Request>) - Constructor for class ai.preferred.venom.job.LazyPriorityJobQueue
Constructs an instance of lazy scheduler without a default handler.
LazyPriorityJobQueue(Iterator<Request>, Handler) - Constructor for class ai.preferred.venom.job.LazyPriorityJobQueue
Constructs an instance of lazy scheduler with a default handler.
LazyScheduler - Class in ai.preferred.venom.job
Deprecated.
LazyScheduler(Iterator<Request>) - Constructor for class ai.preferred.venom.job.LazyScheduler
Deprecated.
Constructs an instance of lazy scheduler without a default handler.
LazyScheduler(Iterator<Request>, Handler) - Constructor for class ai.preferred.venom.job.LazyScheduler
Deprecated.
Constructs an instance of lazy scheduler with a default handler.
LOW - ai.preferred.venom.job.Priority
Low priority.
LOWEST - ai.preferred.venom.job.Priority
Lowest priority.

M

MimeTypeValidator - Class in ai.preferred.venom.validator
This class determines the validity of a response by its mime type.
MimeTypeValidator(String) - Constructor for class ai.preferred.venom.validator.MimeTypeValidator
Constructs mime type validator.
MimeTypeValidator(Pattern) - Constructor for class ai.preferred.venom.validator.MimeTypeValidator
Constructs mime type validator.
MysqlFileManager - Class in ai.preferred.venom.storage
This class implements a FileManager that writes response content to a file on the file system and a record in MySQL database pointing to the record and allows retrieving the file using an id or request.
MysqlFileManager(String, String, String, String, File) - Constructor for class ai.preferred.venom.storage.MysqlFileManager
Constructs an instance of MysqlFileManager.
MysqlFileManager(String, String, String, String, File, int) - Constructor for class ai.preferred.venom.storage.MysqlFileManager
Constructs an instance of MysqlFileManager.
MysqlFileManager(String, String, String, String, String) - Constructor for class ai.preferred.venom.storage.MysqlFileManager
Constructs an instance of MysqlFileManager.
MysqlFileManager.CompletedThreadedCallback - Class in ai.preferred.venom.storage
A callback wrapper for to run complete multithreaded.

N

NORMAL - ai.preferred.venom.job.Priority
Normal priority.

O

offer(Job) - Method in class ai.preferred.venom.job.AbstractPriorityJobQueue
 
offer(Job) - Method in class ai.preferred.venom.job.FIFOJobQueue
 
offer(Job, long, TimeUnit) - Method in class ai.preferred.venom.job.AbstractPriorityJobQueue
 
offer(Job, long, TimeUnit) - Method in class ai.preferred.venom.job.FIFOJobQueue
 
onContentReceived(ContentDecoder, IOControl) - Method in class ai.preferred.venom.fetcher.AsyncResponseConsumer
 
onEntityEnclosed(HttpEntity, ContentType) - Method in class ai.preferred.venom.fetcher.AsyncResponseConsumer
 
onResponseReceived(HttpResponse) - Method in class ai.preferred.venom.fetcher.AsyncResponseConsumer
 
options(String) - Static method in class ai.preferred.venom.request.VRequest.Builder
Creates a new instance of builder with method type options.
OPTIONS - ai.preferred.venom.request.Request.Method
OPTIONS method.
outputReady(IOSession) - Method in class ai.preferred.venom.socks.SocksIOEventDispatch
 

P

peek() - Method in class ai.preferred.venom.job.AbstractJobQueue
 
PipelineValidator - Class in ai.preferred.venom.validator
 
PipelineValidator(Validator...) - Constructor for class ai.preferred.venom.validator.PipelineValidator
Constructs pipeline validator.
PipelineValidator(List<Validator>) - Constructor for class ai.preferred.venom.validator.PipelineValidator
Constructs pipeline validator.
poll() - Method in class ai.preferred.venom.job.FIFOJobQueue
 
poll() - Method in class ai.preferred.venom.job.LazyPriorityJobQueue
 
poll() - Method in class ai.preferred.venom.job.PriorityJobQueue
 
poll(long, TimeUnit) - Method in class ai.preferred.venom.job.FIFOJobQueue
 
poll(long, TimeUnit) - Method in class ai.preferred.venom.job.LazyPriorityJobQueue
 
poll(long, TimeUnit) - Method in class ai.preferred.venom.job.PriorityJobQueue
 
post(String) - Static method in class ai.preferred.venom.request.VRequest.Builder
Creates a new instance of builder with method type post.
POST - ai.preferred.venom.request.Request.Method
POST method.
prepareRetry() - Method in class ai.preferred.venom.job.Job
This method is called before the job is scheduled for a retry.
prepareRetry() - Method in interface ai.preferred.venom.job.JobAttribute
This method is called before the job is scheduled for a retry.
prepareRetry() - Method in class ai.preferred.venom.job.PriorityJobAttribute
 
prependHeaders(Map<String, String>) - Method in class ai.preferred.venom.request.HttpFetcherRequest
Prepend headers to the current headers.
prependHeaders(Map<String, String>) - Method in class ai.preferred.venom.request.StorageFetcherRequest
Prepend headers to the current headers.
Priority - Enum in ai.preferred.venom.job
Job priorities, list in descending order of priority: HIGHEST, HIGH, NORMAL, LOW, LOWEST.
PriorityJobAttribute - Class in ai.preferred.venom.job
This class provides an implementation of job attribute with comparable priority.
PriorityJobAttribute() - Constructor for class ai.preferred.venom.job.PriorityJobAttribute
Constructs an instance of PriorityJobAttribute.
PriorityJobAttribute(Priority) - Constructor for class ai.preferred.venom.job.PriorityJobAttribute
Constructs an instance of PriorityJobAttribute.
PriorityJobAttribute(Priority, Priority) - Constructor for class ai.preferred.venom.job.PriorityJobAttribute
Constructs an instance of PriorityJobAttribute.
PriorityJobQueue - Class in ai.preferred.venom.job
This class provides an implementation of scheduler with a priority sensitive queue.
PriorityJobQueue() - Constructor for class ai.preferred.venom.job.PriorityJobQueue
 
PriorityJobScheduler - Class in ai.preferred.venom.job
Deprecated.
PriorityJobScheduler() - Constructor for class ai.preferred.venom.job.PriorityJobScheduler
Deprecated.
 
ProxyProvider - Interface in ai.preferred.venom
This interface allows the user to define proxies to be used for requests.
put(Request, Response) - Method in interface ai.preferred.venom.storage.FileManager
Puts record into database.
put(Session.Key<T>, T) - Method in class ai.preferred.venom.Session.Builder
Adds a session variable into store.
put(Job) - Method in class ai.preferred.venom.job.AbstractPriorityJobQueue
 
put(Job) - Method in class ai.preferred.venom.job.FIFOJobQueue
 
put(Request, Response) - Method in class ai.preferred.venom.storage.DummyFileManager
 
put(Request, Response) - Method in class ai.preferred.venom.storage.MysqlFileManager
 
put(String) - Static method in class ai.preferred.venom.request.VRequest.Builder
Creates a new instance of builder with method type put.
PUT - ai.preferred.venom.request.Request.Method
PUT method.

R

Record<T> - Interface in ai.preferred.venom.storage
This interface represents only the most basic of a record and the fields that should be retrievable from database.
register(Callback) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
Register any callbacks that will be called when a page has been fetched.
register(Pattern, Handler) - Method in class ai.preferred.venom.UrlRouter
Adds a url pattern, and the handler to be used.
register(Pattern, Handler, Validator) - Method in class ai.preferred.venom.UrlRouter
Adds a url pattern, and the handler to be used.
register(Pattern, Validator) - Method in class ai.preferred.venom.UrlRouter
Adds a url pattern, and the handler to be used.
releaseResources() - Method in class ai.preferred.venom.fetcher.AsyncResponseConsumer
 
remainingCapacity() - Method in class ai.preferred.venom.job.AbstractJobQueue
 
removeAttribute(String) - Method in class ai.preferred.venom.socks.SocksIOSession
 
removeHeader(String) - Method in class ai.preferred.venom.request.VRequest.Builder
Remove a header from this request.
removeHeaders() - Method in class ai.preferred.venom.request.VRequest.Builder
Remove all headers from this request.
removeProxy() - Method in class ai.preferred.venom.request.CrawlerRequest
Remove the proxy from this request.
Request - Interface in ai.preferred.venom.request
 
Request.Method - Enum in ai.preferred.venom.request
The method of the request to be made.
Response - Interface in ai.preferred.venom.response
 
ResponseDecompressor - Class in ai.preferred.venom.utils
Modified from: org.apache.http.client.protocol.ResponseContentEncoding.
ResponseDecompressor() - Constructor for class ai.preferred.venom.utils.ResponseDecompressor
Constructs a decompressor.
Retrievable - Interface in ai.preferred.venom.response
This interface represents that the response can be/ has been stored.

S

Scheduler - Class in ai.preferred.venom.job
This interface represents only the adding part a scheduler.
Scheduler(BlockingQueue<Job>) - Constructor for class ai.preferred.venom.job.Scheduler
Constructs an instance of Scheduler.
Session - Class in ai.preferred.venom
 
SESSION_KEY - Static variable in class ai.preferred.venom.socks.SocksIOSession
SOCKS session key.
Session.Builder - Class in ai.preferred.venom
Builder for Session.
Session.Key<T> - Class in ai.preferred.venom
A class representing the key for a session.
setAcknowledge() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
Set the acknowledge time to current nano time.
setAttribute(String, Object) - Method in class ai.preferred.venom.socks.SocksIOSession
 
setBody(String) - Method in class ai.preferred.venom.request.VRequest.Builder
Sets the request body to be used.
setBufferStatus(SessionBufferStatus) - Method in class ai.preferred.venom.socks.SocksIOSession
 
setComplete() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
Set the complete time to current nano time.
setConnectionRequestTimeout(int) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
The timeout in milliseconds used when requesting a connection from the connection manager.
setConnectTimeout(int) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
Determines the timeout in milliseconds until a connection is established.
setContentType(ContentType) - Method in class ai.preferred.venom.storage.StorageRecord.Builder
Sets the response content type for the record.
setDateCreated(long) - Method in class ai.preferred.venom.storage.StorageRecord.Builder
Sets the date the record was created.
setEvent(int) - Method in class ai.preferred.venom.socks.SocksIOSession
 
setEventMask(int) - Method in class ai.preferred.venom.socks.SocksIOSession
 
setFetcher(Fetcher) - Method in class ai.preferred.venom.Crawler.Builder
Sets the Fetcher to be used, if not set, default will be chosen.
setFileManager(FileManager<?>) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
Sets the FileManager to be used.
setHandlerRouter(HandlerRouter) - Method in class ai.preferred.venom.Crawler.Builder
Sets HandlerRouter to be used.
setHeaders(Map<String, String>) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
Sets the headers to be used when fetching items.
setHeaders(Map<String, String>) - Method in class ai.preferred.venom.fetcher.StorageFetcher.Builder
Sets the headers to be used when fetching items.
setJobAttribute(JobAttribute) - Method in class ai.preferred.venom.job.Job
Adds or replace the current job attribute if the class of attribute is already present in the map.
setJobQueue(BlockingQueue<Job>) - Method in class ai.preferred.venom.Crawler.Builder
Sets the JobQueue to be used, if not set, default will be chosen.
setMaxConnections(int) - Method in class ai.preferred.venom.Crawler.Builder
The number of concurrent connections allowed out of the client.
setMaxConnections(int) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
Sets the maximum allowable connections at an instance.
setMaxRouteConnections(int) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
Sets the maximum allowable connections at an instance for a particular route (host).
setMaxTries(int) - Method in class ai.preferred.venom.Crawler.Builder
Sets number of times to retry for a request.
setMD5(String) - Method in class ai.preferred.venom.storage.StorageRecord.Builder
Sets the md5 hash of the response content for the record.
setName(String) - Method in class ai.preferred.venom.Crawler.Builder
Sets the name for crawler thread.
setNumIoThreads(int) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
Number of httpclient dispatcher threads.
setParallelism(int) - Method in class ai.preferred.venom.Crawler.Builder
Sets the parallelism level.
setPropRetainProxy(double) - Method in class ai.preferred.venom.Crawler.Builder
Sets the proportion of max tries where a specified proxy, if specified will be used.
setProxy(HttpHost) - Method in class ai.preferred.venom.request.HttpFetcherRequest
Sets proxy to be used for this request.
setProxy(HttpHost) - Method in class ai.preferred.venom.request.VRequest.Builder
Sets the proxy to be used, this will override the proxy selected in Fetcher for this request.
setProxyProvider(ProxyProvider) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
Sets the ProxyProvider to be used.
setRedirectStrategy(RedirectStrategy) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
Sets the redirection strategy for a response received by the fetcher.
setRequestBody(Map<String, String>) - Method in class ai.preferred.venom.storage.StorageRecord.Builder
Sets the request body for the record.
setRequestHeaders(Map<String, String>) - Method in class ai.preferred.venom.storage.StorageRecord.Builder
Sets the request headers for the record.
setRequestMethod(Request.Method) - Method in class ai.preferred.venom.storage.StorageRecord.Builder
Sets the request method for the record.
setResponseContent(byte[]) - Method in class ai.preferred.venom.storage.StorageRecord.Builder
Sets the response content for the record.
setResponseHeaders(Header[]) - Method in class ai.preferred.venom.storage.StorageRecord.Builder
Sets the response headers for the record.
setScheduler(BlockingQueue<Job>) - Method in class ai.preferred.venom.Crawler.Builder
Deprecated.
setSession(Session) - Method in class ai.preferred.venom.Crawler.Builder
Sets the Session to be used, if not set, defaults to Session.EMPTY_SESSION.
setSize(int) - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
Set the size of the response in bytes.
setSleepScheduler(SleepScheduler) - Method in class ai.preferred.venom.Crawler.Builder
Sets the SleepScheduler to be used, if not set, default will be chosen.
setSleepScheduler(SleepScheduler) - Method in class ai.preferred.venom.request.VRequest.Builder
Sets the sleep scheduler to be used, this will override the sleep scheduler defined in Crawler for this request.
setSocketTimeout(int) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
Defines the socket timeout (SO_TIMEOUT) in milliseconds, which is the timeout for waiting for data or, put differently, a maximum period inactivity between two consecutive data packets).
setSocketTimeout(int) - Method in class ai.preferred.venom.socks.SocksIOSession
 
setSslContext(SSLContext) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
Sets the ssl context for an encrypted response.
setStart() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
Set the start time to current nano time.
setStatusCode(int) - Method in class ai.preferred.venom.storage.StorageRecord.Builder
Sets the response status code for the record.
setStopCodes(@javax.validation.constraints.NotNull int...) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
Set a list of stop code that will interrupt crawling.
setThreadFactory(ThreadFactory) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
Set the thread factory that creates the httpclient dispatcher threads.
setUrl(String) - Method in class ai.preferred.venom.request.VRequest.Builder
Sets the url to be fetched.
setUrl(String) - Method in class ai.preferred.venom.storage.StorageRecord.Builder
Sets the url for the record.
setUserAgent(UserAgent) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
Sets the UserAgent to be used, if not set, default will be chosen.
setValidator(Validator) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
Sets the Validator to be used.
setValidator(Validator) - Method in class ai.preferred.venom.fetcher.StorageFetcher.Builder
Sets the Validator to be used.
setValidator(Validator...) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
Sets the multiple validators to be used.
setValidator(Validator...) - Method in class ai.preferred.venom.fetcher.StorageFetcher.Builder
Sets the multiple validators to be used.
setValidatorRouter(ValidatorRouter) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
Sets ValidatorRouter to be used.
setWorkerManager(WorkerManager) - Method in class ai.preferred.venom.Crawler.Builder
Sets the WorkerManager to be used, if not set, default will be chosen.
shutdown() - Method in class ai.preferred.venom.socks.SocksIOSession
 
shutdown() - Method in class ai.preferred.venom.utils.InlineExecutorService
 
shutdownNow() - Method in class ai.preferred.venom.utils.InlineExecutorService
 
singleton(Session.Key<T>, T) - Static method in class ai.preferred.venom.Session
Constructs a singleton session.
size() - Method in class ai.preferred.venom.job.AbstractJobQueue
 
SleepScheduler - Class in ai.preferred.venom
 
SleepScheduler(long) - Constructor for class ai.preferred.venom.SleepScheduler
Constructs a sleep scheduler with fix sleep time.
SleepScheduler(long, long) - Constructor for class ai.preferred.venom.SleepScheduler
Constructs a sleep scheduler with range of sleep time.
SocksConnectingIOReactor - Class in ai.preferred.venom.socks
This IOReactor makes sure that the supplied IOEventDispatch is decorated with SocksIOEventDispatch.
SocksConnectingIOReactor() - Constructor for class ai.preferred.venom.socks.SocksConnectingIOReactor
Creates an instance of SocksConnectingIOReactor with default configuration.
SocksConnectingIOReactor(IOReactorConfig) - Constructor for class ai.preferred.venom.socks.SocksConnectingIOReactor
Creates an instance of SocksConnectingIOReactor with the given configuration.
SocksConnectingIOReactor(IOReactorConfig, ThreadFactory) - Constructor for class ai.preferred.venom.socks.SocksConnectingIOReactor
Creates an instance of SocksConnectingIOReactor with the given configuration.
SocksHttpRoutePlanner - Class in ai.preferred.venom.socks
This route planners ensures that the connection to https server via socks proxy works.
SocksHttpRoutePlanner(HttpRoutePlanner) - Constructor for class ai.preferred.venom.socks.SocksHttpRoutePlanner
Decorates HttpRoutePlanner.
SocksIOEventDispatch - Class in ai.preferred.venom.socks
This class wraps and handles IO dispatch related to SocksIOSession.
SocksIOEventDispatch(IOEventDispatch) - Constructor for class ai.preferred.venom.socks.SocksIOEventDispatch
Decorates IOEventDispatch.
SocksIOSession - Class in ai.preferred.venom.socks
The class establishes Socks4a connection and delegates the interface calls to a decorated IOSession.
SocksIOSession(IOSession) - Constructor for class ai.preferred.venom.socks.SocksIOSession
Decorates IOSession, sets default user ID for a SOCKS proxy.
SocksIOSession(IOSession, String) - Constructor for class ai.preferred.venom.socks.SocksIOSession
Decorates IOSession, allows to specify user ID.
SocksIOSessionStrategy - Class in ai.preferred.venom.socks
Socks + TSL/SSL layering strategy.
SocksIOSessionStrategy(SSLIOSessionStrategy) - Constructor for class ai.preferred.venom.socks.SocksIOSessionStrategy
 
start() - Method in class ai.preferred.venom.Crawler
Starts the crawler by starting a new thread to poll for jobs.
start() - Method in class ai.preferred.venom.fetcher.AsyncFetcher
 
start() - Method in interface ai.preferred.venom.fetcher.Fetcher
Fetcher starter.
start() - Method in class ai.preferred.venom.fetcher.StorageFetcher
 
startAndClose() - Method in class ai.preferred.venom.Crawler
Starts the crawler by starting a new thread to poll for jobs and close it after the jobQueue has reached 0.
StatusOkValidator - Class in ai.preferred.venom.validator
This class determines the validity of a response by its status code.
StatusOkValidator() - Constructor for class ai.preferred.venom.validator.StatusOkValidator
 
STOP - ai.preferred.venom.validator.Validator.Status
Stop processing this job.
StopCodeException - Exception in ai.preferred.venom.fetcher
 
StopCodeException(int, String) - Constructor for exception ai.preferred.venom.fetcher.StopCodeException
Constructs a stop code exception with a message.
StopCodeException(int, String, Throwable) - Constructor for exception ai.preferred.venom.fetcher.StopCodeException
Constructs a stop code exception with a message and a cause.
StopCodeException(int, String, Throwable, boolean, boolean) - Constructor for exception ai.preferred.venom.fetcher.StopCodeException
Constructs a stop code exception with a message and a cause.
StopCodeException(int, Throwable) - Constructor for exception ai.preferred.venom.fetcher.StopCodeException
Constructs a stop code exception with a cause.
StorageException - Exception in ai.preferred.venom.storage
 
StorageException(String) - Constructor for exception ai.preferred.venom.storage.StorageException
Constructs a storage exception with a message.
StorageException(String, Throwable) - Constructor for exception ai.preferred.venom.storage.StorageException
Constructs a storage exception with a message and a cause.
StorageException(Throwable) - Constructor for exception ai.preferred.venom.storage.StorageException
Constructs a storage exception with a cause.
StorageFetcher - Class in ai.preferred.venom.fetcher
This class holds the implementation to provide how items are fetched from a database, to validate the item and to store it if specified.
StorageFetcher.Builder - Class in ai.preferred.venom.fetcher
A builder for StorageFetcher class.
StorageFetcherRequest - Class in ai.preferred.venom.request
 
StorageFetcherRequest(Request) - Constructor for class ai.preferred.venom.request.StorageFetcherRequest
Constructs an instance of StorageFetcherRequest.
StorageRecord<T> - Class in ai.preferred.venom.storage
This class implements a default storage record.
StorageRecord.Builder<T> - Class in ai.preferred.venom.storage
A builder for StorageRecord class.
StorageResponse - Class in ai.preferred.venom.response
 
StorageResponse(Record<?>, String) - Constructor for class ai.preferred.venom.response.StorageResponse
Constructs a base response.
StorageUtil - Class in ai.preferred.venom.storage
A utility for storage needs.
submit(Runnable) - Method in interface ai.preferred.venom.Worker
Submits a Runnable task for execution and returns a Future representing that task.
submit(Runnable, T) - Method in interface ai.preferred.venom.Worker
Submits a Runnable task for execution and returns a Future representing that task.
submit(Callable<T>) - Method in interface ai.preferred.venom.Worker
Submits a value-returning task for execution and returns a Future representing the pending results of the task.

T

take() - Method in class ai.preferred.venom.job.AbstractJobQueue
 
ThreadedWorkerManager - Class in ai.preferred.venom
 
ThreadedWorkerManager() - Constructor for class ai.preferred.venom.ThreadedWorkerManager
Constructs a threaded worker manager with a specified executor.
ThreadedWorkerManager(ExecutorService) - Constructor for class ai.preferred.venom.ThreadedWorkerManager
Constructs a threaded worker manager with a specified executor.
ThreadedWorkerManager.AbstractManagedBlockingWorker - Class in ai.preferred.venom
This abstract class exposes the methods to allow submitting tasks for multithreading and implements inline blocking method.
timeout(IOSession) - Method in class ai.preferred.venom.socks.SocksIOEventDispatch
 

U

Unwrappable - Interface in ai.preferred.venom.request
This interface represents that the request can be unwrapped.
Unwrappable - Interface in ai.preferred.venom.response
This interface represents that the response can be unwrapped.
unwrapRequest(Request) - Static method in interface ai.preferred.venom.request.Unwrappable
Unwrap all wrapped request to an instance of base request.
upgrade(HttpHost, IOSession) - Method in class ai.preferred.venom.socks.SocksIOSessionStrategy
 
UrlRouter - Class in ai.preferred.venom
This class provides an implementation to select a handler based on the url from which they were fetched.
UrlRouter() - Constructor for class ai.preferred.venom.UrlRouter
Constructs a url router without default handler.
UrlRouter(Handler) - Constructor for class ai.preferred.venom.UrlRouter
Constructs a url router with default handler.
UrlUtil - Class in ai.preferred.venom.utils
A utility for managing URLs.
UserAgent - Interface in ai.preferred.venom.uagent
This interface allows the user to define a user agent string.

V

VALID - ai.preferred.venom.validator.Validator.Status
The response is valid.
ValidationException - Exception in ai.preferred.venom.fetcher
 
ValidationException(Validator.Status, Response, String) - Constructor for exception ai.preferred.venom.fetcher.ValidationException
Constructs a validation exception with a message.
ValidationException(Validator.Status, Response, String, Throwable) - Constructor for exception ai.preferred.venom.fetcher.ValidationException
 
ValidationException(Validator.Status, Response, String, Throwable, boolean, boolean) - Constructor for exception ai.preferred.venom.fetcher.ValidationException
 
ValidationException(Validator.Status, Response, Throwable) - Constructor for exception ai.preferred.venom.fetcher.ValidationException
 
Validator - Interface in ai.preferred.venom.validator
 
Validator.Status - Enum in ai.preferred.venom.validator
The allowed return status of validation.
ValidatorRouter - Interface in ai.preferred.venom
This interface allows the user to map request to handler.
valueOf(String) - Static method in enum ai.preferred.venom.job.Priority
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum ai.preferred.venom.request.Request.Method
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum ai.preferred.venom.validator.Validator.Status
Returns the enum constant of this type with the specified name.
values() - Static method in enum ai.preferred.venom.job.Priority
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum ai.preferred.venom.request.Request.Method
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum ai.preferred.venom.validator.Validator.Status
Returns an array containing the constants of this enum type, in the order they are declared.
VRequest - Class in ai.preferred.venom.request
An implementation of HTTP request.
VRequest(VRequest.Builder<?>) - Constructor for class ai.preferred.venom.request.VRequest
Constructs an instance of venom request.
VRequest(String) - Constructor for class ai.preferred.venom.request.VRequest
Constructs an instance of venom request.
VRequest(String, Map<String, String>) - Constructor for class ai.preferred.venom.request.VRequest
Constructs an instance of venom request.
VRequest.Builder<T extends VRequest.Builder<T>> - Class in ai.preferred.venom.request
A builder for VRequest class.
VResponse - Class in ai.preferred.venom.response
 
VResponse(Response) - Constructor for class ai.preferred.venom.response.VResponse
Constructs a VResponse.

W

Worker - Interface in ai.preferred.venom
 
WorkerManager - Interface in ai.preferred.venom
 
A B C D E F G H I J K L M N O P R S T U V W 
All Classes All Packages