A B C D E F G H I J K L M N O P R S T U V W
All Classes All Packages
All Classes All Packages
All Classes All Packages
A
- AbstractJobQueue - Class in ai.preferred.venom.job
- AbstractJobQueue(BlockingQueue<Job>) - Constructor for class ai.preferred.venom.job.AbstractJobQueue
-
Constructs an instance of AbstractJobQueue.
- AbstractManagedBlockingWorker() - Constructor for class ai.preferred.venom.ThreadedWorkerManager.AbstractManagedBlockingWorker
- AbstractPriorityJobQueue - Class in ai.preferred.venom.job
- AbstractPriorityJobQueue() - Constructor for class ai.preferred.venom.job.AbstractPriorityJobQueue
-
Constructs an instance of AbstractJobQueue.
- add(Request) - Method in class ai.preferred.venom.job.Scheduler
-
Adds a request to the queue.
- add(Request, JobAttribute...) - Method in class ai.preferred.venom.job.Scheduler
-
Adds a request to the queue.
- add(Request, Handler, JobAttribute...) - Method in class ai.preferred.venom.job.Scheduler
-
Adds a request to the queue.
- add(Request, Handler, Priority) - Method in class ai.preferred.venom.job.Scheduler
-
Deprecated.
- add(Request, Handler, Priority, Priority) - Method in class ai.preferred.venom.job.Scheduler
-
Deprecated.
- add(Request, Priority) - Method in class ai.preferred.venom.job.Scheduler
-
Deprecated.
- add(Request, Priority, Priority) - Method in class ai.preferred.venom.job.Scheduler
-
Deprecated.
- add(Request, Handler) - Method in class ai.preferred.venom.job.Scheduler
-
Adds a request to the queue.
- addHeader(String, String) - Method in class ai.preferred.venom.request.VRequest.Builder
-
Adds header to be used
- addHeaders(Map<String, String>) - Method in class ai.preferred.venom.request.VRequest.Builder
-
Add headers to be used
- ai.preferred.venom - package ai.preferred.venom
- ai.preferred.venom.fetcher - package ai.preferred.venom.fetcher
- ai.preferred.venom.job - package ai.preferred.venom.job
- ai.preferred.venom.request - package ai.preferred.venom.request
- ai.preferred.venom.response - package ai.preferred.venom.response
- ai.preferred.venom.socks - package ai.preferred.venom.socks
- ai.preferred.venom.storage - package ai.preferred.venom.storage
- ai.preferred.venom.uagent - package ai.preferred.venom.uagent
- ai.preferred.venom.utils - package ai.preferred.venom.utils
- ai.preferred.venom.validator - package ai.preferred.venom.validator
- ALWAYS_VALID - Static variable in interface ai.preferred.venom.validator.Validator
-
A validator that always return valid.
- AsyncFetcher - Class in ai.preferred.venom.fetcher
-
This class holds the implementation to provide how items are fetched from the web, to validate the item and to store it if specified.
- AsyncFetcher.Builder - Class in ai.preferred.venom.fetcher
-
A builder for async fetcher class.
- AsyncResponseConsumer - Class in ai.preferred.venom.fetcher
-
On top of the abstract class, this class handles the parsing of a response from the web service.
- awaitTermination(long, TimeUnit) - Method in class ai.preferred.venom.utils.InlineExecutorService
B
- BaseResponse - Class in ai.preferred.venom.response
- BaseResponse(int, String, byte[], ContentType, Header[], HttpHost) - Constructor for class ai.preferred.venom.response.BaseResponse
-
Constructs a base response.
- build() - Method in class ai.preferred.venom.Crawler.Builder
-
Builds the crawler with the options specified.
- build() - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Builds the fetcher with the options specified.
- build() - Method in class ai.preferred.venom.fetcher.StorageFetcher.Builder
-
Builds the fetcher with the options specified.
- build() - Method in class ai.preferred.venom.request.VRequest.Builder
-
Builds the request with the options specified.
- build() - Method in class ai.preferred.venom.Session.Builder
-
Create a new instance of session.
- build() - Method in class ai.preferred.venom.storage.StorageRecord.Builder
-
Builds the storage record with the details specified.
- build(Request.Method, String) - Static method in class ai.preferred.venom.request.VRequest
-
Create a new instance of builder with a method and url.
- buildDefault() - Static method in class ai.preferred.venom.Crawler
-
Builds a new default instance of Crawler.
- buildDefault() - Static method in class ai.preferred.venom.fetcher.AsyncFetcher
-
Create an instance of AsyncFetcher with default options.
- builder() - Static method in class ai.preferred.venom.Crawler
-
Creates a new instance of Builder.
- builder() - Static method in class ai.preferred.venom.fetcher.AsyncFetcher
-
Create an instance of builder.
- builder() - Static method in class ai.preferred.venom.Session
-
Create a new instance of builder.
- builder(FileManager<?>) - Static method in class ai.preferred.venom.fetcher.StorageFetcher
-
Create an instance of builder.
- builder(T) - Static method in class ai.preferred.venom.storage.StorageRecord
-
Create an instance of builder.
- Builder() - Constructor for class ai.preferred.venom.Session.Builder
- Builder(Request.Method, String) - Constructor for class ai.preferred.venom.request.VRequest.Builder
-
Constructs an instance of builder.
- buildResult(HttpContext) - Method in class ai.preferred.venom.fetcher.AsyncResponseConsumer
C
- Callback - Interface in ai.preferred.venom.fetcher
- cancelled(Request) - Method in interface ai.preferred.venom.fetcher.Callback
-
Method to be call upon cancellation of request.
- cancelled(Request) - Method in class ai.preferred.venom.storage.MysqlFileManager.CompletedThreadedCallback
- cancelled(Request) - Method in class ai.preferred.venom.storage.FileManagerCallback
- channel() - Method in class ai.preferred.venom.socks.SocksIOSession
- clearEvent(int) - Method in class ai.preferred.venom.socks.SocksIOSession
- close() - Method in class ai.preferred.venom.Crawler
- close() - Method in class ai.preferred.venom.fetcher.AsyncFetcher
- close() - Method in class ai.preferred.venom.fetcher.StorageFetcher
- close() - Method in class ai.preferred.venom.socks.SocksIOSession
- close() - Method in class ai.preferred.venom.storage.DummyFileManager
- close() - Method in class ai.preferred.venom.storage.MysqlFileManager
- close() - Method in class ai.preferred.venom.ThreadedWorkerManager
- compareTo(PriorityJobAttribute) - Method in class ai.preferred.venom.job.PriorityJobAttribute
- completed(Request, Response) - Method in interface ai.preferred.venom.fetcher.Callback
-
Method to be call upon completion of request.
- completed(Request, Response) - Method in class ai.preferred.venom.storage.MysqlFileManager.CompletedThreadedCallback
- completed(Request, Response) - Method in class ai.preferred.venom.storage.FileManagerCallback
- connected(IOSession) - Method in class ai.preferred.venom.socks.SocksIOEventDispatch
- Crawler - Class in ai.preferred.venom
-
This class handles the coordination between classes during the pre and post fetching of a page such as executing threads, calling to fetcher and manipulating the priority of a scheduled request.
- Crawler.Builder - Class in ai.preferred.venom
-
A builder for crawler class.
- CrawlerRequest - Class in ai.preferred.venom.request
-
This class allows the removal of proxy from request.
- CrawlerRequest(Request) - Constructor for class ai.preferred.venom.request.CrawlerRequest
-
Constructs an instance of crawler request with an underlying request.
D
- decompress(HttpResponse) - Method in class ai.preferred.venom.utils.ResponseDecompressor
-
Decompress http response.
- DEFAULT - Static variable in enum ai.preferred.venom.job.Priority
-
The default starting priority for a job.
- DEFAULT_CHARSET - Static variable in class ai.preferred.venom.response.VResponse
-
The default charset to be used to decode response.
- DefaultUserAgent - Class in ai.preferred.venom.uagent
- DefaultUserAgent() - Constructor for class ai.preferred.venom.uagent.DefaultUserAgent
- delete(String) - Static method in class ai.preferred.venom.request.VRequest.Builder
-
Creates a new instance of builder with method type delete.
- DELETE - ai.preferred.venom.request.Request.Method
-
DELETE method.
- determineRoute(HttpHost, HttpRequest, HttpContext) - Method in class ai.preferred.venom.socks.SocksHttpRoutePlanner
- disableCompression() - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Disables request for compress pages and to decompress pages after it is fetched.
- disableCookies() - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Disables cookie storage.
- disconnected(IOSession) - Method in class ai.preferred.venom.socks.SocksIOEventDispatch
- downgrade() - Method in enum ai.preferred.venom.job.Priority
-
Returns the priority one level below the current priority if priority is higher than the default floor or the lowest available priority.
- downgrade(Priority) - Method in enum ai.preferred.venom.job.Priority
-
Returns the priority one level below the current priority if priority is higher than the specified floor or the lowest available priority.
- drainTo(Collection<? super Job>) - Method in class ai.preferred.venom.job.AbstractJobQueue
- drainTo(Collection<? super Job>, int) - Method in class ai.preferred.venom.job.AbstractJobQueue
- DummyFileManager - Class in ai.preferred.venom.storage
-
This class implements a FileManager that writes response content to a file on the file system.
- DummyFileManager(File) - Constructor for class ai.preferred.venom.storage.DummyFileManager
-
Constructs an instance of DummyFileManager.
- DummyFileManager(String) - Constructor for class ai.preferred.venom.storage.DummyFileManager
-
Constructs an instance of DummyFileManager.
E
- EMPTY_CALLBACK - Static variable in interface ai.preferred.venom.fetcher.Callback
-
An instance of empty callback.
- EMPTY_SESSION - Static variable in class ai.preferred.venom.Session
-
An instance of an empty session.
- EmptyContentValidator - Class in ai.preferred.venom.validator
-
This class determines the validity of a response by its content length.
- EmptyContentValidator() - Constructor for class ai.preferred.venom.validator.EmptyContentValidator
- enableSocksProxy() - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Enables SOCKS protocol for proxies (socks://).
- execute(Runnable) - Method in class ai.preferred.venom.utils.InlineExecutorService
- execute(IOEventDispatch) - Method in class ai.preferred.venom.socks.SocksConnectingIOReactor
- executeBlockingIO(Runnable) - Method in class ai.preferred.venom.ThreadedWorkerManager.AbstractManagedBlockingWorker
- executeBlockingIO(Runnable) - Method in interface ai.preferred.venom.Worker
-
Performs the given task inline, and increase available threads in the pool by one for the execution of other tasks.
F
- failed(Request, Exception) - Method in interface ai.preferred.venom.fetcher.Callback
-
Method to be call upon failure of request.
- failed(Request, Exception) - Method in class ai.preferred.venom.storage.MysqlFileManager.CompletedThreadedCallback
- failed(Request, Exception) - Method in class ai.preferred.venom.storage.FileManagerCallback
- FatalHandlerException - Exception in ai.preferred.venom
-
This class defines fatal runtime exception for
Handler
. - FatalHandlerException() - Constructor for exception ai.preferred.venom.FatalHandlerException
-
Constructs a new fatal handler exception with
null
as its detail message. - FatalHandlerException(String) - Constructor for exception ai.preferred.venom.FatalHandlerException
-
Constructs a new fatal handler exception with the specified detail message.
- FatalHandlerException(String, Throwable) - Constructor for exception ai.preferred.venom.FatalHandlerException
-
Constructs a new fatal handler exception with the specified detail message and cause.
- FatalHandlerException(Throwable) - Constructor for exception ai.preferred.venom.FatalHandlerException
-
Constructs a new fatal handler exception with the specified cause and a detail message of
(cause==null ? null : cause.toString())
(which typically contains the class and detail message ofcause
). - fetch(Request) - Method in interface ai.preferred.venom.fetcher.Fetcher
-
Fetch the desired HTTP page given in
Request
. - fetch(Request, Callback) - Method in interface ai.preferred.venom.fetcher.Fetcher
-
Fetch the desired HTTP page given in
Request
. - fetch(Request) - Method in class ai.preferred.venom.fetcher.AsyncFetcher
- fetch(Request) - Method in class ai.preferred.venom.fetcher.StorageFetcher
- fetch(Request, Callback) - Method in class ai.preferred.venom.fetcher.AsyncFetcher
- fetch(Request, Callback) - Method in class ai.preferred.venom.fetcher.StorageFetcher
- Fetcher - Interface in ai.preferred.venom.fetcher
-
This interface represents only the most basic of fetching a request.
- FIFOJobQueue - Class in ai.preferred.venom.job
-
This class provides and implementation of scheduler with a first in first out queue.
- FIFOJobQueue() - Constructor for class ai.preferred.venom.job.FIFOJobQueue
-
Constructs an instance of FIFOJobQueue.
- FIFOQueueScheduler - Class in ai.preferred.venom.job
-
Deprecated.
- FIFOQueueScheduler() - Constructor for class ai.preferred.venom.job.FIFOQueueScheduler
-
Deprecated.
- FileManager<T> - Interface in ai.preferred.venom.storage
-
This interface represents the basic functions a FileManager should have.
- FileManagerCallback - Class in ai.preferred.venom.storage
-
This class implements the default callback for file managers.
- FileManagerCallback(FileManager<?>) - Constructor for class ai.preferred.venom.storage.FileManagerCallback
-
Constructs an instance of file manager callback.
- fillInStackTrace() - Method in exception ai.preferred.venom.fetcher.StopCodeException
- fillInStackTrace() - Method in exception ai.preferred.venom.fetcher.ValidationException
- FLOOR - Static variable in enum ai.preferred.venom.job.Priority
-
The default lowest priority for a job.
G
- get() - Method in class ai.preferred.venom.uagent.DefaultUserAgent
- get() - Method in interface ai.preferred.venom.uagent.UserAgent
-
Gets the user agent for a request.
- get(Request) - Method in interface ai.preferred.venom.ProxyProvider
-
Returns the get proxy from the list.
- get(Request) - Method in interface ai.preferred.venom.storage.FileManager
-
Returns latest record matching request.
- get(Session.Key<T>) - Method in class ai.preferred.venom.Session
-
Returns the session variable from the store.
- get(Request) - Method in class ai.preferred.venom.storage.DummyFileManager
- get(Request) - Method in class ai.preferred.venom.storage.MysqlFileManager
- get(Integer) - Method in class ai.preferred.venom.storage.MysqlFileManager
- get(Object) - Method in class ai.preferred.venom.storage.DummyFileManager
- get(String) - Static method in class ai.preferred.venom.request.VRequest.Builder
-
Creates a new instance of builder with method type get.
- get(T) - Method in interface ai.preferred.venom.storage.FileManager
-
Returns record by the internal record id.
- GET - ai.preferred.venom.request.Request.Method
-
GET method.
- getAcknowledge() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
-
Get the acknowledge time of the request.
- getAttribute(String) - Method in class ai.preferred.venom.socks.SocksIOSession
- getBaseUrl() - Method in class ai.preferred.venom.response.BaseResponse
- getBaseUrl() - Method in interface ai.preferred.venom.response.Response
-
Deprecated.
- getBaseUrl() - Method in class ai.preferred.venom.response.StorageResponse
- getBaseUrl() - Method in class ai.preferred.venom.response.VResponse
- getBaseUrl(Request) - Static method in class ai.preferred.venom.utils.UrlUtil
-
Deprecated.
- getBody() - Method in class ai.preferred.venom.request.CrawlerRequest
- getBody() - Method in class ai.preferred.venom.request.HttpFetcherRequest
- getBody() - Method in interface ai.preferred.venom.request.Request
-
Returns the request body of the request or null if none specified.
- getBody() - Method in class ai.preferred.venom.request.StorageFetcherRequest
- getBody() - Method in class ai.preferred.venom.request.VRequest
- getCallback() - Method in class ai.preferred.venom.storage.DummyFileManager
- getCallback() - Method in interface ai.preferred.venom.storage.FileManager
-
Get callback upon completion of request.
- getCallback() - Method in class ai.preferred.venom.storage.MysqlFileManager
- getComplete() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
-
Get the complete time of the request.
- getContent() - Method in class ai.preferred.venom.response.BaseResponse
- getContent() - Method in interface ai.preferred.venom.response.Response
-
Returns raw content of the response.
- getContent() - Method in class ai.preferred.venom.response.StorageResponse
- getContent() - Method in class ai.preferred.venom.response.VResponse
- getContentType() - Method in class ai.preferred.venom.response.BaseResponse
- getContentType() - Method in interface ai.preferred.venom.response.Response
-
Returns the content type of the content fetched.
- getContentType() - Method in class ai.preferred.venom.response.StorageResponse
- getContentType() - Method in class ai.preferred.venom.response.VResponse
- getContentType() - Method in interface ai.preferred.venom.storage.Record
- getContentType() - Method in class ai.preferred.venom.storage.StorageRecord
- getContentType(HttpEntity) - Method in class ai.preferred.venom.fetcher.AsyncResponseConsumer
- getDateCreated() - Method in interface ai.preferred.venom.storage.Record
- getDateCreated() - Method in class ai.preferred.venom.storage.StorageRecord
- getDiagnostics() - Method in class ai.preferred.venom.request.HttpFetcherRequest
-
Get diagnostic information for this request.
- getEventMask() - Method in class ai.preferred.venom.socks.SocksIOSession
- getFileExtension(Response) - Static method in class ai.preferred.venom.storage.StorageUtil
-
Get file extension from a response.
- getFileExtension(String) - Static method in class ai.preferred.venom.storage.StorageUtil
-
Get file extension from mime type name.
- getHandler() - Method in class ai.preferred.venom.job.Job
-
Get the handler to handle the response of the job.
- getHandler(Request) - Method in interface ai.preferred.venom.HandlerRouter
-
Returns the handler to be used for a specified request.
- getHandler(Request) - Method in class ai.preferred.venom.UrlRouter
- getHeaders() - Method in class ai.preferred.venom.request.CrawlerRequest
- getHeaders() - Method in class ai.preferred.venom.request.HttpFetcherRequest
- getHeaders() - Method in interface ai.preferred.venom.request.Request
-
Returns the headers set for the request.
- getHeaders() - Method in class ai.preferred.venom.request.StorageFetcherRequest
- getHeaders() - Method in class ai.preferred.venom.request.VRequest
- getHeaders() - Method in class ai.preferred.venom.response.BaseResponse
- getHeaders() - Method in interface ai.preferred.venom.response.Response
-
Returns the headers that were used to trigger this response.
- getHeaders() - Method in class ai.preferred.venom.response.StorageResponse
- getHeaders() - Method in class ai.preferred.venom.response.VResponse
- getHtml() - Method in class ai.preferred.venom.response.VResponse
-
Returns the html in string format.
- getHtml(Charset) - Method in class ai.preferred.venom.response.VResponse
-
Returns the html in string format.
- getId() - Method in interface ai.preferred.venom.storage.Record
- getId() - Method in class ai.preferred.venom.storage.StorageRecord
- getInner() - Method in class ai.preferred.venom.request.CrawlerRequest
- getInner() - Method in class ai.preferred.venom.request.HttpFetcherRequest
- getInner() - Method in class ai.preferred.venom.request.StorageFetcherRequest
- getInner() - Method in interface ai.preferred.venom.request.Unwrappable
-
Returns the unwrapped version of this request.
- getInner() - Method in interface ai.preferred.venom.response.Unwrappable
-
Returns the unwrapped version of this request.
- getInner() - Method in class ai.preferred.venom.response.VResponse
- getJobAttribute(Class<T>) - Method in class ai.preferred.venom.job.Job
-
Get the job attribute for a specific attribute class or return
null
if not found. - getJsoup() - Method in class ai.preferred.venom.response.VResponse
-
Returns a jsoup document of this response.
- getJsoup(Charset) - Method in class ai.preferred.venom.response.VResponse
-
Returns a jsoup document of this response.
- getLatency() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
-
Get the latency between sending of the request and the first response.
- getLocalAddress() - Method in class ai.preferred.venom.socks.SocksIOSession
- getMD5() - Method in class ai.preferred.venom.storage.StorageRecord
-
Get md5 hash of the response content.
- getMethod() - Method in class ai.preferred.venom.request.CrawlerRequest
- getMethod() - Method in class ai.preferred.venom.request.HttpFetcherRequest
- getMethod() - Method in interface ai.preferred.venom.request.Request
-
Returns the method type of the request.
- getMethod() - Method in class ai.preferred.venom.request.StorageFetcherRequest
- getMethod() - Method in class ai.preferred.venom.request.VRequest
- getPriority() - Method in class ai.preferred.venom.job.PriorityJobAttribute
-
Get the priority in this attribute.
- getProxy() - Method in class ai.preferred.venom.request.CrawlerRequest
- getProxy() - Method in class ai.preferred.venom.request.HttpFetcherRequest
- getProxy() - Method in interface ai.preferred.venom.request.Request
-
Returns the proxy set to be used for the request or default to fetcher if none specified.
- getProxy() - Method in class ai.preferred.venom.request.StorageFetcherRequest
- getProxy() - Method in class ai.preferred.venom.request.VRequest
- getProxy() - Method in class ai.preferred.venom.response.BaseResponse
- getProxy() - Method in interface ai.preferred.venom.response.Response
-
Returns the proxy that was used to trigger this response.
- getProxy() - Method in class ai.preferred.venom.response.StorageResponse
- getProxy() - Method in class ai.preferred.venom.response.VResponse
- getQueue() - Method in class ai.preferred.venom.job.AbstractJobQueue
-
Get the BlockingQueue backing this scheduler.
- getRecord() - Method in interface ai.preferred.venom.response.Retrievable
-
Returns record archive of this response that has been insert into a persistent storage.
- getRecord() - Method in class ai.preferred.venom.response.StorageResponse
- getRemoteAddress() - Method in class ai.preferred.venom.socks.SocksIOSession
- getRequest() - Method in class ai.preferred.venom.job.Job
-
Get the request of this job.
- getRequestBody() - Method in interface ai.preferred.venom.storage.Record
- getRequestBody() - Method in class ai.preferred.venom.storage.StorageRecord
- getRequestHeaders() - Method in interface ai.preferred.venom.storage.Record
- getRequestHeaders() - Method in class ai.preferred.venom.storage.StorageRecord
- getRequestMethod() - Method in interface ai.preferred.venom.storage.Record
- getRequestMethod() - Method in class ai.preferred.venom.storage.StorageRecord
- getResponse() - Method in exception ai.preferred.venom.fetcher.ValidationException
-
Get the response validated.
- getResponseContent() - Method in interface ai.preferred.venom.storage.Record
- getResponseContent() - Method in class ai.preferred.venom.storage.StorageRecord
- getResponseHeaders() - Method in interface ai.preferred.venom.storage.Record
- getResponseHeaders() - Method in class ai.preferred.venom.storage.StorageRecord
- getScheduler() - Method in class ai.preferred.venom.Crawler
-
Get the instance of scheduler used.
- getSize() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
-
Get the size the response.
- getSleepScheduler() - Method in class ai.preferred.venom.request.CrawlerRequest
- getSleepScheduler() - Method in class ai.preferred.venom.request.HttpFetcherRequest
- getSleepScheduler() - Method in interface ai.preferred.venom.request.Request
-
Returns information about the amount of sleep before this request is made.
- getSleepScheduler() - Method in class ai.preferred.venom.request.StorageFetcherRequest
- getSleepScheduler() - Method in class ai.preferred.venom.request.VRequest
- getSleepTime() - Method in class ai.preferred.venom.SleepScheduler
-
Get the amount of time to wait specified in this class.
- getSocketTimeout() - Method in class ai.preferred.venom.socks.SocksIOSession
- getSpeed() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
-
Get download speed in bytes per second.
- getStart() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
-
Get the start time of the request.
- getStatus() - Method in exception ai.preferred.venom.fetcher.ValidationException
-
Get the validation status of the response.
- getStatus() - Method in class ai.preferred.venom.socks.SocksIOSession
- getStatusCode() - Method in class ai.preferred.venom.response.BaseResponse
- getStatusCode() - Method in interface ai.preferred.venom.response.Response
-
Returns status code of the response.
- getStatusCode() - Method in class ai.preferred.venom.response.StorageResponse
- getStatusCode() - Method in class ai.preferred.venom.response.VResponse
- getStatusCode() - Method in interface ai.preferred.venom.storage.Record
- getStatusCode() - Method in class ai.preferred.venom.storage.StorageRecord
- getTryCount() - Method in class ai.preferred.venom.job.Job
-
Get attempt number of this job.
- getUrl() - Method in class ai.preferred.venom.request.CrawlerRequest
- getUrl() - Method in class ai.preferred.venom.request.HttpFetcherRequest
- getUrl() - Method in interface ai.preferred.venom.request.Request
-
Returns the url of the request.
- getUrl() - Method in class ai.preferred.venom.request.StorageFetcherRequest
- getUrl() - Method in class ai.preferred.venom.request.VRequest
- getUrl() - Method in class ai.preferred.venom.response.BaseResponse
- getUrl() - Method in interface ai.preferred.venom.response.Response
-
Returns the url used to fetch the response, if the request is redirected, this will be the final requested url.
- getUrl() - Method in class ai.preferred.venom.response.StorageResponse
- getUrl() - Method in class ai.preferred.venom.response.VResponse
- getURL() - Method in interface ai.preferred.venom.storage.Record
- getURL() - Method in class ai.preferred.venom.storage.StorageRecord
- getValidator(Request) - Method in interface ai.preferred.venom.ValidatorRouter
-
Returns the handler to be used for a specified request.
- getValidator(Request) - Method in class ai.preferred.venom.UrlRouter
- getWorker() - Method in class ai.preferred.venom.ThreadedWorkerManager
- getWorker() - Method in interface ai.preferred.venom.WorkerManager
-
Get the result collector in use.
H
- handle(Request, VResponse, Scheduler, Session, Worker) - Method in interface ai.preferred.venom.Handler
-
This function is called when the request is fetched successfully.
- Handler - Interface in ai.preferred.venom
-
This interface represents the method call when the response is makeValidResponse the parsing logic will be up to individual implementations.
- HandlerRouter - Interface in ai.preferred.venom
-
This interface allows the user to map request to handler.
- hasBufferedInput() - Method in class ai.preferred.venom.socks.SocksIOSession
- hasBufferedOutput() - Method in class ai.preferred.venom.socks.SocksIOSession
- head(String) - Static method in class ai.preferred.venom.request.VRequest.Builder
-
Creates a new instance of builder with method type head.
- HEAD - ai.preferred.venom.request.Request.Method
-
HEAD method.
- HIGH - ai.preferred.venom.job.Priority
-
High priority.
- HIGHEST - ai.preferred.venom.job.Priority
-
Highest priority.
- HttpFetcherRequest - Class in ai.preferred.venom.request
- HttpFetcherRequest(Request) - Constructor for class ai.preferred.venom.request.HttpFetcherRequest
-
Constructs an instance of http fetcher request.
- HttpFetcherRequest.Diagnostics - Class in ai.preferred.venom.request
-
This class contains the diagnostic information for this request.
I
- InlineExecutorService - Class in ai.preferred.venom.utils
- InlineExecutorService() - Constructor for class ai.preferred.venom.utils.InlineExecutorService
- inputReady(IOSession) - Method in class ai.preferred.venom.socks.SocksIOEventDispatch
- INSTANCE - Static variable in class ai.preferred.venom.validator.EmptyContentValidator
-
An instance of this validator.
- INSTANCE - Static variable in class ai.preferred.venom.validator.StatusOkValidator
-
An instance of this validator.
- interrupt() - Method in class ai.preferred.venom.Crawler
-
Interrupts crawler, fetcher and worker threads.
- interrupt() - Method in interface ai.preferred.venom.Interruptible
-
Interrupt the underlying mechanisms of the class.
- interrupt() - Method in class ai.preferred.venom.ThreadedWorkerManager
- interruptAndClose() - Method in class ai.preferred.venom.Crawler
-
Interrupts then close this object.
- Interruptible - Interface in ai.preferred.venom
- INVALID_BLOCKED - ai.preferred.venom.validator.Validator.Status
-
The response is invalid as it is blocked.
- INVALID_CONTENT - ai.preferred.venom.validator.Validator.Status
-
The response has invalid content.
- INVALID_STATUS_CODE - ai.preferred.venom.validator.Validator.Status
-
The response is invalid due to the status code.
- isAcknowledged() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
-
Check if request has been started and acknowledged.
- isClosed() - Method in class ai.preferred.venom.socks.SocksIOSession
- isCompleted() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
-
Check if request has been started, acknowledged and completed.
- isEmpty() - Method in class ai.preferred.venom.job.LazyPriorityJobQueue
- isLayeringRequired() - Method in class ai.preferred.venom.socks.SocksIOSessionStrategy
- isShutdown() - Method in class ai.preferred.venom.utils.InlineExecutorService
- isStarted() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
-
Check if request has been started.
- isTerminated() - Method in class ai.preferred.venom.utils.InlineExecutorService
- isValid(Request, Response) - Method in interface ai.preferred.venom.validator.Validator
-
Method will be called when a response need validation.
- isValid(Request, Response) - Method in class ai.preferred.venom.validator.EmptyContentValidator
- isValid(Request, Response) - Method in class ai.preferred.venom.validator.MimeTypeValidator
- isValid(Request, Response) - Method in class ai.preferred.venom.validator.PipelineValidator
- isValid(Request, Response) - Method in class ai.preferred.venom.validator.StatusOkValidator
- iterator() - Method in class ai.preferred.venom.job.AbstractJobQueue
J
- Job - Class in ai.preferred.venom.job
-
This class will be placed in a scheduler for queuing requests.
- Job(Request) - Constructor for class ai.preferred.venom.job.Job
-
Constructs a basic job.
- Job(Request, Handler) - Constructor for class ai.preferred.venom.job.Job
-
Constructs a basic job.
- Job(Request, Handler, JobAttribute...) - Constructor for class ai.preferred.venom.job.Job
-
Constructs a basic job.
- JobAttribute - Interface in ai.preferred.venom.job
-
This interface represents attributes that can be added to jobs to manipulate the crawling process.
K
- Key() - Constructor for class ai.preferred.venom.Session.Key
L
- LazyPriorityJobQueue - Class in ai.preferred.venom.job
-
This class provides and implementation of scheduler with a priority sensitive queue and polls from iterator when queue is empty.
- LazyPriorityJobQueue(Iterator<Request>) - Constructor for class ai.preferred.venom.job.LazyPriorityJobQueue
-
Constructs an instance of lazy scheduler without a default handler.
- LazyPriorityJobQueue(Iterator<Request>, Handler) - Constructor for class ai.preferred.venom.job.LazyPriorityJobQueue
-
Constructs an instance of lazy scheduler with a default handler.
- LazyScheduler - Class in ai.preferred.venom.job
-
Deprecated.
- LazyScheduler(Iterator<Request>) - Constructor for class ai.preferred.venom.job.LazyScheduler
-
Deprecated.Constructs an instance of lazy scheduler without a default handler.
- LazyScheduler(Iterator<Request>, Handler) - Constructor for class ai.preferred.venom.job.LazyScheduler
-
Deprecated.Constructs an instance of lazy scheduler with a default handler.
- LOW - ai.preferred.venom.job.Priority
-
Low priority.
- LOWEST - ai.preferred.venom.job.Priority
-
Lowest priority.
M
- MimeTypeValidator - Class in ai.preferred.venom.validator
-
This class determines the validity of a response by its mime type.
- MimeTypeValidator(String) - Constructor for class ai.preferred.venom.validator.MimeTypeValidator
-
Constructs mime type validator.
- MimeTypeValidator(Pattern) - Constructor for class ai.preferred.venom.validator.MimeTypeValidator
-
Constructs mime type validator.
- MysqlFileManager - Class in ai.preferred.venom.storage
-
This class implements a FileManager that writes response content to a file on the file system and a record in MySQL database pointing to the record and allows retrieving the file using an id or request.
- MysqlFileManager(String, String, String, String, File) - Constructor for class ai.preferred.venom.storage.MysqlFileManager
-
Constructs an instance of MysqlFileManager.
- MysqlFileManager(String, String, String, String, File, int) - Constructor for class ai.preferred.venom.storage.MysqlFileManager
-
Constructs an instance of MysqlFileManager.
- MysqlFileManager(String, String, String, String, String) - Constructor for class ai.preferred.venom.storage.MysqlFileManager
-
Constructs an instance of MysqlFileManager.
- MysqlFileManager.CompletedThreadedCallback - Class in ai.preferred.venom.storage
-
A callback wrapper for to run complete multithreaded.
N
O
- offer(Job) - Method in class ai.preferred.venom.job.AbstractPriorityJobQueue
- offer(Job) - Method in class ai.preferred.venom.job.FIFOJobQueue
- offer(Job, long, TimeUnit) - Method in class ai.preferred.venom.job.AbstractPriorityJobQueue
- offer(Job, long, TimeUnit) - Method in class ai.preferred.venom.job.FIFOJobQueue
- onContentReceived(ContentDecoder, IOControl) - Method in class ai.preferred.venom.fetcher.AsyncResponseConsumer
- onEntityEnclosed(HttpEntity, ContentType) - Method in class ai.preferred.venom.fetcher.AsyncResponseConsumer
- onResponseReceived(HttpResponse) - Method in class ai.preferred.venom.fetcher.AsyncResponseConsumer
- options(String) - Static method in class ai.preferred.venom.request.VRequest.Builder
-
Creates a new instance of builder with method type options.
- OPTIONS - ai.preferred.venom.request.Request.Method
-
OPTIONS method.
- outputReady(IOSession) - Method in class ai.preferred.venom.socks.SocksIOEventDispatch
P
- peek() - Method in class ai.preferred.venom.job.AbstractJobQueue
- PipelineValidator - Class in ai.preferred.venom.validator
- PipelineValidator(Validator...) - Constructor for class ai.preferred.venom.validator.PipelineValidator
-
Constructs pipeline validator.
- PipelineValidator(List<Validator>) - Constructor for class ai.preferred.venom.validator.PipelineValidator
-
Constructs pipeline validator.
- poll() - Method in class ai.preferred.venom.job.FIFOJobQueue
- poll() - Method in class ai.preferred.venom.job.LazyPriorityJobQueue
- poll() - Method in class ai.preferred.venom.job.PriorityJobQueue
- poll(long, TimeUnit) - Method in class ai.preferred.venom.job.FIFOJobQueue
- poll(long, TimeUnit) - Method in class ai.preferred.venom.job.LazyPriorityJobQueue
- poll(long, TimeUnit) - Method in class ai.preferred.venom.job.PriorityJobQueue
- post(String) - Static method in class ai.preferred.venom.request.VRequest.Builder
-
Creates a new instance of builder with method type post.
- POST - ai.preferred.venom.request.Request.Method
-
POST method.
- prepareRetry() - Method in class ai.preferred.venom.job.Job
-
This method is called before the job is scheduled for a retry.
- prepareRetry() - Method in interface ai.preferred.venom.job.JobAttribute
-
This method is called before the job is scheduled for a retry.
- prepareRetry() - Method in class ai.preferred.venom.job.PriorityJobAttribute
- prependHeaders(Map<String, String>) - Method in class ai.preferred.venom.request.HttpFetcherRequest
-
Prepend headers to the current headers.
- prependHeaders(Map<String, String>) - Method in class ai.preferred.venom.request.StorageFetcherRequest
-
Prepend headers to the current headers.
- Priority - Enum in ai.preferred.venom.job
-
Job priorities, list in descending order of priority: HIGHEST, HIGH, NORMAL, LOW, LOWEST.
- PriorityJobAttribute - Class in ai.preferred.venom.job
-
This class provides an implementation of job attribute with comparable priority.
- PriorityJobAttribute() - Constructor for class ai.preferred.venom.job.PriorityJobAttribute
-
Constructs an instance of PriorityJobAttribute.
- PriorityJobAttribute(Priority) - Constructor for class ai.preferred.venom.job.PriorityJobAttribute
-
Constructs an instance of PriorityJobAttribute.
- PriorityJobAttribute(Priority, Priority) - Constructor for class ai.preferred.venom.job.PriorityJobAttribute
-
Constructs an instance of PriorityJobAttribute.
- PriorityJobQueue - Class in ai.preferred.venom.job
-
This class provides an implementation of scheduler with a priority sensitive queue.
- PriorityJobQueue() - Constructor for class ai.preferred.venom.job.PriorityJobQueue
- PriorityJobScheduler - Class in ai.preferred.venom.job
-
Deprecated.
- PriorityJobScheduler() - Constructor for class ai.preferred.venom.job.PriorityJobScheduler
-
Deprecated.
- ProxyProvider - Interface in ai.preferred.venom
-
This interface allows the user to define proxies to be used for requests.
- put(Request, Response) - Method in interface ai.preferred.venom.storage.FileManager
-
Puts record into database.
- put(Session.Key<T>, T) - Method in class ai.preferred.venom.Session.Builder
-
Adds a session variable into store.
- put(Job) - Method in class ai.preferred.venom.job.AbstractPriorityJobQueue
- put(Job) - Method in class ai.preferred.venom.job.FIFOJobQueue
- put(Request, Response) - Method in class ai.preferred.venom.storage.DummyFileManager
- put(Request, Response) - Method in class ai.preferred.venom.storage.MysqlFileManager
- put(String) - Static method in class ai.preferred.venom.request.VRequest.Builder
-
Creates a new instance of builder with method type put.
- PUT - ai.preferred.venom.request.Request.Method
-
PUT method.
R
- Record<T> - Interface in ai.preferred.venom.storage
-
This interface represents only the most basic of a record and the fields that should be retrievable from database.
- register(Callback) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Register any callbacks that will be called when a page has been fetched.
- register(Pattern, Handler) - Method in class ai.preferred.venom.UrlRouter
-
Adds a url pattern, and the handler to be used.
- register(Pattern, Handler, Validator) - Method in class ai.preferred.venom.UrlRouter
-
Adds a url pattern, and the handler to be used.
- register(Pattern, Validator) - Method in class ai.preferred.venom.UrlRouter
-
Adds a url pattern, and the handler to be used.
- releaseResources() - Method in class ai.preferred.venom.fetcher.AsyncResponseConsumer
- remainingCapacity() - Method in class ai.preferred.venom.job.AbstractJobQueue
- removeAttribute(String) - Method in class ai.preferred.venom.socks.SocksIOSession
- removeHeader(String) - Method in class ai.preferred.venom.request.VRequest.Builder
-
Remove a header from this request.
- removeHeaders() - Method in class ai.preferred.venom.request.VRequest.Builder
-
Remove all headers from this request.
- removeProxy() - Method in class ai.preferred.venom.request.CrawlerRequest
-
Remove the proxy from this request.
- Request - Interface in ai.preferred.venom.request
- Request.Method - Enum in ai.preferred.venom.request
-
The method of the request to be made.
- Response - Interface in ai.preferred.venom.response
- ResponseDecompressor - Class in ai.preferred.venom.utils
-
Modified from: org.apache.http.client.protocol.ResponseContentEncoding.
- ResponseDecompressor() - Constructor for class ai.preferred.venom.utils.ResponseDecompressor
-
Constructs a decompressor.
- Retrievable - Interface in ai.preferred.venom.response
-
This interface represents that the response can be/ has been stored.
S
- Scheduler - Class in ai.preferred.venom.job
-
This interface represents only the adding part a scheduler.
- Scheduler(BlockingQueue<Job>) - Constructor for class ai.preferred.venom.job.Scheduler
-
Constructs an instance of Scheduler.
- Session - Class in ai.preferred.venom
- SESSION_KEY - Static variable in class ai.preferred.venom.socks.SocksIOSession
-
SOCKS session key.
- Session.Builder - Class in ai.preferred.venom
-
Builder for Session.
- Session.Key<T> - Class in ai.preferred.venom
-
A class representing the key for a session.
- setAcknowledge() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
-
Set the acknowledge time to current nano time.
- setAttribute(String, Object) - Method in class ai.preferred.venom.socks.SocksIOSession
- setBody(String) - Method in class ai.preferred.venom.request.VRequest.Builder
-
Sets the request body to be used.
- setBufferStatus(SessionBufferStatus) - Method in class ai.preferred.venom.socks.SocksIOSession
- setComplete() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
-
Set the complete time to current nano time.
- setConnectionRequestTimeout(int) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
The timeout in milliseconds used when requesting a connection from the connection manager.
- setConnectTimeout(int) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Determines the timeout in milliseconds until a connection is established.
- setContentType(ContentType) - Method in class ai.preferred.venom.storage.StorageRecord.Builder
-
Sets the response content type for the record.
- setDateCreated(long) - Method in class ai.preferred.venom.storage.StorageRecord.Builder
-
Sets the date the record was created.
- setEvent(int) - Method in class ai.preferred.venom.socks.SocksIOSession
- setEventMask(int) - Method in class ai.preferred.venom.socks.SocksIOSession
- setFetcher(Fetcher) - Method in class ai.preferred.venom.Crawler.Builder
-
Sets the Fetcher to be used, if not set, default will be chosen.
- setFileManager(FileManager<?>) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Sets the FileManager to be used.
- setHandlerRouter(HandlerRouter) - Method in class ai.preferred.venom.Crawler.Builder
-
Sets HandlerRouter to be used.
- setHeaders(Map<String, String>) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Sets the headers to be used when fetching items.
- setHeaders(Map<String, String>) - Method in class ai.preferred.venom.fetcher.StorageFetcher.Builder
-
Sets the headers to be used when fetching items.
- setJobAttribute(JobAttribute) - Method in class ai.preferred.venom.job.Job
-
Adds or replace the current job attribute if the class of attribute is already present in the map.
- setJobQueue(BlockingQueue<Job>) - Method in class ai.preferred.venom.Crawler.Builder
-
Sets the JobQueue to be used, if not set, default will be chosen.
- setMaxConnections(int) - Method in class ai.preferred.venom.Crawler.Builder
-
The number of concurrent connections allowed out of the client.
- setMaxConnections(int) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Sets the maximum allowable connections at an instance.
- setMaxRouteConnections(int) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Sets the maximum allowable connections at an instance for a particular route (host).
- setMaxTries(int) - Method in class ai.preferred.venom.Crawler.Builder
-
Sets number of times to retry for a request.
- setMD5(String) - Method in class ai.preferred.venom.storage.StorageRecord.Builder
-
Sets the md5 hash of the response content for the record.
- setName(String) - Method in class ai.preferred.venom.Crawler.Builder
-
Sets the name for crawler thread.
- setNumIoThreads(int) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Number of httpclient dispatcher threads.
- setParallelism(int) - Method in class ai.preferred.venom.Crawler.Builder
-
Sets the parallelism level.
- setPropRetainProxy(double) - Method in class ai.preferred.venom.Crawler.Builder
-
Sets the proportion of max tries where a specified proxy, if specified will be used.
- setProxy(HttpHost) - Method in class ai.preferred.venom.request.HttpFetcherRequest
-
Sets proxy to be used for this request.
- setProxy(HttpHost) - Method in class ai.preferred.venom.request.VRequest.Builder
-
Sets the proxy to be used, this will override the proxy selected in Fetcher for this request.
- setProxyProvider(ProxyProvider) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Sets the ProxyProvider to be used.
- setRedirectStrategy(RedirectStrategy) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Sets the redirection strategy for a response received by the fetcher.
- setRequestBody(Map<String, String>) - Method in class ai.preferred.venom.storage.StorageRecord.Builder
-
Sets the request body for the record.
- setRequestHeaders(Map<String, String>) - Method in class ai.preferred.venom.storage.StorageRecord.Builder
-
Sets the request headers for the record.
- setRequestMethod(Request.Method) - Method in class ai.preferred.venom.storage.StorageRecord.Builder
-
Sets the request method for the record.
- setResponseContent(byte[]) - Method in class ai.preferred.venom.storage.StorageRecord.Builder
-
Sets the response content for the record.
- setResponseHeaders(Header[]) - Method in class ai.preferred.venom.storage.StorageRecord.Builder
-
Sets the response headers for the record.
- setScheduler(BlockingQueue<Job>) - Method in class ai.preferred.venom.Crawler.Builder
-
Deprecated.
- setSession(Session) - Method in class ai.preferred.venom.Crawler.Builder
-
Sets the Session to be used, if not set, defaults to
Session.EMPTY_SESSION
. - setSize(int) - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
-
Set the size of the response in bytes.
- setSleepScheduler(SleepScheduler) - Method in class ai.preferred.venom.Crawler.Builder
-
Sets the SleepScheduler to be used, if not set, default will be chosen.
- setSleepScheduler(SleepScheduler) - Method in class ai.preferred.venom.request.VRequest.Builder
-
Sets the sleep scheduler to be used, this will override the sleep scheduler defined in Crawler for this request.
- setSocketTimeout(int) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Defines the socket timeout (
SO_TIMEOUT
) in milliseconds, which is the timeout for waiting for data or, put differently, a maximum period inactivity between two consecutive data packets). - setSocketTimeout(int) - Method in class ai.preferred.venom.socks.SocksIOSession
- setSslContext(SSLContext) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Sets the ssl context for an encrypted response.
- setStart() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
-
Set the start time to current nano time.
- setStatusCode(int) - Method in class ai.preferred.venom.storage.StorageRecord.Builder
-
Sets the response status code for the record.
- setStopCodes(@javax.validation.constraints.NotNull int...) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Set a list of stop code that will interrupt crawling.
- setThreadFactory(ThreadFactory) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Set the thread factory that creates the httpclient dispatcher threads.
- setUrl(String) - Method in class ai.preferred.venom.request.VRequest.Builder
-
Sets the url to be fetched.
- setUrl(String) - Method in class ai.preferred.venom.storage.StorageRecord.Builder
-
Sets the url for the record.
- setUserAgent(UserAgent) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Sets the UserAgent to be used, if not set, default will be chosen.
- setValidator(Validator) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Sets the Validator to be used.
- setValidator(Validator) - Method in class ai.preferred.venom.fetcher.StorageFetcher.Builder
-
Sets the Validator to be used.
- setValidator(Validator...) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Sets the multiple validators to be used.
- setValidator(Validator...) - Method in class ai.preferred.venom.fetcher.StorageFetcher.Builder
-
Sets the multiple validators to be used.
- setValidatorRouter(ValidatorRouter) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Sets ValidatorRouter to be used.
- setWorkerManager(WorkerManager) - Method in class ai.preferred.venom.Crawler.Builder
-
Sets the WorkerManager to be used, if not set, default will be chosen.
- shutdown() - Method in class ai.preferred.venom.socks.SocksIOSession
- shutdown() - Method in class ai.preferred.venom.utils.InlineExecutorService
- shutdownNow() - Method in class ai.preferred.venom.utils.InlineExecutorService
- singleton(Session.Key<T>, T) - Static method in class ai.preferred.venom.Session
-
Constructs a singleton session.
- size() - Method in class ai.preferred.venom.job.AbstractJobQueue
- SleepScheduler - Class in ai.preferred.venom
- SleepScheduler(long) - Constructor for class ai.preferred.venom.SleepScheduler
-
Constructs a sleep scheduler with fix sleep time.
- SleepScheduler(long, long) - Constructor for class ai.preferred.venom.SleepScheduler
-
Constructs a sleep scheduler with range of sleep time.
- SocksConnectingIOReactor - Class in ai.preferred.venom.socks
-
This IOReactor makes sure that the supplied
IOEventDispatch
is decorated withSocksIOEventDispatch
. - SocksConnectingIOReactor() - Constructor for class ai.preferred.venom.socks.SocksConnectingIOReactor
-
Creates an instance of SocksConnectingIOReactor with default configuration.
- SocksConnectingIOReactor(IOReactorConfig) - Constructor for class ai.preferred.venom.socks.SocksConnectingIOReactor
-
Creates an instance of SocksConnectingIOReactor with the given configuration.
- SocksConnectingIOReactor(IOReactorConfig, ThreadFactory) - Constructor for class ai.preferred.venom.socks.SocksConnectingIOReactor
-
Creates an instance of SocksConnectingIOReactor with the given configuration.
- SocksHttpRoutePlanner - Class in ai.preferred.venom.socks
-
This route planners ensures that the connection to https server via socks proxy works.
- SocksHttpRoutePlanner(HttpRoutePlanner) - Constructor for class ai.preferred.venom.socks.SocksHttpRoutePlanner
-
Decorates
HttpRoutePlanner
. - SocksIOEventDispatch - Class in ai.preferred.venom.socks
-
This class wraps and handles IO dispatch related to
SocksIOSession
. - SocksIOEventDispatch(IOEventDispatch) - Constructor for class ai.preferred.venom.socks.SocksIOEventDispatch
-
Decorates
IOEventDispatch
. - SocksIOSession - Class in ai.preferred.venom.socks
-
The class establishes Socks4a connection and delegates the interface calls to a decorated
IOSession
. - SocksIOSession(IOSession) - Constructor for class ai.preferred.venom.socks.SocksIOSession
-
Decorates
IOSession
, sets default user ID for a SOCKS proxy. - SocksIOSession(IOSession, String) - Constructor for class ai.preferred.venom.socks.SocksIOSession
-
Decorates
IOSession
, allows to specify user ID. - SocksIOSessionStrategy - Class in ai.preferred.venom.socks
-
Socks + TSL/SSL layering strategy.
- SocksIOSessionStrategy(SSLIOSessionStrategy) - Constructor for class ai.preferred.venom.socks.SocksIOSessionStrategy
- start() - Method in class ai.preferred.venom.Crawler
-
Starts the crawler by starting a new thread to poll for jobs.
- start() - Method in class ai.preferred.venom.fetcher.AsyncFetcher
- start() - Method in interface ai.preferred.venom.fetcher.Fetcher
-
Fetcher starter.
- start() - Method in class ai.preferred.venom.fetcher.StorageFetcher
- startAndClose() - Method in class ai.preferred.venom.Crawler
-
Starts the crawler by starting a new thread to poll for jobs and close it after the jobQueue has reached 0.
- StatusOkValidator - Class in ai.preferred.venom.validator
-
This class determines the validity of a response by its status code.
- StatusOkValidator() - Constructor for class ai.preferred.venom.validator.StatusOkValidator
- STOP - ai.preferred.venom.validator.Validator.Status
-
Stop processing this job.
- StopCodeException - Exception in ai.preferred.venom.fetcher
- StopCodeException(int, String) - Constructor for exception ai.preferred.venom.fetcher.StopCodeException
-
Constructs a stop code exception with a message.
- StopCodeException(int, String, Throwable) - Constructor for exception ai.preferred.venom.fetcher.StopCodeException
-
Constructs a stop code exception with a message and a cause.
- StopCodeException(int, String, Throwable, boolean, boolean) - Constructor for exception ai.preferred.venom.fetcher.StopCodeException
-
Constructs a stop code exception with a message and a cause.
- StopCodeException(int, Throwable) - Constructor for exception ai.preferred.venom.fetcher.StopCodeException
-
Constructs a stop code exception with a cause.
- StorageException - Exception in ai.preferred.venom.storage
- StorageException(String) - Constructor for exception ai.preferred.venom.storage.StorageException
-
Constructs a storage exception with a message.
- StorageException(String, Throwable) - Constructor for exception ai.preferred.venom.storage.StorageException
-
Constructs a storage exception with a message and a cause.
- StorageException(Throwable) - Constructor for exception ai.preferred.venom.storage.StorageException
-
Constructs a storage exception with a cause.
- StorageFetcher - Class in ai.preferred.venom.fetcher
-
This class holds the implementation to provide how items are fetched from a database, to validate the item and to store it if specified.
- StorageFetcher.Builder - Class in ai.preferred.venom.fetcher
-
A builder for StorageFetcher class.
- StorageFetcherRequest - Class in ai.preferred.venom.request
- StorageFetcherRequest(Request) - Constructor for class ai.preferred.venom.request.StorageFetcherRequest
-
Constructs an instance of StorageFetcherRequest.
- StorageRecord<T> - Class in ai.preferred.venom.storage
-
This class implements a default storage record.
- StorageRecord.Builder<T> - Class in ai.preferred.venom.storage
-
A builder for StorageRecord class.
- StorageResponse - Class in ai.preferred.venom.response
- StorageResponse(Record<?>, String) - Constructor for class ai.preferred.venom.response.StorageResponse
-
Constructs a base response.
- StorageUtil - Class in ai.preferred.venom.storage
-
A utility for storage needs.
- submit(Runnable) - Method in interface ai.preferred.venom.Worker
-
Submits a Runnable task for execution and returns a Future representing that task.
- submit(Runnable, T) - Method in interface ai.preferred.venom.Worker
-
Submits a Runnable task for execution and returns a Future representing that task.
- submit(Callable<T>) - Method in interface ai.preferred.venom.Worker
-
Submits a value-returning task for execution and returns a Future representing the pending results of the task.
T
- take() - Method in class ai.preferred.venom.job.AbstractJobQueue
- ThreadedWorkerManager - Class in ai.preferred.venom
- ThreadedWorkerManager() - Constructor for class ai.preferred.venom.ThreadedWorkerManager
-
Constructs a threaded worker manager with a specified executor.
- ThreadedWorkerManager(ExecutorService) - Constructor for class ai.preferred.venom.ThreadedWorkerManager
-
Constructs a threaded worker manager with a specified executor.
- ThreadedWorkerManager.AbstractManagedBlockingWorker - Class in ai.preferred.venom
-
This abstract class exposes the methods to allow submitting tasks for multithreading and implements inline blocking method.
- timeout(IOSession) - Method in class ai.preferred.venom.socks.SocksIOEventDispatch
U
- Unwrappable - Interface in ai.preferred.venom.request
-
This interface represents that the request can be unwrapped.
- Unwrappable - Interface in ai.preferred.venom.response
-
This interface represents that the response can be unwrapped.
- unwrapRequest(Request) - Static method in interface ai.preferred.venom.request.Unwrappable
-
Unwrap all wrapped request to an instance of base request.
- upgrade(HttpHost, IOSession) - Method in class ai.preferred.venom.socks.SocksIOSessionStrategy
- UrlRouter - Class in ai.preferred.venom
-
This class provides an implementation to select a handler based on the url from which they were fetched.
- UrlRouter() - Constructor for class ai.preferred.venom.UrlRouter
-
Constructs a url router without default handler.
- UrlRouter(Handler) - Constructor for class ai.preferred.venom.UrlRouter
-
Constructs a url router with default handler.
- UrlUtil - Class in ai.preferred.venom.utils
-
A utility for managing URLs.
- UserAgent - Interface in ai.preferred.venom.uagent
-
This interface allows the user to define a user agent string.
V
- VALID - ai.preferred.venom.validator.Validator.Status
-
The response is valid.
- ValidationException - Exception in ai.preferred.venom.fetcher
- ValidationException(Validator.Status, Response, String) - Constructor for exception ai.preferred.venom.fetcher.ValidationException
-
Constructs a validation exception with a message.
- ValidationException(Validator.Status, Response, String, Throwable) - Constructor for exception ai.preferred.venom.fetcher.ValidationException
- ValidationException(Validator.Status, Response, String, Throwable, boolean, boolean) - Constructor for exception ai.preferred.venom.fetcher.ValidationException
- ValidationException(Validator.Status, Response, Throwable) - Constructor for exception ai.preferred.venom.fetcher.ValidationException
- Validator - Interface in ai.preferred.venom.validator
- Validator.Status - Enum in ai.preferred.venom.validator
-
The allowed return status of validation.
- ValidatorRouter - Interface in ai.preferred.venom
-
This interface allows the user to map request to handler.
- valueOf(String) - Static method in enum ai.preferred.venom.job.Priority
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum ai.preferred.venom.request.Request.Method
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum ai.preferred.venom.validator.Validator.Status
-
Returns the enum constant of this type with the specified name.
- values() - Static method in enum ai.preferred.venom.job.Priority
-
Returns an array containing the constants of this enum type, in the order they are declared.
- values() - Static method in enum ai.preferred.venom.request.Request.Method
-
Returns an array containing the constants of this enum type, in the order they are declared.
- values() - Static method in enum ai.preferred.venom.validator.Validator.Status
-
Returns an array containing the constants of this enum type, in the order they are declared.
- VRequest - Class in ai.preferred.venom.request
-
An implementation of HTTP request.
- VRequest(VRequest.Builder<?>) - Constructor for class ai.preferred.venom.request.VRequest
-
Constructs an instance of venom request.
- VRequest(String) - Constructor for class ai.preferred.venom.request.VRequest
-
Constructs an instance of venom request.
- VRequest(String, Map<String, String>) - Constructor for class ai.preferred.venom.request.VRequest
-
Constructs an instance of venom request.
- VRequest.Builder<T extends VRequest.Builder<T>> - Class in ai.preferred.venom.request
-
A builder for VRequest class.
- VResponse - Class in ai.preferred.venom.response
- VResponse(Response) - Constructor for class ai.preferred.venom.response.VResponse
-
Constructs a VResponse.
W
- Worker - Interface in ai.preferred.venom
- WorkerManager - Interface in ai.preferred.venom
All Classes All Packages