- get(Request) - Method in interface ai.preferred.venom.ProxyProvider
-
Returns the get proxy from the list.
- get(String) - Static method in class ai.preferred.venom.request.VRequest.Builder
-
Creates a new instance of builder with method type get.
- get(Session.Key<T>) - Method in class ai.preferred.venom.Session
-
Returns the session variable from the store.
- get(int) - Method in interface ai.preferred.venom.storage.FileManager
-
Returns record by the internal record id.
- get(Request) - Method in interface ai.preferred.venom.storage.FileManager
-
Returns latest record matching request.
- get() - Method in class ai.preferred.venom.uagent.DefaultUserAgent
-
- get() - Method in interface ai.preferred.venom.uagent.UserAgent
-
Gets the user agent for a request.
- getAcknowledge() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
-
Get the acknowledge time of the request.
- getBaseUrl() - Method in class ai.preferred.venom.response.BaseResponse
-
- getBaseUrl() - Method in interface ai.preferred.venom.response.Response
-
Returns the base form of the url used in this request.
- getBaseUrl() - Method in class ai.preferred.venom.response.VResponse
-
- getBody() - Method in class ai.preferred.venom.request.CrawlerRequest
-
- getBody() - Method in class ai.preferred.venom.request.HttpFetcherRequest
-
- getBody() - Method in interface ai.preferred.venom.request.Request
-
Returns the request body of the request or null if none specified.
- getBody() - Method in class ai.preferred.venom.request.VRequest
-
- getCallback() - Method in interface ai.preferred.venom.storage.FileManager
-
Get callback upon completion of request.
- getComplete() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
-
Get the complete time of the request.
- getContent() - Method in class ai.preferred.venom.response.BaseResponse
-
- getContent() - Method in interface ai.preferred.venom.response.Response
-
Returns raw content of the response.
- getContent() - Method in class ai.preferred.venom.response.VResponse
-
- getContentType(HttpEntity) - Method in class ai.preferred.venom.fetcher.AsyncResponseConsumer
-
- getContentType() - Method in class ai.preferred.venom.response.BaseResponse
-
- getContentType() - Method in interface ai.preferred.venom.response.Response
-
Returns the content type of the content fetched.
- getContentType() - Method in class ai.preferred.venom.response.VResponse
-
- getDateCreated() - Method in interface ai.preferred.venom.storage.Record
-
- getDiagnostics() - Method in class ai.preferred.venom.request.HttpFetcherRequest
-
Get diagnostic information for this request.
- getHandler(Request) - Method in interface ai.preferred.venom.HandlerRouter
-
Returns the handler to be used for a specified request.
- getHandler() - Method in class ai.preferred.venom.job.BasicJob
-
- getHandler() - Method in interface ai.preferred.venom.job.Job
-
Get the handler to handle the response of the job.
- getHandler(Request) - Method in class ai.preferred.venom.UrlRouter
-
- getHeaders() - Method in class ai.preferred.venom.request.CrawlerRequest
-
- getHeaders() - Method in class ai.preferred.venom.request.HttpFetcherRequest
-
- getHeaders() - Method in interface ai.preferred.venom.request.Request
-
Returns the headers set for the request.
- getHeaders() - Method in class ai.preferred.venom.request.VRequest
-
- getHeaders() - Method in class ai.preferred.venom.response.BaseResponse
-
- getHeaders() - Method in interface ai.preferred.venom.response.Response
-
Returns the headers that were used to trigger this response.
- getHeaders() - Method in class ai.preferred.venom.response.VResponse
-
- getHtml() - Method in class ai.preferred.venom.response.VResponse
-
Returns the html in string format.
- getHtml(Charset) - Method in class ai.preferred.venom.response.VResponse
-
Returns the html in string format.
- getId() - Method in interface ai.preferred.venom.storage.Record
-
- getInner() - Method in class ai.preferred.venom.request.CrawlerRequest
-
- getInner() - Method in class ai.preferred.venom.request.HttpFetcherRequest
-
- getInner() - Method in interface ai.preferred.venom.request.Unwrappable
-
Returns the unwrapped version of this request.
- getInner() - Method in interface ai.preferred.venom.response.Unwrappable
-
Returns the unwrapped version of this request.
- getInner() - Method in class ai.preferred.venom.response.VResponse
-
- getJsoup() - Method in class ai.preferred.venom.response.VResponse
-
Returns a Jsoup document of this response.
- getJsoup(Charset) - Method in class ai.preferred.venom.response.VResponse
-
Returns a Jsoup document of this response.
- getLatency() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
-
Get the latency between sending of the request and the first
response.
- getMD5() - Method in interface ai.preferred.venom.storage.Record
-
- getMethod() - Method in class ai.preferred.venom.request.CrawlerRequest
-
- getMethod() - Method in class ai.preferred.venom.request.HttpFetcherRequest
-
- getMethod() - Method in interface ai.preferred.venom.request.Request
-
Returns the method type of the request.
- getMethod() - Method in class ai.preferred.venom.request.VRequest
-
- getPriority() - Method in class ai.preferred.venom.job.BasicJob
-
- getPriority() - Method in interface ai.preferred.venom.job.Job
-
Get the current priority set for this job.
- getPriority() - Method in enum ai.preferred.venom.job.Priority
-
Get the current priority level.
- getProxy() - Method in class ai.preferred.venom.request.CrawlerRequest
-
- getProxy() - Method in class ai.preferred.venom.request.HttpFetcherRequest
-
- getProxy() - Method in interface ai.preferred.venom.request.Request
-
Returns the proxy set to be used for the request or default to
fetcher if none specified.
- getProxy() - Method in class ai.preferred.venom.request.VRequest
-
- getProxy() - Method in class ai.preferred.venom.response.BaseResponse
-
- getProxy() - Method in interface ai.preferred.venom.response.Response
-
Returns the proxy that was used to trigger this response.
- getProxy() - Method in class ai.preferred.venom.response.VResponse
-
- getProxyList() - Method in interface ai.preferred.venom.ProxyProvider
-
Returns a list of all proxies.
- getRequest() - Method in class ai.preferred.venom.job.BasicJob
-
- getRequest() - Method in interface ai.preferred.venom.job.Job
-
Get the request of this job.
- getRequestBody() - Method in interface ai.preferred.venom.storage.Record
-
- getRequestHeaders() - Method in interface ai.preferred.venom.storage.Record
-
- getRequestMethod() - Method in interface ai.preferred.venom.storage.Record
-
- getResponse() - Method in exception ai.preferred.venom.fetcher.ValidationException
-
Get the response validated.
- getResponseContent() - Method in interface ai.preferred.venom.storage.Record
-
- getResponseHeaders() - Method in interface ai.preferred.venom.storage.Record
-
- getScheduler() - Method in class ai.preferred.venom.Crawler
-
Get the instance of scheduler used.
- getSize() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
-
Get the size the response.
- getSleepScheduler() - Method in class ai.preferred.venom.request.CrawlerRequest
-
- getSleepScheduler() - Method in class ai.preferred.venom.request.HttpFetcherRequest
-
- getSleepScheduler() - Method in interface ai.preferred.venom.request.Request
-
Returns information about the amount of sleep before this request
is made.
- getSleepScheduler() - Method in class ai.preferred.venom.request.VRequest
-
- getSleepTime() - Method in class ai.preferred.venom.SleepScheduler
-
Get the amount of time to wait specified in this class.
- getSourceId() - Method in class ai.preferred.venom.response.BaseResponse
-
- getSourceId() - Method in interface ai.preferred.venom.response.Retrievable
-
Returns the id of the row where an archive of this response
has been insert into a persistent storage.
- getSpeed() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
-
Get download speed in bytes per second.
- getStart() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
-
Get the start time of the request.
- getStatus() - Method in exception ai.preferred.venom.fetcher.ValidationException
-
Get the validation status of the response.
- getStatusCode() - Method in class ai.preferred.venom.response.BaseResponse
-
- getStatusCode() - Method in interface ai.preferred.venom.response.Response
-
Returns status code of the response.
- getStatusCode() - Method in class ai.preferred.venom.response.VResponse
-
- getStatusCode() - Method in interface ai.preferred.venom.storage.Record
-
- getStreamResponseContent() - Method in interface ai.preferred.venom.storage.Record
-
- getTryCount() - Method in class ai.preferred.venom.job.BasicJob
-
- getTryCount() - Method in interface ai.preferred.venom.job.Job
-
Get attempt number of this job.
- getUrl() - Method in class ai.preferred.venom.request.CrawlerRequest
-
- getUrl() - Method in class ai.preferred.venom.request.HttpFetcherRequest
-
- getUrl() - Method in interface ai.preferred.venom.request.Request
-
Returns the url of the request.
- getUrl() - Method in class ai.preferred.venom.request.VRequest
-
- getURL() - Method in interface ai.preferred.venom.storage.Record
-
- getValidator() - Method in class ai.preferred.venom.response.BaseResponse
-
- getValidator() - Method in interface ai.preferred.venom.response.Response
-
Returns the instance of validator used to validate this response.
- getValidator() - Method in class ai.preferred.venom.response.VResponse
-
- getValidator(Request) - Method in class ai.preferred.venom.UrlRouter
-
- getValidator(Request) - Method in interface ai.preferred.venom.ValidatorRouter
-
Returns the handler to be used for a specified request.
- getWorker() - Method in class ai.preferred.venom.ThreadedWorkerManager
-
- getWorker() - Method in interface ai.preferred.venom.WorkerManager
-
Get the result collector in use.
- Scheduler - Interface in ai.preferred.venom.job
-
This interface represents only the most basic of a scheduler.
- Session - Class in ai.preferred.venom
-
- Session.Builder - Class in ai.preferred.venom
-
Builder for Session.
- Session.Key<T> - Class in ai.preferred.venom
-
A class representing the key for a session.
- setAcknowledge() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
-
Set the acknowledge time to current nano time.
- setBody(String) - Method in class ai.preferred.venom.request.VRequest.Builder
-
Sets the request body to be used.
- setComplete() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
-
Set the complete time to current nano time.
- setConnectionRequestTimeout(int) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
The timeout in milliseconds used when requesting a connection
from the connection manager.
- setConnectTimeout(int) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Determines the timeout in milliseconds until a connection is established.
- setFetcher(Fetcher) - Method in class ai.preferred.venom.Crawler.Builder
-
Sets the Fetcher to be used, if not set, default will be chosen.
- setFileManager(FileManager) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Sets the FileManager to be used.
- setHandlerRouter(HandlerRouter) - Method in class ai.preferred.venom.Crawler.Builder
-
Sets HandlerRouter to be used.
- setHeaders(Map<String, String>) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Sets the headers to be used when fetching items.
- setMaxConnections(int) - Method in class ai.preferred.venom.Crawler.Builder
-
The number of concurrent connections allowed out of the client.
- setMaxConnections(int) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Sets the maximum allowable connections at an instance.
- setMaxRouteConnections(int) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Sets the maximum allowable connections at an instance for
a particular route (host).
- setMaxTries(int) - Method in class ai.preferred.venom.Crawler.Builder
-
Sets number of times to retry for a request.
- setName(String) - Method in class ai.preferred.venom.Crawler.Builder
-
Sets the name for crawler thread.
- setNumIoThreads(int) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Number of httpclient dispatcher threads.
- setParallism(int) - Method in class ai.preferred.venom.Crawler.Builder
-
Sets the parallelism level.
- setPropRetainProxy(double) - Method in class ai.preferred.venom.Crawler.Builder
-
Sets the proportion of max tries where a specified proxy, if specified will be used.
- setProxy(HttpHost) - Method in class ai.preferred.venom.request.HttpFetcherRequest
-
Sets proxy to be used for this request.
- setProxy(HttpHost) - Method in class ai.preferred.venom.request.VRequest.Builder
-
Sets the proxy to be used, this will override the
proxy selected in Fetcher for this request.
- setProxyProvider(ProxyProvider) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Sets the ProxyProvider to be used.
- setRedirectStrategy(RedirectStrategy) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Sets the redirection strategy for a response received by the fetcher.
- setScheduler(AbstractQueueScheduler) - Method in class ai.preferred.venom.Crawler.Builder
-
Sets the Scheduler to be used, if not set, default will be chosen.
- setSession(Session) - Method in class ai.preferred.venom.Crawler.Builder
-
Sets the Session to be used, if not set, defaults to none.
- setSize(int) - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
-
Set the size of the response in bytes.
- setSleepScheduler(SleepScheduler) - Method in class ai.preferred.venom.Crawler.Builder
-
Sets the SleepScheduler to be used, if not set, default will be chosen.
- setSleepScheduler(SleepScheduler) - Method in class ai.preferred.venom.request.VRequest.Builder
-
Sets the sleep scheduler to be used, this will override the
sleep scheduler defined in Crawler for this request.
- setSocketTimeout(int) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Defines the socket timeout (SO_TIMEOUT
) in milliseconds,
which is the timeout for waiting for data or, put differently,
a maximum period inactivity between two consecutive data packets).
- setSourceId(String) - Method in class ai.preferred.venom.response.BaseResponse
-
Sets the source id where the raw response is saved.
- setSslContext(SSLContext) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Sets the ssl context for an encrypted response.
- setStart() - Method in class ai.preferred.venom.request.HttpFetcherRequest.Diagnostics
-
Set the start time to current nano time.
- setStopCodes(int...) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Set a list of stop code that will interrupt crawling.
- setThreadFactory(ThreadFactory) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Set the thread factory that creates the httpclient dispatcher
threads.
- setUrl(String) - Method in class ai.preferred.venom.request.VRequest.Builder
-
Sets the url to be fetched.
- setUserAgent(UserAgent) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Sets the UserAgent to be used, if not set, default will be chosen.
- setValidator(Validator) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Sets the Validator to be used.
- setValidator(Validator...) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Sets the multiple validators to be used.
- setValidator(Validator) - Method in class ai.preferred.venom.response.BaseResponse
-
Sets the validator used to validate this response.
- setValidatorRouter(ValidatorRouter) - Method in class ai.preferred.venom.fetcher.AsyncFetcher.Builder
-
Sets ValidatorRouter to be used.
- setWorkerManager(WorkerManager) - Method in class ai.preferred.venom.Crawler.Builder
-
Sets the WorkerManager to be used, if not set, default will be chosen.
- shutdown() - Method in class ai.preferred.venom.utils.InlineExecutorService
-
- shutdownNow() - Method in class ai.preferred.venom.utils.InlineExecutorService
-
- singleton(Session.Key<T>, T) - Static method in class ai.preferred.venom.Session
-
Constructs a singleton session.
- size() - Method in class ai.preferred.venom.job.AbstractQueueScheduler
-
- SleepScheduler - Class in ai.preferred.venom
-
- SleepScheduler(long) - Constructor for class ai.preferred.venom.SleepScheduler
-
Constructs a sleep scheduler with fix sleep time.
- SleepScheduler(long, long) - Constructor for class ai.preferred.venom.SleepScheduler
-
Constructs a sleep scheduler with range of sleep time.
- start() - Method in class ai.preferred.venom.Crawler
-
Starts the crawler by starting a new thread to poll for jobs.
- start() - Method in class ai.preferred.venom.fetcher.AsyncFetcher
-
- start() - Method in interface ai.preferred.venom.fetcher.Fetcher
-
Fetcher starter.
- startAndClose() - Method in class ai.preferred.venom.Crawler
-
Starts the crawler by starting a new thread to poll for jobs and close it
after the queue has reached 0.
- StatusOkValidator - Class in ai.preferred.venom.validator
-
This class determines the validity of a response by its status code.
- StatusOkValidator() - Constructor for class ai.preferred.venom.validator.StatusOkValidator
-
- StopCodeException - Exception in ai.preferred.venom.fetcher
-
- StopCodeException(int, String) - Constructor for exception ai.preferred.venom.fetcher.StopCodeException
-
Constructs a stop code exception with a message.
- StopCodeException(int, String, Throwable) - Constructor for exception ai.preferred.venom.fetcher.StopCodeException
-
Constructs a stop code exception with a message and a cause.
- StopCodeException(int, Throwable) - Constructor for exception ai.preferred.venom.fetcher.StopCodeException
-
Constructs a stop code exception with a cause.
- StopCodeException(int, String, Throwable, boolean, boolean) - Constructor for exception ai.preferred.venom.fetcher.StopCodeException
-
Constructs a stop code exception with a message and a cause.
- StorageException - Exception in ai.preferred.venom.storage
-
- StorageException(String) - Constructor for exception ai.preferred.venom.storage.StorageException
-
Constructs a storage exception with a message.
- StorageException(String, Throwable) - Constructor for exception ai.preferred.venom.storage.StorageException
-
Constructs a storage exception with a message and a cause.
- StorageException(Throwable) - Constructor for exception ai.preferred.venom.storage.StorageException
-
Constructs a storage exception with a cause.
- submit(Callable<T>) - Method in interface ai.preferred.venom.Worker
-
Submits a value-returning task for execution and returns a
Future representing the pending results of the task.
- submit(Runnable, T) - Method in interface ai.preferred.venom.Worker
-
Submits a Runnable task for execution and returns a Future
representing that task.
- submit(Runnable) - Method in interface ai.preferred.venom.Worker
-
Submits a Runnable task for execution and returns a Future
representing that task.