Package | Description |
---|---|
ai.preferred.venom.fetcher |
Modifier and Type | Method and Description |
---|---|
static AsyncFetcher.Builder |
AsyncFetcher.builder()
Create an instance of builder.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.compressed(boolean compressed)
Set whether to request for compress pages and to decompress pages
after it is fetched.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.connectionRequestTimeout(int connectionRequestTimeout)
The timeout in milliseconds used when requesting a connection
from the connection manager.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.connectTimeout(int connectTimeout)
Determines the timeout in milliseconds until a connection is established.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.fileManager(@NotNull FileManager fileManager)
Sets the FileManager to be used.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.headers(@NotNull Map<String,String> headers)
Sets the headers to be used when fetching items.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.numIoThreads(int numIoThreads)
Number of httpclient dispatcher threads.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.proxyProvider(@NotNull ProxyProvider proxyProvider)
Sets the ProxyProvider to be used.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.register(@NotNull Callback callback)
Register any callbacks that will be called when a page has been fetched.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.router(@NotNull ValidatorRouter router)
Sets ValidatorRouter to be used.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.socketTimeout(int socketTimeout)
Defines the socket timeout (
SO_TIMEOUT ) in milliseconds,
which is the timeout for waiting for data or, put differently,
a maximum period inactivity between two consecutive data packets). |
AsyncFetcher.Builder |
AsyncFetcher.Builder.soTimeout(int soTimeout)
Determines the default socket timeout value for non-blocking I/O operations.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.stopCodes(int... codes)
Set a list of stop code that will interrupt crawling.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.threadFactory(@NotNull ThreadFactory threadFactory)
Set the thread factory that creates the httpclient dispatcher
threads.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.userAgent(@NotNull UserAgent userAgent)
Sets the UserAgent to be used, if not set, default will be chosen.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.validator(@NotNull Validator validator)
Sets the Validator to be used.
|
Copyright © 2018. All rights reserved.