static AsyncFetcher.Builder |
AsyncFetcher.builder() |
Create an instance of builder.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.disableCompression() |
Disables request for compress pages and to decompress pages
after it is fetched.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.disableCookies() |
Disables cookie storage.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.enableSocksProxy() |
Enables SOCKS protocol for proxies (socks://).
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.register(@NotNull Callback callback) |
Register any callbacks that will be called when a page has been fetched.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setConnectionRequestTimeout(int connectionRequestTimeout) |
The timeout in milliseconds used when requesting a connection
from the connection manager.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setConnectTimeout(int connectTimeout) |
Determines the timeout in milliseconds until a connection is established.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setFileManager(FileManager<?> fileManager) |
Sets the FileManager to be used.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setHeaders(@NotNull Map<String,String> headers) |
Sets the headers to be used when fetching items.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setMaxConnections(int maxConnections) |
Sets the maximum allowable connections at an instance.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setMaxRouteConnections(int maxRouteConnections) |
Sets the maximum allowable connections at an instance for
a particular route (host).
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setNumIoThreads(int numIoThreads) |
Number of httpclient dispatcher threads.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setProxyProvider(ProxyProvider proxyProvider) |
Sets the ProxyProvider to be used.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setRedirectStrategy(org.apache.http.client.RedirectStrategy redirectStrategy) |
Sets the redirection strategy for a response received by the fetcher.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setSocketTimeout(int socketTimeout) |
Defines the socket timeout (SO_TIMEOUT ) in milliseconds,
which is the timeout for waiting for data or, put differently,
a maximum period inactivity between two consecutive data packets).
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setSslContext(SSLContext sslContext) |
Sets the ssl context for an encrypted response.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setStopCodes(@javax.validation.constraints.NotNull int... codes) |
Set a list of stop code that will interrupt crawling.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setThreadFactory(@NotNull ThreadFactory threadFactory) |
Set the thread factory that creates the httpclient dispatcher
threads.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setUserAgent(@NotNull UserAgent userAgent) |
Sets the UserAgent to be used, if not set, default will be chosen.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setValidator(@NotNull Validator validator) |
Sets the Validator to be used.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setValidator(@NotNull Validator... validators) |
Sets the multiple validators to be used.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setValidatorRouter(ValidatorRouter router) |
Sets ValidatorRouter to be used.
|