Package org.archive.modules.fetcher
package org.archive.modules.fetcher
-
ClassDescriptionCookie store using bdb for storage.A
Listimplementation that wraps aCollection.Server and Host cache.Processor to resolve 'dns:' URIs.Fetches documents and directory listings using FTP.HTTP fetcher that uses Apache HttpComponents.HTTP Fetcher that uses Jetty HttpClient to support HTTP/2 and HTTP/3.Implementation ofDnsResolverthat uses the server cache which is normally expected to have been populated by FetchDNS.Collector of statistics for a 'subset' of a crawl, such as a server (host:port), host, or frontier group (eg queue).Constant flag codes to be used, in lieu of per-protocol codes (like HTTP's 200, 404, etc.), when network/internal/ out-of-band conditions occur.WHOIS Fetcher (RFC 3912).In-memory cookie store, mostly for testing.