Package ai.preferred.venom.request

Class CrawlerRequest

  • All Implemented Interfaces:
    Request, Unwrappable
    public class CrawlerRequest
extends Object
implements Request, Unwrappable
    This class allows the removal of proxy from request.
    Author:
    Maksim Tkachenko, Ween Jiann Lee

    • Constructor Detail

      • CrawlerRequest

        public CrawlerRequest​(Request request)
        Constructs an instance of crawler request with an underlying request.
        Parameters:
        request - An instance of the underlying request

    • Method Detail

      • getMethod

        public final Request.Method getMethod()
        Description copied from interface: Request
        Returns the method type of the request.
        Specified by:
        getMethod in interface Request
        Returns:
        method type

      • getBody

        public final String getBody()
        Description copied from interface: Request
        Returns the request body of the request or null if none specified.
        Specified by:
        getBody in interface Request
        Returns:
        request body

      • getUrl

        public final String getUrl()
        Description copied from interface: Request
        Returns the url of the request.
        Specified by:
        getUrl in interface Request
        Returns:
        url

      • getHeaders

        public final Map<String,​String> getHeaders()
        Description copied from interface: Request
        Returns the headers set for the request.
        Specified by:
        getHeaders in interface Request
        Returns:
        a map of the headers set

      • getProxy

        public final org.apache.http.HttpHost getProxy()
        Description copied from interface: Request
        Returns the proxy set to be used for the request or default to fetcher if none specified.
        Specified by:
        getProxy in interface Request
        Returns:
        proxy

      • removeProxy

        public final void removeProxy()
        Remove the proxy from this request.

      • getSleepScheduler

        public final SleepScheduler getSleepScheduler()
        Description copied from interface: Request
        Returns information about the amount of sleep before this request is made.
        Specified by:
        getSleepScheduler in interface Request
        Returns:
        an instance of SleepScheduler

      • getInner

        public final Request getInner()
        Description copied from interface: Unwrappable
        Returns the unwrapped version of this request.
        Specified by:
        getInner in interface Unwrappable
        Returns:
        an instance of request