Package com.cedarsoftware.util

Class UrlUtilities

java.lang.Object
com.cedarsoftware.util.UrlUtilities
public final class UrlUtilities extends Object
Useful utilities for working with UrlConnections and IO. Anyone using the deprecated api calls for proxying to urls should update to use the new suggested calls. To let the jvm proxy for you automatically, use the following -D parameters: http.proxyHost http.proxyPort (default: 80) http.nonProxyHosts (should always include localhost) https.proxyHost https.proxyPort Example: -Dhttp.proxyHost=proxy.example.org -Dhttp.proxyPort=8080 -Dhttps.proxyHost=proxy.example.org -Dhttps.proxyPort=8080 -Dhttp.nonProxyHosts=*.foo.com|localhost|*.td.afg
Author:
Ken Partlow, John DeRegnaucourt ([email protected])
Copyright (c) Cedar Software LLC

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

License

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

  • Field Details

  • Method Details

    • clearGlobalUserAgent

      public static void clearGlobalUserAgent()

    • clearGlobalReferrer

      public static void clearGlobalReferrer()

    • setReferrer

      public static void setReferrer(String referer)

    • getReferrer

      public static String getReferrer()

    • setUserAgent

      public static void setUserAgent(String agent)

    • getUserAgent

      public static String getUserAgent()

    • setDefaultConnectTimeout

      public static void setDefaultConnectTimeout(int millis)

    • setDefaultReadTimeout

      public static void setDefaultReadTimeout(int millis)

    • getDefaultConnectTimeout

      public static int getDefaultConnectTimeout()

    • getDefaultReadTimeout

      public static int getDefaultReadTimeout()

    • readErrorResponse

      public static void readErrorResponse(URLConnection c)

    • disconnect

      public static void disconnect(HttpURLConnection c)

    • getCookies

      public static void getCookies(URLConnection conn, Map<String,Map<String,Map<String,String>>> store)
      Retrieves and stores cookies returned by the host on the other side of the open java.net.URLConnection.

      The connection MUST have been opened using the connect() method or a IOException will be thrown.

      Parameters:
      conn - a java.net.URLConnection - must be open, or IOException will be thrown

    • setCookies

      public static void setCookies(URLConnection conn, Map<String,Map<String,Map<String,String>>> store)
      Prior to opening a URLConnection, calling this method will set all unexpired cookies that match the path or subpaths for thi underlying URL

      The connection MUST NOT have been opened method or an IOException will be thrown.

      Parameters:
      conn - a java.net.URLConnection - must NOT be open, or IOException will be thrown
      Throws:
      IOException - if the connection has already been opened (thrown as unchecked)

    • getCookieDomainFromHost

      public static String getCookieDomainFromHost(String host)

    • getContentFromUrlAsString

      public static String getContentFromUrlAsString(String url)
      Get content from the passed in URL. This code will open a connection to the passed in server, fetch the requested content, and return it as a String.
      Parameters:
      url - URL to hit
      Returns:
      UTF-8 String read from URL or null in the case of error.

    • getContentFromUrlAsString

      public static String getContentFromUrlAsString(URL url, boolean allowAllCerts)
      Get content from the passed in URL. This code will open a connection to the passed in server, fetch the requested content, and return it as a String.
      Parameters:
      url - URL to hit
      allowAllCerts - true to not verify certificates
      Returns:
      UTF-8 String read from URL or null in the case of error.

    • getContentFromUrlAsString

      public static String getContentFromUrlAsString(String url, Map inCookies, Map outCookies, boolean trustAllCerts)
      Get content from the passed in URL. This code will open a connection to the passed in server, fetch the requested content, and return it as a String.
      Parameters:
      url - URL to hit
      inCookies - Map of session cookies (or null if not needed)
      outCookies - Map of session cookies (or null if not needed)
      trustAllCerts - if true, SSL connection will always be trusted.
      Returns:
      String of content fetched from URL.

    • getContentFromUrlAsString

      public static String getContentFromUrlAsString(URL url, Map inCookies, Map outCookies, boolean trustAllCerts)
      Get content from the passed in URL. This code will open a connection to the passed in server, fetch the requested content, and return it as a String.
      Parameters:
      url - URL to hit
      inCookies - Map of session cookies (or null if not needed)
      outCookies - Map of session cookies (or null if not needed)
      trustAllCerts - if true, SSL connection will always be trusted.
      Returns:
      String of content fetched from URL.

    • getContentFromUrl

      public static byte[] getContentFromUrl(String url)
      Get content from the passed in URL. This code will open a connection to the passed in server, fetch the requested content, and return it as a byte[].
      Parameters:
      url - URL to hit
      Returns:
      byte[] read from URL or null in the case of error.

    • getContentFromUrl

      public static byte[] getContentFromUrl(URL url, boolean allowAllCerts)
      Get content from the passed in URL. This code will open a connection to the passed in server, fetch the requested content, and return it as a byte[].
      Parameters:
      url - URL to hit
      Returns:
      byte[] read from URL or null in the case of error.

    • getContentFromUrl

      public static byte[] getContentFromUrl(String url, Map inCookies, Map outCookies, boolean allowAllCerts)

    • getContentFromUrl

      public static byte[] getContentFromUrl(URL url, Map inCookies, Map outCookies, boolean allowAllCerts)
      Get content from the passed in URL. This code will open a connection to the passed in server, fetch the requested content, and return it as a byte[].
      Parameters:
      url - URL to hit
      inCookies - Map of session cookies (or null if not needed)
      outCookies - Map of session cookies (or null if not needed)
      allowAllCerts - override certificate validation?
      Returns:
      byte[] of content fetched from URL.

    • copyContentFromUrl

      public static void copyContentFromUrl(String url, OutputStream out)
      Convenience method to copy content from a String URL to an output stream.

    • copyContentFromUrl

      public static void copyContentFromUrl(URL url, OutputStream out, Map<String,Map<String,Map<String,String>>> inCookies, Map<String,Map<String,Map<String,String>>> outCookies, boolean allowAllCerts)
      Copy content from a URL to an output stream.

    • getContentFromUrl

      public static byte[] getContentFromUrl(String url, Map inCookies, Map outCookies)
      Get content from the passed in URL. This code will open a connection to the passed in server, fetch the requested content, and return it as a byte[].
      Parameters:
      url - URL to hit
      inCookies - Map of session cookies (or null if not needed)
      outCookies - Map of session cookies (or null if not needed)
      Returns:
      byte[] of content fetched from URL.

    • getConnection

      public static URLConnection getConnection(String url, boolean input, boolean output, boolean cache)
      Parameters:
      input - boolean indicating whether this connection will be used for input
      output - boolean indicating whether this connection will be used for output
      cache - boolean allow caching (be careful setting this to true for non-static retrievals).
      Returns:
      URLConnection established URL connection.
      Throws:
      IOException - if an I/O error occurs (thrown as unchecked)
      MalformedURLException - if the URL is invalid (thrown as unchecked)

    • getConnection

      public static URLConnection getConnection(URL url, boolean input, boolean output, boolean cache)
      Parameters:
      input - boolean indicating whether this connection will be used for input
      output - boolean indicating whether this connection will be used for output
      cache - boolean allow caching (be careful setting this to true for non-static retrievals).
      Returns:
      URLConnection established URL connection.

    • getConnection

      public static URLConnection getConnection(URL url, Map inCookies, boolean input, boolean output, boolean cache, boolean allowAllCerts)
      Gets a connection from a url. All getConnection calls should go through this code.
      Parameters:
      inCookies - Supply cookie Map (received from prior setCookies calls from server)
      input - boolean indicating whether this connection will be used for input
      output - boolean indicating whether this connection will be used for output
      cache - boolean allow caching (be careful setting this to true for non-static retrievals).
      Returns:
      URLConnection established URL connection.
      Throws:
      IOException - if an I/O error occurs (thrown as unchecked)

    • getActualUrl

      public static URL getActualUrl(String url)