Package org.apache.flink.table.resource

Class ResourceManager

    • Field Detail

      • localResourceDir

        protected final org.apache.flink.core.fs.Path localResourceDir

      • resourceInfos

        protected final Map<org.apache.flink.table.resource.ResourceUri,​URL> resourceInfos

      • userClassLoader

        protected final org.apache.flink.util.MutableURLClassLoader userClassLoader

    • Constructor Detail

      • ResourceManager

        public ResourceManager​(org.apache.flink.configuration.ReadableConfig config,
                       org.apache.flink.util.MutableURLClassLoader userClassLoader)

    • Method Detail

      • createResourceManager

        public static ResourceManager createResourceManager​(URL[] urls,
                                                    ClassLoader parent,
                                                    org.apache.flink.configuration.ReadableConfig config)

      • registerJarResources

        public void registerJarResources​(List<org.apache.flink.table.resource.ResourceUri> resourceUris)
                          throws IOException
        Due to anyone of the resource in list maybe fail during register, so we should stage it before actual register to guarantee transaction process. If all the resources are available, register them into the ResourceManager.
        Throws:
        IOException

      • registerFileResource

        public String registerFileResource​(org.apache.flink.table.resource.ResourceUri resourceUri)
                            throws IOException
        Register a file resource into ResourceManager and return the absolute local file path without the scheme.

        If the file is remote, it will be copied to a local file, with file name suffixed with a UUID.

        Parameters:
        resourceUri - resource with type as ResourceType.FILE, the resource uri might or might not contain the uri scheme, or it could be a relative path.
        Returns:
        the absolute local file path.
        Throws:
        IOException

      • declareFunctionResources

        public void declareFunctionResources​(Set<org.apache.flink.table.resource.ResourceUri> resourceUris)
                              throws IOException
        Declare a resource for function and add it to the function resource infos. If the file is remote, it will be copied to a local file. The declared resource will not be added to resources and classloader if it is not used in the job.
        Parameters:
        resourceUris - the resource uri for function.
        Throws:
        IOException

      • unregisterFunctionResources

        public void unregisterFunctionResources​(List<org.apache.flink.table.resource.ResourceUri> resourceUris)
        Unregister the resource uri in function resources, when the reference count of the resource is 0, the resource will be removed from the function resources.
        Parameters:
        resourceUris - the uris to unregister in function resources.

      • registerPythonResources

        public void registerPythonResources()

      • createUserClassLoader

        public URLClassLoader createUserClassLoader​(List<org.apache.flink.table.resource.ResourceUri> resourceUris)

      • getResources

        public Map<org.apache.flink.table.resource.ResourceUri,​URL> getResources()

      • getLocalJarResources

        public Set<URL> getLocalJarResources()
        Get the local jars' URL. Return the URL corresponding to downloaded jars in the local file system for the remote jar. For the local jar, return the registered URL.

      • exists

        public boolean exists​(org.apache.flink.core.fs.Path filePath)
               throws IOException
        Check whether the Path exists.
        Throws:
        IOException

      • syncFileResource

        public void syncFileResource​(org.apache.flink.table.resource.ResourceUri resourceUri,
                             Consumer<String> resourceGenerator)
                      throws IOException
        Generate a local file resource by the given resource generator and then synchronize to the path identified by the given ResourceUri. The path passed to resource generator should be a local path retrieved from the given ResourceUri.

        NOTE: if the given ResourceUri represents a remote file path like "hdfs://path/to/file.json", then the retrieved local path will be "/localResourceDir/file-${uuid}.json"

        Parameters:
        resourceUri - the file resource uri to synchronize to
        resourceGenerator - a consumer that generates a local copy of the file resource
        Throws:
        IOException

      • checkPath

        protected void checkPath​(org.apache.flink.core.fs.Path path,
                         org.apache.flink.table.resource.ResourceType expectedType)
                  throws IOException
        Throws:
        IOException

      • getURLFromPath

        @VisibleForTesting
protected URL getURLFromPath​(org.apache.flink.core.fs.Path path)
                      throws IOException
        Throws:
        IOException