com.databricks.sdk.mixin

Class DbfsExt

java.lang.Object

com.databricks.sdk.service.files.DbfsAPI

com.databricks.sdk.mixin.DbfsExt

public class DbfsExt
extends DbfsAPI

Utility methods for the DBFS API.

This class provides utility methods for the DBFS API that are not part of the official API surface. These methods are subject to change without notice.

Constructor Summary

Constructors

Constructor and Description
DbfsExt(ApiClient apiClient)
DbfsExt(DbfsService mock)

Method Summary

Modifier and Type	Method and Description
OutputStream	getOutputStream(String path) Returns an OutputStream that writes to the given file in DBFS.
InputStream	open(String path) Construct a new java.io.BufferedReader for the given file.
byte[]	readAllBytes(Path path) Returns the contents of the given file as a byte array.
List<String>	readAllLines(Path path, Charset cs) Returns the contents of the given file as a list of strings, where each string corresponds to a line in the file.
Iterable<FileInfo>	recursiveList(String path) Recursively lists files in DBFS, starting from the provided directory.
Path	write(Path path, byte[] bytes) Writes the given bytes to the given file in DBFS.

Methods inherited from class com.databricks.sdk.service.files.DbfsAPI

addBlock, addBlock, close, close, create, create, delete, delete, getStatus, getStatus, impl, list, list, mkdirs, mkdirs, move, move, put, put, read, read

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DbfsExt

public DbfsExt(ApiClient apiClient)

DbfsExt

public DbfsExt(DbfsService mock)

Method Detail

open

public InputStream open(String path)

Construct a new java.io.BufferedReader for the given file.

The returned implementation of InputStream internally buffers calls to the DBFS API to reduce the number of requests made. The buffer has a maximum size of 1 MB, corresponding to the maximum number of bytes that can be read in a single call to the DBFS API. The buffer is refilled when read() is called after the buffer has been exhausted.

Parameters:

path - the path to the file to read

Returns:

an InputStream that reads from the given file in DBFS

readAllBytes

public byte[] readAllBytes(Path path)
                    throws IOException

Returns the contents of the given file as a byte array.

This method is analogous to Files.readAllBytes(path) in Java 8, but it reads the file from DBFS instead of the local filesystem.

Parameters:

path - the path to the file to read

Returns:

the contents of the file as a byte array

Throws:

IOException - if an I/O error occurs

readAllLines

public List<String> readAllLines(Path path,
                                 Charset cs)
                          throws IOException

Returns the contents of the given file as a list of strings, where each string corresponds to a line in the file.

This method is analogous to Files.readAllLines(path, cs) in Java 8, but it reads the file from DBFS instead of the local filesystem.

Parameters:

path - the path to the file to read

cs - the charset to use when decoding the file

Returns:

the contents of the file as a list of strings

Throws:

IOException - if an I/O error occurs

getOutputStream

public OutputStream getOutputStream(String path)

Returns an OutputStream that writes to the given file in DBFS.

This method first creates the given file in DBFS if it does not exist. If the file exists, this method overwrites the existing file. Then, this method constructs an OutputStream that writes to the file in DBFS. The returned OutputStream buffers writes to the file to reduce the number of requests made to the DBFS API. The buffer has a maximum size of 1 MB, corresponding to the maximum number of bytes that can be written in a single call to the DBFS AddBlock API, and is flushed when full.

Parameters:

path - the path to the file to read

Returns:

an OutputStream that writes to the given file in DBFS

write

public Path write(Path path,
                  byte[] bytes)
           throws IOException

Writes the given bytes to the given file in DBFS.

This methods is analogous to Files.write(path, bytes) in Java 8, but it writes the file to DBFS instead of the local filesystem.

Parameters:

path - the path to the file to write

bytes - the bytes to write

Returns:

the path to the file in DBFS

Throws:

IOException - if an I/O error occurs

recursiveList

public Iterable<FileInfo> recursiveList(String path)

Recursively lists files in DBFS, starting from the provided directory.

Parameters:

path - the path to the directory to list

Returns:

an iterable of FileInfo objects, one for each file and directory listing in the directory, recursively