FitsBlock

Instance Constructors

new FitsBlock(hdfsPath: Path, conf: Configuration, hduIndex: Int)

hdfsPath
: (Path) Hadoop path containing informations on the file to read.
conf
: (Configuration) Hadoop configuration containing informations on the run.
hduIndex
: (Int) Index of the HDU to read (zero-based).

Value Members

final def !=(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def !=(arg0: Any): Boolean

Definition Classes
Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def ==(arg0: Any): Boolean

Definition Classes
Any
def BlockBoundaries: (Long, Long, Long, Long)

Return the indices of the first and last bytes of the HDU: hdu_start=header_start, data_start, data_stop, hdu_stop
Return the indices of the first and last bytes of the HDU: hdu_start=header_start, data_start, data_stop, hdu_stop
returns
(Long, Long, Long, Long), the split of the HDU.
final def asInstanceOf[T0]: T0

Definition Classes
Any
val blockBoundaries: (Long, Long, Long, Long)
val blockHeader: Array[String]
def buildHeader(buffer: Array[Byte], prevline: String = ""): List[String]
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
val data: FSDataInputStream
val empty_hdu: Boolean
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
val fs: FileSystem
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
def getColTypes(header: Array[String], col: Int = 0): List[String]

Return the types of elements for each column as a list.
Return the types of elements for each column as a list.
col
: (Int) Column index used for the recursion.
returns
(List[String]), list with the types of elements for each column as given by the header.
def getColumnName(header: Array[String], colIndex: Int): String

Get the name of a column with index colIndex of a HDU.
Get the name of a column with index colIndex of a HDU.
header
: (Array[String]) The header of the HDU.
colIndex
: (Int) Index (zero-based) of a column.
returns
(String), the name of the column.
def getColumnType(header: Array[String], colIndex: Int): String

Get the type of the elements of a column with index colIndex of a HDU.
Get the type of the elements of a column with index colIndex of a HDU.
header
: (Array[String]) The header of the HDU.
colIndex
: (Int) Index (zero-based) of a column.
returns
(String), the type (FITS convention) of the elements of the column.
def getElementFromBuffer(subbuf: Array[Byte], fitstype: String): Any

Companion to readLineFromBuffer.
Companion to readLineFromBuffer. Convert one array of bytes corresponding to one element of the table into its primitive type.
subbuf
: (Array[Byte]) Array of byte describing one element of the table.
fitstype
: (String) The type of this table element according to the header.
returns
the table element converted from binary.
def getHeaderComments(header: Array[String]): HashMap[String, String]

Get the comments of the header.
Get the comments of the header. We assume the comments are written after a backslash (\).
header
: (Array[String]) The header of the HDU.
returns
(HashMap[String, String]), a map of keyword/comment.
def getHeaderKeywords(header: Array[String]): Array[String]

Return the KEYWORDS of the header.
Return the KEYWORDS of the header.
returns
(Array[String]), array with the KEYWORDS of the HDU header.
def getHeaderNames(header: Array[String]): HashMap[String, String]

Get the names of the header.
Get the names of the header. We assume that the names are inside quotes 'my_name'.
header
: (Array[String]) The header of the HDU.
returns
(HashMap[String, String]), a map of keyword/name.
def getHeaderValues(header: Array[String]): HashMap[String, Int]

Return the (KEYWORDS, VALUES) of the header
Return the (KEYWORDS, VALUES) of the header
returns
(HashMap[String, Int]), map array with (keys, values_as_int).
def getNCols(header: Array[String]): Long

Get the number of column of a HDU.
Get the number of column of a HDU. We rely on what's written in the header, meaning here we do not access the data directly.
header
: (Array[String]) The header of the HDU.
returns
(Long), the number of rows as written in KEYWORD=TFIELDS.
def getNHDU: Int

Return the number of HDUs in the file.
Return the number of HDUs in the file.
returns
(Int) the number of HDU.
def getNRows(header: Array[String]): Long

Get the number of row of a HDU.
Get the number of row of a HDU. We rely on what's written in the header, meaning here we do not access the data directly.
header
: (Array[String]) The header of the HDU.
returns
(Long), the number of rows as written in KEYWORD=NAXIS2.
def getSizeRowBytes(header: Array[String]): Int

Get the size (bytes) of each row of a HDU.
Get the size (bytes) of each row of a HDU. We rely on what's written in the header, meaning here we do not access the data directly.
header
: (Array[String]) The header of the HDU.
returns
(Int), the size (bytes) of one row as written in KEYWORD=NAXIS1.
def getSplitLocation(fitstype: String): Int

Companion routine to rowSplitLocations.
Companion routine to rowSplitLocations. Returns the size of a primitive according to its type from the FITS header.
fitstype
: (String) Element type according to FITS standards (I, J, K, E, D, L, A, etc)
returns
(Int), the size (bytes) of the element.
def hashCode(): Int

Definition Classes
AnyRef → Any
val isHDUBelowMax: Boolean
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
val ncols: Int
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
val numberOfHdus: Int
def readHeader: Array[String]

Read the header of a HDU.
Read the header of a HDU. The cursor needs to be at the start of the header. We assume that each header row has a standard size of 80 Bytes, and the total size of the header is 2880 Bytes.
returns
(Array[String) the header is an array of Strings, each String being one line of the header.
def readHeader(position: Long): Array[String]

Read a header at a given position
Read a header at a given position
position
: (Long) The byte index to seek in the file. Need to correspond to a valid header position. Use in combination with BlockBoundaries._1 for example.
returns
(Array[String) the header is an array of Strings, each String being one line of the header.
def readLineFromBuffer(buf: Array[Byte], col: Int = 0): List[_]

Convert binary row into row.
Convert binary row into row. You need to have the cursor at the beginning of a row. Example
```
// Set the cursor at the beginning of the data block
setCursor(BlockBoundaries._2)
// Initialise your binary row
val buffer = Array[Byte](size_of_one_row_in_bytes)
// Read the first binary row into buffer
data.read(buffer, 0, size_of_one_row_in_bytes)
// Convert buffer
val myrow = readLineFromBuffer(buffer)
```
buf
: (Array[Byte]) Row of byte read from the data block.
col
: (Int=0) Index of the column (used for the recursion).
returns
(List[_]) The row as list of elements (float, int, string, etc.) as given by the header.
def registerBlockBoundaries(sep: String = ";;"): Unit

Register the boundaries of the HDU in the Hadoop configuration.
Register the boundaries of the HDU in the Hadoop configuration. By doing this, we broadcast the values to the executors. It is sent as a long String, and can be read properly afterwards using retrieveBlockBoundaries. Make sure you use the same separators.
sep
: (String) Line separator used to form the String. Default is ;;
def registerHeader(sep: String = ";;"): Unit

Register the header in the Hadoop configuration.
Register the header in the Hadoop configuration. By doing this, we broadcast the header to the executors. The header is sent as a long String, and can be read properly afterwards using retrieveHeader. Make sure you use the same separators.
sep
: (String) Line separator used to form the String. Default is ;;
def resetCursorAtData: Unit

Place the cursor at the beginning of the data of the block
def resetCursorAtHeader: Unit

Place the cursor at the beginning of the header of the block
def retrieveBlockBoundaries(sep: String = ";;"): (Long, Long, Long, Long)

Retrieve the blockboundaries from the Hadoop configuration.
Retrieve the blockboundaries from the Hadoop configuration. Make sure you use the same separators as in registerBlockBoundaries.
sep
: (String) Line separator used to split the String. Default is ;;
returns
the block boundaries as Tuple4 of Long. See BlockBoundaries.
def retrieveHeader(sep: String = ";;"): Array[String]

Retrieve the header from the Hadoop configuration.
Retrieve the header from the Hadoop configuration. Make sure you use the same separators as in registerHeader.
sep
: (String) Line separator used to split the String. Default is ;;
returns
the header as Array[String]. See readHeader.
def rowSplitLocations(col: Int = 0): List[Int]

Description of a row in terms of bytes indices.
Description of a row in terms of bytes indices. rowSplitLocations returns an array containing the position of elements (byte index) in a row. Example if we have a row with [20A, E, E], one will have rowSplitLocations -> [0, 20, 24, 28] that is a string on 20 Bytes, followed by 2 floats on 4 bytes each.
col
: (Int) Column position used for the recursion. Should be left at 0.
returns
(List[Int]), the position of elements (byte index) in a row.
val rowTypes: List[String]
def setCursor(position: Long): Unit

Set the cursor at the position (byte index, Long).
Set the cursor at the position (byte index, Long).
position
: (Long) The byte index to seek in the file.
val splitLocations: List[Int]
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toString(): String

Definition Classes
AnyRef → Any
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )

class FitsBlock extends AnyRef

Instance Constructors

new FitsBlock(hdfsPath: Path, conf: Configuration, hduIndex: Int)

Value Members

final def !=(arg0: AnyRef): Boolean

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: AnyRef): Boolean

final def ==(arg0: Any): Boolean

def BlockBoundaries: (Long, Long, Long, Long)

final def asInstanceOf[T0]: T0

val blockBoundaries: (Long, Long, Long, Long)

val blockHeader: Array[String]

def buildHeader(buffer: Array[Byte], prevline: String = ""): List[String]

def clone(): AnyRef

val data: FSDataInputStream

val empty_hdu: Boolean

final def eq(arg0: AnyRef): Boolean

def equals(arg0: Any): Boolean

def finalize(): Unit

val fs: FileSystem

final def getClass(): Class[_]

def getColTypes(header: Array[String], col: Int = 0): List[String]

def getColumnName(header: Array[String], colIndex: Int): String

def getColumnType(header: Array[String], colIndex: Int): String

def getElementFromBuffer(subbuf: Array[Byte], fitstype: String): Any

def getHeaderComments(header: Array[String]): HashMap[String, String]

def getHeaderKeywords(header: Array[String]): Array[String]

def getHeaderNames(header: Array[String]): HashMap[String, String]

def getHeaderValues(header: Array[String]): HashMap[String, Int]

def getNCols(header: Array[String]): Long

def getNHDU: Int

def getNRows(header: Array[String]): Long

def getSizeRowBytes(header: Array[String]): Int

def getSplitLocation(fitstype: String): Int

def hashCode(): Int

val isHDUBelowMax: Boolean

final def isInstanceOf[T0]: Boolean

val ncols: Int

final def ne(arg0: AnyRef): Boolean

final def notify(): Unit

final def notifyAll(): Unit

val numberOfHdus: Int

def readHeader: Array[String]

def readHeader(position: Long): Array[String]

def readLineFromBuffer(buf: Array[Byte], col: Int = 0): List[_]

def registerBlockBoundaries(sep: String = ";;"): Unit

def registerHeader(sep: String = ";;"): Unit

def resetCursorAtData: Unit

def resetCursorAtHeader: Unit

def retrieveBlockBoundaries(sep: String = ";;"): (Long, Long, Long, Long)

def retrieveHeader(sep: String = ";;"): Array[String]

def rowSplitLocations(col: Int = 0): List[Int]

val rowTypes: List[String]

def setCursor(position: Long): Unit

val splitLocations: List[Int]

final def synchronized[T0](arg0: ⇒ T0): T0

def toString(): String

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

Inherited from AnyRef

Inherited from Any

Ungrouped