org.elasticsearch.index.store

Class Store.MetadataSnapshot

java.lang.Object

org.elasticsearch.index.store.Store.MetadataSnapshot

All Implemented Interfaces:

Iterable<StoreFileMetaData>, StreamableReader<Store.MetadataSnapshot>, Writeable<Store.MetadataSnapshot>

Enclosing class:

Store

public static final class Store.MetadataSnapshot
extends Object
implements Iterable<StoreFileMetaData>, Writeable<Store.MetadataSnapshot>

Represents a snapshot of the current directory build from the latest Lucene commit. Only files that are part of the last commit are considered in this datastrucutre. For backwards compatibility the snapshot might include legacy checksums that are derived from a dedicated checksum file written by older elasticsearch version pre 1.3

Note: This class will ignore the segments.gen file since it's optional and might change concurrently for safety reasons.

See Also:

StoreFileMetaData

Field Summary

Fields

Modifier and Type	Field and Description
static Store.MetadataSnapshot	EMPTY

Constructor Summary

Constructors

Constructor and Description
Store.MetadataSnapshot(Map<String,StoreFileMetaData> metadata, Map<String,String> commitUserData, long numDocs)
Store.MetadataSnapshot(StreamInput in)

Method Summary

Methods

Modifier and Type	Method and Description
Map<String,StoreFileMetaData>	asMap()
boolean	contains(String existingFile) Returns true iff this metadata contains the given file.
StoreFileMetaData	get(String name)
Map<String,String>	getCommitUserData()
long	getNumDocs() Returns the number of documents in this store snapshot
StoreFileMetaData	getSegmentsFile() Returns the segments file that this metadata snapshot represents or null if the snapshot is empty.
String	getSyncId() Returns the sync id of the commit point that this MetadataSnapshot represents.
static void	hashFile(org.apache.lucene.util.BytesRefBuilder fileHash, InputStream in, long size) Computes a strong hash value for small files.
static org.apache.lucene.util.BytesRef	hashFile(org.apache.lucene.store.Directory directory, String file) Computes a strong hash value for small files.
Iterator<StoreFileMetaData>	iterator()
Store.MetadataSnapshot	readFrom(StreamInput in) Reads a copy of an object with the same type form the stream input The caller object remains unchanged.
Store.RecoveryDiff	recoveryDiff(Store.MetadataSnapshot recoveryTargetSnapshot) Returns a diff between the two snapshots that can be used for recovery.
int	size() Returns the number of files in this snapshot
void	writeTo(StreamOutput out) Writes the current object into the output stream out

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

EMPTY

public static final Store.MetadataSnapshot EMPTY

Constructor Detail

Store.MetadataSnapshot

public Store.MetadataSnapshot(Map<String,StoreFileMetaData> metadata,
                      Map<String,String> commitUserData,
                      long numDocs)

Store.MetadataSnapshot

public Store.MetadataSnapshot(StreamInput in)
                       throws IOException

Throws:

IOException

Method Detail

getNumDocs

public long getNumDocs()

Returns the number of documents in this store snapshot

hashFile

public static org.apache.lucene.util.BytesRef hashFile(org.apache.lucene.store.Directory directory,
                                       String file)
                                                throws IOException

Computes a strong hash value for small files. Note that this method should only be used for files < 1MB

Throws:

IOException

hashFile

public static void hashFile(org.apache.lucene.util.BytesRefBuilder fileHash,
            InputStream in,
            long size)
                     throws IOException

Computes a strong hash value for small files. Note that this method should only be used for files < 1MB

Throws:

IOException

iterator

public Iterator<StoreFileMetaData> iterator()

Specified by:

iterator in interface Iterable<StoreFileMetaData>

get

public StoreFileMetaData get(String name)

asMap

public Map<String,StoreFileMetaData> asMap()

recoveryDiff

public Store.RecoveryDiff recoveryDiff(Store.MetadataSnapshot recoveryTargetSnapshot)

Returns a diff between the two snapshots that can be used for recovery. The given snapshot is treated as the recovery target and this snapshot as the source. The returned diff will hold a list of files that are:

• identical: they exist in both snapshots and they can be considered the same ie. they don't need to be recovered
• different: they exist in both snapshots but their they are not identical
• missing: files that exist in the source but not in the target

This method groups file into per-segment files and per-commit files. A file is treated as identical if and on if all files in it's group are identical. On a per-segment level files for a segment are treated as identical iff:

• all files in this segment have the same checksum
• all files in this segment have the same length
• the segments .si files hashes are byte-identical Note: This is a using a perfect hash function, The metadata transfers the .si file content as it's hash

The .si file contains a lot of diagnostics including a timestamp etc. in the future there might be unique segment identifiers in there hardening this method further.

The per-commit files handles very similar. A commit is composed of the segments_N files as well as generational files like deletes (_x_y.del) or field-info (_x_y.fnm) files. On a per-commit level files for a commit are treated as identical iff:

• all files belonging to this commit have the same checksum
• all files belonging to this commit have the same length
• the segments file segments_N files hashes are byte-identical Note: This is a using a perfect hash function, The metadata transfers the segments_N file content as it's hash

NOTE: this diff will not contain the segments.gen file. This file is omitted on recovery.

size

public int size()

Returns the number of files in this snapshot

writeTo

public void writeTo(StreamOutput out)
             throws IOException

Description copied from interface: Writeable

Writes the current object into the output stream out

Specified by:

writeTo in interface Writeable<Store.MetadataSnapshot>

Throws:

IOException

getCommitUserData

public Map<String,String> getCommitUserData()

contains

public boolean contains(String existingFile)

Returns true iff this metadata contains the given file.

getSegmentsFile

public StoreFileMetaData getSegmentsFile()

Returns the segments file that this metadata snapshot represents or null if the snapshot is empty.

getSyncId

public String getSyncId()

Returns the sync id of the commit point that this MetadataSnapshot represents.

Returns:

sync id if exists, else null

readFrom

public Store.MetadataSnapshot readFrom(StreamInput in)
                                throws IOException

Description copied from interface: StreamableReader

Reads a copy of an object with the same type form the stream input The caller object remains unchanged.

Specified by:

readFrom in interface StreamableReader<Store.MetadataSnapshot>

Throws:

IOException