org.apache.hadoop.hbase

Class MetaTableAccessor

java.lang.Object

org.apache.hadoop.hbase.MetaTableAccessor

@InterfaceAudience.Private
public class MetaTableAccessor
extends Object

Read/write operations on region and assignment information store in hbase:meta. Some of the methods of this class take ZooKeeperWatcher as a param. The only reason for this is because when used on client-side (like from HBaseAdmin), we want to use short-living connection (opened before each operation, closed right after), while when used on HM or HRS (like in AssignmentManager) we want permanent connection.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	MetaTableAccessor.Visitor Implementations 'visit' a catalog table row.

Field Summary

Fields

Modifier and Type	Field and Description
protected static char	META_REPLICA_ID_DELIMITER The delimiter for meta columns for replicaIds > 0

Constructor Summary

Constructors

Constructor and Description
MetaTableAccessor()

Method Summary

Methods

Modifier and Type	Method and Description
static void	addDaughter(HConnection hConnection, HRegionInfo regionInfo, ServerName sn, long openSeqNum) Adds a daughter region entry to meta.
static Put	addDaughtersToPut(Put put, HRegionInfo splitA, HRegionInfo splitB) Adds split daughters to the Put
static Put	addLocation(Put p, ServerName sn, long openSeqNum, int replicaId)
static void	addRegionsToMeta(HConnection hConnection, List<HRegionInfo> regionInfos) Adds a hbase:meta row for each of the specified new regions.
static void	addRegionToMeta(HConnection hConnection, HRegionInfo regionInfo) Adds a hbase:meta row for the specified new region.
static void	addRegionToMeta(HConnection hConnection, HRegionInfo regionInfo, HRegionInfo splitA, HRegionInfo splitB) Adds a (single) hbase:meta row for the specified new region and its daughters.
static void	addRegionToMeta(Table meta, HRegionInfo regionInfo) Adds a hbase:meta row for the specified new region to the given catalog table.
static void	addRegionToMeta(Table meta, HRegionInfo regionInfo, HRegionInfo splitA, HRegionInfo splitB) Adds a (single) hbase:meta row for the specified new region and its daughters.
static void	deleteFromMetaTable(HConnection hConnection, List<Delete> deletes) Delete the passed deletes from the hbase:meta table.
static void	deleteMergeQualifiers(HConnection hConnection, HRegionInfo mergedRegion) Deletes merge qualifiers for the specified merged region.
static void	deleteRegion(HConnection hConnection, HRegionInfo regionInfo) Deletes the specified region from META.
static void	deleteRegions(HConnection hConnection, List<HRegionInfo> regionsInfo) Deletes the specified regions from META.
static List<Result>	fullScan(HConnection hConnection) Performs a full scan of hbase:meta.
static void	fullScan(HConnection hConnection, MetaTableAccessor.Visitor visitor) Performs a full scan of hbase:meta.
static void	fullScan(HConnection hConnection, MetaTableAccessor.Visitor visitor, byte[] startrow) Performs a full scan of a catalog table.
static void	fullScanMetaAndPrint(HConnection hConnection)
static List<Result>	fullScanOfMeta(HConnection hConnection) Performs a full scan of a hbase:meta table.
static PairOfSameType<HRegionInfo>	getDaughterRegions(Result data) Returns the daughter regions by reading the corresponding columns of the catalog table Result.
protected static byte[]	getFamily() Returns the column family used for meta columns.
static HRegionInfo	getHRegionInfo(Result data) Returns HRegionInfo object from the column HConstants.CATALOG_FAMILY:HConstants.REGIONINFO_QUALIFIER of the catalog table Result.
static PairOfSameType<HRegionInfo>	getMergeRegions(Result data) Returns the merge regions by reading the corresponding columns of the catalog table Result.
static byte[]	getMetaKeyForRegion(HRegionInfo regionInfo) Returns the row key to use for this regionInfo
static Pair<HRegionInfo,ServerName>	getRegion(HConnection hConnection, byte[] regionName) Deprecated. use getRegionLocation(HConnection, byte[]) instead
static int	getRegionCount(org.apache.hadoop.conf.Configuration c, String tableName) Count regions in hbase:meta for passed table.
protected static byte[]	getRegionInfoColumn() Returns the column qualifier for serialized region info
static HRegionLocation	getRegionLocation(HConnection hConnection, byte[] regionName) Returns the HRegionLocation from meta for the given region
static HRegionLocation	getRegionLocation(HConnection hConnection, HRegionInfo regionInfo) Returns the HRegionLocation from meta for the given region
static RegionLocations	getRegionLocations(Result r) Returns an HRegionLocationList extracted from the result.
static Result	getRegionResult(HConnection hConnection, byte[] regionName) Gets the result in hbase:meta for the specified region.
static Pair<HRegionInfo,HRegionInfo>	getRegionsFromMergeQualifier(HConnection hConnection, byte[] regionName) Get regions from the merge qualifier of the specified merged region
static Scan	getScanForTableName(TableName tableName) This method creates a Scan object that will only scan catalog rows that belong to the specified table.
static byte[]	getSeqNumColumn(int replicaId) Returns the column qualifier for seqNum column for replicaId
static byte[]	getServerColumn(int replicaId) Returns the column qualifier for server column for replicaId
static NavigableMap<HRegionInfo,Result>	getServerUserRegions(HConnection hConnection, ServerName serverName)
static byte[]	getStartCodeColumn(int replicaId) Returns the column qualifier for server start code column for replicaId
static List<HRegionInfo>	getTableRegions(ZooKeeperWatcher zkw, HConnection hConnection, TableName tableName) Gets all of the regions of the specified table.
static List<HRegionInfo>	getTableRegions(ZooKeeperWatcher zkw, HConnection hConnection, TableName tableName, boolean excludeOfflinedSplitParents) Gets all of the regions of the specified table.
static List<Pair<HRegionInfo,ServerName>>	getTableRegionsAndLocations(ZooKeeperWatcher zkw, HConnection hConnection, TableName tableName)
static List<Pair<HRegionInfo,ServerName>>	getTableRegionsAndLocations(ZooKeeperWatcher zkw, HConnection hConnection, TableName tableName, boolean excludeOfflinedSplitParents)
static Delete	makeDeleteFromRegionInfo(HRegionInfo regionInfo) Generates and returns a Delete containing the region info for the catalog table
static Put	makePutFromRegionInfo(HRegionInfo regionInfo) Generates and returns a Put containing the region into for the catalog table
static void	mergeRegions(HConnection hConnection, HRegionInfo mergedRegion, HRegionInfo regionA, HRegionInfo regionB, ServerName sn) Merge the two regions into one in an atomic operation.
static void	mutateMetaTable(HConnection hConnection, List<Mutation> mutations) Execute the passed mutations against hbase:meta table.
static void	mutateRegions(HConnection hConnection, List<HRegionInfo> regionsToRemove, List<HRegionInfo> regionsToAdd) Adds and Removes the specified regions from hbase:meta
static void	overwriteRegions(HConnection hConnection, List<HRegionInfo> regionInfos) Overwrites the specified regions from hbase:meta
protected static HRegionInfo	parseRegionInfoFromRegionName(byte[] regionName) Returns an HRI parsed from this regionName.
static void	putsToMetaTable(HConnection hConnection, List<Put> ps) Put the passed ps to the hbase:meta table.
static void	removeRegionReplicasFromMeta(Set<byte[]> metaRows, int replicaIndexToDeleteFrom, int numReplicasToRemove, HConnection hConnection) Deletes some replica columns corresponding to replicas for the passed rows
static void	splitRegion(HConnection hConnection, HRegionInfo parent, HRegionInfo splitA, HRegionInfo splitB, ServerName sn) Splits the region into two in an atomic operation.
static boolean	tableExists(HConnection hConnection, TableName tableName) Checks if the specified table exists.
static void	updateRegionLocation(HConnection hConnection, HRegionInfo regionInfo, ServerName sn, long updateSeqNum) Updates the location of the specified region in hbase:meta to be the specified server hostname and startcode.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

META_REPLICA_ID_DELIMITER

protected static final char META_REPLICA_ID_DELIMITER

The delimiter for meta columns for replicaIds > 0

See Also:

Constant Field Values

Constructor Detail

MetaTableAccessor

public MetaTableAccessor()

Method Detail

fullScanOfMeta

public static List<Result> fullScanOfMeta(HConnection hConnection)
                                   throws IOException

Performs a full scan of a hbase:meta table.

Returns:

List of Result

Throws:

IOException

fullScan

public static void fullScan(HConnection hConnection,
            MetaTableAccessor.Visitor visitor)
                     throws IOException

Performs a full scan of hbase:meta.

Parameters:

hConnection - connection we're using

visitor - Visitor invoked against each row.

Throws:

IOException

fullScan

public static List<Result> fullScan(HConnection hConnection)
                             throws IOException

Performs a full scan of hbase:meta.

Parameters:

hConnection - connection we're using

Returns:

List of Result

Throws:

IOException

getRegion

@Deprecated
public static Pair<HRegionInfo,ServerName> getRegion(HConnection hConnection,
                                                byte[] regionName)
                                              throws IOException

Deprecated. use getRegionLocation(HConnection, byte[]) instead

Gets the region info and assignment for the specified region.

Parameters:

hConnection - connection we're using

regionName - Region to lookup.

Returns:

Location and HRegionInfo for regionName

Throws:

IOException

getRegionLocation

public static HRegionLocation getRegionLocation(HConnection hConnection,
                                byte[] regionName)
                                         throws IOException

Returns the HRegionLocation from meta for the given region

Parameters:

hConnection - connection we're using

regionName - region we're looking for

Returns:

HRegionLocation for the given region

Throws:

IOException

getRegionLocation

public static HRegionLocation getRegionLocation(HConnection hConnection,
                                HRegionInfo regionInfo)
                                         throws IOException

Returns the HRegionLocation from meta for the given region

Parameters:

hConnection - connection we're using

regionInfo - region information

Returns:

HRegionLocation for the given region

Throws:

IOException

getMetaKeyForRegion

public static byte[] getMetaKeyForRegion(HRegionInfo regionInfo)

Returns the row key to use for this regionInfo

parseRegionInfoFromRegionName

protected static HRegionInfo parseRegionInfoFromRegionName(byte[] regionName)
                                                    throws IOException

Returns an HRI parsed from this regionName. Not all the fields of the HRI is stored in the name, so the returned object should only be used for the fields in the regionName.

Throws:

IOException

getRegionResult

public static Result getRegionResult(HConnection hConnection,
                     byte[] regionName)
                              throws IOException

Gets the result in hbase:meta for the specified region.

Parameters:

hConnection - connection we're using

regionName - region we're looking for

Returns:

result of the specified region

Throws:

IOException

getRegionsFromMergeQualifier

public static Pair<HRegionInfo,HRegionInfo> getRegionsFromMergeQualifier(HConnection hConnection,
                                                         byte[] regionName)
                                                                  throws IOException

Get regions from the merge qualifier of the specified merged region

Returns:

null if it doesn't contain merge qualifier, else two merge regions

Throws:

IOException

tableExists

public static boolean tableExists(HConnection hConnection,
                  TableName tableName)
                           throws IOException

Checks if the specified table exists. Looks at the hbase:meta table hosted on the specified server.

Parameters:

hConnection - connection we're using

tableName - table to check

Returns:

true if the table exists in meta, false if not

Throws:

IOException

getTableRegions

public static List<HRegionInfo> getTableRegions(ZooKeeperWatcher zkw,
                                HConnection hConnection,
                                TableName tableName)
                                         throws IOException

Gets all of the regions of the specified table.

Parameters:

zkw - zookeeper connection to access meta table

hConnection - connection we're using

tableName - table we're looking for

Returns:

Ordered list of HRegionInfo.

Throws:

IOException

getTableRegions

public static List<HRegionInfo> getTableRegions(ZooKeeperWatcher zkw,
                                HConnection hConnection,
                                TableName tableName,
                                boolean excludeOfflinedSplitParents)
                                         throws IOException

Gets all of the regions of the specified table.

Parameters:

zkw - zookeeper connection to access meta table

hConnection - connection we're using

tableName - table we're looking for

excludeOfflinedSplitParents - If true, do not include offlined split parents in the return.

Returns:

Ordered list of HRegionInfo.

Throws:

IOException

getScanForTableName

public static Scan getScanForTableName(TableName tableName)

This method creates a Scan object that will only scan catalog rows that belong to the specified table. It doesn't specify any columns. This is a better alternative to just using a start row and scan until it hits a new table since that requires parsing the HRI to get the table name.

Parameters:

tableName - bytes of table's name

Returns:

configured Scan object

getTableRegionsAndLocations

public static List<Pair<HRegionInfo,ServerName>> getTableRegionsAndLocations(ZooKeeperWatcher zkw,
                                                             HConnection hConnection,
                                                             TableName tableName)
                                                                      throws IOException,
                                                                             InterruptedException

Parameters:

zkw - zookeeper connection to access meta table

hConnection - connection we're using

tableName - table we're looking for

Returns:

Return list of regioninfos and server.

Throws:

IOException

InterruptedException

getTableRegionsAndLocations

public static List<Pair<HRegionInfo,ServerName>> getTableRegionsAndLocations(ZooKeeperWatcher zkw,
                                                             HConnection hConnection,
                                                             TableName tableName,
                                                             boolean excludeOfflinedSplitParents)
                                                                      throws IOException,
                                                                             InterruptedException

Parameters:

zkw - ZooKeeperWatcher instance we're using to get hbase:meta location

hConnection - connection we're using

tableName - table to work with

Returns:

Return list of regioninfos and server addresses.

Throws:

IOException

InterruptedException

getServerUserRegions

public static NavigableMap<HRegionInfo,Result> getServerUserRegions(HConnection hConnection,
                                                    ServerName serverName)
                                                             throws IOException

Parameters:

hConnection - connection we're using

serverName - server whose regions we're interested in

Returns:

List of user regions installed on this server (does not include catalog regions).

Throws:

IOException

fullScanMetaAndPrint

public static void fullScanMetaAndPrint(HConnection hConnection)
                                 throws IOException

Throws:

IOException

fullScan

public static void fullScan(HConnection hConnection,
            MetaTableAccessor.Visitor visitor,
            byte[] startrow)
                     throws IOException

Performs a full scan of a catalog table.

Parameters:

hConnection - connection we're using

visitor - Visitor invoked against each row.

startrow - Where to start the scan. Pass null if want to begin scan at first row. hbase:meta, the default (pass false to scan hbase:meta)

Throws:

IOException

getFamily

protected static byte[] getFamily()

Returns the column family used for meta columns.

Returns:

HConstants.CATALOG_FAMILY.

getRegionInfoColumn

protected static byte[] getRegionInfoColumn()

Returns the column qualifier for serialized region info

Returns:

HConstants.REGIONINFO_QUALIFIER

getServerColumn

public static byte[] getServerColumn(int replicaId)

Returns the column qualifier for server column for replicaId

Parameters:

replicaId - the replicaId of the region

Returns:

a byte[] for server column qualifier

getStartCodeColumn

public static byte[] getStartCodeColumn(int replicaId)

Returns the column qualifier for server start code column for replicaId

Parameters:

replicaId - the replicaId of the region

Returns:

a byte[] for server start code column qualifier

getSeqNumColumn

public static byte[] getSeqNumColumn(int replicaId)

Returns the column qualifier for seqNum column for replicaId

Parameters:

replicaId - the replicaId of the region

Returns:

a byte[] for seqNum column qualifier

getRegionLocations

public static RegionLocations getRegionLocations(Result r)

Returns an HRegionLocationList extracted from the result.

Returns:

an HRegionLocationList containing all locations for the region range or null if we can't deserialize the result.

getHRegionInfo

public static HRegionInfo getHRegionInfo(Result data)

Returns HRegionInfo object from the column HConstants.CATALOG_FAMILY:HConstants.REGIONINFO_QUALIFIER of the catalog table Result.

Parameters:

data - a Result object from the catalog table scan

Returns:

HRegionInfo or null

getDaughterRegions

public static PairOfSameType<HRegionInfo> getDaughterRegions(Result data)
                                                      throws IOException

Returns the daughter regions by reading the corresponding columns of the catalog table Result.

Parameters:

data - a Result object from the catalog table scan

Returns:

a pair of HRegionInfo or PairOfSameType(null, null) if the region is not a split parent

Throws:

IOException

getMergeRegions

public static PairOfSameType<HRegionInfo> getMergeRegions(Result data)
                                                   throws IOException

Returns the merge regions by reading the corresponding columns of the catalog table Result.

Parameters:

data - a Result object from the catalog table scan

Returns:

a pair of HRegionInfo or PairOfSameType(null, null) if the region is not a split parent

Throws:

IOException

getRegionCount

public static int getRegionCount(org.apache.hadoop.conf.Configuration c,
                 String tableName)
                          throws IOException

Count regions in hbase:meta for passed table.

Parameters:

c - Configuration object

tableName - table name to count regions for

Returns:

Count or regions in table tableName

Throws:

IOException

makePutFromRegionInfo

public static Put makePutFromRegionInfo(HRegionInfo regionInfo)
                                 throws IOException

Generates and returns a Put containing the region into for the catalog table

Throws:

IOException

makeDeleteFromRegionInfo

public static Delete makeDeleteFromRegionInfo(HRegionInfo regionInfo)

Generates and returns a Delete containing the region info for the catalog table

addDaughtersToPut

public static Put addDaughtersToPut(Put put,
                    HRegionInfo splitA,
                    HRegionInfo splitB)

Adds split daughters to the Put

putsToMetaTable

public static void putsToMetaTable(HConnection hConnection,
                   List<Put> ps)
                            throws IOException

Put the passed ps to the hbase:meta table.

Parameters:

hConnection - connection we're using

ps - Put to add to hbase:meta

Throws:

IOException

deleteFromMetaTable

public static void deleteFromMetaTable(HConnection hConnection,
                       List<Delete> deletes)
                                throws IOException

Delete the passed deletes from the hbase:meta table.

Parameters:

hConnection - connection we're using

deletes - Deletes to add to hbase:meta This list should support #remove.

Throws:

IOException

removeRegionReplicasFromMeta

public static void removeRegionReplicasFromMeta(Set<byte[]> metaRows,
                                int replicaIndexToDeleteFrom,
                                int numReplicasToRemove,
                                HConnection hConnection)
                                         throws IOException

Deletes some replica columns corresponding to replicas for the passed rows

Parameters:

metaRows - rows in hbase:meta

replicaIndexToDeleteFrom - the replica ID we would start deleting from

numReplicasToRemove - how many replicas to remove

hConnection - connection we're using to access meta table

Throws:

IOException

mutateMetaTable

public static void mutateMetaTable(HConnection hConnection,
                   List<Mutation> mutations)
                            throws IOException

Execute the passed mutations against hbase:meta table.

Parameters:

hConnection - connection we're using

mutations - Puts and Deletes to execute on hbase:meta

Throws:

IOException

addRegionToMeta

public static void addRegionToMeta(HConnection hConnection,
                   HRegionInfo regionInfo)
                            throws IOException

Adds a hbase:meta row for the specified new region.

Parameters:

hConnection - connection we're using

regionInfo - region information

Throws:

IOException - if problem connecting or updating meta

addRegionToMeta

public static void addRegionToMeta(Table meta,
                   HRegionInfo regionInfo)
                            throws IOException

Adds a hbase:meta row for the specified new region to the given catalog table. The HTable is not flushed or closed.

Parameters:

meta - the HTable for META

regionInfo - region information

Throws:

IOException - if problem connecting or updating meta

addRegionToMeta

public static void addRegionToMeta(Table meta,
                   HRegionInfo regionInfo,
                   HRegionInfo splitA,
                   HRegionInfo splitB)
                            throws IOException

Adds a (single) hbase:meta row for the specified new region and its daughters. Note that this does not add its daughter's as different rows, but adds information about the daughters in the same row as the parent. Use splitRegion(org.apache.hadoop.hbase.client.HConnection, HRegionInfo, HRegionInfo, HRegionInfo, ServerName) if you want to do that.

Parameters:

meta - the HTable for META

regionInfo - region information

splitA - first split daughter of the parent regionInfo

splitB - second split daughter of the parent regionInfo

Throws:

IOException - if problem connecting or updating meta

addRegionToMeta

public static void addRegionToMeta(HConnection hConnection,
                   HRegionInfo regionInfo,
                   HRegionInfo splitA,
                   HRegionInfo splitB)
                            throws IOException

Adds a (single) hbase:meta row for the specified new region and its daughters. Note that this does not add its daughter's as different rows, but adds information about the daughters in the same row as the parent. Use splitRegion(HConnection, HRegionInfo, HRegionInfo, HRegionInfo, ServerName) if you want to do that.

Parameters:

hConnection - connection we're using

regionInfo - region information

splitA - first split daughter of the parent regionInfo

splitB - second split daughter of the parent regionInfo

Throws:

IOException - if problem connecting or updating meta

addRegionsToMeta

public static void addRegionsToMeta(HConnection hConnection,
                    List<HRegionInfo> regionInfos)
                             throws IOException

Adds a hbase:meta row for each of the specified new regions.

Parameters:

hConnection - connection we're using

regionInfos - region information list

Throws:

IOException - if problem connecting or updating meta

addDaughter

public static void addDaughter(HConnection hConnection,
               HRegionInfo regionInfo,
               ServerName sn,
               long openSeqNum)
                        throws NotAllMetaRegionsOnlineException,
                               IOException

Adds a daughter region entry to meta.

Parameters:

regionInfo - the region to put

sn - the location of the region

openSeqNum - the latest sequence number obtained when the region was open

Throws:

NotAllMetaRegionsOnlineException

IOException

mergeRegions

public static void mergeRegions(HConnection hConnection,
                HRegionInfo mergedRegion,
                HRegionInfo regionA,
                HRegionInfo regionB,
                ServerName sn)
                         throws IOException

Merge the two regions into one in an atomic operation. Deletes the two merging regions in hbase:meta and adds the merged region with the information of two merging regions.

Parameters:

hConnection - connection we're using

mergedRegion - the merged region

regionA -

regionB -

sn - the location of the region

Throws:

IOException

splitRegion

public static void splitRegion(HConnection hConnection,
               HRegionInfo parent,
               HRegionInfo splitA,
               HRegionInfo splitB,
               ServerName sn)
                        throws IOException

Splits the region into two in an atomic operation. Offlines the parent region with the information that it is split into two, and also adds the daughter regions. Does not add the location information to the daughter regions since they are not open yet.

Parameters:

hConnection - connection we're using

parent - the parent region which is split

splitA - Split daughter region A

splitB - Split daughter region A

sn - the location of the region

Throws:

IOException

updateRegionLocation

public static void updateRegionLocation(HConnection hConnection,
                        HRegionInfo regionInfo,
                        ServerName sn,
                        long updateSeqNum)
                                 throws IOException

Updates the location of the specified region in hbase:meta to be the specified server hostname and startcode.

Uses passed catalog tracker to get a connection to the server hosting hbase:meta and makes edits to that region.

Parameters:

hConnection - connection we're using

regionInfo - region to update location of

sn - Server name

Throws:

IOException

deleteRegion

public static void deleteRegion(HConnection hConnection,
                HRegionInfo regionInfo)
                         throws IOException

Deletes the specified region from META.

Parameters:

hConnection - connection we're using

regionInfo - region to be deleted from META

Throws:

IOException

deleteRegions

public static void deleteRegions(HConnection hConnection,
                 List<HRegionInfo> regionsInfo)
                          throws IOException

Deletes the specified regions from META.

Parameters:

hConnection - connection we're using

regionsInfo - list of regions to be deleted from META

Throws:

IOException

mutateRegions

public static void mutateRegions(HConnection hConnection,
                 List<HRegionInfo> regionsToRemove,
                 List<HRegionInfo> regionsToAdd)
                          throws IOException

Adds and Removes the specified regions from hbase:meta

Parameters:

hConnection - connection we're using

regionsToRemove - list of regions to be deleted from META

regionsToAdd - list of regions to be added to META

Throws:

IOException

overwriteRegions

public static void overwriteRegions(HConnection hConnection,
                    List<HRegionInfo> regionInfos)
                             throws IOException

Overwrites the specified regions from hbase:meta

Parameters:

hConnection - connection we're using

regionInfos - list of regions to be added to META

Throws:

IOException

deleteMergeQualifiers

public static void deleteMergeQualifiers(HConnection hConnection,
                         HRegionInfo mergedRegion)
                                  throws IOException

Deletes merge qualifiers for the specified merged region.

Parameters:

hConnection - connection we're using

mergedRegion -

Throws:

IOException

addLocation

public static Put addLocation(Put p,
              ServerName sn,
              long openSeqNum,
              int replicaId)