@InterfaceAudience.Private public class MetaTableAccessor extends Object
Read/write operations on hbase:meta
region as well as assignment information stored
to hbase:meta
.
Some of the methods of this class take ZooKeeperWatcher as a param. The only reason for this is when this class is used on client-side (e.g. HBaseAdmin), we want to use short-lived connection (opened before each operation, closed right after), while when used on HM or HRS (like in AssignmentManager) we want permanent connection.
HBASE-10070 adds a replicaId to HRI, meaning more than one HRI can be defined for the same table range (table, startKey, endKey). For every range, there will be at least one HRI defined which is called default replica.
For each table there is single row named for the table with a 'table' column family. The column family currently has one column in it, the 'state' column: table:state => contains table state Then for each table range ('Region'), there is a single row, formatted as: <tableName>,<startKey>,<regionId>,<encodedRegionName>. This row is the serialized regionName of the default region replica. Columns are: info:regioninfo => contains serialized HRI for the default region replica info:server => contains hostname:port (in string form) for the server hosting the default regionInfo replica info:server_<replicaId> => contains hostname:port (in string form) for the server hosting the regionInfo replica with replicaId info:serverstartcode => contains server start code (in binary long form) for the server hosting the default regionInfo replica info:serverstartcode_<replicaId> => contains server start code (in binary long form) for the server hosting the regionInfo replica with replicaId info:seqnumDuringOpen => contains seqNum (in binary long form) for the region at the time the server opened the region with default replicaId info:seqnumDuringOpen_<replicaId> => contains seqNum (in binary long form) for the region at the time the server opened the region with replicaId info:splitA => contains a serialized HRI for the first daughter region if the region is split info:splitB => contains a serialized HRI for the second daughter region if the region is split info:merge* => contains a serialized HRI for a merge parent region. There will be two or more of these columns in a row. A row that has these columns is undergoing a merge and is the result of the merge. Columns listed in marge* columns are the parents of this merged region. Example columns: info:merge0001, info:merge0002. You make also see 'mergeA', and 'mergeB'. This is old form replaced by the new format that allows for more than two parents to be merged at a time. TODO: Add rep_barrier for serial replication explaination.
The actual layout of meta should be encapsulated inside MetaTableAccessor methods, and should not leak out of it (through Result objects, etc)
限定符和类型 | 类和说明 |
---|---|
static interface |
MetaTableAccessor.CloseableVisitor
Implementations 'visit' a catalog table row but with close() at the end.
|
static class |
MetaTableAccessor.DefaultVisitorBase
A Visitor that skips offline regions and split parents
|
static class |
MetaTableAccessor.QueryType |
static class |
MetaTableAccessor.ReplicationBarrierResult |
static class |
MetaTableAccessor.TableVisitorBase
A Visitor for a table.
|
static interface |
MetaTableAccessor.Visitor
Implementations 'visit' a catalog table row.
|
限定符和类型 | 字段和说明 |
---|---|
static byte[] |
REPLICATION_PARENT_QUALIFIER |
构造器和说明 |
---|
MetaTableAccessor() |
限定符和类型 | 方法和说明 |
---|---|
static Put |
addLocation(Put p,
ServerName sn,
long openSeqNum,
int replicaId) |
static Put |
addRegionInfo(Put p,
RegionInfo hri) |
static void |
addRegionsToMeta(Connection connection,
List<RegionInfo> regionInfos,
int regionReplication)
Adds a hbase:meta row for each of the specified new regions.
|
static void |
addRegionToMeta(Connection connection,
RegionInfo regionInfo)
Adds a (single) hbase:meta row for the specified new region and its daughters.
|
static void |
addReplicationBarrier(Put put,
long openSeqNum) |
static void |
addSplitsToParent(Connection connection,
RegionInfo regionInfo,
RegionInfo splitA,
RegionInfo splitB)
Adds daughter region infos to hbase:meta row for the specified region.
|
static void |
deleteMergeQualifiers(Connection connection,
RegionInfo mergeRegion)
Deletes merge qualifiers for the specified merge region.
|
static void |
deleteRegionInfo(Connection connection,
RegionInfo regionInfo)
Deletes the specified region from META.
|
static void |
deleteRegionInfos(Connection connection,
List<RegionInfo> regionsInfo)
Deletes the specified regions from META.
|
static void |
deleteTableState(Connection connection,
TableName table)
Remove state for table from meta
|
static void |
fullScanMetaAndPrint(Connection connection) |
static List<Result> |
fullScanRegions(Connection connection)
Performs a full scan of
hbase:meta for regions. |
static void |
fullScanRegions(Connection connection,
MetaTableAccessor.Visitor visitor)
Performs a full scan of
hbase:meta for regions. |
static void |
fullScanTables(Connection connection,
MetaTableAccessor.Visitor visitor)
Performs a full scan of
hbase:meta for tables. |
static List<RegionInfo> |
getAllRegions(Connection connection,
boolean excludeOfflinedSplitParents)
Lists all of the regions currently in META.
|
static byte[] |
getCatalogFamily()
Returns the column family used for meta columns.
|
static PairOfSameType<RegionInfo> |
getDaughterRegions(Result data)
Returns the daughter regions by reading the corresponding columns of the catalog table
Result.
|
static List<RegionInfo> |
getMergeRegions(Cell[] cells) |
static List<RegionInfo> |
getMergeRegions(Connection connection,
byte[] regionName) |
static Table |
getMetaHTable(Connection connection)
Callers should call close on the returned
Table instance. |
static byte[] |
getMetaKeyForRegion(RegionInfo regionInfo)
Returns the row key to use for this regionInfo
|
static byte[] |
getParentsBytes(List<RegionInfo> parents) |
static Pair<RegionInfo,ServerName> |
getRegion(Connection connection,
byte[] regionName)
已过时。
use
getRegionLocation(Connection, byte[]) instead |
static int |
getRegionCount(org.apache.hadoop.conf.Configuration c,
TableName tableName)
Count regions in
hbase:meta for passed table. |
static int |
getRegionCount(Connection connection,
TableName tableName)
Count regions in
hbase:meta for passed table. |
static RegionInfo |
getRegionInfo(Result data)
Returns RegionInfo object from the column
HConstants.CATALOG_FAMILY:HConstants.REGIONINFO_QUALIFIER of the catalog
table Result.
|
static RegionInfo |
getRegionInfo(Result r,
byte[] qualifier)
Returns the RegionInfo object from the column
HConstants.CATALOG_FAMILY and
qualifier of the catalog table result. |
static byte[] |
getRegionInfoColumn()
Returns the column qualifier for serialized region info
|
static HRegionLocation |
getRegionLocation(Connection connection,
byte[] regionName)
Returns the HRegionLocation from meta for the given region
|
static HRegionLocation |
getRegionLocation(Connection connection,
RegionInfo regionInfo)
Returns the HRegionLocation from meta for the given region
|
static RegionLocations |
getRegionLocations(Result r)
Returns an HRegionLocationList extracted from the result.
|
static Result |
getRegionResult(Connection connection,
byte[] regionName)
Gets the result in hbase:meta for the specified region.
|
static long[] |
getReplicationBarrier(Connection conn,
byte[] regionName) |
static MetaTableAccessor.ReplicationBarrierResult |
getReplicationBarrierResult(Connection conn,
TableName tableName,
byte[] row,
byte[] encodedRegionName) |
static long[] |
getReplicationBarriers(Result result) |
static Scan |
getScanForTableName(Connection connection,
TableName tableName)
已过时。
|
static byte[] |
getSeqNumColumn(int replicaId)
Returns the column qualifier for seqNum column for replicaId
|
static byte[] |
getServerColumn(int replicaId)
Returns the column qualifier for server column for replicaId
|
static ServerName |
getServerName(Result r,
int replicaId)
Returns a
ServerName from catalog table Result . |
static NavigableMap<RegionInfo,Result> |
getServerUserRegions(Connection connection,
ServerName serverName) |
static byte[] |
getStartCodeColumn(int replicaId)
Returns the column qualifier for server start code column for replicaId
|
static List<Pair<String,Long>> |
getTableEncodedRegionNameAndLastBarrier(Connection conn,
TableName tableName) |
static List<String> |
getTableEncodedRegionNamesForSerialReplication(Connection conn,
TableName tableName) |
static List<RegionInfo> |
getTableRegions(Connection connection,
TableName tableName)
Gets all of the regions of the specified table.
|
static List<RegionInfo> |
getTableRegions(Connection connection,
TableName tableName,
boolean excludeOfflinedSplitParents)
Gets all of the regions of the specified table.
|
static List<Pair<RegionInfo,ServerName>> |
getTableRegionsAndLocations(Connection connection,
TableName tableName)
Do not use this method to get meta table regions, use methods in MetaTableLocator instead.
|
static List<Pair<RegionInfo,ServerName>> |
getTableRegionsAndLocations(Connection connection,
TableName tableName,
boolean excludeOfflinedSplitParents)
Do not use this method to get meta table regions, use methods in MetaTableLocator instead.
|
static byte[] |
getTableStartRowForMeta(TableName tableName,
MetaTableAccessor.QueryType type) |
static TableState |
getTableState(Connection conn,
TableName tableName)
Fetch table state for given table from META table
|
static TableState |
getTableState(Result r)
Decode table state from META Result.
|
static Map<TableName,TableState> |
getTableStates(Connection conn)
Fetch table states from META table
|
static byte[] |
getTableStopRowForMeta(TableName tableName,
MetaTableAccessor.QueryType type) |
static boolean |
hasMergeRegions(Cell[] cells) |
static Put |
makePutForReplicationBarrier(RegionInfo regionInfo,
long openSeqNum,
long ts) |
static Put |
makePutFromRegionInfo(RegionInfo regionInfo,
long ts)
Generates and returns a Put containing the region into for the catalog table
|
static Put |
makePutFromTableState(TableState state,
long ts)
Construct PUT for given state
|
static void |
mergeRegions(Connection connection,
RegionInfo mergedRegion,
Map<RegionInfo,Long> parentSeqNum,
ServerName sn,
int regionReplication)
Merge regions into one in an atomic operation.
|
static void |
overwriteRegions(Connection connection,
List<RegionInfo> regionInfos,
int regionReplication)
Overwrites the specified regions from hbase:meta.
|
static RegionInfo |
parseRegionInfoFromRegionName(byte[] regionName)
Returns an HRI parsed from this regionName.
|
static void |
putsToMetaTable(Connection connection,
List<Put> ps)
Put the passed
ps to the hbase:meta table. |
static void |
removeRegionReplicasFromMeta(Set<byte[]> metaRows,
int replicaIndexToDeleteFrom,
int numReplicasToRemove,
Connection connection)
Deletes some replica columns corresponding to replicas for the passed rows
|
static Result |
scanByRegionEncodedName(Connection connection,
String regionEncodedName)
Scans META table for a row whose key contains the specified regionEncodedName,
returning a single related
Result instance if any row is found, null otherwise. |
static void |
scanMeta(Connection connection,
MetaTableAccessor.Visitor visitor,
TableName tableName,
byte[] row,
int rowLimit)
Performs a scan of META table for given table starting from given row.
|
static void |
scanMetaForTableRegions(Connection connection,
MetaTableAccessor.Visitor visitor,
TableName tableName) |
static void |
splitRegion(Connection connection,
RegionInfo parent,
long parentOpenSeqNum,
RegionInfo splitA,
RegionInfo splitB,
ServerName sn,
int regionReplication)
Splits the region into two in an atomic operation.
|
static boolean |
tableExists(Connection connection,
TableName tableName)
Checks if the specified table exists.
|
static void |
updateRegionLocation(Connection connection,
RegionInfo regionInfo,
ServerName sn,
long openSeqNum,
long masterSystemTime)
Updates the location of the specified region in hbase:meta to be the specified server hostname
and startcode.
|
static void |
updateTableState(Connection conn,
TableName tableName,
TableState.State actual)
Updates state in META
|
public static void fullScanRegions(Connection connection, MetaTableAccessor.Visitor visitor) throws IOException
hbase:meta
for regions.connection
- connection we're usingvisitor
- Visitor invoked against each row in regions family.IOException
public static List<Result> fullScanRegions(Connection connection) throws IOException
hbase:meta
for regions.connection
- connection we're usingIOException
public static void fullScanTables(Connection connection, MetaTableAccessor.Visitor visitor) throws IOException
hbase:meta
for tables.connection
- connection we're usingvisitor
- Visitor invoked against each row in tables family.IOException
public static Table getMetaHTable(Connection connection) throws IOException
Table
instance.connection
- connection we're using to access MetaTable
for hbase:meta
IOException
@Deprecated public static Pair<RegionInfo,ServerName> getRegion(Connection connection, byte[] regionName) throws IOException
getRegionLocation(Connection, byte[])
insteadconnection
- connection we're usingregionName
- Region to lookup.regionName
IOException
public static HRegionLocation getRegionLocation(Connection connection, byte[] regionName) throws IOException
connection
- connection we're usingregionName
- region we're looking forIOException
public static HRegionLocation getRegionLocation(Connection connection, RegionInfo regionInfo) throws IOException
connection
- connection we're usingregionInfo
- region informationIOException
public static byte[] getMetaKeyForRegion(RegionInfo regionInfo)
public static RegionInfo parseRegionInfoFromRegionName(byte[] regionName) throws IOException
IOException
public static Result getRegionResult(Connection connection, byte[] regionName) throws IOException
connection
- connection we're usingregionName
- region we're looking forIOException
public static Result scanByRegionEncodedName(Connection connection, String regionEncodedName) throws IOException
Result
instance if any row is found, null otherwise.connection
- the connection to query META table.regionEncodedName
- the region encoded name to look for at META.Result
instance with the row related info in META, null otherwise.IOException
- if any errors occur while querying META.@Nullable public static List<RegionInfo> getMergeRegions(Connection connection, byte[] regionName) throws IOException
regionName
row.IOException
@Nullable public static List<RegionInfo> getMergeRegions(Cell[] cells)
cells
.public static boolean hasMergeRegions(Cell[] cells)
cells
; i.e.
the column in cell
matches the regex 'info:merge.*'.public static boolean tableExists(Connection connection, TableName tableName) throws IOException
connection
- connection we're usingtableName
- table to checkIOException
public static List<RegionInfo> getAllRegions(Connection connection, boolean excludeOfflinedSplitParents) throws IOException
connection
- to connect withexcludeOfflinedSplitParents
- False if we are to include offlined/splitparents regions,
true and we'll leave out offlined regions from returned listIOException
public static List<RegionInfo> getTableRegions(Connection connection, TableName tableName) throws IOException
connection
- connection we're usingtableName
- table we're looking forRegionInfo
.IOException
public static List<RegionInfo> getTableRegions(Connection connection, TableName tableName, boolean excludeOfflinedSplitParents) throws IOException
connection
- connection we're usingtableName
- table we're looking forexcludeOfflinedSplitParents
- If true, do not include offlined split
parents in the return.RegionInfo
.IOException
public static byte[] getTableStartRowForMeta(TableName tableName, MetaTableAccessor.QueryType type)
tableName
- table we're working withpublic static byte[] getTableStopRowForMeta(TableName tableName, MetaTableAccessor.QueryType type)
tableName
- table we're working with@Deprecated public static Scan getScanForTableName(Connection connection, TableName tableName)
tableName
- bytes of table's namepublic static List<Pair<RegionInfo,ServerName>> getTableRegionsAndLocations(Connection connection, TableName tableName) throws IOException
connection
- connection we're usingtableName
- table we're looking forIOException
public static List<Pair<RegionInfo,ServerName>> getTableRegionsAndLocations(Connection connection, @Nullable TableName tableName, boolean excludeOfflinedSplitParents) throws IOException
connection
- connection we're usingtableName
- table to work with, can be null for getting all regionsexcludeOfflinedSplitParents
- don't return split parentsIOException
public static NavigableMap<RegionInfo,Result> getServerUserRegions(Connection connection, ServerName serverName) throws IOException
connection
- connection we're usingserverName
- server whose regions we're interested inIOException
public static void fullScanMetaAndPrint(Connection connection) throws IOException
IOException
public static void scanMetaForTableRegions(Connection connection, MetaTableAccessor.Visitor visitor, TableName tableName) throws IOException
IOException
public static void scanMeta(Connection connection, MetaTableAccessor.Visitor visitor, TableName tableName, byte[] row, int rowLimit) throws IOException
connection
- connection we're usingvisitor
- visitor to calltableName
- table withing we scanrow
- start scan from this rowrowLimit
- max number of rows to returnIOException
public static byte[] getCatalogFamily()
public static byte[] getRegionInfoColumn()
public static byte[] getServerColumn(int replicaId)
replicaId
- the replicaId of the regionpublic static byte[] getStartCodeColumn(int replicaId)
replicaId
- the replicaId of the regionpublic static byte[] getSeqNumColumn(int replicaId)
replicaId
- the replicaId of the region@Nullable @InterfaceAudience.Private public static ServerName getServerName(Result r, int replicaId)
ServerName
from catalog table Result
.r
- Result to pull frompublic static PairOfSameType<RegionInfo> getDaughterRegions(Result data)
data
- a Result object from the catalog table scan@Nullable public static RegionLocations getRegionLocations(Result r)
public static RegionInfo getRegionInfo(Result data)
data
- a Result object from the catalog table scan@Nullable public static RegionInfo getRegionInfo(Result r, byte[] qualifier)
HConstants.CATALOG_FAMILY
and
qualifier
of the catalog table result.r
- a Result object from the catalog table scanqualifier
- Column family qualifier@Nullable public static TableState getTableState(Connection conn, TableName tableName) throws IOException
conn
- connection to usetableName
- table to fetch state forIOException
public static Map<TableName,TableState> getTableStates(Connection conn) throws IOException
conn
- connection to useIOException
public static void updateTableState(Connection conn, TableName tableName, TableState.State actual) throws IOException
conn
- connection to usetableName
- table to look forIOException
@Nullable public static TableState getTableState(Result r) throws IOException
IOException
public static int getRegionCount(org.apache.hadoop.conf.Configuration c, TableName tableName) throws IOException
hbase:meta
for passed table.c
- Configuration objecttableName
- table name to count regions fortableName
IOException
public static int getRegionCount(Connection connection, TableName tableName) throws IOException
hbase:meta
for passed table.connection
- Connection objecttableName
- table name to count regions fortableName
IOException
public static Put makePutFromRegionInfo(RegionInfo regionInfo, long ts) throws IOException
IOException
public static void putsToMetaTable(Connection connection, List<Put> ps) throws IOException
ps
to the hbase:meta
table.connection
- connection we're usingps
- Put to add to hbase:metaIOException
public static void removeRegionReplicasFromMeta(Set<byte[]> metaRows, int replicaIndexToDeleteFrom, int numReplicasToRemove, Connection connection) throws IOException
metaRows
- rows in hbase:metareplicaIndexToDeleteFrom
- the replica ID we would start deleting fromnumReplicasToRemove
- how many replicas to removeconnection
- connection we're using to access meta tableIOException
public static void addSplitsToParent(Connection connection, RegionInfo regionInfo, RegionInfo splitA, RegionInfo splitB) throws IOException
splitRegion(Connection, RegionInfo, long, RegionInfo, RegionInfo, ServerName, int)
if you want to do that.connection
- connection we're usingregionInfo
- RegionInfo of parent regionsplitA
- first split daughter of the parent regionInfosplitB
- second split daughter of the parent regionInfoIOException
- if problem connecting or updating metapublic static void addRegionToMeta(Connection connection, RegionInfo regionInfo) throws IOException
splitRegion(Connection, RegionInfo, long, RegionInfo, RegionInfo, ServerName, int)
if you want to do that.connection
- connection we're usingregionInfo
- region informationIOException
- if problem connecting or updating metapublic static void addRegionsToMeta(Connection connection, List<RegionInfo> regionInfos, int regionReplication) throws IOException
connection
- connection we're usingregionInfos
- region information listIOException
- if problem connecting or updating metapublic static void mergeRegions(Connection connection, RegionInfo mergedRegion, Map<RegionInfo,Long> parentSeqNum, ServerName sn, int regionReplication) throws IOException
connection
- connection we're usingmergedRegion
- the merged regionparentSeqNum
- Parent regions to merge and their next open sequence id used
by serial replication. Set to -1 if not needed by this table.sn
- the location of the regionIOException
public static void splitRegion(Connection connection, RegionInfo parent, long parentOpenSeqNum, RegionInfo splitA, RegionInfo splitB, ServerName sn, int regionReplication) throws IOException
connection
- connection we're usingparent
- the parent region which is splitparentOpenSeqNum
- the next open sequence id for parent region, used by serial
replication. -1 if not necessary.splitA
- Split daughter region AsplitB
- Split daughter region Bsn
- the location of the regionIOException
public static Put makePutFromTableState(TableState state, long ts)
state
- new statepublic static void deleteTableState(Connection connection, TableName table) throws IOException
connection
- to use for deletiontable
- to delete state forIOException
public static void updateRegionLocation(Connection connection, RegionInfo regionInfo, ServerName sn, long openSeqNum, long masterSystemTime) throws IOException
Uses passed catalog tracker to get a connection to the server hosting hbase:meta and makes edits to that region.
connection
- connection we're usingregionInfo
- region to update location ofopenSeqNum
- the latest sequence number obtained when the region was opensn
- Server namemasterSystemTime
- wall clock time from master if passed in the open region RPCIOException
public static void deleteRegionInfo(Connection connection, RegionInfo regionInfo) throws IOException
connection
- connection we're usingregionInfo
- region to be deleted from METAIOException
public static void deleteRegionInfos(Connection connection, List<RegionInfo> regionsInfo) throws IOException
connection
- connection we're usingregionsInfo
- list of regions to be deleted from METAIOException
public static void overwriteRegions(Connection connection, List<RegionInfo> regionInfos, int regionReplication) throws IOException
connection
- connection we're usingregionInfos
- list of regions to be added to METAIOException
public static void deleteMergeQualifiers(Connection connection, RegionInfo mergeRegion) throws IOException
connection
- connection we're usingmergeRegion
- the merged regionIOException
public static Put addRegionInfo(Put p, RegionInfo hri) throws IOException
IOException
public static Put addLocation(Put p, ServerName sn, long openSeqNum, int replicaId) throws IOException
IOException
public static byte[] getParentsBytes(List<RegionInfo> parents)
public static Put makePutForReplicationBarrier(RegionInfo regionInfo, long openSeqNum, long ts) throws IOException
IOException
public static void addReplicationBarrier(Put put, long openSeqNum) throws IOException
IOException
public static long[] getReplicationBarriers(Result result)
public static MetaTableAccessor.ReplicationBarrierResult getReplicationBarrierResult(Connection conn, TableName tableName, byte[] row, byte[] encodedRegionName) throws IOException
IOException
public static long[] getReplicationBarrier(Connection conn, byte[] regionName) throws IOException
IOException
public static List<Pair<String,Long>> getTableEncodedRegionNameAndLastBarrier(Connection conn, TableName tableName) throws IOException
IOException
public static List<String> getTableEncodedRegionNamesForSerialReplication(Connection conn, TableName tableName) throws IOException
IOException
Copyright © 2007–2019 The Apache Software Foundation. All rights reserved.