org.apache.hadoop.hbase.replication
Class ReplicationZookeeper

java.lang.Object
  extended by org.apache.hadoop.hbase.replication.ReplicationZookeeper
All Implemented Interfaces:
Closeable

@InterfaceAudience.Private
public class ReplicationZookeeper
extends Object
implements Closeable

This class serves as a helper for all things related to zookeeper in replication.

The layout looks something like this under zookeeper.znode.parent for the master cluster:

 replication/
  state      {contains true or false}
  clusterId  {contains a byte}
  peers/
    1/   {contains a full cluster address}
      peer-state  {contains ENABLED or DISABLED}
    2/
    ...
  rs/ {lists all RS that replicate}
    startcode1/ {lists all peer clusters}
      1/ {lists hlogs to process}
        10.10.1.76%3A53488.123456789 {contains nothing or a position}
        10.10.1.76%3A53488.123456790
        ...
      2/
      ...
    startcode2/
    ...
 


Field Summary
static byte[] ENABLED_ZNODE_BYTES
          ZNode content if enabled state.
 
Constructor Summary
ReplicationZookeeper(Abortable abortable, org.apache.hadoop.conf.Configuration conf, ZooKeeperWatcher zk)
          Constructor used by clients of replication (like master and HBase clients)
ReplicationZookeeper(Server server, AtomicBoolean replicating)
          Constructor used by region servers, connects to the peer cluster right away.
 
Method Summary
 void addLogToList(String filename, String peerId)
          Add a new log to the list of hlogs in zookeeper
 void addPeer(String id, String clusterKey)
          Add a new peer to this cluster
 void close()
           
 boolean connectToPeer(String peerId)
          This method connects this cluster to another one and registers it in this region server's replication znode
 SortedMap<String,SortedSet<String>> copyQueuesFromRS(String znode)
          This methods copies all the hlogs queues from another region server and returns them all sorted per peer cluster (appended with the dead server's znode)
 SortedMap<String,SortedSet<String>> copyQueuesFromRSUsingMulti(String znode)
          It "atomically" copies all the hlogs queues from another region server and returns them all sorted per peer cluster (appended with the dead server's znode).
 void deleteOwnRSZNode()
          Delete this cluster's queues
 void deleteRsQueues(String znode)
          Recursive deletion of all znodes in specified rs' znode
 void deleteSource(String peerZnode, boolean closeConnection)
          Delete a complete queue of hlogs
 void disablePeer(String id)
          Disable replication to the peer
 void enablePeer(String id)
          Enable replication to the peer
 long getHLogRepPosition(String peerId, String hlog)
          Get the position of the specified hlog in the specified peer znode
 List<String> getListHLogsForPeerForRS(String rs, String id)
          Get the list of hlogs for the specified region server and peer cluster
 List<String> getListOfReplicators()
          Get the list of the replicators that have queues, they can be alive, dead or simply from a previous run
 List<String> getListPeersForRS(String rs)
          Get the list of peer clusters for the specified server names
 ReplicationPeer getPeer(String peerId)
          Helper method to connect to a peer
 Map<String,ReplicationPeer> getPeerClusters()
          Get a map of all peer clusters
 boolean getPeerEnabled(String id)
          Check whether the peer is enabled or not.
 String getPeersZNode()
          Get the full path to the peers' znode
 UUID getPeerUUID(String peerId)
          Returns the UUID of the provided peer id.
 List<String> getRegisteredRegionServers()
          Get a list of all the other region servers in this cluster and set a watch
 boolean getReplication()
          Get the replication status of this cluster.
 List<ServerName> getSlavesAddresses(String peerClusterId)
          Returns all region servers from given peer
 UUID getUUIDForCluster(ZooKeeperWatcher zkw)
          Get the UUID for the provided ZK watcher.
static String getZNodeName(String fullPath)
          Extracts the znode name of a peer cluster from a ZK path
 ZooKeeperWatcher getZookeeperWatcher()
          Get this cluster's zk connection
 boolean isPeerPath(String path)
          Determine if a ZK path points to a peer node.
static List<ServerName> listChildrenAndGetAsServerNames(ZooKeeperWatcher zkw, String znode)
          Lists the children of the specified znode, retrieving the data of each child as a server address.
 Map<String,String> listPeers()
          Map of this cluster's peers for display.
 List<String> listPeersIdsAndWatch()
          List this cluster's peers' IDs
 boolean lockOtherRS(String znode)
          Try to set a lock in another server's znode.
 void registerRegionServerListener(ZooKeeperListener listener)
           
 void removeLogFromList(String filename, String clusterId)
          Remove a log from the list of hlogs in zookeeper
 void removePeer(String id)
          Remove the peer from zookeeper.
 void setReplication(boolean newState)
          Set the new replication state for this cluster
 void writeReplicationStatus(String filename, String clusterId, long position)
          Set the current position of the specified cluster in the current hlog
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

ENABLED_ZNODE_BYTES

public static final byte[] ENABLED_ZNODE_BYTES
ZNode content if enabled state.

Constructor Detail

ReplicationZookeeper

public ReplicationZookeeper(Abortable abortable,
                            org.apache.hadoop.conf.Configuration conf,
                            ZooKeeperWatcher zk)
                     throws org.apache.zookeeper.KeeperException
Constructor used by clients of replication (like master and HBase clients)

Parameters:
conf - conf to use
zk - zk connection to use
Throws:
IOException
org.apache.zookeeper.KeeperException

ReplicationZookeeper

public ReplicationZookeeper(Server server,
                            AtomicBoolean replicating)
                     throws IOException,
                            org.apache.zookeeper.KeeperException
Constructor used by region servers, connects to the peer cluster right away.

Parameters:
server -
replicating - atomic boolean to start/stop replication
Throws:
IOException
org.apache.zookeeper.KeeperException
Method Detail

listPeersIdsAndWatch

public List<String> listPeersIdsAndWatch()
List this cluster's peers' IDs

Returns:
list of all peers' identifiers

listPeers

public Map<String,String> listPeers()
Map of this cluster's peers for display.

Returns:
A map of peer ids to peer cluster keys

getSlavesAddresses

public List<ServerName> getSlavesAddresses(String peerClusterId)
Returns all region servers from given peer

Parameters:
peerClusterId - (byte) the cluster to interrogate
Returns:
addresses of all region servers

listChildrenAndGetAsServerNames

public static List<ServerName> listChildrenAndGetAsServerNames(ZooKeeperWatcher zkw,
                                                               String znode)
                                                        throws org.apache.zookeeper.KeeperException
Lists the children of the specified znode, retrieving the data of each child as a server address. Used to list the currently online regionservers and their addresses. Sets no watches at all, this method is best effort. Returns an empty list if the node has no children. Returns null if the parent node itself does not exist.

Parameters:
zkw - zookeeper reference
znode - node to get children of as addresses
Returns:
list of data of children of specified znode, empty if no children, null if parent does not exist
Throws:
org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

connectToPeer

public boolean connectToPeer(String peerId)
                      throws IOException,
                             org.apache.zookeeper.KeeperException
This method connects this cluster to another one and registers it in this region server's replication znode

Parameters:
peerId - id of the peer cluster
Throws:
org.apache.zookeeper.KeeperException
IOException

getPeer

public ReplicationPeer getPeer(String peerId)
                        throws IOException,
                               org.apache.zookeeper.KeeperException
Helper method to connect to a peer

Parameters:
peerId - peer's identifier
Returns:
object representing the peer
Throws:
IOException
org.apache.zookeeper.KeeperException

removePeer

public void removePeer(String id)
                throws IOException
Remove the peer from zookeeper. which will trigger the watchers on every region server and close their sources

Parameters:
id -
Throws:
IllegalArgumentException - Thrown when the peer doesn't exist
IOException

addPeer

public void addPeer(String id,
                    String clusterKey)
             throws IOException
Add a new peer to this cluster

Parameters:
id - peer's identifier
clusterKey - ZK ensemble's addresses, client port and root znode
Throws:
IllegalArgumentException - Thrown when the peer doesn't exist
IllegalStateException - Thrown when a peer already exists, since multi-slave isn't supported yet.
IOException

enablePeer

public void enablePeer(String id)
                throws IOException
Enable replication to the peer

Parameters:
id - peer's identifier
Throws:
IllegalArgumentException - Thrown when the peer doesn't exist
IOException

disablePeer

public void disablePeer(String id)
                 throws IOException
Disable replication to the peer

Parameters:
id - peer's identifier
Throws:
IllegalArgumentException - Thrown when the peer doesn't exist
IOException

getPeerEnabled

public boolean getPeerEnabled(String id)
Check whether the peer is enabled or not. This method checks the atomic boolean of ReplicationPeer locally.

Parameters:
id - peer identifier
Returns:
true if the peer is enabled, otherwise false
Throws:
IllegalArgumentException - Thrown when the peer doesn't exist

getReplication

public boolean getReplication()
                       throws org.apache.zookeeper.KeeperException
Get the replication status of this cluster. If the state znode doesn't exist it will also create it and set it true.

Returns:
returns true when it's enabled, else false
Throws:
org.apache.zookeeper.KeeperException

setReplication

public void setReplication(boolean newState)
                    throws org.apache.zookeeper.KeeperException
Set the new replication state for this cluster

Parameters:
newState -
Throws:
org.apache.zookeeper.KeeperException

addLogToList

public void addLogToList(String filename,
                         String peerId)
                  throws org.apache.zookeeper.KeeperException
Add a new log to the list of hlogs in zookeeper

Parameters:
filename - name of the hlog's znode
peerId - name of the cluster's znode
Throws:
org.apache.zookeeper.KeeperException

removeLogFromList

public void removeLogFromList(String filename,
                              String clusterId)
Remove a log from the list of hlogs in zookeeper

Parameters:
filename - name of the hlog's znode
clusterId - name of the cluster's znode

writeReplicationStatus

public void writeReplicationStatus(String filename,
                                   String clusterId,
                                   long position)
Set the current position of the specified cluster in the current hlog

Parameters:
filename - filename name of the hlog's znode
clusterId - clusterId name of the cluster's znode
position - the position in the file
Throws:
IOException

getRegisteredRegionServers

public List<String> getRegisteredRegionServers()
Get a list of all the other region servers in this cluster and set a watch

Returns:
a list of server nanes

getListOfReplicators

public List<String> getListOfReplicators()
Get the list of the replicators that have queues, they can be alive, dead or simply from a previous run

Returns:
a list of server names

getListPeersForRS

public List<String> getListPeersForRS(String rs)
Get the list of peer clusters for the specified server names

Parameters:
rs - server names of the rs
Returns:
a list of peer cluster

getListHLogsForPeerForRS

public List<String> getListHLogsForPeerForRS(String rs,
                                             String id)
Get the list of hlogs for the specified region server and peer cluster

Parameters:
rs - server names of the rs
id - peer cluster
Returns:
a list of hlogs

lockOtherRS

public boolean lockOtherRS(String znode)
Try to set a lock in another server's znode.

Parameters:
znode - the server names of the other server
Returns:
true if the lock was acquired, false in every other cases

copyQueuesFromRSUsingMulti

public SortedMap<String,SortedSet<String>> copyQueuesFromRSUsingMulti(String znode)
It "atomically" copies all the hlogs queues from another region server and returns them all sorted per peer cluster (appended with the dead server's znode).

Parameters:
znode -
Returns:
HLog queues sorted per peer cluster

copyQueuesFromRS

public SortedMap<String,SortedSet<String>> copyQueuesFromRS(String znode)
This methods copies all the hlogs queues from another region server and returns them all sorted per peer cluster (appended with the dead server's znode)

Parameters:
znode - server names to copy
Returns:
all hlogs for all peers of that cluster, null if an error occurred

deleteSource

public void deleteSource(String peerZnode,
                         boolean closeConnection)
Delete a complete queue of hlogs

Parameters:
peerZnode - znode of the peer cluster queue of hlogs to delete

deleteRsQueues

public void deleteRsQueues(String znode)
Recursive deletion of all znodes in specified rs' znode

Parameters:
znode -

deleteOwnRSZNode

public void deleteOwnRSZNode()
Delete this cluster's queues


getHLogRepPosition

public long getHLogRepPosition(String peerId,
                               String hlog)
                        throws org.apache.zookeeper.KeeperException
Get the position of the specified hlog in the specified peer znode

Parameters:
peerId - znode of the peer cluster
hlog - name of the hlog
Returns:
the position in that hlog
Throws:
org.apache.zookeeper.KeeperException

getPeerUUID

public UUID getPeerUUID(String peerId)
Returns the UUID of the provided peer id. Should a connection loss or session expiration happen, the ZK handler will be reopened once and if it still doesn't work then it will bail and return null.

Parameters:
peerId - the peer's ID that will be converted into a UUID
Returns:
a UUID or null if there's a ZK connection issue

getUUIDForCluster

public UUID getUUIDForCluster(ZooKeeperWatcher zkw)
                       throws org.apache.zookeeper.KeeperException
Get the UUID for the provided ZK watcher. Doesn't handle any ZK exceptions

Parameters:
zkw - watcher connected to an ensemble
Returns:
the UUID read from zookeeper
Throws:
org.apache.zookeeper.KeeperException

registerRegionServerListener

public void registerRegionServerListener(ZooKeeperListener listener)

getPeerClusters

public Map<String,ReplicationPeer> getPeerClusters()
Get a map of all peer clusters

Returns:
map of peer cluster keyed by id

isPeerPath

public boolean isPeerPath(String path)
Determine if a ZK path points to a peer node.

Parameters:
path - path to be checked
Returns:
true if the path points to a peer node, otherwise false

getZNodeName

public static String getZNodeName(String fullPath)
Extracts the znode name of a peer cluster from a ZK path

Parameters:
fullPath - Path to extract the id from
Returns:
the id or an empty string if path is invalid

getZookeeperWatcher

public ZooKeeperWatcher getZookeeperWatcher()
Get this cluster's zk connection

Returns:
zk connection

getPeersZNode

public String getPeersZNode()
Get the full path to the peers' znode

Returns:
path to peers in zk

close

public void close()
           throws IOException
Specified by:
close in interface Closeable
Throws:
IOException


Copyright © 2013 The Apache Software Foundation. All Rights Reserved.