Skip to content

HDFS-16540 Data locality is lost when DataNode pod restarts in kubern… #4170

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 28, 2022
Merged

HDFS-16540 Data locality is lost when DataNode pod restarts in kubern… #4170

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added .BUILDING.txt.swp
View file
Open in desktop
Binary file not shown.
17 changes: 16 additions & 1 deletion ...oop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1171,6 +1171,7 @@ public void registerDatanode(DatanodeRegistration nodeReg)
nodeN = null;
}

boolean updateHost2DatanodeMap = false;
if (nodeS != null) {
if (nodeN == nodeS) {
// The same datanode has been just restarted to serve the same data
Expand All @@ -1189,7 +1190,11 @@ public void registerDatanode(DatanodeRegistration nodeReg)
nodes with its data cleared (or user can just remove the StorageID
value in "VERSION" file under the data directory of the datanode,
but this is might not work if VERSION file format has changed
*/
*/
// Check if nodeS's host information is same as nodeReg's, if not,
// it needs to update host2DatanodeMap accordringly.
updateHost2DatanodeMap = !nodeS.getXferAddr().equals(nodeReg.getXferAddr());

NameNode.stateChangeLog.info("BLOCK* registerDatanode: " + nodeS
+ " is replaced by " + nodeReg + " with the same storageID "
+ nodeReg.getDatanodeUuid());
Expand All @@ -1199,6 +1204,11 @@ nodes with its data cleared (or user can just remove the StorageID
try {
// update cluster map
getNetworkTopology().remove(nodeS);

// Update Host2DatanodeMap
if (updateHost2DatanodeMap) {
getHost2DatanodeMap().remove(nodeS);
}
if(shouldCountVersion(nodeS)) {
decrementVersionCount(nodeS.getSoftwareVersion());
}
Expand All @@ -1217,6 +1227,11 @@ nodes with its data cleared (or user can just remove the StorageID
nodeS.setDependentHostNames(
getNetworkDependenciesWithDefault(nodeS));
}

if (updateHost2DatanodeMap) {
getHost2DatanodeMap().add(nodeS);
}

getNetworkTopology().add(nodeS);
resolveUpgradeDomain(nodeS);

Expand Down
29 changes: 29 additions & 0 deletions ...hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestDatanodeManager.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,35 @@ public void testNumVersionsCorrectAfterReregister()
mapToCheck.get("version1").intValue(), 1);
}

/**
* This test checks that if a node is re-registered with a different ip, its
* host2DatanodeMap is correctly updated with the new ip.
*/
@Test
public void testHost2NodeMapCorrectAfterReregister()
throws IOException, InterruptedException {
//Create the DatanodeManager which will be tested
FSNamesystem fsn = Mockito.mock(FSNamesystem.class);
Mockito.when(fsn.hasWriteLock()).thenReturn(true);
Configuration conf = new Configuration();
DatanodeManager dm = mockDatanodeManager(fsn, conf);

String storageID = "someStorageID1";
String ipOld = "someIPOld" + storageID;
String ipNew = "someIPNew" + storageID;

dm.registerDatanode(new DatanodeRegistration(
new DatanodeID(ipOld, "", storageID, 9000, 0, 0, 0),
null, null, "version"));

dm.registerDatanode(new DatanodeRegistration(
new DatanodeID(ipNew, "", storageID, 9000, 0, 0, 0),
null, null, "version"));

assertNull("should be no node with old ip", dm.getDatanodeByHost(ipOld));
assertNotNull("should be a node with new ip", dm.getDatanodeByHost(ipNew));
}

/**
* This test sends a random sequence of node registrations and node removals
* to the DatanodeManager (of nodes with different IDs and versions), and
Expand Down