by In this tutorial, we focus on HDFS snapshots. Common use cases of HDFS snapshots include backups and protection against user errors.
Create a snapshot of HDFS directory:
HDFS directories must be enabled for snapshots in order for snapshots to be created. Steps are:
- From the Clusters tab -> select HDFS service.
- Go to the File Browser tab. Select the file directory.
- Verify the Snapshottable Path and click Enable Snapshots.
With Command line:
|
1 2 3 4 |
su - hdfs Hdfs dfsadmin -allowsnapshot /user/mandy/snapshot_test # Another Method hadoop dfsadmin -allowSnapshot /user/mandy/snapshot_test |
Once directory has been enabled for snapshots, take a snapshot.
- To take a snapshot, click Take Snapshot, specify the name of the snapshot, and click Take Snapshot.
The snapshot is added to the snapshot list.
With Command line:
|
1 2 3 4 5 6 |
hdfs dfs -createsnapshot /user/mandy/snapshot_test # List the snapshots [mandy@cm ~]$ hdfs lsSnapshottableDir drwxr-xr-x 0 mandy hdfs 0 2018-05-24 02:40 1 65536 /user/mandy/snapshot_test [mandy@cm ~]$ hdfs dfs -ls /user/mandy/snapshot_test/.snapshot Found 1 itemsdrwxr-xr-x - mandy hdfs 0 2018-05-24 02:40 /user/mandy/snapshot_test/.snapshot/s20180524-024025.045 |
Remove the file from snapshot:
Now, let’s “accidentally” remove a file inside the snapshotable directory:
|
1 2 |
[mandy@cm ~]$ hdfs dfs -rm -r /user/mandy/snapshot_test/snap1.txt 18/05/24 03:05:51 INFO fs.TrashPolicyDefault: Moved: 'hdfs://cluster-ha/user/mandy/snapshot_test/snap1.txt' to trash at: hdfs://cluster-ha/user/mandy/.Trash/Current/user/mandy/snapshot_test/snap1.txt |
Recover the file from snapshot:
To restore a snapshot, click drop down button near folder name again and select restore from snapshot.
Select the snap from and select the restore method.
Restore in progress.
With Command line:
Recovering from the snapshot is as simple as copying the file.
|
1 |
[mandy@cm ~]$ hdfs dfs -cp /user/mandy/snapshot_test/.snapshot/s20180524-030219.005 /user/mandy/snapshot_test |
You can read the content of the file or list the file.
Disable Snapshot:
Try to remove a snapshotable directory by typing a following command as the hdfs user. As expected, the directory can’t be deleted because is snapshottable and it already contains a snapshot. Remove the snapshot first and re-try again.
Delete the snapshot.
From command line.
|
1 2 3 4 5 6 |
[hdfs@cm root]$ hdfs dfsadmin -disallowSnapshot /user/mandy/snapshot_test disallowSnapshot: The directory /user/mandy/snapshot_test has snapshot(s). Please redo the operation after removing all the snapshots. # Delete Snaphsot first [mandy@cm ~]$ hdfs dfs -deleteSnapshot /user/mandy/snapshot_test/ s20180524-024025.045 # Now You can delete the [hdfs@cm root]$ hdfs dfsadmin -disallowSnapshot /user/mandy/snapshot_test |
Thanks
Mandy
Leave a Reply