How do I recover files that I’ve deleted?
We make nightly, weekly and monthly snapshots of the entire home directory space of the NFS fileserver. Nightly snapshots are retained for one week, weekly snapshots for one month and monthly snapshots for up to one year. On the Linux workstations and compute nodes every home directory has a hidden snapshots directory at the following path: /home/<username>/.zfs/snapshot. You cannot see the .zfs directory, but you can list its contents and act on it by using the full path, for example:
user@homes-01:~$ ls -ltr ~/.zfs/snapshot
total 528
drwx------ 32 root root 44 Nov 20 2018 zfs-auto-snap_monthly-2018-11-24-11h11
drwx------ 32 root root 44 Dec 20 2018 zfs-auto-snap_monthly-2018-12-24-11h11
drwx------ 33 root root 46 Jan 15 2019 zfs-auto-snap_monthly-2019-01-24-11h11
drwx------ 34 root root 51 Feb 20 2019 zfs-auto-snap_monthly-2019-02-24-11h11
drwx------ 34 root root 52 Mar 21 2019 zfs-auto-snap_monthly-2019-03-24-12h11
drwx------ 34 root root 52 Apr 10 2019 zfs-auto-snap_monthly-2019-04-24-12h11
drwx------ 35 root root 53 May 20 13:38 zfs-auto-snap_monthly-2019-05-24-12h11
drwx------ 36 root root 57 Jun 20 11:01 zfs-auto-snap_monthly-2019-06-24-12h11
drwx------ 40 root root 68 Jul 17 13:13 zfs-auto-snap_monthly-2019-07-24-12h11
drwx------ 40 root root 71 Aug 23 11:14 zfs-auto-snap_monthly-2019-08-24-12h11
drwx------ 42 root root 75 Sep 19 11:06 zfs-auto-snap_weekly-2019-10-01-12h11
drwx------ 42 root root 75 Sep 19 11:06 zfs-auto-snap_monthly-2019-09-24-12h11
drwx------ 42 root root 76 Oct 3 16:04 zfs-auto-snap_weekly-2019-10-08-12h11
drwx------ 43 root root 78 Oct 11 17:05 zfs-auto-snap_weekly-2019-10-15-12h11
drwx------ 43 root root 78 Oct 11 17:05 zfs-auto-snap_daily-2019-10-14-12h11
drwx------ 43 root root 78 Oct 17 09:30 zfs-auto-snap_weekly-2019-10-22-12h11
drwx------ 43 root root 78 Oct 17 09:30 zfs-auto-snap_monthly-2019-10-24-12h11
drwx------ 43 root root 78 Oct 17 09:30 zfs-auto-snap_daily-2019-10-28-12h11
drwx------ 43 root root 78 Oct 17 09:30 zfs-auto-snap_daily-2019-10-27-12h11
drwx------ 43 root root 78 Oct 17 09:30 zfs-auto-snap_daily-2019-10-23-12h11
drwx------ 43 root root 78 Oct 17 09:30 zfs-auto-snap_daily-2019-10-21-12h11
drwx------ 43 root root 78 Oct 17 09:30 zfs-auto-snap_daily-2019-10-17-12h11
user@homes-01:~$
total 528
drwx------ 32 root root 44 Nov 20 2018 zfs-auto-snap_monthly-2018-11-24-11h11
drwx------ 32 root root 44 Dec 20 2018 zfs-auto-snap_monthly-2018-12-24-11h11
drwx------ 33 root root 46 Jan 15 2019 zfs-auto-snap_monthly-2019-01-24-11h11
drwx------ 34 root root 51 Feb 20 2019 zfs-auto-snap_monthly-2019-02-24-11h11
drwx------ 34 root root 52 Mar 21 2019 zfs-auto-snap_monthly-2019-03-24-12h11
drwx------ 34 root root 52 Apr 10 2019 zfs-auto-snap_monthly-2019-04-24-12h11
drwx------ 35 root root 53 May 20 13:38 zfs-auto-snap_monthly-2019-05-24-12h11
drwx------ 36 root root 57 Jun 20 11:01 zfs-auto-snap_monthly-2019-06-24-12h11
drwx------ 40 root root 68 Jul 17 13:13 zfs-auto-snap_monthly-2019-07-24-12h11
drwx------ 40 root root 71 Aug 23 11:14 zfs-auto-snap_monthly-2019-08-24-12h11
drwx------ 42 root root 75 Sep 19 11:06 zfs-auto-snap_weekly-2019-10-01-12h11
drwx------ 42 root root 75 Sep 19 11:06 zfs-auto-snap_monthly-2019-09-24-12h11
drwx------ 42 root root 76 Oct 3 16:04 zfs-auto-snap_weekly-2019-10-08-12h11
drwx------ 43 root root 78 Oct 11 17:05 zfs-auto-snap_weekly-2019-10-15-12h11
drwx------ 43 root root 78 Oct 11 17:05 zfs-auto-snap_daily-2019-10-14-12h11
drwx------ 43 root root 78 Oct 17 09:30 zfs-auto-snap_weekly-2019-10-22-12h11
drwx------ 43 root root 78 Oct 17 09:30 zfs-auto-snap_monthly-2019-10-24-12h11
drwx------ 43 root root 78 Oct 17 09:30 zfs-auto-snap_daily-2019-10-28-12h11
drwx------ 43 root root 78 Oct 17 09:30 zfs-auto-snap_daily-2019-10-27-12h11
drwx------ 43 root root 78 Oct 17 09:30 zfs-auto-snap_daily-2019-10-23-12h11
drwx------ 43 root root 78 Oct 17 09:30 zfs-auto-snap_daily-2019-10-21-12h11
drwx------ 43 root root 78 Oct 17 09:30 zfs-auto-snap_daily-2019-10-17-12h11
user@homes-01:~$
In the ~/.zfs/snapshot/ directory you will see a number of subdirectories named in the following format: zfs-auto-snap_<daily|weekly|monthly>-YYYY-MM-DD-HHhMM
Within those snapshots you will find a representation of your home directory as it was on that day.
So, to recover a file that was in your home directory on the Fourth of September 19th at 11 am named “foo” that was deleted sometime before the 3rd of October one would simply issue the following command: