list-zfs-snapshots

A small and simple script to visualize your snapshot space usage and help you identify a minimum amount of snapshots you need to delete to free up some space

To execute simply run the script (tested in Kubuntu 18.04)

./list_snapshots.sh

If you only want information of a certain data set (and it's nested datasets)

./list_snapshots.sh [dataset]

For example:

./list_snapshots.sh tank/documents

TL;DR for the following section

Huge numbers are highlighted and easy to find. The USED column shows how much space can be freed by deleting that snapshot. Read last section to learn how to identify short lived data that is referenced by multiple snapshots.

Example output :

(output has also highlighting for huge numbers that does not show here but greatly increases visibility of hot spots)

  CREATION              TYPE        NAME           WRITTEN  REFER   USED  RATIO  LREFER  LUSED
Mo Dez 18  0:23 2017  filesystem  tank/projects      1,76M   176G   623G  1.13x    190G   691G

Fr Mai 18 11:29 2018  snapshot    tank/projects@1     437G   437G  1,70G  1.14x    488G      -
...

This is the header line (TYPE=filesystem) that is output for each data set (alphabetically) followed by all it's snapshots in chronological order.

In the header line you can see:

• WRITTEN: 1,76 megabyte of data was written since the last snapshot
• REFER: 176G gigabyte of compressed/real data is currently referenced by this data set and it's nested data sets (if any)
• USED: 623G gigabyte of compressed/real data is used by this data set, it's nested data sets and all their snapshots.
• RATIO: 1.13x The current overall compression ratio for this data set. Will be 1.00x if compression is of.
• LREFER: 190G gigabyte of uncompressed data is currently referenced by this data set and it's nested data sets
• LUSED: 691G gigabyte of uncompressed data is currently referenced by this data set, it's nested data sets and all their snapshots.

What you can learn from the header line:

• You currently saved 691G - 623G = 68G data due to using compression on this data set
• You can free up 623G - 176G = 447G data by deleting all snapshots of this data set (also all snapshots of nested data sets)
• After deleting all snapshots you still will be saving 190G - 176G = 14G data due to using compression
• So basically if REFER and USED differ greatly this is a good indicator, that this data set could free some space by deleting the right snapshots. If they are similar, most data is still in use.

In the first snapshot line you can see that:

• WRITTEN: 437 gigabyte was written between the last snapshot and this one. (Or between data set creation and snapshot, if it is the first one)
• REFER: 437 gigabyte is referred by this snapshot. Meaning this is all data this particular snapshot is pointing at. Other snapshots might point to the same data though.
• USED: 1,70 gigabyte is the amount of data, ONLY this snapshot refers to. This is the easiest way to quickly free up data. Deleting this snapshot will free exactly this amount of space. USE zfs diff \<snapshotA> \<snapshotB> To find out what this actually references first. As deleting this will permanently delete this data.
• RATIO: 1.14x The compression ratio of data referenced in this snapshot
• LREFER: 488 gigabyte uncompressed data is referred by this snapshot.
• LUSED: - Not available for snapshots

What you can learn from the snapshot line:

• If you delete this snapshot. It will free 1.70 gigabyte of data.
• 437 gigabyte data was created between this and the last snapshot. See next section to find out how to see if some if this could be freed by deleting multiple snapshots.

Find short lived data referenced by multiple snapshots

TYPE        NAME              WRITTEN  REFER   USED 
filesystem  tank/projects           0  45,7G  91,0G

snapshot    tank/projects@1     1,00G  1,00G    16k 
snapshot    tank/projects@2     50,3G  51,3G   200M 
snapshot    tank/projects@3     2,00G  51,5G    16k 
snapshot    tank/projects@4     40,0G  45,7G  1,70G 
snapshot    tank/projects@5     13,7M  45,7G  13,7M

This list of snapshots reveals that roughly 45 gigabyte can be freed if snapshot 2 and 3 are being deleted. You will find this situation in large lists of frequent snapshots if data was created and deleted rather quickly, but was present long enough to end up in multiple snapshots.

How to tell that 45 gigabyte can be freed

• Snapshot 2 has written 50,3 gigabyte of data. This will be highlighted in red by the script as it is more than 10 gigabyte. So these events are easily spotted.
• Unfortunately none of the USED values are promising much release by deleting a single snapshot.
• However looking at the snapshots closely following snapshot 2 you can see that snapshot 4 writes 40 gigabyte of data BUT the REFER column does not grow! Instead it even shrinks. This means, while 40 gigabyte was written between 3 and 4, another ~45 gigabyte was also deleted. Else the REFER column would now be at 91.5G
• Since the USED values are small, these 45 gigabyte of deleted data needs to be referenced by more than one snapshot. In this case it is obviously snapshot 2 and 3 that both reference most of this data. Deleting both should free the 45 gigabyte of data.
• So to do this, make a diff of 3 and 4. Check that there is nothing important in it. And then delete 3.

TYPE        NAME              WRITTEN  REFER   USED 
filesystem  tank/projects           0  45,7G  91,0G

snapshot    tank/projects@1     1,00G  1,00G    16k 
snapshot    tank/projects@2     50,3G  51,3G  45,3G 
snapshot    tank/projects@4     42,0G  45,7G  1,70G 
snapshot    tank/projects@5     13,7M  45,7G  13,7M

• Afterwards the USED column of snapshot 2 will now show 45,3G. Make another diff of 2 and 4, and if you are fine with loosing this data as well: Also delete snapshot 2 freeing the full 45,3 gigabyte of data.

``` TYPE NAME WRITTEN REFER USED filesystem tank/projects 0 45,7G 45,7G

snapshot tank/projects@1 1,00G 1,00G 16k snapshot tank/projects@4 44,7G 45,7G 1,70G snapshot tank/projects@5 13,7M 45,7G 13,7M ```

Note: This was an especially hard case as a huge amount of data was written & deleted in the same snapshot. Often you will only see a drop in the REFER column, which is much easier to spot. So after a huge write, check for a drop in REFER or if WRITTEN and REFER don't add up, to find short lived data