chjatwork

updated

12 years ago

C

chjatwork

2 Intern

•

356 Posts

0

11088

January 6th, 2014 04:00

Deleting Individual Files from /ifs/.snapshots

Question?

I have a home directory created under /ifs/home/[domain name]/[username].  I have snapshots running on the entire /ifs/home/[domain name] directory.  Months ago I created a .ISO file under my account to test out the quota functionality on the individual home folders.  Months later forgetting that I have created this .ISO and thought I deleted it, but forgetting that snapshots had saved it and the snapshot is not set to expire.  I need to free this space in my home directory, so I am wondering if I can just go in and delete the individual file that is causing my home folder to be so large with compromising the entire snapshot saved for that day on the entire /ifs/home/[domain name]?  I figure you can, but I don't want to assume doing this will be OK.  Please let me know.

Thank you,

  • dynamox

    11 Legend

    •

    20419 Posts

    •

    87439 Points

    5380

    1

    Posted January 6th, 2014 08:00

    no

  • chughh

    122 Posts

    5380

    0

    Posted January 6th, 2014 05:00

    Depending upon OneFS version  you need to check  isi snapshot list for eg

    isi snapshot list

    ID   Name                                        Path

    ---------------------------------------------------------------------

    2    forrob                                      /ifs/data

    6    forrob1                                     /ifs/data

    148  SIQ-Failover-source1-2013-12-11_21-51-13    /ifs/data/target1dir

    164  te: 2013Dec13, 03:26:04 PM                  /ifs/data/folder

    165  testtime                                    /ifs/data/folder

    319  SIQ-e25ee6c12cff3fc52a09071643212b75-latest /ifs/data/source1dir

    321  SIQ-5ab4b40fd512e90b6d1c1e8dedb40e7f-latest /ifs/data/source2dir

    323  SIQ-Failover-source2-2013-12-23_15-49-25    /ifs/data/target2dir

    325  SIQ-Failover-source1-2013-12-23_16-07-57    /ifs/data/target1dir

    ---------------------------------------------------------------------

    you can  delete snapshot that would free up space..

    isi snapshot snapshots delete --snapshot=

  • chjatwork

    2 Intern

    •

    356 Posts

    5380

    0

    Posted January 6th, 2014 08:00

    @ chughh,

    Yeah i am aware you can delete an entire snapshot.  I am asking can I just delete a single file WITHIN the entire snapshot and not have any problems being able to recover the other files that I didn't tamper with?

    i.e. /ifs/.snapshot/[DomainName]-backup-2013-11-15-_00-00/home/[DomainName]/[UserName]/TestFile.iso

    Thank you,

  • christopher_ime

    6 Operator

    •

    1962 Posts

    5380

    0

    Posted January 6th, 2014 23:00

    chjatwork wrote:

    Months ago I created a .ISO file under my account to test out the quota functionality on the individual home folders. 

    You have apparently in the quota setting specified "Usage Accounting: Include Snapshot Data".  I am of course not judging that choice (sure track and enforce usage on exactly what physical disk space the users' data is representing/consuming), but I simply wanted to remind you that it had been enabled (and maybe even the Data-Protection Overhead).  Thus, you are now looking into solutions to reduce the amount of data that the snapshot(s) represent which can only be accomplished by deleting entire snapshots as others have already suggested.

    chjatwork wrote:

    the snapshot is not set to expire.  I need to free this space in my home directory,

    The worst case scenario from a capacity perspective for a (pointer-based) snapshot is if everything in the directory it is preserving a point-in-time of is modified; it is then basically a clone.  Having set your snapshot to *not* expire of course increases the likelihood of that; certainly it increases the amount of data the snapshot has to preserve (SnapshotIQ strategically chooses either RoW and/or CoW) when compared to, for example only, snapshot(s) that is refreshed after a week.

    So unless you are prepared for this worst case scenario and you still have requirements to retain your snapshots indefinitely, maybe it is time to increase the quota or maybe reconsider leaving the option enabled to include snapshot (and/or data-protection) overhead in the quota calculation.  Alternatively, maybe reconsider the non-expiry of your snapshots.  Of course, these are business decisions you have to make but is often a balance.

  • chjatwork

    2 Intern

    •

    356 Posts

    5380

    0

    Posted January 7th, 2014 03:00

    @ Peter_Sero,

    Hey ok this is a OneFS 7.1 feature only?  And chughh is right that there is no way to just delete a single file within a snapshot in OneFS 7.0.1.4?  The only way to insure that this file in question is deleted it to delete the ENTIRE snapshot for that day, essentially deleting everything saved that day within that snapshot?

    Let me make sure I understand you right about OneFS 7.1 new snapshot functionality... you can essentially make a full shadow copy backup of a single file within a snapshot job saved?  So in the event I have to delete the ENTIRE snapshot because of one file that is compromising a single persons disk space I could run this command to essentially make a shadow copy of everything saved within that snapshot minus the file in question?  So that I can reclaim the space for this single user?

    Hopefully I have an understand for what you are trying to relay.

    Thank you,

  • Peter_Sero

    6 Operator

    •

    1169 Posts

    5380

    0

    Posted January 7th, 2014 03:00

    Ho chjatwork,

    with OneFS 7.1 you can explicitly clone files from snapshots into what is called "shadow store".

    As this is managed by reference counting (Ref: 7.1 CLI Admin Guide, pp 373+374),

    deleting the underlying snapshot will leave the clone intact -- which becomes the primary

    instance of a file if the very original file had already been deleted.

    So with a bit of scripting one can transmogrify snapshots into clones (need to

    create physical directories and populate those with the file clones).

    Then you can delete individual files from the clones!

    and ShadowStoreDelete will clean up the space eventually.

    I admit,  it's probably not worth the hassle for saving only a few

    dozen GB or so...

    Cheers

    -- Peter

  • chughh

    122 Posts

    5380

    1

    Posted January 7th, 2014 03:00

    I am sorry to say but there is no option to delete a file from snapshot..

  • Peter_Sero

    6 Operator

    •

    1169 Posts

    5380

    0

    Posted January 7th, 2014 04:00

    > Hey ok this is a OneFS 7.1 feature only?

    It appears in the 7.0 release notes, but details only in the 7.1 CLI admin... Confusing, yeah -- better check with support for actual functionality in 7.0.

    > Let me make sure I understand you right about OneFS 7.1 new snapshot functionality... you can essentially make a full shadow copy backup of a single file within a snapshot job saved?

    /from/ a snapshot, but the the path of the clone will be /outside/ the snapshot path.

    > So in the event I have to delete the ENTIRE snapshot because of one file that is compromising a single persons disk space I could run this command to essentially make a shadow copy of everything saved within that snapshot minus the file in question?  So that I can reclaim the space for this single user?


    Kind of. Clones are per file, not for a complete tree, this is why I said some scripting will be needed...

    It is really a hack, stretching OneFS's new features to the limit. To be considered only if you are in a very painful situation...

    BTW, the 7.1 Release Notes state that the max number of snapshot has been increased from 2,048 to 20,000, so you might organize the snapshots more fine-grained. With say 1000 users and 7 days retention time, you can have even snapshots per user. Snapshots with longer retention times can be taken at larger intervals to reduce the total number -- see the snapshot best practices in the 7.1 CLI Admin guide p 272.

    -- P.

  • chjatwork

    2 Intern

    •

    356 Posts

    3329

    0

    Posted January 7th, 2014 04:00

    @ Christopher Imes,

    Your right I have this setting selected:  "Usage Accounting: Include Snapshot Data"  I need to be able to account for the overhead that users are using.  I am guessing you think this is a bad idea?

    Starting a couple months ago I did change the retention period for snapshots from non-expire to a 7-day expiration.  I notice how out of control having the snapshot set to non-expire can get.  We are currently looking into a storage unit that does a better job of handling data for a longer period of time.  The snapshots in questions are the snapshots that still linger from before the policy was changed.  It just brought up a much larger question for me of what would happen if I had no choice in the matter and I had to retain all of this data, but wanted to make this single adjustment, and that was to be able to delete a single file from a snapshot job.


    Maybe this is worth EMC's time to look into changing?



    Thank you,



  • chjatwork

    2 Intern

    •

    356 Posts

    2996

    0

    Posted January 7th, 2014 04:00

    @ Peter_Sero,

    Hey thanks of the FYI on the snapshot max numbers.  I am have snapshot policies running for directories not individuals, so hopefully we can keep the directories at bay.