Temp File System

Edit this Page via GitHub       Comment by Filing an Issue      Have Questions? Ask them here.

The temp file system is maintained by SciComp for temporary storage of research data during active analysis. This is a large, high-performance storage system designed to be used as temporary storage for cluster jobs. It is not designed to be as available or as robust as the home or fast file systems.

Data on this platform is not backed up. This storage platform is not appropriate for storing the primary or only copy of any data.

The limitations of temp:

  • the storage is only accessible on cluster nodes at /hpc/temp
  • data on temp is not backed up in any way
  • data on temp is subject to an automated lifecycle policy and will be deleted 30 days after they are created

There are no charges to the investigator for data stored here.

Due to the temporary nature of this storage, the lack of charges, limits of capacity, and automatic deletion policy, it is very important that lab leadership understand these aspect before storing data here. To have a folder created for your lab and your use, please send an email to scicomp@fredhutch.org and include lab owner, PI, and/or data steward.

FAQ

Q: Can I use temp rather than Fast for lab data to avoid storage charges?

A: No. The temp storage platform is not backed up and has automatic deletions. All data stored on temp will be deleted in 30 days.

Q: What will happen to my existing data in scratch?

A: scratch will be retired between July and October of 2024. Watch for further announcements with a more exact time frame. You can start moving your data off of scratch right away. Data that is not moved or copied elsewhere will be deleted.

Q: If I have data in Economy storage, but need to analyze it, can I copy it to temp?

A: Yes! This is the main use case for temp storage.

Q: Can I touch my files to avoid automatic deletion?

A: No. The lifecycle policy is managed by the storage platform and is based on file creation time, not read or modify time stamps.

Q: What is a good way to automate data management for my jobs so I don’t worry about data in temp?

A: Use a workflow manager. Please see DaSL’s PROOF

Edit this Page via GitHub       Comment by Filing an Issue      Have Questions? Ask them here.