Persistent Dataset

Hi all. So here is an example of how to create and use one of our persistent dataset types; an image backed dataset. This is a dataset that you create using a temporary file system, capture, and then reuse in other profiles. Each time you use it, you get a private copy on your node(s) that you can read and write, but which is thrown away when your experiment ends. Note that you can update the "master copy" of the dataset later if needed.  Hopefully this html with images will come across okay!

Step 1:  Start an instance of the public Temp-Datastore profile. This example profile will provide you with an extra 30GB file system mounted at /mydata.

Step 2:  Login to your node (or use the shell in your browser) and populate /mydata. When you are done, log off of the node so that the data can be captured cleanly.

Step 3: Go to Actions->Create Dataset. Choose a name for your dataset and which project to associate it with. Click on Image Backed dataset. From the drop down menu, choose the Temp-Datastore experiment that you started up  in Step 1. You will see the next three fields filled in automatically:

Step 4: Click create. There will be a delay of a few seconds, be patient! In a bit you will see the image capture progress modal. When the image capture process is completed you will see the following (and you will receive an email message):

Step 5: Dismiss the modal, you will see the details of your new dataset. The most important bit of data is the "urn" of your dataset. This is what you need to use your dataset in another profile.

Step 6: Now make a copy of the Image-Dataset public profile. On the edit page for your new profile, click on the Source button, and then replace the urn in the blockstore tag with urn of your new dataset:

Step 7: Click on Accept and then Save your profile changes. Then click on Instantiate. When your experiment has setup, you can log into your node, and cd to /mydata to see your data.

Step 8: If later, you want to update the contents of your dataset, then go back to the Show Dataset page shown in Step 5, and click on Snapshot.