I’ve been working on a project on my local computer and versioning my data via DVC to an S3 bucket.
Now my project is too big and I need to take advantage of the nearly unlimited seeming RAM available on AWS Sagemaker.
Also, I’ve linked my Gitlab to my notebook instances in Sagemaker so I can see the folder containing the dvc files which I’ve pushed to my Gitlab.
However, everything I see on Sagemaker tutorials says I need to point to the actual file in S3 (see here: https://stackoverflow.com/a/56060184/4691538).
To that end, can someone tell me how to load a file into Sagemaker which is under data versioning control using DVC and stored in S3? Cheers!