Thanks for the quick reply. I like the suggestion per project and per team creds.
I also like the ability to reproduce from source data if intermediates are lost. One of my main concerns of course is not losing the source data and also ensuring that we don’t lose the final generated artifacts that would be deployed to production. One thing we’re evaluating is whether we can configure DVC & S3 so that we can store those in S3 or if we should rely on external archiving where a majority of users would be granted only read access.
With S3, I believe we could turn on versioning at the bucket level and not grant
s3:DeleteObject permissions. This would ensure users could create and update objects but never delete and we’d have a version history for all updates if we needed to roll back.
The DVC docs for S3 list
s3:DeleteObject as a required permission. Is this necessary for “every day” workflows?
Specifically, I’m trying to understand how this relates to my mental model of git history. Once a specific commit hash is in the repo it is there forever unless you’re rewriting the history which is not an “every day” workflow.