dvc add --external /mnt/my_storage I am getting
ERROR: file '../../../../mnt/my_storage/data.csv' not owned by user!
I have machine_1 with a project I am working under git control and dvc initialized.
Machine_2 is a remote file server on the same network mounted to /mnt/my_storage/ via samba. I’d like to keep the training data on /mnt/my_storage, but track it in DVC.
dvc hashes all files in /mnt/my_storage and creats a my_storage.dvc in my project folder.
Files in one subdirectory are hashed and in another I am getting above mentioned error.
The files on machine_2 are owned by a different user, but I have write access to /mnt/my_storage on file system level.
What is this error telling me?
I’m not too familiar with SMB so can’t tell you exactly how or if it’s possible to fix the Linux permissions but I can tell you what’ happening:
To DVC /mnt/ is just another local path so it tries to move the add targets to your local DVC cache, and then link them back to their original position (mechanism explained in https://dvc.org/doc/command-reference/add). I guess we need to own them in order to do all that.
A possible workaround is to change your project link types (
cache.type) and see if other kinds of linking works. See https://dvc.org/doc/command-reference/config#cache
Another thing you may want to do is to setup your cache external to the project, somewhere in the SMB mount. Otherwise you aren’t really keeping those files outside of your local fs. Please see https://dvc.org/doc/use-cases/shared-development-server#configure-the-external-shared-cache.
What directory is that one? The one that works. Is it also in the SMB mount?
Which file system are you referring to here? You mean you make files and directories in /mnt/my_storage/xyz? Those would be owned by you but pre-existing files are still not under your full control.
Thanks, I’ll follow-up on your suggestions and if I find a solution I’ll post it here.