Read only mode add dataset

Questions
1

My dataset folder only read mode. I dont want to active write mode. So, how can ı add and track my dataset?

2

Hi @merve,

Are you getting some specific error when dvc adding it? Generally, to add data into dvc, you need to own the file/directory, so if you don’t - you need to copy it and then dvc add it.

3

ı used simplicity dvc add mydatasetpath
error : ERROR: unexpected error - [Errno 30] Read-only file system: mydatasetpath/abc.jpg

4

Could you show full log for dvc add mydatasetopath -v, please? Also, please show dvc doctor output.

5

dvc doctor output;
DVC version: 1.11.16 (pip)

Platform: Python 3.6.9 on Linux-4.15.0-135-generic-x86_64-with-Ubuntu-18.04-bionic
Supports: http, https
Cache types: hardlink, symlink
Cache directory: overlay on overlay
Caches: local
Remotes: None
Workspace directory: overlay on overlay
Repo: dvc (no_scm)

and dvc add mydatasetopath -v output is
2021-02-17 15:28:59,058 DEBUG: Analytics is enabled.
2021-02-17 15:28:59,120 DEBUG: Trying to spawn ‘[‘daemon’, ‘-q’, ‘analytics’, ‘/tmp/tmpt3or8f1b’]’
2021-02-17 15:28:59,122 DEBUG: Spawned ‘[‘daemon’, ‘-q’, ‘analytics’, ‘/tmp/tmpt3or8f1b’]’

6

Thanks!

Unfortunately that dvc add mydatasetopath -v log doesn’t look right, try dvc add mydatasetopath -v &> log.log and post the contents of log.log

7 
2021-02-17 15:50:06,877 DEBUG: Check for update is enabled.
2021-02-17 15:50:06,878 DEBUG: fetched: [(3,)]
2021-02-17 15:50:22,785 DEBUG: Path '/jup/datasets/TRAIN/PLATE/LOCATION/VERSION2/FRONT/jenoptic-1' inode '768770473'
2021-02-17 15:50:22,785 DEBUG: fetched: [('41882c1af0439fd46eeb5f0f90a11338', '2718846001', 'e27dcd51611a843f0744ba096799f52e.dir', '1613576997068446720')]
2021-02-17 15:50:22,817 DEBUG: Assuming '/jup/.dvc/cache/e2/7dcd51611a843f0744ba096799f52e.dir' is unchanged since it is read-only
2021-02-17 15:50:22,817 DEBUG: {'datasets/TRAIN/PLATE/LOCATION/VERSION2/FRONT/jenoptic-1': 'modified'}
2021-02-17 15:50:22,867 DEBUG: Path '/jup/datasets/TRAIN/PLATE/LOCATION/VERSION2/FRONT/jenoptic-1' inode '768770473'
2021-02-17 15:50:22,867 DEBUG: fetched: [('41882c1af0439fd46eeb5f0f90a11338', '2718846001', 'e27dcd51611a843f0744ba096799f52e.dir', '1613577022786244352')]
2021-02-17 15:50:22,868 DEBUG: Assuming '/jup/.dvc/cache/e2/7dcd51611a843f0744ba096799f52e.dir' is unchanged since it is read-only
2021-02-17 15:50:22,868 DEBUG: Computed stage: 'datasets/TRAIN/PLATE/LOCATION/VERSION2/FRONT/jenoptic-1.dvc' md5: 'None'
2021-02-17 15:50:22,871 DEBUG: Saving 'datasets/TRAIN/PLATE/LOCATION/VERSION2/FRONT/jenoptic-1' to '.dvc/cache/e2/7dcd51611a843f0744ba096799f52e.dir'.
2021-02-17 15:50:22,871 DEBUG: Assuming '/jup/.dvc/cache/e2/7dcd51611a843f0744ba096799f52e.dir' is unchanged since it is read-only
2021-02-17 15:50:22,871 DEBUG: cache '/jup/.dvc/cache/2f/6cc7cf7fab02868e12f9d4707d719e' expected 'HashInfo(name='md5', value='2f6cc7cf7fab02868e12f9d4707d719e', dir_info=None, size=None, nfiles=None)' actual 'None'
2021-02-17 15:50:22,882 DEBUG: fetched: [(2,)]
2021-02-17 15:50:22,894 ERROR: unexpected error - [Errno 30] Read-only file system: '/jup/datasets/TRAIN/PLATE/LOCATION/VERSION2/FRONT/jenoptic-1/images/PLATE_LOC_F_D_jenoptic-1_V2-01BLJ36_3511612000020101_2020-12-10_12-50-18-150_0_0_0_73_0-0-0-0_0_0_0_D60DD87C000015372101_1_0.jpg'
------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.6/shutil.py", line 550, in move
    os.rename(src, real_dst)
OSError: [Errno 18] Invalid cross-device link: '/jup/datasets/TRAIN/PLATE/LOCATION/VERSION2/FRONT/jenoptic-1/images/PLATE_LOC_F_D_jenoptic-1_V2-01BLJ36_3511612000020101_2020-12-10_12-50-18-150_0_0_0_73_0-0-0-0_0_0_0_D60DD87C000015372101_1_0.jpg' -> '/jup/.dvc/cache/2f/6cc7cf7fab02868e12f9d4707d719e.iV3dGZaD64MuEyY4E4Y4QZ'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/dvc/main.py", line 90, in main
    ret = cmd.run()
  File "/usr/local/lib/python3.6/dist-packages/dvc/command/add.py", line 24, in run
    desc=self.args.desc,
  File "/usr/local/lib/python3.6/dist-packages/dvc/repo/__init__.py", line 54, in wrapper
    return f(repo, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/dvc/repo/scm_context.py", line 4, in run
    result = method(repo, *args, **kw)
  File "/usr/local/lib/python3.6/dist-packages/dvc/repo/add.py", line 85, in add
    _process_stages(repo, stages, no_commit, pbar)
  File "/usr/local/lib/python3.6/dist-packages/dvc/repo/add.py", line 124, in _process_stages
    stage.commit()
  File "/usr/local/lib/python3.6/dist-packages/funcy/decorators.py", line 39, in wrapper
    return deco(call, *dargs, **dkwargs)
  File "/usr/local/lib/python3.6/dist-packages/dvc/stage/decorators.py", line 36, in rwlocked
    return call()
  File "/usr/local/lib/python3.6/dist-packages/funcy/decorators.py", line 60, in __call__
    return self._func(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.6/dist-packages/dvc/stage/__init__.py", line 478, in commit
    out.commit()
  File "/usr/local/lib/python3.6/dist-packages/dvc/output/base.py", line 297, in commit
    self.cache.save(self.path_info, self.cache.tree, self.hash_info)
  File "/usr/local/lib/python3.6/dist-packages/funcy/decorators.py", line 39, in wrapper
    return deco(call, *dargs, **dkwargs)
  File "/usr/local/lib/python3.6/dist-packages/dvc/cache/base.py", line 40, in use_state
    return call()
  File "/usr/local/lib/python3.6/dist-packages/funcy/decorators.py", line 60, in __call__
    return self._func(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.6/dist-packages/dvc/cache/base.py", line 317, in save
    self._save(path_info, tree, hash_info, save_link, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/dvc/cache/base.py", line 324, in _save
    self._save_dir(path_info, tree, hash_info, save_link, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/dvc/cache/base.py", line 298, in _save_dir
    entry_info, tree, entry_hash, save_link=False, **kwargs
  File "/usr/local/lib/python3.6/dist-packages/dvc/cache/base.py", line 194, in _save_file
    self.tree.move(path_info, cache_info, mode=self.CACHE_MODE)
  File "/usr/local/lib/python3.6/dist-packages/dvc/tree/local.py", line 176, in move
    move(from_info, to_info, mode=mode)
  File "/usr/local/lib/python3.6/dist-packages/dvc/utils/fs.py", line 102, in move
    shutil.move(src, tmp)
  File "/usr/lib/python3.6/shutil.py", line 565, in move
    os.unlink(src)
OSError: [Errno 30] Read-only file system: '/jup/datasets/TRAIN/PLATE/LOCATION/VERSION2/FRONT/jenoptic-1/images/PLATE_LOC_F_D_jenoptic-1_V2-01BLJ36_3511612000020101_2020-12-10_12-50-18-150_0_0_0_73_0-0-0-0_0_0_0_D60DD87C000015372101_1_0.jpg'
------------------------------------------------------------
2021-02-17 15:50:22,920 DEBUG: Version info for developers:
DVC version: 1.11.16 (pip)
---------------------------------
Platform: Python 3.6.9 on Linux-4.15.0-135-generic-x86_64-with-Ubuntu-18.04-bionic
Supports: http, https
Cache types: hardlink, symlink
Cache directory: overlay on overlay
Caches: local
Remotes: None
Workspace directory: overlay on overlay
Repo: dvc (no_scm)

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
2021-02-17 15:50:22,922 DEBUG: Analytics is enabled.
2021-02-17 15:50:22,989 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/tmp/tmp8z6k9p1y']'
2021-02-17 15:50:22,991 DEBUG: Spawned '['daemon', '-q', 'analytics', '/tmp/tmp8z6k9p1y']'
8

Thanks!

Ok, so your whole dvc repository is on a read-only filesystem. That won’t work, as dvc needs to move files around and that is not permitted by the read-only filesystem. You need to move your dvc project to a normal filesystem and then copy the data you want to add into it and then dvc add it there.

9

thank you I will try

1 Like