DVC pull fails without sudo

Bug Reports
1

I am currently using ssh for the remote dvc storage. I had initialized it using the below commands

git commit -m 'initialize DVC'
python -m dvc remote add -d myremote ssh://xxxxx@xxxxx:xxxxx
git commit .dvc/config -m "initialize DVC local remote"
python -m dvc remote modify myremote user xxxxx
python -m dvc remote modify myremote keyfile ~/.ssh/id_rsa
python -m dvc remote modify myremote ask_password true

Now my colleagues have to use sudo everytime they try to do a dvc pull.
All my clients run on ubuntu 16.04
Any idea how the use of sudo can be avoided

2

Hi @shaijujanardhanan !

Could you please provide more info? Why do your colleagues have to use sudo and what error is thrown otherwise?

Thanks,
Ruslan

3

Hi @kupruser
Please find the trace below

    Debug: SELECT count from state_info WHERE rowid=1
Debug: fetched: [(0,)]
Debug: UPDATE state_info SET count = 0 WHERE rowid = 1
Error: Traceback (most recent call last):
  File "/home/shaiju/.local/lib/python2.7/site-packages/dvc/command/data_sync.py", line 29, in do_run
    force=self.args.force)
  File "/home/shaiju/.local/lib/python2.7/site-packages/dvc/project.py", line 803, in pull
    with_deps=with_deps)
  File "/home/shaiju/.local/lib/python2.7/site-packages/dvc/project.py", line 782, in fetch
    jobs=jobs)['local']
  File "/home/shaiju/.local/lib/python2.7/site-packages/dvc/project.py", line 646, in _used_cache
    jobs=jobs)
  File "/home/shaiju/.local/lib/python2.7/site-packages/dvc/project.py", line 605, in _collect_used_cache
    jobs=jobs)
  File "/home/shaiju/.local/lib/python2.7/site-packages/dvc/project.py", line 556, in _collect_dir_cache
    show_checksums=False)
  File "/home/shaiju/.local/lib/python2.7/site-packages/dvc/data_cloud.py", line 127, in pull
    show_checksums=show_checksums)
  File "/home/shaiju/.local/lib/python2.7/site-packages/dvc/remote/local.py", line 673, in pull
    remote.exists(remote.md5s_to_path_infos(['000']))
  File "/home/shaiju/.local/lib/python2.7/site-packages/dvc/remote/ssh.py", line 106, in exists
    port=self.port)
  File "/home/shaiju/.local/lib/python2.7/site-packages/dvc/remote/ssh.py", line 98, in ssh
    password=self.password)
  File "/usr/local/lib/python2.7/dist-packages/paramiko/client.py", line 424, in connect
    passphrase,
  File "/usr/local/lib/python2.7/dist-packages/paramiko/client.py", line 630, in _auth
    key_filename, pkey_class, passphrase,
  File "/usr/local/lib/python2.7/dist-packages/paramiko/client.py", line 551, in _key_from_filepath
    key = klass.from_private_key_file(key_path, password)
  File "/usr/local/lib/python2.7/dist-packages/paramiko/pkey.py", line 206, in from_private_key_file
    key = cls(filename=filename, password=password)
  File "/usr/local/lib/python2.7/dist-packages/paramiko/rsakey.py", line 48, in __init__
    self._from_private_key_file(filename, password)
  File "/usr/local/lib/python2.7/dist-packages/paramiko/rsakey.py", line 169, in _from_private_key_file
    data = self._read_private_key_file('RSA', filename, password)
  File "/usr/local/lib/python2.7/dist-packages/paramiko/pkey.py", line 278, in _read_private_key_file
    with open(filename, 'r') as f:
IOError: [Errno 13] Permission denied: '/home/xxx/.ssh/id_rsa'

Error: Failed to pull data from the cloud: [Errno 13] Permission denied: '/home/xxx/.ssh/id_rsa'
4

@shaijujanardhanan Thanks! So the issue here is simply wrong permissions on your /home/xxx/.ssh/id_rsa and it was not caused by the dvc itself. Did you generate your key as root? Here is an example from my machine:

$ ls -la ~/.ssh/id_rsa
-rw------- 1 efiop efiop 1675 вер 14  2015 /home/efiop/.ssh/id_rsa

You should see the same, with the exception of user name :slight_smile:

If it is not the same, try using this command:

chmod 600 ~/.ssh/id_rsa

and trying dvc pull again.

Please let us know if it worked for you.

Thanks,
Ruslan

5

HI @kupruser
I executed chmod against the id_rsa key as you had mentioned. But was still getting the permission issue. After which i gave read only permission for others on the id_rsa key which finally resolved the issue. Please find the result of ls -al below.

xxx@xxxx:$ ls -la /home/xxx/.ssh/id_rsa
-rw----r-- 1 xxx xxx 1679 Mar 12  2018 /home/xxxx/.ssh/id_rsa

But i think this is not a safe approach to adopt. Is there a way that i can add a new keyfile to an existing dvc remote. This would solve the issue as all my colleagues keys can be added

6

Hi @kupruser

I was able to get over this issue by unsetting the keyfile for the remote

1 Like
7

@shaijujanardhanan Glad it worked! :slightly_smiling_face: What do you mean by “unsetting” though?

8

Hi Ruslan,
Initially the configuration was set as below

python -m dvc remote modify myremote keyfile ~/.ssh/id_rsa

To remove the restriction introduced by the key i did the following
python -m dvc remote modify myremote unset keyfile

1 Like
9

Sorry for the delay.

Ah, I think I know what the problem was. When you’ve specified ~/.ssh/id_rsa, dvc config evaluated it into /home/you/.ssh/id_rsa, which made it work for you, but not for your colleagues, since they don’t have permissions to access your id_rsa. This is a bug. Created https://github.com/iterative/dvc/issues/1608 to track it.

Thank you for your feedback!