Facing issue on dvc push

I have a 100 GB dataset on a machine. I set remote location and all other required configurations. Now, I need to push it to its remote location, (dvc push) but its throwing error,

ERROR: unexpected error - Failure
Having any troubles? Hit us up at Support | Platinum Services For MLOps, we are always happy to help!

Anyone knows what am I missing here?

Could you show the output of dvc push -v?

2024-06-11 17:16:46,172 DEBUG: v2.56.0 (pip), CPython 3.8.10 on Linux-5.15.0-101-generic-x86_64-with-glibc2.29
2024-06-11 17:16:46,172 DEBUG: command: /home/user/env/bin/dvc push -v
2024-06-11 17:16:46,469 DEBUG: Preparing to transfer data from '/opt/source/model_u_test_u_1_6522cc/.dvc/cache' to '/media/dest/model_u_test_u_1_6522cc'
2024-06-11 17:16:46,469 DEBUG: Preparing to collect status from '/media/dest/model_u_test_u_1_6522cc'
2024-06-11 17:16:46,470 DEBUG: Collecting status from '/media/dest/model_u_test_u_1_6522cc'
2024-06-11 17:16:46,478 DEBUG: Querying 1 oids via object_exists                                                                                                                                           
2024-06-11 17:16:46,780 ERROR: unexpected error - Failure                                                                                                                                                  
Traceback (most recent call last):
  File "/home/user/env/lib/python3.8/site-packages/dvc/cli/__init__.py", line 210, in main
    ret = cmd.do_run()
  File "/home/user/env/lib/python3.8/site-packages/dvc/cli/command.py", line 26, in do_run
    return self.run()
  File "/home/user/env/lib/python3.8/site-packages/dvc/commands/data_sync.py", line 60, in run
    processed_files_count = self.repo.push(
  File "/home/user/env/lib/python3.8/site-packages/dvc/repo/__init__.py", line 65, in wrapper
    return f(repo, *args, **kwargs)
  File "/home/user/env/lib/python3.8/site-packages/dvc/repo/push.py", line 92, in push
    result = self.cloud.push(
  File "/home/user/env/lib/python3.8/site-packages/dvc/data_cloud.py", line 154, in push
    return self.transfer(
  File "/home/user/env/lib/python3.8/site-packages/dvc/data_cloud.py", line 135, in transfer
    return transfer(src_odb, dest_odb, objs, **kwargs)
  File "/home/user/env/lib/python3.8/site-packages/dvc_data/hashfile/transfer.py", line 203, in transfer
    status = compare_status(
  File "/home/user/env/lib/python3.8/site-packages/dvc_data/hashfile/status.py", line 178, in compare_status
    dest_exists, dest_missing = status(
  File "/home/user/env/lib/python3.8/site-packages/dvc_data/hashfile/status.py", line 134, in status
    exists = hashes.intersection(
  File "/home/user/env/lib/python3.8/site-packages/dvc_data/hashfile/status.py", line 42, in _indexed_dir_hashes
  File "/home/user/env/lib/python3.8/site-packages/tqdm/std.py", line 1195, in __iter__
    for obj in iterable:
  File "/home/user/env/lib/python3.8/site-packages/dvc_objects/db.py", line 359, in list_oids_exists
    in_remote = self.fs.exists(paths, batch_size=jobs)
  File "/home/user/env/lib/python3.8/site-packages/dvc_objects/fs/base.py", line 365, in exists
    return fut.result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 444, in result
    return self.__get_result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/home/user/env/lib/python3.8/site-packages/dvc_objects/executors.py", line 134, in batch_coros
    result = fut.result()
  File "/home/user/env/lib/python3.8/site-packages/fsspec/asyn.py", line 589, in _exists
    await self._info(path)
  File "/home/user/env/lib/python3.8/site-packages/sshfs/utils.py", line 27, in wrapper
    return await func(*args, **kwargs)
  File "/home/user/env/lib/python3.8/site-packages/sshfs/spec.py", line 127, in _info
    attributes = await channel.stat(path)
  File "/home/user/env/lib/python3.8/site-packages/asyncssh/sftp.py", line 4573, in stat
    return await self._handler.stat(path, flags)
  File "/home/user/env/lib/python3.8/site-packages/asyncssh/sftp.py", line 2695, in stat
    return cast(SFTPAttrs,  await self._make_request(
  File "/home/user/env/lib/python3.8/site-packages/asyncssh/sftp.py", line 2454, in _make_request
    result = self._packet_handlers[resptype](self, resp)
  File "/home/user/env/lib/python3.8/site-packages/asyncssh/sftp.py", line 2470, in _process_status
    raise exc
asyncssh.sftp.SFTPFailure: Failure

2024-06-11 17:16:46,851 DEBUG: link type reflink is not available ([Errno 95] no more link types left to try out)
2024-06-11 17:16:46,852 DEBUG: Removing '/opt/source/.VphzLbBFxtgnKoMCvp9SuF.tmp'
2024-06-11 17:16:46,852 DEBUG: Removing '/opt/source/.VphzLbBFxtgnKoMCvp9SuF.tmp'
2024-06-11 17:16:46,853 DEBUG: Removing '/opt/source/.VphzLbBFxtgnKoMCvp9SuF.tmp'
2024-06-11 17:16:46,853 DEBUG: Removing '/opt/source/model_u_test_u_1_6522cc/.dvc/cache/.jzQSyAaZ2vRGRzShH8xv79.tmp'
2024-06-11 17:16:46,869 DEBUG: Version info for developers:
DVC version: 2.56.0 (pip)
Platform: Python 3.8.10 on Linux-5.15.0-101-generic-x86_64-with-glibc2.29
	dvc_data = 0.47.1
	dvc_objects = 0.25.0
	dvc_render = 0.4.0
	dvc_task = 0.2.1
	scmrepo = 1.6.0
	http (aiohttp = 3.8.4, aiohttp-retry = 2.8.3),
	https (aiohttp = 3.8.4, aiohttp-retry = 2.8.3),
	ssh (sshfs = 2023.4.1)
	Global: /home/user/.config/dvc
	System: /etc/xdg/dvc
Cache types: hardlink, symlink
Cache directory: ext4 on /dev/nvme0n1p2
Caches: local
Remotes: ssh
Workspace directory: ext4 on /dev/nvme0n1p2
Repo: dvc (no_scm)
Repo.site_cache_dir: /var/tmp/dvc/repo/d3584894478ce49f7926200c5098a117

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
2024-06-11 17:16:46,871 DEBUG: Analytics is enabled.
2024-06-11 17:16:46,903 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/tmp/tmpmntpcwv0']'
2024-06-11 17:16:46,904 DEBUG: Spawned '['daemon', '-q', 'analytics', '/tmp/tmpmntpcwv0']'

Are you able to upload files to that server without DVC?

Yes. I am able to upload transfer files using rsync and scp

Since the error is raised by asyncssh, you could take a look at that library and see if you are able to upload files using asyncssh without dvc.

1 Like