Dvc cannot connect to s3

Bug Reports
1

Environment:
Ubuntu18.04
python3.6
boto3==1.17.49
botocore==1.20.49
dvc==2.3.0

I have created (s3-compatible) MINIO bucket with read/write access named dvc and configured the dvc as follows

12:31

[core]
remote = dvc
[‘remote “dvc”’]
url = s3://dvc
endpointurl = http://192.168.3.32:9000

my config file;

[core]
    remote = dvc
['remote "dvc"']
    url = s3://dvc
    endpointurl = http://192.168.3.32:9000

When i do dvc add i got core dump error

Checked solutions

2

Hi @merve!
Would it be possible to post the log for command run with -v option?

3

2021-06-04 10:59:27,163 DEBUG: Check for update is enabled.
2021-06-04 10:59:29,385 DEBUG: state save (773292125, 1608142828000000000, 3766) 6be79816758ab0b2773391e2b7c80533
2021-06-04 10:59:29,408 DEBUG: state save (773292126, 1608142638000000000, 5188) 2c8c0e4d3e959e818bb2a86f694acfa2
2021-06-04 10:59:29,416 DEBUG: state save (773292127, 1608142526000000000, 3977) 01b5f7180045d33a527096ad103027a4
2021-06-04 10:59:29,424 DEBUG: state save (773292128, 1608142889000000000, 5012) 1952c68a414998260439c11c9591d252
2021-06-04 10:59:29,433 DEBUG: state save (773292129, 1608142620000000000, 4389) 33d6b2dde40a3edb8af103554c9e1a1d
2021-06-04 10:59:29,441 DEBUG: state save (773292130, 1608142607000000000, 6829) 51d3fa54ce6b9c392149e3a434c02c35
Adding…Bus error (core dumped)

4

Hey Merve! Would you mind sharing dvc doctor and pip freeze | grep s3fs. Also please post the full logs (including the command).

5 
dvc doctor
DVC version: 2.3.0 (pip)
---------------------------------
Platform: Python 3.6.9 on Linux-4.15.0-143-generic-x86_64-with-Ubuntu-18.04-bionic
Supports: All remotes
Cache types: <https://error.dvc.org/no-dvc-cache>
Caches: local
Remotes: s3
Workspace directory: nfs4 on 192.168.3.13:/mnt/raid/users/process_team
Repo: dvc, git

pip freeze | grep s3fs
/usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
/usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
s3fs==2021.5.0
6

@merve
so the failure occurs when using dvc add?
Would it be possible to provide dvc add -v result?

7

@Paffciu
this error is before push it is when i do dvc add -v

and it gives bus error(core dumped)

8

this error is before push it is when i do dvc add -v

Unless you are working with external outputs, dvc add doesn’t interact with the s3 at all so the issue doesn’t seem to be something relevant with ‘connecting to the s3’.

There isn’t much that comes to my mind, though perhaps you could try python -Xfaulthandler -m dvc add -v and try to get some results (unlikely, but might give a traceback).