Dvc exp push origin fails to push to AWS CodeCommit

Questions
1

Hi there! I’ve just started using DVC and am looking to set up DVC experiments to track my experiments. I have got the following experiments that I have run and logged using dvclive, which I am now trying to push:

dvc_experiments_test_branch:                                          
	c50b40a [boozy-aqua]
	72f3a5a [fined-zest]
	349b33e [herby-lama]

But I am getting the following error when running dvc exp push origin -v:

2024-04-12 09:45:16,851 DEBUG: v3.49.0 (pip), CPython 3.10.12 on Linux-6.5.0-1016-aws-x86_64-with-glibc2.35
2024-04-12 09:45:16,851 DEBUG: command: /home/ubuntu/Documents/dvc_test_env/bin/dvc exp push origin -v
2024-04-12 09:45:17,115 DEBUG: git push experiment ['refs/exps/98/558612ded7fb57ebf0bc1837b3adbe76c32fc9/boozy-aqua:refs/exps/98/558612ded7fb57ebf0bc1837b3adbe76c32fc9/boozy-aqua', 'refs/exps/98/558612ded7fb57ebf0bc1837b3adbe76c32fc9/fined-zest:refs/exps/98/558612ded7fb57ebf0bc1837b3adbe76c32fc9/fined-zest', 'refs/exps/98/558612ded7fb57ebf0bc1837b3adbe76c32fc9/herby-lama:refs/exps/98/558612ded7fb57ebf0bc1837b3adbe76c32fc9/herby-lama'] -> 'origin'
2024-04-12 09:45:20,057 ERROR: unexpected error - The remote server unexpectedly closed the connection.
Traceback (most recent call last):
  File "/home/ubuntu/Documents/dvc_test_env/lib/python3.10/site-packages/dvc/cli/__init__.py", line 211, in main
    ret = cmd.do_run()
  File "/home/ubuntu/Documents/dvc_test_env/lib/python3.10/site-packages/dvc/cli/command.py", line 27, in do_run
    return self.run()
  File "/home/ubuntu/Documents/dvc_test_env/lib/python3.10/site-packages/dvc/commands/experiments/push.py", line 54, in run
    result = self.repo.experiments.push(
  File "/home/ubuntu/Documents/dvc_test_env/lib/python3.10/site-packages/dvc/repo/experiments/__init__.py", line 364, in push
    return push(self.repo, *args, **kwargs)
  File "/home/ubuntu/Documents/dvc_test_env/lib/python3.10/site-packages/dvc/repo/__init__.py", line 58, in wrapper
    return f(repo, *args, **kwargs)
  File "/home/ubuntu/Documents/dvc_test_env/lib/python3.10/site-packages/dvc/repo/scm_context.py", line 143, in run
    return method(repo, *args, **kw)
  File "/home/ubuntu/Documents/dvc_test_env/lib/python3.10/site-packages/dvc/repo/experiments/push.py", line 111, in push
    push_result = _push(repo, git_remote, exp_ref_set, force)
  File "/home/ubuntu/Documents/dvc_test_env/lib/python3.10/site-packages/dvc/repo/experiments/push.py", line 153, in _push
    results: Mapping[str, SyncStatus] = repo.scm.push_refspecs(
  File "/home/ubuntu/Documents/dvc_test_env/lib/python3.10/site-packages/scmrepo/git/__init__.py", line 307, in _backend_func
    result = func(*args, **kwargs)
  File "/home/ubuntu/Documents/dvc_test_env/lib/python3.10/site-packages/scmrepo/git/backend/dulwich/__init__.py", line 663, in push_refspecs
    result = client.send_pack(
  File "/home/ubuntu/Documents/dvc_test_env/lib/python3.10/site-packages/dulwich/client.py", line 2125, in send_pack
    ref_status = self._handle_receive_pack_tail(
  File "/home/ubuntu/Documents/dvc_test_env/lib/python3.10/site-packages/dulwich/client.py", line 934, in _handle_receive_pack_tail
    for chan, data in _read_side_band64k_data(proto.read_pkt_seq()):
  File "/home/ubuntu/Documents/dvc_test_env/lib/python3.10/site-packages/dulwich/client.py", line 501, in _read_side_band64k_data
    for pkt in pkt_seq:
  File "/home/ubuntu/Documents/dvc_test_env/lib/python3.10/site-packages/dulwich/protocol.py", line 272, in read_pkt_seq
    pkt = self.read_pkt_line()
  File "/home/ubuntu/Documents/dvc_test_env/lib/python3.10/site-packages/dulwich/protocol.py", line 215, in read_pkt_line
    raise HangupException
dulwich.errors.HangupException: The remote server unexpectedly closed the connection.

2024-04-12 09:45:20,089 DEBUG: link type reflink is not available ([Errno 95] no more link types left to try out)
2024-04-12 09:45:20,089 DEBUG: Removing '/home/ubuntu/Documents/.0xFMgvC9qXuZBpDPIH5A0g.tmp'
2024-04-12 09:45:20,089 DEBUG: Removing '/home/ubuntu/Documents/.0xFMgvC9qXuZBpDPIH5A0g.tmp'
2024-04-12 09:45:20,089 DEBUG: Removing '/home/ubuntu/Documents/.0xFMgvC9qXuZBpDPIH5A0g.tmp'
2024-04-12 09:45:20,089 DEBUG: Removing '/home/ubuntu/Documents/raids-u109-castle-repo/.dvc/cache/files/md5/.KMGEqzV2_6k4yD-zTyW6Zw.tmp'
2024-04-12 09:45:20,100 DEBUG: Version info for developers:
DVC version: 3.49.0 (pip)
-------------------------
Platform: Python 3.10.12 on Linux-6.5.0-1016-aws-x86_64-with-glibc2.35
Subprojects:
	dvc_data = 3.15.1
	dvc_objects = 5.1.0
	dvc_render = 1.0.1
	dvc_task = 0.4.0
	scmrepo = 3.3.1
Supports:
	http (aiohttp = 3.9.3, aiohttp-retry = 2.8.3),
	https (aiohttp = 3.9.3, aiohttp-retry = 2.8.3),
	s3 (s3fs = 2024.3.1, boto3 = 1.34.51)
Config:
	Global: /home/ubuntu/.config/dvc
	System: /etc/xdg/dvc
Cache types: hardlink, symlink
Cache directory: ext4 on /dev/nvme0n1p1
Caches: local
Remotes: s3
Workspace directory: ext4 on /dev/nvme0n1p1
Repo: dvc, git
Repo.site_cache_dir: /var/tmp/dvc/repo/c615bcd70e1f35acafcebbca7063e140

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
2024-04-12 09:45:20,102 DEBUG: Analytics is enabled.
2024-04-12 09:45:20,150 DEBUG: Trying to spawn ['daemon', 'analytics', '/tmp/tmp03zu_gaf', '-v']
2024-04-12 09:45:20,156 DEBUG: Spawned ['daemon', 'analytics', '/tmp/tmp03zu_gaf', '-v'] with pid 4332

My git repo is in AWS CodeCommit and the remote storage for DVC is set as an S3 bucket. For information, when just tracking a model file with DVC (not experiments), dvc push works fine and I can see my models in S3.

I would really appreciate some help on this! If you need any further information or printouts in order to help, please let me know :slight_smile: