How to force dvc get

From my CI pipeline, I would like to download some files tracked using DVC.

data
└── test_data
    ├── tests_subfolder_1
    │   ├── file1.test
    │   └── file2.test
    ├── tests_subfolder_1.dvc
    ├── tests_subfolder_2

… etc.

When I run dvc get . data/test_data --out data/, I get the error:

ERROR: failed to get 'data/test_data/' from '.' - unable to remove 'data/test_data/test_subfolder.dvc' without a confirmation. Use `-f` to force.

This is surprising to me, because I would expect test_subfolder.dvc and test_subfolder/ to be able to coexist.

However, I don’t know where to put the -f argument? I tried:

  • dvc -f get . data/test_data --out data/
  • dvc get . data/tests_data --out data/ --force

I also tried to work around the problem with:
dvc get . data/test_data/* --out data/

However, my final solution was:
ls data/test_data/*.dvc | sed -e 's/\.dvc$//' | xargs -I {} dvc get . {} --out data/test_data/

This work-around feels over-complicated. What was I supposed to do?

Hi

-f or --force can go anywhere after dvc get.

However, I’m trying to understand your scenario here and I don’t quite get it. You are trying to “download” data already tracked in your own project, and put it in the same location it already exists? That’s what dvc get . data/test_data --out data/ looks like.

TBH I’m not even sure why we support . as the URL provided to get (and import). See the ref. here: https://dvc.org/doc/command-reference/get

Also, is there a data/test_data.dvc file in your project? Otherwise the command should fail, I believe. Maybe the error message is wrong and this is the problem. If so would you mind opening a bug report for that in https://github.com/iterative/dvc/issues? :slightly_smiling_face:

p.s. please note we’ve officially released DVC 1.0! Your project looks like it’s still using 0.x — we highly recommend migrating.