I accidentally deleted the cache folder in my project. And then I tested dvc gc after a series of experiments and runs of dvc repro. The error that appeared was something like this - no unused local cache. And accordingly, dvc gc -w did not delete anything. How can I fix it?
dvc gc -w removes files from your cache directory that are not referenced in your current workspace. If you have already deleted the entire cache folder, dvc gc will not do anything.
Can you please clarify what you are expecting to be deleted when you run dvc gc?
Hi, let me add some details to my colleague’s question. I ran several repros and exps commands in my repo but when I try to clear cache of exps nothing changes and I get message:
No unused ‘local’ cache to remove.
No unused ‘legacy’ cache to remove.
Here is the final output:
I am allowed to add only one media, but i descrived all steps here
Please help us to understand the behavior of dvc gc. My dvc version is DVC version: 3.2.1.
Please update to the latest DVC release (3.41.0) and then retry using dvc gc
Thanks, your advice helped me. But could you give me deeper explanation of the results, please? After I ran dvc gc -A, system reported about 3 removed objects.
But nothing was deleted from .dvc/cache/runs (where I expected), and at the same time in other directory - .dvc/cache/files/md5 3 folders were cleaned.
-
Now I guess that real results of dvc outputs for exps and repros are cached in .dvc/cache/files/md5. Is my suggestion right? Than what does .dvc/cache/runs store?
-
I saved repo before running dvc gc -A and understood that only versions of last repros or exp runs before git commiting were saved. For my repro:
Am I rigth thinking that git commit references results of last dvc exp run or dvc repro, because both save results to dvc cache? -
Is there a way to run dvc exp without storing results in cache like dvc repro with --no-commit option?
Than what does .dvc/cache/runs store?
dvc gc does not affect .dvc/cache/runs. That directory is used for storing DVC run cache. Run cache is how DVC identifies whether or not a stage needs to be re-run or has not changed, none of the actual output data for dvc repro or dvc exp run is stored in the run cache.
Am I rigth thinking that git commit references results of last dvc exp run or dvc repro, because both save results to dvc cache?
I’m not sure I understand your question. What do you mean by “saved your repo”? If you use dvc gc -A, any DVC files which are referenced by Git commits in your repo will be preserved.
With -A, DVC will go through your Git history, and check each commit for any .dvc or dvc.lock files. Any DVC tracked data used by those .dvc/dvc.lock files will be preserved, and dvc gc will remove unused data from the DVC cache.
Is there a way to run dvc exp without storing results in cache like dvc repro with --no-commit option?
No, if you need this behavior you can just use dvc repro. dvc exp run needs to store the results of your experiment somewhere, and this is done by storing any DVC tracked results in the DVC cache.
Thank you, very much!