DVC doesn't seem to notice that I delete or restore a folder

Questions
1

This is probably a bug in my wetware, but I don’t understand what’s happening here.

dev on git by-19-channels [$] via conda company 
> ll data/production_models 
total 256K
drwxr-xr-x 1 john john  152 Apr 10 10:37 .
drwxr-xr-x 1 john john  294 Apr 10 10:36 ..
drwxrwxr-x 1 john john   18 Apr 10 09:59 epi_46621521
drwxr-xr-x 1 john john   30 Apr  6 10:04 cond_202304041327
drwxr-xr-x 1 john john  120 Apr 10 10:04 epi_mc_202304061050
-rw-rw-r-- 1 john john 249K Apr 10 10:04 pcaer.pkl
-rw-rw-r-- 1 john john 1.2K Apr 10 10:04 transformer.pkl

I move epi_46621521 to the Trash using the file browser.

dev on git by-19-channels [$] via conda company 
> ll data/production_models
total 256K
drwxr-xr-x 1 john john  128 Apr 10 10:38 .
drwxr-xr-x 1 john john  294 Apr 10 10:36 ..
drwxr-xr-x 1 john john   30 Apr  6 10:04 cond_202304041327
drwxr-xr-x 1 john john  120 Apr 10 10:04 epi_mc_202304061050
-rw-rw-r-- 1 john john 249K Apr 10 10:04 pcaer.pkl
-rw-rw-r-- 1 john john 1.2K Apr 10 10:04 transformer.pkl

dev on git by-19-channels [$] via conda company 
> dvc status
Data and pipelines are up to date.

DVC didn’t seem to notice.
I restore the folder from the trash.

dev on git by-19-channels [$] via conda company 
> ll data/production_models
total 256K
drwxr-xr-x 1 john john  152 Apr 10 10:38 .
drwxr-xr-x 1 john john  294 Apr 10 10:36 ..
drwxrwxr-x 1 john john   18 Apr 10 09:59 epi_46621521
drwxr-xr-x 1 john john   30 Apr  6 10:04 cond_202304041327
drwxr-xr-x 1 john john  120 Apr 10 10:04 epi_mc_202304061050
-rw-rw-r-- 1 john john 249K Apr 10 10:04 pcaer.pkl
-rw-rw-r-- 1 john john 1.2K Apr 10 10:04 transformer.pkl

dev on git by-19-channels [$] via conda company 
> dvc status               
Data and pipelines are up to date.

Again, DVC didn’t notice a change.
I try just rming the folder.

dev on git by-19-channels [$] via conda company 
> rm -rf data/production_models/epi_46621521 

dev on git by-19-channels [$] via conda company 
> dvc status
Data and pipelines are up to date.

And DVC didn’t notice that either.

Just as a sanity check:

dev on git by-19-channels [$] via conda company 
> touch data/production_models/test.txt

dev on git by-19-channels [$] via conda company 
> dvc status
data.dvc:                                                                                                                                                                                                                                                   
	changed outs:
		modified:           data

dev on git by-19-channels [$] via conda company 
> rm data/production_models/test.txt 

dev on git by-19-channels [$] via conda company 
> mkdir data/production_models/test

dev on git by-19-channels [$] via conda company 
> dvc status
Data and pipelines are up to date.                                                                                                                                                                                                                          

dev on git by-19-channels [$] via conda company 
>

I must be missing something, right?

DVC 2.51.0
Linux pop-os 6.2.6-76060206-generic #202303130630~1679424972~22.04~4a8cde1 SMP PREEMPT_DYNAMIC Tue M x86_64 x86_64 x86_64 GNU/Linux
2

I’m not 100% sure how it’s supposed to work either, but when I track a folder rather than an individual file I see similar behaviour. In order for it to work properly I find I have to

dvc add folder

(where folder is a folder with an associated folder.dvc file)

every time files in a tracked folder are added/deleted or changed.

Its quite annoying, I’d originally assumed DVC would automatically scan for changes but it doesn’t appear to (maybe this is a bug)?

1 Like
3

I think the problem was the folders were empty, which I didn’t realize.
So it seems folders aren’t tracked unless they have files within. Or maybe as you said, if they have a .dvc file.