Update same output dir in different stages

Hi @mauro,

Your case is similar to a recent question which you can find here: Managing pipelines operating per dataset element. It lists some options to workaround the current overlapping output limitation of DVC (with regular files/dirs).

In your case the problem is harder for DVC though, because we work at the file level and have no understanding of internal data formats. So DVC can’t see what’s inside results-file1*, much less track only certain parts of that file per stage.

Is there perhaps any way to create a “view” of a Zarr file that the file system sees as regular dirs and files? Perhaps using symlinks? If so, that could be what DVC tracks (as a proxy to results-file*.

Related issues you may want to read and perhaps participate in: