Is it possible to output the stage file in a different directory than CWD?

Questions
1

I typically run scripts from my project’s root directory, and use parameters to point to inputs and outputs as needed. Wrapping with dvc run generates a stage file in the root directory, which clutters the project.

I tried using the -f option in dvc run to specify a different path for the stage file, but it ended up causing problems with the data paths. In particular it appears that DVC considers the dependencies and output paths to be relative to the stage file.

My current solution is to make a runs directory from where I execute all dvc run commands. All the stage files end up in there. But this is a bit clunky and isn’t the pattern I usually use.

I’m wondering if I’m missing something here? Is there a way to store stage files to somewhere other than where the dvc run command was executed?

2

Hi @garrin !

Try using -c|--cwd option with dvc run to specify the directory that you want your command to be ran from. Indeed, dvc runs a command specified in the stage file from the directory that stage file is located in. It also treats paths for dependencies and outputs as relative to that directory. This is done to avoid confusion between paths in your command and paths to deps/outs in dvc run. For example, if your project looks like:

data
code
runs

you could use this command from the root directory:

dvc run -d ../data/input -o ../data/output \
        -c runs -f stage.dvc \
        python ../code/script.py ../data/input ../data/output

This will create a runs/stage.dvc file. Notice how paths specified by -d and -o are the same as the ones passed to the script.py as arguments.

Thanks,
Ruslan

3

For the record, all commands support -f option to specify path and name of the DVC-file.

2 Likes