I tried to run experiments by comand:
dvc exp run --run-all
How I understand execute files are located in my case in .dvc\tmp\exps\tmp105umt89\src\stages.
But there are src.utils.files module in my directory and this files tree copy to .dvc\tmp\exps\tmp105umt89 folder. I am using VS code and I use options *Terminal > Integrated > Env to add the workspace folder to pythonpath. It works good with dvc repro but it’s not working with dvc run exp.
The main reason is that I can’t use relative import in python modulus which run as scripts. These experiments folders don’t have constant pathes.
So, how can I run the exps in this case? Does the dvc have an option to add path to pythonpath localy?
In the latest DVC release you can use the -C/--copy-paths option to add your python module directories so they are copied into the temporary execution workspace like:
dvc exp run --queue --copy-paths src/utils
See: exp run
Alternatively, you could add your entire src directory as the stage dependency (which would also make DVC copy the directory into the execution workspace)
Initially, I have these scripts files in my stage deps and this files copy to my execution workspace how I understand. I can see these scripy files in my tmp directory as I described before. But sys.path doesn’t have this directory when the copy to tmp dir.
To clarify my point. Initially I have a worksace dir which is in sys.path because I use specific settind of my vscode so the workspace is added to my sys.path.
If I understand correctly the tree of workspace files are copied to something like that .dvc\tmp\exps\tmp\105umt89 for each experimetns. And this directory is not in my sys.path so the mains sripts (preapare.py, train.py, …) don’t see needed modules.
I solved my issue by coping my scripts in src dir on the top level. But interesting that this approach doesn’t use in the dvc course. And the course’s stages files run from src/stages. I didn’t run the course files. but when I try to use this approach I have problem.