Is it possible to have more than one params.yaml? If so, how should they be organised and named?
The context is I am considering of breaking up my dvc.yaml to multiple pipelines and would like to do the same for the params as well. Is the answer to save a params.yaml at the same place where the pipeline for dvc.yaml will live? something like pipelines/pip1/dvc.yaml,params.yaml and pipelines/pip2/dvc.yaml,params.yaml
1 Like
You can have multiple parameters files but only one file named params.yaml
per directory.
Any valid YAML, JSON, TOML, or Python file can be a params file. It’s up to you how to organize or name them. A good way to think about them is as any other config file in your project (so probably git-tracked).
Please see params and dvc.yaml Files for more details.
That’s a straightforward approach, yes
Thanks for the quick response. I assume the suggestion then is to break up the pipelines currently in one dvc.yaml into separate pipelines within their own directory and create a params.yaml at the same directory as the dvc.yaml that defines each pipeline.
Let me know if I understood it wrong.
@nsorros Yes, this is a viable approach. Remember, that if you are using params file that is not named params.yaml
, you need to address those params properly. For example if you have file other_params.yaml
and want to use foo
param from inside, you need to address it with -p other_params.yaml:foo
when defining the stage.