Multiple params.yaml

Is it possible to have more than one params.yaml? If so, how should they be organised and named?

The context is I am considering of breaking up my dvc.yaml to multiple pipelines and would like to do the same for the params as well. Is the answer to save a params.yaml at the same place where the pipeline for dvc.yaml will live? something like pipelines/pip1/dvc.yaml,params.yaml and pipelines/pip2/dvc.yaml,params.yaml

1 Like

You can have multiple parameters files but only one file named params.yaml per directory.

Any valid YAML, JSON, TOML, or Python file can be a params file. It’s up to you how to organize or name them. A good way to think about them is as any other config file in your project (so probably git-tracked).

Please see https://dvc.org/doc/command-reference/params and https://dvc.org/doc/user-guide/project-structure/pipelines-files#parameter-dependencies for more details.

That’s a straightforward approach, yes :+1:

Thanks for the quick response. I assume the suggestion then is to break up the pipelines currently in one dvc.yaml into separate pipelines within their own directory and create a params.yaml at the same directory as the dvc.yaml that defines each pipeline.

Let me know if I understood it wrong.

@nsorros Yes, this is a viable approach. Remember, that if you are using params file that is not named params.yaml, you need to address those params properly. For example if you have file other_params.yaml and want to use foo param from inside, you need to address it with -p other_params.yaml:foo when defining the stage.