Custom params not working when templating pipeline

[lucas]

Hi, there. I have this question that seems to be a bug, but I really don’t know if I’m messing things up, or I’m mistaken.

I want to use a custom params file to track all the hyperparameters of the model in the framework I’m using (detectron2’ config.yaml) . Just to be clear: I don’t want to use the standard-named “params.yaml”. I’ve already tried to rename the configuration file to params.yaml and it does work, but I’m not allowed to use that because of our project’s naming conventions.

Just to make things easier, I’m working with an example that I found on the internet that should work, just with an echo command.

$ dvc run -n hello "echo Hello"

It creates the following dvc.yaml:

stages:
  hello:
    cmd: echo Hello

I would like that it says hello to a value of a parameter defined inside a custom param file, for example:

custom.yaml
NAME: Lucas

Then, change the original dvc.yaml to the following:

stages:
  hello:
    params:
      - custom.yaml:
    cmd: echo Hello ${NAME}

When I run dvc repro, it gives me the following error message:
ERROR: failed to parse 'stages.hello.cmd' in 'dvc.yaml': Could not find 'NAME'

It turns out that if I rename custom.yaml to params.yaml, erasing (or not) the whole “params” section in the dvc.yaml, it works as expected, printing: " Hello Lucas".

My first intuition was that it wasn’t reading my custom.yaml at all. So I tried this third version of dvc.yaml:

stages:
  hello:
    params:
      - custom.yaml:
    cmd: echo Hello

Note that I erased the template part from the command, and I intentionally introduced a syntax error on custom.yaml, like this:
NAME: "Lucas

Then, dvc repro gives this error.
ERROR: failed to reproduce 'hello': Unable to read parameters from 'custom.yaml': unable to read: 'custom.yaml', YAML file structure is corrupted: ...

But if fix the syntax error, and restore the template part of the command, it goes back to the first error:
ERROR: failed to parse ‘stages.hello.cmd’ in ‘dvc.yaml’: Could not find ‘NAME’

So, it seems to parse the command first, and if there are no errors, then it tries to parse the custom.yaml … or I don’t know

Any help will be really appreciated.

[YanxiangGao]

@lucas

    params:
      - custom.yaml:
    cmd: echo Hello ${NAME}

Hi, Lucas, you can use vars instead of params

    vars:
      - custom.yaml:NAME
    cmd: echo Hello ${NAME}

[lucas]

Hi, YanxiangGao, thank you very much for your help! It did work! but I’m not getting what I need.

Your suggestion worked perfectly for the example, but dvc didn’t want to take it any further. I’m still not sure if I got it wrong, or dvc isn’t working as expected.

I need to import the whole file. I mean all the params or vars, or just a subset of them, but not just one as the example we just tried.

In the docs I’ve read that it could be achieved using the following ways:

For the whole file:

    vars:
      - custom.yaml:

For some params or vars:

vars:
      - custom.yaml:
        - NAME
        - LASTNAME

as an example.
No one of them worked. I also tried:

params:
      - custom.yaml:NAME

But it didn’t work either.

Once again, thank you very much for your help!!! Any pointers or suggestion will be appreciated!

[YanxiangGao]

@lucas

stages:
  hello:
    params:
      - custom.yaml:
    cmd: echo Hello ${NAME}

Hello, @lucas , you do not need to add a : after custom.yaml, it’s invalid in yaml format, some thing like

stages:
  hello:
    vars:
      - custom.yaml
    cmd: echo Hello ${NAME} > ${LASTNAME}

works for me

[lucas]

Ok, YanxiangGao, great! It works as you proposed, thank you very much!!

I think I misunderstood the syntax, since from the docs (params section) I’ve understood that a colon is needed in order to import all params from the custom params file.

Params documentation link

But it doesn’t work anyway (with or without colon).

Checking the documentation of the “vars” section, it says exactly what you’ve suggested, and works Great!

For now, it’s enough for me. I mean, I can start to work with this.

Anyway, I think there’s a bug in parameters functionality in dvc, or documentation should be properly fixed in the Templating section, since it only works when the default parameters file (params.yaml) is used.

[nini]

Great thread! Thanks for the help. I have multiple sub directories with their own dvc.yaml and params.yaml files. After having initially only 1 of these sub directories with working templating within stages with ${var}, but this did not work until i added vars: -custom.yaml before the places referenced in commands. As well as adding an indent on my params and a custom.yaml header for the referenced parameters.

This also did not work using the params.yaml file as it would mess up over the python api since multiple variables were named the same.

Changing the name to custom.yaml ended up working to reference the vars since I guess params gets overwritten within the workspace.