Creating sections of experiments

I am new to DVC and I have following use case. Say I have different kinds of experiments (For example T1, T2, T3). Now all these experiment share same code but some change in params (For example, param1, param2). Now for each combination of param1 and param2 (which corresponds to T1, T2 or T3), I need to perform hyper-parameter tuning and then apply the best performing experiment (using dvc exp apply).

So, how should I organise above use case? What would be the best practice? I have following two options:

  1. Create different branch for T1, T2 and T3 and perform tuning in the corresponding branch.
  2. Perform all experiments in one branch an apply whatever experiment needs to be applied based on requirement.

I think either can be a “best practice” and there’s no right answer here. It depends on whether you want to work with them as 3 separate sets of experiments or one mega-tuning. There functionally is very little difference, so it’s entirely up to you.

1 Like

Is the following case possible:

I want to keep different sets of experiments in different branch

For example, I created a branch b1 from main and performed few experiments in branch b1 and committed all the dvc files generated.

Now, when I go to main branch via git checkout main and dvc checkout. The command dvc exp show shows the experiment done in the branch b1. But, what I want is to keep the experiments separated (in different sections).

Is something like above possible?

Update (What I found in docs)

We can tag each commits to create sections instead of creating branches as shown in this link.

Please let me know if there is any better technique.

@anupam dvc exp show should show the experiments from whatever branch you are in. Do you have different commits in main and b1? If there’s an actual difference between the branches, then you should only see the experiments for that branch.