# Best practice for hyperparameters sweep
**URL:** https://discuss.dvc.org/t/best-practice-for-hyperparameters-sweep/244
**Category:** Questions
**Created:** [October 7, 2019, 10:35am UTC](https://discuss.dvc.org/t/best-practice-for-hyperparameters-sweep/244 "2019-10-07T10:35:43Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@alex](https://discuss.dvc.org/u/alex)
#### Post date: [October 7, 2019, 10:35am UTC](https://discuss.dvc.org/t/best-practice-for-hyperparameters-sweep/244/1 "2019-10-07T10:35:43Z")
Hi,
I wonder what is best practice for hyperparameter optimization, to do e.g. a parameter sweep, to capture both the models and the metrics. I’d like to try 10 different values for the number of clusters (and also a few other parameters), and then plot figures visualizing each clustering.
Do you just execute `dvc run` in a loop from a script? Then that script essentially becomes the “Makefile”?
Also, do you create a new branch for each experiment, or just a new folder with a unique name?
Thanks!
Alex
---
### Author:  [@Paffciu](https://discuss.dvc.org/u/Paffciu)
#### Post date: [October 7, 2019, 11:03am UTC](https://discuss.dvc.org/t/best-practice-for-hyperparameters-sweep/244/2 "2019-10-07T11:03:02Z")
Hi @alex,
we have some GitHub issues related to this subject.
[https://github.com/iterative/dvc/issues/2532](https://github.com/iterative/dvc/issues/2532) here is one related to your question.
Also here:
[https://github.com/iterative/dvc/issues/2379](https://github.com/iterative/dvc/issues/2379)
Would you mind sharing your use case and describing how would you expect such workflow to behave? The more feedback we gather, the faster we can develop some initial draft on how to proceed with this subject.
---
### Author:  [@alex](https://discuss.dvc.org/u/alex)
#### Post date: [October 7, 2019, 12:33pm UTC](https://discuss.dvc.org/t/best-practice-for-hyperparameters-sweep/244/3 "2019-10-07T12:33:18Z")
Thanks for the links, they address exactly the questions I had! Unfortunately, no solutions yet 🙂
I’ve used a number of ad-hoc solutions, and I was not satisfied with any of them.
For every output file I would like to be able to find exactly how the file was produced, i.e. version of code (git commit), hyperparameters (e.g. command-line parameters to the executables), and data (I have several “datasets” that go through identical pipelines). I experimented with long file- and directory-names (called “experiment-as-a-folder” in issue #2532), and included values of all parameters in a filename (e.g. summaryFigures/dataA\_numclusters5\_epochs10/). Much of this information is already in the DVC file, but the workflow is not “clicking” for me yet.
I think I’d like to separate a generic workflow/pipeline/DAG definition from each of its “instances” (with the hashes and all).
I will watch those two issues on GitHub.
Thanks again!
Alex
---
### Author:  [@dmitry](https://discuss.dvc.org/u/dmitry)
#### Post date: [October 7, 2019, 6:15pm UTC](https://discuss.dvc.org/t/best-practice-for-hyperparameters-sweep/244/4 "2019-10-07T18:15:18Z")
Hi Alex!
> [@alex](#):
>
> I think I’d like to separate a generic workflow/pipeline/DAG definition from each of its “instances” (with the hashes and all).
This is good insight! It would be great if you can share this in the issue tracker (under any of the issues @Paffciu mentioned).
---
### Author:  [@shcheklein](https://discuss.dvc.org/u/shcheklein)
#### Post date: [October 23, 2019, 10:04pm UTC](https://discuss.dvc.org/t/best-practice-for-hyperparameters-sweep/244/5 "2019-10-23T22:04:07Z")
@alex could you please elaborate on the the suggested split (generic workflow vs instances)? Did you mean taking `md5` checksum values out of DVC-files and saving them somewhere? What benefits do you see in doing this?
---
### Author:  [@alex](https://discuss.dvc.org/u/alex)
#### Post date: [October 24, 2019, 5:47am UTC](https://discuss.dvc.org/t/best-practice-for-hyperparameters-sweep/244/6 "2019-10-24T05:47:16Z")
@shcheklein I don’t have a full picture yet, but I’d say that DVC-files would store checksums. Maybe some generic pipeline will instantiate “runs” and save the run-command and track the command result in a DVC-file. I guess I’m looking for some tool that would auto-generate and manage huge amount of DVC-files.
---
### Author:  [@shcheklein](https://discuss.dvc.org/u/shcheklein)
#### Post date: [October 24, 2019, 8:43pm UTC](https://discuss.dvc.org/t/best-practice-for-hyperparameters-sweep/244/7 "2019-10-24T20:43:03Z")
Thanks, @alex! So, am I understand correct that would be some DSL or bash or whatnot to actually execute runs (instantiate DAG)? Could you take a look at this ticket - [https://github.com/iterative/dvc/issues/1018](https://github.com/iterative/dvc/issues/1018) ? what do you think?
---
### Author:  [@alex](https://discuss.dvc.org/u/alex)
#### Post date: [October 29, 2019, 10:45am UTC](https://discuss.dvc.org/t/best-practice-for-hyperparameters-sweep/244/8 "2019-10-29T10:45:59Z")
Yes, ticket [#1018](https://github.com/iterative/dvc/issues/1018) is very relevant. I found no definite solution/recommendation there, but very useful references to snakemake and makepp.
---
### Author:  [@shcheklein](https://discuss.dvc.org/u/shcheklein)
#### Post date: [November 20, 2019, 10:52pm UTC](https://discuss.dvc.org/t/best-practice-for-hyperparameters-sweep/244/9 "2019-11-20T22:52:02Z")
@alex we’ve prioritized the issue and moved the discussion here [https://github.com/iterative/dvc/issues/2799](https://github.com/iterative/dvc/issues/2799) . Please, feel free to chime in. We would really appreciate your involvement.