# Git Flow for DVC 🌿 **URL:** \ **Category:** General\ **Created:** [December 9, 2020, 9:36am UTC](https://discuss.dvc.org/t/git-flow-for-dvc/578 "2020-12-09T09:36:27Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![rabefabi](https://avatars.discourse-cdn.com/v4/letter/r/74df32/32.png) [@rabefabi](https://discuss.dvc.org/u/rabefabi)\ **Post date:** [December 9, 2020, 9:36am UTC](https://discuss.dvc.org/t/git-flow-for-dvc/578/1 "2020-12-09T09:36:27Z")
Hey everyone, Over the past couple of months we have started using DVC in our small team. With a handful of developers all coding, training models & committing in the same repository, we soon realized the need for a workflow. We came up with a concept inspired by _Git Flow_, which I outlined in this post: [https://git.rz.uni-augsburg.de/rabefabi/git-flow-for-dvc](https://git.rz.uni-augsburg.de/rabefabi/git-flow-for-dvc) By publishing this post, I’m following [Cunninghams’ Law](https://meta.wikimedia.org/wiki/Cunningham%27s_Law), hoping to get feedback on how you use DVC in a team, and what your opinions on [our yet unresolved points](https://git.rz.uni-augsburg.de/rabefabi/git-flow-for-dvc#discussion) (or the concept in general) are. Cheers, Fabian ---
**Author:** ![Paffciu](https://avatars.discourse-cdn.com/v4/letter/p/d07c76/32.png) [@Paffciu](https://discuss.dvc.org/u/Paffciu)\ **Post date:** [December 9, 2020, 10:33am UTC](https://discuss.dvc.org/t/git-flow-for-dvc/578/2 "2020-12-09T10:33:05Z")
Hi @rabefabi! Thank you very much for sharing your experiences! Your setup seems to me to be robust and though through. About unresolved problems: Problem #1. Using some kind of CI run on debug dataset, to verify each pull request does not break existing code, even if its kind of configuration change definitely seems like a reasonable idea. Problem #2. > Also, we’re unsure how the [DVC garbage collection](https://dvc.org/doc/command-reference/gc) treats data from executions which are “just in the run cache”. What do you mean by `just in the run cache`? Are you using only `-O` options for `dvc run`? We should probably have more info on `gc` and `run-cache` interaction in our docs. Problem #3. Currently its a bit cumbersome. Editing `params.yam` seems like a way to switch between test run and “master” run. What might be interesting for you is that currently we are developing `experiments` feature which might help you with your desired goal. With latest `dvc` version, you could try out (for example) `dvc exp run {stage_name} --params epochs=3` - this way you would reproduce given `stage_name` with `epochs=3`. ---
**Author:** ![elle](https://avatars.discourse-cdn.com/v4/letter/e/ccd318/32.png) [@elle](https://discuss.dvc.org/u/elle)\ **Post date:** [December 9, 2020, 8:51pm UTC](https://discuss.dvc.org/t/git-flow-for-dvc/578/3 "2020-12-09T20:51:14Z")
@rabefabi this is so cool- do you mind if we share it on our twitter to get more eyes on it? Really love how you are using CI for your DVC pipelines ---
**Author:** ![rabefabi](https://avatars.discourse-cdn.com/v4/letter/r/74df32/32.png) [@rabefabi](https://discuss.dvc.org/u/rabefabi)\ **Post date:** [December 10, 2020, 9:44pm UTC](https://discuss.dvc.org/t/git-flow-for-dvc/578/4 "2020-12-10T21:44:51Z")
Thanks for taking the time to give feedback! #### Problem 2 > What do you mean by `just in the run cache` ? Are you using only `-O` options for `dvc run` ? No, we’re using the `-o` option, but maybe I’ve misunderstood how DVC works here. My understanding is: The CI Runner successfully executed the DVC pipeline, and now has the new outputs (trained model, processed data, …) as well as a new `dvc.lock` in its working directory. To make these outputs accessible from the developers local machine, I see two options: a) The CI Runner does `ci-runner: dvc push --run-cache`, afterwards the I do `dev-local: dvc pull --run-cache && dvc repro`. - Disadvantages: - AFAIK I can’t pull the run cache for just one stage (I need to pull all the preprocessed data as well as the trained model, even if I only want `dvc pull --run-cache model_training`) - Looking at a git branch, it’s not immediately obvious whether a successful `dvc repro` has been executed here (compared to a commit with a changed `dvc.lock`) - I couldn’t find anything in the [garbage collection docs](https://dvc.org/doc/command-reference/gc) regarding how to clean the run cache, or which runs stay cached after cleaning - Advantages: - No extra commit necessary, to check out the results of an experiment, I can pull the commit with which an experiment was started b) `ci-runner: git add dvc.lock && git commit && git push && dvc push` - Disadvantages: - CI-Runner needs to be configured to be able to commit to a branch - The “results” (the dvc.lock) is stored in a separate commit from the one which started the experiments (relevant for e.g., MLFLow which stores the commit hash of an experiment) - Advantages - The changed `dvc.lock` in a branch tells me that this experiment has been successfully executed - I can pull single dvc stages ( `dvc pull model_training`) without downloading all preprocessed data For all these reasons we opted for option b) #### Problem 3 Your new solution (`dvc exp run {stage_name} --params epochs=3`) seems ideal, thanks for the hint! Together with the [parameterlike dependencies](https://discuss.dvc.org/t/dvc-yaml-deps-outs-section-accessible-from-code/577/2) we can streamline our project a lot ---
**Author:** ![rabefabi](https://avatars.discourse-cdn.com/v4/letter/r/74df32/32.png) [@rabefabi](https://discuss.dvc.org/u/rabefabi)\ **Post date:** [December 10, 2020, 9:46pm UTC](https://discuss.dvc.org/t/git-flow-for-dvc/578/5 "2020-12-10T21:46:20Z")
Of course you can share it, the bigger the discussion gets, the better 🙂 ---
**Author:** ![elle](https://avatars.discourse-cdn.com/v4/letter/e/ccd318/32.png) [@elle](https://discuss.dvc.org/u/elle)\ **Post date:** [December 11, 2020, 8:02pm UTC](https://discuss.dvc.org/t/git-flow-for-dvc/578/6 "2020-12-11T20:02:59Z")
we shared on twitter, hope we get some eyes on it. we’ll put in our monthly newsletter, too. [https://twitter.com/DVCorg/status/1337486256947646466](https://twitter.com/DVCorg/status/1337486256947646466) i love the idea for a debugging dataset. get all the format/input-output errors out of the way before you get to your full dataset on CI.