How to continually update a model with new data

Questions
1

I have a reinforcement learning use case with the following initial steps:

  1. Construct initial episodes
  2. Add episodes to initial experience replay buffer
  3. Train initial model

Then the following steps which repeat every day:

  1. Construct most recent time steps
  2. Add new time steps to existing experience replay buffer
  3. Fine-tune existing model
  4. Compare existing model and fine-tuned model and keep the one that’s best

What would be the best way to do this using DVC? Is it even possible given that circular dependencies are not allowed?

2

You can use persist: true to prevent outputs from being deleted during stage execution (see https://dvc.org/doc/command-reference/repro#description), so you can feed in new data each day and have the subsequent stages read from their existing outputs before writing out newly modified versions.

3

Thanks! I think that should work