Hi @maartenb sorry for late response,
So the problem here is that both stages (while different due to parameters use) use the same dependency:
- my_script.py
dvc is locking it and hence you get an error. So this is a little bit different use case than the one linked by you in your post, where the pipelines are separate completely.
I am afraid there is no straightforward workaround for that, besides creating several copies of the script and naming them differently, though that would probably not be very elegant.
Having said so, you might be interested in exp command, which should allow you to achieve your goal:
You could get parallel execution with the following commands:
dvc exp run setup@sim1 --queue
dvc exp run setup@sim2 --queue
dvc exp run -j {2}
Please read on exp and share your thoughts on whether this is something you might find useful.