# True parallel dvc repro of independent branches on shared workspace (Databricks) — rwlock prevents concurrent execution

**URL:** <https://discuss.dvc.org/t/true-parallel-dvc-repro-of-independent-branches-on-shared-workspace-databricks-rwlock-prevents-concurrent-execution/2706>\
**Category:** Bug Reports\
**Created:** [July 17, 2026, 3:52am UTC](https://discuss.dvc.org/t/true-parallel-dvc-repro-of-independent-branches-on-shared-workspace-databricks-rwlock-prevents-concurrent-execution/2706 "2026-07-17T03:52:45Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![daniel.castillo99](https://avatars.discourse-cdn.com/v4/letter/d/e19b73/32.png) [@daniel.castillo99](https://discuss.dvc.org/u/daniel.castillo99)\
**Post date:** [July 17, 2026, 3:52am UTC](https://discuss.dvc.org/t/true-parallel-dvc-repro-of-independent-branches-on-shared-workspace-databricks-rwlock-prevents-concurrent-execution/2706/1 "2026-07-17T03:52:45Z")

</div>

**Environment**

- DVC version: 3.67.1 (pip)
- Platform: Databricks Asset Bundle (Azure).

**Problem**

I have a pipeline with two independent branches (ModelA and ModelB) that share a common root but diverge completely after `data_explorer`:

```auto
dvc dag
+------------+ +------------+
| transform | | transform | (same stage)
+------------+ +------------+
      | |
+------------+ +------------+
| data_expl | | data_expl | (same stage)
+------------+ +------------+
      | |
+------------+ +------------+
| train_A | | train_B | ← INDEPENDENT
+------------+ +------------+
      ... ...
+------------+ +------------+
| hyper_A | | hyper_B |
+------------+ +------------+
          \ /
        +----------+
        | assembly |
        +----------+

```

Each branch takes ~3 hours. I’m using **Databricks multi-task jobs** to orchestrate the pipeline. When I configure `tunning_A`(hyper\_A ) and `tunning_B`(hyper\_B) as parallel tasks (both `depends_on: setup`), I get:

```auto
ERROR: Unable to acquire lock. Most likely another DVC process is running
or was terminated abruptly. (.dvc/tmp/rwlock)

```

**Root cause identified**

All Databricks tasks in the same job share the same workspace path (`/Workspace/Users/.../.bundle/project/files/`), so they share the same rwlock. DVC acquires an exclusive lock for the **entire** `dvc repro` operation, making concurrent execution on the same workspace impossible.

**Question**

The [official docs](https://dvc.org/doc/command-reference/repro#parallel-stage-execution) say:

> “you can launch dvc repro multiple times concurrently (e.g. in separate terminals)”

But this seems to only work in truly separate working directories (different clones). Is there:

1. A way to configure DVC’s lock timeout/retry behavior so one process **waits** instead of immediately failing?
2. A `--no-lock` flag or equivalent for `dvc repro` when the user guarantees the stages are independent?
3. A recommended pattern for running independent DAG branches in parallel within a CI/CD system that shares a single workspace (not multiple git clones)?
4. Would running each branch task in a separate temp directory (copy of the workspace files) on local disk, then merging dvc.lock results, be a supported/safe approach?

**What I’ve tried**

- Cleaning rwlock before each task: helps with stale locks from aborted runs but not with active concurrent locks
- Sequential execution: works but totals ~6h instead of ~3h

Any guidance appreciated.