Dvc exp show, dvc queue status execution time

Questions
1

Hello,

How long should dvc queue status and dvc exp show take to run? Dvc queue status is going on ten minutes now for 20 experiments or so. Earlier it took 5 minutes for dvc exp show. These seem like they should be quick commands like git status which I spam constantly.

Any tips for how to speed this up?

Thanks,
Greg

2

Dvc queue logs same issue

3

Unfortunately this is a known issue, the performance for all 3 commands are affected due to how we collect/store information about the queued experiments. This is being actively worked on and will hopefully be resolved soon, but there aren’t any workarounds at the moment.

You can follow these issues for updates:

1 Like
4

Is there an older version of DVC which doesn’t have this issue? On a different project I have been using 1.16 or 1.11 (I don’t remember) which seems to be quick. Is there a 2.x version of DVC which doesn’t have the issue?

5

You could use 2.13.0 or earlier, but those releases were prior to dvc queue related commands being introduced in DVC. In 2.13.0 and earlier there was a naive implementation for dvc exp run --queue/--run-all which would allow you to run multiple experiments, but was not really a functional task queue.

(There is no dvc queue command in those releases, so dvc queue status and dvc queue logs are unavailable)

6

Okay thanks for the advise. That makes sense because I started seeing the issue when upgrading from 2.9.2 to >= 2.30