`dvc plots diff` of two experiments only works if they were run from the same queue

Questions
1

I assume I have a misunderstanding of how DVC is supposed to work, but this is what I observe vs. my (perhaps incorrect) expectations:

Run one experiment

dvc exp run

param is set to 0 in the params.yaml

Run another experiment

dvc exp run -S param=1

the experiments table now shows:

dvc exp show -A

...
45492cd                    10:20 AM....     
├── 6a0cb2c [mopey-mesh]   11:16 AM  ... 1
├── 7e6cef2 [glial-skis]   11:08 AM  ... 0   
...

I have defined plots:

plots:
  - ${work_path}${results_file}:
      y: filtered

And I compare the results:

dvc plots diff mopey-mesh glial-skis

image
image466×360 24.8 KB

Only one line is shown on the graph - the first experiment glial-skis appears to be “inaccessible” after running the second one. I expected to be able to plot any pair of experiments by name.

I can make this do what I want by queuing two experiments, running the queue and then doing the plots diff. But that seems to be more trouble than it should be.

Am I doing something wrong, or am I misunderstanding how comparing experiments should work?

Thanks :slight_smile:

dvc doctor:

DVC version: 3.61.0 (exe)

Platform: Python 3.12.10 on Windows-11-10.0.22631-SP0
Subprojects:

Supports:
azure (adlfs = 2024.12.0, knack = 0.12.0, azure-identity = 1.23.0),
gdrive (pydrive2 = 1.21.3),
gs (gcsfs = 2025.5.1),
hdfs (fsspec = 2025.5.1, pyarrow = 20.0.0),
http (aiohttp = 3.12.13, aiohttp-retry = 2.9.1),
https (aiohttp = 3.12.13, aiohttp-retry = 2.9.1),
oss (ossfs = 2025.5.0),
s3 (s3fs = 2025.5.1, boto3 = 1.38.27),
ssh (sshfs = 2025.2.0),
webdav (webdav4 = 0.10.0),
webdavs (webdav4 = 0.10.0),
webhdfs (fsspec = 2025.5.1)
Config:
Global: C:\Users\\AppData\Local\iterative\dvc
System: C:\ProgramData\iterative\dvc
Cache types: hardlink, symlink
Cache directory: NTFS on Z:
Caches: local
Remotes: None
Workspace directory: NTFS on Z:
Repo: dvc, git
Repo.site_cache_dir: C:\ProgramData\iterative\dvc\Cache\repo\44ee357d0fb1b1ee20d01b45cb96b2e8

2

Can you try to run it with -v , what do you see in logs?

Can you try to pass sha (hash) instead of experiment names?

1 Like
3

Thanks - it appears both revisions are being “looked at”:

dvc plots diff -vv mopey-mesh glial-skis
2025-08-13 09:40:47,123 DEBUG: v3.61.0 (exe), CPython 3.12.10 on Windows-11-10.0.22631-SP0
2025-08-13 09:40:47,123 DEBUG: command: plots diff -vv mopey-mesh glial-skis
2025-08-13 09:40:47,123 TRACE: Namespace(quiet=0, verbose=2, cprofile=False, cprofile_dump=None, yappi=False, yappi_separate_threads=False, viztracer=False, viztracer_depth=None, viztracer_async=False, pdb=False, instrument=False, instrument_open=False, show_stack=False, cd='.', wait_for_lock=False, cmd='diff', targets=None, experiment=False, revisions=['mopey-mesh', 'glial-skis'], template=None, x=None, y=None, header=None, title=None, x_label=None, y_label=None, out=None, show_vega=False, json=False, split=False, open=False, html_template=None, func=<class 'dvc.commands.plots.CmdPlotsDiff'>, parser=DvcParser(prog='dvc', usage=None, description='Data Version Control', formatter_class=<class 'dvc.cli.formatter.RawTextHelpFormatter'>, conflict_handler='error', add_help=False))
2025-08-13 09:40:47,726 TRACE: switching fs to revision 6a0cb2c
2025-08-13 09:40:47,850 TRACE: Context during resolution of stage get:
{'server_path': '<redacted>', 'source_file': '<redacted>', 'check_fcs': False, 'downselect': {'selector': 1}, 'process': {'num_points': 16000, 'filter': {'ripple_db': 0.01, 'cutoff': 0.0173, 'attenuation_db': 40, 'type': 'bessel', 'Ntaps': 64, 'window_fraction': 0.05}}, 'work_path': './work/', 'exp_name_file': 'exp_name.txt', 'raw_file': 'data.raw', 'ing_file': 'data.ing', 'pre_file': 'data.pre', 'results_file': 'results.csv', 'metrics_file': 'metrics.json'}
2025-08-13 09:40:47,850 TRACE: Context during resolution of stage ingest:
{'server_path': '<redacted>', 'source_file': '<redacted>', 'check_fcs': False, 'downselect': {'selector': 1}, 'process': {'num_points': 16000, 'filter': {'ripple_db': 0.01, 'cutoff': 0.0173, 'attenuation_db': 40, 'type': 'bessel', 'Ntaps': 64, 'window_fraction': 0.05}}, 'work_path': './work/', 'exp_name_file': 'exp_name.txt', 'raw_file': 'data.raw', 'ing_file': 'data.ing', 'pre_file': 'data.pre', 'results_file': 'results.csv', 'metrics_file': 'metrics.json'}
2025-08-13 09:40:47,864 TRACE: Context during resolution of stage preprocess:
{'server_path': '<redacted>', 'source_file': '<redacted>', 'check_fcs': False, 'downselect': {'selector': 1}, 'process': {'num_points': 16000, 'filter': {'ripple_db': 0.01, 'cutoff': 0.0173, 'attenuation_db': 40, 'type': 'bessel', 'Ntaps': 64, 'window_fraction': 0.05}}, 'work_path': './work/', 'exp_name_file': 'exp_name.txt', 'raw_file': 'data.raw', 'ing_file': 'data.ing', 'pre_file': 'data.pre', 'results_file': 'results.csv', 'metrics_file': 'metrics.json'}
2025-08-13 09:40:47,869 TRACE: Context during resolution of stage process:
{'server_path': '<redacted>', 'source_file': '<redacted>', 'check_fcs': False, 'downselect': {'selector': 1}, 'process': {'num_points': 16000, 'filter': {'ripple_db': 0.01, 'cutoff': 0.0173, 'attenuation_db': 40, 'type': 'bessel', 'Ntaps': 64, 'window_fraction': 0.05}}, 'work_path': './work/', 'exp_name_file': 'exp_name.txt', 'raw_file': 'data.raw', 'ing_file': 'data.ing', 'pre_file': 'data.pre', 'results_file': 'results.csv', 'metrics_file': 'metrics.json'}
2025-08-13 09:40:47,873 TRACE:   119.61 ms in collecting stages from /
2025-08-13 09:40:47,874 TRACE:    20.10 mks in collecting stages from /experiments
2025-08-13 09:40:47,875 TRACE:    25.60 mks in collecting stages from /expt
2025-08-13 09:40:47,875 TRACE:    19.60 mks in collecting stages from /support
2025-08-13 09:40:47,876 TRACE:    29.70 mks in collecting stages from /tests
2025-08-13 09:40:47,876 TRACE:    10.10 mks in collecting stages from /tests/data
2025-08-13 09:40:47,877 TRACE:    35.50 mks in collecting stages from /utils
2025-08-13 09:40:48,268 TRACE: switching fs to revision 7e6cef2
2025-08-13 09:40:48,334 TRACE: Context during resolution of stage get:
{'server_path': '<redacted>', 'source_file': '<redacted>', 'check_fcs': False, 'downselect': {'selector': 0}, 'process': {'num_points': 16000, 'filter': {'ripple_db': 0.01, 'cutoff': 0.0173, 'attenuation_db': 40, 'type': 'bessel', 'Ntaps': 64, 'window_fraction': 0.05}}, 'work_path': './work/', 'exp_name_file': 'exp_name.txt', 'raw_file': 'data.raw', 'ing_file': 'data.ing', 'pre_file': 'data.pre', 'results_file': 'results.csv', 'metrics_file': 'metrics.json'}
2025-08-13 09:40:48,339 TRACE: Context during resolution of stage ingest:
{'server_path': '<redacted>', 'source_file': '<redacted>', 'check_fcs': False, 'downselect': {'selector': 0}, 'process': {'num_points': 16000, 'filter': {'ripple_db': 0.01, 'cutoff': 0.0173, 'attenuation_db': 40, 'type': 'bessel', 'Ntaps': 64, 'window_fraction': 0.05}}, 'work_path': './work/', 'exp_name_file': 'exp_name.txt', 'raw_file': 'data.raw', 'ing_file': 'data.ing', 'pre_file': 'data.pre', 'results_file': 'results.csv', 'metrics_file': 'metrics.json'}
2025-08-13 09:40:48,344 TRACE: Context during resolution of stage preprocess:
{'server_path': '<redacted>', 'source_file': '<redacted>', 'check_fcs': False, 'downselect': {'selector': 0}, 'process': {'num_points': 16000, 'filter': {'ripple_db': 0.01, 'cutoff': 0.0173, 'attenuation_db': 40, 'type': 'bessel', 'Ntaps': 64, 'window_fraction': 0.05}}, 'work_path': './work/', 'exp_name_file': 'exp_name.txt', 'raw_file': 'data.raw', 'ing_file': 'data.ing', 'pre_file': 'data.pre', 'results_file': 'results.csv', 'metrics_file': 'metrics.json'}
2025-08-13 09:40:48,349 TRACE: Context during resolution of stage process:
{'server_path': '<redacted>', 'source_file': '<redacted>', 'check_fcs': False, 'downselect': {'selector': 0}, 'process': {'num_points': 16000, 'filter': {'ripple_db': 0.01, 'cutoff': 0.0173, 'attenuation_db': 40, 'type': 'bessel', 'Ntaps': 64, 'window_fraction': 0.05}}, 'work_path': './work/', 'exp_name_file': 'exp_name.txt', 'raw_file': 'data.raw', 'ing_file': 'data.ing', 'pre_file': 'data.pre', 'results_file': 'results.csv', 'metrics_file': 'metrics.json'}
2025-08-13 09:40:48,352 TRACE:    54.52 ms in collecting stages from /
2025-08-13 09:40:48,353 TRACE:    21.20 mks in collecting stages from /experiments
2025-08-13 09:40:48,354 TRACE:    21.20 mks in collecting stages from /expt
2025-08-13 09:40:48,354 TRACE:    20.10 mks in collecting stages from /support
2025-08-13 09:40:48,354 TRACE:    30.30 mks in collecting stages from /tests
2025-08-13 09:40:48,354 TRACE:    10.50 mks in collecting stages from /tests/data
2025-08-13 09:40:48,354 TRACE:    20.00 mks in collecting stages from /utils
file:///Z:/mjt/phdwork/src/dvc_plots/index.html
2025-08-13 09:40:48,425 DEBUG: Analytics is enabled.
2025-08-13 09:40:48,439 TRACE: Saving analytics report to C:\Users\<redacted>\AppData\Local\Temp\tmpbonh8vy1
2025-08-13 09:40:48,439 DEBUG: Trying to spawn ['daemon', 'analytics', 'C:\\Users\\<redacted>\\AppData\\Local\\Temp\\tmpbonh8vy1', '-vv']
2025-08-13 09:40:48,535 DEBUG: Spawned ['daemon', 'analytics', 'C:\\Users\\<redacted>\\AppData\\Local\\Temp\\tmpbonh8vy1', '-vv'] with pid 18892
2025-08-13 09:40:48,548 TRACE: Process 18388 exiting with 0

Plotting with the hashes produces much the same result, but with the single line labelled with the hash of mopey-mesh.

dvc plots diff 6a0cb2c 7e6cef2

image
image446×360 24.8 KB

Reversing the order of the hashes on the command line produces the same plot (so it is not order-dependent).

Mistyping either hash gives an error, confirming that both are being parsed and something attempted for them:

$ dvc plots diff 7e6cef0 6a0cb2c
ERROR: unknown Git revision '7e6cef0'
$ dvc plots diff 7e6cef2 6a0cb20
ERROR: unknown Git revision '6a0cb20'
4

Hmm, can’t reproduce it on the example-get-started project. It gives me a file with two plots. Would it be possible to isolate it a bit, simplify the project to keep the structure so that it is reproducible? Also, could you try the latest DVC please - 3.62, just in case …

5

We have a good debug message if we fail to read data/plots.

But I don’t see any in the logs. I wonder if this is happening because dvc failed to load the stages properly. Could you checkout to 7e6cef2 and then run dvc plots show --vv and see what happens?

6

Thanks - I tried that - it did not create any plot:

dvc plots show -vv mopey-mesh glial-skis2025-08-14 16:17:02,034 DEBUG: v3.61.0 (exe), CPython 3.12.10 on Windows-11-10.0.22631-SP02025-08-14 16:17:02,049 DEBUG: command: plots show -vv mopey-mesh glial-skis<snip>

2025-08-14 16:17:02,539 TRACE:   104.64 ms in collecting stages from <redacted>
2025-08-14 16:17:02,539 TRACE:    14.70 mks in collecting stages from <redacted>\experiments
2025-08-14 16:17:02,547 TRACE:    15.40 mks in collecting stages from <redacted>\expt
2025-08-14 16:17:02,549 TRACE:    13.10 mks in collecting stages from <redacted>\support
2025-08-14 16:17:02,550 TRACE:   685.80 mks in collecting stages from <redacted>\tests
2025-08-14 16:17:02,551 TRACE:    10.10 mks in collecting stages from <redacted>\tests\data
2025-08-14 16:17:02,551 TRACE:    20.00 mks in collecting stages from <redacted>\utils
2025-08-14 16:17:02,753 DEBUG: Analytics is disabled.2025-08-14 16:17:02,753 TRACE: Process 1160 exiting with 0

Interestingly, the stages are now being collected from a “proper” Windows path with backslashes. is of the form d:\path\path\path.

7

Good idea, thanks - I will try and replicate with the example project, maybe later tonight…

8

plots show takes file paths as arguments. I was asking log for just dvc plots show -vv.

If dvc is reading from a Git revision, it uses forward-slashes matching Git’s convention. When reading from a local filesystem, it will use whatever the OS path conventions are (which is backslash on Windows).

9

plots show takes file paths as arguments. I was asking log for just dvc plots show -vv.

Doh, sorry was confusing it with plots diff!

No plots were loaded, visualization file will not be created.
Enter text (Ctrl+D to end):
2025-08-15 09:32:51,992 DEBUG: v3.61.0 (exe), CPython 3.12.10 on Windows-11-10.0.22631-SP0
2025-08-15 09:32:51,992 DEBUG: command: plots show -vv
2025-08-15 09:32:51,993 TRACE: Namespace(quiet=0, verbose=2, cprofile=False, cprofile_dump=None, yappi=False, yappi_separate_threads=False, viztracer=False, viztracer_depth=None, viztracer_async=False, pdb=False, instrument=False, instrument_open=False, show_stack=False, cd='.', wait_for_lock=False, cmd='show', targets=[], template=None, x=None, y=None, header=None, title=None, x_label=None, y_label=None, out=None, show_vega=False, json=False, split=False, open=False, html_template=None, func=<class 'dvc.commands.plots.CmdPlotsShow'>, parser=DvcParser(prog='dvc', usage=None, description='Data Version Control', formatter_class=<class 'dvc.cli.formatter.RawTextHelpFormatter'>, conflict_handler='error', add_help=False))
2025-08-15 09:32:52,478 TRACE: switching fs to workspace
2025-08-15 09:32:52,588 TRACE: Context during resolution of stage get:
{'server_path': '[REDACTED]/', 'source_file': '[REDACTED]', 'raw_file': 'data.raw', 'ing_file': 'data.ing', 'pre_file': 'data.pre', 'results_file': 'results.csv', 'metrics_file': 'metrics.json'}
2025-08-15 09:32:52,593 TRACE: Context during resolution of stage ingest:
{'server_path': '[REDACTED]/', 'source_file': '[REDACTED]', 'raw_file': 'data.raw', 'ing_file': 'data.ing', 'pre_file': 'data.pre', 'results_file': 'results.csv', 'metrics_file': 'metrics.json'}
2025-08-15 09:32:52,596 TRACE: Context during resolution of stage preprocess:
{'server_path': '[REDACTED]/', 'source_file': '[REDACTED]', 'raw_file': 'data.raw', 'ing_file': 'data.ing', 'pre_file': 'data.pre', 'results_file': 'results.csv', 'metrics_file': 'metrics.json'}
2025-08-15 09:32:52,599 TRACE: Context during resolution of stage process:
{'server_path': '[REDACTED]/', 'source_file': '[REDACTED]', 'raw_file': 'data.raw', 'ing_file': 'data.ing', 'pre_file': 'data.pre', 'results_file': 'results.csv', 'metrics_file': 'metrics.json'}
2025-08-15 09:32:52,608 TRACE:   124.30 ms in collecting stages from [REDACTED]\src
2025-08-15 09:32:52,609 TRACE:    15.00 mks in collecting stages from [REDACTED]\src\experiments
2025-08-15 09:32:52,609 TRACE:    15.40 mks in collecting stages from [REDACTED]\src\expt
2025-08-15 09:32:52,610 TRACE:    12.70 mks in collecting stages from [REDACTED]\src\support
2025-08-15 09:32:52,611 TRACE:   882.80 mks in collecting stages from [REDACTED]\src\tests
2025-08-15 09:32:52,612 TRACE:    15.50 mks in collecting stages from [REDACTED]\src\tests\data
2025-08-15 09:32:52,613 TRACE:    32.60 mks in collecting stages from [REDACTED]\src\utils
2025-08-15 09:32:52,620 DEBUG: Analytics is disabled.
2025-08-15 09:32:52,620 TRACE: Process 15204 exiting with 0