Error with metrics show -a

Bug Reports
#1

I have been working on a small ML project for a week now. But when I tried to get all the metrics results of the different branches I created with the command “dvc metrics show -a” it showed an error message:
“ERROR: unexpected error - ‘a8fc2b321054a00b995f235770ae077828ab3b5a’”

This error message appeared the first time, two days ago, before that it worked perfectly fine. Also, it is still able to show the metrics of the current branch, when I don’t use “-a”.

I already checked if the error code is a hash from git or dvc, but I couldn’t find a commit with the same code. I also tried going back to a commit where it used to be working before, but it still gives this error message and doesn’t show the metrics.

I suspect that I might have changed something in one of the dvc files which I shouldn’t have changed. But I have no idea how I could find out what exactly is causing this error message. Does anyone have a suggestion on how to proceed?

(I am using DVC version 0.61.1)

#2

Hi @jmvanham !

Could you please post full log of dvc metrics show -a -v ?

#3 
ERROR: unexpected error - 'a8fc2b321054a00b995f235770ae077828ab3b5a'
------------------------------------------------------------
Traceback (most recent call last):
  File "/home/da-guest/mnist2/env/lib/python3.6/site-packages/dvc/main.py", line 42, in main
    ret = cmd.run()
  File "/home/da-guest/mnist2/env/lib/python3.6/site-packages/dvc/command/metrics.py", line 52, in run
    recursive=self.args.recursive,
  File "/home/da-guest/mnist2/env/lib/python3.6/site-packages/dvc/repo/metrics/__init__.py", line 18, in show
    return show(self.repo, *args, **kwargs)
  File "/home/da-guest/mnist2/env/lib/python3.6/site-packages/dvc/repo/__init__.py", line 33, in wrapper
    ret = f(repo, *args, **kwargs)
  File "/home/da-guest/mnist2/env/lib/python3.6/site-packages/dvc/repo/metrics/show.py", line 283, in show
    repo, target, recursive, typ, xpath, branch
  File "/home/da-guest/mnist2/env/lib/python3.6/site-packages/dvc/repo/metrics/show.py", line 174, in _collect_metrics
    outs = [out for stage in repo.stages for out in stage.outs]
  File "/home/da-guest/mnist2/env/lib/python3.6/site-packages/funcy/objects.py", line 28, in __get__
    res = instance.__dict__[self.fget.__name__] = self.fget(instance)
  File "/home/da-guest/mnist2/env/lib/python3.6/site-packages/dvc/repo/__init__.py", line 421, in stages
    return get_stages(self.graph)
  File "/home/da-guest/mnist2/env/lib/python3.6/site-packages/funcy/objects.py", line 28, in __get__
    res = instance.__dict__[self.fget.__name__] = self.fget(instance)
  File "/home/da-guest/mnist2/env/lib/python3.6/site-packages/dvc/repo/__init__.py", line 370, in graph
    return self._collect_graph()
  File "/home/da-guest/mnist2/env/lib/python3.6/site-packages/dvc/repo/__init__.py", line 321, in _collect_graph
    stages = stages or self.collect_stages()
  File "/home/da-guest/mnist2/env/lib/python3.6/site-packages/dvc/repo/__init__.py", line 403, in collect_stages
    self.root_dir, dvcignore=self.dvcignore
  File "/home/da-guest/mnist2/env/lib/python3.6/site-packages/dvc/scm/git/tree.py", line 150, in walk
    for x in self._walk(tree, topdown):
  File "/home/da-guest/mnist2/env/lib/python3.6/site-packages/dvc/scm/git/tree.py", line 122, in _walk
    for i in _iter_tree(tree):
  File "/home/da-guest/mnist2/env/lib/python3.6/site-packages/dvc/scm/git/tree.py", line 27, in _iter_tree
    node = submodules[node.hexsha]
KeyError: 'a8fc2b321054a00b995f235770ae077828ab3b5a'
------------------------------------------------------------
#4

@jmvanham Thanks! Are you using git submodules in your repo, btw?

#5

I didn’t use git submodules in this example

#6

@jmvanham And how did you install dvc? From pip? If so, could you show pip freeze | grep gitpython please?

#7

This gives:
GitPython==3.0.2

#8

Thanks! Hm, I have the same one installed. And how about pip freeze | grep gitdb?

#9

One more question, maybe you’ve cloned this repo as a shallow clone? Or is there anything unusual about your repo instance or how you’ve cloned/created it?

#10

pip freeze | grep gitdb gives gitdb2==2.0.6

I created this repository locally and didn’t move or push it

#11

@jmvanham Thanks! So the piece of code that is raising that exception is https://github.com/iterative/dvc/blob/0.61.2/dvc/scm/git/tree.py#L27 , which is a workaround for gitpython bug. Are you familiar with pdb? If so, could you attach and take a look at tree.repo.submodules? It seems like you do have submodules for some reason somewhere in repo. Maybe you’ve cloned something into your current repo?

#12

@kupruser, you were right about the submodule. I cloned a directory in my directory last, but I didn’t track it with git. So I didn’t recognize it as a submodule. I just removed this directory, and now it is working again! Thanks for your help

1 Like
#13

Good news! But still, it seems like we should be able to work with that. Could you please create an issue for that in https://github.com/iterative/dvc/issues ? We will investigate. :slight_smile: