DVC experiment breaks GIT-lfs

Questions
1

I have several files stored in git-lfs and several in DVC.

I run dvc exp run command and it breaks all files tracked by git-lfs and I don’t know why.

ermolaev@df783b0a927d:~/projects/***$ git status .
On branch ***
Your branch is ahead of '***' by 2 commits.
  (use "git push" to publish your local commits)

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   ***/data/classifier_model/model.onnx
        modified:   ***/data/detector_model/model.onnx
        modified:   ***/data/knee_type_model/model.onnx
        modified:   ***/data/segmentator_model/model.onnx

no changes added to commit (use "git add" and/or "git commit -a")
ermolaev@df783b0a927d:~/projects/***$ git diff .

What’s happening? How can I get rid of this?

2

It’s hard to say without more info. Are you able to provide a minimal reproducible example? Also, can you provide the output of dvc exp run -v and dvc doctor?

3

I don’t know what happend, but problem has disappeared :grin:
Thank you for response :slight_smile:

4

I faced with this problem again. Pipeline of the commands is:

$ dvc exp run <...> --temp --name exp
$ dvc exp branch exp
$ git status
>> all git-lfs files have been changed

Looking inside what exact changes were made I found out the following comment: line-endings changed since checkout

5

I decided to try split dvc exp run command into repro & exp-save. Here is another question about it - Saving repo snapshot.

But finally I’ve gotten the same problem with the changing git-lfs endings after dvc exp save. It’s interesting that count of changes LFS files is not the same - sometimes it may be 3 files, sometimes only 1 file, but I did’t get all tracked files or none of them in any attempt.

6

Are you on Windows? Also, can I ask why you are using both git-lfs and dvc together instead of one or the other?

7

No, ubuntu 24.04

Because of legacy, generally. I need to move a lot of different bin-files under DVC and rewrite their build jobs. Actually, I’ve already started doing this because of this problem, but this problem pretty crucial when integrating DVC in any existing project.

8

I’m not able to reproduce it. Is it possible to avoid the exp commands until you migrate off of git-lfs? exp commands do git operations, which inherently act on the whole repo, to make sure you can reproduce that repo state. Other commands should not be touching the lfs files.

9

Ok, I’ll try to create demo-repo later to illustrate this problem. Not sure that will be able, but will try. Can you share then how your .gitattributes file looks like? Maybe my config is not the same as you has.

I can migrate all to DVC on this project, but it will be a problem in other legacy project. And I wanted to understand how to cope this problem before started work on it.

10

Here’s a reproduction script that doesn’t show any change to lfs files for me:

DIR=$(mktemp -d)
cd $DIR
git init
echo lfs > lfs-file
git lfs install
git lfs track lfs-file
git add .
git commit -m "track lfs-file"
dvc init
cat > dvc.yaml <<- EOM
stages:
  test:
    cmd: echo dvc
EOM
dvc exp run -n test
dvc exp branch test
git status