I’m trying to use dvc in a github codespace and gdrive remote storage. However once I try to “dvc push”
I get this error
1 Like
Hi @mouadAqsone ! I haven’t looked in depth, but looks like the authorization method we use for GDrive remotes (GitHub - iterative/PyDrive2: Google Drive API Python wrapper library. Maintained fork of PyDrive.) doesn’t work inside a GitHub codespace.
As a workaround, I would suggest that you run a first-time dvc push on your local machine. This would create a credentials files in that local machine (How to Setup a Google Drive DVC Remote) and you could use Codespaces secrets (Managing encrypted secrets for your codespaces - GitHub Docs) to expose the contents of that file as an environment variable (GDRIVE_CREDENTIALS_DATA) .
I think that would make dvc push work inside the codespace
2 Likes
Hello @daavoo, I’ve finally came back to this as it’s important for our company to work on codespaces. I have made the codespaces secrets with the full content of the credentials files (default.json).
But a dvc pull still just creates a new port and tries to open a new window from which I have to agree to access for the gdrive ( and when I do, i get nothing )
this is my error message
/usr/local/python/3.10.4/lib/python3.10/site-packages/oauth2client/_helpers.py:255: UserWarning: Cannot access /home/codespace/.cache/pydrive2fs/710796635688-iivsgbgsb6uv1fap6635dhvuei09o66c.apps.googleusercontent.com/default.json: No such file or directory
warnings.warn(_MISSING_FILE_MESSAGE.format(filename))
Your browser has been opened to visit:
…
I’m not very familiar with these types of things, but the logical thing that came to my mind is just copy the default.json file into this folder. Is there a better way than that with codespace Secrets ?
I know it’s super confusing with GDrive, but there are two different credentials files:
-
The one you downloaded from the Google Cloud console (this step is optional though, and I’m not sure if you did it or not).
-
The one that is generated during the Authorization process.
You need to get the second one and put it either as an env variable (should be possible to set an env variable in codespaces secrets?) or copy in some location and set up the gdrive_user_credentials_file path. Could you please make sure that you are indeed using the second file? What is the structure of the file (field names, etc) that you are copying?
Btw, I definitely recommend taking a look into Dev Containers, and specifically DVC feature GitHub - iterative/features: A collection of development container 'features' . It simplifies the Codespaces setup with DVC. You can find a bit more sophisticated example here- hackathon/.devcontainer.json at main · shcheklein/hackathon · GitHub
Hi @shcheklein following up from discord. Thanks for the suggestions you’ve already offered. I’m stuck on finding my auth file (default.json) on my local machine so that I can copy it into the correct location on a remote machine. I have both a Windows and a Mac and I’ve looked in the places specified here: Google Drive But so far have been unable to find it. For example, is there a command to ask dvc to return the auth file location?
On macos it should be in ~/Library/Caches , check if there is a directory called dvc there.
Unfortunately there is no command to ask dvc for it 
1 Like
Yes that is specified in the manual, and I’ve looked there. I’ve even done a global search for default.json.
Is there perhaps a way to trigger reauth on a local machine and specify where the auth file should be located?
Yes, you could set the gdrive_user_credentials_file, see the description here Google Drive Remote Config Options please.
Btw, is there a chance that you are using service account to do auth? Also, what version of DVC do you use (could you share the dvc version).
Problem solved. Was horrible to upgrade on Windows given $PATH crap but have finally managed to generate an auth token and scp to remote machine. Thanks for your help!
2 Likes