Create and run multiple experiments from python

Questions
1

Hi,
I see that the python API is read-only.
how can I have python iterate over parameters and run an experiment for each,
i.e. doing gridsearch or something similar?

os.system(“dvc exp run -n exp_name”) doesn’t seem very ‘integrated’ and it has generated issues with git index lock and dvc locks.
thanks!

2

I would like to see a feature like this. I am doing what you described. You can use the Repo class from dvc.repo to check for experiment names and stuff like that. Hope this helps

def run_experiment(name, params):
    command = f"dvc exp run --queue -n {name}"
    for key, val in params.items():
        command += f" -S {key}={val}"
    result = subprocess.run(command, capture_output=True, shell=True)
    print(result)


def main():
    # dvc repo, determine current branch
    repo = Repo(".")
    branch = repo.scm.active_branch()
    # get experiments on this commit, including queue
    taken_names = repo.experiments.ls()[branch]
    taken_names += [a.name for a in repo.experiments.celery_queue.iter_queued()]
    experiment_params = get_random_search_params()  # list of dictionaries
    print(f"Setting up {len(experiment_params)} experiments...")
    for params in experiment_params:
        # get new name
        for i in range(9999):
            name = f"exp_{i}"
            if name not in taken_names:
                taken_names.append(name)
                break
        # run experiment
        print(name, params)
        run_experiment(name, params)
1 Like
3

@marc_wagner: Hi! Indeed, the Python API has no convenient way to do that, but I guess using the Hydra integration on the CLI would solve your issue. Have a look at our blog for a quick intro, or the full docs.