Notebooks

Merge projects

iPython Project Created 4 days ago Free
How to merge several projects to a single one
Free Signup

Merge projects

This script will merge several projects to a single one

Input:

  • List of existing projects

Output:

  • New Project

Configuration

Edit the following settings for your own case

In [1]:
import supervisely_lib as sly
import os
from tqdm import tqdm
In [2]:
src_project_names = ['lemons_annotated', 'roads_annotated']
dst_project_name = 'merged_project'

# Context
team_name = "jupyter_tutorials"
workspace_name = "cookbook"

# Obtain server address and your api_token from environment variables
# Edit those values if you run this notebook on your own PC
address = os.environ['SERVER_ADDRESS']
token = os.environ['API_TOKEN']
In [3]:
# Initialize API object
api = sly.Api(address, token)

Verify input values

Test that context (team / workspace / project) exists

In [4]:
# Get IDs of team and workspace

team = api.team.get_info_by_name(team_name)
if team is None:
    raise RuntimeError("Team {!r} not found".format(team_name))

workspace = api.workspace.get_info_by_name(team.id, workspace_name)
if workspace is None:
    raise RuntimeError("Workspace {!r} not found".format(workspace_name))
    
print("Team: id={}, name={}".format(team.id, team.name))
print("Workspace: id={}, name={}".format(workspace.id, workspace.name))
Out [4]:
Team: id=30, name=jupyter_tutorials
Workspace: id=76, name=cookbook
In [5]:
# Check if all input projects exist
src_projects = []
for project_name in src_project_names:
    src_project = api.project.get_info_by_name(workspace.id, project_name)
    if src_project is not None:
        src_projects.append(src_project)
    else:
        raise RuntimeError("Project {!r} not found".format(project_name))
In [6]:
# Check if destination project name already exists. If yes - generate new free name
if api.project.exists(workspace.id, dst_project_name):
    dst_project_name = api.project.get_free_name(workspace.id, dst_project_name)
    
# create remote project
dst_project = api.project.create(workspace.id, dst_project_name)
print("Project: id={}, name={!r}".format(dst_project.id, dst_project.name))
Out [6]:
Project: id=1143, name='merged_project'

Create output Project

Generate ProjectMeta for new Project by merging all source ProjectMeta's

In [7]:
destination_meta = sly.ProjectMeta()
for project in src_projects:
    src_meta_json = api.project.get_meta(project.id)
    src_meta = sly.ProjectMeta.from_json(src_meta_json)
    destination_meta = destination_meta.merge(src_meta)

api.project.update_meta(dst_project.id, destination_meta.to_json())

Create output datasets and add images

In [8]:
# add datasets, images and annotations to destination project
for project in src_projects:
    for dataset in api.dataset.get_list(project.id):
        
        # generate dataset name in destination porject if it exists
        dst_dataset_name = dataset.name
        if api.dataset.exists(dst_project.id, dst_dataset_name):
            dst_dataset_name = api.project.get_free_name(workspace.id, dst_project_name)
        
        # create new dataset in destination project
        dst_dataset = api.dataset.create(dst_project.id, dst_dataset_name)
        
        print("Processing: project = {!r}, dataset = {!r}".format(project.name, dataset.name))
        
        # add images and annotations from source dataset to destination dataset
        for image in tqdm(api.image.get_list(dataset.id)):
            # add image to destination dataset 
            dst_image = api.image.add(dst_dataset.id, image.name, image.hash)
            
            # get image annotation
            ann_info = api.annotation.download(image.id)
            ann_json = ann_info.annotation
            
            # upload annotation to destination image
            api.annotation.upload(dst_image.id, ann_json)
Out [8]:
 17%|█▋        | 1/6 [00:00<00:00,  7.14it/s]
Processing: project = 'lemons_annotated', dataset = 'ds1'
100%|██████████| 6/6 [00:00<00:00,  7.45it/s]
 10%|█         | 1/10 [00:00<00:01,  7.72it/s]
Processing: project = 'roads_annotated', dataset = 'ds1'
100%|██████████| 10/10 [00:01<00:00,  7.64it/s]
In [9]:
print("Project {!r} has been sucessfully uploaded".format(dst_project.name))
print("Number of uploaded images: ", api.project.get_images_count(dst_project.id))
Out [9]:
Project 'merged_project' has been sucessfully uploaded
Number of uploaded images:  16

More Info

ID
27
First released
4 days ago
Last updated
3 hours ago

Owner

s