Notebooks

Split project by tags

iPython Project Created 3 months ago Free
Notebook creates new project only with images that contain given tags
Free Signup

Filter project by tags

Notebook creates new project with only the images from the source project that contain given tags

Input:

  • Source project
  • Set of tags

Output:

  • New project with images that contain given tags

Configuration

Edit the following settings for your own case

In [1]:
import supervisely_lib as sly
import os
from tqdm import tqdm
In [2]:
team_name = "jupyter_tutorials"
workspace_name = "cookbook"
project_name = "lemons_annotated_01"

dst_project_name = "lemons_annotated_01_filtered"

filter_tag_names = {'train', 'tag_name1', 'tag_name2'}

# Obtain server address and your api_token from environment variables
# Edit those values if you run this notebook on your own PC
address = os.environ['SERVER_ADDRESS']
token = os.environ['API_TOKEN']
In [3]:
# Initialize API object
api = sly.Api(address, token)

Verify input values

Test that context (team / workspace / project) exists

In [4]:
# Get IDs of team, workspace and project by names

team = api.team.get_info_by_name(team_name)
if team is None:
    raise RuntimeError("Team {!r} not found".format(team_name))

workspace = api.workspace.get_info_by_name(team.id, workspace_name)
if workspace is None:
    raise RuntimeError("Workspace {!r} not found".format(workspace_name))
    
project = api.project.get_info_by_name(workspace.id, project_name)
if project is None:
    raise RuntimeError("Project {!r} not found".format(project_name))
    
print("Team: id={}, name={}".format(team.id, team.name))
print("Workspace: id={}, name={}".format(workspace.id, workspace.name))
print("Project: id={}, name={}".format(project.id, project.name))
Out [4]:
Team: id=30, name=jupyter_tutorials
Workspace: id=76, name=cookbook
Project: id=1294, name=lemons_annotated_01

Get Source ProjectMeta

In [5]:
project = api.project.get_info_by_name(workspace.id, project_name)
meta_json = api.project.get_meta(project.id)
meta = sly.ProjectMeta.from_json(meta_json)
print("Source ProjectMeta: \n", meta)
Out [5]:
Source ProjectMeta: 
 ProjectMeta:
Object Classes
+-------+--------+----------------+
|  Name | Shape  |     Color      |
+-------+--------+----------------+
|  kiwi | Bitmap |  [255, 0, 0]   |
| lemon | Bitmap | [81, 198, 170] |
+-------+--------+----------------+
Image Tags
+-------+------------+-----------------+
|  Name | Value type | Possible values |
+-------+------------+-----------------+
| train |    none    |       None      |
|  val  |    none    |       None      |
+-------+------------+-----------------+
Object Tags
+------+------------+-----------------+
| Name | Value type | Possible values |
+------+------------+-----------------+
+------+------------+-----------------+

Construct Destination ProjectMeta

In [6]:
dst_meta = meta
print("Destination ProjectMeta:\n", dst_meta)
Out [6]:
Destination ProjectMeta:
 ProjectMeta:
Object Classes
+-------+--------+----------------+
|  Name | Shape  |     Color      |
+-------+--------+----------------+
|  kiwi | Bitmap |  [255, 0, 0]   |
| lemon | Bitmap | [81, 198, 170] |
+-------+--------+----------------+
Image Tags
+-------+------------+-----------------+
|  Name | Value type | Possible values |
+-------+------------+-----------------+
| train |    none    |       None      |
|  val  |    none    |       None      |
+-------+------------+-----------------+
Object Tags
+------+------------+-----------------+
| Name | Value type | Possible values |
+------+------------+-----------------+
+------+------------+-----------------+

Create Destination project

In [7]:
# check if destination project already exists. If yes - generate new free name
if api.project.exists(workspace.id, dst_project_name):
    dst_project_name = api.project.get_free_name(workspace.id, dst_project_name)
print("Destination project name: ", dst_project_name)
Out [7]:
Destination project name:  lemons_annotated_01_filtered
In [8]:
dst_project = api.project.create(workspace.id, dst_project_name)
api.project.update_meta(dst_project.id, dst_meta.to_json())
print("Destination project has been created: id={}, name={!r}".format(dst_project.id, dst_project.name))
Out [8]:
Destination project has been created: id=1326, name='lemons_annotated_01_filtered'

Iterate over all images, filter them by tags and add to destination project

In [9]:
num_filtered_images = 0
for dataset in api.dataset.get_list(project.id):
    print('Dataset: {}'.format(dataset.name), flush=True)
    dst_dataset = api.dataset.create(dst_project.id, dataset.name)

    images = api.image.get_list(dataset.id)
    with tqdm(total=len(images), desc="Process annotations") as progress_bar:
        for batch in sly.batched(images):
            image_ids = [image_info.id for image_info in batch]
            image_names = [image_info.name for image_info in batch]
            
            ann_infos = api.annotation.download_batch(dataset.id, image_ids)
            
            image_names_after_filter = []
            image_ids_after_filter = []
            anns_after_filter = []
            for img_name, img_id, ann_info in zip(image_names, image_ids, ann_infos):
                ann_json = ann_info.annotation
                ann = sly.Annotation.from_json(ann_json, meta)
                
                if any(ann.img_tags.has_key(tag_name) for tag_name in filter_tag_names):
                    image_names_after_filter.append(img_name)
                    image_ids_after_filter.append(img_id)
                    anns_after_filter.append(ann)

            dst_image_infos = api.image.upload_ids(dst_dataset.id, image_names_after_filter, image_ids_after_filter)
            dst_image_ids = [image_info.id for image_info in dst_image_infos]
            api.annotation.upload_anns(dst_image_ids, anns_after_filter)
            
            num_filtered_images += len(image_names_after_filter)
            progress_bar.update(len(batch))
Out [9]:
Dataset: ds1
Process annotations: 100%|██████████| 6/6 [00:00<00:00, 28.80it/s]
In [10]:
print("Project {!r} has been sucessfully uploaded".format(dst_project.name))
print("Project contains {} images".format(api.project.get_images_count(dst_project.id)))
print("Number of filtered images:", num_filtered_images)
Out [10]:
Project 'lemons_annotated_01_filtered' has been sucessfully uploaded
Project contains 4 images
Number of filtered images: 4

More Info

ID
89
First released
3 months ago
Last updated
A month ago

Owner

s