Using Multi-Label Classifiers as a Single-Label Classifier

fastai
pytorch
python
Author

Sean Dokko

Published

December 28, 2022

In this post, we’ll create a simple multi-label classifier. Much of the code is a composition of a few examples I’ve seen online and through the Fastbook reading material.

This exercise is from the further reading section of chapter 6 of the Deep Learning for Coders with fastai & PyTorch book.

The purpose of this exercise was to convert an existing imlementation of a single-label classifier into a multi-label classifier. As far as functionality goes, they are identical, since the training data has a single label anyways (this is purely for the sake of experimentation). We’ll implement both, to see if there is a difference in results.

from fastai.vision.all import *
from duckduckgo_search import ddg_images
from fastcore.all import *

def search_images(term, max_images=100):
    print(f"Searching for '{term}'")
    return L(ddg_images(term, max_results=max_images)).itemgot('image')
urls = search_images('polar bear')
Searching for 'polar bear'
from fastdownload import download_url
dest = 'polar_bear.jpg'
download_url(urls[0], dest, show_progress=False)

from fastai.vision.all import *
im = Image.open(dest)
im.to_thumb(256,256)

path = Path('images')
searches = 'polar bear','grizzly bear','black bear', 'arctic wolf'
path = Path('images')
from time import sleep

for o in searches:
    dest = (path/o)
    dest.mkdir(exist_ok=True, parents=True)
    download_images(dest, urls=search_images(o))
    sleep(10)
    resize_images(path/o, max_size=400, dest=path/o)
Searching for 'polar bear'
Searching for 'grizzly bear'
Searching for 'black bear'
Searching for 'arctic wolf'
failed = verify_images(get_image_files(path))
failed.map(Path.unlink)
len(failed)
8

Originally, for the single-label classifier, the DataBlock construction looks like this:

dls = DataBlock(
    blocks=(ImageBlock, CategoryBlock), 
    get_items=get_image_files, 
    splitter=RandomSplitter(),
    get_y=parent_label,
    item_tfms=[Resize(192, method='squish')]
).dataloaders(path, bs=32)

dls.show_batch(max_n=12)

learn = vision_learner(dls, resnet18, metrics=[accuracy])
learn.fine_tune(3)
epoch train_loss valid_loss accuracy time
0 0.999492 0.113413 0.950000 00:36
epoch train_loss valid_loss accuracy time
0 0.114955 0.027196 0.991667 00:54
1 0.065354 0.024253 0.991667 00:52
2 0.050956 0.020718 0.991667 00:55

Let us construct the multilabel DataBlock. The key difference here is the use of MultiCategoryBlock and the custom get_y function that wraps the label.

def get_y(r): 
    label = parent_label(r)
    return [label]

dls = DataBlock(
    blocks=(ImageBlock, MultiCategoryBlock), 
    get_items=get_image_files, 
    splitter=RandomSplitter(),
    get_y=get_y,
    item_tfms=[Resize(192, method='squish')]
).dataloaders(path, bs=32)

dls.show_batch(max_n=12)

Let’s ensure that the right labels are being populated

dls.vocab
['arctic wolf', 'black bear', 'grizzly bear', 'polar bear']

In addition, for the metrics parameter, it is important to use the accuracy_multi helper function.

learn = vision_learner(dls, resnet18, metrics=[accuracy_multi])
learn.fine_tune(3)
epoch train_loss valid_loss accuracy_multi time
0 0.692847 0.221365 0.922917 00:43
epoch train_loss valid_loss accuracy_multi time
0 0.215801 0.128911 0.954167 01:01
1 0.145919 0.096342 0.985417 00:58
2 0.101925 0.086828 0.985417 01:02

Conclusively

After training both models with identical hyperparameters, we have the accuracy of both leaners.

Single Category Classifier: 0.991667

Multi Category Classifier: 0.985417

The multi category classifier, for its added complexity, works quite well, with the delta between the two results being 0.00625 in this example. Pretty small!

A potential use case for using a multi-category classifier right off the bat might be to justify the ease of adoption for the introduction of multi-category labels in the future, since refactoring might become become more challenging during the later stages of development.