Candid Code - Using Multi-Label Classifiers as a Single-Label Classifier

In this post, we’ll create a simple multi-label classifier. Much of the code is a composition of a few examples I’ve seen online and through the Fastbook reading material.

This exercise is from the further reading section of chapter 6 of the Deep Learning for Coders with fastai & PyTorch book.

The purpose of this exercise was to convert an existing imlementation of a single-label classifier into a multi-label classifier. As far as functionality goes, they are identical, since the training data has a single label anyways (this is purely for the sake of experimentation). We’ll implement both, to see if there is a difference in results.

from fastai.vision.all import *

from duckduckgo_search import ddg_images
from fastcore.all import *

def search_images(term, max_images=100):
    print(f"Searching for '{term}'")
    return L(ddg_images(term, max_results=max_images)).itemgot('image')

urls = search_images('polar bear')

Searching for 'polar bear'

from fastdownload import download_url
dest = 'polar_bear.jpg'
download_url(urls[0], dest, show_progress=False)

from fastai.vision.all import *
im = Image.open(dest)
im.to_thumb(256,256)

path = Path('images')

searches = 'polar bear','grizzly bear','black bear', 'arctic wolf'
path = Path('images')
from time import sleep

for o in searches:
    dest = (path/o)
    dest.mkdir(exist_ok=True, parents=True)
    download_images(dest, urls=search_images(o))
    sleep(10)
    resize_images(path/o, max_size=400, dest=path/o)

Searching for 'polar bear'
Searching for 'grizzly bear'
Searching for 'black bear'
Searching for 'arctic wolf'

failed = verify_images(get_image_files(path))
failed.map(Path.unlink)
len(failed)

Originally, for the single-label classifier, the DataBlock construction looks like this:

dls = DataBlock(
    blocks=(ImageBlock, CategoryBlock), 
    get_items=get_image_files, 
    splitter=RandomSplitter(),
    get_y=parent_label,
    item_tfms=[Resize(192, method='squish')]
).dataloaders(path, bs=32)

dls.show_batch(max_n=12)

learn = vision_learner(dls, resnet18, metrics=[accuracy])
learn.fine_tune(3)

epoch	train_loss	valid_loss	accuracy	time
0	0.999492	0.113413	0.950000	00:36

epoch	train_loss	valid_loss	accuracy	time
0	0.114955	0.027196	0.991667	00:54
1	0.065354	0.024253	0.991667	00:52
2	0.050956	0.020718	0.991667	00:55

Let us construct the multilabel DataBlock. The key difference here is the use of MultiCategoryBlock and the custom get_y function that wraps the label.

def get_y(r): 
    label = parent_label(r)
    return [label]

dls = DataBlock(
    blocks=(ImageBlock, MultiCategoryBlock), 
    get_items=get_image_files, 
    splitter=RandomSplitter(),
    get_y=get_y,
    item_tfms=[Resize(192, method='squish')]
).dataloaders(path, bs=32)

dls.show_batch(max_n=12)

Let’s ensure that the right labels are being populated

dls.vocab

['arctic wolf', 'black bear', 'grizzly bear', 'polar bear']

In addition, for the metrics parameter, it is important to use the accuracy_multi helper function.

learn = vision_learner(dls, resnet18, metrics=[accuracy_multi])
learn.fine_tune(3)

epoch	train_loss	valid_loss	accuracy_multi	time
0	0.692847	0.221365	0.922917	00:43

epoch	train_loss	valid_loss	accuracy_multi	time
0	0.215801	0.128911	0.954167	01:01
1	0.145919	0.096342	0.985417	00:58
2	0.101925	0.086828	0.985417	01:02

Conclusively

After training both models with identical hyperparameters, we have the accuracy of both leaners.

Single Category Classifier: 0.991667

Multi Category Classifier: 0.985417

The multi category classifier, for its added complexity, works quite well, with the delta between the two results being 0.00625 in this example. Pretty small!

A potential use case for using a multi-category classifier right off the bat might be to justify the ease of adoption for the introduction of multi-category labels in the future, since refactoring might become become more challenging during the later stages of development.