Automatically tag images after uploading

Getting editors to properly tag the images they upload can be quite a challenge, if you need/want them tagged of course.

The Computer Vision API can be of assistance. It only supports English and Simplified Chinese at the moment though, as you can read in the API reference. You could of course use the translation API to translate the analysis results.

To make this work you will need to add the Project Oxford Vision NuGet package to your project.

First you need to add some extra properties to your image content type e.g.

[UIHint(UIHint.LongString)]
public virtual string Caption { get; set; }

[BackingType(typeof(PropertyStringList))]
public virtual string[] ImageCategories { get; set; }

public virtual bool IsAdultContent { get; set; }

public virtual bool IsBwImg { get; set; }

public virtual bool IsRacyContent { get; set; }

[BackingType(typeof(PropertyStringList))]
public virtual string[] Tags { get; set; }

Next you will need an “imagehelper” where you process the image in Azure and set the properties. Do it Async, so the editor won’t be kept waiting.

public async Task TagImagesAsync(ImageFile imageFile)
{
	AnalysisResult analysisResult = await this.UploadAndAnalyzeImageAsync(imageFile).ConfigureAwait(false);

	this.ProcessAnalysisResult(analysisResult, imageFile);
}
private async Task<AnalysisResult> UploadAndAnalyzeImageAsync(ImageData imageData)
{
    VisionServiceClient visionServiceClient = new VisionServiceClient(this.subscriptionKey);

    VisualFeature[] visualFeatures =
	{
	    VisualFeature.Adult, VisualFeature.Categories, VisualFeature.Color,
	    VisualFeature.Description, VisualFeature.ImageType,
	    VisualFeature.Tags
	};

	return
		await visionServiceClient.AnalyzeImageAsync(imageData.BinaryData.OpenRead(), visualFeatures).ConfigureAwait(false);

}
private void ProcessAnalysisResult(AnalysisResult result, ImageFile imageFile)
{
    if (result == null)
    {
	return;
    }

    ImageFile clone = imageFile.CreateWritableClone() as ImageFile;

    if (clone == null)
    {
	return;
    }

    if (result.Adult != null)
    {
	clone.IsAdultContent = result.Adult.IsAdultContent;
	clone.IsRacyContent = result.Adult.IsRacyContent;
    }

    if (result.Categories != null && result.Categories.Length > 0)
    {
	clone.ImageCategories = result.Categories.Select(c => c.Name).ToArray();
    }

    if ((clone.Tags == null || clone.Tags.Length == 0) && result.Tags != null)
    {
	clone.Tags = result.Tags.Where(t => t.Confidence > 0.5).Select(t => t.Name).ToArray();
    }

    if (result.Description != null)
    {
	clone.Caption = result.Description.Captions.OrderByDescending(c => c.Confidence).FirstOrDefault()?.Text;

	if (clone.Tags == null || clone.Tags.Length == 0)
	{
	    clone.Tags = result.Description.Tags;
	}
    }

this.contentRepository.Save(
	clone,
	SaveAction.Publish | SaveAction.ForceCurrentVersion,
	AccessLevel.NoAccess);
}

Tie it all together in an initialization module

private static void OnPublishedContent(object sender, ContentEventArgs contentEventArgs)
{
	if (contentEventArgs == null)
	{
		return;
	}

	ImageFile contentData = contentEventArgs.Content as ImageFile;

	if (contentData == null)
	{
		return;
	}

	ImageHelper imageHelper = new ImageHelper();

	#pragma warning disable 4014
	imageHelper.TagImagesAsync(contentData);
	#pragma warning restore 4014
}

The complete code can be found on gist

Or you can use the NuGet package created by Allan.

4 thoughts on “Automatically tag images after uploading

  1. I’m curious – why during the OnPublished event? Can authors override these values later? I’m not sure I tried calling my Async method without an await except as part of the event handler itself, so that may be the trick.

    Like

    1. As I’m calling the method async, without awaiting the result, the image needed to be published before being able to add the tags from the result. And I tried a lot of options, waiting for the result to come back, but the “image uploader” always gave the message that it failed, though it didn’t. Which I found quite confusing 🙂

      Like

      1. I found it confusing as well, though I don’t get that error with my version of this solution any more. Using the Saving event also allows me to update the fields without forcing a new version of the content.

        Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s