Storing Files in Azure Cloud Storage through the Sitecore Media Library – Part 3 – Attach Media Handler

This wasn’t supposed to be a blog post series, and my original reason for this code was to just offload the hosting of large files. But whilst writing the first draft of the blog post I started to investigate various areas since the potential for this module became greater. It was supposed to stop at part two, but I supposed I suffer from the same thing as most of you: programmers OCD. And so a series is born and expect another post or two to follow until a final solution is attained.

In my previous code I noted that the code did not work with the attach/detach handlers in the Media Library:

Media-Attach-Handler

As after a bit of investigation I managed to track down the pipeline I needed to tap into.

Detach

It turns out that for “Detach” there was nothing to do. For file based media Sitecore does not do anything in a standard installation, so we will ignore it to. For media stored in the database, it correctly removes the reference though.

Attach

For “Attach” event, the code calls processors/attachFile pipeline. The code in this pipeline largely follows the same pattern as the uiUpload pipeline, albeit they use different args Classes as parameters.

It was decided to clean the code up a little (but still there is further to go) and make use of a Helper class to remove repetition of code and create a custom args object to allow the custom pipelines from multiple other pipelines.

Here are the specific changes to handle the Attach event. We’ll delete the previous file from Blob storage and then upload the replacement.

a. Configuration

<processors>
  <attachFile>
    <processor mode="on" type="Sitecore.Custom.Pipelines.AttachFile.DeletePreviousMedia, Sitecore.Custom"
                patch:before="*[@type='Sitecore.Pipelines.Attach.UpdatePath,Sitecore.Kernel']" />
    <processor mode="on" type="Sitecore.Custom.Pipelines.AttachFile.ProcessMedia, Sitecore.Custom" />
  </attachFile>
</processors>

b. Custom Args

using System.Collections.Generic;
using Sitecore.Data.Items;
using Sitecore.Pipelines;

namespace Sitecore.Custom.Pipelines.MediaProcessor
{
    public class MediaProcessorArgs : PipelineArgs
    {
        public IEnumerable<Item> UploadedItems { get; set; }
    }
}

c. Delete Processor

Before the new file is processed and the File Path field updated with the new local path, let’s do some clean up and delete the existing file from Azure Blob storage

using Sitecore.Services.Interfaces;
using Sitecore.Services.Media;
using Sitecore.Diagnostics;
using Sitecore.Pipelines.Attach;
using Sitecore.StringExtensions;

namespace Sitecore.Custom.Pipelines.AttachFile
{
    public class DeletePreviousMedia
    {
        /// <summary>Deletes media from Cloud storage that was previously associated with item</summary>
        public void Process(AttachArgs args)
        {
            Assert.ArgumentNotNull(args, "args");

            if (!args.MediaItem.FileBased)
                return;

            Log.Info("Deleting '{0}' from Cloud storage".FormatWith(args.MediaItem.FilePath), this);

            ICloudStorage storage = new AzureStorage();
            storage.Delete(args.MediaItem.FilePath);
        }
    }
}

d. ProcessMedia Processor

This is the main place that refactoring of the original code has taken place, since it essentially calls the exact same code but the pipeline args are of a different Type. To accommodate, we have a new PipelineHelper which deals with handling the upload and the logic itself has not changed.

using Sitcore.Custom.Helpers;
using Sitecore.Data.Items;
using Sitecore.Diagnostics;
using Sitecore.Pipelines.Attach;
using System.Collections.Generic;

namespace Sitecore.Custom.Pipelines.AttachFile
{
    public class ProcessMedia
    {
        public void Process(AttachArgs args)
        {
            Assert.ArgumentNotNull(args, "args");

            if (!args.MediaItem.FileBased)
                return;

            var helper = new PipelineHelper();
            helper.StartMediaProcessorJob(new List<Item> { args.MediaItem });
        }
    }
}
using Sitecore.Custom.Pipelines.MediaProcessor;
using Sitecore.Data.Items;
using Sitecore.Pipelines;
using System.Collections.Generic;

namespace Sitecore.Custom.Helpers
{
    public class PipelineHelper
    {
        /// <summary>Creates and starts a Sitecore Job to run as a long running background task</summary>
        /// <param name="args">The UploadArgs</param>
        public void StartMediaProcessorJob(IEnumerable<Item> uploadedItems)
        {
            var args = new MediaProcessorArgs { UploadedItems = uploadedItems };
            var jobOptions = new Sitecore.Jobs.JobOptions("CustomMediaProcessor", "MediaProcessing",
                                                          Sitecore.Context.Site.Name,
                                                          this, "RunMediaProcessor", new object[] { args });
            Sitecore.Jobs.JobManager.Start(jobOptions);
        }

        /// <summary>Calls Custom Pipeline with the supplied args</summary>
        /// <param name="args">The UploadArgs</param>
        public void RunMediaProcessor(MediaProcessorArgs args)
        {
            CorePipeline.Run("custom.MediaProcessor", args);
        }
    }
}

Code

The updated code can be found in this Githib repository: https://github.com/jammykam/Sitecore-CloudMediaLibrary

Incidently, if you were at the Sitecore London Technical User Group (January 2016) then above is the code from my demonstration.

Beware that this code will be changing, I already have some ideas in mind, some which I mentioned before and a few more listed below. You may want to keep an eye on the Github repository and/or follow the blog.

TODO:

Expect some follow up posts, most likely concentrating on the following areas. Maybe more. Send me a message/leave a comment if you have suggestions.

  • Versioned Media
  • Workflow
  • Alternate Providers (at least, refactor the code to allow the provider to be easily switched out)

4 comments

  1. Adam Weber · April 15, 2016

    Great series of posts! I wonder if you’d find any value in this approach I developed a few years ago: http://www.sitecore.net/learn/blogs/technical-blogs/sitecorebetter/posts/2013/07/sitecore-media-library-and-azure-blob-storage.aspx

    Instead of extending processors and pipelines, you go a step deeper and extend the storage provider. In theory you could still incorporate your size restrictions, i.e. only uploading larger files to Azure while retaining smaller files in the database.

    I would be curious to know if this type of approach would help fulfill the requirements you outlined while simplifying and reducing the amount of extension points you have to consider.

    cheers!
    adam

    • jammykam · April 15, 2016

      Hey Adam. I hadn’t seen that post, great approach. I’ll take a look at it in more detail, but perhaps looking at the `FileSystemDataProvider` instead. The problem with overriding the Sql Provider would be the GetBlobStream method, esp given my original requirement for very large files (over 1GB). Streaming this from Azure and then onto the client would lead to trouble. The `FileSystemDataProvider` doesn’t seem to override the BlobStream methods so should be interesting to see how this deals with file based media (if at all). Let me know if you have any insights 🙂

  2. Pingback: Storing Files in Azure Cloud Storage through the Sitecore Media Library – Part One – Uploading Files | jammykam
  3. Pingback: Storing Files in Azure Cloud Storage through the Sitecore Media Library – Part Two – Link Management | jammykam

Leave a comment