Chunk Upload Server Validation

[Richard]

Good afternoon,

I'm using a chunk upload to upload files, some of which could be greater than 2GB.

When the user has selected a file I want to run a series of checks of the filename against a database before deciding whether to allow the upload to take place.  If it fails that validation I want it to return a message to the view explaining why it failed.

I've set the Upload as follows:

        @(Html.Kendo().Upload()
            .Name("files")
            .HtmlAttributes(new { aria_label = "files"})
            .Enable(true)
            .Multiple(false)
            .Async(a => a
                .Save("Async_ChunkSave", "Source")
                .Remove("Async_Remove", "Source")
                .AutoUpload(true)
                .ChunkSize(4194304)
            )
            .Events(events => events
                .Upload("onUpload")
                .Complete("onComplete")
                .Error("onUploadError"))
        )

I'm using the Upload event to send some extra data from the view:

public async Task<ActionResult> Async_ChunkSaveAsync(IEnumerable<IFormFile> files, string metaData, string uploadData)

Am I right in thinking that although it has a foreach file in files, it will only be one file since chunkData.FileName can only be one file - presumably the same name as the file's.

                    if (files != null)
                    {
                        foreach (var file in files)
                        {
                            path = Path.Combine(fullFolderPath, chunkData.FileName);
                            AppendToFile(path, file);
                        }
                    }

Where is the best place to validate the filename against the database, before it starts handling some potentially very large files? And then return an appropriate error message to the View, and halt the upload?

Kind regards,

Richard

 

[Mihaela]

Hello Richard,

Thank you for the provided code snippets.

I would recommend validating the filename on the server and returning the respective error message back to the client, as per the example below:

• When the first file chunk is received in the "Async_ChunkSave" Action, access the filename through the "metaData" and return StatusCode and error message if it is not valid.

        public async Task<ActionResult> Async_ChunkSaveAsync(IEnumerable<IFormFile> files, string metaData, string uploadData)
        {
            if (metaData == null)
            {
                return await Save(files);
            }

            ChunkMetaData chunkData;
            var options = new JsonSerializerOptions
            {
                PropertyNameCaseInsensitive = true
            };
            chunkData = JsonSerializer.Deserialize<ChunkMetaData>(metaData, options);

            if (chunkData.FileName == "test.png") //Validate the file name
            {
                return StatusCode(500, "Invalid file name.");
            }
            ...
        }

• Handle the Error event of the Upload that triggers when the upload or the remove operation fails and get the error message from the server response:

<script>
    function onError(e) {
        let errorMsg = e.XMLHttpRequest.response; //get the error message from the server
    }
</script>

• You could display the error message as a tooltip or by using the Notification component:
  • Tooltip example:

<script>
    function onError(e) {
        let errorMsg = e.XMLHttpRequest.response; //get the error message from the server
        $(".upload-error").remove(); //reset any previously shown error messages from the server
        $(".k-upload").append('<div class="k-tooltip k-tooltip-error upload-error" style="margin:0.5em"><span class="k-icon k-i-exclamation-circle"> </span>' + errorMsg + '<div class="k-callout k-callout-n"></div></div>').show(); //Show the error message
</script>

• Notification example:
  

<div id="notification-container"></div>

@(Html.Kendo().Notification()
    .Name("notification")
    .HideOnClick(true)
    .AppendTo("#notification-container")
    .Position(p => p.Pinned(true).Top(30).Right(30))
    .Stacking(NotificationStackingSettings.Down)
    .Templates(t =>
    {
        t.Add().Type("error").ClientTemplateID("errorTemplate");
    })
)

<script id="errorTemplate" type="text/x-kendo-template">
    <div class="upload-error">
        <h3>#= title #</h3>
        <p>#= message #</p>
    </div>
</script>

@(Html.Kendo().Upload()
  .Name("files")
  .Events(ev => ev.Error("onError"))
  ...
)

<script>
    function onError(e) {
        let errorMsg = e.XMLHttpRequest.response; //get the error message from the server
        // Display the message.
        var notification = $("#notification").data("kendoNotification"); //Get a reference of the Notification
        notification.show({
            title: "Invalid data",
            message: errorMsg //Pass the server response error message
        }, "error");
    }
</script>

 

Regarding your query about the IEnumerable<IFormFile> files in the Save Action, your assumption is correct - it will be a single file when using the chunk upload feature.

I hope that helps.

 

Regards, Mihaela Progress Telerik

[Richard]

Afternoon Mihaela,

Many thanks for your suggestions.  I'm trying out the Notification example - my Upload is on a page of a Wizard and I want to automatically go to the final step when complete, but stay on the same step when there is an error. I found the tooltip looked neat but didn't fully show in the wizard step.  The notification can be placed elsewhere so doesn't affect the step window size. I've have also had to trap that there is an error in the OnError event, and then check for that in the OnComplete event to stop it going to the final step when there is an error.  

In the error event, I've also managed to use e.files[0].uid to removeFileByUid.

Is there a way to hide the default dropzone of <em class="k-dropzone-hint">Drop files here to upload</em>.  I've seen this suggested, but it isn't working:

$("#files").closest(".k-upload").find(".k-dropzone em").remove();

Kind regards,

Richard

 

[Mihaela]

Hello Richard,

Thank you for your feedback.

Regarding the default file drop zone, there is a logged feature request in our Feedback portal to implement a built-in option to disable the drop zone of the Upload:

https://feedback.telerik.com/kendo-jquery-ui/1562711-add-an-option-to-disable-dropzone-of-upload

Feel free to cast your vote since the more votes the item gets, the higher its chances of getting approved for implementation. 

Meanwhile, you could remove it as follows:

<style>
     .k-upload {
       border-width: 0;
       display: inline-block;
     }

     .k-upload .k-dropzone {
       padding: 0;
     }
</style>

<script>
    $(document).ready(function() {
      //Removing the "Drop files here message"
      $("#files").closest(".k-upload").find(".k-dropzone-hint").remove();
    });
</script>

Here is a REPL sample for your reference: 

https://netcorerepl.telerik.com/mHarudEU56Skv2Wp44

Best,

Mihaela

[Richard]

Hi Mihaela,

Thanks for your suggestion to hide the dropzone.  I added the background-color in order to hide the colour surrounding the button:

    .k-upload .k-dropzone {
        padding: 0;
        background-color: transparent;
    }

Could you explain which bit of the code is handling the transfer of chunks from the client to the server?

Is it this bit:

        JsonSerializer serializer = new JsonSerializer();
        ChunkMetaData chunkData;
        using (StreamReader streamReader = new StreamReader(ms))
        {
            chunkData = (ChunkMetaData)serializer.Deserialize(streamReader, typeof(ChunkMetaData));
        }

Or this:

AppendToFile(path, file);

For very large files, I'm trying to work out which process is going to take the time.

Kind regards,

Richard

[Mihaela]

Hi Richard,

When uploading a file, the Save request payload is formatted as follows:

When the chunk is received on the server, the "metaData" JSON object is deserialized by using Newtonsoft.Json:

MemoryStream ms = new MemoryStream(Encoding.UTF8.GetBytes(metaData));
JsonSerializer serializer = new JsonSerializer();
ChunkMetaData chunkData;
using (StreamReader streamReader = new StreamReader(ms))
{
  chunkData = (ChunkMetaData)serializer.Deserialize(streamReader, typeof(ChunkMetaData));
}

Another option to deserialize the "metaData" is to use the System.Text.Json namespace:

using System.Text.Json;

ChunkMetaData chunkData;
var options = new JsonSerializerOptions
{
  PropertyNameCaseInsensitive = true //The "metaData" JSON is Camel case-formatted
};
chunkData = JsonSerializer.Deserialize<ChunkMetaData>(metaData, options);

The deserialized "metaData" object allows you to get the "TotalChunks", "ChunkIndex", and "UploadUid" properties for the server response:

FileResult fileBlob = new FileResult();
fileBlob.uploaded = chunkData.TotalChunks - 1 <= chunkData.ChunkIndex;
fileBlob.fileUid = chunkData.UploadUid;
return Json(fileBlob);

Once the "metaData" object is deserialized, the respective file chunk is saved through the AppendToFile() Action method:

string path = Path.Combine("App_Data", chunkData.FileName);
AppendToFile(path, file);

You could use Stopwatch to measure the time execution of this Action, as discussed in the SO thread.

Best,

Mihaela

 

[Richard]

Hi Mihaela,

Many thanks for your advice with this - you have been really helpful.

Do you think it's possible to have the server side Action method running as a background task? I would display a message telling the user to check back later once the file has uploaded, depending on the size of the file.

Kind regards,

Richard

[Mihaela]

Hi Richard,

Thank you for your positive feedback! :)

You can run a service on the background that will process the uploaded files asynchronously by using BackgroundService Class. For more information, refer to the following articles:

• Background tasks with hosted services in ASP.NET Core
• .NET Core Background Services blogpost

Best,

Mihaela