It is possible to perform custom validation for a particular file extension. You can write the necessary code to perform custom validation in the ValidatingFile server-side event. Please have a look into the following C# code which performs a custom validation for ZIP files.
sender, ValidateFileEventArgs e)
// check only the zip files
//Your Custom Validation Code
// zip files must not be validated for file size by the internal validator