I am having the exact same problem and in addition, when the button is first rendered, it does not have a grey box surrounding it (screenshot 1), but after selecting a file, a box suddenly appears (screenshot 2). I have set the following 2 options:
I am using Kendo UI Complete v2013.2.918 (using the Bootstrap theme)
Any additional styling I have is from Twitter Bootstrap 2.3.2 - I have not specifically overridden any styles related to the upload widget
I am using Firefox 25.0 on a Windows 8 Pro (64 bit) system
I recently upgraded to this version of Kendo UI; in the previous one (Q2 final release) both problems did not occur.
I can live with the active "blue" active button, but the box surrounding the upload button totally messes up my application.