To change the background of the appointment, the default control template of the appointment dialog would need to be extracted and modified. You could follow the steps from the Editing Control Templates article, and the needed template has x:Key="EditAppointmentTemplate". In it, set the Background property of the main Grid layout to the needed color. Then, create a style that targets the telerik:SchedulerDialog element and set its Template property to the modified control template. Finally, create an additional style with telerik:RadScheduleView as target type and set its EditAppointmentDialogStyle to the style that has the extracted control template set in.
To change the BorderBrush brush, follow the steps from the Customizing Dialog Windows article since, by default, the RadScheduleView dialogs are displayed in RadWindow instances.
With that said, I have prepared a sample project for you to test, and I hope that the provided information is of help to you.