After much debugging I found the issue.
First: It's specific to a radmenu that exapands with a webservice. I tested it with other user controls and it worked perfectly. I created a new page with nothing but the user control, I also created new usercontrols with a rad menu without a webservice and it worked fine.
Second: It only occurs after the lower menu gets clicked once. Once that usercontrol gets focus for the first time, the z-index of the entire div gets updated, but never reverts back to the original index. This is where the issue is at the core, the changing zindex.
How I fixed it was by adding a line to the client side menu click event.
document.getElementById(sender.get_id()).style.zIndex = (getMenuPageIndex(sender.get_id()) * -1) + 3000;
The function "getMenuPageIndex", is a method that parses and returns an integer from the user control name itself. I used the id of the user control to create a type of index by adding an integer. The fully rendered it includes the radmenu id as well as the usercontrol id and allows me to easily access the index that I set on the page level.
Once I have that index, I'm able to reset the zindex of the div itself, while incrementing 1 for each element found on the page allowing the menus to overlap from top to bottom and not the other way around. This way the uppermost menu gets priority and cascades downward.
The classes that I applied were in response to your first suggestion. After that didn't work, I kept trying out different methods to force the zindex. But since it was occurring on the clickevent, any classes I added were being overwritten by the bug.
Since I've got you here, I'll mention some other things that I noticed.
1) The close animation doesn't occur when the expandmode is set to webservice. It animates on expand, but just disappears on close.
2) When a height is set the width is forced. This is handled differently by different browsers (IE and Firefox).
Thanks for your help,