1. It depends at what stage in the ASP.NET page life-cycle the TextBoxID is set to the RadCaptcha ValidatedTextBoxID property. For example, if it is set in the button's click event and at the same time this button serves as the one which will validate the code (i.e. CausesValidation="true" and the ValidationGroup is the same as the one set to the Captcha control) the input will not be validated correctly, because the RadCaptcha gets the text from the custom textbox in the LoadPostData
method which is fired before the Load and after the Init event. This means that the RadCaptcha tried to get the input from the custom textbox before the ID was actually assigned to the RadCaptcha.ValidatedTextBoxID
Please set the ValidatedTextBoxID property before the LoadPostData is executed (i.e. Page_Init), or set it when the page validation does not occur, and let me know if the code is validated correctly.
3. When an AJAX request is fired, the server-side page goes through its full page lifecycle. This means that a new RadCaptcha code will be generated, but because it's an AJAX update it would not be shown on the image (the old code is still shown because the RadCaptcha is not affected by the partial page update). So, the user sees the old code and enters it correctly but still gets an Error Message, because this old code is not valid anymore. To overcome this issue, the Captcha control should be placed inside the UpdatePanel.
the Telerik team