I am getting odd behavior from Q2 2009.
Similar problem different results, 3 ways
1. I am using a table and entering values through the design surface. Textbox10 is 73.4, Textbox11 is 73.3. Textbox12 has an expression of =sum([textbox10.value] - [textbox11.value]) In design mode and html mode I get 0 for Textbox 12
2. Method 2. No values entered into the textboxes in the table, no expression in textbox12.
Code behind says
Textbox12 says 1.09999999999999. It says the same thing if I tell it to format to 2 decimal places.
3. Method3. If I add similar items as number 2, Textbox6.value is 1.09999999999999 TextBox12.value is 1.10000000000001
Any ideas? This was easy in previous releases.
Similar problem different results, 3 ways
1. I am using a table and entering values through the design surface. Textbox10 is 73.4, Textbox11 is 73.3. Textbox12 has an expression of =sum([textbox10.value] - [textbox11.value]) In design mode and html mode I get 0 for Textbox 12
2. Method 2. No values entered into the textboxes in the table, no expression in textbox12.
Code behind says
| Public Sub New() |
| InitializeComponent() |
| TextBox10.Value = CDbl(75.5) |
| TextBox11.Value = CDbl(74.4) |
| TextBox12.Value = TextBox5.Value - TextBox8.Value |
| End Sub |
Textbox12 says 1.09999999999999. It says the same thing if I tell it to format to 2 decimal places.
3. Method3. If I add similar items as number 2, Textbox6.value is 1.09999999999999 TextBox12.value is 1.10000000000001
| TextBox5.Value = CDbl(75.5) |
| TextBox8.Value = CDbl(74.4) |
| TextBox6.Value = TextBox5.Value - TextBox8.Value |
| TextBox10.Value = CDbl(75.5) |
| TextBox11.Value = CDbl(74.4) |
| TextBox12.Value = TextBox14.Value - TextBox15.Value |
Any ideas? This was easy in previous releases.
2 Answers, 1 is accepted
0
Joel
Top achievements
Rank 2
Rank 2
answered on 20 Jul 2009, 07:24 PM
Update. I get the 1.10000000000001 if I use 55.6 and 55.5 for the values. It has nothing to do with the position of the box in the row.
Sorry for the confusion.
Sorry for the confusion.
0
Hello Joel,
This is how .NET works and you can read more info on floating point operations in this article. Using decimal instead would not yield in such results.
As for your first "method" - division of two integers is also an int and therefore only the part before the delimiter is taken into account. Your expression in this case should return float values in order to achieve what you want.
Best wishes,
Steve
the Telerik team
Instantly find answers to your questions on the new Telerik Support Portal.
Check out the tips for optimizing your support resource searches.
This is how .NET works and you can read more info on floating point operations in this article. Using decimal instead would not yield in such results.
As for your first "method" - division of two integers is also an int and therefore only the part before the delimiter is taken into account. Your expression in this case should return float values in order to achieve what you want.
Best wishes,
Steve
the Telerik team
Instantly find answers to your questions on the new Telerik Support Portal.
Check out the tips for optimizing your support resource searches.