This is a migrated thread and some comments may be shown as answers.

Subtracting 2 fields

2 Answers 99 Views
General Discussions
This is a migrated thread and some comments may be shown as answers.
Joel
Top achievements
Rank 2
Joel asked on 20 Jul 2009, 07:17 PM
I am getting odd behavior from Q2 2009.

Similar problem different results, 3 ways
1.  I am using a table and entering values through the design surface.  Textbox10 is 73.4, Textbox11 is 73.3.  Textbox12 has an expression of =sum([textbox10.value] - [textbox11.value])  In design mode and html mode I get 0 for Textbox 12


2.  Method 2.  No values entered into the textboxes in the table, no expression in textbox12.
Code behind says
    Public Sub New() 
        InitializeComponent() 
        TextBox10.Value = CDbl(75.5) 
        TextBox11.Value = CDbl(74.4) 
        TextBox12.Value = TextBox5.Value - TextBox8.Value 
 
    End Sub 

Textbox12 says 1.09999999999999.  It says the same thing if I tell it to format to 2 decimal places.

3.  Method3.  If I add similar items as number 2, Textbox6.value is 1.09999999999999 TextBox12.value is 1.10000000000001
        TextBox5.Value = CDbl(75.5) 
        TextBox8.Value = CDbl(74.4) 
        TextBox6.Value = TextBox5.Value - TextBox8.Value 
 
        TextBox10.Value = CDbl(75.5) 
        TextBox11.Value = CDbl(74.4) 
        TextBox12.Value = TextBox14.Value - TextBox15.Value 
 


Any ideas?  This was easy in previous releases.


2 Answers, 1 is accepted

Sort by
0
Joel
Top achievements
Rank 2
answered on 20 Jul 2009, 07:24 PM
Update.  I get the 1.10000000000001 if I use 55.6 and 55.5 for the values.  It has nothing to do with the position of the box in the row.

Sorry for the confusion.
0
Steve
Telerik team
answered on 21 Jul 2009, 09:36 AM
Hello Joel,

This is how .NET works and you can read more info on floating point operations in this article. Using decimal instead would not yield in such results.
As for your first "method" - division of two integers is also an int and therefore only the part before the delimiter is taken into account. Your expression in this case should return float values in order to achieve what you want.

Best wishes,
Steve
the Telerik team

Instantly find answers to your questions on the new Telerik Support Portal.
Check out the tips for optimizing your support resource searches.
Tags
General Discussions
Asked by
Joel
Top achievements
Rank 2
Answers by
Joel
Top achievements
Rank 2
Steve
Telerik team
Share this question
or