Subtracting 2 fields

3 posts, 0 answers
  1. Joel
    Joel avatar
    165 posts
    Member since:
    Oct 2006

    Posted 20 Jul 2009 Link to this post

    I am getting odd behavior from Q2 2009.

    Similar problem different results, 3 ways
    1.  I am using a table and entering values through the design surface.  Textbox10 is 73.4, Textbox11 is 73.3.  Textbox12 has an expression of =sum([textbox10.value] - [textbox11.value])  In design mode and html mode I get 0 for Textbox 12

    2.  Method 2.  No values entered into the textboxes in the table, no expression in textbox12.
    Code behind says
        Public Sub New() 
            TextBox10.Value = CDbl(75.5) 
            TextBox11.Value = CDbl(74.4) 
            TextBox12.Value = TextBox5.Value - TextBox8.Value 
        End Sub 

    Textbox12 says 1.09999999999999.  It says the same thing if I tell it to format to 2 decimal places.

    3.  Method3.  If I add similar items as number 2, Textbox6.value is 1.09999999999999 TextBox12.value is 1.10000000000001
            TextBox5.Value = CDbl(75.5) 
            TextBox8.Value = CDbl(74.4) 
            TextBox6.Value = TextBox5.Value - TextBox8.Value 
            TextBox10.Value = CDbl(75.5) 
            TextBox11.Value = CDbl(74.4) 
            TextBox12.Value = TextBox14.Value - TextBox15.Value 

    Any ideas?  This was easy in previous releases.

  2. Joel
    Joel avatar
    165 posts
    Member since:
    Oct 2006

    Posted 20 Jul 2009 Link to this post

    Update.  I get the 1.10000000000001 if I use 55.6 and 55.5 for the values.  It has nothing to do with the position of the box in the row.

    Sorry for the confusion.
  3. DevCraft R3 2016 release webinar banner
  4. Steve
    Steve avatar
    10941 posts

    Posted 21 Jul 2009 Link to this post

    Hello Joel,

    This is how .NET works and you can read more info on floating point operations in this article. Using decimal instead would not yield in such results.
    As for your first "method" - division of two integers is also an int and therefore only the part before the delimiter is taken into account. Your expression in this case should return float values in order to achieve what you want.

    Best wishes,
    the Telerik team

    Instantly find answers to your questions on the new Telerik Support Portal.
    Check out the tips for optimizing your support resource searches.
Back to Top