Subtracting 2 fields

3 posts, 0 answers
  1. Joel
    Joel avatar
    166 posts
    Member since:
    Oct 2006

    Posted 20 Jul 2009 Link to this post

    I am getting odd behavior from Q2 2009.

    Similar problem different results, 3 ways
    1.  I am using a table and entering values through the design surface.  Textbox10 is 73.4, Textbox11 is 73.3.  Textbox12 has an expression of =sum([textbox10.value] - [textbox11.value])  In design mode and html mode I get 0 for Textbox 12


    2.  Method 2.  No values entered into the textboxes in the table, no expression in textbox12.
    Code behind says
        Public Sub New() 
            InitializeComponent() 
            TextBox10.Value = CDbl(75.5) 
            TextBox11.Value = CDbl(74.4) 
            TextBox12.Value = TextBox5.Value - TextBox8.Value 
     
        End Sub 

    Textbox12 says 1.09999999999999.  It says the same thing if I tell it to format to 2 decimal places.

    3.  Method3.  If I add similar items as number 2, Textbox6.value is 1.09999999999999 TextBox12.value is 1.10000000000001
            TextBox5.Value = CDbl(75.5) 
            TextBox8.Value = CDbl(74.4) 
            TextBox6.Value = TextBox5.Value - TextBox8.Value 
     
            TextBox10.Value = CDbl(75.5) 
            TextBox11.Value = CDbl(74.4) 
            TextBox12.Value = TextBox14.Value - TextBox15.Value 
     


    Any ideas?  This was easy in previous releases.


  2. Joel
    Joel avatar
    166 posts
    Member since:
    Oct 2006

    Posted 20 Jul 2009 Link to this post

    Update.  I get the 1.10000000000001 if I use 55.6 and 55.5 for the values.  It has nothing to do with the position of the box in the row.

    Sorry for the confusion.
  3. DevCraft banner
  4. Steve
    Admin
    Steve avatar
    10941 posts

    Posted 21 Jul 2009 Link to this post

    Hello Joel,

    This is how .NET works and you can read more info on floating point operations in this article. Using decimal instead would not yield in such results.
    As for your first "method" - division of two integers is also an int and therefore only the part before the delimiter is taken into account. Your expression in this case should return float values in order to achieve what you want.

    Best wishes,
    Steve
    the Telerik team

    Instantly find answers to your questions on the new Telerik Support Portal.
    Check out the tips for optimizing your support resource searches.
Back to Top