The expected behavior of Image Comparison

[gnuftw]

01.[CodedStep(@"ImageComp")]
02.public void Comparison_CodedStep()
03.{
04.    System.Drawing.Bitmap actualbmp = new System.Drawing.Bitmap(@"pink3.png");
05.    System.Drawing.Bitmap expectedbmp = new System.Drawing.Bitmap(@"pink5.png");
06.    ArtOfTest.Common.PixelMap expected = ArtOfTest.Common.PixelMap.FromBitmap(expectedbmp);
07.    ArtOfTest.Common.PixelMap actual = ArtOfTest.Common.PixelMap.FromBitmap(actualbmp);
08.
09.    Assert.IsTrue(expected.Compare(actual, 0.0));
10.}

Above is my code for the image comparison I am carrying out with the 2 attached PNGs. Both pink squares are 300 by 300 pixels. Each black square contained inside each pink square is 50 x 50 pixels. In examining the images, it is inherently obvious that they are different, the black squares are in completely separate locations. But, with a tolerance of 0, the test I have this coded step in still passes. So it brings me to my question...

How is this test passing?

What is the Compare function testing for? A summation of pixel values?

Is there a workaround to this? How can I make this test fail without modifying the images?

~ Max

[Cody]

Hi Max,

Test Studio doesn't really know how to compare two images via code. It's probably doing a .ToStrhing() on both objects and getting back a string for the type of object. Since they are both the same type, the two strings compare equal.

We do not have an API method for comparing two images in code. We have a non-coded step that can compare an image found in the app to one that was previously recorded, but not any two random images like you're attempting to do. There are other tools and libraries out there that can do this.


Regards,
Cody
Telerik

 

[gnuftw]

I don't see how what I am doing is any different than the Image Comparison page from the documentation. In both cases, the 2 PixelMaps being compared were both PixelMaps are originally Bitmaps. It shouldn't make a difference whether both of them came from disk versus the method described in Perform an Image Comparison in Code. 

You say it is probably doing a .ToString() on the objects. Is this a guarantee or a guess? 

Nonetheless, I will look into other libraries available to do the comparison I need.

Thanks!

~ Max

[Cody]

Hi Max,

You say it is probably doing a .ToString() on the objects. Is this a guarantee or a guess?

I admit it's a guess on my part based on my experience how other similar comparisons do work.

Thank you for reminding me of that KB article. I had forgotten about it and the functionality it points out. After deep research I have a real answer for you.

Test Studio does image comparison by doing a mathematical difference of the color histogram of the two images. Your two sample images calculate to absolutely identical color histograms as shown in the 4 attached screen shots. As a result Test Studio considers them equal. If your images had a different amount of each color they would compare differently.

Regards,
Cody
Telerik

 

[gnuftw]

Alright, that totally makes sense.

Thanks for your effort to get this explanation to me!

~ Max

[Cody]

Hello,

I'm glad I could help. And my apologies for not getting there sooner.

Regards,
Cody
Telerik