Screen Capture Control

[Jeff]

Does Telerik have a screen capture control, component, or utility of some kind?

For our company-wide support ticket application we built, I want to create the ability for the user to click a button, draw (or select) a window to capture the screen contents, and automatically add the image to the RadUpload tools temp folder to be uploaded.

Before I design it myself, I figured I'd check to see if Telerik has a solution I can implement.

 

[Vessy]

Hello Jeff,

Currently we do not offer a screen capturing component in our suit, but you can take a look at the following resources and see if any of the suggested approaches is applicable to your needs:

• https://www.c-sharpcorner.com/UploadFile/0c1bb2/capture-screen-shots-of-page-using-Asp-Net-C-Sharp-without-script/
• https://stackoverflow.com/questions/30214374/screenshot-in-c-asp-net

If you wish you could submit a feature request for such component at our Feedback Portal, so we can collect the overall interest on such control and plan it for implementation accordingly:

https://feedback.telerik.com/aspnet-ajax?typeId=2&listMode=Recent

 

Regards,
Vessy
Progress Telerik

[Jeff]

Thank you for your response.  I did run across both of those resources.  It appears those are only related to Windows Forms Applications.   Our is Web Forms.

The html2canvass package is nice, as well, but it offers no mobile support. 

I will add it as a feature request, and continue my search.  Looks like what I want to do is not so easy (if not impossible without 3rd party apps) due to security reasons.

[Jeff]

@Vessy

Is there a way to add a blob, or stream, or bytes directly to the RadAsyncUpload.UploadedFiles collection?  

I have some logic which will return the selected screen/window contents to each of the types mentioned above.  Now I just need to get that data into the UploadedFiles collection.

Thanks,

Jeff.

[Vessy]

Hi Jeff,

Can you, please, elaborate a bit the exact logic that you have implemented? RadAsyncUpload does not allow to update its UploadFiles collection manually, but if you have the stream on the server-side you can directly work with it and save it as an image.
If you have the stream on the client-side, you can store it in a hidden field and continue with the saving on the server-side, by accessing the hidden field value.

[Jeff]

Hi Vessy,

I jave implemented the use of the getMediaDevice screen capture API, resulting in producing a blob, or bytes, or a stream, or the actual jpeg, all client aide.  So, i have options.

I would be excited to know how I can get this image, in whichever format necessary, into my RadAsyncUpload control.

Thank you,

Jeff.

[Jeff]

Additionally, I'm able to create a thumbnail version of the image, from the blob, and append it to the body, showing at the bottom of the page.  I can also view the blob (full size image) in the network tab of the dev tools.

The code I have also shows that I can send it to a web page as a Post request, but I haven't quite figured out how to use that, just yet.

[Jeff]

Quick correction:  It's navigator.mediaDevices.getDisplayMedia that I'm using.  Sorry about the confusion.

[Jeff]

For even further clarity, here is the exact SO solution:  

https://stackoverflow.com/a/60231897/16577485

[Vessy]

Hi Jeff,

Thanks a lot for the update. I am afraid that as mentioned in my previous reply, RadAsyncApload does not allow manual updating of its UploadFiles collection, so adding manually a file to it from a stream would not be possible.

If you have the stream on the client-side, though, you could follow a similar logic to save the copied image:

• save it into an asp:hiddenField
• accessing its value of the server-side (you can make a manual ajax request) 
• use the standard Image.Save method to create an image from stream (refer this post)

[Jeff]

Vessy,

Thank you for the reply.  I accept that I cannot manually accomplish this, so I'm trying another route.

I could just add my own server side logic, as you've described, but I'm trying to avoid that route, if at all possible.  My preference is to use the existing logic we have to iterate the UploadedFiles collection, then insert a record for each into the database, then upload/save the file.

With that being said, I have been exploring drag and drop functionality to accomplish this.  I create a screenshot, then using appendChild to the bottom of the page to display a 300px thumbnail.  I'm attempting to allow the user to drag this preview image, and drop it into the dropzone of the RadAsyncUpload control.

I've done so by adding draggable events to the newly appended child, and they're all firing, except the "drop" event.  Haven't figured out why, yet. 

Before I potentially waste too much time exploring this, do you believe it's possible?

 

onScreenShotClick = async () => {
                
                // take the screenshot
                var screenshotJpegBlob = await takeScreenshotJpegBlob()

                screenshotJpegBlob.name = "testImageFromBlob.jpeg"
                screenshotJpegBlob.lastModified = new Date();

                const myFile = new File([screenshotJpegBlob], screenshotJpegBlob.name, {
                    type: screenshotJpegBlob.type,
                });

                // show preview with max size 300 x 300 px
                var previewCanvas = await blobToCanvas(screenshotJpegBlob, 300, 300)


                previewCanvas.setAttribute('draggable', true);
                previewCanvas.addEventListener('dragstart', (e) => {
                    console.log('dragstart', e);
                    e.dataTransfer.ClientFileSelected = myFile;
                    console.log(e.dataTransfer.ClientFileSelected);
                })



                previewCanvas.addEventListener('dragover', (e) => {
                    console.log('dragover')
                    e.preventDefault();
                })



                previewCanvas.addEventListener('dragenter', (e) => {
                    console.log('dragenter')
                    e.preventDefault();
                })



                // Drop event is not firing  -jk 20210805
                previewCanvas.addEventListener('drop', (e) => {
                    e.preventDefault();
                    console.log('drop', e)
                    //this.appendChild(e.dataTransfer.ClientFileSelected);
                })

                let link = document.createElement('a');
                link.appendChild(previewCanvas);
                const openImage = () => { window.open(URL.createObjectURL(screenshotJpegBlob), "_blank") };
                link.href = '#';
                link.addEventListener('click', openImage);                

                await document.body.appendChild(link)
            }

[Vessy]

Hi Jeff,

I made a little research and found that the following SO thread coping with a similar problem:

• https://stackoverflow.com/questions/7699987/html5-canvas-ondrop-event-isnt-firing

You can try adding "return false;" after preventing the drag/drop events and see what the result will be.

If you get the mentioned in the same thread exception, though ("[04:16:42.298] uncaught exception: [Exception... "Component returned failure code.."), there is nothing much you can do. The only remaining option would be to extract the already existing logic that you want to use and follow the approach described in my previous reply.

[Jeff]

Vessy,

Thank you for that.  I implemented some of the logic in that post with no luck.  I did not get the exception you mentioned.  I did get the drop event to fire, but no luck getting the desired element to drop into the dropzone.

Prior to this, I began following your suggested approach.  My knowledge of using binary data between the client and the server is quite limited.  When I mentioned I have a "stream", it turns out that stream returns a video stream.

What I'm left with is either a blob, or bytes; they work great.  It's just a struggle to get it to the server.

I keep getting a Request Validation error, and discovered that I can disable this validation for that specific control.  It's clear that I shouldn't go that route.  Request Validation in ASP.NET | Microsoft Docs

I have not implemented the Ajax request, and am beginning to believe that is what may be causing me grief.  Today, I will add the Ajax logic and see if that gets it resolved.

Currently, on the client, I'm using the blob.stream() method and saving the result in the HiddenField.  When I try to convert that value to a byte array, server-side, and then convert that byte array to a MemoryStream to output an image to save, I get a "Parameter is not valid" exception.

I've also tried the blob.text() method, which resolves to a promise where I can add the blob data as a string into the HiddenField.  Again, when I try to convert that string to a byte array, then a stream, I get the same "Parameter is not valid" exception.

 

Blob.stream() documentation:  Blob.stream() - Web APIs | MDN (mozilla.org)

The Blob interface's stream() method returns a ReadableStream which upon reading returns the data contained within the Blob.

 

Blob.text() documentation:  Blob.text() - Web APIs | MDN (mozilla.org)

The text() method in the Blob interface returns a Promise that resolves with a string containing the contents of the blob, interpreted as UTF-8.

 

// Client Side Code:

//blob.text() method:
let screenshotJpegBlob = await takeScreenshotJpegBlob()
let blobHidden = document.getElementById('<%= HiddenField1.ClientID %>')
screenshotJpegBlob.text().then(text => { blobHidden.value = text })


//blob.stream() method:
let blobHidden = document.getElementById('<%= HiddenField1.ClientID %>')
let blobStream = screenshotJpegBlob.stream();
blobHidden.Value = blobStream;

 

//Server-side code

//Button-Click event handler:
byte[] blobBytesArray = Encoding.UTF8.GetBytes(HiddenField1.Value);
ByteArrayToImage(blobBytesArray).Save("c:\\temp\\myImage.jpeg", System.Drawing.Imaging.ImageFormat.Jpeg);


// Function returning image:
public Image ByteArrayToImage(byte[] bytesArray)
{
      using (MemoryStream stream = new MemoryStream(bytesArray))
      {
          Image image = Image.FromStream(stream);
          return image;
       }
}

 

I've mentioned all of this in hopes that there is some obvious problem with this logic, that I'm missing due to not being proficient with these types of things.

 

Thank you,

Jeff.

[Jeff]

Quick update:  

I'm now getting the base64string conversion from the blob, client-side.  I can console.log it, and view it in dev tools.  However, when I try to use the HiddenField Value server-side, it's empty.

Is this because I need to get the value of the hidden field using a manual ajax request?

let blobHidden = document.getElementById('<%= HiddenField1.ClientID %>')
                let reader = new FileReader();
                reader.readAsDataURL(screenshotJpegBlob);
                reader.onloadend = async () => {
                    let base64String = reader.result;
                    console.log('Base64 String - ', base64String);
                    blobHidden.Value = base64String;
                };

 

Server side shows the HiddenField value is an empty string:

var filePath = "c:\\temp\\myImage.jpeg";
            var bytes = Convert.FromBase64String(HiddenField1.Value);
            using (var imageFile = new FileStream(filePath, FileMode.Create))
            {
                imageFile.Write(bytes, 0, bytes.Length);
                imageFile.Flush();
            }

 

[Jeff]

Yet another update:

I switched from using a HiddenField for storing the base64 string, representing the blob data, to a RadTextBox, and like magic, the Text property is available server-side, and I can convert and save the image from this string.   I did have to remove the "Tag" from the beginning of the base64 string before it would work.

var base64String = BlobTextBox.Text;
var bytes = Convert.FromBase64String(base64String.Substring(base64String.LastIndexOf(',') + 1));

So... success, kind of.

My guess as to why the HiddenField Value is not available at the server is because I'm not using a manual Ajax Request.  Would this be true?

Since I don't fully understand Ajax Requests, I'll be spending some time digging in to them.  Any guidance is much appreciated.

[Vessy]

Hi Jeff,

When you are making a manual AJAX request via the ajaxRequest method, the AJAX initiator is the AjaxManager itself and you need to add the HiddenField ID to the updated controls collection of the AjaxManager in order to be able to access its new value on the server-side.

You can find useful information about the main concept of the AjaxManager here:

• AjaxManager Overview
• AjaxUpdatedControl Properties

 

For example:

<telerik:RadAjaxManager ID="RadAjaxManager1" runat="server">
    <AjaxSettings>
        <telerik:AjaxSetting AjaxControlID="RadAjaxManager1">
            <UpdatedControls>
                <telerik:AjaxUpdatedControl ControlID="HiddenField1" />
            </UpdatedControls>
        </telerik:AjaxSetting>
    </AjaxSettings>
</telerik:RadAjaxManager>