In this article, we look at Array Buffers and how they are used to handle raw binary data in JavaScript.
In JavaScript, an ArrayBuffer is defined as an object that represents a generic, fixed-length raw binary data buffer. It allows you to work with binary data directly without worrying about character encoding. This means the ArrayBuffer handles raw binary data, such as working with network protocol files.
To understand it better without using too many unfamiliar words, ArrayBuffer is used to keep and arrange binary data. It handles raw binary data, such as working with network protocol files and manipulating raw data.
Think of the ArrayBuffer as a box of chocolates. The box size is fixed and can only contain a certain number of chocolates; regardless of whether any chocolates are in it, the size is fixed. The box still takes up space and has a defined capacity; it cannot be compromised, shrunk or grown. The box is the ArrayBuffer, while the chocolates are the binary data.
ArrayBuffer is used to handle raw binary data effectively. It is particularly useful when dealing with network protocols, file input and output (I/O), and other situations where you need to work with binary data.
The goal when using ArrayBuffer is to to seamlessly handle raw binary data during processes, for cases such as :
To create an array buffer, you only need the ArrayBuffer constructor and the size (in bytes).
Here is how it works:
const buffer = new ArrayBuffer(8); //creates a buffer of 8 bytes
console.log(buffer.byteLength); //output: 8
In the example above, we created a fixed-size block of memory with 8 bytes. It is currently empty because it does not contain any usable data.
As explained in the introduction, ArrayBuffer doesn’t allow direct manipulation, but that doesn’t mean it can’t be manipulated. To interact with an ArrayBuffer, you need typed arrays like Uint8Array, Int16Array or Float32Array, or you can use DataView for more flexible manipulation.
Typed arrays are a special kind of array that allows you to store and manipulate raw binary data in a specific format like 8-bit integers, 16-bit floats and more. They only allow one type of data and have a fixed size.
Here are the list of types:
The choice of the type to be used depends on these factors:
Let’s manipulate the array buffer we created in the previous example (remember it is empty) using a typed array of Uint8Array, Int16Array and Float32Array.
const buffer = new ArrayBuffer(8); //creates a buffer of 8 bytes
const uint8view = new Uint8Array(buffer); //creates a view as 8-bit unsigned integers
uint8view[0] = 65; //store value at index 0
uint8view[1] = 66; //store value at index 1
console.log(uint8view); //Output: Uint8Array(8) [65, 66, 0, 0, 0, 0, 0, 0]
const int16View = new Int16Array(buffer); //creates a view as 16-bit unsigned integers
int16View[0] = 300; // store 300 at byte 0,
int16View[1] = -500; // store -500 at byte 1
console.log(int16View); // Output: Int16Array(4) [300, -500, 0, 0]
const float32View = new Float32Array(buffer);
float32View[0] = 3.14; //store a floating-point number at byte 0,
console.log(float32View); //Output: Float32Array(2) [ 3.140000104904175, 0 ]
Unlike typed arrays, which only allow one type of data and a fixed size, DataView allows you to read and write different types of data sizes, making it much easier to work with ArrayBuffer.
Here is a list of types in DataView:
The offset is the index or position of a byte in the array buffer. See it as a slot to uniquely store a byte in a buffer while value is the byte to be stored.
Let’s see how to use DataView.
const buffer = new ArrayBuffer(8); //Create an 8-byte memory block
const view = new DataView(buffer); //Create a DataView for easy reading/writing
In the code above, a new ArrayBuffer with 8 bytes of memory is created. It is initially empty and is stored in a variable name buffer. We then used DataView to read and write different data types for the ArrayBuffer.
view.setInt8(0, 100); // Store the value 100 at the first byte (position 0) as an 8-bit signed integer
view.setUint16(1, 500); // Store the value 500 at the next two bytes (positions 1 and 2) as a 16-bit unsigned integer
view.setFloat32(3, 3.14); // Store the value 3.14 at the next four bytes (positions 3 to 6) as a 32-bit floating-point number
console.log(view); // Output the DataView object showing the total byte length (8 bytes)
In the code above, we store three different data types in the buffer: setInt8, setUint16 and setFloat32 using DataView.
setInt8(0, 100) has an offset of 0 and a value of 100; it stores 100 at byte position 0. setInt8 can either be positive or negative, but 100 is positive.
setUint16(1, 500) has an offset of 1 and a value of 500. It stores 500 in byte positions 1 and 2 (2 bytes). This is because in binary, 500 is 00000001 11110100. The first byte takes position 1, while the second byte takes position 2.
setFloat32(3, 3.14) has an offset of 3 and a value of 3.14; it stores 3.14 in byte position 3 to 6 (4 bytes). This is because of how floating point numbers are represented in the computer. Float32 uses the IEEE 754 standard and takes 4 bytes according to the computer. In binary, 3.14 is 01000000 01001100 11110011 11010101. The first byte takes position 1, the second byte takes position 2, the third byte takes position 3, and the fourth byte takes position 4.
All these are different data types stored in a buffer, which can only be achieved using DataView.
console.log(view.getInt8(0)); // 100
console.log(view.getUint16(1)); // 500
console.log(view.getFloat32(3)); // 3.14
In the code above, we are reading the code of different data types stored in the buffer and getting them according to their offset.
JavaScript provides several methods to convert between strings and ArrayBuffers, which is essential for tasks such as encoding, decoding and managing data streams. We can’t talk about conversion between strings and ArrayBuffer without talking about TextEncoder and TextDecoder.
To convert a string to an ArrayBuffer, you typically need to encode the string into a binary format. The most common encoding is UTF-8.
Here is how to achieve it:
function stringToArrayBuffer(str) {
let encoder = new TextEncoder();
return encoder.encode(str).buffer;
}
let myString = "Hello, World";
let buffer = stringToArrayBuffer(myString);
console.log(buffer); //ArrayBuffer containing UTF-8 encoded string.
The code above is a function for converting a string to an array buffer using TextEncoder.
To convert an ArrayBuffer to a string, you need to decode the binary data. The TextDecoder object is used to decode an ArrayBuffer back to a string.
function arrayBufferToString(buffer) {
let decoder = new TextDecoder();
return decoder.decode(new Uint8Array(buffer));
}
let decodedString = arrayBufferToString(buffer);
console.log(decodedString); //"Hello, World"
In the code above, we used the function to convert the arrayBuffer to string using TextDecoder.
While UTF-8 is the most common encoding, sometimes you might need to handle other encodings. The TextEncoder and TextDecoder can be used with different encodings by simply specifying the encoding type as shown below:
let encoder = new TextEncoder("utf-16");
let buffer = encoder.encode("Hello, World!");
let string = decoder.decode(buffer);
console.log(string); //"Hello, World"
Web APIs often require data to be in a binary format. For instance, when using the Fetch API to download binary files, the response is an ArrayBuffer. Converting this binary data to a string might be necessary for further processing or display.
fetch("example.com/data")
.then((response) => response.arrayBuffer())
.then((buffer) => {
let text = arrayBufferToString(buffer);
console.log(text);
});
ArrayBuffer is important when dealing with performance-intensive tasks, especially when they involve binary data.
Here are some of its applications in modern software:
let fileInput = document.querySelector('input[type="file"]');
fileInput.addEventListener("change", function (event) {
let file = event.target.files[0];
let reader = new FileReader();
reader.onload = function () {
let arrayBuffer = reader.result;
let text = arrayBufferToString(arrayBuffer);
console.log(text);
};
reader.readAsArrayBuffer(file);
});
In the code above, input[type="file"] is an HTML tag for uploading a file. The file is read as an array buffer and later converted to a string to be displayed in the console.
It is clear that ArrayBuffer is important for managing high-performance applications. However, these best practices should be considered for optimal use:
subarray() instead of slice() when possible.Here is how to do that:
let buffer = new ArrayBuffer(8);
buffer = null; // Memory Cleared!
response.body.getReader().Here is the code explaining this:
async function fetchLargeFile(url) {
const response = await fetch(url);
const reader = response.body.getReader();
const chunks = [];
let receivedLength = 0;
while (true) {
const { done, value } = await reader.read();
if (done) break;
chunks.push(value);
receivedLength += value.length;
console.log(`Received ${receivedLength} bytes so far...`);
}
// Combine all chunks into a single Uint8Array
const fullData = new Uint8Array(receivedLength);
let position = 0;
for (const chunk of chunks) {
fullData.set(chunk, position);
position += chunk.length;
}
console.log("Download complete!");
return fullData;
}
// Example usage
fetchLargeFile("https://example.com/large-file.bin").then((data) => {
console.log("Final file size:", data.length);
});
We created an async function called fetchLargeFile which accepts a URL as an argument. We use getReader to read the files in small chunks instead of downloading them at once. We create an array called chunks to collect and collate all the small chunks. We use a while loop to read the file in small parts and track the process. We then store each chunk in the chunk array and update the progress. We combine all chunks into one final file and log Download complete! when the process is completed.
SharedArrayBuffer. Some applications (e.g., Web Workers) require multitasking instead of one single way of processing; it is recommended to use SharedArrayBuffer to share data between threads without copying. This is most often used for high-performance tasks like video processing or real-time communication.Here is how to make use of SharedArrayBuffer:
const buffer = new SharedArrayBuffer(1024); // Shared across threads
const uint8Array = new Uint8Array(buffer);
ArrayBuffer in JavaScript helps handle binary data efficiently. Although it may seem complex to understand, it can help develop high-performance software applications when implemented correctly. In fact, it is the technology behind many successful streaming platforms and other real-time applications.
Chris Nwamba is a Senior Developer Advocate at AWS focusing on AWS Amplify. He is also a teacher with years of experience building products and communities.