Bytes and Blobs

David Flanagan

http://davidflanagan.com

@__DavidFlanagan

HTML5 Binary APIs

Blob

interface Blob {
    readonly attribute unsigned long long size;
    readonly attribute DOMString type;
    Blob slice(start, end, contentType);
};

• Binary Large Object
• Minimal API: size and type, and can be chopped up
• May live on disk: requires async APIs
• A binary data interchange mechanism.
• Use cases follow

File Upload

Manual Client-side Cache

Local Image Display

Audio Synthesis

Decode Binary Data

Image Processing

File

interface File : Blob {
    readonly attribute DOMString name;
    readonly attribute Date lastModifiedDate;
};

• All Files are Blobs
• Not all Blobs are Files

How to Get a File

• With an <input> element:

  <input type="file"
         onchange="process(this.files[0])">
  

• With Drag-and-Drop:

  droptarget.ondrop = function(e) {
      var files = e.dataTransfer.files; 
      upload(files);
  }
  

• User grants permission to read the file(s) by selecting them.
• Or read from a private filesystem sandbox (discussed later)

Example: File Properties

<input type="file" accept="image/*" multiple
       onchange="fileinfo(this.files)"/>
<script>
// Log information about a list of selected files
function fileinfo(files) { // files is array-like
  for(var i = 0; i < files.length; i++) { 
    console.log(files[i].name,
                files[i].size,
                files[i].type,
                files[i].lastModifiedDate);
  }
}
</script>

Example from JavaScript: The Definitive Guide

Uploading a File or Blob

function upload(url, blob, f) {
  var r = new XMLHttpRequest();
  r.open("POST", url);
  r.send(blob);  // XHR2           
  r.upload.onprogress = f; 
}
function uploadFiles(url, files) {
  var r = new XMLHttpRequest();
  r.open("POST", url);
  var fd = new FormData(); // XHR2
  for(var i=0; i < files.length; i++)
    fd.append(files[i].name, files[i]);
  r.send(fd);  // multipart/form-data
}

Downloading a Blob

• Brand new feature of XHR2
• Buggy in Chrome 10, NYI in FF4

function getBlob(url, cb) {
  var r = new XMLHttpRequest();
  r.open("GET", url);          
  // Or use "arraybuffer"
  r.responseType = "blob"   // XHR2
  r.send(null);            
  r.onload = function() {   // XHR2
    if (cb) cb(r.response); // XHR2
  }
}

Example from JavaScript: The Definitive Guide

Blob URLs

• URL.createObjectURL()
• webkitURL.createObjectURL()
• Return a short, opaque blob:// URL
• Browser implements HTTP semantics for blob:// URLs
• Use URL.revokeObjectURL() when done

Example: Local Images

<input type="file" allow="image/*"
       onchange="display(this,this.files[0])"/>
<script>
function display(elt, f) {
  var img = document.createElement("img");
  img.src = URL.createObjectURL(f);
  img.onload = function() {
    URL.revokeObjectURL(this.src);
  };
  elt.parentNode.replaceChild(img, elt);
}
</script>

Reading Blob Content

• Use a FileReader object:
• readAsText()
• readAsBinaryString()
• readAsArrayBuffer()
• readAsDataURL()
• These are asynchronous methods

Reading a Text File

<input type="file" onchange="read(this.files[0])"/>
<pre id="output"></pre>
<script>
function read(f) {
  var reader = new FileReader();
  reader.readAsText(f);         
  reader.onload = function() {
    var text = reader.result;
    var node = document.createTextNode(text);
    var out = document.getElementById("output");
    out.innerHTML = "";
    out.appendChild(node);
  }
  reader.onerror = function(e) { console.log(e); }
}
</script>

Example from JavaScript: The Definitive Guide

Try it out:

Select a text file:

File contents:

readAsArrayBuffer()

• Like readAsText(), but reader.result is an ArrayBuffer.
• ArrayBuffer is a chunk of bytes in memory:

  interface ArrayBuffer {
      readonly attribute unsigned long byteLength;
  };
  

• ArrayBufferView is a view of a portion of an ArrayBuffer:

  interface ArrayBufferView {
      readonly attribute ArrayBuffer buffer;
      readonly attribute unsigned long byteOffset;
      readonly attribute unsigned long byteLength;
  };
  

• Typed Arrays are array-like ArrayBufferView subtypes...

Typed Arrays

• Int8Array, Uint8Array
• Int16Array, Uint16Array
• Int32Array, Uint32Array
• Float32Array, Float64Array

var bytes = new Uint8Array(buffer);
bytes = new Uint8Array(b, b.byteLength-1024, 1024);
var ints = new Int32Array(4096); // new buffer

• Typed arrays use native endianness
• Multi-byte element types useless for network data

DataView

• Another ArrayBufferView that understands endianness:

  var data = new DataView(buffer);
  var i = data.getInt32(0); // Big-endian
  data.setInt32(0,i,true);  // Reverse it
  var x = data.getFloat64(4, true); // LE
  var y = data.getFloat32(12);      // BE
  

• Terrible API, but easy to improve upon:
• https://github.com/davidflanagan/BufferView
• Includes example that lists contents of a .zip file

Creating Blobs

// Use WebKitBlobBuilder in Chrome 12
var bb = new BlobBuilder();
// Append Strings, ArrayBuffers or Blobs
bb.append("Hello"); // Any # of times
// Convert to a Blob, with optional type
var blob = bb.getBlob("text/plain");

Audio Synthesis (1)

// This code works in Chrome 
header = new Uint8Array([
    0x52,0x49,0x46,0x46, // "RIFF"
    0, 0, 0, 0,          // put total size here
    0x57,0x41,0x56,0x45, // "WAVE"
    0x66,0x6d,0x74,0x20, // "fmt "
    16,0,0,0,            // size of the following
    1, 0,                // PCM format
    1, 0,                // Mono: 1 channel
    0,80,0,0,            // 20,000 samples per second
    0,80,0,0,            // byte rate: one byte per sample 
    1, 0,                // aligned on any byte
    8, 0,                // 8 bits per sample
    0x64,0x61,0x74,0x61, // "data"
    0, 0, 0, 0           // put number of samples here
]).buffer;  // Note: we just want the ArrayBuffer

Audio Synthesis (2)

if (!window.BlobBuilder && window.WebKitBlobBuilder)
    window.BlobBuilder = window.WebKitBlobBuilder;
if (!window.URL && window.webkitURL)
    window.URL = window.webkitURL;

function makeWave(samples) {
    var bb = new BlobBuilder();
    var dv = new DataView(header);
    dv.setInt32(4, 36 + samples.length, true);
    dv.setInt32(40, samples.length, true);
    bb.append(header);
    bb.append(samples.buffer);
    return bb.getBlob("audio/wav");
}

Audio Synthesis (3)

// play a note of the specifed frequency and duration
// We've hardcoded 20,000 samples per second
function playnote(frequency, duration) {
    var samplespercycle = 20000/frequency;
    var samples = new Uint8Array(20000 * duration);
    var da = 2*Math.PI/samplespercycle;
    for(var i = 0, a=0; i < samples.length; i++,a+=da) 
        samples[i] = Math.floor(Math.sin(a) * 255);
    var blob = makeWave(samples);
    var url = URL.createObjectURL(blob);
    var player = new Audio(url);
    player.play();
    player.addEventListener("ended",function () {
                              URL.revokeObjectURL(url);
                            }, false);
}

Local (Sandboxed) Files

// Chrome 12 requires --unlimited-quota-for-files
// and maybe also --allow-file-access-from-files
function loadBlob(path, callback) {
  webkitRequestFileSystem( // Ask for filesystem
    PERSISTENT, 50*1024*1024, 
    function(fs) { // When we get filesystem
      fs.root.getFile( // Ask for a fs entry
        path, {},      // With this name
        function(entry) {  // When we get entry
          entry.file(      // Ask for the file
            function(f) {  // When we get file
              callback(f); // Pass to callback
            });
        });
    });
}

Writing Local Files

function saveBlob(blob, path, done) {
  webkitRequestFileSystem( // Ask for fs
    PERSISTENT,     // Save across sessions
    50*1024*1024,   // Allow 50Mb 
    function(fs) {  // When we get the fs
      fs.root.getFile( // Ask for a fs entry
        path,          // With specified path
        {create:true}, // Create if needed.
        function(entry) { // When we get entry
          entry.createWriter( // Ask for a writer
            function(writer) { // When we get writer
              writer.write(blob); // Write blob
              writer.onwrite = done;
            });
        });
    });
}

Blob as Messages

• HTML5 "structured clone algorithm" works with blobs:

  // Messages to and from WebWorkers
  worker.postMessage(blob); // Send Blob
  worker.onmessage = function(e) {
    var blob = e.data;      // Receive Blob
  }
  // In the worker
  postMessage(blob);  // Send Blob to parent
  onmessage = function(e) {
   var blob = e.data; // Receive Blob from parent
  }
  // Cross-origin Messaging
  // (Some browsers are still text-only)
  win.postMessage(blob, origin);
  win.onmessage = function(e) {
    var blob = trusted(e.origin)?e.data:null;
  }
  

Blobs and Databases

• localStorage and sessionStorage localStorage.audioclip = blob; // NYI
• Blobs can be stored and retrieved from an IndexedDB:

  object_store.put(blob);        // Store a Blob
  var request = object_store.get(key);
  request.onsuccess = function() {
      var blob = request.result; // Look up a Blob
  }
  

• IndexedDB API is actually much more complicated than that

Summary