03 4 / 2013

Blink. Chrome’s new rendering engine

Chrome is departing WebKit as its rendering engine. This is big news for web developers, so I thought I’d write up my personal take on the matter. Please realize these are my own thoughts and not those of Google.

The new engine is called Blink. It’s open source and based WebKit.

"You’re kidding, right!?"

That was my reaction when I first heard the news. It was quickly followed by: “Won’t this segment the web even further?” and “Great. An additional rendering engine I have to test.” Being a web developer, I feel your pain.

Honestly, I was extremely skeptical about the decision at first. After several conversations with various members of the web platform team here at Google, I was slowly convinced it might not be such a terrible idea after all. In fact, I’m now convinced it’s a good idea for the long term health and innovation of browsers.

Reflecting on Chrome’s mission

Many of you will be in the same skeptic boat I was. But I think it’s worth remembering the continuing goals of the Chromium project.

From day one the Chrome team’s mission has been to build the best browser possible. Speed, security, and simplicity are in its blood. Over the last four years, I have gained a deep respect for our engineering team. They’re some of the most brilliant engineers in the world. If their consensus is that Chrome cannot be the best browser it can be with WebKit at its core, I fully trust and support that decision. After all, these folks know how to build browsers. If you think about it some more things start to make sense. The architecture of today’s web browser is dramatically different than it was back in 2001 (when WebKit was designed).

The irony in all of this is that we were soon destined to have three render engines with Opera’s impending move to WebKit. Even today, Mozilla/Samsung announced their work on a new engine, called Servo. So, we were at three engines. Now we have 4+. Interesting times indeed.

Things we can all look forward to

Ultimately, Chrome is engineering driven project and I’m personally excited about the potential this change offers. Here are a few:

Improved performance & security

Many ideas and proposals have sprung up about things like out of process iframes, moving DOM to JS, multi-threaded layout, faster DOM bindings,…. Big architectural changes and refactorings means Chrome gets smaller, more secure, and faster over time.

Increased transparency, accountability, responsibility

Every feature added to the web platform has a cost. Through efforts like Chromium Feature Dashboard - chromestatus.com, developers will be in the full know about what features we’re adding. New APIs are going under a fine comb before being released. There’s an extensive process for adding new features.

By the way, watch for chromestatus.com to get much more robust in the coming months. I’m personally helping with that project. Look forward to it’s v2 :)

No vendor prefixes

What a debacle vendor prefixes have been! Features in Blink are going to be implemented unprefixed and kept behind the “Enable experimental web platform features” flag until they’re ready for prime time. This is a great thing for authors.

Testing Testing Testing

More conformance testing is a win. Period. There’s a huge benefit to all browser vendors when things are interoperable. Blink will be no exception.

Conclusion

I see Blink as an opportunity to take browser engines to the next level. Innovation needs to happen at all levels of the stack, not just shiny new HTML5 features.

Having multiple rendering engines—similar to having multiple browsers—will spur innovation and over time improve the health of the entire open web ecosystem.

If you have burning questions for Blink’s engineering leads (Darin Fisher, Eric Seidel), post them. There will be a live video Q&A tomorrow (Thursday, April 4th) at 1PM PST: developers.google.com/live

Tags:

Permalink 10 notes

13 9 / 2012

Creating .webm video from getUserMedia()

There’s a ton of motivation for being able to record live video. One scenario: you’re capturing video from the webcam. You add some post-production touchups in your favorite online video editing suite. You upload the final product to YouTube and share it out to friends. Stardom proceeds.

MediaStreamRecorder is a WebRTC API for recording getUserMedia() streams (example code). It allows web apps to create a file from a live audio/video session.

MediaStreamRecorder is currently unimplemented in the Chrome. However, all is not lost thanks to Whammy.js. Whammy is a library that encodes .webm video from a list of .webp images, each represented as dataURLs.

As a proof of concept, I’ve created a demo that captures live video from the webcam and creates a .webm file from it.

LAUNCH DEMO

The demo also uses a[download] to let users download their file.

Creating webp images from <canvas>

The first step is to feed getUserMedia() data into a <video> element:

var video = document.querySelector('video');
video.autoplay = true; // Make sure we're not frozen!

// Note: not using vendor prefixes!
navigator.getUserMedia({video: true}, function(stream) {
  video.src = window.URL.createObjectURL(stream);
}, function(e) {
  console.error(e);
});

Next, draw an individual video frame into a <canvas>:

var canvas = document.querySelector('canvas');
ctx.drawImage(video, 0, 0, canvas.width, canvas.height);

Chrome supports canvas.toDataURL("image/webp"). This allows us to read back the <canvas> as a .webp image and encode is as a dataURL, all in one swoop:

var url = canvas.toDataURL('image/webp', 1); // Second param is quality.

Since this only gives us an single frame, we need to repeat the draw/read pattern using a requestAnimationFrame() loop. That’ll give us webp frames at 60fps:

var rafId;
var frames = [];
var CANVAS_WIDTH = canvas.width;
var CANVAS_HEIGHT = canvas.height;

function drawVideoFrame(time) {
  rafId = requestAnimationFrame(drawVideoFrame);
  ctx.drawImage(video, 0, 0, CANVAS_WIDTH, CANVAS_HEIGHT);
  frames.push(canvas.toDataURL('image/webp', 1));
};

rafId = requestAnimationFrame(drawVideoFrame); // Note: not using vendor prefixes!

\m/

The last step is to bring in Whammy. The library includes a static method fromImageArray() that creates a Blob (file) from an array of dataURLs. Perfect! That’s just what we have.

Let’s package all of this goodness up in a stop() method:

function stop() {
  cancelAnimationFrame(rafId);  // Note: not using vendor prefixes!

  // 2nd param: framerate for the video file.
  var webmBlob = Whammy.fromImageArray(frames, 1000 / 60);

  var video = document.createElement('video');
  video.src = window.URL.createObjectURL(webmBlob);

  document.body.appendChild(video);
}

When stop() is called, the requestAnimationFrame() recursion is terminated and the .webm file is created.

Performance and Web Workers

Encoding webp images using canvas.toDataURL('image/webp') takes ~120ms on my MBP. When you do something crazy like this in requestAnimationFrame() callback, the framerate of the live getUserMedia() video stream noticeably drops. It’s too much for the UI thread to handle.

Having the browser encode webp in C++ is far faster than encoding the .webp image in JS.

My tests using libwebpjs in a Web Worker were horrendously slow. The idea was to each frame as a Uint8ClampedArray (raw pixel arrays), save them in an array, and postMessage() that data to the Worker. The worker was responsible for encoding each pixel array into webp. The whole process took up to 20+ seconds to encode a single second’s worth of video. Not worth it.

It’s too bad CanvasRenderingContext2D doesn’t exist in the Web Worker context. That would solved a lot of the perf issues.

08 9 / 2012

Mashups using CORS and responseType=’document’

I always forget that you can request a resource as a Document using XHR2. Combine this with CORS and things get pretty nice. No need to parse HTML strings and turn them into DOM yourself.

For html5rocks.com, we support CORS on all of our content. It’s trivial to pull down the tutorials page and query the DOM directly using querySelector()/querySelectorAll() on the XHR’s response.

Demo: http://jsbin.com/ibepag/1

https://gist.github.com/3581825

Tags:

Permalink 3 notes

06 8 / 2012

Five things you didn’t know the web could do

It’s been awhile since my last blog post. That’s because I’ve been busy writing articles all over the web! Check out my post on netmagazine, “Five things you didn’t know the web could do”.

Tags:

Permalink 2 notes

23 5 / 2012

Data Binding Using data-* Attributes

Custom data-* attributes in HTML5 are pretty rad. They’re especially handy for stashing small amounts of data and retaining minimal state on the DOM. Turns out, they can also be used for one-way data binding!

I’ve been using a nifty trick in recent projects that I thought would be worth sharing. The technique is to use a data attribute to store values (i.e. the data model) and :before/:after pseudo elements to render the values as generated content (i.e. the view). I call it "poor man’s data binding" because it’s not true data binding in the traditional sense, but the semantics are similar. Count it!

Here we go:

<style>
  input {
    vertical-align: middle;
    margin: 2em;
    font-size: 14px;
    height: 20px;
  }
  input::after {
    content: attr(data-value) '/' attr(max);
    position: relative;
    left: 135px;
    top: -20px;
  }
</style>

<input type="range" min="0" max="100" value="25">

<script>
  var input = document.querySelector('input');

  input.dataset.value = input.value; // Set an initial value.

  input.addEventListener('change', function(e) {
    this.dataset.value = this.value;
  });
</script>

image TRY IT

Notice the 25/100 updates as you move the slider, but the <input> is the only markup on the page.

The magic line is the content: attr(data-value) '/' attr(max). It uses CSS attr() to pull out the data-value and max attributes; both set using markup on the <input>. As those values change, the generated content is automatically updated. Sick data binding bro.

Really the only benefit of this technique is that we’re not including extraneous markup. For comparison, here’s the same gig, but using an extra element:

<input type="range" min="0" max="100" value="25"><span></span>

<script>
  var input = document.querySelector('input');
  input.addEventListener('change', function(e) {
    document.querySelector('span').textContent = this.value + '/' + this.max;
  });
</script>

Less elegant, but it works.

Last but not least, here’s a more complex example that uses CSS transitions to change the height of a div container when clicked. As the height changes, requestAnimationFrame() updates the data-height of the div and the pseudo element picks that up.

image TRY IT

I’m sure if HTML was conceived in the age of web apps, we’d have proper DOM/JS data binding by now. Fortunately, initiatives like MDV and Web Components are on their way. One day this stuff will be a reality and native to HTML!

Data binding is a technique for automatically synchronizing data between two sources. On the web, data binding typically manifests itself as updating DOM (UI) in response to events: XHRs, user input, or other business logic doing its thing. Take the canonical todo list for example. When I mark an item as done, the completed count increments. When it’s unchecked, the count decrements. That’s data binding!

If you want true two-way data binding, checkout one of the popular MVC frameworks like Angular, Knockout, or Ember.

Tags:

Permalink 18 notes

23 4 / 2012

idb.filesystem.js - Bringing the HTML5 Filesystem API to More Browsers

The HTML5 Filesystem API is a versatile API that addresses many of the uses cases that the other offline APIs don’t. It can remedy their shortcomings, like making it difficult to dynamically caching a page. I’m looking at you AppCache!

My ♥ for the API is deep—so much so that I wrote a book and released a library called filer.js to help promote its adoption. While filer aims to make the API more consumable, it fails to address the elephant in the room: browser support.

Introducing idb.filesystem.js

Today, I’m happy to bring the HTML5 Filesystem API to more browsers by releasing idb.filesystem.js.

idb.filesystem.js is a well tested JavaScript polyfill implementation of the Filesystem API intended for browsers that lack native support. Right now that’s everyone but Chrome. The library works by using IndexedDB as an underlying storage layer. This means any browser supporting IndexedDB, now supports the Filesystem API! All you need to do is make Filesystem API calls and the rest is magic.

Demos

I’ve thrown together two demo apps to demonstrate the library’s usage. The first is a basic example. It allows you to create empty files/folders, drag files into the app from the desktop, and navigate into a folder or preview a file by clicking its name:

imageTry the demo in Firefox 11+

Want to use filer.js’s API with idb.filesystem.js? No problem. 90% of filer.js works out of the box with idb.filesystem.js. In fact, the second demo is a slightly modified version of filer.js’s playground app, showing that the two libraries can work in harmony. \m/

What’s exciting is that both of these apps work in FF, Chrome, and presumably other browsers that implement storing binary data in IndexedDB.

I look forward to your feedback and pull requests!

27 12 / 2011

Introducing filer.js

Some 1300+ lines of code, 106 tests, and a year after I first started it, I’m happy to officially unleash filer.js (https://github.com/ebidel/filer.js); a wrapper library for the HTML5 Filesystem API.

Unlike other libraries [1, 2], filer.js takes a different approach and incorporates some lessons I learned while implementing the Google Docs Python client library. Namely, the library reuses familiar UNIX commands (cp, mv, rm) for its API. My goal was to a.) make the HTML5 API more approachable for developers that have done file I/O in other languages, and b.) make repetitive operations (renaming, moving, duplicating) easier.

So, say you wanted to list the files in a given folder. There’s an ls() for that:

var filer = new Filer();
filer.init({size: 1024 * 1024}, onInit.bind(filer), onError);

function onInit(fs) {
  filer.ls('/', function(entries) {
    // entries is an Array of file/directories in the root folder.
  }, onError);
}

function onError(e) { ... }

A majority of filer.js calls are asynchronous. That’s because the underlying HTML5 API is also asynchronous. However, the library is extremely versatile and tries to be your friend whenever possible. In most cases, callbacks are optional. filer.js is also good at accepting multiple types when working with entries. It accepts entries as string paths, filesystem: URLs, or as the FileEntry/DirectoryEntry object.

For example, ls() is happy to take your filesystem: URL or your DirectoryEntry:

// These will produce the same results.
filer.ls(filer.fs.root.toURL(), function(entries) { ... });
filer.ls(filer.fs.root, function(entries) { ... });
filer.ls('/', function(entries) { ... });

The library clocks in at 24kb (5.6kb compressed). I’ve thrown together a complete sample app to demonstrate most of filer.js's functionality:

imageTry the DEMO

Lastly, there’s room for improvement:

  1. Incorporate Chrome’s Quota Management API
  2. Make usage in Web Workers more friendly (there is a synchronous API).

I look forward to your feedback and pull requests!

22 12 / 2011

Making file inputs a pleasure to look at

I’ve seen a lot of people ask how to 1.) apply custom styles to a <input type="file"> and 2.) programmatically open the browser’s file dialog with JavaScript. Turns out, the first is a cinch WebKit. The second comes with a couple of caveats.

If you want to skip ahead, there’s a demo.

Custom file inputs in WebKit

The first example on that demo page shows how to style your basic file input into something great. To achieve magnificence, we start with some standard issue markup:

<input type="file" class="button" multiple>

followed by some semi-rowdy CSS that to hide the ::-webkit-file-upload-button pseudo-element and create a fake button using :before content:

.button::-webkit-file-upload-button {
  visibility: hidden;
}
.button:before {
  content: 'Select some files';
  display: inline-block;
  background: -webkit-linear-gradient(top, #f9f9f9, #e3e3e3);
  border: 1px solid #999;
  border-radius: 3px;
  padding: 5px 8px;
  outline: none;
  white-space: nowrap;
  -webkit-user-select: none;
  cursor: pointer;
  text-shadow: 1px 1px #fff;
  font-weight: 700;
  font-size: 10pt;
}
.button:hover:before {
  border-color: black;
}
.button:active:before {
  background: -webkit-linear-gradient(top, #e3e3e3, #f9f9f9);
}

Reference: i'm just a reference

Since this one is only available in WebKit, I’ve left out the other vendor prefixes.

Programmatically opening a file dialog

No browser that I know of lets you simulate a manual click on a file input without user intervention. The reason is security. Browsers require that a user make an explicit manual click (user-initiated click) somewhere on the page. However, once that happens, it’s straightforward to hijack the click and route it to a file input.

My second technique (see this tweet) for styling a file input works across the modern browsers. It requires a bit of extra markup but allows us to “send” the user’s click to a file input.

The trick is to hide the <input type="file"> by setting it to visibility: hidden; and subbing in an extra <button> to hand the user’s actual click:

<style>
#fileElem {
  /* Note: display:none on the input won't trigger the click event in WebKit.
    Setting visibility: hidden and height/width:0 works great. */
  visibility: hidden;
  width: 0;
  height: 0;
}
#fileSelect {
  /* style the button any way you want */
}
</style>

<input type="file"  multiple>
<button >Select some files</button>

<script>
document.querySelector('#fileSelect').addEventListener('click', function(e) {
  // Use the native click() of the file input.
  document.querySelector('#fileElem').click();
}, false);
</script>

Reference: i'm just a reference

You’ll be even cooler if you use custom events:

function click(el) {
  var evt = document.createEvent('Event');
  evt.initEvent('click', true, true);
  el.dispatchEvent(evt);
}

document.querySelector('#fileSelect').onclick = function(e) {
  // Simulate the click on fileInput with a custom event.
  click(document.querySelector('#fileElem'));
};
Caveat

Most browsers require the fileInput.click() to be called within 1000ms of the user-initiated click. For example, waiting 1.5s will fail because it’s too long after the user initiates a click:

document.querySelector('#fileSelect').onclick = function(e) {
  setTimeout(function() {
    document.querySelector('#fileElem').click(); // Will fail.
  }, 1500);
};

The cap gives you the chance to call window.open, adjust UI, whatever before the file dialog opens.

Live demo

Tags:

Permalink 21 notes

28 11 / 2011

Web Audio API how-to: Playing audio based on user interaction

One thing the Web Audio API does particularly well is play sound. Of course, this is something you’d expect from an audio API :). That said, the API is complex and it’s not immediately obvious on the best way to do something simple like load a sound file and play it based on a button click. That task alone can involve a number of new platform features likes XHR2, FileReader API, and ArrayBuffers.

So…I threw together a quick example on how to load a audio file and play/stop it based on the user clicking a button:

<!DOCTYPE html>
<!-- Author: Eric Bidelman (ericbidelman@chromium.org) -->
<html>
<head>
  <meta charset="utf-8" />
  <meta http-equiv="X-UA-Compatible" content="chrome=1" />
  <title>Web Audio API: Simple load + play</title>
</head>
<body>
  <p>Example of using the Web Audio API to load a sound file
  and start playing on user-click.</p>
  <input type="file" accept="audio/*">
  <button onclick="playSound()" disabled>Start</button>
  <button onclick="stopSound()" disabled>Stop</button>
<script>
var context = new window.webkitAudioContext();
var source = null;
var audioBuffer = null;

function stopSound() {
  if (source) {
    source.noteOff(0);
  }
}

function playSound() {
  // source is global so we can call .noteOff() later.
  source = context.createBufferSource();
  source.buffer = audioBuffer;
  source.loop = false;
  source.connect(context.destination);
  source.noteOn(0); // Play immediately.
}

function initSound(arrayBuffer) {
  context.decodeAudioData(arrayBuffer, function(buffer) {
    // audioBuffer is global to reuse the decoded audio later.
    audioBuffer = buffer;
    var buttons = document.querySelectorAll('button');
    buttons[0].disabled = false;
    buttons[1].disabled = false;
  }, function(e) {
    console.log('Error decoding file', e);
  }); 
}

// User selects file, read it as an ArrayBuffer and pass to the API.
var fileInput = document.querySelector('input[type="file"]');
fileInput.addEventListener('change', function(e) {  
  var reader = new FileReader();
  reader.onload = function(e) {
    initSound(this.result);
  };
  reader.readAsArrayBuffer(this.files[0]);
}, false);

// Load file from a URL as an ArrayBuffer.
// Example: loading via xhr2: loadSoundFile('sounds/test.mp3');
function loadSoundFile(url) {
  var xhr = new XMLHttpRequest();
  xhr.open('GET', url, true);
  xhr.responseType = 'arraybuffer';
  xhr.onload = function(e) {
    initSound(this.response); // this.response is an ArrayBuffer.
  };
  xhr.send();
}
</script>
</body>
</html>

Demo

As you can see, this example includes two different ways to get an audio file into the Web Audio API: via an <input type="file"> and an XMLHttpRequest. Both methods call initSound(), which is passed an ArrayBuffer containing the audio file. That method then decodes the audio and stores the result in a global variable (so it can be reused as play/stop are pressed).

That’s it! Straightforward once you see it, right!?

Tags:

Permalink 2 notes

01 8 / 2011

Reading .mp3 ID3 tags in JavaScript

For a recent project, I needed to read an .mp3’s ID3 metadata (song title, artist, year, album) in pure JS. The idea was to have the user select a song file, and boom!, its info would display to the user. Good news…totally easy with the FileReader API and JS typed arrays.

Initially, I did a quick search to find some examples, but all of the examples I found used older techniques like manipulating a binary string. Ugh! We have better tools now for working with binary data in JS. Typed arrays to the rescue, specifically DataView.

DataView is a cousin to ArrayBufferView, which is a “view” of a portion of an ArrayBuffer. Array buffers represent chunks of bytes in memory. Multiple views can be created from a single ArrayBuffer. For example, one could create a Int8Array and a Float32Array from the same underlying data. Hence, “views”. This property makes them extremely versatile for binary data.

For my purposes, DataView turned out to be a perfect container for pulling sections of bytes as a string. However, I found the API to be a bit unintuitive in practice. Fortunately, I stumbled upon Christopher Chedeau's jDataView a while back and things started making sense. jDataView is an excellent wrapper for DataView, improving much of its jankiness and adding a few extra utility methods for things like seeking and getting at data (e.g. getString(), getChar()).

Here’s all the code that I needed:

document.querySelector('input[type="file"]').onchange = function(e) {
  var reader = new FileReader();

  reader.onload = function(e) {
    var dv = new jDataView(this.result);

    // "TAG" starts at byte -128 from EOF.
    // See http://en.wikipedia.org/wiki/ID3
    if (dv.getString(3, dv.byteLength - 128) == 'TAG') {
      var title = dv.getString(30, dv.tell());
      var artist = dv.getString(30, dv.tell());
      var album = dv.getString(30, dv.tell());
      var year = dv.getString(4, dv.tell());
    } else {
      // no ID3v1 data found.
    }
  };

  reader.readAsArrayBuffer(this.files[0]);
};

Pretty slick.

DataView is implemented in Chrome 9+ and Webkit nightlies. However, jDataView provides the DataView API for all the browsers using the best available option between Strings, JS typed arrays, and DataView.