background preloader

Capturing Audio & Video in HTML5

Capturing Audio & Video in HTML5
Introduction Audio/Video capture has been the "Holy Grail" of web development for a long time. For many years we've had to rely on browser plugins (Flash or Silverlight) to get the job done. Come on! HTML5 to the rescue. This tutorial introduces a new API, navigator.getUserMedia(), which allows web apps to access a user's camera and microphone. The road to getUserMedia() If you're not aware of its history, the way we arrived at the getUserMedia() API is an interesting tale. Several variants of "Media Capture APIs" have evolved over the past few years. I'll try to summarize what happened in 2011... Round 1: HTML Media Capture HTML Media Capture was the DAP's first go at standardizing media capture on the web. If you wanted to let users take a snapshot of themselves with the webcam, that's possible with capture=camera: Recording a video or audio is similar: Kinda nice right? Support: Android 3.0 browser - one of the first implementations. Round 2: device element Take a breath. Round 3: WebRTC OK.

Media Capture and Streams 4. MediaStream API 4.1 Introduction The MediaStream interface is used to represent streams of media data, typically (but not necessarily) of audio and/or video content, e.g. from a local camera. Each MediaStream object can contain zero or more tracks, in particular audio and video tracks. Each track in a MediaStream object has a corresponding MediaStreamTrack object. A MediaStreamTrack represents content comprising one or more channels, where the channels have a defined well known relationship to each other (such as a stereo or 5.1 audio signal). A channel is the smallest unit considered in this API specification. A MediaStream object has an input and an output. Each track in a MediaStream object can be disabled, meaning that it is muted in the object's output. A MediaStream can be finished , indicating that its inputs have forever stopped providing data. The output of a MediaStream object MUST correspond to the tracks in its input. Both MediaStream and MediaStreamTrack objects can be cloned.

HTML5: camera access navigator.getUserMedia - WebRTC Prompts the user for permission to use a media device such as a camera or microphone. If the user provides permission, the successCallback is invoked on the calling application with a LocalMediaStream object as its argument. Syntax navigator.getUserMedia ( constraints, successCallback, errorCallback ); Example Here's an example of using getUserMedia() with browser prefixes. Parameters constraints The constraints parameter is a MediaStreamConstraints object with two Boolean members: video and audio. If the value or the member is not specified in the object, the value for the member defaults to false. successCallback The getUserMedia function will call the function specified in the successCallback with the LocalMediaStream object that contains the media stream. function(localMediaStream) { var video = document.querySelector('video'); video.src = window.URL.createObjectURL(localMediaStream); video.onloadedmetadata = function(e) { // Do something with the video here. }; }, errorCallback Permissions

Camera and Video Control with HTML5 Client-side APIs on mobile and desktop devices are quickly providing the same APIs. Of course our mobile devices got access to some of these APIs first, but those APIs are slowly making their way to the desktop. One of those APIs is the getUserMedia API, providing developers access to the user's camera. Let me show you how to get simple camera access from within your browser! The HTML Please read my note about the HTML structure below: <video id="video" width="640" height="480" autoplay></video><button id="snap">Snap Photo</button><canvas id="canvas" width="640" height="480"></canvas> Each of these elements should be created once confirmation of camera support is confirmed, but for the sake of this tutorial, I wanted to show you what the elements look like with basic HTML. The JavaScript Since the HTML elements above are already created, the JavaScript portion will look smaller than you think: Taking a photo is only marginally more difficult.

Capturing Audio & Video in HTML5 Many browsers now have the ability to access video and audio input from the user. However, depending on the browser it might be a full dynamic and inline experience, or it could be delegated to another app on the user's device. On top of that, not every device even has a camera. So how can you create an experience that uses a user generated image that works well everywhere? Start simple and progressively If you want to progressively enhance your experience, you need to start with something that works everywhere. Ask for a URL This is the best supported but least satisfying option. Though, if you want to manipulate the image in any way, things are a bit more complicated. File input You can also use a simple file input element, including an accept filter that indicates you only want image files. This method works on all platforms. <input type="file" accept="image/*" id="file-input"><script> const fileInput = document.getElementById('file-input'); Drag and drop Paste from clipboard if (file ! Tips

Face detection using HTML5, javascript, webrtc, websockets, Jetty and OpenCV Through HTML5 and the corresponding standards, modern browsers get more standarized features with every release. Most people have heard of websockets that allows you to easily setup a two way communication channel with a server, but one of the specifications that hasn't been getting much coverage is the webrtc specificiation. With the webrtc specification it will become easier to create pure HTML/Javascript real-time video/audio related applications where you can access a user's microphone or webcam and share this data with other peers on the internet. Update: in the newest versions of the webrtc spec we can also access the microphone! As with a lot of HTML5 related specification, the webrtc one isn't quite finished yet, and support amongst browsers is minimal. For this we need to take the following steps: In other words, we're going to create a real-time face detection system, where the frontend is completely provided by 'standard' HTML5/Javascript functionality.