HTML 5 Video Scripting with Javascript

Javascript: Scene List, Progress Bar, Fast Forward and Backward, Volume controls.



Studies of a Flying Heron at Lake Ossiach

Our Video Divided into Scenes

Here again comes our heron with 600 kbit/s and 848x480 pixels. This time we will be able to fully disfruit the video by slow motion, one-frame-steps and a scene directory (as far as your browser does already support all of it).




Volume: 100%

Apart from the rewind function and the scene/shot selection, which we have implemented ourselves (see below), all the functionality is directly supported via Javascript.





Reference; the video tag: Javascript Functions, Properties and Event Handlers

For all of you who are a little bit impatient and who just wanna know the methods, properties and event handlers used with the video object in order to set something up themselves quickly here comes a table with the most used functions sorted by topic. Of course we have also elaborated oncarrying methods and concepts.

basic features and properties
video.play() start to play (can jump to the beginning if already playing)
video.pause() pause (a video.pause() followed by a video.play() may not take the desired effect.)
video.paused true if the video is paused, false otherwise.
<video oncanplay='myfn()' initialization function myfn() is called as soon as the video has been loaded.
<video ontimeupdate='myfn()' trigger called while the video is playing and whenever there is a seek
<video onseeked='myfn()' frame the program has jumped to is ready to be viewed and to be read out
<video onerror='myfn()' define error handling routine
video.duration the duration of the video in seconds
video.currentTime actual point in time of playback, seeks to the given point in time
video.videoWidth, video.videoHeight width and height of the shown frame or image
video.playBackRate 1 ~ default playback rate; 0.5 ~ half, 2 ~ doubled playback rate, -1,-2,-3,..: rewind
video.muted true whenever the sound has been disabled
video.volume volume as fraction or real number (value between 0 or 1)

A video tag can be inserted with document.createElement("VIDEO") into the document; its properties can be controlled via video.src, .autoplay, .loop, .controls und .preload (see my introductory article). video.addEventListener('canplay', fnname, false) does finally add an event listener to our new video object. The function invoked by this mechanism expects an event object e as parameter. You can thereby access the video object via e.target. "error" is another event listener for a function like function(e) { alert(e.code); }.

e.code
1 == MEDIA_ERROR_ABORTED the user has aborted fetching the video
2 == MEDIA_ERROR_NETWORK network error
3 == MEDIA_ERR_DECODE error at decodation time
4 == MEDIA_ERR_SRC_NOT_SUPPORTED media format not supported

You can detect the progress in fetching the video by video.networkState.

video.networkState
0 == NETWORK_EMPTY not yet initialized
1 == NETWORK_IDLE audio/video is active and has selected a resource, but is not using the network
2 == NETWORK_LOADING actively fetches the source
3 == NETWORK_NO_SOURCE no source in a supported format can be spotted

According to the status field video.networkState the following events exist: loadstart, progress (downloading), suspend (loading of video suspended/finished), abort (downloading aborted), error, emptied (on reload), stalled (trying to get data but none available).

video.currentSrc denotes the source from out of which the source is being fetched. video.canPlayType(type) == '' / 'maybe' / 'probably' determines whether the given format can not be loaded at all, can maybe or probably be loaded.

In order to start a video at the right time video.readyState exists:

video.readyState
0 == HAVE_NOTHING no data
1 == HAVE_METADATA duration, width, height and other metadata of the video have been fetched.
2 == HAVE_CURRENT_DATA There has not been sufficiently data loaded in order to start or continue playback.
3 == HAVE_FUTURE_DATA enough data to start playback
4 == HAVE_ENOUGH_DATA it should be possible to play the media stream without interruption till the end.

Again we have matching events like loadedmetadata, loadeddata, waiting, playing, canplay and canplaythrough. At last we do also have events for the given user actions: play, pause, seeking, timeupdate, ended, ratechange and durationchange.

You may determine whether a given time period has already been buffered, played or whether it is seekable (seek = jump to a certain point in time). buffered, played and seekable are TimeRanges objects. The number of time periods can be determined with timeranges.length; the start and end point of a given time frame with timeranges.start(i) and timeranges.end(i). We do additionally have the boolean status flags seeking and ended which are set while seeking to a new position or whenever the end of the video has been reached. video.startTime at last gives the position from which on the last playback time period had been started.

Interestingly it is also possible to read out the actual video frame pixel by pixel, to alter it by pixelwise operations and then show the result inside a canvas object. However this can not be done directly. At first you need to copy the actual frame with drawImage from the video into a canvas which will need to be hidden from the user. From there you can read the image with getImageData pixel by pixel. You may finally want to create an altered image with createImageData which thereupon needs to be filled by hand. The pixels are stored all in a one dimensional array with an entry for red, green, blue and alpha (opacity) in sequence for each pixel. After filling the newly created pixel array you may show the results with putImageData.

var bgCanvas = document.createElement('canvas'); var bgc = bgCanvas.getContext('2d'); var redc = redCanvas.getContext('2d'), bgc.drawImage(video,0,0); pixAll = bgc.getImageData(0,0,bgCanvas.width,bgCanvas.height); pixRed = bgc.createImageData(bgCanvas.width,bgCanvas.height); for( i=0; i < pixAll.data.length; i+=4 ) { pixRed.data[i] = pixAll.data[i]; pixRed.data[i+1] = pixRed.data[i+2] = 0; pixRed.data[i+3] = 255; } redc.putImageData( pixRed, 0, 0 );


Sourcecodes of the Embedded Video

Let us first have a look at the HTML control elements for the video. There is nothing really special to be noted except that the buttons have been labeled with Unicode characters. Some of the Unicode characters have been selected with kcharselect while others can be looked up by their name like the RIGHT-POINTING TRIANGLE. Let us also have a short look at the embedded Javascript hook 'oncanplay' of the video tag as well as the onclick and onchange events of the slider, the option list and the push buttons.

If you want to have multiple videos in one web page you will need to assign them different id-s. The id-value (here: 'ReiherVideo') needs then to be exchanged for all onclick and onchange function invocations. Other elements do not need to be changed as they are identified automatically by InitControls(this) via their 'class=' name inside the VideoControls-container.

<table class=VideoControls> <tr><td> <video id=ReiherVideo preload=auto oncanplay='InitControls(this)' style='min-width:848px;min-height:480px'> <source src='data/Reiher-848x480-13064.mp4' type='video/mp4'> <source src='data/Reiher-848x480-13000kb.webm' type='video/webm'> <source src='data/Reiher-848x480-20000.ogg' type='video/ogg'> <!--source src='data/Reiher-848x480-600kb.mp4' type='video/mp4'> <source src='data/Reiher-848x480-600kb.webm' type='video/webm'> <source src='data/Reiher-848x480-600kb.ogg' type='video/ogg'--> </video> <td> <select class='scenes' onchange='SelectScene(ReiherVideo,this)' size=8> <option value='0:00' selected>0:00 Abflug</option> <option value='0:03'>0:03 Reiher über freiem Wasser</option> <option value='0:07'>0:07 Landeanflug</option> <option value='0:08.6'>0:08 Landung</option> <option value='0:10'>0:10 sitzend (am Baumstamm)</option> </select> <br><br> <input type=button class=butStop value='&#x25FC;' onclick='RewindVideo(ReiherVideo);'> <input type=button class=fstbwdbut value='&#x00AB;' onclick='PlayPause(ReiherVideo,this,-2)'> <input type=button class=butStepBwd style='letter-spacing: -0.4ex;' value='&nbsp;&#x23D0;&lsaquo;&nbsp;' onclick='FrameStep(ReiherVideo,this,-0.04)'> <input type=button class=snailbut value='~' onclick='PlayPause(ReiherVideo,this,0.2)'> <input type=button class=playbutton value='&#x25B6;' onclick='PlayPause(ReiherVideo,this,1)'> <input type=button class=butStepFwd style='letter-spacing: -0.4ex;' value='&nbsp;&rsaquo;&#x23D0;&nbsp;' onclick='FrameStep(ReiherVideo,this,+0.04)'> <input type=button class=fstfwdbut value='&#x00BB;' onclick='PlayPause(ReiherVideo,this,2)'> <br> <table><tr> <td><input type=range class=volumeRange min=0 max=1 step=0.1 value=1 onchange='AdjustVolume(ReiherVideo,this.value)'> <td><input type=button value='&#x266B;' onclick='Mute(ReiherVideo,this)'> <td style="white-space: nowrap;">Volume: <span class=volumeTxt> 100% </span> </table> <tr><td> <input type="range" class='videopos' style="width:100%" min=0 max=0 step=0.04 onchange='GotoPos(ReiherVideo,this.value)'> </table>

Let us start by playing the video without considering the rewind functionality. We have the procedure PlayPause which calls PlayVideo and PauseVideo. We will never call video.play() and video.pause() directly because PlayVideo and PauseVideo need to adjust the button labels. video.play() should only be called if the video does not play yet because it may otherwise cause the player to start from the beginning. Even a video.pause() followed by a video.play() would not do what we want as the video would remain stopped (Apparently it takes some time for the .pause() action to take effect.).

If we click onto a scene in the scene list at the right side SelectScene is called. SelectScene does only need to set video.currentTime in order to make the jump. After that we call UpdateTime like each time after making a change to video.currentTime in order to make the time slider show the correct playback position (only exception to this rule is GotoPos which is called by direct clicks on the time slider so that there is no need to update the slider any more.).

UpdateTime is by the way also the hook which is directly called during video playback in order to update the time slider position. UpdateTime calls CheckForScene in sequence which sorts out the actually played scene basing on the playback position. It does so by accessing the DOM-tree of the scene selector where the option fields are child nodes listed in the correct order as they are displayed. Access to the DOM-tree is only necessary if we leave the current scene which is retained as long as our playback position lies withing curSceneStart and curSceneEnd

Somewhat tricky is also the implementation of the playback buttons because it is always the button with the currently selected playback speed that needs to turn into a pause button (We have different speeds like normal, fast forward, rewind and slow motion.). This can simply be done by resetting the labels of all buttons each time and by only turning the pressed button into a pause button afterwards.

The matching of the current playback speed requires ratedelta because the video.playbackRate may be adjusted a little bit whenever we set it especially when setting it to a real number (rounding).

Let us now explain the rewind function. It basically works by setting video.currentTime 25 times per second which can be done via setInterval. Unfortunately a simple implementation like this would not work. If the video position is updated faster than the browser can show it the display will remain the same during the whole rewind. The browser in deed waits some time if many changes to video.currentTime happen within a small time frame. The procedure UpdateTime detects whenever a new frame is shown and causes video.currentTime not to be changed again in the meantime. The variable last_frame_shown is set to true if the last frame has already been shown and a new frame may then be displayed.

The implementation of the other handles like Mute and AdjustVolume is very streight forward; I will leave their discovery to the reader.

In order not to leave you in doubt about the usability of the presented code: it is licensed by LGPL and may be used for free and of course free of any charge.

function getTime(opt) { minsec = opt.value.split(':'); return parseFloat( minsec[0]*60 + minsec[1] ); } function CheckForScene(rec,pos) { if( rec.curSceneStart <= pos && pos <= rec.curSceneEnd ) return; var scene, nextscene, sno, cn; rec.curSceneEnd = 0; nextscene = scene = null; for(sno in rec.scenes.childNodes) { cn = rec.scenes.childNodes[sno]; if(cn.tagName=="OPTION") { scene = nextscene; nextscene = cn; rec.curSceneStart = rec.curSceneEnd; rec.curSceneEnd = getTime(nextscene); if( rec.curSceneStart <= pos && pos < rec.curSceneEnd && scene ) { scene.selected = true; return; } }} rec.curSceneStart = rec.curSceneEnd; rec.curSceneEnd = rec.video.duration; scene = nextscene; if( rec.curSceneStart <= pos && pos <= rec.curSceneEnd && scene ) scene.selected = true; } var videos = {}; function getElement(base,clnm) { var ary = base.getElementsByClassName(clnm); if(ary.length!=1) return null; return ary[0]; } function InitControls(video) { var cur = video; do { cur = cur.parentNode; } while ( cur && cur.className != 'VideoControls' ); if(!cur) { alert("No container with class VideoControls found for video "+video.id); return; } if( video.id in videos ) return; var videopos = getElement(cur,'videopos'); var scenes = getElement(cur,'scenes'); if( !videopos || !scenes ) { alert("no distinct "+(videopos?"videopos":"")+(videopos&&scenes?" and ":"")+(scenes?"scenes":"")+" object found for video "+video.id); return; } videopos.max = video.duration; videopos.style.width = video.videoWidth; // video.offsetWidth || video.innerWidth; var StartScene = null, SecondScene = null; for( i in scenes.childNodes ) { var thisopt = scenes.childNodes[i]; if(thisopt.tagName=="OPTION") { if(!StartScene) StartScene = thisopt; else if(!SecondScene) SecondScene = thisopt; else break; } } if(!StartScene) { alert("no start scene defined for video "+video.id); return; } var sceneStart = getTime(StartScene), sceneEnd; if(SecondScene) sceneEnd = getTime(SecondScene); else sceneEnd = video.duration; StartScene.selected = true; var buttons = {}, bno, elmt, buttonClasses = ['butStop','fstbwdbut','butStepBwd','snailbut','playbutton','butStepFwd','fstfwdbut']; for( bno in buttonClasses ) { elmt = getElement(cur,buttonClasses[bno]); if(elmt) buttons[buttonClasses[bno]] = elmt; else alert("not found:"+buttonClasses[bno]); } var volumeTxt = getElement(cur,'volumeTxt'); volumeTxt.innerHTML = Math.round(video.volume*100)+'%' var volumeRange = getElement(cur,'volumeRange'); volumeRange.value = video.volume; //videopos.style.width = cur.offsetWidth || cur.innerWidth; videos[video.id] = { 'videopos' : videopos, 'scenes' : scenes, 'StartScene' : StartScene, 'SecondScene' : SecondScene, 'curScene' : StartScene, 'curSceneStart' : sceneStart, 'curSceneEnd' : sceneEnd, 'rewinding' : false, 'last_frame_shown' : true, 'rewindTimer' : null, 'rewindStartTime' : null, 'rewindStartPos' : null, 'ratedelta' : 0, 'volumeTxt' : volumeTxt, 'buttons' : buttons, 'video' : video }; //var j=0; for( i in videos ) j+=1; alert(j); video.addEventListener('timeupdate', UpdateTime, false) } var rewind_speed = 1; function UpdateTime(e) { var video = e.target; var rec = videos[video.id]; if( rec.last_frame_shown && rec.rewinding && ( rec.videopos.value < video.currentTime - 25/1000.0 || video.currentTime + 25/1000.0 < rec.videopos.value ) ) { video.currentTime = rec.videopos.value; rec.last_frame_shown = false; } else rec.last_frame_shown = true; rec.videopos.value = video.currentTime; CheckForScene(rec,video.currentTime); if( video.currentTime >= video.duration ) RestoreButtons(rec); } function ShowFrameWhileRewinding(rec) { var timediff = new Date().getTime() - rec.rewindStartTime; var newpos = rec.rewindStartPos - ( timediff * rewind_speed / 1000.0 ); if( newpos < 0 ) { newpos = 0; StopRewinding(rec); } if(rec.last_frame_shown) { rec.video.currentTime = newpos; rec.last_frame_shown = false; } rec.videopos.value = newpos; CheckForScene(rec,newpos); } function StartRewinding(rec,but) { rec.rewindStartTime = new Date().getTime(); // in ms rec.rewindStartPos = rec.video.currentTime; if(rec.rewindTimer===null) rec.rewindTimer = setInterval(ShowFrameWhileRewinding,1000/25,rec); rec.rewinding = true; RestoreButtons(rec); if(!(but.className in savedicon)) savedicon[but.className] = but.value; but.value = String.fromCharCode('0x25AE','0x25AE'); } function StopRewinding(rec) { clearInterval(rec.rewindTimer); rec.rewindTimer=null; rec.rewinding=false; RestoreButtons(rec); } var savedicon = {}; function PlayPause(video,but,rate) { var rec = videos[video.id]; if( rate < 0 ) { var playing = !video.paused || rec.rewinding; video.pause(); if( playing && rec.rewinding ) StopRewinding(rec); else StartRewinding(rec,but); } else { if(rec.rewinding) StopRewinding(rec); if(video.paused||video.playbackRate + rec.ratedelta != rate) PlayVideo(rec,video,but,rate); else PauseVideo(rec,video); } } function RestoreButtons(rec) { var but; for(butid in rec.buttons) { but = rec.buttons[butid]; if(butid in savedicon) but.value = savedicon[butid]; } } function PlayVideo(rec,video,but,rate) { if(video.paused) video.play(); video.playbackRate = rate; rec.ratedelta = rate - video.playbackRate; RestoreButtons(rec); if(!(but.className in savedicon)) savedicon[but.className] = but.value; but.value = String.fromCharCode('0x25AE','0x25AE'); } function PauseVideo(rec,video) { video.pause(); RestoreButtons(rec); } function FrameStep(video,but,step) { var rec = videos[video.id]; PauseVideo(rec,video); video.currentTime = video.currentTime + step; UpdateTime(video); } function RewindVideo(video) { var rec = videos[video.id]; if(rec.rewinding) StopRewinding(rec); else PauseVideo(rec,video); video.currentTime = 0; UpdateTime(video); } function GotoPos(video,newpos) { var rec = videos[video.id]; if(rec.rewinding) StopRewinding(rec); else if(!video.paused) PauseVideo(rec,video); video.currentTime = newpos; CheckForScene(rec,newpos); } function SelectScene(video,selector) { var minsec = selector.value.split(':'); video.currentTime = parseFloat( minsec[0]*60 + minsec[1] ); UpdateTime(video); } function Mute(video,but) { video.muted = !video.muted; but.style.color = video.muted ? 'silver' : 'black'; } function AdjustVolume(video,value) { var rec = videos[video.id]; video.volume = value; if(rec.volumeTxt) rec.volumeTxt.innerHTML = Math.round(value*100)+'%' }