[MUSIC PLAYING]
FRANCOIS BEAUFORT:
You are tired.
I can feel it, so I
will cut to the chase.
Media on the web matters.
And I'm not just saying
it because I deeply
believe in this.
Let me share some
numbers with you.
Almost 40,000 years of video
are watched every day in Chrome.
You may want to take
some time to digest this.
30% of all time on
Chrome for Android
is spent watching video,
and it is about 15%
of all time on
Chrome for Android.
It is huge.
And I'm not the only
one to have noticed it.
Media PWAs across the world
have seen business impact.
Spotify globally
and Ghana in India
are both in great success.
And it's just the beginning.
In the next 20 minutes
or so, Angie and I
will watch you [INAUDIBLE].
We think will help you
build great and modern media
experiences.
We'll cover multitasking with
picture-in-picture, bandwidth
saving with the 81 video codec,
a brand new way of switching
codec seamlessly, and finally,
playback quality predictability
with the media capabilities API.
Let me tell you a little
bit about myself first
and how I work.
I love to multitask
on my computers
doing many thing
at the same time--
browsing, obviously,
writing some code,
sharing news on social
media platforms,
watching educational
videos, and so on.
This is what I do
on a daily basis.
And I'm quite sure I'm not
the only one to do that.
But you may wonder,
Francois, this looks cool,
but are you good at it?
Does that make you
more productive?
This does not matter.
I love to do that, and I
want to be efficient at it.
But it is not always easy.
For instance, watching a
video while I'm coding,
how's that work?
First, I have to open a
separate YouTube window,
move it to a corner
of my screen,
and making sure other
windows are not covering it.
And only then I
can start to enjoy.
But can you imagine
my frustration
when window position
are not remembered,
or when some new window open in
the middle of my little video?
But that's OK.
That's OK, because today,
a brand new web API
called picture-in-picture solved
that very specific use case.
Picture-in-picture,
also known as PIP,
is a common feature
in television
that allows users to watch
video in a floating window
always on top of other
window, so that they
can keep an eye on what they're
watching while interacting
with other sites' obligations.
The BBC website actually
picture-in-picture a month ago.
And they're quite happy with
the early result they got.
Now, like I said earlier,
I like to write code.
So let me show some code.
To enter picture-in-picture
on the video element,
you simply have to call
request picture-in-picture
on a video element.
And because this
call is asynchronous,
it will return a promise that
can either resolve or reject.
And I'll explain why
it can reject in a bit.
The important thing
to notice here
is that the user has to
interact with the page first
to be able to enter
picture-in-picture.
In this example,
I use the button.
Making this button
a toggle button
is quite straightforward.
By checking if the
video element is not
the document that
picture-in-picture element,
in other words, already
in picture-in-picture,
I'll proceed as before.
If it is, let's call document
dot exit picture-in-picture.
Requesting picture-in-picture
may reject for several reasons.
The most common ones being
the video metadata not
loaded yet, picture-in-picture
not supported by the platform,
or simply not
allowed by the user.
The full list of reasons is
available in the documentation.
Updating your website
when the video
is playing in [INAUDIBLE]
picture-in-picture is crucial.
And you may think that
waiting for the request
picture-in-picture promise
to wait is good enough,
but it is not.
What if the video enters
picture-in-picture from
another path, for instance?
What if the user clicked
the browser context menu,
for instance, or the browser
triggered picture-in-picture
automatically, like Chrome
does on full-screen video
on Android?
This is why I strongly
recommend you update your layout
and enter and leave
picture-in-picture event
listeners.
Now, having a 4K video
playing in a small window
may not be what you want.
So to adjust the
quality of the video
based on the
picture-in-picture window size,
you can simply check
the width and height
attribute of the
picture-in-picture window
available in the enter
picture-in-picture event.
Too many
picture-in-picture, I know.
Adding a resize
amount to this object
would also let you know
when to user resize
the picture-in-picture
window so that you
can update the video quality.
By the way, Angie
will work you later
on how to change seamlessly
the video quality
and the codec container
to help with that.
And finally, defining the
availability of your custom
picture-in-picture
button should be
as easy as taking the
Boolean value of document dot
picture-in-picture
enable, but it is not.
Because you want your
website to be perfect.
So you also have to check
if the HTML video attribute
desirable picture-in-picture
is present.
And finally, for
real this time, you
have to check if the video
is actually ready to play.
And only then, you'll get
a perfect implementation
for your custom
picture-in-picture button
in your media player.
I'm glad to say that we have
shaped picture-in-picture API
last month Chrome 70 in
Linux, Mac and Windows.
Chrome OS and Android
are coming soon.
And we're looking forward to
see all the browser vendors
implement this API as well.
You'll find all documentation
and sample units at this URL.
Now, what if I tell you this
is just the tip of the iceberg?
In Chrome 71, currently, beta
will support media stream video
in picture-in-picture.
These two little
lines up java script
do what you think it does.
Your web cam video stream
in a picture-in-picture
in picture window.
And this already makes me happy.
And in case you're wondering,
those are real fake glasses.
[LAUGHTER]
But wait, there's more.
Soon, a brand new web API,
called screen capture,
will allow a website in Chrome
to capture a screen to a media
stream for recording or
sharing over the network.
This API will enable a
number of web applications,
including screen sharing.
Imagine now if you combine these
APIs with picture-in-picture.
Let's have a look at this code.
After getting the
screen, we display media.
And the voice audio stream
will get user media.
I create from scratch
a new media stream,
but contains the
screen video stream,
including my picture-in-picture
window and my voice
as the audio stream.
This code is simply
gorgeous, in my opinion.
This is it.
There's nothing more.
So let me show what this code
does with short demo I've
created just for you.
And can we switch
to demo, please?
So on the left is me,
Francois, the hardcore gamer.
On the right, it's still me, but
this time, as a casual viewer.
And what you're going
to see on the left
is me sharing my
screen, including
the picture-in-picture
window, while I'm
playing the diner game.
So this is me.
Hi, mom.
And let's say [INAUDIBLE].
So like I said, I'm
a hardcore game--
OK, sorry, I lied.
[LAUGHTER]
So keep in mind that
the code you've just
seen with 10 more
lines of java script
involving the media we called
API and some web sockets
are pretty much the
entire code for this demo.
Can we switch back to
the slides, please?
The demo is available
for you on Glitch
if you want to play
with it later on.
Now, some of you may
have noticed that
the picture-in-picture window
contains only two buttons--
a play/pause button
and a close button.
Those are blue there.
We've talked to other
brother vendors interested
in picture-in-picture
about this,
and we're happy to
share that the media
session API we've talked
about last year [INAUDIBLE]..
We'll be using a new feature to
add and customize some actions
to picture-in-picture window.
Think of seat backwards,
forward, previous track,
next track, and even new ones.
If you are already using
it for your mobile website,
this will come for free.
To illustrate these
upcoming possibilities,
imagine a web app that shows the
poster image of a podcast show,
for instance, in a
picture-in-picture window,
all right up, and use
these window media
controls to tailor
the media experience.
I think that looks cool.
And this is coming as well.
So to summarize,
picture-in-picture
is great for multitasking.
And in near future,
it may also be
used to record your
screen with your webcam
or even to create a
custom media center always
on top of other windows that
users can access easily.
I think we all agree
that picture-in-picture
improves a lot the user
experience in general.
And you know what else
improves it a lot?
Video codec.
And talk about this,
let me introduce
Angie Chiang, a software
engineer working
on video compression.
[MUSIC PLAYING]
[APPLAUSE]
ANGIE CHIANG: Thank you.
Thank you, Francois.
Hi, everyone.
I'm Angie from
Google's web engine.
Today, I'm going
to share with you
a new generation video codec,
AV1, was launched recently.
So we have three
main goals for AV1.
First off, we want AV1 to
provide state-of-the-art
compression efficiency
among other codecs.
Secondly, we want AV1 to
be accessible by everyone,
so we made it an open source
project, and it's royalty-free.
Finally, we want to deploy
AV1 right away and quickly.
So before I jump into
how we did or will
do to achieve these goals,
let me explain a little bit
why video compression
is important for users.
To visualize the importance of
compression for video service,
let me give you an example.
Using Edge 264, a five
minute HD-compressed video
will take about 300 megabytes.
On the other hand, the
uncompressed version
will take about 25
gigabytes, 80 times larger
than the compressed version.
This means, without
video compression,
watching video online will
eat up all your internet
bandwidth, not to mention the
sky-rocket costs of storage
on the cloud.
So more compression gives the
user better user experience.
We have seen the proof
of this with VP9,
the predecessor of AV1.
YouTube did a comprehensive AV
experiment when launching VP9.
Compared to H.264, performance
improved in a number of ways,
ultimately result in
higher watch time.
This is due to VP9's
outperforming coding
efficiency.
With AV1, we have done it again.
A new generate codec
provides 30% [INAUDIBLE]
reduction over VP9 across a
variety of video qualities.
This means, given the
[INAUDIBLE] quality,
AV1's video size will
be 30% smaller than VP9.
This project integrated
over 100 algorithm tools,
including the technologies
from open source projects,
like Mozilla's
[INAUDIBLE] project
and Cisco's SOAR project.
And again, AV1 is an open source
project, and it's royalty-free.
So its development community
alliance for open media
has attracted 40 companies
to join and to contribute
their technologies into AV1.
There are content providers,
streaming service providers,
and power companies which
cover a wide spectrum
of the ecosystem for
video on the web.
Having Google, Apple,
Microsoft, and Mozilla
means we can have AV1 to work
everywhere on the web platform.
So we have been working
hard toward this goal.
In this quarter, AV1 decoder
is supported by Chrome browser.
And we started sorting
video content from YouTube.
Firefox and Edge are also
launched in the beta platform
last month.
And we plan to integrate
the AV1 with web RTC
and to deploy AV1
into Android platform
in the following years.
[INAUDIBLE] support is
also under development,
first arriving in 2020 for
TVs and mobile handsets.
For web developers,
using AV1 is easy.
You simply need to code its type
supported function to make sure
the browser support AV1.
Then you can start
play the video.
So AV1 is very exciting.
But rolling out AV1 is not easy.
To ease the process of
the codec transition.
a new change type
function is supported
on Chrome browser, which allow
the browser to switch one
codec to the other seamlessly.
So for example, when
AV1 is not performing
on some low-end devices,
the string service
can always fall back to
VP9, which is less complex
and may have
[INAUDIBLE] support.
Here is a simple call
for using change type.
So initially, we add a
source buffer using VP9.
And later on, we can call
a change type function
to switch to AV1.
Let me give you some
demo about change type.
So we start playing
video using Edge 264.
You can see that from
the top left corner,
by clicking the button,
we can switch to AV1.
Now, we are using AV1.
By clicking the
button again, now, we
switch back to Edge 264.
You can't tell there's
a transition, right?
I think this is really cool.
Let's go back to the slide.
This is a support slide,
where you can find information
about AV1 decoder
and change type.
Please, welcome back
on stage Francois,
who's going to talk about
playback predictability
with media capabilities API.
[APPLAUSE]
Francois.
[MUSIC PLAYING]
Thank you.
FRANCOIS BEAUFORT: Thank you.
Thank you.
Thank you.
Did you ever wish you
could predict the future?
I know I did.
Be some kind of fortune teller.
I personally would love
to jump into my future,
have a quick look,
and just come back.
Out of curiosity, I
asked people around me
what they would do with
this kind of power.
Some said they would
love to know if they
would become grandparents.
Some would like to see
if their project is
going to be successful.
And some obviously
would simply look
for upcoming lottery numbers.
Now brace yourself.
In a sense, the
media capabilities
API allows you to
predict the future,
but only for a media
playback on the web.
Until recently, web
developer had to rely
solely on web API such
as is type supported
or can play type to discover
whether media could be decoded.
While this told them whether
media could be played at all,
it didn't provide an
indication of whether the media
playback would
drop lots of frame
or rapidly drain device battery.
In the absence of
the signal, developer
had to create their
own heuristic,
or just assume that if
a device can play back
a combination of
codec and resolution,
it could do so smoothly
and power efficiently.
For users with less
capable devices,
this often led to a
very poor experience.
By using the media
capabilities API today,
you can get more information
about the client's
video decoded performance
and make an informed decision
about which codec and resolution
to deliver to the user.
In other words, it helps
ensure adaptive video streaming
[INAUDIBLE] selection that
will play back smoothly
onto the specific device.
Here's how it works in Chrome.
The media capabilities API use
metrics from previous playback
to predict whether future
playback in the same codec
and at the same resolution
will be smoothly decoded.
When you ask this API about a
specific media configuration,
it will return asynchronously
three Booleans.
Is this configuration supported?
This is the same result
returned by these types of body
you can use it for
instance to detect
whether everyone video correct
is supported by the way
easily by going to be smooth?
It is currently true if
less than 10% of frame
have been previously dropped
for this media configuration.
Is playback going to
be power efficient?
Am I basically going to
drain the device battery?
If true, if more
than 50% of frame
have been decoded by the
hardware for this media
configuration.
Now, warning, this is not
some kind of magic API
that will tell you what to play.
You are in control and
have to make decisions
about which media configuration
to play eventually based
on the result of this API.
Speaking of results, YouTube
experimented with the media
capabilities API to prevent the
adaptive [INAUDIBLE] algorithm
from selecting a resolution
that a device could not
playback exclusively.
For users who were part
of the experimental group,
the mean time between
rebuffers, also known as MTBR,
increased by 7.1%.
While the average
resolution served
to the aggregate group
measured by video height
only declined by 0.5%.
These results, as obscure
as it may be to some of you,
basically show that some
users on low-end devices
saw less frequently on YouTube
the frustrating buffering
animation.
So just for that, thank you.
The media capabilities
API is available today
in Chrome, Firefox, and
Safari Tech Preview.
As you saw, you'll find all
documentation and sample
you need at this URL.
Now, if you remember
one thing from this talk
about those features, it is
[INAUDIBLE] the web matters.
And the web platform
is the best place
to serve efficient and
delightful major experiences.
One more thing-- you can
find all audio and video
updates in Chrome by simply
searching for Chrome media
updates on your
favorite search engine.
This will allow you to stay up
to date with the amazing media
features that Chrome is
adding to the web platform
every release.
And with that, I humbly
thank you for your time.
[APPLAUSE]
[MUSIC PLAYING]
