- [Todd] Hello everyone, and welcome
to the Center on
Technology and Disability's
latest assistive technology webinar.
Today we are looking at some
new technology possibilities,
with presenter Jaclyn Wickham, a teacher
turned technologist and
founder of AcclimateVR,
an ed tech startup
developing virtual reality
based learning modules to
help students on the spectrum
develop independent learning skills,
practice social skills,
build safety awareness,
and navigate challenging
community locations.
Today, she will present on the many ways
virtual and augmented reality can offer
new learning opportunities
in the classroom.
We'll let her run through
her presentation first,
though feel free to ask questions
via the chat box as we go.
We will have a Q and A session at the end,
and make sure we get as many
questions answered as we can.
Just keep in mind CTD always records
and archives the webinar,
PowerPoint and resources
on the CTD website by the following day.
If you're looking to revisit
the webinar and materials.
Welcome everyone and let's get started.
Jaclyn, the floor is yours.
- [Jacyln] Hey everyone.
My name is Jaclyn, and I am an education
technology specialist and
founder of AcclimateVR.
As Todd mentioned, we are at AcclimateVR,
we are an ed tech startup
focused on creating
virtual reality content learning
modules for students with
autism and other
developmental disabilities.
So just to give you a bit of background
as to the journey that kind of
led me to found AcclimateVR,
I have a background in education.
I started as an elementary
teacher, and then transitioned
into education technology
and instructional design.
So I transitioned into
ed tech after teaching
elementary school for about five years.
And realizing, after
realizing the benefits
of technology in the classroom.
And as an ed tech specialist,
I had the opportunity to travel to schools
across New York City, helping teachers
implement technology
into their instruction.
And it was during that time that I became
really interested in ed
tech and app development,
and ed tech entrepreneurship.
I had a lot of ideas for educational apps.
So I ended up at a graduate
program called ITP over at NYU
which was essentially a coding boot camp,
where I learned to program,
and I had the opportunity
to explore a lot of new
and emerging technologies,
like augmented and virtual reality.
And while I was there, I
ended up taking a course
on developing assistive technology,
and that's when I really started to become
really interested in assistive tech.
And I ended up teaming up
with an occupational therapist
there for a couple of
projects in that course.
And through our collaboration,
was born the idea for AcclimateVR.
So it started as my
graduate thesis project,
and now has sort of grown
into a full-time passion.
So our mission at
AcclimateVR is to leverage
virtual reality technology
to help students
with autism and other
developmental disabilities
experience real world situations
and develop everyday living
skills in a safe environment.
And our VR teaching tools are designed
to help children and teens
on the autism spectrum
develop independent living skills,
practice social skills,
build safety awareness,
and navigate challenging
community locations.
And so we're focusing on
these four different areas,
and I'll elaborate more on
this later in the presentation.
But first I'd just like to start
with a quick agenda just to let you know
what we'll be covering in today's webinar.
So I wanted to start out with just talking
a little bit about the
difference between VR and AR.
Because we will actually
be talking about some
augmented reality applications
as well as virtual reality today.
And I want to also make
sure that everybody's
comfortable with those definitions,
and talk a little bit about
the different types of VR.
And then we will transition
into talking more
about the practical applications of VR
in the assistive technology space.
And then I'll tell you a little bit more
about AcclimateVR and what we're building,
and what we're learning
from our pilot studies.
And at the end, we'll
transition into talking
about augmented reality and
some specific applications
in the assistive technology space.
So to start out with, I'd love to,
as I mentioned, we're just gonna start
about talking a little
bit about VR versus AR.
So I'd like to just open up the,
take a quick poll, and
open up the chat box here.
So hearing from you, from the audience,
what do you think the
difference is between AR and VR?
I'd love to hear your thoughts first,
before we jump into this section.
I'll give you just a few
minutes to answer the poll.
Okay, so we've got some answers coming in.
Some people are not sure at
all, which is completely fine.
We're gonna talk about the
difference in just a minute.
Okay, some of you seem to be pretty clear
on understanding of one
being totally immersive
and the other partially immersive.
Right.
Okay, so so far, it looks like
we are about half and half.
Some of you seem pretty
clear on the definition,
and some of you aren't sure.
So it's a good thing that we're gonna stop
and just talk about the difference before
we jump into looking at
different applications.
So thank you for those of
you who submitted a response.
So this is actually an
infographic by Earthly Mission.
And it does a really
nice job of explaining
the difference between
augmented and virtual reality.
So the way to really think about it is
to kind of think of the difference
between the definition
of augmented reality,
we think of augmented it
means improved or enhanced,
whereas virtual means
something that's not physically
existing but that's made
by software to appear so.
So AR is the blending of
virtual reality and real life.
And so developers will
actually basically create
images within app locations that blend in
with the contents in the real world.
Whereas virtual reality
is all about the creation
of a virtual world that
users can interact with,
and it's closed and fully immersive.
And so you can think of
it as augmented reality
essentially puts virtual
things into users real worlds,
augmenting them, whereas VR puts
users inside virtual
worlds, immersing them.
So, in AR, the user still continues
to be in touch with the real world,
whereas with VR, the user
is completely immersed.
And you, I'd like to just take
a look at some examples of both,
just so that that
distinction is really clear.
You are all probably familiar actually
with some common applications
of augmented reality.
If any of you have ever been on Instagram
or SnapChat, you will have probably seen
some of these face filters.
This is a perfect example
of augmented reality.
You can see the image of yourself through
your smartphone's
camera, and then you have
the ability to add on these
sort of silly face filters.
So you're overlaying digital elements
onto the real world, augmenting it.
And just another example here.
If some of you may remember
the Pokemon Go craze
from a couple years
ago, and that's another
perfect example of augmented reality,
where we have the ability
to point your smartphone,
use the camera on your smartphone
and you will see that there are these
sort of digital avatars layered
onto the real world scene
there, augmenting your view.
And you may also be familiar with,
there are a lot of different
AR apps for education
like this one, where this one uses,
it's essentially like anatomy flash cards.
And they superimpose small
images on the normal field
of vision to help
students learn vocabulary.
So that's another example
of augmented reality.
Now, virtual reality, on the other hand,
you can see that the user wears a headset.
Sometimes you'll hear it called
googles or a head-mounted display,
but the point is that
they're very isolated
or immersed in a virtual world.
So virtual reality is simply
an artificial experience
that kind of tricks
your mind into thinking
that it's somewhere else
by stimulating your senses.
So it transports you to another world
that can be experienced or interacted with
through the use of sensory devices.
And depending on the type of headset
that you are using, sometimes you can,
users can actually interact
with the virtual world
using hand controllers or
buttons on the headset.
So just another quick
poll out of curiosity.
I'm curious if any of
you have ever tried VR,
and if so, which headset you've tried.
I'll give you just a minute
to see the responses come in here.
So we'll start with the yes and no.
Okay, so it looks like,
still coming in here.
Okay, so we have, it seems that quite
a few of you have actually tried it.
We also have some who have never tried it.
And it looks like we're,
I do see some answers coming
in on the chat box as well.
We have, okay, it looks like we've had
some people try the Oculus,
Google Cardboard, oh great.
Okay, yeah, Second Life,
somebody asked if Second Life counts.
That would absolutely count.
That's a perfect example
of a computer-generated VR,
which is actually what we're about
to transition into in just a second.
Samsung Gear, great.
So yeah, you'll see there
are a lot of different
VR headsets out on the market right now.
And they all offer different features.
Google Cardboard is essentially the most,
the easiest to get started with,
as it's just basically a
cardboard set of goggles
that you place your smartphone into.
And then there are some more advanced
headsets like the Oculus
Rift out there as well.
Most of the examples here use mobile VR,
where you can just place
your phone into the headset.
Okay.
Great, that's really helpful.
Thank you for those of you
who submitted a response.
The reason I was asking is I was curious
just to see if the types of
VR that you've tried out,
and so if you've ever tried VR,
as it looks like some of you have,
there are, you've probably experienced
one of two different
types of virtual reality.
So there is a difference.
There is what we call
computer-generated VR.
and computer-generated VR environments
can be created in two different ways.
I'm sorry, or basically,
are created by computers,
and they can look and
feel like the real world.
Such as a flight simulator,
or they can be completely artificial.
Think of an imagined distant planet.
But any case, they are computer-generated
simulations created by
computer, and you can think of it
almost like a video game type atmosphere.
And Second Life, which somebody mentioned,
is a perfect example of
computer-generated VR.
And often in computer-generated
VR environments,
you'll see different
avatars that look like this,
that the user can interact with.
So there's also another, you
may have also heard of VR,
360 degree images and videos referred
to as virtual reality content.
And this is basically immersive
or spherical videos or images.
They are recordings where a view
in every direction is
recorded at the same time,
shot using an omnidirectional camera
or a collection of cameras.
You can see some examples
here on the slide.
And 360 degree video is typically recorded
using either a special
rig of multiple cameras
or using a dedicated camera that contains
multiple camera lenses
embedded into the device
and then filming overlapping
angles simultaneously.
So you can see here, this is an example
of a video captured by
one of these 360 cameras.
So it starts out,
you can see the video footage is recorded
with a fisheye lens, and
then through a method
known as video stitching,
basically they stitched
the footage together
into one spherical place.
But I wanted to show you
a quick example of that,
just so you get an idea of the difference.
Sorry, let me see if I can.
We'll go through this
way, sorry about that.
The,
you can see here, this
is where it's converted.
And the end result will
look something like
this, where you'll get a video,
(static crackling)
and it's often referred to as VR,
as if you were watching this
content through a VR headset,
you can imagine that it would,
if you were able to turn your head
and look in all different directions,
it would look like you were actually
at this real world scene.
So we've talked a little
bit about the differences
between VR and AR and the types of VR.
So I'd like to jump into talking about
what most of you are probably here for,
is to hear a little bit more about some
of the practical applications of VR,
specifically in the
assistive technology space.
Both of these technologies, VR and AR,
have the potential to
create new applications
that support people with disabilities.
So often we hear about virtual reality
games and advertisement, but VR really has
a huge potential beyond just
its entertainment value.
It's making a huge impact
in the health sector.
Examples include cognitive
behavioral therapy,
exposure therapy, treating
PTSD, treating phobias
and anxiety disorders, as
well as stress and meditation.
And it's also being used as a way
to help people with disabilities
explore the world that might be difficult
or impossible in real life.
So I'd like to talk a
bit about some specific
examples of applications, and also a bit,
a bit about the research
behind VR therapy.
I should mention before
you jump into this section
that a lot of my research,
my personal research
and experience has focused
on how VR can support
individuals and children and
teens on the autism spectrum.
Which is mainly what we'll be
looking at examples of today.
But please keep in mind
that many of these examples
and applications that we will be
looking at could also support people
with other intellectual or cognitive
and developmental disabilities
in addition to autism.
So we'll be kind of looking
through a narrow lens today,
specifically at examples to
support people with autism.
But just keep in mind that a lot of these
could be used or applied more broadly.
So, why consider using
VR therapy for autism?
First of all, it creates a safe
and controlled environment.
So as a lot of you probably know,
chaotic situations can be extremely
stressful for those on the spectrum.
And they sometimes tend to have trouble
ignoring irrelevant
stimuli, and can easily
get overwhelmed in unfamiliar places
that are full of new
sights or sounds or smells.
So a VR learning environment
is much more controlled.
The student can enter and leave at will,
simply by putting on or
taking off the headset.
So there's a low risk of overstimulation.
And learning is much more productive
under low stress conditions.
So not only are the students
much more comfortable,
but they're also more
likely to make progress.
This second point here is a big one,
accessible practice and
unlimited repetition.
Opportunities to practice
social interaction
are much more limited for
students with autism often,
than for their neurotypical peers.
And going out into the community
can often be really difficult,
because more supervision is required
to ensure their safety.
They also get, sometimes
can get less exposure
to everyday social situations in school
if they're separated from
the mainstream classroom,
either deliberately or otherwise.
But in contrast, VR can be
at their fingertips anytime,
both at school and at home.
And it also lets them
practice these same situations
as many times as they want,
in a safe environment,
over and over again, until that skill,
until they're comfortable with the skill.
And that's something that doesn't
always happen in the real world.
A couple other reasons to consider
using VR for autism, it
can create an individual,
I'm sorry, I skipped ahead there.
There's an emphasis on
visual and auditory cues.
And sometimes people with
autism will tend to respond
better to sight and sound than touch.
So VR can be a great match.
The ability to create
individualized experiences.
So, as many of you know, autism includes
a wide spectrum of symptoms.
You know, it can be from slight
social and intellectual differences
to a total inability to communicate.
And not every student needs
the same kind of teaching.
So VR allows for a complete
individualization for each student.
And the therapist can
select the simulations
that are right for the student,
and the simulation itself can respond
according to what each individual does.
So that no two students will necessarily
have the exact same experience.
And as I'll mention, I'll
mention these two in just a bit,
when we get into the research.
But it's also, VR is also
beneficial for people with autism,
as a lot of them have shown preference
for computer interaction,
and it also allows non-verbal
interaction possibilities.
So we'll get into that
in just a moment here.
So because of all of these considerations,
which we just spoke about,
academic researchers,
as it turns out, have
actually been investigating
the use of VR as a teaching tool
for individuals on the autism spectrum
since the 1990s actually,
when VR head-mounted displays
were huge and clunky, and
tethered to a computer.
And I'll show you kind
of what they used to,
they used to look a lot like this.
So,
and the reason that researchers
have been exploring,
as I mentioned is kind of intuitive,
kids with autism need to practice
basic life skills in the
real world, and it's hard.
They need a place to go,
someone to take them there,
time to learn and support.
With VR, we can make a world
that's convenient and controlled,
and perfectly designed
for learning some of
these essential skills.
So one of the first
studies on the use of VR
with individuals with autism was conducted
by a pioneering researcher
named Dorothy Strickland,
and this is actually just a
quick writeup of her case study.
They were, sorry I'm
hearing some feedback there.
Okay.
So, one of the, as I mentioned,
this is one of the first studies.
And they, in the study,
they examined whether
children with autism could
tolerate and respond to VR.
So they had two children,
age seven and nine,
both mild or moderate on the spectrum,
and they used VR headsets that displayed
a computer generated simulation
of crossing the street.
So this is a perfect example of what
we were talking about
earlier, which would be
computer-generated VR, you can see
it's sort of a video game-like atmosphere.
And the results were
actually really promising.
The students were able
to wear the headset,
which of course was much
bulkier and heavier.
And I can actually show you just another
quick picture here from the same study.
You can see
right here, it looked a lot like that.
So the fact that the
students were able to,
to wear that was pretty impressive,
considering what they looked
like and felt like back then.
Because of course it's
probably natural for everybody,
most people typically wonder
if students with autism
would experience any
negative sensory effects.
So they were able to wear
it and verbally identify
the objects and walk around
the computer generated world.
But it wasn't really
clear whether the children
would translate or
generalize their experience
to a real street crossing situation.
So there's still some research
to be done in that area.
And there have been a lot
of studies in between.
But more recently, we have,
yeah, I just saw the
question come in about
what about vertigo or dizziness,
which is a great question.
It's natural to ask that,
especially considering
some of, that some students
have some sensory issues.
And it depends on the
headset that you're using.
Some people do experience vertigo with VR.
But the study that I actually
have pulled up right now,
that was one of his focuses.
So this is Dr. Nigel Newbutt,
from the University of West England.
And the main focus of his pilot
study was to basically test,
to test whether the students were able to,
or I'm sorry, the participants were able
to wear the headsets and,
so they tried it with
the Oculus, Oculus Rift.
He tested it with 29
individuals with autism.
And he did it in two phases.
So the first phase had a virtual
scenario for about 10 minutes.
And the second phase was 25 minutes.
And the scenarios were computer-generated,
so you can see the graphics
looked something like this here.
This is one of the
graphics that he showed.
And the participants for the most part,
as I mentioned, there were 29,
four of the participants
were unable to complete
the first phase because of
dizziness and tiredness.
Which was the question that just came in.
But most experienced no
negative physiological effects.
So overall, the group reported
a strong sense of presence and immersion.
And I'll just come back here for a second.
So, across, there have
been as I mentioned,
there have been a lot
of studies in between.
I showed you one of the oldest studies
and one of the most recent.
But basically, across
technologies and studies,
study participants have demonstrated
their ability to engage with simulations
with actually very few negative
emotional or physiological reactions.
And beyond that, VR learning experiences
have been shown to produce learning gains
measured not only by
tests of social judgment,
but also by assessments of activities
performed in the real world.
There is a lot more work
to do in order to find out
if students are actually transferring
or generalizing these
skills to the real world.
But if you're interested to learn more
about some of these studies,
you can actually go to our website,
acclimatevr.com/research,
and you'll see a form there
that will ask you to sign up for our blog.
If you sign up for the blog,
you'll hear about all the different,
different things we're working on
and building at Acclimate,
and what we're up to.
And once you sign up for
that, you can actually,
you will be sent a free downloadable
version of our Research Roundup.
So we basically combed all of the research
that's out there about VR and
autism, and did essentially
like a literature review
of all the research.
So if you are, if you are
interested in learning more,
please feel free to head
over to our website,
and take a look at our Research Roundup.
So as I mentioned, research has shown
that virtual reality is very promising,
and has a lot of potential as a tool
for therapists to be able to administer
virtual reality exposure therapy
in a safe and controlled manner.
But we haven't really,
it hasn't really been
widely available to
date until recent years.
So, you know, as before, like I said,
they looked a lot more like space helmets.
But in the last few years,
we've seen huge advancements
in the technology.
It's cheaper than ever.
And you now only need something as simple
as a Google Cardboard
and your mobile phone,
which is a $12 viewer, and your phone,
to be able to experience it.
So the first time ever
VR is actually becoming
a reality in school settings.
So I'd like to talk about
some of the applications
that are being used, how it's being used
in school settings specifically,
and also to of course support people
with autism and other disabilities.
So,
sorry.
So these are some of the
specific applications
of VR in the assistive tech space.
It's being used for joint,
to work with students
on joint attention skills,
sensory-based experiences,
exposure therapy to reduce anxiety,
social stories,
community-based instruction,
social skills, executive functioning,
daily and independent living skills,
safety awareness, and social modeling.
And I have a few examples of each.
Not of each, I'm sorry, a few
examples of some of these.
But I will also show you what
we're building at AcclimateVR,
which incorporates a lot
of these different skills.
So the first example I
wanted to just show you is,
let me just pull this up,
so this is actually, where this plays,
this is a startup called Floreo,
and they are DC-based, I believe.
And they are exploring, doing a lot
of exploration with computer-generated VR.
And you'll see this is actually an example
of an activity to build
joint attention skills.
So for those of you who don't know,
joint attention is an early developing
social communicative
skill in which two people,
usually a young child and an adult,
use gestures and gaze to share attention
with respect to interesting
objects or events.
And that can sometimes be really difficult
for kids with autism.
So I'll just show you this quick.
This is I think one of
their early prototypes.
And you'll see that this is
computer-generated VR here.
- [Woman] There's an elephant over there.
All right, let's see.
Let's play with one of the animals.
- [Emma] Wow, I wonder
what's gonna happen.
- [Woman] All right, let's look at Emma.
What's the elephant doing?
- [Emma] Neat, look at that!
- [Woman] Nice job watching
the elephant with Emma.
- [Jaclyn] Okay, so that's just,
again, if you're interested
in learning more about them,
they're called Floreo, and they're doing
a lot of cool things with
computer-generated VR.
And I think a big, very
big possibility for VR,
which we're actually
working on at AcclimateVR
is taking social stories.
A lot of you, especially if
you're an occupational therapist
or work with students with special needs,
a lot of you have seen these
types of social stories.
And imagine being able to
take something like this,
which students typically
use to learn through,
you know, these paper social stories
and scaffolding skills or
working through role playing.
But imagine if we could take this
and actually make it into a
virtual reality experience
where they actually see
a view of the railroad
and being able to go over the crosswalk,
and listen for cars, and be able to hear
the environmental sounds.
So,
the ability, so social
stories is a big one
that we're seeing different
uses of for VR applications.
So, one other really interesting
use that's kind of a little bit different
is this video, which was put out by the,
let me just pull this up here,
Autism TMI Virtual Experience.
And this is posted or published
by the National Autistic Society.
And basically what they
did was they created
a virtual reality experience
using 360 degree video
to show other, to show people
what it's like to have autism.
So they tried to recreate the experience
and show what it would be
like to have sensory overload.
So it was kind of interesting.
I'll show you that here.
- [Woman] I'm just gonna
get a ticket, okay?
- [Jaclyn] So we can't slow it down.
(static crackling)
(panting)
So you can imagine that if
you were wearing a headset
and watching that, it would
be a really interesting way
to explore that perspective
of somebody with autism.
Obviously, that's a pretty extreme piece,
but I thought that was
an interesting example.
So those are just a few, a few examples
of some of the work that's being done
with computer-generated VR.
And now I'd like to transition
into AcclimateVR's approach,
and talk a little bit about what
we're building over at AcclimateVR.
So, I'd like to start by
just talking a little bit
about my main inspiration for Acclimate,
and my inspiration for building Acclimate.
As I mentioned, it started
as my graduate thesis.
And it all kind of started
over a conversation
I was having over coffee one morning
with a colleague of mine, Kaitlin.
And she's an occupational therapist.
She works with students with
autism on a daily basis.
And she was telling me about her morning,
which she had spent at the subway station
across from her school with a group of her
middle school students,
teaching them how to buy
and swipe a Metrocard, read a subway map,
ask for help if they're lost,
and basically how to safely ride
public transportation to and from school.
And this is a normal morning for Kaitlin.
Trips like these occur on a weekly basis
and vary in location,
ranging from subway station
to the cafe or a
restaurant down the street,
to the local grocery store.
Interacting with the cashier,
or ordering from a restaurant menu,
or going to a doctor's office,
these are all skills that
many people take for granted.
But for children and adolescents
on the autism spectrum,
developing independence
and building the skills
that go along with it can
be hugely challenging.
And for many of the spectrum,
it's not so easy to pick up things
on the fly just by observing others.
Additionally, many students with,
the nuances of navigating
unfamiliar places
and interacting with strangers are not
always intuitive or self-explanatory.
So additionally, a lot
of students with autism
have challenges with sensory processing.
So, for a child who is
oversensitive to stimulation,
an ordinary trip to the supermarket
or a neighborhood can be
very anxiety-inducing.
So what Kaitlin's doing when she's taking
her students to these community locations
is called, it's a very
widely known instructional strategy
called community-based instruction.
So like many, it's used by a
lot of occupational therapists
and educators who work
with students with autism.
And so it's an evidence-based
instructional strategy
that helps students develop
functional independence
as they approach the
transition into adulthood.
So evidence shows that the
best way to help students
develop skills to become more independent
is to take them out of the classroom
and into the real world,
or the natural settings
where a skill is ultimately
expected to be demonstrated.
And a lot of times these field trips,
sorry, not field trips, these
trips are very structured.
And by no means a field trip.
As you can see from the
data collection form above.
So, Kaitlin the occupational therapist,
she invited me to observe one of these
community-based instruction trips.
And it was really interesting to watch.
The therapists each had a clipboard
and a Velcro wallet for
each of their 15 students.
And as we walked around, they worked with
individual students on specific tasks,
based on their individual
goals in their IEPs,
keeping track of how much
prompting each student needed.
But what I learned from Kaitlin,
and from numerous
conversations and surveys
and interviews with a lot of
other occupational therapists,
is that community-based instruction
presents a lot of challenges.
One of the biggest challenges is access.
Because a lot of schools don't have access
to community places nearby where they can
take students to practice.
So what I saw at some
of the schools I visited
were rooms like these
with mock grocery stores
or cafes where students could practice
the skills through role playing.
And time is also a big
issue, as many sessions
only last for 30 minutes.
So it's not always an option time-wise
for therapists to take students
out into the community.
Safety is also a concern.
One therapist told me that sometimes
what they'll do is stand 15 feet back
around the corner from the student
to see how the student
will do independently
without the teacher or the therapist
standing over their shoulder.
But they do have to worry about safety.
And lastly,
logistics, organizing the
trips is quite a process.
It requires permission slips,
and reimbursement forms,
and partnerships with local
businesses in the community.
And it can really just be
a logistical nightmare.
So Kaitlin, the therapist,
and I began kind of thinking
about how we might address
some of these challenges.
And we thought what if we could create
an immersive environment to simulate
a physical presence in the
real-world community settings,
and allow students to repeatedly
practice some of these targeted skills
within a safe, controllable environment.
And what if they could
have repeated exposure
to real world places and situations
before going out into the community
without real world consequences.
So that's when, which led us to consider
virtual reality as a platform.
And with VR, students can practice
far more often and with less stress,
gradually building towards readiness
to go out into the world
and try their new skills.
So after, after we did the research,
I submitted an institutional
review board proposal form,
and we ended up doing a usability study
at the Manhattan Children's Center.
It's a school that provides education
to children with autism spectrum disorder.
And throughout the development
and design process,
I had access to and
was able to collaborate
with a team of five
occupational therapists
throughout the design and development.
And so the OTs were involved
in every step of the process.
And what we ended up with was...
We tried to figure out
which community setting
would be best to build out
for our initial prototype.
A cafe, or a grocery store.
And we ended up deciding
on a grocery store,
just because the shopping experience
requires students to use
so many different skills.
And here you can see a chart
of all the executive functioning skills
which can be applied to the
experience of a shopping trip.
And our next step was to figure out
what these instructional
scenarios would look like.
So over and over again,
throughout the surveys
and interviews, we kept
hearing how important
it was for virtual experiences
used during therapy
to mimic as closely as possible
what students would hear and
see in real world settings.
And while computer-generated simulations
allow for a wide range
of user interactions,
they don't always capture
the visual intensity
or the ambient noise
or environmental sounds
or chaos of the real world.
And that's exactly what a lot of students
with autism struggle with so much.
So we decided to try something different.
As you've probably noticed,
a lot of the examples
that I've shown you so far
were computer-generated VR.
But we wanted to try using
360 degree images and videos
so that students, we could
actually prepare learners
for the conditions that they
would encounter in real life.
And complete with real people
and environmental sounds.
So you can think of it almost as,
what we're building at AcclimateVR
is almost like immersive video modeling.
Many of you may be familiar with
the concept of video modeling.
It's a well-known therapy technique
that has been shown to help teach students
communication, social,
cognitive, and play skills.
And basically we're taking that concept,
and making the video immersive.
And so I can show you
just very quickly here
what some of the video footage
that we took looks like.
So you can see here
that with the 360 video,
the student is able to see
all the way 360 degree view,
and hear the sounds of the grocery store,
and see the people in the grocery store
to get used to that environment.
So it's almost like exposure therapy.
And we wanted to take it a step
further, the video modeling,
and unlike traditional
video-based modeling,
which is often used as a tool
for students with special needs,
we wanted to give the
opportunity for students
to actively participate in tasks
within the 360 degree video scene.
So what we did was we built
in embedded assessments
into each VR experience
to provide opportunities
for practice and to check understanding
of the student's targeted skill.
So I'm just gonna show you very quickly
what those embedded assessments look like.
So here you can see a user testing session
that we had with a student with autism.
And you'll notice that he is presented
with a sort of check for understanding
as he's watching the videos.
So I'm just gonna turn the volume up here.
- [Narrator] Look down to see your cart.
(loudspeaker blares)
Check your list.
Do you need anything in the bread section?
- [Woman] Great job
getting the wheat bread.
It's crossed off your
list, and in your cart.
- [Cashier] Great, you have a great day.
- [Narrator] Now, it's your turn to pay.
- [Jaclyn] Okay, so you
can see from that example
where the student was
working on finding items
on the grocery list, that he was presented
with positive feedback when
he made the correct choice.
And the bread was suddenly in the cart.
So that was a check for understanding
to make sure that he was understanding
the skills that he's being taught.
Another feature
that we wanted to
include, based on surveys
with therapists was the
ability for teachers
or therapists to see what
the student is seeing.
So when a student participates
in a VR experience,
the therapist or teacher has the ability
to observe the student's experience
mirrored on a separate device's screen.
And this feature allows educators
to track whether students are meeting
learning objectives for targeted skills,
and provide feedback to
students in real time.
So I'm just gonna show you
one more usability testing,
and we'll quickly look
at an example of AR,
and then we'll get to questions,
I know there have been
some questions coming in.
So in this case, this is an
actual student user test.
And you can hear, you can actually
hear the student's responses.
So listen carefully, and you'll hear
the student actually
interacting with the cashier.
Which indicated to us
that he was definitely
experiencing a sense of
presence or immersion.
And this is another example
of one of the embedded
assessments that a student would see
within the 360 degree video scene.
This skill was working
on paying at the cashier.
- [Cashier] Thank you very
much, you have a great day.
- [Narrator] Now it's your turn to pay.
- Hi, How are you today?
- [Student] I'm doing okay!
Yeah, I have one friend.
- Great.
(scanner beeps)
(scanner beeps)
- [Student] Apples.
- All right, your total is $18.36 please.
- [Student] Okay.
Bye.
- [Narrator] Your money from your wallet
is on the counter.
How much will you give the cashier?
$1 is not enough.
Try again.
$5 is not enough.
You need to give the cashier more money.
- [Student] 19.36, $20.
- All right, out of $20, thank you.
Okay, 64 cents is your change.
And here is your receipt.
Thank you very much.
Have a good day now.
- [Student] I did it!
- [Jaclyn] So you'll see
the student's very excited
at the end after completing
the task successfully.
So we're running at that sharing time,
so I'm just gonna skip ahead
to the quick example of some AR.
But just to give you an idea of what
we're working on for the
future before we do that,
we're looking at actually being able
to differentiate the VR experiences
by controlling some of the sensory inputs.
So for example, some students might see
just a still photo with no noise,
or a list with pictures
or without pictures,
you receive more prompting
or less prompting.
So that's something that's on
the roadmap for the future for us.
And we're currently
doing some pilot studies.
And if you're interested
in learning more about
what we're learning from
those pilot studies,
you can check out our website
on www.acclimatevr.com.
And as I mentioned, we're
running a bit short on time,
so I'm just gonna skip over to AR.
So we talked a little bit about
the difference between AR and VR.
And I wanted to just look
at a couple practical
applications of VR in the
assistive technology space.
So this is a really interesting project,
called the Color Black Moment.
And you'll see it's used to increase
emotional responsiveness in children.
So I'll just show you really quickly,
I won't show you the whole thing.
But just to give you an idea.
It's a virtual reality cube that you place
your camera on the device.
- Hi my name's Kevin
Child, CEO for Color Black.
And I'm gonna show you
my software, Moment.
It works off this thing right
here, called a Merge Cube.
You can get it on Amazon
or Walmart for only $15.
It comes with little mini kid
games in augmented reality.
And my wife and I created the
first professional app for it.
Me being a software developer,
my wife being a school psychologist.
I'm gonna launch it right here on my iPad,
but it works on any ILS
and active link device.
There's three tabs on here,
and the first one is emotion.
This is for autism and mental health.
And all you do is just put the cube
right in front of it, and now you have
all the emotions in the palm of your hand.
And so now it becomes a
straight talking piece
and you can engage with each emotion.
So you can go right here
and you can talk to sad,
and you can figure out,
hey, why is he sad?
And as you see from the side,
I'm using the unique
movement of the cube itself.
And while we're working on sad,
if you want to divulge even further,
you can click on it, and
it's six sub-emotions
of that main emotion.
So you have more, you're sad
because you're in front
of a lost dog sign.
You're sad, is he being
bullied or talked about?
You're sad, maybe it's self-image.
He's looking at himself and doesn't
feel comfortable and he's sad about it.
All of these are straight talking pieces.
- [Jaclyn] So that's a really interesting
project to check out, if
you'd like to learn more,
you can check out their website.
And the last example I'll
give, this is actually
an article that we came
across by Craig Smith.
And he was talking about the potential
for what's called the ARKit.
And if you're not familiar with the ARKit,
it's basically,
it's basically a kit for developers to use
that they can create
augmented reality experiences.
And so what Craig Smith
did in this article,
was he basically, and booted the link,
I'll have to share that link with you.
He talked about different possibilities
for using the ARKit for
students with autism.
So looking at things like being able
to create visual schedules overlaid
on a real-world scene like this.
Like if a student was preparing a meal,
and had to do the different
steps of the meal.
You would be able to see that
overlay onto the real world.
Emotional regulation,
being able to identify
a feeling in real life during a,
and then being presented
with opportunities
or suggestions for how
to handle that feeling.
And also, social modeling that's actually
out in the real world.
So if a student was out on the playground,
and you could have a video
overlaid on top of the swing,
where they could learn
to share and take turns.
And this is another example.
Just skip down to this one here.
With visual focus.
So,
being able to demonstrate social skills
or emotional regulation skills.
And use visual timetables
to help organize the day.
So visual schedules overlaid on top
of what you're seeing in the real world.
And this was just an idea that
we had for a bravery game.
And I think that would be,
sensory customization, this
one is also very interesting.
So being able to, to show,
to adjust the color of the ceiling light.
And have customized texture
in front of your eyes,
and have each of these different
sensory elements be customized.
So we're running short on time.
If you're interested in learning
a little bit more about it,
I'm just gonna kind of scroll down
through some of these pictures, so you can
keep seeing some of them.
Some of these ideas for
how augmented reality
could be used with students with autism,
definitely take a look at this article.
It's definitely interesting.
And I believe we are about out of time,
so I will open up, I know
we've had some questions
come in on the chat box here.
Just let me scroll up here.
If you have any recent questions,
I know some people asked
throughout the presentation.
But if you have any questions,
feel free to type them now.
And if I don't get to your question,
feel free to contact me after the session,
and I'd happily answer over email as well.
So let's see.
It looks like we had
some questions about...
Okay great, so we had another example
of a first-person perspective of a child
with autism, thank you for sharing that.
All right, so question just came in.
Let's see.
- [Todd] Jaclyn, there
was a question about
are you aware of any
protocol that helps determine
which students are good candidates for VR?
- [Jaclyn] Oh thank you, I
missed that one, perfect.
So we, during a lot of
our, it's a great question,
during a lot of our pilot
studies and usability testing
as we're working with schools,
we typically will leave
that up to the therapist.
Usually the occupational
therapist or the teachers
will know best the
needs of their students.
We haven't actually developed
like a specific protocol,
but we have found that students who have
really extreme sensory processing issues
are sometimes not the best candidates,
because the, like, there was one student
that we worked with, one
who did put the headset on,
it kind of bothered him
having that on his face.
And that's actually only
happened one or two times.
And for the most part, we haven't had
any negatives sensory issues.
But again, usually we just
typically will leave it
up to the discretion of the therapist
or the teacher working with the students.
That's a great question.
Any other questions
coming in before we leave?
- [Todd] There's some new
comments towards the bottom
about double view, and whether or not
it's easily overcome by most students?
There's some questions about
accessibility standards.
- [Jaclyn] I don't know why I'm not
seeing those questions come in.
Okay, so it was about.
Let me see.
Yeah, I'm still just seeing
the Merge is the last part I'm seeing.
So it was about, can you repeat
the question one more time, Todd?
- [Todd] Yeah, there
was one question about
is the double view, quote
unquote, double view
easily overcome by most students?
And then there was also a question on,
do you have any insight into
the upcoming accessibility
standards for individuals with other
disabilities related to VR and AR?
- [Jaclyn] Okay, so for the double view,
I don't, we haven't experienced any issues
with that during our usability testing.
So I would say yes, the answer
to that would be yes,
it is easily overcome.
Interestingly we, the
students that we've actually,
we've done a lot of usability testing.
And with most of the
students we've worked with,
like I said, they haven't
really experienced
any dizziness or double view,
or any sort of vertigo
or anything like that.
In fact, it's kind of the opposite.
Most of the time we have to,
to beg them to take the headset off.
They are really, they
tend to be very, very,
very engaged when they
have the headsets on.
And very sort of entranced by the content.
So we haven't really experienced,
as I mentioned, there was the one student
who had some issues wearing the headset.
But other than that, we haven't really had
any issues with double vision
or vertigo or dizziness or anything.
And, let's see.
I'm so sorry, Todd, I'm still not seeing
that one other question.
- [Todd] Yeah, that's okay.
You were asking about accessibility,
and I feel like honestly,
I don't want to presume,
but my guess is super early
for accessibility
standards to be out there.
I haven't read, I've done some research.
I haven't, do you know anything?
Have you seen, I'm not even sure
where that would be as a resource.
- [Jaclyn] I don't, I mean
I've seen a lot of resources
being posted in terms of best protocols,
you know, for developers
and things like that,
in terms of developing
apps to meet standards.
But no, I think it might be
too early for that still.
- [Todd] A bunch of thank yous in here
for the presentation and the links.
Some other people have
shared some great stuff here.
So if you haven't looked in the chat box,
hopefully nobody else is having
issues with the chat box.
We will probably post a bunch
of these links that other people
have added for visual
impairments and such.
Thank you so much for
sharing that everyone.
Somebody's suggesting that the FCC
may have some stuff.
- [Jaclyn] Oh, my chat box
just, sorry to interrupt.
My chat box just got
caught up for some reason.
So yeah, I see somebody just asked where
they would find the developer protocols.
I don't have that offhand, but
I do have it saved somewhere.
So I can see if I can find that link
to share after the presentation.
And thank you so much, I'm--
- [Todd] Are you--
- [Jaclyn] Sorry, go ahead Todd.
- [Todd] I was just gonna
ask Alice if she was asking
about the AR developer protocols
or VR, or probably both.
- [Jaclyn] Yeah, I would guess both.
And I think I've seen
some resources for both.
So I can take a look and see
if I can find those links.
Perfect, yeah.
- [Todd] I think that's
it, I think we're there.
Jaclyn so much for this
great presentation.
I learned a lot today.
As we said, we will share the other
information that the folks have shared.
Please, it always helps us
to get survey information
back from you, feedback from you folks,
if you could click the
link on the SurveyMonkey,
we're always happy to hear
your comments and criticisms
and suggestions for other topics.
So thank you so much everyone
for joining us today.
We will be posting this on
the CTD website tomorrow.
Jaclyn, again, thank you
so much for your expertise.
- [Jaclyn] Yes, thanks
so much for inviting.
- [Todd] And I think we'll let you go.
