RIGHT. WELCOME. WELCOME TO OUR SESSION
DEVELOPING MOBILE AUGMENTED
REALITY APPLICATIONS WITH AZURE
SPATIAL ANCHORS.
IN THIS SESSION,
WE'LL FIRST TELL YOU WHY YOU SHOULD
CARE ABOUT SPATIAL ANCHORS AND WHY
IT IS AN IMPORTANT TECHNOLOGY AND
WHY WE NEED TO ACTUALLY TALK ABOUT
MOBILE AS WELL. AND THEN I WILL
TELL YOU A COUPLE OF EXAMPLES, A
BUNCH OF SCENARIOS FOR CONSUMER
AS WELL AS ENTERPRISE USE CASES
AND PARIS WILL ACTUALLY EXPLAIN
A LITTLE BIT ABOUT THE UNDERLINING
TECHNOLOGY. WE WILL HAVE A DEMO.
LATER ON GET FOR NATIVE IOS AND
ANDROID APPLICATION AND FIRST OF
ALL, THEN PARIS TELL YOU ABOUT U.
S. BEST PRACTICES AND AT THE END,
WE SHOULD HAVE TIME FOR SOME Q AND
A. PARIS, PLEASE INTRODUCE YOURSELF.
>> AWESOME. HI, EVERYONE. I'M PARIS
AND I'M AN ENGINEER ON THE AZURE
MIX TEAM. I WORK ON CLIENT SIDE
FOR SPATIAL ANCHORS. I TOUCH S IT
, S IT K AND THE DEVELOPER EXPERIENCE.
>> I'M RENE. WE ARE DIGITAL TRANSFORMATION
FIRM PRETTY MUCH FOCUS ON THE MICROSOFT
STACK FROM AZURE IOT AND 365 AND
INTELLIGENT EDGE, AND WE HAVE BEEN
DOING MIXED REALITY SINCE 2015 BEING
PART OF THE MIXED REALITY DEPARTMENT
PROGRAM BUT EVEN BEFORE THAT, I
HAVE BEEN DOING MOBILE AR. I CREATED
OPEN SOURCE PROJECT, AL2 KID. I
DID 9-10 YEARS AGO. YOU MIGHT NOTICE,
THE SLIDE SAYS MICROSOFT MVP, BE
CLEAR, THESE ARE ONE THAT MICROSOFT
PROVIDE TO INDEPENDENT EXPERTS.
I DON'T WORK WITH MICROSOFT. I LOVE
THE TECHNOLOGY AND PART OF THE COMMUNITY.
IT IS AMAZING BECAUSE I CAN BE HERE
ON STAGE AND TALK TO YOU ABOUT INTERESTING
TECHNOLOGY. FOR EXAMPLE, WHY YOU
SHOULD CARE ABOUT MOBILE AR. ALL
RIGHT. WHY IS MOBILE AR RELEVANT
? IF WE HAVE DEVICES LIKE HOLOLENS,
WHY CARE ABOUT MOBILE AR IN THE
END? LET'S TAKE A LOOK AT NUMBERS
HERE. THEY PREDICT BY NEXT YEAR,
100 MILLION CONSUMER ALSO SHOP IN
AR, ON-LINE AS WELL AS RETAIL. AND
ESPECIALLY FOR RETAIL EXPERIENCES,
I CAN IMAGINE A LOT OF VALUE ADDED
BY PERSISTENT CONTENT THAT CAN ALSO
BE SHARED ACROSS DEVICE, RIGHT?
MORE ABOUT THAT LATER IN A COUPLE
OF SLIDES, I WILL SHOW YOU SOME
SCENARIOS. BUT ALSO TAKE A LOOK
AT THE RIGHT HAND SIDE OF THAT SLIDE.
AND YOU CAN SEE THAT DATA, THAT
DATA IS AN ARM COMPANY, THEY SAY
SEPTEMBER 2018 THAT 26 OF ALL ACTIVE
IOS AND ANDROID DEVICES SUPPORTED
AR KID OR AR CORE. BY NOW, IT MIGHT
BE FAIR TO SAY IN MAY 2019, PROBABLY
MORE THAN ONE THOUGHT OF ALL ACTIVE
IOS AND AN DRID DEVICES THAT HAVE
THE BUILT-IN ADVANCE AR CAPABILITIES
THAT USE WITH SPECIAL ANCHORS. DO
THE MATH YOURSELF BUT HUGE MARKET
TO TAP INTO. IF WE HAVE THESE DEVICES
WITH THESE ADVANCE CAPABILITIES
BUILT IN, WHY WOULD WE USE AZURE?
RIGHT? WHY DOES AZURE MAKE SENSE
HERE? WELL, ALL OF THESE DEVICES
HAVE A VIEW OF THE DIGITAL WORLD
IN THEIR OWN DEVICE. TAKING IT OUT
OF THE DEVICE VIEW BASICALLY AN
LAUNCHING THOSE TOGETHER WITH ALL
OF THE DIFFERENT DEVICES THAT CAN
BE ENABLED WITH THE CLOUD. AZURE
SPATIAL ANCHORS AND YOU CAN HAVE
THESE UNIFIED VIEW OF THE DIGITAL
WORLD. AND OF COURSE, YOU CAN SECURELY
STORE IT WHICH MEANS YOU CAN ENABLE
THESE PERSISTED EXPERIENCES. RIGHT?
YOU CAN STORE IT SECURELY IN AZURE.
SO THESE DEVICES PERCEIVE THE REAL
WORLD IN A CERTAIN WAY. A COUPLE
OF YOU ARE PRETTY MUCH FAMILIAR
WITH IT. JUST TO MAKE SURE WE ARE
ON THE SAME PAGE. HERE IS A VIDEO,
ANDROID ENABLED PHONE, TO SEE HOW
IT PER SIEVES THE -- PERCEIVES THE
REAL WORLD. SEE THE DOTS, FUTURE
POINTS. THESE PLAINES, THESE GRID
OVERLAY. THAT GRID OVERLAY, CURRENTLY
AR SUPPORTS PLAINES AND HORIZONTAL
AND VERTICAL PLAINES SO THAT BASICALLY,
THEY CAN RECONSTRUCT. VERTICAL AND
HORIZONTAL. AND YOU CAN PLAY ON
THIS, LIKE THIS WEIRD GUY, THAT
IS SCANNED IN 3D. OF COURSE, WORKS
OUTDOOR, RIGHT? SCAN IT OUTDOORS.
AND SO THAT IS THE VIEW OF MY ANDROID
DEVICE. IF I USE A NON-IOS DEVICE,
NOT ANDROID, SLIGHTLY DIFFERENT
VIEW OF THE REAL WORLD. OR HOLOLENS,
HERE IS SIMILAR SCENE WITH THE HOLOLENS,
WITH THAT, OF COURSE, MORE SENSORS,
MORE DETAILED VIEW OF THE REAL WORLD
AND RECONSTRUCT THE SPATIAL MESH
IN A LITTLE MORE DETAIL. DIGITAL
REPRESENTATION. THIS IS HOW THE
DEVICE SEES THE WORLD AND SEE THE
CONTENT AND HAVE IT INTERACT WITH
THE REAL WORLD BUT WHAT WE REALLY
WANT IS WE WANT TO STEP OUT FROM
OUR LOCAL VIEW AND STEP INTO THE
GLOBAL VIEW, RIGHT? HAVING THE EXPERIENCES
FROM ALL DIFFERENT DEVICES PROVIDING
CROSS-PLATFORM EXPERIENCES WHERE
WE CAN PLACE CONTENT. SO WE CAN
DROP VIRTUAL CONTENT IN THE REAL
WORLD ON THESE SURFACES. WE CAN
ALSO ANCHOR THAT CONTEND THERE.
WHICH MEANS THAT CONTENT WILL STAY
AT THE EXACT SAME POSITION. SO YOU
MIGHT ASK, WHAT IS SPATIAL ANCHOR?
WHAT IS THAT CONCEPT OF AN ANCHOR?
LET'S TALK A LITTLE BIT ABOUT THAT.
ANCHOR IS BASICALLY A SPECIAL POINT
IN THE REAL WORLD, BUT A TRACKING
SYSTEM WILL MAKE SURE OF THE POSITION
AND ORIENTATION OF THE POINT WILL
STAY IN RELATION TO THE REAL WORLD
WHICH MEANS EVEN IF YOU WALK AROUND,
GO BACK AND FORWARD, THIS ANCHOR
WILL STAY AT THE EXACT SAME POSITION.
SO THE TRACKING SYSTEM, THEN ASSIGNS
COORDINATES, ANCHOR COORDINATES
THEM, AND THEN ON THAT ANCHOR, THE
CORE NATION SYSTEM, ATTACH VIRTUAL
CONTENT TO IT. THAT MEANS IF THE
CONTENT IS ANCHORED, IF THAT CONTENT
IS RELATED TO THE ANCHOR, IT WILL
MAKE SURE THAT IT STAYS AT THAT
POSITION, EVEN IF YOU WALK AROUND.
AGAIN, THE ANCHOR IS REALLY JUST
A LOCATION. IT IS NOT CONTENT. AND
YOU CAN ATTACH ANY CONTENT TO IT
IF YOU WANT. THAT IS SPATIAL ANCHOR,
JUST LOCAL DEVICE. HOLOLENS SUPPORTS
IT. AR KITS SUPPORT IT. AR CORE,
MAGICALLY, ALL OF THE PLATFORMS
HAVE A CONCEPT OF SPATIAL ANCHOR.
CALL IT MAYBE A LITTLE DIFFERENTLY
BUT ALL OF THE CONCEPT. WHAT IS
A CLOUD SPATIAL ANCHOR THEN? AS
A SPATIAL ANCHOR, YOU CAN UPLOAD
THE ANCHORS THAT YOU CREATED ON
THE DEVICE TO ACTUAL SPATIAL ANCHOR.
THEN YOU CAN HAVE YOUR FRIEND, FOR
EXAMPLE, COME IN, USE ITS OWN DEVICE.
DOWNLOAD THE ANCHOR FROM AZURE SPECIAL
ANCHOR, THEN IT WILL MAP THAT ANCHOR
FROM THE SPECIAL ANCHOR INTO THE
DEVICE'S LOCAL TRACKING SYSTEM.
AND THEN, WELL CAN SEE THE SAME
CONTENT AT EXACT SAME POSITION.
AND YOU CAN ENABLE THESE KIND OF
SHARED CROSS-PLATFORM EXPERIENCES
WITH AZURE PLATFORM ANCHORS THAT
NORMALIZES THESE DIFFERENT VIEW,
RIGHT? AND YOU CAN PERSIST IT. RIGHT?
SO I CAN COME BACK, A WEEK LATER,
CONTENT STILL THERE. I CAN SEE IT
AT THE EXACT SAME POSITION. PRETTY
HARD RESEARCH TOPIC AT THE MOMENT.
A COUPLE OF START-UPS IN ITS SPACE
AS WELL AS THE BIG PLAYERS. MICROSOFT,
AS SPECIAL SPATIAL, IT IS THE ONLY
ONE THAT PROVIDES SPECIAL ASSISTANCE.
LEAP FORWARD COMPARED TO ALL OF
THOSE IN THE MARKET. PARIS WILL
EXPLAIN LATER HOW THE DATA LOOKS
UNDER THE HOOD. AND SPATIAL ANCHOR,
HOW IT IS STORED. I WANT TO GO ON
AND SHOW YOU SOME TESTS AND SOME
MORE EXAMPLES THAT MAKE ABSTRACT
CONCEPT LITTLE BIT MORE FEASIBLE.
ALL RIGHT. WHAT I SAID, RIGHT, DOWN
TO THE ANCHOR AND YOU CAN SEE THE
SAME DATA A WEEK LATER AT EXACT
SAME SPOT. RIGHT. HERE IS THE VIDEO,
I RECORDED WITH MY ANDROID PHONE
AS WELL AS THE HOLOLENS. THIS IS
ABOUT CROSS-PLATFORM SHARED EXPERIENCES.
SOME OF YOU MIGHT HAVE SEEN IT ON-LINE
BEFORE. IF NOT, TAKE A LOOK. BASICALLY,
I CREATED THAT ANCHOR WITH MY ANDROID
PHONE. THAT IS RECALLED FROM ANDROID
PHONE, I CREATE THE ANCHOR, UPLOAD
TO AZURE SPECIAL ANCHOR SERVICE.
THIS IS THE FIRST VIEW OF THE HOLOLENS,
NOW. I LOAD THAT ANCHOR FROM AZURE
SPATIAL EANGOR SERVICE INTO THE
HOLOLENS AN SEE THE CONTENT AT EXACT
SAME POSITION. AND YOU CAN SEE MY
PHONE IN FRONT. REALLY ALIGNED REALLY
WELL. SO WHEN I DID THESE KIND OF
TESTS, I WAS REALLY SURPRISED HOW
IT ACTUALLY IS. IT IS FANTASTIC.
WE ARE TALKING ABOUT ARRANGED POSITION.
AND AGAIN, THAT IS NOT USING ANY
GPS. IT IS PURELY BASED ON COMPUTER
VISION, THE AZURE SPATIAL ANCHOR
AT VISION. IF WE USE GPS, WE WOULD
PROBABLY GET MIDDLE RANGE KIND OF
POSITION, RIGHT? AND WITH THIS,
WITH COMPUTER INNOVATION, WE GET
IN THE CENTIMETER RANGE OF POSITION.
THOUGH YOU CAN STILL COMBINE BOTH,
RIGHT? TOTALLY MAKES SENSE, ACTUALLY.
IF YOU THINK ABOUT, YOU CAN USE
GPS FOR FURTHER NARROWING DOWN THE
SPACE. AND YOU CAN SAY, I'M ONLY
INTERESTED IN THE ANCHOR IDEAS THAT
ARE 100 METERS AROUND ME. JUST LOAD
THOSE AND SEARCH FOR THEM LATER
ON. IT TOTALLY MAKES SENSE TO MAKE
GPS OR WI-FI ID OR FINGERPRINTING
AND THESE TECHNOLOGIES AND COMBINE
WITH AZURE AND GET TO THE DETAIL
MATCHING AND COURSE. THAT IS CROSS-PLATFORM
SHARED EXPERIENCES, LET'S TAKE A
LOOK AT PERSISTENCE, SOME OF YOU
MIGHT KNOW THE FAMOUS PIPE PLACE
MARKET IN SEATTLE DOWN THE STREET.
WHAT I DID IN MARCH, DOING THE MVP
SUMMIT, WENT THERE, CREATED AZURE
SPATIAL ANCHOR AND SAVED IT, OF
COURSE. THERE IS THE ANCHOR. A COUPLE
OF DAYS LATER, I WENT BACK AND RELOCALLIZED
IT. AND AS YOU CAN SEE, WALLAH!
IT IS RIGHT THERE. AT THE EXACT
SAME POSITION. AND YOU ALSO MIGHT
BE ABLE TO NOTICE THE FLOOR PATTERN
CHANGED. WHEN I CREATED THE ANCHOR,
IT WAS WET. WHEN RELOCATED, DRIED
UP. A LOT OF PEOPLE MOVING AROUND.
A LOT OF DYNAMIC SCENE CHANGES THAT
IS CHALLENGING FOR COMPUTER INNOVATION
UNDER THE HOOD. I WAS SURPRISED
HOW WELL IT WORKS AND HOW WELL IT
IS. NO GPS AGAIN. PURELY AZURE SPATIAL
ANCHORS. OKAY. SO DONE. LET'S TALK
ABOUT ENTERPRISE SCENARIOS. AS I
MENTION, WE HAVE BEEN WORKING WITH
ENTERPRISE, MANY YEARS. AS I DID,
EXTRACTED USES, WHICH I DO IN A
COUPLE OF DIFFERENT CATEGORIES TAND
TALKING FIRST ENTERPRISE AND THEN
CONSUMER SCENARIOS. LET'S TALK ABOUT
THE FIRST USE CASE CATEGORY. MULTI
USERS SHARED EXPERIENCES WITH PERSISTENT
CONTENT. AS YOU CAN SEE, ON THE
PHOTO HERE, THESE COLOR DESIGN REVIEWS.
THE LADY IS WEARING A HOLOLENS.
TWO OTHERS ARE USING DIFFERENT DAB
TABLETS. THE GOOD THING, SAME CONTENT
AT SAME LOCATION EVEN THOUGH TO
USE DIFFERENT DEVICES. THE BUSINESS
VALUE HAS REALLY, THAT WE CAN VISUALIZE
THESE KIND OF COMPLEX 3D DATA AND
CROSS FUNCTIONAL TEAMS. THEREFORE,
INCREASE BUSINESS VALUE, DEFINITELY
INCREASE COST AND QUALITY BECAUSE
YOU CAN, YOU KNOW, MUCH EASIER COLLABORATE
ON THAT VIRTUAL CONTENT. OR OF COURSE,
SEEING REALTIME SENSOR ON THE ACTUAL
EQUIPMENT. AND YOU CAN, FOR EXAMPLE,
HAVE LIKE HERE IN THE PHOTO, A CONTROL
PANEL ON A SENSOR AND SEE THE ACTUAL
SENSE LAID OFF IN REALTIME AT THE
ACTUAL MACHINERY. IF YOU THINK ABOUT
WHAT YOU NEED TO DO NOW, THERE IS
LIKE A COMPLEX CONTROL ROOM PANEL.
AND YOU NEED TO SEARCH THROUGH ALL
OF THE DATA AND SO ON. WITH THAT,
YOU CAN EASILY LOOK UP TO THE DATA.
SEE IT ON THE MACHINES. RIGHT ON
THE SPOT. OF COURSE, MUCH FASTER
INFORMATION AND SAVE A LOT OF TIME.
GUIDANCE. SO I'M GOING TO SHOW YOU
A VIDEO OF AN APP THAT WE DID FOR
A CLIENT A WHILE AGO WHICH IS FOR
AUTOMOTIVE SEAT ASSEMBLY. THE CHALLENGE
THERE WAS BASICALLY THEY HAVE QUITE
A BIT OF TURNOVER RATE. NEW EMPLOYEES
COME IN AND TRAINED. A LIMITED AMOUNT
OF TRAINERS ON STAFF. WHAT WE DID
WITH THE APP, ENABLING AND EMPOWERING
THE TRAINEES TO TRAIN THESE EASY
STEPS THEMSELVES SO YOU CAN SEE
THE HOLLOW GRAPHIC OVERLAYS TO LEARN
HOW TO ASSEMBLE AN AUTOMOTIVE SEAT.
YOU MIGHT WONDER, HOW DO WE KNOW
WHERE TO POSITION THE VIRTUAL CONCEPT
ON TOP OF THE REAL SEAT. YOU CAN
SEE, IT IS REALLY NICELY ALIGNED.
WHAT WE ARE USING IS A MARKER, RIGHT?
YOU CAN SEE THE MARKER THAT IS HIGHLIGHTED.
SOME CALL THEM AL MARKET. IMAGE
TARGET. CODE. WHATEVER. THIS ADDITIONAL
MARK WILL HELP US TO FIND THE REFERENCE
POINT IN THE REAL WORLD AND KNOW
THE OFFSET TO ACTUALLY POSITION
THE CONTENT. WHY AM I HERE TELLING
YOU THAT? WITH AZURE SPECIAL ANCHORS
WE CAN GET RID OF THOSE. IF YOU
CREATED SOME OF THE MARKERS BEFORE,
IT IS QUITE CUMBERSOME. ADD A DESIGN
AND THEN YOU NEED TO PRINT THEM
AND TEST THE SIZES. CHANGE SOMETHING,
REPRINT AND REITERATE THE PROCESS.
AZURE SPATIAL ANCHOR, WHAT WE DO,
USE A PHONE, GO INSIDE AND SCAN
THE POSITION IN THE ROOM AND CREATE
AN ANCHOR. THAT WILL BE OUR REFERENCE
POINT. WE CAN CREATE THIS TRACKING
DATA ON THE FLY AND PERSIST IT.
WE DON'T HAVE TO DEAL WITH UGLY
TRADITIONAL MARKERS ANYMORE. THE
OTHER BENEFIT, OF COURSE, NO LIKE
EXTRA KIND OF THINGY ON TOP OF THAT
WHICH MIGHT BE ANNOYING, IF YOU
THINK ABOUT AUTOMOTIVE OR AUTO TRADE
SHOWS WHERE I REALLY DON'T WANT
TO HAVE ANY STICKERS ATTACHED TO
ANYTHING. THAT IS REALLY, REALLY
A BIG VALUE, I THINK. ANOTHER CATEGORY
IS WAY FINDING. PARIS WILL EXPLAIN
LATER ON HOW IT WORKS UNDER THE
HOOD, WITH THE GRAPH. JUST THINK
ABOUT IF YOU CREATE AN ANCHOR AND
ACTUALLY MULTIPLE ANCHORS IN THE
SAME SESSION IT WILL BE CONNECTED.
SO YOU CAN HAVE THESE KIND OF WAY
FINDINGS. GO TO THIS ONE, THIS ANCHOR
AND SO ON. TAKE A LOOK AT THE PHOTO
THAT IS THE BOEING PLANT IN EVERETT
HERE. A HUGE FACILITY AND ACTUALLY
OFFER TOURS. IF YOU HAVE NOT BEEN
THERE, I HIGHLY RECOMMEND YOU TAKE
A TOUR. REALLY, REALLY AMAZING.
CAN YOU IMAGINE FIRST DAY ON SITE.
ALL YOU HAVE IS THIS MAP. THEY TELL
YOU, FIND THE COLLEAGUE IN BC45,
WHATEVER. I WOULD HIGHLY STRUGGLE.
I WOULD PROBABLY HAVE TO INTERRUPT
CO-WORKERS ASK THEM, WHERE DO YOU
NEED TO GO AND SO ON AND SO FORTH.
WITH AZURE SPECIAL ANCHOR, CREATE
THE WAY FINDING EXPERIENCES, RIGHT?
THEY CAN PULL OUT THE PHONE AND
ACTUALLY GET NAVIGATED WHERE THEY
NEED TO GO. SAVE A LOT OF TIME.
INCREASING EMPLOYEE PRODUCTIVITY
AND BUSINESS VALUE, OF COURSE, WITH
AZURE SPECIAL ANCHOR, ACTUALLY SECURELY
MAP YOUR FACILITY IN 3D. BECAUSE
IT IS NOT STORING ANY PHOTOS OR
STUFF. REALLY JUST STORING A POINT.
PARIS WILL EXPLAIN THAT TO YOU MUCH
LATER. IT IS REALLY THE HIGH BENEFIT
HERE, YOU CAN SECURELY CEMENT YOUR
3D ENVIRONMENT. ALL RIGHT. THIS
IS ANOTHER SCENARIO. THINK BEFORE
, ABOUT THIS ONE. ENGINEER IN CONTROL
ROOM. MACHINE IS FAILING. CONTROL
ROOM PANEL, LED, OR WHATEVER, IT
IS BLINKING. THIS MACHINE IS FAILING.
WHAT THEY NEED TO DO NOW IS NEED
TO GO THROUGH SOME INSTRUCTIONS.
OR SOME MAPS TO FIND, OKAY, MAYBE
BLUEPRINT, RIGHT? WHERE DO I NEED
TO GO? WHERE IS THE MACHINE? WHERE
IS THE SENSOR AN SO ON AND SO FORTH.
NOW THINK ABOUT, IF WE WOULD ENABLE
THEM, WITH AN APP ON A PHONE OR
A HOLOLENS, THEY CAN JUST LOOK IT
UP ON THE CONTROL ROOM PANEL, FOR
EXAMPLE, TAKE THE PHONE, HOLD IT
UP IN FRONT OF THE CONTROL ROOM
PANEL. THEN COMPUTER VISION. RECOGNIZES
WHICH KIND OF SENSOR IT IS. THEN
IT LOADS THE DATA FROM AZURE SPECIAL
ANCHORS AN GUIDES THE USE OF THE
ACTUAL MACHINE. OKAY. SO, GETTING
THERE MUCH FASTER WILL SAVE A LOT
OF MONEY. IF YOU ARE WORKING WITH
ENTERPRISES, LIKE SUDDEN FACILITIES
IF THEY JUST SHUT DOWN FOR A COUPLE
OF MINUTES, IT COST HUNDRED THOUSAND
FOLLOWERS. HUGE TIME SAVER THERE.
A LOT OF BUSINESS POTENTIAL THERE.
THEN NAVIGATING WAREHOUSES. THAT
IS PRETTY CLEAR, RIGHT? THINK ABOUT,
PICK UPS, AND PROVIDE DYNAMIC INFORMATION.
GO THERE, AND, YOU KNOW, PICK THAT
KIND OF STUFF UP. AND SO WE CAN
BUSINESS VALUE INCREASE EFFICIENCY
AND ALSO LESS PACKAGING IN THE END.
WELL LET'S TALK ABOUT CONSUMER SCENARIOS.
LIKE I MENTIONED, THERE ARE SO MANY
DEVICES OUT THERE. THAT ARE ENABLED
WITH IT. ANDROIDS, AR CORE, iPhone,
AR KIT, HOLOLENS, A HUGE MARKET.
OF COURSE, EVERYONE HAS A SMART
PHONE AND MANY OF THEM ACTUALLY
SUPPORT THESE ADVANCE AR CAPABILITIES.
HERE IS THE FIRST ONE, AGAIN, WAY
FINDING, THINK ABOUT SPATIAL LIST.
YOU CAN GO TO THE GROCERY STORE
AND CREATE THESE ANCHORS AT CERTAIN
LOCATION, CERTAIN AISLES WITH THE
FAVORITE PRODUCTS. OR, FOR EXAMPLE,
WHEN MY WIFE SENDS ME GROCERY SHOPPING
I ALWAYS STRUGGLE. BECAUSE I NEVER,
I DON'T GO OFTEN. I ALWAYS STRUGGLE
TO FIND THE STUFF. IT WOULD BE AWESOME
IF CREATES THESE ANKERS AN PULL
OUT MY PHONE AND TELLS ME, GO THERE,
PICK THIS UP, AND THAT UP. IT WOULD
SAVE ME A LOT OF TIME. IT WOULD
BE HAPPY TO HAVE THAT. LOOK AT THIS,
A COUPLE OF DAYS AGO, I GOT THE
NOTE ON THE LEFT SIDE. I WENT INTO
THE STORE AND CREATED THE ANCHORS.
QUITE A WAY TO DEMO, THAT YOU ARE
GOING TO GET LATER. CREATE THESE
ANCHORS WITH, LIKE WHAT DO YOU NEED
TO BUY. THINK ABOUT, YOU CAN SHARE
THAT ANCHOR DATA WITH THE STORE.
THE STORE CAN ACCESS THAT IF YOU
WANT. AND SO THEY COULD SAY, OKAY,
WE NOTICED THAT YOU BUY THE PRODUCT
OFTEN, HERE IS $2 DISCOUNT AND DROP
THAT INFO. DROP THAT AT THE ANCHOR,
RIGHT? AT THE POSITION. SO FOR RETAIL,
THAT IS A HUGE POTENTIAL, I THINK.
AS WE ALL KNOW, BRICK AND MORTAR
STORES HAVE BEEN STRUGGLING WITH
THE COMPETITION THE LAST COUPLE
OF YEARS. WITH THESE ADVANCE AR
CAPABILITIES, AND SCENARIOS WE CAN
ACTUALLY, YOU KNOW, INCREASE FOOT
TRAFFIC. BRING MORE PEOPLE IN. AND
IT DOESN'T HAVE TO BE ONLY WAY FINDING,
ALSO THINK ABOUT SAMPLING VIRTUAL
PRODUCTS RIGHT AT THE LOCATION.
FOR EXAMPLE, YOU HAVE SOME MODULAR
COMPONENTS AND THEN ACTUALLY SEE
THE REAL OBJECT AND THEN, YOU KNOW,
SEE SOME VIRTUAL ACCESS, THEY DON'T
HAVE IN STOCK AT THE MOMENT BUT
BUY LATER. A LOT OF THESE SCENARIOS
ARE POSSIBLE FOR RETAIL . WE JUST
TALKED ABOUT VISUAL CONTENT. JUST
ABOUT TALKED ABOUT ADDING VISUAL
CONTENT TO ANCHORS. LIKE I MENTIONED,
ANCHORS ARE REALLY JUST LOCATION
IN THE REAL WORLD. SO WE CAN ACTUALLY,
NOT JUST ATTACH VISUAL CONTENT BUT
AUDIO CONTENT. WE COULD, FOR EXAMPLE,
USE SPATIAL SOUND. NAVIGATING. IF
NOT FAMILIAR WITH SOUND, IT IS BASICALLY
A WAY THAT YOU AS A HUMAN, YOU KNOW
WHERE SOUND IS COMING FROM IN THE
WORLD. SO YOU KNOW, OKAY, I HEAR
THE SOUND HERE. AND YOU NEED TO
GO IN THAT DIRECTION. WE CAN USE
THAT. WE CAN DROP SPATIAL SOUND
AT ANCHORS AND THEN, YOU KNOW, POWER
EVERYONE TO NAVIGATE. LET ME SHOW
YOU A REALLY QUICK VIDEO FROM AN
APP CALLED, SEEING EYE. IT IS MICROSOFT
PRODUCT. WHICH WAS JUST ENHANCED
WITH SOME REALLY NICE FEATURES.
TAKE A LOOK AT THE VIDEO. [ MUSIC
] >> NAVIGATE TO MY NEXT MEETING.
>> YOUR NEXT MEETING WITH JULIET
IN STUDIO C. WOULD YOU LIKE ME TO
GUIDE YOU? >> YES. >> ARE YOU STARTING
FROM THE LOBBY OF STUDIO C? >> YES.
>> AND YOUR PHONE, TO MAP THE ROOM.
FOLLOW THE AUDIO BREAD CRUMBS TO
CONFERENCE ROOM 1200. >> WHERE CAN
I SIT? >> HAND YOUR PHONE TO MAP
THE ROOM. TABLE AT 12 O'CLOCK, EMPTY
SEAT LOCATED AT 11 O'CLOCK. >> HEY,
GLAD YOU'RE HERE. LET'S GET STARTED.
>> THAT IS PRETTY AWESOME, ISN'T
IT? I REALLY LOVE IT. [ APPLAUSE
] I DON'T TAKE CREDIT FROM IT, RIGHT?
IT IS FROM THE SEEING EYE TEAM.
I LOVE WHAT THEY DO THERE. USING
AZURE SPATIAL ANCHORS TO EMPOWER
EVERYONE. THESE NAVIGATIONS. I LOVE
WHERE MICROSOFT IS GOING WITH THE
WHOLE AI FOR GOOD MOVEMENT. IT IS
REALLY AWESOME. ALL RIGHTY. LET'S
GO ON. THE FINDING, THINK ABOUT
NAVIGATING AREAS LIKE MUSEUM, THEME
PARK, FESTIVAL AND SO ON. HERE IS
A VIDEO THAT I GOT FROM OUR FRIENDS
AT STUDIO 05 IN THEET , NETHERLANDS.
A THEME PARK THEY MADE. A CONCEPT.
BUT COULD MAKE IT REAL WITH AZURE
SPATIAL ANCHORS. YOU HAVE YOUR FAVORITE
THEME PARK BIGGER, SHOWS YOU WHERE
YOU NEED TO GO. IT CAN PULL IN DYNAMIC
INFORMATION SO IT CAN, FOR EXAMPLE,
TELL YOU, OKAY, IF YOU GO LEFT,
15 MINUTE WAITING TIMES. RIGHT,
30 MINUTE WAITING TIMES. OF COURSE,
YOU GO LEFT. AND YEAH, YOU SAVE
A LOT OF TIME. YOU WOULD BE PRETTY
HAPPY, I GUESS. A LOT OF BUSINESS
VALUE IN TERMS OF LIKE ENGAGING
CUSTOMER EXPERIENCES.... THERE
IS UGLY PARKING SIGN, WHICH RUINS
THE WHOLE THING. WE CAN ENHANCE
IT WITH DIGITAL CONTENT ON IT. FOR
EXAMPLE, OUR GOOD OLD FRIEND CLIPY.
RIGHT? [LAUGHTER] AND YOU KNOW WHAT?
IT DOESN'T HAVE TO BE CLIPY IN 2D.
BUT WE CAN HAVE CLIPY IN 3D. ALL
RIGHT. BUT JOKE ASIDE, THINK ABOUT
ALL OF THESE SOCIAL MEDIA APPS,
THEY OFFER US. THESE ARE VERY LIKE,
ONLY AN EXPERIENCE FOR YOURSELF.
IF YOU USE THESE AR FILTERS, YOU
CAN CREATE THESE. AR FILTERS BUT
JUST FOR YOURSELF. IF YOU THINK
ABOUT WITH AZURE SPATIAL ANCHORS,
YOU CAN ACTUALLY SHARE THEM WITH
OTHER PEOPLE AND EVEN CAN PERSIST
THEM. RIGHT? THEY WOULD WORK AT
ANY LOCATION. NOT JUST SOME SPECIFIC
HAND PICKED LOCATION AS WE SEE THESE
DAYS. AND SO, YOU KNOW, AND THE
GOOD THING IS, YOU CAN EVEN HAVE
CUSTOMIZABLE PERSISTENCE. SO YOU
CAN DEFINE EXPIRATION DATE. OKAY,
VIRTUAL CONTENT DIES OFF A COUPLE
MINUTE, HOUR, DAYS. SO HUGE POTENTIAL
FOR THESE KIND OF SOCIAL ENGAGING
APPLICATIONS. ARCHITECTURAL VISUALIZATION,
THIS IS A PHOTO FROM SWITZERLAND,
WHEN THEY SET UP THESE POLES CALLED
[INDISCERNIBLE]. THESE POLES DEFINE
HOW BIG OF A BUILDING WILL BE BEFORE
IT IS BUILT. RIGHT? SO THEN ACTUALLY
NEIGHBORS CAN ALSO COME IN AND POSE
AGAINST IT. OH, THAT IS A HUGE BUILDING.
I DON'T WANT TO HAVE IT THERE. THINK
ABOUT, IT IS QUITE A TEDIOUS AND
LABORIOUS WORK TO SET THEM UP. THINK
ABOUT, SET THEM UP VIRTUALLY. CONSTRUCTION,
ARCHITECTURE, SET THEM UP AS AZURE
SPATIAL ANCHORS. AR CODE. DOWNLOAD
AN APP. SEE THAT STUFF. IT DOESN'T
HAVE TO ONLY BE THIS UGLY POLE.
THEY COULD VISUALIZE THE REAL BUILDING,
CAT MODEL OF THE BUILDING HOW IT
LOCKS WHEN BUILT. THE NEIGHBORS
MIGHT ACTUALLY POSE AGAINST IT.
THEY DON'T JUST LIKE SEE SOME POLES
AND IMAGINE THEIR HEAD, THAT IS
A HUGE, NO, IT WILL SEE, A NICE
BUILDING. MUCH BETTER COMMITMENT.
ALSO THINK ABOUT NOT JUST THE PRIVATE
SECTOR BUT THE PUBLIC SECTOR. SOMETIMES
HIGHLY CONTROVERSIAL PROJECTS LIKE
EXPANSIONS, BRIDGE, AND THAT KIND
OF STUFF. IN GERMANY, THERE IS A
LOT OF CONTROVERSIALS AROUND THAT
STUFF ALL OF THE TIME. SO HAVING
THESE KIND OF APPLICATIONS, THEN
VISUALIZE IT ON SITE AND HAVE THESE
PERSISTED EXPERIENCES CAN SAVE YOU
A LOT OF MONEY. SO BUSINESS VALUE
IS REALLY HERE, YOU KNOW, NOT JUST
WITH YOU SEEING THE WORK, AND SETTING
UP THE POLES FOR THIS SPECIFIC EXAMPLES.
IF YOU BROADEN THE RANGE, VISUALIZATION,
BETTER COMMITMENT IN THE END COST
SAVING. WELL, MARKETING AND ADS,
RIGHT? THAT IS CLEAR. HERE IS A
VIDEO. DROP AND REPLY LOGO. RECORD
THE VIDEO. SHARE IT ON LINE. HAVE
A LITTLE BIT OF WIND. YEAH, I MEAN,
YOU CAN DO A LOT OF ENGAGING CUSTOMER
EXPERIENCES THERE. OF COURSE IF
WE TALK ABOUT ADVERTISE MANY, THINK
ABOUT PRIVACY. THE GOOD THING IS
ALL SAFE AND REIF -- PRIVATE. IT
IS CONNECTED TO TENANT, WHICH MEANS
THAT EVERYONE CAN HAVE THEIR OWN
VIEW, OWN LAYER OF THE VIRTUAL WORLD.
POINTS OF INTEREST, SO HERE IS A
VIDEO. TOOK MY ROAD BIKE. WENT OUT
TO THE PLACE THAT I LIVE, BEAUTIFUL
ROCKS THERE. AND WENT UP THE HILL
AND THEN, YOU KNOW, DROP SPATIAL
ANCHOR THERE WITH POINT OF INTEREST
INFORMATION TELLING YOU ABOUT THIS
BEAUTIFUL REGION. JUST TEXT IN THIS
CASE, RIGHT? THINK ABOUT TOURIST
AGENCIES. CANNOT JUST SET UP TEXT
THERE, COULD HAVE NICE ENGAGING
EXPERIENCES, COULD BE A VIDEO, INTERACTIVE
CONTENT. THESE REALLY ENGAGING EXPERIENCES
FOR USERS. AND OF COURSE, YOU KNOW,
YOU CAN REDUCE THE COST OF SETTING
UP THESE SIGNS. BUT I THINK THAT
EVEN HIGHER VALUE IS, YOU KNOW,
PROVIDING THESE ENGAGING EXPERIENCES
AND, YOU KNOW, THERE ARE SO MANY
TOURIST REGIONS COMPETING ALL OF
THE TIME. IT IS LIKE, USING INNOVATIVE
TECHNOLOGY TO SET THESE MORE FOR
THE COMPETITION. AND OF COURSE,
GAMES. I MEAN, AZURE SPATIAL ANCHORS
SCREAMS GAMES, RIGHT? THAT IS PERFECT
GAMES. SO THINK ABOUT HIDE AND SEEK
GAMES. YOU KNOW, KIND OF SCAVENGER
HUNT GAMES. ANY KIND OF LOCATION
BASE, SOCIAL ENGAGING GAMES COULD
BE POSSIBLE IF THESE MULTI USER
CROSS-PLATFORM EXPERIENCES. AND
THE REALLY, REALLY NICE THING, WHAT
I LOVE ABOUT AZURE SPATIAL ANCHORS
IS THAT DEMO THE GAME. ANYONE CAN
BUILD A REAL POKÉMON GO NOW BASICALLY.
HERE IS A VIDEO. CREATED SOME ANCHORS
ALL OVER THE PLACE. AND THEN ATTACHED
SOME VIRTUAL FRUITS TO IT AND PERSISTED
THEM. SO STILL THERE. THE GOOD THING
IS, NO ONE WILL SLIP ON A VIRTUAL
BANANA. IT IS SAFE, LEFT IT THERE.
ON THE SIDEWALK. SO YOU KNOW FRUITY
ANCHORS ALL OVER THE PLACE. AND
BELLEVUE AND REDMOND AND, YEAH,
I MIGHT COME BACK AT SOME POINT
AND DEVELOP MY FRUIT HUNT GAME.
I HAVE THE BASIS FOR THAT NOW. ANYWAY,
I HOPE THAT THE LAST COUPLE OF EXAMPLES
GAVE YOU SOME INSPIRATION, SPARKED
SOME IDEAS FOR YOUR OWN PROJECTS.
PARIS IS ACTUALLY EXPLAINING HOW
TO BUILD YOUR OWN. >> AWESOME. THANK
YOU, RENE. [ APPLAUSE ] >> SO I'M
STILL WAITING FOR THE BANANA GAME.
BUT TO HELP EVERYONE BUILD THAT
GAME, I WANT TO GET INTO A LITTLE
BIT MORE DETAIL ON AZURE SPATIAL
ANCHORS AND HOW YOU CAN BUILD APPS
FOR IT. SO TO START, LET'S REVISIT
THIS DIAGRAM. BECAUSE IT IS REALLY
IMPORTANT THAT WE ARE ALTOGETHER
ON WHAT IS A SPATIAL ANCHOR AND
WHAT IS A CLOUD SPATIAL ANCHOR.
SPATIAL ANCHOR, LIKE RENE DISCUSSED
IS A COMMON FRAME OF REFERENCE BETWEEN
THE DIGITAL AND PHYSICAL WORLD.
YOU HAVE THE SPATIAL ANCHOR AND
YOU CAN RENDER A HOLOGRAM THERE.
THE SPATIAL ANCHOR IS THAT LOCATION
NOT THE ROCKETSHIP. IF YOU WANT
TO SHARE THAT LOCATION WITH AZURE
SPATIAL ANCHORS, YOU CAN SEND IT
UP TO AZURE SPATIAL ANCHORS THAT
I WILL ABBREVIATE FROM ASA FOR NOW
ON. SOMETIMES. WHEN YOUR FRIEND
COMES, GET THE LOCATION SENT TO
THEM. AND THEN RENDER A HOLOGRAM
THERE AS WELL. SO TO GIVE YOU A
LITTLE INSIGHT ON TO WHAT IS GOING
ON HERE. I WANT TO SHOW YOU A PICTURE
THAT I KNOW REALLY WELL. BECAUSE
THIS IS THE LOBBY OF MY BUILDING
AT WORK. I WANT TO THEN SHOW YOU
WHAT AZURE SPATIAL ANCHOR SEES WHEN
IT LOOKS AT THAT. SO IN THIS GIFT
THERE ARE THREE MAIN THINGS GOING
ON. FIRST IN GREEN, OUR FEATURE
POINTS. FEATURE POINTS ARE AREAS
OF HIGH VISUAL CONTRAST. THERE ARE
AREAS WHERE THERE IS AN EDGE OR
CORNER THAT WENT FROM DARK TO LIGHT.
OR A PLACE WHERE MAYBE IF YOU LOOK
AT YOUR BADGE, YOU'RE WRITING WAS
ON IT AND WENT FROM WHITE TO BLACK.
WE GET FEATURE POINTS BY PASSING
IN CAMERA FRAMES TO OUR ALGORITHMS.
WHAT YOU ARE SEEING HERE, IN RED,
ARE EACH OF THE CAMERA FRAMES THAT
A MOBILE DEVICE PASSES INTO AZURE
SPATIAL ANCHORS. NEXT IN THE BOTTOM
LEFT, YOU ARE SEEING A SPATIAL ANCHOR.
SO THAT WHITE SPHERE IS A PLACE
IN THE REAL WORLD THAT SOMEBODY
SAID, I CARE ABOUT THIS. I'M GOING
TO PLACE AN ANCHOR HERE. THAT I
CAN LATER RETRIEVE IT. I WANT TO
NOTE THAT ALL OF THE PROCESSING
THAT WE ARE DOING FROM CAMERA FRAMES
TO FEATURE POINT IS HAPPENING ON
THE DEVICE. WE ARE NOT SENDING THESE
IMAGES UP TO THE CLOUD. WITH THAT
IN MIND, I WANT TO TALK ABOUT THREE
THINGS NOW. FIRST, HOW DO YOU BUILD
UPPER COLLABORATION? NEXT, HOW DO
WE EXTEND THAT TO FIND WAY FINDING
APP. FINALLY, A LITTLE BIT ABOUT
UX WHEN BUILDING WITH ASA. TO START,
WE ARE CROSS-PLATFORM. SO YOU CAN
BUILD FOR HOLOLENS, IOS OR ANDROID.
YOU CAN WRITE OR NATIVE CODE. IN
THIS, JAVA ANDROID APP. THE CONCEPTS
ARE PRETTY UNIVERSAL ACROSS ALL
PLATFORMS AN DEVELOPMENT ENVIRONMENTS.
SO LET'S SHOW A DEMO FIRST. FOR
TODAY, WE BUILT AN APP THAT WAS
SIMILAR TO RENE'S FACTORY FLOOR
EXAMPLE WHERE HE SHOWED YOU HOW
BY PLACING NOTES ON A CAR ASSEMBLY
LINE, YOU ARE ABLE TO SAVE MONEY
FOR THE BUSINESS BECAUSE THEY CAN
JUST RENDER THAT HOLOGRAM RIGHT
ON THE CAR SEAT. SO I'M NOW GOING
TO SWITCH OVER AND I'M GOING TO
SHOW YOU AN ANDROID APP. BEFORE
TODAY, I PLACED A NOTE UP HERE.
AND WHAT YOU ARE GOING TO SEE, I
AM GOING TO OPEN UP THE APP AND
SCAN AROUND FOR KEY FRAMES. WE ARE
GOING TO SCAN AROUND FOR FEATURE
POINTS FROM CAMERA FRAMES. HERE
WE ARE. OPENING THE APP. AND WE
ARE LOOKING FOR THAT ANCHOR RIGHT
NOW. AR CORE IS RIGHT NOW LOOKING
FOR A PLAINES THAT IT JUST FOUND.
THERE WE GO. HELLO, WORLD! WE NOW
HAVE AN ANCHOR AT THAT PLACE WE
FOUND. LET'S PLACE A NEW ANCHOR,
SO WE'LL DO A SMILEY FACE. RIGHT
NOW AZURE SPATIAL ANCHORS IS GETTING
CAMERA FRAMES AND TURNING THEM INTO
FUTURE POINTS. WE WENT YELLOW WHEN
WE HAD ENOUGH DATA TO ENVIRONMENT
TO SAVE TO AZURE ANCHORS. WE WENT
BLUE WHEN THE ANCHOR IS SUCCESSFULLY
SAVED. SO LET'S IMAGINE A WEEK PASSES
AND WE WANT TO FIND THESE SAME TWO
ANCHORS. RENE IS GOING TO COME UP
NOW AND WE ARE GOING TO SHOW YOU
HOW BOTH ON THIS ANDROID DEVICE
AND IOS DEVICE FIND THE SAME TWO
ANCHORS. SO I'M GOING TO OPEN UP
MY APP. RENE IS GOING TO OPEN UP
HIS. AND HERE WE GO. SMILEY FACE
AND HELLO WORLD. AND RENE GOT IT.
MY DEMO WORKED. AWESOME. [ APPLAUSE
] SO LET'S GO BACK AND LET'S TALK
A LITTLE BIT ABOUT HOW WE BUILT
THIS. I'M GOING TO SHOW YOU A DIAGRAM
THAT IS OUR ARCHITECTURE FOR THE
APP. SO THERE IS THREE MAIN THINGS
GOING ON IN THIS DIAGRAM. FIRST
WE HAVE SOME CLIENT CODE. SO WE
HAVE AN IOS APP THAT WE WROTE IN
SWIFT USING AR AND ANDROID SCRIPT
IN JAVA. NEXT WE HAVE AZURE SPATIAL
ANCHORS WHICH IS HOW WE ARE SAVING
ANCHORS AN FINDING THEM LATER. AND
FINALLY WE HAVE WHAT WE ARE CALLING
THE SHARING SERVICE. THIS IS NOT
ACTUAL AZURE SERVICE BUT JUST CALLING
IT THAT. THIS CASE, TWO THING, AZURE
APP SERVICE, WEB API, LET'S YOU
DO POST-CALLS AND GET CALLS. AND
TABLE STORAGE AS DATABASE. TOGETHER
THEY HELP US SAVE INFORMATION ABOUT
ANCHORS THAT WE NEED TO SAVE. I
WILL GIVE YOU MORE DETAIL ON THAT
BUT BEFORE WE DO THAT, I WANT TO
JUMP INTO SOME API KEY CONCEPTS
FOR ASA. SO AT THE ROOT OF ASA IS
THE CLOUD SPATIAL ANCHOR SESSION.
AND FIRST PROVIDES CORE SERVICES.
SO THROUGH IT, YOU CREATE ANCHORS,
LOCATE THEM, UPDATE THEIR PROPERTIES
OR DELETE THEM. AND SECOND, IT CAPTURES
DATA ABOUT THE ENVIRONMENT. WHEN
I SAY THAT, I MEAN, ON MOBILE DEVICES
YOU ARE PASSING IN CAMERA FRAMES
TO THE SESSION. ON HOLOLENS, WE
ARE GETTING THE MAP FROM HOLOLENS
AN PASSING IT OURSELVES. NEXT WE
HAVE THE ANCHOR. AND THE REALLY
KEY THING THAT I WANT TO CALL OUT
HERE, IT LINKS TO THE UNDERLYING
AI PLATFORM ANCHOR. IF WRITING A
HOLOLENS APP AND WORLD ANCHOR, YOU
SET THAT AS PROPERTY ON THE CLOUD
SPATIAL ANCHOR. OR IF YOU ARE ON
OIS, AR ANCHOR OR ANDROID ANCHOR.
YOU DON'T HAVE TO DO CONVERSION
MATH. YOU JUST SET THE ANCHOR THAT
YOU ARE USED TO WORKING WITH ON
THE PLATFORM OF YOUR CHOICE AS A
PROPERTY ON OUR CLOUD SPATIAL ANCHOR
OBJECT. NEXT IT HOLDS A UNIQUE ID
AND EXPIRATION DATE AND A BAG OF
PROPERTIES. WHEN YOU ARE READY TO
CREATE AN ANCHOR, AND BY THAT, I
MEAN YOU HAVE AN ANCHOR AND READY
TO SAVE IT TO THE CLOUD YOU ARE
GOING TO CALL CREATE ANCHOR ASYNC
ON THIS ANCHOR. IN RETURN, WE'LL
GIVE YOU A CLOUD SPATIAL ANCHOR
IN A CALLBACK. THE DIFFERENCE BETWEEN
THE ANCHOR YOU ARE ABOUT TO SAVE
AND THE ANCHOR THAT WE GIVE YOU
BACK IS THAT IT HAS AN IDEA SIDE.
SO WHEN YOU ARE JUST WRITING YOUR
APP, YOU WON'T HAVE IDs BUT ONLY
ONCE YOU ACTUALLY SAVE THEM. WHEN
YOU ARE READY TO LOCATE ANCHOR,
FOR EXAMPLE, UP HERE, WE WANT TO
DEFINE THOSE TWO, WE ARE GOING TO
USE THE CONCEPT OF A WATCHER. A
WATCHER IS SOMETHING THAT LOOKS
FOR ANCHORS THAT MEET THE SPECIFIED
CRITERIA. I PROMISE I WILL TELL
YOU WHAT CRITERIA ARE IN A SECOND.
BUT WHEN THEY FIND ANCHORS THAT
MEET THE SPECIFIED CRITERIA, YOU
WILL GET BACK THOSE OBJECTS THROUGH
A DELEGATE OR A CALLBACK. THE INTERESTING
THING ABOUT THE ANCHORS THAT COME
BACK IS THAT THEY WILL HAVE THAT
UNDERLYING AR PLATFORM ASSIGNED
AS A PROPERTY. HERE YOU SAW, WE
HAVE IOS APP AND ANDROID APP. I
CREATED AN ANCHOR, CREATED A CLOUD
SPATIAL ANCHOR IN ANDROID AND ABLE
TO FIND IT ON IOS. WHEN I WAS ON
IOS, I WAS GIVEN BACK THAT AR ANCHOR
EVEN THOUGH THAT I CREATED THE CLOUD
SPATIAL ANCHOR WITH THE AR CORE
ANCHOR. SO THAT IS THE ASPECT OF
CROSS-PLATFORM THAT WE TALK ABOUT.
THAT WE DO THE CONVERSION BETWEEN
DIFFERENT SYSTEMS AND YOU GET TO
WORK WITH WHATEVER YOU WANT. THE
CRITERIA THAT I PROMISED ARE ANCHOR
LOCATE CRITERIA. IT IS AN ARRAY
OF IDs. SO YOU MIGHT BE LOOKING
AT THIS AND SAYING, OKAY, I CREATED
ANCHOR. AND I GET BACK AN ID. LATER,
WHEN I WANT TO FIND ANCHOR, I NEED
TO GIVE YOU IDs, SO WHERE DO I PUT
THOSE IDS? THAT IS WHERE WE HAVE
THE SHARING SERVICE. THE SHARING
SERVICE IS JUST DATABANK OF IDS.
BECAUSE IT IS NOT REALLY IMPORTANT
WHAT THE IMPLEMENTATION DETAILS
FOR THIS, WE ARE GOING TO SIMPLIFY
IT AND THINK OF IT AS A DATABASE.
DATABASE THAT STORES IDs. SO WHAT
ELSE IS GOING ON IN HERE? THERE
IS THREE MAIN THINGS. FIRST WE ARE
INITIALIZING A SESSION. THEN WE
ARE CREATING AN ANCHOR. AND FINALLY
WE ARE LOCATING THE ANCHORS. IF
WE WALK THROUGH HOW DATA IS FLOWING,
FIRST WE SAVE THE ANCHOR TO ASA.
IN RETURN, WE GET AN ID. WE SAVE
THAT ID TO THE DATABASE. AND WE
CREATED AN ANCHOR. WHEN WE ARE READY
TO LOCATE ANCHORS WE ARE GOING TO
GET THE IDs FROM THE DATABASE AND
TELL THEM TO ASA THROUGH THE CRITERIA
THAT WE SET ON A WATCHER AND GET
BACK THE ANCHORS. IN BETWEEN STEPS
FIVE AND SIX, YOU ARE SCANNING YOUR
PHONE AROUND TO GET ENVIRONMENT
DATA. ONE THING THAT I WANT TO CALL
OUT BEFORE I MOVE ON, YOU WRITE
THIS YOURSELF. SO IN THIS APP THAT
WE WROTE, WE JUST USE THE KEY OF
BUILD 2019. AND THAT IS HOW WE SELECTED
THE GROUPING OF ANCHORS THAT WE
WANTED TO LOOK FOR. AND YOU COULD
USE OTHER THINGS, YOU COULD USE
GPS TO KIND OF NARROW IT DOWN TO
ANCHORS MAYBE AROUND YOU. AND THEN
YOU LOOK FOR THOSE. WHERE YOU COULD
USE USER ID OR SOME OTHER APP SPECIFIC
PROPERTY. WITH THAT PREFACE, I WANT
TO SHOW YOU FOUR SLIDES OF CODE,
THAT WILL STEP YOU THROUGH ONE,
TWO, AND THREE. AND WE ARE GOING
TO GO FORWARD AND MY FAVORITE IDE,
WHICH IS POWERPOINT. [LAUGHTER]
SO HERE WE ARE IN STEP ONE. INITIALIZING
A SESSION. WE ARE IN ON CREATE.
WHICH IS START OR VIEW LOAD IOS.
TO START, WE ARE GOING TO SET UP
OUR AR COORDINATE SCENE. I AM USING
ELIPSYS HERE TO HAVE ON GITHUB OR
MANY AR SAMPLE APPS. BUT BASICALLY,
WE HAVE SCENE VIEW AND SCENE AND
FOREIGN CONCEPTS. NEXT WE ARE GOING
TO INITIALIZE THE CLOUD SPATIAL
ANCHOR SESSION. FIRST, THE SESSION.
WE ARE GOING TO SET A LOG LEVEL
AND WE ARE GOING TO SET AN ACCOUNT
ID AND ACCOUNT KEY WHICH YOU GET
FROM AZURE. COM. AFTER THAT, WE
ARE GOING TO SET UP FOR FRAME LISTENER.
OR THE UPDATE LISTENER. AND WHAT
THIS DOES, IT GIVES FRAMES TO AZURE
SPATIAL ANCHORS. SO IN THE ONE I
SHOWED YOU, WE HAD THE MANY KEY
FRAMES OR CAMERA FRAMES TURNED INTO
FEATURE POINTS. THIS IS HOW WE GIVE
AZURE SPATIAL ANCHORS THOSE CAMERA
FRAMES. THIS UPDATE LISTENER IS
A JAVA CONCEPT BASICALLY IT WILL
BE CALLED EVERY TIME AR CORE HAS
A NEW CAMERA FRAME TO GIVE US. IN
RETURN, WE ARE GOING TO CALL PROCESS
FRAME ON THAT CAMERA FRAME. SO THIS
IS ONE SIDE OF THE HANDSHAKE THAT
I AM ABOUT TO SHOW YOU. WE GIVE
CAMERA FRAMES TO AZURE SPATIAL ANCHORS.
IN RETURN, AZURE SPATIAL ANCHORS
GIVE US BACK A FLOAT THAT TELLS
US HOW MUCH INFORMATION ABOUT THE
ENVIRONMENT IT HAS. WE ARE DOING
THIS CLOUD SESSION. SESSION UPDATED
LISTENER. THIS WILL BE CALLED THE
AZURE SPATIAL ANCHORS AN TELL US
WHEN WE ARE READY TO CREATE AN ANCHOR.
SESSION UPDATED WILL BE A FLOAT
AND WHEN IT GETS TO ONE, THAT IS
US, THAT IS AZURE SPATIAL ANCHORS
TELLING YOU, YOU CAN SAVE AN ENANCHOR
AND WE KNOW ABOUT THE ENVIRONMENT.
GIVE AZURE ANCHORS IN RETURN, GIVE
BACK A FLOAT, 0-2. WHEN 2 , TO 1,
SAVE ENOUGH. NOW THAT WE HAVE INITIALIZED
THE SESSION, WE ARE READY TO CREATE
AN ANCHOR. BEFORE WE CREATE ANCHOR,
WE NEED TO CHOOSE A PLACE IN THE
REAL WORLD TO PLACE THE ANCHOR.
WHAT WE WILL DO, SET THE ON TAP
AR PLAIN LISTENER. AR CORE CONCEPT.
WHAT IT WILL DO, SAY, WHENEVER THE
USER TAPS ON THE PHONE AND TAPPING
ON A PLAIN, SO A HORIZONTAL SPACE,
OR VIRTUAL CALL SPACE, WRITE THE
HANDLE TAP METHOD. WE WRITE IT OURSELVES
MUCH LET'S GO IN AND WRITE THAT.
HANDLE TAP METHOD WHICH WILL LET
US CREATE AN ANCHOR. WE WILL GETTS
PASSED IN A HIT RESULT. WITH THAT
HIT RESULT, CREATE AR CORE ANKOR.
CREATING AR CORE ANCHOR IS ARRAY
CAST DOWN INTO THE REAL WORLD. WE
WILL SAY, HIT RESULT. ANCHOR, CAST
DOWN FROM 2D SPACE TO 3D SPACE AND
NOW HAVE OUR SPATIAL ANCHOR. AFTER
THAT, WE CAN RENDER A WHITE SPHERE.
YOU SAW HOW WE TAP TO PLACE FOR
A WHITE SPHERE FOR A LITTLE BIT.
THAT IS WAS US WITH A LOCAL AIR
CORE ANCHOR . NEXT HERE COMES THE
MAGIC. WE ARE GOING TO CREATE THE
CLOUD SPATIAL ANCHOR AND WE ARE
GOING TO SET THE LOCAL ANCHOR AS
THAT AR CORE ANCHOR. HERE IS US
SETTING THE UNDERLYING AR PLATFORM
ANCHOR ON THE CLOUD SPATIAL ANCHOR.
AFTER THAT, WE ARE GOING TO PROMPT
THE USER TO PUT A NOTE. THAT WAS
THE SMILEY FACE THAT I TYPED IN.
WE ARE GOING TO THEN SET IT AS A
PROPERTY ON THE CLOUD SPATIAL ANCHOR.
SO WE HAVE A STRING STRING DICTIONARY
AND YOU CAN SET THINGS YOU WANT
ON THERE. NEXT WE ARE GOING TO WAIT
UNTIL WE HAVE AN UPDATE ABOUT THE
ENVIRONMENT. WHEN I TALKED ABOUT
THAT HANDSHAKE, HERE IS CAMERA FRAMES,
GIVE US BACK A NUMBER. WE ARE WAITING
UNTIL WE GET ABOVE ONE SO WE KNOW
ENOUGH ABOUT THE ENVIRONMENT. ONCE
WE GET TO THERE, WE CAN SAVE IT
TO AZURE SPATIAL ANCHORS. WE'LL
SAVE IT AND WAIT FOR A CALLBACK.
THAT CALLBACK WILL HAVE THE CLOUD
SPATIAL ANCHOR WITH AN IDEA SIGNED.
WE ARE GOING TO UPDATE THE SPHERE
COLOR TO BLUE BECAUSE WE SUCCESSFULLY
SAVED THE ANCHOR. AND THEN WE ARE
GOING TO SAVE THE ID TO THE SHARING
SERVICE. SO WITH THIS, WE CRAE ,
CREATED ANCHOR IN THE REAL WORLD,
USING AR CORE. SAVED TO AZURE SPATIAL
ANCHORS AND ID TO THE SHARING SERVICE.
ON THE OTHER SIDE OF THINGS WE ARE
GOING TO LOCATE THE ANCHOR. WE ARE
GOING TO START FROM THE SHARING
SERVICE. WE ARE THEN GOING TO DO
A GIT CALL. LIKE WE TALKED ABOUT
BEFORE, IN THIS CASE, WE ARE JUST
USING A KEY. YOU COULD USE GPS OR
ANOTHER WAY TO FILTER DOWN THE ANCHORS
YOU ARE INTERESTED IN. ONCE YOU
HAVE THE ANCHOR ID YOU ARE INTERESTED
IN, YOU ARE GOING TO CREATE A WATCHER.
TO CREATE A WATCHER, YOU CREATE
YOUR CRITERIA, YOU SET THE IDENTIFIER
AS THE LIST OF I'D FER YS YOU ARE
INTERESTED IN. THEN TELL AZURE SPATIAL
ANCHOR, CLOUD SESSION, CREATE WATCHER.
AS YOU PAN YOUR PHONE AROUND AND
WE GET FUTURE POINTS IN THE REAL
WORLD, WE ARE GOING TO BE TRYING
TO MATCH THE FEATURE POINTS TO THE
FEATURE POINTS THAT WE SAVED THAT
LINE UP WITH THOSE IDENTIFIERS.
WHEN WE FIND ANCHORS, WE NEED TO
HAVE A WAY TO TELL YOU ABOUT THEM.
LET'S GO BACK TO THE CODE FOR INITIALIZING
THE SESSION AND ADD SOMETHING THAT
WE FORGOT. WE ARE GOING TO ADD ANCHOR
LOCATED LISTENER. SO THIS IS A CALLBACK
THAT WILL GET CALLED ONCE WE FOUND
AN ANCHOR. THERE ARE SEVERAL STATUSES
THAT CAN OCCUR HERE. WE ARE GOING
TO SHOW LOCATED. ANOTHER OPTION
IS NOT LOCATED DOES NOT EXIST AND
THAT COULD COME UP IF YOU GIVE US
A BAD IDENTIFIER THAT WE DON'T KNOW
ABOUT. SO HERE WE FOUND ONE AND
WE HAVE THE LOCATE ANCHOR STATUS
OF LOCATED. NOW WE ARE GOING TO
GET THE AR CORE ANCHOR FROM THAT
CLOUD SPATIAL ANCHOR. AND I WANT
TO HIGHLIGHT AGAIN, THIS IS THE
REALLY COOL PART. AND YOU ARE GOING
FROM A CLOUD SPATIAL ANCHOR AND
JUST GRABBING THE PROPERTY OF THAT
LOCAL ANCHOR AND THAT IS A POINT
IN THE REAL WORLD. SO IT DOESN'T
MATTER WHETHER IT IS CREATED IN
HOLOLENS, IOS OR ANDROID, YOU GET
IT ON WHATEVER PLATFORM YOU WANT
IT ON. WE CAN THEN RENDER GREEN
SPHERE THERE, GET THAT NOTE, THAT
IS ON A DICTIONARY AND RENDER IT
AS WELL. AND NOW WE HAVE COMPLETED
THE APP. SO WITH THIS, WE CAN INITIALIZE
THE SESSION, CREATE A CLOUD ANCHOR,
CLOUD SPATIAL ANCHOR AN LOCATE THAT
ANCHOR. SO HOPE THIS IS A GOOD STARTING
POINT FOR YOU TO BUILD APPS THAT
ENABLE YOU TO PLACE PERSISTENT CONTENT
IN THE REAL WORLD. BUT ONE THING
I DIDN'T SHOW YOU HERE IS HOW DO
WE DO WAY FINDING? HOW DO THE ANCHORS
BY LOCATING ID LINE UP WITH THE
ABILITY TO WAY FINDER ON A LOCATION?
AND THE THING I HAVEN'T TOLD YOU
YET IS THAT ANCHORS ARE CONNECTED
IN A GRAPH. SO TO ILLUSTRATE THAT
POINT, I'M GOING TO WALK YOU THROUGH
A SCENARIO. HERE WE HAVE IN THAT
BOX, A SESSION, A SESSION IS BASICALLY
YOU OPEN UP YOUR APP AND YOU ARE
SCANNING AROUND. FIRST WE CREATE
ANCHOR READY. WE TAPPED ON THE REAL
WORLD, CREATED ANCHOR. WE THEN MOVE
SOMEWHERE AND WE CREATED ANCHOR
B. IN THE GRAPH, A AND B ARE NOW
CONNECTED. AND THEY ARE CONNECTED
BECAUSE WE KNEW THE LOCATION OF
BOTH ANCHORS, AT THE SAME TIME.
IN THE SAME SESSION. WE THEN CLOSE
THAT SESSION AND START A NEW SESSION.
SO WE CLOSE THE APP AND WE OPEN
THE APP AGAIN. IF WE FIND ANCHOR
A, A AND B ARE STILL CONNECTED IN
THE GRAPH. IF WE THEN MOVE SOMEWHERE,
AND CREATE ANCHOR C, A AND C WILL
NOW BE CONNECTED. THEY'LL BE CONNECTED
BECAUSE WE KNEW THE LOCATION OF
BOTH OF THEM IN THE REAL WORLD.
IN THE SAME SESSION. NOW LET'S MOVE
SOMEWHERE ELSE. SO WE WALKED OVER
TO THAT CORNER. WE LOCATE ANCHOR
D. AND ANCHOR D IS AN ANCHOR THAT
WE PLACED A MONTH AGO AND IT HASN'T
INTERACTED BY ANYTHING SINCE. WE
ARE ABLE TO LOCATE IT BY ID. AND
WHAT HAPPENS THEN IS C&D ARE CONNECTED.
AND A AND D ARE CONNECTED. AND THEY
ARE CONNECTED AGAIN BECAUSE WE KNEW
THE LOCATION OF ALL OF THEM AT THE
SAME TIME. SO WHAT IS THIS MEANING
FOR YOU WHEN WRITING APPS? I SHOW
YOU HOW TO LOCATE ANCHORS BY ID.
ANOTHER WAY TO LOCATE ANCHORS. AND
THAT IS BY LOCATING ANCHORS CONNECTED
TO AN ANCHOR THAT YOU'VE ALREADY
LOCATED. SO LET'S IMAGINE FIRST
LOCATE AN ANCHOR BY ID. AND IN THAT
CALLBACK, WE GET BACK A CLOUD SPATIAL
ANCHOR. IT HAS THAT LOCAL ANCHOR
PROPERTY ON IT. AFTER THAT, WE WILL
THEN CREATE A NEW METHOD, LOCATE
NEARBY ANCHORS AND PASSED IN THAT
ANCHOR. THAT IS AN ANCHOR THAT IS
LOCATED. WE'LL CREATE A NEW CRY
TIERUE CALLED THIS NEAR ANCHOR CRITERIA,
SET THE SOURCE ANCHOR OF THE ANCHOR
THAT HAS BEEN ALREADY FOUND. AND
THEN WE WILL GO BACK TO ANCHOR LOCATE
CRITERIA, SET THE NEAR ANKOR AS
THAT NEAR ANCHOR CRY TIERUE AND
CREATE A WATCHER. SEMANTICALLY,
WHAT THAT MEANS, YOU FOUND THAT
CLOUD SPATIAL ANCHOR AND YOU WANT
TO FIND ANCHORS CONNECTED TO IT,
CONNECTED TO IT . AND YOU SET PROPERTY.
FIND ANCHOR WITHIN 2 METERS, 5 METER,
10 METERS. IF WE GO BACK TO DIAGRAM,
LET'S CHOOSE D TO BE SOURCE ANKOR.
WE CHOOSE D TO BE SOURCE ANCHOR,
WE WILL GET BACK A AND C. WE WILL
ALSO GET BACK B. BECAUSE ALL OF
THEM ARE CONNECTED IN THE GRAPH
. THINK ABOUT THIS ON LARGER SCALE,
FACTORY FLOOR, AMUSEMENT PARK, IF
THE ENTIRE AREA WAS CONNECTED WITH
ANCHOR, START IN ONE LOCATION AND
GET TO THE OTHER AND YOU CAN GET
TO THE OTHER IN THE REAL WORLD.
THESE ARE REAL POINTS. AS AN EXAMPLE
OF THIS, WE BUILT A DEMO APP INTERNALLY
AND WE ARE SEEING HERE THAT WE HAVE
A START LOCATION OF THE HALLWAY
CHAIRS AND WE WANT TO GET SOMEWHERE.
WE KNOW WHERE HALLWAY CHAIRS ARE,
WE CAN DO A LOCATE ANCHORS BY ID
CALL THERE. HERE WE ARE, CHOOSE
THE PLACE YOU WANT TO GET TO. WE
WANT TO GET T. LOCATE ANCHOR ON
HALLWAY CHAIRS. FIND IT. WE FOUND
IT. NOW A NEARBY QUERY. WE ARE FINDING
NEARBY ANCHORS. FIND T AND DISPLAY
ANCHOR TELLING, YOU YEAH, GO THAT
WAY. THE T IS OVER THERE. WE ARE
ABLE TO FIND IT. AND WE MADE IT
TO T. SUCCESSFULLY. SO WITH THIS
EXTENSION OF THAT APP THAT I SHOWED
YOU, I HOPE YOU CAN SEE HOW BY USING
NEARBY QUERIES TO FAN OUT IN THE
GRAPH, YOU CAN BOTH BUILD PERSISTENT
EXPERIENCES AND WAY FINDING EXPERIENCES.
SO WITH THAT SAID, I WANT TO MOVE
ON TO A THIRD POINT. AND IT IS ABOUT
UX CONSIDERATIONS. BEFORE WE JUMP
IN TOO MUCH I WANT TO SHOW YOU A
GIF AGAIN THAT YOU SEEN. I WANT
TO CALL IT SOMETHING NEW, THE FEATURE
POINTS THAT WE GET ARE SUPER, SUPER
IMPORTANT FOR HOW WE ARE ABLE TO
LOCATE ANCHORS. SO HERE IS US FINDING
THIS REALLY INTERESTING LOBBY. WE
HAD LOTS OF INTERESTING FEATURE
POINTS. IF YOU DIDN'T HAVE A GOOD
AREA TO LOOK AT, YOU PROBABLY WOULDN'T
HAVE AS MUCH SUCCESS. SO IN THE
RIGHT EXAMPLE, WHERE THE ROOM IS
ALL BLANK WALLS, WE DON'T ENCOURAGE
YOU TO USE AZURE SPATIAL ANCHORS
AS MUCH THERE. WE WANT YOU TO TARGET
INTERESTING VISUAL FEATURES AND
TO AVOID JUST A BLANK WHITE WALL.
SECOND POINT I WANT TO TALK ABOUT
IS CONSIDERING THE SCENARIO. SO
WHEN YOU CONSIDER THE SCENARIO,
I MEAN, THINK ABOUT WHO IS COMING
AFTER YOU AND GOING TO BE FINDING
YOUR ANCHORS. FOR EXAMPLE ARE YOU
BUILDING AN APP THAT LET'S YOU PLACE
AN ANCHOR AT THE TOP OF THE HIKING
TRAIL? YOU WANT SOMEONE COMING FROM
THE NORTH AND THE SOUTH AND THE
EAST AND THE WEST TO BE ABLE TO
FIND THAT ANCHOR. IF SO YOU PROBABLY
WANT TO PAN AROUND AND DO A 360
AS PLACING THE ANCHOR. RECORDED
THE ENVIRONMENT DATA ABOUT THE ENTIRE
SCENE. BUT MAYBE YOU ARE NOT. MAYBE
THERE IS AN FOR DATA VISUALIZATION
ON A CONFERENCE ROOM. AND IN THAT
CASE, IS IT AS IMPORTANT THAT YOU
AIM AT THE WALLS OR THE SIDE CORNERS?
MAYBE NOT. MAYBE IT IS MORE IMPORTANT
THAT YOU GET USERS TO SIT DOWN AND
PLACE AN ANCHOR IN A CENTRAL LOCATION
AND MAYBE BE SITTING WHILE DO SO.
OR HOW ABOUT IF YOU ARE BUILDING
AN APP FOR A MUSEUM WHERE YOU WANT
TO PLACE INFORMATION ON THE PAINTINGS,
SO THAT USERS CAN LEARN MORE ABOUT
THE FAMOUS PAINTING OR SEEING THEM.
IN THAT CASE, MAYBE YOU JUST WANT
TO SCAN AROUND THE PAINTING AND
IT IS NOT JUST AS IMPORTANT AS YOU
PAN AT THE BENCH BEHIND THE PAINTING.
SO AS AN EXAMPLE OF CONSIDERING
THE SCENARIO, WE FOUND THAT THERE
IS TWO MAIN WAYS THAT USERS FIND
ANCHORS. PROBABLY MANY MORE. SO
I WANT TO NOTE, DON'T TAKE THIS
AS SILVER BULLET. UX IS HARD. THE
FIRST IS A TARGET SCENARIO. AND
IN THE TARGET SCENARIO, YOU ARE
ABLE TO SHOW A PICTURE WHEN YOU
TELL THE USER TO LOOK FOR AN ANCHOR.
THIS CAN BE USEFUL BECAUSE MAYBE
YOU ARE BUILDING THAT MUSEUM APP.
AND YOU CAN GET ADMIN TO TAKE A
PICTURE OF THE APP, TAKE A PICTURE
OF THE PAINTING, WHEN THE USER GOES,
I KNOW THAT REMBRANT, I'LL PAINT
IT. POINT MY PHONE RIGHT THERE AND
FIND THE SPATIAL ANCHOR EASILY.
BUT THAT DOESN'T ALWAYS WORK. OFTENTIMES
YOU ARE IN MORE OF A ROOM SCENARIO
WHERE THE ANCHORS CAN BE ANYWHERE
AROUND YOU. AS AN EXAMPLE OF THIS,
WE BUILT AN ESCAPE THE ROOM EXPERIENCE.
AND IN IT, THE ANCHORS COULD BE
ABOVE YOU, BELOW YOU, BEHIND YOU
OR IN THE CORNER. THAT CASE, THERE
IS NO INTERESTING PICTURE TO SHOW
THEM. THE ANCHORS COULD BE ANYWHERE.
SO IT IS MORE IMPORTANT TO BUILD
UX THAN ENCOURAGE THE USER TO PAN
ALL AROUND THEM INCLUDING DOWN AND
UP INSTEAD OF JUST LOOKING AT ONE
THING. SO WITH THAT, I WANT TO SAY
THAT WE ARE LIVE. SO WE HAVE DOCS
UP, EXAMPLES ON GITHUB ON ALL PLATFORMS
AND LOTS OF OTHER RESOURCES. DEMO
CODE IS ON GITHUB. PLEASE TAKE A
PICTURE OF THIS AND CHECK IT OUT.
WE ALSO HAVE SAMPLES, DOCS, STACK
OVERFLOW, WE WOULD LOVE TO HEAR
YOU ON USER VOICE. OR THROUGH THAT
FEEDBACK EMAIL. WE TRY TO BE SUPER
RESPONSIVE THERE. AND A REALLY OPEN
TO WORKING WITH CUSTOMERS. ALSO,
CHECK OUT RENE'S TWITTER BECAUSE
HE HAS LOTS OF REALLY COOL VIDEOS.
>> THERE ARE SOME OTHER FUN BUILD
SESSIONS RELATED TO THIS WE ASK
YOU TO CHECK OUT AS WELL. >> IT
IS LIKE A 360. >> ALL RIGHT. AND
WE SHOULD ACTUALLY HAVE, SOME TIME
FOR QUESTIONS. GO AHEAD. PLEASE
STEP UP TO THE MICROPHONES AND ASK
YOUR QUESTIONS. >> AWESOME. AND
ASENTE, ENGINEER ON THE TEAM WILL
BE HERE TO ANSWER QUESTIONS AND
STICK AROUND AFTER THE SESSION.
>> I WANTED TO ASK, HOW ARE YOU
GETTING THE FEATURE POINTS FROM
ARK AND AR CORE? >> WE USE FOR SOME
SLAM AND BUILD ON TOP OF THEM TO
DO OUR OWN FEATURE POINTS FROM THE
CAMERA FRAMES YOU PASS IN. >> OKAY,
THAT IS LOCALLY ON THE DEVICE. >>
YES . >> YOU KNOW, SO WHEN UPLOADING
AN ANCHOR POINT, UPLOADING AN ANCHOR,
THE FEATURE POINTS THAT TRAVEL ALONG
WITH THAT DEFINITION, THOSE FEATURE
POINTS STORED WITH THAT ANCHOR ONLY?
OR ARE THEY STORED IN SORT OF A
GLOBAL EXPANDING AND ENRICHMENT
GLOBAL MESH OF FEATURE POINTS? >>
YOU WANT TO TAKE IT? >> THEY ARE
STORED TODAY WITH THE ANCHOR. YOU
UPLOAD IT. >> WITH THE ANCHOR ONLY.
>> YES. >> SO THERE IS NO REAL CONCEPT
AT THIS POINT OF THE FEATURE POINT
CLOUD. >> CORRECT. NO. >> WHICH
YOU CAN QUERY ANCHOR POINTS FROM
FEATURE POINTS. THAT IS SOMETHING
THAT YOU THINK IS A ROADMAP ITEM?
>> WE DO HAVE GRAPHICS EXPANSION
IN THE ROADMAP. WE HAVEN'T TACKLED
IT YET. >> AS OF TODAY, I QUERY
ANCHOR POINT BUSINESS SPECIFIC ENUMMERRATION
OR OTHER ATTRIBUTES BY GPS, ETC.,
THROUGH FEATURE POINTS. >> YES.
>> THANK YOU. >> HOW IS THE DIRECTION
BETWEEN TWO CONNECTED ANCHOR POINTS?
DOES IT MATTER HOW FAR THEY ARE
FROM EACH OTHER? WHEN YOU SAY ONE,
AND THEN ONE IN ONE SESSION. IS
IT KEEPING TRACK OF WHICH DIRECTION
I WENT? DOES IT DO THAT? >> SO YEAH,
BASICALLY, IT, WHEN YOU PLACE ONE
ANCHOR SESSION, WE ARE USING THE
TRACKER FROM AIR CORE OR AR KIT.
AND YOU ARE MORE SUSCEPTIBLE TO
ERRORS. IF YOU RECOMMEND GO FURTHER
ALONE, YOU MIGHT DROP ANCHORS WHILE
GOING THERE. WE HAVE PLACES TO FIND
THEM AS YOU GO. >> IN THE DIRECTION
ARROW THAT YOU WERE SHOWING, JUST
A BASICALLY A STRAIGHT LINE. NOT
REALLY TAKING INTO ACCOUNT. >> CORRECT.
>> THAT IS AN SPECIFIC . >> THAT
IS AN SPECIFIC. >> YES. >> AWESOME.
>> ANYMORE QUESTIONS? WE HAVE FOUR
MINUTES. >> YES. THIS GENTLEMAN.
>> SO DO YOU HAVE A DEVICE PACK
ON WHEN THESE COULD BE DISPLAYED?
SOME OF THE OLDER DEVICES DOESN'T
SHOW OFF THESE PROPERLY . >> SO
TODAY, ANY AR KIT ENABLED CASE OR
AR CORE, SHOULD BE ABLE TO HANDLE
THE STK. >> WHAT IS THE BEST RECOMMENDATION
TO SHOW THE HOLO MESSAGE? IF YOU
HAVE THAT ANCHOR, SHOWS THAT. IF
YOU HAVE ANCHOR ON THE [INDISCERNIBLE],
IT IS DIFFICULT TO GET TO OUR DATA,
I GUESS. >> CAN YOU REPEAT THE QUESTION?
CAN YOU REPEAT THE QUESTION? GET
CLOSER TO THE MIC MAYBE. >> WHAT
IS THE BEST RECOMMENDATION TO LOOK
AT THE, HOW DO YOU KNOW THE ANGLE
WHICH YOU ARE DISPLAYING SOME DATA?
I HAVE A DASHBOARD DISPLAYING REALTIME
DATA BUT MOST OF THE TIME IT IS
VERY DIFFICULT TO PULL IT UP BECAUSE
PEOPLE ARE NOT USED TO IT OR THEY
ARE HANDLING OR FIDDLING WITH THE
DEVICE. >> I STILL, LET ME FOLLOW-UP
WITH YOU AFTER. I'M NOT SURE IF
I UNDERSTAND THE QUESTION. >> THANK
YOU . >> COOL. AWESOME. >> THERE
IS ANOTHER ONE. >> LAST ONE. >>
THIS MAY NOT BE WORTH EVERYBODY'S
TIME. I'M CURIOUS AS A NEWBIE. IF
I MAYBE MADE AN APP AND GAVE IT
TO MY KID, YOU CAN PUT DINOSAURS
ALL OVER THE HOUSE. YOU WILL FIND
YOUR DINOSAURS. IS HE ABLE TO FIND
THOSE AFTER HE CLEANS HIS ROOM,
FOR EXAMPLE? IN THE LOCATION POINTS
CHANGE? >> SO WITHIN REASON, YES,
BUT IF YOU REMODEL THE ROOM, YOU
WILL PROBABLY NOT FIND THEM. THE
ALGORITHMS ARE ACTUALLY PURE VISUAL.
YOU CAN IMAGINE, IF YOU GO TO A
ROOM, RECOGNIZE AS A HUMAN, SAME
THING FOR THE CAMERA. IF YOU GO
IN AND THE ROOM IS VASTLY DIFFERENT,
YOU WILL PROBABLY HAVE TROUBLE FINDING
THEM. >> OKAY. THANKS. >> THIS IS
KIND OF RELATED BUT FEATURE POINTS
ARE VERY SPECIFIC TO LIGHTING AND
TO THE DATA MODEL. THAT IS BEING
USED AT THAT PARTICULAR TIME. AND
SO GOOGLE UPDATES THEIRS EVERY,
WHO KNOWS, WEEK, MONTH, WHATEVER.
IOS WILL ALSO, APPLE WILL ALSO DO
THAT. SO HOW HAVE YOU FOUND LIKE
AS YOU'RE DEVELOPING THIS, THE ACCURACY,
ARE YOU ABLE TO LIKE AS THEY PROGRESS
AND CHANGE, AS THE LIGHTING CHANGES,
ARE YOU GUYS ABLE TO KEEP THAT POINT
HERE. THE LIGHTS ARE 10 VERSUS 100.
>> SO TODAY, WE ARE, LIKE REALLY
STARTING ON ALL OF THIS, ALL OF
THIS TO DO PERSISTENCE. WE ARE LIKE
REALLY BETTING HEAVILY ON IT. WE
HAVEN'T DONE ALL OF THE EXPERIMENTS.
TODAY YOU BASICALLY RELY ON MULTIPLE
ANCHORS FOR DIFFERENT CONDITIONS.
THAT WILL BE BEST BET TO GET RECALL.
>> BUT YOU MIGHT REMEMBER IN THE
VIDEO I SHOWED YOU AT THE MARKET.
IT WAS A DIFFERENT FLOOR PANEL.
IT WAS DRY AND WET. DIFFERENT CONTRAST
FOR FINDING FEATURE POINTS. IT WORKED
SURPRISINGLY WELL. ALWAYS HAS LIMITATIONS.
BUT THE LEVEL WE ARE RIGHT NOW,
IT IS IMPRESSIVE, IN FACT. >> SO
RELATED TO THAT, WHEN YOU ARE, THE
ONE THAT YOU SHOWED WITH THE, SHOWING
AN ASSEMBLY, IF, IT IS GREAT AS
LONG AS THE ASSEMBLY IS IN THE SAME
PLACE BUT IF THAT, IF THEY MOVE
THAT ASSEMBLY, IT IS STILL THE SAME
ASSEMBLY, BUT THEY MOVE IT SLIGHTLY
IN THE ROOM BECAUSE THEY ARE CHANGING
THEIR SHOP FLOOR. DO YOU HAVE TO
THEN REORIENT EVERYTHING? OR DOES
IT, CAN YOU, ARE ALL OF THE FEATURE
POINTS JUST LOCATED ON THE ASSEMBLY
ITSELF? >> REALLY MATTERS, YOU KNOW,
DURING THE CLOUD SPECIAL ANCHOR
SESSION, WHAT YOU REALLY CAPTURE.
RIGHT? IF YOU WOULD JUST FOCUS ON
AUTO MOATIVE SEAT, -- AUTOMOTIVE
SEAT, THEN YOU WON'T. IF YOU CAPTURE,
CHANCE MUCH HIGHER THAT IT WILL
RELOCATE THAT. BUT YOU WANT TO ADD
SOMETHING TO THAT? >> THE ANCHOR
POSE IS RELATIVE TO THE FEATURE
POINTS. LIKE HE SAID, IF WHERE THE
SEAT IS, YOU ARE ONLY TAKING THAT
INTO ACCOUNT? IF YOU MOVE IT, THE
POST WILL BE RELATIVE TO THAT. WILL
BE MOVING WITH IT AT THAT POINT.
>> ALL RIGHT. WE ARE RIGHT AT THE
END. THANKS A LOT FOR ATTENDING.
