Party like it's 1970: conversational interfaces are back (into your app)
Catalan, Adrian
"Behold the promise land of AI. Machine learning has brought natural language processing right into our backyards and with it the possibilities of different ways of interaction such as conversations. But the challenges have evolved as recognizing the input can be done via an API (like DialogFlow) we also need to consider the interface and how to give an experience so natural that it seem like a human conversation. ""Oh hi how are you holding up? because I'm a potato"" it's not longer enough we need conversationally-enabled Android apps built with a different set of design concepts. In this talk we'll review examples of conversational interfaces, benefits of having one on your app, and the core principles for the design process of conversational experiences."
 Hello, everyone. My name is Sade Reang. I'm from Guatemala. I'm a GDE on Android, Firebase, and IoT. And I'm here to share a little bit about conversational interfaces. And the title make a reference to when this started. It's not something new. So my slides are going to be available, already are available on that URL in case you want to check it out. But let's get into business. So let's start with a very brief history lesson. Remember in the 70s? Everything started around 1952. There was some work in Bell Labs and Audrey software was able to recognize a string of digits. This was like the start of everything. Although it seems like something really easy in our days, at that time it was a breakthrough, recognizing the digits, strings of digits. And although things started in the 50s, it was until around the 70s where hidden models were developed. This helped a lot to recognize a speech and move away from pattern matching. This has happened several times in history. For a while we tried to recognize things using patterns. That's like one of the first steps. And then we find a way to move from pattern recognition into something not that literal, let's say. And around the 70s, also, several DARPA projects started to develop in this area. In the 80s, there was still a lot of brute force pattern matching and template work. But also hidden models became popular at last. It took around 10 years for this to happen. And around mid-80s, 1984, speech works was one of the first software working on IVR. IVR is when you call center and some computer replies like, if you know the extension number, dial it, or press zero for an operator. And it was until the 90s, four years later, that an industry started. In 1997, Dragon was the first recognition software of continuous speech. This was a big breakthrough for the industry. And in our days, 2007, Siri, Amazon and Google Home. And here we are in 2018, where it's really common to use Google Assistant or Alexa or Siri or any Assistant. And we are like in this phase where it's common to speak to our phones and wait for something to happen. We're going to cover several things. The first one is going to be the sign process. And I would like to ask a question because of the light, I can barely see everyone, but I would like to know, how many of you are developers? Raise your hands. Almost all of the audience. UX designers? No one? Oh, just one hand in the back. Great. Marketing? No? Okay. So I'm a developer, but in this talk, I'm covering several UX design, several parts of the UX design process. So the first step would be create a persona. Many of the things that I'm going to be covering in the talk may seem intuitive, but it's important to keep in mind that we need to have a checklist when working with conversational interface. So the idea here is to identify the main traits of the user that's going to be using the conversation, who is going to be talking with our app, and from there, keep doing more like specific segments in order to identify better and build this profile of a persona. If you have ever worked with branding or marketing, it's a common practice. However, in software development, it's not that common. And sometimes it's easier for developers to target a broad spectrum of users instead of the one group that's going to be using the app. Well, in this case, besides creating a persona, we need to focus on some specific traits. Because it's going to be a conversation, we need to focus on things like tone, style, technique, and voice. Even if we're using just one language, it's not the same way everywhere. There are accents, there are words, there are several things that change from region to region. And it's funny, mostly from my experience, I've been privileged enough to travel a lot mostly because of conferences, but there are several words that mean one thing like literally, but have another meaning in another region. Happened to me a lot in Latin America when speaking in different countries. So keep in mind that we need to focus on all this besides, of course, words and phrases, and the way in which we're going to communicate with our user. A lot of approaches here in the design process suggest that we think outside of box. It's like a cliché to hear or read that. Instead of thinking outside of box, for conversational interfaces, we need to destroy that imaginary box that we have and build something around tone, style, technique, and voice. So in order to do that, traditional thinking won't help us because we're not building a visual interface. We might have a visual interface for the conversation, but our main selling point is going to be a conversation. So we need to write a screenplay. It's similar to, like, working with a script or not a software script, but a script with several actors. We need to focus on the several parts of our conversation, which role will be playing each one of the participants, the user, and the software that we're developing, and write all that in a form of a screenplay. In this way, it will be easier for us to describe the flow that the user will be taking. We might be used to write or understand or use mock-ups and understand flow for our apps, but for conversational interfaces, although there are flows similar because the interface is different, a screenplay is needed. And there are several basic principles that we put in practice every time we have a conversation that, for some reason, it's not obvious when developing for a chatbot or other conversational interface. The first one and the most obvious is turn-taking. If we're having a conversation, in some moment, we're going to be speaking, and at the next, we're going to listen for the other party to speak. Also, context is really important. Mostly, that's why we have a programming languages and we don't program in natural language because of context. Computers are terrible understanding context. For some reason, we have not been able to develop something to make computers understand context, but humans, we are good with context. Also, threading and reading between the lines. This is something that's a challenge. Mostly because when we're having a conversation, there are several channels, but usually, it's not the same having a conversation in person. Let's say at the conference, this might be a great chance for a lot of people to meet in real life or having the same conversation over Twitter or over an IM app. When we're having a conversation, face-to-face in real life, usually, we can get some cues from our body language. But when having the same conversation over Twitter or social media or instant messaging app, we don't have those cues. The natural replacement is using some emojis. But still, it's not exactly the same. So, when developing the conversational interfaces, it's important to keep in mind the cooperation, much like when we're having a conversation, we are both in the same boat trying to achieve something and all this trace that I mentioned. Also, there are no errors. And this is an important thing to keep in mind. When we're having a conversation, we don't say to the other person, like, no, that's wrong. But not in terms of what you're saying. I consider it's wrong, but you are saying it wrong. That sounds weird. But anyway, what I'm trying to say is, when we're having a conversation, there are no errors. We might have different point of views, but both parties, or if it's a group conversation, if there are many people involved, are putting something to the table and adding some value to the conversation. There are no errors. In the same way, when dealing with an app, it's either an Android app or an Assistant app, something like Google Home or Alexa. If for some reason, the device says you are wrong, it's going to be something hard on the user. Instead of that, we have several branches. If the user, for some reason, doesn't reply what we're expecting, or if they reply it's wrong in some way, we're not going to say that's an error, like what we usually do in UI interface. Instead of that, we're going to take a different road, a different branch, and show a different part of the flow. This is important to keep in mind. We're not showing like a big red X saying to the user, you're wrong. Instead of that, we're taking a different path. And also, much like real life, interacting with a conversational app sometimes will add some challenges. Some of them might be temporarily, other might be for long term. So we need to consider that there will be some noise, that sometimes both the user or the app will need something to be repeated, and that in some moments, we will need a graphical reply. In other moments, it will be only audio. So with all this in mind, we need to keep those cues to understand. This is the main goal. We need to understand what the user is trying to say, and we need to be able to provide everything the user needs to understand what our app is trying to say. Understanding is key, and it's interesting, mostly because there are several barriers. When we're working remotely, developing software that is, we're working with themes that are located in several parts of the world, usually we communicate using English. And for many of us, it's not our first language. So we need to deal with a couple of things like accent, like the words and phrases that we're using, like cultural barriers. The same thing might happen here when developing a conversational interface. So keep in mind that understanding is critical, and the users are going to help us. Users are here to do something, to achieve something, and focus on that task instead of focus in using our app. Users know how to talk. Everybody knows how to ask for something for a conversational interface that might be, let's say, turning the lights on or turning the TV off. And we don't need to be taught how to say that or how to speak to the app. What we might need is a little help with the onboarding process. There might be several ways of saying something, so our app should consider all these different options and make the user's life a bit easier. Besides that, users also know what they want, how do they want to be done, and basically we just need to leverage on that, mostly because we share the same goal and we're trying to understand each other. As I've said before, the context is really important. The context matters. It's not the same to... Might not be the same to say the same phrase in two different contexts. We are basically good to make a... To infer this context, but as I've said, the computer, it's not as good as humans, so we need to deal with this. If we're having a conversation and someone pops in, it's a bit easier to get into track and understand what's happening. But that context might lead us to different directions. So if we don't have a clear understanding of what's the goal, if the user, it's not understanding what the app is trying to say in this context, things are not going to work out. And it's important to have a backup plan in cases when something happening, like the user, it's not replying. There's important to have a timeout. When we're having a conversation, it's funny because if you're having a conversation face to face, because we are doing this in a synchronous way, the timeout that we have, it's almost non-existent. If we're speaking face to face and for some reason, I say something and the other person just stares at me, it's going to be awkward, right? Sometimes when we're having a conversation, we're thinking about something and it might... We might spend a couple of seconds thinking, I don't remember this, and it's quite common. But when we're having the same conversation over, let's say, an I am app, like, what's app, that's not synchronous. So we might have to wait for a while and it's okay. When we're dealing with a conversational interface, usually it's synchronous, but the user might not reply because something happened, like some interruption, like someone appeared and started a conversation in person with me or I'm dealing with something and the app is still waiting for some input. In this case, we're going to deal with some timeout and that's just one example, but it's important to keep in mind that having a backup plan might let us into a good path, might bring us back into a happy path of the user of our app. Here's an example. This is a sample dialogue with the Google home and an app called number genie. You're supposed to guess a number between 0 and 100. In this case, user says nothing, like around here, and the reply from the app is, I didn't hear a number. It's not an error. It's not you're wrong. It's I didn't hear. That might be because noise, because you didn't say anything or because any reason. But that's what I meant when I say there are no errors. Then it's silent or muffled. It might happen that there's too much noise or maybe it's not the user speaking. Maybe it's a dog barking or something. So the next reply, it's, I don't know, if you're still there, what's your guess? The wording, the phrases, the tone, it's really important for the conversation to be in the happy path. We need to provide the canonical happy path, but consider all these possible branches, and also we need to consider that it might be a first-time experience or a return user. We already know how to have a conversation, but we might need a bit of help on an onboarding process, or if we have used this app several times, we would like to, we will appreciate for our expertise to be considered from the app. This is another example. This is for the game to quit. In this case, the user tried to guess 21 right here. It's not 21, so the user give up. In this case, the app should catch that the user is giving up, quitting, and give a good ending for the conversation. Sure, I'll tell you the number anyway. It was nice. Besides that, I would like to provide some snacks. Some people call these best practices. I think each one of these bullets might imply a discussion, so that's why I'm calling all of these snacks. I would be happy to discuss any ideas that you have after the talk, but let's review some of these ideas on how we are going to be able to build something nice and pretty when we don't have exactly some graphical interface. We should avoid written language. This is important, mostly because when we are developing a graphical UI, usually we focus on some wording that we're used to see in a written way. In this case, we might have a supporting graphical interface, but the main interface is going to be a conversational one just maybe with the Google Assistant that is. We might have some feedback on the app on the phone, but usually it's just the voice from the Assistant speaking to us. It's important to kick off the conversation in a good way. We need to keep a balance on engaging with the user, but not so much that we're going to nag the user, and it's something similar with the notifications. For maybe about three or four years, I've been living without notifications, and it's an interesting experience with the new do not disturb mode in Android. It's been easier for me to configure that on my phone, but for some apps, I leave the notifications on, like maps, mostly when I'm traveling, but lately I've been getting so many notifications from apps that I ended up blocking the app. This in mind, the idea here is to engage with the user just in the correct amount of interactions. We also will need to guide the user through the conversation, although we're not explaining how to have a conversation, nor we are leading all the conversation, while just guiding. And keep our text-to-speech interactions short and clear just in case the user is not able to understand for any reason. It's also a good idea to avoid any data points unrelated to the user query. Usually, when we're having a conversation, we like to be on a direct path. Many people, like me, tend to ramble when speaking. So when having a conversation with someone who rambles a lot, it's a good idea to stay in the path. The same thing with a conversational interface. We should not ramble. We should just give the user whatever they need for this query, for whatever they're trying to achieve. And follow the natural turn taking in order to keep the conversation like something that we will have with a person. It's also a good idea to use conversational makers to keep the user engaged. And finally, with all this solved, we're going to add some... With some machine learning. As the abstract said, we don't need to deal anymore with NLP. Basically everything is solved, everything, let's say. And there are many tools that help us dealing with the language processing. So instead of developing the whole stack of recognizing the input for the conversation to happen, we can use a tool, a framework, like Dialog Flow. There are many other options. For me, this is one of the best available tools. And basically Dialog Flow works with two things. Intent matching and entity extraction. It's going to categorize whatever the user is trying to achieve into an intent. And also it's going to identify something that's relevant for that task to be achieved. In order to do that, or in order to take the next step, we're going to take those intent and entities and do something with that. And eventually reply to the user. Basically the flow is this. The user is going to input something either with a key or a microphone, and that's going to turn into a query. Dialog Flow is going to receive that query, identify the intent, and process it. It might be with something simple, like Firebase Cloud Function, or AWS Lambda, or whatever you would like to use for serverless. Or it might be something more complex with your own backend. Either way, we're going to have external APIs, maybe a database, some code to do this processing. And when we have all this settled, we're going to reply to the user and provide a fulfillment with actionable data. The idea here is that Dialog Flow is doing all the heavy lifting, and we only need to identify what's the user is trying to achieve, what's the main goal, and which pieces of data are relevant to that goal. We're going to start with an invoke procedure. We need to say hello and start experience with the agent, with the software that we're building our app. In Dialog Flow, we're going to define these intents. This is just an example, an intent for movie recommendations, and we need to provide training phrases. Like, I want to watch something, or tell me what to watch, or recommend me a movie. These are different ways of saying the same thing. The idea here is to get a movie recommendation. Then we need to provide the entities, in this case, it's a movie, and we're going to provide some synonyms. Like, for sci-fi, we don't have a synonym, but for comedy, we have funny movies, or for drama, we have serious movies. And eventually, we're going to use that webhook data that I mentioned before with a backend to provide a fulfillment. The response, it's also specified here, because we are providing the user something like a phrase. Keep in mind that we're doing a conversation here. We're not just providing some unconnected pieces of data. We're having a full conversation here. So the response might be, you should give it a try and watch a comedy, grab the popcorn, it's movie night, just in case the user is looking for a random recommendation, or have out a sci-fi for tonight. We use a context to move between intents. Might be that the user is looking forward for something more than just a movie recommendation. Maybe we would like to discuss, like, a specific actor or actress, or maybe the user is looking for a recommendation, like, based on movies that they have already seen before. With this context in mind, we can achieve more fluent conversations. Instead of one request and reply, it's a conversation that is going to be, it's going to take many turns between the app and the user and eventually provide some recommendation, but with all the context shared between those interactions. One of the main things that I was looking forward on Google recently launched at the IEO is that using the Google Home, it's really nice, but you need to say, okay, Google, or hey, Google, for each one of the requests. Now they are launching this new feature for you to not say that in every request just to start a new conversation. And finally, this is all the code that we're going to see. In order to integrate this into our Android app, we need to use an AI service. We're going to be using an access token. Dial flow is not that different from other API providers. It got its own SDK. It also needs some registration, provide an access token. In this case, we're going to click a button and listen for input. And afterwards, this is the result. And here we're just showing what the app is providing. We have a query, we have an action, we have several parameters. And all this depends on whatever we configure on Dial Flow. It's interesting for me mostly because we are dealing with configuration. Not everything is done in code. Although if you really like to code, this might sound like different. But it's like a trend that I've been seeing for a while that it works better if some parts of the process are just configuration, like defining the intents and identifying the entities and other parts of the process are dealing with both the input and the output with code. And basically, that's all that we need in order to deal with our interaction with Dial Flow. It supports several languages and it provides several more features than what I cover here. So to summarize, many of the ideas behind conversational interfaces are things that have been around for a while. Many of the things that are useful are quite intuitive. So in order to build an app that got some conversation in any point for any goal for the user, we need to keep in mind that it's important to build a persona using tone, words, phrases, and the technique that we would like for that conversation. There are no errors. There's the happy path and many other paths. So we need a backup plan in order to get the user back in track in case anything happens. The user is there to help us, already knows how to have a conversation, already knows how to achieve things. We need to leverage that. And basically, the idea here is having a script for a couple of interactions. If it's your first approach, I would suggest writing at least three different scenarios, the happy path, and then considering several options in order to build the full screenplay that we're going to have. And there are many other small things like avoiding written language, keeping the text-to-speech to shortest interactions, but mainly the idea here is make the experience the most wonderful that we can build for the user. So that was what I had. Thank you very much. Do we have time for questions? Yes. Are there any questions? Because we still have time. Over there. Perfect. Thank you. I was just curious about if you have a conversational interface, what's going to happen with all the flows, the existing flows that you have? Because it basically leaves us with no job? Well, it really depends on what are we looking forward to. I mean, there are several things that are going to be automated, but I don't see this happening as quick cuts for all the fields are for some fields. This might be not completely related to the presentation, but in my opinion, in a couple of years, software development is not going to be as good as it's now in terms of wages, mostly because many things are going to be automated, but it's not the only job that's going to change. It's not going to disappear, it's going to change. With that in mind, many of the apps that we're developing nowadays are going to have something like a conversational interface inside. It's going to integrate with the flow that we already have. Let's say we're building a movie recommendation app. Right now, there are like a screen with several buttons and images. Instead of that, we're also going to have first, maybe a small button that says chat with someone. It's going to be that agent recommending movies. Eventually, it's going to be like 50-50. Eventually, if anything goes like the way it seems going now, it's going to be fully automated with a conversational interface. I'm not sure it's going to happen. Chatbots have been around for a couple of years that are not as adopted as I would have thought, but those are my ideas in the coming weeks.
 Hello, everyone. My name is Sade Reang. I'm from Guatemala. I'm a GDE on Android, Firebase,  and IoT. And I'm here to share a little bit about conversational interfaces. And the title  make a reference to when this started. It's not something new. So my slides are going  to be available, already are available on that URL in case you want to check it out.  But let's get into business. So let's start with a very brief history lesson.  Remember in the 70s? Everything started around 1952. There was some work in Bell Labs and  Audrey software was able to recognize a string of digits. This was like the start of everything.  Although it seems like something really easy in our days, at that time it was a breakthrough,  recognizing the digits, strings of digits. And although things started in the 50s, it  was until around the 70s where hidden models were developed. This helped a lot to recognize  a speech and move away from pattern matching. This has happened several times in history.  For a while we tried to recognize things using patterns. That's like one of the first steps.  And then we find a way to move from pattern recognition into something not that literal,  let's say. And around the 70s, also, several DARPA projects started to develop in this  area. In the 80s, there was still a lot of brute force pattern matching and template  work. But also hidden models became popular at last. It took around 10 years for this  to happen. And around mid-80s, 1984, speech works was one of the first software working  on IVR. IVR is when you call center and some computer replies like, if you know the extension  number, dial it, or press zero for an operator. And it was until the 90s, four years later,  that an industry started. In 1997, Dragon was the first recognition software of continuous  speech. This was a big breakthrough for the industry. And in our days, 2007, Siri, Amazon  and Google Home. And here we are in 2018, where it's really common to use Google Assistant  or Alexa or Siri or any Assistant. And we are like in this phase where it's common to  speak to our phones and wait for something to happen. We're going to cover several things.  The first one is going to be the sign process. And I would like to ask a question because  of the light, I can barely see everyone, but I would like to know, how many of you are  developers? Raise your hands. Almost all of the audience. UX designers? No one? Oh, just  one hand in the back. Great. Marketing? No? Okay. So I'm a developer, but in this talk,  I'm covering several UX design, several parts of the UX design process. So the first step  would be create a persona. Many of the things that I'm going to be covering in the talk  may seem intuitive, but it's important to keep in mind that we need to have a checklist  when working with conversational interface. So the idea here is to identify the main traits  of the user that's going to be using the conversation, who is going to be talking with our app,  and from there, keep doing more like specific segments in order to identify better and build  this profile of a persona. If you have ever worked with branding or marketing, it's a  common practice. However, in software development, it's not that common. And sometimes it's  easier for developers to target a broad spectrum of users instead of the one group that's going  to be using the app. Well, in this case, besides creating a persona, we need to focus on some  specific traits. Because it's going to be a conversation, we need to focus on things  like tone, style, technique, and voice. Even if we're using just one language, it's not  the same way everywhere. There are accents, there are words, there are several things  that change from region to region. And it's funny, mostly from my experience, I've been  privileged enough to travel a lot mostly because of conferences, but there are several words  that mean one thing like literally, but have another meaning in another region. Happened  to me a lot in Latin America when speaking in different countries. So keep in mind that  we need to focus on all this besides, of course, words and phrases, and the way in which we're  going to communicate with our user. A lot of approaches here in the design process  suggest that we think outside of box. It's like a cliché to hear or read that. Instead  of thinking outside of box, for conversational interfaces, we need to destroy that imaginary  box that we have and build something around tone, style, technique, and voice. So in order  to do that, traditional thinking won't help us because we're not building a visual interface.  We might have a visual interface for the conversation, but our main selling point is going to be  a conversation. So we need to write a screenplay. It's similar to, like, working with a script  or not a software script, but a script with several actors. We need to focus on the several  parts of our conversation, which role will be playing each one of the participants, the  user, and the software that we're developing, and write all that in a form of a screenplay.  In this way, it will be easier for us to describe the flow that the user will be taking. We might  be used to write or understand or use mock-ups and understand flow for our apps, but for  conversational interfaces, although there are flows similar because the interface is  different, a screenplay is needed. And there are several basic principles that we put in  practice every time we have a conversation that, for some reason, it's not obvious when  developing for a chatbot or other conversational interface. The first one and the most obvious  is turn-taking. If we're having a conversation, in some moment, we're going to be speaking,  and at the next, we're going to listen for the other party to speak. Also, context is  really important. Mostly, that's why we have a programming languages and we don't program  in natural language because of context. Computers are terrible understanding context. For some  reason, we have not been able to develop something to make computers understand context, but  humans, we are good with context. Also, threading and reading between the lines. This is something  that's a challenge. Mostly because when we're having a conversation, there are several channels,  but usually, it's not the same having a conversation in person. Let's say at the conference, this  might be a great chance for a lot of people to meet in real life or having the same conversation  over Twitter or over an IM app. When we're having a conversation, face-to-face in real  life, usually, we can get some cues from our body language. But when having the same conversation  over Twitter or social media or instant messaging app, we don't have those cues. The natural  replacement is using some emojis. But still, it's not exactly the same. So, when developing  the conversational interfaces, it's important to keep in mind the cooperation, much like  when we're having a conversation, we are both in the same boat trying to achieve something  and all this trace that I mentioned. Also, there are no errors. And this is an important  thing to keep in mind. When we're having a conversation, we don't say to the other person,  like, no, that's wrong. But not in terms of what you're saying. I consider it's wrong,  but you are saying it wrong. That sounds weird. But anyway, what I'm trying to say is, when  we're having a conversation, there are no errors. We might have different point of views, but  both parties, or if it's a group conversation, if there are many people involved, are putting  something to the table and adding some value to the conversation. There are no errors. In  the same way, when dealing with an app, it's either an Android app or an Assistant app,  something like Google Home or Alexa. If for some reason, the device says you are wrong,  it's going to be something hard on the user. Instead of that, we have several branches.  If the user, for some reason, doesn't reply what we're expecting, or if they reply it's  wrong in some way, we're not going to say that's an error, like what we usually do in  UI interface. Instead of that, we're going to take a different road, a different branch,  and show a different part of the flow. This is important to keep in mind. We're not showing  like a big red X saying to the user, you're wrong. Instead of that, we're taking a different  path. And also, much like real life, interacting with a conversational app sometimes will add  some challenges. Some of them might be temporarily, other might be for long term. So we need to  consider that there will be some noise, that sometimes both the user or the app will need  something to be repeated, and that in some moments, we will need a graphical reply. In  other moments, it will be only audio. So with all this in mind, we need to keep those  cues to understand. This is the main goal. We need to understand what the user is trying  to say, and we need to be able to provide everything the user needs to understand what  our app is trying to say. Understanding is key, and it's interesting, mostly because  there are several barriers. When we're working remotely, developing software that is, we're  working with themes that are located in several parts of the world, usually we communicate  using English. And for many of us, it's not our first language. So we need to deal with  a couple of things like accent, like the words and phrases that we're using, like cultural  barriers. The same thing might happen here when developing a conversational interface.  So keep in mind that understanding is critical, and the users are going to help us. Users  are here to do something, to achieve something, and focus on that task instead of focus in  using our app. Users know how to talk. Everybody knows how to ask for something for a conversational  interface that might be, let's say, turning the lights on or turning the TV off. And we  don't need to be taught how to say that or how to speak to the app. What we might need  is a little help with the onboarding process. There might be several ways of saying something,  so our app should consider all these different options and make the user's life a bit easier.  Besides that, users also know what they want, how do they want to be done, and basically  we just need to leverage on that, mostly because we share the same goal and we're trying to  understand each other. As I've said before, the context is really important. The context  matters. It's not the same to... Might not be the same to say the same phrase in two  different contexts. We are basically good to make a... To infer this context, but as  I've said, the computer, it's not as good as humans, so we need to deal with this. If  we're having a conversation and someone pops in, it's a bit easier to get into track and  understand what's happening. But that context might lead us to different directions. So  if we don't have a clear understanding of what's the goal, if the user, it's not understanding  what the app is trying to say in this context, things are not going to work out. And it's  important to have a backup plan in cases when something happening, like the user, it's not  replying. There's important to have a timeout. When we're having a conversation, it's funny  because if you're having a conversation face to face, because we are doing this in a synchronous  way, the timeout that we have, it's almost non-existent. If we're speaking face to face  and for some reason, I say something and the other person just stares at me, it's going  to be awkward, right? Sometimes when we're having a conversation, we're thinking about  something and it might... We might spend a couple of seconds thinking, I don't remember  this, and it's quite common. But when we're having the same conversation over, let's say,  an I am app, like, what's app, that's not synchronous. So we might have to wait for a while and it's  okay. When we're dealing with a conversational interface, usually it's synchronous, but the  user might not reply because something happened, like some interruption, like someone appeared  and started a conversation in person with me or I'm dealing with something and the app  is still waiting for some input. In this case, we're going to deal with some timeout and that's  just one example, but it's important to keep in mind that having a backup plan might let  us into a good path, might bring us back into a happy path of the user of our app. Here's  an example. This is a sample dialogue with the Google home and an app called number genie.  You're supposed to guess a number between 0 and 100. In this case, user says nothing,  like around here, and the reply from the app is, I didn't hear a number. It's not an error.  It's not you're wrong. It's I didn't hear. That might be because noise, because you didn't  say anything or because any reason. But that's what I meant when I say there are no errors.  Then it's silent or muffled. It might happen that there's too much noise or maybe it's  not the user speaking. Maybe it's a dog barking or something. So the next reply, it's, I don't  know, if you're still there, what's your guess? The wording, the phrases, the tone, it's  really important for the conversation to be in the happy path. We need to provide the  canonical happy path, but consider all these possible branches, and also we need to consider  that it might be a first-time experience or a return user. We already know how to have  a conversation, but we might need a bit of help on an onboarding process, or if we have  used this app several times, we would like to, we will appreciate for our expertise to  be considered from the app. This is another example. This is for the game to quit. In  this case, the user tried to guess 21 right here. It's not 21, so the user give up. In  this case, the app should catch that the user is giving up, quitting, and give a good ending  for the conversation. Sure, I'll tell you the number anyway. It was nice. Besides that,  I would like to provide some snacks. Some people call these best practices. I think each one  of these bullets might imply a discussion, so that's why I'm calling all of these snacks.  I would be happy to discuss any ideas that you have after the talk, but let's review  some of these ideas on how we are going to be able to build something nice and pretty  when we don't have exactly some graphical interface. We should avoid written language.  This is important, mostly because when we are developing a graphical UI, usually we focus  on some wording that we're used to see in a written way. In this case, we might have  a supporting graphical interface, but the main interface is going to be a conversational  one just maybe with the Google Assistant that is. We might have some feedback on the app  on the phone, but usually it's just the voice from the Assistant speaking to us. It's important  to kick off the conversation in a good way. We need to keep a balance on engaging with  the user, but not so much that we're going to nag the user, and it's something similar  with the notifications. For maybe about three or four years, I've been living without notifications,  and it's an interesting experience with the new do not disturb mode in Android. It's been  easier for me to configure that on my phone, but for some apps, I leave the notifications  on, like maps, mostly when I'm traveling, but lately I've been getting so many notifications  from apps that I ended up blocking the app. This in mind, the idea here is to engage with  the user just in the correct amount of interactions. We also will need to guide the user through  the conversation, although we're not explaining how to have a conversation, nor we are leading  all the conversation, while just guiding. And keep our text-to-speech interactions short  and clear just in case the user is not able to understand for any reason. It's also a  good idea to avoid any data points unrelated to the user query. Usually, when we're having  a conversation, we like to be on a direct path. Many people, like me, tend to ramble  when speaking. So when having a conversation with someone who rambles a lot, it's a good  idea to stay in the path. The same thing with a conversational interface. We should  not ramble. We should just give the user whatever they need for this query, for whatever they're  trying to achieve. And follow the natural turn taking in order to keep the conversation  like something that we will have with a person. It's also a good idea to use conversational  makers to keep the user engaged. And finally, with all this solved, we're going to add some ... With some machine learning. As the abstract said, we don't need to deal anymore with NLP.  Basically everything is solved, everything, let's say. And there are many tools that help  us dealing with the language processing. So instead of developing the whole stack of recognizing  the input for the conversation to happen, we can use a tool, a framework, like Dialog  Flow. There are many other options. For me, this is one of the best available tools. And  basically Dialog Flow works with two things. Intent matching and entity extraction. It's  going to categorize whatever the user is trying to achieve into an intent. And also it's going  to identify something that's relevant for that task to be achieved. In order to do that,  or in order to take the next step, we're going to take those intent and entities and do something  with that. And eventually reply to the user. Basically the flow is this. The user is going  to input something either with a key or a microphone, and that's going to turn into a query. Dialog  Flow is going to receive that query, identify the intent, and process it. It might be with  something simple, like Firebase Cloud Function, or AWS Lambda, or whatever you would like  to use for serverless. Or it might be something more complex with your own backend. Either  way, we're going to have external APIs, maybe a database, some code to do this processing.  And when we have all this settled, we're going to reply to the user and provide a fulfillment  with actionable data. The idea here is that Dialog Flow is doing all the heavy lifting,  and we only need to identify what's the user is trying to achieve, what's the main goal,  and which pieces of data are relevant to that goal. We're going to start with an invoke procedure.  We need to say hello and start experience with the agent, with the software that we're  building our app. In Dialog Flow, we're going to define these intents. This is just an example,  an intent for movie recommendations, and we need to provide training phrases. Like, I  want to watch something, or tell me what to watch, or recommend me a movie. These are  different ways of saying the same thing. The idea here is to get a movie recommendation.  Then we need to provide the entities, in this case, it's a movie, and we're going to provide  some synonyms. Like, for sci-fi, we don't have a synonym, but for comedy, we have funny  movies, or for drama, we have serious movies. And eventually, we're going to use that webhook  data that I mentioned before with a backend to provide a fulfillment. The response, it's  also specified here, because we are providing the user something like a phrase. Keep in  mind that we're doing a conversation here. We're not just providing some unconnected  pieces of data. We're having a full conversation here. So the response might be, you should  give it a try and watch a comedy, grab the popcorn, it's movie night, just in case the  user is looking for a random recommendation, or have out a sci-fi for tonight.  We use a context to move between intents. Might be that the user is looking forward for something  more than just a movie recommendation. Maybe we would like to discuss, like, a specific  actor or actress, or maybe the user is looking for a recommendation, like, based on movies  that they have already seen before. With this context in mind, we can achieve more fluent  conversations. Instead of one request and reply, it's a conversation that is going to  be, it's going to take many turns between the app and the user and eventually provide  some recommendation, but with all the context shared between those interactions. One of  the main things that I was looking forward on Google recently launched at the IEO is  that using the Google Home, it's really nice, but you need to say, okay, Google, or hey,  Google, for each one of the requests. Now they are launching this new feature for you  to not say that in every request just to start a new conversation. And finally, this is all  the code that we're going to see. In order to integrate this into our Android app, we  need to use an AI service. We're going to be using an access token. Dial flow is not  that different from other API providers. It got its own SDK. It also needs some registration,  provide an access token. In this case, we're going to click a button and listen for input.  And afterwards, this is the result. And here we're just showing what the app is providing.  We have a query, we have an action, we have several parameters. And all this depends on  whatever we configure on Dial Flow. It's interesting for me mostly because we are dealing with  configuration. Not everything is done in code. Although if you really like to code, this  might sound like different. But it's like a trend that I've been seeing for a while  that it works better if some parts of the process are just configuration, like defining  the intents and identifying the entities and other parts of the process are dealing with  both the input and the output with code. And basically, that's all that we need in order  to deal with our interaction with Dial Flow. It supports several languages and it provides  several more features than what I cover here. So to summarize, many of the ideas behind  conversational interfaces are things that have been around for a while. Many of the  things that are useful are quite intuitive. So in order to build an app that got some  conversation in any point for any goal for the user, we need to keep in mind that it's  important to build a persona using tone, words, phrases, and the technique that we would like  for that conversation. There are no errors. There's the happy path and many other paths.  So we need a backup plan in order to get the user back in track in case anything happens.  The user is there to help us, already knows how to have a conversation, already knows  how to achieve things. We need to leverage that. And basically, the idea here is having  a script for a couple of interactions. If it's your first approach, I would suggest  writing at least three different scenarios, the happy path, and then considering several  options in order to build the full screenplay that we're going to have. And there are many  other small things like avoiding written language, keeping the text-to-speech to shortest interactions,  but mainly the idea here is make the experience the most wonderful that we can build for the  user. So that was what I had. Thank you very much.  Do we have time for questions?  Yes. Are there any questions? Because we still have time.  Over there.  Perfect. Thank you.  I was just curious about if you have a conversational interface, what's going to happen with all  the flows, the existing flows that you have? Because it basically leaves us with no job?  Well, it really depends on what are we looking forward to. I mean, there are several things  that are going to be automated, but I don't see this happening as quick cuts for all the  fields are for some fields. This might be not completely related to the presentation,  but in my opinion, in a couple of years, software development is not going to be as good as  it's now in terms of wages, mostly because many things are going to be automated, but  it's not the only job that's going to change. It's not going to disappear, it's going to  change. With that in mind, many of the apps that we're developing nowadays are going to  have something like a conversational interface inside. It's going to integrate with the flow  that we already have. Let's say we're building a movie recommendation app. Right now, there  are like a screen with several buttons and images. Instead of that, we're also going  to have first, maybe a small button that says chat with someone. It's going to be that agent  recommending movies. Eventually, it's going to be like 50-50. Eventually, if anything  goes like the way it seems going now, it's going to be fully automated with a conversational  interface. I'm not sure it's going to happen. Chatbots have been around for a couple of  years that are not as adopted as I would have thought, but those are my ideas in the coming  weeks.
_DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_0_707.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_708_1115.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_1116_2915.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_2916_4067.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_4068_4883.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_4884_5207.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_5208_7739.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_7740_9203.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_9204_9815.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_9816_9911.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_9912_10331.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_10332_11951.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_11952_15095.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_15096_17735.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_17736_18935.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_18936_20363.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_20364_20699.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_20700_21731.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_21732_22031.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_22032_22235.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_22236_23735.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_23736_26303.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_26304_28775.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_28776_29495.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_29496_29783.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_29784_32855.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_32856_33539.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_33540_34691.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_34692_34871.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_34872_35399.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_35400_37139.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_37140_38831.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_38832_39083.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_39084_39275.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_39276_39323.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_39324_39755.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_39756_40175.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_40176_40391.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_40392_41159.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_41160_43055.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_43056_43943.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_43944_45287.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_45288_48455.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_48456_52019.png _DCBerlin18_107_Catalan_PARTY_LIKE_IT_S_1970_CONVE_3kA3vG4rLLo_52020_52033.png
Hello, everyone. My name is Sade Reang. I'm from Guatemala. I'm a GDE on Android, Firebase, and IoT. And I'm here to share a little bit about conversational interfaces. And the title make a reference to when this started. It's not something new. So my slides are going to be available, already are available on that URL in case you want to check it out. But let's get into business. So let's start with a very brief history lesson. Remember in the 70s? Everything started around 1952. There was some work in Bell Labs and Audrey software was able to recognize a string of digits. This was like the start of everything. Although it seems like something really easy in our days, at that time it was a breakthrough, recognizing the digits, strings of digits. And although things started in the 50s, it was until around the 70s where hidden models were developed. This helped a lot to recognize a speech and move away from pattern matching. This has happened several times in history. For a while we tried to recognize things using patterns. That's like one of the first steps. And then we find a way to move from pattern recognition into something not that literal, let's say. And around the 70s, also, several DARPA projects started to develop in this area. In the 80s, there was still a lot of brute force pattern matching and template work. But also hidden models became popular at last. It took around 10 years for this to happen. And around mid-80s, 1984, speech works was one of the first software working on IVR. IVR is when you call center and some computer replies like, if you know the extension number, dial it, or press zero for an operator. And it was until the 90s, four years later, that an industry started. In 1997, Dragon was the first recognition software of continuous speech. This was a big breakthrough for the industry. And in our days, 2007, Siri, Amazon and Google Home. And here we are in 2018, where it's really common to use Google Assistant or Alexa or Siri or any Assistant. And we are like in this phase where it's common to speak to our phones and wait for something to happen. We're going to cover several things. The first one is going to be the sign process. And I would like to ask a question because of the light, I can barely see everyone, but I would like to know, how many of you are developers? Raise your hands. Almost all of the audience. UX designers? No one? Oh, just one hand in the back. Great. Marketing? No? Okay. So I'm a developer, but in this talk, I'm covering several UX design, several parts of the UX design process. So the first step would be create a persona. Many of the things that I'm going to be covering in the talk may seem intuitive, but it's important to keep in mind that we need to have a checklist when working with conversational interface. So the idea here is to identify the main traits of the user that's going to be using the conversation, who is going to be talking with our app, and from there, keep doing more like specific segments in order to identify better and build this profile of a persona. If you have ever worked with branding or marketing, it's a common practice. However, in software development, it's not that common. And sometimes it's easier for developers to target a broad spectrum of users instead of the one group that's going to be using the app. Well, in this case, besides creating a persona, we need to focus on some specific traits. Because it's going to be a conversation, we need to focus on things like tone, style, technique, and voice. Even if we're using just one language, it's not the same way everywhere. There are accents, there are words, there are several things that change from region to region. And it's funny, mostly from my experience, I've been privileged enough to travel a lot mostly because of conferences, but there are several words that mean one thing like literally, but have another meaning in another region. Happened to me a lot in Latin America when speaking in different countries. So keep in mind that we need to focus on all this besides, of course, words and phrases, and the way in which we're going to communicate with our user. A lot of approaches here in the design process suggest that we think outside of box. It's like a cliché to hear or read that. Instead of thinking outside of box, for conversational interfaces, we need to destroy that imaginary box that we have and build something around tone, style, technique, and voice. So in order to do that, traditional thinking won't help us because we're not building a visual interface. We might have a visual interface for the conversation, but our main selling point is going to be a conversation. So we need to write a screenplay. It's similar to, like, working with a script or not a software script, but a script with several actors. We need to focus on the several parts of our conversation, which role will be playing each one of the participants, the user, and the software that we're developing, and write all that in a form of a screenplay. In this way, it will be easier for us to describe the flow that the user will be taking. We might be used to write or understand or use mock-ups and understand flow for our apps, but for conversational interfaces, although there are flows similar because the interface is different, a screenplay is needed. And there are several basic principles that we put in practice every time we have a conversation that, for some reason, it's not obvious when developing for a chatbot or other conversational interface. The first one and the most obvious is turn-taking. If we're having a conversation, in some moment, we're going to be speaking, and at the next, we're going to listen for the other party to speak. Also, context is really important. Mostly, that's why we have a programming languages and we don't program in natural language because of context. Computers are terrible understanding context. For some reason, we have not been able to develop something to make computers understand context, but humans, we are good with context. Also, threading and reading between the lines. This is something that's a challenge. Mostly because when we're having a conversation, there are several channels, but usually, it's not the same having a conversation in person. Let's say at the conference, this might be a great chance for a lot of people to meet in real life or having the same conversation over Twitter or over an IM app. When we're having a conversation, face-to-face in real life, usually, we can get some cues from our body language. But when having the same conversation over Twitter or social media or instant messaging app, we don't have those cues. The natural replacement is using some emojis. But still, it's not exactly the same. So, when developing the conversational interfaces, it's important to keep in mind the cooperation, much like when we're having a conversation, we are both in the same boat trying to achieve something and all this trace that I mentioned. Also, there are no errors. And this is an important thing to keep in mind. When we're having a conversation, we don't say to the other person, like, no, that's wrong. But not in terms of what you're saying. I consider it's wrong, but you are saying it wrong. That sounds weird. But anyway, what I'm trying to say is, when we're having a conversation, there are no errors. We might have different point of views, but both parties, or if it's a group conversation, if there are many people involved, are putting something to the table and adding some value to the conversation. There are no errors. In the same way, when dealing with an app, it's either an Android app or an Assistant app, something like Google Home or Alexa. If for some reason, the device says you are wrong, it's going to be something hard on the user. Instead of that, we have several branches. If the user, for some reason, doesn't reply what we're expecting, or if they reply it's wrong in some way, we're not going to say that's an error, like what we usually do in UI interface. Instead of that, we're going to take a different road, a different branch, and show a different part of the flow. This is important to keep in mind. We're not showing like a big red X saying to the user, you're wrong. Instead of that, we're taking a different path. And also, much like real life, interacting with a conversational app sometimes will add some challenges. Some of them might be temporarily, other might be for long term. So we need to consider that there will be some noise, that sometimes both the user or the app will need something to be repeated, and that in some moments, we will need a graphical reply. In other moments, it will be only audio. So with all this in mind, we need to keep those cues to understand. This is the main goal. We need to understand what the user is trying to say, and we need to be able to provide everything the user needs to understand what our app is trying to say. Understanding is key, and it's interesting, mostly because there are several barriers. When we're working remotely, developing software that is, we're working with themes that are located in several parts of the world, usually we communicate using English. And for many of us, it's not our first language. So we need to deal with a couple of things like accent, like the words and phrases that we're using, like cultural barriers. The same thing might happen here when developing a conversational interface. So keep in mind that understanding is critical, and the users are going to help us. Users are here to do something, to achieve something, and focus on that task instead of focus in using our app. Users know how to talk. Everybody knows how to ask for something for a conversational interface that might be, let's say, turning the lights on or turning the TV off. And we don't need to be taught how to say that or how to speak to the app. What we might need is a little help with the onboarding process. There might be several ways of saying something, so our app should consider all these different options and make the user's life a bit easier. Besides that, users also know what they want, how do they want to be done, and basically we just need to leverage on that, mostly because we share the same goal and we're trying to understand each other. As I've said before, the context is really important. The context matters. It's not the same to... Might not be the same to say the same phrase in two different contexts. We are basically good to make a... To infer this context, but as I've said, the computer, it's not as good as humans, so we need to deal with this. If we're having a conversation and someone pops in, it's a bit easier to get into track and understand what's happening. But that context might lead us to different directions. So if we don't have a clear understanding of what's the goal, if the user, it's not understanding what the app is trying to say in this context, things are not going to work out. And it's important to have a backup plan in cases when something happening, like the user, it's not replying. There's important to have a timeout. When we're having a conversation, it's funny because if you're having a conversation face to face, because we are doing this in a synchronous way, the timeout that we have, it's almost non-existent. If we're speaking face to face and for some reason, I say something and the other person just stares at me, it's going to be awkward, right? Sometimes when we're having a conversation, we're thinking about something and it might... We might spend a couple of seconds thinking, I don't remember this, and it's quite common. But when we're having the same conversation over, let's say, an I am app, like, what's app, that's not synchronous. So we might have to wait for a while and it's okay. When we're dealing with a conversational interface, usually it's synchronous, but the user might not reply because something happened, like some interruption, like someone appeared and started a conversation in person with me or I'm dealing with something and the app is still waiting for some input. In this case, we're going to deal with some timeout and that's just one example, but it's important to keep in mind that having a backup plan might let us into a good path, might bring us back into a happy path of the user of our app. Here's an example. This is a sample dialogue with the Google home and an app called number genie. You're supposed to guess a number between 0 and 100. In this case, user says nothing, like around here, and the reply from the app is, I didn't hear a number. It's not an error. It's not you're wrong. It's I didn't hear. That might be because noise, because you didn't say anything or because any reason. But that's what I meant when I say there are no errors. Then it's silent or muffled. It might happen that there's too much noise or maybe it's not the user speaking. Maybe it's a dog barking or something. So the next reply, it's, I don't know, if you're still there, what's your guess? The wording, the phrases, the tone, it's really important for the conversation to be in the happy path. We need to provide the canonical happy path, but consider all these possible branches, and also we need to consider that it might be a first-time experience or a return user. We already know how to have a conversation, but we might need a bit of help on an onboarding process, or if we have used this app several times, we would like to, we will appreciate for our expertise to be considered from the app. This is another example. This is for the game to quit. In this case, the user tried to guess 21 right here. It's not 21, so the user give up. In this case, the app should catch that the user is giving up, quitting, and give a good ending for the conversation. Sure, I'll tell you the number anyway. It was nice. Besides that, I would like to provide some snacks. Some people call these best practices. I think each one of these bullets might imply a discussion, so that's why I'm calling all of these snacks. I would be happy to discuss any ideas that you have after the talk, but let's review some of these ideas on how we are going to be able to build something nice and pretty when we don't have exactly some graphical interface. We should avoid written language. This is important, mostly because when we are developing a graphical UI, usually we focus on some wording that we're used to see in a written way. In this case, we might have a supporting graphical interface, but the main interface is going to be a conversational one just maybe with the Google Assistant that is. We might have some feedback on the app on the phone, but usually it's just the voice from the Assistant speaking to us. It's important to kick off the conversation in a good way. We need to keep a balance on engaging with the user, but not so much that we're going to nag the user, and it's something similar with the notifications. For maybe about three or four years, I've been living without notifications, and it's an interesting experience with the new do not disturb mode in Android. It's been easier for me to configure that on my phone, but for some apps, I leave the notifications on, like maps, mostly when I'm traveling, but lately I've been getting so many notifications from apps that I ended up blocking the app. This in mind, the idea here is to engage with the user just in the correct amount of interactions. We also will need to guide the user through the conversation, although we're not explaining how to have a conversation, nor we are leading all the conversation, while just guiding. And keep our text-to-speech interactions short and clear just in case the user is not able to understand for any reason. It's also a good idea to avoid any data points unrelated to the user query. Usually, when we're having a conversation, we like to be on a direct path. Many people, like me, tend to ramble when speaking. So when having a conversation with someone who rambles a lot, it's a good idea to stay in the path. The same thing with a conversational interface. We should not ramble. We should just give the user whatever they need for this query, for whatever they're trying to achieve. And follow the natural turn taking in order to keep the conversation like something that we will have with a person. It's also a good idea to use conversational makers to keep the user engaged. And finally, with all this solved, we're going to add some... With some machine learning. As the abstract said, we don't need to deal anymore with NLP. Basically everything is solved, everything, let's say. And there are many tools that help us dealing with the language processing. So instead of developing the whole stack of recognizing the input for the conversation to happen, we can use a tool, a framework, like Dialog Flow. There are many other options. For me, this is one of the best available tools. And basically Dialog Flow works with two things. Intent matching and entity extraction. It's going to categorize whatever the user is trying to achieve into an intent. And also it's going to identify something that's relevant for that task to be achieved. In order to do that, or in order to take the next step, we're going to take those intent and entities and do something with that. And eventually reply to the user. Basically the flow is this. The user is going to input something either with a key or a microphone, and that's going to turn into a query. Dialog Flow is going to receive that query, identify the intent, and process it. It might be with something simple, like Firebase Cloud Function, or AWS Lambda, or whatever you would like to use for serverless. Or it might be something more complex with your own backend. Either way, we're going to have external APIs, maybe a database, some code to do this processing. And when we have all this settled, we're going to reply to the user and provide a fulfillment with actionable data. The idea here is that Dialog Flow is doing all the heavy lifting, and we only need to identify what's the user is trying to achieve, what's the main goal, and which pieces of data are relevant to that goal. We're going to start with an invoke procedure. We need to say hello and start experience with the agent, with the software that we're building our app. In Dialog Flow, we're going to define these intents. This is just an example, an intent for movie recommendations, and we need to provide training phrases. Like, I want to watch something, or tell me what to watch, or recommend me a movie. These are different ways of saying the same thing. The idea here is to get a movie recommendation. Then we need to provide the entities, in this case, it's a movie, and we're going to provide some synonyms. Like, for sci-fi, we don't have a synonym, but for comedy, we have funny movies, or for drama, we have serious movies. And eventually, we're going to use that webhook data that I mentioned before with a backend to provide a fulfillment. The response, it's also specified here, because we are providing the user something like a phrase. Keep in mind that we're doing a conversation here. We're not just providing some unconnected pieces of data. We're having a full conversation here. So the response might be, you should give it a try and watch a comedy, grab the popcorn, it's movie night, just in case the user is looking for a random recommendation, or have out a sci-fi for tonight. We use a context to move between intents. Might be that the user is looking forward for something more than just a movie recommendation. Maybe we would like to discuss, like, a specific actor or actress, or maybe the user is looking for a recommendation, like, based on movies that they have already seen before. With this context in mind, we can achieve more fluent conversations. Instead of one request and reply, it's a conversation that is going to be, it's going to take many turns between the app and the user and eventually provide some recommendation, but with all the context shared between those interactions. One of the main things that I was looking forward on Google recently launched at the IEO is that using the Google Home, it's really nice, but you need to say, okay, Google, or hey, Google, for each one of the requests. Now they are launching this new feature for you to not say that in every request just to start a new conversation. And finally, this is all the code that we're going to see. In order to integrate this into our Android app, we need to use an AI service. We're going to be using an access token. Dial flow is not that different from other API providers. It got its own SDK. It also needs some registration, provide an access token. In this case, we're going to click a button and listen for input. And afterwards, this is the result. And here we're just showing what the app is providing. We have a query, we have an action, we have several parameters. And all this depends on whatever we configure on Dial Flow. It's interesting for me mostly because we are dealing with configuration. Not everything is done in code. Although if you really like to code, this might sound like different. But it's like a trend that I've been seeing for a while that it works better if some parts of the process are just configuration, like defining the intents and identifying the entities and other parts of the process are dealing with both the input and the output with code. And basically, that's all that we need in order to deal with our interaction with Dial Flow. It supports several languages and it provides several more features than what I cover here. So to summarize, many of the ideas behind conversational interfaces are things that have been around for a while. Many of the things that are useful are quite intuitive. So in order to build an app that got some conversation in any point for any goal for the user, we need to keep in mind that it's important to build a persona using tone, words, phrases, and the technique that we would like for that conversation. There are no errors. There's the happy path and many other paths. So we need a backup plan in order to get the user back in track in case anything happens. The user is there to help us, already knows how to have a conversation, already knows how to achieve things. We need to leverage that. And basically, the idea here is having a script for a couple of interactions. If it's your first approach, I would suggest writing at least three different scenarios, the happy path, and then considering several options in order to build the full screenplay that we're going to have. And there are many other small things like avoiding written language, keeping the text-to-speech to shortest interactions, but mainly the idea here is make the experience the most wonderful that we can build for the user. So that was what I had. Thank you very much. Do we have time for questions? Yes. Are there any questions? Because we still have time. Over there. Perfect. Thank you. I was just curious about if you have a conversational interface, what's going to happen with all the flows, the existing flows that you have? Because it basically leaves us with no job? Well, it really depends on what are we looking forward to. I mean, there are several things that are going to be automated, but I don't see this happening as quick cuts for all the fields are for some fields. This might be not completely related to the presentation, but in my opinion, in a couple of years, software development is not going to be as good as it's now in terms of wages, mostly because many things are going to be automated, but it's not the only job that's going to change. It's not going to disappear, it's going to change. With that in mind, many of the apps that we're developing nowadays are going to have something like a conversational interface inside. It's going to integrate with the flow that we already have. Let's say we're building a movie recommendation app. Right now, there are like a screen with several buttons and images. Instead of that, we're also going to have first, maybe a small button that says chat with someone. It's going to be that agent recommending movies. Eventually, it's going to be like 50-50. Eventually, if anything goes like the way it seems going now, it's going to be fully automated with a conversational interface. I'm not sure it's going to happen. Chatbots have been around for a couple of years that are not as adopted as I would have thought, but those are my ideas in the coming weeks.
this talk, I'm covering several UX design, several parts of the UX design process. So the one of the participants, the user, and the don't program in natural language because of what they want, how do they want to be done, and computer, it's not as good as humans, so we need consider that it might be a first-time experience user give up. In this case, the app should catch having a conversation, we like to be on a direct stack of recognizing the input for the recommend me a movie. These are different ways of So the response might be, you should give it a order to integrate this into our Android app, we need to use an AI service. We're going to be conversational interfaces are things that have have something like a conversational interface with a conversational interface. I'm not sure