 Hey! Hey! Hey! Hey! Hey! Hey! Hey! Hi everyone, good morning. Welcome to the workshop on Gemini and Vertex AI agent builder under Google Cloud AI for Impact, APAC hackathon, powered by Google Cloud and Hack for Skill. I am Shweta and I will be your host for today. Okay, so before we start with the session, I would first like to give you a brief overview of what this hackathon is all about, including the timeline, report section, themes, and other things. So AI for Impact Hackathon is a unique opportunity to collaborate, innovate, and create groundbreaking solutions using Google Cloud's Gen AI tools to address pressing social challenges across the APAC region. So you have various tools to explore, including Gemma, Gemini, Vertex AI, and Gemini Code Assist. And you have the chance to create innovative solutions that can drive actual meaningful change and improve the lives of various communities living in the APAC region. So whether you are a developer, freelancer, IT practitioner or maybe an AI based or tech based startup solely based out of APAC region, we encourage you to join and build for a better tomorrow with AI. So with respect to the timeline, we are currently in the registration and idea submission phase. And here's what you need to know in this current phase. So first step is register yourself. That is going to be the first and very easy step. Second is form a team. So here you can also participate solely because if you want you can also form a team of up to four members then the next step is that you have to submit your idea and the deadline for both registration as well as idea submission is november 17 so make sure you do not miss out as we only have a week left. And as of now, for idea submission, you can just go to your dashboard and answer the questions that are written there and only have to submit your idea and not the prototype. Then the shortlisted teams will be moving on to the prototype phase where they have to integrate Google Clouds, Gen AI tools and everything, and develop their prototype. And by the end of November, we'll announce the initial shortlisted tips. And we are going to have the grand finale in the later half of this episode. So now let's talk about what's in it for you. What are the rewards and benefits? Well, apart from getting a chance to create an impact we have got some incredible rewards lined up and trust me they are worth the hassle so number one is cash pool price worth us dollar 20 000 and second is so cash pool price is one thing but i believe the real win here is the chance to be mentored by some of the best minds at google clubs so we'll be conducting several workshops like this one in next week as well and apart from that the top shortlisted teams of this cyclotron will also get a chance to receive feedback and mentorship from google for startups team so you know that is something that that is there as a real win for all the top teams apart from that for all the valid idea, you have got certificates and credits for the Google Cloud platform. The winners will also get a chance to present their solutions at the next big Google event. So earlier this year as well, Google Cloud conducted the GNI hackathons and the winners got a chance to present their solution at the Google I.O. Connect event happening in different cities. So similarly, you will also get exclusive passes for Google events for this hackathon as well. And last, you are getting an opportunity to network with like-minded individuals from across the APAC region and gain hands-on experience on Google Cloud's AI tools through these workshops. So this hackathon is basically focused on creating a social impact. So under that we have several themes for you, which you can choose one and submit your idea, starting from empowering healthcare workers, forecasting natural disaster, creating a greener world, tech for better education, helping people with disabilities, transforming public service and solutions for community empowerment. And if you think that you have observed any other social challenge that you want to solve for, then you can select the open innovation and social good problem statement and you can submit your idea under that. So now moving on to today's workshop, I am thrilled to have Aditi Jain, Customer Engineer at Google with us to take forward the workshop. We'll also have a Q&A session in the end. So make sure that you type out all your queries and all your questions in the comments section during the presentation. And after that, we will take a look. So without further ado, let's welcome Aditi. ADITI Jainaini- Thank you. Thank you so much, Veda. Hey, everyone. Am I audible? Just a quick check. ADITI Jainaini- Yes, you're audible. ado let's welcome everything thank you thank you so much for that hey everyone am i audible just a quick check uh yes you're all perfect uh so i'll probably take over and start sharing my screen and basically get this started give me a second i think my screen should be visible now uh shweta, quick check. Are you able to see the screen? Perfect. Hey everyone, myself Aditi. I am a custom engineer in Google, which means I'm basically a solution cloud, a pre-sale solution architect working with our sales teams here with our customers to help them solve multiple problems across Google Cloud, whether it's AI, data, infra, etc. I've been part of Google for almost two and a half years now in total, and been working in the overall Cloud industry for almost nine years now in total. So moving forward, this is the agenda for today. Very quickly. We'll start with talking about Vertex AI, our foundation AI ML offerings in GCP. Then we'll quickly move forward to do a quick little deep dive around Vertex AI search and conversation, how you can use Vertex AI search and conversation on how easy it is to get started, what you can build and towards the end, we'll also try and get you a very quick walkthrough of the platform on GCP console. Along with that, we have built a very small, quick chatbot for the demo which we have built using Vertex AI Search and Conversation so that at least we'll be able to get you a gist of how exactly you can create these chatbots and, and of course, just easily mingle with your ideas that you have been thinking about. Then we'll move forward and talk about Gemini, our Gemini offerings, how you can get started with Gemini, what are the ways you can simply just, you know, either use Gemini for fine tuning, basically pre-trained Gemini models on GCP, and how exactly is it you can do it on the GCP consoles towards the end. So this is basically the agenda for today. Now, let's move forward to the very first topic of today, which is setting the foundation on Vertex AI. Now, before I talk about Vertex AI or our solution areas on GCP or the platform on GCP, I would first want to spend like quick two minutes on different consumer and enterprise perspective on these use cases, right? And how Google has focused on developing these different solutions for different user needs. So on the left side, as you see on the slide, we have consumers and enthusiasts. So consumer as in us using Gemini or maybe generative AI for our own personal use. Maybe you want to plan a trip. You want to basically, you know, plan a neighborhood block party. You want to write a novel, just want to get started on the idea how to get started, right? So basically brainstorming from an end user perspective, who's not very much into AI, just want to use AI for getting their work done a little bit more impactfully. Of course, enthusiasts, maybe students who wants to explore AI, so they can definitely get started with our offerings like Google AI Studio, start developing on top of it and start exploring our offerings on that perspective as well. This is one side of users which we see in the AI platform. And then second is enterprises and developers like you, who are either working with big organization and trying to build some solution for them, or maybe yourself, and you're trying to basically see how enterprises will basically fit into this overall AI ML solution areas. So if we specifically talk about enterprises, their needs are very different from the consumer use cases, right? So enterprises probably would want to understand and see how exactly they can use these AI ML solutions to build something from scratch for themselves, or maybe their end customers if they're into service based background, or maybe they want to integrate these AI ML solutions, maybe a chatbot right into their existing application. Similarly, they would also want to keep a cost into the mind as well, which is primarily a very big decision point for any of the enterprises who are working on AI, right? So these are some of the areas where enterprise are now exploring AI ML as well. If I talk about products perspective, so from consumer perspective, we have products like Gemini app, which is publicly available. So you can just simply go on Google and type Gemini.google.com. And you will be simply able to access Gemini public application for consumer perspective, right? And hence, it's a publicly open application. Hence, from a security standpoint, not very fit for a firm and organization or enterprises. Then if I talk about solutions on enterprise side, we have, of course, the talk of the town for today, which is Vertex AI platform, which has everything and anything that Google Cloud has to offer from a model perspective, whether these are foundation models that Google has built or any third party OSS models that we already have launched or kept onto our platform. Or maybe you just simply want to get started on pre-training and fine tuning your own models on platform. Right. So you can do all of this Gemini models, whether you talk about Gemini 1.0, 1.5 Pro, Gemini Flash, etc. All of them are also available on Vertex AI and on GCP, so that you can just simply get started by simply pre-tuning our models, and then start using them in your own use cases and applications. Now, moving forward, now that we've briefly talked about different AI users, what are the solutions that would make sense for them from Google GCP perspective? Let's just shift the focus a little bit more on the enterprise needs and we'll do here as well for Vertex AI. So Vertex AI, if you ask me what Vertex AI is, Vertex AI is an AI ML platform that has anything and everything that you can think about from an AI ML perspective. If you want to simply launch and run your own custom model, you can do that on Vertex AI. If you want to use pre existing models from Google, which we call foundation models like Chirp, Gemini, Gemma, etc. You can use them on Vertex AI as well. And of course, if you want to, in addition to Google AI Studio and Vertex, we already have integrated Gemini in our powerful platform like Collab. If anybody of you have been using Collab, then Collab Enterprises and enhanced version of Collab, which is also available on Google Cloud. Firebase, if you're into building mobile application or Android applications, you can easily use Firebase for maintaining and managing. It's a very known platform. And of course, Flutter. On top of that, Vertex AI also integrates via an SDK and offers Python integration to let you automate ML workflows as well. If you want to basically create ML automation, you can just simply design those workflow using Python on Vertex AI SDK. Then apart from this, we have Vertex AI Workbench, which is a Jupyter notebook-based development environment available on GCP as well, which you can use anybody who's into AI ML or who's an AI ML developer, you can just simply use to create an entire machine learning workflow on the workbench and then get started. So with these different integrations of Vertex AI within Google Cloud solutions and even with solutions like Flutter and Fireways and Collag, any developer who's more into developing application, they can be an AI developer. So that's basically a nutshell of how exactly we have built and focused on Vertex AI to make it a little bit more accessible on Vertex. Also a quick plugin, if you feel like I'm going too slow or I'm going too fast, just drop a ping and then we'll try to adjust the speed as well. Now let's take a look at various offerings from a Gen AI perspective on GCP as well. So the idea is that we as Google understand that generative AI business value would require a comprehensive platform and not just a single solution. Hence, we have built the slide that you currently see are the solutions that we have built focusing on those business problems. So we have built solutions to help basically meet different business needs and different user needs. So if I start from the bottom on the screen, which you see here, we have Google Cloud AI infrastructure. So anybody who is coming from an AI background, we know that to be able to run any AI-based LLM model or a simple model, we need GPUs to run them a little bit more effectively from a performance POV. Hence, we would need the AI infrastructure to be able to run these models. So we offer that as well. So GPUs like H100s, A100s, we have our own native GPUs, T4, etc. are available on GCP as a platform for anybody to get started on GCP. You just simply spin up either a GKE container or a virtual machine, which has GPU enabled in them and then start using and running these models on GCP platform. Now, moving forward a step ahead, which is basically this layer. This is where our Vertex AI platform comes into. It has three major dissections if you want to call it. So the very first one is Vertex AI Model Garden. So Vertex AI Model Garden is basically a platform basically which is which combines diverse foundation models Google owned. And then of course we have some third party and open source models also available on Vertex AI Model Garden. So this is different models and either start building on top of them or just basically integrate these models via APIs into their own applications, right? So Model Garden would be a way to go. Then we have something called Vertex AI Model Builder. So if you want to, you know, prompt tune, you want to serve a model, you want to fine tune a model performance, if you want to access notebooks, if you want to train a model. Anything from a AI ML pipeline perspective, monitoring perspective, you would use a Vertex AI Model Builder to able to perform all of these tasks, whether it's workflows or basically automating the overall machine learning pipeline. This is where Vertex AI Model Builder will be able to help you with. Then moving forward a step ahead, then we have Vertex AI Agent Builder. This is what we will be exploring today specifically. So Vertex AI Agent Builder is a platform which can help you build out of the box custom agents. Two basic themes where you can build around your chatbots on. One is search. So if you want to build a search-based model, all you have to do is just simply integrate the backend data source with Vertex AI Search. And then just spin up a frontend where you can access this search from. And then you will be able to ask questions and search through multiple data using natural language prompts through search. And similarly, for chatbots, right, a normal use case chatbot where you want to build a calling chatbot, a chat conversation for retail customers or many examples like healthcare, a simple chatbot where you want to build a calling chatbot or chat conversation for retail customers or many examples like healthcare a simple chat chatbot on a website right or maybe internal chatbot for multiple use cases you can build that on vertex ai agent builder as well now going a little bit top on this so as i was mentioning earlier we have gemini as a model as well so what we have done is we have integrated Gemini in a lot of our own Google Cloud solution as well. So if you are somebody who have already been using BigQuery, for an example, you might have noticed that we have integrated Gemini with BigQuery as a product as well. So now you can just simply use Gemini on a chat prompt and then get the SQL query to run on BigQuery. Just simply use natural language to write what exactly you want to retrieve out of BigQuery, and Gemini will be able to help you write the SQL version of it. As simple as you start using these products. Similarly, we have integrated Gemini in our Google Workspace product as well. So customers who are specifically using Google workspace for their organization communication needs, we have integrated Gemini and Google workspace, which I think the followed workshop will probably cover in a little bit deeper. But just to give you a brief, Gemini for Google workspace is integrated in all of our workspace applications, which mean if you're somebody who has been using workspace, it has integrated it has integrated in Gmail, Google Docs, Google Slides, Sheets, even AppSheet as a platform, and even Google Drive, which from a consumer end user perspective, makes the life a little bit more easier. For example, writing emails, writing comprehensive document, analyzing a lot of amount of data in Sheets, which is generally not easy for a human eye to just capture patterns. Instead, you can just use Gemini, ask questions based on your data in sheets. So for example, let's say you have 1000s and 1000s of rows in the sheet and you want to understand a pattern in that sheet data, you can just simply use the integration of Gemini in Google Sheet and ask a question on that data, maybe analyzing a pattern, maybe asking one particular information. And Gemini will be able to help you do that very easily. Right. And then third is, of course, building your own generative AI powered agents, which is your own chatbots that you can also do with our Vertex AI and Gemini platform. The top layer that you see here are remaining solutions like contact center AI, healthcare data engine, search for retail media healthcare. Apart from this, we have solutions like document AI, right? So all of these solutions are out of the box, like speech to text, text to speech. So if you're somebody who has been working on Google Cloud products, then you probably know speech to text and text to speech is an oldest product that we have actually been working with. So all of these are also part of our overall AI solution platform. Now that we have looked at the very high level overview of Vertex AI and Google Cloud AI platform, let's do a quick deep dive around how do we get started with specifically on Vertex AI search and conversation, which is building agents or chatbots for search or normal conversation based use cases. Now, basically, GenApp or when we call agent builder allows users or developers to build very low code and Gen AI enabled app on two types. It's very low code. When I say low code, it's actually a three or four step process to just simply build a chatbot and which really require very low or almost none of the coding skills to be able to build these chatbots and get started with these chatbots on platform. One use case could be building enterprise search applications, which have a lot of data at the back end attached to it. And then you can just simply use to build those enterprise search apps to build a search engine, maybe for your own organization or maybe for end customers. So the good part is attaching the data with enterprise search. You can easily work with data like unstructured data. For example, let's say you have a lot of PDF files, text files. You can simply attach them to Vertex AI search. Structured data like JSON or maybe BigQuery rows, that also is available as a data source and even website data. So if you would simply want to attach your website and build a search chatbot for a website, you can just simply add your website's data source and Google's tool will automatically use indexing for the site and will be able to help you answer the questions as well. So once you build an application using search and attaching multiple data sources, your end users will be able to search that data. And not just on carefully selected keywords, they can even use natural language, which is simply asking questions how we converse and ask those queries in the chatbot. And actually, InGen will be able to process and will be able to return the answer to that query with the linked resources and also summaries and conclusions. So think about use cases. If I particularly talk about use cases, think about use cases where let's say you're working for a healthcare organization as an example. And when we talk about healthcare organization, they generally deal with a lot of amount of data, for example let's say you you have built an application for maybe doctors because they need to get access to a lot of patient information very handily right so what you can do is if they have maintained uh their patient files in some sorts of format you can just simply connect that data source it can be connected with google cloud storage it can as i mentioned. It can be connected with Google Cloud Storage. It can, as I mentioned earlier, it can be directly connected with files as well, which are stored in Google Cloud Storage. And once you connect them, you can just simply ask questions. So for example, if I am a doctor and I want to see, okay, what was this patient's history when I last met them? I can just simply search, putting it in the search chatbot that I have created and just get the answer in a natural language problem. Instead of going back, search for the patient information and then opening a particular file. So this is one use case, very, very random use case. They can be much more and better use cases that you can start building using our search AI and conversation. So this was the search use case that I just mentioned about. Another use case of Vertex AI agent builder could be building information chatbots applications, right? Which is primarily focused on providing intelligent chatbot, which can answer questions related to a specific information, right? Which can be information about your business. If you're building a chatbot for your website, it can be a chatbot created internally for your internal processes. If a new employee joins, for example, and they want to look for very particular information, they can just use that chatbot to find the information in the organization, right? So imagine taking all of your product documentation and helpdesk material and building a chatbot that allows your end users to very quickly get answers without having to scan dozens and hundreds of pages. As simple as that. So this is how Vertex AI agent builder can actually help you build grounded data stores and then help build AI applications. So as you can see on the slide as well, it's very simple three or four step process to get started with Vertex AI Agent Builder. You just simply create a data store first, which is attaching your source with Vertex AI Agent Builder to serve as a source of truth for your AI application, whether it's a search-focused application or information bot-focused application. And then second step is simply loading the data store from multiple sources. Could be BigQuery, could be cloud storage, even Google Drive for that matter. And then simply just start using it. And then once Vertex AI simply process all of this data for content and meaning, your app will be ready. So you can create search applications, recommendations, application, chatbots, etc. And then just simply start. also building and using Vertex AI agent builder for multiple use cases in the market. Now moving forward, the current page that you see, when we worked on Vertex AI agent builder, we realized it's very important for our customers who are using these solutions to build their own platform to be able to create grounded generation of responses, which means the response that Vertex AI is coming back or the agent builder is coming back with, it should be grounded in nature, which means it should be relevant to the use case that the customer is trying to build on. Hence, we build the solution to make sure that you're able to attach your own data to basically meet where your need is. So Vertex AI Search will be a great option if you want an end-to-end, out-of-the-box customizable solution for search and answer generation on a variety of corpus types, depending on your use case, of course. And if you want to build your own solution, you just need more control. And we totally understand that, right? Which is why we have enabled building your own rag with the AI chart builder as well. So you can just simply by integrating your own rags, you'll be able to simply get more accurate and reliable responses from Vertex AI agent builder whenever you build your application and attach your data sources to it. So as you can see this particular slide from processing and annotating, embedding, indexing even and retrieving all of this task is done by Vertex AI. So you don't have to manually do an indexing to be able to better find, to basically to be able to better the search response and performance. Indexing is automatically taken care by Vertex AI as a platform. All you have to do is just attach your data to it. Of course, start working towards it in terms of once Vertex AI process the data and you are good to go in terms of building these chartboards. Very quick view, this slide just very simply show you, and I will be showing you this in a Google Cloud console as well, but I've still attached the screenshot for a very quick reference. So as I was talking about, very simple three-step process of getting started with building these agents on GCP platform. First, you connect your source. So as I was talking earlier, we have multiple source available, right? Whether you want to attach a website URL, Vertex AI will automatically index the URL and will be able to get you a faster response. BigQuery as a data source, cloud storage, if you have a lot of files, right? For example, let's say you want to build a chatbot or search chatbot for your internal employees to be able to find relevant information on your organization, policies, etc. So you can just simply upload all of the policies into a cloud storage bucket, for an example, and just integrate that, add that as a data store for Vertex AI search and then just get started. Healthcare API, Google Drive, or any other APIs which you can manually integrate or import data from by calling those APIs as well. Then once the data store is attached, you can just simply start building the application. As I mentioned earlier, you can build search, chat, or even recommendation or normal agent-based chatbot applications as well. That's pretty much very easy to get started with our Vertex AI chat and search conversation. Now that we have very quickly spend some time on Vertex AI search and conversation, let's now move forward towards understanding Gemini's overall plethora of offerings on GCP. Now, before I talk about different models available, different Gemini versions available on GCP, again, connecting the dots from what I was talking about earlier, which is we have focused our AI MLB solutions from a different user needs perspective. So if I talk about consumers, again, Gemini models are available as an app or web application to our consumers directly in a public availability. Then of course we have Gemini is available in Google apps. If you look at Google search application, you get Gemini responses as well. Then of course, we have already integrated Gemini in Gmail, Docs, etc. This is where we are focusing on making our AI and ML solutions available to our consumer-based users. Then of course, we have Gemini available for developers. Developers who specifically want to use Gemini API, either through Vertex AI Platform, Vertex AI Studio, and they want to basically fine-tune their responses, want to use Gemini API into their own solution that they have built, they can just very easily get started with that using in simple APIs. Then of course, the third level of users, which is businesses and enterprises. For them, we have of course, build Gemini for Workspace and of course, Gemini for Google Cloud. Gemini models available on Vertex AI as a platform, and then of course, Gemini for Google Workspace, where you can start using Gemini in all of our offerings. This is Gemini in all of our offerings. So this is Gemini for you. Gemini is our most capable and generalized multi-model that Google has ever built. And it has been a really great effort by our teams across Google, including our DeepMind team, which is very specifically focused on building these solutions, and of course, Google Research. Now, Gemini is integrated in our most of Google Cloud products and services, as I was mentioning earlier. So example, as I mentioned earlier, could be BigQuery for SQL assistance, conversational assistant through a chat interface cloud console can also even help so if anybody of you have used google cloud console then then you know that we have already built an integrated gemini in the console as well through a conversational chatbot assistance right where you can just simply get the help on the console with anything related to Google Cloud, right? So Gemini is now available from being an assistant to an AI powered code assistant even, right? So we have something called Gemini code assistance also available. And we have already integrated the code assistance with Gemini into multiple IDs platforms as well. So if you're somebody who have been using Visual Studio code, then you will be able to integrate simply and access Gemini's code assistant in Visual Studio as well. We have already indicated this within JetBrains IDE. We have already available in our own Cloud workstations, Cloud Shell. If you're somebody who already have used Google Cloud, then you're definitely a little bit more known about Google Cloud Shell, Cloud Spanner, and BigQuery. So all of these integrations are already done. And of course, we are working more and more to enhance our offerings from a Gemini POV. This slide basically will give you a very high level overview of Model Garden. So Model Garden again is an offering within Vertex AI as a platform where you can get access to all open sourced, not all actually, a lot of open source Google owned foundation models as well as partner models available through Model Garden. So, of course, we have our own foundation models also available on Model Garden, which you see on the left side, including Gemini and the latest version, which is Gemini 1.5 Pro, Gemini Flash, Gemini Ultra, and Imagine, Kodi, Chirp, all of these models, foundation models that Google has built, all of them you'll be able to access on Model Garden. Any open source models that you might want to use for an example, Meta's Llama 2, Mistral, Ditto, Gemma models, which we have created, and we have made Gemma as an open source, so you can just simply start using Gemma as well. And then any third-party models like Anthropic, Clouds 2, and of course, we are adding more and more model support from a third party perspective in the platform as we move forward just to get you an understanding of how these models are like which models are available this is what the categorization on a very high level looks like on model garden right so at the bottom you will see all of our partner and open ecosystem model, which includes Lama 2, Falcon, etc. Then we have Google domain-specific models that we have built, which is MedLM specifically built for life sciences and healthcare, a large language model which has been trained on medical data. So it understand medical terminologies, etc. to be able to better perform when it comes to medical data. And then SecPalm specifically built and focused across cybersecurity. Then we have Google task specific model. So I think these are the older oldest AIML based offerings available. If you are somebody who already know Google's platform, then you totally you might have worked with these solution areas before as well. So we have speech to text, text to speech, of course, then we have natural language model. We have translation, doc AI, OCR, etc. So all of these models are also available. Then we built our foundation model, which I was talking about earlier, Palm to Imagine. Imagine is basically our image to text and text to image based model. Chirp specifically made for audio to text based audio to text based model processing specifically. Kodi is built for, of course, your code assistance and embeddings as well. And then on top of that, we have Gemini foundation model. So any customer who doesn't want a single model for different different tasks, we have made Gemini. So as you know, Gemini can work and can process data in a form of image, audio, video, etc. So Gemini is a multi hence we call that multi multi model. And we have different versions of Gemini also available as as we speak now. So in a total of available models on Gemini, we have somewhere around 130 plus enterprise ready foundation models in Model Garden available as of today. And of course, we are working on adding more and more as we move forward. A very quick overview of getting started with Model Garden. All you have to do is just simply search Model Garden on GCP Cloud Console. Once you do that, you'll be able to see all of our models which are available here. Of course, starting with Google's foundation models on top, and then of course, our different models from third-party and open-sourced, which are at the bottom. Of course, you have two options whenever it comes to accessing and deploying a model or running a model. Either you can run a model on Vertex AI platform, which is a managed platform just like any managed services offering, helps you streamline the overall operation tasks of running and managing these models. Then of course, if you don't want to go with the platform and want to control and manage your models and running these models yourselves, then you can just simply get started on compute engine or Kubernetes engine and start using and running these model on Google Cloud. With that, I'm through with my content and then I'll probably move to the demo and then we'll take the questions towards the end. Give me one second, I'm going to change my screen. Let me just refresh this. So this is a chatbot that we have built on GCP as a platform, on GCP itself. It's a demo chatbot that we have built, focused on the retail use case, which I was talking about earlier. So if you want to build a simple retail-based infobot, which gives your end user some information about maybe on your website, internal, some processes. This is a very simple example that we have built. After I show you the demo of this end chatbot, I'll quickly show you around the platform as well on GCP. So I'm going to start with the chatbot. As I mentioned, it's a retail-focused chatbot. So it basically has all the information about retail store called Symbol Bikes, of course, hypothetical store. And as you see on the left side, if you see right, we already have attached these documents via our cloud storage as a source for this chatbot to be able to answer those questions. So as you can see, it can answer questions on return policy. And generally, it's totally up to you whether you want to show this or not, but we have just kept this for this demo purposes, so that at least you know what sort of data sources we have connected with this chatbot, right? So we have connected a document for return policies, services, any store information or frequently asked questions. So any customer who might anybody who might want to ask questions, they can just get answers across all of these different topics. So let's say I want to ask the chatbot, are there any stores available in Miami? So let's see the chatbot comes back with here. Since I already had added a data store information about my stores and locations, the chatbot was able to come back with a response because we had a store in Miami. Now, let's say I want to ask a very process-related question. that I purchased before, can I return to? Yes. Since I'm asking for a return, the childbot automatically came back with a return policy attached to it as well, which is it offers a 30-day return of policy on all items. You can just simply return the item to the store where you purchased it or just basically ship it back to the right store. And these chatbots that you have created are similar to the chatbot that you probably see on multiple websites as well. Earlier, if you talk about creating these chatbots from scratch with the use task, because you had to code and then create a web interface, et cetera, but now simply you just put a front-end, maybe a front-end running on an app engine, or maybe a front-end running on a compute virtual machine, depending on how quickly you want to build it. And then on the back end, you can just simply create or attach a data store and then just simply get started with it. So as many data you want to attach to it and the chatbot will be able to response with more and more information. So this is a front end, very simple chatbot. And I'll just quickly show you on GCP as well, a very quick process in terms of how easy is it to get started with building agents in GCP. Give me one second. I just searched for Vertex AI agent builder. You can see it's currently loading, but you can see all the apps that we have already built. If you want to create a new app, you can just simply get started with creating an application. I'll just quickly show you how easy is it to get started with building these. As I was showing earlier, you can select the type of the application that you want to build, whether it's a search application chart, recommendation. Let's say we want to build a search application, generic, media focused or healthcare focused. The reason we have segregated this is because we understand that healthcare-based customers have FHIR and NorvHIR data and then the understanding of these terms would be different. Hence, different configurations and content specifically selected. Let's say we want to build a generic chatbot. You can select the enterprise edition feature, which are basically emit search website search etc which lets you search for websites as well advanced these are something which we have added very recently which is adding advanced feature to our bots as well you can give give give a name to the chatbot you want to give a company name you can give the company name as well. Then you can select where you want to deploy. As of today, you can either create a global chatbot. Basically, if you are coming from an organization or you're building a solution for an organization who have users across the globe, then you can pick global which will make sure that we have endpoints in the back end of this chatbot into multiple locations, or just restrict to US or EU as well. At this point in time, these are the only regions where we have the endpoints available. Let's say I just picked up US as a region. The second step is you just simply attach data stores. So as I was talking earlier, we are adding more and more data stores as we speak today with Vertex AI Agent Builder. So you can pick from all of these options wherever you want to add data sources on. So let's say you want to build a cloud storage. You want to attach data storage as a cloud storage. Then you can just simply select the data type, structure, unstructured, media data, etc. And then ingestion, whether the data should be ingested in Vertix AI once, which is processing of content, or periodic, which is basically a preview function. It's not currently GA yet. And then you can give the path of the folder and file. Once you give that, you just click on continue and you have your app configured. And all you have to do is just build a front end and then point this app via an API call to basically make sure that you're able to access this. These are the current data stores that we have built. If you want to basically monitor your application performance, you can just get all of your data inbuilt directly from here. By traffic, latency of your chatbot, any error rate which is happening, you can just simply see the API matrix here as well, which is how many API calls are being made, etc., from your front end or any of the website that you might have put the chatbot on. These integrations are available, and this is how easy is it to get started with Vertex AI chat builder. Question, Chweta, am I still audible? I think I'm just getting a prompt that I've lost connection. If anybody can give me thumbs up on the chat as well, that would work. Yes, you're perfectly audible. Shweta, am I audible? Yes, you're audible. Okay, perfect. Thank you. Thank you so much. This was a demo on the Vertex AI agent builder. Very quickly, we have also built another demo on Vertex AI search. I just quickly go over that and then we'll, of course, move forward to the Q&A piece. Just to quickly show you how exactly we have built this. The previous example was for our Infobot chat builder, right, which is simply asking question about website or anything that we want the chatbot to answer questions on. This is specifically built for a biomedical literature, specifically for a healthcare-based use case, right? So coming back to the example which I was giving earlier, which is let's say you're working for a doctor or let's say you have built an application for doctors who want information on their patients, right? So you can build a search-based application. And this is how this application that we have built, this is how its architecture looks like. So we, of course, have configured access control on this and then the for application layer. And for the front end, we have configured App Engine. On the back end, we have integrated enterprise search where the document the big query metadata search is happening right so very simple straightforward architecture nothing complex uh as such i'll quickly move forward to the bot so this is a bot so for this board particularly what we have done is uh we have integrated questions already here right if you don't want to do this, you can just simply put in a search bar and then the customers or the end users can simply type their own questions in natural language in the chatbot and sorry, and the search bot would be able to take that as well. So here, since it's a health care or biomedical literature based at what we have already asked these questions added these questions, which can be asked. So if I press on any of the question, which is what are the classes of anti-arythmating drug? So as you can see, the answer came in and at the bottom, the best part, basically, when it comes to any of the search idea. So whenever you want to create any search based chatbots, you want to make sure that whatever information or the answers that your chatbots are giving, you add the relevant source to it as well. Then the end user, whom server is asking the question on what this information, they have a source to go back to to validate the answer. You can even do that. With Vertex AI searchbots, you can just simply get those source links towards the end as well so as you can see uh for this particular answer it added sources all i have to do is just simply click on the sources and it will expand these sources for me so as you can see here view paper so here the bottom it will give me the source right so you can very easily get started with these simple applications and even more complex one depending on your use case, what you're trying to build. And just to show you a corpus, which is the papers that we had uploaded on this search board are these ones. So these 10 papers were uploaded to create this very quick chatbot. And all of them can be accessed through here. And some of them even have a public link. So as you can see, search bots can access both information, text-based or a document-based source or data store that you add or even if you simply want to add a website link, search will be able to answer questions based on that as well. So as you can see, this is the information which was available in the emerging paper, which is basically fetched from this particular website that I just opened. So that's it pretty much from my end on Vertix AI, Gemini. I really hope that you were able to get some understanding on both of these solution areas. I see we have a few questions. Give me one second. I can pick the question and show on the screen so that everyone knows what you are answering. Yeah, thank you. So I will start with. Okay, so is asking what are some of the common use cases for Gemini in real world applications? Okay, so I think you said Misty, right? Yeah, Misty, it tends to be very honest, depending on which model you want to use. For example, within Gemini, we have created different versions of Gemini right for example we start with Gemini 1.4 then we come back with Gemini 1.5 which is more complex more bigger model you can answer more complex and solve more complex problems right and then we have Gemini flash so for example if you're creating a use case which needs fast responses. Let's say you build basically scenarios of online or live chatbots, right? They need faster responses from the model. Then Gemini Flash would help. For an example, let's say you are... Healthcare could be one example which I was talking about earlier. Second could be, let's say you are working in a build back field and you want to basically build an application where the customer or the customer or the lawyer basically can go back to and then ask this legal information from one, get all of this legal information about cases, about your organization from one place, right? You can just simply use gemini flash the back end and then fetch information drops right uh this is another use case we have seen customers using gemini for uh more complex problems which is uh you know uh integrating with their own applications and gemini basically helping them process more complex situations so in an actual the answer may vary depending on what use case you are trying to implement. And if you want to explore more where you can ask Gemini, where you can start using Gemini from, just simply search Gemini use cases in GCP, and you'll be able to get more. It's a Google Cloud page. So it has real-world Genial problems where Gemini can help. I can probably share this link. I can paste the link as well, like in the private chat. Yes, you can paste in the private chat and then paste in the conversation. Perfect. So this is the link where you can explore different real world Gemini use cases and maybe pick something of your own and start building on top of it. I hope this answers the question. Thank you. Next is Siddharth is asking, how do Gemini and Vertex AI manage scalability for large data workloads? Good question. Really great question. So when it comes to large data workloads, I'll answer this from a Vertex AI perspective because it's a platform. Generally, whatever platform you are using on the back end, whether you are running the models on Vertex AI as a platform, or you're running these models on TKE or computer engine. Depending on what you have chosen, Vertex AI is a platform, it's a pass offering, which means it automatically is auto-scaled based on the need and the traffic that is coming into the models. The scaling piece, the performance piece is automatically taken care of by Vertex AI. You don't have to manually configure anything. And we already worked with a lot of large enterprises who have built on top of Vertex AI. So if you're using Vertex AI, then then vertex ai as a platform will take care of automatically scaling these workloads depending on the traffic and the need but if you're running these models on let's say a computer engine or a gte depending on the world if you're running these models on a gp uh compute engine based uh bm which has gpus on it then of course the scaling piece would be taken care by you, right? Depending on what your scaling requirements, whether it's a production environment, testing environment, what is the user base you're trying to cover, right? So both the options available, but I always recommend my customers. If you're specifically building to enterprises, right? And you expect a lot of used traffic, then I always recommend our customers to go with Vertex AI as a platform because you just simply consider the required and then Vertex AI will take care of everything from a platform to manage these requirements. Moving on to the next question. Amar is asking what customization options are available within Vertex AI agent building? When you say customization, as you can see, it's a very out of the box solution, which is you can just get started with them very quickly. When it comes to customization, use cases can be across search and infobots depending on where you want to integrate these. Customization basically will vary in terms of what is the use case you're trying to implement, whether you want to build an infobot based solution, a chatbot which might help you. It can also even look at the transaction and help you get some transaction information as well. So from a use case perspective, the possibilities are simply from a use case where you want to integrate these chatbots in. So I hope that answers your question. But primarily from a search and info bot perspective, you can just basically pick out any use case wherever you want to build a legal healthcare based use case and you will be able to get started with that very easily. So next question is from Ganesh and he's asking, can Gemini handle multilingual data in books effectively? Great question Ganesh, the answer is yes. As of today, we are adding more and more languages to be very honest. But as of today, I think we have somewhere around 40 plus languages already supported within Gemini, which includes some of our native Indian languages as well as well as global languages as we move forward right if you give me a minute I'll just probably share a link of the language support as well for everybody's reference give me one second please Here. I've placed the link. I think you can move forward with an answer. The links are available in the comment section. Yes. So moving on to the next question. Yes. What are the best practices for deploying general in a distributed cloud environment? See, depending on, of course, your use case, but if you're using a distributed cloud environment, two ways generally to use Google's own proprietary foundation models. Either you use these models on the ModelGuard platform, which is through Vertix AI, or if you're looking to access these into your own environment, could be across any of the Cloud platform, then you can just simply start using them through Gemini API. So for any environment which is out of Google Cloud, any other Cloud platform, or something else, you can just simply use a Gemini API, and you'll be able to use them. I think there was a Gemini API best practices as well. If I find that document, I just probably put the link there as well. So just one quick sort of plug-in. If you need documentation, I think we've added a lot of documentation on Google Cloud, different offerings. So any question you might have, just simply start searching for these and you'll pretty much end up finding a good detailed public documentation on any of your queries. And then very easy to get started on them. Moving to the next, can Google map data be used as input to the model and ask questions about the surrounding area, live traffic details? Google map is, as an API, is integrable across these solutions, so that's also possible. I've seen a lot of customers who have built these AI-based application with a Google map data. For example, a very simple use case could be an application in our cars, if you think about it. So our car providers who are manufacturing these cars, all of these applications, they definitely have some telemetric data on the maps, location, et cetera. That's also possible. Next is, if I can use Go language, base, JKI, SDK to connect to Vertex AI, or do we have to learn Python to integrate with Vertex AI? Based on what I remember, I can of course double check, but apart from Python, we have Go, Rest, a lot of ways of getting started with Vertex AI platform. Give me just one minute. I'll just probably share and double set this thing as well. But based on what I remember, yes, you can use Go as well with Vertex AI. You don't need to only know Python. I'll probably double check this one but based on what I remembered who is supported but I'm not able to find the correct link but I probably work with whether and see if I can share the link after the link. Yes definitely all the links that you have shared until now and whatever links that are there we'll share with the participants after the session in our document. Sure moving on to the next version. This is from the file. What do you that you can just simply start building on. agent builder I will share here so for any building any search or rag experiences you can just simply use a suite of API's which are available on vertex bi to build your own augmented generation retrieval on applications so and that's also not quite a lot of coding as such. You just very simply use our API to get started with building bootlegs. I've just shared the link for that as well. I hope this helps. Next is, is there any real-time processing AI tool which could be installed and happier with Kooten. I mean, I might need a little bit more context on this. But when you see real-time processing, pretty much a lot of these models support real-time. Depending on what type of data you're processing, the solution might change that you're working with a vision model, text-based model, or if you want to work with a model to understand all of text, audio, video, et cetera, then Gemini flash could be the one which you can probably start using, right? Because Gemini flash, it was shorter and not shorter. It's a smaller version of Gemini, which is comparatively faster and created specifically for real time use cases. So maybe start exploring my flash, and if you have any specific questions, then probably we can take that back offline if possible. So next is, how does Vertex AI leverage Gemini's multi-model capabilities and mixture of experts architecture to improve model performance and scalability in real world applications? Good question. Scalability, as I mentioned earlier, since scalability is primarily an infrastructure job, so Vertex AI takes care of that in the form of being a pass offering or a platform. When it comes to performance of a model, to be very honest, it varies. It depends a lot on what type of data are you working with a model on, how huge the data is, what kind of response you're expecting from the model. The definition of performance changes depending on the model, the data that you're working with, and what is the environment and what is the use case you're trying to implement with that right so so far to be very honest we have uh lived into and even if you just simply search gemini for gcp and you basically will land on the gemini page it talks about the performance of gemini and it's a publicly posted page uh compared to our other private previous models as well right and how many tokens it can process in a single request. So all of this information can be found, but in a nutshell, I've seen Gemini performing really good so far with the customers who have implemented Gemini in their own use cases. I think Navin was talking about real direct processing, right? So he has added a follow-up question that is called signal processing if you want to add something otherwise we can move to the next question um i probably need some more another comment where he least trying to explain. Are you reffing the healthcare, the signals from the ECG devices, Naveen? Okay, let Naveen come back and I'll give another question in the meantime. Also, to the earlier question, Go is supporting with.HDI. I just found out, got the link as well. I didn't, my colleague is also on the call. She helped me with the link. So I just shared the link as well for the Go support. So Go language is supported with Vertex CI, so you don't have to learn Python to get started with that. OK. Naveena said, yes, I mean, this is what he made it with respect to the health care. Got it. So yes, you can, when it comes to real time signal processing, to be very honest, it would work. In the short answer is yes. The question is, depending on the image size, the model, the response time might vary, but the very short answer is yes. You can process signals data as well with Gemini, but you probably will have to use our Medgemini particularly, which is trained on medical data and not the generalized version of it because that will not be able to understand these medical terms, etc. Offline tools, I might need a little bit of context there, like what exactly your use case? Why do you need an offline access? But probably we can connect on Discord channel and see if I can answer your questions there. So next question is, yes, what are the key differences between Gemini 1.5 Flash and Gemini 1.5 Pro? Got it. Good question, shrijan i think uh answer is two major differences uh one is gemini 1.5 pro is much larger and much complex model which means if you're trying to solve a very complex use case which needs a lot of processing and the model to be able to understand and tone down the problem, then Gemini 1.5 Pro would make sense. And I always recommend and Gemini flash specifically built for use cases, which is which needs real time response, right? Because when we had launched Gemini 1.5 Pro earlier, a lot of customers were coming back and saying that, you know, since it's a very big model and very complex model, it will take some time to respond, process and then come back. Hence, we were seeing this coming from a lot of customers. And that's the reason we came back to Gemini Flash, which is specifically built for real-time new cases, right? For example, if you're building a web application to answer questions in real time or to process data in real time, then Gemini flash would be the right choice and gemini 1.5 pro is more is going more complex problems where you need where you don't need to work with a lot of real time and where the response time is not very important so that's the key difference in terms of choosing the right model depending on your use i think we're five minutes over shweta just Jig. I think you probably had to run as well. Is there a question that we have? I believe there is one question that we just got. If we take this as a last question, and if some of the questions are still left unanswered, we'll share with Arithi over our doc over email after the session, and we'll get back to the participants later. Sure. Can Mordex AI build chatbot can provide a professional response like give some charts, graph or insights that related to it? Specifically charts and graphs, to be very honest, I have not seen. But depending on your source, for example, even let's say you attaching a chart uh you'll probably attach that either maybe in a document or as a png file right so depending on how you have attached the data store gemini will be able to also the chat bot will be able to come back with the response right if it's part of a document then gemini always uh comes attach the source link at the bottom. Right. I haven't I do the best of my knowledge. I don't think a response will be on some charts and graph spaces. But I do remember that we were working on this. It's currently available in preview, which means will not be available to all the users, but should be out very soon. It's something that is being worked upon, to be very users, but should be out very soon. It's something which is being worked upon, to be very honest, and should be out soon for the Jamai version. At least in workspace, it's available. I'll double check on the Vertex CIP on the chat box. Great. Thank you so much, Aditi, for your time and for the detailed session. I'm sure the participants have now much better clarity on how to integrate who will give us any tools in their solutions and better solutions for the side. Thanks again. I'll just take up a few questions related to the hackathon. Right. So thank you for that. Thank you. Okay. So for everyone who has been asking about the slides for this session, the entire recording of the entire session, starting from the introduction and all the slides that are presented as well as the Q&A will be uploaded in the resources section. So you can just log into your dashboard, go to the resources section and check out the entire recording of the workshop. And I saw some more questions around the hackathon that is, was it online or offline? So the entire hackathon is online, all the workshops and everything as well as the grand finale will be online. Make sure that you keep a check on your email so that you do not miss out on any important information. As per the deadline, we have 17th of November is the final date to register and submit your ideas. Okay, so this is the feedback form for all the participants who have joined us today. You can just plan the QR code and let us know your feedback on the entire workshop. What did you like? What did you not like? And all the Q&A session and everything. And we will get back to you over the email and whatever queries are still left we will make sure that we pick those up and share with the with our speaker and we'll get back to you with the responses great so with that let's wrap up this workshop uh all the very best for your ideas of issues.
Workshop-1 (English) | AI for Impact APAC Hackathon 2024 by Google Cloud
Welcome to the 1st workshop of AI for Impact APAC Hackathon 2024.

The AI for Impact Hackathon, presented by Google Cloud and powered by Hack2skill, is a unique opportunity to leverage the transformative power of AI to address pressing social challenges across the APAC region. By utilizing Google’s GenAI tools, including Gemma, Gemini, Vertex AI, and Gemini Code Assist, participants will have the chance to create innovative solutions that can drive meaningful change and improve lives. By harnessing the potential of Google’s cutting-edge technologies, develop solutions that address a wide range of societal issues, from healthcare and education to sustainability and accessibility.

To know more and participate, visit: https://vision.hack2skill.com/event/aiforimpact_reg?utm_source=H2Ssocial&utm_medium=H2Ssocial

Register and submit your AI idea before 17th November, 2024.