 Hello everyone. My name is Taz, tech leader for AWS Data Analytics, here today with Nitin Arora from Amazon Finance and Amazon Vice President and Distinguished Engineer, Ippo. Welcome to the Data Foundation in the Age of Generative AI breakout session. Now, really quick, who here is feeling lucky and can guess the approximate number of people we have in this room? Yes. All right, any other guesses? 300. 300, couple more? 275. 275. 275. 275. Good one. All right, one last one. 252. All right, one last one. All right, no prizes for guessing it right, but you are in Vegas, you know what to do. It's about close to 400, so thank you for playing along. The thing is, over the past three decades, the world of data has gone through many evolutions, marked by key defining moments. And I would probably not be wrong when I say that between the 400 of us over here today, we have witnessed the birth of several of these milestones, like data warehousing, big data, NoSQL, machine learning, and unless someone has been living under a rock, you've probably heard about generative AI too. Data has been the driving force behind these technologies and that is no less true for generative AI or Gen AI. But let's be honest, data is not the first thing that comes to mind when you hear about Gen AI, does it? So today, we will see how Gen AI impacts and influences data engineering, and how AWS is scaling and evolving its data foundation capabilities to meet the demands of building a GenAI application. And by way of agenda, in the next 60 minutes, we'll start with a quick understanding of the idea of a data foundation and how AWS delivers it. We'll then cover several data foundation capabilities that AWS is scaling to meet the demands of building a Gen AI application. Then we'll follow up with a real world use case presented by Nitin from Amazon Finance on how building their data foundation on AWS accelerated their journey to deploy a data mesh and then quickly enhance it with Gen AI features. And then we'll wrap it up with Ippo taking us through a deep dive on how AWS is evolving its data foundation capabilities to meet the demands of the future. Now, I have been in the data business for close to around 25 years. And although the term data foundation may seem simple enough, it is one of the harder things to articulate. So like, picture this, for example, you have someone new joining your team and they ask you, hey, can you tell me a little bit more about our data foundation strategy? How do you respond to that? A good way of understanding it is to break it down into a what, how, why and who questions. What is it? It's a behind the scenes organizational strategy that centers around the ingestion, integration, processing, transformation, and governance of your data. Who is it for? It is of course for the employees, but it's also for the partners that work with the organization data, and in many cases extends to the customers that work with the organizational data, and in many cases, extends to the customers that work with that data. Why is it needed? This one's simple, for data-driven decision-making. And the loaded question of how, how do you deliver it? This is primarily through a broad and well-interconnected set of data sources and solutions. And AWS delivers the how through a comprehensive set of data, machine learning, and AI services with direct and native integration capabilities for data within and outside of AWS and comprehensive data governance to do self-service analytics. And the AWS Data Foundation goes beyond technology. It understands the critical role of people and processes in an organization and incorporates these factors to accelerate what is called the data flywheel effect, where data is treated as a product designed to deliver a rich customer experience with scope to improvise based on feedback that is channeled back into the data. And the benefits can be broadly classified or categorized into business and technology. Business benefits include tactical outcomes like efficiently organizing quality data, making it easier to trust, which in turn makes it easy to monetize it. And strategic benefits such as having the ability to quickly adopt new frameworks, for example, data mesh, and easily pivot to a new way of doing things, Gen AI. And the technology benefits include the findability, accessibility of this data for effective discovery and access control at scale, better interoperability between different data types and different processing applications, and increased reusability with a well-defined usage contract for repeated use. And these benefits are true for all kinds of data architecture patterns, whether you are building a complex data mesh or a simple data pipeline, something like this. A conventional data pipeline with the typical ingestion processing governance components. So how do data foundation capabilities and data pipelines change in the age of Gen AI? We'll address this through the lens of how AWS is scaling its data foundation capabilities to meet this change. Before we get into it, let's quickly level set on what is Gen AI. Very simply, it's the use of artificial intelligence to create new content using AI models trained on large amounts of data. And we see a trend of popular application building patterns here at AWS. The pre-training your own model approach is when a customer builds their own model from the ground up, pre-trained on large amounts of data. The continued pre-training process tweaks the model parameters to accommodate newly available unlabeled input data. This is followed by the fine-tuning a pre-trained model approach. This is where you take a pre-trained model and train it further on a smaller labeled data set. And RAG, retrieval augmented generation, one of the more popular methods due to its ease of use is an example of in-context learning where you do not alter the model parameters or train the model on a specific data set, but instead you guide the model's behavior by passing it additional context. Context can be either situational or semantic or even both. And these building patterns influence the capabilities of a data foundation. But it is not all apples to apples. Along with your data being the differentiator, there are several aspects of building Gen.AI applications that require scaling the capabilities of a data foundation. So let's see how the scope of a simple data pipeline is influenced when you build a GenAI application. One of the very first things that GenAI introduces into the data pipeline is the need for additional data sources. And these are primarily in the form of unstructured data. Now unstructured data is not a new construct. However, it is the dominant part of Gen AI pipeline. And with more than 80% of the world's data in unstructured format, it is now front and center. Unstructured data is data that doesn't confirm to a predefined format and is not organized according to a preset data model. So metadata discovery for unstructured data can be challenging. Let's walk through a high level architecture of a solution that we can build on AWS to process the metadata for unstructured data. We start by ingesting the raw input data into an object store like Amazon S3. Then depending on the type of data, we can use one of several AWS AI services that are designed to extract metadata from this unstructured data. The most common use services are Amazon Comprehend, Amazon Transcribe, Amazon Textract, and Amazon SageMaker. The metadata is then extracted to an S3 bucket and we can then optionally apply additional transformations to enrich and refine the data sets with an AWS Glue ETL job. Next, we build a metadata catalog in S3 that houses the extracted metadata attributes that help in the discovery of this data. And for discovering and accessing the catalog itself, we can use the AWS crew data catalog and can also apply Amazon Athena as a query engine and deploy governance with AWS Lake Formation and Datazone. Coming back to our pipeline, again, depending on the Gen AI application building approach you take, your data processing phase is also influenced. You could be working with feature engineering to extract information from your data warehouse or data lakes, or you could be running inference for your continued pre-training approach. Or you could be managing vector data for a RAC based application for some real time meaningful context. The thing is, all this processing requires advanced forms of data integration and data stores. So let's take a vector data management example with the help of a RAC application. You start by using the Amazon Bedrock service to generate vector embeddings from unstructured data sources and save these embeddings within a vectorized database index. You then have a GenAI client that submits a question to a Lambda query encoder function, which in turn makes a call to a Bedrock API. In our case, we are using the retrieve and generate API. The API queries the knowledge base and then generates a response, which is returned as context to another knowledge base backed Lambda function, along with the original question from the client. Together, these are submitted as a prompt to a large language model, which then generates a more factually accurate response. As I mentioned earlier, RAG with vector data is a popular approach. Let's talk about vector data management a little bit more. Vector data management involves taking your domain data and breaking it into chunks in a process called tokenization. You take the chunks and pass them through a large language model to generate numerical vectors or number arrays. These are then stored in a vector database where the vectors place elements with related meaning close to each other into multi-dimensional vector space. And then searching for meaningful or semantically related data and returning it to a GenAI prompt is simply reduced to a mathematical function of finding the smallest distance between these vectors. In our previous RAG example, the vector storage and vector search capabilities were provided by the Amazon OpenSearch service. It's an elastic search based open source solution that is already widely used for search analysis and log analysis. So choosing a vector store that is familiar is a big plus as you do not take on any new licensing costs. And when your vectors and business data are stored in the same place, your applications deliver higher performance with very little data movement. And in the same spirit of familiar tooling, reduce licensing and faster experiences, AWS offers vector capabilities across many of its fully managed database services, including SQL databases, such as Amazon Aurora and Amazon RDS, NoSQL databases like Amazon document DB and Amazon Neptune, a graph database. Returning to our data pipeline, some Gen AI applications also incorporate advanced learning techniques such as reinforcement learning from human feedback, RLHF, a machine learning technique that uses, you guessed it, human feedback for a highly personalized and accurate response. Capturing this information in the right kind of data stores and efficient pipelines helps with latency and accuracy. Let's go back to a RAGA application example and see how we can tweak a data pipeline to incorporate user personalization context. So before submitting the client question to the encoder, the application can reference additional pieces of information to provide more relevant and meaningful user context. It can query a customer 360 data warehouse's Amazon Redshift to get additional unique customer attributes or it can also maintain a database in Amazon DynamoDB to store the state and history of the conversation. You're good. All right, moving on. Now, a customer 360 provides a complete and unified view of a customer's profile that can drive business decisions. So like for example, you can use C360 for your generative AI application to create marketing campaigns that are more likely to resonate with a specific customer demographic. However, building a C360 solution is non-trivial. It needs access to data that is spread across different types of databases and data stores. It also needs to be able to add new data sets over time. And needs to ensure the correct change data capture techniques are in place. SQL databases and NoSQL databases typically store user operation data like user profiles, conversation threads, and transaction history. To ingest this data into a C360, you would probably build complex data integration pipelines that connect to a source database, identify what has changed, and then load that into the target data store. Zero ETL data integration capabilities by AWS takes away this heavy lifting. They replicate data from your SQL and NoSQL databases on AWS into the Redshift data warehouse using CDC techniques without the need to provision any infrastructure. Likewise, a generative AI application many times needs most up-to-date real-time information on the latest user transactions for a more complete customer view. Streaming ingestion between Redshift and services such as Amazon Managed Kafka and Amazon Kinesis data streams, and the auto-copy feature between a S3 data lake and Redshift gives the GenAI application the most timely context for downstream processing. Now, because the GenAI application opens itself up to a much wider variety of data sources and data personas, data governance becomes a function of the entire end-to-end data pipeline. Behind the scenes, data sharing, data privacy, data quality, data cataloging play an important role towards delivering comprehensive data governance for a GenAI application. Data sharing on AWS makes data available without having to move the data or create copies of it. Whether you are looking to securely share data from a data warehouse or a data lake or a data marketplace, services such as Redshift, Lake Formation, AWS Data Exchange and Data Zone make it simple to do point-to-point or centralized data sharing. Data sharing also gets easier when your data is well-organized and catalogued. You can build technical and business catalogs with DataZone and Glue Data Catalog for different data types and natively integrate with many AWS services to help you quickly deploy governance best practices for data sharing. And finally, no GenAI application can be taken seriously if user privacy and data quality are not managed. So AWS has scaled the AWS Glue service to detect and process sensitive data such as PII data to increase compliance, and has also equipped the Glue service with machine learning algorithms and rules-based automation capabilities to monitor, detect, and mitigate quality issues. So to summarize, we covered several dimensions of how AWS scales its data foundation capabilities for an enhanced Gen AI experience. We looked at data processing features for structured and unstructured data to improve Gen AI functions such as training large language models. Then we covered data integration capabilities like zero ETL, auto copy and streaming ingestion that advanced the user experience, followed by databases that are optimized to include support for vectorized structures that add important context to the RAG approach, and rounded it up with data governance features that deliver a secure and reliable experience. But we are not stopping here. We continue to evolve our data foundation capabilities and Ippo will cover those right after we hear from Nitin from Amazon Finance on how a strong data foundation helped them build the data mesh and then quickly enhance it with Gen AI features. Thank you for your time. Handing over to Nitin. Hello everyone I'm Nitin Arora. I'm a senior engineering manager for a team called finance automation at Amazon. I lead teams that build and operate data services communication and work management software for finance operations. Now what exactly is finance operations at Amazon? Well in short it's an organization responsible for three key things. First it ensures that we pay our vendors and our employees on time with accuracy. Second it ensures that our customers are paying Amazon timely. And third, we handle every financial transaction at the highest bar of controllership and security. Finance automation is finance operations tech team. Our technologies empower finance operations to support Amazon's growth with better productivity, automation, and self-service. We support all Amazon business lines such as AWS, retail, advertising. Finance operations consist of thousands of employees. Our analysts who are a part of the finance operations team are responsible to support vendors globally across multiple business lines in a variety of languages. They respond to general questions from vendors. They handle multiple disputes or disagreements with the goal to build a strong partnership with world's largest ecosystem of suppliers. Let's start with a quick show of hands. How many of you have bought anything or something from Amazon? Pretty much all of you. So, wow, first of all, thank you and really appreciate you being our customers. So now let me provide you some context about the scale of our work. Customers like you are ordering various things at Amazon and to meet the customer demand, to meet meet your demand Amazon is ordering large volume of products from vendors in bulk we distribute these products to fulfillment centers and pay vendors for the goods that we've received for the most part this works pretty straightforward where it gets complicated is an Amazon scheme we need to handle different complicated scenarios such as defective products returns shipping damages which adds complicated is an Amazon scale we need to handle different complicated scenarios such as defective products returns shipping damages which adds complexity to the vendor payment process there are hundreds of teams and distributed systems across Amazon that are responsible to manage a complex supply chain functions like ordering inventory inventory, receiving, returns, and finance is responsible to track events across billions and billions of transactions and piece together the financial picture. Now that is just Amazon retail, one of business lines that finance operations supports. Amazon's vast scale has required specialized teams and distributed systems, each managing a specific business function. While this structure has really allowed Amazon to scale rapidly in trade-off, this has led to data silos. To create the complete picture of a financial transaction, finance needs to stitch together events across hundreds of these systems. Well this translates to trillions of events and petabytes of data to be processed every month and naturally our finance users want this data in near real time. Why? To support operational monitoring, machine learning and data analytics use cases across the company. Let me give you some examples. We are paying hundreds of millions of dollars in vendor payments every day. We use real-time data to monitor that these payments are made accurately and timely. We use machine learning to continuously monitor for payment patterns to detect and prevent any fraudulent transactions. We use data analytics to analyze our cash and collection practices to improve Amazon's cash flow. Such data forms a big part of how Amazon is making key decisions and running its finance operations effectively day to day. When we started on our journey to build the modern data foundation, we identified three key objectives. First, we faced a landscape of fragmented data, multiple teams, multiple copies, multiple locations, leading to inefficiencies and waste. Our goal was to create a single source of truth for business analytics and machine learning by minimizing the redundant copies of the data. Second, to truly minimize the data duplication, now we needed to make the data more accessible and discoverable. We required a central data catalog where users can find the data sets that they need. We wanted to ensure that every data set present on the centralized data catalog was reconciled and validated through a series of data quality checks. Third, with data from hundreds of systems, we required robust permissioning, stringent data security, and stronger governance. We required comprehensive audit trails to ensure that the right people have the access to the right data at all times. I mentioned about Amazon's distributed landscape and its trade-offs, which was data silos across the organization across multiple data producers. Well to address this reality we needed to identify a way such that data can be easily but securely be connected across domains and systems. We selected Data Mesh as our foundational strategy as it advocates for decentralized approach to data management. It advocates for self-service data sharing where business owners can come in, find the data that they need and make a request. That request is dispatched to a data producer who decides the right level of access to be granted. The central data catalog or the central catalog is place where we implement discovery, data governance, and store good documentation of data sets so that users can understand the quality, context, and the usage guidelines of data. Doing these allows us to treat data sets like data products. On this slide, you will see two sides of our Data Mesh Foundation, data producers and data consumers. Data producers are responsible for bringing data from those hundreds of upstream systems across various domains such as payables and receivables into the data mesh. The producers are responsible for data management, data storage, data quality, all those types of things that are required to finally produce good data sets, good data products into the data mesh. On the right hand side are the data consumers. Consumers are the teams that are doing analytics, advanced analytics and machine learning types of use cases. Our goal is simple, integrated once, consume indefinitely. We want to integrate that high quality data set into the data machine a way that is trusted in a way that business analytics and machine learning teams can consume the data easily and start solving business problems as quickly as possible. The central data catalog that you now see on the slide is the place where we implement permissions, apply guardrails for consistent metadata management, define governance policies and audit usage. Throughout the implementation, we are using AWS data integration capabilities such as Lake Formation and Redshift Data Share to share the data without actually moving or copying it. On benefits leveraging the data architecture of AWS we've achieved incredible business agility to evolve our data infrastructure with ever-changing business or technology demands. The time taken to make the data available and consumable has improved from many months now down to just few days. We've achieved faster performance for less cost, saving millions of dollars at this point by reducing our compute and storage footprint thanks to the AWS data integration patterns. We've moved from what used to be hours of long-running ETL jobs to now sub 15 minute job that has improved the data freshness and allowing business to make faster decisions. Our decentralized architecture has helped us to move away from the concept of large monolithic structure of teams to now move the data teams into the business or domain-specific organizations who have the specialized knowledge of data, allowing them to innovate faster while a centralized, horizontal data services team is responsible for data mesh, data governance, and applying consistent guardrails across the organization. Well with this data foundation we can now take the advantage of Generative AI much easier and faster. With our strong data foundation, good data, comprehensive catalog, reconciled data, we are now positioned to integrate these data products into our business processes. Generative AI is a huge opportunity for Amazon Finance. It has a potential to improve many business workflows, allows us to make faster and smarter decisions, and improves the productivity with better customer experience overall. As we looked at finance operations, our first priority was to improve the productivity of the analysts that are multitasking between several customer or vendor requests. At a high level, our data foundation and generative AI provides excellent capabilities to understand the intent of a customer request, identify solutions that align with these business policy documents, combine and present financial data across multiple sources, and finally execute or draft the next steps for review. Our goal is to automate any human routine task and position human as the final decision makers. To meet this end state, first we must identify how do we understand the business context from these policy documents, integrate data and then recommend actions for human in-loop execution. So how did we get started? Well we had the data mesh set up so what was missing was a good understanding of business rules from the policy document. The data in the policy document is highly unstructured, but a great source of reference to understand every human work that is happening at finance operations. We took a two-step approach, working towards our end state. In step one, our goal was to understand the business context from the policy document, while also enabling a conversational experience for the analyst, where they can search and retrieve solutions instead of having to browse through hundreds of pages of documents. In step two, we merge the financial data with the business context to suggest a precise action that an analyst should take to resolve a customer issue. Let's take a small detour and talk about how did we actually understand the context behind these policy documents and integrate data. Well, to build a comprehensive understanding, we utilize the power of AWS Vector Datastore. As you can see from this slide, with unstructured documents stored on S3, we extracted that data, chunked that data, converted into vector embeddings and loaded into a knowledge database. We chose Titan embedding models based on performance, accuracy, and cost benchmarking. Finance was already using OpenSearch for real-time analytics and high-throughput transactional search. Building on what Taz was mentioning before, the addition of vector database capabilities to OpenSearch allowed us to adapt quickly without building any additional data infrastructure. We chose OpenSearch based on performance, based on its ability to search across diverse document types and handle large-scale data volumes specific to our use case. Now, with evolving business, the policy documents change over time. They are not static. So we built a RAC pipeline, which allows us to search and retrieve policy information based on user prompts from analysts using a conversational interface. In step two, we built on the existing RAC foundation and as you can notice on the slide, we've now added the capability to pass the financial data which can allow specific recommendations with data for our analyst to resolve a customer request. But why is this significant? This approach combines the business contextual understanding from the policy document with precise financial data, the high quality data from the data mesh to enable targeted problem solving. I shared the data mesh architecture before. And now you will see that we've added the vector data store and LLMs connected to the centralized data catalog, an integral part of our data mesh. Once again, this is how strong data foundation has allowed us to adapt generative AI quickly and easily. With the combination of generative AI and data, humans are no longer the consolidators of data. It's LLMs that are consolidating the data for our use case. We're in the early stages of our journey, but our initial results are promising and encouraging. They are showing that we are accelerating the time it takes to understand the context of a customer problem. We're accelerating the time to get answers. We are accelerating on what should happen next. We launched a Gen AI chatbot to Q&A on the policy documents and improved the cycle time by more than 80%, improving productivity for hundreds of analysts across a major business channel. The human productivity continues to improve as we onboard more channels to the chatbot and achieve our vision to enable a virtual assistant for every finance operations employee. By integrating the data, we started to enable analysts with well-defined summaries and suggested next steps which will allow analysts to resolve customer queries across thousands of emails with improved productivity and elevated customer experience. Obviously we are not done yet and we have a way to go. We have more domains, datasets, business policy documents to onboard into the data mesh. We need LLMs to answer to different types of issues, policies, and a wide range of prompts with high quality data. As we add more sources, domains, we need to get better at discovery. We need stronger governance. We need the ability to semantically search the data within the data mesh. With good metadata, we can now generate queries and execute them using natural processing language. We still have cost optimization opportunities. About 60% of our pipelines are in scope to benefit with zero ETL and optimize the cost further. I second-task and reiterate that good data foundations on AWS are must-haves for generative AI. I'll end by saying that we strongly believe that our investments in the area of evolving our data foundations in the age of generative AI will transform how we improve the productivity for analysts and redefine the bar of customer experience at finance operations. I hope this was helpful. Thank you. I'll now hand over to Ippo, who's going to talk about AWS foundation or future of AWS foundations in AWS. Thank you. Thank you. Hello, everybody. Thank you, Nitin. So my name is Ippo. I'm an engineer, distinguished engineer at AWS. My responsibility is kind of the technical evolution of our analytical services at AWS. I guess Matt this morning said in his position he's not supposed to have favorite child but EC2 is his favorite child for me I come from Redshift and we're potentially can say that this is my favorite child so as Nitin said with our counterparts on amazon.com data is what it fuels analytical and AI experiences services and in particular it is your data that allows you to go and build specialized customized tailored applications analytical and AI applications that drive your business and you have been has been choosing over the years over the past decade or more you have been choosing our comprehensive set of services to develop these applications. You have been using a multitude of our services and you appreciate the depth of these services and we have especially our price, our performance, our security posture that differentiates us from other vendors. But over the past years one thing that we have heard over and over from you, from the customers, was that even though you really appreciate the depth of these services and the capabilities, you would really like to leverage the same set of services through more unified experiences. And you can categorize, you could say that there are two layers in what you were asking, unification that you were asking. The first one was on the data layer. You wanted the data unified data management layer. And on top of that you wanted a unified development experience that can allow you to develop applications faster and we are excited today this morning Matt announced our evolution of the SageMaker service the next generation of Amazon SageMaker that now it is becoming the center of your data analytics and AI on AWS. What we are doing, we are essentially doing, we're adding, we're enhancing SageMaker with a bunch of capabilities. There are three basic components that I would like to talk about today. The first component is the SageMaker Unified Studio. It's a unified IDE, a unified development experience that can allow you to develop data processing applications. It allows you to run efficient SQL analytics using Redshift, it allows you to go and do development of models, it also allows you to develop generative AI applications and as Matt said this morning we will keep on adding capabilities in this unified development experience by bringing in streaming, by bringing in search, by bringing in business intelligence. On the layer below that we are evolving the capability of our data zone service and we are offering SageMaker data and AI governance capabilities in a easy to use very very powerful capability that we're going to talk about. And then at the bottom of it or even at the core of it we are introducing SageMaker Lakehouse which is an evolution of our existing Lake Formation Glue Data Catalog, Technical Catalog that now allows you to manage and access data from Redshift managed storage, from open file formats in S3, or even the newly announced S3 tables that we talked about this morning, and a bunch of other capabilities that I'm gonna talk. And we're making it super easy to ingest, to move data into this lake house. And by doing that, we are allowing you to collaborate and build faster. You have one unified data and AI development environment, and we have a very broad set of tools that allow you to build generative AI applications. We are bringing down the data silos between warehouses and the lakes by offering a unified data access, data management layer that can be also accessed by open APIs such as the Apache Iceberg REST API. And we are doing that and we are allowing you to set up your enterprise security and governance by offering, for example, abilities such as fine-grained access controls in this unified data management layer. I'm an engineer and there are many talks about the next generation of Amazon SageMaker throughout this week. And I'm pretty sure many of them, most of them, will index on the unified studio experience because it's also easy to demonstrate. I love it. I have been using it. It's very, very impressive. But since I am an engineer, I will actually start from the diverse order. I will start from the unified data management layer. You have been using a bunch of tools from our service, a bunch of our services in order to address your use cases. You have use cases that require the performance, the high throughput, the ability to run multi-statement transactions and obviously to offer low latency by accessing data from say main memory and you have been using data warehouses in order to achieve that. But at the same time you also love data lakes. As Matt said this morning we have thousands, if I remember he said, hundreds of thousands customers that have built data lakes on S3. You like the flexibility of the storage, you love the flexibility of S3, you love the ability of the open, the flexibility of the open file formats. 15 years ago I was in my previous employer, I was part of developing Parquet back then. It was early on, we were developing, innovating on Parquet and I could not imagine back then the impact of this thing that would have in the industry. Pretty much everything right now is on Parquet file format and it is impressive to see this thing, how it has evolved over the years. And you also like the ability that you can have different tools, specialized tool to access this data. But you don't want to be thinking about this split brain. You don't want to be thinking about warehouses and lakes and have catalogs in different places and setting having to set up governance in two different places and whatnot. What we have been asking is a unified solution what people you want us to bring the warehouse capabilities and the lake capabilities together. I was advocating when we were developing this capability over the past year or more I was advocating to call it where lake. Because warehouse and the data lake were lake. But my suggestion did not go wrong. So we turned it around and we called it Lake House. So Lake House is a... we are bringing in the capabilities of the warehouse of the lake and we allow you to bring any service to operate on top of this data and allows you to set up consistent governance access controls to this data. And that's what it is. The SageMaker Lakehouse is a unified data management layer. It is open. You can access all of your data on AWS on analytical services on AWS through an open API. We are offering an Iceberg REST API so you can consume it these things from our one piece services as well as third-party services as long as they can speak the Iceberg REST API and it is secure. Let's dive a little into some of the capabilities. The first thing we believe at AWS is that you need to have flexibility, you have different needs, you need to have flexibility and options in the storage. We are offering three different types of storage for analytical data. The first one is very commonly used, the data link. You can have a general purpose S3 bucket and you can store your data. Most of you, probably with the ease of hand, most of you you will have a data lake somewhere in your ecosystem. You can access the data and one of the file formats that has been coming increasingly popular is Iceberg. And you can access your data using the Iceberg protocol such as the REST API. On AWS, we are offering optimizations on your Iceberg data lakes. For example, you can go to the Glue Data Catalog, enable automatic table optimizations, and we start making sure that we are keeping your data combed and whatnot. We are also enabling automatic statistics collection so that we can populate the technical catalog with statistics that then the query engines such as Redshift can take them and use them in cost-based query optimization in order to make sure that it generates efficient query execution plan and has good performance. And you can access the data from one-party services such as EMR, Spark, Athena, Glue, SageMaker, Redshift, as well as third-party services such as Apache Iceberg, Presto and what have you. Today this morning Matt also announced that we are offering also a managed iceberg as S3 tables which is a managed iceberg service by powered by S3. It's a new class for Apache Iceberg Data Lakes. It has high throttles on the written right throttles, RPS, which allows you to have very good performance right off the bat and it is a fully managed iceberg table maintenance. Estuary tables are part of the lake house, of the SageMirme lake house. And we offer management, automated management for these S3 tables. For example, right off the bat, you have ability to combat the data. And another request that has been highly requested was the ability to have snapshot retention. So you can say that this snapshot, when it expires, go and drop it and also clean up the S3 data that are associated and are not being referenced in full-cutter versions of this iceberg data. But we're also bringing the Redshift managed storage into the lake house. In Redshift, we have over a decade of experience and technology on analytics optimized proprietary format and we are opening it up to the SageMaker Lakehouse. If you have an existing Redshift data somewhere, we are starting today, you have a button to publish this database to this namespace into the lake house and you can allow now any other compute to go connect to these databases and start reading and writing to it you can even go to the lake house and create new database powered by the Redshift manage format and in this proprietary storage layer, which can be accessed through the same Iceberg REST API, we have many many years of experience and technology, underlying technology based of machine learning, which allows us to continuously monitor and optimize the system, for example, changing the physical design of the database in order to ensure to give you optimal performance. The Redshift Manas Torus is an analytics optimized format that is also very amenable for near real-time operational analytics applications. For example if you have some stream application streaming that inserts little data continuously into your analytical store, doing it on the Retsch with Mana storage is very efficient. And not only that, but performance does not degrade as you keep on inserting small, small data because of the way that the format is implemented. It also gives you multi-statement, multi-transaction, kind of transactional consistency. And what we have seen is that by having, for example, Spark writing to Redshift Managed Torus, you can get up to 3.5 times better performance than having Spark writing to, say say Iceberg on S3. And when it comes to low latency queries, we have a very common like you have a business intelligence application or you have many concurrent users that want to get answers very quickly, the Redshift managed storage offers up to seven times higher throughput than any other option out there. So that's what it is. The SageMaker Lakehouse is a unified technical catalog. You can put the Redshift managed storage, you can put the open file formats in S3, you can put your data in S3 tables and you can access it from any service that speaks 1P analytical service on AWS, as well as 3P services that speak the Iceberg Crest API. It's open and it is also secure. We are also spending a lot of our energy into making sure that you can bring in data into the SageMaker Lakehouse very easy. In particular, over the past two years now we have been innovating a lot around the zero ETL technology. We have GA the ability to ingest data, zero ETL data from Dynamo, from Aurora MySQL, Aurora Postgres, RDS MySQL into Redshift and by the mechanism of publishing that to the lake house. And I can tell you that every morning one of the reports I get is what is the latency? What is the other latency of all the customers that have been using Zero ETL and they are ingesting data, writing data to Aurora, what is the latency it takes to appear on the other side and this data to become available for analysis on the other side, on the lake house. And then does anybody have a guess? It's actually on average it's single digit seconds, around eight or nine seconds today on average to move the data from Aurora to the analytical side. And today as Matt announced this morning we are also excited to extend the capabilities of our zero ETL capabilities by now enabling zero ETL from enterprise applications. We are starting with eight applications that being Salesforce, SAP, ServiceNow, Facebook Ads, Zendesk, Zoho, Instagram Ads as well as Salesforce Management, I think the Pardot is Management Cloud, Marketing Cloud. And those now in the price applications can be zero ETL into the SageMaker Lakehouse and you can consume it on the other side with any tool of your choice. We are also bringing the federation connections that we have in the Lakehouse and we are also bringing in streaming. We have been having streaming support in streaming ingestion into Redshift and again to the Lakehouse. We have been supporting Kinesis data streams as well as Managed Kafka for some time now. As of last month we opened it up to Apache Kafka as well as Confluent and we have customers that are ingesting gigabytes per second using streaming ingestion to Redshift and Redshift managed storage and buy that to the lake house. And all these things are become of a unified technical and business catalog and in the studio you can go navigate your data and you can see all the data that are indexed by the Lakehouse. You have a dynamic catalog hierarchy and you can have managed catalogs, you can spin up a catalog that is supported by your S3 bucket or by Redshift managed storage, but you can also bring in existing Redshift installations as well as the S3 tables and the federated sources. The way it works, it's for example in Redshift, you go, you have say we have exabytes of data managed in Redshift, you press a button, it goes to the Lakehouse as yet another catalog in the lake house. And then you can have other existing Redshift compute, you can spin up another serverless compute to start reconnecting and reading writing to that, a third one some other data scientist that is very scary, writes these very crazy joins that you don't want to be isolated from the rest of your production and you can go and connect to that and start reading and writing. You can also use EMR Spark now or Glue or Athena or even third-party applications that can speak Iceberg APIs. And you can set up your access controls at this layer. For example, we give you ability to do column level security, row level security or a combination of them or we give you the ability to do tag-based access controls to all the data that are at an index in the SageMaker Lakehouse. And on top of that we have the data and AI governance powered by data zone and queue that allows you to create projects and you can put in data assets, models, compute, then AI applications and with a git based collaboration it allows you to go and develop and create applications on top of that. And we are adding that we have we have this also gives you, it's a business catalog that allows you to go in and discover data sets in a Q-based format. You can say, hey, find me the data sets that have information about my sales. You can subscribe to that, put them into the project and have your team collaborate and develop over there. And all of these are happening using the Iceberg REST API. Iceberg has been taking off over the past few years and we have many, there are many services first party and third party that can speak to that. So we are putting a REST API on top of that and you can come in with a SageMaker Unified Studio or analytical services or third-party services. I did ask for two hours for this session because I have too many things to say. Unfortunately they told me I have only one and I have only four minutes. So let's go to the development experience. The unified studio brings in together previously we were asking you to jump around consoles, go and do something on the EMR console or the Glue console, on the Redshift console or Athena or in a Glue data catalog and that was creating, it was challenging. So what we did and also developed generative AI applications on top of that. So the unified studio is a single data and AI development environment that allows you for example to go and create applications, machine learning applications, generative AI applications, it can allow you to do data preparation if you are a data engineer, you can run SQL queries, you can also use Bedrock IDE to develop generative AI applications. And you get access to the projects where you have put in your data, your code, your models and your compute. If you have an existing EMR cluster or some glue endpoints or Redshift endpoints, you can bring them in, you can keep on using, if you have developed some processes, you can keep on using those, but you are also evolving the way you can consume this setup you have already created. So you can train and deploy AI models using SageMaker AI, you can build generative AI models using SageMaker AI. You can build generative AI applications with Bedrock IDE which is in preview. You can prepare and integrate data using EMR or Glue and you can run SQL queries using Redshift. And all of these operations are accelerated with Amazon Q Developer, which is the most capable generative AI assistant for software developer. We have put Q everywhere there, you have a very nice natural interface to go and develop. So that's it. I tried very comprehensively to explain, to give you a glimpse of how we are evolving what is the next generation of Amazon SageMaker. If you are using our existing services you don't have to go and change something, we are bringing in the existing services and we have been integrating them in Amazon SageMaker and we are excited we want to hear from you, we want to see what you're gonna develop with that and looking forward for your feedback. There are many sessions over this week about SageMaker, the next generation of SageMaker. Please attend, please send us feedback and please develop on us. Thank you.
AWS re:Invent 2024 - Data foundation in the age of generative AI (ANT302)
An unparalleled level of interest in generative AI is driving organizations of all sizes to rethink their data strategy. While there is a need for data foundation constructs such as data pipelines, data architectures, data stores, and data governance to evolve, there are also business elements that need to stay constant such as organizations wanting to be cost-efficient while efficiently collaborating across their data estate. In this session, learn how laying your data foundation on AWS provides the guidance and the building blocks to balance both needs and empowers organizations to grow their data strategy for building generative AI applications.

Learn more:
AWS re:Invent: https://go.aws/reinvent.
More AWS events: https://go.aws/3kss9CP 

Subscribe:
More AWS videos: http://bit.ly/2O3zS75
More AWS events videos: http://bit.ly/316g9t4

About AWS:
Amazon Web Services (AWS) hosts events, both online and in-person, bringing the cloud computing community together to connect, collaborate, and learn from AWS experts. AWS is the world's most comprehensive and broadly adopted cloud platform, offering over 200 fully featured services from data centers globally. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—are using AWS to lower costs, become more agile, and innovate faster.

#AWSreInvent  #AWSreInvent2024