 Thank you. Please welcome the Vice President of Technology at AWS, Mylon Thompson-Bukovec. Hello, everybody. Welcome. My name is Mylon Thompson-Bukovec, and I'm a Vice President of Technology at AWS. Now, I have been working on AWS Cloud Services for a long time, since 2010, and I have spent much of that time working with customers like yourself on the right data strategy for your business. Today, I want to share with you three different data patterns that have emerged among AWS customers across industries for data-driven innovation. The success of any modern business is tied to your effective use of data to differentiate. Petabytes of data power the customer experiences of NFL's NextGen stats, AstraZeneca's drug research, Pinterest's visual search engine, and Netflix's streaming business, among many other examples. Now, you have heard the phrase, good data means good AI, a lot recently. But that is not new. The importance of good data is nothing new to any of us in this room. When Amazon first started selling books way back in 1994, data was used at the very start to make important decisions about inventory management and personalization for book recommendations. That is still true today. And data drives everything from the MLI models that let your packages get there on time to how we in AWS manage the operations of our services. Data is the cornerstone to all of Amazon businesses, and that is true for you as well. Netflix transitioned from DVD rentals to a streaming business in 2007. And that was a huge shift for rethinking all of how they use data for the digital age. We had ride sharing companies like Lyft that disrupted transportation by using data to drive algorithms that matched up drivers with riders and optimized routes based on real-time changing traffic patterns. And Airbnb used data to create a marketplace that brought together hosts and guests. Now, the most common thread between all of these companies and many others is that they all used vast amounts of data for their business. There were major technology shifts, ranging from the introduction of Elastic Cloud Storage and Amazon S3 to open source innovations like Hadoop and OTFs like Apache Iceberg that made it easier than ever to use data for your business. Our first customer back in 2006 for S3 was SmugMug and SmugMug is a company that is in process of transforming the world of photography. Every day, people from all walks of life, from every corner of the world, pause for a moment to capture something they find special. And though the people, places and perspectives may vary drastically, that shared experience of capturing something meaningful. A moment worth remembering. Not only has the power to connect us all, but to change the world. We understand the value and power of these moments. And we're here to preserve what matters most, whenever and wherever you need us. to two decades, and really it's a unique perspective, given he was the first customer of S3, I'd like to invite on stage one of the very earliest AWS customers and my friend, Don McCaskill, the CEO of SmugMug, to the stage. Don? So as you can see on this slide and many of the future slides, these are the photos shot by that great set of photographers that we just saw the video for. I'm thrilled to be here today. As you heard, I'm the very first S3 customer, so we've been doing this a long time, for more than 18 years. Pretty remarkable. And one of the things that I want to talk about is our size and scale. When we started working with AWS all those years ago, we were an enormous customer for them at a whopping 70 million photos. Today, we're at 50 billion photos. So you can imagine the size of the scale over the years. In fact, we now have more than 100 million people a month using our services and our platforms, all built on top of AWS. So let that sink in for a minute. When we started 18 years ago, we had 70 million photos, and now we have more than 100 million people every month using it. The size and ability for us to grow and scale on AWS has been transformational for us. We have a couple hundred petabytes of data under management now, so it's really getting enormous. Now, some of these big numbers were helped a lot by our acquisition of another platform, Flickr, about six years ago, which we now own and operate on top of AWS as well. And when we first got started way back in 2006, we were most excited about storing all of those photos. 70 million, which seemed huge at the time, 50 billion now. But it eventually dawned on us that there was a lot more we could do with S3 than just storing photos. Now, back then, there was no sort of terminology or best practice guides or anything about big data. In fact, I don't think the term even existed in 2006. So we didn't have anybody to draft off of. We didn't have best practices or consultants or anything like that to figure this out. We had to learn by doing, and we made a lot of mistakes along the way. But it quickly dawned on us that we could use S3 for a lot more than just storing photos, that having an unlimited, bottomless, inexpensive storage tier meant that we could transform the way we managed our data. Our first big light bulb moment was realizing that if we got out of our data center mindset and we had seven data centers at the time, that we could keep all of our data. We'd gotten in some sort of bad habits around optimizing our data footprint because it was just so costly to run in our own data centers where we would keep fresh data around for say one to three months and then summarize it and age it out and eventually delete it because we just didn't have the storage capacity to keep it around but being on s3 enabled us to say take our web access logs and keep them for forever we knew that that that data was super useful in sort of a one-month time frame or maybe a three-month time frame, but it didn't really dawn on us that being able to compare data year over year, or in our case, decade over decade, would turn out to be super valuable as well. So we learned to keep all of our data, stop deleting it. Ooh, big win for us. And our next light bulb moment was realizing that we should really keep the data in its original format. We had similarly, like being used to being in our data centers, we tried to optimize the format either to make it smaller or maybe to make it work with our current software stack, which wasn't very future forward and future looking because of course software evolved and our software stack would evolve. And so we learned quickly the hard way that we should just keep all of our original data in its original format because that gave us a lot more flexibility and we didn't have to worry quite so much about cost because it was in S3 and on top of that we realized that we had better get better at keeping our metadata about all of this data we were accumulating around, things like timestamps and the origin of the data generation and all of that sort of stuff. So we got good at keeping all the data and keeping it in its original formats. And luckily, we were so glad that we did, because we had no idea back then that tools like Athena would come along that would literally let us query the original format data without even having to transform it. It was transformational. And then sort of our next big light bulb moment or epiphany was when we realized that we had inadvertently restricted our data and put up sort of access controls and gatekeepers and everything in front of it that were limiting our team's ability to get their job done and ask the data questions and get answers from it. And we did it out of an abundance of caution because we wanted things to be safe and secure. And we do deal with sensitive data. We're dealing with consumers and customers all the time. But there's a lot of data in there, average order value, sales volume, most popular photo of the day, or things like that that weren't sensitive, but we were still keeping it all locked up. So we learned that we really need to get as much of the data unlocked and share it with our people, our product teams, our business leaders, so they could make the fast, quick, great decisions that the data really existed to serve. And then last, but certainly not least, after we made it accessible, we discovered that that wasn't quite enough, right? We needed to go one step further and make it easy to get access to. It turned out that some of our sales, marketing, finance teams didn't love creating SQL queries to get access to some of this data. So we had to meet them where their tooling lived. We had to integrate with tools like QuickSight and create dashboards and natural language interfaces and things to really enable all of those teams to really unlock all of the value that we were now creating and storing in these vast, vast pools of data. So just recently, a couple of weeks ago, I mentioned that we had acquired Flickr six years ago. A couple of weeks ago, a team at Flickr analyzed tens of billions of data points across hundreds of millions of customers to answer a crucial, critical question for us that had long eluded us. When we acquired Flickr it was an amazing community but it was kind of a disaster of a business. It was in decline, it was losing a lot of money, and today I'm super thrilled that it is growing again and for the first time in 20 years it's profitable but it should be growing faster and we'd like it to be even more profitable so we can continue to invest in that wonderful community. So this tiny team at Flickr took advantage of all this work we'd done to create huge pools of accessible, easy-to-get-to data and answered this massive question for us, which is we have more than 100 million people using the service and platform who don't pay us anything. And then we have a much smaller cohort of members who gladly pay us a monthly or annual subscription fee. And we haven't really understood the behavioral differences between the two. We think we do now as a result of having all this data unlocked and accessible to our teams. We think we've identified a set of behaviors that are unique for those paying customers and we are hard at work designing new product experiences to really unlock that value. We're really excited about it. So I'll just leave you with a couple of our suggestions. If you want to transform your organization like we're transforming ours, keep as much data as possible. Don't transform it or if you do transform it immediately, at least keep the original data around, especially keep the metadata, and make it accessible to as many people in your organization as you can, and meet your people where they are. After all, it's the people making the decisions that the data enables for massive transformation. Thank you so much. Thank you, John. Amazing. Thanks. John McHasso. So today I said that we were going to talk about three patterns that are incredibly common across the breadth and the variety of AWS businesses. Almost every cloud journey starts with the first step of aggregate, what Dawn talked about, which is bringing together all of your data sources in S3 so your application owners can take advantage of the diversity of data and the depth of data in your business. Now, this is a very big change from on-premises data architectures. It moves companies away from expensive, vertically integrated data and compute all tied together, and it lets you scale all your data at a massive rate separate from the compute that uses your data. Application owners in the aggregate model have access to the data in a federated ownership model, and they can use it from everything from fraud analytics to knowledge bases, and it lets you take your proprietary business information and bring it to any business application. Many of our customers use aggregate data pattern on S3 very successfully at a very high scale, but we have other customers that take it to the next step, and that is the curate data pattern. In a curate data pattern, application developers no longer have direct access to all of the aggregated data. Instead, application owners select from a much smaller subset of curated high-quality data sets offered to internal or external data marketplaces. In a curate data pattern, you have the ability to control the quality of your data and you have the ability to control the access to it, which means you can manage the governance and you can add a lot more context into your product and your data. Now, sometimes AWS customers build on that data product concept with something called the extend data pattern. And what that does is it adds a data API layer to not just access curated data sets, but to add semantic meaning and standardize on the usage of their data. In Extend, application developers or newly developed agentic systems interact with the data using an API that operates on a single data model. It provides connectors and hooks for application extensibility, and it standardizes how to collect and use data for applications. Now, when you're aggregating data, you're basically sending your data from many different sources, like sensor feeds, data processing pipelines, data streams, log feeds, data warehouses, databases into Amazon S3 to store and use for business applications. This is where many of our customers actually shift another part of their architectures from a legacy batch system to using real-time data streaming services. Many of our customers now are using AWS services like our Manage Kafka service or Kinesis Data Streams to aggregate all that data in real-time from all of these different sources to consolidate, process, and analyze large volumes of information in just one data pipeline. Now, many of you in the audience today are already using the aggregate data pattern, and you are in good company. It is the reason we have over a million data links running on AWS. And we find that customers like the aggregate data pattern because the federated ownership model works well for your organization. It works well culturally and it works well with your different business units when you have so many different application owners. And when you have Amazon S3 as a foundation of aggregate, this data pattern will give you the most flexibility in how you want to use your data across a distributed organization with many application owners. For one example of aggregate, Georgia Pacific, a pulp and paper manufacturer, streams real-time data from over 85,000 vibration sensors on their production line, so they can correlate the telemetry streams from their connected equipment and production lines with other business data like scheduled maintenance so that they can build accurate predictive maintenance models. That is an example of aggregation of your data, again, using a real-time streaming model. Now, there are a lot of advantages to aggregate. By centralizing your data in Amazon S3, which is the common storage layer across all three of these data patterns, application and other members of the team, like data scientists or AI researchers, they have access to a wide variety of raw and processed data sets to use for experiments and application innovation, simply the act of bringing your data together into one place can significantly change the speed of your business. For example, Invista used to run analytics on their visual inspection site, stored in silos in each plant site. Before their migration to AWS, it took their data scientists two months to get their hands on just one plant site's historical data to perform analytics. And by bringing all of that data into AWS, now they can do their analytics in minutes or hours on an aggregated data set because it's more self-service. However, if you are using the aggregate model today, or you plan to, the key to success in the aggregate model is to apply standards in a federated model of data ownership. Okay? So I'm going to give you some examples of different standards. It starts with putting all of the data in S3. That is one standard. For example, Roche, a pioneer in healthcare, they use Amazon S3 to store all their data, genome sequences, radiology images, voice recordings, videos, medical publications, you name it. And when they standardize, they standardize in a second way, which is to do a single ETL pipeline, okay? And so they're using S3, but they run all their data through a single ETL pipeline, and that gives them standardization because it enforces consistent and accurate results across different document types, which helps their different users like analysts or business users accelerate the time to get to the data that they need for the right task at hand. In addition, I have to say many of our largest AWS customers, and that means Netflix, New Bank, Lyft, Pinterest, I can go on and on. They are now standardizing on a file format called Apache Parquet. And they're using it to store their business data, with many of them using open source Apache Iceberg to represent that Parquet data as a table to update it and to interact with it with SQL queries. Now, any text or numerical data that can be represented as a table to update it and to interact with it with SQL queries. Now, any text or numerical data that can be represented as a table, like credit scoring, transaction records, inventory reporting, all that data can and is very often stored in Parquet files in S3. It is a very common form of standardization in data lakes today. In fact, that means that Parquet is one of my most fastest growing data types in S3. I have exabytes of Parquet data stored in S3 today, exabytes of Parquet. Parquet is not a very big file type. It's a compressed format. I have exabytes of it. Customers make over 15 million requests per second to Parquet data, and we end up serving over just hundreds of petabytes of Parquet every day for requests. That should give you a sense of how that has gotten standardized in the largest data lakes running on AWS. Now, this combination of Parquet files and S3 and Apache Iceberg, it's what's powering these data lakes for some of the most data intensive companies. Pinterest has over an exabyte of data in their data lake. They have thousands of business critical iceberg table, and they are an example of a company that has standardized all their tabular data on Parquet. Now, as I said earlier, AWS is constantly evolving to build what you need for your business. And here at reInvent, we are going to be introducing a set of new capabilities that make the aggregate data pattern even more easy to use and even more powerful for your business. I'm going to touch on just a few of these, but there's many others that you will learn about this week at reInvent. Today, as Matt said this morning, we are introducing a new capability called Amazon S3 Tables. That means we are introducing fully managed Apache Iceberg Table support in S3 tables. That means we are introducing fully managed Apache Iceberg table support in S3, native to S3 for your Parquet data. If you store your Parquet data in this new bucket type, you are going to get three times faster performance for your queries and up to 10 times faster transaction or more transactions per second compared to doing it on your own, doing Iceberg on your own with Amazon S3. Now, S3 tables takes care of the work of maintaining the Apache Iceberg table layer that sits on top of your Parquet data. And that means it's even easier for you to standardize on Parquet files in your aggregate data pattern and then use Iceberg, any Iceberg compliant client to access your Iceberg tables. So for the first time ever, as of today, business users can now build business dashboards directly on Parquet data stored in S3 using Q4 QuickSight preview integration with Table Buckets. I am so excited about this. Let's take a look now at what a business user can do with Q4 QuickSight and S3 table buckets. I'm going to go ahead and get a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of. Pretty amazing. I have talked to so many customers who are excited about Q for QuickSight. The ability to type in a natural language question and drill down into beautiful dashboards is so compelling. And now you can take that same capability and use it directly on your S3 table buckets and all the parquet that is the business data that runs your company. Now, I am also excited to say that there's another big change to S3 that we are announcing today, and that is S3 metadata. When you have a lot of data, one of the big challenges that you have is finding the data that you need. And by introducing S3 metadata, we are providing a capability where we generate and store both system and custom metadata for any object. And we're giving you access to that metadata in an S3 table. This is going to be incredibly easy to do aggregate with S3 metadata because you're going to be able to run a SQL query in any Iceberg compliant client and find the objects that you want to use for one of your business owners. So when you turn on S3 metadata for new objects, which is available in preview for you today and coming soon for the existing data you have in S3, whenever a new object is added, we automatically store information that we generate from the system, like the time that an object was uploaded or its size, or as well as any custom metadata that you put in your object header. You can also, after the fact of adding the object to storage, add a custom tag to your metadata, and what that can do is store things like data classification or other properties that help with the data lineage for the object. So a custom tag can be, I don't know, a streaming source, it could be the application that generated it, it could be a data classification. All of this metadata is captured and accessible in real time, and again, queriable by SQL in any Iceberg compatible tool. Now, we plan to extend this ability so that customers can add whatever semantic meaning you want in the near future to your metadata. We want to have unlimited metadata in S3 for you, whatever semantic meaning you want in the near future to your metadata. We want to have unlimited metadata in S3 for you. And that means object metadata, which at the end of the day is context about your data, will evolve into its own data lake, constantly queried, constantly searched to find data sets that your business owners want or your auditors need to audit. With S3 metadata today, you can find objects that match the results of a SQL query. So, it can be objects that have the same custom tag for data classification, they're of a certain size, they originated from a specific data source, like a stream. And this idea of metadata at the layer of S3 means that we are now introducing data understanding into your object storage, so you have the power of object context in your data layer itself. Now, with Curate Data Pattern, instead of having all these individual application owners developing and using their own data sets from aggregated data, the company standardizes on a few data sets that are used across the company. The data and business catalogs like AWS Glue Data Catalog and Amazon Data Zone become an important standardization to the Curate data pattern. Application data developers then access production data sets via an internal or external data market place for their applications, and they no longer go directly to the aggregated data and storage. Now, Curate makes it easier than aggregate to govern usage as well as audit for compliance because it shrinks the surface area for your exposure to just a few data products. A great example of a company that's using the Curate data pattern is Cox Automotive, who is transforming the way the world buys, sells, and uses vehicles. Just a fact for you, more than 60% of automobile shoppers visit Cox properties like Kelley Blue Book or Auto Trader to start their car purchasing process. They aggregate all of their data in S3, but they built a really rich set of data products over that data, which is used for everything from analytics to powering vehicle valuations. Now, Cox Automotive centralizes and catalogs petabytes of data in an internal product that they call the Cox Auto Data Marketplace for developers to use. This is the distribution vehicle for your data products, the internal data marketplace. And they say that the thing that makes Curate Data Patterns so successful for Cox Automotive is the quality of their data, the richness of the metadata in their data, the richness of the metadata in their data catalogs for data discovery, and the hub of having a data marketplace so all their application owners can come to one place to find it. That is a very common set of best practices for people who are using data products and the Curate Data pattern. Another customer that's built an internal data marketplace is Siemens. Siemens centralizes their data in S3, and they have hundreds of petabytes of data from over 300 sources, and those are used to build data products across more than 70,000 internal data consumers. Now, while the data providers are responsible for data ingestion into S3 and maintaining the technical metadata of the datasets, they have this central data marketplace that they call Data Lake to Go that lets consumers find and explore datasets that they need, like a research that wants to build an ML model, or somebody who's trying to optimize their rail operations. Now, before this, before they had data products, it took their internal customers weeks to contact the owner of the data and request access. This is probably very familiar to many of you in this room. But by changing that into an aggregate model with data products, they self-serve their data access now, and people can find anything they need, and they're seeing massive productivity gains. With data products and data like to go, developers were able to build an application that they call intelligent document mapping. And it takes a variety of different documents in their data set, and it builds step-by-step operations which include analyzing the design, creating a bill of materials, even enhancing the drawings. And this application that they built on a data product has really helped them save money and time. Before, their engineers at the factories had to do all of these steps manually. And by putting all of this data into data products, they're saving more than 15,000 hours of work, almost 1 million euros per year, just from centralizing the use on a few well-curated data products. Now, we have many customers using data products in the curate pattern, and to share his story, I'd like to invite onstage Moeed Waheed, the Executive VP and CTO for Experience Financial Services and Data Business. Moeed, welcome. Thank you, Mylan. Good morning, everyone. I'm super excited to walk you through the journey of how we build the right data strategy to build high quality data strategy to build high-quality data products that significantly unleash opportunities for our customers. Experian is a purpose-driven company. We are there when people are making life's important decisions like buying a car, buying a home, or sending kids to college. At least these are the very important decisions I made in my life. We serve 1.5 billion consumers and 200 million businesses with the credit data with the accuracy of three nines across the globe. Our obsession to precision and reliability empowers our clients to make confident decisions every single day. As we embark on our technology modernization journey, our goal was simple, to build the first, best, and only product to delight our customers. I'm going to structure my conversation today, six years of our journey, in four phases. The pre-cloud, aggregate, curate, and AI or extend phase. During the pre-cloud phase, we faced several challenges. We had a monolithic code base, which was preventing us from innovation, and we were using closed- source technologies. Our ETL processes were lengthy and complex, causing significant delays. The data duplication was widespread because of the siloed data across. The data velocity was limited to weekly, monthly, in some cases, quarterly. There is significant underutilization of our data. And a few business processes used to take more than 24 hours. The reason because we had to gather the data from various ETL jobs. These inefficiencies was major roadblock in achieving our business goals. As we embarked on our technology modernization journey to address these challenges that we faced during the pre-cloud phase. My vision was to build a data platform that powers heterogeneous workload, like transactional workload, real-time batch and analytics. First, we hydrated the data from OLTP to our data lake on S3. We invested a lot in automation, automation of our data pipelines using Airflow and S3. This is like super important step that we took standardizing on S3 and Airflow. And for the SQL engine, we invested in Spark and Athena, and we've standardized on Parquet as a data format. The most significant step that we took was from the day one, we enhanced our security posture. We encrypted the data at file, field and disk level. Not only that, we encrypted the data at rest, in transit, and in flight. Today, we perform over 10 billion encryptions per day. For the first time, we empowered our customers to perform advanced analytics using our credit and fraud data. This process used to take 20 days. Now it takes less than two days. However, despite having an incredibly talented team, we face some challenges with data discovery and governance. While we are making some really good progress, recognizing that we need to improve our data velocity to streaming, we embarked on the next phase, building on the progress that we made on aggregate phase. This phase made a significant transformation. We put Bureau on the cloud. We established a unified metadata management using Glue that allowed us to maintain a comprehensive and a consistent understanding of our data assets. Most importantly, we developed an intelligent data lifecycle based on when and how the data is accessed to optimize the storage. to deliver fresher data and help our customers to make decisions faster. And now we process 1 billion events per day. And we are operating at scale, and we also created internal and external marketplaces for our customers to build products on our hundreds of petabytes of data. The next evolution is extending this catalog to AI and ML workload, allowing our customers to power the advanced analytics so that they can derive greater value from our data. This phase not only laid the foundation for our current capabilities, but also set the stage for advancements in data-driven innovations. Building upon the success of the curate phase, in this phase, we extended our data to be accessed through API and implemented federated data governance through data mesh architecture. We also established several KPIs. Observability is the key for operational excellence. To name a few, cost to build, cost to store, time to live, time to archive. By having a unified catalog of data assets and ML and AI models, we were able to innovate and deliver amazing product, Experian Assistant, in less than 90 days. Experian Assistant is embedded in our Ascent platform. This is an on-demand Gen A Assistant designed to accelerate your modeling lifecycle while ensuring responsible use of AI, privacy, transparency, and security. We are hard at work on AI governance and evolving our data strategy for LLMs and multimodal. This phase represents a significant leap in our data capabilities, ensuring that we'll continue to deliver cutting-edge solutions for our customers. There are multiple dimensions in our data strategy that we evolved over the last six years. Every transformation stage has presented exciting opportunities for us to innovate and bring the best products to the market. last six years, every transformation stage has presented exciting opportunities for us to innovate and bring the best products to the market. As we evolve from the pre-cloud to the AI phase, our speed to build products increased exponentially. Our data grew from 100x, but our cost grew only 10x. And we leveraged managed AWS services to accelerate the transformation. Our investment in automation helped us build diverse data set, enforcing governance and compliance has become much more easier. Thank you. Amazing what Experian is doing with the Curate data pattern and soon to come using Extend. So when you think about the capabilities that you use to create data products, one of the things that we are launching right here at reInvent is Amazon SageMaker. And Amazon SageMaker is going to make a huge difference to the curate data pattern because it's going to simplify the processing of data into data products. It's going to bring together data analytics and AI into one workbench so you can do everything from model development to training to data processing to SQL analytics, assisted by Q. This is going to be a game changer for you to build data products. It gives you unified access to your data, whether it's stored in an ASRI data lake or a Redshift data warehouse or it's a federated data source, and you have governance built in to help you with anything that you are doing from an enterprise auditing perspective. Now, once you do have data products, you can do amazing things. United Airlines Data Hub lets their data scientists and analysts self-serve on data requests, and it's helping drive a data-oriented culture into their organization. Those clean data products are also what you want to use with RAG for AI inference-based applications. The PGA Gulf Tour does it this way. They use Bedrock to retrieve and integrate both structured data, like scoring or stats, and unstructured data, like commentary or social media, to help fans closely follow players through real-time updates, betting, multilingual commentary, all powered by clean, curated data products. Now, many companies find great success with curated data products. Now, many companies find great success with curated data products. They publish these clean data sets into an internal or an external marketplace, and they let other parts of their business leverage them. And other companies like Experience that you just heard are working to take their product strategy a step further with an API layer. So, rather than directly accessing the curated data sets, the API acts as an intermediary. It helps you control and govern the use of underlying data, and it can also be used as a form of standardization, to standardize on the formats of data ingestion and retrieval, and that means your application owners all over your company aren't building and owning their own pipelines. They are just using your data API. Now, making this transition to an API-driven model is a heavy lift, but it can provide a ton of value back to your business. You have that centralized control over data governance, security, and access policies. You can monitor usage patterns. You can apply rate limiting. You can enforce compliance for your data regulations with a data API. And once you build that data API layer, you are taking ownership of the quality, the recency and the availability of your business's data and you have to make sure that you have the talent and the resources to do so. But it also means that if you are rolling out a data API, you can take advantage of all the innovations that you're hearing right now on AI and benefit every user of your API. For example, as part of building your data API, your engineers can use LLMs to create summaries to enrich metadata, to find examples of personally identifiable information, or build an agentic workflow that standardizes data into a common parquet scheme or other data processing tasks. And you know the easiest way to do this is with a bedrock agent. This is the next stage of evolution for data processing. AI agents leverage foundation models now, like LLMs, to reason about and take action on goals you set out for them in these agentic workflows. They utilize a variety of tools that connect to different data sources like knowledge bases and APIs, and they are incredibly predominant now in the evolving architectures for data stewards. Moody's is a leading financial provider of analysis tools, and it is working on a proof of concept. It uses Amazon Bedrock's multi-agent orchestration, which Matt announces in preview earlier, and they have a workflow that involves a cohort of specialized data agents that collaborate to synthesize data from multiple data sources, including the product APIs and customers-owned data sources. These Bedrock agents work with different APIs, like the data APIs for Orbis, which is one of their data providers, and it gives both data and context around company financials, corporate structures, patents, and more. So again, this is the next evolution of where data is going for data processing, and what that means is that you can take advantage of that. You can put it behind your data API, and the rest of your organization can leverage the investments that you've already made. Now, to tell us more about how to build a next-generation data API, I'd like to invite Ravi Langathan, EVP of the Salesforce Data Cloud to this stage. Ravi, welcome. Thank you, Milan. Thank you, Milan, for the wonderful introduction. At Salesforce, we have always been thinking about our customer journeys, and we have really embraced the whole shift towards data-driven applications. That's really why Fisher and Paykel, as an example, is able to connect the IoT signals from their refrigerator to the next field service agent. It's the same is true when you really think about a major airline like Air India merging with Vistara and bringing customers together. SiriusXM, as an example, knows when to reach me at the right moment with the right offer to move me forward when it comes to subscription re-enablement. This has been a foundational shift for us. That's one of the reasons why we are seeing amazing adoption. Just this past quarter, we had more than 2.3 quadrillion records that we processed. More important than that is the fact that we had 1.1 trillion activations. What are these activations? Just think about it. This is the touch point that our customers have had with their customers in interesting ways to really lift their business forward. Now, we did not come about this in vacuum. We really lean into our customers to try to understand what did they do. At the heart of it, we all understand as data practitioners, enterprise data is trapped and siloed in applications, thousands of them. In fact, in Salesforce ourselves, we have tons of them. There is, of course, the combination of lakes and warehouses that people have created, creating a chasm between where the information is versus where the action needs to be with the customers. The integration complexity is paramount to understand here through the lens of the different patterns. Some were doing it for AI, some were doing it for BI use cases, some were just building it for decisioning, so on and so forth. And everybody always thinks about it as a data problem at salesforce we had a fundamentally different learning as we dive deep into it first it was about metadata it's also about the semantic data it's about the security and last but not least how do you really action all of this somewhere so that's why we fundamentally reimagined the entire salesforce stack on Salesforce platform from the ground up with every piece of information that's relevant for the enterprise. Whether it is inside Salesforce or outside Salesforce, we wanted to activate all of that information. We wanted to create an inversion from data to insights which gets orchestrated into applications and now across to the agents. With AgentForce now you're able to bring together humans, agents, data and CRM all in the same flow. Let's see how this all comes together in action. So here I'm role-playing. I'm Ken, a bank customer and I have my credit cards. Now I'm able to have a nice conversation here to bring together all of my activities from my credit card details and transactions from a warehouse. Equally, I'm also able to understand my APRs and the fees. When was the last time you guys remember where it is that you signed up for? This is unstructured data in S3 that is coming seamlessly in context. Now, there is also interesting nuances around agents, that is guardrails. Not all agents are created equal, and security is paramount. Here, as Ken is going to go about and asking for a credit line increase, it's super important to understand the role of humans. Seamlessly, because the data is the same, the metadata is the same, the context is the same, we are now going to be able to transfer Ken to a human agent in the form of Alex. With this part of the Salesforce service console, Alex is able to see everything that has just happened in context just so far. She's able to re-eason over what is going on. Normally she would have swivel shared over ten applications to answer. Now, thanks to her own employee facing agent, she's have swivel-chaired over 10 applications to answer. Now, thanks to her own employee-facing agent, she's not only able to see the basic information about the requests and what Ken has in his view, she's able to see much more deeper. With the Mule API, she's able to see what's the credit score. She's able to see much more deeper activities that he has done, including the credit card activities of his spouse. Now, organizations also have a lot of policies that are documents that are sitting everywhere. This unstructured content needs to come to life in this flow where she's able to understand the policies of credit scores with how much limits are allowed. This is foundationally important to understand how this all comes together in an agentic reasoning way. Now she's asking a loaded question to say, what should I do with Ken? It's bringing the structured data from CRM, the structured data from the warehouse, the unstructured data from the S3. It's also bringing all of the APIs to bear to say what should be the next steps. But it's super easy with Salesforce, thanks to our flow, where you can now automate all of that behind the scenes to go and really manifest this new change for credit line increase. That is the power of humans, data, AI, CRM, all coming together in this agentic ecosystem. Now, you're all technologists. You might be wondering, how is this all coming together? First and foremost, our fundamental philosophy is open and extensible. What do we really mean by open and extensible? First, every piece of Salesforce information is now available in data cloud in 60 seconds or less in the form of a parquet with an iceberg. Now, we didn't stop there because we really needed to understand the semantics. As I said before, the source objects have to be serialized down. As an example, we know which specific folder in S3 has the order and which one has the order details. The semantic information is piped down, which is allowing all our applications to seamlessly access the information with the right model. As Milan alluded, this is the data API of Salesforce. We call it data as a service. That's now ubiquitous across our ecosystem for all our applications, sales, service marketing, everything to consume. That's why we were able to unlock the demo you just saw before. The second aspect of it is this also allowed us to reimagine the entirety of the enterprise data. Thanks to partnerships with Redshift as example, we are now able to take any piece of information in Redshift, any table, and any schema, and make that seamlessly available with zero copy and in the form of both the metadata and the data. So you can use it in Flow, Apex, or the agent itself. Equally, anything that is there in the Salesforce ecosystem can now be manifesting itself in Redshift for the next query execution for your analytics workload. This has been a foundational shift for us to embrace the entirety of the enterprise. Now it's great, we got all the data, what's next? We really need to think about how we think about processing that information. As data practitioners, we always think about data quality and governance in the sense of how do we structure our pipelines. But now more important than ever is how are you going to harmonize? There's going to be tons of other data, tons of customer information from all of these applications. So thanks to the Salesforce information model, we have 700 plus objects right in there, which is going to get you started in your journey of how you think about harmonization of your schemas. Once you harmonize the schema, there's still the nuance around customers, accounts, and entities that you need to do with regard to identity resolution before you can take action so that you know the right Ken and the right Ravi that you're working with. When we looked at unstructured information, we don't see it as any different. It's equally important for us to really give structure to the unstructured. That's one of the reasons why, in addition to the classic rag pattern underneath that you would expect that we would have, we would also be able to seamlessly marry that with the structured information. Now, as we move forward, it's super important to think about governance. It's great that we got all the data from all over the enterprise. It's great that we have harmonized it. What are we going to do? It's really important to really embrace and build a new system of governance. What we had to do was connect the RBAC system that you typically have in enterprise applications with a new generative AI-led policy system. This system is automatically scanning all your data, all your metadata, to really propose what should be the right classification, what should be the right policies, and how these policies should manifest themselves in the context of these different agents. The demonstration you saw where the employee agent had much more broader security primitives than the consumer agent is a reflection of the system. We were able to do both object level, field level, record level, and masking all in the demonstration as you saw before. This is foundational as we all embark on this agentic journey. Now, in many ways, we have blurred the lines by breaking open all the strapped data and all the silos, and data fluidity is the norm, not data center of gravity as we see it. Once you have that, it's super important to think about harmonization and unification. Then comes the whole notion around governance, tagging, classification, and security. But more important than that is how are you going to activate it? How are you going to use it everywhere in your organization? Whether it is for transactional use case in sales or service, or for analytical use case in Tableau, or for a prediction use case with the builder that you might have, with a prediction model or a propensity score, or the agentic use cases that we are looking at across the board. So thank you. Amazing, huh? That is a data API in action in Salesforce right now. Now, the great thing about these three patterns of aggregate, curate, and extend is that you can choose to deploy one of them across your whole organization, or you can apply different data patterns to different business units in your organization. It depends on what they need. The starting place in the cornerstone is always going to be Amazon S3, but if you think that the fraud analytics organization might always want access to raw data, they can use the aggregate pattern. And if you think that extend is a better approach for users of highly regulated data, you can give extend to your marketing team to use as their data pattern. Now, because you are building on AWS, you know we are constantly innovating, so that it is easier for you to use any of these patterns in your business. And that this week means S3 tables, S3 metadata, the next generation of Amazon SageMaker, Bedrock agent, multi-agent capabilities, and they're all there to help you apply the data pattern that you need. With AWS, it is easy to build data-driven decision making into the DNA of your company. You have choice, you have flexibility in how you apply your data patterns on AWS, and that is critical to how you evolve and you accelerate your business with data. Thank you very much, everyone, and have fun here at reInvent.
AWS re:Invent 2024 - Modern data patterns for modern data strategies (INV201)
CTOs and CDOs must adapt their data strategies to manage both the explosive growth of data and its mission-critical usage to drive application experiences that span personalized user experiences, petabyte-scale business analytics, and real-time AI inference in applications in every industry. To keep pace and optimize for the business, leaders must balance governance with flexibility and scale by separating data growth from data usage using emerging patterns like data semantic layering. In this talk, discover how AWS customers manage data growth and usage at exabyte scale, and explore AWS capabilities to evolve your data strategy for this rapidly changing landscape.

Learn more about AWS events: https://go.aws/3kss9CP
 
Subscribe: 
More AWS videos: http://bit.ly/2O3zS75 
More AWS events videos: http://bit.ly/316g9t4

ABOUT AWS
Amazon Web Services (AWS) hosts events, both online and in-person, bringing the cloud computing community together to connect, collaborate, and learn from AWS experts. AWS is the world’s most comprehensive and broadly adopted cloud platform, offering over 200 fully featured services from data centers globally. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—are using AWS to lower costs, become more agile, and innovate faster.

#AWSreInvent  #AWSreInvent2024 #AWSEvents