Query-Driven Learning for Next Generation Predictive Modeling and Analytics
Savva, Fotis
As data-size is increasing exponentially, new paradigm shifts have to emerge allowing fast exploitation of data by every- body. Large-scale predictive analytics is restricted to wealthy organizations as small-scale enterprises (SMEs) struggle to compete and are inundated by the sheer monetary cost of either procuring data infrastructures or analyzing datasets over the Cloud. The aim of this work is to study mechanisms which can democratize analytics, in the sense of making them affordable, while at the same time ensuring high efficiency, scalability, and accuracy. The crux of this proposal lies in developing query-driven solutions that can be used off the Cloud thus minimizing costs. Our query-driven approach will learn and adapt on-the-fly machine learning models, based solely on query-answer interactions, which can be used for answering analytical queries. In this abstract we describe the methodology followed for the implementation and evaluation of the system designed.
 Today I'll be presenting my research on query-driven learning for next generation analytics. So first of all, let's start with an observation. An exponential increase in data translates to an exponential increase in costs. So we have both monetary costs and computational costs. We have computational costs because queries are executed over large datasets, with their answers being returned within minutes, hours, or days, depending on how big your data is. There is research proving the existence of an interactivity constraint, sent somewhere around 500 milliseconds to two seconds, saying that any answer returned to the user over that limit might negatively affect the productivity of data analysts. To examine the monetary costs, we've investigated Google's BigQuery pricing, and we've seen that as dataset sizes increase from 1TB to 50TB, organizations executing 10 to 100 queries daily have to spend multiple thousands of dollars to keep on leveraging these kinds of technologies. So we need to find solutions that are able to address both these computational costs and the monetary costs. One way to address the computational cost is using approximate query processing, or AQP in short. So AQP provides approximate answers at a fraction of the time needed for the computation of the actual answer. The way this is performed is that AQP engines construct multiple samples to which the queries are executed against instead of the queries being executed over the actual data. So the answers being returned are approximate answers, and any query that cannot be answered by the sampling-based AQP engine is forwarded to the actual data store and then the actual answer is retrieved. However, these kinds of solutions make use of samples, so they still require huge amounts of storage. So for 1TB of data and with just 1% of sampling ratio, we still need to store 10GB of data. They also have an inherent trade-off between accuracy and sampling ratio, meaning that if we want more and more accurate answers, we will have to sample more and more of the dataset. This can end up breaking the interactivity constraint, as our queries have to be executed over larger and larger datasets. They also make use of the same infrastructure as the data store engine, so the samples have to be stored in the same engine as the actual data. So a way we came up with to address these problems, these shortcomings, is to leverage past queries and train machine learning models to predict the answers of any new queries instead of creating samples. So the queries are now executed against functions, and the answers are predicted using those machine learning models, those functions. Using this approach, we make no use of data as we require no samples. We can get accurate estimations of the answers with just an average relative error of 3%. Our answers can be returned in an efficient manner, as the average response time is just 0.1 milliseconds. And our solution is extremely lightweight, as what's being stored in this case are the machine learning models and not any of the data required. And our solution is also data store agnostic, meaning that we do not care where the queries are executed against, whether they are executed against a no SQL database or an SQL database, as what's being used are the queries and their answers. So a way to implement this methodology is the following. Imagine an analytic query being expressed with an SQL statement. This SQL statement is initially parsed, and the filtering parameters of this SQL statement are used to construct a feature vector. So in the end, we have a number of feature vectors with the corresponding answers, which translates to queries and their corresponding answers that we've retrieved by actually executing those queries. We then use those feature vectors and partition them using a clustering algorithm. This clustering algorithm produces subgroups to which within each subgroup, the queries belonging to each subgroup, are more similar to each other than in other subgroups. We then train a regression model over each one of those subgroups, which is more appropriate to answer those kind of queries. Then when a new query comes in, its feature vector is extracted, and it gets mapped to the closest cluster. We find what's the most appropriate model to use, and we estimate its answer in an efficient manner. So a way to view the complete analytics stack now is the following. Imagine that we have query-driven learning, the sampling-based approaches, and data stores. We can use all of them side by side, and the user gets to decide which one she wants to execute the query against. If she wants more accuracy, she can move from the left to the right to get more and more accurate results, and if she needs more speed, she can execute the queries from right to the left. A useful analogy to this is the following. You can imagine this as the cache, RAM, and disk memories of a normal PC. Again, if you want more and more accurate results, you move from the left to the right, and if you want more speed, you move from the disk to the cache memory. We've already addressed the computational cost, however, we have the monetary cost to address as well. So normally, we would use BigQuery or, let's say, Redshift to perform all of these queries. The on-demand pricing for those engines is $5 per terabyte. Instead, using our technology, we can deploy this as what's now called function as a service. Those function as a service are a much cheaper alternative, and we would only get charged for 40 cents per one million function calls. Method of vocation. So this translates to one million potential queries that we could execute using our query-driven learning mechanism than going to the BigQuery or Redshift. So you can see that we can save a lot of money by executing a portion of the queries initially intended for the BigQuery clusters or Redshift clusters using our solution. So in short, we have managed to address both the computational cost, as our solution requires no extra storage and can provide efficient estimations to queries in just 0.1 milliseconds. And we have also addressed the monetary cost using a deployment with the function as a service provided by most cloud providers. That would be all for my presentation. Thank you very much.
 Today I'll be presenting my research on query-driven learning for next generation analytics.  So first of all, let's start with an observation.  An exponential increase in data translates to an exponential increase in costs.  So we have both monetary costs and computational costs.  We have computational costs because queries are executed over large datasets,  with their answers being returned within minutes, hours, or days,  depending on how big your data is.  There is research proving the existence of an interactivity constraint,  sent somewhere around 500 milliseconds to two seconds,  saying that any answer returned to the user over that limit  might negatively affect the productivity of data analysts.  To examine the monetary costs, we've investigated Google's BigQuery pricing,  and we've seen that as dataset sizes increase from 1TB to 50TB,  organizations executing 10 to 100 queries daily have to spend multiple thousands of dollars  to keep on leveraging these kinds of technologies.  So we need to find solutions that are able to address both these computational costs and the monetary costs.  One way to address the computational cost is using approximate query processing, or AQP in short.  So AQP provides approximate answers at a fraction of the time needed for the computation of the actual answer.  The way this is performed is that AQP engines construct multiple samples  to which the queries are executed against instead of the queries being executed over the actual data.  So the answers being returned are approximate answers,  and any query that cannot be answered by the sampling-based AQP engine  is forwarded to the actual data store and then the actual answer is retrieved.  However, these kinds of solutions make use of samples, so they still require huge amounts of storage.  So for 1TB of data and with just 1% of sampling ratio, we still need to store 10GB of data.  They also have an inherent trade-off between accuracy and sampling ratio,  meaning that if we want more and more accurate answers, we will have to sample more and more of the dataset.  This can end up breaking the interactivity constraint, as our queries have to be executed over larger and larger datasets.  They also make use of the same infrastructure as the data store engine,  so the samples have to be stored in the same engine as the actual data.  So a way we came up with to address these problems, these shortcomings,  is to leverage past queries and train machine learning models to predict the answers of any new queries instead of creating samples.  So the queries are now executed against functions, and the answers are predicted using those machine learning models, those functions.  Using this approach, we make no use of data as we require no samples.  We can get accurate estimations of the answers with just an average relative error of 3%.  Our answers can be returned in an efficient manner, as the average response time is just 0.1 milliseconds.  And our solution is extremely lightweight, as what's being stored in this case are the machine learning models and not any of the data required.  And our solution is also data store agnostic, meaning that we do not care where the queries are executed against,  whether they are executed against a no SQL database or an SQL database,  as what's being used are the queries and their answers.  So a way to implement this methodology is the following.  Imagine an analytic query being expressed with an SQL statement.  This SQL statement is initially parsed, and the filtering parameters of this SQL statement are used to construct a feature vector.  So in the end, we have a number of feature vectors with the corresponding answers,  which translates to queries and their corresponding answers that we've retrieved by actually executing those queries.  We then use those feature vectors and partition them using a clustering algorithm.  This clustering algorithm produces subgroups to which within each subgroup,  the queries belonging to each subgroup, are more similar to each other than in other subgroups.  We then train a regression model over each one of those subgroups,  which is more appropriate to answer those kind of queries.  Then when a new query comes in, its feature vector is extracted,  and it gets mapped to the closest cluster.  We find what's the most appropriate model to use, and we estimate its answer in an efficient manner.  So a way to view the complete analytics stack now is the following.  Imagine that we have query-driven learning, the sampling-based approaches, and data stores.  We can use all of them side by side, and the user gets to decide which one she wants to execute the query against.  If she wants more accuracy, she can move from the left to the right to get more and more accurate results,  and if she needs more speed, she can execute the queries from right to the left.  A useful analogy to this is the following.  You can imagine this as the cache, RAM, and disk memories of a normal PC.  Again, if you want more and more accurate results, you move from the left to the right,  and if you want more speed, you move from the disk to the cache memory.  We've already addressed the computational cost, however, we have the monetary cost to address as well.  So normally, we would use BigQuery or, let's say, Redshift to perform all of these queries.  The on-demand pricing for those engines is $5 per terabyte.  Instead, using our technology, we can deploy this as what's now called function as a service.  Those function as a service are a much cheaper alternative,  and we would only get charged for 40 cents per one million function calls.  Method of vocation.  So this translates to one million potential queries that we could execute using our query-driven learning mechanism  than going to the BigQuery or Redshift.  So you can see that we can save a lot of money by executing a portion of the queries  initially intended for the BigQuery clusters or Redshift clusters using our solution.  So in short, we have managed to address both the computational cost,  as our solution requires no extra storage and can provide efficient estimations to queries in just 0.1 milliseconds.  And we have also addressed the monetary cost using a deployment with the function as a service  provided by most cloud providers.  That would be all for my presentation. Thank you very much.
Fotis_Savva_0_143.png Fotis_Savva_144_263.png Fotis_Savva_264_323.png Fotis_Savva_324_371.png Fotis_Savva_372_419.png Fotis_Savva_420_2207.png Fotis_Savva_2208_2507.png Fotis_Savva_2508_2975.png Fotis_Savva_2976_4859.png Fotis_Savva_4860_10259.png Fotis_Savva_10260_12611.png Fotis_Savva_12612_12707.png Fotis_Savva_12708_12841.png
Today I'll be presenting my research on query-driven learning for next generation analytics. So first of all, let's start with an observation. An exponential increase in data translates to an exponential increase in costs. So we have both monetary costs and computational costs. We have computational costs because queries are executed over large datasets, with their answers being returned within minutes, hours, or days, depending on how big your data is. There is research proving the existence of an interactivity constraint, sent somewhere around 500 milliseconds to two seconds, saying that any answer returned to the user over that limit might negatively affect the productivity of data analysts. To examine the monetary costs, we've investigated Google's BigQuery pricing, and we've seen that as dataset sizes increase from 1TB to 50TB, organizations executing 10 to 100 queries daily have to spend multiple thousands of dollars to keep on leveraging these kinds of technologies. So we need to find solutions that are able to address both these computational costs and the monetary costs. One way to address the computational cost is using approximate query processing, or AQP in short. So AQP provides approximate answers at a fraction of the time needed for the computation of the actual answer. The way this is performed is that AQP engines construct multiple samples to which the queries are executed against instead of the queries being executed over the actual data. So the answers being returned are approximate answers, and any query that cannot be answered by the sampling-based AQP engine is forwarded to the actual data store and then the actual answer is retrieved. However, these kinds of solutions make use of samples, so they still require huge amounts of storage. So for 1TB of data and with just 1% of sampling ratio, we still need to store 10GB of data. They also have an inherent trade-off between accuracy and sampling ratio, meaning that if we want more and more accurate answers, we will have to sample more and more of the dataset. This can end up breaking the interactivity constraint, as our queries have to be executed over larger and larger datasets. They also make use of the same infrastructure as the data store engine, so the samples have to be stored in the same engine as the actual data. So a way we came up with to address these problems, these shortcomings, is to leverage past queries and train machine learning models to predict the answers of any new queries instead of creating samples. So the queries are now executed against functions, and the answers are predicted using those machine learning models, those functions. Using this approach, we make no use of data as we require no samples. We can get accurate estimations of the answers with just an average relative error of 3%. Our answers can be returned in an efficient manner, as the average response time is just 0.1 milliseconds. And our solution is extremely lightweight, as what's being stored in this case are the machine learning models and not any of the data required. And our solution is also data store agnostic, meaning that we do not care where the queries are executed against, whether they are executed against a no SQL database or an SQL database, as what's being used are the queries and their answers. So a way to implement this methodology is the following. Imagine an analytic query being expressed with an SQL statement. This SQL statement is initially parsed, and the filtering parameters of this SQL statement are used to construct a feature vector. So in the end, we have a number of feature vectors with the corresponding answers, which translates to queries and their corresponding answers that we've retrieved by actually executing those queries. We then use those feature vectors and partition them using a clustering algorithm. This clustering algorithm produces subgroups to which within each subgroup, the queries belonging to each subgroup, are more similar to each other than in other subgroups. We then train a regression model over each one of those subgroups, which is more appropriate to answer those kind of queries. Then when a new query comes in, its feature vector is extracted, and it gets mapped to the closest cluster. We find what's the most appropriate model to use, and we estimate its answer in an efficient manner. So a way to view the complete analytics stack now is the following. Imagine that we have query-driven learning, the sampling-based approaches, and data stores. We can use all of them side by side, and the user gets to decide which one she wants to execute the query against. If she wants more accuracy, she can move from the left to the right to get more and more accurate results, and if she needs more speed, she can execute the queries from right to the left. A useful analogy to this is the following. You can imagine this as the cache, RAM, and disk memories of a normal PC. Again, if you want more and more accurate results, you move from the left to the right, and if you want more speed, you move from the disk to the cache memory. We've already addressed the computational cost, however, we have the monetary cost to address as well. So normally, we would use BigQuery or, let's say, Redshift to perform all of these queries. The on-demand pricing for those engines is $5 per terabyte. Instead, using our technology, we can deploy this as what's now called function as a service. Those function as a service are a much cheaper alternative, and we would only get charged for 40 cents per one million function calls. Method of vocation. So this translates to one million potential queries that we could execute using our query-driven learning mechanism than going to the BigQuery or Redshift. So you can see that we can save a lot of money by executing a portion of the queries initially intended for the BigQuery clusters or Redshift clusters using our solution. So in short, we have managed to address both the computational cost, as our solution requires no extra storage and can provide efficient estimations to queries in just 0.1 milliseconds. And we have also addressed the monetary cost using a deployment with the function as a service provided by most cloud providers. That would be all for my presentation. Thank you very much.
So we have both monetary costs and computational queries are executed over large datasets, with technologies. So we need to find solutions that approximate answers at a fraction of the time executed over the actual data. So the answers query that cannot be answered by the answers, we will have to sample more and more of stored in the same engine as the actual data. So Using this approach, we make no use of data as we relative error of 3%. Our answers can be returned in this case are the machine learning models and implement this methodology is the following. vectors with the corresponding answers, which feature vectors and partition them using a query-driven learning, the sampling-based intended for the BigQuery clusters or Redshift monetary cost using a deployment with the