nosql-transaction problem
54049216
Transaction management across data stores
Companies have evolved from a world where they only had SQL databases to a world where they use different kinds of data stores, such as keyvalue data stores, documentoriented data stores and graph databases. The reason why they have started to introduce this diversity of persistency models is because different NoSQL technologies bring different data models with associated query languages and/or APIs. However, they are confronted now with a problem in which they have the data scattered across different data stores. This problem lies in that when a business action requires to update the data, the data reside in different data stores, and they are subject to inconsistencies in the event of failure and/or concurrent access. These inconsistencies appear due to the lack of transactional consistency that was guaranteed in traditional SQL databases but is not guaranteed either within the NoSQL data stores or across data stores and databases. CoherentPaaS comes to remedy this need. CoherentPaaS provides an ultrascalable transactional management layer that can be integrated with any data store with multi versioning capabilities. The layer has been integrated with six different data stores, three NoSQL data stores and three SQLlike databases. In this paper, we describe this generic ultrascalable transactional management layer and focus on its API and how it can be integrated in different ways with different data stores and databases.
Int. J. High Perform. Comput. Netw.
Ricardo Jimenez-Peris Marta Patino-Martinez Ivan Brondino Valerio Vianello
212657006
CONVERGENCE FROM SQL NoSQL to NewSQL
Management of database includes software data management developing tools such as oldsql and now trending nosql and next generation data language newsql. As the database had been revolutionised from 40 years starting from oldSql to nosql and now to the upcoming newsql, oldsql includes features like relational model to handle data entry app and dimensional model to access data warehouse apps but it does not have features like object model, document model and graph model which is advantage of nosql whose examples are db2, hadoop, mongodb etc. The sql only use to deal with transaction problem, and cannot deal for big velocity applications does not have capacity of storing bulk data and update today's requirement of data, sql is accepted as the only standard rdbms language. We are in the middle of a database revolution that is "nosql" notable as "nosequel" It is open source distributed in nature as well as it is having high performance in a linear way that is horizontally scalable ,fast ,information retrieval database and is portable. But it can't solve large transaction problem. So to overcome the problem of both oldsql and nosql new sql is coming up with solution combination of both. We also investigate that which database is suitable for what job? choosing the right sql database is important according to users need to notice that the architecture of real world systems is not only driven by performance requirements, but has to completly include many other standard trademark requirements for selecting proper database. Such as national intelligence, cyber security, education system, marketing, business and medical informatics. Companies such as Google, Amazonetc are analyzing large volumes of data for business analysis and decisions, impacting existing and future technology all of them have different requirements, so through this paper we would conclude which query language would be better for individual need.
E Vijayan Sourabh Tolani Prasenjeet Bhadra
46175291
A Proposal of Transaction Processing Method for MongoDB
At present, to deal with a large amount of variety data in the database, various NoSQL databases have been proposed and put to practical use. However, since most of them support the transaction processing only on the single data, there is the problem that the plural data cannot be updated in a lump with maintaining the ACID properties. To solve this problem, in this paper, we propose a method to process plural data as a single transaction for MongoDB, which is a kind of document oriented NoSQL database. Concretely, each data has both of the before and after update fields, and the state of the transaction is managed. Then, in the case of before the commit, the former data is queried; after the commit, the latter data is queried. By this method, we show that the plural data can be updated as a transaction with the specified isolation level.
KES
Tsukasa Kudou M Ishino Kenji Saotome Nobuhiro Kataoka
202739471
MaharajaMacFS A scale out HDFS compatible file system built over the FlexLog
The Hadoop Distributed File System (HDFS) [8] is limited in the scalability of both transaction throughput and number of files by its single node metadata service. The current version of HDFS relies on a federated structure to avoid this bottleneck, but it is usually extremely inconvenient and often times inefficient to have files divided across name spaces. This paper presents a solution to that problem in the form of a scale out HDFS compatible file system called MaharajaMacFS. MaharajaMacFS consists of a fat client file system library operating over the FlexLog, a novel distributed metadata store from the Yale Distributed Systems Group, and Apache Cassandra [6] the open source distributed NoSQL database. Some inconclusive preliminary results from tests on Amazon Web Services Elastic Cloud Compute (AWS EC2) are discussed.
Soham Sankaran
11069368
Multi partition Distributed Transactions over Cassandra Like Database with Tunable Contention Control
The amounts of data being processed today are enormous and they require specialized systems to store them, access them and do computations. Therefore, a number of NoSql databases and big data platforms were built to address this problem. They usually lack transaction support which features atomicity, consistency, isolation, durability and at the same time they are distributed, scalable, and fault tolerant. In this paper, we present a novel transaction processing framework based on Cassandra storage model. It uses Paxos protocol to provide atomicity and consistency of transactions and Cassandra specific read and write paths improvements to provide read committed isolation level and durability. Unlike built in Light Weight Transactions (LWT) support in Cassandra, our algorithm can span multiple data partitions and provides tunable contention control. We verified correctness and efficiency both theoretically and by executing tests over different workloads. The results presented in this paper prove the usability and robustness of the designed system.
BDAS
Marek Lewandowski Jacek Lewandowski
9117087
A Blockchain Based on Gossip a Position Paper
A blockchain is an append only sequence of blocks of arbitrary data. The two most popular approaches to blockchains are permissionless blockchains based on Proof of Work (PoW) and permissioned blockchains based on Byzantine consensus or Byzantine Fault Tolerance (BFT) The first is based on competitions between anonymous participants solving cryptopuzzles, while the latter is a cooperative approach based on mutual trust between participants. Major problems with PoW approaches include that the energy per transaction is enormous, the transaction rate is very low, and the latency is very high. A major problem with BFT is that membership is closed. Various other approaches to blockchains have been proposed to address these problems. In this paper we propose yet another approach, based on gossip (aka epidemiological protocols) [1] Gossip is an approach to agreement in so called eventually consistent systems, and is particularly popular in NoSQL Key Value Stores such as Dynamo, Cassandra, and so on. In a basic gossip protocol, there is a fixed group of participants. Periodically, each participant randomly selects a peer and exchanges state. This state is reconciled in a way so that all nonfaulty participants eventually converge to the same state. It can be shown that this approach is efficient, converging in O(logN) gossip rounds where N is the number of participants, even in the face of participants failures and message loss [1] Moreover, gossip protocols are amenable to open and dynamic membership whereby the membership itself is gossiped along with other state [4] To make the application of gossip to blockchains more concrete, we propose, for simplicity, a setting similar to the Bitcoin blockchain. Blocks are 1 MByte. We assume participants have access to a good source of time, such as a GPS clock. We divide time into 10 minute epochs. Each hour would have six such epochs starting at the top of the hour. Each epoch is further subdivided into 10 second gossip rounds, allowing for 60 rounds of gossip. The intention is for the participants in an epoch to use gossip to agree upon a block. Note that the transaction rate would be the same as for the Bitcoin blockchain here we only try to address energy consumption. Each participant, in the background, collects transactions from clients, and at the top of an epoch fills a block with new transactions. The block also contains a history hash: a hash of the content of the previous block on the chain, as determined by the participant. The participant then starts gossiping its block and learning about the blocks of other participants. Before we go into further specifics about how the participants may end up agreeing on a particular block, we point out that gossip is prone to Byzantine attack. We will introduce and address various attacks as we go. The first question is how participants would end up agreeing on a block. The straightforward
Robbert van Renesse
15140220
Optimization and application in medical big document data of Apriori algorithm based on MapReduce
For the challenges of redundancy, multi dimension, complex and heterogeneous in medical documents ,and to solve the problem that the value hidden in the huge amounts of medical document data can't be mined, this paper proposed a system called MSPM based on NOSQL and MapReduce. Through storage of key value pairs,complex and heterogeneous datas are summed up in a unified and convenient format of transaction for Apriori. Then Apriori is executed in parallel through MapReduce.At last,with the strategies of generating all the candidate sets non recursively and constraint count for candidate sets of interest, it can solve the problem of low speed, high overhead and poor effectiveness for Apriori algorithm in the application of medical data. Testing results has shown the algorithm of optimization is available.
2016 International Conference on Computer Communication and Informatics (ICCCI)
Li Wei Liu Guangming Shao Yachao Li Junlong Zuo You
199086437
An improved NSGA2 to solve a bi objective optimization problem of multi state electronic transaction network
Abstract An electronic transaction network (ETN) plays a very important role in communications among trading partners. Transmission reliability is of concern to system supervisors. This study adopts a binary state physical line allocation strategy, minimizing cost and maximizing transmission reliability for an ETN with a known network structure, in which the ETN is represented by arcs and nodes. The strategy is to allocate adequate binary state physical lines to arcs. Particularly, the physical lines allocated to the same arc could be in correlated failure owing to maintenance. That is, the ETN can be modeled as a multi state flow network with correlated failures for reliability evaluation. For solving this bi objective optimization problem, an improved fast non dominated sorting genetic algorithm II (iNSGA2) integrating the NSGA2 and k means algorithm, is proposed, where the k means is utilized to expand the search space of the NSGA2. A set of non dominated solutions is found by the iNSGA2, and then, the technique for order preference by similarity to an ideal solution (TOPSIS) is adopted to determine the compromise alternative from the set. By solving this problem, the system supervisor can improve ETN stability at a reasonable expense without changing the network structure.
Reliab. Eng. Syst. Saf.
Cheng-Ta Yeh
19669755
Transaction processing in consistency aware user's applications deployed on NoSQL databases
NoSQL databases are capable of storing and processing big data which is characterized by various properties such as volume, variety and velocity. Such databases are used in a variety of user applications that need large volume of data which is highly available and efficiently accessible. But they do not enforce or require strong data consistency nor do they support transactions. This paper investigates into the transaction processing in consistency aware applications hosted on MongoDB and Riak which are two representatives of Document and Key Value NoSQL databases, respectively. It develops new transaction schemes in order to provide NoSQL databases with transactional facilities as well as to analyze the effects of transactions on data consistency and efficiency in user's applications. The proposed schemes are evaluated using the YCSB T benchmark which is based on Yahoo! Cloud Services Benchmark (YCSB) Experimental results show that using the proposed schemes, strong consistency can be achieved in MongoDB and Riak without severely affecting their efficiency. We also conduct experiments in order to analyse the level of consistencyof MongoDB and Riak transactional systems.
Human centric Computing and Information Sciences
Maria Teresa Gonzalez-Aparicio Adewole Ogunyadeka Muhammad Younas Javier Tuya Ruben Casado
52390326
Optimal Bail Out Dividend Problem with Transaction Cost and Capital Injection Constraint
We consider the optimal bail out dividend problem with fixed transaction cost for a Levy risk model with a constraint on the expected present value of injected capital. To solve this problem, we first consider the optimal bail out dividend problem with transaction cost and capital injection and show the optimality of reflected c 1 c 2 policies. We then find the optimal Lagrange multiplier, by showing that in the dual Lagrangian problem the complementary slackness conditions are met. Finally, we present some numerical examples to support our results.
Mauricio Junca Harold A Moreno-Franco Jos'e Luis P'erez