5146790
Modeling skew in data streams
Data stream applications have made use of statistical summaries to reason about the data using nonparametric tools such as histograms, heavy hitters, and join sizes. However, relatively little attention has been paid to modeling stream data parametrically, despite the potential this approach has for mining the data. The challenges to do model fitting at streaming speeds are both technical -- how to continually find fast and reliable parameter estimates on high speed streams of skewed data using small space -- and conceptual -- how to validate the goodness-of-fit and stability of the model online.In this paper, we show how to fit hierarchical (binomial multifractal) and non-hierarchical (Pareto) power-law models on a data stream. We address the technical challenges using an approach that maintains a sketch of the data stream and fits least-squares straight lines; it yields algorithms that are fast, space-efficient, and provide approximations of parameter value estimates with a priori quality guarantees relative to those obtained offline. We address the conceptual challenge by designing fast methods for online goodness-of-fit measurements on a data stream; we adapt the statistical testing technique of examining the quantile-quantile (q-q) plot, to perform online model validation at streaming speeds.As a concrete application of our techniques, we focus on network traffic data which has been shown to exhibit skewed distributions. We complement our analytic and algorithmic results with experiments on IP traffic streams in AT&T's Gigascope®® data stream management system, to demonstrate practicality of our methods at line speeds. We measured the stability and robustness of these models over weeks of operational packet data in an IP network. In addition, we study an intrusion detection application, and demonstrate the potential of online parametric modeling.
2511721
Fractal Modeling of IP Network Traffic at Streaming Speeds
This paper describes how to fit fractal models, online, on IP traffic data streams. Our approach relies on maintaining a sketch of the data stream and fitting straight lines: it yields algorithms that are fast, space-efficient, and accurate. We implemented our methods in AT&T’s Gigascope data stream management system, to demonstrate their practicality at streaming line speeds.
16403469
PSALM: A Data Model for Pervasive Visualization
In this poster, we investigate how to achieve visualization in a pervasive visualization environment, especially on new platforms such as PDAs, cell phones, and other mobile devices that have very limited resources and usually are not considered appropriate platforms for visualization. We will use a clinic application as an example and our target application. In this system, patient data, such as CT and MRI, are stored in a central server in the hospital and physicians can access these data using a cell phone for a quick preview, a pocket PC or laptop for some initial diagnosis, and a powerful workstation for any further diagnosis.
487639
Intelligent Multi-hop Video Communications
In this paper, we study an intelligent multi-hop communications architecture for quality video delivery over networks. The key to this architecture is the insertion of relays with different levels of intelligence along the end-to-end path that coordinate with each other and the end-systems. We study traffic management and content adaptation within this architecture. Compared with the end-to-end model, the multi-hop model can achieve higher throughput without increasing the risk of overloading the network and the end-to-end delay. Experimental results show a throughput gain of 19.3% on a concatenation of two best-effort networks, and 37.8% for a concatenation of a best-effort network and a guaranteed service network. With the error withdrawal feature at the relay, abrupt quality drops are avoided. Analysis and simulations suggest that the multi-hop traffic management can gain high throughput, and the content adaptation should be sender-centric with occasional relay supports.
15778343
Smart rebinning for compression of concentric mosaics
Concentric mosaics offer a quick solution to construct a virtual copy of a real environment, and navigate in the virtual environment. However, the huge amount of data associated with concentric mosaics is a heavy burden for its application. A 3D wavelet transform-based compressor has been proposed in previous work to compress the concentric mosaics. In this paper, we greatly improve the performance of the 3D wavelet coder with a data rearrangement mechanism called “smart rebinning”. The proposed scheme first aligns the concentric mosaic image shots along the horizontal direction and then rebins the shots into multi-perspective panoramas. Smart rebinning greatly improves the cross shot correlation and enables the coder to better explore the redundancy among shots. Experimental results show that the performance of the 3D wavelet coder improves an average of 4.3dB with the use of smart rebinning. The proposed coder outperforms MPEG-2 coding of concentric mosaics by an average of 3.7dB.
13420664
Music Classification Using Significant Repeating Patterns
With the popularity of multimedia applications, a large amount of music data has been accumulated on the Internet. Automatic classification of music data becomes a critical technique for providing an efficient and effective retrieval of music data. In this paper, we propose a new approach for classifying music data based on their contents. In this approach, we focus on monophonic music features represented as rhythmic and melodic sequences. Moreover, we use repeating patterns of music data to do music classification. For each pattern discovered from a group of music data, we employ a series of measurements to estimate its usefulness for classifying this group of music data. According to the patterns contained in a music piece, we determine which class it should be assigned to. We perform a series of experiments and the results show that our approach performs on average better than the approach based on the probability distribution of contextual information in music.
3736444
Domain-specific keyphrase extraction
Document keyphrases provide semantic metadata characterizing documents and producing an overview of the content of a document. They can be used in many text-mining and knowledge management related applications. This paper describes a Keyphrase Identification Program (KIP), which extracts document keyphrases by using prior positive samples of human identified domain keyphrases to assign weights to the candidate keyphrases. The logic of our algorithm is: the more keywords a candidate keyphrase contains and the more significant these keywords are, the more likely this candidate phrase is a keyphrase. To obtain prior positive inputs, KIP first populates its glossary database using manually identified keyphrases and keywords. It then checks the composition of all noun phrases of a document, looks up the database and calculates scores for all these noun phrases. The ones having higher scores will be extracted as keyphrases.