2316797
MMSS: multi-modal story-oriented video summarization
We propose multi-modal story-oriented video summarization (MMSS) which, unlike previous works that use fine-tuned, domain-specific heuristics, provides a domain-independent, graph-based framework. MMSS uncovers correlation between information of different modalities which gives meaningful story-oriented news video summaries. MMSS can also be applied for video retrieval, giving performance that matches the best traditional retrieval techniques (OKAPI and LSI), with no fine-tuned heuristics such as tf/idf.
1997586
Automatic multimedia cross-modal correlation discovery
Given an image (or video clip, or audio song), how do we automatically assign keywords to it? The general problem is to find correlations across the media in a collection of multimedia objects like video clips, with colors, and/or motion, and/or audio, and/or text scripts. We propose a novel, graph-based approach, "MMG", to discover such cross-modal correlations.Our "MMG" method requires no tuning, no clustering, no user-determined constants; it can be applied to any multimedia collection, as long as we have a similarity function for each medium; and it scales linearly with the database size. We report auto-captioning experiments on the "standard" Corel image database of 680 MB, where it outperforms domain specific, fine-tuned methods by up to 10 percentage points in captioning accuracy (50% relative improvement).
10479007
Segmenting Motion Capture Data into Distinct Behaviors
Much of the motion capture data used in animations, commercials, and video games is carefully segmented into distinct motions either at the time of capture or by hand after the capture session. As we move toward collecting more and longer motion sequences, however, automatic segmentation techniques will become important for processing the results in a reasonable time frame.We have found that straightforward, easy to implement segmentation techniques can be very effective for segmenting motion sequences into distinct behaviors. In this paper, we present three approaches for automatic segmentation. The first two approaches are online, meaning that the algorithm traverses the motion from beginning to end, creating the segmentation as it proceeds. The first assigns a cut when the intrinsic dimensionality of a local model of the motion suddenly increases. The second places a cut when the distribution of poses is observed to change. The third approach is a batch process and segments the sequence where consecutive frames belong to different elements of a Gaussian mixture model. We assess these three methods on fourteen motion sequences and compare the performance of the automatic methods to that of transitions selected manually.
2592435
Automatic image captioning
We examine the problem of automatic image captioning. Given a training set of captioned images, we want to discover correlations between image features and keywords, so that we can automatically find good keywords for a new image. We experiment thoroughly with multiple design alternatives on large datasets of various content styles, and our proposed methods achieve up to a 45% relative improvement on captioning accuracy over the state of the art.
6657323
"GeoPlot": spatial data mining on video libraries
Are "tornado" touchdowns related to "earthquakes"? How about to "floods", or to "hurricanes"? In Informedia [14], using a gazetteer on news video clips, we map news onto points on the globe and find correlations between sets of points. In this paper we show how to find answers to such questions, and how to look for patterns on the geo-spatial relationships of news events. The proposed tool is "GeoPlot", which is fast to compute and gives a lot of useful information which traditional text retrieval can not find.We describe our experiments on 2-year worth of video data (~ 20 Gbytes). There we found that GeoPlot can find unexpected correlations that text retrieval would never find, such as those between "earthquake" and "volcano", and "tourism" and "wine".In addition, GeoPlot provides a good visualization of a data set's characteristics. Characteristics at all scales are shown in one plot and a wealth of information is given, for example, geo-spatial clusters, characteristic scales, and intrinsic (fractal) dimensions of the events' locations.
7681917
ViVo: visual vocabulary construction for mining biomedical images
Given a large collection of medical images of several conditions and treatments, how can we succinctly describe the characteristics of each setting? For example, given a large collection of retinal images from several different experimental conditions (normal, detached, reattached, etc.), how can data mining help biologists focus on important regions in the images or on the differences between different experimental conditions? If the images were text documents, we could find the main terms and concepts for each condition by existing IR methods (e.g., tf/idf and LSI). We propose something analogous, but for the much more challenging case of an image collection: We propose to automatically develop a visual vocabulary by breaking images into n /spl times/ n tiles and deriving key tiles ("ViVos") for each image and condition. We experiment with numerous domain-independent ways of extracting features from tiles (color histograms, textures, etc.), and several ways of choosing characteristic tiles (PCA, ICA). We perform experiments on two disparate biomedical datasets. The quantitative measure of success is classification accuracy: Our "ViVos" achieve high classification accuracy (up to 83 %for a nine-class problem on feline retinal images). More importantly, qualitatively, our "ViVos" do an excellent job as "visual vocabulary terms": they have biological meaning, as corroborated by domain experts; they help spot characteristic regions of images, exactly like text vocabulary terms do for documents; and they highlight the differences between pairs of images.
171830
Inconsistencies, Negations and Changes in Ontologies
Ontology management and maintenance are considered cornerstone issues in current Semantic Web applications in which semantic integration and ontological reasoning play a fundamental role. The ability to deal with inconsistency and to accommodate change is of utmost importance in real-world applications of ontological reasoning and management, wherein the need for expressing negated assertions also arises naturally. For this purpose, precise, formal definitions of the the different types of inconsistency and negation in ontologies are required. Unfortunately, ontology languages based on Description Logics (DLs) do not provide enough expressive power to represent axiom negations. Furthermore, there is no single, well-accepted notion of inconsistency and negation in the Semantic Web community, due to the lack of a common and solid foundational framework. In this paper, we propose a general framework accounting for inconsistency, negation and change in ontologies. Different levels of negation and inconsistency in DL-based ontologies are distinguished. We demonstrate how this framework can provide a foundation for reasoning with and management of dynamic ontologies.
3648465
Optimal base-station locations in two-tiered wireless sensor networks
We consider generic two-tiered wireless sensor networks (WSNs) consisting of sensor clusters deployed around strategic locations, and base-stations (BSs) whose locations are relatively flexible. Within a sensor cluster, there are many small sensor nodes (SNs) that capture, encode, and transmit relevant information from a designated area, and there is at least one application node (AN) that receives raw data from these SNs, creates a comprehensive local-view, and forwards the composite bit-stream toward a BS. This paper focuses on the topology control process for ANs and BSs, which constitute the upper tier of two-tiered WSNs. Since heterogeneous ANs are battery-powered and energy-constrained, their node lifetime directly affects the network lifetime of WSNs. By proposing algorithmic approaches to locate BSs optimally, we can maximize the topological network lifetime of WSNs deterministically, even when the initial energy provisioning for ANs is no longer always proportional to their average bit-stream rate. The obtained optimal BS locations are under different lifetime definitions according to the mission criticality of WSNs. By studying intrinsic properties of WSNs, we establish the upper and lower bounds of maximal topological lifetime, which enable a quick assessment of energy provisioning feasibility and topology control necessity. Numerical results are given to demonstrate the efficacy and optimality of the proposed topology control approaches designed for maximizing network lifetime of WSNs.
5675255
RDFS(FA): Connecting RDF(S) and OWL DL
Semantic Web (SW) languages are supposed to be compatible with each other in a meaningful way, so as to facilitate machine understanding. Recent research, however, shows that the semantics of the standard SW annotation language RDF (as well as its ontological extension RDFS) and that of the standard SW ontology language OWL DL are not compatible with each other. This paper investigates some issues behind this incompatibility and proposes a novel modification of RDF(S) as a firm semantic foundation for many of the latest description logics-based SW ontology languages, including OWL DL. Furthermore, the bidirectional one-to-one mapping between RDFS(FA) axioms in strata 0-2 and OWL DL axioms has been established, which enables RDFS(FA)-agents and OWL DL-agents to communicate with each other more easily. As a result, the introduction of RDFS(FA) clarifies the vision of the semantic Web and solidifies RDF(S)'s proposed role as the base of the semantic Web
33668284
f-SWRL: A Fuzzy Extension of SWRL
Although the combination of OWL and Horn rules results in the creation of a highly expressive language, i.e. SWRL, there are still many occasions where this language fails to accurately represent knowledge of our world. In particular, SWRL fails at representing vague and imprecise knowledge and information. Such type of information is apparent in many applications like multimedia processing and retrieval, information fusion, etc. In this paper, we propose f-SWRL, a fuzzy extension to SWRL to include fuzzy assertions (such as ‘Mary is tall in the degree of 0.9') and fuzzy rules (such as ‘being healthy is more important than being rich to determine if one is happy').
2042793
Peer Collaboration in Wireless Ad Hoc Networks
Voluntary peer collaboration is often assumed in media access, route discovery, packet forwarding, and upper-layer protocols for wireless ad hoc networks. This assumption is seriously challenged when some peers are autonomous, selfish, or malicious in large-scale, heterogeneous networks. In this paper, based on the latest advances in identity-based cryptography, we design a lightweight and cheat-resistant micropayment scheme to stimulate and compensate collaborative peers that sacrifice their resources to relay packets for other peers. We also demonstrate that when security and collaboration measures are properly enforced, profitable collaboration is a preferable strategy for all peers.