15348655
Exploring benefits of non-linear time compression
In comparison to text, audio-video content is much more challenging to browse. Time-compression has been suggested as a key technology that can support browsing-time compression speeds up the playback of audio-video content without causing the pitch to change. Simple forms of time-compression are starting to appear in commercial streaming-media products from Microsoft and Real Networks.In this paper we explore the potential benefits of more recent and advanced types of time compression, called non-linear time compression. The most advanced of these algorithms exploit fine-grain structure of human speech (e.g., phonemes) to differentially speedup segments of speech, so that the overall speedup can be higher. In this paper we explore what are the actual gains achieved by end-users from these advanced algorithms. Our results indicate that the gains are actually quite small in common cases and come with significant system complexity and some audio/video synchronization issues.
5542322
Rendering with concentric mosaics
An image based system and process for rendering novel views of a real or synthesized 3D scene based on a series of concentric mosaics depicting the scene. In one embodiment, each concentric mosaic represents a collection of consecutive slit images of the surrounding 3D scene taken from a different viewpoint tangent to a circle on a plane within the scene. Novel views from viewpoints within circular regions of the aforementioned circle plane defined by the concentric mosaics are rendered using these concentric mosaics. Specifically, a slit image can be identified by a ray originating at its viewpoint on the circle plane and extending toward the longitudinal midline of the slit image. Each of the rays associated with the slit images needed to construct a novel view will either coincide with one of the rays associated with a previously captured slit image, or it will pass between two of the concentric circles on the circle plane. If it coincides, then the previously captured slit image associated with the coinciding ray can be used directly to construct part of the novel view. If the ray passes between two of the concentric circles of the plane, then the needed slit image is interpolated using the two previously captured slit images associated with the rays originating from the adjacent concentric circles that are parallel to the non-coinciding ray. If the objects in the 3D scene are close to the camera, depth correction is applied to reduce image distortion for pixels located above and below the circle plane. In another embodiment, a single camera is used to capture a sequence of images. Each image includes image data that has a ray direction associated therewith. To render an image at a novel viewpoint, multiple ray directions from the novel viewpoint are chosen. Image data is combined from the sequence of images by selecting image data that has a ray direction substantially aligning with the ray direction from the novel viewpoint.
5953229
Building an intelligent camera management system
Given rapid improvements in storage devices, network infrastructure and streaming-media technologies, a large number of corporations and universities are recording lectures and making them available online for anytime, anywhere access. However, producing high-quality lecture videos is still labor intensive and expensive. Fortunately, recent technology advances are making it feasible to build automated camera management systems to capture lectures. In this paper we report our design of such a system, including system configuration, audio-visual tracking techniques, software architecture, and user study. Motivated by different roles in a professional video production team, we have developed a multi-cinematographer single-director camera management system. The system performs lecturer tracking, audience tracking, and video editing all fully automatically, and offers quality close to that of human-operated systems.
14821600
Comparing presentation summaries: slides vs. reading vs. listening
As more audio and video technical presentations go online, it becomes imperative to give users effective summarization and skimming tools so that they can find the presentation they want and browse through it quickly. In a previous study, we reported three automated methods for generating audio-video summaries and a user evaluation of those methods. An open question remained about how well various text/image only techniques will compare to the audio-video summarizations. This study attempts to fill that gap.
This paper reports a user study that compares four possible ways of allowing a user to skim a presentation: 1) PowerPoint slides used by the speaker during the presentation, 2) the text transcript created by professional transcribers from the presentation, 3) the transcript with important points highlighted by the speaker, and 4) a audio-video summary created by the speaker. Results show that although some text-only conditions can match the audio-video summary, users have a marginal preference for audio-video (ANOVA f=3.067, p=0.087). Furthermore, different styles of slide-authoring (e.g., detailed vs. big-points only) can have a big impact on their effectiveness as summaries, raising a dilemma for some speakers in authoring for on-demand previewing versus that for live audiences.
10647458
Auto-summarization of audio-video presentations
As streaming audio-video technology becomes widespread, there is a dramatic increase in the amount of multimedia content available on the net. Users face a new challenge: How to examine large amounts of multimedia content quickly. One technique that can enable quick overview of multimedia is video summaries; that is, a shorter version assembled by picking important segments from the original.
We evaluate three techniques for automatic creation of summaries for online audio-video presentations. These techniques exploit information in the audio signal (e.g., pitch and pause information), knowledge of slide transition points in the presentation, and information about access patterns of previous users. We report a user study that compares automatically generated summaries that are 20%-25% the length of full presentations to author generated summaries. Users learn from the computer-generated summaries, although less than from authors' summaries. They initially find computer-generated summaries less coherent, but quickly grow accustomed to them.
2560002
Automating lecture capture and broadcast: technology and videography
Abstract.Our goal is to help automate the capture and broadcast of lectures to online audiences. Such systems have two interrelated design components. The technology component includes hardware and associated software. The aesthetic component comprises the rules and idioms that human videographers follow to make a video visually engaging; these rules guide hardware placement and software algorithms. We report the design of a complete system that captures and broadcasts lectures automatically and report on a user study and a detailed set of video-production rules obtained from professional videographers who critiqued the system, which has been deployed in our organization for 2 years. We describe how the system can be generalized to a variety of lecture room environments differing in room size and number of cameras. We also discuss gaps between what professional videographers do and what is technologically feasible today.
11874775
Microarchitecture Level Interconnect Modeling Considering Layout Optimization
— In this paper, we study microarchitecture-level interconnect modeling for power and performance. Considering structural interconnects, layer assignment, and concurrent repeater and Flip-Flop (FF) insertion, we develop cycle-accurate microarchitecture-level power and throughput simulation and obtain an accurate modeling of interconnects at the early design stage. Experiment show that the simulation reduces over-estimation by up to 2.24X compared to the conventional power estimation based on purely stochastic interconnects and fixed switching factor. Furthermore, we optimize throughput with consideration of FF insertion for interconnects and floorplanning optimization. We show that throughput is not always higher for an increased clock frequency, and there exists an optimal clock frequency to maximize throughput for a given microarchitecture and given floorplan. In addition, floorplan optimized for IPC (instructions per cycle)-critical interconnects has little on the total interconnect length but improves throughput by 23.49%. As FF insertion becomes necessary to achieve the clock frequency specified by ITRS, we conclude that the traditional design flow optimizing IPC and clock frequency separately is no longer valid, and coupled microarchitecture and layout optimization may improve both power efficiency and throughput.
14096392
Mapping DAG-based applications to multiclusters with background workload
Before an application modelled as a directed acyclic graph (DAG) is executed on a heterogeneous system, a DAG mapping policy is often enacted. After mapping, the tasks (in the DAG-based application) to be executed at each computational resource are determined. The tasks are then sent to the corresponding resources, where they are orchestrated in the pre-designed pattern to complete the work. Most DAG mapping policies in the literature assume that each computational resource is a processing node of a single processor, i.e. the tasks mapped to a resource are to be run in sequence. Our studies demonstrate that if the resource is actually a cluster with multiple processing nodes, this assumption will cause a mis-perception in the tasks' execution time and execution order. This will disturb the pre-designed cooperation among tasks so that the expected performance cannot be achieved. In this paper, a DAG mapping algorithm is presented for multicluster architectures. Each constituent cluster in the multicluster is shared by background workload (from other users) and has its own independent local scheduler. The multicluster DAG mapping policy is based on theoretical analysis and its performance is evaluated through extensive experimental studies. The results show that compared with conventional DAG mapping policies, the new scheme that we present can significantly improve the scheduling performance of a DAG-based application in terms of the schedule length.
928092
The impact of predictive inaccuracies on execution scheduling
This paper investigates the underlying impact of predictive inaccuracies on execution scheduling, with particular reference to execution time predictions. This study is conducted from two perspectives: from that of job selection and from that of resource allocation, both of which are fundamental components in execution scheduling. A new performance metric, termed the degree of misperception, is introduced to express the probability that the predicted execution times of jobs display different ordering characteristics from their real execution times due to inaccurate prediction. Specific formulae are developed to calculate the degree of misperception in both job selection and resource allocation scenarios. The parameters which influence the degree of misperception are also extensively investigated. The results presented in this paper are of significant benefit to scheduling approaches that take into account predictive data; the results are also of importance to the application of these scheduling techniques to real-world high-performance systems.
17606652
Solving the Chan-Vese Model by a Multiphase Level Set Algorithm Based on the Topological Derivative
In this work, we specifically solve the Chan-Vese active contour model by multiphase level set methods. We first develop a fast algorithm based on calculating the variational energy of the Chan-Vese model without the length term. We check whether the energy decreases or not when we move a point to another segmented region. Then we draw a connection between this algorithm and the topological derivative, a concept emerged from the shape optimization field. Furthermore, to include the length term of the Chan-Vese model, we apply a preprocessing step on the image by using nonlinear diffusion. We show numerical experiments to demonstrate the efficiency and the robustness of our algorithm.
16360889
Optimising static workload allocation in multiclusters
Summary form only given. Workload allocation and job dispatching are two fundamental components in static job scheduling for distributed systems. We address the static workload allocation techniques for two types of job stream in multicluster systems, namely, nonreal-time job streams and soft-real-time job streams, which request different qualities of service. Two workload allocation strategies (called ORT and OMR) are developed by establishing and numerically solving two optimisation equation sets. The ORT strategy achieves the optimised mean response time for the nonreal-time job stream; while the OMR strategy can gain the optimised mean miss rate for the soft-real-time job stream over multiple clusters (these strategies can also be applied in a single cluster system). The effectiveness of both strategies is demonstrated through theoretical analysis. The proposed workload allocation schemes are combined with two job dispatching strategies (weighted random and weighted round-robin) to generate new static job scheduling algorithms for multicluster environments. These algorithms are evaluated through extensive experimental studies and the results show that compared with static approaches without the optimisation techniques, the proposed workload allocation schemes can significantly improve the performance of static job scheduling in multiclusters, in terms of both the mean response time (for the nonreal-time jobs) and the mean miss rate (for soft-real-time jobs).