Activity Driven Session Management in 3DTI
[2012 - present] Related Publications: IEEE Multimedia Magazine 2013, MMSYS'2013
Current interactive tele-presence systems are designed and optimized for one particular type of cyber-physical activity such as conversation, video chat, or gaming. However, with the emerging new 3D tele-immersive (TI) systems, such as our own TI system, called TEEVE (TEle-immersion for EVErybody), we observe that the same TI system platform is being used for very different activities. In this project, we classify the TI activities with respect to their physical characteristics, qualitatively analyze the cyber side of TI activities, and argue that one needs to consider very different performance profiles of the same TI system platform in order to achieve high quality of experience (QoE) for different cyber-physical TI activities. Finally, we develop an evolutionary optimization approach in the global control plane for optimizing the content distribution considering the activity-driven QoS requirements, participants' demands and underlying network infrastructure.
4D TeleCast: Large Scale Dissemination of Multi-view Multi-stream 3D Contents
[2011 - present] Related Publications: ICDCS'2012
In addition to interactive participants in the 3DTI environments, there is a large number of passive viewers that (a) watch the interactive activities in 3DTI shared environments without participating in the activities, and (b) select views of the activities at run time. These participants introduce following challenges: (1) require a large number of concurrent 3D views as well as viewers, (2) require multi-stream and multi-view dependency preservation at the viewers for high quality 3D experiences despite of Internet dynamics, and (3) introduce dynamic viewer behavior such as view changes and large-scale simultaneous viewer arrivals or departures. To address these challenges, we develop 4D TeleCast, a novel 3D content distribution and session management framework providing the functionality of multi-view selections. We divide the problem space into two: (1) overlay construction problem that aims to minimize the cost of distribution of 3D contents, and maximize the number of concurrent viewers, and (2) view synchronization problem that aims to preserve the multi-stream dependencies in a view.
In-Network Filtering of Click Spam using OpenFlow
[2012-present] Related Publications: (under preparation)
(Summer Internship with Microsoft Research)
One of our on-going efforts is to provide application layer support from the lower layers using OpenFlow technology. We use a data mining technique to learn the temporal and spatial patterns of the harmful clients generating spam traffics (e.g., click spams), to design a filtering logic at the network ingress-point based on OpenFlow. The learned patterns help to overcome two shortcomings in OpenFlow-based switches: 1) low layer-3 memory due to the use of expensive TCAM, and 2) rule insertion latency due to the separation of the control plane as well as the necessity of rule optimization via recompilation on every insertion.
FlowDiff: Flow By Flow Data Center Diagnosis
[2011-present] Related Publications: ICDCS'2013 (under submission)
(Summer Internship with NEC Laboratories America, Inc.)
Large-scale data centers comprise thousands of servers running complex applications, whose interactions with each other and with the infrastructure is not always known. Small workload changes or simple operator tasks may have unpredictable results and lead to expensive failures and performance degradations that cost millions of dollars. We propose FlowDiff , a scalable, efficient, automated, and non-intrusive tool for detecting anomalous data center behavior. FlowDiff collects information from all entities involved in the operation of a data center - applications, operators, and infrastructure - and continually builds behavioral models for the operation. By comparing current models with pre-computed, known-to-be-stable models, FlowDiff is able to detect many operational problems, ranging from host and network failures to unauthorized access. We use OpenFlow-based passive logging scheme at the controller in our implementation to ensure unobtrusiveness.
DIAMOND: Bandwidth Efficient Online Correlation Based Anomaly Detection in Distributed Interactive Environments
[2010-2011] Related Publications: ISM'2010
Distributed Interactive Multimedia Environments (DIMEs) show important dependency constraints between application and underlying system components over time. For example, the video frame rate and the underlying bandwidth usage have a strong performance dependency. Performance dependencies must also be considered among distributed components. These dependencies over a time-span form correlation relationships. Violations of such correlation relationships represent collective anomalies. Users and most specifically DIME application developers face problems of finding (detecting), localizing such anomalies, and adapting against them in real-time. Current practices are to collect joint application-system metadata characterizing behaviors of application and system components while a DIME session is running, and then analyze them offline. Our goal is to provide a framework, called DIAMOND, that allows for real-time and unobtrusive collection and organization of joint application system metadata in order to assist in finding such correlation violations in the system. DIAMOND works in four steps: (a) real-time metadata collection, (b) metadata processing to allow efficient computation of correlation constraints, (c) metadata distribution for efficient clustering of distributed metadata, and (d) anomaly detection, localization, and evolution monitoring based on violations of correlation relationships.
RESCUE: Scaling Data Plane Logging in Large Scale Network
[2009-2010] Related Publications: MILCOM'2011
Understanding and troubleshooting wide area networks(such as military backbone networks and ISP networks are challenging tasks due to their large, distributed, and highly dynamic nature. Building a system that can record and replay fine-grained behaviors of such networks would simplify this problem by allowing operators to recreate the sequence and precise ordering of events (e.g., packet-level forwarding decisions, route changes, failures) taking place in their networks. However, doing this at large scales seems intractable due to the vast amount of information that would need to be logged. In this paper, we propose a scalable and reliable framework to monitorfine-grained data-plane behavior within a large network. We give a feasible architecture for a distributed logging facility, a tree-based data structure for log compression and show how this logged information helps network operators to detect and debug anomalous behavior of the network. Experimental results obtained through trace-driven simulations and Click software router experiments show that our design is lightweight in terms of processing time, memory requirement and control overhead, yet still achieves over 99% precision in capturing network events.
QoS (Quality of Service) impact on QoE (Quality of Experience) in 3D Tele-immersive Interactive Environments
[2009-2011] Related Publications: MM'2009, ISM'2010, MM'2011
Despite the intensity of user-involved interaction in DIMEs, the existing evaluation frameworks remain very much system-centric. As a step toward the human-centric paradigm, we present a conceptual framework of Quality of Experience (QoE) in DIMEs, to model, measure, and understand user experience and its relationship with the traditional Quality of Service (QoS) metrics. A multi-displinary approach is taken to build up the framework based on the theoretical results from various fields including psychology, cognitive sciences, sociology, and information technology. We introduce a mapping methodology to quantify the correlations between QoS and QoE. We also conduct a psychophysical study that measures the perceptual thresholds of a new factor called Color-plus-Depth Level-of-Detail peculiar to polygon-based 3D tele-immersive video. The results demonstrate the existence of Just Noticeable Degradation and Just Unacceptable Degradation thresholds on the factor. Our experimental results show that the adaptation scheme can reduce resource usage while considerably enhancing theoverall perceived visual quality.
CloudInsight: Learning Based Debugging and Troubleshooting for Virtual Cloud Data Center
[2010-2011] Related Publications: SRDS'2011
(Summer Internship with NEC Laboratories America, Inc.)
The cloud-computing paradigm also introduces some new challenges in system management. Due to the decoupled ownership of applications and infrastructures, if a problem occurs, there is no visibility for either cloud users or providers to understand the whole context of the incident and solve it quickly. To this end, we develop a software solution, CloudInsight, to provide some visibility through the middle virtualization layer for both cloud users and providers to address their problems quickly. CloudInsight automatically tracks each VM instance’s configuration status and maintains their life-cycle configuration records in a configuration management database (CMDB). When a user reports a problem, the algorithms automatically analyze CMDB to probabilistically determine the root cause and invoke a recovery process by interacting with the cloud user.
Q-Tree: Multi-Attribute Based Query Solution for Large Scale Distributed Interactive Environments
[2008-2010] Related Publications: ICDCS'2009
A multimedia session control plane is frequently in need of monitoring various components, and resources to allow real-time queries about them. Though there exist several distributed query and aggregation systems, the clustered structure of 3DTI interactive frameworks and their time-sensitive nature and application requirements pose different challenges on this distributed search. Multi-attribute composite range queries are one of the key features in this class. To design such a query engine with minimum traffic overhead, low service latency, and with static and dynamic nature of large datasets, we propos Q-Tree, a multi-attribute based range query framework. In order to serve efficient queries, Q-Tree builds a single topology-aware tree overlay by connecting the participating 3DTI sites, and assigns dynamic range intervals on each overlay node (i.e., site) in a hierarchical manner. The range structure is self-adaptable based on the load condition of the networks and the participating sites. Finally, the monitoring metadata from each site is distributed to the tree nodes based on the range structure, which provides a provision for the fast and efficient range queries. Q-Tree is robust against failure and it is self-configurable at session run-time.
[2012 - present] Related Publications: IEEE Multimedia Magazine 2013, MMSYS'2013
Current interactive tele-presence systems are designed and optimized for one particular type of cyber-physical activity such as conversation, video chat, or gaming. However, with the emerging new 3D tele-immersive (TI) systems, such as our own TI system, called TEEVE (TEle-immersion for EVErybody), we observe that the same TI system platform is being used for very different activities. In this project, we classify the TI activities with respect to their physical characteristics, qualitatively analyze the cyber side of TI activities, and argue that one needs to consider very different performance profiles of the same TI system platform in order to achieve high quality of experience (QoE) for different cyber-physical TI activities. Finally, we develop an evolutionary optimization approach in the global control plane for optimizing the content distribution considering the activity-driven QoS requirements, participants' demands and underlying network infrastructure.
4D TeleCast: Large Scale Dissemination of Multi-view Multi-stream 3D Contents
[2011 - present] Related Publications: ICDCS'2012
In addition to interactive participants in the 3DTI environments, there is a large number of passive viewers that (a) watch the interactive activities in 3DTI shared environments without participating in the activities, and (b) select views of the activities at run time. These participants introduce following challenges: (1) require a large number of concurrent 3D views as well as viewers, (2) require multi-stream and multi-view dependency preservation at the viewers for high quality 3D experiences despite of Internet dynamics, and (3) introduce dynamic viewer behavior such as view changes and large-scale simultaneous viewer arrivals or departures. To address these challenges, we develop 4D TeleCast, a novel 3D content distribution and session management framework providing the functionality of multi-view selections. We divide the problem space into two: (1) overlay construction problem that aims to minimize the cost of distribution of 3D contents, and maximize the number of concurrent viewers, and (2) view synchronization problem that aims to preserve the multi-stream dependencies in a view.
In-Network Filtering of Click Spam using OpenFlow
[2012-present] Related Publications: (under preparation)
(Summer Internship with Microsoft Research)
One of our on-going efforts is to provide application layer support from the lower layers using OpenFlow technology. We use a data mining technique to learn the temporal and spatial patterns of the harmful clients generating spam traffics (e.g., click spams), to design a filtering logic at the network ingress-point based on OpenFlow. The learned patterns help to overcome two shortcomings in OpenFlow-based switches: 1) low layer-3 memory due to the use of expensive TCAM, and 2) rule insertion latency due to the separation of the control plane as well as the necessity of rule optimization via recompilation on every insertion.
FlowDiff: Flow By Flow Data Center Diagnosis
[2011-present] Related Publications: ICDCS'2013 (under submission)
(Summer Internship with NEC Laboratories America, Inc.)
Large-scale data centers comprise thousands of servers running complex applications, whose interactions with each other and with the infrastructure is not always known. Small workload changes or simple operator tasks may have unpredictable results and lead to expensive failures and performance degradations that cost millions of dollars. We propose FlowDiff , a scalable, efficient, automated, and non-intrusive tool for detecting anomalous data center behavior. FlowDiff collects information from all entities involved in the operation of a data center - applications, operators, and infrastructure - and continually builds behavioral models for the operation. By comparing current models with pre-computed, known-to-be-stable models, FlowDiff is able to detect many operational problems, ranging from host and network failures to unauthorized access. We use OpenFlow-based passive logging scheme at the controller in our implementation to ensure unobtrusiveness.
DIAMOND: Bandwidth Efficient Online Correlation Based Anomaly Detection in Distributed Interactive Environments
[2010-2011] Related Publications: ISM'2010
Distributed Interactive Multimedia Environments (DIMEs) show important dependency constraints between application and underlying system components over time. For example, the video frame rate and the underlying bandwidth usage have a strong performance dependency. Performance dependencies must also be considered among distributed components. These dependencies over a time-span form correlation relationships. Violations of such correlation relationships represent collective anomalies. Users and most specifically DIME application developers face problems of finding (detecting), localizing such anomalies, and adapting against them in real-time. Current practices are to collect joint application-system metadata characterizing behaviors of application and system components while a DIME session is running, and then analyze them offline. Our goal is to provide a framework, called DIAMOND, that allows for real-time and unobtrusive collection and organization of joint application system metadata in order to assist in finding such correlation violations in the system. DIAMOND works in four steps: (a) real-time metadata collection, (b) metadata processing to allow efficient computation of correlation constraints, (c) metadata distribution for efficient clustering of distributed metadata, and (d) anomaly detection, localization, and evolution monitoring based on violations of correlation relationships.
RESCUE: Scaling Data Plane Logging in Large Scale Network
[2009-2010] Related Publications: MILCOM'2011
Understanding and troubleshooting wide area networks(such as military backbone networks and ISP networks are challenging tasks due to their large, distributed, and highly dynamic nature. Building a system that can record and replay fine-grained behaviors of such networks would simplify this problem by allowing operators to recreate the sequence and precise ordering of events (e.g., packet-level forwarding decisions, route changes, failures) taking place in their networks. However, doing this at large scales seems intractable due to the vast amount of information that would need to be logged. In this paper, we propose a scalable and reliable framework to monitorfine-grained data-plane behavior within a large network. We give a feasible architecture for a distributed logging facility, a tree-based data structure for log compression and show how this logged information helps network operators to detect and debug anomalous behavior of the network. Experimental results obtained through trace-driven simulations and Click software router experiments show that our design is lightweight in terms of processing time, memory requirement and control overhead, yet still achieves over 99% precision in capturing network events.
QoS (Quality of Service) impact on QoE (Quality of Experience) in 3D Tele-immersive Interactive Environments
[2009-2011] Related Publications: MM'2009, ISM'2010, MM'2011
Despite the intensity of user-involved interaction in DIMEs, the existing evaluation frameworks remain very much system-centric. As a step toward the human-centric paradigm, we present a conceptual framework of Quality of Experience (QoE) in DIMEs, to model, measure, and understand user experience and its relationship with the traditional Quality of Service (QoS) metrics. A multi-displinary approach is taken to build up the framework based on the theoretical results from various fields including psychology, cognitive sciences, sociology, and information technology. We introduce a mapping methodology to quantify the correlations between QoS and QoE. We also conduct a psychophysical study that measures the perceptual thresholds of a new factor called Color-plus-Depth Level-of-Detail peculiar to polygon-based 3D tele-immersive video. The results demonstrate the existence of Just Noticeable Degradation and Just Unacceptable Degradation thresholds on the factor. Our experimental results show that the adaptation scheme can reduce resource usage while considerably enhancing theoverall perceived visual quality.
CloudInsight: Learning Based Debugging and Troubleshooting for Virtual Cloud Data Center
[2010-2011] Related Publications: SRDS'2011
(Summer Internship with NEC Laboratories America, Inc.)
The cloud-computing paradigm also introduces some new challenges in system management. Due to the decoupled ownership of applications and infrastructures, if a problem occurs, there is no visibility for either cloud users or providers to understand the whole context of the incident and solve it quickly. To this end, we develop a software solution, CloudInsight, to provide some visibility through the middle virtualization layer for both cloud users and providers to address their problems quickly. CloudInsight automatically tracks each VM instance’s configuration status and maintains their life-cycle configuration records in a configuration management database (CMDB). When a user reports a problem, the algorithms automatically analyze CMDB to probabilistically determine the root cause and invoke a recovery process by interacting with the cloud user.
Q-Tree: Multi-Attribute Based Query Solution for Large Scale Distributed Interactive Environments
[2008-2010] Related Publications: ICDCS'2009
A multimedia session control plane is frequently in need of monitoring various components, and resources to allow real-time queries about them. Though there exist several distributed query and aggregation systems, the clustered structure of 3DTI interactive frameworks and their time-sensitive nature and application requirements pose different challenges on this distributed search. Multi-attribute composite range queries are one of the key features in this class. To design such a query engine with minimum traffic overhead, low service latency, and with static and dynamic nature of large datasets, we propos Q-Tree, a multi-attribute based range query framework. In order to serve efficient queries, Q-Tree builds a single topology-aware tree overlay by connecting the participating 3DTI sites, and assigns dynamic range intervals on each overlay node (i.e., site) in a hierarchical manner. The range structure is self-adaptable based on the load condition of the networks and the participating sites. Finally, the monitoring metadata from each site is distributed to the tree nodes based on the range structure, which provides a provision for the fast and efficient range queries. Q-Tree is robust against failure and it is self-configurable at session run-time.
