Connecting end-points to âservicesâ with processing embedded in the network ... Intra-cloud service. Traditional unicast .... CDN/web server updating. â Gaming.
SAHARA and OASIS: Storage Services in the New Composed Internet
Randy H. Katz Computer Science Division Electrical Engineering and Computer Science Department University of California, Berkeley Berkeley, CA 94720-1776
New Opportunity • New things you can do inside the network • Connecting end-points to “services” with processing embedded in the network fabric • “agents” not protocols, executing inside the network • Location-aware, data format aware • Controlled violation of layering necessary! • Distributed architecture aware of network topology • No single technical architecture likely to dominate: think overlays, system of systems
Composition Scenario: Universal In-box
– Message type (phone, email, fax) – Access network (data, telephone, pager) – Terminal device (computer, phone, pager, fax) – User preferences & rules – Message translation & storage
Separate end device and network from end-to-end communications service: indirection via composition of translators with access
Overlays: Creating New Interdomain Services • Deploy new services above the routing layer – E.g., interdomain multicast management and peering – E.g., alternative connectivity for performance, resilience Isolated Intra-cloud service Administrative
Administrative domain
Planet-Lab domain
Admin domain
Admin domain
Admin domain
Traditional unicast peering
“Elevator” Statement • New mechanisms, techniques for end-toend services w/ desirable, predictable, enforceable properties spanning potentially distrusting service providers • Architecture for service composition and inter-operation across separate administrative domains, supporting peering and brokering, and diverse business, valueexchange, access-control models
Layered Reference Model for Service Composition
Middleware Services End-to-End Network With Desirable Properties
Enhanced Paths (Inter-domain)
Enhanced Links (Intra-domain)
IP Network
Connectivity Plane
Service Composition
Applications Services
Application Plane
End-User Applications
Overlay Network “Links”
Routing as a Composed Service • Routing as a Reachability “Service” – Paths between composed service instances--“links” within an overlay network – Multi-provider environment, no centralized control
• Desirable Enhanced Properties – Context Awareness: discovery/exploitation of net relationships – Agility: converge quickly in response to global changes to retain good reachability “performance” – Trust: verify believability of routing advertisements – Performance: “guaranteed” bandwidth and latency – Reliability: detect service composition path failures quickly to enable fast recomposition to maintain E2E service – Scalability and Interoperability: Adapt protocols via processing between admin domains
Research Strategy Convergence Issues Shared Bottlenecks New Control Plane
Existing Interdomain Routing
Overlay Routing
New Gen Routing Infrastructure
QoS and Performance Failure Detection and Recovery Measurement-Based Algorithms Validity and Security
New Directions
• OASIS: Overlays and Active Services for Internetworked Storage – Wide-Area Network-attached Storage Services, particularly for disaster recovery
• Composed Services and Resource Management – Authorization Control Across Administrative Domains (Suzuki, Moreno + students) – Radio Resource Allocation Across Service Providers (Matsunaga + students)
Storage as a Network Service • Large capacity for rich content, video, scientific datasets, visualizations, etc. • Centralized repository for cost-effective management • Distributed “caches” for performance and resilience to loss and failure • Storage virtualization: network elements transparently map large monolithic “logical” storage into distributed physical storage • Experimental environment: Petabyte store, campusscale distribution, CITRIS-scale distribution, Planet-Lab
Emergence of Programmable Network Elements • First Gen Network Appliances, Directors – Storage Virtualizers, Intrusion Detectors, Traffic Shapers, Server Load Balancers, MIE accountants • Packet input • Protocol-specific packet characterization • Action invocation – Linux Firewall code--masquerade, redirect, accept, deny, reject, return, queue
• Packet transformation • Packet redirection to output
• Next Gen: Third Party Programmable beyond rules • Generalized “virtual machine” model for this class of devices
Generic PNE Architecture Buffers Buffers
CP CP CP CP
Classification Processor
Tag Mem
CP CP CP AP
Rules & Programs
Interconnection Fabric
Output Ports
Input Ports
Buffers
Action Processor
OASIS: Wide-Area Storage Services PNE
PNE
LAN
Sensor Net
WAN
Smart Smart Storage Smart Storage Smart Storage Smart Storage “Smart” Storage Storage
Programs
Smart Smart Storage Smart Storage Smart Storage Smart Storage “Smart” Storage Storage
PNE
LAN PNE
Programs Programs
Control Plane
Programs
Sensor Net
Storage Wide-Area Networking
Metro Area
• SAN technology works well @ metro-area scale • Challenges for extension to wide-area storage – Comparable performance as small area storage applications
Gather
Synchronous
• Digital animation editing • Large dataset visualization
Asynchronous • •
•
N-to-1 disk copies (KaZaa) Recreate dataset from multiple sources/disks (scientific experiment) Restore backup
Scatter
Synchronous
• State dissemination – CDN/web server updating – Gaming
• Updating mapping tables
Asynchronous • •
Disaster-recovery application Experimental data unloading
The SAHARA Project • • • • • •
Service Architecture for Heterogeneous Access, Resources, and Applications
Recent Publications • •
• • •
•
J. Shih, R. H. Katz, “Evaluating the Tradeoffs of Congestion Pricing for Voice Calls,” 2002 International Symposium on Performance Evaluation of Computer and Telecommunication Systems (SPECTS 2002), San Diego, California, (July 2002). B. Raman, R. H. Katz, “Emulation-based Evaluation of an Architecture for WideArea Service Composition,” 2002 International Symposium on Performance Evaluation of Computer and Telecommunication Systems (SPECTS 2002), San Diego, California, (July 2002). Z. Mao, R. Govindan, G. Varghese, R. H. Katz, “Route Flap Damping Exacerbates Internet Routing Convergence.” ACM SIGCOMM Conference, Pittsburgh, PA, (August 2002). Y. Chen, R. H. Katz, J. D. Kubiatowicz, “SCAN: a Dynamic Scalable and Efficient Content Distribution Network,” International Conference on Pervasive Computing (Pervasive 2002), Zurich, Switzerland, (August 2002). B. Raman, S. Agarwal, Y. Chen, M. Caesar, W. Cui, P. Johansson, K. Lai, T. Lavian, S. Machiraju, Z. Mao, G. Porter, T. Roscoe, M. Seshadri, J. Shih, K. Sklower, L. Subramanian, T. Suzuki, S. Zhuang, A. D. Joseph, R. H. Katz, I. Stoica, “The SAHARA Model for Service Composition Across Multiple Providers,” International Conference on Pervasive Computing (Pervasive 2002), Zurich, Switzerland, (August 2002), Invited Paper. Z. Mao, R. H. Katz, “A Framework for Universal Service Access using Device Ensembles,” CRA Grace Murray Hopper Celebration of Women in Computer Science Conference, Vancouver, BC, (October 2002). Mao selected as Hopper Young Investigator (Best Student Paper).
Recent Publications • • • • •
•
L. Subramanian, I. Stoica, H. Balakrishnan, R. H. Katz, “OverQoS: Offering QoS using Overlays,” First Workshop on Hot Topics in Networking (HotNets02), Princeton, NJ, (October 2002). Y. Chen, L. Qui, R. H. Katz, “On the Clustering of Web Content for Efficient Replication,” 10th IEEE Conference on Network Protocols (ICNP 2002), Paris, France, (November 2002). W. Cui, I. Stoica, R. H. Katz, “Backup Path Allocation Based on a Correlated Link Failure Probability Model in Overlay Networks,” 10th IEEE Conference on Network Protocols (ICNP 2002), Paris. France, (November 2002). S. Agarwal, C. N. Chuah, R. H. Katz, “OPCA: Robust Interdomain Policy Routing and Traffic Control,” Proceedings OpenArch 2003, San Francisco, CA, (April 2003). S. Zhuang, K. Lai, I. Stoica, R. H. Katz, S. Shenker, “Host Mobility using an Internet Indirection Infrastructure,” First International Conference on Mobile Systems, Applications, and Services (ACM/USENIX Mobisys), San Francisco, CA, (May 2003). B. Raman, R. H. Katz, “Load Balancing and Stability Issues in Algorithms for Service Composition,” IEEE Infocomm Conference, San Francisco, California, (July 2003).
SAHARA and OASIS Randy H. Katz
Thank You!
