Software architecture for big data and the cloud

Software architecture for Big Data and the Cloud is designed to be a single resource that brings together research on how software architectures can solve the challenges imposed by building big data software systems. The challenges of big data on the software architecture can relate to scale, securi...

Full description

Bibliographic Details
Other Authors: Mistrík, Ivan (Editor)
Format: eBook
Language:English
Published: Cambridge, MA Morgan Kaufmann 2017
Edition:First edition
Subjects:
Online Access:
Collection: O'Reilly - Collection details see MPG.ReNa
LEADER 16474nmm a2200637 u 4500
001 EB001945431
003 EBX01000000000000001108333
005 00000000000000.0
007 cr|||||||||||||||||||||
008 210123 ||| eng
020 |a 9780128093382 
050 4 |a QA76.758 
100 1 |a Mistrík, Ivan  |e editor 
245 0 0 |a Software architecture for big data and the cloud  |c edited by Ivan Mistrik [and 4 others] 
250 |a First edition 
260 |a Cambridge, MA  |b Morgan Kaufmann  |c 2017 
300 |a xxxviii, 432 pages 
505 0 |a Note continued: 9.4.1.Findings and Lessons Learned -- 9.4.2.Threats to Validity -- 9.5.Conclusion -- References -- ch. 10 Big Data: A Practitioners Perspective / Fiona O'Sullivan -- 10.1.Big Data Is a New Paradigm -- Differences With Traditional Data Warehouse, Pitfalls and Consideration -- 10.1.1.Differences With Traditional Data Warehouse -- 10.1.2.Pitfalls -- 10.1.3.Considerations -- 10.2.Product Considerations for Big Data -- Use of Open Source Products for Big Data, Pitfalls and Considerations -- 10.2.1.The Use of Open Source Product for Big Data -- 10.2.2.Pitfalls -- 10.2.3.Considerations -- 10.3.Use of Cloud for hosting Big Data -- Why to Use Cloud, Pitfalls and Consideration -- 10.3.1.Why to Use Cloud? -- 10.3.2.Pitfalls -- 10.3.3.Consideration -- 10.4.Big Data Implementation -- Architecture Definition, Processing Framework and Migration Pattern From Data Warehouse to Big Data -- 10.4.1.Patterns for Transitioning From Data Warehouse to Big Data 
505 0 |a Note continued: 10.5.Conclusion -- References -- pt. 3 TECHNOLOGIES -- ch. 11 A Taxonomy and Survey of Stream Processing Systems / Rajkumar Buyya -- 11.1.Introduction -- 11.2.Stream Processing Platforms: A Brief Background -- 11.2.1.Requirements of Stream Processing Platforms/Engines -- 11.2.2.Generic Model of Modern Stream Processing Platforms/Engines -- 11.3.Taxonomy -- 11.3.1.Functional Aspects -- 11.3.2.Nonfunctional Aspects -- 11.4.A Survey of Stream Processing Platforms -- 11.4.1.Data Stream Management Systems -- 11.4.2.Complex Event Processing Systems -- 11.4.3.Stream Processing Platforms/Engines -- 11.5.Comparison Study of the Stream Processing Platforms -- 11.5.1.Scalability -- 11.5.2.Messaging & Distribution -- 11.5.3.Data Processing/Stream Processors -- 11.5.4.Fault Tolerance -- 11.6.Conclusions and Future Directions -- References -- ch. 12 Architecting Cloud Services for the Digital Me in a Privacy-Aware Environment / Andreas Wortmann -- 12.1.Introduction 
505 0 |a Note continued: 17.1.Motivation and Contributions -- 17.2.Background Knowledge -- 17.2.1.Data Organization in Version Control Systems -- 17.2.2.Remote Data Integrity Checking (RDIC) -- 17.3.System and Adversarial Model -- 17.4.Auditable Version Control Systems -- 17.4.1.Definition of AVCS -- 17.4.2.An AVCS Construction -- 17.5.Discussion -- 17.6.Other RDIC Approaches for Version Control Systems -- 17.7.Evaluation -- 17.7.1.Theoretical Evaluation -- 17.7.2.Experimental Evaluation -- 17.8.Conclusion -- References -- ch. 18 Scientific Workflow Management System for Clouds / Rajkumar Buyya -- 18.1.Introduction -- 18.2.Background -- 18.3.Workflow Management Systems for Clouds -- 18.4.Cloudbus Workflow Management System -- 18.5.Cloud-Based Extensions to the Workflow Engine -- 18.6.Performance Evaluation -- 18.6.1.WRPS -- 18.6.2.Montage -- 18.6.3.Setup of Experimental Infrastructure -- 18.6.4.Montage Setup -- 18.6.5.Results -- 18.7.Summary and Conclusions -- References 
505 0 |a Note continued: 3.5.Architecture Example -- Context Management in the IoT -- 3.6.Big Data Origins and Characteristics -- 3.7.The Systems That Capture and Process Big Data -- 3.8.Operating Across Organizational Silos -- 3.9.Architecture Example -- Local Processing of Big Data -- 3.10.Architecture Example -- Creating a Multichannel View -- 3.11.Application Independent Data -- 3.12.Metadata and Governance -- 3.13.Conclusions -- 3.14.Outlook and Future Directions -- References -- ch. 4 Domain-Driven Design of Big Data Systems Based on a Reference Architecture / Ioannis N. Athanasiadis -- 4.1.Introduction -- 4.2.Domain-Driven Design Approach -- 4.3.Related Work -- 4.4.Feature Model of Big Data Systems -- 4.4.1.Data -- 4.4.2.Information Management -- 4.4.3.Interface and Visualization -- 4.4.4.Data Processing -- 4.4.5.Data Storage -- 4.4.6.Data Analysis -- 4.4.7.Feature Constraints -- 4.5.Deriving the Application Architectures and Example -- 4.5.1.Feature Modeling 
505 0 |a Note continued: 12.2.Example -- 12.3.Challenges -- 12.3.1.Service Composition -- 12.3.2.Technology Abstraction -- 12.3.3.Service and Data Integration -- 12.3.4.Trusted Use of Personal Data -- 12.4.Preliminaries -- 12.5.System-of-Systems Approach -- 12.5.1.Persistence Service -- 12.5.2.DataConversion Service -- 12.5.3.Privacy Service -- 12.5.4.LookUp Service -- 12.5.5.PersonalData Service -- 12.6.Generative Approach -- 12.7.Related Work -- 12.7.1.Service Composition -- 12.7.2.Technology Abstraction -- 12.7.3.Service and Data Integration -- 12.7.4.Trusted Use of Personal Data -- 12.8.Discussion -- 12.9.Conclusion -- References -- ch. 13 Reengineering Data-Centric Information Systems for the Cloud -- A Method and Architectural Patterns Promoting Multitenancy / Olaf Zimmermann -- 13.1.Introduction -- 13.2.Context and Problem: Multitenancy in Cloud Computing -- 13.3.Solution Overview: Reengineering Method and Process 
505 0 |a Note continued: 14.3.Data Intensive Computing -- 14.3.1.Big Compute Versus Big Data -- 14.3.2.Data Intensive Applications -- 14.3.3.Data Intensive Frameworks -- 14.3.4.MapReduce and GFS -- 14.4.Apache Hadoop -- 14.4.1.Hadoop VI -- 14.4.2.Hadoop 2.0 -- 14.5.Apache Spark -- 14.5.1.Resilient Distributed Datasets -- 14.5.2.Data Flow and Programming With Spark -- 14.5.3.Spark Processing Engines -- 14.5.4.Hadoop Ecosystem Taxonomy -- 14.6.The Role of Cloud Computing -- 14.7.The Future of Big Data Platforms -- 14.7.1.Big Data Applications -- 14.7.2.Big Data Frameworks and Hardware -- 14.7.3.Big Data on the Road to Exascale -- 14.8.Conclusion -- References -- ch. 15 A Taxonomy and Survey of Fault-Tolerant Workflow Management Systems in Cloud and Distributed Computing Environments / Rajkumar Buyya -- 15.1.Introduction -- 15.2.Background -- 15.2.1.Workflow Management Systems -- 15.2.2.Workflow Scheduling -- 15.3.Introduction to Fault-Tolerance 
505 0 |a Note continued: 4.5.2.Design Rule Modeling -- 4.5.3.Associating Design Decisions With Features -- 4.5.4.Generation of the Application Architecture and the Deployment Diagram -- 4.5.5.Deriving Big Data Architectures of Existing Systems -- 4.6.Conclusion -- References -- ch. 5 An Architectural Model-Based Approach to Quality-Aware DevOps in Cloud Applications / Ralf Reussner -- 5.1.Introduction -- 5.2.A Cloud-Based Software Application -- 5.3.Differences in Architectural Models Among Development and Operations -- 5.4.The iObserve Approach -- 5.5.Addressing the Differences in Architectural Models -- 5.5.1.The iObserve Megamodel -- 5.5.2.Descriptive and Prescriptive Architectural Models in iObserve -- 5.5.3.Static and Dynamic Content in Architectural Models -- 5.6.Applying iObserve to CoCoME -- 5.6.1.Applying the iObserve Megamodel -- 5.6.2.Applying Descriptive and Prescriptive Architectural Models -- 5.6.3.Applying Live Visualization -- 5.7.Limitations 
505 0 |a Note continued: 15.9.1.Data Management Tools -- 15.9.2.Security and Fault-Tolerance Management Tools -- 15.9.3.Cloud Development Tools -- 15.9.4.Support Systems -- 15.10.Summary -- References -- pt. 4 RESOURCE MANAGEMENT -- ch. 16 The HARNESS Platform: A Hardware- and Network-Enhanced Software System for Cloud Computing / Alexander Wolf -- 16.1.Introduction -- 16.2.Related Work -- 16.3.Overview -- 16.4.Managing Heterogeneity -- 16.4.1.Hierarchical Resource Management -- 16.4.2.Agnostic Resource Management -- 16.4.3.Ranking Allocation Requests -- 16.4.4.HARNESS API -- 16.5.Prototype Description -- 16.5.1.The Platform Layer -- 16.5.2.The Infrastructure Layer -- 16.5.3.The Virtual Execution Layer -- 16.6.Evaluation -- 16.6.1.Executing HPC Applications on the Cloud -- 16.6.2.Resource Scheduling with Network Constraints -- 16.7.Conclusion -- Project Resources -- References -- Acknowledgements -- ch. 17 Auditable Version Control Systems in Untrusted Public Clouds / Jun Dai 
505 0 |a Note continued: 5.8.Related Work -- 5.9.Conclusion -- References -- ch. 6 Bridging Ecology and Cloud: Transposing Ecological Perspective to Enable Better Cloud Autoscaling / Rami Bahsoon -- 6.1.Introduction -- 6.2.Motivation -- 6.3.Natural Ecosystem -- 6.4.Transposing Ecological Principles, Theories and Models to Cloud Ecosystem -- 6.5.Ecology-Inspired Self-Aware Pattern -- 6.6.Opportunities and Challenges -- 6.7.Related Work -- 6.8.Conclusion -- References -- Acknowledgement -- pt. 2 ANALYZING AND EVALUATING -- ch. 7 Evaluating Web PKIs / Mark Ryan -- 7.1.Introduction -- 7.2.An Overview of PKI -- 7.3.Desired Features and Security Concerns -- 7.4.Existing Proposals -- 7.4.1.Classic -- 7.4.2.Difference Observation -- 7.4.3.Scope Restriction -- 7.4.4.Certificate Management Transparency -- 7.5.Observations -- 7.5.1.Property Perspective -- 7.5.2.System Perspective -- 7.6.Conclusion -- References 
505 0 |a Machine generated contents note: ch. 1 Introduction. Software Architecture for Cloud and Big Data: An Open Quest for the Architecturally Significant Requirements / Ivan Mistrik -- 1.1.A Perspective into Software Architecture for Cloud and Big Data -- 1.2.Cloud Architecturally Significant Requirements and Their Design Implications -- 1.2.1.Dynamism and Elasticity as Cloud Architecturally Significant Requirements -- 1.2.2.Multitenancy as Cloud Architecturally Significant Requirement -- 1.2.3.Service Level Agreements (SLAs) Constraints as Cloud Architecturally Significant Requirement -- 1.2.4.Cloud Marketplaces as Architecturally Significant Requirement -- 1.2.5.Seeking Value as Cloud Architecturally Significant Requirement -- 1.3.Big Data Management as Cloud Architecturally Significant Requirement -- 1.3.1.Big Data Analytics Enabled by the Cloud and Its Architecturally Significant Requirements 
505 0 |a Note continued: ch. 8 Performance Isolation in Cloud-Based Big Data Architectures / Alp Oral -- 8.1.Introduction -- 8.2.Background -- 8.2.1.Cloud Computing -- 8.2.2.Big Data Architecture -- 8.3.Case Study and Problem Statement -- 8.3.1.Case Study -- 8.3.2.Problem Statement -- 8.4.Performance Monitoring in Cloud-Based Systems -- 8.5.Application Framework for Performance Isolation -- 8.6.Evaluation of the Framework -- 8.6.1.Evaluation Results -- 8.7.Discussion -- 8.8.Related Work -- 8.9.Conclusion -- References -- ch. 9 From Legacy to Cloud: Risks and Benefits in Software Cloud Migration / Patricia Lago -- 9.1.Introduction -- 9.2.Research Method -- 9.2.1.Pilot Study -- 9.2.2.Search Strategy -- 9.2.3.Data Extraction -- 9.2.4.Data Analysis Method -- 9.3.Results -- 9.3.1.Overview of Primary Studies and Quality Evaluation -- 9.3.2.Benefits and Risks -- 9.3.3.General Measures -- 9.3.4.Models and Frameworks for Cloud Migration -- 9.4.Discussion 
505 0 |a Note continued: 1.3.2.Architecturally Significant Requirements in Realm of Competing Big Data Technologies -- References -- pt. 1 CONCEPTS AND MODELS -- ch. 2 Hyperscalability -- The Changing Face of Software Architecture / Ian Gorton -- 2.1.Introduction -- 2.2.Hyperscalable Systems -- 2.2.1.Scalability -- 2.2.2.Scalability Limits -- 2.2.3.Scalability Costs -- 2.2.4.Hyperscalability -- 2.3.Principles of Hyperscalable Systems -- 2.3.1.Automate and Optimize to Control Costs -- 2.3.2.Simple Solutions Promote Scalability -- 2.3.3.Utilize Stateless Services -- 2.3.4.Observability is Fundamental to Success at Hyperscale -- 2.4.Related Work -- 2.5.Conclusions -- References -- ch. 3 Architecting to Deliver Value From a Big Data and Hybrid Cloud Architecture / Tim Vincent -- 3.1.Introduction -- 3.2.Supporting the Analytics Lifecycle -- 3.3.The Role of Data Lakes -- 3.4.Key Design Features That Make a Data Lake Successful 
505 0 |a Note continued: 15.3.1.Necessity for Fault-Tolerance in Distributed Systems -- 15.4.Taxonomy of Faults -- 15.5.Taxonomy of Fault-Tolerant Scheduling Algorithms -- 15.5.1.Replication -- 15.5.2.Resubmission -- 15.5.3.Checkpointing -- 15.5.4.Provenance -- 15.5.5.Rescue Workflow -- 15.5.6.User-Defined Exception Handling -- 15.5.7.Alternate Task -- 15.5.8.Failure Masking -- 15.5.9.Slack Time -- 15.5.10.Trust-Based Scheduling Algorithms -- 15.6.Modeling of Failures in Workflow Management Systems -- 15.7.Metrics Used to Quantify Fault-Tolerance -- 15.8.Survey of Workflow Management Systems and Frameworks -- 15.8.1.Askalon -- 15.8.2.Pegasus -- 15.8.3.Triana -- 15.8.4.UNICORE 6 -- 15.8.5.Kepler -- 15.8.6.Cloudbus Workflow Management System -- 15.8.7.Traverna -- 15.8.8.The e-Science Central (e-SC) -- 15.8.9.SwinDeW-C -- 15.8.10.Big Data Workflow Frameworks: MapReduce, Hadoop, and Spark -- 15.8.11.Other Workflow Management Systems -- 15.9.Tools and Support Systems 
505 0 |a Note continued: pt. 5 LOOKING AHEAD -- ch. 19 Outlook and Future Directions / Ivan Mistrik -- 19.1.New or Advanced Applications -- 19.2.Advanced Supporting Technologies -- 19.3.Architecturally Significant Requirements -- 19.4.Challenges for the Architecting Process -- 19.5.Further Reading -- References 
505 0 |a Includes bibliographical references and indexes 
505 0 |a Note continued: 13.4.Solution Detail 1: Architectural Patterns in the Method -- 13.4.1.Architectural Reengineering Steps for the Cloud (Architectural Refactoring) -- 13.4.2.Multitenancy Requirements and Patterns for Cloud Environments -- 13.4.3.The Multitenancy Capable Model -- 13.4.4.The Multitenancy Capable Controller -- 13.4.5.The Multitenancy Capable View -- 13.5.Solution Detail 2: Testing and Code Reviews -- 13.5.1.Testing for Multitenancy Defects -- 13.5.2.Code Review for Multitenancy Defects -- 13.5.3.Summary -- 13.6.Case Study (Implementation) -- 13.6.1.Multitenancy Transformation Without Patterns -- 13.6.2.Multitenancy Transformation With Patterns -- 13.6.3.Comparison -- 13.7.Discussion -- 13.8.Related Work -- 13.9.Summary and Conclusions -- Appendix 13.A Architectural Refactoring (AR) Reference -- References -- ch. 14 Exploring the Evolution of Big Data Technologies / Georgios Theodoropoulos -- 14.1.Introduction -- 14.2.Big Data in Our Daily Lives 
653 |a Big data / fast 
653 |a COMPUTERS / Software Development & Engineering / General / bisacsh 
653 |a Données volumineuses 
653 |a Software engineering / http://id.loc.gov/authorities/subjects/sh87007398 
653 |a Architecture logicielle 
653 |a Génie logiciel 
653 |a Software architecture / http://id.loc.gov/authorities/subjects/sh2004001071 
653 |a Software architecture / fast 
653 |a Infonuagique 
653 |a Cloud computing / http://id.loc.gov/authorities/subjects/sh2008004883 
653 |a Software engineering / fast 
653 |a Big data / http://id.loc.gov/authorities/subjects/sh2012003227 
653 |a Cloud computing / fast 
041 0 7 |a eng  |2 ISO 639-2 
989 |b OREILLY  |a O'Reilly 
776 |z 0128093382 
776 |z 9780128093382 
776 |z 0128054670 
776 |z 9780128054673 
856 4 0 |u https://learning.oreilly.com/library/view/~/9780128093382/?ar  |x Verlag  |3 Volltext 
082 0 |a 005.1/2 
082 0 |a 620 
520 |a Software architecture for Big Data and the Cloud is designed to be a single resource that brings together research on how software architectures can solve the challenges imposed by building big data software systems. The challenges of big data on the software architecture can relate to scale, security, integrity, performance, concurrency, parallelism, and dependability, amongst others. Big data handling requires rethinking architectural solutions to meet functional and non-functional requirements related to volume, variety and velocity. The book's editors have varied and complementary backgrounds in requirements and architecture, specifically in software architectures for cloud and big data, as well as expertise in software engineering for cloud and big data. This book brings together work across different disciplines in software engineering, including work expanded from conference tracks and workshops led by the editors