Stream processing with Apache Flink fundamentals, implementation, and operation of streaming applications
"Get started with Apache Flink, the open source framework that powers some of the world's largest stream processing applications. With this practical book, you'll explore the fundamental concepts of parallel stream processing and discover how this technology differs from traditional b...
| Main Authors: | , |
|---|---|
| Format: | eBook |
| Language: | English |
| Published: |
Sebastopol, CA
O'Reilly Media, Inc.
2019
|
| Edition: | First edition |
| Subjects: | |
| Online Access: | |
| Collection: | O'Reilly - Collection details see MPG.ReNa |
Table of Contents:
- Includes bibliographical references and index
- Highly Available SetupData Transfer in Flink; Credit-Based Flow Control; Task Chaining; Event-Time Processing; Timestamps; Watermarks; Watermark Propagation and Event Time; Timestamp Assignment and Watermark Generation; State Management; Operator State; Keyed State; State Backends; Scaling Stateful Operators; Checkpoints, Savepoints, and State Recovery; Consistent Checkpoints; Recovery from a Consistent Checkpoint; Flink's Checkpointing Algorithm; Performace Implications of Checkpointing; Savepoints; Summary; Chapter 4. Setting Up a Development Environment for Apache Flink; Required Software
- Chapter 2. Stream Processing FundamentalsIntroduction to Dataflow Programming; Dataflow Graphs; Data Parallelism and Task Parallelism; Data Exchange Strategies; Processing Streams in Parallel; Latency and Throughput; Operations on Data Streams; Time Semantics; What Does One Minute Mean in Stream Processing?; Processing Time; Event Time; Watermarks; Processing Time Versus Event Time; State and Consistency Models; Task Failures; Result Guarantees; Summary; Chapter 3. The Architecture of Apache Flink; System Architecture; Components of a Flink Setup; Application Deployment; Task Execution
- Explicitly Providing Type InformationDefining Keys and Referencing Fields; Field Positions; Field Expressions; Key Selectors; Implementing Functions; Function Classes; Lambda Functions; Rich Functions; Including External and Flink Dependencies; Summary; Chapter 6. Time-Based and Window Operators; Configuring Time Characteristics; Assigning Timestamps and Generating Watermarks; Watermarks, Latency, and Completeness; Process Functions; TimerService and Timers; Emitting to Side Outputs; CoProcessFunction; Window Operators; Defining Window Operators; Built-in Window Assigners
- Run and Debug Flink Applications in an IDEImport the Book's Examples in an IDE; Run Flink Applications in an IDE; Debug Flink Applications in an IDE; Bootstrap a Flink Maven Project; Summary; Chapter 5. The DataStream API (v1.7); Hello, Flink!; Set Up the Execution Environment; Read an Input Stream; Apply Transformations; Output the Result; Execute; Transformations; Basic Transformations; KeyedStream Transformations; Multistream Transformations; Distribution Transformations; Setting the Parallelism; Types; Supported Data Types; Creating Type Information for Data Types
- Cover; Copyright; Table of Contents; Preface; What You Will Learn in This Book; Conventions Used in This Book; Using Code Examples; O'Reilly Online Learning; How to Contact Us; Acknowledgments; Chapter 1. Introduction to Stateful Stream Processing; Traditional Data Infrastructures; Transactional Processing; Analytical Processing; Stateful Stream Processing; Event-Driven Applications; Data Pipelines; Streaming Analytics; The Evolution of Open Source Stream Processing; A Bit of History; A Quick Look at Flink; Running Your First Flink Application; Summary