After almost 5 months of hard work by the Flink community, the team is happy to roll out the newest release Apache Flink 1.5.0. This is a major release of the 1.x series featuring advanced capabilities along with over 750+ bugs and issues fixed.
Apache Flink is an open-source big data processing framework used for real-time analytics, stream processing and batch processing applications.This framework is capable of delivering fast, efficient, accurate, and high fault tolerance in handling huge massive streams of events. With more than 330 active contributors, Apache Flink is one of the most active stream processing projects of Apache Software Foundation.
Key new features and improvements:
Rewritten Flink’s Deployment and Process Model
- Added dynamic support for allocation and release of resources on YARN and Mesos.
- Simplified deployment on Kubernetes.
- Requests for job submission, cancellation, job status to the JobManager happen through REST.
Broadcast State
- Connects broadcasted stream such as context data, machine learning models with other streams.
- Broadcasted states can be checkpointed and restored.
- Unblocks implementation of “dynamic patterns” feature.
Improvements to Flink’s Network Stack
- Added Credit-based flow control for high throughput.
- Improved performance by lowering latencies without reduction in throughput.
Task-Local State Recovery
- Keeps copy of the application state on the local disk of each machine.
- Improved failure recovery.
Extending Join Support for SQL and Table API
- Support for joining of tables on bounded time ranges in both event-time and processing-time.
- Supports full-history matching similar to standard SQL statements.
SQL CLI Client
- Added SQL CLI client support for processing exploratory queries on data streams.
- Service added for streaming and batch SQL queries.
Various other features and improvements
- Supports OpenStack’s S3-like file system
- Improved reading and writing of JSON messages from and to connectors
- Applications rescaling improved without manual triggers
- Improved watermarks and latency measures
For the complete list of features and improvements, please review the release notes on the official Apache Flink page.
Read Next
Flink Complex Event Processing
Top 5 programming languages for crunching Big Data effectively