Apache Spark 4.3 发布，已登陆 Databricks Runtime 19
Spark 4.3 来了，实时流处理、历史数据管道、小数据性能都升级了，还改成季度发版，做数据的可以试试 Runtime 19。
Databricks 推出 Apache Spark 4.3，随 Databricks Runtime 19 一起提供。新版本在实时处理上引入流式聚合和 transformWithState，让 Real-Time Mode 支持有状态计算。数据处理方面新增 SCD Type 2 Auto CDC，可自动保留历史变更记录。小数据场景通过 Project Feather 提升迭代速度，Spark Connect 也扩展了可接入的应用范围。此后 Spark 将改为每季度一次的功能发布节奏。
Introducing Apache Spark® 4.3, now available in Databricks Runtime 19!
The release brings major new capabilities across real-time processing, history-preserving pipelines, small-data performance, Spark Connect, and SQL.
What’s new:
- Bring state to Real-Time Mode with streaming aggregation and transformWithState - Preserve history with SCD Type 2 Auto CDC - Iterate faster on small data with Project Feather - Reach Spark from more applications with expanded Spark Connect capabilities
It also begins a new quarterly feature-release cadence, bringing completed improvements to users sooner.
Get started: https://t.co/FF22P4x06r
