Releases: GoogleCloudDataproc/spark-bigquery-connector
Releases · GoogleCloudDataproc/spark-bigquery-connector
0.39.1
- PR #1236: Fixing unshaded artifacts, added shading verification
- PR #1239: Allow GCS bucket to be supplied including a scheme. Thanks @tom-s-powell !
- Issue #1126: Fixing Kryo serialization issues
- Issue #1223: Fix gRPC status creates 'object not serializable' errors
Contributors
tom-s-powell
Assets 9
0.36.3
0.39.0
- PR #1221: Adding support for table snapshots. Thanks @tom-s-powell !
- PR #1222: Add option to request lz4 compressed ReadRowsResponse
- PR #1225: Fixing multi-release jar shading
- PR #1227: Optimizing dynamic partition overwrite for Time partitioned tables
- BigQuery API has been upgraded to version 2.40.1
- BigQuery Storage API has been upgraded to version 3.5.1
- GAX has been upgraded to version 2.48.1
- gRPC has been upgraded to version 1.64.0
Contributors
tom-s-powell
Assets 9
0.36.2
0.38.0
- PR #1205: Sending Identity token in the read API header
- Issue #1195: Support map type with complex value
- Issue #1215: Support predicate pushdown for DATETIME
- BigQuery API has been upgraded to version 2.39.0
- BigQuery Storage API has been upgraded to version 3.5.0
- GAX has been upgraded to version 2.47.0
- Arrow has been upgraded to version 16.0.0
- gRPC has been upgraded to version 1.63.0
- Netty has been upgraded to version 4.1.109.Final
0.37.0
⚠️ Starting version 0.38.0 of the connector, thespark-2.4-bigqueryversion won't be released as Spark 2.4 is
well-supported by thespark-bigquery-with-dependenciesconnectors.- PR #1156: Propagate stats for BigLake Managed tables
- PR #1181: Add caching during protobuf generation
- PR #1190: Enable connection sharing for atLeastOnce writes
- Issue #1182: Fix query check logic
- BigQuery API has been upgraded to version 2.38.1
- BigQuery Storage API has been upgraded to version 3.3.1
- GAX has been upgraded to version 2.45.0
- Arrow has been upgraded to version 15.0.1
- gRPC has been upgraded to version 1.62.2
- Netty has been upgraded to version 4.1.107.Final
- Protocol Buffers has been upgraded to version 3.25.3
0.36.1
0.36.0
- PR #1155: allow lazy materialization of query on load
- PR #1163: Added config to set the BigQuery Job timeout
- PR #1166: Fix filters by adding surrounding parenthesis. Thanks @tom-s-powell !
- PR #1171: fix read, write issues with Timestamp
- Issue #1116: BigQuery write fails with MessageSize is too large
- BigQuery API has been upgraded to version 2.36.0
- GAX has been upgraded to version 2.40.0
- gRPC has been upgraded to version 1.61.0
- Netty has been upgraded to version 4.1.106.Final
- Protocol Buffers has been upgraded to version 3.25.2
Contributors
tom-s-powell
Assets 10
0.35.1
0.35.0
- PR #1115: Added new connector,
spark-3.5-bigqueryaimed to be used in Spark 3.5. This connector implements new APIs and capabilities provided by the Spark Data Source V2 API. - PR #1117: Make read session caching duration configurable
- PR #1118: Improve read session caching key
- PR #1122: Set traceId on write
- PR #1124: Added
SparkListenerEvents for Query and Load jobs running on BigQuery - PR #1127: Fix job labeling for mixed case Dataproc job names
- PR #1136: Consider projections for biglake stats
- PR #1143: Enable async write for default stream
- BigQuery API has been upgraded to version 2.35.0
- BigQuery Storage API has been upgraded to version 2.47.0
- GAX has been upgraded to version 2.38.0
- gRPC has been upgraded to version 1.60.0
- Netty has been upgraded to version 4.1.101.Final
- Protocol Buffers has been upgraded to version 3.25.1