54 Pull requests merged by 19 people

• Skip BoundedTrie on Dataflow till service is have BoundedTrie

  #33921 merged Feb 10, 2025

• Add AlloyDB password to Xlang GCP and Direct Postcommit workflows.

  #33933 merged Feb 10, 2025

• Update Python Dependencies

  #33923 merged Feb 10, 2025

• Fix error in Dataflow Cost Benchmark README

  #33930 merged Feb 10, 2025

• Fix unsubscribe emails on Contact Us page

  #33926 merged Feb 10, 2025

• [AnomalyDetection] Add base classes and specifiable protocol

  #33845 merged Feb 10, 2025

• Add unassign issue feature

  #33924 merged Feb 10, 2025

• Add alloydb password secret to Python postcommit test workflow.

  #33927 merged Feb 10, 2025

• Move interactive-specific TestStreamService to interactive protos.

  #33858 merged Feb 7, 2025

• Change the period for checking if the workflow is flaky

  #33897 merged Feb 7, 2025

• Fix LoadTests Go CoGBK / GBK Flink Batch jobs

  #33898 merged Feb 7, 2025

• Reduce frequency of issue report

  #33906 merged Feb 7, 2025

• Fix Postcommit Java IO Performance tests workflow and BigQuery Stress Test

  #33915 merged Feb 7, 2025

• Expand yaml provider documentation.

  #33907 merged Feb 6, 2025

• Cherry pick Add support for bounded trie metric in legacy worker

  #33890 merged Feb 6, 2025

• Fix PostCommit Java ValidatesRunner ULR job

  #33899 merged Feb 6, 2025

• Remove unused choose_rc_commit workflow

  #33887 merged Feb 6, 2025

• Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.17.58 to 1.17.59 in /sdks

  #33895 merged Feb 6, 2025

• Build docker artifacts from self-hosted runners

  #33547 merged Feb 6, 2025

• [java] Add Optional coder

  #33019 merged Feb 6, 2025

• Bump golang.org/x/sync from 0.10.0 to 0.11.0 in /sdks

  #33896 merged Feb 6, 2025

• Bump github.com/aws/aws-sdk-go-v2/config from 1.29.5 to 1.29.6 in /sdks

  #33892 merged Feb 6, 2025

• Reduce time spent backing off dry run query

  #33888 merged Feb 6, 2025

• Add support for bounded trie metric in legacy worker

  #33474 merged Feb 6, 2025

• Cherry-pick: Add AlloyDB vector writer. (#33841)

  #33880 merged Feb 5, 2025

• Force specifying release branch

  #33884 merged Feb 5, 2025

• Add ALLOYDB_PASSWORD secret to beam_PreCommit_Python_ML.yml.

  #33882 merged Feb 5, 2025

• Cherry pick #33876 - update Go version to 1.23.6

  #33877 merged Feb 5, 2025

• Update build Go version to 1.23.6

  #33876 merged Feb 5, 2025

• Add AlloyDB vector writer.

  #33841 merged Feb 5, 2025

• Bump cloud.google.com/go/bigquery from 1.66.0 to 1.66.2 in /sdks

  #33861 merged Feb 5, 2025

• Update release branch cut to still update container and snapshot versions to 2.XX.0

  #33872 merged Feb 5, 2025

• Update CHANGES.md to include the 2.64.0 release

  #33874 merged Feb 5, 2025

• Fix release version

  #33871 merged Feb 5, 2025

• Restore "attributes of a Beam committer" to prior accuracy

  #33814 merged Feb 5, 2025

• Fix cut_release_branch workflow

  #33869 merged Feb 5, 2025

• Fix postcommit python dependency workflow

  #33865 merged Feb 5, 2025

• Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.17.55 to 1.17.58 in /sdks

  #33860 merged Feb 5, 2025

• Bump golang.org/x/oauth2 from 0.25.0 to 0.26.0 in /sdks

  #33862 merged Feb 5, 2025

• sdks/python: enable recursive deletion for GCSFileSystem Paths

  #33611 merged Feb 5, 2025

• Fix user mailing list link on contribution page

  #33856 merged Feb 5, 2025

• Bump cloud.google.com/go/pubsub from 1.45.3 to 1.47.0 in /sdks

  #33832 merged Feb 4, 2025

• Bump google.golang.org/api from 0.218.0 to 0.219.0 in /sdks

  #33789 merged Feb 4, 2025

• Bump github.com/docker/docker from 27.5.0+incompatible to 27.5.1+incompatible in /sdks

  #33791 merged Feb 4, 2025

• Bump cloud.google.com/go/spanner from 1.73.0 to 1.75.0 in /sdks

  #33846 merged Feb 4, 2025

• Fix Postcommit Python Arm workflow

  #33849 merged Feb 4, 2025

• [#31438] Trigger Precusor work for Prism.

  #33763 merged Feb 4, 2025

• [Go SDK + Protos] Fix Proto Spec for Pane encoding + Go SDK implementation.

  #33840 merged Feb 4, 2025

• Write how-to doc on dataflow cost benchmarking

  #33702 merged Feb 4, 2025

• Bump beam-master container version

  #33848 merged Feb 4, 2025

• Optimize failed key processing by indexing workitems by shardingkey + workid

  #33755 merged Feb 4, 2025

• Revert "[Flink] disable operator chaining in validatesRunner"

  #33838 merged Feb 4, 2025

• Calculate byte size via sampling in StateBackedIterable if size is not cheap to calculate

  #33780 merged Feb 4, 2025

• [Dataflow Streaming] Enable Windmill GetWork Response Batching by default

  #33847 merged Feb 4, 2025

26 Pull requests opened by 17 people

• Enable timeout setting for Python TestPipeline (#29646)

  #33866 opened Feb 5, 2025

• Add `types.Unlias` to types assertions and types switches to get an underlying type instead of types.Alias

  #33868 opened Feb 5, 2025

• [DO NOT MERGE] Run all PostCommit and PreCommit Tests against Release Branch

  #33870 opened Feb 5, 2025

• [#31438] Non-processing time triggers pass w/accumulation.

  #33881 opened Feb 5, 2025

• Enable AlloyDB tests.

  #33885 opened Feb 5, 2025

• Use BoundedTrie metric to track lineage in IO

  #33891 opened Feb 6, 2025

• Include window coder to repr of WindowedValueCoder

  #33900 opened Feb 6, 2025

• Change UnboundedSourceAsSdfWrapperFn to share the cache across instances.

  #33901 opened Feb 6, 2025

• Fix: update prerequisites link to correct location on contribution page

  #33904 opened Feb 6, 2025

• [Release] Add 2.63.0 to Beam Website

  #33905 opened Feb 6, 2025

• Eliminate usage of deprecated org.mockito.Matcherss

  #33908 opened Feb 6, 2025

• Bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.75.4 to 1.76.0 in /sdks

  #33910 opened Feb 7, 2025

• Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.17.59 to 1.17.60 in /sdks

  #33911 opened Feb 7, 2025

• Bump golang.org/x/text from 0.21.0 to 0.22.0 in /sdks

  #33912 opened Feb 7, 2025

• Bump google.golang.org/protobuf from 1.36.4 to 1.36.5 in /sdks

  #33913 opened Feb 7, 2025

• Bump github.com/nats-io/nats.go from 1.38.0 to 1.39.0 in /sdks

  #33914 opened Feb 7, 2025

• [DO NOT MERGE] V2.63.0 rc1 validations

  #33916 opened Feb 7, 2025

• Remove type suppressions from MetricsPusher and MetricsSink

  #33918 opened Feb 7, 2025

• [Portable BQ] default null list to empty list

  #33919 opened Feb 7, 2025

• multi: utilize `cmp` package for comparison in Go test files

  #33922 opened Feb 8, 2025

• Update pkglist with more recent python versions matching pythonTips page

  #33929 opened Feb 10, 2025

• Create examples/java/webapis/README.md

  #33931 opened Feb 10, 2025

• Support custom coders in Reshuffle

  #33932 opened Feb 10, 2025

• Revert emitting lineage as boundedtrie from python

  #33935 opened Feb 10, 2025

• Improve docstring for fire_interval

  #33936 opened Feb 10, 2025

• Reduce Iceberg IT parallel forks

  #33937 opened Feb 10, 2025

13 Issues closed by 6 people

• The PostCommit Java ValidatesRunner Dataflow V2 job is flaky

  #33720 closed Feb 10, 2025

• [Bug]: beam_Publish_Beam_SDK_Snapshots and beam_PostCommit_Python_Arm are extremely flaky due to failing to build wheels

  #33425 closed Feb 10, 2025

• The PreCommit Java job is flaky

  #30683 closed Feb 10, 2025

• [Bug]: Correct unsuscribe emails (dev and user mailing lists)

  #30496 closed Feb 10, 2025

• Wait.on() in Python

  #21137 closed Feb 7, 2025

• The PostCommit Java Dataflow V2 job is flaky

  #30729 closed Feb 7, 2025

• The PostCommit Java ValidatesRunner ULR job is flaky

  #33859 closed Feb 6, 2025

• [Bug]: Direct Runner sometimes rises `RuntimeError: dictionary changed size during iteration`

  #27501 closed Feb 5, 2025

• The PostCommit Python Dependency job is flaky

  #30799 closed Feb 5, 2025

• [Bug]: Python GCSFileSystem.delete does not recursively delete

  #27605 closed Feb 5, 2025

• [Bug]: Incorrect user mail list address on contribution page

  #26013 closed Feb 5, 2025

• [Bug]: Correct user mailing list email link

  #33855 closed Feb 4, 2025

• The PostCommit Python Arm job is flaky

  #30760 closed Feb 4, 2025

17 Issues opened by 9 people

• The Assign or close an issue job is flaky

  #33934 opened Feb 10, 2025

• [Bug]: pkglist needs updating to the latest python tips recommendations

  #33928 opened Feb 10, 2025

• [Feature Request]: Add unassign issue

  #33925 opened Feb 9, 2025

• [Task]: Support non-ascii BigQuery table name

  #33920 opened Feb 7, 2025

• [Feature Request]: [Portable BigQueryIO] support fetching Schema from dynamic destinations when setting useBeamSchema()

  #33917 opened Feb 7, 2025

• The PreCommit Java GCP IO Direct job is flaky

  #33909 opened Feb 7, 2025

• [Bug]: prerequisites incorrect link on contribution page

  #33903 opened Feb 6, 2025

• [Feature Request]: Support for BigTable from YAML

  #33902 opened Feb 6, 2025

• [Failing Test]: TypeError for EncodedFIle in Python SDF

  #33889 opened Feb 5, 2025

• [Bug]: BigQueryIO with AvroFormat fails on nullable nested record types

  #33875 opened Feb 5, 2025

• [Bug]: Managed BQ read transform fails to attach config row

  #33864 opened Feb 5, 2025

• Performance Regression or Improvement: gbk_python_batch_load_test_2gb_of_100KB_records:runtime

  #33857 opened Feb 4, 2025

• [Failing Test]: AttributeError for MaybeReshuffle in Python SDK

  #33854 opened Feb 4, 2025

• [Failing Test]: Namespace error for prism_runner in Python STD

  #33853 opened Feb 4, 2025

• [Failing Test]: ValueError in yaml for Python SDK

  #33852 opened Feb 4, 2025

• [Failing Test]: portable_runner_test in Python SDK

  #33851 opened Feb 4, 2025

• [Task]: Migrate Beam I/O Metrics to use a platform that allows public embedding

  #33850 opened Feb 4, 2025

48 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• [Managed Iceberg] unbounded source

  #33504 commented on Feb 10, 2025 • 12 new comments

• [Do not merge] Update slf4j version

  #33574 commented on Feb 6, 2025 • 5 new comments

• [Managed Iceberg] custom equals method for SerializedDataFile

  #33554 commented on Feb 10, 2025 • 4 new comments

• The LoadTests Go CoGBK Flink Batch job is flaky

  #30502 commented on Feb 10, 2025 • 0 new comments

• [Bug]: Unexpected Empty Records in Nested Arrays When Using BigQueryIO.write() with .withAutoSchemaUpdate(true) and .ignoreUnknownValues()

  #33842 commented on Feb 10, 2025 • 0 new comments

• [Bug]: Building wheels failing on self hosted actions, working on github's runners

  #33506 commented on Feb 10, 2025 • 0 new comments

• Support writing to Pubsub with ordering key; Add PubsubMessage SchemaCoder

  #31608 commented on Feb 8, 2025 • 0 new comments

• BigQueryIO uniformize direct and export reads

  #32360 commented on Feb 8, 2025 • 0 new comments

• Add portable Mqtt source and sink transforms

  #32385 commented on Feb 4, 2025 • 0 new comments

• Enable configuration of a CDC mutation info Callable for CDC Writes into BigQuery

  #32878 commented on Feb 4, 2025 • 0 new comments

• add generics support to AutoValueUtils helpers

  #32977 commented on Feb 4, 2025 • 0 new comments

• Add histogram to metrics container

  #33043 commented on Feb 4, 2025 • 0 new comments

• Add kafka sdf poll metrics

  #33045 commented on Feb 8, 2025 • 0 new comments

• Read RabbitMQ messages with headers containing nested objects

  #33072 commented on Feb 8, 2025 • 0 new comments

• Adding Google Storage Requester pays feature to Golang SDK.

  #33236 commented on Feb 10, 2025 • 0 new comments

• Tour of Beam: update GroupByKey example

  #33242 commented on Feb 10, 2025 • 0 new comments

• Disable keepalive pings.

  #33317 commented on Feb 10, 2025 • 0 new comments

• Force portable TestStream to use the Nested Coder context throughout.

  #33334 commented on Feb 8, 2025 • 0 new comments

• [docs] Add new Python multi-lang quickstart using the SchemaTransform framework

  #33360 commented on Feb 4, 2025 • 0 new comments

• [java] BQ: Add avro schema to BQ TableSchema conversion

  #33389 commented on Feb 7, 2025 • 0 new comments

• Remove type coercion of `FinalizeWrite` in iobase.py

  #33614 commented on Feb 4, 2025 • 0 new comments

• Fix incorrect nullness in FlinkJobInvoker and JobInvoker

  #33713 commented on Feb 4, 2025 • 0 new comments

• Upgrade to hive 4.0.1

  #33750 commented on Feb 4, 2025 • 0 new comments

• [Spark] Skip unused outputs of ParDo in SparkRunner (#33771)

  #33772 commented on Feb 4, 2025 • 0 new comments

• The PostRelease Nightly Snapshot job is flaky

  #30505 commented on Feb 4, 2025 • 0 new comments

• [Task]: Add new Python SchemaTransform multi-language documentation

  #33358 commented on Feb 4, 2025 • 0 new comments

• [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled"

  #33435 commented on Feb 4, 2025 • 0 new comments

• [Bug]: Iceberg integration test warning log spam

  #33776 commented on Feb 4, 2025 • 0 new comments

• [Tracking Umbrella] Prism Runner areas for contribution.

  #29650 commented on Feb 4, 2025 • 0 new comments

• The PostCommit Python ValidatesContainer Dataflow With RC job is flaky

  #30525 commented on Feb 5, 2025 • 0 new comments

• The PostCommit Java Nexmark Dataflow job is flaky

  #30606 commented on Feb 5, 2025 • 0 new comments

• The PostCommit Java Examples Dataflow V2 job is flaky

  #33646 commented on Feb 5, 2025 • 0 new comments

• JUnit5 support

  #18733 commented on Feb 6, 2025 • 0 new comments

• The PostCommit Java ValidatesRunner Dataflow job is flaky

  #33722 commented on Feb 6, 2025 • 0 new comments

• The PostCommit Java ValidatesRunner Dataflow JavaVersions job is flaky

  #33723 commented on Feb 6, 2025 • 0 new comments

• [Bug]: UnboundedSourceAsSDFWrapperFn caches readers per DoFn instance

  #32968 commented on Feb 6, 2025 • 0 new comments

• The IcebergIO Integration Tests job is flaky

  #31931 commented on Feb 6, 2025 • 0 new comments

• The PostCommit Java ValidatesRunner Dataflow V2 Streaming job is flaky

  #32224 commented on Feb 6, 2025 • 0 new comments

• Performance Regression or Improvement: pytorch_image_classification_benchmarks-resnet152-GPU-mean_load_model_latency_milli_secs:mean_load_model_latency_milli_secs

  #33704 commented on Feb 6, 2025 • 0 new comments

• [Bug][Go]: Metrics incremented in Setup methods are not recalled.

  #27038 commented on Feb 7, 2025 • 0 new comments

• [Bug]: Using WriteToBigQuery FILE_LOADS in a streaming pipeline does not remove temporary tables

  #31329 commented on Feb 7, 2025 • 0 new comments

• Use cmp in fullvalue_test.go

  #20154 commented on Feb 8, 2025 • 0 new comments

• [Task][Go SDK]: Add String UTF8 check to vet runner for serialization.

  #24949 commented on Feb 8, 2025 • 0 new comments

• The Clean Up Prebuilt SDK Images job is flaky

  #30514 commented on Feb 9, 2025 • 0 new comments

• [Task]: Enable to set timeout for Python TestPipeline

  #29646 commented on Feb 9, 2025 • 0 new comments

• Performance Regression or Improvement: pytorch_image_classification_benchmarks-resnet152-GPU-mean_inference_batch_latency_micro_secs:mean_inference_batch_latency_micro_secs

  #33670 commented on Feb 9, 2025 • 0 new comments

• The PostCommit Java IO Performance Tests job is flaky

  #30527 commented on Feb 10, 2025 • 0 new comments

• The LoadTests Go GBK Flink Batch job is flaky

  #30507 commented on Feb 10, 2025 • 0 new comments