test new release

test

Vitess 6 General Availability - TEST

Incompatible Changes

This release includes the following changes, which may result in incompatibilities when upgrading from a previous release:

Changes to Local Examples

The local examples have been updated to use the new VReplication workflows for MoveTables and Reshard. This obsoletes the previous workflows of Vertical Split Clone and Horizontal Sharding. The earlier examples remain available in the “local.legacy” example directory for now. vitessio#6016 vitessio#6015 vitessio#6052 vitessio#6072 vitessio#6106

Change to Helm Chart

The helm chart has been updated to default to using Kubernetes as the Topology Server provider. This change was required to remove the dependency on etcd-operator, which is no longer under active support. vitessio#5989

Earlier Kubernetes Operator Retired

Support for the Vitess Kubernetes Operator has been removed due to a lack of maintainers. Adoption is understood to be low, but users are encouraged to migrate to using the Helm charts which continue to be supported, or PlanetScale’s Vitess Operator (provided by a third party) vitessio#5873

SQL Support: Keywords are no longer lowercase

Vitess no longer lower cases SQL keywords. In corner cases, this may affect applications which made specific use of SQL keywords, and then referred to a column in the result set vitessio#5897

SQL Support: Parenthesis are now removed from the AST
Vitess now simplifies expressions to remove parentheses from expressions. In corner cases, this may affect applications which use an expression that contains parens and then refer to it as a column in the result. vitessio#5906

Statement Based Replication, SplitQuery and UpdateStream
Code for the unsupported features of Statement-based replication (SBR), SplitQuery and UpdateStream has now been removed. Users using SBR are encouraged to switch to using Row-based replication instead. vitessio#5940 vitessio#5972 vitessio#5928

Messaging: First Class Citizen + API Improvements
The messaging feature has had several breaking changes and feature additions
Direct inserts are now supported: INSERT INTO msg_table SELECT col1, col2...
time_scheduled and time_created are no longer required nor automatically populated.
priority is a new required field. Messages with a lower priority will be processed first.
Message retention is now measured after time_acked instead of time_scheduled
New table options vt_min_backoff and vt_max_backoff set bounds on exponential backoff for message retries
33% backoff jitter has been added for messages that are getting postponed to prevent thundering herds.
Users of the messages feature are encouraged to read through the full details in issue vitessio#5947 vitessio#5948 vitessio#6114

RPM and DEB Packages
Vitess binaries and examples are now installed in system standard paths, rather than the /vt directory vitessio#5836

Vitess Go Driver
Values of type sqltypes.Timestamp are now automatically converted to time.Time values
vitessio#5788

Simplified VTtablet Schema Load Process
The process of loading vttablet schema has been simplified, as many of the statistics loaded were expensive to gather but remained unused. This may lead to compatibility breakage in the use of the features: Sequences, VStreamer and Messages vitessio#5951

Planned Reparent
The implementation of Planned Reparent and Emergency Reparent have been improved, with more graceful handing of failure modes. While this change is functionality neutral, it is important that Vitess components are upgraded in the recommended order. vitessio#6050
Bugs Fixed

• Binlog Client: Vitess components now randomize the server_id when acting as a MySQL replica. vitessio#5998

• heartbeat: Prevent a scenario where tables could be created on a replica leading to errant transactions. Now tables are only created on the master with binlog ON, using the app_user not dba_user vitessio#6053

• Messages: For the mysql protocol, buffered results coming from vitess are flushed a minimum of every 100ms, and configurable by mysql_server_flush_delay. This helps prevent against a case where intermittent streaming could delay messages from being sent. vitessio#5879

• mysqlctl: Correctly escape and quote passwords that contain special characters. vitessio#5761

• VReplication: Several improvements to failure handling and miscellaneous fixes vitessio#5753 vitessio#5792 vitessio#5801 vitessio#5842 vitessio#5935 vitessio#5905 vitessio#5845 vitessio#5942

• Vtcompose: Add a check to make sure that files can be read vitessio#6024

• Vtcompose: refactor vtcompose for improved testability vitessio#5994

• Vindexes: Support for a region_json Vindex vitessio#6017 vitessio#5806 vitessio#5955

• Vtctl: Before applying a VSchema, vtctl now correctly checks that the schema exists. vitessio#5931

• Vtctl: Vtctl now exits early if the topo_global_root was set incorrectly vitessio#6119

• Vtctl: EnsureVSchema and GetOrCreateShard now pass in the cell name. This helps prevent against corruption, which could occur from auto-rebuilding of cells automatically when creating a new shard or keyspace. vitessio#5930

• VTExplain: An issue where incorrectly formatted comments could lead to a hang has been fixed. vitessio#5984

• VTGate: Leading zeros were incorrectly dropped in microsecond timestamps when using prepared statements vitessio#5901

• VTGate: Setting the transaction_mode in a mysql session is now case insensitive. vitessio#5963

• VTGate: When updating consistent indexes, VTGate applies an optimization where it checks if anything really changed before applying an update. In the case that this check results in errors, VTGate will now skip the optimization and execute the update regardless. vitessio#5883

• VTGate: Remove allocation of unused bytes vitessio#5874

• VTGate: The gateway interface functionality has been simplified in anticipation of two new gateway implementations: TabletGateway and VTDirectGateway vitessio#5978 vitessio#5992

• VTGate: The process of watching for VSchema updates from the topology server now recovers successfully if it receives an invalid update. This helps prevent against a scenario where a malformed vschema could take down all vtgate servers. vitessio#6051

• VTGate: Previously VTGate performed health checks against all keyspaces, not just keyspaces listed in keyspaces_to_watch. Healthchecks are now limited correctly. vitessio#5815

• Vtgate: SQL Support: database() is now only rewritten against dual vitessio#5793

• Vtgate: Plan Cache is now disabled for statements with nil instructions vitessio#6045

• Vtgate: Make sure to only accept DDL that can be parsed, at least partially vitessio#6023

• Vtgate: Log when accepting a new connection fails vitessio#5881

• Vtgate: Prepared Statements did not correctly handle signed integers vitessio#5974

• Vtgate: Executor statistics did not correctly account for dividing by zero, leading to a potential panic. vitessio#5904

• Vtgate: The error message for unsupported DDL creation has been improved vitessio#6109

• Vtgate: The metric QueriesProcessedByTable was not correctly incremented with streaming queries processed by vtgate vitessio#6057

• GRPC Client: The default keep alive and keep alive timeout values have been changed from unlimited to 10 seconds. This helps prevent cases where queries could hang while waiting for a failed pod or node. In addition, the defaultMaxMessageSize is now set to 16MB vitessio#5922 vitessio#3890

• VTTablet: local_metadata is now populated after an agent has been initialized. This helps prevent against the ambiguous case where both -init_populate_metadata and -restore_from_backup are specified. vitessio#5783

• VTTablet: A hang could occur when shutting down a tablet server when the Topo Server is unavailable. Tablet servers will now time out correctly. vitessio#5865

• Vttablet Health Check: A warning is now logged when the vttablet StreamHealth RPC fails vitessio#5917

• VTTablet Planned Reparent: A new context is now created for calling UndoDemoteMaster. This helps prevent a scenario where the existing context may have expired. vitessio#5841

• VTTablet: A panic could occur when calling NewDBConnNoPool vitessio#6086

• Vttablet: Queries of type STREAM_EXECUTE were inadvertently dropped from logging vitessio#6056

• Zookeeper Topo: Trailing newlines are now ignored when specifying -topo_zk_auth_file
  vitessio#6030

• Misc: Stats counters would erroneously print warnings to log files when adding a negative value. This warning did not consider the case where counters are reused. vitessio#5988

• Misc: Binary log events with the special name of "mysql.rds_*" are now skipped. This helps improve compatibility with Amazon RDS flavors vitessio#5966

Functionality Added or Changed

• Docker: sysstat and strace are now bundled in vitess/lite images vitessio#5814

• Docker: new lite images based on Red Hat UBI 7 are now available vitessio#6100

• Example Code: An are you alive example has been added vitessio#5646

• Example Code: The demo application has been migrated from Python to Go vitessio#6022

• Example Code: A set of notes and scripts on how to use Vitess with common frameworks vitessio#6034

• Helm charts: now work with the latest Kubernetes version. An issue where the 308_final.yml script did not delete a tablet has also been fixed. vitessio#6048 vitessio#6083 vitessio#6075 vitessio#6074

• Helm charts: Helm charts now enable proxy_tablets by default vitessio#6112

• Java Client: The deprecated VTGateBlockingConn client has now been removed. The newer VTGateBlockingConnection client should be used instead. vitessio#5973

• Get Started Guides: Examples now use the VReplication workflows MoveTables and Reshard instead of Vertical Split Clone and Horizontal Resharding. vitessio#6016 vitessio#6015 vitessio#6052 vitessio#6072 vitessio#6106

• Local Example: Scripts that start services have been refactored to ./scripts/vttablet-up.sh, ./scripts/mysqlctl-up.sh and start only one daemon at a time. The motivation for doing this was to make the usage more closely mimic init scripts. vitessio#5808

• Messaging: add new priority field vitessio#5967

• Messaging: Setting both a minimum and maximum backoff time is now supported (rather than exponentially backing off forever) vitessio#5979

• Messaging: Messaging has been refactored to use Row-based Replication vstreamer feed vitessio#5913

• Messaging: A 33% Jitter has been added to postpone backoff vitessio#6092

• ...