Skip to content

Commit

Permalink
[yugabyte#9867]: YSQL: Fix double type overflow in case of SET yb_tra…
Browse files Browse the repository at this point in the history
…nsaction_priority_lower_bound/yb_transaction_priority_upperr_bound command

Summary:
YB transaction priority is stored in uint64_t variable.
the `SET yb_transaction_priority_lower_bound/yb_transaction_priority_upperr_bound` command uses `double` to store factor (0..1) for priority bounds calculation. `double` has the following limitations for representing integers

```
Precision limitations on integer values:
- Integers from −2^53 to 2^53 (−9,007,199,254,740,992 to 9,007,199,254,740,992) can be exactly represented
- Integers between 2^53 and 2^54 = 18,014,398,509,481,984 round to a multiple of 2 (even number)
-Integers between 2^54 and 2^55 = 36,028,797,018,963,968 round to a multiple of 4
```

Due to this limitations the following statement causes overflow

```
  return minValue + value * static_cast<double>(maxValue - minValue);
```

Solutions is to use `long double` instead of `double` because it can exactly represents int up to `2^113`

Test Plan:
Run existing unit test multiple times

```
./yb_build.sh --cxx-test pgwrapper_pg_libpq-test --gtest_filter PgLibPqTest.MultiBankAccountSerializable -n 20
```

Reviewers: alex

Reviewed By: alex

Subscribers: yql

Differential Revision: https://phabricator.dev.yugabyte.com/D12789
  • Loading branch information
d-uspenskiy committed Aug 30, 2021
1 parent 5557a8b commit 5b46a35
Showing 1 changed file with 2 additions and 3 deletions.
5 changes: 2 additions & 3 deletions src/yb/yql/pggate/pg_txn_manager.cc
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ uint64_t txn_priority_regular_upper_bound = yb::kRegularTxnUpperBound;
uint64_t txn_priority_regular_lower_bound = yb::kRegularTxnLowerBound;
// Converts double value in range 0..1 to uint64_t value in range [minValue, maxValue]
uint64_t ConvertBound(double value, uint64_t minValue, uint64_t maxValue) {
uint64_t ConvertBound(long double value, uint64_t minValue, uint64_t maxValue) {
if (value <= 0.0) {
return minValue;
}
Expand All @@ -58,8 +58,7 @@ uint64_t ConvertBound(double value, uint64_t minValue, uint64_t maxValue) {
return maxValue;
}
// Have to cast to double to avoid a warning on implicit cast that changes the value.
return minValue + value * static_cast<double>(maxValue - minValue);
return minValue + value * (maxValue - minValue);
}
uint64_t ConvertRegularPriorityTxnBound(double value) {
Expand Down

0 comments on commit 5b46a35

Please sign in to comment.