Skip to content

[libc++] Fix UB in bitwise logic of {std, ranges}::{fill, fill_n} algorithms #122410

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Feb 5, 2025

Conversation

winner245
Copy link
Contributor

@winner245 winner245 commented Jan 10, 2025

This PR addresses an undefined behavior that arises when using the std::fill and std::fill_n algorithms, as well as their ranges counterparts ranges::fill and ranges::fill_n, with vector<bool, Alloc> that utilizes a custom-sized allocator with small integral types.

When assigning to a vector<bool, Alloc> via the fill or fill_n algorithms, if the underlying __storage_type (i.e., Alloc::size_type if available) of the word is a small unsigned integral type (e.g., uint8_t, uint16_t, unsigned short), the internal bitwise logic in __algorithm/fill_n.h is subject to integral promotion. The problematic code located in __algorithm/fill_n.h is as follows:

__storage_type __m = (~__storage_type(0) << __first.__ctz_) & (~__storage_type(0) >> (__clz_f - __dn));

Here, the evaluation ~__storage_type(0) is first promoted to ~int(0), which evaluates to -1. This leads to a subsequent bitwise left shift of a negative integer value, resulting in UB prior to C++20. Although starting in C++20, bitwise left shift of negative integer values are defined as modulo 2^N operation, we must address this issue for backward compatibility with standards prior to C++20.

Since this UB typically results in a compilation warning, it can be difficult to detect. Running the new tests added in this PR without applying my fix will cause only the stage 2 CI (generic-gcc-cxx11, gcc-14, g++-14) to fail, as it treats the shift-negative-value warning as an error using the -Werror=shift-negative-value compilation flag (all other stage 1 and 2 CIs pass):

  # | /__w/llvm-project/llvm-project/build/generic-gcc-cxx11/libcxx/test-suite-install/include/c++/v1/__algorithm/fill_n.h:44:50: error: left shift of negative value [-Werror=shift-negative-value]
  # |    44 |     __storage_type __m     = (~__storage_type(0) << __first.__ctz_) & (~__storage_type(0) >> (__clz_f - __dn));
  # |       |                              ~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~

To reproduce this UB, we should enable the warning as an error with the following compilation flags in both Clang and GCC: -Werror=shift-negative-value and run it for any C++ standards prior to C++20:

Godbolt Link

int main() { 
  using __storage_type = unsigned short;
  auto y = ~__storage_type(0) << 1;
 (void)y;
}

When compiling this code, Clang will produce the following error message (gcc exhibits a similar behavior):

<source>:3:31: error: shifting a negative signed value is undefined [-Werror,-Wshift-negative-value]
    3 |   auto y = ~__storage_type(0) << 1;
      |            ~~~~~~~~~~~~~~~~~~ ^
1 error generated.

Solution: since integral promotions in bitwise arithmetic for small integral types are inevitable, to resolve this UB, we need to ensure that the result of each bitwise operation is explicitly cast back to the unsigned __storage_type. This prevents left shifts of negative values due to integral promotions.

Additional Note: This UB was first observed while working on #119801 and has led to more serious ambiguous call bugs in {std, ranges}::{count, find} algorithms, as demonstrated in issue #122528. Thus, fixing the UB for fill/fill_n is a prerequisite for addressing the bugs in the {count, find} algorithms.

@winner245 winner245 force-pushed the fix-bitwise-logic-in-algo branch 3 times, most recently from 2ecddb8 to c84a156 Compare January 10, 2025 04:59
@winner245 winner245 changed the title [libc++] Fix UB in bitwise logic of algorithms [libc++] Fix UB in bitwise logic of {std, ranges}::{fill, fill_n} algorithms Jan 11, 2025
@winner245 winner245 force-pushed the fix-bitwise-logic-in-algo branch 6 times, most recently from 8030070 to fa1fb7f Compare January 12, 2025 04:16
@winner245 winner245 marked this pull request as ready for review January 13, 2025 14:43
@winner245 winner245 requested a review from a team as a code owner January 13, 2025 14:43
@llvmbot llvmbot added the libc++ libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi. label Jan 13, 2025
@llvmbot
Copy link
Member

llvmbot commented Jan 13, 2025

@llvm/pr-subscribers-libcxx

Author: Peng Liu (winner245)

Changes

This PR addresses an undefined behavior that arises when using the std::fill and std::fill_n algorithms, as well as their ranges counterparts ranges::fill and ranges::fill_n, with vector&lt;bool, Alloc&gt; that utilizes a custom-sized allocator with small integral types.

When assigning to a vector&lt;bool, Alloc&gt; via the fill or fill_n algorithms, if the underlying __storage_type (i.e., Alloc::size_type if available) of the word is a small unsigned integral type (e.g., uint8_t, uint16_t, unsigned short), the internal bitwise logic in __algorithm/fill_n.h is subject to integral promotion. The problematic code located in __algorithm/fill_n.h is as follows:

__storage_type __m = (~__storage_type(0) << __first.__ctz_) & (~__storage_type(0) >> (__clz_f - __dn));

Here, the evaluation ~__storage_type(0) is first promoted to ~int(0), which evaluates to -1. This leads to a subsequent bitwise left shift of a negative integer value, resulting in UB prior to C++20. Although starting in C++20, bitwise left shift of negative integer values are defined as modulo 2^N operation, we must address this issue for backward compatibility with standards prior to C++20.

Since this UB typically results in a compilation warning, it can be difficult to detect. Running the new tests added in this PR without applying my fix will cause only the stage 2 CI (generic-gcc-cxx11, gcc-14, g++-14) to fail, as it treats the shift-negative-value warning as an error using the -Werror=shift-negative-value compilation flag (all other stage 1 and 2 CIs pass):

  # | /__w/llvm-project/llvm-project/build/generic-gcc-cxx11/libcxx/test-suite-install/include/c++/v1/__algorithm/fill_n.h:44:50: error: left shift of negative value [-Werror=shift-negative-value]
  # |    44 |     __storage_type __m     = (~__storage_type(0) &lt;&lt; __first.__ctz_) &amp; (~__storage_type(0) &gt;&gt; (__clz_f - __dn));
  # |       |                              ~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~

To reproduce this UB, we should enable the warning as an error with the following compilation flags in both Clang and GCC: -Werror=shift-negative-value and run it for any C++ standards prior to C++20:

Godbolt Link

int main() { 
  using __storage_type = unsigned short;
  auto y = ~__storage_type(0) &lt;&lt; 1;
 (void)y;
}

When compiling this code, Clang will produce the following error message (gcc exhibits a similar behavior):

&lt;source&gt;:3:31: error: shifting a negative signed value is undefined [-Werror,-Wshift-negative-value]
    3 |   auto y = ~__storage_type(0) &lt;&lt; 1;
      |            ~~~~~~~~~~~~~~~~~~ ^
1 error generated.

Solution: since integral promotions in bitwise arithmetic for small integral types are inevitable, to resolve this UB, we need to ensure that the result of each bitwise operation is explicitly cast back to the unsigned __storage_type. This prevents left shifts of negative values due to integral promotions.

Additional Note: This UB was first observed while working on #119801 and has led to more serious ambiguous call bugs in {std, ranges}::{count, find} algorithms, as demonstrated in issue #122528. Thus, fixing the UB for fill/fill_n is a prerequisite for addressing the bugs in the {count, find} algorithms.


Patch is 20.89 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/122410.diff

7 Files Affected:

  • (modified) libcxx/include/__algorithm/fill_n.h (+2-2)
  • (modified) libcxx/include/__fwd/bit_reference.h (+18)
  • (modified) libcxx/test/std/algorithms/alg.modifying.operations/alg.fill/fill.pass.cpp (+35)
  • (modified) libcxx/test/std/algorithms/alg.modifying.operations/alg.fill/fill_n.pass.cpp (+118-102)
  • (modified) libcxx/test/std/algorithms/alg.modifying.operations/alg.fill/ranges.fill.pass.cpp (+47-5)
  • (modified) libcxx/test/std/algorithms/alg.modifying.operations/alg.fill/ranges.fill_n.pass.cpp (+43-1)
  • (added) libcxx/test/support/sized_allocator.h (+58)
diff --git a/libcxx/include/__algorithm/fill_n.h b/libcxx/include/__algorithm/fill_n.h
index 5069a72783f348..5bccf5c9bd0bee 100644
--- a/libcxx/include/__algorithm/fill_n.h
+++ b/libcxx/include/__algorithm/fill_n.h
@@ -41,7 +41,7 @@ __fill_n_bool(__bit_iterator<_Cp, false> __first, typename _Cp::size_type __n) {
   if (__first.__ctz_ != 0) {
     __storage_type __clz_f = static_cast<__storage_type>(__bits_per_word - __first.__ctz_);
     __storage_type __dn    = std::min(__clz_f, __n);
-    __storage_type __m     = (~__storage_type(0) << __first.__ctz_) & (~__storage_type(0) >> (__clz_f - __dn));
+    __storage_type __m     = std::__middle_mask<__storage_type>(__first.__ctz_, __clz_f - __dn);
     if (_FillVal)
       *__first.__seg_ |= __m;
     else
@@ -56,7 +56,7 @@ __fill_n_bool(__bit_iterator<_Cp, false> __first, typename _Cp::size_type __n) {
   // do last partial word
   if (__n > 0) {
     __first.__seg_ += __nw;
-    __storage_type __m = ~__storage_type(0) >> (__bits_per_word - __n);
+    __storage_type __m = std::__trailing_mask<__storage_type>(__bits_per_word - __n);
     if (_FillVal)
       *__first.__seg_ |= __m;
     else
diff --git a/libcxx/include/__fwd/bit_reference.h b/libcxx/include/__fwd/bit_reference.h
index 237efb6db66429..c927e65747af08 100644
--- a/libcxx/include/__fwd/bit_reference.h
+++ b/libcxx/include/__fwd/bit_reference.h
@@ -10,6 +10,8 @@
 #define _LIBCPP___FWD_BIT_REFERENCE_H
 
 #include <__config>
+#include <__type_traits/enable_if.h>
+#include <__type_traits/is_unsigned.h>
 
 #if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
 #  pragma GCC system_header
@@ -20,6 +22,22 @@ _LIBCPP_BEGIN_NAMESPACE_STD
 template <class _Cp, bool _IsConst, typename _Cp::__storage_type = 0>
 class __bit_iterator;
 
+template <class _StorageType, __enable_if_t<is_unsigned<_StorageType>::value, int> = 0>
+_LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 _StorageType __leading_mask(unsigned __shift) {
+  return static_cast<_StorageType>(static_cast<_StorageType>(~static_cast<_StorageType>(0)) << __shift);
+}
+
+template <class _StorageType, __enable_if_t<is_unsigned<_StorageType>::value, int> = 0>
+_LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 _StorageType __trailing_mask(unsigned __shift) {
+  return static_cast<_StorageType>(static_cast<_StorageType>(~static_cast<_StorageType>(0)) >> __shift);
+}
+
+template <class _StorageType, __enable_if_t<is_unsigned<_StorageType>::value, int> = 0>
+_LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 _StorageType __middle_mask(unsigned __lshift, unsigned __rshift) {
+  return static_cast<_StorageType>(
+      std::__leading_mask<_StorageType>(__lshift) & std::__trailing_mask<_StorageType>(__rshift));
+}
+
 _LIBCPP_END_NAMESPACE_STD
 
 #endif // _LIBCPP___FWD_BIT_REFERENCE_H
diff --git a/libcxx/test/std/algorithms/alg.modifying.operations/alg.fill/fill.pass.cpp b/libcxx/test/std/algorithms/alg.modifying.operations/alg.fill/fill.pass.cpp
index 619dc7242a3660..a1797fab6cc140 100644
--- a/libcxx/test/std/algorithms/alg.modifying.operations/alg.fill/fill.pass.cpp
+++ b/libcxx/test/std/algorithms/alg.modifying.operations/alg.fill/fill.pass.cpp
@@ -19,6 +19,7 @@
 #include <cstddef>
 #include <vector>
 
+#include "sized_allocator.h"
 #include "test_macros.h"
 #include "test_iterators.h"
 
@@ -46,6 +47,37 @@ struct Test {
   }
 };
 
+TEST_CONSTEXPR_CXX20 void test_bititer_with_custom_sized_types() {
+  {
+    using Alloc = sized_allocator<bool, std::uint8_t, std::int8_t>;
+    std::vector<bool, Alloc> in(100, false, Alloc(1));
+    std::vector<bool, Alloc> expected(100, true, Alloc(1));
+    std::fill(in.begin(), in.end(), true);
+    assert(in == expected);
+  }
+  {
+    using Alloc = sized_allocator<bool, std::uint16_t, std::int16_t>;
+    std::vector<bool, Alloc> in(200, false, Alloc(1));
+    std::vector<bool, Alloc> expected(200, true, Alloc(1));
+    std::fill(in.begin(), in.end(), true);
+    assert(in == expected);
+  }
+  {
+    using Alloc = sized_allocator<bool, std::uint32_t, std::int32_t>;
+    std::vector<bool, Alloc> in(200, false, Alloc(1));
+    std::vector<bool, Alloc> expected(200, true, Alloc(1));
+    std::fill(in.begin(), in.end(), true);
+    assert(in == expected);
+  }
+  {
+    using Alloc = sized_allocator<bool, std::uint64_t, std::int64_t>;
+    std::vector<bool, Alloc> in(200, false, Alloc(1));
+    std::vector<bool, Alloc> expected(200, true, Alloc(1));
+    std::fill(in.begin(), in.end(), true);
+    assert(in == expected);
+  }
+}
+
 TEST_CONSTEXPR_CXX20 bool test() {
   types::for_each(types::forward_iterator_list<char*>(), Test<char>());
   types::for_each(types::forward_iterator_list<int*>(), Test<int>());
@@ -93,6 +125,9 @@ TEST_CONSTEXPR_CXX20 bool test() {
       assert(in == expected);
     }
   }
+
+  test_bititer_with_custom_sized_types();
+
   return true;
 }
 
diff --git a/libcxx/test/std/algorithms/alg.modifying.operations/alg.fill/fill_n.pass.cpp b/libcxx/test/std/algorithms/alg.modifying.operations/alg.fill/fill_n.pass.cpp
index 7d6770de702bf3..582889dbb21d10 100644
--- a/libcxx/test/std/algorithms/alg.modifying.operations/alg.fill/fill_n.pass.cpp
+++ b/libcxx/test/std/algorithms/alg.modifying.operations/alg.fill/fill_n.pass.cpp
@@ -15,159 +15,175 @@
 
 #include <algorithm>
 #include <cassert>
+#include <vector>
 
+#include "sized_allocator.h"
 #include "test_macros.h"
 #include "test_iterators.h"
 #include "user_defined_integral.h"
 
 #if TEST_STD_VER > 17
 TEST_CONSTEXPR bool test_constexpr() {
-    const std::size_t N = 5;
-    int ib[] = {0, 0, 0, 0, 0, 0}; // one bigger than N
-
-    auto it = std::fill_n(std::begin(ib), N, 5);
-    return it == (std::begin(ib) + N)
-        && std::all_of(std::begin(ib), it, [](int a) {return a == 5; })
-        && *it == 0 // don't overwrite the last value in the output array
-        ;
-    }
+  const std::size_t N = 5;
+  int ib[]            = {0, 0, 0, 0, 0, 0}; // one bigger than N
+
+  auto it = std::fill_n(std::begin(ib), N, 5);
+  return it == (std::begin(ib) + N) && std::all_of(std::begin(ib), it, [](int a) { return a == 5; }) &&
+       *it == 0 // don't overwrite the last value in the output array
+      ;
+}
 #endif
 
 typedef UserDefinedIntegral<unsigned> UDI;
 
 template <class Iter>
-void
-test_char()
-{
-    char a[4] = {};
-    Iter it = std::fill_n(Iter(a), UDI(4), char(1));
-    assert(base(it) == a + 4);
-    assert(a[0] == 1);
-    assert(a[1] == 1);
-    assert(a[2] == 1);
-    assert(a[3] == 1);
+void test_char() {
+  char a[4] = {};
+  Iter it   = std::fill_n(Iter(a), UDI(4), char(1));
+  assert(base(it) == a + 4);
+  assert(a[0] == 1);
+  assert(a[1] == 1);
+  assert(a[2] == 1);
+  assert(a[3] == 1);
 }
 
 template <class Iter>
-void
-test_int()
-{
-    int a[4] = {};
-    Iter it = std::fill_n(Iter(a), UDI(4), 1);
-    assert(base(it) == a + 4);
-    assert(a[0] == 1);
-    assert(a[1] == 1);
-    assert(a[2] == 1);
-    assert(a[3] == 1);
+void test_int() {
+  int a[4] = {};
+  Iter it  = std::fill_n(Iter(a), UDI(4), 1);
+  assert(base(it) == a + 4);
+  assert(a[0] == 1);
+  assert(a[1] == 1);
+  assert(a[2] == 1);
+  assert(a[3] == 1);
 }
 
-void
-test_int_array()
-{
-    int a[4] = {};
-    assert(std::fill_n(a, UDI(4), static_cast<char>(1)) == a + 4);
-    assert(a[0] == 1);
-    assert(a[1] == 1);
-    assert(a[2] == 1);
-    assert(a[3] == 1);
+void test_int_array() {
+  int a[4] = {};
+  assert(std::fill_n(a, UDI(4), static_cast<char>(1)) == a + 4);
+  assert(a[0] == 1);
+  assert(a[1] == 1);
+  assert(a[2] == 1);
+  assert(a[3] == 1);
 }
 
 struct source {
-    source() : i(0) { }
+  source() : i(0) {}
 
-    operator int() const { return i++; }
-    mutable int i;
+  operator int() const { return i++; }
+  mutable int i;
 };
 
-void
-test_int_array_struct_source()
-{
-    int a[4] = {};
-    assert(std::fill_n(a, UDI(4), source()) == a + 4);
-    assert(a[0] == 0);
-    assert(a[1] == 1);
-    assert(a[2] == 2);
-    assert(a[3] == 3);
+void test_int_array_struct_source() {
+  int a[4] = {};
+  assert(std::fill_n(a, UDI(4), source()) == a + 4);
+  assert(a[0] == 0);
+  assert(a[1] == 1);
+  assert(a[2] == 2);
+  assert(a[3] == 3);
 }
 
 struct test1 {
-    test1() : c(0) { }
-    test1(char xc) : c(xc + 1) { }
-    char c;
+  test1() : c(0) {}
+  test1(char xc) : c(xc + 1) {}
+  char c;
 };
 
-void
-test_struct_array()
-{
-    test1 test1a[4] = {};
-    assert(std::fill_n(test1a, UDI(4), static_cast<char>(10)) == test1a + 4);
-    assert(test1a[0].c == 11);
-    assert(test1a[1].c == 11);
-    assert(test1a[2].c == 11);
-    assert(test1a[3].c == 11);
+void test_struct_array() {
+  test1 test1a[4] = {};
+  assert(std::fill_n(test1a, UDI(4), static_cast<char>(10)) == test1a + 4);
+  assert(test1a[0].c == 11);
+  assert(test1a[1].c == 11);
+  assert(test1a[2].c == 11);
+  assert(test1a[3].c == 11);
 }
 
-class A
-{
-    char a_;
+class A {
+  char a_;
+
 public:
-    A() {}
-    explicit A(char a) : a_(a) {}
-    operator unsigned char() const {return 'b';}
+  A() {}
+  explicit A(char a) : a_(a) {}
+  operator unsigned char() const { return 'b'; }
 
-    friend bool operator==(const A& x, const A& y)
-        {return x.a_ == y.a_;}
+  friend bool operator==(const A& x, const A& y) { return x.a_ == y.a_; }
 };
 
-void
-test5()
-{
-    A a[3];
-    assert(std::fill_n(&a[0], UDI(3), A('a')) == a+3);
-    assert(a[0] == A('a'));
-    assert(a[1] == A('a'));
-    assert(a[2] == A('a'));
+void test5() {
+  A a[3];
+  assert(std::fill_n(&a[0], UDI(3), A('a')) == a + 3);
+  assert(a[0] == A('a'));
+  assert(a[1] == A('a'));
+  assert(a[2] == A('a'));
 }
 
-struct Storage
-{
-  union
-  {
+struct Storage {
+  union {
     unsigned char a;
     unsigned char b;
   };
 };
 
-void test6()
-{
+void test6() {
   Storage foo[5];
   std::fill_n(&foo[0], UDI(5), Storage());
 }
 
+TEST_CONSTEXPR_CXX20 void test_bititer_with_custom_sized_types() {
+  {
+    using Alloc = sized_allocator<bool, std::uint8_t, std::int8_t>;
+    std::vector<bool, Alloc> in(100, false, Alloc(1));
+    std::vector<bool, Alloc> expected(100, true, Alloc(1));
+    std::fill_n(in.begin(), in.size(), true);
+    assert(in == expected);
+  }
+  {
+    using Alloc = sized_allocator<bool, std::uint16_t, std::int16_t>;
+    std::vector<bool, Alloc> in(200, false, Alloc(1));
+    std::vector<bool, Alloc> expected(200, true, Alloc(1));
+    std::fill_n(in.begin(), in.size(), true);
+    assert(in == expected);
+  }
+  {
+    using Alloc = sized_allocator<bool, std::uint32_t, std::int32_t>;
+    std::vector<bool, Alloc> in(200, false, Alloc(1));
+    std::vector<bool, Alloc> expected(200, true, Alloc(1));
+    std::fill_n(in.begin(), in.size(), true);
+    assert(in == expected);
+  }
+  {
+    using Alloc = sized_allocator<bool, std::uint64_t, std::int64_t>;
+    std::vector<bool, Alloc> in(200, false, Alloc(1));
+    std::vector<bool, Alloc> expected(200, true, Alloc(1));
+    std::fill_n(in.begin(), in.size(), true);
+    assert(in == expected);
+  }
+}
+
+int main(int, char**) {
+  test_char<cpp17_output_iterator<char*> >();
+  test_char<forward_iterator<char*> >();
+  test_char<bidirectional_iterator<char*> >();
+  test_char<random_access_iterator<char*> >();
+  test_char<char*>();
 
-int main(int, char**)
-{
-    test_char<cpp17_output_iterator<char*> >();
-    test_char<forward_iterator<char*> >();
-    test_char<bidirectional_iterator<char*> >();
-    test_char<random_access_iterator<char*> >();
-    test_char<char*>();
+  test_int<cpp17_output_iterator<int*> >();
+  test_int<forward_iterator<int*> >();
+  test_int<bidirectional_iterator<int*> >();
+  test_int<random_access_iterator<int*> >();
+  test_int<int*>();
 
-    test_int<cpp17_output_iterator<int*> >();
-    test_int<forward_iterator<int*> >();
-    test_int<bidirectional_iterator<int*> >();
-    test_int<random_access_iterator<int*> >();
-    test_int<int*>();
+  test_int_array();
+  test_int_array_struct_source();
+  test_struct_array();
 
-    test_int_array();
-    test_int_array_struct_source();
-    test_struct_array();
+  test5();
+  test6();
 
-    test5();
-    test6();
+  test_bititer_with_custom_sized_types();
 
 #if TEST_STD_VER > 17
-    static_assert(test_constexpr());
+  static_assert(test_constexpr());
 #endif
 
   return 0;
diff --git a/libcxx/test/std/algorithms/alg.modifying.operations/alg.fill/ranges.fill.pass.cpp b/libcxx/test/std/algorithms/alg.modifying.operations/alg.fill/ranges.fill.pass.cpp
index 5dc375e0e8dc0d..870f05bc555143 100644
--- a/libcxx/test/std/algorithms/alg.modifying.operations/alg.fill/ranges.fill.pass.cpp
+++ b/libcxx/test/std/algorithms/alg.modifying.operations/alg.fill/ranges.fill.pass.cpp
@@ -20,9 +20,12 @@
 #include <cassert>
 #include <ranges>
 #include <string>
+#include <vector>
 
+#include "sized_allocator.h"
 #include "almost_satisfies_types.h"
 #include "test_iterators.h"
+#include "test_macros.h"
 
 template <class Iter, class Sent = sentinel_wrapper<Iter>>
 concept HasFillIt = requires(Iter iter, Sent sent) { std::ranges::fill(iter, sent, int{}); };
@@ -53,7 +56,7 @@ constexpr void test_iterators() {
     }
     {
       int a[3];
-      auto range = std::ranges::subrange(It(a), Sent(It(a + 3)));
+      auto range                = std::ranges::subrange(It(a), Sent(It(a + 3)));
       std::same_as<It> auto ret = std::ranges::fill(range, 1);
       assert(std::all_of(a, a + 3, [](int i) { return i == 1; }));
       assert(base(ret) == a + 3);
@@ -69,12 +72,47 @@ constexpr void test_iterators() {
     {
       std::array<int, 0> a;
       auto range = std::ranges::subrange(It(a.data()), Sent(It(a.data())));
-      auto ret = std::ranges::fill(range, 1);
+      auto ret   = std::ranges::fill(range, 1);
       assert(base(ret) == a.data());
     }
   }
 }
 
+// The `ranges::{fill, fill_n}` algorithms require `vector<bool, Alloc>::iterator` to satisfy the
+// `std::indirectly_writable` concept when used with `vector<bool, Alloc>`, which is only met since C++23.
+#if TEST_STD_VER >= 23
+TEST_CONSTEXPR_CXX20 void test_bititer_with_custom_sized_types() {
+  {
+    using Alloc = sized_allocator<bool, std::uint8_t, std::int8_t>;
+    std::vector<bool, Alloc> in(100, false, Alloc(1));
+    std::vector<bool, Alloc> expected(100, true, Alloc(1));
+    std::ranges::fill(in, true);
+    assert(in == expected);
+  }
+  {
+    using Alloc = sized_allocator<bool, std::uint16_t, std::int16_t>;
+    std::vector<bool, Alloc> in(200, false, Alloc(1));
+    std::vector<bool, Alloc> expected(200, true, Alloc(1));
+    std::ranges::fill(in, true);
+    assert(in == expected);
+  }
+  {
+    using Alloc = sized_allocator<bool, std::uint32_t, std::int32_t>;
+    std::vector<bool, Alloc> in(200, false, Alloc(1));
+    std::vector<bool, Alloc> expected(200, true, Alloc(1));
+    std::ranges::fill(in, true);
+    assert(in == expected);
+  }
+  {
+    using Alloc = sized_allocator<bool, std::uint64_t, std::int64_t>;
+    std::vector<bool, Alloc> in(200, false, Alloc(1));
+    std::vector<bool, Alloc> expected(200, true, Alloc(1));
+    std::ranges::fill(in, true);
+    assert(in == expected);
+  }
+}
+#endif
+
 constexpr bool test() {
   test_iterators<cpp17_output_iterator<int*>, sentinel_wrapper<cpp17_output_iterator<int*>>>();
   test_iterators<cpp20_output_iterator<int*>, sentinel_wrapper<cpp20_output_iterator<int*>>>();
@@ -94,19 +132,19 @@ constexpr bool test() {
     };
     {
       S a[5];
-      std::ranges::fill(a, a + 5, S {true});
+      std::ranges::fill(a, a + 5, S{true});
       assert(std::all_of(a, a + 5, [](S& s) { return s.copied; }));
     }
     {
       S a[5];
-      std::ranges::fill(a, S {true});
+      std::ranges::fill(a, S{true});
       assert(std::all_of(a, a + 5, [](S& s) { return s.copied; }));
     }
   }
 
   { // check that std::ranges::dangling is returned
     [[maybe_unused]] std::same_as<std::ranges::dangling> decltype(auto) ret =
-        std::ranges::fill(std::array<int, 10> {}, 1);
+        std::ranges::fill(std::array<int, 10>{}, 1);
   }
 
   { // check that std::ranges::dangling isn't returned with a borrowing range
@@ -131,6 +169,10 @@ constexpr bool test() {
     }
   }
 
+#if TEST_STD_VER >= 23
+  test_bititer_with_custom_sized_types();
+#endif
+
   return true;
 }
 
diff --git a/libcxx/test/std/algorithms/alg.modifying.operations/alg.fill/ranges.fill_n.pass.cpp b/libcxx/test/std/algorithms/alg.modifying.operations/alg.fill/ranges.fill_n.pass.cpp
index 10ff385d474281..fc3289772b886b 100644
--- a/libcxx/test/std/algorithms/alg.modifying.operations/alg.fill/ranges.fill_n.pass.cpp
+++ b/libcxx/test/std/algorithms/alg.modifying.operations/alg.fill/ranges.fill_n.pass.cpp
@@ -18,9 +18,12 @@
 #include <cassert>
 #include <ranges>
 #include <string>
+#include <vector>
 
+#include "sized_allocator.h"
 #include "almost_satisfies_types.h"
 #include "test_iterators.h"
+#include "test_macros.h"
 
 template <class Iter>
 concept HasFillN = requires(Iter iter) { std::ranges::fill_n(iter, int{}, int{}); };
@@ -48,6 +51,41 @@ constexpr void test_iterators() {
   }
 }
 
+// The `ranges::{fill, fill_n}` algorithms require `vector<bool, Alloc>::iterator` to satisfy the
+// `std::indirectly_writable` concept when used with `vector<bool, Alloc>`, which is only met since C++23.
+#if TEST_STD_VER >= 23
+TEST_CONSTEXPR_CXX20 void test_bititer_with_custom_sized_types() {
+  {
+    using Alloc = sized_allocator<bool, std::uint8_t, std::int8_t>;
+    std::vector<bool, Alloc> in(100, false, Alloc(1));
+    std::vector<bool, Alloc> expected(100, true, Alloc(1));
+    std::ranges::fill_n(std::ranges::begin(in), in.size(), true);
+    assert(in == expected);
+  }
+  {
+    using Alloc = sized_allocator<bool, std::uint16_t, std::int16_t>;
+    std::vector<bool, Alloc> in(200, false, Alloc(1));
+    std::vector<bool, Alloc> expected(200, true, Alloc(1));
+    std::ranges::fill_n(std::ranges::begin(in), in.size(), true);
+    assert(in == expected);
+  }
+  {
+    using Alloc = sized_allocator<bool, std::uint32_t, std::int32_t>;
+    std::vector<bool, Alloc> in(200, false, Alloc(1));
+    std::vector<bool, Alloc> expected(200, true, Alloc(1));
+    std::ranges::fill_n(std::ranges::begin(in), in.size(), true);
+    assert(in == expected);
+  }
+  {
+    using Alloc = sized_allocator<bool, std::uint64_t, std::int64_t>;
+    std::vector<bool, Alloc> in(200, false, Alloc(1));
+    std::vector<bool, Alloc> expected(200, true, Alloc(1));
+    std::ranges::fill_n(std::ranges::begin(in), in.size(), true);
+    assert(in == expected);
+  }
+}
+#endif
+
 constexpr bool test() {
   test_iterators<cpp17_output_iterator<int*>, sentinel_wrapper<cpp17_output_iterator<int*>>>();
   test_iterators<cpp20_output_iterator<int*>, sentinel_wrapper<cpp20_output_iterator<int*>>>();
@@ -68,7 +106,7 @@ constexpr bool test() {
     };
 
     S a[5];
-    std::ranges::fill_n(a, 5, S {});
+    std::ranges::fill_n(a, 5, S{});
     assert(std::all_of(a, a + 5, [](S& s) { return s.copied; }));
   }
 
@@ -79,6 +117,10 @@ constexpr bool test() {
     assert(std::all_of(a.begin(), a.end(), [](auto& s) { return s == "long long string so no SSO"; }));
   }
 
+#if TEST_STD_VER >= 23
+  test_bititer_with_custom_sized_types();
+#endif
+
   return true;
 }
 
diff --git a/libcxx/test/support/sized_allocator.h b/libcxx/test/support/sized_allocator.h
new file mode 100644
index 00000000000000..a877252e82962c
--- /dev/null
+++ b/libcxx/test/support/sized_allocator.h
@@ -0,0 +1,58 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef TEST_SUPPORT_SIZED_ALLOCATOR_H
+#define TEST_SUPPORT_SIZED_ALLOCATOR_H
+
+#include <cstddef>
+#include <limits>
+#include <memory>
+#include <new>
+
+#include "test_macros.h"
+
+template <typename T, typename SIZE_TYPE = std::size_t, typename DIFF_TYPE = std::ptrdiff_t>
+class sized_allocator {
+  template <typename U, typename Sz, typename Diff>
+  friend class sized_allocator;
+
+public:
+  using value_type                  = T;
+  using size_type                   = SIZE_TYPE;
+  using difference_type             = DIFF_TYPE;
+  using propagate_on_container_swap = std::true_type;
+
+  TEST_CONSTEXPR_CXX20 explicit sized_allocator(int d = 0) : data_(d)...
[truncated]

@winner245 winner245 force-pushed the fix-bitwise-logic-in-algo branch 9 times, most recently from 0f1d23a to 5349b7c Compare January 15, 2025 21:49
Copy link

github-actions bot commented Jan 15, 2025

✅ With the latest revision this PR passed the C/C++ code formatter.

@winner245 winner245 force-pushed the fix-bitwise-logic-in-algo branch 5 times, most recently from b708ad2 to ac6addf Compare January 16, 2025 04:05
Copy link
Member

@ldionne ldionne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with comments applied. If you do find that we're missing coverage and add some tests, I'd like to have another quick look before merging.

Thanks!

@winner245 winner245 force-pushed the fix-bitwise-logic-in-algo branch from 2e64e4f to 787d514 Compare January 20, 2025 20:53
Copy link
Member

@ldionne ldionne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with tiny comment applied! Thanks!

@ldionne ldionne added the pending-ci Merging the PR is only pending completion of CI label Jan 29, 2025
@winner245 winner245 force-pushed the fix-bitwise-logic-in-algo branch from 787d514 to 30a9274 Compare January 29, 2025 20:32
@ldionne ldionne merged commit cf9806e into llvm:main Feb 5, 2025
81 checks passed
@winner245 winner245 deleted the fix-bitwise-logic-in-algo branch February 5, 2025 18:21
Icohedron pushed a commit to Icohedron/llvm-project that referenced this pull request Feb 11, 2025
…orithms (llvm#122410)

This PR addresses an undefined behavior that arises when using the
`std::fill` and `std::fill_n` algorithms, as well as their ranges
counterparts `ranges::fill` and `ranges::fill_n`, with `vector<bool, Alloc>`
that utilizes a custom-sized allocator with small integral types.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
libc++ libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi. pending-ci Merging the PR is only pending completion of CI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants