-
-
Notifications
You must be signed in to change notification settings - Fork 297
Array-walking is aligned #1191
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Array-walking is aligned #1191
Conversation
This addresses one assert failure in #1185 |
This looks fine. I guess the problem here is that we were continuing to assert something that's actually not true, or otherwise properly handled by the follow-up changes after zero-copy-arrays first landed? |
Correct. I am not even sure if it was still unhandled by the end of the first PR, but somewhere after the |
Because the intervals are static, they can be calculated once.
The assumptions in this assert have long-since been corrected. Fixed-size array-walking is correct for all sizes and alignments.
131abc7
to
91122a7
Compare
I need to add an assert elsewhere, I think? |
Added appropriate debug asserts for "YES this pointer is inbounds of the array!" These will be lighter weight than certain other notorious debug asserts we had. |
Oh lol, this debug assert is wrong. |
5f2a248
to
8a5e5a1
Compare
pgrx/src/datum/array.rs
Outdated
|
||
// Make sure a previous iter didn't push out of bounds! | ||
if !is_null { | ||
debug_assert!(*ptr < array.raw.end_ptr()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that the comparison is wrong if ptr
wrapped around the address space and other such edge cases. Might not be worth giving a shit about, since handling that correctly is annoying.
Actually, I have some code in a non-work project that cares about that, so I tweaked it for pgrx (hopefully not breaking it in the process) which gives https://gist.github.com/thomcc/b2730e3b1a56c4d8cdebf29814ea2351. It can probably be simplified for our use though (we perhaps don't need the Actually, Something like this on RawArray is simpler but still sufficient.size
param), and has not been tested after I tweaked it for pgrx.
#[inline]
pub(crate) fn is_in_bounds(&self, p: *const u8) -> bool {
let base = self.ptr.as_ptr();
let mem_size = unsafe { crate::varlena::varsize_any(base.cast()) };
// Note: This intentionally underflows if `p < base`.
// If that happens, the wraparound will cause the result to
// be greater than `mem_size` (we assume that the size postgres
// gave us is reasonable, e.g. less than `isize::MAX`).
(p as usize).wrapping_sub(base as usize) <= mem_size
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you go that route, we have a couple other places where we have similar debug assertions comparing against end_ptr
, might be worth fixing them too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we land that in a separate PR? I probably want it but we should discuss it as its own beast because that raises questions about more of our codebase and this patch has already scope-creeped a bit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure. We talked out of band (I don't think it raises other questions about most of the code base, just places where we might have broken address arithmetic)
/// Array elements are [`pg_sys::varlena`] types, which are pass-by-reference | ||
pub(super) struct PassByVarlena { | ||
pub(super) align: usize, | ||
pub(super) align: Align, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Carrying a full usize
is unnecessary here and in any case makes the result that much slightly-less typesafe.
Co-authored-by: Thom Chovioloni <[email protected]>
2950c60
to
543e8b1
Compare
* Precompute fixed-size array-walking Because the intervals are static, they can be calculated once. * Remove now-unnecessary assert The assumptions in this assert have long-since been corrected. Fixed-size array-walking is correct for all sizes and alignments. * Canonize Align::pad fn Remove code duplication and use more conventional forms. Co-authored-by: Thom Chovioloni <[email protected]>
This is the final release of v0.10.0. Thanks everyone for the beta testing, pull requests, issues, and patience. As always, install `cargo-pgrx` with `cargo install cargo-pgrx --locked` and update your extension Cargo.toml files to use the `0.10.0` pgrx dependencies. This release includes support for Postgres 16RC1. Support for the previous betas has been removed. As such, a fresh `cargo pgrx init` is required. ## What's Changed Since v0.10.0-beta.4 * Fix `GetMemoryChunkContext` port by @workingjubilee in #1273 * Better error messages when `pg_config` isn't found. by @eeeebbbbrrrr in #1271 * Make `PostgresHash` also need `Eq` by @workingjubilee in #1264 * Memoize git hash and extension metadata by @levkk in #1274 * move to pg16rc1 by @eeeebbbbrrrr in #1276 * Fix bgworker template up to 0.10.0-beta.4 by @workingjubilee in #1270 ## New Contributors * @levkk made their first contribution in #1274 **Changelog**: v0.10.0-beta.4...v0.10.0 --- v0.10.0's full set of changes throughout the entire beta period are: * Postgres 16beta1 Support by @eeeebbbbrrrr in #1169 * Support building against macOS universal binaries by @clowder in #1166 * list specific versions in feature gates by @eeeebbbbrrrr in #1175 * Fix bug with converting a `pg_sys::Datum` into a `pgrx::Date` by @eeeebbbbrrrr in #1177 * Fix Arrays with leading nulls by @eeeebbbbrrrr in #1180 * Disable hello_versioned_so test by @workingjubilee in #1192 * doc: fix link broken by @yihong0618 in #1181 * fcinfo: fix incorrect length set in unsafe code by @Sasasu in #1190 * update to pg16beta2 support by @eeeebbbbrrrr in #1188 * Array-walking is aligned by @workingjubilee in #1191 * Implement PGRXSharedMemory for Deque by @feikesteenbergen in #1170 * Include security labels header by @daamien in #1189 * Fixes macos-11 tests by @BradyBonnette in #1197 * Pgcentralfoundation updates again by @eeeebbbbrrrr in #1200 * Update version to 0.10.0-beta.0 by @eeeebbbbrrrr in #1201 * Testing help by @eeeebbbbrrrr in #1203 * Type testability cleanup by @eeeebbbbrrrr in #1204 * Try to smartly propagate fs errors by @workingjubilee in #1186 * Fix issue #1209 by @eeeebbbbrrrr in #1210 * Type roundtrip tests by @eeeebbbbrrrr in #1185 * Update version to 0.10.0-beta.1 by @eeeebbbbrrrr in #1213 * Add a workaround for the pg16/homebrew/icu4c situation by @thomcc in #1206 * Add security policy by @johnrballard in #1207 * `AnyNumeric` is no longer backed by Postgres-allocated memory by @eeeebbbbrrrr in #1216 * Modularize pgrx::spi by @workingjubilee in #1219 * Stop SpiClient soundness from regressing by @workingjubilee in #1214 * Add foreign table headers by @workingjubilee in #1226 * Modularize the interior of pgrx-pg-sys by @workingjubilee in #1227 * Initial valgrind support by @thomcc in #1218 * Add support for handling SIGINT and SIGCHLD from bgworker by @JelteF in #1229 * Ignores UI tests for MUSL environments by @BradyBonnette in #1235 * Add a env flag that can be set to skip `#[pg_test]`-generated tests. by @thomcc in #1239 * Fix issue #1076: Properly handle dependency graph of `Result<T, _>` by @eeeebbbbrrrr in #1241 * Cleanup the error when cargo-pgrx version doesn't match Cargo.toml by @eeeebbbbrrrr in #1240 * Add operator and cache related api by @VoVAllen in #1242 * Addresses cargo-pgrx error reporting by @BradyBonnette in #1238 * Update version to 0.10.0-beta.2 by @eeeebbbbrrrr in #1244 * Bump cargo-metadata and clap-cargo by @thomcc in #1246 * Derive Clone for Inet by @JelteF in #1251 * Correct docs for datetime `From` impls by @workingjubilee in #1253 * Only enable line tables for profile.dev by @thomcc in #1249 * Remove references to master branch by @thomcc in #1243 * Ensure bindgen gets all the `cppflags` it needs (on macOS, anyway) by @thomcc in #1247 * update for pg16beta3 support by @eeeebbbbrrrr in #1254 * Update version to 0.10.0-beta.3 by @eeeebbbbrrrr in #1255 * Add proptest support by @workingjubilee in #1258 * Misc reformatting and typo fixes by @workingjubilee in #1260 * spi: simplify (optimize?) Datum preparation by @vrmiguel in #1256 * Assume commutation when deriving PostgresEq by @workingjubilee in #1261 * Demand Ord for PostgresOrd by @workingjubilee in #1262 * Fix pgrx install causing postgresql coredump by @Sasasu in #1263 * Update version to 0.10.0-beta.4 by @workingjubilee in #1267 ## New Contributors * @clowder made their first contribution in #1166 * @yihong0618 made their first contribution in #1181 * @Sasasu made their first contribution in #1190 * @daamien made their first contribution in #1189 * @johnrballard made their first contribution in #1207 * @VoVAllen made their first contribution in #1242 * @vrmiguel made their first contribution in #1256 **Full Changelog**: v0.9.8...v0.10.0
* Precompute fixed-size array-walking Because the intervals are static, they can be calculated once. * Remove now-unnecessary assert The assumptions in this assert have long-since been corrected. Fixed-size array-walking is correct for all sizes and alignments. * Canonize Align::pad fn Remove code duplication and use more conventional forms. Co-authored-by: Thom Chovioloni <[email protected]>
This is the final release of v0.10.0. Thanks everyone for the beta testing, pull requests, issues, and patience. As always, install `cargo-pgrx` with `cargo install cargo-pgrx --locked` and update your extension Cargo.toml files to use the `0.10.0` pgrx dependencies. This release includes support for Postgres 16RC1. Support for the previous betas has been removed. As such, a fresh `cargo pgrx init` is required. ## What's Changed Since v0.10.0-beta.4 * Fix `GetMemoryChunkContext` port by @workingjubilee in pgcentralfoundation#1273 * Better error messages when `pg_config` isn't found. by @eeeebbbbrrrr in pgcentralfoundation#1271 * Make `PostgresHash` also need `Eq` by @workingjubilee in pgcentralfoundation#1264 * Memoize git hash and extension metadata by @levkk in pgcentralfoundation#1274 * move to pg16rc1 by @eeeebbbbrrrr in pgcentralfoundation#1276 * Fix bgworker template up to 0.10.0-beta.4 by @workingjubilee in pgcentralfoundation#1270 ## New Contributors * @levkk made their first contribution in pgcentralfoundation#1274 **Changelog**: pgcentralfoundation/pgrx@v0.10.0-beta.4...v0.10.0 --- v0.10.0's full set of changes throughout the entire beta period are: * Postgres 16beta1 Support by @eeeebbbbrrrr in pgcentralfoundation#1169 * Support building against macOS universal binaries by @clowder in pgcentralfoundation#1166 * list specific versions in feature gates by @eeeebbbbrrrr in pgcentralfoundation#1175 * Fix bug with converting a `pg_sys::Datum` into a `pgrx::Date` by @eeeebbbbrrrr in pgcentralfoundation#1177 * Fix Arrays with leading nulls by @eeeebbbbrrrr in pgcentralfoundation#1180 * Disable hello_versioned_so test by @workingjubilee in pgcentralfoundation#1192 * doc: fix link broken by @yihong0618 in pgcentralfoundation#1181 * fcinfo: fix incorrect length set in unsafe code by @Sasasu in pgcentralfoundation#1190 * update to pg16beta2 support by @eeeebbbbrrrr in pgcentralfoundation#1188 * Array-walking is aligned by @workingjubilee in pgcentralfoundation#1191 * Implement PGRXSharedMemory for Deque by @feikesteenbergen in pgcentralfoundation#1170 * Include security labels header by @daamien in pgcentralfoundation#1189 * Fixes macos-11 tests by @BradyBonnette in pgcentralfoundation#1197 * Pgcentralfoundation updates again by @eeeebbbbrrrr in pgcentralfoundation#1200 * Update version to 0.10.0-beta.0 by @eeeebbbbrrrr in pgcentralfoundation#1201 * Testing help by @eeeebbbbrrrr in pgcentralfoundation#1203 * Type testability cleanup by @eeeebbbbrrrr in pgcentralfoundation#1204 * Try to smartly propagate fs errors by @workingjubilee in pgcentralfoundation#1186 * Fix issue pgcentralfoundation#1209 by @eeeebbbbrrrr in pgcentralfoundation#1210 * Type roundtrip tests by @eeeebbbbrrrr in pgcentralfoundation#1185 * Update version to 0.10.0-beta.1 by @eeeebbbbrrrr in pgcentralfoundation#1213 * Add a workaround for the pg16/homebrew/icu4c situation by @thomcc in pgcentralfoundation#1206 * Add security policy by @johnrballard in pgcentralfoundation#1207 * `AnyNumeric` is no longer backed by Postgres-allocated memory by @eeeebbbbrrrr in pgcentralfoundation#1216 * Modularize pgrx::spi by @workingjubilee in pgcentralfoundation#1219 * Stop SpiClient soundness from regressing by @workingjubilee in pgcentralfoundation#1214 * Add foreign table headers by @workingjubilee in pgcentralfoundation#1226 * Modularize the interior of pgrx-pg-sys by @workingjubilee in pgcentralfoundation#1227 * Initial valgrind support by @thomcc in pgcentralfoundation#1218 * Add support for handling SIGINT and SIGCHLD from bgworker by @JelteF in pgcentralfoundation#1229 * Ignores UI tests for MUSL environments by @BradyBonnette in pgcentralfoundation#1235 * Add a env flag that can be set to skip `#[pg_test]`-generated tests. by @thomcc in pgcentralfoundation#1239 * Fix issue pgcentralfoundation#1076: Properly handle dependency graph of `Result<T, _>` by @eeeebbbbrrrr in pgcentralfoundation#1241 * Cleanup the error when cargo-pgrx version doesn't match Cargo.toml by @eeeebbbbrrrr in pgcentralfoundation#1240 * Add operator and cache related api by @VoVAllen in pgcentralfoundation#1242 * Addresses cargo-pgrx error reporting by @BradyBonnette in pgcentralfoundation#1238 * Update version to 0.10.0-beta.2 by @eeeebbbbrrrr in pgcentralfoundation#1244 * Bump cargo-metadata and clap-cargo by @thomcc in pgcentralfoundation#1246 * Derive Clone for Inet by @JelteF in pgcentralfoundation#1251 * Correct docs for datetime `From` impls by @workingjubilee in pgcentralfoundation#1253 * Only enable line tables for profile.dev by @thomcc in pgcentralfoundation#1249 * Remove references to master branch by @thomcc in pgcentralfoundation#1243 * Ensure bindgen gets all the `cppflags` it needs (on macOS, anyway) by @thomcc in pgcentralfoundation#1247 * update for pg16beta3 support by @eeeebbbbrrrr in pgcentralfoundation#1254 * Update version to 0.10.0-beta.3 by @eeeebbbbrrrr in pgcentralfoundation#1255 * Add proptest support by @workingjubilee in pgcentralfoundation#1258 * Misc reformatting and typo fixes by @workingjubilee in pgcentralfoundation#1260 * spi: simplify (optimize?) Datum preparation by @vrmiguel in pgcentralfoundation#1256 * Assume commutation when deriving PostgresEq by @workingjubilee in pgcentralfoundation#1261 * Demand Ord for PostgresOrd by @workingjubilee in pgcentralfoundation#1262 * Fix pgrx install causing postgresql coredump by @Sasasu in pgcentralfoundation#1263 * Update version to 0.10.0-beta.4 by @workingjubilee in pgcentralfoundation#1267 ## New Contributors * @clowder made their first contribution in pgcentralfoundation#1166 * @yihong0618 made their first contribution in pgcentralfoundation#1181 * @Sasasu made their first contribution in pgcentralfoundation#1190 * @daamien made their first contribution in pgcentralfoundation#1189 * @johnrballard made their first contribution in pgcentralfoundation#1207 * @VoVAllen made their first contribution in pgcentralfoundation#1242 * @vrmiguel made their first contribution in pgcentralfoundation#1256 **Full Changelog**: pgcentralfoundation/pgrx@v0.9.8...v0.10.0
The alignment-handling code has been correct for some time.
Remove the questioning assert, extract the align-up pattern into a function on
layout::Align
, and precompute the fixed-size offset to use for fixed-size elements in arrays.