rpms/ruby
6
0
Fork 0
Code Issues Pull Requests Releases Activity
f77e9e6b4e
ruby/ruby-3.2.0-Detect-compaction-support-during-runtime.patch
Jarek Prokop 7993206359 Undefine GC compaction and related methods for ppc64le. 
The following falcon source:
<https://github.com/socketry/falcon/blob/v0.42.3/lib/falcon/command/serve.rb#L153-L155>
contains the following snippet:
~~~
if GC.respond_to?(:compact)
	GC.compact
end
~~~

The `if` guard for this feature is recommended from documentation since Ruby >= 3.2:
<https://docs.ruby-lang.org/en/3.2/GC.html#method-c-compact>:
```
 To test whether GC compaction is supported, use the idiom:
 ~~~
 GC.respond_to?(:compact)
 ~~~
```

Gems make use of this idiom in various situations. However on Ruby 3.0
without this patch the `GC.respond_to?` method will return true for
`GC.compact` and compaction related methods even though an attempt to call
them will raise "NotImplementedError" exception.

We observe this behavior due to the methods being implemented and the
NotImplementedError is not returned because methods are not implemented,
but because it is a runtime behavior from inside the called methods.

The applied patchset undefines the methods in C code to ensure even the
`GC.respond_to?` will return false and accomodate libraries that call
`GC.compact` like falcon does:
<https://github.com/socketry/falcon/blob/v0.42.3/lib/falcon/command/serve.rb#L153-L155>

The methods are undefined on build time on affected platforms that do
not support compaction. Currently this is only affecting ppc64le.

The important portion of the patch undefines the `GC.compact` and other
methods, so we can expect the following on ppc64le architecture:
Before:
~~~
$ ruby -e 'p GC.respond_to? :compact'
true
~~~

After:
~~~
$ ruby -e 'p GC.respond_to? :compact'
false
~~~

As long as Ruby gems guard the call to `GC.compact` correctly, there
should no longer be a situation where GC reports compaction methods
as implemented even though it will throw an exception when trying to use the methods.

Most importantly, this is only an issue with ppc64le. The other
architectures such as x86_64, aarch64, s390x should continue to be able
to compact. This is guarded on compile-time.

However the patches touch the source for pre-generated files.
Mainly meaning the file `gc.rb` that is the source for the
`gc.rbinc` and `miniprelude.c` need to be re-generated.
The files use Ruby to generate the C counterparts. To prevent cyclic dependency,
they are re-generated and patches are created against the old version.
The approach to regenerate the files for 3.0 is the same as for 3.1
with the exception of also having to first patch with the file
`ruby-3.1.0-Use-mmap-for-allocating-heap-pages-in-the-GC.patch`,
since the subsequent patches are made on top of that patch.

The mmap patch changes parts of gc.c, that the required patch
also changes, so it had to be backported on top of that patch.

The sequence of actions is as follows:
~~~
tar -Jxvf ./ruby-3.0.7.tar.xz
git clone https://github.com/ruby/ruby.git
cd ruby && git checkout v3_0_7
patch -p1 < ../ruby-3.1.0-Use-mmap-for-allocating-heap-pages-in-the-GC.patch
patch -p1 < ../ruby-3.2.0-define-unsupported-gc-compaction-methods-as-rb_f_notimplement.patch
./autogen.sh && ./configure
make gc.rbinc miniprelude.c
cd ..
diff -u {ruby-3.0.7,ruby}/gc.rbinc > ruby-3.2.0-define-unsupported-gc-compaction-methods_generated-files.patch
diff -u {ruby-3.0.7,ruby}/miniprelude.c >> ruby-3.2.0-define-unsupported-gc-compaction-methods_generated-files.patch
~~~
Firstly we unpact the upstream tar archive, which creates `ruby-3.0.7`
directory.
Then we clone the upstream git repository and check out the Ruby version
tag that is source for the tar archive which creates `ruby` repository.

The `ruby-3.0.7` directory contains the pre-generated files we will use as
the original for the patch.

In the `ruby` repository we apply the patches mentioned in the code
snippet.

Then, generate configure script with autogen.sh, and we create the
gc.rbinc and miniprelude.c files that are generated.

Then create the patch for the generated files. `diff` is used as the
generated files are not tracked via git in upstream, and the diff is
executed against the expanded upstream tar archive that does not ship a
git repository with it. While it would be possible to `git add` the
files before patching them, this ensures that the files are applicable
without a problem on top of the archive.

This commit is partially backported from Ruby 3.1's Fedora counterparts:
b7b5473796
ca94aff023
649a6e3083
instead of re-doing the entire patchset from 3.2 branch back to 3.0.

There is less disruption in GC related code when comparing 3.0 to 3.1
than there is 3.1 to 3.2. Therefore the upstream sources for the patches
were first applied in the Ruby upstream git repository on top of tag v3_1_2.
Namely, the following is the patch source:
https://github.com/ruby/ruby/pull/5934
0de1495f35
0c36ba5319
available in Fedora as part of:
649a6e3083

The version of Ruby that is the version when the mentioned Fedora commits
were introduced. And then cherry picked on top of patch
"ruby-3.1.0-Use-mmap-for-allocating-heap-pages-in-the-GC.patch"
applied in v3_0_7 tag that is present here in downstream as the
difference brought by the patch is significant enough to disrupt
patch application. Git allows easier resolution of the patches here.

In short,
1. Take the upstream compaction patches, cherry-pick them on top of v3_1_2 so that
the changes are present in the git tree, as they have not been backported
to 3.1 upstream.
2. Apply the "ruby-3.1.0-Use-mmap-for-allocating-heap-pages-in-the-GC.patch" on top of v3_0_7.
3. Cherry pick the 3.1 patches to v3_0_7 and resolve differences.

There might have been a few sections where git marked changes from both
patches as a conflict to resolve.
I preferred the approach where, despite the checks introduced
from the 3.1 branch patches should be enough to guard against bad
behavior around compaction, the "mmap" patch's checks are retained for safety.
One of the reasons are that the checks are also present outside the
compact methods in other parts of `gc.c` code.

References for the specific Fedora patches:
b7b5473796
649a6e3083
<b7b5473796>
<649a6e3083>
Uptream bug: <https://bugs.ruby-lang.org/issues/18779>
Upstream PR: <https://github.com/ruby/ruby/pull/5934>

ca94aff023
<ca94aff023>
Related upstream issue: <https://bugs.ruby-lang.org/issues/18829>
<https://github.com/ruby/ruby/pull/6019>
<2c19086323>

Resolves: RHEL-83136
2025-03-27 15:03:49 +01:00

284 lines
9.5 KiB
Diff
Raw Blame History

From ec992161c3ca346061fb5b1b9c44402a03105b79 Mon Sep 17 00:00:00 2001
From: Jarek Prokop <jprokop@redhat.com>
Date: Wed, 22 Jun 2022 01:03:49 +0200
Subject: [PATCH] Detect compaction support during runtime.
The patch is created by backporting 3 commits from
the upstream Ruby master branch in the chronological order below.
https://github.com/ruby/ruby/commit/52d42e702375446746164a0251e1a10bce813b78
https://github.com/ruby/ruby/commit/79eaaf2d0b641710613f16525e4b4c439dfe854e
https://github.com/ruby/ruby/commit/2c190863239bee3f54cfb74b16bb6ea4cae6ed20
== How to create this patch ==
Download Ruby source code.
```
$ git clone https://github.com/ruby/ruby.git
$ cd ruby
```
First create a commit squashed from the 3 commits above.
Checkout the second commmit above, and create a temporary branch.
```
$ git checkout 79eaaf2d0b641710613f16525e4b4c439dfe854e
$ git checkout -b wip/detect-compaction-runtime-tmp
```
Cherry pick the third commit on the second commit.
```
$ git cherry-pick 2c190863239bee3f54cfb74b16bb6ea4cae6ed20
```
Squash the last 3 commits on the branch.
```
$ git rebase -i 2223eb082afa6d05321b69df783d4133b9aacba6
```
Then checkout Ruby 3.1.2 branch.
Create a new branch.
Merge the Fedora Ruby's
ruby-3.2.0-define-unsupported-gc-compaction-methods-as-rb_f_notimplement.patch.
```
$ git checkout v3_1_2
$ git checkout -b wip/detect-compaction-runtime
$ patch -p1 <
~/fed/ruby/ruby-3.2.0-define-unsupported-gc-compaction-methods-as-rb_f_notimplement.patch
$ git add gc.c gc.rb test/ruby/test_gc_compact.rb
$ git commit
```
Merge the squashed one commit on the
`wip/detect-compaction-runtime-tmp` branch
into the `wip/detect-compaction-runtime` branch.
```
$ git cherry-pick <the squashed commit above>
```
Fix conflicts seeing the difference by `git show <the squashed commit
above>`
on another terminal.
```
$ vi gc.c
$ git add gc.c
$ git commit
```
== Notes for the patch ==
```
+# define GC_COMPACTION_SUPPORTED (GC_CAN_COMPILE_COMPACTION && USE_MMAP_ALIGNED_ALLOC)
```
We use the USE_MMAP_ALIGNED_ALLOC instead of HEAP_PAGE_ALLOC_USE_MMAP on
the line above. Because while the Ruby on the master branch replaced the
USE_MMAP_ALIGNED_ALLOC with HEAP_PAGE_ALLOC_USE_MMAP, Ruby 3.1.2 doesn't.
See <https://github.com/ruby/ruby/commit/fe21b7794af0cdb7ebd502e2c0da38c68fd89839>.
```
+ rb_define_singleton_method(rb_mGC, "verify_compaction_references", gc_verify_compaction_references, -1);
```
We added the line in the case that GC_COMPACTION_SUPPORTED is true.
Because while the Ruby on the master branch defines the
GC.verify_compaction_references in the gc.rb in
the case that GC_COMPACTION_SUPPORTED is true, Ruby 3.1.2
doesn't define it in the gc.rb.
See <https://github.com/ruby/ruby/commit/b96a3a6fd2093e1dbea5491c002da515652dd347>.
```
+ OPT(GC_COMPACTION_SUPPORTED);
```
We added the line to expose the C macro to Ruby level.
In Ruby the macro existance can then be checked like so:
```Ruby
GC::OPTS.include?("GC_COMPACTION_SUPPORTED")
```
It will return `true` if the GC_COMPACTION_SUPPORTED evaluates to `true` on the
C level, `false` otherwise.
See <https://github.com/ruby/ruby/blob/b96a3a6fd2093e1dbea5491c002da515652dd347/gc.c#L14091>
== Original commit messages ==
This is a combination of 3 commits.
This is the 1st commit message:
~~~
Rename GC_COMPACTION_SUPPORTED
Naming this macro GC_COMPACTION_SUPPORTED is misleading because it
only checks whether compaction is supported at compile time.
[Bug #18829]
~~~
This is the commit message #2:
~~~
Include runtime checks for compaction support
Commit 0c36ba53192c5a0d245c9b626e4346a32d7d144e changed GC compaction
methods to not be implemented when not supported. However, that commit
only does compile time checks (which currently only checks for WASM),
but there are additional compaction support checks during run time.
This commit changes it so that GC compaction methods aren't defined
during run time if the platform does not support GC compaction.
[Bug #18829]
~~~
This is the commit message #3:
~~~
Suppress code unused unless GC_CAN_COMPILE_COMPACTION
~~~
---
gc.c | 58 ++++++++++++++++++++++++++++++++++++++++++++++------------
1 file changed, 46 insertions(+), 12 deletions(-)
diff --git a/gc.c b/gc.c
index 6a5c739cc4..a3e40d9aac 100644
--- a/gc.c
+++ b/gc.c
@@ -4632,7 +4632,25 @@ gc_unprotect_pages(rb_objspace_t *objspace, rb_heap_t *heap)
static void gc_update_references(rb_objspace_t * objspace, rb_heap_t *heap);
static void invalidate_moved_page(rb_objspace_t *objspace, struct heap_page *page);
-static void read_barrier_handler(intptr_t address)
+#ifndef GC_CAN_COMPILE_COMPACTION
+#if defined(__wasi__) /* WebAssembly doesn't support signals */
+# define GC_CAN_COMPILE_COMPACTION 0
+#else
+# define GC_CAN_COMPILE_COMPACTION 1
+#endif
+#endif
+
+#if defined(__MINGW32__) || defined(_WIN32)
+# define GC_COMPACTION_SUPPORTED 1
+#else
+/* If not MinGW, Windows, or does not have mmap, we cannot use mprotect for
+ * the read barrier, so we must disable compaction. */
+# define GC_COMPACTION_SUPPORTED (GC_CAN_COMPILE_COMPACTION && USE_MMAP_ALIGNED_ALLOC)
+#endif
+
+#if GC_CAN_COMPILE_COMPACTION
+static void
+read_barrier_handler(uintptr_t address)
{
VALUE obj;
rb_objspace_t * objspace = &rb_objspace;
@@ -4651,6 +4669,7 @@ static void read_barrier_handler(intptr_t address)
}
RB_VM_LOCK_LEAVE();
}
+#endif
#if defined(_WIN32)
static LPTOP_LEVEL_EXCEPTION_FILTER old_handler;
@@ -8562,6 +8581,8 @@ gc_start_internal(rb_execution_context_t *ec, VALUE self, VALUE full_mark, VALUE
/* For now, compact implies full mark / sweep, so ignore other flags */
if (RTEST(compact)) {
+ GC_ASSERT(GC_COMPACTION_SUPPORTED);
+
/* If not MinGW, Windows, or does not have mmap, we cannot use mprotect for
* the read barrier, so we must disable compaction. */
#if !defined(__MINGW32__) && !defined(_WIN32)
@@ -8732,7 +8753,7 @@ gc_move(rb_objspace_t *objspace, VALUE scan, VALUE free)
return (VALUE)src;
}
-#if GC_COMPACTION_SUPPORTED
+#if GC_CAN_COMPILE_COMPACTION
static int
compare_free_slots(const void *left, const void *right, void *dummy)
{
@@ -9412,7 +9433,7 @@ gc_update_references(rb_objspace_t * objspace, rb_heap_t *heap)
static VALUE type_sym(size_t type);
-#if GC_COMPACTION_SUPPORTED
+#if GC_CAN_COMPILE_COMPACTION
/*
* call-seq:
* GC.latest_compact_info -> {:considered=>{:T_CLASS=>11}, :moved=>{:T_CLASS=>11}}
@@ -9453,7 +9474,7 @@ gc_compact_stats(VALUE self)
# define gc_compact_stats rb_f_notimplement
#endif
-#if GC_COMPACTION_SUPPORTED
+#if GC_CAN_COMPILE_COMPACTION
static void
root_obj_check_moved_i(const char *category, VALUE obj, void *data)
{
@@ -9532,7 +9553,7 @@ gc_compact(VALUE self)
# define gc_compact rb_f_notimplement
#endif
-#if GC_COMPACTION_SUPPORTED
+#if GC_CAN_COMPILE_COMPACTION
/*
* call-seq:
* GC.verify_compaction_references(toward: nil, double_heap: false) -> hash
@@ -10056,7 +10077,7 @@ gc_disable(rb_execution_context_t *ec, VALUE _)
return rb_gc_disable();
}
-#if GC_COMPACTION_SUPPORTED
+#if GC_CAN_COMPILE_COMPACTION
/*
* call-seq:
* GC.auto_compact = flag
@@ -10077,6 +10098,7 @@ gc_set_auto_compact(VALUE _, VALUE v)
rb_raise(rb_eNotImpError, "Automatic compaction isn't available on this platform");
}
#endif
+ GC_ASSERT(GC_COMPACTION_SUPPORTED);
ruby_enable_autocompact = RTEST(v);
return v;
@@ -10085,7 +10107,8 @@ gc_set_auto_compact(VALUE _, VALUE v)
# define gc_set_auto_compact rb_f_notimplement
#endif
-#if GC_COMPACTION_SUPPORTED
+
+#if GC_CAN_COMPILE_COMPACTION
/*
* call-seq:
* GC.auto_compact -> true or false
@@ -12926,11 +12949,21 @@ Init_GC(void)
rb_define_singleton_method(rb_mGC, "malloc_allocated_size", gc_malloc_allocated_size, 0);
rb_define_singleton_method(rb_mGC, "malloc_allocations", gc_malloc_allocations, 0);
#endif
- rb_define_singleton_method(rb_mGC, "compact", gc_compact, 0);
- rb_define_singleton_method(rb_mGC, "auto_compact", gc_get_auto_compact, 0);
- rb_define_singleton_method(rb_mGC, "auto_compact=", gc_set_auto_compact, 1);
- rb_define_singleton_method(rb_mGC, "latest_compact_info", gc_compact_stats, 0);
- rb_define_singleton_method(rb_mGC, "verify_compaction_references", gc_verify_compaction_references, -1);
+ if (GC_COMPACTION_SUPPORTED) {
+ rb_define_singleton_method(rb_mGC, "compact", gc_compact, 0);
+ rb_define_singleton_method(rb_mGC, "auto_compact", gc_get_auto_compact, 0);
+ rb_define_singleton_method(rb_mGC, "auto_compact=", gc_set_auto_compact, 1);
+ rb_define_singleton_method(rb_mGC, "latest_compact_info", gc_compact_stats, 0);
+ rb_define_singleton_method(rb_mGC, "verify_compaction_references", gc_verify_compaction_references, -1);
+ }
+ else {
+ rb_define_singleton_method(rb_mGC, "compact", rb_f_notimplement, 0);
+ rb_define_singleton_method(rb_mGC, "auto_compact", rb_f_notimplement, 0);
+ rb_define_singleton_method(rb_mGC, "auto_compact=", rb_f_notimplement, 1);
+ rb_define_singleton_method(rb_mGC, "latest_compact_info", rb_f_notimplement, 0);
+ /* When !GC_COMPACTION_SUPPORTED, this method is not defined in gc.rb */
+ rb_define_singleton_method(rb_mGC, "verify_compaction_references", rb_f_notimplement, -1);
+ }
#if GC_DEBUG_STRESS_TO_CLASS
rb_define_singleton_method(rb_mGC, "add_stress_to_class", rb_gcdebug_add_stress_to_class, -1);
@@ -12954,6 +12987,7 @@ Init_GC(void)
OPT(MALLOC_ALLOCATED_SIZE);
OPT(MALLOC_ALLOCATED_SIZE_CHECK);
OPT(GC_PROFILE_DETAIL_MEMORY);
+ OPT(GC_COMPACTION_SUPPORTED);
#undef OPT
OBJ_FREEZE(opts);
}
Reference in New Issue View Git Blame Copy Permalink