number of CPUs will never be set to less than 1
this now outputs build flag overrides to be used with %make_build etc.
add documentation
Signed-off-by: Michel Alexandre Salim <salimma@fedoraproject.org>
A number of cases have cropped up where things like documentation
builds fail with out of tree builds because tools like Doxygen like
encoding the build path into the documentation metadata.
This causes failures in Koji and other places where noarch documentation
subpackages from archful packages fail because they fail the comparison
check due to this.
This allows us to maintain our own BuildRoot Policy scripts in an easier way.
This change needs to be coordinated with the addition of the files to python-srpm-macros.
redhat-rpm-config requires python-srpm-macros, so no significant change is expected for the packagers.
This also moves the Python BRPs to the end of the list which should be fine.
The script was dropped in rpm 4.17.0 beta1, so we can't very well
try to use it on builds. For now I'm intentionally dropping just the
macro definition: in case the Python community wants to bring it
back, all they need to do is revive the script and add the macro
definition for it.
On armv7hl, enabling annobin can in some cases lead to corrupt unwind
information in generated object files:
https://bugzilla.redhat.com/show_bug.cgi?id=1951492
The root-cause of this is still unclear and until this is sorted out,
it is best to disable annobin on the architecture.
If the file we are trying to mangle a shebang in has
0 lines (as reported by `wc`) `read` command fails
to read the first line and the script fails silently.
Text files without newlines should not be executable as there
is no way for them to contain a shebang.
Avoid using the full path and instead rely on PATH being correctly set
up to find the executable.
This fixes byte compilation for python2.7 when doing flatpak module
builds where python2.7 can be in either /usr/bin or /app/bin, depending
on how it's compiled.
Some packages require being able to redefine the compiler variables
set to add additional base arguments that must be used everywhere.
This change makes it possible for that to work correctly.
The kernel doesn't do a complete build for all arches. And example is i686 in Fedora, where only kernel-headers is built.
kernel-srpm-macros provides %kernel_arches so that the various packages have a consistent way to check for this.
A new script brp-fix-pyc-reproducibility creates an opt-in way of how to fix
problems with the reproducibility of byte-compiled Python files. The script
uses marshalparser [0] which currently doesn't provide solutions for all issues
but can fix at least problems with reference flags. For more info see
this Bugzilla [1].
If you want to use this new feature, you need to define
`%py_reproducible_pyc_path` to specify a path you want to fix `.pyc`
files in (recursively) and build-require /usr/bin/marshalparser.
if you forget to build-require the parser. The error message is:
```
+ /usr/lib/rpm/redhat/brp-python-bytecompile '' 1 0
Bytecompiling .py files below /builddir/build/BUILDROOT/tldr-0.5-2.fc33.x86_64/usr/lib/python3.9 using /usr/bin/python3.9
+ /usr/lib/rpm/redhat/brp-fix-pyc-reproducibility /builddir/build/BUILDROOT/tldr-0.5-2.fc33.x86_64
ERROR: If %py_reproducible_pyc_path is defined, you have to also BuildRequire: /usr/bin/marshalparser !
error: Bad exit status from /var/tmp/rpm-tmp.UUJr4v (%install)
```
A build fails if the parser is not able to parse any of the `.pyc` files.
And finally, if a build is properly configured it produces fixed `.pyc` files.
Currently, `.pyc` files in the tldr package contain a lot of unused reference flags:
```
$ dnf install -y tldr
$ marshalparser --unused /usr/lib/python3.9/site-packages/__pycache__/tldr.cpython-39.pyc
… long output …
190 - Flag_ref(byte=9610, type='TYPE_SHORT_ASCII_INTERNED', content=b'init', usages=0)
191 - Flag_ref(byte=9633, type='TYPE_SHORT_ASCII_INTERNED', content=b'source', usages=0)
192 - Flag_ref(byte=9651, type='TYPE_SHORT_ASCII_INTERNED', content=b'argv', usages=0)
193 - Flag_ref(byte=9657, type='TYPE_SHORT_ASCII_INTERNED', content=b'print_help', usages=0)
194 - Flag_ref(byte=9669, type='TYPE_SHORT_ASCII_INTERNED', content=b'stderr', usages=0)
195 - Flag_ref(byte=9682, type='TYPE_SHORT_ASCII_INTERNED', content=b'parse_args', usages=0)
196 - Flag_ref(byte=9737, type='TYPE_SHORT_ASCII_INTERNED', content=b'encode', usages=0)
197 - Flag_ref(byte=9782, type='TYPE_SHORT_ASCII_INTERNED', content=b'parser', usages=0)
198 - Flag_ref(byte=9790, type='TYPE_SHORT_ASCII_INTERNED', content=b'options', usages=0)
199 - Flag_ref(byte=9799, type='TYPE_SHORT_ASCII_INTERNED', content=b'rest', usages=0)
200 - Flag_ref(byte=9821, type='TYPE_SHORT_ASCII_INTERNED', content=b'result', usages=0)
202 - Flag_ref(byte=10022, type='TYPE_SHORT_ASCII_INTERNED', content=b'__main__', usages=0)
203 - Flag_ref(byte=10102, type='TYPE_SHORT_ASCII_INTERNED', content=b'argparse', usages=0)
204 - Flag_ref(byte=10433, type='TYPE_SHORT_ASCII_INTERNED', content=b'__name__', usages=0)
205 - Flag_ref(byte=10463, type='TYPE_SHORT_ASCII_INTERNED', content=b'<module>', usages=0)
```
This new feature fixes them:
```
$ marshalparser --unused /usr/lib/python3.9/site-packages/__pycache__/tldr.cpython-39.pyc
<empty output>
```
[0] https://github.com/fedora-python/marshalparser
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1686078
This cuts the kernel provide generation time from ~33s to 2.5s on my laptop.
Tighten the path matching rule a bit while at it - it doesn't matter that
much with parametric generator but there's no point looking at entries
we don't generate dependencies on.
Resolves: rhbz#1595265
The problem this change is intended to solve is with how `real_libdir`
is calculated. Let's assume we want to recursively byte-compile all
`*.py` files in
`/builddir/build/BUILDROOT/python-scales-1.0.9-250.fc32.x86_64/usr/lib/python3.8`.
Then, `real_libdir` is this path without `$RPM_BUILD_ROOT` with
the filename at the end which displays in the error message like this:
```
Bytecompiling .py files below /builddir/build/BUILDROOT/python-scales-1.0.9-250.fc32.x86_64/usr/lib/python3.8 using /usr/bin/python3.8
*** Error compiling '/builddir/build/BUILDROOT/python-scales-1.0.9-250.fc32.x86_64/usr/lib/python3.8/site-packages/greplin/bar.py'...
File "/usr/lib/python3.8/bar.py", line 1
import sin from math
^
SyntaxError: invalid syntax
```
`/usr/lib/python3.8/bar.py` is obviously wrong.
One of the new features of the `compileall2` module (which will
be available in stdlib in Python 3.9) is that the path byte-compiled to
`*.pyc` files is calculated for each file. This means that by using
`-s` and `-p` we can strip `$RPM_BUILD_ROOT` and prepend `/` for each
file individually which will fix the problem.
```
Bytecompiling .py files below /builddir/build/BUILDROOT/python-scales-1.0.9-250.fc32.x86_64/usr/lib/python3.8 using /usr/bin/python3.8
*** Error compiling '/builddir/build/BUILDROOT/python-scales-1.0.9-250.fc32.x86_64/usr/lib/python3.8/site-packages/greplin/bar.py'...
File "/usr/lib/python3.8/site-packages/greplin/bar.py", line 1
import sin from math
^
SyntaxError: invalid syntax
```
This change has an effect only for Python >= 3.4.
If %source_date_epoch_from_changelog is true, RPM can set the SOURCE_DATE_EPOCH
environment variable to the timestamp of the topmost changelog entry. The
SOURCE_DATE_EPOCH can be in turn used by various projects to override otherwise
dynamically generated timestamps.
E.g. this might help to have stable timestamps in generated
documentation etc.
Rpm 4.15 removes various language-specific macros. Python side is
already covered by the versioned python macros but this is not the
case with Perl, macros. Add them here temporarily to avoid breaking
the world, but these really belong to perl-macros or such.
Once upon a time these did differ (for various bad reasons) but the
version in rpm has been identical to ours for many years, lets shed
some old baggage.
– permit extraction of multiple archives in a single specfile
– %forgemeta, %forgesetup, %forgeautosetup: add a “-z <number>” switch to
select a specific set of rpm variables
(for example forgeurl<number> and version<number>)
– %forgemeta, %forgesetup: add a “-a” switch to process all sets in one go
(makes no sense in forgeautosetup, as you need so select specific patches)
– %forgemeta: deprecate the “-u” switch, use “-z” it’s better
(“-u” was awkward and mainly used by the %gometa macro. %gometa will now
call the lua code directly)
– %forgesetup: use “-v” for verbose processing, be quiet by default, drop “-q”
(align with %forgemeta, %forgeautosetup and %autosetup)
– %forgesetup, %forgeautosetup: only pass flags that make sense to
%setup/%autosetup; reorder to match what works in el7
– factor out complex or common lua code in separate lua modules, to allow:
– code reuse in other macros without cut and pasting
– direct lua routine invocation from other macros without going through a
rpm macro
– rpm syntax errors that point to a line in an actual lua file
– %forgemeta: refactor the logic to drop as much forge-specific code as
possible, use a single logic flow with tables of constants
– %forgemeta: export more computed info in rpm variables, such as the
%{_builddir} subdirectory an archive was extracted to (lifesaver when
processing multiple archives)
– %forgemeta: prepend secondary distprefixes with .S, to make clear they do
not apply to the main archive
– %forgemeta: add versionx to secondary distprefixes, if relevant
– %forgemeta: make tar.bz2 the default archive format
Caveats:
– forge services implement full-release downloads via tags. However the actual
syntax of such tags is not standardised. If the macro does not guess the
correct tag an upstream uses for a specific release, you will need to
set the tag value explicitly.
– GitHub lets upstreams move their projects to new URLs, keeping the old URL
active. It all works transparently *except* the top directory inside
generated archives always matches the new project name (even when accessed
by compatibility URLs, and even for releases that antedate the renaming).
Therefore, if macro processing of a GitHub archive suddenly fails, start by
checking if upstream didn’t rename itself.
Multicall usage example (with “-a”)
– to process a specific bloc in one of the macros use “-z <suffix>”
– suffix 0 and no suffix are synonyms
– therefore, calling the macros without “-a” or “-z” just works for the
general use case when you have a single archive to process
– caveat: forge services implement full-release
%<--
%global forgeurl0 https://gitlab.com/osslugaru/lugaru
Version: 1.2
%global forgeurl1 https://gitlab.com/osslugaru/lugaru
%global tag1 1.1
%global forgeurl2 https://gitlab.com/osslugaru/lugaru
%global commit2 68488b0a11df90dca703c67e5592b93c6a269957
%global forgeurl3 https://gitlab.com/osslugaru/lugaru
%global branch3 v1.1
%global forgeurl4 https://github.com/google/trillian
%global version4 1.0.8
%global forgeurl5 https://github.com/kubernetes/apiextensions-apiserver
%global version5 1.9.6
%global tag5 kubernetes-%{version5}
%global forgeurl6 https://github.com/jdbranham/grafana-diagram
%global version6 1.3
%global commit6 440689793ab6da82019c5ee43b49438dfef976d5
%global forgeurl7 https://github.com/rethinkdb/rethinkdb-go
%global version7 1.4.2
%global branch7 v1
%global forgeurl8 https://code.googlesource.com/gocloud
%global version8 0.20.0
%global forgeurl9 https://code.googlesource.com/gocloud
%global tag9 v0.27.0
%global forgeurl10 https://code.googlesource.com/google-api-go-client
%global commit10 24928b980e6919be4c72647aacd53ebcbb8c4bab
%global version10 0
%global forgeurl11 https://code.googlesource.com/google-api-go-client
%global branch11 dartman
%global forgeurl12 https://bitbucket.org/nielsenb/pdfocr
%global version12 0.3.0
%global commit12 4f5750d202d33267094621630836f1215a5efa66
%global forgeurl13 https://bitbucket.org/nielsenb/pdfocr
%global version13 0.1.4
%global tag13 v0.1.4
%global commit13 c0359843a3420769940e12019ebd68891a053bd8
%global forgeurl14 https://bitbucket.org/creachadair/shell
%global commit14 3dcd505a7ca5845388111724cc2e094581e92cc6
%global forgeurl15 https://bitbucket.org/kirbyvisp/vdjpuzzle2/
%global branch15 js-edits
%global commit15 36a3850eb4a04c05e0f7e29e7d0c196f373eb672
%forgemeta -ia
Name: testing
Release: 1%{?dist}
Summary: A test package
URL: %{forgeurl}
License: Public domain
Source0: %{forgesource0}
Source1: %{forgesource1}
Source2: %{forgesource2}
Source3: %{forgesource3}
Source4: %{forgesource4}
Source5: %{forgesource5}
Source6: %{forgesource6}
Source7: %{forgesource7}
Source8: %{forgesource8}
Source9: %{forgesource9}
Source10: %{forgesource10}
Source11: %{forgesource11}
Source12: %{forgesource12}
Source13: %{forgesource13}
Source14: %{forgesource14}
Source15: %{forgesource15}
%description
A test package
%prep
%forgesetup -a
%build
exit 1
%install
%files
%doc
%<--
Merges: https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/35
Sadly, ldc ppc64 support has detoriated so much that it no longer works
with any current llvm version. I'll keep an eye on things and re-enable
it once it's fixed, but right now it's just broken and upstream is
suggesting to disable the support for now.
https://github.com/ldc-developers/ldc/issues/2356
This patch adds two additional rpm macros, __brp_mangle_shebangs_exclude_file
and __brp_mangle_shebangs_exclude_from_file, to specify files from which
to read the extended regexps used for excluding shebangs and target
files.
Additionally, this adds documentation in the macros file and
--help/--usage/-?/-h to brp-mangle-shebangs, so that it's possible to
actually discover what the intended behavior is without reading the
script itself.
Signed-off-by: Peter Jones <pjones@redhat.com>
