Implements weekly upstream rebase cadence automation: - GitLab CI scheduled pipeline (every 6 hours) - Automatic GitHub release detection - JIRA ticket creation via API - Spec file updates with changelog - Automated MR creation with OSCI Bot integration - TEST_MODE support for safe testing Scripts: - check-upstream-release.sh: Poll GitHub API for new releases - create-release-mr.sh: Main automation orchestrator - update-spec.py: Safe spec file updater - create-jira.sh: JIRA API client - test-workflow.sh: Local testing utility Documentation: - scripts/README.md: Technical documentation - AUTOMATION-SETUP.md: Step-by-step setup guide
243 lines
8.1 KiB
Markdown
243 lines
8.1 KiB
Markdown
# bootc CentOS Stream Release Automation
|
|
|
|
This directory contains scripts for automating bootc releases to CentOS Stream.
|
|
|
|
## Overview
|
|
|
|
The automation runs as a GitLab CI scheduled pipeline (every 6 hours) that:
|
|
1. Checks GitHub for new bootc releases
|
|
2. Creates a JIRA ticket for tracking
|
|
3. Downloads tarballs from GitHub
|
|
4. Updates the spec file and sources
|
|
5. Creates a GitLab Merge Request
|
|
6. OSCI Bot handles CI checks and auto-merges if all tests pass
|
|
|
|
## Architecture
|
|
|
|
```
|
|
┌─────────────────────────────────────────────┐
|
|
│ GitLab Scheduled Pipeline (every 6 hours) │
|
|
└──────────────┬──────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────┐
|
|
│ check_new_release job │
|
|
│ → scripts/check-upstream-release.sh │
|
|
│ → Compares GitHub latest vs spec version │
|
|
│ → Outputs: release.env │
|
|
└──────────────┬──────────────────────────────┘
|
|
│ if NEW_RELEASE=true
|
|
▼
|
|
┌─────────────────────────────────────────────┐
|
|
│ create_release_mr job │
|
|
│ → scripts/create-release-mr.sh │
|
|
│ ├─ Download tarballs from GitHub │
|
|
│ ├─ Create JIRA (scripts/create-jira.sh) │
|
|
│ ├─ Upload sources via centpkg │
|
|
│ ├─ Update spec (scripts/update-spec.py) │
|
|
│ ├─ Commit and push branch │
|
|
│ └─ Create GitLab MR via API │
|
|
└──────────────┬──────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────┐
|
|
│ OSCI Bot Workflow (automatic) │
|
|
│ → Runs required checks │
|
|
│ → Auto-approves if tests pass │
|
|
│ → Auto-merges with Red Hat approval │
|
|
└─────────────────────────────────────────────┘
|
|
```
|
|
|
|
## Scripts
|
|
|
|
### `check-upstream-release.sh`
|
|
Polls the GitHub API for the latest bootc release and compares it with the current spec file version.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./scripts/check-upstream-release.sh
|
|
```
|
|
|
|
**Output:**
|
|
Creates `release.env` with:
|
|
- `NEW_RELEASE` (true/false)
|
|
- `VERSION` (e.g., "1.5.2")
|
|
- `RELEASE_TAG` (e.g., "v1.5.2")
|
|
- `CURRENT_VERSION` (current spec version)
|
|
|
|
### `create-release-mr.sh`
|
|
Main automation script that orchestrates the entire release process.
|
|
|
|
**Prerequisites:**
|
|
- `release.env` must exist (created by check-upstream-release.sh)
|
|
- Environment variables configured (see below)
|
|
- `centpkg` installed and configured
|
|
- Git credentials set up
|
|
|
|
**What it does:**
|
|
1. Downloads tarballs from GitHub releases
|
|
2. Creates JIRA ticket via API
|
|
3. Creates working branch (`rebase-X.Y.Z`)
|
|
4. Uploads sources to CentOS lookaside cache
|
|
5. Updates spec file (Version, Release, changelog)
|
|
6. Commits and pushes changes
|
|
7. Creates GitLab MR with assignee
|
|
|
|
**Usage:**
|
|
```bash
|
|
# Normally run by GitLab CI, but can be run manually:
|
|
export VERSION=1.5.2
|
|
export RELEASE_TAG=v1.5.2
|
|
./scripts/create-release-mr.sh
|
|
```
|
|
|
|
### `update-spec.py`
|
|
Python helper to safely update the spec file.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./scripts/update-spec.py <spec_file> <version> <jira_ticket> [author]
|
|
```
|
|
|
|
**Example:**
|
|
```bash
|
|
./scripts/update-spec.py bootc.spec 1.5.2 RHEL-104567
|
|
```
|
|
|
|
**What it does:**
|
|
- Updates `Version:` field
|
|
- Resets `Release:` to `1%{?dist}`
|
|
- Adds properly formatted changelog entry
|
|
- Preserves spec file formatting
|
|
|
|
### `create-jira.sh`
|
|
Creates a JIRA issue for the release via REST API.
|
|
|
|
**Usage:**
|
|
```bash
|
|
export JIRA_API_TOKEN="your-token-here"
|
|
./scripts/create-jira.sh <version> [upstream_url]
|
|
```
|
|
|
|
**Example:**
|
|
```bash
|
|
./scripts/create-jira.sh 1.5.2
|
|
# Outputs: RHEL-104567
|
|
```
|
|
|
|
**Returns:** JIRA ticket key (e.g., `RHEL-104567`) on stdout
|
|
|
|
## GitLab CI/CD Variables
|
|
|
|
Configure these in **Settings → CI/CD → Variables**:
|
|
|
|
| Variable | Description | Type |
|
|
|----------|-------------|------|
|
|
| `JIRA_API_TOKEN` | JIRA API token for bootc-release-bot | Masked |
|
|
| `GITLAB_API_TOKEN` | GitLab API token (scope: `api`, `write_repository`) | Masked |
|
|
| `CENTOS_SSH_PRIVATE_KEY` | SSH private key for centpkg lookaside uploads | File, Masked |
|
|
| `BOT_EMAIL` | bootc-release-bot@redhat.com | Variable |
|
|
| `ASSIGNEE_USER_ID` | GitLab user ID for MR assignee (e.g., `1234567`) | Variable |
|
|
|
|
## GitLab Schedule Setup
|
|
|
|
1. Go to **Settings → CI/CD → Schedules**
|
|
2. Click **New schedule**
|
|
3. Configure:
|
|
- **Description:** "Check for new bootc releases"
|
|
- **Interval:** Custom → `0 */6 * * *` (every 6 hours)
|
|
- **Target branch:** `c10s`
|
|
- **Active:** ✅ Yes
|
|
|
|
## Manual Testing
|
|
|
|
### Test the release checker
|
|
```bash
|
|
./scripts/check-upstream-release.sh
|
|
cat release.env
|
|
```
|
|
|
|
### Test spec file update
|
|
```bash
|
|
./scripts/update-spec.py bootc.spec 1.5.2 RHEL-TEST-001 "Your Name <your@email.com>"
|
|
git diff bootc.spec
|
|
git restore bootc.spec # Undo changes
|
|
```
|
|
|
|
### Test JIRA creation (requires API token)
|
|
```bash
|
|
export JIRA_API_TOKEN="your-token"
|
|
./scripts/create-jira.sh 1.5.2
|
|
```
|
|
|
|
### Trigger pipeline manually
|
|
1. Go to **CI/CD → Pipelines**
|
|
2. Click **Run pipeline**
|
|
3. Select branch: `c10s`
|
|
4. Add variable: `CI_PIPELINE_SOURCE` = `web`
|
|
5. Click **Run pipeline**
|
|
|
|
## Troubleshooting
|
|
|
|
### Pipeline fails at check_new_release
|
|
- Verify GitHub API is accessible: `curl -s https://api.github.com/repos/containers/bootc/releases/latest | jq .tag_name`
|
|
- Check spec file exists and is parseable: `rpmspec -q --queryformat='%{version}\n' bootc.spec`
|
|
|
|
### Pipeline fails at create_release_mr
|
|
- **JIRA creation fails:** Check `JIRA_API_TOKEN` is valid and bot has permissions
|
|
- The automation continues with placeholder `RHEL-XXXXX` if JIRA fails
|
|
- **centpkg upload fails:** Check `CENTOS_SSH_PRIVATE_KEY` is configured correctly
|
|
- **Git push fails:** Verify bot has Developer role on the GitLab project
|
|
- **MR creation fails:** Check `GITLAB_API_TOKEN` has `api` and `write_repository` scopes
|
|
|
|
### MR created but not auto-merging
|
|
- Check OSCI Bot comments on the MR for test failures
|
|
- Ensure another Red Hatter approves the MR (required for auto-merge)
|
|
- Review failed tests and fix issues
|
|
|
|
### No new releases detected
|
|
- Verify upstream has actually released a new version: https://github.com/containers/bootc/releases
|
|
- Check current spec version matches latest upstream: `rpmspec -q --queryformat='%{version}\n' bootc.spec`
|
|
|
|
## Bot Account Setup
|
|
|
|
The automation uses `bootc-release-bot` account:
|
|
|
|
**GitLab:**
|
|
- Developer role on `gitlab.com/redhat/centos-stream/rpms/bootc`
|
|
- API token with `api` and `write_repository` scopes
|
|
- SSH key for centpkg uploads
|
|
|
|
**JIRA:**
|
|
- Account with issue creation permissions in RHEL project
|
|
- API token configured in GitLab CI/CD variables
|
|
|
|
**Git config:**
|
|
```bash
|
|
git config user.name "bootc-release-bot"
|
|
git config user.email "bootc-release-bot@redhat.com"
|
|
```
|
|
|
|
## Weekly Cadence
|
|
|
|
With the 6-hour scheduled pipeline, new releases are detected within:
|
|
- **Best case:** <6 hours after upstream release
|
|
- **Average:** 3 hours after upstream release
|
|
- **Worst case:** ~6 hours after upstream release
|
|
|
|
This ensures the weekly cadence is maintained with minimal manual intervention.
|
|
|
|
## Success Metrics
|
|
|
|
- ✅ Zero missed upstream releases
|
|
- ✅ MRs created within 6 hours of upstream release
|
|
- ✅ Auto-merge rate >90% (indicates healthy CI)
|
|
- ✅ Manual intervention required <10% of releases
|
|
|
|
## Support
|
|
|
|
For issues or questions:
|
|
- Check GitLab pipeline logs for detailed error messages
|
|
- Review OSCI Bot comments on MRs for test failures
|
|
- Contact the bootc team for bot account or permission issues
|