Notizen zur Installation und dem Setup eines neuen Doku-Projekts

Bootstrap von Sphinx

~ $ virtualenv venv
created virtual environment CPython3.9.13.final.0-64 in 6768ms
...

~ $ . venv/bin/activate

(venv) ~ $ pip install Sphinx
Collecting Sphinx
...
Installing collected packages: s........
Successfully installed ...
WARNING: You are using pip version 22.0.4; however, version 22.2.2 is available.
You should consider upgrading via the '/home/LOCAL_USER/venv/bin/python -m pip install --upgrade pip' command.

(venv) ~ $ sphinx-quickstart MY_projekt
Welcome to the Sphinx 5.2.3 quickstart utility.

Please enter values for the following settings (just press Enter to
accept a default value, if one is given in brackets).

Selected root path:MY_projekt

You have two options for placing the build directory for Sphinx output.
Either, you use a directory "_build" within the root path, or you separate
"source" and "build" directories within the root path.
> Separate source and build directories (y/n) [n]: y

The project name will occur in several places in the built documentation.
> Project name: Mein Doku Projekt

Creating file /home/LOCAL_USER/my_projekt/source/conf.py.
Creating file /home/LOCAL_USER/my_projekt/source/index.rst.
Creating file /home/LOCAL_USER/my_projekt/Makefile.
Creating file /home/LOCAL_USER/my_projekt/make.bat.

Finished: An initial directory structure has been created.

You should now populate your master file /home/LOCAL_USER/my_projekt/source/index.rst and create other documentation
source files. Use the Makefile to build the docs, like so:
   make builder
where "builder" is one of the supported builders, e.g. html, latex or linkcheck.

Vergleiche auch https://www.sphinx-doc.org/en/master/tutorial/index.html

Einchecken in Gitea

  • Ein Repository my_projekt anlegen

  • Einchecken:

~ $ cd MY_projekt
~/my_projekt $ git init
Initialized empty Git repository in /home/LOCAL_USER/my_projekt/.git/
~/my_projekt $ git add *
~/my_projekt $ git commit -m "initial commit"
[main (root-commit) 43a000b] initial commit
 4 files changed, 102 insertions(+)
 create mode 100644 Makefile
 create mode 100644 make.bat
 create mode 100644 source/conf.py
 create mode 100644 source/index.rst
~/my_projekt $ git remote add origin https://MY_GIT_HOSTNAME/gitea/MY_GIT_USER/my_projekt.git
~/my_projekt $ git push origin main
Password for 'https://MY_GIT_USER@MY_GIT_HOSTNAME':
Enumerating objects: 7, done.
Counting objects: 100% (7/7), done.
Delta compression using up to 8 threads
Compressing objects: 100% (7/7), done.
Writing objects: 100% (7/7), 1.72 KiB | 1.72 MiB/s, done.
Total 7 (delta 0), reused 0 (delta 0), pack-reused 0
remote: . Processing 1 references
remote: Processed 1 references in total
To https://MY_GIT_HOSTNAME/gitea/MY_GIT_USER/my_projekt.git
 * [new branch]      main -> main

Konfigurieren des Webhook

Ziel ist es, dass bei einem Checkin per git in Gitea automatisch auf dem Webserver ein Checkout gemacht und ein Build gestartet wird (ähnlich wie ein Github-Runner).

Im Projekt

  • Im Projekt einen Folder webhook anlegen und zwei Dateien hinterlegen:

tea_runner.py
  1#!/usr/bin/env python3
  2
  3# credits to: https://github.com/DavesCodeMusings/tea-runner
  4#   curl -X POST --data-binary '{}' -H  "Content-Type: application/json" $URL
  5
  6"""
  7  Run tasks based on webhooks configured in Gitea.
  8
  9  Command-line options:
 10    --debug, -d  Send more detailed log output to console.
 11
 12  Configuration file (config.ini) options:
 13
 14    [runner]
 15    ALLOWED_IP_RANGE=xxx.xxx.xxx.xxx/mm
 16    # Only respond to requests made from this range of IP addresses. Eg. 192.168.1.0/24
 17    LISTEN_IP=xxx.xxx.xxx.xxx
 18    # IP address for incoming requests. Defaults to 0.0.0.0 (Any).
 19    LISTEN_PORT=xxxx
 20    # TCP port number used for incoming requests. Defaults to 1706.
 21
 22    BASE_DIR=xxx
 23    # Location of an existing repository
 24"""
 25
 26GIT_BIN = '/usr/bin/git'
 27MAKE_BIN = '/usr/bin/make'
 28
 29import logging
 30from argparse import ArgumentParser
 31from configparser import ConfigParser
 32from flask import Flask, request, jsonify
 33from werkzeug import utils
 34from waitress import serve
 35from tempfile import TemporaryDirectory
 36from os import access, X_OK, chdir, path
 37from sys import exit
 38from subprocess import run, DEVNULL
 39from os import path
 40from ipaddress import ip_address, ip_network
 41
 42print("Tea Runner")
 43
 44# Debug is a command-line option, but most configuration comes from config.ini
 45arg_parser = ArgumentParser()
 46arg_parser.add_argument('-d', '--debug', action='store_true', help='display debugging output while running')
 47args = arg_parser.parse_args()
 48
 49if args.debug:
 50    logging.basicConfig(format='%(levelname)s: %(message)s', level=logging.DEBUG)
 51else:
 52    logging.basicConfig(format='%(levelname)s: %(message)s', level=logging.INFO)
 53
 54config = ConfigParser()
 55config.read('config.ini')
 56
 57if not access(GIT_BIN, X_OK):
 58    logging.error("git binary not found or not executable")
 59    exit(1)
 60
 61if not access(MAKE_BIN, X_OK):
 62    logging.error(f"{MAKE_BIN}: binary not found or not executable")
 63    exit(1)
 64
 65
 66def git_clone(src_url, dest_dir):
 67    """
 68      Clone a remote git repository into a local directory.
 69        :src_url (string): HTTP(S) url used to clone the repo.
 70        :dest_dir (string): Path to the local directory.
 71        :returns (boolean): True if command returns success.
 72    """
 73
 74    logging.info('git clone ' + src_url)
 75    chdir(dest_dir)
 76    clone_result = run([GIT_BIN, 'clone', src_url, '.'],
 77                       stdout=None if args.debug else DEVNULL, stderr=None if args.debug else DEVNULL)
 78    return clone_result.returncode == 0
 79
 80
 81def git_pull(src_url, dest_dir):
 82    """
 83      Pull a remote git repository into a local directory.
 84        :src_url (string): HTTP(S) url used to clone the repo.
 85        :dest_dir (string): Path to the local directory.
 86        :returns (boolean): True if command returns success.
 87    """
 88
 89    logging.info('git pull ' + src_url)
 90    chdir(dest_dir)
 91    clone_result = run([GIT_BIN, 'pull'],
 92                       stdout=None if args.debug else DEVNULL, stderr=None if args.debug else DEVNULL)
 93    return clone_result.returncode == 0
 94
 95
 96app = Flask(__name__)
 97
 98
 99@app.before_request
100def check_authorized():
101    """
102      Only respond to requests from ALLOWED_IP_RANGE if it's configured in config.ini
103    """
104
105    if config.has_option('runner', 'ALLOWED_IP_RANGE'):
106        allowed_ip_range = ip_network(config['runner']['ALLOWED_IP_RANGE'])
107        requesting_ip = ip_address(request.remote_addr)
108        if requesting_ip not in allowed_ip_range:
109            logging.info('Dropping request from unauthorized host ' + request.remote_addr)
110            return jsonify(status='forbidden'), 403
111        else:
112            logging.info('Request from ' + request.remote_addr)
113
114
115@app.before_request
116def check_media_type():
117    """
118      Only respond requests with Content-Type header of application/json
119    """
120    if not request.headers.get('Content-Type').lower().startswith('application/json'):
121        logging.error('"Content-Type: application/json" header missing from request made by ' + request.remote_addr)
122        return jsonify(status='unsupported media type'), 415
123
124
125@app.route('/test', methods=['POST'])
126def test():
127    logging.debug('Content-Type: ' + request.headers.get('Content-Type'))
128    logging.debug(request.get_json(force=True))
129    return jsonify(status='success', sender=request.remote_addr)
130
131
132@app.route('/sphinx/build', methods=['POST'])
133def sphinx_build():
134    body = request.get_json()
135    logging.info('Working dir: ' + config.get('runner', 'BASE_DIR', fallback='/tmp'))
136    work_dir = config.get('runner', 'BASE_DIR', fallback='/tmp')
137    if git_pull(body['repository']['clone_url'], work_dir):
138        logging.info('sphinx build')
139        chdir(work_dir)
140        result = run([MAKE_BIN, 'html'],
141                     stdout=None if args.debug else DEVNULL, stderr=None if args.debug else DEVNULL)
142        if result.returncode != 0:
143            return jsonify(status='sphinx build failed'), 500
144    else:
145        return jsonify(status='git pull failed'), 500
146
147    return jsonify(status='success')
148
149
150if __name__ == '__main__':
151    logging.info('Limiting requests to: ' + config.get('runner', 'ALLOWED_IP_RANGE', fallback='<any>'))
152    serve(app, host=config.get('runner', 'LISTEN_IP', fallback='0.0.0.0'),
153          port=config.getint('runner', 'LISTEN_PORT', fallback=1706))
config.ini
[runner]
ALLOWED_IP_RANGE = 192.168.xx.xx/24
LISTEN_PORT = xxxx
BASE_DIR = /www/WEBSERVER_DIR/my_projekt

Bemerkung

Großen Dank an https://github.com/DavesCodeMusings/tea-runner! Das Projekt hat als Vorlage für meinen tea_runner gedient. An Stelle des Docker-Builds ist hier ein Sphinx-Build getreten.

Auf dem Server

  • manueller Checkout des Projekts in das WEBSERVER_DIR

  • Anlegen eines Python-Virtualenv / venv

  • Erzeugen eines ssh-Keys (z.B. id_gitrunner)

  • SSH-Command für git, um den Key einzustellen: git config core.sshCommand "ssh -i ....../id_gitrunner "

  • Starten des Webhook-Servers (tea_runner.py):

cd /www/WEBSERVER_DIR/my_projekt/
source venv/bin/activate
cd webhook
nohup python tea_runner.py -d > ../logs/webhook.log 2>&1 &

Gitea konfigurieren

  • Im Gitea-Projekt unter Einstellungen > Webhooks einen neuen Webhook erstellen. URL ist die des gestarteten Tearunner-Dient auf dem Server

Weitere Erfahrungen / Notizen