scrum / devops

DevOps

Environments, infrastructure, deployment workflow, repository conventions, monitoring and secrets policy.

Environments

Services

Deploy method

Manual + Vercel

Repo hosting

GitHub

environments

localDeveloper machines

active

—

Python 3.11 virtual environment

—

Direct connection to VM (SSH / DB)

—

Local Bronze file copies for testing

—

.env with dev credentials

productionShared VM + Raspberry Pi

active

—

Apache Airflow — webserver + scheduler

—

PostgreSQL — Silver & Gold layers

—

Bronze — timestamped file system on VM

—

recup.py on Raspberry Pi (every minute)

infrastructure

service

role

location

status

Airflow, PostgreSQL, Bronze filesystem

On-premise

active

Raspberry Pi

Sensor polling — recup.py, 1-min interval

On-site

active

MySQL pidb

Static metadata — buildings, rooms, devices

On-premise

active

Meteo2 sFTP

Daily weather CSV download (~7 am)

External

active

GitHub

Source control, issue tracker, project board

Cloud

active

Vercel

Project website — auto-deploy from main

Cloud

active

deployment workflow

Work on a feature branch

One branch per issue — branch name references issue number

Open a pull request → review → merge

At least one reviewer required before merging to main

SSH into VM → git pull

Manual deployment — no automated CD on the VM

Restart Airflow if DAGs changed

airflow scheduler restart / webserver restart as needed

Vercel auto-deploys the website

Triggered automatically ~60 s after merge to main

repository

Branch strategy

main

Always stable and deployable

No direct push — PR + review required

feature/*

One branch per issue or grouped task

Squash-merged via PR

Commit conventions

Prefix: ADD / MOD / FIX / REF / DOCS followed by short description

Reference the GitHub issue number where relevant — e.g. ADD ingestion pipeline (#12)

Keep commits focused — one logical change per commit

Never commit secrets, .env files, or generated artefacts

monitoring

tool

what it covers

status

Airflow UI

DAG run status, task logs, retry history

active

fact_sensor_reliability

Sensor uptime, error count, missing readings

designed

PostgreSQL logs

Slow queries, connection errors

active

Failure alerting

Email / Teams notification on DAG failure

planned

secrets & config

All credentials stored in .env files on the VM and on local machines

.env is gitignored — never committed, never shared in chat history

Shared securely via a private Teams message — not email

If a team member leaves, DB passwords are rotated

No hard-coded connection strings anywhere in the codebase