25 Commits

Author SHA1 Message Date
abceb07c1a add profile scope
Some checks failed
App Image CI / Build app image (push) Successful in -1m9s
Docker Image CI / Build build images (arch) (push) Successful in -6s
Docker Image CI / Build build images (artix) (push) Successful in -32s
NPM Audit Check / Check NPM audit (push) Failing after -2m9s
2025-09-28 21:47:29 -05:00
2c71563efc hide cancel button from public view
All checks were successful
NPM Audit Check / Check NPM audit (push) Successful in -2m13s
App Image CI / Build app image (push) Successful in -1m9s
2025-09-26 14:43:59 -05:00
f39ce0977f upsert user
All checks were successful
App Image CI / Build app image (push) Successful in -1m28s
NPM Audit Check / Check NPM audit (push) Successful in -2m13s
2025-09-26 14:38:17 -05:00
45ebb7d9cf Merge pull request 'add authentication support' (#1) from auth-system into master
All checks were successful
App Image CI / Build app image (push) Successful in -1m9s
Docker Image CI / Build build images (arch) (push) Successful in -3s
Docker Image CI / Build build images (artix) (push) Successful in -29s
NPM Audit Check / Check NPM audit (push) Successful in -2m12s
Reviewed-on: #1
2025-09-24 23:40:05 -05:00
fe49302eca add session store
All checks were successful
App Image CI / Build app image (pull_request) Successful in -1m28s
NPM Audit Check / Check NPM audit (pull_request) Successful in -2m13s
2025-09-24 23:31:24 -05:00
77441fe7ed create public build endpoint
All checks were successful
App Image CI / Build app image (pull_request) Successful in -1m26s
NPM Audit Check / Check NPM audit (pull_request) Successful in -2m14s
2025-09-23 19:15:19 -05:00
b20bbf40fc Track and display build authors
All checks were successful
App Image CI / Build app image (pull_request) Successful in -1m26s
NPM Audit Check / Check NPM audit (pull_request) Successful in -2m14s
2025-09-23 16:09:31 -05:00
24d7b37583 require login to view (if enabled)
All checks were successful
App Image CI / Build app image (pull_request) Successful in -1m26s
NPM Audit Check / Check NPM audit (pull_request) Successful in -2m13s
2025-09-23 15:02:25 -05:00
5e95f22245 footer position fixed
All checks were successful
App Image CI / Build app image (push) Successful in -1m27s
NPM Audit Check / Check NPM audit (push) Successful in -2m12s
2025-09-23 12:55:16 -05:00
12cf09d408 add clone button for build parameters
All checks were successful
App Image CI / Build app image (push) Successful in -1m12s
Docker Image CI / Build build images (arch) (push) Successful in -5s
Docker Image CI / Build build images (artix) (push) Successful in -38s
NPM Audit Check / Check NPM audit (push) Successful in -2m13s
2025-09-17 23:34:55 -05:00
8f19175c3a qol improvements to build.js 2025-09-17 23:15:38 -05:00
eeac35a0a6 upgrade express, others 2025-09-17 22:50:48 -05:00
b5d57228f0 use sqids 2025-09-17 22:45:50 -05:00
4435c0baee npm audit fix
All checks were successful
App Image CI / Build app image (push) Successful in -1m42s
Docker Image CI / Build build images (arch) (push) Successful in -42s
Docker Image CI / Build build images (artix) (push) Successful in -1m0s
NPM Audit Check / Check NPM audit (push) Successful in -2m15s
2025-07-22 11:12:49 -05:00
766e4a9594 npm audit check
Some checks failed
App Image CI / Build app image (push) Successful in -1m39s
NPM Audit Check / Check NPM audit (push) Failing after -2m20s
2025-07-22 11:10:23 -05:00
b13d34cbdb move to gitea
All checks were successful
App Image CI / Build app image (push) Successful in -2m7s
Docker Image CI / Build build images (arch) (push) Successful in -15s
Docker Image CI / Build build images (artix) (push) Successful in -48s
2025-04-29 21:04:32 -05:00
9722c4f992 use build package 2025-03-18 21:47:15 -05:00
96b9d3210c back to development: 0.1.4 2025-01-15 22:02:55 -05:00
14f0d494be 0.1.3: visual fix 2025-01-15 21:56:22 -05:00
7d07873536 0.1.2: visual improvements 2025-01-15 21:32:07 -05:00
69868d0f7c fix example docker-compose.yml 2025-01-14 20:31:27 -05:00
e1230b3fde Change workdir to archery 2025-01-14 20:30:06 -05:00
c83b7535c1 Read db password from env 2025-01-14 20:22:59 -05:00
4a3ce70751 Include some barebones documentation 2025-01-14 20:22:44 -05:00
97440f1352 prepare 0.1.0 release 2025-01-14 20:04:52 -05:00
24 changed files with 1266 additions and 775 deletions

View File

@@ -17,8 +17,6 @@ jobs:
runs-on: ubuntu-latest
env:
DH_REGISTRY: docker.io
GH_REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
REPOSITORY: ${{ github.event.repository.name }}
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
permissions:
@@ -42,60 +40,24 @@ jobs:
registry: ${{ env.DH_REGISTRY }}
username: ${{ env.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ${{ env.GH_REGISTRY }}
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata for release Docker image
if: startsWith(github.ref, 'refs/tags/v')
id: meta
uses: docker/metadata-action@v5
with:
images: |
${{ env.DOCKER_USERNAME }}/${{ env.REPOSITORY }}
${{ env.GH_REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=raw,value=latest
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=semver,pattern={{major}}
- name: Extract metadata for develop Docker image
if: "!startsWith(github.ref, 'refs/tags/v')"
id: meta-develop
uses: docker/metadata-action@v5
with:
images: |
${{ env.GH_REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=pr
- name: Build and push release Docker image
if: startsWith(github.ref, 'refs/tags/v')
uses: docker/build-push-action@v6
with:
target: deploy
push: true
push: ${{ startsWith(github.ref, 'refs/tags/v') && 'true' || 'false' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
platforms: linux/amd64
cache-from: type=gha,scope=${{ github.workflow }}
cache-to: type=gha,mode=max,scope=${{ github.workflow }}
- name: Build and push develop Docker image
if: "!startsWith(github.ref, 'refs/tags/v')"
uses: docker/build-push-action@v6
with:
target: deploy
push: true
tags: ${{ steps.meta-develop.outputs.tags }}
labels: ${{ steps.meta-develop.outputs.labels }}
platforms: linux/amd64
cache-from: type=gha,scope=${{ github.workflow }}
cache-to: type=gha,mode=max,scope=${{ github.workflow }}

View File

@@ -39,22 +39,22 @@ jobs:
run: echo "DEPLOY=true" >> $GITHUB_ENV
- name: Checkout repository
uses: actions/checkout@v4
uses: https://github.com/actions/checkout@v4
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v3
uses: https://github.com/docker/setup-buildx-action@v3
with:
install: true
- name: Log in to the Docker Hub
uses: docker/login-action@v3
uses: https://github.com/docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Build Docker image
uses: docker/build-push-action@v6
uses: https://github.com/docker/build-push-action@v6
with:
push: ${{ env.DEPLOY }}
pull: true

View File

@@ -0,0 +1,27 @@
name: NPM Audit Check
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
schedule:
- cron: '0 16 * * 5'
jobs:
npm_audit:
name: Check NPM audit
runs-on: ubuntu-latest
timeout-minutes: 20
strategy:
fail-fast: true
permissions:
contents: read
steps:
- name: Checkout repository
uses: https://github.com/actions/checkout@v4
- name: NPM Audit
run: npm audit

View File

@@ -5,13 +5,13 @@ WORKDIR /build
COPY ./package*json ./
RUN npm ci
COPY . .
RUN node --experimental-strip-types build.ts && \
RUN npm run build && \
npm exec tsc && \
npm ci --only=production --omit=dev
FROM base as deploy
FROM base AS deploy
WORKDIR /srv/abt
WORKDIR /srv/archery
RUN apk add --no-cache docker-cli
COPY --from=build-env /build .

171
build.ts
View File

@@ -1,171 +0,0 @@
import fs from 'fs';
import path from 'path';
import child_process from 'child_process';
import uglifyjs from "uglify-js";
import * as sass from 'sass';
import * as csso from 'csso';
const spawn = child_process.spawn;
const fsp = fs.promises;
const STYLESDIR = 'styles';
const SCRIPTSDIR = 'scripts';
const IMAGESDIR = path.join('assets', 'images');
const STYLEOUTDIR = process.env.STYLEOUTDIR || path.join(import.meta.dirname, 'assets', 'css');
const SCRIPTSOUTDIR = process.env.SCRIPTSOUTDIR || path.join(import.meta.dirname, 'assets', 'js');
const IMAGESOUTDIR = process.env.IMAGESOUTDIR || path.join(import.meta.dirname, 'assets', 'webp');
const STYLEOUTFILE = process.env.STYLEOUTFILE || 'styles.css';
const SQUASH = new RegExp('^[0-9]+-');
async function emptyDir(dir: string) {
await Promise.all((await fsp.readdir(dir, { withFileTypes: true })).map(f => path.join(dir, f.name)).map(p => fsp.rm(p, {
recursive: true,
force: true
})));
}
async function mkdir(dir: string | string[]) {
if (typeof dir === 'string') {
await fsp.mkdir(dir, { recursive: true });
}
else {
await Promise.all(dir.map(mkdir));
}
}
// Process styles
async function styles() {
await mkdir([STYLEOUTDIR, STYLESDIR]);
await emptyDir(STYLEOUTDIR);
let styles: string[] = [];
let files = await fsp.readdir(STYLESDIR);
await Promise.all(files.map(f => new Promise(async (res, reject) => {
let p = path.join(STYLESDIR, f);
console.log(`Processing style ${p}`);
let style = sass.compile(p).css;
if (f.charAt(0) !== '_') {
if (SQUASH.test(f)) {
styles.push(style);
}
else {
let o = path.join(STYLEOUTDIR, f.substring(0, f.lastIndexOf('.')) + '.css');
await fsp.writeFile(o, csso.minify(style).css);
console.log(`Wrote ${o}`);
}
}
res(0);
})));
let out = csso.minify(styles.join('\n')).css;
let outpath = path.join(STYLEOUTDIR, STYLEOUTFILE);
await fsp.writeFile(outpath, out);
console.log(`Wrote ${outpath}`);
}
// Process scripts
async function scripts() {
await mkdir([SCRIPTSOUTDIR, SCRIPTSDIR]);
await emptyDir(SCRIPTSOUTDIR);
let files = await fsp.readdir(SCRIPTSDIR);
await Promise.all(files.map(f => new Promise(async (res, reject) => {
let p = path.join(SCRIPTSDIR, f);
let o = path.join(SCRIPTSOUTDIR, f);
console.log(`Processing script ${p}`);
try {
await fsp.writeFile(o, uglifyjs.minify((await fsp.readFile(p)).toString()).code);
console.log(`Wrote ${o}`);
}
catch (ex) {
console.log(`error writing ${o}: ${ex}`);
}
res(0);
})));
}
// Process images
async function images(dir = '') {
let p = path.join(IMAGESDIR, dir);
await mkdir(p);
if (dir.length === 0) {
await mkdir(IMAGESOUTDIR)
await emptyDir(IMAGESOUTDIR);
}
let files = await fsp.readdir(p, {
withFileTypes: true
});
if (files.length) {
await Promise.all(files.map(f => new Promise(async (res, reject) => {
if (f.isFile()) {
let outDir = path.join(IMAGESOUTDIR, dir);
let infile = path.join(p, f.name);
let outfile = path.join(outDir, f.name.substring(0, f.name.lastIndexOf('.')) + '.webp');
await mkdir(outDir);
console.log(`Processing image ${infile}`)
let process = spawn('cwebp', ['-mt', '-q', '50', infile, '-o', outfile]);
let timeout = setTimeout(() => {
reject('Timed out');
process.kill();
}, 30000);
process.on('exit', async (code) => {
clearTimeout(timeout);
if (code === 0) {
console.log(`Wrote ${outfile}`);
res(null);
}
else {
reject(code);
}
});
}
else if (f.isDirectory()) {
images(path.join(dir, f.name)).then(res).catch(reject);
}
})));
}
}
function isAbortError(err: unknown): boolean {
return typeof err === 'object' && err !== null && 'name' in err && err.name === 'AbortError';
}
(async function () {
await Promise.all([styles(), scripts(), images()]);
if (process.argv.indexOf('--watch') >= 0) {
console.log('watching for changes...');
(async () => {
try {
const watcher = fsp.watch(STYLESDIR);
for await (const _ of watcher)
await styles();
} catch (err) {
if (isAbortError(err))
return;
throw err;
}
})();
(async () => {
try {
const watcher = fsp.watch(SCRIPTSDIR);
for await (const _ of watcher)
await scripts();
} catch (err) {
if (isAbortError(err))
return;
throw err;
}
})();
(async () => {
try {
const watcher = fsp.watch(IMAGESDIR, {
recursive: true // no Linux ☹️
});
for await (const _ of watcher)
await images();
} catch (err) {
if (isAbortError(err))
return;
throw err;
}
})();
}
})();

View File

@@ -1 +1,15 @@
{}
{
"db": {
"host": "postgres",
"password": "correcthorse"
},
"web": {
"port": 8080,
"oidc": {
"server": "https://gitea.artixlinux.org/",
"clientId": "fakefake-fake-fake-fake-fakefakefake",
"clientSecret": "thisisnotarealsecret",
"appBaseUrl": "http://localhost:8080"
}
}
}

View File

@@ -7,8 +7,10 @@ services:
context: ./
dockerfile: Dockerfile
volumes:
- ./config:/srv/abt/config
- ./config:/srv/archery/config
- /var/run/docker.sock:/var/run/docker.sock
environment:
PASSWORD: ${POSTGRES_PASSWORD}
restart: "no"
ports:
- 8080:8080

983
package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,6 @@
{
"name": "archery",
"version": "0.0.2",
"version": "0.2.2",
"description": "Build Arch packages through a web interface",
"keywords": [
"docker",
@@ -20,25 +20,33 @@
"type": "module",
"main": "index.ts",
"dependencies": {
"body-parser": "^1.20.3",
"body-parser": "^2.2.0",
"ejs": "3.1.10",
"express": "^4.21.2",
"express": "^5.1.0",
"express-session": "1.18.2",
"express-ws": "^5.0.2",
"pg": "^8.13.1",
"ky": "1.10.0",
"passport": "0.7.0",
"passport-openidconnect": "0.1.2",
"pg": "^8.16.3",
"pg-hstore": "^2.3.4",
"sequelize": "^6.37.5"
"sequelize": "^6.37.7",
"sqids": "0.3.0"
},
"devDependencies": {
"@types/csso": "5.0.4",
"@types/express": "^5.0.0",
"@types/express": "^5.0.3",
"@types/express-session": "^1.18.2",
"@types/express-ws": "3.0.5",
"@types/node": "^22.10.5",
"@types/uglify-js": "3.17.5",
"csso": "5.0.5",
"sass": "1.83.1",
"uglify-js": "3.19.3"
"@types/node": "^24.5.2",
"@types/passport": "1.0.17",
"@types/passport-openidconnect": "0.1.3",
"forking-build-shit": "1.0.4"
},
"peerDependencies": {
"typescript": "5.7.3"
},
"scripts": {
"build": "npx build-shit",
"watch": "npx build-shit --watch"
}
}

6
readme.md Normal file
View File

@@ -0,0 +1,6 @@
# Archery
## Build system for pacman
Quickly trigger build jobs on a remote system (perhaps one with more RAM?) for debugging packaging issues. Switch between Artix and Arch to compare builds. Enable testing or staging repos in a single click.
Consult [docker-compose.yml](docker-compose.yml) and [config.example.json](config/config.example.json) to get an understanding of how to deploy Archery.

View File

@@ -2,23 +2,18 @@ document.addEventListener('DOMContentLoaded', function () {
const logContainer = this.getElementById('logs');
const followLogsBtn = this.getElementById('followCheckmark');
const buildStatusTxt = this.getElementById('buildStatus');
/**
* Get the correct path to establish a ws connection
* @param {Location} loc
*/
function wsPath(loc) {
return loc.pathname.replace(/\/$/, '') + '/ws';
}
let started = logContainer.childElementCount > 0;
/**
* Add log line to the DOM
* @param {string} str
* @param {string[]} str
*/
function appendLine(str, e = false) {
const p = document.createElement('p');
p.appendChild(document.createTextNode(str));
logContainer.appendChild(p);
function appendLines(str, e = false) {
str.forEach(line => {
const p = document.createElement('p');
p.appendChild(document.createTextNode(line));
logContainer.appendChild(p);
});
}
/**
@@ -30,6 +25,18 @@ document.addEventListener('DOMContentLoaded', function () {
}
}
function handleScrollToggleClick() {
window.scrollTo(0, followLogsBtn.checked ? document.body.scrollHeight : 0);
}
/**
* Split string by newline char
* @param {string} str
*/
function splitLines(str) {
return str.split('\n').map(line => line.substring(line.lastIndexOf('\r') + 1));
}
/**
* Establish websocket connection
*/
@@ -37,19 +44,25 @@ document.addEventListener('DOMContentLoaded', function () {
const loc = window.location;
let new_uri = loc.protocol === 'https:' ? 'wss:' : 'ws:';
new_uri += "//" + loc.host;
new_uri += wsPath(loc);
new_uri += loc.pathname + 'ws';
var ws = new WebSocket(new_uri);
ws.onmessage = function (message) {
console.log('Got message: ', message);
const buildEvent = JSON.parse(message.data);
if (!started) {
started = true;
buildStatusTxt.replaceChild(document.createTextNode('running'), buildStatusTxt.firstChild);
}
if (buildEvent.type === 'finish') {
ws.close();
buildStatusTxt.replaceChild(document.createTextNode(buildEvent.message), buildStatusTxt.firstChild);
appendLines([`finished: ${buildEvent.message}`]);
const cancelBtn = document.getElementById('cancelRow');
cancelBtn.parentElement.removeChild(cancelBtn);
}
else {
appendLine(buildEvent.message, buildEvent.type === 'err');
appendLines(splitLines(buildEvent.message), buildEvent.type === 'err');
scrollToBottom();
}
}
@@ -57,5 +70,5 @@ document.addEventListener('DOMContentLoaded', function () {
connect();
followLogsBtn.checked = false;
followLogsBtn.addEventListener('change', scrollToBottom);
followLogsBtn.addEventListener('change', handleScrollToggleClick);
});

7
scripts/copy.js Normal file
View File

@@ -0,0 +1,7 @@
document.addEventListener('DOMContentLoaded', function () {
for (let btn of document.getElementsByClassName('copybtn')) {
btn.addEventListener('click', e => {
navigator.clipboard.writeText(e.target.previousElementSibling.innerText);
});
}
});

246
src/DB.ts
View File

@@ -1,9 +1,13 @@
import { Sequelize, DataTypes, Op, } from 'sequelize';
import { Store } from 'express-session'
import { notStupidParseInt } from './Web.ts';
import type { ModelStatic, Filterable } from 'sequelize';
import type { LogType } from './BuildController.ts';
import type { SessionData } from 'express-session'
type Status = 'queued' | 'running' | 'cancelled' | 'success' | 'error';
type Dependencies = 'stable' | 'testing' | 'staging';
type Callback = (err?: unknown, data?: any) => any
interface DBConfig {
db?: string;
@@ -24,6 +28,14 @@ interface Build {
endTime?: Date;
status: Status;
pid?: number;
sqid?: string;
uuid: string;
}
interface User {
id: string;
username: string;
displayName?: string;
}
interface LogChunk {
@@ -33,7 +45,7 @@ interface LogChunk {
chunk: string
}
const MONTH = 1000 * 60 * 60 * 24 * 24;
const MONTH = 1000 * 60 * 60 * 24 * 30;
const FRESH = {
[Op.or]: [
{ startTime: { [Op.gt]: new Date(Date.now() - MONTH) } },
@@ -42,16 +54,32 @@ const FRESH = {
}
const SELECT = ['id', 'repo', 'commit', 'distro', 'dependencies', 'startTime', 'endTime', 'status'];
class DB {
function handleCallback<T>(err: unknown, data: T, cb?: Callback): T {
if (cb) {
cb(err, data);
}
if (err) {
throw err;
}
return data;
}
class DB extends Store {
private build: ModelStatic<any>;
private logChunk: ModelStatic<any>;
private user: ModelStatic<any>;
private session: ModelStatic<any>;
private sequelize: Sequelize;
private ttl: number;
constructor(config: DBConfig = {}) {
this.sequelize = new Sequelize(config.db || 'archery', config.user || 'archery', config.password || '', {
super();
this.ttl = notStupidParseInt(process.env['COOKIETTL']) || 1000 * 60 * 60 * 24 * 30;
this.sequelize = new Sequelize(config.db || 'archery', config.user || 'archery', process.env.PASSWORD || config.password || '', {
host: config.host || 'localhost',
port: config.port || 5432,
dialect: 'postgres'
dialect: 'postgres',
logging: false
});
this.build = this.sequelize.define('builds', {
id: {
@@ -96,6 +124,10 @@ class DB {
pid: {
type: DataTypes.INTEGER,
allowNull: true
},
uuid: {
type: DataTypes.STRING,
unique: true
}
});
@@ -126,21 +158,82 @@ class DB {
}
});
this.user = this.sequelize.define('users', {
id: {
type: DataTypes.STRING,
primaryKey: true,
},
username: {
type: DataTypes.STRING,
},
displayName: {
type: DataTypes.STRING,
allowNull: true
}
});
this.session = this.sequelize.define('session', {
sid: {
type: DataTypes.STRING,
primaryKey: true,
},
sessionData: {
type: DataTypes.JSONB,
}
});
this.build.belongsTo(this.user);
this.user.hasMany(this.build);
this.sync();
}
private async sync(): Promise<void> {
await this.user.sync();
await this.build.sync();
await this.logChunk.sync();
await this.session.sync();
if (!(await this.getUser('-1'))) {
await this.createUser({
id: '-1',
username: '???',
displayName: 'Anonymous User'
});
}
}
public async createBuild(repo: string, commit: string, patch: string, distro: string, dependencies: string): Promise<number> {
public async getUser(id: string): Promise<User> {
return await this.user.findByPk(id);
}
public async createUser(user: User): Promise<string> {
await this.user.create({
id: user.id,
username: user.username,
displayName: user.displayName || null
});
return user.id;
}
public async upsertUser(user: User): Promise<string> {
await this.user.upsert({
id: user.id,
username: user.username,
displayName: user.displayName || null
});
return user.id;
}
public async createBuild(repo: string, commit: string, patch: string, distro: string, dependencies: string, author: string, uuid: string): Promise<number> {
const buildRec = await this.build.create({
repo,
commit: commit || null,
patch: patch || null,
distro,
dependencies
dependencies,
uuid,
userId: author || '-1'
});
return buildRec.id;
}
@@ -187,14 +280,26 @@ class DB {
}
public async getBuild(id: number): Promise<Build> {
return await this.build.findByPk(id);
return await this.build.findByPk(id, {
include: this.user
});
}
public async getBuildByUuid(uuid: string): Promise<Build> {
return await this.build.findOne({
where: {
uuid
},
include: this.user
});
}
public async getBuilds(): Promise<Build[]> {
return await this.build.findAll({
attributes: SELECT,
order: [['id', 'DESC']],
where: FRESH
where: FRESH,
include: this.user
});
}
@@ -205,7 +310,8 @@ class DB {
where: {
...FRESH,
status
}
},
include: this.user
});
}
@@ -216,7 +322,8 @@ class DB {
where: {
...FRESH,
distro
}
},
include: this.user
});
}
@@ -227,7 +334,8 @@ class DB {
where: {
...FRESH,
...filterable
}
},
include: this.user
});
}
@@ -250,7 +358,8 @@ class DB {
{ repo: { [Op.iLike]: `%${query}%` } }
]
},
limit: 100
limit: 100,
include: this.user
});
}
@@ -258,8 +367,117 @@ class DB {
await this.build.destroy({
where: {
startTime: { [Op.lt]: new Date(Date.now() - MONTH * 6) }
}
},
force: true
});
await this.session.destroy({
where: {
updatedAt: { [Op.lt]: new Date(Date.now() - this.ttl) }
},
force: true
});
}
public getTTL(sessionData: SessionData) {
if (sessionData?.cookie?.expires) {
const ms = Number(new Date(sessionData.cookie.expires)) - Date.now();
return ms;
}
else {
return this.ttl;
}
}
public async set(sid: string, sessionData: SessionData, cb?: Callback): Promise<void> {
const ttl = this.getTTL(sessionData);
try {
if (ttl > 0) {
await this.session.upsert({
sid,
sessionData
});
handleCallback(null, null, cb);
return;
}
await this.destroy(sid, cb);
}
catch (err) {
return handleCallback(err, null, cb);
}
}
public async get(sid: string, cb?: Callback): Promise<SessionData> {
try {
return handleCallback(null, ((await this.session.findByPk(sid))?.sessionData) as SessionData || null, cb);
}
catch (err) {
return handleCallback(err, null, cb);
}
}
public async destroy(sid: string, cb?: Callback): Promise<void> {
try {
await this.session.destroy({
where: {
sid
},
force: true
});
handleCallback(null, null, cb);
}
catch (err) {
handleCallback(err, null, cb);
}
}
public async clear(cb?: Callback): Promise<void> {
try {
await this.session.destroy({
truncate: true,
force: true
});
handleCallback(null, null, cb);
}
catch (err) {
handleCallback(err, null, cb);
}
}
public async length(cb?: Callback): Promise<number> {
try {
return handleCallback(null, await this.session.count(), cb);
}
catch (err) {
handleCallback(err, null, cb);
}
}
public async touch(sid: string, sessionData: SessionData, cb?: Callback): Promise<void> {
try {
await this.session.update({},
{
where: {
sid
}
}
);
handleCallback(null, null, cb);
}
catch (err) {
handleCallback(err, null, cb);
}
}
public async all(cb?: Callback): Promise<SessionData[]> {
try {
const all = await this.session.findAll({
attributes: ['sessionData']
});
return handleCallback(null, all.map(row => row.sessionData as SessionData), cb);
}
catch (err) {
handleCallback(err, null, cb);
}
}
public async close(): Promise<void> {
@@ -269,4 +487,4 @@ class DB {
export default DB;
export { DB };
export type { DBConfig, Status, Build };
export type { DBConfig, Status, Build, LogChunk, User };

View File

@@ -1,14 +1,34 @@
import * as http from "http";
import crypto from 'crypto';
import type { Express } from "express";
import type { Express } from 'express';
import express from 'express';
import expressWs from "express-ws";
import bodyParser from "body-parser";
import type { DB } from "./DB.ts";
import expressWs from 'express-ws';
import session from 'express-session';
import ky from 'ky';
import passport from 'passport';
import OpenIDConnectStrategy from 'passport-openidconnect';
import bodyParser from 'body-parser';
import Sqids from 'sqids';
import type { DB, LogChunk, Build, User } from "./DB.ts";
import type { BuildController, BuildEvent } from "./BuildController.ts";
interface WebConfig {
sessionSecret?: string;
port?: number;
secure?: boolean;
oidc?: {
server: string;
clientId: string;
clientSecret: string;
appBaseUrl: string;
};
}
interface OpenIdConfiguration {
issuer: string;
authorization_endpoint: string;
token_endpoint: string;
userinfo_endpoint: string;
}
/**
@@ -29,16 +49,32 @@ function timeElapsed(date1: Date, date2: Date) {
return `${hours}:${minutes}:${seconds}`;
}
function splitLines(lines: LogChunk[]) {
return lines.map(logChunk => logChunk.chunk.split('\n')).flat().map(line => line.substring(line.lastIndexOf('\r') + 1));
}
class Web {
private _webserver: http.Server | null = null;
private db: DB;
private buildController: BuildController;
private app: expressWs.Application;
private port: number;
private options:WebConfig;
constructor(options: WebConfig = {}) {
this.options = options;
}
initialize = async () => {
const options = this.options;
const sessionSecret = process.env['SESSIONSECRET'] || options.sessionSecret;
const sqids = new Sqids({
minLength: 6,
alphabet: 'abcdefghijkmnprstuvwxyz'
});
const app: Express = express();
const wsApp = this.app = expressWs(app).app;
const oidc = await this.initializeOIDC(options);
this.port = notStupidParseInt(process.env.PORT) || options['port'] as number || 8080;
app.set('trust proxy', 1);
@@ -59,15 +95,154 @@ class Web {
});
});
const createBuildPages = (slug: string, getBuildFn: (str: string) => Promise<Build>) => {
app.get(`/${slug}/:id/`, async (req, res) => {
const build = await getBuildFn(req.params.id);
if (!build) {
res.sendStatus(404);
return;
}
build.sqid = sqids.encode([build.id]);
const log = splitLines(await this.db.getLog(build.id));
if (req?.user) {
res.locals.shareable = `${req.protocol}://${req.host}/b/${build.uuid}/`;
}
res.render('build', {
page: {
title: 'Archery',
titlesuffix: `Build #${build.id}`,
description: `Building ${build.repo} on ${build.distro}`,
},
user: req?.user,
build,
log,
ended: build.status !== 'queued' && build.status !== 'running',
public: !!oidc && !req?.user
});
});
app.get(`/${slug}/:id/logs{/}`, async (req, res) => {
const build = await getBuildFn(req.params.id);
if (!build) {
res.sendStatus(404);
return;
}
const log = (await this.db.getLog(build.id)).map(logChunk => logChunk.chunk).join('\n');
res.set('Content-Type', 'text/plain').send(log);
});
app.get(`/${slug}/:id/patch{/}`, async (req, res) => {
const build = await getBuildFn(req.params.id);
if (!build || !build.patch) {
res.sendStatus(404);
return;
}
res.set('Content-Type', 'text/plain').send(build.patch);
});
wsApp.ws(`/${slug}/:id/ws`, async (ws, req) => {
const build = await getBuildFn(req.params.id);
if (!build || (build.status !== 'queued' && build.status !== 'running')) {
return ws.close();
}
console.log('WS Opened');
const eventListener = (be: BuildEvent) => {
if (be.id === build.id) {
ws.send(JSON.stringify(be));
}
};
this.buildController.on('log', eventListener);
ws.on('close', () => {
console.log('WS Closed');
this.buildController.removeListener('log', eventListener);
});
});
}
app.get('/healthcheck', (_, res) => {
res.send('Healthy');
});
if (oidc) {
if (!sessionSecret) {
throw new Error('sessionSecret must be set.');
}
app.use(session({
name: 'sessionId',
secret: sessionSecret,
resave: true,
saveUninitialized: false,
store: this.db,
cookie: {
maxAge: notStupidParseInt(process.env['COOKIETTL']) || 1000 * 60 * 60 * 24 * 30, // 30 days
httpOnly: true,
secure: !!options.secure
}
}));
passport.use(oidc);
app.use(passport.initialize());
app.use(passport.session());
passport.serializeUser(function (user: User, done) {
done(null, user.id);
});
passport.deserializeUser(async (id: string, done) => {
const user = await this.db.getUser(id);
done(null, {
id: user.id,
username: user.username,
displayName: user.displayName
});
});
app.get('/login', (req, res) => {
if (req?.user) {
return res.redirect('/');
}
res.append('X-Robots-Tag', 'none');
res.render('login-required', {
page: {
title: 'Archery',
titlesuffix: 'Log In',
description: 'Authentication required',
}
});
});
app.post('/login', passport.authenticate('openidconnect'));
app.get('/cb', passport.authenticate('openidconnect', { successRedirect: '/', failureRedirect: '/login', failureMessage: true }));
app.get('/logout', (req, res) => {
req.logOut((err) => {
if (err) {
console.error(`Failed to log out user: ${err}`);
}
res.redirect('/login');
});
});
createBuildPages('b', (id) => this.db.getBuildByUuid(id));
app.use((req, res, next) => {
if (!req?.user) {
res.redirect('/login');
return;
}
next();
});
}
app.get('/', async (req, res) => {
try {
const builds = 'q' in req.query ? await this.db.searchBuilds(req.query.q as string) : await this.db.getBuildsBy(req.query);
builds.forEach(b => {
b.sqid = sqids.encode([b.id]);
});
res.render('index', {
page: {
title: 'Archery',
titlesuffix: 'Dashboard',
description: 'PKGBUILD central'
},
user: req?.user,
builds,
timeElapsed
});
@@ -78,50 +253,37 @@ class Web {
}
});
app.get('/build/?', (_, res) => {
app.get('/build{/}', async (req, res) => {
const query = ('id' in req.query && typeof req.query.id === 'string' && await this.db.getBuild(sqids.decode(req.query.id)?.[0])) || req.query;
res.render('build-new', {
page: {
title: 'Archery',
titlesuffix: 'New Build',
description: 'Kick off a build'
}
description: 'Kick off a build',
},
user: req?.user,
query
});
});
app.post('/build/?', async (req, res) => {
app.post('/build{/}', async (req, res) => {
const buildId = await this.db.createBuild(
req.body.repo,
req.body.commit || null,
req.body.patch || null,
req.body.distro || 'arch',
req.body.dependencies || 'stable'
req.body.dependencies || 'stable',
req?.user?.['id'],
crypto.randomUUID()
);
res.redirect(`/build/${buildId}`);
res.redirect(`/build/${sqids.encode([buildId])}/`);
this.buildController.triggerBuild();
});
app.get('/build/:num/?', async (req, res) => {
const build = await this.db.getBuild(parseInt(req.params.num));
if (!build) {
res.sendStatus(404);
return;
}
const log = (await this.db.getLog(build.id)).map(logChunk => logChunk.chunk.split('\n')).flat();
createBuildPages('build', (id) => this.db.getBuild(sqids.decode(id)?.[0]));
res.render('build', {
page: {
title: 'Archery',
titlesuffix: `Build #${req.params.num}`,
description: `Building ${build.repo} on ${build.distro}`
},
build,
log,
ended: build.status !== 'queued' && build.status !== 'running'
});
});
app.get('/build/:num/cancel', async (req, res) => {
const build = await this.db.getBuild(parseInt(req.params.num));
app.get('/build/:id/cancel', async (req, res) => {
const build = await this.db.getBuild(sqids.decode(req.params.id)?.[0]);
if (!build) {
res.sendStatus(404);
return;
@@ -132,47 +294,10 @@ class Web {
catch (ex) {
console.error(ex);
}
res.redirect(`/build/${build.id}`);
});
app.get('/build/:num/logs/?', async (req, res) => {
const build = await this.db.getBuild(parseInt(req.params.num));
if (!build) {
res.sendStatus(404);
return;
}
const log = (await this.db.getLog(build.id)).map(logChunk => logChunk.chunk).join('\n');
res.set('Content-Type', 'text/plain').send(log);
});
app.get('/build/:num/patch/?', async (req, res) => {
const build = await this.db.getBuild(parseInt(req.params.num));
if (!build || !build.patch) {
res.sendStatus(404);
return;
}
res.set('Content-Type', 'text/plain').send(build.patch);
});
app.get('/healthcheck', (_, res) => {
res.send('Healthy');
});
wsApp.ws('/build/:num/ws', (ws, req) => {
console.log('WS Opened');
const eventListener = (be: BuildEvent) => {
if (be.id === notStupidParseInt(req.params.num)) {
ws.send(JSON.stringify(be));
}
};
this.buildController.on('log', eventListener);
ws.on('close', () => {
console.log('WS Closed');
this.buildController.removeListener('log', eventListener);
});
res.redirect(`/build/${req.params.id}/`);
});
this._webserver = this.app.listen(this.port, () => console.log(`archery is running on port ${this.port}`));
}
close = () => {
@@ -183,9 +308,33 @@ class Web {
setDB = (db: DB) => {
this.db = db;
if (!this._webserver) {
this._webserver = this.app.listen(this.port, () => console.log(`archery is running on port ${this.port}`));
}
initializeOIDC = async (options: WebConfig): Promise<OpenIDConnectStrategy | false> => {
if (!options.oidc || !options.oidc.server || !options.oidc.clientId || !options.oidc.clientSecret) {
return false;
}
const server = options.oidc.server.endsWith('/') ? options.oidc.server : `${options.oidc.server}/`;
const baseUrl = options.oidc.appBaseUrl.endsWith('/') ? options.oidc.appBaseUrl : `${options.oidc.appBaseUrl}/`;
const openidconf = await ky.get(`${server}.well-known/openid-configuration`).json<OpenIdConfiguration>();
return new OpenIDConnectStrategy({
issuer: openidconf.issuer,
authorizationURL: openidconf.authorization_endpoint,
tokenURL: openidconf.token_endpoint,
userInfoURL: openidconf.userinfo_endpoint,
clientID: options.oidc.clientId,
clientSecret: options.oidc.clientSecret,
callbackURL: `${baseUrl}cb`,
scope: 'profile'
}, async (_: string, profile: passport.Profile, cb: OpenIDConnectStrategy.VerifyCallback) => {
const userObj: User = {
id: profile.id,
username: profile.username,
displayName: profile.displayName
};
await this.db.upsertUser(userObj);
return cb(null, userObj);
});
}
setBuildController = (buildController: BuildController) => {
@@ -194,5 +343,5 @@ class Web {
}
export default Web;
export { Web };
export { Web, notStupidParseInt };
export type { WebConfig };

View File

@@ -11,7 +11,7 @@ interface compositeConfig {
db?: DBConfig
}
const config: compositeConfig = JSON.parse(await fs.promises.readFile(process.env.config || path.join('config', 'config.json'), 'utf-8'));
const config: compositeConfig = JSON.parse(await fs.promises.readFile(process.env.config || process.env.CONFIG || path.join(process.cwd(), 'config', 'config.json'), 'utf-8'));
const web = new Web(config.web);
const buildController = new BuildController();
@@ -19,6 +19,7 @@ await new Promise((resolve) => setTimeout(resolve, 1500));
const db = new DB(config.db);
web.setDB(db);
web.setBuildController(buildController);
web.initialize();
buildController.setDB(db);
process.on('SIGTERM', () => {

View File

@@ -54,6 +54,7 @@ table {
background-color: #2a2a2a;
border-collapse: collapse;
width: 100%;
min-width: 42em;
}
td,
@@ -117,10 +118,19 @@ input[type="submit"] {
}
}
span,
a {
&:has(+ button.copybtn) {
float: left;
margin-right: .75em;
}
}
.sidebar {
background: var(--primary);
width: 16em;
width: 13rem;
height: 100%;
position: fixed;
top: 0;
@@ -183,12 +193,16 @@ input[type="submit"] {
}
.content {
margin-left: 16em;
margin-left: 13rem;
padding: 1em;
&.no-padding {
padding: 0;
}
&:has(+ .content.footer) {
padding-bottom: 3em;
}
}
.overflow-x {
@@ -200,6 +214,8 @@ input[type="submit"] {
display: grid;
grid-template-columns: auto 1fr;
gap: .65em;
word-break: keep-all;
white-space: nowrap;
.span-2 {
grid-column: span 2;
@@ -207,12 +223,13 @@ input[type="submit"] {
}
.logs {
width: fit-content;
min-width: 100%;
font-family: 'Courier New', monospace;
font-size: 0.9em;
border-left: #d96a14 .2em solid;
background: #2a2a2a;
margin-top: 1em;
overflow-x: auto;
overflow-wrap: anywhere;
p {
padding: .1em .5em;
@@ -227,9 +244,13 @@ input[type="submit"] {
.content.footer {
background: var(--primary-dark);
color: var(--color-text);
margin-top: .5em;
padding: .25em 1em;
position: fixed;
bottom: 0;
left: 0;
right: 0;
a {
color: var(--color-text);
text-decoration: underline;
@@ -253,10 +274,10 @@ input[type="submit"] {
right: 1.3em;
bottom: 1.6em;
padding: 1em;
background-color: rgba(15,15,15,.8);
background-color: rgba(15, 15, 15, .8);
cursor: pointer;
input {
cursor: inherit;
}
}
}

View File

@@ -15,4 +15,32 @@
.content {
margin-left: 0;
}
.logs {
font-size: 0.75em;
}
}
@media screen and (max-width:28rem) {
body {
font-size: 13pt;
}
.grid-2col {
grid-template-columns: 1fr;
.span-2 {
grid-column: initial;
}
&> :nth-last-child(n+2):nth-child(odd) {
font-weight: bold;
font-size: 1.1em;
&::after {
content: ":";
}
}
}
}

View File

@@ -10,19 +10,19 @@
<div class="content">
<h1>Start a build</h1>
<form action="/build" method="post" class="grid-2col">
<label for="repoTxt">Repo</label> <input type="text" name="repo" id="repoTxt" required />
<label for="commitTxt">Commit</label> <input type="text" name="commit" id="commitTxt" />
<label for="patchTxt">Patch</label> <textarea name="patch" id="patchTxt"></textarea>
<label for="repoTxt">Repo</label> <input type="text" name="repo" id="repoTxt" required <% if (typeof query.repo === 'string') { %>value="<%= query.repo %>"<% } %> />
<label for="commitTxt">Commit</label> <input type="text" name="commit" id="commitTxt" <% if (typeof query.commit === 'string') { %>value="<%= query.commit %>"<% } %> />
<label for="patchTxt">Patch</label> <textarea name="patch" id="patchTxt"><% if (typeof query.patch === 'string') { %><%= query.patch %><% } %></textarea>
<label for="distroTxt">Distro</label>
<select name="distro" id="distroTxt" required>
<option value="arch">Arch</option>
<option value="artix">Artix</option>
<option value="arch" <% if (query.distro === 'arch') { %>selected<% } %>>Arch</option>
<option value="artix" <% if (query.distro === 'artix') { %>selected<% } %>>Artix</option>
</select>
<label for="dependsTxt">Dependencies</label>
<select name="dependencies" id="dependsTxt" required>
<option value="stable">Stable</option>
<option value="testing">Testing</option>
<option value="staging">Staging</option>
<option value="stable" <% if (query.dependencies === 'stable') { %>selected<% } %>>Stable</option>
<option value="testing" <% if (query.dependencies === 'testing') { %>selected<% } %>>Testing</option>
<option value="staging" <% if (query.dependencies === 'staging') { %>selected<% } %>>Staging</option>
</select>
<div class="span-2"><button type="submit">Build</button></div>
</form>

View File

@@ -10,25 +10,34 @@
<div class="content">
<h1>Build #<%= build.id %></h1>
<h2 id="buildStatus"><%= build.status %></h2>
<div class="grid-2col">
<label>Repo</label> <span><%= build.repo %></span>
<label>Commit</label> <span><% if (build.commit) { %><%= build.commit %><% } else { %>latest<% } %></span>
<label>Patch</label> <span><% if (build.patch) { %><a href="/build/<%= build.id %>/patch">patch file</a><% } else { %>none<% } %></span>
<label>Distro</label> <span><%= build.distro %></span>
<label>Dependencies</label> <span><%= build.dependencies %></span>
<label>Start time</label> <span class="to-local-time"><%= build.startTime %></span>
<div class="overflow-x">
<div class="grid-2col">
<label>Repo</label> <span><span><%= build.repo %></span> <button class="copybtn">Copy</button></span>
<label>Commit</label> <span><% if (build.commit) { %><%= build.commit %><% } else { %>latest<% } %></span>
<label>Patch</label> <span><% if (build.patch) { %><a href="/build/<%= build.sqid %>/patch">patch file</a><% } else { %>none<% } %></span>
<label>Distro</label> <span><%= build.distro %></span>
<label>Dependencies</label> <span><%= build.dependencies %></span>
<label>Start time</label> <span class="to-local-time"><%= build.startTime %></span>
<% if (build.userId && build.userId !== '-1') { %>
<label>Triggered by</label> <span><%= build.user.displayName %> (<%= build.user.username %>)</span>
<% } %>
<% if (locals.shareable) { %>
<label>Shareable link</label> <span><a href="<%= shareable %>"><%= shareable %></a> <button class="copybtn">Copy</button></span>
<% } %>
</div>
</div>
<% if (!ended) { %>
<% if (build.sqid && !public) { %>
<div>
<a href="/build/<%= build.id %>/cancel" class="button">Cancel build</a>
<a href="/build?id=<%= build.sqid %>" class="button">Clone</a>
</div>
<% } %>
<p><a href="/build/<%= build.id %>/logs">Full logs</a></p>
<div class="logs" id="logs">
<% (log || []).forEach(line => { %>
<p><%= line %></p>
<% }) %>
<% if (!ended && !public) { %>
<div id="cancelRow">
<a href="cancel" class="button">Cancel build</a>
</div>
<% } %>
<p><a href="logs">Full logs</a></p>
<pre class="overflow-x"><div class="logs" id="logs"><% (log || []).forEach(line => { %><p><%= line %></p><% }) %></div></pre>
<% if (!ended) { %>
<label id="followCheckmarkContainer" title="Follow logs">
@@ -38,8 +47,9 @@
</div>
<%- include("footer", locals) %>
<script src="/assets/js/timezone.js?v1" nonce="<%= cspNonce %>"></script>
<script src="/assets/js/copy.js?v1" nonce="<%= cspNonce %>"></script>
<% if (!ended) { %>
<script src="/assets/js/build.js?v1" nonce="<%= cspNonce %>"></script>
<script src="/assets/js/build.js?v3" nonce="<%= cspNonce %>"></script>
<% } %>
</body>
</html>

View File

@@ -1,8 +1,9 @@
<div class="content footer">
<footer>
<ul>
<li><a href="/">Home</a></li>
<li>Developed by Cory Sanin</li>
<li><a href="https://github.com/CorySanin/archery">Source</a></li>
<li><a href="https://git.sanin.dev/corysanin/archery">Source</a></li>
</ul>
</footer>
</div>

View File

@@ -14,7 +14,7 @@
<link rel="canonical" href="<%= page.canonical%>"/>
<% } %>
<link rel="shortcut icon" href="/assets/svg/favicon.svg">
<link rel="stylesheet" href="/assets/css/styles.css?v1">
<link rel="stylesheet" href="/assets/css/styles.css?v4">
<script nonce="<%= cspNonce %>">
document.addEventListener("DOMContentLoaded", function() {
document.body.classList.remove('preload');

View File

@@ -20,7 +20,7 @@
</tr>
<% builds.forEach(build => { %>
<tr>
<td><a href="/build/<%= build.id %>"><%= build.repo %></a></td>
<td><a href="/build/<%= build.sqid %>/"><%= build.repo %></a></td>
<td><%= build.distro %></td>
<td class="to-local-time"><%= build.startTime %></td>
<td><%= timeElapsed(build.startTime, build.endTime) %></td>

19
views/login-required.ejs Normal file
View File

@@ -0,0 +1,19 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<%- include("head", locals) %>
</head>
<body class="preload">
<%- include("navigation", locals) %>
<div class="content">
<h1>Authentication Required</h1>
You must log in to access that page.
<form action="/login" method="post">
<button type="submit">Log In</button>
</form>
</div>
<%- include("footer", locals) %>
</body>
</html>

View File

@@ -14,6 +14,9 @@
<li><a href="/?distro=arch">Arch Builds</a></li>
<li><a href="/?distro=artix">Artix Builds</a></li>
<li><a href="/build">New Build</a></li>
<% if (locals.user) { %>
<li><a href="/logout">Log Out</a></li>
<% } %>
</ul>
</nav>
</div>