commit
1252c5b115
14 changed files with 1143 additions and 0 deletions
@ -0,0 +1,11 @@
|
||||
__pycache__/ |
||||
*.pyc |
||||
*.pyo |
||||
*.pyd |
||||
.env |
||||
.venv/ |
||||
venv/ |
||||
.git/ |
||||
.gitignore |
||||
data/ |
||||
storage/ |
||||
@ -0,0 +1,37 @@
|
||||
# Runtime data — never commit |
||||
data/ |
||||
storage/ |
||||
|
||||
# Python bytecode |
||||
__pycache__/ |
||||
*.pyc |
||||
*.pyo |
||||
*.pyd |
||||
|
||||
# Environment files |
||||
.env |
||||
.env.* |
||||
|
||||
# Virtual environments |
||||
.venv/ |
||||
venv/ |
||||
env/ |
||||
|
||||
# Package build artifacts |
||||
*.egg-info/ |
||||
dist/ |
||||
build/ |
||||
|
||||
# Test caches |
||||
.pytest_cache/ |
||||
.mypy_cache/ |
||||
.ruff_cache/ |
||||
|
||||
# Editor metadata |
||||
.idea/ |
||||
.vscode/ |
||||
*.swp |
||||
*~ |
||||
|
||||
# macOS |
||||
.DS_Store |
||||
@ -0,0 +1,17 @@
|
||||
FROM python:3.12-slim |
||||
|
||||
WORKDIR /app |
||||
|
||||
ENV PYTHONDONTWRITEBYTECODE=1 \ |
||||
PYTHONUNBUFFERED=1 |
||||
|
||||
COPY requirements.txt . |
||||
RUN pip install --no-cache-dir -r requirements.txt |
||||
|
||||
COPY app ./app |
||||
|
||||
RUN mkdir -p /app/data /app/storage |
||||
|
||||
EXPOSE 8000 |
||||
|
||||
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"] |
||||
@ -0,0 +1,406 @@
|
||||
# Cellar |
||||
|
||||
**Author:** Matteo Benedetto |
||||
|
||||
`Cellar` is a minimal self-hosted prototype for archiving exported [Bottles](https://usebottles.com/) environments and restoring them on another Linux system. |
||||
|
||||
The name refers to a wine cellar — the place where bottles are stored and preserved. |
||||
|
||||
The project currently provides: |
||||
|
||||
- a `FastAPI` server |
||||
- a local `SQLite` database for archive metadata |
||||
- local file storage for uploaded bottle backups |
||||
- a minimal Python CLI client |
||||
- a `Docker Compose` setup for running the server quickly |
||||
|
||||
The current goal is simple: make it possible to back up a local Bottles bottle, upload it to a Cellar server, list what is stored, and reinstall a bottle by name. |
||||
|
||||
--- |
||||
|
||||
## Current status |
||||
eventi ora è no c'è una cosa comune cinque |
||||
This is an early MVP. |
||||
|
||||
Implemented features: |
||||
|
||||
- health endpoint |
||||
- archive upload |
||||
- archive listing |
||||
- archive detail lookup |
||||
- archive download |
||||
- archive deletion |
||||
- local Bottles scan from the client |
||||
- interactive backup + upload wizard from the client |
||||
- install a bottle from the server with a simple command: |
||||
- `python client.py install <bottle-name>` |
||||
|
||||
Not implemented yet: |
||||
|
||||
- authentication |
||||
- permissions / multi-user separation |
||||
- editing metadata already stored in the database |
||||
- archive versioning |
||||
- deduplication |
||||
- tags normalization |
||||
- server-side search/filtering |
||||
- direct Bottles GUI integration |
||||
|
||||
--- |
||||
|
||||
## Project structure |
||||
|
||||
- [Dockerfile](Dockerfile) — container image for the API |
||||
- [docker-compose.yml](docker-compose.yml) — local deployment |
||||
- [requirements.txt](requirements.txt) — Python dependencies |
||||
- [client.py](client.py) — minimal CLI client |
||||
- [app/main.py](app/main.py) — API routes |
||||
- [app/models.py](app/models.py) — SQLAlchemy model |
||||
- [app/schemas.py](app/schemas.py) — API response schemas |
||||
- [app/database.py](app/database.py) — database setup |
||||
- [app/storage.py](app/storage.py) — file storage helpers |
||||
- [app/config.py](app/config.py) — environment/config handling |
||||
- [data/](data/) — SQLite database location |
||||
- [storage/](storage/) — uploaded archive files |
||||
|
||||
--- |
||||
|
||||
## Architecture |
||||
|
||||
### Server |
||||
The server is built with `FastAPI` and stores metadata in `SQLite`. |
||||
|
||||
Each uploaded archive creates: |
||||
|
||||
1. a physical file in [storage/](storage/) |
||||
2. a metadata record in the SQLite database in [data/](data/) |
||||
|
||||
Stored metadata includes: |
||||
|
||||
- archive display name |
||||
- bottle name |
||||
- description |
||||
- tags |
||||
- architecture |
||||
- runner |
||||
- Windows version |
||||
- original file name |
||||
- stored file name |
||||
- content type |
||||
- file size |
||||
- SHA256 digest |
||||
- creation time |
||||
|
||||
### Client |
||||
The client is intentionally minimal and uses the Python standard library. |
||||
|
||||
It can: |
||||
|
||||
- list archives on the server |
||||
- upload an existing archive file |
||||
- download an archive by id |
||||
- scan local Bottles bottles |
||||
- run a wizard to choose a local bottle, create a backup, and upload it |
||||
- install a remote bottle by name |
||||
|
||||
### Storage format |
||||
The current client creates a `.tar.gz` backup of the selected bottle directory. |
||||
|
||||
The install flow: |
||||
|
||||
1. finds the archive by bottle name |
||||
2. downloads it |
||||
3. extracts it safely |
||||
4. places it into the local Bottles directory |
||||
|
||||
Because Bottles backups contain `dosdevices` symlinks, extraction uses a trusted tar extraction mode while still enforcing a path traversal safety check. |
||||
|
||||
--- |
||||
|
||||
## Requirements |
||||
|
||||
Recommended local environment: |
||||
|
||||
- Docker |
||||
- Docker Compose |
||||
- Python 3.12+ |
||||
- Bottles installed locally if you want to use `scan-local`, `wizard-upload`, or `install` |
||||
|
||||
Expected Bottles path used by the client by default: |
||||
|
||||
- `~/.var/app/com.usebottles.bottles/data/bottles/bottles` |
||||
|
||||
This matches the Flatpak Bottles installation layout. |
||||
|
||||
--- |
||||
|
||||
## Running the server |
||||
|
||||
From the project root run: |
||||
|
||||
`docker compose up --build` |
||||
|
||||
The API is exposed on: |
||||
|
||||
- `http://127.0.0.1:8080` |
||||
|
||||
Interactive API docs are available at: |
||||
|
||||
- `http://127.0.0.1:8080/docs` |
||||
|
||||
--- |
||||
|
||||
## Environment configuration |
||||
|
||||
Current environment variables used by the API container: |
||||
|
||||
- `DATABASE_URL` |
||||
- `STORAGE_DIR` |
||||
|
||||
Current defaults in [docker-compose.yml](docker-compose.yml): |
||||
|
||||
- database: `sqlite:////app/data/bottle_archive.db` |
||||
- storage: `/app/storage` |
||||
|
||||
Host directories are mounted as volumes: |
||||
|
||||
- [data/](data/) → `/app/data` |
||||
- [storage/](storage/) → `/app/storage` |
||||
|
||||
--- |
||||
|
||||
## API endpoints |
||||
|
||||
### `GET /health` |
||||
Simple health check. |
||||
|
||||
### `GET /archives` |
||||
Returns the list of archives ordered by creation time descending. |
||||
|
||||
### `POST /archives` |
||||
Uploads a new archive file and stores metadata. |
||||
|
||||
Form fields supported: |
||||
|
||||
- `file` |
||||
- `name` |
||||
- `bottle_name` |
||||
- `description` |
||||
- `tags` |
||||
- `arch` |
||||
- `runner` |
||||
- `windows_version` |
||||
|
||||
### `GET /archives/{archive_id}` |
||||
Returns one archive metadata entry. |
||||
|
||||
### `GET /archives/{archive_id}/download` |
||||
Downloads the archive file. |
||||
|
||||
### `DELETE /archives/{archive_id}` |
||||
Deletes the archive file and the corresponding metadata row. |
||||
|
||||
--- |
||||
|
||||
## Client commands |
||||
|
||||
All commands below assume you are inside the project root. |
||||
|
||||
### List remote archives |
||||
`python client.py list` |
||||
|
||||
### Upload an existing archive file |
||||
`python client.py upload /path/to/archive.tar.gz --name "Empire Earth Gold" --bottle-name "Empire-Earth-Gold"` |
||||
|
||||
Optional extra metadata: |
||||
|
||||
`python client.py upload /path/to/archive.tar.gz --name "Empire Earth Gold" --bottle-name "Empire-Earth-Gold" --description "Backup from desktop" --tags "games,gog" --arch win32 --runner soda-9.0-1 --windows-version win7` |
||||
|
||||
### Download an archive by id |
||||
`python client.py download 1 ./downloads/` |
||||
|
||||
### Scan local Bottles bottles |
||||
`python client.py scan-local` |
||||
|
||||
JSON output: |
||||
|
||||
`python client.py scan-local --json` |
||||
|
||||
### Interactive wizard: choose local bottle, back it up, upload it |
||||
`python client.py wizard-upload` |
||||
|
||||
### Install a bottle from the server by name |
||||
`python client.py install Empire-Earth-Gold` |
||||
|
||||
Replace existing local bottle: |
||||
|
||||
`python client.py install Empire-Earth-Gold --replace` |
||||
|
||||
Use a custom Bottles directory: |
||||
|
||||
`python client.py install Empire-Earth-Gold --bottles-dir /custom/path` |
||||
|
||||
--- |
||||
|
||||
## Example workflow |
||||
|
||||
### Backup and upload |
||||
1. Start the server: |
||||
- `docker compose up --build` |
||||
2. Scan local Bottles: |
||||
- `python client.py scan-local` |
||||
3. Start the wizard: |
||||
- `python client.py wizard-upload` |
||||
4. Choose one of the local bottles |
||||
5. Confirm archive metadata |
||||
6. The client creates a temporary `.tar.gz` backup and uploads it |
||||
|
||||
### Restore on another machine |
||||
1. Start the same server or connect to a reachable one |
||||
2. Install the bottle by name: |
||||
- `python client.py install Empire-Earth-Gold` |
||||
3. Check local Bottles: |
||||
- `python client.py scan-local` |
||||
|
||||
--- |
||||
|
||||
## Known limitations |
||||
|
||||
### 1. No authentication |
||||
At the moment, **anyone who can reach the API can list, download, upload, and delete archives**. |
||||
|
||||
This is acceptable only for a local prototype or a trusted LAN environment. |
||||
|
||||
### 2. No metadata editing |
||||
Once an archive is inserted in the database, there is currently **no API or client command to update its metadata**. |
||||
|
||||
That means fields like: |
||||
|
||||
- `name` |
||||
- `description` |
||||
- `tags` |
||||
- `runner` |
||||
- `windows_version` |
||||
|
||||
cannot yet be corrected or refined without direct database manipulation or deleting and re-uploading the entry. |
||||
|
||||
### 3. No versions/history model |
||||
Multiple uploads of the same bottle are just separate rows. There is no version chain or “latest version” logic yet. |
||||
|
||||
### 4. Local storage only |
||||
The server stores everything on local disk. There is no S3/MinIO/remote object storage integration yet. |
||||
|
||||
### 5. Flatpak Bottles path assumption |
||||
The default client path targets a Flatpak Bottles installation. Native Bottles layouts may need `--bottles-dir`. |
||||
|
||||
--- |
||||
|
||||
## Next phases |
||||
|
||||
The next development phases should focus on two critical areas. |
||||
|
||||
### Phase 1: Authentication system integration |
||||
This is the most important missing feature. |
||||
|
||||
The project needs an authentication layer so the server can be used safely beyond a private local test. |
||||
|
||||
Recommended additions: |
||||
|
||||
- user accounts |
||||
- password hashing |
||||
- token-based authentication |
||||
- per-user archive ownership |
||||
- permission checks on download/delete/edit operations |
||||
- optional admin role |
||||
|
||||
Possible implementation direction: |
||||
|
||||
- FastAPI auth routes |
||||
- JWT access tokens or session tokens |
||||
- password hashing with a standard library such as `passlib`/`bcrypt` |
||||
- `users` table + relation from archive rows to owners |
||||
|
||||
Minimum target for this phase: |
||||
|
||||
- `POST /auth/register` |
||||
- `POST /auth/login` |
||||
- protected archive routes |
||||
- client support for login and storing a token locally |
||||
|
||||
### Phase 2: Database entry editing system |
||||
The second required feature is a proper metadata editing workflow. |
||||
|
||||
Right now archive metadata is write-once. That is too limiting. |
||||
|
||||
The project needs a system to edit database entries safely from the API and the client. |
||||
|
||||
Recommended additions: |
||||
|
||||
- `PATCH /archives/{id}` endpoint |
||||
- validation rules for editable fields |
||||
- audit fields like `updated_at` |
||||
- optional edit history |
||||
- client command such as: |
||||
- `python client.py edit 12 --name "..." --tags "..."` |
||||
|
||||
Editable fields should include at least: |
||||
|
||||
- `name` |
||||
- `bottle_name` |
||||
- `description` |
||||
- `tags` |
||||
- `arch` |
||||
- `runner` |
||||
- `windows_version` |
||||
|
||||
This phase is important because real archives often need cleanup after upload. |
||||
|
||||
--- |
||||
|
||||
## Recommended roadmap |
||||
|
||||
### Short term |
||||
- add authentication |
||||
- add metadata editing |
||||
- add archive search/filter endpoints |
||||
- add better error messages in client install/upload flows |
||||
|
||||
### Medium term |
||||
- add versioning for the same bottle |
||||
- add per-user archive namespaces |
||||
- add pagination on `GET /archives` |
||||
- add import/export manifest support |
||||
- add `upload-local <bottle-name>` non-interactive command |
||||
|
||||
### Long term |
||||
- replace local storage with S3/MinIO |
||||
- add a small web UI |
||||
- add full-text search on archive metadata |
||||
- add deduplication / retention policies |
||||
- integrate directly with Bottles workflows more tightly |
||||
|
||||
--- |
||||
|
||||
## Security note |
||||
|
||||
Until authentication is implemented, do **not** expose this API publicly. |
||||
|
||||
Use it only: |
||||
|
||||
- on localhost |
||||
- on a trusted home LAN |
||||
- behind an authenticated reverse proxy if temporarily needed |
||||
|
||||
--- |
||||
|
||||
## Development notes |
||||
|
||||
The project is currently intentionally simple. |
||||
|
||||
That simplicity is useful because it makes the next steps clear: |
||||
|
||||
1. secure access |
||||
2. allow metadata editing |
||||
3. improve archive lifecycle management |
||||
|
||||
Those two immediate next phases — **authentication** and **database entry editing** — should be considered required before treating this project as more than a local prototype. |
||||
@ -0,0 +1,10 @@
|
||||
from pathlib import Path |
||||
import os |
||||
|
||||
BASE_DIR = Path(__file__).resolve().parent.parent |
||||
DATA_DIR = Path(os.getenv("DATA_DIR", BASE_DIR / "data")) |
||||
STORAGE_DIR = Path(os.getenv("STORAGE_DIR", BASE_DIR / "storage")) |
||||
DATABASE_URL = os.getenv( |
||||
"DATABASE_URL", |
||||
f"sqlite:///{(DATA_DIR / 'bottle_archive.db').as_posix()}", |
||||
) |
||||
@ -0,0 +1,18 @@
|
||||
from sqlalchemy import create_engine |
||||
from sqlalchemy.orm import declarative_base, sessionmaker |
||||
|
||||
from app.config import DATABASE_URL |
||||
|
||||
connect_args = {"check_same_thread": False} if DATABASE_URL.startswith("sqlite") else {} |
||||
|
||||
engine = create_engine(DATABASE_URL, connect_args=connect_args) |
||||
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine) |
||||
Base = declarative_base() |
||||
|
||||
|
||||
def get_db(): |
||||
db = SessionLocal() |
||||
try: |
||||
yield db |
||||
finally: |
||||
db.close() |
||||
@ -0,0 +1,114 @@
|
||||
from __future__ import annotations |
||||
|
||||
from contextlib import asynccontextmanager |
||||
|
||||
from fastapi import Depends, FastAPI, File, Form, HTTPException, UploadFile, status |
||||
from fastapi.responses import FileResponse |
||||
from sqlalchemy import select |
||||
from sqlalchemy.orm import Session |
||||
|
||||
from app.config import DATA_DIR, STORAGE_DIR |
||||
from app.database import Base, engine, get_db |
||||
from app.models import Archive |
||||
from app.schemas import ArchiveRead |
||||
from app.storage import delete_file, ensure_storage_dirs, resolve_path, save_upload |
||||
|
||||
|
||||
@asynccontextmanager |
||||
async def lifespan(_: FastAPI): |
||||
DATA_DIR.mkdir(parents=True, exist_ok=True) |
||||
STORAGE_DIR.mkdir(parents=True, exist_ok=True) |
||||
Base.metadata.create_all(bind=engine) |
||||
ensure_storage_dirs() |
||||
yield |
||||
|
||||
|
||||
app = FastAPI( |
||||
title="Bottle Archive Server", |
||||
version="0.1.0", |
||||
description="Local archive server for exported Bottles backups.", |
||||
lifespan=lifespan, |
||||
) |
||||
|
||||
|
||||
@app.get("/health") |
||||
def health() -> dict[str, str]: |
||||
return {"status": "ok"} |
||||
|
||||
|
||||
@app.get("/archives", response_model=list[ArchiveRead]) |
||||
def list_archives(db: Session = Depends(get_db)) -> list[Archive]: |
||||
return list(db.scalars(select(Archive).order_by(Archive.created_at.desc()))) |
||||
|
||||
|
||||
@app.post("/archives", response_model=ArchiveRead, status_code=status.HTTP_201_CREATED) |
||||
async def create_archive( |
||||
file: UploadFile = File(...), |
||||
name: str = Form(...), |
||||
bottle_name: str | None = Form(default=None), |
||||
description: str | None = Form(default=None), |
||||
tags: str | None = Form(default=None), |
||||
arch: str | None = Form(default=None), |
||||
runner: str | None = Form(default=None), |
||||
windows_version: str | None = Form(default=None), |
||||
db: Session = Depends(get_db), |
||||
) -> Archive: |
||||
if not file.filename: |
||||
raise HTTPException(status_code=400, detail="Uploaded file must have a filename.") |
||||
|
||||
stored_name, size_bytes, digest = await save_upload(file) |
||||
|
||||
archive = Archive( |
||||
name=name, |
||||
bottle_name=bottle_name, |
||||
description=description, |
||||
tags=tags, |
||||
arch=arch, |
||||
runner=runner, |
||||
windows_version=windows_version, |
||||
file_name=file.filename, |
||||
stored_name=stored_name, |
||||
content_type=file.content_type, |
||||
size_bytes=size_bytes, |
||||
sha256=digest, |
||||
) |
||||
db.add(archive) |
||||
db.commit() |
||||
db.refresh(archive) |
||||
return archive |
||||
|
||||
|
||||
@app.get("/archives/{archive_id}", response_model=ArchiveRead) |
||||
def get_archive(archive_id: int, db: Session = Depends(get_db)) -> Archive: |
||||
archive = db.get(Archive, archive_id) |
||||
if archive is None: |
||||
raise HTTPException(status_code=404, detail="Archive not found.") |
||||
return archive |
||||
|
||||
|
||||
@app.get("/archives/{archive_id}/download") |
||||
def download_archive(archive_id: int, db: Session = Depends(get_db)) -> FileResponse: |
||||
archive = db.get(Archive, archive_id) |
||||
if archive is None: |
||||
raise HTTPException(status_code=404, detail="Archive not found.") |
||||
|
||||
path = resolve_path(archive.stored_name) |
||||
if not path.exists(): |
||||
raise HTTPException(status_code=404, detail="Stored file not found.") |
||||
|
||||
return FileResponse( |
||||
path=path, |
||||
media_type=archive.content_type or "application/octet-stream", |
||||
filename=archive.file_name, |
||||
) |
||||
|
||||
|
||||
@app.delete("/archives/{archive_id}", status_code=status.HTTP_204_NO_CONTENT) |
||||
def delete_archive(archive_id: int, db: Session = Depends(get_db)) -> None: |
||||
archive = db.get(Archive, archive_id) |
||||
if archive is None: |
||||
raise HTTPException(status_code=404, detail="Archive not found.") |
||||
|
||||
delete_file(archive.stored_name) |
||||
db.delete(archive) |
||||
db.commit() |
||||
@ -0,0 +1,25 @@
|
||||
from sqlalchemy import DateTime, Integer, String, Text, func |
||||
from sqlalchemy.orm import Mapped, mapped_column |
||||
|
||||
from app.database import Base |
||||
|
||||
|
||||
class Archive(Base): |
||||
__tablename__ = "archives" |
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, index=True) |
||||
name: Mapped[str] = mapped_column(String(255), nullable=False) |
||||
bottle_name: Mapped[str | None] = mapped_column(String(255), nullable=True) |
||||
description: Mapped[str | None] = mapped_column(Text, nullable=True) |
||||
tags: Mapped[str | None] = mapped_column(Text, nullable=True) |
||||
arch: Mapped[str | None] = mapped_column(String(32), nullable=True) |
||||
runner: Mapped[str | None] = mapped_column(String(128), nullable=True) |
||||
windows_version: Mapped[str | None] = mapped_column(String(32), nullable=True) |
||||
file_name: Mapped[str] = mapped_column(String(255), nullable=False) |
||||
stored_name: Mapped[str] = mapped_column(String(255), unique=True, nullable=False) |
||||
content_type: Mapped[str | None] = mapped_column(String(255), nullable=True) |
||||
size_bytes: Mapped[int] = mapped_column(Integer, nullable=False) |
||||
sha256: Mapped[str] = mapped_column(String(64), nullable=False, index=True) |
||||
created_at: Mapped[str] = mapped_column( |
||||
DateTime(timezone=True), server_default=func.now(), nullable=False |
||||
) |
||||
@ -0,0 +1,28 @@
|
||||
from datetime import datetime |
||||
|
||||
from pydantic import BaseModel, ConfigDict |
||||
|
||||
|
||||
class ArchiveBase(BaseModel): |
||||
name: str |
||||
bottle_name: str | None = None |
||||
description: str | None = None |
||||
tags: str | None = None |
||||
arch: str | None = None |
||||
runner: str | None = None |
||||
windows_version: str | None = None |
||||
|
||||
|
||||
class ArchiveCreate(ArchiveBase): |
||||
pass |
||||
|
||||
|
||||
class ArchiveRead(ArchiveBase): |
||||
model_config = ConfigDict(from_attributes=True) |
||||
|
||||
id: int |
||||
file_name: str |
||||
content_type: str | None = None |
||||
size_bytes: int |
||||
sha256: str |
||||
created_at: datetime |
||||
@ -0,0 +1,46 @@
|
||||
from __future__ import annotations |
||||
|
||||
from hashlib import sha256 |
||||
from pathlib import Path |
||||
from uuid import uuid4 |
||||
|
||||
from fastapi import UploadFile |
||||
|
||||
from app.config import STORAGE_DIR |
||||
|
||||
CHUNK_SIZE = 1024 * 1024 |
||||
|
||||
|
||||
def ensure_storage_dirs() -> None: |
||||
STORAGE_DIR.mkdir(parents=True, exist_ok=True) |
||||
|
||||
|
||||
async def save_upload(file: UploadFile) -> tuple[str, int, str]: |
||||
ensure_storage_dirs() |
||||
|
||||
suffix = Path(file.filename or "archive.tar.gz").suffixes |
||||
extension = "".join(suffix) if suffix else ".bin" |
||||
stored_name = f"{uuid4().hex}{extension}" |
||||
destination = STORAGE_DIR / stored_name |
||||
|
||||
digest = sha256() |
||||
size = 0 |
||||
|
||||
with destination.open("wb") as buffer: |
||||
while chunk := await file.read(CHUNK_SIZE): |
||||
size += len(chunk) |
||||
digest.update(chunk) |
||||
buffer.write(chunk) |
||||
|
||||
await file.close() |
||||
return stored_name, size, digest.hexdigest() |
||||
|
||||
|
||||
def delete_file(stored_name: str) -> None: |
||||
path = STORAGE_DIR / stored_name |
||||
if path.exists(): |
||||
path.unlink() |
||||
|
||||
|
||||
def resolve_path(stored_name: str) -> Path: |
||||
return STORAGE_DIR / stored_name |
||||
@ -0,0 +1,412 @@
|
||||
#!/usr/bin/env python3 |
||||
from __future__ import annotations |
||||
|
||||
import argparse |
||||
import json |
||||
import shutil |
||||
import sys |
||||
import tarfile |
||||
import tempfile |
||||
import urllib.error |
||||
import urllib.parse |
||||
import urllib.request |
||||
from datetime import datetime |
||||
from pathlib import Path |
||||
from typing import Any |
||||
|
||||
DEFAULT_SERVER = "http://127.0.0.1:8080" |
||||
DEFAULT_BOTTLES_DIR = Path.home() / ".var/app/com.usebottles.bottles/data/bottles/bottles" |
||||
|
||||
|
||||
def request_json(url: str, method: str = "GET", data: bytes | None = None, headers: dict[str, str] | None = None): |
||||
req = urllib.request.Request(url, data=data, method=method) |
||||
for key, value in (headers or {}).items(): |
||||
req.add_header(key, value) |
||||
with urllib.request.urlopen(req) as response: |
||||
return json.loads(response.read().decode("utf-8")) |
||||
|
||||
|
||||
def print_archives(server: str) -> int: |
||||
archives = request_json(f"{server}/archives") |
||||
if not archives: |
||||
print("No archives found.") |
||||
return 0 |
||||
|
||||
print(f"{'ID':<4} {'Name':<30} {'Bottle':<30} {'Arch':<8} {'Runner':<18} {'Size(MB)':>10}") |
||||
print("-" * 110) |
||||
for item in archives: |
||||
size_mb = item["size_bytes"] / (1024 * 1024) |
||||
print( |
||||
f"{item['id']:<4} " |
||||
f"{(item['name'] or '-')[:30]:<30} " |
||||
f"{(item.get('bottle_name') or '-')[:30]:<30} " |
||||
f"{(item.get('arch') or '-'):<8} " |
||||
f"{(item.get('runner') or '-')[:18]:<18} " |
||||
f"{size_mb:>10.2f}" |
||||
) |
||||
return 0 |
||||
|
||||
|
||||
def upload_archive(server: str, file_path: Path, fields: dict[str, str]) -> int: |
||||
boundary = "----BottleArchiveBoundary7MA4YWxkTrZu0gW" |
||||
data = [] |
||||
|
||||
def add_field(field_name: str, value: str): |
||||
data.extend( |
||||
[ |
||||
f"--{boundary}\r\n".encode(), |
||||
f'Content-Disposition: form-data; name="{field_name}"\r\n\r\n'.encode(), |
||||
value.encode(), |
||||
b"\r\n", |
||||
] |
||||
) |
||||
|
||||
for key, value in fields.items(): |
||||
if value: |
||||
add_field(key, value) |
||||
|
||||
filename = file_path.name |
||||
mime = "application/gzip" if filename.endswith((".tar.gz", ".tgz", ".gz")) else "application/octet-stream" |
||||
data.extend( |
||||
[ |
||||
f"--{boundary}\r\n".encode(), |
||||
f'Content-Disposition: form-data; name="file"; filename="{filename}"\r\n'.encode(), |
||||
f"Content-Type: {mime}\r\n\r\n".encode(), |
||||
file_path.read_bytes(), |
||||
b"\r\n", |
||||
f"--{boundary}--\r\n".encode(), |
||||
] |
||||
) |
||||
|
||||
payload = b"".join(data) |
||||
archive = request_json( |
||||
f"{server}/archives", |
||||
method="POST", |
||||
data=payload, |
||||
headers={"Content-Type": f"multipart/form-data; boundary={boundary}"}, |
||||
) |
||||
print(f"Uploaded archive #{archive['id']}: {archive['name']}") |
||||
return 0 |
||||
|
||||
|
||||
def download_archive(server: str, archive_id: int, output: Path) -> int: |
||||
url = f"{server}/archives/{archive_id}/download" |
||||
req = urllib.request.Request(url, method="GET") |
||||
with urllib.request.urlopen(req) as response: |
||||
if output.is_dir(): |
||||
filename = response.headers.get_filename() or f"archive-{archive_id}.bin" |
||||
destination = output / filename |
||||
else: |
||||
destination = output |
||||
destination.write_bytes(response.read()) |
||||
print(f"Downloaded to {destination}") |
||||
return 0 |
||||
|
||||
|
||||
def find_remote_archive(server: str, bottle_ref: str) -> dict[str, Any]: |
||||
archives = request_json(f"{server}/archives") |
||||
bottle_ref_lower = bottle_ref.lower() |
||||
|
||||
for item in archives: |
||||
if (item.get("bottle_name") or "").lower() == bottle_ref_lower: |
||||
return item |
||||
for item in archives: |
||||
if (item.get("name") or "").lower() == bottle_ref_lower: |
||||
return item |
||||
|
||||
raise FileNotFoundError(f"No remote archive found for '{bottle_ref}'") |
||||
|
||||
|
||||
def safe_extract_tar(archive_path: Path, target_dir: Path) -> None: |
||||
with tarfile.open(archive_path, "r:gz") as tar: |
||||
for member in tar.getmembers(): |
||||
member_path = (target_dir / member.name).resolve() |
||||
if not str(member_path).startswith(str(target_dir.resolve())): |
||||
raise ValueError("Unsafe archive path detected.") |
||||
tar.extractall(target_dir, filter="fully_trusted") |
||||
|
||||
|
||||
def install_archive(server: str, bottle_ref: str, bottles_dir: Path, replace: bool) -> int: |
||||
archive = find_remote_archive(server, bottle_ref) |
||||
bottle_name = archive.get("bottle_name") or archive.get("name") or bottle_ref |
||||
target_dir = bottles_dir / bottle_name |
||||
|
||||
if target_dir.exists(): |
||||
if not replace: |
||||
print( |
||||
f"Bottle already exists: {target_dir}\n" |
||||
"Use --replace to overwrite it.", |
||||
file=sys.stderr, |
||||
) |
||||
return 1 |
||||
shutil.rmtree(target_dir) |
||||
|
||||
bottles_dir.mkdir(parents=True, exist_ok=True) |
||||
|
||||
with tempfile.TemporaryDirectory(prefix="bottle-install-") as tmp_dir: |
||||
tmp_path = Path(tmp_dir) |
||||
download_path = tmp_path / f"archive-{archive['id']}.tar.gz" |
||||
download_archive(server, int(archive["id"]), download_path) |
||||
|
||||
extract_dir = tmp_path / "extract" |
||||
extract_dir.mkdir(parents=True, exist_ok=True) |
||||
safe_extract_tar(download_path, extract_dir) |
||||
|
||||
candidates = [path for path in extract_dir.iterdir() if path.is_dir()] |
||||
if not candidates: |
||||
raise FileNotFoundError("Archive did not contain a bottle directory.") |
||||
|
||||
source_dir = candidates[0] |
||||
bottle_yml = source_dir / "bottle.yml" |
||||
if not bottle_yml.exists(): |
||||
raise FileNotFoundError("Archive does not look like a valid bottle backup.") |
||||
|
||||
shutil.move(str(source_dir), str(target_dir)) |
||||
|
||||
print(f"Installed bottle '{bottle_name}' to {target_dir}") |
||||
return 0 |
||||
|
||||
|
||||
def parse_bottle_yml(bottle_yml: Path) -> dict[str, str]: |
||||
data: dict[str, str] = {} |
||||
wanted_keys = {"Name", "Arch", "Runner", "Environment", "Windows"} |
||||
|
||||
for line in bottle_yml.read_text(encoding="utf-8", errors="ignore").splitlines(): |
||||
if ":" not in line or line.startswith(" "): |
||||
continue |
||||
key, value = line.split(":", 1) |
||||
if key in wanted_keys: |
||||
data[key] = value.strip().strip("'") |
||||
|
||||
return data |
||||
|
||||
|
||||
def collect_local_bottles(bottles_dir: Path) -> list[dict[str, Any]]: |
||||
if not bottles_dir.exists(): |
||||
return [] |
||||
|
||||
bottles: list[dict[str, Any]] = [] |
||||
for directory in sorted(path for path in bottles_dir.iterdir() if path.is_dir()): |
||||
bottle_yml = directory / "bottle.yml" |
||||
if not bottle_yml.exists(): |
||||
continue |
||||
|
||||
metadata = parse_bottle_yml(bottle_yml) |
||||
bottles.append( |
||||
{ |
||||
"name": metadata.get("Name", directory.name), |
||||
"directory": directory.name, |
||||
"path": str(directory), |
||||
"arch": metadata.get("Arch", "-"), |
||||
"runner": metadata.get("Runner", "-"), |
||||
"environment": metadata.get("Environment", "-"), |
||||
"windows": metadata.get("Windows", "-"), |
||||
} |
||||
) |
||||
|
||||
return bottles |
||||
|
||||
|
||||
def scan_local_bottles(bottles_dir: Path, as_json: bool) -> int: |
||||
bottles = collect_local_bottles(bottles_dir) |
||||
if as_json: |
||||
print(json.dumps(bottles, indent=2)) |
||||
return 0 |
||||
|
||||
if not bottles: |
||||
print(f"No local bottles found in {bottles_dir}") |
||||
return 0 |
||||
|
||||
print(f"Local Bottles directory: {bottles_dir}") |
||||
print(f"{'Dir':<24} {'Name':<28} {'Arch':<8} {'Runner':<18} {'Env':<12} {'Windows':<10}") |
||||
print("-" * 110) |
||||
for bottle in bottles: |
||||
print( |
||||
f"{bottle['directory'][:24]:<24} " |
||||
f"{bottle['name'][:28]:<28} " |
||||
f"{bottle['arch']:<8} " |
||||
f"{bottle['runner'][:18]:<18} " |
||||
f"{bottle['environment'][:12]:<12} " |
||||
f"{bottle['windows'][:10]:<10}" |
||||
) |
||||
return 0 |
||||
|
||||
|
||||
def create_bottle_backup(bottle: dict[str, Any], output_dir: Path | None = None) -> Path: |
||||
source_dir = Path(bottle["path"]) |
||||
target_dir = output_dir or Path(tempfile.gettempdir()) |
||||
timestamp = datetime.now().strftime("%Y%m%d-%H%M%S") |
||||
archive_name = f"{bottle['directory']}-{timestamp}.tar.gz" |
||||
archive_path = target_dir / archive_name |
||||
|
||||
with tarfile.open(archive_path, "w:gz") as tar: |
||||
tar.add(source_dir, arcname=source_dir.name) |
||||
|
||||
return archive_path |
||||
|
||||
|
||||
def prompt_text(label: str, default: str) -> str: |
||||
value = input(f"{label} [{default}]: ").strip() |
||||
return value or default |
||||
|
||||
|
||||
def choose_bottle(bottles: list[dict[str, Any]]) -> dict[str, Any]: |
||||
print("Available local bottles:") |
||||
for index, bottle in enumerate(bottles, start=1): |
||||
print( |
||||
f" {index}. {bottle['directory']} " |
||||
f"(arch={bottle['arch']}, runner={bottle['runner']}, windows={bottle['windows']})" |
||||
) |
||||
|
||||
while True: |
||||
raw = input("Select a bottle number: ").strip() |
||||
try: |
||||
choice = int(raw) |
||||
except ValueError: |
||||
print("Please enter a valid number.") |
||||
continue |
||||
|
||||
if 1 <= choice <= len(bottles): |
||||
return bottles[choice - 1] |
||||
|
||||
print("Choice out of range.") |
||||
|
||||
|
||||
def wizard_upload(server: str, bottles_dir: Path) -> int: |
||||
bottles = collect_local_bottles(bottles_dir) |
||||
if not bottles: |
||||
print(f"No local bottles found in {bottles_dir}") |
||||
return 0 |
||||
|
||||
bottle = choose_bottle(bottles) |
||||
display_name = prompt_text("Archive name", bottle["name"]) |
||||
bottle_name = prompt_text("Bottle name", bottle["directory"]) |
||||
description = prompt_text("Description", f"Backup of {bottle['name']}") |
||||
tags = prompt_text("Tags", "bottles,backup") |
||||
|
||||
print(f"\nCreating backup for {bottle['directory']}...") |
||||
archive_path = create_bottle_backup(bottle) |
||||
print(f"Backup created: {archive_path}") |
||||
|
||||
try: |
||||
return upload_archive( |
||||
server, |
||||
archive_path, |
||||
{ |
||||
"name": display_name, |
||||
"bottle_name": bottle_name, |
||||
"description": description, |
||||
"tags": tags, |
||||
"arch": str(bottle.get("arch", "")), |
||||
"runner": str(bottle.get("runner", "")), |
||||
"windows_version": str(bottle.get("windows", "")), |
||||
}, |
||||
) |
||||
finally: |
||||
archive_path.unlink(missing_ok=True) |
||||
|
||||
|
||||
def build_parser() -> argparse.ArgumentParser: |
||||
parser = argparse.ArgumentParser(description="Minimal client for Bottle Archive Server") |
||||
parser.add_argument("--server", default=DEFAULT_SERVER, help="Base server URL") |
||||
|
||||
subparsers = parser.add_subparsers(dest="command", required=True) |
||||
|
||||
subparsers.add_parser("list", help="List uploaded bottle archives") |
||||
|
||||
upload_parser = subparsers.add_parser("upload", help="Upload a bottle archive") |
||||
upload_parser.add_argument("file", type=Path, help="Path to the archive file") |
||||
upload_parser.add_argument("--name", required=True, help="Display name") |
||||
upload_parser.add_argument("--bottle-name", help="Bottle name") |
||||
upload_parser.add_argument("--description", help="Description") |
||||
upload_parser.add_argument("--tags", help="Tags") |
||||
upload_parser.add_argument("--arch", help="Architecture") |
||||
upload_parser.add_argument("--runner", help="Runner") |
||||
upload_parser.add_argument("--windows-version", help="Windows version") |
||||
|
||||
download_parser = subparsers.add_parser("download", help="Download an archive by id") |
||||
download_parser.add_argument("archive_id", type=int, help="Archive id") |
||||
download_parser.add_argument("output", type=Path, help="Output file or directory") |
||||
|
||||
install_parser = subparsers.add_parser( |
||||
"install", |
||||
help="Install a bottle from the server by bottle name or archive name", |
||||
) |
||||
install_parser.add_argument("bottle", help="Bottle name or archive name to install") |
||||
install_parser.add_argument( |
||||
"--bottles-dir", |
||||
type=Path, |
||||
default=DEFAULT_BOTTLES_DIR, |
||||
help="Path to the local Bottles directory", |
||||
) |
||||
install_parser.add_argument("--replace", action="store_true", help="Replace existing local bottle") |
||||
|
||||
scan_parser = subparsers.add_parser("scan-local", help="List bottles installed on this computer") |
||||
scan_parser.add_argument( |
||||
"--bottles-dir", |
||||
type=Path, |
||||
default=DEFAULT_BOTTLES_DIR, |
||||
help="Path to the local Bottles directory", |
||||
) |
||||
scan_parser.add_argument("--json", action="store_true", help="Print results as JSON") |
||||
|
||||
wizard_parser = subparsers.add_parser( |
||||
"wizard-upload", |
||||
help="Choose a local bottle, create a backup, and upload it to the server", |
||||
) |
||||
wizard_parser.add_argument( |
||||
"--bottles-dir", |
||||
type=Path, |
||||
default=DEFAULT_BOTTLES_DIR, |
||||
help="Path to the local Bottles directory", |
||||
) |
||||
|
||||
return parser |
||||
|
||||
|
||||
def main() -> int: |
||||
parser = build_parser() |
||||
args = parser.parse_args() |
||||
server = args.server.rstrip("/") |
||||
|
||||
try: |
||||
if args.command == "list": |
||||
return print_archives(server) |
||||
if args.command == "upload": |
||||
return upload_archive( |
||||
server, |
||||
args.file, |
||||
{ |
||||
"name": args.name, |
||||
"bottle_name": args.bottle_name or "", |
||||
"description": args.description or "", |
||||
"tags": args.tags or "", |
||||
"arch": args.arch or "", |
||||
"runner": args.runner or "", |
||||
"windows_version": args.windows_version or "", |
||||
}, |
||||
) |
||||
if args.command == "download": |
||||
return download_archive(server, args.archive_id, args.output) |
||||
if args.command == "install": |
||||
return install_archive(server, args.bottle, args.bottles_dir, args.replace) |
||||
if args.command == "scan-local": |
||||
return scan_local_bottles(args.bottles_dir, args.json) |
||||
if args.command == "wizard-upload": |
||||
return wizard_upload(server, args.bottles_dir) |
||||
parser.error("Unknown command") |
||||
return 2 |
||||
except urllib.error.HTTPError as exc: |
||||
detail = exc.read().decode("utf-8", errors="ignore") |
||||
print(f"HTTP {exc.code}: {detail or exc.reason}", file=sys.stderr) |
||||
return 1 |
||||
except urllib.error.URLError as exc: |
||||
print(f"Connection error: {exc.reason}", file=sys.stderr) |
||||
return 1 |
||||
except FileNotFoundError as exc: |
||||
print(f"File error: {exc}", file=sys.stderr) |
||||
return 1 |
||||
|
||||
|
||||
if __name__ == "__main__": |
||||
raise SystemExit(main()) |
||||
@ -0,0 +1,15 @@
|
||||
services: |
||||
api: |
||||
build: |
||||
context: . |
||||
dockerfile: Dockerfile |
||||
container_name: bottle-archive-api |
||||
ports: |
||||
- "8080:8000" |
||||
environment: |
||||
DATABASE_URL: sqlite:////app/data/bottle_archive.db |
||||
STORAGE_DIR: /app/storage |
||||
volumes: |
||||
- ./data:/app/data |
||||
- ./storage:/app/storage |
||||
restart: unless-stopped |
||||
Loading…
Reference in new issue