Initial commit — Cellar MVP
This commit is contained in:
@@ -0,0 +1,11 @@
|
||||
__pycache__/
|
||||
*.pyc
|
||||
*.pyo
|
||||
*.pyd
|
||||
.env
|
||||
.venv/
|
||||
venv/
|
||||
.git/
|
||||
.gitignore
|
||||
data/
|
||||
storage/
|
||||
+37
@@ -0,0 +1,37 @@
|
||||
# Runtime data — never commit
|
||||
data/
|
||||
storage/
|
||||
|
||||
# Python bytecode
|
||||
__pycache__/
|
||||
*.pyc
|
||||
*.pyo
|
||||
*.pyd
|
||||
|
||||
# Environment files
|
||||
.env
|
||||
.env.*
|
||||
|
||||
# Virtual environments
|
||||
.venv/
|
||||
venv/
|
||||
env/
|
||||
|
||||
# Package build artifacts
|
||||
*.egg-info/
|
||||
dist/
|
||||
build/
|
||||
|
||||
# Test caches
|
||||
.pytest_cache/
|
||||
.mypy_cache/
|
||||
.ruff_cache/
|
||||
|
||||
# Editor metadata
|
||||
.idea/
|
||||
.vscode/
|
||||
*.swp
|
||||
*~
|
||||
|
||||
# macOS
|
||||
.DS_Store
|
||||
+17
@@ -0,0 +1,17 @@
|
||||
FROM python:3.12-slim
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
ENV PYTHONDONTWRITEBYTECODE=1 \
|
||||
PYTHONUNBUFFERED=1
|
||||
|
||||
COPY requirements.txt .
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
COPY app ./app
|
||||
|
||||
RUN mkdir -p /app/data /app/storage
|
||||
|
||||
EXPOSE 8000
|
||||
|
||||
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
@@ -0,0 +1,406 @@
|
||||
# Cellar
|
||||
|
||||
**Author:** Matteo Benedetto
|
||||
|
||||
`Cellar` is a minimal self-hosted prototype for archiving exported [Bottles](https://usebottles.com/) environments and restoring them on another Linux system.
|
||||
|
||||
The name refers to a wine cellar — the place where bottles are stored and preserved.
|
||||
|
||||
The project currently provides:
|
||||
|
||||
- a `FastAPI` server
|
||||
- a local `SQLite` database for archive metadata
|
||||
- local file storage for uploaded bottle backups
|
||||
- a minimal Python CLI client
|
||||
- a `Docker Compose` setup for running the server quickly
|
||||
|
||||
The current goal is simple: make it possible to back up a local Bottles bottle, upload it to a Cellar server, list what is stored, and reinstall a bottle by name.
|
||||
|
||||
---
|
||||
|
||||
## Current status
|
||||
eventi ora è no c'è una cosa comune cinque
|
||||
This is an early MVP.
|
||||
|
||||
Implemented features:
|
||||
|
||||
- health endpoint
|
||||
- archive upload
|
||||
- archive listing
|
||||
- archive detail lookup
|
||||
- archive download
|
||||
- archive deletion
|
||||
- local Bottles scan from the client
|
||||
- interactive backup + upload wizard from the client
|
||||
- install a bottle from the server with a simple command:
|
||||
- `python client.py install <bottle-name>`
|
||||
|
||||
Not implemented yet:
|
||||
|
||||
- authentication
|
||||
- permissions / multi-user separation
|
||||
- editing metadata already stored in the database
|
||||
- archive versioning
|
||||
- deduplication
|
||||
- tags normalization
|
||||
- server-side search/filtering
|
||||
- direct Bottles GUI integration
|
||||
|
||||
---
|
||||
|
||||
## Project structure
|
||||
|
||||
- [Dockerfile](Dockerfile) — container image for the API
|
||||
- [docker-compose.yml](docker-compose.yml) — local deployment
|
||||
- [requirements.txt](requirements.txt) — Python dependencies
|
||||
- [client.py](client.py) — minimal CLI client
|
||||
- [app/main.py](app/main.py) — API routes
|
||||
- [app/models.py](app/models.py) — SQLAlchemy model
|
||||
- [app/schemas.py](app/schemas.py) — API response schemas
|
||||
- [app/database.py](app/database.py) — database setup
|
||||
- [app/storage.py](app/storage.py) — file storage helpers
|
||||
- [app/config.py](app/config.py) — environment/config handling
|
||||
- [data/](data/) — SQLite database location
|
||||
- [storage/](storage/) — uploaded archive files
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
### Server
|
||||
The server is built with `FastAPI` and stores metadata in `SQLite`.
|
||||
|
||||
Each uploaded archive creates:
|
||||
|
||||
1. a physical file in [storage/](storage/)
|
||||
2. a metadata record in the SQLite database in [data/](data/)
|
||||
|
||||
Stored metadata includes:
|
||||
|
||||
- archive display name
|
||||
- bottle name
|
||||
- description
|
||||
- tags
|
||||
- architecture
|
||||
- runner
|
||||
- Windows version
|
||||
- original file name
|
||||
- stored file name
|
||||
- content type
|
||||
- file size
|
||||
- SHA256 digest
|
||||
- creation time
|
||||
|
||||
### Client
|
||||
The client is intentionally minimal and uses the Python standard library.
|
||||
|
||||
It can:
|
||||
|
||||
- list archives on the server
|
||||
- upload an existing archive file
|
||||
- download an archive by id
|
||||
- scan local Bottles bottles
|
||||
- run a wizard to choose a local bottle, create a backup, and upload it
|
||||
- install a remote bottle by name
|
||||
|
||||
### Storage format
|
||||
The current client creates a `.tar.gz` backup of the selected bottle directory.
|
||||
|
||||
The install flow:
|
||||
|
||||
1. finds the archive by bottle name
|
||||
2. downloads it
|
||||
3. extracts it safely
|
||||
4. places it into the local Bottles directory
|
||||
|
||||
Because Bottles backups contain `dosdevices` symlinks, extraction uses a trusted tar extraction mode while still enforcing a path traversal safety check.
|
||||
|
||||
---
|
||||
|
||||
## Requirements
|
||||
|
||||
Recommended local environment:
|
||||
|
||||
- Docker
|
||||
- Docker Compose
|
||||
- Python 3.12+
|
||||
- Bottles installed locally if you want to use `scan-local`, `wizard-upload`, or `install`
|
||||
|
||||
Expected Bottles path used by the client by default:
|
||||
|
||||
- `~/.var/app/com.usebottles.bottles/data/bottles/bottles`
|
||||
|
||||
This matches the Flatpak Bottles installation layout.
|
||||
|
||||
---
|
||||
|
||||
## Running the server
|
||||
|
||||
From the project root run:
|
||||
|
||||
`docker compose up --build`
|
||||
|
||||
The API is exposed on:
|
||||
|
||||
- `http://127.0.0.1:8080`
|
||||
|
||||
Interactive API docs are available at:
|
||||
|
||||
- `http://127.0.0.1:8080/docs`
|
||||
|
||||
---
|
||||
|
||||
## Environment configuration
|
||||
|
||||
Current environment variables used by the API container:
|
||||
|
||||
- `DATABASE_URL`
|
||||
- `STORAGE_DIR`
|
||||
|
||||
Current defaults in [docker-compose.yml](docker-compose.yml):
|
||||
|
||||
- database: `sqlite:////app/data/bottle_archive.db`
|
||||
- storage: `/app/storage`
|
||||
|
||||
Host directories are mounted as volumes:
|
||||
|
||||
- [data/](data/) → `/app/data`
|
||||
- [storage/](storage/) → `/app/storage`
|
||||
|
||||
---
|
||||
|
||||
## API endpoints
|
||||
|
||||
### `GET /health`
|
||||
Simple health check.
|
||||
|
||||
### `GET /archives`
|
||||
Returns the list of archives ordered by creation time descending.
|
||||
|
||||
### `POST /archives`
|
||||
Uploads a new archive file and stores metadata.
|
||||
|
||||
Form fields supported:
|
||||
|
||||
- `file`
|
||||
- `name`
|
||||
- `bottle_name`
|
||||
- `description`
|
||||
- `tags`
|
||||
- `arch`
|
||||
- `runner`
|
||||
- `windows_version`
|
||||
|
||||
### `GET /archives/{archive_id}`
|
||||
Returns one archive metadata entry.
|
||||
|
||||
### `GET /archives/{archive_id}/download`
|
||||
Downloads the archive file.
|
||||
|
||||
### `DELETE /archives/{archive_id}`
|
||||
Deletes the archive file and the corresponding metadata row.
|
||||
|
||||
---
|
||||
|
||||
## Client commands
|
||||
|
||||
All commands below assume you are inside the project root.
|
||||
|
||||
### List remote archives
|
||||
`python client.py list`
|
||||
|
||||
### Upload an existing archive file
|
||||
`python client.py upload /path/to/archive.tar.gz --name "Empire Earth Gold" --bottle-name "Empire-Earth-Gold"`
|
||||
|
||||
Optional extra metadata:
|
||||
|
||||
`python client.py upload /path/to/archive.tar.gz --name "Empire Earth Gold" --bottle-name "Empire-Earth-Gold" --description "Backup from desktop" --tags "games,gog" --arch win32 --runner soda-9.0-1 --windows-version win7`
|
||||
|
||||
### Download an archive by id
|
||||
`python client.py download 1 ./downloads/`
|
||||
|
||||
### Scan local Bottles bottles
|
||||
`python client.py scan-local`
|
||||
|
||||
JSON output:
|
||||
|
||||
`python client.py scan-local --json`
|
||||
|
||||
### Interactive wizard: choose local bottle, back it up, upload it
|
||||
`python client.py wizard-upload`
|
||||
|
||||
### Install a bottle from the server by name
|
||||
`python client.py install Empire-Earth-Gold`
|
||||
|
||||
Replace existing local bottle:
|
||||
|
||||
`python client.py install Empire-Earth-Gold --replace`
|
||||
|
||||
Use a custom Bottles directory:
|
||||
|
||||
`python client.py install Empire-Earth-Gold --bottles-dir /custom/path`
|
||||
|
||||
---
|
||||
|
||||
## Example workflow
|
||||
|
||||
### Backup and upload
|
||||
1. Start the server:
|
||||
- `docker compose up --build`
|
||||
2. Scan local Bottles:
|
||||
- `python client.py scan-local`
|
||||
3. Start the wizard:
|
||||
- `python client.py wizard-upload`
|
||||
4. Choose one of the local bottles
|
||||
5. Confirm archive metadata
|
||||
6. The client creates a temporary `.tar.gz` backup and uploads it
|
||||
|
||||
### Restore on another machine
|
||||
1. Start the same server or connect to a reachable one
|
||||
2. Install the bottle by name:
|
||||
- `python client.py install Empire-Earth-Gold`
|
||||
3. Check local Bottles:
|
||||
- `python client.py scan-local`
|
||||
|
||||
---
|
||||
|
||||
## Known limitations
|
||||
|
||||
### 1. No authentication
|
||||
At the moment, **anyone who can reach the API can list, download, upload, and delete archives**.
|
||||
|
||||
This is acceptable only for a local prototype or a trusted LAN environment.
|
||||
|
||||
### 2. No metadata editing
|
||||
Once an archive is inserted in the database, there is currently **no API or client command to update its metadata**.
|
||||
|
||||
That means fields like:
|
||||
|
||||
- `name`
|
||||
- `description`
|
||||
- `tags`
|
||||
- `runner`
|
||||
- `windows_version`
|
||||
|
||||
cannot yet be corrected or refined without direct database manipulation or deleting and re-uploading the entry.
|
||||
|
||||
### 3. No versions/history model
|
||||
Multiple uploads of the same bottle are just separate rows. There is no version chain or “latest version” logic yet.
|
||||
|
||||
### 4. Local storage only
|
||||
The server stores everything on local disk. There is no S3/MinIO/remote object storage integration yet.
|
||||
|
||||
### 5. Flatpak Bottles path assumption
|
||||
The default client path targets a Flatpak Bottles installation. Native Bottles layouts may need `--bottles-dir`.
|
||||
|
||||
---
|
||||
|
||||
## Next phases
|
||||
|
||||
The next development phases should focus on two critical areas.
|
||||
|
||||
### Phase 1: Authentication system integration
|
||||
This is the most important missing feature.
|
||||
|
||||
The project needs an authentication layer so the server can be used safely beyond a private local test.
|
||||
|
||||
Recommended additions:
|
||||
|
||||
- user accounts
|
||||
- password hashing
|
||||
- token-based authentication
|
||||
- per-user archive ownership
|
||||
- permission checks on download/delete/edit operations
|
||||
- optional admin role
|
||||
|
||||
Possible implementation direction:
|
||||
|
||||
- FastAPI auth routes
|
||||
- JWT access tokens or session tokens
|
||||
- password hashing with a standard library such as `passlib`/`bcrypt`
|
||||
- `users` table + relation from archive rows to owners
|
||||
|
||||
Minimum target for this phase:
|
||||
|
||||
- `POST /auth/register`
|
||||
- `POST /auth/login`
|
||||
- protected archive routes
|
||||
- client support for login and storing a token locally
|
||||
|
||||
### Phase 2: Database entry editing system
|
||||
The second required feature is a proper metadata editing workflow.
|
||||
|
||||
Right now archive metadata is write-once. That is too limiting.
|
||||
|
||||
The project needs a system to edit database entries safely from the API and the client.
|
||||
|
||||
Recommended additions:
|
||||
|
||||
- `PATCH /archives/{id}` endpoint
|
||||
- validation rules for editable fields
|
||||
- audit fields like `updated_at`
|
||||
- optional edit history
|
||||
- client command such as:
|
||||
- `python client.py edit 12 --name "..." --tags "..."`
|
||||
|
||||
Editable fields should include at least:
|
||||
|
||||
- `name`
|
||||
- `bottle_name`
|
||||
- `description`
|
||||
- `tags`
|
||||
- `arch`
|
||||
- `runner`
|
||||
- `windows_version`
|
||||
|
||||
This phase is important because real archives often need cleanup after upload.
|
||||
|
||||
---
|
||||
|
||||
## Recommended roadmap
|
||||
|
||||
### Short term
|
||||
- add authentication
|
||||
- add metadata editing
|
||||
- add archive search/filter endpoints
|
||||
- add better error messages in client install/upload flows
|
||||
|
||||
### Medium term
|
||||
- add versioning for the same bottle
|
||||
- add per-user archive namespaces
|
||||
- add pagination on `GET /archives`
|
||||
- add import/export manifest support
|
||||
- add `upload-local <bottle-name>` non-interactive command
|
||||
|
||||
### Long term
|
||||
- replace local storage with S3/MinIO
|
||||
- add a small web UI
|
||||
- add full-text search on archive metadata
|
||||
- add deduplication / retention policies
|
||||
- integrate directly with Bottles workflows more tightly
|
||||
|
||||
---
|
||||
|
||||
## Security note
|
||||
|
||||
Until authentication is implemented, do **not** expose this API publicly.
|
||||
|
||||
Use it only:
|
||||
|
||||
- on localhost
|
||||
- on a trusted home LAN
|
||||
- behind an authenticated reverse proxy if temporarily needed
|
||||
|
||||
---
|
||||
|
||||
## Development notes
|
||||
|
||||
The project is currently intentionally simple.
|
||||
|
||||
That simplicity is useful because it makes the next steps clear:
|
||||
|
||||
1. secure access
|
||||
2. allow metadata editing
|
||||
3. improve archive lifecycle management
|
||||
|
||||
Those two immediate next phases — **authentication** and **database entry editing** — should be considered required before treating this project as more than a local prototype.
|
||||
@@ -0,0 +1,10 @@
|
||||
from pathlib import Path
|
||||
import os
|
||||
|
||||
BASE_DIR = Path(__file__).resolve().parent.parent
|
||||
DATA_DIR = Path(os.getenv("DATA_DIR", BASE_DIR / "data"))
|
||||
STORAGE_DIR = Path(os.getenv("STORAGE_DIR", BASE_DIR / "storage"))
|
||||
DATABASE_URL = os.getenv(
|
||||
"DATABASE_URL",
|
||||
f"sqlite:///{(DATA_DIR / 'bottle_archive.db').as_posix()}",
|
||||
)
|
||||
@@ -0,0 +1,18 @@
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import declarative_base, sessionmaker
|
||||
|
||||
from app.config import DATABASE_URL
|
||||
|
||||
connect_args = {"check_same_thread": False} if DATABASE_URL.startswith("sqlite") else {}
|
||||
|
||||
engine = create_engine(DATABASE_URL, connect_args=connect_args)
|
||||
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
|
||||
Base = declarative_base()
|
||||
|
||||
|
||||
def get_db():
|
||||
db = SessionLocal()
|
||||
try:
|
||||
yield db
|
||||
finally:
|
||||
db.close()
|
||||
+114
@@ -0,0 +1,114 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from fastapi import Depends, FastAPI, File, Form, HTTPException, UploadFile, status
|
||||
from fastapi.responses import FileResponse
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.config import DATA_DIR, STORAGE_DIR
|
||||
from app.database import Base, engine, get_db
|
||||
from app.models import Archive
|
||||
from app.schemas import ArchiveRead
|
||||
from app.storage import delete_file, ensure_storage_dirs, resolve_path, save_upload
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(_: FastAPI):
|
||||
DATA_DIR.mkdir(parents=True, exist_ok=True)
|
||||
STORAGE_DIR.mkdir(parents=True, exist_ok=True)
|
||||
Base.metadata.create_all(bind=engine)
|
||||
ensure_storage_dirs()
|
||||
yield
|
||||
|
||||
|
||||
app = FastAPI(
|
||||
title="Bottle Archive Server",
|
||||
version="0.1.0",
|
||||
description="Local archive server for exported Bottles backups.",
|
||||
lifespan=lifespan,
|
||||
)
|
||||
|
||||
|
||||
@app.get("/health")
|
||||
def health() -> dict[str, str]:
|
||||
return {"status": "ok"}
|
||||
|
||||
|
||||
@app.get("/archives", response_model=list[ArchiveRead])
|
||||
def list_archives(db: Session = Depends(get_db)) -> list[Archive]:
|
||||
return list(db.scalars(select(Archive).order_by(Archive.created_at.desc())))
|
||||
|
||||
|
||||
@app.post("/archives", response_model=ArchiveRead, status_code=status.HTTP_201_CREATED)
|
||||
async def create_archive(
|
||||
file: UploadFile = File(...),
|
||||
name: str = Form(...),
|
||||
bottle_name: str | None = Form(default=None),
|
||||
description: str | None = Form(default=None),
|
||||
tags: str | None = Form(default=None),
|
||||
arch: str | None = Form(default=None),
|
||||
runner: str | None = Form(default=None),
|
||||
windows_version: str | None = Form(default=None),
|
||||
db: Session = Depends(get_db),
|
||||
) -> Archive:
|
||||
if not file.filename:
|
||||
raise HTTPException(status_code=400, detail="Uploaded file must have a filename.")
|
||||
|
||||
stored_name, size_bytes, digest = await save_upload(file)
|
||||
|
||||
archive = Archive(
|
||||
name=name,
|
||||
bottle_name=bottle_name,
|
||||
description=description,
|
||||
tags=tags,
|
||||
arch=arch,
|
||||
runner=runner,
|
||||
windows_version=windows_version,
|
||||
file_name=file.filename,
|
||||
stored_name=stored_name,
|
||||
content_type=file.content_type,
|
||||
size_bytes=size_bytes,
|
||||
sha256=digest,
|
||||
)
|
||||
db.add(archive)
|
||||
db.commit()
|
||||
db.refresh(archive)
|
||||
return archive
|
||||
|
||||
|
||||
@app.get("/archives/{archive_id}", response_model=ArchiveRead)
|
||||
def get_archive(archive_id: int, db: Session = Depends(get_db)) -> Archive:
|
||||
archive = db.get(Archive, archive_id)
|
||||
if archive is None:
|
||||
raise HTTPException(status_code=404, detail="Archive not found.")
|
||||
return archive
|
||||
|
||||
|
||||
@app.get("/archives/{archive_id}/download")
|
||||
def download_archive(archive_id: int, db: Session = Depends(get_db)) -> FileResponse:
|
||||
archive = db.get(Archive, archive_id)
|
||||
if archive is None:
|
||||
raise HTTPException(status_code=404, detail="Archive not found.")
|
||||
|
||||
path = resolve_path(archive.stored_name)
|
||||
if not path.exists():
|
||||
raise HTTPException(status_code=404, detail="Stored file not found.")
|
||||
|
||||
return FileResponse(
|
||||
path=path,
|
||||
media_type=archive.content_type or "application/octet-stream",
|
||||
filename=archive.file_name,
|
||||
)
|
||||
|
||||
|
||||
@app.delete("/archives/{archive_id}", status_code=status.HTTP_204_NO_CONTENT)
|
||||
def delete_archive(archive_id: int, db: Session = Depends(get_db)) -> None:
|
||||
archive = db.get(Archive, archive_id)
|
||||
if archive is None:
|
||||
raise HTTPException(status_code=404, detail="Archive not found.")
|
||||
|
||||
delete_file(archive.stored_name)
|
||||
db.delete(archive)
|
||||
db.commit()
|
||||
@@ -0,0 +1,25 @@
|
||||
from sqlalchemy import DateTime, Integer, String, Text, func
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.database import Base
|
||||
|
||||
|
||||
class Archive(Base):
|
||||
__tablename__ = "archives"
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, index=True)
|
||||
name: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||
bottle_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
description: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
tags: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
arch: Mapped[str | None] = mapped_column(String(32), nullable=True)
|
||||
runner: Mapped[str | None] = mapped_column(String(128), nullable=True)
|
||||
windows_version: Mapped[str | None] = mapped_column(String(32), nullable=True)
|
||||
file_name: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||
stored_name: Mapped[str] = mapped_column(String(255), unique=True, nullable=False)
|
||||
content_type: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
size_bytes: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
sha256: Mapped[str] = mapped_column(String(64), nullable=False, index=True)
|
||||
created_at: Mapped[str] = mapped_column(
|
||||
DateTime(timezone=True), server_default=func.now(), nullable=False
|
||||
)
|
||||
@@ -0,0 +1,28 @@
|
||||
from datetime import datetime
|
||||
|
||||
from pydantic import BaseModel, ConfigDict
|
||||
|
||||
|
||||
class ArchiveBase(BaseModel):
|
||||
name: str
|
||||
bottle_name: str | None = None
|
||||
description: str | None = None
|
||||
tags: str | None = None
|
||||
arch: str | None = None
|
||||
runner: str | None = None
|
||||
windows_version: str | None = None
|
||||
|
||||
|
||||
class ArchiveCreate(ArchiveBase):
|
||||
pass
|
||||
|
||||
|
||||
class ArchiveRead(ArchiveBase):
|
||||
model_config = ConfigDict(from_attributes=True)
|
||||
|
||||
id: int
|
||||
file_name: str
|
||||
content_type: str | None = None
|
||||
size_bytes: int
|
||||
sha256: str
|
||||
created_at: datetime
|
||||
@@ -0,0 +1,46 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from hashlib import sha256
|
||||
from pathlib import Path
|
||||
from uuid import uuid4
|
||||
|
||||
from fastapi import UploadFile
|
||||
|
||||
from app.config import STORAGE_DIR
|
||||
|
||||
CHUNK_SIZE = 1024 * 1024
|
||||
|
||||
|
||||
def ensure_storage_dirs() -> None:
|
||||
STORAGE_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
async def save_upload(file: UploadFile) -> tuple[str, int, str]:
|
||||
ensure_storage_dirs()
|
||||
|
||||
suffix = Path(file.filename or "archive.tar.gz").suffixes
|
||||
extension = "".join(suffix) if suffix else ".bin"
|
||||
stored_name = f"{uuid4().hex}{extension}"
|
||||
destination = STORAGE_DIR / stored_name
|
||||
|
||||
digest = sha256()
|
||||
size = 0
|
||||
|
||||
with destination.open("wb") as buffer:
|
||||
while chunk := await file.read(CHUNK_SIZE):
|
||||
size += len(chunk)
|
||||
digest.update(chunk)
|
||||
buffer.write(chunk)
|
||||
|
||||
await file.close()
|
||||
return stored_name, size, digest.hexdigest()
|
||||
|
||||
|
||||
def delete_file(stored_name: str) -> None:
|
||||
path = STORAGE_DIR / stored_name
|
||||
if path.exists():
|
||||
path.unlink()
|
||||
|
||||
|
||||
def resolve_path(stored_name: str) -> Path:
|
||||
return STORAGE_DIR / stored_name
|
||||
@@ -0,0 +1,412 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import shutil
|
||||
import sys
|
||||
import tarfile
|
||||
import tempfile
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
DEFAULT_SERVER = "http://127.0.0.1:8080"
|
||||
DEFAULT_BOTTLES_DIR = Path.home() / ".var/app/com.usebottles.bottles/data/bottles/bottles"
|
||||
|
||||
|
||||
def request_json(url: str, method: str = "GET", data: bytes | None = None, headers: dict[str, str] | None = None):
|
||||
req = urllib.request.Request(url, data=data, method=method)
|
||||
for key, value in (headers or {}).items():
|
||||
req.add_header(key, value)
|
||||
with urllib.request.urlopen(req) as response:
|
||||
return json.loads(response.read().decode("utf-8"))
|
||||
|
||||
|
||||
def print_archives(server: str) -> int:
|
||||
archives = request_json(f"{server}/archives")
|
||||
if not archives:
|
||||
print("No archives found.")
|
||||
return 0
|
||||
|
||||
print(f"{'ID':<4} {'Name':<30} {'Bottle':<30} {'Arch':<8} {'Runner':<18} {'Size(MB)':>10}")
|
||||
print("-" * 110)
|
||||
for item in archives:
|
||||
size_mb = item["size_bytes"] / (1024 * 1024)
|
||||
print(
|
||||
f"{item['id']:<4} "
|
||||
f"{(item['name'] or '-')[:30]:<30} "
|
||||
f"{(item.get('bottle_name') or '-')[:30]:<30} "
|
||||
f"{(item.get('arch') or '-'):<8} "
|
||||
f"{(item.get('runner') or '-')[:18]:<18} "
|
||||
f"{size_mb:>10.2f}"
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
def upload_archive(server: str, file_path: Path, fields: dict[str, str]) -> int:
|
||||
boundary = "----BottleArchiveBoundary7MA4YWxkTrZu0gW"
|
||||
data = []
|
||||
|
||||
def add_field(field_name: str, value: str):
|
||||
data.extend(
|
||||
[
|
||||
f"--{boundary}\r\n".encode(),
|
||||
f'Content-Disposition: form-data; name="{field_name}"\r\n\r\n'.encode(),
|
||||
value.encode(),
|
||||
b"\r\n",
|
||||
]
|
||||
)
|
||||
|
||||
for key, value in fields.items():
|
||||
if value:
|
||||
add_field(key, value)
|
||||
|
||||
filename = file_path.name
|
||||
mime = "application/gzip" if filename.endswith((".tar.gz", ".tgz", ".gz")) else "application/octet-stream"
|
||||
data.extend(
|
||||
[
|
||||
f"--{boundary}\r\n".encode(),
|
||||
f'Content-Disposition: form-data; name="file"; filename="{filename}"\r\n'.encode(),
|
||||
f"Content-Type: {mime}\r\n\r\n".encode(),
|
||||
file_path.read_bytes(),
|
||||
b"\r\n",
|
||||
f"--{boundary}--\r\n".encode(),
|
||||
]
|
||||
)
|
||||
|
||||
payload = b"".join(data)
|
||||
archive = request_json(
|
||||
f"{server}/archives",
|
||||
method="POST",
|
||||
data=payload,
|
||||
headers={"Content-Type": f"multipart/form-data; boundary={boundary}"},
|
||||
)
|
||||
print(f"Uploaded archive #{archive['id']}: {archive['name']}")
|
||||
return 0
|
||||
|
||||
|
||||
def download_archive(server: str, archive_id: int, output: Path) -> int:
|
||||
url = f"{server}/archives/{archive_id}/download"
|
||||
req = urllib.request.Request(url, method="GET")
|
||||
with urllib.request.urlopen(req) as response:
|
||||
if output.is_dir():
|
||||
filename = response.headers.get_filename() or f"archive-{archive_id}.bin"
|
||||
destination = output / filename
|
||||
else:
|
||||
destination = output
|
||||
destination.write_bytes(response.read())
|
||||
print(f"Downloaded to {destination}")
|
||||
return 0
|
||||
|
||||
|
||||
def find_remote_archive(server: str, bottle_ref: str) -> dict[str, Any]:
|
||||
archives = request_json(f"{server}/archives")
|
||||
bottle_ref_lower = bottle_ref.lower()
|
||||
|
||||
for item in archives:
|
||||
if (item.get("bottle_name") or "").lower() == bottle_ref_lower:
|
||||
return item
|
||||
for item in archives:
|
||||
if (item.get("name") or "").lower() == bottle_ref_lower:
|
||||
return item
|
||||
|
||||
raise FileNotFoundError(f"No remote archive found for '{bottle_ref}'")
|
||||
|
||||
|
||||
def safe_extract_tar(archive_path: Path, target_dir: Path) -> None:
|
||||
with tarfile.open(archive_path, "r:gz") as tar:
|
||||
for member in tar.getmembers():
|
||||
member_path = (target_dir / member.name).resolve()
|
||||
if not str(member_path).startswith(str(target_dir.resolve())):
|
||||
raise ValueError("Unsafe archive path detected.")
|
||||
tar.extractall(target_dir, filter="fully_trusted")
|
||||
|
||||
|
||||
def install_archive(server: str, bottle_ref: str, bottles_dir: Path, replace: bool) -> int:
|
||||
archive = find_remote_archive(server, bottle_ref)
|
||||
bottle_name = archive.get("bottle_name") or archive.get("name") or bottle_ref
|
||||
target_dir = bottles_dir / bottle_name
|
||||
|
||||
if target_dir.exists():
|
||||
if not replace:
|
||||
print(
|
||||
f"Bottle already exists: {target_dir}\n"
|
||||
"Use --replace to overwrite it.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 1
|
||||
shutil.rmtree(target_dir)
|
||||
|
||||
bottles_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
with tempfile.TemporaryDirectory(prefix="bottle-install-") as tmp_dir:
|
||||
tmp_path = Path(tmp_dir)
|
||||
download_path = tmp_path / f"archive-{archive['id']}.tar.gz"
|
||||
download_archive(server, int(archive["id"]), download_path)
|
||||
|
||||
extract_dir = tmp_path / "extract"
|
||||
extract_dir.mkdir(parents=True, exist_ok=True)
|
||||
safe_extract_tar(download_path, extract_dir)
|
||||
|
||||
candidates = [path for path in extract_dir.iterdir() if path.is_dir()]
|
||||
if not candidates:
|
||||
raise FileNotFoundError("Archive did not contain a bottle directory.")
|
||||
|
||||
source_dir = candidates[0]
|
||||
bottle_yml = source_dir / "bottle.yml"
|
||||
if not bottle_yml.exists():
|
||||
raise FileNotFoundError("Archive does not look like a valid bottle backup.")
|
||||
|
||||
shutil.move(str(source_dir), str(target_dir))
|
||||
|
||||
print(f"Installed bottle '{bottle_name}' to {target_dir}")
|
||||
return 0
|
||||
|
||||
|
||||
def parse_bottle_yml(bottle_yml: Path) -> dict[str, str]:
|
||||
data: dict[str, str] = {}
|
||||
wanted_keys = {"Name", "Arch", "Runner", "Environment", "Windows"}
|
||||
|
||||
for line in bottle_yml.read_text(encoding="utf-8", errors="ignore").splitlines():
|
||||
if ":" not in line or line.startswith(" "):
|
||||
continue
|
||||
key, value = line.split(":", 1)
|
||||
if key in wanted_keys:
|
||||
data[key] = value.strip().strip("'")
|
||||
|
||||
return data
|
||||
|
||||
|
||||
def collect_local_bottles(bottles_dir: Path) -> list[dict[str, Any]]:
|
||||
if not bottles_dir.exists():
|
||||
return []
|
||||
|
||||
bottles: list[dict[str, Any]] = []
|
||||
for directory in sorted(path for path in bottles_dir.iterdir() if path.is_dir()):
|
||||
bottle_yml = directory / "bottle.yml"
|
||||
if not bottle_yml.exists():
|
||||
continue
|
||||
|
||||
metadata = parse_bottle_yml(bottle_yml)
|
||||
bottles.append(
|
||||
{
|
||||
"name": metadata.get("Name", directory.name),
|
||||
"directory": directory.name,
|
||||
"path": str(directory),
|
||||
"arch": metadata.get("Arch", "-"),
|
||||
"runner": metadata.get("Runner", "-"),
|
||||
"environment": metadata.get("Environment", "-"),
|
||||
"windows": metadata.get("Windows", "-"),
|
||||
}
|
||||
)
|
||||
|
||||
return bottles
|
||||
|
||||
|
||||
def scan_local_bottles(bottles_dir: Path, as_json: bool) -> int:
|
||||
bottles = collect_local_bottles(bottles_dir)
|
||||
if as_json:
|
||||
print(json.dumps(bottles, indent=2))
|
||||
return 0
|
||||
|
||||
if not bottles:
|
||||
print(f"No local bottles found in {bottles_dir}")
|
||||
return 0
|
||||
|
||||
print(f"Local Bottles directory: {bottles_dir}")
|
||||
print(f"{'Dir':<24} {'Name':<28} {'Arch':<8} {'Runner':<18} {'Env':<12} {'Windows':<10}")
|
||||
print("-" * 110)
|
||||
for bottle in bottles:
|
||||
print(
|
||||
f"{bottle['directory'][:24]:<24} "
|
||||
f"{bottle['name'][:28]:<28} "
|
||||
f"{bottle['arch']:<8} "
|
||||
f"{bottle['runner'][:18]:<18} "
|
||||
f"{bottle['environment'][:12]:<12} "
|
||||
f"{bottle['windows'][:10]:<10}"
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
def create_bottle_backup(bottle: dict[str, Any], output_dir: Path | None = None) -> Path:
|
||||
source_dir = Path(bottle["path"])
|
||||
target_dir = output_dir or Path(tempfile.gettempdir())
|
||||
timestamp = datetime.now().strftime("%Y%m%d-%H%M%S")
|
||||
archive_name = f"{bottle['directory']}-{timestamp}.tar.gz"
|
||||
archive_path = target_dir / archive_name
|
||||
|
||||
with tarfile.open(archive_path, "w:gz") as tar:
|
||||
tar.add(source_dir, arcname=source_dir.name)
|
||||
|
||||
return archive_path
|
||||
|
||||
|
||||
def prompt_text(label: str, default: str) -> str:
|
||||
value = input(f"{label} [{default}]: ").strip()
|
||||
return value or default
|
||||
|
||||
|
||||
def choose_bottle(bottles: list[dict[str, Any]]) -> dict[str, Any]:
|
||||
print("Available local bottles:")
|
||||
for index, bottle in enumerate(bottles, start=1):
|
||||
print(
|
||||
f" {index}. {bottle['directory']} "
|
||||
f"(arch={bottle['arch']}, runner={bottle['runner']}, windows={bottle['windows']})"
|
||||
)
|
||||
|
||||
while True:
|
||||
raw = input("Select a bottle number: ").strip()
|
||||
try:
|
||||
choice = int(raw)
|
||||
except ValueError:
|
||||
print("Please enter a valid number.")
|
||||
continue
|
||||
|
||||
if 1 <= choice <= len(bottles):
|
||||
return bottles[choice - 1]
|
||||
|
||||
print("Choice out of range.")
|
||||
|
||||
|
||||
def wizard_upload(server: str, bottles_dir: Path) -> int:
|
||||
bottles = collect_local_bottles(bottles_dir)
|
||||
if not bottles:
|
||||
print(f"No local bottles found in {bottles_dir}")
|
||||
return 0
|
||||
|
||||
bottle = choose_bottle(bottles)
|
||||
display_name = prompt_text("Archive name", bottle["name"])
|
||||
bottle_name = prompt_text("Bottle name", bottle["directory"])
|
||||
description = prompt_text("Description", f"Backup of {bottle['name']}")
|
||||
tags = prompt_text("Tags", "bottles,backup")
|
||||
|
||||
print(f"\nCreating backup for {bottle['directory']}...")
|
||||
archive_path = create_bottle_backup(bottle)
|
||||
print(f"Backup created: {archive_path}")
|
||||
|
||||
try:
|
||||
return upload_archive(
|
||||
server,
|
||||
archive_path,
|
||||
{
|
||||
"name": display_name,
|
||||
"bottle_name": bottle_name,
|
||||
"description": description,
|
||||
"tags": tags,
|
||||
"arch": str(bottle.get("arch", "")),
|
||||
"runner": str(bottle.get("runner", "")),
|
||||
"windows_version": str(bottle.get("windows", "")),
|
||||
},
|
||||
)
|
||||
finally:
|
||||
archive_path.unlink(missing_ok=True)
|
||||
|
||||
|
||||
def build_parser() -> argparse.ArgumentParser:
|
||||
parser = argparse.ArgumentParser(description="Minimal client for Bottle Archive Server")
|
||||
parser.add_argument("--server", default=DEFAULT_SERVER, help="Base server URL")
|
||||
|
||||
subparsers = parser.add_subparsers(dest="command", required=True)
|
||||
|
||||
subparsers.add_parser("list", help="List uploaded bottle archives")
|
||||
|
||||
upload_parser = subparsers.add_parser("upload", help="Upload a bottle archive")
|
||||
upload_parser.add_argument("file", type=Path, help="Path to the archive file")
|
||||
upload_parser.add_argument("--name", required=True, help="Display name")
|
||||
upload_parser.add_argument("--bottle-name", help="Bottle name")
|
||||
upload_parser.add_argument("--description", help="Description")
|
||||
upload_parser.add_argument("--tags", help="Tags")
|
||||
upload_parser.add_argument("--arch", help="Architecture")
|
||||
upload_parser.add_argument("--runner", help="Runner")
|
||||
upload_parser.add_argument("--windows-version", help="Windows version")
|
||||
|
||||
download_parser = subparsers.add_parser("download", help="Download an archive by id")
|
||||
download_parser.add_argument("archive_id", type=int, help="Archive id")
|
||||
download_parser.add_argument("output", type=Path, help="Output file or directory")
|
||||
|
||||
install_parser = subparsers.add_parser(
|
||||
"install",
|
||||
help="Install a bottle from the server by bottle name or archive name",
|
||||
)
|
||||
install_parser.add_argument("bottle", help="Bottle name or archive name to install")
|
||||
install_parser.add_argument(
|
||||
"--bottles-dir",
|
||||
type=Path,
|
||||
default=DEFAULT_BOTTLES_DIR,
|
||||
help="Path to the local Bottles directory",
|
||||
)
|
||||
install_parser.add_argument("--replace", action="store_true", help="Replace existing local bottle")
|
||||
|
||||
scan_parser = subparsers.add_parser("scan-local", help="List bottles installed on this computer")
|
||||
scan_parser.add_argument(
|
||||
"--bottles-dir",
|
||||
type=Path,
|
||||
default=DEFAULT_BOTTLES_DIR,
|
||||
help="Path to the local Bottles directory",
|
||||
)
|
||||
scan_parser.add_argument("--json", action="store_true", help="Print results as JSON")
|
||||
|
||||
wizard_parser = subparsers.add_parser(
|
||||
"wizard-upload",
|
||||
help="Choose a local bottle, create a backup, and upload it to the server",
|
||||
)
|
||||
wizard_parser.add_argument(
|
||||
"--bottles-dir",
|
||||
type=Path,
|
||||
default=DEFAULT_BOTTLES_DIR,
|
||||
help="Path to the local Bottles directory",
|
||||
)
|
||||
|
||||
return parser
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = build_parser()
|
||||
args = parser.parse_args()
|
||||
server = args.server.rstrip("/")
|
||||
|
||||
try:
|
||||
if args.command == "list":
|
||||
return print_archives(server)
|
||||
if args.command == "upload":
|
||||
return upload_archive(
|
||||
server,
|
||||
args.file,
|
||||
{
|
||||
"name": args.name,
|
||||
"bottle_name": args.bottle_name or "",
|
||||
"description": args.description or "",
|
||||
"tags": args.tags or "",
|
||||
"arch": args.arch or "",
|
||||
"runner": args.runner or "",
|
||||
"windows_version": args.windows_version or "",
|
||||
},
|
||||
)
|
||||
if args.command == "download":
|
||||
return download_archive(server, args.archive_id, args.output)
|
||||
if args.command == "install":
|
||||
return install_archive(server, args.bottle, args.bottles_dir, args.replace)
|
||||
if args.command == "scan-local":
|
||||
return scan_local_bottles(args.bottles_dir, args.json)
|
||||
if args.command == "wizard-upload":
|
||||
return wizard_upload(server, args.bottles_dir)
|
||||
parser.error("Unknown command")
|
||||
return 2
|
||||
except urllib.error.HTTPError as exc:
|
||||
detail = exc.read().decode("utf-8", errors="ignore")
|
||||
print(f"HTTP {exc.code}: {detail or exc.reason}", file=sys.stderr)
|
||||
return 1
|
||||
except urllib.error.URLError as exc:
|
||||
print(f"Connection error: {exc.reason}", file=sys.stderr)
|
||||
return 1
|
||||
except FileNotFoundError as exc:
|
||||
print(f"File error: {exc}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,15 @@
|
||||
services:
|
||||
api:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
container_name: bottle-archive-api
|
||||
ports:
|
||||
- "8080:8000"
|
||||
environment:
|
||||
DATABASE_URL: sqlite:////app/data/bottle_archive.db
|
||||
STORAGE_DIR: /app/storage
|
||||
volumes:
|
||||
- ./data:/app/data
|
||||
- ./storage:/app/storage
|
||||
restart: unless-stopped
|
||||
@@ -0,0 +1,4 @@
|
||||
fastapi>=0.115.0
|
||||
uvicorn[standard]>=0.30.0
|
||||
sqlalchemy>=2.0.0
|
||||
python-multipart>=0.0.9
|
||||
Reference in New Issue
Block a user