DeveloperThumbnail API

Thumbnail API

The sb-thumbnails crate provides a unified thumbnail generation pipeline shared by desktop, mobile (Tauri), and cloud worker pods. All outputs are capped at 512×512 — AI tagging and vision embeddings consume the 512 thumbnail, never the full-resolution source.

Location: crates/sb-thumbnails/ in the studiobrain-ai repository. Previously located in studiobrain-core (relocated in SBAI-2149).

Feature Flags

The crate uses feature-gated backends. The default feature set is image only. Cloud worker pods enable full; desktop and mobile clients pick the subset they ship.

FeatureBackendSystem dependencies
imageimage cratenone (pure Rust)
svgresvg + usvgnone (pure Rust)
pdfpdfium-renderlibpdfium.{so,dll,dylib}
videoffmpeg-nextlibavcodec, libavformat, …
model3dgltf (parser)none for parsing; render stub TBD

Supported Formats

Format kindFormatsBackend feature
Rasterjpg, jpeg, png, webp, gif, bmp, heic, tiffimage
SVGsvgsvg
PDFpdfpdf
Videomp4, mov, mkv, webm, avivideo
3D Modelgltf, glb, obj, fbxmodel3d

Public API

Core Types

pub use sb_thumbnails::{
    Pipeline, ThumbnailRequest, ThumbnailResult,
    CacheLayout, Error, Result,
    MAX_DIM, STANDARD_SIZES, JPEG_QUALITY,
    detect_format, clamp_size, FormatKind,
};

Constants

ConstantValueDescription
MAX_DIM512Maximum thumbnail dimension (hard cap)
STANDARD_SIZES[64, 256, 512]Default thumbnail sizes produced by the pipeline
JPEG_QUALITY85JPEG quality for raster thumbnails (0..=100)

Pipeline

The top-level pipeline orchestrator.

pub struct Pipeline {
    cache: CacheLayout,
}
 
impl Pipeline {
    pub fn new(cache: CacheLayout) -> Self;
    pub fn cache(&self) -> &CacheLayout;
    pub fn supported_formats(&self) -> Vec<&'static str>;
    pub async fn generate(&self, req: ThumbnailRequest) -> Result<ThumbnailResult>;
}

ThumbnailRequest

A request to generate thumbnails for a single source asset.

pub struct ThumbnailRequest {
    pub source_path: PathBuf,           // Path to the source file
    pub tenant_id: String,              // Tenant scope for cache layout
    pub asset_id: String,               // Stable asset ID for output keys
    pub format_hint: Option<String>,    // Optional mime type or extension hint
    pub sizes: Vec<u32>,                // Sizes to generate (default: 64, 256, 512)
    pub generate_gif_preview: bool,     // Generate animated GIF preview (video/gif)
    pub generate_3d_views: bool,        // Generate 4 view thumbnails (3D models)
}

ThumbnailResult

pub struct ThumbnailResult {
    pub asset_id: String,           // The asset this result is for
    pub format: Option<String>,     // Detected format slug
    pub generated: Vec<PathBuf>,    // List of generated thumbnail file paths
    pub errors: Vec<String>,        // Per-size errors (does not short-circuit)
}

CacheLayout

Computes key paths for the per-tenant cache. Keys are forward-slash-delimited (suitable for S3/Garage). local_path() converts to OS-native separators.

pub struct CacheLayout {
    // Private fields
}
 
impl CacheLayout {
    pub fn new(root: impl Into<PathBuf>, tenant_id: impl Into<String>) -> Result<Self>;
    pub fn root(&self) -> &Path;
    pub fn tenant_id(&self) -> &str;
    pub fn entity_key(&self, entity_type: &str, id: &str) -> String;
    pub fn thumbnail_key(&self, asset_id: &str, size: u32) -> String;
    pub fn gif_preview_key(&self, asset_id: &str) -> String;
    pub fn model3d_view_key(&self, asset_id: &str, view: Model3dView) -> String;
    pub fn local_path(&self, key: &str) -> PathBuf;
}

Format Detection

pub fn detect_format(path: &Path, hint: Option<&str>) -> Option<String>;
pub fn clamp_size(size: u32) -> u32;
 
pub enum FormatKind {
    Raster,
    Svg,
    Pdf,
    Video,
    Model3d,
}
 
impl FormatKind {
    pub fn from_slug(slug: &str) -> Option<Self>;
}

Cache Layout

Generated thumbnails are written to a per-tenant local cache directory and mirrored to the sb-tenant-cache Garage S3 bucket so devices can pull each other’s thumbnails:

~/.studiobrain/cache/{tenant_id}/
  entities/{type}/{id}.md
  thumbnails/{asset_id}_64.jpg
  thumbnails/{asset_id}_256.jpg
  thumbnails/{asset_id}_512.jpg
  thumbnails/{asset_id}_gif.gif      // animated preview (video/gif)
  thumbnails/{asset_id}_3d_v1.jpg    // front
  thumbnails/{asset_id}_3d_v2.jpg    // side
  thumbnails/{asset_id}_3d_v3.jpg    // top
  thumbnails/{asset_id}_3d_v4.jpg    // iso

3D Model Views

For 3D models, four standard views are generated when generate_3d_views is enabled:

ViewModel3dViewKey suffixDescription
FrontModel3dView::Front_3d_v1.jpgDefault orientation, used as primary thumbnail
SideModel3dView::Side_3d_v2.jpg90° Y-axis rotation
BackModel3dView::Back_3d_v3.jpg180° Y-axis rotation
Three-quarterModel3dView::Iso_3d_v4.jpg45° Y + 30° X rotation (isometric)

SBAI-1443: 3D thumbnails are now auto-generated on upload. The sb-server routes detect 3D file extensions and spawn a background task that calls the AI service (/api/asset/analyze/queue), which uses trimesh to render four views at 512×512. Each view is then fed through the standard image embedding pipeline (RAM tagging + embedding vectors), making 3D models searchable the same way as 2D images. The front view is set as the asset’s primary thumbnail_path in the database.

SBAI-3371: The AI-side asset_processor.py now handles 3D model rendering directly via the processors/model3d/ module. The Model3DRenderer class produces deterministic front/side/back/three-quarter views using trimesh.save_image when pyrender is available, falling back to a PIL wireframe projector (no GPU needed), and finally a blank placeholder. All four rendered PNGs are handed through AssetProcessor.process_image() — the same path as normal image uploads — so 3D assets receive identical RAM tagging, captioning, and embedding vectors as 2D images. The result dict includes thumbnails (all four views as base64 PNGs), thumbnail (front view), tags (deduplicated across all views), and ai_analysis (per-view analysis keyed by thumbnail_front, thumbnail_side, etc.).

Thumbnails are stored in the _thumbnails subfolder alongside the asset (e.g. Characters/hero/_thumbnails/front.png). Supported formats: .glb, .gltf, .fbx, .obj.

Usage Example

use sb_thumbnails::{CacheLayout, Pipeline, ThumbnailRequest, STANDARD_SIZES};
 
// Set up cache for a tenant
let cache = CacheLayout::new("/home/user/.studiobrain/cache", "tenant-abc")?;
let pipeline = Pipeline::new(cache);
 
// Generate thumbnails for a source asset
let req = ThumbnailRequest::new(
    "/path/to/source/image.png",
    "tenant-abc",
    "asset-123",
);
 
let result = pipeline.generate(req).await?;
println!("Generated {} thumbnails", result.generated.len());

Video with GIF preview

let req = ThumbnailRequest::new(
    "/path/to/video.mp4",
    "tenant-abc",
    "asset-456",
)
.with_generate_gif_preview(true);
 
let result = pipeline.generate(req).await?;
// result.generated includes: asset-456_64.jpg, asset-456_256.jpg,
// asset-456_512.jpg, and asset-456_gif.gif

3D Model with four views

let req = ThumbnailRequest::new(
    "/path/to/model.glb",
    "tenant-abc",
    "asset-789",
)
.with_generate_3d_views(true);
 
let result = pipeline.generate(req).await?;
// result.generated includes: asset-789_64.jpg, asset-789_256.jpg,
// asset-789_512.jpg, and asset-789_3d_v{1..4}.jpg

Error Types

#[derive(Debug, thiserror::Error)]
pub enum Error {
    UnsupportedFormat(String),      // Format not recognized
    Io(io::Error),                  // Filesystem / IO error
    ImageError(String),             // image crate backend error
    SvgError(String),               // SVG backend error
    PdfError(String),               // PDF backend error
    VideoError(String),             // Video backend error
    Model3dError(String),           // 3D model backend error
    CacheError(String),             // Cache layout / write error
    MissingFeature(&'static str),   // Feature flag not enabled in this build
    InvalidInput(String),           // Malformed request
}

Edition Differences

The sb-thumbnails crate lives in studiobrain-ai and is consumed across all editions. Each edition enables only the feature flags it can ship:

  • Desktop (Tauri): image, svg, optionally pdf (ships libpdfium)
  • Mobile (Tauri): image, svg
  • Cloud worker pods: full (all features)
  • Self-hosted: Same as desktop — enable features matching available system deps