Multimodal AI Tagging Engine
FlyRankImage Relevance & Auto-Tagging Engine
An AI backend integrating Gemini multimodal vision, semantic vector search, and an automated Mismatch Guard safety layer with 100% precision.
- 04Persistence Layer
- 03Safety Gate
- 02Service Layer
- 01HTTP Layer
- Top-1 Precision
- 100%
- Automated Tests
- 57
- Vision & Vectors
- Gemini
- FastAPI + Neon
- Async
01 / The Problem & Design Motivation
Content platforms and CMS editors spend excessive manual hours scouring stock image libraries for images matching blog post contexts. Naive vector search alone often yields semantic false-positives (such as matching a post about wild wolves with a domestic dog or coyote). This system solves both challenges by fusing vision-model auto-tagging, cosine vector similarity ranking, a top-2 category inference Mismatch Guard that actively rejects bad matches, and project-wide AI API cost circuit breakers.
Manual image pairing is slow, and unconstrained vector search frequently produces semantic false-positives. This system pairs vision auto-tagging with a strict Mismatch Guard and cost safeguards.
02 / Key Features
Multimodal Vision Classification
Batch-classifies corpus images using Google Gemini 3.5 Flash Lite, automatically generating rich descriptions, extracted entities, and primary category tags stored in PostgreSQL.
Semantic Vector Embedding Search
Converts article text and image captions into vector embeddings via Gemini Embedding 001, performing cosine similarity ranking across the persistent image corpus.
Mismatch Guard Safety Layer
Enforces a dual-verification safety gate using top-2 category inference and confidence threshold checks, actively rejecting category hallucinations and recording full decision audit trails.
AI Spend Circuit Breaker
Tracks every AI API call cost in real time across token usages and shuts down paid execution when the MAX_BUDGET cap is reached, returning graceful 503 fallback responses.
Decision Trail & Human Overrides
Provides full audit transparency through review endpoints, allowing human review, approval, rejection, or manual override while preserving the automated decision history.
03 / System Architecture
HTTP Layer
FastAPI Async Routers & Pydantic DTOs
Decoupled routers for /images, /posts, /suggestions, /jobs, and /costs, enforcing strict Pydantic input/output schemas with background batch task execution.
Service Layer
Vision & Embedding Services
Orchestrates Google Gemini multimodal vision tagging, text-embedding vector generation, and cosine similarity ranking without direct DB coupling.
Safety Gate
GuardService & Decision Audit Trail
Applies category inference and threshold verification before persisting suggestions, logging rejected near-misses and maintaining full audit trails.
Persistence Layer
SQLAlchemy Async & Neon Cloud Postgres
Fully asynchronous relational database models with Alembic schema migrations, storing raw image vectors, post vectors, suggestions, and API cost logs.
04 / Tech Stack
- Python 3.10+
- FastAPI
- SQLAlchemy (Async)
- Alembic
- PostgreSQL (Neon)
- Google Gemini Vision API
- Gemini Embeddings
- Pydantic
- Pytest (57 Tests)
Explore the Full FlyRank Capstone on GitHub
Explore the complete source code, evaluation suite, test fixtures, and architectural build logs on GitHub.