Applied AI · Backend · Personalization
DeepFeed AI — personalized knowledge discovery with an adaptive AI loop
DeepFeed AI is a personalized knowledge-discovery platform designed to filter information overload into a user-specific feed. The repository combines a FastAPI backend, recommendation and feedback systems, event-driven processing, a Next.js frontend, and an agentic adaptation layer that changes future research plans as the system learns from user behavior.
Architecture
Modular monolith + event-driven processing + agentic adaptation
Core product loop
Discover → rank → interact → adapt
Repository state
Active implementation with backend, frontend, tests, CI, and deployment assets
01 · Product problem
A feed should optimize for the person, not just the source
The system is built around a simple problem: useful information is spread across many sources, but collecting more content does not automatically create more value. The platform needs to decide what is relevant to a specific user, explain why it surfaced an item, and change its behavior as that user's interests evolve.
DeepFeed therefore combines ordinary software-engineering concerns—APIs, persistence, authentication, queues, failure handling, and observability—with recommendation logic and an adaptive agent layer.
02 · System architecture
Modular monolith at the center, asynchronous work around it
The repository documents the platform as a modular monolith with event-driven processing and a separate agentic adaptation concern. The important boundary is not “how many services exist”; it is which layer owns each responsibility and which direction dependencies are allowed to flow.
ClientNext.js frontend
The repository includes a Next.js frontend for the user-facing feed and research experience. The platform boundary is kept separate from backend application logic.
APIFastAPI application
FastAPI exposes health, authentication, user/profile, feed, feedback/admin, agent, statistics, and metrics routes. Middleware adds audit logging, rate limiting, trace IDs, CORS, and a global error envelope.
ApplicationUse-case services
Application services coordinate authentication, content, discovery, feed generation, feedback, profiles, ranking, and agent-driven adaptation without collapsing those responsibilities into route handlers.
DomainModels + interfaces
Domain concepts and provider interfaces separate business rules from database, queue, and external-provider implementations.
InfrastructurePersistence, queues, observability
PostgreSQL/pgvector, Redis, RabbitMQ/Celery, logging, metrics, and tracing sit behind infrastructure boundaries defined by the repository architecture.
Agentic adaptationModel → plan → reflect
Specialized agents update the user model, generate personalized research plans, evaluate recommendation performance, and record adaptation decisions for traceability.
Dependency rule
API handlers call application services; application logic depends on domain abstractions and infrastructure boundaries. The repository explicitly documents that domain code should not import infrastructure and infrastructure should not depend back on application services.
03 · End-to-end platform flow
Discovery becomes a feedback-controlled system
User profile & interests
Explicit interests, profile depth, expertise, and later behavioral signals form the personalization context.
Research planning
ResearchPlanningAgent combines explicit interests with learned topic preferences, expands queries through an LLM when available, and creates a persisted SearchPlan.
Discovery & processing
Discovery/content services ingest and process information before it enters ranking and feed generation.
Ranking
RankingEngine scores candidate content by relevance, credibility, freshness, novelty, and feedback, then persists both recommendations and explanation traces.
Feed interaction
Users consume the ranked feed and generate behavioral signals through likes, bookmarks, ignores, dislikes, reads, and related interactions.
Adaptation
UserModelingAgent turns those signals into evolving topic/source preferences; ReflectionAgent evaluates performance and the AdaptationEngine coordinates the next cycle.
04 · Recommendation engine
Explainable weighted ranking instead of a single opaque score
The implemented RankingEngine scores processed content with five explicit factors. Recommendations are persisted together with RecommendationTrace records containing the score breakdown, matched interests, weights, and explanation.
Relevance
40%
Match explicit interests and learned topic preferences against content topics.
Credibility
20%
Blend source trust with the user-specific source preference.
Freshness
15%
Apply exponential time decay to published/discovered time.
Novelty
15%
Penalize content the user has already interacted with.
Feedback
10%
Reserve an explicit feedback contribution in the weighted ranking formula.
Freshness is implemented with exponential time decay. Credibility combines global source trust with personal source preference, while relevance considers both explicit interests and learned topic preferences.
05 · Agentic adaptation
Observe → Interpret → Decide → Act → Learn
The adaptation layer is not a generic “AI agent” label. The repository contains explicit UserModelingAgent, ResearchPlanningAgent, ReflectionAgent, and AdaptationEngine responsibilities, with adaptation decisions recorded as AdaptationEvents.
Step 1
Observe
Collect recent user-interest signals and recommendation-performance data.
Step 2
Interpret
Aggregate behavior into topic and source preference changes with explicit confidence values.
Step 3
Decide
Generate personalized research queries, source priorities, and recommended adaptation actions.
Step 4
Act
Persist search plans, preference updates, recommendation changes, and adaptation events.
Step 5
Learn
Reflection evaluates engagement and recommendation quality so later research and ranking cycles can adapt.
06 · User modeling
Behavioral signals become evolving topic and source preferences
UserModelingAgent loads recent interest signals, maps them back to content topics and sources, aggregates signal strength, then moves topic weights toward the observed behavior. Confidence grows as more evidence accumulates. Source preferences are separately updated from positive and negative interactions.
Topic adaptation
Likes, saves, long reads, ignores, and dislikes influence learned topic weights. Every change records old/new values and the behavioral evidence behind it.
Source adaptation
Personal source trust evolves independently from global source trust, allowing two users to receive different ranking behavior from the same underlying content pool.
07 · Research planning & reflection
ResearchPlanningAgent
Combines explicit interests with learned topics, optionally expands them through an LLM provider, chooses source priorities from profile depth/expertise, and persists the resulting SearchPlan.
ReflectionAgent
Evaluates recommendation count, feedback breakdown, average score, acceptance rate, likes, and dislikes, then produces insights and recommended adaptation actions for the next cycle.
08 · Reliability & observability
AI behavior sits inside ordinary operational controls
Trace IDs
TraceIDMiddleware attaches request-level identity so failures can be correlated across logs and responses.
Rate limiting
RateLimitMiddleware exists in the API middleware stack rather than relying only on deployment infrastructure.
Audit trail
AuditMiddleware records request activity while adaptation decisions are separately persisted as AdaptationEvents.
Global errors
Unhandled exceptions are converted into structured 500 responses containing a trace_id and stable error envelope.
Structured logging
Backend components use structured log events instead of ad-hoc print debugging.
OpenTelemetry
Tracing setup is initialized at application startup, with metrics exposed through a dedicated router.
09 · Testing & delivery
The repository tests more than the happy-path API
The GitHub Actions workflow starts PostgreSQL/pgvector and RabbitMQ services for the backend pipeline, then exercises multiple validation levels before Docker build checks.
Ruff linting for the Python backend
Backend unit tests with coverage output
Backend integration tests against PostgreSQL/pgvector
Backend end-to-end tests
Next.js frontend type-check and production build
Docker build checks for backend and frontend on main
10 · Engineering decisions
What makes the architecture interesting
Keep the core platform as a modular monolith instead of splitting ordinary business functions into unnecessary microservices.
Move expensive/background work to event-driven processing with RabbitMQ and Celery rather than forcing every operation through synchronous request latency.
Persist recommendation traces so ranking can be explained and inspected rather than returning only a score.
Separate explicit interests from learned behavioral preferences so personalization can adapt without destroying the user-provided model.
Record agent decisions as AdaptationEvents so AI-driven changes have a durable reason, confidence, and before/after context.
Treat tracing, logging, rate limiting, auditability, and error envelopes as application concerns—not something added only after deployment.
Engineering takeaway
DeepFeed is useful portfolio evidence because the AI component is only one part of the system. The repository forces personalization, ranking, agent behavior, APIs, persistence, asynchronous work, traceability, testing, and deployment concerns to coexist inside one coherent architecture. That is the engineering problem the project is intended to explore.