Building a Career Path Recommender: Notes on AI-powered GPS for your Career

Patrick Donahue · Levelbrook Consulting

Traditional career planning often feels like navigating with a static paper map. You can see potential destinations—job titles, industries—but the viable routes, the necessary waypoints, and the dynamic "road conditions" of the market are opaque. This is where the concept of an "AI-powered GPS for your career" becomes a fascinating engineering problem. It’s not about simple keyword matching; it's about modeling career trajectories, understanding skill adjacencies, and illuminating paths that a person might not have known existed.

The technical challenge is rich. We're dealing with a high-dimensional, sparse, and deeply personal dataset. A career isn't a simple linear progression; it's a complex graph of skills acquired, roles held, and individual preferences that change over time. Building a system to navigate this space requires a thoughtful blend of data modeling, machine learning, and user-centric design.

Try the interactive demo

System Blueprint: From Data to Direction

To build a robust recommender, we need a flexible architecture that can handle structured relational data, unstructured text, and real-time user interactions. A stack combining React/TypeScript for the frontend with a Node.js backend, backed by both PostgreSQL and MongoDB on AWS, provides a powerful and pragmatic foundation.

The Data Backbone: Relational Truth and Document Flexibility

The core of the system is its data model, which we can split based on data shape and access patterns:

PostgreSQL for Structured Data: This is our source of truth for the "map." It's perfect for entities with clear relationships: users (auth, core profile), companies, standardized job roles, and, most importantly, a curated skill graph. We can model skills as nodes, with edges representing prerequisites or relationships (e.g., `React` is a type of `JavaScript Framework`). This structured approach allows for powerful, reliable graph traversal queries and transactional integrity.
MongoDB for Unstructured & Event Data: This is our "telemetry log." It excels at storing flexible, schema-less data like raw parsed resume text, user interaction events (clicks, saves, dismissals), and cached recommendation sets. Its write performance makes it ideal for capturing the high volume of feedback signals that are crucial for model training.

This hybrid approach prevents us from forcing unstructured data into rigid SQL tables or sacrificing the integrity and query power of a relational database for our core domain model.

The Recommendation Engine: A Multi-Stage Pipeline

A performant recommendation engine rarely relies on a single, monolithic model. A multi-stage filtering and ranking pipeline is more effective and scalable. A request would flow through a Node.js service orchestrated on AWS (e.g., using ECS or Lambda).

Candidate Generation: The first step is to quickly identify a broad set of a few hundred potentially relevant career moves. This must be fast. We can achieve this by converting user profiles and target job descriptions into embeddings (numerical vector representations). Using an extension like pgvector in PostgreSQL, we can perform an efficient nearest-neighbor search to find roles that are semantically similar to the user's experience.
Re-ranking: This is where the deeper personalization occurs. The initial candidate set is passed to a more sophisticated model. This model can consider more features: the user's explicit preferences, interaction history from MongoDB, the skill gap between the user's current skills and the target role's requirements (calculated via the skill graph), and salary expectations. This re-ranking step produces the final, ordered list of recommendations.
Explanation Generation: A recommendation without justification is a black box that erodes trust. The final step is to use an LLM (e.g., a fine-tuned open-source model or a commercial API) to generate a human-readable explanation for the top recommendations. The prompt would be carefully constructed with structured data: "Generate a one-sentence explanation for why a user with skills [A, B] would be a good fit for a role requiring skills [A, C, D]." This transforms a simple list into actionable advice.

Real-Time UX and Scaling Considerations

The user experience must feel responsive. A user tweaking their profile shouldn't have to wait ten seconds for a full page reload. On the frontend, a React/TypeScript application can manage state optimistically. For the backend communication, Server-Sent Events (SSE) are an excellent fit. When a user action triggers a new recommendation computation, the server can immediately open an SSE connection and stream back results as they are processed by the re-ranking and explanation stages. This gives the UI a dynamic, "live" feel without the complexity of full duplex WebSockets.

At scale, this architecture will face bottlenecks. The read load on PostgreSQL during candidate generation can be mitigated with read replicas. The re-ranking models can be deployed as separate microservices that auto-scale independently. Heavy caching (e.g., with Redis) is non-negotiable for pre-computed user vectors, popular skill paths, and expensive query results. The data ingestion pipeline, which processes millions of job postings, would be built on a message queue like SQS and serverless functions like Lambda to ensure it can handle massive, spiky workloads without impacting the user-facing application.

Navigating Uncertainty: Tradeoffs and The Human Element

Engineering is about tradeoffs, and in a domain as subjective as career advice, pragmatism and user trust are paramount.

Correctness and the Hallucination Guardrail

The single fastest way to destroy trust is to provide a nonsensical recommendation. While LLMs are powerful, they can "hallucinate" plausible-sounding but nonexistent career paths. The system's ground truth cannot be the latent space of a language model. It must be our curated PostgreSQL skill and career graph. The LLM is a powerful tool for natural language understanding and generation—for interpreting the graph—but it should not be allowed to arbitrarily invent new connections. The recommendations must be validated against our structured data.

Explainability as a Core Feature

As mentioned, explainability isn't a nice-to-have; it's a core requirement. A user needs to understand the "why" to evaluate a suggestion. Explanations like, "Because you have strong experience in Python and data analysis, a 'Data Engineer' role is a logical next step. To be a top candidate, focus on learning 'Apache Airflow'" provides a clear rationale and an actionable learning path.

The User as Co-Pilot

Ultimately, the system is a tool to augment the user's own judgment, not replace it. The UI must treat the user as a co-pilot. This means providing explicit controls:

Feedback Mechanisms: Simple "this is relevant" or "not interested" buttons on each recommendation are the most valuable source of training data. This is the essence of Reinforcement Learning from Human Feedback (RLHF) and is critical for refining the personalization models over time.
Parameter Tuning: Allow users to directly influence the algorithm. Toggles like "Show more senior roles," "Focus on remote work," or "Prioritize skill growth over salary" empower the user and make them a partner in the exploration process.

The cold-start problem—recommending for a new user with no history—is best solved by a well-designed onboarding flow that explicitly asks about their interests and aspirations, seeding the model with this crucial initial data.

Reflection: Beyond the Next Job

Building an AI-powered "career GPS" is a compelling problem because it sits at the intersection of data science, UX engineering, and human psychology. It forces us to think deeply about how to model something as complex and personal as a career. The goal isn't just to find the next job listing more efficiently. It's to provide a tool for exploration that illuminates the landscape of possibility, revealing adjacent career paths and the concrete steps required to navigate to them. The ultimate measure of success for such a system is not a prescriptive path, but the sense of agency and clarity it provides to the user navigating their own professional journey.