AI-powered search engine that connects recruiters, founders, and investors with the right people faster. We combine data from LinkedIn, Gmail, and professional networks into a private AI search layer that understands relationships, intent, and relevance. Our MVP is live with: • 10K+ candidates embedded in a pgVector database • TypeScript backend and React frontend • OpenAI embeddings for candidate matching Design an evaluation framework to measure search relevance • Benchmark and experiment with different embedding models (OpenAI, Cohere, SBERT, etc.) • Implement hybrid retrieval (vector + BM25 + re-ranking) • Develop and test fusion strategies (RRF, weighted, or learning-to-rank) • Introduce re-ranking with cross-encoders for top results • Optimize for both precision and recall with latency under 2 seconds • Document findings, metrics, and model decisions • (Later) Help build continuous monitoring and drift detection ⸻ Tech Stack You enjoy • Python (FastAPI, PyTorch, SentenceTransformers) • PostgreSQL + pgVector • OpenAI / Cohere APIs • rank-bm25 or Elasticsearch • ranx / ir-measures (evaluation) • Jupyter, MLflow, or Weights & Biases for experiment tracking • Integration with existing TypeScript backend (via API) Test task(paid) Evaluate and improve the semantic search quality of Yena’s current setup using a small sample dataset. What to Do 1. Dataset Setup • Use a sample CSV of 100 candidate profiles (we’ll provide, or you can mock). • Each profile includes: title, company, location, skills, experience summary. 2. Build Evaluation Sandbox • Use OpenAI or any embedding model to encode candidates. • Create 5–10 realistic search queries (e.g. “Senior Python Engineer in Berlin”). • Manually label 2–3 relevant candidates per query. 3. Compute Search Quality Metrics • Implement Recall@10 and MRR. • Output baseline numbers in JSON/CSV. 4. Suggest or Test One Improvement • Could be model change, BM25 hybridization, or re-ranking. • Compare metrics and describe what changed. 5. Deliverables • Jupyter notebook or Python script • CSV/JSON with metrics results • 1-page written summary: findings, reasoning, next steps