Summary Job Description We already have a working AI chatbot automation built in n8n. What is already implemented AI chatbot workflow built in n8n Chat triggers via messaging channels (Telegram / WhatsApp) Jina.ai web scraping tool integrated AI agent answers user questions based on scraped page content Basic memory handling inside the AI agent The chatbot currently scrapes pages on demand (per request) This part is already done and functional. What needs to be added / improved We are looking for an experienced n8n + AI automation specialist to extend this system with a scalable knowledge architecture. Website crawler & page discovery Automatically crawl a website starting from a base URL Identify and collect relevant pages only (exclude media, junk, duplicate, tracking URLs) Support dynamic structures (blogs, tour pages, category pages, etc.) Allow configuration for: depth include/exclude rules domain limits Vector database integration Store scraped content in a vector database (open to suggestions: Pinecone, Qdrant, Weaviate, Supabase Vector, etc.) Chunk content properly (semantic chunks, metadata included) Store: page URL page title content last updated timestamp Use vector search for retrieval-augmented answers (RAG) instead of live scraping Daily (scheduled) re-scraping system Set up a daily automated job in n8n that: re-scrapes pages detects changes updates vectors only when content has changed Handle removed pages and updated content correctly Log scraping status and errors AI agent integration Modify the existing AI agent to: retrieve answers from the vector database fallback gracefully when information is missing Ensure no hallucinations Answers must be strictly based on vector search results Clean architecture & documentation Clean, reusable n8n workflows Clear naming and structure Basic documentation explaining: how crawling works how vectors are updated how to add a new website later Required Experience Strong experience with n8n Experience building RAG systems Vector databases (any of the major ones) AI agents (OpenAI / Claude / Gemini / OpenRouter) Web scraping best practices Handling large text content and chunking Deliverables Updated n8n workflows Vector database setup and integration Daily scraping automation AI agent updated to use vector search Short documentation Pricing & Estimate (IMPORTANT) Please include in your proposa: Fixed price estimate for the full implementation Time estimate (in days) Brief explanation of: which vector DB you recommend and why how you would handle daily updates efficiently Examples of similar work (if available)