Build a "chat with your website data" agent with Jina Embeddings v5 and Elasticsearch

Agent Builder is available now GA. Get started with an Elastic Cloud Trial, and check out the documentation for Agent Builder here.

Build a “chat with your website” experience in under an hour using Elasticsearch Serverless, Jina Embeddings v5, Elastic Open Web Crawler, and Elastic Agent Builder.

By the end, you’ll have a working agent that can search your crawled pages, cite relevant passages, and answer questions grounded in your content, no custom chunking or embedding pipeline required.

In this guide, you’ll:

1. Start an Elasticsearch Serverless project.
2. Create an index using the new semantic_text field powered by Jina Embeddings v5.
3. Crawl any website using Elastic Crawler Control (a.k.a. Crawly) (an open source UI + API wrapper around the Elastic Open Web Crawler).
4. Chat with that data using the Elastic Agent Builder in Kibana.

What you’ll walk away with:

• A repeatable pattern you can point at any website/docs source.
• Chat that stays grounded in your content.

Prerequisites

• An Elasticsearch Serverless (Search) project + an API key with write permissions.
• Docker + Docker Compose (to run the crawler UI).
• git (to clone the repo).

1. Start an Elasticsearch Serverless project

First, we need a serverless project to host our data.

1. Log in to your Elastic Cloud Console.

2. Click Create project.

3. Select Search as the project type. (This type is optimized for vector search and retrieval.)

4. Give it a name (for example, es-labs-jina-guide), and click Create.

5. Important: Save the Elasticsearch endpoint and API Key provided when the project is created. You’ll need these for the crawler.

2. Create the index

Elasticsearch Serverless supports semantic_text, which handles chunking and embedding generation automatically. We’ll use the .jina-embeddings-v5-text-small model that’s hosted on GPUs on Elastic Inference Service.

Create the index with the semantic_text field. This tells Elastic to automatically vectorize content put into the field property using the inference endpoint we just created.

In Kibana Dev tools run:

3. Run the Elastic Open Crawler

Crawly is one example of how an application can be constructed around the functionalities that the Open Web Crawler provides.

The application wraps the Elastic Open Crawler in a FastAPI service that manages crawler processes and persists execution data. A React front end provides the interface for configuring and monitoring crawls.

What happens under the hood is that the crawler service (check crawler.py) spawns JRuby processes via subprocess.Popen, allowing multiple concurrent crawls. Each execution's configuration, status, and logs are persisted to disk (for now).

Clone the repository:

Create an env.local file with your Elasticsearch credentials:

Start the services:

Access the UI at http://localhost:16700

You don’t necessarily need seed_urls unless you want to be specific, so your config can be as simple as below:

From there, you can start a crawl on any website and check its progress:

Once it's finished, we’re ready to query the content in Elasticsearch directly or use the pages you just crawled for chatting with the website on Agent Builder.

4. Chat with data in Kibana

Now that the data is indexed and vectorized, we can start chatting with the data using the Elastic Agent Builder.

1. Open Kibana, and navigate to Agents (under the "Search" section).
2. Test the agent:
   • In the chat window, ask a question, like,"What is the difference between sparse and dense vectors?"

The agent will search your Jina-embedded data, retrieve the relevant snippets from the Search Labs blog posts, and generate an answer.

You can also chat with the data directly via Kibana API:

Use conversation_id to resume an existing conversation with an agent in Elastic Agent Builder. If you don’t provide it on the initial request, the API starts a new conversation and returns a newly generated ID in the streaming response.

Summary

You now have a working “chat with your website” stack: Your site gets crawled, indexed, auto-embedded with semantic_text + Jina v5, and surfaced through an agent in Kibana that answers questions grounded in your pages.

From here, you can point the same setup at docs, support content, or internal wikis and iterate on relevance in minutes.