MindsDB Query Engine

Description

MindsDB's Query Engine empowers users to construct sophisticated semantic search pipelines using only SQL. This framework allows for seamless integration of data sources, enabling users to shape and prepare data through views and tables. A key capability is the ability to index unstructured content into knowledge bases, which are essentially semantic indexes that match content by meaning rather than keywords.

Once data is indexed, users can expose it through a natural-language agent. This agent acts as an intelligent interface, allowing for queries in plain English. The entire process, from data connection to querying the agent, can be managed without ever leaving a SQL client. This approach democratizes AI capabilities, making them accessible to database professionals and developers familiar with SQL.

The framework supports a robust workflow: connect data sources, index content into knowledge bases, and query through an agent. This is facilitated by a series of SQL statements for creating databases, projects, knowledge bases, and agents. MindsDB offers flexible deployment options, including Docker, and integrates with various clients via a MySQL-compatible wire protocol.

Key features include the ability to create and manage databases (connections to external data sources), projects (namespaces for organizing AI assets), tables, and views for data preparation. Knowledge bases are central to MindsDB's search capabilities, leveraging embedding models, optional reranking models, and vector stores for context-aware retrieval. Agents provide the conversational interface, combining LLMs with access to knowledge bases and tables for natural language querying over both structured and unstructured data.

MindsDB is designed for developers, data scientists, and database administrators who want to embed AI capabilities, particularly semantic search and natural language querying, directly into their data workflows. The value proposition lies in simplifying complex AI implementations into familiar SQL commands, reducing the learning curve and accelerating development cycles for AI-powered applications.

MindsDB Query Engine's Core Features

• Build semantic search pipelines in SQL

• Connect live data sources

• Index unstructured content into knowledge bases

• Query data via natural-language agents

• Manage data with views and tables

• Supports hybrid vector + keyword search

• Integrates with LLMs for agent capabilities

• Data is queried live, not copied

• Flexible deployment options (Docker)

• MySQL-compatible wire protocol for client connections

• Supports standard SQL syntax with extensions

• Knowledge bases match content by meaning

• Agents can query structured and unstructured data

Getting Started with MindsDB Query Engine

1. 1

   Install: Set up MindsDB locally using Docker or pip.

2. 2

   Connect Data: Use CREATE DATABASE statements to link external data sources.

3. 3

   Create Project: Organize AI assets like knowledge bases and agents.

4. 4

   Create Knowledge Base: Define semantic indexes using embedding models.

5. 5

   Index Data: Insert content from tables or views into the knowledge base.

6. 6

   Build Agent: Create a conversational interface using an LLM and data sources.

7. 7

   Query Agent: Ask natural language questions to retrieve information.

MindsDB Query Engine's Use Cases

• Semantic Document Search
• Customer Support Automation
• Data Analysis with Natural Language
• Content Recommendation Systems
• Internal Knowledge Management
• Hybrid Search Implementation

FAQ from MindsDB Query Engine

• What is the MindsDB Query Engine?

  The MindsDB Query Engine is a framework that allows users to build semantic search pipelines and natural language agents entirely within SQL, connecting to various data sources and indexing content for intelligent retrieval.

• How does MindsDB enable semantic search?

  MindsDB uses knowledge bases, which are semantic indexes created with embedding models. These indexes match content by meaning, enabling searches like 'forgot password' to find relevant documents even without exact keyword matches.

• Can I use my existing SQL clients with MindsDB?

  Yes, MindsDB exposes a MySQL-compatible wire protocol, allowing any standard MySQL client, such as MySQL CLI, DBeaver, or TablePlus, to connect and interact with MindsDB.

• Does MindsDB copy my data?

  No, MindsDB never copies your data. It queries external data sources live each time a query is executed, ensuring data remains in its original location.

• What are Projects in MindsDB?

  Projects are namespaces within MindsDB used to organize knowledge bases, agents, views, and jobs, allowing for better management of AI pipelines by use case.

• How do I index unstructured content?

  You create a Knowledge Base using an embedding model and then use INSERT INTO statements to feed text content from your tables or views into it for semantic indexing.

• What is the role of an Agent in MindsDB?

  An Agent acts as a conversational interface over your data. It combines a Large Language Model (LLM) with access to knowledge bases and database tables, enabling natural language queries.

• Can MindsDB perform hybrid search?

  Yes, MindsDB supports hybrid search, which combines semantic similarity (vector embeddings) with exact keyword matching (BM25 full-text index), requiring PGVector as the storage backend.

• How can I keep my knowledge bases up-to-date?

  You can use MindsDB Jobs to schedule SQL statements, such as INSERT statements, to run automatically at specified intervals, ensuring your knowledge bases are updated with new data.

• What LLM providers are supported for Agents?

  MindsDB supports various LLM providers including OpenAI, Anthropic, Google, and AWS Bedrock, allowing flexibility in choosing the best model for your agent.