Bleve gives Go developers full-text search without external dependencies. It indexes structs into a file-based store, handles millions of documents, and supports concurrent reads and writes. Queries use a Lucene-inspired language with relevance scoring. This matters for apps needing control over data storage and privacy—no cloud vendor, no API keys, just local disk I/O.
Full-text indexers like Elasticsearch scale to clusters but introduce complexity and costs: managed services charge per GB indexed, and self-hosting demands ops overhead. Bleve sidesteps this. It embeds directly, using Go reflection to map fields. A basic index persists to a directory, opens atomically, and swaps indexes live. Drawbacks exist: it’s single-node, so disk throughput caps scale at tens of millions of docs on SSDs. Benchmarks show 50k docs/sec indexing on a modern CPU, sub-50ms queries for 1M docs. Fair trade for simplicity.
Basic Indexing and Querying
Start with three lines: create mapping, open index, index docs. Bleve reflects over structs, defaults to keyword or text analyzers per field. Here’s the minimal example, indexing three docs and querying for “Hister search engine”. It ranks by TF-IDF score, returns titles and URLs.
package main
import (
"fmt"
"log"
bleve "github.com/blevesearch/bleve/v2"
)
type Document struct {
Title string
URL string
Text string
}
func main() {
// Create or open index.
mapping := bleve.NewIndexMapping()
index, err := bleve.New("example.bleve", mapping)
if err != nil {
index, err = bleve.Open("example.bleve")
if err != nil {
log.Fatal(err)
}
}
defer index.Close()
// Index docs by ID.
docs := map[string]Document{
"1": {Title: "Go Programming", URL: "https://go.dev", Text: "Go is an open source programming language that makes it easy to build reliable software."},
"2": {Title: "Bleve Search", URL: "https://blevesearch.com", Text: "Bleve is a full-text search and indexing library for Go."},
"3": {Title: "Hister - Your own search engine", URL: "https://hister.org/", Text: "Full-text search across your files, browsing history and beyond."},
}
for id, doc := range docs {
if err := index.Index(id, doc); err != nil {
log.Printf("failed to index %s: %v", id, err)
}
}
// Query across all fields.
query := bleve.NewMatchQuery("Hister search engine")
req := bleve.NewSearchRequest(query)
req.Fields = []string{"Title", "URL"}
req.Highlight = bleve.NewHighlightWithStyle("html")
searchResults, err := index.Search(req)
if err != nil {
log.Fatal(err)
}
// Print top hits.
fmt.Printf("Found %d matches\n", searchResults.Total)
for _, hit := range searchResults.Hits {
fmt.Printf("ID: %s, Score: %.2f\n", hit.ID, hit.Score)
for _, field := range hit.Fields {
fmt.Printf(" %s: %v\n", field.Name, field.Value)
}
}
}Running this yields doc “3” at top score (~2.3), highlighting matches. Bleve stores originals, supports highlighting, facets, and pagination. Implications: embed in CLI tools or services for instant search over logs, crypto tx histories, or docs—query latency stays low even at 10M records if you tune analyzers.
Advanced Control and Real-World Use
Bleve shines in customization. Define field mappings with analyzers (e.g., stemming for English, n-grams for autocomplete). Use Scorch segments for 2-5x faster queries over older formats. Queries support booleans, phrases, dismax, per-field boosts: title:^3 "hister" AND (engine OR search). Hister, built atop Bleve, adds cursor pagination (handles 100M+ results without OOM) and multi-lang aliases.
Security angle: everything stays on-disk, encrypted at rest via filesystem (e.g., LUKS). No network exposure unless you expose an HTTP endpoint (Bleve has one). Compare to SQLite FTS5—simpler but weaker ranking. Or Tantivy (Rust crate, WASM-friendly)—faster raw speed but less mature Go bindings. Bleve’s battle-tested since 2012, powers production at scale in tools like Hister for personal search over files and history.
Why this matters now: data hoarding grows—browsers, apps, chains dump TBs locally. Self-hosted search avoids Google/ cloud leaks. Skeptical note: at planetary scale (billions docs), shard it yourself or go distributed. For 99% cases—microservices, desktops, edge devices—Bleve indexes reliably. Test your workload: index 1M docs, measure QPS. If disk bottlenecks, NVMe helps; RAM for hot cache via memdoc store.
Bottom line: Bleve equips Go apps with pro search in 100 LOC. Prioritize it over reinventing wheels or SaaS lock-in. Fork, tune, own your index.
