Part of Vulos
Reap the open web — a decentralized crawl → extract → index engine that stores the index, not the pages.
Vuna is a network of volunteer nodes that crawl a distributed, subscribable list of URLs, run pluggable extractors over each page, and contribute the result — keyword postings, vector embeddings, link/knowledge-graph edges, a snippet, and a pointer back to the live URL — into a shared index replicated across the network. Pages themselves are never archived, which is what collapses storage from datacenter scale to roughly 2 GB per volunteer node at billion-page scale. One engine, two verticals so far — web search/RAG and retail price-radar — riding the KOTVA substrate's identity, PUB, DHT and SEARCH primitives. No token, no new crypto.
Shown: the desktop app's search view. All data shown is mock — vuna-query and vuna-node are Wave-2 stubs (see status). Every screen is in the gallery below.
One engine, pluggable everything
Crawl, extract and index are three separable stages joined by seam traits in vuna-core — so a new embedding model, a new vertical, or a new ranking strategy is a new implementation of an existing trait, not a fork of the engine.
Crawl → extract → index
A polite crawler (robots.txt, per-host rate limits, body caps) hands pages to pluggable extractors, which hand structured output to a keyword + vector + graph index — one pipeline, swappable at every joint.
Index, not pages
Keyword postings, embeddings, link/graph edges, a snippet, and a pointer back to the live URL — never the page body. That single decision is what turns a petabyte-scale archive into single-digit gigabytes per volunteer.
Multi-model spaces
Each embedding model is an independent, parallel vector space; the keyword index and link graph stay shared and model-agnostic. Shards keep chunk text, so adopting a new space is local recompute — never a re-crawl.
Web search/RAG extractor
Emits chunks and links from a page for full-text search and retrieval-augmented generation — the default vertical, and the one most of the test suite exercises today.
Retail radar extractor
Reads a store's own JSON-LD/OpenGraph data for price and stock observations. The store signs nothing, so vuna-core's quorum reconciler treats agreement across k distinct, anchored observers as the only ground truth.
Distributed URL frontier
A subscribable list of URLs, deduplicated and DHT-assigned across nodes — the ad-filter-list model applied to a crawl queue, so no single operator owns what gets crawled next.
v0 preview — honest about what's real
63 tests passing describes what compiles and behaves against its own fixtures, not 63 features shipped. There are zero live nodes and zero real users today — read this table before the pitch above.
web (chunks + links) and retail (JSON-LD/OG) extractors.kotva-core — crawl→index→publish loop.It does not yet crawl-to-query end to end. See the architecture chapter for the crate-by-crate detail and viability for the two open risks — compute-volunteer supply and small-network Sybil resistance — neither of which this design claims to have solved.
Every surface
The desktop app is a single screen today: a search box over local + peer results, and a node panel that surfaces exactly what your box is doing — no hidden telemetry. Real product screenshots, running on a mock corpus.
Quick start
One Cargo workspace for the engine, one Tauri + React app for the desktop shell. No Docker, no database server, no external service required to run the tests.
git clone https://github.com/vul-os/vuna cd vuna # run the whole engine test suite — offline, no network, no keys cargo test --workspace # 63 tests green # build every crate cargo build --workspace # desktop app — mock data today, real UI cd app && npm install npm run tauri dev # or: npm run dev (plain Vite, no Tauri)
- 1BuildA Rust toolchain (1.75+) plus Node for the desktop shell.
vuna-corehas minimal deps; heavy dependencies — tantivy, HNSW, reqwest, embedding runtimes — live only in the crate that needs them. - 2Test the engine
cargo test --workspaceruns all 63 tests across core, crawl, extract, index and frontier.vuna-queryandvuna-nodeare stubs — not exercised end to end yet. - 3Preview the appThe Tauri desktop shell builds and runs today against a mock corpus —
app/lib/api.tsfalls back to it when there is no live daemon, so the UI can be built and previewed on its own.
Index, not pages — and why that's the whole idea
Storing only the derived index instead of the pages collapses storage from datacenter-operator scale — hundreds of petabytes, for a "reliable" open-web archive — to single-digit gigabytes per volunteer node, even at billion-page scale. Compute, not storage, becomes the recurring cost — and it's embarrassingly parallel.
No page archive
Vuna never stores the page body — only keyword postings, vectors, graph edges, a snippet and a pointer back to the live URL. The author's copy stays the only copy; the index is derived and rebuildable, never authoritative.
Distributed frontier, DHT-assigned
URL lists are published and subscribed like ad-filter lists. Assignment across nodes runs over KOTVA's DHT, so no single operator holds the crawl queue and redundancy is a replication factor, not a backup job.
KOTVA substrate, not new crypto
Identity, PUB, DHT and SEARCH all come from KOTVA — the same substrate Envoir speaks. vuna-node is the one crate that binds it; every other crate compiles and tests fully offline.
| Corpus | 1 copy | ×3 redundant / 10k nodes |
|---|---|---|
| 1B pages | ~6 TB | ~1.8 GB / node |
| 10B pages (Google-ish) | ~60 TB | ~18 GB / node |
~6 KB/page: keyword ~2 KB · embeddings (int8) ~2.5 KB · graph ~0.3 KB · metadata ~0.7 KB. The honest trade-offs — compute-vs-disk volunteer economics, Sybil resistance at small scale, freshness — are in the viability chapter. Read it before believing the pitch.
Part of Vulos
Vulos is a family of open, self-hostable apps. Vuna is fully standalone — it never requires Vulos infrastructure to run a node or the desktop app — and it shares its identity/DHT/PUB substrate, KOTVA, with siblings like envoir. No hub, no account, no coupling.
Explore Vulos →