| title | Setup Guide |
|---|---|
| description | Install msgvault, choose a source, and explore your first archived messages. |
Install msgvault, bring in a small first batch, then open it in the browser or terminal. The Gmail walkthrough below includes Google OAuth setup. For other mail providers, chats, meetings, contacts, or local exports, use Choose a Source after installation.
Already using 0.19? Read the changelog’s upgrade notes before opening an existing archive with a newer build.
macOS / Linux:
curl -fsSL https://msgvault.io/install.sh | bashWindows (PowerShell):
powershell -ExecutionPolicy ByPass -c "irm https://msgvault.io/install.ps1 | iex"The installer detects your OS and architecture, downloads the latest release from GitHub Releases, verifies the SHA-256 checksum, and installs the binary.
Windows releases include native AMD64 and ARM64 packages. On Windows ARM64, the PowerShell installer selects the native package when the release provides one and falls back to the AMD64 package under emulation for older releases.
!!! tip "Running on a headless server?" msgvault works on headless machines (SSH, VPS, NAS, Docker), but OAuth requires a browser for the initial authorization. You'll authorize on your local machine and copy the token file to the server. See Headless Server Setup for the copy-token workflow, or jump to the Remote Deployment guide for a full NAS/server setup with Docker Compose.
Verify the installation:
msgvault --help# Using pixi (recommended)
pixi global install msgvault
# Using conda
conda install -c conda-forge msgvaultOn macOS and Linux, source builds require Go 1.27+, Bun 1.3.14+, Node.js
(20.19+ on 20.x, 22.13+ on 22.x, or 24+), and a C/C++ compiler (GCC or Clang).
Bun builds the browser application embedded in the binary, and Node runs the
embed validator that make install invokes. CGO is required because msgvault
uses mattn/go-sqlite3 (SQLite with FTS5) and duckdb-go/v2 (Parquet
analytics), both of which compile native extensions.
On Debian/Ubuntu also install libsqlite3-dev, which provides the sqlite3.h
header needed to compile the default sqlite_vec extension:
sudo apt install -y libsqlite3-devgit clone https://github.com/kenn-io/msgvault.git
cd msgvault
make installOn macOS and Linux this installs to ~/.local/bin or $GOPATH/bin. For a
debug build use make build, or make build-release for an optimized binary
with stripped debug symbols.
On Windows, use the native PowerShell helper:
.\scripts\build.ps1 # Debug build
.\scripts\build.ps1 -Release # Optimized, stripped buildIt detects AMD64 or ARM64 automatically and writes msgvault.exe in the
repository root. See Development for the
one-time MSYS2 compiler prerequisites.
Verify the installation:
msgvault --helpThis section is for Gmail and other Google-backed sources. Skip it for local imports or a provider with its own authorization guide.
Create a Google Cloud project, enable the Gmail API, and download your client_secret.json. If you plan to archive Google Calendar, enable the Google Calendar API too. See the full OAuth Setup Guide.
msgvault stores all data (config, database, tokens, attachments) in a single directory. The default location depends on your platform:
| Platform | Data directory | Config file |
|---|---|---|
| macOS / Linux | ~/.msgvault/ |
~/.msgvault/config.toml |
| Windows | C:\Users\<you>\.msgvault\ |
C:\Users\<you>\.msgvault\config.toml |
!!! tip
The .msgvault directory is created automatically the first time you run any msgvault command. If you're unsure of the exact path, run msgvault add-account you@gmail.com; the error message may show you where to create the config file.
To store data on a different drive or location, use the --home flag or set the MSGVAULT_HOME environment variable. If MSGVAULT_HOME is set, paths in the table above are relative to that directory instead:
Per-command (any platform):
msgvault sync --home E:/msgvaultWindows (PowerShell, persistent):
$env:MSGVAULT_HOME = "E:\msgvault"
# Or set it permanently:
[Environment]::SetEnvironmentVariable("MSGVAULT_HOME", "E:\msgvault", "User")macOS / Linux (persistent):
export MSGVAULT_HOME=/mnt/data/msgvaultThe --home flag takes priority over MSGVAULT_HOME. See Configuration for all options.
macOS / Linux:
[oauth]
client_secrets = "/path/to/client_secret.json"Windows: use forward slashes in the path:
[oauth]
client_secrets = "C:/Users/you/Downloads/client_secret.json"msgvault add-account you@gmail.comThis opens your browser for OAuth consent. For headless servers, see the copy-token workflow.
If you plan to deploy to a remote host (NAS, cloud VM, etc.), run msgvault setup after this step to generate a ready-to-run deployment bundle with Docker Compose and remote configuration. See the Remote Deployment guide.
To sync mail from a non-Gmail provider (Fastmail, Outlook, Yahoo, self-hosted, etc.), use add-imap:
msgvault add-imap --host imap.fastmail.com --username you@fastmail.comYou will be prompted for your password (or set MSGVAULT_IMAP_PASSWORD / pipe via stdin for scripting). The command tests the connection before saving credentials. Use an app-specific password if your provider supports them.
Common IMAP servers:
| Provider | Host | Port | Notes |
|---|---|---|---|
| Fastmail | imap.fastmail.com |
993 | App password recommended |
| Outlook / Hotmail | outlook.office365.com |
993 | Use add-o365 for OAuth (recommended); or app password with 2FA |
| Yahoo | imap.mail.yahoo.com |
993 | App password required |
| iCloud | imap.mail.me.com |
993 | App-specific password required |
| Gmail (IMAP) | imap.gmail.com |
993 | Use add-account for Gmail API instead |
| Self-hosted | your server hostname | 993 |
For STARTTLS connections (port 143), add --starttls:
msgvault add-imap --host mail.example.com --username you@example.com --starttlsAfter adding the account, list its folders and start a sync:
msgvault list-folders you@fastmail.com
msgvault sync-full you@fastmail.comIMAP accounts are stored in the same database as Gmail accounts. All tools
(Web UI, TUI, search, MCP, and REST API) work with IMAP messages the same way.
To start with only part of a large account, see
IMAP Folder Sync for --folder and --skip-folder examples.
!!! tip "Microsoft 365 / Outlook.com"
For Outlook, Hotmail, Live.com, and Microsoft 365 accounts, add-o365 provides OAuth-based access without app passwords. It auto-detects the correct IMAP host and configures XOAUTH2 authentication. See the OAuth Setup guide for details.
!!! tip "Yahoo App Passwords" Yahoo requires an App Password for IMAP access. Your regular Yahoo password will not work. To generate one:
1. Go to [Yahoo Account Security](https://login.yahoo.com/account/security)
2. Under **Generate and manage app passwords**, click **Generate app password**
3. Enter `msgvault` as the app name and copy the generated password
4. Use this password when `add-imap` prompts for your credentials
!!! note
IMAP sync always performs a full scan of the mailbox. The sync (incremental) command falls back to a full sync for IMAP accounts because IMAP does not provide a change-tracking API like Gmail's History API. Messages already in the database are skipped efficiently.
# Test with a small batch first
msgvault sync-full you@gmail.com --limit 100
# Or sync a specific date range
msgvault sync-full you@gmail.com --after 2024-01-01 --before 2024-02-01
# Sync everything (no limit)
msgvault sync-full you@gmail.comThe initial full sync downloads every message and attachment from Gmail, so it can take a while. In testing we have observed roughly 50 messages per second on fast internet, but the Gmail API has per-user quotas that may throttle throughput further. An account with hundreds of thousands of messages and large attachments may take several hours; an account with millions of messages could take significantly longer. We recommend starting with --limit or a date range to verify everything works before kicking off the full run.
The good news: syncs are resumable (see below), and once the initial sync is complete, incremental syncs only fetch new and changed messages, which is much faster.
msgvault stores raw MIME data compressed with zlib (typically 3-5x compression). As a rough guide:
| Gmail usage (Settings → Storage) | SQLite DB on disk | Parquet cache | Attachments |
|---|---|---|---|
| 5 GB | ~1-2 GB | < 10 MB | varies |
| 25 GB | ~5-10 GB | < 50 MB | varies |
| 100 GB | ~20-40 GB | < 100 MB | varies |
Gmail's "storage used" number includes attachments at full size. Your on-disk footprint depends on the ratio of message text to attachments:
- Message metadata + bodies go into the SQLite database, compressed ~3-5x.
- Attachments are extracted and stored as-is (PDFs, images, etc. are already compressed). Identical attachments across messages are deduplicated by content hash.
- Parquet analytics cache is a lightweight projection for the Web UI and TUI — typically a few MB even for large archives.
Use --limit or a date range for your first sync to gauge the ratio for your mailbox before committing to a full sync. After syncing, msgvault stats shows the actual sizes. See Data Storage for details on compression and storage layers.
| Flag | Description |
|---|---|
--limit N |
Download at most N messages |
--after YYYY-MM-DD |
Only messages after this date |
--before YYYY-MM-DD |
Only messages before this date |
--query |
Gmail search query filter |
--noresume |
Start fresh instead of resuming |
--verbose |
Show detailed progress |
After the initial full sync, use incremental sync for efficient updates. It uses the Gmail History API to fetch only new and changed messages:
msgvault sync you@gmail.com
# Or sync all accounts at once
msgvault syncIf a sync is interrupted (network error, Ctrl+C), run the same command again. It resumes from the last checkpoint:
# This resumes automatically
msgvault sync-full you@gmail.com --after 2024-01-01 --before 2024-02-01Checkpoint data is stored in the sync_checkpoints table. Use --noresume to discard checkpoints and start over.
msgvault uses token bucket rate limiting to respect Gmail API quotas. The default is 5 requests per second, configurable in config.toml:
[sync]
rate_limit_qps = 5Reduce this value if you encounter rate limit errors during large syncs.
Sync operations are read-only. They use only messages.list and messages.get Gmail APIs. No write operations are performed. Your Gmail data remains untouched.
# Search your archive
msgvault search from:alice@example.com
# Open the analytical Web UI
msgvault serve
# Launch the interactive TUI
msgvault tui
# View stats
msgvault stats
Semantic search, visual and document attachment search, and the people sweep are opt-in. Put the API keys you have in the environment and let setup choose the rest:
export VOYAGE_API_KEY="..." # text, people, and visual search
export MISTRAL_API_KEY="..." # document attachments
export OPENAI_API_KEY="..." # people sweep (and text search without a Voyage key)
msgvault setup providers # one consent per provider, then config.toml is written
msgvault setup status # what is on, what is off, and whyThe people sweep additionally requires msgvault setup providers --allow-sensitive
to permit sensitive archive excerpts and sensitive personal inferences.
See Recommended Configuration for the values it writes and the probe steps the hosted lanes still need.
To archive Calendar events alongside email, authorize Calendar access and run a calendar sync:
msgvault add-calendar you@gmail.com
msgvault sync-calendar you@gmail.comCalendar sync is read-only. Events become searchable with
--message-type calendar_event; see Google Calendar for the
full workflow, scheduled sync, and headless-server setup.
Build the analytical cache and start the daemon:
msgvault build-cache
msgvault serveOpen the API server URL printed at startup. With the default loopback bind,
the browser is trusted locally. A daemon bound to another interface must use an
API key; the browser presents a login screen and stores only an in-memory
daemon session. The release binary contains the complete UI, so no frontend
runtime or separately installed web files are required.
For a server or NAS, prefer HTTPS at a reverse proxy and configure only that
proxy in server.trusted_proxies. Plain HTTP is an explicit private-network
tradeoff because its session cookie cannot be marked Secure. See Web UI
for URL discovery, search/index states, keyboard controls, and deployment
examples.
To archive Teams chats and channels, add Microsoft Graph permissions to your Microsoft app registration, authorize Teams, then sync:
msgvault add-teams user@example.com
msgvault sync-teams user@example.comTeams messages become searchable with --message-type teams. See
Microsoft Teams for required Graph permissions, scheduling,
and inline media backfill.
To archive Discord guild channels and threads, create a dedicated bot with Message Content Intent, View Channels, and Read Message History, then register and sync a guild:
msgvault add-discord --guild 123456789012345678
msgvault sync-discord 123456789012345678Discord messages become searchable with --message-type discord. The bot API
is guild-only and does not expose personal direct-message history. See
Discord for least-privilege setup, scheduling, filters,
repair behavior, and attachment limits.
Create a backup repository before relying on the archive as your source of truth:
msgvault backup init --repo ~/Backups/msgvault
msgvault backup create --repo ~/Backups/msgvault
msgvault backup verify --repo ~/Backups/msgvaultRecord the repository in config.toml so future commands can omit --repo:
[backup]
repo = "~/Backups/msgvault"See Backup for restore, verification, scheduling, and secret-handling details.