monolith and send them back to the chat as attachments.
- Python 99.4%
- Shell 0.5%
- Dockerfile 0.1%
| .github/workflows | ||
| .pi | ||
| docs | ||
| tests | ||
| .dockerignore | ||
| .DS_Store | ||
| .env.example | ||
| .gitignore | ||
| bot.py | ||
| CHANGELOG.md | ||
| database.py | ||
| dc_fingerprint.py | ||
| docker-compose.yml | ||
| Dockerfile | ||
| icon.png | ||
| LICENSE | ||
| README.md | ||
| requirements.txt | ||
| set_admin.py | ||
| transports_view.py | ||
| update.sh | ||
Delta Chat WebPreview Bot
Delta Chat bot designed to save web pages as complete, single self-contained HTML files (including images, CSS, fonts, and assets) using monolith and send them back to the chat as attachments.
Features
- 🖼️ Animated WebP Support: Automatically preserves multi-frame animation when downloading, converting, and resizing GIF images or animated WebP files, converting them into featherweight Animated WebP files (maintaining frame rate, loops, and transparency).
- 📄 Compressed Reader Mode (
/preview <url>): Compile webpages into highly compressed, clutter-free reader views using Mozilla's Readability. All images are automatically downloaded, optimized, resized, and inlined as Base64 (with responsive/lazy-loading attributes like srcset and data-src stripped to guarantee offline/local rendering). - ⚡ AI Summarization (
/tldr [url], Voice Messages & Previews): Generate 1-2 paragraph AI summaries (or key bullet points) of articles, text messages, or voice/audio recordings using Google Gemini API (GEMINI_API_KEY). Previews generated with/previewor/webxdcautomatically include a concise 1-paragraph TL;DR above the link. Replying/tldrto any message (links, text, or voice notes) or entering/tldr <text>generates a summary directly. Summaries are automatically cached in SQLite for 24 hours per URL/text/audio hash, language, and mode to conserve API quota and provide instant (<10ms) responses. - 🎙️ Voice Message & Audio Intelligence (
/ai&/tldr): Transcribe, summarize, or query voice notes and audio files (OGG/Opus, MP3, AAC, M4A, WAV, FLAC, WebM). Reply/aito any voice message without a prompt to get a transcription and overview, or ask specific questions (e.g./ai what did they decide about the deadline?). Reply/tldrto a voice note to receive a concise summary of key points and action items. Supports audio attachments up to 20 MB. - 📑 PDF & Video Analysis (
/ai): Send or reply/aito a PDF document or a video (MP4, MOV, WebM, MPEG, AVI, 3GP, FLV, WMV) to get a summary, or ask a question about it (e.g./ai what is the total on page 3?). The file goes to Gemini directly, up to 14 MB; PDFs and videos get a longer time budget. Photos and audio keep their 20 MB limit. - 🌐 Per-Chat Summary Language (
/lang <code>): Set target summary language for each chat (e.g./lang AUTO(default, matches article language),/lang RU,/lang EN,/lang DE). - 📱 WebXDC App Packaging (
/webxdc <url>): Generate a standalone WebXDC application (.xdcZIP container withindex.html,manifest.toml, andicon.png) from any web page and send it into the chat for interactive offline viewing. - ⚡ Full Page Archiving (
/archive <url>): Save complete pages as full interactive archives with JavaScript enabled usingmonolith. Proactively compresses and optimizes heavy base64-encoded image payloads post-generation to keep files tiny. - 💬 Quote Reply Parsing: Reply with
/previewor/archive(without a URL) to any message containing links, and the bot will automatically extract and capture the first link in the quoted text. - ⏱️ Rate Limiting: Protects against abuse by rate-limiting regular users (15-second debounce) while allowing admins unlimited generations.
- 🔄 Multiple Mail Relays: Supports multiple mail servers. Relay selection and failover are handled by the Delta Chat core (2.61+), which sends via the newest relay first and falls back to the next one if a relay is unreachable.
- 🛡️ Secure Administration: Claim ownership with
/initadmin. Admins bypass rate limits and have exclusive control over relays and statistics. - 🧹 Automatic Cache Rotation & Efficiency Tracking: Keeps disk usage low by automatically purging compiled HTML cache previews, banners, and AI summaries older than 24 hours. Tracks granular cache hits and misses in SQLite across OG cards, reader files, and AI summaries, reporting real-time cache efficiency and hit ratios via
/stats. - 💾 Direct File Downloads: Automatically detects URLs pointing to document files (PDF, EPUB, DjVu, MS Office, LibreOffice, plain text/data files). Instead of attempting an HTML preview, the bot offers a
/downloadcommand in groups or directly downloads/attaches the file in private chats (up to 50 MB limits, with chunked streaming). - 🤖 Jina.ai Fallback Support: Integrated Jina AI Reader (
r.jina.ai) to resolve webpage titles, preview banners, and markdown-to-HTML text contents if the target site blocks standard user agents or readability parser fails to extract meaningful data (e.g. on anti-bot challenge pages), displaying a🤖🌐prefix. When Reader is requested, the bot checks the configured API key balance in a background thread, so the reply is not delayed, at most once an hour and privately warns the administrator in Delta Chat below 1,000,000 tokens. HTTP 402 (Payment Required) triggers a separate warning. Each warning is limited to once every 24 hours, with cooldowns persisted across restarts; there is no background polling. It automatically filters boilerplate navigation, ads, and footers, and strips tracking pixels/broken images to preserve privacy and presentation. - 📺 Invidious & YT Bot Redirection: Detects Invidious instances (alternative YouTube front-ends) by checking page description metadata. If
YT Botis present in the chat, the bot extracts the video ID and redirects it to a standardyoutu.belink to be processed byYT Bot, completely bypassing WebPreview generation. It automatically learns detected instance domains and supports manual domain registration/management via admin commands. - 🏛️ Web Archive / KaraKeep / Archive.today / Ghostarchive Integration (
/keep <url>): Asynchronously save webpages across archiving services. When executed by the bot administrator with a configured KaraKeep instance, the URL is saved to KaraKeep first (with confirmation sent privately). The bot then attempts to archive the URL to the Web Archive (Wayback Machine) using a 120s timeout and official Save Page Now 2 (SPN2) authenticated API support. Only if the Web Archive fails or is unavailable does the bot fall back to Archive.today (with active mirror failover and optional proxy routing viaARCHIVE_TODAY_PROXY_URL) and Ghostarchive (ghostarchive.org). - 📱 Telegram Post Previews (
t.me): Detects Telegram post links and scrapes them directly from the static public preview feed (https://t.me/s/{channel}/{post_id}), bypassing any JavaScript requirements. Extracts the post author, text content with preserved line breaks, and media/thumbnails. Caches the extracted markdown in the local SQLite database so/previewand/archivecommands run entirely offline/instantaneously. When TG Bridge (deltachat_telegram_bridge) is present in the chat, Web Preview automatically yields Telegram post links to TG Bridge to avoid duplicate previews and ensure rich posts, media albums, and videos are delivered via TG Bridge's native MTProto / WebXDC integration. - 🗺️ Google Maps Links: Previews of Google Maps links (including
maps.app.goo.glshort links) show a real map of the place with a marker, drawn from OpenStreetMap tiles; names and addresses are looked up via Nominatim when the link has no coordinates. - 📸 Instagram & OGInstagram Embeds: Detects Instagram links (
instagram.com,instagr.am,oginstagram.comfor posts, reels, stories, profiles) and seamlessly resolves them via the OGInstagram embed proxy. Bypasses Instagram login walls and CDN hotlink blocks, extracting author names, full captions, accessibility/alt descriptions, and direct unblocked images. Preview cards cleanly display the post caption right beneath the image above the source author link, without cluttering action buttons. - 🔇 Chat-Specific Toggle (
/webpreview [on|off]): Turn off automatic link previews in specific group or private chats. When disabled, the bot reacts only when explicit commands like/preview,/archive, or/keepare run manually (by typing the command or replying to a link). - 🐳 Docker Ready: Built with a multi-stage Docker build compiling Rust-based
monolithand packing it into a slim Python runtime.
Setup
-
Clone the repository:
git clone https://github.com/mrgluek/deltachat_webpreview cd deltachat_webpreview -
Configure Email and Admin (Optional): Skip this step and the bot creates its own account at first start: the Delta Chat core (2.62+) picks the fastest chatmail relay from its built-in list and later adds further relays in the background.
To choose the account yourself, initialize it with an email and password, or with
DCACCOUNT:server.tldfor an account on a specific chatmail relay:docker compose run --rm webpreview_bot python bot.py init bot-email@example.com your_password docker compose run --rm webpreview_bot python bot.py init DCACCOUNT:server.tldThe admin can be set on the server (
docker compose run --rm webpreview_bot python set_admin.py, see Admin Management) or later inside the bot chat with the/initadmincommand. -
Start the Bot:
docker compose up -d docker compose logs -fNote: If it's a new account, a QR code will be printed to the logs for linking your Delta Chat device.
Updates: run
./update.sh(e.g. from cron) to deploy the newest release: the highestvX.Y.Ztag, from GitHub or, when GitHub is unreachable, from the Forgejo mirror;-frebuilds the container without changes. Commits onmainthat are not released yet are not deployed. In.env.local,UPDATE_REF=v2.17.1stays on that release (to roll back; remove it to follow new releases again),UPDATE_REF=mainfollows the branch, andMONITOR_URLpings a healthcheck on start, success and failure. -
Claim Admin Ownership: Send
/initadminto the bot in a private message to become the administrator.
Configuration
The bot can be configured using environment variables in docker-compose.yml or a .env file (you can copy .env.example to .env as a template):
| Variable | Description | Default |
|---|---|---|
DISPLAY_NAME |
Custom display name for the Delta Chat bot profile. | WebPreview Bot |
STATUS_TEXT |
Custom bio/status description for the Delta Chat bot profile. | I generate single-file HTML web previews in chats and groups... |
ALLOWED_BOT_EMAILS |
Comma-separated list of allowed bot emails. | (Empty) |
OGINSTAGRAM_HOST |
Hostname of the OGInstagram proxy instance for Instagram post/reel/profile embeds. | oginstagram.com |
JINA_API_KEY |
Optional API Key for Jina Reader (r.jina.ai) to raise rate limits (from 20 req/min to 500+). |
(Empty) |
JINA_PROXY_URL |
Optional dedicated proxy server URL for routing Jina Reader requests (if unset, Jina queries run directly). | (Empty) |
GEMINI_API_KEY |
Optional Google Gemini API Key for enabling article TL;DR summarization (/tldr), AI question answering (/ai), and automatic short summaries in previews. |
(Empty) |
GEMINI_SEARCH_API_KEY |
Optional paid Gemini key that enables /google (answers grounded in live Google Search). Search is not available on the free tier, so this key is used for /google only; everything else stays on GEMINI_API_KEY. If it is rejected (HTTP 401/402/403, e.g. prepaid credits depleted) /google pauses for an hour. |
(Empty) |
GEMINI_SEARCH_MODELS |
Optional comma-separated models /google tries in order. Empty uses GEMINI_MODELS without Gemma (Gemma cannot use Google Search). |
(Empty) |
GEMINI_SEARCH_THINKING |
Reasoning depth for /google: minimal, low, medium, high, or empty for the model default. Reasoning tokens are billed as output, so lower is cheaper and faster. |
low |
GOOGLE_CACHE_MINUTES |
Minutes an identical /google question (same text and chat language) is answered from memory instead of a new paid request. 0 turns it off. |
60 |
GOOGLE_DAILY_LIMIT |
Most /google searches per rolling 24 hours for all users together. 0 means no limit. |
100 |
GOOGLE_USER_DAILY_LIMIT |
Most /google searches per user per rolling 24 hours; the bot admin is exempt. 0 means no limit. |
10 |
GEMINI_MODELS |
Optional comma-separated list of Gemini/Gemma models for automatic fallback on rate limit (HTTP 429), overload (5xx), timeouts or retired models (HTTP 404, 24h cooldown). | gemini-4-hydrogen,gemini-3.8-flash,gemini-3.7-flash,gemini-3.6-flash,gemini-3.5-flash,gemini-3.5-flash-lite,gemini-3.1-flash-lite,gemma-4-31b-it,gemma-4-26b-a4b-it |
GEMINI_TOOLS |
Optional comma-separated built-in Gemini tools for /ai only: code_execution (runs Python for exact maths), url_context (the model reads links itself), google_search (live web search, needs a paid Gemini key). Off by default. If a model or key rejects the tools, the same model is retried without them. Gemma models never get tools; answers are not cached for 24h while google_search is on. |
(Empty) |
AI_CONTEXT_CHARS |
Characters of article or quoted text /ai sends to the model as context (1000-200000). A larger value lets /ai read long articles whole, at the cost of slower answers and more tokens. |
16000 |
AI_MAX_OUTPUT_TOKENS |
Output cap for /ai answers in tokens (512-16384). Thinking models spend part of it on reasoning; answer length is set by AI_ANSWER_STYLE. |
4096 |
AI_ANSWER_STYLE |
Optional wording that tells /ai and /google how long and deep to answer; it goes into the prompt as is (max 1500 characters). Empty uses the built-in default: a direct answer without filler for simple questions, a reasoned answer for complex ones. |
(built-in) |
GEMINI_TIME_BUDGET |
Total seconds the whole Gemini chain may spend before handing over to OpenRouter. The chain also stops after 2 timeouts in a row. | 30 |
OPENROUTER_API_KEY |
Optional OpenRouter API key. Used as the last-resort fallback when every Gemini model fails (or as the only AI backend if GEMINI_API_KEY is empty). Text and images only — voice/audio summaries still require Gemini. |
(Empty) |
OPENROUTER_MODELS |
Optional comma-separated OpenRouter models tried in order (each empty answer is retried once, the whole chain is capped at 60s). Answers from moderation classifiers (*safety*, *guard*) are discarded. Every AI reply names the model that produced it, e.g. 🤖 **AI** *(gemini-3.8-flash)*:. |
nvidia/nemotron-3-ultra-550b-a55b:free,nvidia/nemotron-3-super-120b-a12b:free,qwen/qwen3.8-27b:free,google/gemma-4-31b-it:free,dots-studio/dots-3-note-preview:free,openrouter/free |
PROXY_URL |
Optional proxy server URL to route matching domains through (e.g. http://127.0.0.1:8118 or socks5://127.0.0.1:9050). RU_PROXY is also recognized as an alias. This proxy is also passed to Jina Reader via X-Proxy-Url to route Jina's crawler for matching domains. |
(Empty) |
RU_PROXY |
Optional alias for PROXY_URL (shares configuration with YT Bot). |
(Empty) |
PROXY_DOMAINS |
Comma-separated list of domain suffixes to route via PROXY_URL. Automatically supports Cyrillic/internationalized domains and Punycode (e.g. .ru, .su, .рф). |
.ru |
INSTAGRAM_PROXY_URL |
Optional dedicated proxy URL (e.g. home residential router) for routing Instagram/OGInstagram requests and preview image downloads to bypass Cloudflare/datacenter IP limits. | (Empty) |
KARAKEEP_URL |
The base URL of your KaraKeep instance (e.g., https://keep.gluek.info). |
(Empty) |
KARAKEEP_API_KEY |
The API Key for authenticating with your KaraKeep instance. | (Empty) |
KARAKEEP_TAGS |
Optional comma-separated list of tags to automatically attach to saved bookmarks (e.g., deltachat). |
(Empty) |
ARCHIVE_TODAY_MIRRORS |
Comma-separated list of Archive.today mirror URLs to try sequentially when archiving. | https://archive.ph,https://archive.is,https://archive.today,https://archive.li,https://archive.vn,https://archive.md |
ARCHIVE_TODAY_PROXY_URL |
Optional dedicated proxy URL (e.g. socks5://127.0.0.1:9050 or http://proxy:8080) to route Archive.today & Ghostarchive requests through to bypass Cloudflare/datacenter 429/403 blocks (falls back to PROXY_URL if set). |
(Empty) |
WAYBACK_ACCESS_KEY |
Optional Internet Archive S3 Access Key for authenticated SPN2 API saves (bypasses anonymous 403 blocks & raises rate limits). | (Empty) |
WAYBACK_SECRET_KEY |
Optional Internet Archive S3 Secret Key for authenticated SPN2 API saves. | (Empty) |
DELETE_DEVICE_AFTER |
Local message retention duration (in seconds) in the bot database. | 3600 (1 hour) |
DOWNLOAD_LIMIT |
Automatic download limit for incoming message attachments (in bytes). At 1 the core also defers every plain text message, which delays replies. |
65536 (64 KB) |
Tip
Internet Archive SPN2 API (
WAYBACK_ACCESS_KEY&WAYBACK_SECRET_KEY): Unauthenticated Save Page Now requests can occasionally hit403 Forbiddenor strict rate limits. Adding free S3 keys from archive.org/account/s3.php enables authenticated Save Page Now 2 (SPN2) API requests (POST https://web.archive.org/save/withAuthorization: LOW <access_key>:<secret_key>), polling task statuses until snapshot completion with prioritized processing.
Commands
/preview <url>— Save page in highly compressed reader-mode format (using Mozilla's Readability)./webxdc <url>— Save page as a WebXDC app 📱 (.xdcZIP containingindex.html,manifest.toml,icon.png)./archive <url>— Save page as a full monolith-based dynamic archive (with JS enabled, optimized images). (Note:/previewjsis also supported as an alias to/archive)/tldr [url]— Generate AI summary (TL;DR) ⚡ for an article, text, or voice/audio message (supports replies and dynamic/tldr_[hash]links). Commands are case-insensitive (e.g./tldror/TLDR)./ai [text]— Ask AI a question, analyze a topic, explain an article, inspect/describe photos, or transcribe and query voice notes and audio recordings 🤖🖼️🎙️. Commands are case-insensitive (e.g./aior/AI). Also works on PDF documents and videos 📑🎬 (up to 14 MB)./google <question>— Answer from live Google Search results, with up to 3 source links 🔎. Reply/googleto any message to look it up. NeedsGEMINI_SEARCH_API_KEY; answers are never cached. The admin sees a cost estimate for/googlein/stats(searches, tokens, search queries and an approximate dollar amount from the logged token counts)./lang [code]— Set preferred summary language for current chat (e.g./lang AUTO(default),/lang RU,/lang EN,/lang DE)./download <url>— Download file directly and send as attachment (supported for PDF, office documents, text files)./keep <url>— Asynchronously save webpage to KaraKeep (admin private notification), Web Archive (120s timeout), Archive.today (multi-mirror with optional proxy), and Ghostarchive. Also supports dynamic/keep_[hash]links.- Emoji reactions — react to a message with a link: ⚡ TL;DR, 🖥️ preview, 📦 WebXDC, 🏛️ Web Archive (bot answers with a “🏛️ Saved to Web Archive!” reaction), 💾 KaraKeep (admin only, bot answers with a “💾 Saved to KaraKeep!” reaction). 3 or more 👎 on an answer sends the administrator its domain for
/preview_exclude. React 🗑️ to a bot answer to delete it (the requester or the admin). /stats— Show generation counters, total traffic size, and disk space (disk space is admin-only)./source— Show primary and backup source code links 🔌./donate— Support project development ❤️./help— Show available commands and greeting info./initadmin— Claim administrative ownership (private chat only)./transports— Show mail relays: relay used for sending, send queue, incoming status and storage (Admin only)./addtransport— Add a backup mail relay (Admin only, private 1:1 chat only)./rmtransport <addr>— Remove a mail relay (Admin only)./invidious_add <domain/url>— Register a custom Invidious instance domain (Admin only)./invidious_rm <domain/url>— Deregister an Invidious instance domain (Admin only). (Note:/invidious_removeis also supported as an alias)/invidious_list— List registered Invidious instance domains (Admin only)./jina <api_key>— Check Jina AI API key remaining token balance (Admin only; defaults to env-configured key if<api_key>is omitted)./keep <url>— Save URL to KaraKeep (admin), Web Archive, and Archive.today fallback (Admin command and preview button; also supports quote replies)./webpreview [on|off]— Enable or disable automatic link previews in the current chat. Defaults to enabled; acceptson/off,1/0, ortrue/false.
Relay selection and failover are handled by the Delta Chat core (2.61+): it keeps sending through the relay that last worked and moves to the next one only if it cannot connect, so the relay used for sending can differ from the account's main address. /transports shows which relay that is, the core's send queue, and per relay the incoming status, the last received message and mailbox storage. The former /setprimary and /resilient commands are deprecated and only reply with this explanation.
Target-Specific Commands in Group Chats
In group chats where multiple bots are present, you can address this bot specifically to prevent other bots from responding. Append the @web or @wp suffix to any command, for example:
/help@webor/help@wp/stats@webor/stats@wp
A plain /help sent in a group chat is answered in a private 1:1 chat with the sender, so several bots don't flood the group with help texts. Use /help@web to show the help in the group itself.
Admin Management
Admin functions can be performed directly through chat commands, or managed via the server CLI:
Set Administrator
docker compose exec webpreview_bot python set_admin.py
Transport (Mail Relay) CLI Initialization
Although we recommend using /addtransport in chat, you can also add a backup relay via the command line:
- Stop the bot:
docker compose stop webpreview_bot - Add relay:
docker compose run --rm webpreview_bot python bot.py init transport backup-email@example.com password - Start the bot:
docker compose up -d
Development & Testing
The repository ships with a comprehensive tests/ directory with unittest suites:
| File | What it covers |
|---|---|
tests/test_database.py |
Config roundtrip, admin fingerprinting, buffered transport statistics, and 30-day record retention pruning |
tests/test_transport_commands.py |
Transport commands, private chat enforcement for /addtransport and /initadmin, deprecated /resilient and /setprimary, and error sanitization |
tests/test_url_validation.py |
_is_internal_or_invalid_url – valid domains, private IPs, DNS resolution/rebinding SSRF protection, and SafeRedirectHandler |
tests/test_google_maps.py |
_is_google_maps_url, _parse_google_maps_url, geocoder fallback – Google Maps link previews |
tests/test_instagram_parser.py |
_is_instagram_url, _fetch_instagram_og_data – OGInstagram parsing, alt text, direct media fallback |
tests/test_telegram_parser.py |
_is_telegram_url, _fetch_telegram_og_data – static preview parsing, truncation, newlines |
tests/test_invidious.py |
_extract_youtube_id_from_invidious, _clean_domain, Invidious database helpers |
tests/test_proxy_and_jina.py |
Proxy routing, Jina headers, _is_valid_image_url (skipping blob:, data:, localhost, internal IPs, SVGs), octet-stream logic, cache saving, OG fallback |
tests/test_webpreview.py |
/webpreview command, DB status checks, and on_new_message auto-preview toggling |
tests/test_image_compression.py |
Animated GIF / WebP frame iteration, animation preservation, and image resizing |
tests/test_auto_account.py |
Automatic account creation on serve without a configured account (mocked RPC) |
To run locally (inside the virtualenv):
python -m unittest discover -s tests -p "test_*.py" -v
CI runs automatically on every push and pull-request via GitHub Actions (.github/workflows/tests.yml).
Releasing
Merge into main, then tag the merge commit: git tag -a vX.Y.Z -m "..." <commit> && git push origin vX.Y.Z. Pushing the tag runs .github/workflows/release.yml: it checks that VERSION in bot.py matches the tag, publishes the GitHub release with that version's CHANGELOG.md section, and sends the ntfy notification and the Home Assistant add-on sync. The server's update.sh deploys only tagged releases.
Support & Development
If you find this bot useful, consider supporting its development:
- Git Repository: mrgluek/deltachat_webpreview
- Forgejo Mirror: gluek/deltachat_webpreview
- Donations: Use the
/donatecommand in Delta Chat.
License
MIT, see LICENSE.