Delta Chat WebPreview Bot

Delta Chat bot designed to save web pages as complete, single self-contained HTML files (including images, CSS, fonts, and assets) using monolith and send them back to the chat as attachments.

Features

• 🖼️ Animated WebP Support: Automatically preserves multi-frame animation when downloading, converting, and resizing GIF images or animated WebP files, converting them into featherweight Animated WebP files (maintaining frame rate, loops, and transparency).
• 📄 Compressed Reader Mode (/preview <url>): Compile webpages into highly compressed, clutter-free reader views using Mozilla's Readability. All images are automatically downloaded, optimized, resized, and inlined as Base64 (with responsive/lazy-loading attributes like srcset and data-src stripped to guarantee offline/local rendering).
• ⚡ AI Summarization (/tldr [url], Voice Messages & Previews): Generate 1-2 paragraph AI summaries (or key bullet points) of articles, text messages, or voice/audio recordings using Google Gemini API (GEMINI_API_KEY). Previews generated with /preview or /webxdc automatically include a concise 1-paragraph TL;DR above the link. Replying /tldr to any message (links, text, or voice notes) or entering /tldr <text> generates a summary directly. Summaries are automatically cached in SQLite for 24 hours per URL/text/audio hash, language, and mode to conserve API quota and provide instant (<10ms) responses.
• 🎙️ Voice Message & Audio Intelligence (/ai & /tldr): Transcribe, summarize, or query voice notes and audio files (OGG/Opus, MP3, AAC, M4A, WAV, FLAC, WebM). Reply /ai to any voice message without a prompt to get a transcription and overview, or ask specific questions (e.g. /ai what did they decide about the deadline?). Reply /tldr to a voice note to receive a concise summary of key points and action items. Supports audio attachments up to 20 MB.
• 🌐 Per-Chat Summary Language (/lang <code>): Set target summary language for each chat (e.g. /lang AUTO (default, matches article language), /lang RU, /lang EN, /lang DE).
• 📱 WebXDC App Packaging (/webxdc <url>): Generate a standalone WebXDC application (.xdc ZIP container with index.html, manifest.toml, and icon.png) from any web page and send it into the chat for interactive offline viewing.
• ⚡ Full Page Archiving (/archive <url>): Save complete pages as full interactive archives with JavaScript enabled using monolith. Proactively compresses and optimizes heavy base64-encoded image payloads post-generation to keep files tiny.
• 💬 Quote Reply Parsing: Reply with /preview or /archive (without a URL) to any message containing links, and the bot will automatically extract and capture the first link in the quoted text.
• ⏱️ Rate Limiting: Protects against abuse by rate-limiting regular users (15-second debounce) while allowing admins unlimited generations.
• 🔄 Multiple Mail Relays: Supports multiple mail servers. Relay selection and failover are handled by the Delta Chat core (2.61+), which sends via the newest relay first and falls back to the next one if a relay is unreachable.
• 🛡️ Secure Administration: Claim ownership with /initadmin. Admins bypass rate limits and have exclusive control over relays and statistics.
• 🧹 Automatic Cache Rotation & Efficiency Tracking: Keeps disk usage low by automatically purging compiled HTML cache previews, banners, and AI summaries older than 24 hours. Tracks granular cache hits and misses in SQLite across OG cards, reader files, and AI summaries, reporting real-time cache efficiency and hit ratios via /stats.
• 💾 Direct File Downloads: Automatically detects URLs pointing to document files (PDF, EPUB, DjVu, MS Office, LibreOffice, plain text/data files). Instead of attempting an HTML preview, the bot offers a /download command in groups or directly downloads/attaches the file in private chats (up to 50 MB limits, with chunked streaming).
• 🤖 Jina.ai Fallback Support: Integrated Jina AI Reader (r.jina.ai) to resolve webpage titles, preview banners, and markdown-to-HTML text contents if the target site blocks standard user agents or readability parser fails to extract meaningful data (e.g. on anti-bot challenge pages), displaying a 🤖🌐 prefix. It automatically filters boilerplate navigation, ads, and footers, and strips tracking pixels/broken images to preserve privacy and presentation.
• 📺 Invidious & YT Bot Redirection: Detects Invidious instances (alternative YouTube front-ends) by checking page description metadata. If YT Bot is present in the chat, the bot extracts the video ID and redirects it to a standard youtu.be link to be processed by YT Bot, completely bypassing WebPreview generation. It automatically learns detected instance domains and supports manual domain registration/management via admin commands.
• 🏛️ Web Archive / KaraKeep / Archive.today / Ghostarchive Integration (/keep <url>): Asynchronously save webpages across archiving services. When executed by the bot administrator with a configured KaraKeep instance, the URL is saved to KaraKeep first (with confirmation sent privately). The bot then attempts to archive the URL to the Web Archive (Wayback Machine) using a 120s timeout and official Save Page Now 2 (SPN2) authenticated API support. Only if the Web Archive fails or is unavailable does the bot fall back to Archive.today (with active mirror failover and optional proxy routing via ARCHIVE_TODAY_PROXY_URL) and Ghostarchive (ghostarchive.org).
• 📱 Telegram Post Previews (t.me): Detects Telegram post links and scrapes them directly from the static public preview feed (https://t.me/s/{channel}/{post_id}), bypassing any JavaScript requirements. Extracts the post author, text content with preserved line breaks, and media/thumbnails. Caches the extracted markdown in the local SQLite database so /preview and /archive commands run entirely offline/instantaneously. When TG Bridge (deltachat_telegram_bridge) is present in the chat, Web Preview automatically yields Telegram post links to TG Bridge to avoid duplicate previews and ensure rich posts, media albums, and videos are delivered via TG Bridge's native MTProto / WebXDC integration.
• 📸 Instagram & OGInstagram Embeds: Detects Instagram links (instagram.com, instagr.am, oginstagram.com for posts, reels, stories, profiles) and seamlessly resolves them via the OGInstagram embed proxy. Bypasses Instagram login walls and CDN hotlink blocks, extracting author names, full captions, accessibility/alt descriptions, and direct unblocked images. Preview cards cleanly display the post caption right beneath the image above the source author link, without cluttering action buttons.
• 🔇 Chat-Specific Toggle (/webpreview [on|off]): Turn off automatic link previews in specific group or private chats. When disabled, the bot reacts only when explicit commands like /preview, /archive, or /keep are run manually (by typing the command or replying to a link).
• 🐳 Docker Ready: Built with a multi-stage Docker build compiling Rust-based monolith and packing it into a slim Python runtime.

Setup

1. Clone the repository:

   git clone https://github.com/mrgluek/deltachat_webpreview
   cd deltachat_webpreview
   

2. Initialize Account: Run the initialization command once to set up the bot's email and password:

   docker compose run --rm webpreview_bot python bot.py init bot-email@example.com your_password
   

3. Start the Bot:

   docker compose up -d
   docker compose logs -f
   

   Note: If it's a new account, a QR code will be printed to the logs for linking your Delta Chat device.

4. Claim Admin Ownership: Send /initadmin to the bot in a private message to become the administrator.

Configuration

The bot can be configured using environment variables in docker-compose.yml or a .env file (you can copy .env.example to .env as a template):

Variable	Description	Default
DISPLAY_NAME	Custom display name for the Delta Chat bot profile.	WebPreview Bot
STATUS_TEXT	Custom bio/status description for the Delta Chat bot profile.	I generate single-file HTML web previews in chats and groups...
ALLOWED_BOT_EMAILS	Comma-separated list of allowed bot emails.	(Empty)
OGINSTAGRAM_HOST	Hostname of the OGInstagram proxy instance for Instagram post/reel/profile embeds.	oginstagram.com
JINA_API_KEY	Optional API Key for Jina Reader (r.jina.ai) to raise rate limits (from 20 req/min to 500+).	(Empty)
JINA_PROXY_URL	Optional dedicated proxy server URL for routing Jina Reader requests (if unset, Jina queries run directly).	(Empty)
GEMINI_API_KEY	Optional Google Gemini API Key for enabling article TL;DR summarization (/tldr), AI question answering (/ai), and automatic short summaries in previews.	(Empty)
GEMINI_MODELS	Optional comma-separated list of Gemini/Gemma models for automatic fallback on rate limit (HTTP 429), overload (5xx), timeouts or retired models (HTTP 404, 24h cooldown).	gemini-3.8-flash,gemini-3.7-flash,gemini-3.6-flash,gemini-3.5-flash,gemini-3.5-flash-lite,gemini-3.1-flash-lite,gemma-4-31b-it,gemma-4-26b-a4b-it
GEMINI_TIME_BUDGET	Total seconds the whole Gemini chain may spend before handing over to OpenRouter. The chain also stops after 2 timeouts in a row.	30
OPENROUTER_API_KEY	Optional OpenRouter API key. Used as the last-resort fallback when every Gemini model fails (or as the only AI backend if GEMINI_API_KEY is empty). Text and images only — voice/audio summaries still require Gemini.	(Empty)
OPENROUTER_MODELS	Optional comma-separated OpenRouter models tried in order (each empty answer is retried once, the whole chain is capped at 60s). Answers from moderation classifiers (*safety*, *guard*) are discarded. Every AI reply names the model that produced it, e.g. 🤖 **AI** *(gemini-3.8-flash)*:.	nvidia/nemotron-3-ultra-550b-a55b:free,nvidia/nemotron-3-super-120b-a12b:free,qwen/qwen3.8-27b:free,google/gemma-4-31b-it:free,dots-studio/dots-3-note-preview:free,openrouter/free
PROXY_URL	Optional proxy server URL to route matching domains through (e.g. http://127.0.0.1:8118 or socks5://127.0.0.1:9050). RU_PROXY is also recognized as an alias. This proxy is also passed to Jina Reader via X-Proxy-Url to route Jina's crawler for matching domains.	(Empty)
RU_PROXY	Optional alias for PROXY_URL (shares configuration with YT Bot).	(Empty)
PROXY_DOMAINS	Comma-separated list of domain suffixes to route via PROXY_URL. Automatically supports Cyrillic/internationalized domains and Punycode (e.g. .ru, .su, .рф).	.ru
INSTAGRAM_PROXY_URL	Optional dedicated proxy URL (e.g. home residential router) for routing Instagram/OGInstagram requests and preview image downloads to bypass Cloudflare/datacenter IP limits.	(Empty)
KARAKEEP_URL	The base URL of your KaraKeep instance (e.g., https://keep.gluek.info).	(Empty)
KARAKEEP_API_KEY	The API Key for authenticating with your KaraKeep instance.	(Empty)
KARAKEEP_TAGS	Optional comma-separated list of tags to automatically attach to saved bookmarks (e.g., deltachat).	(Empty)
ARCHIVE_TODAY_MIRRORS	Comma-separated list of Archive.today mirror URLs to try sequentially when archiving.	https://archive.ph,https://archive.is,https://archive.today,https://archive.li,https://archive.vn,https://archive.md
ARCHIVE_TODAY_PROXY_URL	Optional dedicated proxy URL (e.g. socks5://127.0.0.1:9050 or http://proxy:8080) to route Archive.today & Ghostarchive requests through to bypass Cloudflare/datacenter 429/403 blocks (falls back to PROXY_URL if set).	(Empty)
WAYBACK_ACCESS_KEY	Optional Internet Archive S3 Access Key for authenticated SPN2 API saves (bypasses anonymous 403 blocks & raises rate limits).	(Empty)
WAYBACK_SECRET_KEY	Optional Internet Archive S3 Secret Key for authenticated SPN2 API saves.	(Empty)
DELETE_DEVICE_AFTER	Local message retention duration (in seconds) in the bot database.	3600 (1 hour)
DOWNLOAD_LIMIT	Automatic download limit for incoming message attachments (in bytes).	1 (1 byte)

Tip

Internet Archive SPN2 API (WAYBACK_ACCESS_KEY & WAYBACK_SECRET_KEY): Unauthenticated Save Page Now requests can occasionally hit 403 Forbidden or strict rate limits. Adding free S3 keys from archive.org/account/s3.php enables authenticated Save Page Now 2 (SPN2) API requests (POST https://web.archive.org/save/ with Authorization: LOW <access_key>:<secret_key>), polling task statuses until snapshot completion with prioritized processing.

Commands

• /preview <url> — Save page in highly compressed reader-mode format (using Mozilla's Readability).
• /webxdc <url> — Save page as a WebXDC app 📱 (.xdc ZIP containing index.html, manifest.toml, icon.png).
• /archive <url> — Save page as a full monolith-based dynamic archive (with JS enabled, optimized images). (Note: /previewjs is also supported as an alias to /archive)
• /tldr [url] — Generate AI summary (TL;DR) ⚡ for an article, text, or voice/audio message (supports replies and dynamic /tldr_[hash] links). Commands are case-insensitive (e.g. /tldr or /TLDR).
• /ai [text] — Ask AI a question, analyze a topic, explain an article, inspect/describe photos, or transcribe and query voice notes and audio recordings 🤖🖼️🎙️. Commands are case-insensitive (e.g. /ai or /AI).
• /lang [code] — Set preferred summary language for current chat (e.g. /lang AUTO (default), /lang RU, /lang EN, /lang DE).
• /download <url> — Download file directly and send as attachment (supported for PDF, office documents, text files).
• /keep <url> — Asynchronously save webpage to KaraKeep (admin private notification), Web Archive (120s timeout), Archive.today (multi-mirror with optional proxy), and Ghostarchive. Also supports dynamic /keep_[hash] links.
• /stats — Show generation counters, total traffic size, and disk space (disk space is admin-only).
• /source — Show primary and backup source code links 🔌.
• /donate — Support project development ❤️.
• /help — Show available commands and greeting info.
• /initadmin — Claim administrative ownership (private chat only).
• /transports — Show configured mail relays & stats (Admin only).
• /addtransport — Add a backup mail relay (Admin only, private 1:1 chat only).
• /rmtransport <addr> — Remove a mail relay (Admin only).
• /invidious_add <domain/url> — Register a custom Invidious instance domain (Admin only).
• /invidious_rm <domain/url> — Deregister an Invidious instance domain (Admin only). (Note: /invidious_remove is also supported as an alias)
• /invidious_list — List registered Invidious instance domains (Admin only).
• /jina <api_key> — Check Jina AI API key remaining token balance (Admin only; defaults to env-configured key if <api_key> is omitted).
• /keep <url> — Save URL to KaraKeep (admin), Web Archive, and Archive.today fallback (Admin command and preview button; also supports quote replies).
• /webpreview [on|off] — Enable or disable automatic link previews in the current chat. Defaults to enabled; accepts on/off, 1/0, or true/false.

Relay selection and failover are handled by the Delta Chat core (2.61+): it sends via the newest relay first and falls back to the next one if a relay is unreachable. /transports lists relays in that order. The former /setprimary and /resilient commands are deprecated and only reply with this explanation.

Target-Specific Commands in Group Chats

In group chats where multiple bots are present, you can address this bot specifically to prevent other bots from responding. Append the @web or @wp suffix to any command, for example:

• /help@web or /help@wp
• /stats@web or /stats@wp

A plain /help sent in a group chat is answered in a private 1:1 chat with the sender, so several bots don't flood the group with help texts. Use /help@web to show the help in the group itself.

Admin Management

Admin functions can be performed directly through chat commands, or managed via the server CLI:

Set Administrator

docker compose exec webpreview_bot python set_admin.py

Transport (Mail Relay) CLI Initialization

Although we recommend using /addtransport in chat, you can also add a backup relay via the command line:

1. Stop the bot: docker compose stop webpreview_bot
2. Add relay: docker compose run --rm webpreview_bot python bot.py init transport backup-email@example.com password
3. Start the bot: docker compose up -d

Development & Testing

The repository ships with a comprehensive tests/ directory with unittest suites:

File	What it covers
tests/test_database.py	Config roundtrip, admin fingerprinting, buffered transport statistics, and 30-day record retention pruning
tests/test_transport_commands.py	Transport commands, private chat enforcement for /addtransport and /initadmin, deprecated /resilient and /setprimary, and error sanitization
tests/test_url_validation.py	_is_internal_or_invalid_url – valid domains, private IPs, DNS resolution/rebinding SSRF protection, and SafeRedirectHandler
tests/test_instagram_parser.py	_is_instagram_url, _fetch_instagram_og_data – OGInstagram parsing, alt text, direct media fallback
tests/test_telegram_parser.py	_is_telegram_url, _fetch_telegram_og_data – static preview parsing, truncation, newlines
tests/test_invidious.py	_extract_youtube_id_from_invidious, _clean_domain, Invidious database helpers
tests/test_proxy_and_jina.py	Proxy routing, Jina headers, _is_valid_image_url (skipping blob:, data:, localhost, internal IPs, SVGs), octet-stream logic, cache saving, OG fallback
tests/test_webpreview.py	/webpreview command, DB status checks, and on_new_message auto-preview toggling
tests/test_image_compression.py	Animated GIF / WebP frame iteration, animation preservation, and image resizing

To run locally (inside the virtualenv):

python -m unittest discover -s tests -p "test_*.py" -v

CI runs automatically on every push and pull-request via GitHub Actions (.github/workflows/tests.yml).

Support & Development

If you find this bot useful, consider supporting its development:

• Git Repository: mrgluek/deltachat_webpreview
• Forgejo Mirror: gluek/deltachat_webpreview
• Donations: Use the /donate command in Delta Chat.