English | 简体中文
DyFetch is a Python tool for downloading watermark-free videos, image/live-photo posts, user profiles, collections, and music-page videos from Douyin, with optional video frame extraction and a Tkinter desktop GUI.
- Media downloads: Single videos (
/video/<id>), image and live-photo posts (/note/<id>), user profiles (/user/<sec_user_id>: published posts, liked posts, collections), collection pages (/collection/<id>,/mix/detail/<id>), music pages (/music/<id>), andv.douyin.comshort links. - Extra assets & metadata: Saves background audio (
_music.mp3), cover images (_cover.jpg), and raw metadata (_data.json) alongside downloaded posts. - Automatic frame extraction: Slices downloaded videos into JPG frames under
frames/at a configurable FPS usingffmpeg(bundled viaimageio-ffmpegout of the box), plus a standalone OpenCV frame tool (cut.py). - Browser fallback: When Douyin web APIs return empty bodies (for example when cookies expire),
downloader.pycan fall back to Playwright to inspect page network responses and DOM links so public posts remain downloadable. - Rolling session renewal: Reads
sid_guardto report exact cookie expiry at startup and mergesSet-Cookieheaders back into local credential files on every request. - Messy input cleaning: Strips share captions, fixes full-width characters and HTML entities, deduplicates query parameters, and expands bare
aweme_idorsec_user_idvalues into full URLs.
Current limitations:
downloader.pydoes not record live streams (live.douyin.comURLs are skipped; the legacyDouYinCommand.pyonly parses live room info and prints FLV pull URLs). Full profile downloads require a real/user/<sec_user_id>URL rather than/user/self.
| Entry / Path | Role |
|---|---|
ui.py |
Tkinter desktop GUI (no local web server or port). Manages download queues, live logs, task history, and settings (ui_run_config.yml), driving downloader.py as a subprocess. |
downloader.py |
Main async CLI downloader (UnifiedDownloader v3.2). Handles X-Bogus API requests, Playwright fallback, cookie renewal, SQLite incremental records (data.db), and ffmpeg frame slicing. |
cookie_extractor.py |
Playwright login helper. Opens Chromium for QR/phone login (or connects via CDP port 9222) and writes cookies.txt, storage_state.json, and config_simple.yml. |
get_cookies_manual.py |
Interactive helper for pasting cookies copied from browser DevTools without installing Playwright. |
cut.py |
Optional standalone OpenCV frame exporter for slicing local video directories at custom frame intervals. |
DouYinCommand.py |
Legacy synchronous CLI wrapper kept for compatibility (supports author avatar downloads, HTTP Range resume, and interactive live-stream URL lookup). |
dy-downloader/ |
Standalone earlier modular async CLI package (v1.0.0) with its own run.py, requirements.txt, and dy_downloader.db. Supports single video, gallery, and user post downloads only; kept for reference. |
Requires Python 3.8+ (Tkinter is included with standard Windows/macOS Python installers for ui.py).
# 1. Install core dependencies for downloader.py and ui.py
pip install -r requirements.txt
# 2. (Recommended) Install Playwright + Chromium for cookie extraction and browser fallback
pip install playwright
playwright install chromium
# 3. (Optional) Install OpenCV + tqdm if you use the standalone cut.py script
pip install opencv-python tqdmFrame slicing in downloader.py uses imageio-ffmpeg (already in requirements.txt) if no system ffmpeg binary is on PATH.
Douyin web APIs require a logged-in session for paging user profiles, liked posts, and non-public items.
python cookie_extractor.py
# Or non-interactive mode (opens Chromium directly for QR scan):
python cookie_extractor.py --autoAfter signing in inside the opened browser window, credentials are saved to cookies.txt, storage_state.json, and optionally config_simple.yml. When downloader.py runs, it also syncs cookies.pkl.
python get_cookies_manual.pyFollow the on-screen F12 → Network instructions to paste the Cookie header from www.douyin.com. It validates key fields (ttwid, sessionid, msToken, etc.) and writes config_simple.yml, cookies.txt, and a timestamped cookies_backup_*.json.
Set cookies in config_simple.yml (or config.yml) as auto, a raw cookie string, or a key-value map. With cookies: auto, downloader.py loads cookies.txt and cookies.pkl automatically (preferring whichever has a newer sid_guard expiry) and falls back to Playwright when needed.
python ui.pyPaste one or more Douyin links (or share text), press Enter to add them to the queue, and click 开始下载 (Ctrl+Enter to start, Esc to stop). GUI parameter overrides are saved to ui_run_config.yml without modifying your config_simple.yml or config.yml.
# Download a single video or image post
python downloader.py -c config_simple.yml -u "https://www.douyin.com/video/7320876060210373923"
python downloader.py -c config_simple.yml -u "https://www.douyin.com/note/7320876060210373923"
# Download all published posts on a user profile
python downloader.py -c config_simple.yml -u "https://www.douyin.com/user/MS4wLjABAAAA..."
# Override output directory or backfill frames for previously downloaded videos
python downloader.py -c config_simple.yml -u "https://www.douyin.com/user/MS4wLjABAAAA..." -p "./Downloaded/"
python downloader.py -c config_simple.yml -u "https://www.douyin.com/user/MS4wLjABAAAA..." --slice-backfill
# Renew and save session cookies without downloading media
python downloader.py -c config_simple.yml --refresh-session| Flag | Description |
|---|---|
-c, --config |
YAML config path (default: config.yml, falls back to config_simple.yml if missing). |
-u, --url |
One or more Douyin URLs, share strings, aweme_id, or sec_user_id values. |
-p, --path |
Output directory (overrides path in config). |
--auto-cookie |
Force cookies: auto and allow Playwright cookie lookup. |
--cookie |
Pass a raw Cookie header string on the command line. |
--slice-backfill / --no-slice-backfill |
Enable or skip frame extraction for existing video folders before downloading. |
--refresh-session |
Refresh session cookies via browser/API and write them back to disk without downloading. |
Everyday runs use config_simple.yml (copy from config.example.yml if needed; config.yml contains all incremental/limit keys):
link:
- https://www.douyin.com/user/SEC_USER_ID
path: ./Downloaded/
music: true # Save background audio (video posts)
cover: true # Save cover image
json: true # Save raw metadata JSON
slice_videos: true # Extract JPG frames with ffmpeg
slice_fps: 10 # Frames per second
slice_subdir: frames # Subfolder name under each post folder
slice_backfill_on_run: false # Backfill missing frames for older downloads
auto_install_ffmpeg: true # Auto-install imageio-ffmpeg if ffmpeg is missing
start_time: "" # Date filter YYYY-MM-DD (inclusive), empty = no filter
end_time: ""
mode: # User profile sections: post / like / mix
- post
number: # Item limits (0 = unlimited)
post: 0
like: 0
allmix: 0
mix: 0
music: 0
database: true # Record downloads in SQLite (data.db)
increase: # Skip items already recorded in data.db
post: false
like: false
mix: false
music: false
retry_times: 3 # API retry attempts
cookies: auto # auto / cookie string / cookie dict
profile_link_mode: all # all = /user/... always downloads the profile
# modal = download only the single modal_id/vid post
user_posts_browser_recovery: true # Use Playwright when profile API paging is incomplete
user_posts_browser_headless: false
single_aweme_browser_recovery: true
single_aweme_browser_headless: trueSee USAGE.md for the full configuration reference, link routing rules, and DouYinCommand.py / cut.py usage.
downloader.py writes each post into its own folder keyed by timestamp, sanitized description, and aweme_id:
<path>/<author_nickname>/<YYYY-MM-DD_HH-MM-SS>_<description>_<aweme_id>/
├── <folder_name>.mp4 # Video post
├── clip_1.mp4 # Motion/video clip inside an image post (if present)
├── image_1.jpg # Static image inside an image post
├── <folder_name>_music.mp3 # Background audio (video posts)
├── <folder_name>_cover.jpg # Cover image
├── <folder_name>_data.json # Raw metadata JSON
└── frames/ # Extracted video frames (when slice_videos: true)
└── video_<stem>_000001.jpg
Existing media files on disk are skipped automatically on reruns.
DyFetch/
├── ui.py # Tkinter desktop GUI (drives downloader.py)
├── downloader.py # Main async CLI downloader (UnifiedDownloader v3.2)
├── cookie_extractor.py # Playwright cookie login & CDP extractor
├── get_cookies_manual.py # Manual DevTools cookie helper
├── cut.py # Standalone OpenCV frame exporter
├── DouYinCommand.py # Legacy synchronous CLI wrapper
├── config_simple.yml # Concise daily configuration
├── config.yml # Full configuration with incremental & count fields
├── config.example.yml # Configuration template
├── apiproxy/ # Douyin API endpoints, X-Bogus signer, SQLite wrapper, Cookie manager
├── utils/ # URL cleaner (urlfix.py) and rotating logger
├── tests/ # Isolated regression suite (regress_downloader.py)
├── dy-downloader/ # Standalone earlier modular async CLI + pytest suite
├── claudedocs/ # Functional specification document
├── USAGE.md # Detailed usage & configuration manual
└── YOLO.md # Architecture & maintenance guide
The root regression script runs in a temporary directory without touching your local data.db or cookies:
python tests/regress_downloader.py # 20 offline checks
python tests/regress_downloader.py --network # + 4 live Douyin API/download checks
python ui.py --selftest # Tkinter GUI startup smoke checkTo run the standalone dy-downloader unit tests (must execute inside dy-downloader/ so the root utils/ package does not shadow dy-downloader/utils/):
cd dy-downloader
pip install -r requirements.txt
python -m pytest tests -q- 0 items downloaded / empty API responses / HTTP 403: Check the startup session validity line (
登录态有效期至 ...orCookie 登录状态已于 ... 过期). If expired, runpython cookie_extractor.py(or click 重新登录 inui.py). Note that after a failed automatic browser refresh,downloader.pywaits 6 hours before retrying (tracked in.session_refresh_state.json); runpython downloader.py --refresh-sessionto force an immediate check after logging in. - User profile downloads fewer posts than displayed on Douyin: Anonymous sessions only receive the first page of profile posts. Sign in via
cookie_extractor.py, or setuser_posts_browser_headless: falseso you can log in when Playwright opens the profile window. /user/selflink fails: Replacehttps://www.douyin.com/user/self?...with your actualhttps://www.douyin.com/user/MS4wLjABAAAA...profile URL and setmode: [like](orpost/mix).- No frames generated under
frames/: Checkslice_videos: trueand verifyffmpeg -versionworks orimageio-ffmpegis installed (pip install imageio-ffmpeg).
This tool is intended for personal learning, research, and authorized content backup only. Respect Douyin's Terms of Service and content creators' copyrights; do not use it for unauthorized redistribution or commercial scraping.
DyFetch builds upon jiji262/douyin-downloader (MIT License) for dy-downloader/, the apiproxy/ layout, and DouYinCommand.py, with downloader.py and ui.py substantially rewritten and extended. dy-downloader/utils/xbogus.py includes Evil0ctal's X-Bogus implementation from Evil0ctal/Douyin_TikTok_Download_API under the Apache License 2.0.
Released under the MIT License — see LICENSE. Third-party attributions and license notices are documented in THIRD-PARTY-NOTICES.md and licenses/Apache-2.0.txt.

