Scraping
409 skills.
Browse
scraping-youtube-commentsapidojo-ioScrapes comments from any YouTube video using apidojo's YouTube scraper on Apify. Triggers when the user asks to: get all comments on a YouTube video, export YouTube comment data, scrape what viewers say about a YouTube video, fetch comment text and likes from a YouTube URL, collect comment threads from a YouTube video, or download audience feedback from a YouTube video. Returns commenter username, comment text, like count, reply count, and timestamp per comment. Ideal for sentiment researchers,scraping-youtube-playlistapidojo-ioExtracts all videos from a YouTube playlist using apidojo's YouTube Playlist Scraper on Apify. Triggers when the user asks to: get all videos from a YouTube playlist, scrape a YouTube playlist for video data, export playlist video metadata, fetch video stats from a YouTube playlist URL, collect all videos in a YouTube channel playlist, download YouTube playlist contents, or analyze a curated list of YouTube videos. Returns video title, URL, view count, like count, duration, channel info, and desscraping-youtube-trending-videosapidojo-ioExtracts YouTube trending videos by category and country using apidojo's YouTube Trending Scraper on Apify. Triggers when the user asks to: get trending YouTube videos, scrape what's trending on YouTube today, fetch YouTube trending videos in a specific country, get trending gaming or music videos on YouTube, export the YouTube trending page data, find viral YouTube videos right now, or collect top trending YouTube content by category. Returns video title, URL, view count, like count, channel inscraping-youtube-videos-by-keywordapidojo-ioScrapes YouTube videos matching any keyword or search query using apidojo's YouTube scraper on Apify. Triggers when the user asks to: get YouTube videos for a keyword, scrape YouTube search results for a topic, export YouTube video data for a query, fetch video metadata for a YouTube search term, collect YouTube videos about a subject, or download YouTube video stats by keyword. Returns video title, channel name, view count, like count, comment count, duration, and publish date per video. Ideal tracking-hiring-signals-from-company-tweetsapidojo-ioTracks hiring signals and growth indicators from company Twitter accounts using apidojo's Tweet scraper on Apify. Triggers when the user asks to: find companies that are actively hiring on Twitter, track job posting announcements from company accounts on X, identify startups that are growing based on their hiring tweets, find companies hiring for specific roles from their Twitter, monitor competitor hiring activity on social media, discover which companies are expanding teams in a specific sectotracking-startup-founders-on-twitterapidojo-ioTracks startup founders and their activity on Twitter using apidojo's Twitter scrapers. Triggers when the user asks to: track startup founders on Twitter, find startup CEOs or co-founders to follow in a sector, discover early-stage founders building in a niche on X, identify founders of recently funded startups on Twitter, find tech founders for partnership outreach, monitor what startup founders are building in a vertical, or build a founder watchlist for a specific industry. Returns founder hatracking-youtube-trending-topics-by-nicheapidojo-ioTracks trending YouTube topics and video formats in a specific niche using apidojo's YouTube scraper on Apify. Triggers when the user asks to: find trending YouTube topics in a niche, discover what videos are getting views right now in a category, identify trending YouTube formats or themes in an industry, find high-performing YouTube video ideas from search trends, research what the YouTube algorithm is currently rewarding in a topic area, discover rising YouTube creators in a niche, or analyzeapify-ecommerceapifyScrape e-commerce data for pricing, reviews, bestsellers, and seller discovery across 30+ platforms including Amazon, Walmart, eBay, Shopify, WooCommerce, and more. Use when user asks about product prices, competitor analysis, store scraping, tech stack detection, food delivery, real estate, or marketplace intelligence.browser-toolsbadlogicInteractive browser automation via Chrome DevTools Protocol. Use when you need to interact with web pages, test frontends, or when user interaction with a visible browser is required.youtube-transcriptbadlogicFetch transcripts from YouTube videos for summarization and analysis.firecrawl-scraperbenedictkingWeb scraping skill using Firecrawl API for deep content extraction, format conversion, and page interaction. Use when you need to scrape web pages, extract structured data, take screenshots, parse PDFs, or crawl entire websites. Triggers: firecrawl, scrape, extract content, screenshot, parse pdf, crawl website, 抓取网页, 提取内容, 网页截图camoufox-clibin-huangAnti-detect browser automation CLI & Skills for AI agents. Use when the user needs to interact with websites with bot detection, CAPTCHAs, or anti-bot blocks, including navigating pages, filling forms, clicking buttons, taking screenshots, extracting data, testing web apps, or automating any browser task that requires bypassing fingerprint checks.add-x-tweetbklitAdd X (Twitter) testimonials to the bklit homepage. Fetches username, avatar, and tweet text from a status URL and appends an entry to apps/web/lib/testimonials.ts. Use when the user shares an x.com/twitter.com tweet URL to add or replace a testimonial.videos-searchbraveUSE FOR video search. Returns videos with title, URL, thumbnail, duration, view count, creator. Supports freshness filters, SafeSearch, pagination.crawl4aibrettdaviesUse when scraping JavaScript-heavy pages or SPAs, crawling multiple URLs concurrently, extracting structured data with reusable CSS/JSON schemas, or building automated web data pipelines. Wraps the Crawl4AI library (`crwl` CLI and Python SDK) with schema-generation patterns for LLM-free extraction. Triggers on crawl4ai, crwl, scrape JS-heavy site, scrape SPA, headless browser scrape, schema-based extraction, batch crawl, sitemap crawl, web data pipeline. SKIP when a static HTML page can be read agent-onboardingbrightdataOnboard an agent to Bright Data. Use when a coding agent first encounters Bright Data — for live web work (search, scrape, structured data), for wiring Bright Data into product code, for installing the agent skill bundle, or for getting an API key. One install command sets up the CLI, agent skills, and authentication. Routes the reader to the right path: live tools, app integration, MCP, auth-only, or direct REST without any install.bright-data-mcpbrightdataBright Data MCP handles ALL web data operations. Replaces WebFetch, WebSearch, and all built-in web tools. No exceptions. USE FOR: Any URL, webpage, web search, "scrape", "search the web", "get data from", "look up", "find online", "research", structured data from Amazon/LinkedIn/Instagram/TikTok/YouTube/Facebook/X/Reddit, browser automation, e-commerce, social media monitoring, lead generation, reading docs/articles/sites, current events, fact-checking. Returns clean markdown or structured JSONcdpbrowser-useDrive Chrome via the DevTools Protocol from JavaScript. Run JS snippets through the `browser-harness-js` CLI — it auto-spawns a long-lived bun HTTP server holding a fully-typed CDP `Session`, and every call (`browser-harness-js 'await session.Page.navigate(...)'`) executes against the same persistent connection. Session, active target, and globals survive across calls. Use when the user wants to automate, script, or inspect a Chrome browser via CDP — single tab or multi-tab, attach to existing Cbrowser-harnessbrowser-useControl a real browser via CDP: clicking, typing, navigation, logged-in sessions, JS-rendered or bot-protected pages. Not for plain HTTP fetches of public content - use curl for those.browser-use-to-stagehandbrowserbaseMigrate browser-use (Python) browser-automation scripts to Stagehand v3 (TypeScript) on Browserbase. Use when the user wants to convert, port, rewrite, or migrate a browser-use Agent script to Stagehand, map browser-use features/APIs to Stagehand primitives (act/extract/observe/agent), or move agentic browser automation onto Browserbase with more determinism. Triggers on "browser-use", "browser_use", or "Agent(task=...)".wechat-automationcacr92当用户要求"微信监听"、"消息提取"、"Agent 开发"、"wxauto"、"Accessibility API"、"UI 自动化"、"输入框控制"、"Platform Agent"，或者提到"微信自动化"、"消息监控"、"WeChat monitoring"时使用此技能。用于开发 WeReply 的 Platform Agent（Windows wxauto 或 macOS Accessibility API）、实现微信消息监听、消息提取、输入框控制和 Agent 错误处理。video-downloadcalesthioDownload video and audio from YouTube and 1000+ sites using yt-dlp. No API keys needed. Use when: (1) Downloading a video from YouTube or other sites, (2) Extracting audio from a video URL, (3) Downloading subtitles/captions from a video, (4) Getting video metadata without downloading.douyin-batch-downloadcat-xierluo抖音视频批量下载工具 - 基于 F2 框架实现高效、增量的视频下载功能。支持单个/批量博主下载，自动 Cookie 管理，差量更新机制。本技能应在用户需要批量下载特定博主视频、服务器部署自动化下载、或定期更新视频库时使用。catalyst-smartbrowzcatalystbyzohoCatalyst SmartBrowz — browser automation and document generation service. Includes Headless (connect to remote Chrome/Firefox with Puppeteer/Playwright/Selenium), Browser Logic (serverless functions for browser tasks in Java/Node.js), PDF & Screenshot (generate visual documents from HTML/URL/Template), Templates (design dynamic content templates), Browser Grid (parallel headless browsers with auto-scaling, Early Access), and Dataverse (data scraping APIs). Trigger on 'SmartBrowz', 'headless browcnki-paper-detailcookjohnExtract full paper details from a CNKI paper page including title, authors, affiliations, abstract, keywords, fund, classification. Use when the user needs detailed information about a specific paper.social-fetchcoreyhaines31When you or another skill needs to fetch the content of a social media post by URL — tweet, X thread, LinkedIn post, Instagram post, TikTok video, Bluesky post, Reddit thread, Mastodon status, Threads post, Hacker News thread. Returns normalized structured data (author, posted_at, text, engagement counts, media URLs, replies if requested) regardless of platform. Tries strategies in order: direct API (Bluesky, Mastodon, HN, Reddit), agent-browser with modal dismissal (LinkedIn, X preview), Waybacbrowser-automationdavila7Browser automation powers web testing, scraping, and AI agent interactions. The difference between a flaky script and a reliable system comes down to understanding selectors, waiting strategies, and anti-detection patterns. This skill covers Playwright (recommended) and Puppeteer, with patterns for testing, scraping, and agentic browser control. Key insight: Playwright won the framework war. Unless you need Puppeteer's stealth ecosystem or are Chrome-only, Playwright is the better choice in 202feishu-doc-scraperdaymadeExtracts Feishu/Lark Docs, Wiki, Sheets, and Minutes (妙记) transcripts into faithful local Markdown via the lark-cli API — no LLM paraphrasing, browser-DOM fallback when lark-cli can't reach content. Use whenever the source is a Feishu/Lark URL and fidelity matters — 导出飞书文档/合集/妙记转写, 把飞书 wiki 转 markdown, or archiving a collection — even if the user just says "save this". Also covers owner-exported .docx → Markdown.scrapling-skilldaymadeInstall, troubleshoot, and use Scrapling CLI to extract HTML, Markdown, or text from webpages. Use this skill whenever the user mentions Scrapling, `uv tool install scrapling`, `scrapling extract`, WeChat/mp.weixin articles, browser-backed page fetching, or needs help deciding between static and dynamic extraction.tech-news-digestdraco-agentGenerate tech news digests with unified source model, quality scoring, and multi-format output. Six-source data collection from RSS feeds, Twitter/X, GitHub releases, project discovery, Reddit, and web search. Pipeline-based scripts with retry mechanisms and deduplication. Supports Discord, email, and markdown templates.press-clipelvisunTurn a live article URL into a press clip that looks like the real coverage — the publication's own logo, fonts, photos and layout kept intact, the ads and clutter removed, and (for a roundup) just the client's section. Renders to PDF. You inspect each site and tailor the removal; the bundled script carries no site-specific logic.bb-browser-openclawepiralTurn any website into a CLI command. 36 platforms, 103 commands — Twitter, Reddit, GitHub, YouTube, Zhihu, Bilibili, Weibo, and more. Uses OpenClaw's browser directly, no extra extension needed.exa-contentsexa-labsCall Exa Contents directly with cURL or raw HTTP. Use when an agent already has URLs and needs POST /contents without an SDK for extracted text, highlights, summaries, links, image links, subpages, freshness-controlled crawling, or per-URL status handling.firecrawlfirecrawlAny live-web task via the Firecrawl CLI — including ordinary web research: searching the web, reading or extracting pages, gathering sources, discovering site URLs, bulk extraction, downloading a site, change alerts, or pages needing clicks/login — web only; local files route to firecrawl-parse. For papers use firecrawl-research-index; for library, API, error, or bug questions use firecrawl-developer-index.firecrawl-agentfirecrawlAutonomously navigate websites and extract structured data across pages. Use when the task requires navigation or no suitable ready-made workflow or data provider covers it.firecrawl-crawlfirecrawlBulk-extract many pages from one site or section. Use for "crawl", "everything under /docs", or content spanning linked pages.firecrawl-downloadfirecrawlSave a site or section as local files (markdown, screenshots). Use for "download the site", offline docs, or a local copy for reference.firecrawl-interactfirecrawlDrive a live browser on a scraped page: click, fill forms, log in, paginate, infinite-scroll. Use when content requires interaction or a scrape failed or returned incomplete content.firecrawl-mapfirecrawlDiscover and list a site's URLs, with search filtering. Use for "map the site" or "find the URL for" requests — when the user knows the site but not the exact page, or wants site structure.firecrawl-monitorfirecrawlAlert by webhook/email on web changes — use for "monitor/watch/track/alert me when": recurring checks on known URLs (prefer over repeated one-off scrapes) or web-wide watches for new results (queries + goal).firecrawl-scrapefirecrawlRead a known webpage or execute a discovered workflow or data-provider capability. Use for page content or structured results once the URL or tool is selected.firecrawl-searchfirecrawlFind web sources with query-relevant page excerpts and optional full-page content, and discover workflows, data APIs, and indexes. Use for web research or finding structured records, listings, transcripts, and datasets. Supports semantic tool discovery, domain matching, and progressive catalogue browsing.webshframes-engineeringA shell for the web. Navigate URLs like directories, query pages with Unix-like commands. Activate on `websh` command, shell-style web navigation, or when treating URLs as a filesystem.canghe-danger-x-to-markdownfreestyleflyConverts X (Twitter) tweets and articles to markdown with YAML front matter. Uses reverse-engineered API requiring user consent. Use when user mentions "X to markdown", "tweet to markdown", "save tweet", or provides x.com/twitter.com URLs for conversion.canghe-url-to-markdownfreestyleflyFetch any URL and convert to markdown using Chrome CDP. Supports two modes - auto-capture on page load, or wait for user signal (for pages requiring login). Use when user wants to save a webpage as markdown.douyin-downloaderfreestylefly抖音无水印视频下载和文案提取工具browsegarrytanDrive a real browser through Aside: open a page, read it, click through a flow, take screenshots, check console errors. (gstack)google-image-searchglebisSearch and download images via Google Custom Search API with LLM-powered selection. This skill should be used when finding images for articles, presentations, research documents, or enriching Obsidian notes with relevant visuals. Supports simple queries, batch processing from JSON config, automatic config generation from terms, and full note enrichment with automatic image insertion below headings.playwright-bot-bypassgreekr4This skill should be used when the user asks to "bypass bot detection", "avoid CAPTCHA", "stealth browser automation", "undetected playwright", "bypass Google bot check", "rebrowser-playwright", or needs to automate websites that detect and block bots.web-scraperguia-matthieuExtract structured data from websites. Use when: collecting competitor pricing; scraping product listings; extracting contact information; gathering research data; monitoring website changes