github/scientific-agent-skills
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(parallel-web): migrate skill docs from parallel-cli to google_research.py (Step 4)

Browse Source 
Replace all parallel-cli commands and PARALLEL_API_KEY references with
google_research.py + Vertex AI equivalents:

- SKILL.md: swap env vars, setup instructions, and routing table
- web-search.md: --fast mode (gemini-2.5-flash), pre-synthesized output
- deep-research.md: synchronous command with --timeout 1800, poll recovery
- web-extract.md: google_research.py extract with --objective support

Residual: data-enrichment.md unchanged (out of scope per step2 decision log).
This commit is contained in:
Yuhuan He
2026-07-01 16:59:19 -07:00
parent 332e043259
commit 0ce5914927
4 changed files with 105 additions and 184 deletions
Download Patch File Download Diff File Expand all files Collapse all files

88
skills/parallel-web/SKILL.md
View File

@@ -1,103 +1,79 @@
---
name: parallel-web
description: "All-in-one web toolkit powered by parallel-cli, with a strong emphasis on academic and scientific sources. Use this skill whenever the user needs to search the web, fetch/extract URL content, enrich data with web-sourced fields, or run deep research reports. Covers: web search (fast lookups, research, current info — prioritizing peer-reviewed papers, preprints, and scholarly databases), URL extraction (fetching pages, articles, academic PDFs), bulk data enrichment (adding fields to CSV/lists from the web), and deep research (exhaustive multi-source reports grounded in academic literature). Also handles setup, status checks, and result retrieval. Use this skill for ANY web-related task — even if the user doesn't mention 'parallel' or 'web' explicitly. If they want to look something up, fetch a page, enrich a dataset, investigate a topic, find academic papers, check citations, or review scientific literature, this is the skill to use."
compatibility: Requires parallel-cli and internet access.
required_environment_variables: [{"name": "PARALLEL_API_KEY", "prompt": "Parallel API key.", "required_for": "full functionality"}]
metadata: {"version": "1.1", "author": "K-Dense, Inc.", "openclaw": {"primaryEnv": "PARALLEL_API_KEY", "envVars": [{"name": "PARALLEL_API_KEY", "required": true, "description": "Parallel API key."}]}}
description: "All-in-one web toolkit powered by Google Deep Research (Vertex AI), with a strong emphasis on academic and scientific sources. Use this skill whenever the user needs to search the web, fetch/extract URL content, or run deep research reports. Covers: web search (fast lookups, research, current info — prioritizing peer-reviewed papers, preprints, and scholarly databases), URL extraction (fetching pages, articles, academic PDFs), and deep research (exhaustive multi-source reports grounded in academic literature). Also handles status checks and result retrieval by interaction ID. Use this skill for ANY web-related task — even if the user doesn't mention 'search' or 'web' explicitly. If they want to look something up, fetch a page, investigate a topic, find academic papers, check citations, or review scientific literature, this is the skill to use."
compatibility: Requires google-genai (pip) and Vertex AI access.
required_environment_variables: [{"name": "GOOGLE_CLOUD_PROJECT", "prompt": "GCP project ID for Vertex AI.", "required_for": "full functionality"}, {"name": "GOOGLE_APPLICATION_CREDENTIALS", "prompt": "Path to Vertex service-account JSON (or use ADC).", "required_for": "full functionality"}]
metadata: {"version": "2.0", "author": "K-Dense, Inc.", "openclaw": {"primaryEnv": "GOOGLE_CLOUD_PROJECT", "envVars": [{"name": "GOOGLE_CLOUD_PROJECT", "required": true, "description": "GCP project ID."}, {"name": "GOOGLE_APPLICATION_CREDENTIALS", "required": false, "description": "Service-account JSON path. Not needed if using ADC."}]}}
---
# Parallel Web Toolkit
# Google Deep Research Web Toolkit
A unified skill for all web-powered tasks: searching, extracting, enriching, and researching — with academic and scientific sources as the default priority.
A unified skill for all web-powered tasks: searching, extracting, and researching — with academic and scientific sources as the default priority. Powered by Google Deep Research via Vertex AI.
## Routing — pick the right capability
Read the user's request and match it to one of the capabilities below. For web search, extract, enrichment, and deep research, read the corresponding reference file for detailed instructions.
Read the user's request and match it to one of the capabilities below. For web search, extract, and deep research, read the corresponding reference file for detailed instructions.
| User wants to... | Capability | Where |
|---|---|---|
| Look something up, research a topic, find current info | **Web Search** | `references/web-search.md` |
| Fetch content from a specific URL (webpage, article, PDF) | **Web Extract** | `references/web-extract.md` |
| Add web-sourced fields to a list of companies/people/products | **Data Enrichment** | `references/data-enrichment.md` |
| Get an exhaustive, multi-source report (user says "deep research", "exhaustive", "comprehensive") | **Deep Research** | `references/deep-research.md` |
| Install or authenticate parallel-cli | **Setup** | Below |
| Check status of a running research/enrichment task | **Status** | Below |
| Retrieve completed research results by run ID | **Result** | Below |
| Retrieve a research report by interaction ID (poll after timeout) | **Poll** | Below |
| Install dependencies or verify auth | **Setup** | Below |
### Decision guide
- **Default to Web Search** for a single lookup, research question, or "what is X?" query. It's fast and cost-effective. When the query touches a scientific or technical topic, include academic domains (see `references/web-search.md`) to surface peer-reviewed and preprint sources alongside general results.
- **Use Web Extract** when the user provides a URL or asks you to read/fetch a specific page. Prefer this over the built-in WebFetch tool. Particularly useful for extracting full text from academic PDFs, preprint servers, and journal articles.
- **Use Data Enrichment** when the user has **multiple entities** (a CSV, a list of companies/people/products, or even a short inline list) and wants to find or add the same kind of information for each one. The key signal is a repeated lookup across a set of items — e.g., "find the CEO for each of these companies" or "get the founding year for Apple, Stripe, and Anthropic." Even if the user doesn't say "enrich," use `parallel-cli enrich` whenever the task is the same query applied to multiple entities. Do NOT use Web Search in a loop for this — the enrichment pipeline handles batching, parallelism, and structured output automatically.
- **Use Deep Research only** when the user explicitly asks for deep, exhaustive, or comprehensive research. It is 10-100x slower and more expensive than Web Search — never default to it. Deep research is especially valuable for literature reviews and multi-paper synthesis.
- If `parallel-cli` is not found when running any command, follow the Setup section below.
- **Default to Web Search** for a single lookup, research question, or "what is X?" query. It's fast and cost-effective.
- **Use Web Extract** when the user provides a URL or asks you to read/fetch a specific page. Particularly useful for academic PDFs, preprint servers, and journal articles.
- **Use Deep Research only** when the user explicitly asks for deep, exhaustive, or comprehensive research. It takes 5–20 minutes — never default to it.
- Data enrichment (batch entity lookups) is **not available** in this skill version. If the user needs it, let them know and suggest an alternative approach.
### Academic source priority
Across all capabilities, prefer academic and scientific sources when the query is technical or scientific in nature. This means:
Across all capabilities, prefer academic and scientific sources when the query is technical or scientific in nature:
- Peer-reviewed journal articles and conference proceedings over blog posts or news articles
- Preprints (arXiv, bioRxiv, medRxiv) when peer-reviewed versions aren't available
- Institutional and government sources (NIH, WHO, NASA, NIST) over commercial sites
- Primary research over secondary summaries
When citing academic sources, include author names and publication year where available (e.g., [Smith et al., 2025](url)) in addition to the standard citation format. If a DOI is present, prefer the DOI link.
## Context chaining
Several capabilities support multi-turn context via `interaction_id`. When a research or enrichment task completes, it returns an `interaction_id`. If the user asks a follow-up question related to that task, pass `--previous-interaction-id` to carry context forward automatically. This avoids restating what was already found.
When citing academic sources, include author names and publication year where available in addition to the standard citation format.
---
## Setup
If `parallel-cli` is not installed, install and authenticate:
If `google-genai` is not installed, install it:
```bash
curl -fsSL https://parallel.ai/install.sh | bash
uv pip install google-genai
# or: pip install google-genai
```
If unable to install that way, use uv instead:
Verify auth. The script needs `GOOGLE_CLOUD_PROJECT` and either `GOOGLE_APPLICATION_CREDENTIALS` (service-account JSON path) or Application Default Credentials (ADC via `gcloud auth application-default login`).
Check if a `.env` file exists in the project root containing `GOOGLE_CLOUD_PROJECT` and `GOOGLE_APPLICATION_CREDENTIALS`. If so, load it before running:
```bash
uv tool install "parallel-web-tools[cli]"
dotenv -f .env run python scripts/google_research.py search "test" --fast
```
Then authenticate. First, check if a `.env` file exists in the project root and contains `PARALLEL_API_KEY`. If so, load it with `dotenv`:
If `dotenv` isn't available: `pip install python-dotenv[cli]` or `uv pip install python-dotenv[cli]`.
If env vars are already in the environment, run directly:
```bash
dotenv -f .env run parallel-cli auth
python scripts/google_research.py search "test" --fast
```
If `dotenv` isn't available, install it with `pip install python-dotenv[cli]` or `uv pip install python-dotenv[cli]`.
---
If there's no `.env` file or it doesn't contain the key, fall back to interactive login:
## Poll a research result by interaction ID
If a `research` command timed out, the script printed an `interaction_id` to stderr. Use it to fetch the completed report later:
```bash
parallel-cli login
python scripts/google_research.py poll "$INTERACTION_ID" -o "$FILENAME.md"
```
Or set the key manually: `export PARALLEL_API_KEY="your-key"`
Verify with:
```bash
parallel-cli auth
```
If `parallel-cli` is not found after install, add `~/.local/bin` to PATH.
## Check task status
```bash
parallel-cli research status "$RUN_ID" --json
```
Report the current status to the user (running, completed, failed, etc.).
## Get completed result
```bash
parallel-cli research poll "$RUN_ID" --json
```
Present results in a clear, organized format.
Report the file path and offer to read the file if the user wants to review the results.

87
skills/parallel-web/references/deep-research.md
View File

@@ -4,77 +4,48 @@ Research topic: $ARGUMENTS
## When to use (vs web search)
ONLY use this capability when the user explicitly requests deep/exhaustive research. Deep research is 10-100x slower and more expensive than web search. For normal "research X" requests, quick lookups, or fact-checking, use **web search** instead.
ONLY use this capability when the user explicitly requests deep/exhaustive research. Deep research takes 5–20 minutes and is significantly more expensive than web search. For normal "research X" requests, quick lookups, or fact-checking, use **web search** instead.
## Step 1: Start the research
## Command
Frame the research objective to prioritize academic literature. If the user's query is scientific or technical, prepend context to the arguments that steers toward scholarly sources — e.g., instead of `"effects of sleep deprivation"`, use `"peer-reviewed research and clinical studies on the effects of sleep deprivation"`. This nudges the research agent toward primary literature without excluding relevant non-academic sources.
Choose a descriptive filename based on the topic (e.g., `mrna-vaccine-platforms-2026`, `gut-microbiome-depression`). Use lowercase with hyphens, no spaces.
```bash
parallel-cli research run "$ARGUMENTS" --processor pro-fast --no-wait --json
```
If this is a **follow-up** to a previous research or enrichment task where you know the `interaction_id`, add context chaining:
```bash
parallel-cli research run "$ARGUMENTS" --processor lite --no-wait --json --previous-interaction-id "$INTERACTION_ID"
```
By chaining `interaction_id` values across requests, each follow-up question automatically has the full context of prior turns — so you can drill deeper into a topic without restating what was already researched. Use `--processor lite` for follow-ups since the heavy research was already done in the initial turn and the follow-up just needs to build on that context.
This returns instantly. Do NOT omit `--no-wait` — without it the command blocks for minutes and will time out.
Processor options (choose based on user request):
| Processor | Expected latency | Use when |
|-----------|-----------------|----------|
| `pro-fast` | 30s - 5 min | Default — good balance of depth and speed |
| `ultra-fast` | 1 - 10 min | Deeper analysis, more sources (~2x cost) |
| `ultra` | 5 - 25 min | Maximum depth, only when explicitly requested (~3x cost) |
Parse the JSON output to extract the `run_id`, `interaction_id`, and monitoring URL. Immediately tell the user:
- Deep research has been kicked off
- The expected latency for the processor tier chosen (from the table above)
- The monitoring URL where they can track progress
Tell them they can background the polling step to continue working while it runs.
## Step 2: Poll for results
Choose a descriptive filename based on the topic (e.g., `ai-chip-market-2026`, `react-vs-vue-comparison`). Use lowercase with hyphens, no spaces.
```bash
parallel-cli research poll "$RUN_ID" -o "$FILENAME" --timeout 540
python scripts/google_research.py research "$ARGUMENTS" -o "$FILENAME.md" --timeout 1800
```
Important:
- Use `--timeout 540` (9 minutes) to stay within tool execution limits
- Do NOT pass `--json` — the full output is large and will flood context. The `-o` flag writes results to files instead.
- The `-o` flag generates two output files:
- `$FILENAME.json` — metadata and basis
- `$FILENAME.md` — formatted markdown report
- The poll command prints an **executive summary** to stdout when the research completes. Share this executive summary with the user — it gives them a quick overview without having to open the files.
- Use `--timeout 1800` (30 minutes) — academic queries regularly take 15+ minutes.
- The command is **synchronous**: it handles polling internally. Do not run a separate poll step unless the command times out (see below).
- The script prints `interaction_id` and progress to stderr. Note the `interaction_id` — you will need it if the command times out.
- For scientific or technical queries, prepend context to steer toward scholarly sources. For example, instead of `"effects of sleep deprivation"`, use `"peer-reviewed research and clinical studies on the effects of sleep deprivation"`.
### If the poll times out
## If the command times out
Higher processor tiers can take longer than 9 minutes. If the poll exits without completing:
1. Tell the user the research is still running server-side
2. Re-run the same `parallel-cli research poll` command to continue waiting
If the command exits with a timeout error, it will print the `interaction_id` and a recovery command. Wait a few minutes, then poll:
```bash
python scripts/google_research.py poll "$INTERACTION_ID" -o "$FILENAME.md"
```
Tell the user the research is still running server-side, and re-run `poll` again if it hasn't completed yet.
## What the output looks like
The script writes a comprehensive markdown report to `$FILENAME.md` and prints it to stdout. The report contains:
- Full narrative with inline `[[N]](url)` citations
- A `## Sources` section (often 50–100+ entries for deep queries)
There is no separate `.json` metadata file or executive summary — the markdown is the complete deliverable.
## Response format
**After step 1:** Share the monitoring URL (for tracking progress only — it is not the final report).
**After the command completes:**
**After step 2:**
1. Share the **executive summary** that the poll command printed to stdout
2. Briefly assess the source quality: how many of the cited sources are peer-reviewed journals, preprints, or institutional reports vs. news articles or blog posts. If academic coverage is thin, flag this to the user and suggest a follow-up search targeting specific databases (e.g., "I can search PubMed/arXiv specifically for more primary sources on this").
3. Tell the user the two generated file paths:
- `$FILENAME.md` — formatted markdown report
- `$FILENAME.json` — metadata and basis
4. Share the `interaction_id` and tell the user they can ask follow-up questions that build on this research (e.g., "drill deeper into X" or "compare that to Y")
1. **Assess source quality** from the `## Sources` section and the narrative text. Since URLs are grounding redirects rather than direct publisher links, assess quality from source titles and in-text mentions (e.g., "Nature", "PubMed", "PMC", "arXiv"). Count roughly how many are peer-reviewed journals/preprints vs. news/blog. Flag to the user if academic coverage is thin and offer a follow-up targeted web search.
Do NOT re-share the monitoring URL after completion — the results are in the files, not at that link.
2. Tell the user the report file path: `$FILENAME.md`
Ask the user if they would like to read through the files for more detail. Do NOT read the file contents into context unless the user asks.
3. Ask if the user wants to review the full report. **Do NOT read the file into context unless the user asks** — reports are often 5,000–10,000+ words and will flood the context window.
**Remember the `interaction_id`** — if the user asks a follow-up question that relates to this research, use it as `--previous-interaction-id` in the next research or enrichment command.
4. Note the `interaction_id` from stderr for potential follow-up queries (visible in stderr output as `[research] interaction_id=...`).

43
skills/parallel-web/references/web-extract.md
View File

@@ -4,42 +4,43 @@ Extract content from: $ARGUMENTS
## Command
Choose a short, descriptive filename based on the URL or content (e.g., `vespa-docs`, `react-hooks-api`). Use lowercase with hyphens, no spaces.
Choose a short, descriptive filename based on the URL or content (e.g., `vespa-docs`, `attention-is-all-you-need`). Use lowercase with hyphens, no spaces.
```bash
parallel-cli extract "$ARGUMENTS" --json -o "$FILENAME.json"
python scripts/google_research.py extract "$ARGUMENTS" -o "$FILENAME.md"
```
Options if needed:
- `--objective "focus area"` to focus on specific content
- `--objective "focus area"` to focus on specific content (highly recommended for academic papers)
## Academic content handling
When extracting from academic sources (arXiv, PubMed, journal sites, conference proceedings), use `--objective` to focus on the most valuable sections:
When extracting from academic sources (arXiv, PubMed, journal sites, conference proceedings), use `--objective` to target the most valuable sections:
```bash
parallel-cli extract "$URL" --json --objective "extract abstract, methodology, key findings, and conclusions" -o "$FILENAME.json"
python scripts/google_research.py extract "$URL" \
--objective "extract title, authors, publication date, abstract, methodology, key findings, and conclusions" \
-o "$FILENAME.md"
```
For arXiv papers, prefer the `/abs/` URL (which has structured metadata) over the raw PDF URL when available. If the user provides a PDF link, extract it directly — parallel-cli handles PDFs.
For arXiv papers, prefer the `/abs/` URL (which has structured metadata) over the raw PDF URL when available. The script handles both.
## What the output looks like
The script returns a structured markdown document with:
- Title, authors, publication date (for academic papers)
- Abstract and key findings as prose sections
- A `## Sources` section with the source URL
The output is richer than raw page scraping — it is a structured synthesis of the page content.
## Response format
Return content as:
Present the extracted content to the user. For academic papers, confirm key metadata (title, authors, venue, date) are present before presenting the full extraction.
**[Page Title](URL)**
For academic papers, include structured metadata when available:
- **Authors:** list of authors
- **Published:** date and venue/journal
- **DOI:** if available
- **Abstract:** the paper's abstract
Then the extracted content verbatim, with these rules:
- Keep content verbatim - do not paraphrase or summarize
- Parse lists exhaustively - extract EVERY numbered/bulleted item
- Strip only obvious noise: nav menus, footers, ads
Keep the content faithful to the source:
- Do not paraphrase or re-summarize the extracted text
- Preserve all facts, names, numbers, dates, quotes
- For academic papers, preserve figure/table captions and references
- For academic papers, preserve figure/table captions and key numerical results
After the response, mention the output file path (`$FILENAME.json`) so the user knows it's available for follow-up questions.
After presenting the content, mention the output file path (`$FILENAME.md`) so the user knows it's saved for follow-up questions.

71
skills/parallel-web/references/web-search.md
View File

@@ -7,67 +7,40 @@ Search the web for: $ARGUMENTS
Choose a short, descriptive filename based on the query (e.g., `ai-chip-news`, `react-vs-vue`). Use lowercase with hyphens, no spaces.
```bash
parallel-cli search "$ARGUMENTS" -q "<keyword1>" -q "<keyword2>" --json --max-results 10 --excerpt-max-chars-total 27000 -o "$FILENAME.json"
python scripts/google_research.py search "$ARGUMENTS" --fast -o "$FILENAME.md"
```
The first argument is the **objective** — a natural language description of what you're looking for. It replaces multiple keyword searches with a single call for broad or complex queries. Add `-q` flags for specific keyword queries to supplement the objective. The `-o` flag saves the full results to a JSON file for follow-up questions.
`--fast` uses `gemini-2.5-flash + google_search` for a ~10s synthesized response. This is the default and recommended mode for all search queries.
Options if needed:
- `--after-date YYYY-MM-DD` for time-sensitive queries
- `--include-domains domain1.com,domain2.com` to limit to specific sources
- `--timeout N` to extend the timeout (default 600s; rarely needed with `--fast`)
## What the output looks like
The script writes a pre-synthesized markdown report directly to `$FILENAME.md` and prints the same content to stdout. The output already contains:
- Inline citations in `[[N]](url)` format
- A `## Sources` section listing all referenced URLs with titles
**Do not re-parse or re-synthesize the output.** Present it as-is — the synthesis and citation linking are already done.
## Academic source strategy
For scientific or technical queries, run **two searches** to ensure academic sources surface alongside general results:
For scientific or technical queries, append academic context to the query string to improve source targeting:
1. **Academic-focused search** — append `--include-domains` with scholarly domains:
```bash
parallel-cli search "$ARGUMENTS" -q "<keyword1>" --json --max-results 10 --excerpt-max-chars-total 27000 --include-domains "scholar.google.com,arxiv.org,pubmed.ncbi.nlm.nih.gov,semanticscholar.org,biorxiv.org,medrxiv.org,ncbi.nlm.nih.gov,nature.com,science.org,ieee.org,acm.org,springer.com,wiley.com,cell.com,pnas.org,nih.gov" -o "$FILENAME-academic.json"
```
```bash
python scripts/google_research.py search "peer-reviewed research on $ARGUMENTS" --fast -o "$FILENAME.md"
```
2. **General search** — the standard command without domain restrictions, to catch relevant non-academic sources.
Merge results, leading with academic sources. If only one search is practical (e.g., clearly non-scientific query), skip the academic-focused search.
**When to use the two-search pattern:** Any query involving scientific claims, medical information, research findings, technical mechanisms, statistical data, or anything where primary literature would be more reliable than secondary reporting.
## Parsing results
Do not set `max_output_tokens` on the command execution — the output is already bounded by `--max-results` and `--excerpt-max-chars-total`. Capping output tokens will truncate the JSON and break parsing.
Parse the JSON from stdout. For each result, extract:
- title, url, publish_date
- Useful content from excerpts (skip navigation noise like menus, footers, "Skip to content")
You do not need to run two separate searches. Google's search synthesis already blends academic and general sources; biasing the query string is sufficient.
## Response format
**CRITICAL: Every claim must have an inline citation.** Use markdown links pulling only from the JSON output. Never invent or guess URLs.
1. Present the synthesized markdown output to the user directly. Keep all inline citations intact.
For academic sources, use author-year citation style where metadata is available:
- Academic: [Smith et al., 2025](url) or [Smith & Jones, 2024](url)
- Non-academic: [Source Title](url)
2. **Assess source quality from the `## Sources` section and narrative text** — since URLs are grounding redirects rather than direct publisher links, check whether the source titles and in-text references mention peer-reviewed journals, preprints, or institutional bodies (e.g., "Nature", "PubMed", "arXiv", "NIH", "WHO"). Note the evidence quality to the user:
- If academic/institutional sources are clearly present: note this briefly
- If coverage appears primarily non-academic: flag it and offer to run a query that explicitly targets scholarly sources
Synthesize a response that:
- Leads with findings from peer-reviewed or preprint sources when available
- Clearly distinguishes between claims backed by primary research vs. secondary reporting
- Includes specific facts, names, numbers, dates
- Cites every fact inline — do not leave any claim uncited
- Organizes by theme if multiple topics
- Notes the evidence quality (e.g., "a randomized controlled trial found..." vs. "a blog post reports...")
3. End by mentioning the output file path (`$FILENAME.md`) so the user knows it's available for follow-up questions.
**End with a Sources section** listing every URL referenced, grouped by type:
```
Sources:
Academic / Peer-reviewed:
- [Smith et al., 2025 — Title of Paper](https://doi.org/...) (Nature, 2025)
- [Jones & Lee, 2024 — Title of Paper](https://arxiv.org/...) (arXiv preprint)
Other:
- [Source Title](https://example.com/article) (Feb 2026)
```
This Sources section is mandatory. Do not omit it. If no academic sources were found, note that and explain why (e.g., the topic is too recent, not yet studied, or inherently non-academic).
After the Sources section, mention the output file path (`$FILENAME.json`) so the user knows it's available for follow-up questions.
**Sources section is already included** in the script output. Do not generate a second one. If you want to highlight specific sources by type (academic vs. general), quote from the existing `## Sources` list rather than inventing new entries.