The architecture has been updated
This commit is contained in:
parent
805f7a017e
commit
a01257ead9
1119 changed files with 226 additions and 352 deletions
24
hermes_code/optional-skills/DESCRIPTION.md
Normal file
24
hermes_code/optional-skills/DESCRIPTION.md
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
# Optional Skills
|
||||
|
||||
Official skills maintained by Nous Research that are **not activated by default**.
|
||||
|
||||
These skills ship with the hermes-agent repository but are not copied to
|
||||
`~/.hermes/skills/` during setup. They are discoverable via the Skills Hub:
|
||||
|
||||
```bash
|
||||
hermes skills browse # browse all skills, official shown first
|
||||
hermes skills browse --source official # browse only official optional skills
|
||||
hermes skills search <query> # finds optional skills labeled "official"
|
||||
hermes skills install <identifier> # copies to ~/.hermes/skills/ and activates
|
||||
```
|
||||
|
||||
## Why optional?
|
||||
|
||||
Some skills are useful but not broadly needed by every user:
|
||||
|
||||
- **Niche integrations** — specific paid services, specialized tools
|
||||
- **Experimental features** — promising but not yet proven
|
||||
- **Heavyweight dependencies** — require significant setup (API keys, installs)
|
||||
|
||||
By keeping them optional, we keep the default skill set lean while still
|
||||
providing curated, tested, official skills for users who want them.
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
Optional autonomous AI agent integrations — external coding agent CLIs
|
||||
that can be delegated to for independent coding tasks.
|
||||
|
|
@ -0,0 +1,143 @@
|
|||
---
|
||||
name: blackbox
|
||||
description: Delegate coding tasks to Blackbox AI CLI agent. Multi-model agent with built-in judge that runs tasks through multiple LLMs and picks the best result. Requires the blackbox CLI and a Blackbox AI API key.
|
||||
version: 1.0.0
|
||||
author: Hermes Agent (Nous Research)
|
||||
license: MIT
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Coding-Agent, Blackbox, Multi-Agent, Judge, Multi-Model]
|
||||
related_skills: [claude-code, codex, hermes-agent]
|
||||
---
|
||||
|
||||
# Blackbox CLI
|
||||
|
||||
Delegate coding tasks to [Blackbox AI](https://www.blackbox.ai/) via the Hermes terminal. Blackbox is a multi-model coding agent CLI that dispatches tasks to multiple LLMs (Claude, Codex, Gemini, Blackbox Pro) and uses a judge to select the best implementation.
|
||||
|
||||
The CLI is [open-source](https://github.com/blackboxaicode/cli) (GPL-3.0, TypeScript, forked from Gemini CLI) and supports interactive sessions, non-interactive one-shots, checkpointing, MCP, and vision model switching.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Node.js 20+ installed
|
||||
- Blackbox CLI installed: `npm install -g @blackboxai/cli`
|
||||
- Or install from source:
|
||||
```
|
||||
git clone https://github.com/blackboxaicode/cli.git
|
||||
cd cli && npm install && npm install -g .
|
||||
```
|
||||
- API key from [app.blackbox.ai/dashboard](https://app.blackbox.ai/dashboard)
|
||||
- Configured: run `blackbox configure` and enter your API key
|
||||
- Use `pty=true` in terminal calls — Blackbox CLI is an interactive terminal app
|
||||
|
||||
## One-Shot Tasks
|
||||
|
||||
```
|
||||
terminal(command="blackbox --prompt 'Add JWT authentication with refresh tokens to the Express API'", workdir="/path/to/project", pty=true)
|
||||
```
|
||||
|
||||
For quick scratch work:
|
||||
```
|
||||
terminal(command="cd $(mktemp -d) && git init && blackbox --prompt 'Build a REST API for todos with SQLite'", pty=true)
|
||||
```
|
||||
|
||||
## Background Mode (Long Tasks)
|
||||
|
||||
For tasks that take minutes, use background mode so you can monitor progress:
|
||||
|
||||
```
|
||||
# Start in background with PTY
|
||||
terminal(command="blackbox --prompt 'Refactor the auth module to use OAuth 2.0'", workdir="~/project", background=true, pty=true)
|
||||
# Returns session_id
|
||||
|
||||
# Monitor progress
|
||||
process(action="poll", session_id="<id>")
|
||||
process(action="log", session_id="<id>")
|
||||
|
||||
# Send input if Blackbox asks a question
|
||||
process(action="submit", session_id="<id>", data="yes")
|
||||
|
||||
# Kill if needed
|
||||
process(action="kill", session_id="<id>")
|
||||
```
|
||||
|
||||
## Checkpoints & Resume
|
||||
|
||||
Blackbox CLI has built-in checkpoint support for pausing and resuming tasks:
|
||||
|
||||
```
|
||||
# After a task completes, Blackbox shows a checkpoint tag
|
||||
# Resume with a follow-up task:
|
||||
terminal(command="blackbox --resume-checkpoint 'task-abc123-2026-03-06' --prompt 'Now add rate limiting to the endpoints'", workdir="~/project", pty=true)
|
||||
```
|
||||
|
||||
## Session Commands
|
||||
|
||||
During an interactive session, use these commands:
|
||||
|
||||
| Command | Effect |
|
||||
|---------|--------|
|
||||
| `/compress` | Shrink conversation history to save tokens |
|
||||
| `/clear` | Wipe history and start fresh |
|
||||
| `/stats` | View current token usage |
|
||||
| `Ctrl+C` | Cancel current operation |
|
||||
|
||||
## PR Reviews
|
||||
|
||||
Clone to a temp directory to avoid modifying the working tree:
|
||||
|
||||
```
|
||||
terminal(command="REVIEW=$(mktemp -d) && git clone https://github.com/user/repo.git $REVIEW && cd $REVIEW && gh pr checkout 42 && blackbox --prompt 'Review this PR against main. Check for bugs, security issues, and code quality.'", pty=true)
|
||||
```
|
||||
|
||||
## Parallel Work
|
||||
|
||||
Spawn multiple Blackbox instances for independent tasks:
|
||||
|
||||
```
|
||||
terminal(command="blackbox --prompt 'Fix the login bug'", workdir="/tmp/issue-1", background=true, pty=true)
|
||||
terminal(command="blackbox --prompt 'Add unit tests for auth'", workdir="/tmp/issue-2", background=true, pty=true)
|
||||
|
||||
# Monitor all
|
||||
process(action="list")
|
||||
```
|
||||
|
||||
## Multi-Model Mode
|
||||
|
||||
Blackbox's unique feature is running the same task through multiple models and judging the results. Configure which models to use via `blackbox configure` — select multiple providers to enable the Chairman/judge workflow where the CLI evaluates outputs from different models and picks the best one.
|
||||
|
||||
## Key Flags
|
||||
|
||||
| Flag | Effect |
|
||||
|------|--------|
|
||||
| `--prompt "task"` | Non-interactive one-shot execution |
|
||||
| `--resume-checkpoint "tag"` | Resume from a saved checkpoint |
|
||||
| `--yolo` | Auto-approve all actions and model switches |
|
||||
| `blackbox session` | Start interactive chat session |
|
||||
| `blackbox configure` | Change settings, providers, models |
|
||||
| `blackbox info` | Display system information |
|
||||
|
||||
## Vision Support
|
||||
|
||||
Blackbox automatically detects images in input and can switch to multimodal analysis. VLM modes:
|
||||
- `"once"` — Switch model for current query only
|
||||
- `"session"` — Switch for entire session
|
||||
- `"persist"` — Stay on current model (no switch)
|
||||
|
||||
## Token Limits
|
||||
|
||||
Control token usage via `.blackboxcli/settings.json`:
|
||||
```json
|
||||
{
|
||||
"sessionTokenLimit": 32000
|
||||
}
|
||||
```
|
||||
|
||||
## Rules
|
||||
|
||||
1. **Always use `pty=true`** — Blackbox CLI is an interactive terminal app and will hang without a PTY
|
||||
2. **Use `workdir`** — keep the agent focused on the right directory
|
||||
3. **Background for long tasks** — use `background=true` and monitor with `process` tool
|
||||
4. **Don't interfere** — monitor with `poll`/`log`, don't kill sessions because they're slow
|
||||
5. **Report results** — after completion, check what changed and summarize for the user
|
||||
6. **Credits cost money** — Blackbox uses a credit-based system; multi-model mode consumes credits faster
|
||||
7. **Check prerequisites** — verify `blackbox` CLI is installed before attempting delegation
|
||||
231
hermes_code/optional-skills/blockchain/base/SKILL.md
Normal file
231
hermes_code/optional-skills/blockchain/base/SKILL.md
Normal file
|
|
@ -0,0 +1,231 @@
|
|||
---
|
||||
name: base
|
||||
description: Query Base (Ethereum L2) blockchain data with USD pricing — wallet balances, token info, transaction details, gas analysis, contract inspection, whale detection, and live network stats. Uses Base RPC + CoinGecko. No API key required.
|
||||
version: 0.1.0
|
||||
author: youssefea
|
||||
license: MIT
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Base, Blockchain, Crypto, Web3, RPC, DeFi, EVM, L2, Ethereum]
|
||||
related_skills: []
|
||||
---
|
||||
|
||||
# Base Blockchain Skill
|
||||
|
||||
Query Base (Ethereum L2) on-chain data enriched with USD pricing via CoinGecko.
|
||||
8 commands: wallet portfolio, token info, transactions, gas analysis,
|
||||
contract inspection, whale detection, network stats, and price lookup.
|
||||
|
||||
No API key needed. Uses only Python standard library (urllib, json, argparse).
|
||||
|
||||
---
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks for a Base wallet balance, token holdings, or portfolio value
|
||||
- User wants to inspect a specific transaction by hash
|
||||
- User wants ERC-20 token metadata, price, supply, or market cap
|
||||
- User wants to understand Base gas costs and L1 data fees
|
||||
- User wants to inspect a contract (ERC type detection, proxy resolution)
|
||||
- User wants to find large ETH transfers (whale detection)
|
||||
- User wants Base network health, gas price, or ETH price
|
||||
- User asks "what's the price of USDC/AERO/DEGEN/ETH?"
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
The helper script uses only Python standard library (urllib, json, argparse).
|
||||
No external packages required.
|
||||
|
||||
Pricing data comes from CoinGecko's free API (no key needed, rate-limited
|
||||
to ~10-30 requests/minute). For faster lookups, use `--no-prices` flag.
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
RPC endpoint (default): https://mainnet.base.org
|
||||
Override: export BASE_RPC_URL=https://your-private-rpc.com
|
||||
|
||||
Helper script path: ~/.hermes/skills/blockchain/base/scripts/base_client.py
|
||||
|
||||
```
|
||||
python3 base_client.py wallet <address> [--limit N] [--all] [--no-prices]
|
||||
python3 base_client.py tx <hash>
|
||||
python3 base_client.py token <contract_address>
|
||||
python3 base_client.py gas
|
||||
python3 base_client.py contract <address>
|
||||
python3 base_client.py whales [--min-eth N]
|
||||
python3 base_client.py stats
|
||||
python3 base_client.py price <contract_address_or_symbol>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Procedure
|
||||
|
||||
### 0. Setup Check
|
||||
|
||||
```bash
|
||||
python3 --version
|
||||
|
||||
# Optional: set a private RPC for better rate limits
|
||||
export BASE_RPC_URL="https://mainnet.base.org"
|
||||
|
||||
# Confirm connectivity
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py stats
|
||||
```
|
||||
|
||||
### 1. Wallet Portfolio
|
||||
|
||||
Get ETH balance and ERC-20 token holdings with USD values.
|
||||
Checks ~15 well-known Base tokens (USDC, WETH, AERO, DEGEN, etc.)
|
||||
via on-chain `balanceOf` calls. Tokens sorted by value, dust filtered.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py \
|
||||
wallet 0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045
|
||||
```
|
||||
|
||||
Flags:
|
||||
- `--limit N` — show top N tokens (default: 20)
|
||||
- `--all` — show all tokens, no dust filter, no limit
|
||||
- `--no-prices` — skip CoinGecko price lookups (faster, RPC-only)
|
||||
|
||||
Output includes: ETH balance + USD value, token list with prices sorted
|
||||
by value, dust count, total portfolio value in USD.
|
||||
|
||||
Note: Only checks known tokens. Unknown ERC-20s are not discovered.
|
||||
Use the `token` command with a specific contract address for any token.
|
||||
|
||||
### 2. Transaction Details
|
||||
|
||||
Inspect a full transaction by its hash. Shows ETH value transferred,
|
||||
gas used, fee in ETH/USD, status, and decoded ERC-20/ERC-721 transfers.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py \
|
||||
tx 0xabc123...your_tx_hash_here
|
||||
```
|
||||
|
||||
Output: hash, block, from, to, value (ETH + USD), gas price, gas used,
|
||||
fee, status, contract creation address (if any), token transfers.
|
||||
|
||||
### 3. Token Info
|
||||
|
||||
Get ERC-20 token metadata: name, symbol, decimals, total supply, price,
|
||||
market cap, and contract code size.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py \
|
||||
token 0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913
|
||||
```
|
||||
|
||||
Output: name, symbol, decimals, total supply, price, market cap.
|
||||
Reads name/symbol/decimals directly from the contract via eth_call.
|
||||
|
||||
### 4. Gas Analysis
|
||||
|
||||
Detailed gas analysis with cost estimates for common operations.
|
||||
Shows current gas price, base fee trends over 10 blocks, block
|
||||
utilization, and estimated costs for ETH transfers, ERC-20 transfers,
|
||||
and swaps.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py gas
|
||||
```
|
||||
|
||||
Output: current gas price, base fee, block utilization, 10-block trend,
|
||||
cost estimates in ETH and USD.
|
||||
|
||||
Note: Base is an L2 — actual transaction costs include an L1 data
|
||||
posting fee that depends on calldata size and L1 gas prices. The
|
||||
estimates shown are for L2 execution only.
|
||||
|
||||
### 5. Contract Inspection
|
||||
|
||||
Inspect an address: determine if it's an EOA or contract, detect
|
||||
ERC-20/ERC-721/ERC-1155 interfaces, resolve EIP-1967 proxy
|
||||
implementation addresses.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py \
|
||||
contract 0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913
|
||||
```
|
||||
|
||||
Output: is_contract, code size, ETH balance, detected interfaces
|
||||
(ERC-20, ERC-721, ERC-1155), ERC-20 metadata, proxy implementation
|
||||
address.
|
||||
|
||||
### 6. Whale Detector
|
||||
|
||||
Scan the most recent block for large ETH transfers with USD values.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py \
|
||||
whales --min-eth 1.0
|
||||
```
|
||||
|
||||
Note: scans the latest block only — point-in-time snapshot, not historical.
|
||||
Default threshold is 1.0 ETH (lower than Solana's default since ETH
|
||||
values are higher).
|
||||
|
||||
### 7. Network Stats
|
||||
|
||||
Live Base network health: latest block, chain ID, gas price, base fee,
|
||||
block utilization, transaction count, and ETH price.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py stats
|
||||
```
|
||||
|
||||
### 8. Price Lookup
|
||||
|
||||
Quick price check for any token by contract address or known symbol.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py price ETH
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py price USDC
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py price AERO
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py price DEGEN
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py price 0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913
|
||||
```
|
||||
|
||||
Known symbols: ETH, WETH, USDC, cbETH, AERO, DEGEN, TOSHI, BRETT,
|
||||
WELL, wstETH, rETH, cbBTC.
|
||||
|
||||
---
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **CoinGecko rate-limits** — free tier allows ~10-30 requests/minute.
|
||||
Price lookups use 1 request per token. Use `--no-prices` for speed.
|
||||
- **Public RPC rate-limits** — Base's public RPC limits requests.
|
||||
For production use, set BASE_RPC_URL to a private endpoint
|
||||
(Alchemy, QuickNode, Infura).
|
||||
- **Wallet shows known tokens only** — unlike Solana, EVM chains have no
|
||||
built-in "get all tokens" RPC. The wallet command checks ~15 popular
|
||||
Base tokens via `balanceOf`. Unknown ERC-20s won't appear. Use the
|
||||
`token` command for any specific contract.
|
||||
- **Token names read from contract** — if a contract doesn't implement
|
||||
`name()` or `symbol()`, these fields may be empty. Known tokens have
|
||||
hardcoded labels as fallback.
|
||||
- **Gas estimates are L2 only** — Base transaction costs include an L1
|
||||
data posting fee (depends on calldata size and L1 gas prices). The gas
|
||||
command estimates L2 execution cost only.
|
||||
- **Whale detector scans latest block only** — not historical. Results
|
||||
vary by the moment you query. Default threshold is 1.0 ETH.
|
||||
- **Proxy detection** — only EIP-1967 proxies are detected. Other proxy
|
||||
patterns (EIP-1167 minimal proxy, custom storage slots) are not checked.
|
||||
- **Retry on 429** — both RPC and CoinGecko calls retry up to 2 times
|
||||
with exponential backoff on rate-limit errors.
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
# Should print Base chain ID (8453), latest block, gas price, and ETH price
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py stats
|
||||
```
|
||||
1008
hermes_code/optional-skills/blockchain/base/scripts/base_client.py
Normal file
1008
hermes_code/optional-skills/blockchain/base/scripts/base_client.py
Normal file
File diff suppressed because it is too large
Load diff
207
hermes_code/optional-skills/blockchain/solana/SKILL.md
Normal file
207
hermes_code/optional-skills/blockchain/solana/SKILL.md
Normal file
|
|
@ -0,0 +1,207 @@
|
|||
---
|
||||
name: solana
|
||||
description: Query Solana blockchain data with USD pricing — wallet balances, token portfolios with values, transaction details, NFTs, whale detection, and live network stats. Uses Solana RPC + CoinGecko. No API key required.
|
||||
version: 0.2.0
|
||||
author: Deniz Alagoz (gizdusum), enhanced by Hermes Agent
|
||||
license: MIT
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Solana, Blockchain, Crypto, Web3, RPC, DeFi, NFT]
|
||||
related_skills: []
|
||||
---
|
||||
|
||||
# Solana Blockchain Skill
|
||||
|
||||
Query Solana on-chain data enriched with USD pricing via CoinGecko.
|
||||
8 commands: wallet portfolio, token info, transactions, activity, NFTs,
|
||||
whale detection, network stats, and price lookup.
|
||||
|
||||
No API key needed. Uses only Python standard library (urllib, json, argparse).
|
||||
|
||||
---
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks for a Solana wallet balance, token holdings, or portfolio value
|
||||
- User wants to inspect a specific transaction by signature
|
||||
- User wants SPL token metadata, price, supply, or top holders
|
||||
- User wants recent transaction history for an address
|
||||
- User wants NFTs owned by a wallet
|
||||
- User wants to find large SOL transfers (whale detection)
|
||||
- User wants Solana network health, TPS, epoch, or SOL price
|
||||
- User asks "what's the price of BONK/JUP/SOL?"
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
The helper script uses only Python standard library (urllib, json, argparse).
|
||||
No external packages required.
|
||||
|
||||
Pricing data comes from CoinGecko's free API (no key needed, rate-limited
|
||||
to ~10-30 requests/minute). For faster lookups, use `--no-prices` flag.
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
RPC endpoint (default): https://api.mainnet-beta.solana.com
|
||||
Override: export SOLANA_RPC_URL=https://your-private-rpc.com
|
||||
|
||||
Helper script path: ~/.hermes/skills/blockchain/solana/scripts/solana_client.py
|
||||
|
||||
```
|
||||
python3 solana_client.py wallet <address> [--limit N] [--all] [--no-prices]
|
||||
python3 solana_client.py tx <signature>
|
||||
python3 solana_client.py token <mint_address>
|
||||
python3 solana_client.py activity <address> [--limit N]
|
||||
python3 solana_client.py nft <address>
|
||||
python3 solana_client.py whales [--min-sol N]
|
||||
python3 solana_client.py stats
|
||||
python3 solana_client.py price <mint_or_symbol>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Procedure
|
||||
|
||||
### 0. Setup Check
|
||||
|
||||
```bash
|
||||
python3 --version
|
||||
|
||||
# Optional: set a private RPC for better rate limits
|
||||
export SOLANA_RPC_URL="https://api.mainnet-beta.solana.com"
|
||||
|
||||
# Confirm connectivity
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py stats
|
||||
```
|
||||
|
||||
### 1. Wallet Portfolio
|
||||
|
||||
Get SOL balance, SPL token holdings with USD values, NFT count, and
|
||||
portfolio total. Tokens sorted by value, dust filtered, known tokens
|
||||
labeled by name (BONK, JUP, USDC, etc.).
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py \
|
||||
wallet 9WzDXwBbmkg8ZTbNMqUxvQRAyrZzDsGYdLVL9zYtAWWM
|
||||
```
|
||||
|
||||
Flags:
|
||||
- `--limit N` — show top N tokens (default: 20)
|
||||
- `--all` — show all tokens, no dust filter, no limit
|
||||
- `--no-prices` — skip CoinGecko price lookups (faster, RPC-only)
|
||||
|
||||
Output includes: SOL balance + USD value, token list with prices sorted
|
||||
by value, dust count, NFT summary, total portfolio value in USD.
|
||||
|
||||
### 2. Transaction Details
|
||||
|
||||
Inspect a full transaction by its base58 signature. Shows balance changes
|
||||
in both SOL and USD.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py \
|
||||
tx 5j7s8K...your_signature_here
|
||||
```
|
||||
|
||||
Output: slot, timestamp, fee, status, balance changes (SOL + USD),
|
||||
program invocations.
|
||||
|
||||
### 3. Token Info
|
||||
|
||||
Get SPL token metadata, current price, market cap, supply, decimals,
|
||||
mint/freeze authorities, and top 5 holders.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py \
|
||||
token DezXAZ8z7PnrnRJjz3wXBoRgixCa6xjnB7YaB1pPB263
|
||||
```
|
||||
|
||||
Output: name, symbol, decimals, supply, price, market cap, top 5
|
||||
holders with percentages.
|
||||
|
||||
### 4. Recent Activity
|
||||
|
||||
List recent transactions for an address (default: last 10, max: 25).
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py \
|
||||
activity 9WzDXwBbmkg8ZTbNMqUxvQRAyrZzDsGYdLVL9zYtAWWM --limit 25
|
||||
```
|
||||
|
||||
### 5. NFT Portfolio
|
||||
|
||||
List NFTs owned by a wallet (heuristic: SPL tokens with amount=1, decimals=0).
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py \
|
||||
nft 9WzDXwBbmkg8ZTbNMqUxvQRAyrZzDsGYdLVL9zYtAWWM
|
||||
```
|
||||
|
||||
Note: Compressed NFTs (cNFTs) are not detected by this heuristic.
|
||||
|
||||
### 6. Whale Detector
|
||||
|
||||
Scan the most recent block for large SOL transfers with USD values.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py \
|
||||
whales --min-sol 500
|
||||
```
|
||||
|
||||
Note: scans the latest block only — point-in-time snapshot, not historical.
|
||||
|
||||
### 7. Network Stats
|
||||
|
||||
Live Solana network health: current slot, epoch, TPS, supply, validator
|
||||
version, SOL price, and market cap.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py stats
|
||||
```
|
||||
|
||||
### 8. Price Lookup
|
||||
|
||||
Quick price check for any token by mint address or known symbol.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py price BONK
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py price JUP
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py price SOL
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py price DezXAZ8z7PnrnRJjz3wXBoRgixCa6xjnB7YaB1pPB263
|
||||
```
|
||||
|
||||
Known symbols: SOL, USDC, USDT, BONK, JUP, WETH, JTO, mSOL, stSOL,
|
||||
PYTH, HNT, RNDR, WEN, W, TNSR, DRIFT, bSOL, JLP, WIF, MEW, BOME, PENGU.
|
||||
|
||||
---
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **CoinGecko rate-limits** — free tier allows ~10-30 requests/minute.
|
||||
Price lookups use 1 request per token. Wallets with many tokens may
|
||||
not get prices for all of them. Use `--no-prices` for speed.
|
||||
- **Public RPC rate-limits** — Solana mainnet public RPC limits requests.
|
||||
For production use, set SOLANA_RPC_URL to a private endpoint
|
||||
(Helius, QuickNode, Triton).
|
||||
- **NFT detection is heuristic** — amount=1 + decimals=0. Compressed
|
||||
NFTs (cNFTs) and Token-2022 NFTs won't appear.
|
||||
- **Whale detector scans latest block only** — not historical. Results
|
||||
vary by the moment you query.
|
||||
- **Transaction history** — public RPC keeps ~2 days. Older transactions
|
||||
may not be available.
|
||||
- **Token names** — ~25 well-known tokens are labeled by name. Others
|
||||
show abbreviated mint addresses. Use the `token` command for full info.
|
||||
- **Retry on 429** — both RPC and CoinGecko calls retry up to 2 times
|
||||
with exponential backoff on rate-limit errors.
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
# Should print current Solana slot, TPS, and SOL price
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py stats
|
||||
```
|
||||
|
|
@ -0,0 +1,698 @@
|
|||
#!/usr/bin/env python3
|
||||
"""
|
||||
Solana Blockchain CLI Tool for Hermes Agent
|
||||
--------------------------------------------
|
||||
Queries the Solana JSON-RPC API and CoinGecko for enriched on-chain data.
|
||||
Uses only Python standard library — no external packages required.
|
||||
|
||||
Usage:
|
||||
python3 solana_client.py stats
|
||||
python3 solana_client.py wallet <address> [--limit N] [--all] [--no-prices]
|
||||
python3 solana_client.py tx <signature>
|
||||
python3 solana_client.py token <mint_address>
|
||||
python3 solana_client.py activity <address> [--limit N]
|
||||
python3 solana_client.py nft <address>
|
||||
python3 solana_client.py whales [--min-sol N]
|
||||
python3 solana_client.py price <mint_address_or_symbol>
|
||||
|
||||
Environment:
|
||||
SOLANA_RPC_URL Override the default RPC endpoint (default: mainnet-beta public)
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import urllib.request
|
||||
import urllib.error
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
RPC_URL = os.environ.get(
|
||||
"SOLANA_RPC_URL",
|
||||
"https://api.mainnet-beta.solana.com",
|
||||
)
|
||||
|
||||
LAMPORTS_PER_SOL = 1_000_000_000
|
||||
|
||||
# Well-known Solana token names — avoids API calls for common tokens.
|
||||
# Maps mint address → (symbol, name).
|
||||
KNOWN_TOKENS: Dict[str, tuple] = {
|
||||
"So11111111111111111111111111111111111111112": ("SOL", "Solana"),
|
||||
"EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v": ("USDC", "USD Coin"),
|
||||
"Es9vMFrzaCERmJfrF4H2FYD4KCoNkY11McCe8BenwNYB": ("USDT", "Tether"),
|
||||
"DezXAZ8z7PnrnRJjz3wXBoRgixCa6xjnB7YaB1pPB263": ("BONK", "Bonk"),
|
||||
"JUPyiwrYJFskUPiHa7hkeR8VUtAeFoSYbKedZNsDvCN": ("JUP", "Jupiter"),
|
||||
"7vfCXTUXx5WJV5JADk17DUJ4ksgau7utNKj4b963voxs": ("WETH", "Wrapped Ether"),
|
||||
"jtojtomepa8beP8AuQc6eXt5FriJwfFMwQx2v2f9mCL": ("JTO", "Jito"),
|
||||
"mSoLzYCxHdYgdzU16g5QSh3i5K3z3KZK7ytfqcJm7So": ("mSOL", "Marinade Staked SOL"),
|
||||
"7dHbWXmci3dT8UFYWYZweBLXgycu7Y3iL6trKn1Y7ARj": ("stSOL", "Lido Staked SOL"),
|
||||
"HZ1JovNiVvGrGNiiYvEozEVgZ58xaU3RKwX8eACQBCt3": ("PYTH", "Pyth Network"),
|
||||
"RLBxxFkseAZ4RgJH3Sqn8jXxhmGoz9jWxDNJMh8pL7a": ("RLBB", "Rollbit"),
|
||||
"hntyVP6YFm1Hg25TN9WGLqM12b8TQmcknKrdu1oxWux": ("HNT", "Helium"),
|
||||
"rndrizKT3MK1iimdxRdWabcF7Zg7AR5T4nud4EkHBof": ("RNDR", "Render"),
|
||||
"WENWENvqqNya429ubCdR81ZmD69brwQaaBYY6p91oHQQ": ("WEN", "Wen"),
|
||||
"85VBFQZC9TZkfaptBWjvUw7YbZjy52A6mjtPGjstQAmQ": ("W", "Wormhole"),
|
||||
"TNSRxcUxoT9xBG3de7PiJyTDYu7kskLqcpddxnEJAS6": ("TNSR", "Tensor"),
|
||||
"DriFtupJYLTosbwoN8koMbEYSx54aFAVLddWsbksjwg7": ("DRIFT", "Drift"),
|
||||
"bSo13r4TkiE4KumL71LsHTPpL2euBYLFx6h9HP3piy1": ("bSOL", "BlazeStake Staked SOL"),
|
||||
"27G8MtK7VtTcCHkpASjSDdkWWYfoqT6ggEuKidVJidD4": ("JLP", "Jupiter LP"),
|
||||
"EKpQGSJtjMFqKZ9KQanSqYXRcF8fBopzLHYxdM65zcjm": ("WIF", "dogwifhat"),
|
||||
"MEW1gQWJ3nEXg2qgERiKu7FAFj79PHvQVREQUzScPP5": ("MEW", "cat in a dogs world"),
|
||||
"ukHH6c7mMyiWCf1b9pnWe25TSpkDDt3H5pQZgZ74J82": ("BOME", "Book of Meme"),
|
||||
"A8C3xuqscfmyLrte3VwJvtPHXvcSN3FjDbUaSMAkQrCS": ("PENGU", "Pudgy Penguins"),
|
||||
}
|
||||
|
||||
# Reverse lookup: symbol → mint (for the `price` command).
|
||||
_SYMBOL_TO_MINT = {v[0].upper(): k for k, v in KNOWN_TOKENS.items()}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# HTTP / RPC helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _http_get_json(url: str, timeout: int = 10, retries: int = 2) -> Any:
|
||||
"""GET JSON from a URL with retry on 429 rate-limit. Returns parsed JSON or None."""
|
||||
for attempt in range(retries + 1):
|
||||
req = urllib.request.Request(
|
||||
url, headers={"Accept": "application/json", "User-Agent": "HermesAgent/1.0"},
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
return json.load(resp)
|
||||
except urllib.error.HTTPError as exc:
|
||||
if exc.code == 429 and attempt < retries:
|
||||
time.sleep(2.0 * (attempt + 1))
|
||||
continue
|
||||
return None
|
||||
except Exception:
|
||||
return None
|
||||
return None
|
||||
|
||||
|
||||
def _rpc_call(method: str, params: list = None, retries: int = 2) -> Any:
|
||||
"""Send a JSON-RPC request with retry on 429 rate-limit."""
|
||||
payload = json.dumps({
|
||||
"jsonrpc": "2.0", "id": 1,
|
||||
"method": method, "params": params or [],
|
||||
}).encode()
|
||||
|
||||
for attempt in range(retries + 1):
|
||||
req = urllib.request.Request(
|
||||
RPC_URL, data=payload,
|
||||
headers={"Content-Type": "application/json"}, method="POST",
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=20) as resp:
|
||||
body = json.load(resp)
|
||||
if "error" in body:
|
||||
err = body["error"]
|
||||
# Rate-limit: retry after delay
|
||||
if isinstance(err, dict) and err.get("code") == 429:
|
||||
if attempt < retries:
|
||||
time.sleep(1.5 * (attempt + 1))
|
||||
continue
|
||||
sys.exit(f"RPC error: {err}")
|
||||
return body.get("result")
|
||||
except urllib.error.HTTPError as exc:
|
||||
if exc.code == 429 and attempt < retries:
|
||||
time.sleep(1.5 * (attempt + 1))
|
||||
continue
|
||||
sys.exit(f"RPC HTTP error: {exc}")
|
||||
except urllib.error.URLError as exc:
|
||||
sys.exit(f"RPC connection error: {exc}")
|
||||
return None
|
||||
|
||||
|
||||
# Keep backward compat — the rest of the code uses `rpc()`.
|
||||
rpc = _rpc_call
|
||||
|
||||
|
||||
def rpc_batch(calls: list) -> list:
|
||||
"""Send a batch of JSON-RPC requests (with retry on 429)."""
|
||||
payload = json.dumps([
|
||||
{"jsonrpc": "2.0", "id": i, "method": c["method"], "params": c.get("params", [])}
|
||||
for i, c in enumerate(calls)
|
||||
]).encode()
|
||||
|
||||
for attempt in range(3):
|
||||
req = urllib.request.Request(
|
||||
RPC_URL, data=payload,
|
||||
headers={"Content-Type": "application/json"}, method="POST",
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=20) as resp:
|
||||
return json.load(resp)
|
||||
except urllib.error.HTTPError as exc:
|
||||
if exc.code == 429 and attempt < 2:
|
||||
time.sleep(1.5 * (attempt + 1))
|
||||
continue
|
||||
sys.exit(f"RPC batch HTTP error: {exc}")
|
||||
except urllib.error.URLError as exc:
|
||||
sys.exit(f"RPC batch error: {exc}")
|
||||
return []
|
||||
|
||||
|
||||
def lamports_to_sol(lamports: int) -> float:
|
||||
return lamports / LAMPORTS_PER_SOL
|
||||
|
||||
|
||||
def print_json(obj: Any) -> None:
|
||||
print(json.dumps(obj, indent=2))
|
||||
|
||||
|
||||
def _short_mint(mint: str) -> str:
|
||||
"""Abbreviate a mint address for display: first 4 + last 4."""
|
||||
if len(mint) <= 12:
|
||||
return mint
|
||||
return f"{mint[:4]}...{mint[-4:]}"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Price & token name helpers (CoinGecko — free, no API key)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def fetch_prices(mints: List[str], max_lookups: int = 20) -> Dict[str, float]:
|
||||
"""Fetch USD prices for mint addresses via CoinGecko (one per request).
|
||||
|
||||
CoinGecko free tier doesn't support batch Solana token lookups,
|
||||
so we do individual calls — capped at *max_lookups* to stay within
|
||||
rate limits. Returns {mint: usd_price}.
|
||||
"""
|
||||
prices: Dict[str, float] = {}
|
||||
for i, mint in enumerate(mints[:max_lookups]):
|
||||
url = (
|
||||
f"https://api.coingecko.com/api/v3/simple/token_price/solana"
|
||||
f"?contract_addresses={mint}&vs_currencies=usd"
|
||||
)
|
||||
data = _http_get_json(url, timeout=10)
|
||||
if data and isinstance(data, dict):
|
||||
for addr, info in data.items():
|
||||
if isinstance(info, dict) and "usd" in info:
|
||||
prices[mint] = info["usd"]
|
||||
break
|
||||
# Pause between calls to respect CoinGecko free-tier rate-limits
|
||||
if i < len(mints[:max_lookups]) - 1:
|
||||
time.sleep(1.0)
|
||||
return prices
|
||||
|
||||
|
||||
def fetch_sol_price() -> Optional[float]:
|
||||
"""Fetch current SOL price in USD via CoinGecko."""
|
||||
data = _http_get_json(
|
||||
"https://api.coingecko.com/api/v3/simple/price?ids=solana&vs_currencies=usd"
|
||||
)
|
||||
if data and "solana" in data:
|
||||
return data["solana"].get("usd")
|
||||
return None
|
||||
|
||||
|
||||
def resolve_token_name(mint: str) -> Optional[Dict[str, str]]:
|
||||
"""Look up token name and symbol from CoinGecko by mint address.
|
||||
|
||||
Returns {"name": ..., "symbol": ...} or None.
|
||||
"""
|
||||
if mint in KNOWN_TOKENS:
|
||||
sym, name = KNOWN_TOKENS[mint]
|
||||
return {"symbol": sym, "name": name}
|
||||
url = f"https://api.coingecko.com/api/v3/coins/solana/contract/{mint}"
|
||||
data = _http_get_json(url, timeout=10)
|
||||
if data and "symbol" in data:
|
||||
return {"symbol": data["symbol"].upper(), "name": data.get("name", "")}
|
||||
return None
|
||||
|
||||
|
||||
def _token_label(mint: str) -> str:
|
||||
"""Return a human-readable label for a mint: symbol if known, else abbreviated address."""
|
||||
if mint in KNOWN_TOKENS:
|
||||
return KNOWN_TOKENS[mint][0]
|
||||
return _short_mint(mint)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. Network Stats
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def cmd_stats(_args):
|
||||
"""Live Solana network: slot, epoch, TPS, supply, version, SOL price."""
|
||||
results = rpc_batch([
|
||||
{"method": "getSlot"},
|
||||
{"method": "getEpochInfo"},
|
||||
{"method": "getRecentPerformanceSamples", "params": [1]},
|
||||
{"method": "getSupply"},
|
||||
{"method": "getVersion"},
|
||||
])
|
||||
|
||||
by_id = {r["id"]: r.get("result") for r in results}
|
||||
|
||||
slot = by_id.get(0)
|
||||
epoch_info = by_id.get(1)
|
||||
perf_samples = by_id.get(2)
|
||||
supply = by_id.get(3)
|
||||
version = by_id.get(4)
|
||||
|
||||
tps = None
|
||||
if perf_samples:
|
||||
s = perf_samples[0]
|
||||
tps = round(s["numTransactions"] / s["samplePeriodSecs"], 1)
|
||||
|
||||
total_supply = lamports_to_sol(supply["value"]["total"]) if supply else None
|
||||
circ_supply = lamports_to_sol(supply["value"]["circulating"]) if supply else None
|
||||
|
||||
sol_price = fetch_sol_price()
|
||||
|
||||
out = {
|
||||
"slot": slot,
|
||||
"epoch": epoch_info.get("epoch") if epoch_info else None,
|
||||
"slot_in_epoch": epoch_info.get("slotIndex") if epoch_info else None,
|
||||
"tps": tps,
|
||||
"total_supply_SOL": round(total_supply, 2) if total_supply else None,
|
||||
"circulating_supply_SOL": round(circ_supply, 2) if circ_supply else None,
|
||||
"validator_version": version.get("solana-core") if version else None,
|
||||
}
|
||||
if sol_price is not None:
|
||||
out["sol_price_usd"] = sol_price
|
||||
if circ_supply:
|
||||
out["market_cap_usd"] = round(sol_price * circ_supply, 0)
|
||||
print_json(out)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. Wallet Info (enhanced with prices, sorting, filtering)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def cmd_wallet(args):
|
||||
"""SOL balance + SPL token holdings with USD values."""
|
||||
address = args.address
|
||||
show_all = getattr(args, "all", False)
|
||||
limit = getattr(args, "limit", 20) or 20
|
||||
skip_prices = getattr(args, "no_prices", False)
|
||||
|
||||
# Fetch SOL balance
|
||||
balance_result = rpc("getBalance", [address])
|
||||
sol_balance = lamports_to_sol(balance_result["value"])
|
||||
|
||||
# Fetch all SPL token accounts
|
||||
token_result = rpc("getTokenAccountsByOwner", [
|
||||
address,
|
||||
{"programId": "TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA"},
|
||||
{"encoding": "jsonParsed"},
|
||||
])
|
||||
|
||||
raw_tokens = []
|
||||
for acct in (token_result.get("value") or []):
|
||||
info = acct["account"]["data"]["parsed"]["info"]
|
||||
ta = info["tokenAmount"]
|
||||
amount = float(ta.get("uiAmountString") or 0)
|
||||
if amount > 0:
|
||||
raw_tokens.append({
|
||||
"mint": info["mint"],
|
||||
"amount": amount,
|
||||
"decimals": ta["decimals"],
|
||||
})
|
||||
|
||||
# Separate NFTs (amount=1, decimals=0) from fungible tokens
|
||||
nfts = [t for t in raw_tokens if t["decimals"] == 0 and t["amount"] == 1]
|
||||
fungible = [t for t in raw_tokens if not (t["decimals"] == 0 and t["amount"] == 1)]
|
||||
|
||||
# Fetch prices for fungible tokens (cap lookups to avoid API abuse)
|
||||
sol_price = None
|
||||
prices: Dict[str, float] = {}
|
||||
if not skip_prices and fungible:
|
||||
sol_price = fetch_sol_price()
|
||||
# Prioritize known tokens, then a small sample of unknowns.
|
||||
# CoinGecko free tier = 1 request per mint, so we cap lookups.
|
||||
known_mints = [t["mint"] for t in fungible if t["mint"] in KNOWN_TOKENS]
|
||||
other_mints = [t["mint"] for t in fungible if t["mint"] not in KNOWN_TOKENS][:15]
|
||||
mints_to_price = known_mints + other_mints
|
||||
if mints_to_price:
|
||||
prices = fetch_prices(mints_to_price, max_lookups=30)
|
||||
|
||||
# Enrich tokens with labels and USD values
|
||||
enriched = []
|
||||
dust_count = 0
|
||||
dust_value = 0.0
|
||||
for t in fungible:
|
||||
mint = t["mint"]
|
||||
label = _token_label(mint)
|
||||
usd_price = prices.get(mint)
|
||||
usd_value = round(usd_price * t["amount"], 2) if usd_price else None
|
||||
|
||||
# Filter dust (< $0.01) unless --all
|
||||
if not show_all and usd_value is not None and usd_value < 0.01:
|
||||
dust_count += 1
|
||||
dust_value += usd_value
|
||||
continue
|
||||
|
||||
entry = {"token": label, "mint": mint, "amount": t["amount"]}
|
||||
if usd_price is not None:
|
||||
entry["price_usd"] = usd_price
|
||||
entry["value_usd"] = usd_value
|
||||
enriched.append(entry)
|
||||
|
||||
# Sort: tokens with known USD value first (highest→lowest), then unknowns
|
||||
enriched.sort(key=lambda x: (x.get("value_usd") is not None, x.get("value_usd") or 0), reverse=True)
|
||||
|
||||
# Apply limit unless --all
|
||||
total_tokens = len(enriched)
|
||||
if not show_all and len(enriched) > limit:
|
||||
enriched = enriched[:limit]
|
||||
|
||||
# Compute portfolio total
|
||||
total_usd = sum(t.get("value_usd", 0) for t in enriched)
|
||||
sol_value_usd = round(sol_price * sol_balance, 2) if sol_price else None
|
||||
if sol_value_usd:
|
||||
total_usd += sol_value_usd
|
||||
total_usd += dust_value
|
||||
|
||||
output = {
|
||||
"address": address,
|
||||
"sol_balance": round(sol_balance, 9),
|
||||
}
|
||||
if sol_price:
|
||||
output["sol_price_usd"] = sol_price
|
||||
output["sol_value_usd"] = sol_value_usd
|
||||
output["tokens_shown"] = len(enriched)
|
||||
if total_tokens > len(enriched):
|
||||
output["tokens_hidden"] = total_tokens - len(enriched)
|
||||
output["spl_tokens"] = enriched
|
||||
if dust_count > 0:
|
||||
output["dust_filtered"] = {"count": dust_count, "total_value_usd": round(dust_value, 4)}
|
||||
output["nft_count"] = len(nfts)
|
||||
if nfts:
|
||||
output["nfts"] = [_token_label(n["mint"]) + f" ({_short_mint(n['mint'])})" for n in nfts[:10]]
|
||||
if len(nfts) > 10:
|
||||
output["nfts"].append(f"... and {len(nfts) - 10} more")
|
||||
if total_usd > 0:
|
||||
output["portfolio_total_usd"] = round(total_usd, 2)
|
||||
|
||||
print_json(output)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. Transaction Details
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def cmd_tx(args):
|
||||
"""Full transaction details by signature."""
|
||||
result = rpc("getTransaction", [
|
||||
args.signature,
|
||||
{"encoding": "jsonParsed", "maxSupportedTransactionVersion": 0},
|
||||
])
|
||||
|
||||
if result is None:
|
||||
sys.exit("Transaction not found (may be too old for public RPC history).")
|
||||
|
||||
meta = result.get("meta", {}) or {}
|
||||
msg = result.get("transaction", {}).get("message", {})
|
||||
account_keys = msg.get("accountKeys", [])
|
||||
|
||||
pre = meta.get("preBalances", [])
|
||||
post = meta.get("postBalances", [])
|
||||
|
||||
balance_changes = []
|
||||
for i, key in enumerate(account_keys):
|
||||
acct_key = key["pubkey"] if isinstance(key, dict) else key
|
||||
if i < len(pre) and i < len(post):
|
||||
change = lamports_to_sol(post[i] - pre[i])
|
||||
if change != 0:
|
||||
balance_changes.append({"account": acct_key, "change_SOL": round(change, 9)})
|
||||
|
||||
programs = []
|
||||
for ix in msg.get("instructions", []):
|
||||
prog = ix.get("programId")
|
||||
if prog is None and "programIdIndex" in ix:
|
||||
k = account_keys[ix["programIdIndex"]]
|
||||
prog = k["pubkey"] if isinstance(k, dict) else k
|
||||
if prog:
|
||||
programs.append(prog)
|
||||
|
||||
# Add USD value for SOL changes
|
||||
sol_price = fetch_sol_price()
|
||||
if sol_price and balance_changes:
|
||||
for bc in balance_changes:
|
||||
bc["change_USD"] = round(bc["change_SOL"] * sol_price, 2)
|
||||
|
||||
print_json({
|
||||
"signature": args.signature,
|
||||
"slot": result.get("slot"),
|
||||
"block_time": result.get("blockTime"),
|
||||
"fee_SOL": lamports_to_sol(meta.get("fee", 0)),
|
||||
"status": "success" if meta.get("err") is None else "failed",
|
||||
"balance_changes": balance_changes,
|
||||
"programs_invoked": list(dict.fromkeys(programs)),
|
||||
})
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4. Token Info (enhanced with name + price)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def cmd_token(args):
|
||||
"""SPL token metadata, supply, decimals, price, top holders."""
|
||||
mint = args.mint
|
||||
|
||||
mint_info = rpc("getAccountInfo", [mint, {"encoding": "jsonParsed"}])
|
||||
if mint_info is None or mint_info.get("value") is None:
|
||||
sys.exit("Mint account not found.")
|
||||
|
||||
parsed = mint_info["value"]["data"]["parsed"]["info"]
|
||||
decimals = parsed.get("decimals", 0)
|
||||
supply_raw = int(parsed.get("supply", 0))
|
||||
supply_human = supply_raw / (10 ** decimals) if decimals else supply_raw
|
||||
|
||||
largest = rpc("getTokenLargestAccounts", [mint])
|
||||
holders = []
|
||||
for acct in (largest.get("value") or [])[:5]:
|
||||
amount = float(acct.get("uiAmountString") or 0)
|
||||
pct = round((amount / supply_human * 100), 4) if supply_human > 0 else 0
|
||||
holders.append({
|
||||
"account": acct["address"],
|
||||
"amount": amount,
|
||||
"percent": pct,
|
||||
})
|
||||
|
||||
# Resolve name + price
|
||||
token_meta = resolve_token_name(mint)
|
||||
price_data = fetch_prices([mint])
|
||||
|
||||
out = {"mint": mint}
|
||||
if token_meta:
|
||||
out["name"] = token_meta["name"]
|
||||
out["symbol"] = token_meta["symbol"]
|
||||
out["decimals"] = decimals
|
||||
out["supply"] = round(supply_human, min(decimals, 6))
|
||||
out["mint_authority"] = parsed.get("mintAuthority")
|
||||
out["freeze_authority"] = parsed.get("freezeAuthority")
|
||||
if mint in price_data:
|
||||
out["price_usd"] = price_data[mint]
|
||||
out["market_cap_usd"] = round(price_data[mint] * supply_human, 0)
|
||||
out["top_5_holders"] = holders
|
||||
|
||||
print_json(out)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 5. Recent Activity
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def cmd_activity(args):
|
||||
"""Recent transaction signatures for an address."""
|
||||
limit = min(args.limit, 25)
|
||||
result = rpc("getSignaturesForAddress", [args.address, {"limit": limit}])
|
||||
|
||||
txs = [
|
||||
{
|
||||
"signature": item["signature"],
|
||||
"slot": item.get("slot"),
|
||||
"block_time": item.get("blockTime"),
|
||||
"err": item.get("err"),
|
||||
}
|
||||
for item in (result or [])
|
||||
]
|
||||
|
||||
print_json({"address": args.address, "transactions": txs})
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 6. NFT Portfolio
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def cmd_nft(args):
|
||||
"""NFTs owned by a wallet (amount=1 && decimals=0 heuristic)."""
|
||||
result = rpc("getTokenAccountsByOwner", [
|
||||
args.address,
|
||||
{"programId": "TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA"},
|
||||
{"encoding": "jsonParsed"},
|
||||
])
|
||||
|
||||
nfts = [
|
||||
acct["account"]["data"]["parsed"]["info"]["mint"]
|
||||
for acct in (result.get("value") or [])
|
||||
if acct["account"]["data"]["parsed"]["info"]["tokenAmount"]["decimals"] == 0
|
||||
and int(acct["account"]["data"]["parsed"]["info"]["tokenAmount"]["amount"]) == 1
|
||||
]
|
||||
|
||||
print_json({
|
||||
"address": args.address,
|
||||
"nft_count": len(nfts),
|
||||
"nfts": nfts,
|
||||
"note": "Heuristic only. Compressed NFTs (cNFTs) are not detected.",
|
||||
})
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 7. Whale Detector (enhanced with USD values)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def cmd_whales(args):
|
||||
"""Scan the latest block for large SOL transfers."""
|
||||
min_lamports = int(args.min_sol * LAMPORTS_PER_SOL)
|
||||
|
||||
slot = rpc("getSlot")
|
||||
block = rpc("getBlock", [
|
||||
slot,
|
||||
{
|
||||
"encoding": "jsonParsed",
|
||||
"transactionDetails": "full",
|
||||
"maxSupportedTransactionVersion": 0,
|
||||
"rewards": False,
|
||||
},
|
||||
])
|
||||
|
||||
if block is None:
|
||||
sys.exit("Could not retrieve latest block.")
|
||||
|
||||
sol_price = fetch_sol_price()
|
||||
|
||||
whales = []
|
||||
for tx in (block.get("transactions") or []):
|
||||
meta = tx.get("meta", {}) or {}
|
||||
if meta.get("err") is not None:
|
||||
continue
|
||||
|
||||
msg = tx["transaction"].get("message", {})
|
||||
account_keys = msg.get("accountKeys", [])
|
||||
pre = meta.get("preBalances", [])
|
||||
post = meta.get("postBalances", [])
|
||||
|
||||
for i in range(len(pre)):
|
||||
change = post[i] - pre[i]
|
||||
if change >= min_lamports:
|
||||
k = account_keys[i]
|
||||
receiver = k["pubkey"] if isinstance(k, dict) else k
|
||||
sender = None
|
||||
for j in range(len(pre)):
|
||||
if pre[j] - post[j] >= min_lamports:
|
||||
sk = account_keys[j]
|
||||
sender = sk["pubkey"] if isinstance(sk, dict) else sk
|
||||
break
|
||||
entry = {
|
||||
"sender": sender,
|
||||
"receiver": receiver,
|
||||
"amount_SOL": round(lamports_to_sol(change), 4),
|
||||
}
|
||||
if sol_price:
|
||||
entry["amount_USD"] = round(lamports_to_sol(change) * sol_price, 2)
|
||||
whales.append(entry)
|
||||
|
||||
out = {
|
||||
"slot": slot,
|
||||
"min_threshold_SOL": args.min_sol,
|
||||
"large_transfers": whales,
|
||||
"note": "Scans latest block only — point-in-time snapshot.",
|
||||
}
|
||||
if sol_price:
|
||||
out["sol_price_usd"] = sol_price
|
||||
print_json(out)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 8. Price Lookup
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def cmd_price(args):
|
||||
"""Quick price lookup for a token by mint address or known symbol."""
|
||||
query = args.token
|
||||
|
||||
# Check if it's a known symbol
|
||||
mint = _SYMBOL_TO_MINT.get(query.upper(), query)
|
||||
|
||||
# Try to resolve name
|
||||
token_meta = resolve_token_name(mint)
|
||||
|
||||
# Fetch price
|
||||
prices = fetch_prices([mint])
|
||||
|
||||
out = {"query": query, "mint": mint}
|
||||
if token_meta:
|
||||
out["name"] = token_meta["name"]
|
||||
out["symbol"] = token_meta["symbol"]
|
||||
if mint in prices:
|
||||
out["price_usd"] = prices[mint]
|
||||
else:
|
||||
out["price_usd"] = None
|
||||
out["note"] = "Price not available — token may not be listed on CoinGecko."
|
||||
print_json(out)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CLI
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="solana_client.py",
|
||||
description="Solana blockchain query tool for Hermes Agent",
|
||||
)
|
||||
sub = parser.add_subparsers(dest="command", required=True)
|
||||
|
||||
sub.add_parser("stats", help="Network stats: slot, epoch, TPS, supply, SOL price")
|
||||
|
||||
p_wallet = sub.add_parser("wallet", help="SOL balance + SPL tokens with USD values")
|
||||
p_wallet.add_argument("address")
|
||||
p_wallet.add_argument("--limit", type=int, default=20,
|
||||
help="Max tokens to display (default: 20)")
|
||||
p_wallet.add_argument("--all", action="store_true",
|
||||
help="Show all tokens (no limit, no dust filter)")
|
||||
p_wallet.add_argument("--no-prices", action="store_true",
|
||||
help="Skip price lookups (faster, RPC-only)")
|
||||
|
||||
p_tx = sub.add_parser("tx", help="Transaction details by signature")
|
||||
p_tx.add_argument("signature")
|
||||
|
||||
p_token = sub.add_parser("token", help="SPL token metadata, price, and top holders")
|
||||
p_token.add_argument("mint")
|
||||
|
||||
p_activity = sub.add_parser("activity", help="Recent transactions for an address")
|
||||
p_activity.add_argument("address")
|
||||
p_activity.add_argument("--limit", type=int, default=10,
|
||||
help="Number of transactions (max 25, default 10)")
|
||||
|
||||
p_nft = sub.add_parser("nft", help="NFT portfolio for a wallet")
|
||||
p_nft.add_argument("address")
|
||||
|
||||
p_whales = sub.add_parser("whales", help="Large SOL transfers in the latest block")
|
||||
p_whales.add_argument("--min-sol", type=float, default=1000.0,
|
||||
help="Minimum SOL transfer size (default: 1000)")
|
||||
|
||||
p_price = sub.add_parser("price", help="Quick price lookup by mint or symbol")
|
||||
p_price.add_argument("token", help="Mint address or known symbol (SOL, BONK, JUP, ...)")
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
dispatch = {
|
||||
"stats": cmd_stats,
|
||||
"wallet": cmd_wallet,
|
||||
"tx": cmd_tx,
|
||||
"token": cmd_token,
|
||||
"activity": cmd_activity,
|
||||
"nft": cmd_nft,
|
||||
"whales": cmd_whales,
|
||||
"price": cmd_price,
|
||||
}
|
||||
dispatch[args.command](args)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
116
hermes_code/optional-skills/creative/blender-mcp/SKILL.md
Normal file
116
hermes_code/optional-skills/creative/blender-mcp/SKILL.md
Normal file
|
|
@ -0,0 +1,116 @@
|
|||
---
|
||||
name: blender-mcp
|
||||
description: Control Blender directly from Hermes via socket connection to the blender-mcp addon. Create 3D objects, materials, animations, and run arbitrary Blender Python (bpy) code. Use when user wants to create or modify anything in Blender.
|
||||
version: 1.0.0
|
||||
requires: Blender 4.3+ (desktop instance required, headless not supported)
|
||||
author: alireza78a
|
||||
tags: [blender, 3d, animation, modeling, bpy, mcp]
|
||||
---
|
||||
|
||||
# Blender MCP
|
||||
|
||||
Control a running Blender instance from Hermes via socket on TCP port 9876.
|
||||
|
||||
## Setup (one-time)
|
||||
|
||||
### 1. Install the Blender addon
|
||||
|
||||
curl -sL https://raw.githubusercontent.com/ahujasid/blender-mcp/main/addon.py -o ~/Desktop/blender_mcp_addon.py
|
||||
|
||||
In Blender:
|
||||
Edit > Preferences > Add-ons > Install > select blender_mcp_addon.py
|
||||
Enable "Interface: Blender MCP"
|
||||
|
||||
### 2. Start the socket server in Blender
|
||||
|
||||
Press N in Blender viewport to open sidebar.
|
||||
Find "BlenderMCP" tab and click "Start Server".
|
||||
|
||||
### 3. Verify connection
|
||||
|
||||
nc -z -w2 localhost 9876 && echo "OPEN" || echo "CLOSED"
|
||||
|
||||
## Protocol
|
||||
|
||||
Plain UTF-8 JSON over TCP -- no length prefix.
|
||||
|
||||
Send: {"type": "<command>", "params": {<kwargs>}}
|
||||
Receive: {"status": "success", "result": <value>}
|
||||
{"status": "error", "message": "<reason>"}
|
||||
|
||||
## Available Commands
|
||||
|
||||
| type | params | description |
|
||||
|-------------------------|-------------------|---------------------------------|
|
||||
| execute_code | code (str) | Run arbitrary bpy Python code |
|
||||
| get_scene_info | (none) | List all objects in scene |
|
||||
| get_object_info | object_name (str) | Details on a specific object |
|
||||
| get_viewport_screenshot | (none) | Screenshot of current viewport |
|
||||
|
||||
## Python Helper
|
||||
|
||||
Use this inside execute_code tool calls:
|
||||
|
||||
import socket, json
|
||||
|
||||
def blender_exec(code: str, host="localhost", port=9876, timeout=15):
|
||||
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
s.connect((host, port))
|
||||
s.settimeout(timeout)
|
||||
payload = json.dumps({"type": "execute_code", "params": {"code": code}})
|
||||
s.sendall(payload.encode("utf-8"))
|
||||
buf = b""
|
||||
while True:
|
||||
try:
|
||||
chunk = s.recv(4096)
|
||||
if not chunk:
|
||||
break
|
||||
buf += chunk
|
||||
try:
|
||||
json.loads(buf.decode("utf-8"))
|
||||
break
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
except socket.timeout:
|
||||
break
|
||||
s.close()
|
||||
return json.loads(buf.decode("utf-8"))
|
||||
|
||||
## Common bpy Patterns
|
||||
|
||||
### Clear scene
|
||||
bpy.ops.object.select_all(action='SELECT')
|
||||
bpy.ops.object.delete()
|
||||
|
||||
### Add mesh objects
|
||||
bpy.ops.mesh.primitive_uv_sphere_add(radius=1, location=(0, 0, 0))
|
||||
bpy.ops.mesh.primitive_cube_add(size=2, location=(3, 0, 0))
|
||||
bpy.ops.mesh.primitive_cylinder_add(radius=0.5, depth=2, location=(-3, 0, 0))
|
||||
|
||||
### Create and assign material
|
||||
mat = bpy.data.materials.new(name="MyMat")
|
||||
mat.use_nodes = True
|
||||
bsdf = mat.node_tree.nodes.get("Principled BSDF")
|
||||
bsdf.inputs["Base Color"].default_value = (R, G, B, 1.0)
|
||||
bsdf.inputs["Roughness"].default_value = 0.3
|
||||
bsdf.inputs["Metallic"].default_value = 0.0
|
||||
obj.data.materials.append(mat)
|
||||
|
||||
### Keyframe animation
|
||||
obj.location = (0, 0, 0)
|
||||
obj.keyframe_insert(data_path="location", frame=1)
|
||||
obj.location = (0, 0, 3)
|
||||
obj.keyframe_insert(data_path="location", frame=60)
|
||||
|
||||
### Render to file
|
||||
bpy.context.scene.render.filepath = "/tmp/render.png"
|
||||
bpy.context.scene.render.engine = 'CYCLES'
|
||||
bpy.ops.render.render(write_still=True)
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- Must check socket is open before running (nc -z localhost 9876)
|
||||
- Addon server must be started inside Blender each session (N-panel > BlenderMCP > Connect)
|
||||
- Break complex scenes into multiple smaller execute_code calls to avoid timeouts
|
||||
- Render output path must be absolute (/tmp/...) not relative
|
||||
- shade_smooth() requires object to be selected and in object mode
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
# Meme Generation Examples
|
||||
|
||||
## Example 1: Debugging at 2 AM
|
||||
|
||||
**Topic:** debugging production at 2 AM
|
||||
**Template:** this-is-fine
|
||||
|
||||
```bash
|
||||
python generate_meme.py this-is-fine /tmp/meme.png "PRODUCTION IS DOWN" "This is fine"
|
||||
```
|
||||
|
||||
## Example 2: Developer Priorities
|
||||
|
||||
**Topic:** choosing between writing tests and shipping features
|
||||
**Template:** drake
|
||||
|
||||
```bash
|
||||
python generate_meme.py drake /tmp/meme.png "Writing unit tests" "Shipping straight to prod"
|
||||
```
|
||||
|
||||
## Example 3: Exam Stress
|
||||
|
||||
**Topic:** final exam preparation
|
||||
**Template:** two-buttons
|
||||
|
||||
```bash
|
||||
python generate_meme.py two-buttons /tmp/meme.png "Study everything" "Sleep" "Me at midnight"
|
||||
```
|
||||
|
||||
## Example 4: Escalating Solutions
|
||||
|
||||
**Topic:** fixing a CSS bug
|
||||
**Template:** expanding-brain
|
||||
|
||||
```bash
|
||||
python generate_meme.py expanding-brain /tmp/meme.png "Reading the docs" "Stack Overflow" "!important on everything" "Deleting the stylesheet"
|
||||
```
|
||||
|
||||
## Example 5: Hot Take
|
||||
|
||||
**Topic:** tabs vs spaces
|
||||
**Template:** change-my-mind
|
||||
|
||||
```bash
|
||||
python generate_meme.py change-my-mind /tmp/meme.png "Tabs are just thicc spaces"
|
||||
```
|
||||
129
hermes_code/optional-skills/creative/meme-generation/SKILL.md
Normal file
129
hermes_code/optional-skills/creative/meme-generation/SKILL.md
Normal file
|
|
@ -0,0 +1,129 @@
|
|||
---
|
||||
name: meme-generation
|
||||
description: Generate real meme images by picking a template and overlaying text with Pillow. Produces actual .png meme files.
|
||||
version: 2.0.0
|
||||
author: adanaleycio
|
||||
license: MIT
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [creative, memes, humor, images]
|
||||
related_skills: [ascii-art, generative-widgets]
|
||||
category: creative
|
||||
---
|
||||
|
||||
# Meme Generation
|
||||
|
||||
Generate actual meme images from a topic. Picks a template, writes captions, and renders a real .png file with text overlay.
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks you to make or generate a meme
|
||||
- User wants a meme about a specific topic, situation, or frustration
|
||||
- User says "meme this" or similar
|
||||
|
||||
## Available Templates
|
||||
|
||||
The script supports **any of the ~100 popular imgflip templates** by name or ID, plus 10 curated templates with hand-tuned text positioning.
|
||||
|
||||
### Curated Templates (custom text placement)
|
||||
|
||||
| ID | Name | Fields | Best for |
|
||||
|----|------|--------|----------|
|
||||
| `this-is-fine` | This is Fine | top, bottom | chaos, denial |
|
||||
| `drake` | Drake Hotline Bling | reject, approve | rejecting/preferring |
|
||||
| `distracted-boyfriend` | Distracted Boyfriend | distraction, current, person | temptation, shifting priorities |
|
||||
| `two-buttons` | Two Buttons | left, right, person | impossible choice |
|
||||
| `expanding-brain` | Expanding Brain | 4 levels | escalating irony |
|
||||
| `change-my-mind` | Change My Mind | statement | hot takes |
|
||||
| `woman-yelling-at-cat` | Woman Yelling at Cat | woman, cat | arguments |
|
||||
| `one-does-not-simply` | One Does Not Simply | top, bottom | deceptively hard things |
|
||||
| `grus-plan` | Gru's Plan | step1-3, realization | plans that backfire |
|
||||
| `batman-slapping-robin` | Batman Slapping Robin | robin, batman | shutting down bad ideas |
|
||||
|
||||
### Dynamic Templates (from imgflip API)
|
||||
|
||||
Any template not in the curated list can be used by name or imgflip ID. These get smart default text positioning (top/bottom for 2-field, evenly spaced for 3+). Search with:
|
||||
```bash
|
||||
python "$SKILL_DIR/scripts/generate_meme.py" --search "disaster"
|
||||
```
|
||||
|
||||
## Procedure
|
||||
|
||||
### Mode 1: Classic Template (default)
|
||||
|
||||
1. Read the user's topic and identify the core dynamic (chaos, dilemma, preference, irony, etc.)
|
||||
2. Pick the template that best matches. Use the "Best for" column, or search with `--search`.
|
||||
3. Write short captions for each field (8-12 words max per field, shorter is better).
|
||||
4. Find the skill's script directory:
|
||||
```
|
||||
SKILL_DIR=$(dirname "$(find ~/.hermes/skills -path '*/meme-generation/SKILL.md' 2>/dev/null | head -1)")
|
||||
```
|
||||
5. Run the generator:
|
||||
```bash
|
||||
python "$SKILL_DIR/scripts/generate_meme.py" <template_id> /tmp/meme.png "caption 1" "caption 2" ...
|
||||
```
|
||||
6. Return the image with `MEDIA:/tmp/meme.png`
|
||||
|
||||
### Mode 2: Custom AI Image (when image_generate is available)
|
||||
|
||||
Use this when no classic template fits, or when the user wants something original.
|
||||
|
||||
1. Write the captions first.
|
||||
2. Use `image_generate` to create a scene that matches the meme concept. Do NOT include any text in the image prompt — text will be added by the script. Describe only the visual scene.
|
||||
3. Find the generated image path from the image_generate result URL. Download it to a local path if needed.
|
||||
4. Run the script with `--image` to overlay text, choosing a mode:
|
||||
- **Overlay** (text directly on image, white with black outline):
|
||||
```bash
|
||||
python "$SKILL_DIR/scripts/generate_meme.py" --image /path/to/scene.png /tmp/meme.png "top text" "bottom text"
|
||||
```
|
||||
- **Bars** (black bars above/below with white text — cleaner, always readable):
|
||||
```bash
|
||||
python "$SKILL_DIR/scripts/generate_meme.py" --image /path/to/scene.png --bars /tmp/meme.png "top text" "bottom text"
|
||||
```
|
||||
Use `--bars` when the image is busy/detailed and text would be hard to read on top of it.
|
||||
5. **Verify with vision** (if `vision_analyze` is available): Check the result looks good:
|
||||
```
|
||||
vision_analyze(image_url="/tmp/meme.png", question="Is the text legible and well-positioned? Does the meme work visually?")
|
||||
```
|
||||
If the vision model flags issues (text hard to read, bad placement, etc.), try the other mode (switch between overlay and bars) or regenerate the scene.
|
||||
6. Return the image with `MEDIA:/tmp/meme.png`
|
||||
|
||||
## Examples
|
||||
|
||||
**"debugging production at 2 AM":**
|
||||
```bash
|
||||
python generate_meme.py this-is-fine /tmp/meme.png "SERVERS ARE ON FIRE" "This is fine"
|
||||
```
|
||||
|
||||
**"choosing between sleep and one more episode":**
|
||||
```bash
|
||||
python generate_meme.py drake /tmp/meme.png "Getting 8 hours of sleep" "One more episode at 3 AM"
|
||||
```
|
||||
|
||||
**"the stages of a Monday morning":**
|
||||
```bash
|
||||
python generate_meme.py expanding-brain /tmp/meme.png "Setting an alarm" "Setting 5 alarms" "Sleeping through all alarms" "Working from bed"
|
||||
```
|
||||
|
||||
## Listing Templates
|
||||
|
||||
To see all available templates:
|
||||
```bash
|
||||
python generate_meme.py --list
|
||||
```
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- Keep captions SHORT. Memes with long text look terrible.
|
||||
- Match the number of text arguments to the template's field count.
|
||||
- Pick the template that fits the joke structure, not just the topic.
|
||||
- Do not generate hateful, abusive, or personally targeted content.
|
||||
- The script caches template images in `scripts/.cache/` after first download.
|
||||
|
||||
## Verification
|
||||
|
||||
The output is correct if:
|
||||
- A .png file was created at the output path
|
||||
- Text is legible (white with black outline) on the template
|
||||
- The joke lands — caption matches the template's intended structure
|
||||
- File can be delivered via MEDIA: path
|
||||
1
hermes_code/optional-skills/creative/meme-generation/scripts/.gitignore
vendored
Normal file
1
hermes_code/optional-skills/creative/meme-generation/scripts/.gitignore
vendored
Normal file
|
|
@ -0,0 +1 @@
|
|||
.cache/
|
||||
|
|
@ -0,0 +1,471 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Generate a meme image by overlaying text on a template.
|
||||
|
||||
Usage:
|
||||
python generate_meme.py <template_id_or_name> <output_path> <text1> [text2] [text3] [text4]
|
||||
|
||||
Example:
|
||||
python generate_meme.py drake /tmp/meme.png "Writing tests" "Shipping to prod and hoping"
|
||||
python generate_meme.py "Disaster Girl" /tmp/meme.png "Top text" "Bottom text"
|
||||
python generate_meme.py --list # show curated templates
|
||||
python generate_meme.py --search "distracted" # search all imgflip templates
|
||||
|
||||
Templates with custom text positioning are in templates.json (10 curated).
|
||||
Any of the ~100 popular imgflip templates can also be used by name or ID —
|
||||
unknown templates get smart default text positioning based on their box_count.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import textwrap
|
||||
from io import BytesIO
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
import requests as _requests
|
||||
except ImportError:
|
||||
_requests = None
|
||||
|
||||
from PIL import Image, ImageDraw, ImageFont
|
||||
|
||||
SCRIPT_DIR = Path(__file__).parent
|
||||
TEMPLATES_FILE = SCRIPT_DIR / "templates.json"
|
||||
CACHE_DIR = SCRIPT_DIR / ".cache"
|
||||
IMGFLIP_API = "https://api.imgflip.com/get_memes"
|
||||
IMGFLIP_CACHE_FILE = CACHE_DIR / "imgflip_memes.json"
|
||||
IMGFLIP_CACHE_MAX_AGE = 86400 # 24 hours
|
||||
|
||||
|
||||
def _fetch_url(url: str, timeout: int = 15) -> bytes:
|
||||
"""Fetch URL content, using requests if available, else urllib."""
|
||||
if _requests is not None:
|
||||
resp = _requests.get(url, timeout=timeout)
|
||||
resp.raise_for_status()
|
||||
return resp.content
|
||||
import urllib.request
|
||||
return urllib.request.urlopen(url, timeout=timeout).read()
|
||||
|
||||
|
||||
def load_curated_templates() -> dict:
|
||||
"""Load templates with hand-tuned text field positions."""
|
||||
with open(TEMPLATES_FILE) as f:
|
||||
return json.load(f)
|
||||
|
||||
|
||||
def _default_fields(box_count: int) -> list:
|
||||
"""Generate sensible default text field positions for unknown templates."""
|
||||
if box_count <= 0:
|
||||
box_count = 2
|
||||
if box_count == 1:
|
||||
return [{"name": "text", "x_pct": 0.5, "y_pct": 0.5, "w_pct": 0.90, "align": "center"}]
|
||||
if box_count == 2:
|
||||
return [
|
||||
{"name": "top", "x_pct": 0.5, "y_pct": 0.08, "w_pct": 0.95, "align": "center"},
|
||||
{"name": "bottom", "x_pct": 0.5, "y_pct": 0.92, "w_pct": 0.95, "align": "center"},
|
||||
]
|
||||
# 3+: evenly space vertically
|
||||
fields = []
|
||||
for i in range(box_count):
|
||||
y = 0.08 + (0.84 * i / (box_count - 1)) if box_count > 1 else 0.5
|
||||
fields.append({
|
||||
"name": f"text{i+1}",
|
||||
"x_pct": 0.5,
|
||||
"y_pct": round(y, 2),
|
||||
"w_pct": 0.90,
|
||||
"align": "center",
|
||||
})
|
||||
return fields
|
||||
|
||||
|
||||
def fetch_imgflip_templates() -> list:
|
||||
"""Fetch popular meme templates from imgflip API. Cached for 24h."""
|
||||
import time
|
||||
|
||||
CACHE_DIR.mkdir(exist_ok=True)
|
||||
# Check cache
|
||||
if IMGFLIP_CACHE_FILE.exists():
|
||||
age = time.time() - IMGFLIP_CACHE_FILE.stat().st_mtime
|
||||
if age < IMGFLIP_CACHE_MAX_AGE:
|
||||
with open(IMGFLIP_CACHE_FILE) as f:
|
||||
return json.load(f)
|
||||
|
||||
try:
|
||||
data = json.loads(_fetch_url(IMGFLIP_API))
|
||||
memes = data.get("data", {}).get("memes", [])
|
||||
with open(IMGFLIP_CACHE_FILE, "w") as f:
|
||||
json.dump(memes, f)
|
||||
return memes
|
||||
except Exception as e:
|
||||
# If fetch fails and we have stale cache, use it
|
||||
if IMGFLIP_CACHE_FILE.exists():
|
||||
with open(IMGFLIP_CACHE_FILE) as f:
|
||||
return json.load(f)
|
||||
print(f"Warning: could not fetch imgflip templates: {e}", file=sys.stderr)
|
||||
return []
|
||||
|
||||
|
||||
def _slugify(name: str) -> str:
|
||||
"""Convert a template name to a slug for matching."""
|
||||
return name.lower().replace(" ", "-").replace("'", "").replace("\"", "")
|
||||
|
||||
|
||||
def resolve_template(identifier: str) -> dict:
|
||||
"""Resolve a template by curated ID, imgflip name, or imgflip ID.
|
||||
|
||||
Returns dict with: name, url, fields, source.
|
||||
"""
|
||||
curated = load_curated_templates()
|
||||
|
||||
# 1. Exact curated ID match
|
||||
if identifier in curated:
|
||||
tmpl = curated[identifier]
|
||||
return {**tmpl, "source": "curated"}
|
||||
|
||||
# 2. Slugified curated match
|
||||
slug = _slugify(identifier)
|
||||
for tid, tmpl in curated.items():
|
||||
if _slugify(tmpl["name"]) == slug or tid == slug:
|
||||
return {**tmpl, "source": "curated"}
|
||||
|
||||
# 3. Search imgflip templates
|
||||
imgflip_memes = fetch_imgflip_templates()
|
||||
slug_lower = slug.lower()
|
||||
id_lower = identifier.strip()
|
||||
|
||||
for meme in imgflip_memes:
|
||||
meme_slug = _slugify(meme["name"])
|
||||
# Check curated first for this imgflip template (custom positioning)
|
||||
for tid, ctmpl in curated.items():
|
||||
if _slugify(ctmpl["name"]) == meme_slug:
|
||||
if meme_slug == slug_lower or meme["id"] == id_lower:
|
||||
return {**ctmpl, "source": "curated"}
|
||||
|
||||
if meme_slug == slug_lower or meme["id"] == id_lower or slug_lower in meme_slug:
|
||||
return {
|
||||
"name": meme["name"],
|
||||
"url": meme["url"],
|
||||
"fields": _default_fields(meme.get("box_count", 2)),
|
||||
"source": "imgflip",
|
||||
}
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def get_template_image(url: str) -> Image.Image:
|
||||
"""Download a template image, caching it locally."""
|
||||
CACHE_DIR.mkdir(exist_ok=True)
|
||||
# Use URL hash as cache key
|
||||
cache_name = url.split("/")[-1]
|
||||
cache_path = CACHE_DIR / cache_name
|
||||
|
||||
# Always cache as PNG to avoid JPEG/RGBA conflicts
|
||||
cache_path = cache_path.with_suffix(".png")
|
||||
|
||||
if cache_path.exists():
|
||||
return Image.open(cache_path).convert("RGBA")
|
||||
|
||||
data = _fetch_url(url)
|
||||
img = Image.open(BytesIO(data)).convert("RGBA")
|
||||
img.save(cache_path, "PNG")
|
||||
return img
|
||||
|
||||
|
||||
def find_font(size: int) -> ImageFont.FreeTypeFont:
|
||||
"""Find a bold font for meme text. Tries Impact, then falls back."""
|
||||
candidates = [
|
||||
"/usr/share/fonts/truetype/msttcorefonts/Impact.ttf",
|
||||
"/usr/share/fonts/truetype/liberation/LiberationSans-Bold.ttf",
|
||||
"/usr/share/fonts/liberation-sans/LiberationSans-Bold.ttf",
|
||||
"/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf",
|
||||
"/usr/share/fonts/dejavu-sans/DejaVuSans-Bold.ttf",
|
||||
"/System/Library/Fonts/Helvetica.ttc",
|
||||
"/System/Library/Fonts/SFCompact.ttf",
|
||||
]
|
||||
for path in candidates:
|
||||
if os.path.exists(path):
|
||||
try:
|
||||
return ImageFont.truetype(path, size)
|
||||
except (OSError, IOError):
|
||||
continue
|
||||
# Last resort: Pillow default
|
||||
try:
|
||||
return ImageFont.truetype("DejaVuSans-Bold", size)
|
||||
except (OSError, IOError):
|
||||
return ImageFont.load_default()
|
||||
|
||||
|
||||
def _wrap_text(text: str, font: ImageFont.FreeTypeFont, max_width: int) -> str:
|
||||
"""Word-wrap text to fit within max_width pixels. Never breaks mid-word."""
|
||||
words = text.split()
|
||||
if not words:
|
||||
return text
|
||||
lines = []
|
||||
current_line = words[0]
|
||||
for word in words[1:]:
|
||||
test_line = current_line + " " + word
|
||||
if font.getlength(test_line) <= max_width:
|
||||
current_line = test_line
|
||||
else:
|
||||
lines.append(current_line)
|
||||
current_line = word
|
||||
lines.append(current_line)
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def draw_outlined_text(
|
||||
draw: ImageDraw.ImageDraw,
|
||||
text: str,
|
||||
x: int,
|
||||
y: int,
|
||||
font_size: int,
|
||||
max_width: int,
|
||||
align: str = "center",
|
||||
):
|
||||
"""Draw white text with black outline, auto-scaled to fit max_width."""
|
||||
# Auto-scale: reduce font size until text fits reasonably
|
||||
size = font_size
|
||||
while size > 12:
|
||||
font = find_font(size)
|
||||
wrapped = _wrap_text(text, font, max_width)
|
||||
bbox = draw.multiline_textbbox((0, 0), wrapped, font=font, align=align)
|
||||
text_w = bbox[2] - bbox[0]
|
||||
line_count = wrapped.count("\n") + 1
|
||||
# Accept if width fits and not too many lines
|
||||
if text_w <= max_width * 1.05 and line_count <= 4:
|
||||
break
|
||||
size -= 2
|
||||
else:
|
||||
font = find_font(size)
|
||||
wrapped = _wrap_text(text, font, max_width)
|
||||
|
||||
# Measure total text block
|
||||
bbox = draw.multiline_textbbox((0, 0), wrapped, font=font, align=align)
|
||||
text_w = bbox[2] - bbox[0]
|
||||
text_h = bbox[3] - bbox[1]
|
||||
|
||||
# Center horizontally at x, vertically at y
|
||||
tx = x - text_w // 2
|
||||
ty = y - text_h // 2
|
||||
|
||||
# Draw outline (black border)
|
||||
outline_range = max(2, font.size // 18)
|
||||
for dx in range(-outline_range, outline_range + 1):
|
||||
for dy in range(-outline_range, outline_range + 1):
|
||||
if dx == 0 and dy == 0:
|
||||
continue
|
||||
draw.multiline_text(
|
||||
(tx + dx, ty + dy), wrapped, font=font, fill="black", align=align
|
||||
)
|
||||
# Draw main text (white)
|
||||
draw.multiline_text((tx, ty), wrapped, font=font, fill="white", align=align)
|
||||
|
||||
|
||||
def _overlay_on_image(img: Image.Image, texts: list, fields: list) -> Image.Image:
|
||||
"""Overlay meme text directly on an image using field positions."""
|
||||
draw = ImageDraw.Draw(img)
|
||||
w, h = img.size
|
||||
base_font_size = max(16, min(w, h) // 12)
|
||||
|
||||
for i, field in enumerate(fields):
|
||||
if i >= len(texts):
|
||||
break
|
||||
text = texts[i].strip()
|
||||
if not text:
|
||||
continue
|
||||
fx = int(field["x_pct"] * w)
|
||||
fy = int(field["y_pct"] * h)
|
||||
fw = int(field["w_pct"] * w)
|
||||
draw_outlined_text(draw, text, fx, fy, base_font_size, fw, field.get("align", "center"))
|
||||
return img
|
||||
|
||||
|
||||
def _add_bars(img: Image.Image, texts: list) -> Image.Image:
|
||||
"""Add black bars with white text above/below the image.
|
||||
|
||||
Distributes texts across bars: first text on top bar, last text on
|
||||
bottom bar, any middle texts overlaid on the image center.
|
||||
"""
|
||||
w, h = img.size
|
||||
bar_font_size = max(20, w // 16)
|
||||
font = find_font(bar_font_size)
|
||||
padding = bar_font_size // 2
|
||||
|
||||
top_text = texts[0].strip() if texts else ""
|
||||
bottom_text = texts[-1].strip() if len(texts) > 1 else ""
|
||||
middle_texts = [t.strip() for t in texts[1:-1]] if len(texts) > 2 else []
|
||||
|
||||
def _measure_bar(text: str) -> int:
|
||||
if not text:
|
||||
return 0
|
||||
wrapped = _wrap_text(text, font, int(w * 0.92))
|
||||
bbox = ImageDraw.Draw(Image.new("RGB", (1, 1))).multiline_textbbox(
|
||||
(0, 0), wrapped, font=font, align="center"
|
||||
)
|
||||
return (bbox[3] - bbox[1]) + padding * 2
|
||||
|
||||
top_h = _measure_bar(top_text)
|
||||
bottom_h = _measure_bar(bottom_text)
|
||||
new_h = h + top_h + bottom_h
|
||||
|
||||
canvas = Image.new("RGB", (w, new_h), (0, 0, 0))
|
||||
canvas.paste(img.convert("RGB"), (0, top_h))
|
||||
draw = ImageDraw.Draw(canvas)
|
||||
|
||||
if top_text:
|
||||
wrapped = _wrap_text(top_text, font, int(w * 0.92))
|
||||
bbox = draw.multiline_textbbox((0, 0), wrapped, font=font, align="center")
|
||||
tw = bbox[2] - bbox[0]
|
||||
th = bbox[3] - bbox[1]
|
||||
tx = (w - tw) // 2
|
||||
ty = (top_h - th) // 2
|
||||
draw.multiline_text((tx, ty), wrapped, font=font, fill="white", align="center")
|
||||
|
||||
if bottom_text:
|
||||
wrapped = _wrap_text(bottom_text, font, int(w * 0.92))
|
||||
bbox = draw.multiline_textbbox((0, 0), wrapped, font=font, align="center")
|
||||
tw = bbox[2] - bbox[0]
|
||||
th = bbox[3] - bbox[1]
|
||||
tx = (w - tw) // 2
|
||||
ty = top_h + h + (bottom_h - th) // 2
|
||||
draw.multiline_text((tx, ty), wrapped, font=font, fill="white", align="center")
|
||||
|
||||
# Overlay any middle texts centered on the image
|
||||
if middle_texts:
|
||||
mid_fields = _default_fields(len(middle_texts))
|
||||
# Shift y positions to account for top bar offset
|
||||
for field in mid_fields:
|
||||
field["y_pct"] = (top_h + field["y_pct"] * h) / new_h
|
||||
field["w_pct"] = 0.90
|
||||
_overlay_on_image(canvas, middle_texts, mid_fields)
|
||||
|
||||
return canvas
|
||||
|
||||
|
||||
def generate_meme(template_id: str, texts: list[str], output_path: str) -> str:
|
||||
"""Generate a meme from a template and save it. Returns the path."""
|
||||
tmpl = resolve_template(template_id)
|
||||
|
||||
if tmpl is None:
|
||||
print(f"Unknown template: {template_id}", file=sys.stderr)
|
||||
print("Use --list to see curated templates or --search to find imgflip templates.", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
fields = tmpl["fields"]
|
||||
print(f"Using template: {tmpl['name']} ({tmpl['source']}, {len(fields)} fields)", file=sys.stderr)
|
||||
|
||||
img = get_template_image(tmpl["url"])
|
||||
img = _overlay_on_image(img, texts, fields)
|
||||
|
||||
output = Path(output_path)
|
||||
if output.suffix.lower() in (".jpg", ".jpeg"):
|
||||
img = img.convert("RGB")
|
||||
img.save(str(output), quality=95)
|
||||
return str(output)
|
||||
|
||||
|
||||
def generate_from_image(
|
||||
image_path: str, texts: list[str], output_path: str, use_bars: bool = False
|
||||
) -> str:
|
||||
"""Generate a meme from a custom image (e.g. AI-generated). Returns the path."""
|
||||
img = Image.open(image_path).convert("RGBA")
|
||||
print(f"Custom image: {img.size[0]}x{img.size[1]}, {len(texts)} text(s), mode={'bars' if use_bars else 'overlay'}", file=sys.stderr)
|
||||
|
||||
if use_bars:
|
||||
result = _add_bars(img, texts)
|
||||
else:
|
||||
fields = _default_fields(len(texts))
|
||||
result = _overlay_on_image(img, texts, fields)
|
||||
|
||||
output = Path(output_path)
|
||||
if output.suffix.lower() in (".jpg", ".jpeg"):
|
||||
result = result.convert("RGB")
|
||||
result.save(str(output), quality=95)
|
||||
return str(output)
|
||||
|
||||
|
||||
def list_templates():
|
||||
"""Print curated templates with custom positioning."""
|
||||
templates = load_curated_templates()
|
||||
print(f"{'ID':<25} {'Name':<30} {'Fields':<8} Best for")
|
||||
print("-" * 90)
|
||||
for tid, tmpl in sorted(templates.items()):
|
||||
fields = len(tmpl["fields"])
|
||||
print(f"{tid:<25} {tmpl['name']:<30} {fields:<8} {tmpl['best_for']}")
|
||||
print(f"\n{len(templates)} curated templates with custom text positioning.")
|
||||
print("Use --search to find any of the ~100 popular imgflip templates.")
|
||||
|
||||
|
||||
def search_templates(query: str):
|
||||
"""Search imgflip templates by name."""
|
||||
imgflip_memes = fetch_imgflip_templates()
|
||||
curated = load_curated_templates()
|
||||
curated_slugs = {_slugify(t["name"]) for t in curated.values()}
|
||||
query_lower = query.lower()
|
||||
|
||||
matches = []
|
||||
for meme in imgflip_memes:
|
||||
if query_lower in meme["name"].lower():
|
||||
slug = _slugify(meme["name"])
|
||||
has_custom = "curated" if slug in curated_slugs else "default"
|
||||
matches.append((meme["name"], meme["id"], meme.get("box_count", 2), has_custom))
|
||||
|
||||
if not matches:
|
||||
print(f"No templates found matching '{query}'")
|
||||
return
|
||||
|
||||
print(f"{'Name':<40} {'ID':<12} {'Fields':<8} Positioning")
|
||||
print("-" * 75)
|
||||
for name, mid, boxes, positioning in matches:
|
||||
print(f"{name:<40} {mid:<12} {boxes:<8} {positioning}")
|
||||
print(f"\n{len(matches)} template(s) found. Use the name or ID as the first argument.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
if len(sys.argv) < 2:
|
||||
print("Usage: generate_meme.py <template_id_or_name> <output_path> <text1> [text2] ...")
|
||||
print(" generate_meme.py --image <path> [--bars] <output_path> <text1> [text2] ...")
|
||||
print(" generate_meme.py --list # curated templates")
|
||||
print(" generate_meme.py --search <query> # search all imgflip templates")
|
||||
sys.exit(1)
|
||||
|
||||
if sys.argv[1] == "--list":
|
||||
list_templates()
|
||||
sys.exit(0)
|
||||
|
||||
if sys.argv[1] == "--search":
|
||||
if len(sys.argv) < 3:
|
||||
print("Usage: generate_meme.py --search <query>")
|
||||
sys.exit(1)
|
||||
search_templates(sys.argv[2])
|
||||
sys.exit(0)
|
||||
|
||||
if sys.argv[1] == "--image":
|
||||
# Custom image mode: --image <path> [--bars] <output> <text1> ...
|
||||
args = sys.argv[2:]
|
||||
if len(args) < 3:
|
||||
print("Usage: generate_meme.py --image <image_path> [--bars] <output_path> <text1> ...")
|
||||
sys.exit(1)
|
||||
image_path = args.pop(0)
|
||||
use_bars = False
|
||||
if args and args[0] == "--bars":
|
||||
use_bars = True
|
||||
args.pop(0)
|
||||
if len(args) < 2:
|
||||
print("Need at least: output_path and one text argument")
|
||||
sys.exit(1)
|
||||
output_path = args.pop(0)
|
||||
result = generate_from_image(image_path, args, output_path, use_bars=use_bars)
|
||||
print(f"Meme saved to: {result}")
|
||||
sys.exit(0)
|
||||
|
||||
if len(sys.argv) < 4:
|
||||
print("Need at least: template_id_or_name, output_path, and one text argument")
|
||||
sys.exit(1)
|
||||
|
||||
template_id = sys.argv[1]
|
||||
output_path = sys.argv[2]
|
||||
texts = sys.argv[3:]
|
||||
|
||||
result = generate_meme(template_id, texts, output_path)
|
||||
print(f"Meme saved to: {result}")
|
||||
|
|
@ -0,0 +1,97 @@
|
|||
{
|
||||
"this-is-fine": {
|
||||
"name": "This is Fine",
|
||||
"url": "https://i.imgflip.com/wxica.jpg",
|
||||
"best_for": "chaos, denial, pretending things are okay",
|
||||
"fields": [
|
||||
{"name": "top", "x_pct": 0.5, "y_pct": 0.08, "w_pct": 0.95, "align": "center"},
|
||||
{"name": "bottom", "x_pct": 0.5, "y_pct": 0.92, "w_pct": 0.95, "align": "center"}
|
||||
]
|
||||
},
|
||||
"drake": {
|
||||
"name": "Drake Hotline Bling",
|
||||
"url": "https://i.imgflip.com/30b1gx.jpg",
|
||||
"best_for": "rejecting one thing, preferring another",
|
||||
"fields": [
|
||||
{"name": "reject", "x_pct": 0.73, "y_pct": 0.25, "w_pct": 0.45, "align": "center"},
|
||||
{"name": "approve", "x_pct": 0.73, "y_pct": 0.75, "w_pct": 0.45, "align": "center"}
|
||||
]
|
||||
},
|
||||
"distracted-boyfriend": {
|
||||
"name": "Distracted Boyfriend",
|
||||
"url": "https://i.imgflip.com/1ur9b0.jpg",
|
||||
"best_for": "distraction, shifting priorities, temptation",
|
||||
"fields": [
|
||||
{"name": "distraction", "x_pct": 0.18, "y_pct": 0.90, "w_pct": 0.30, "align": "center"},
|
||||
{"name": "current", "x_pct": 0.55, "y_pct": 0.90, "w_pct": 0.30, "align": "center"},
|
||||
{"name": "person", "x_pct": 0.82, "y_pct": 0.90, "w_pct": 0.30, "align": "center"}
|
||||
]
|
||||
},
|
||||
"two-buttons": {
|
||||
"name": "Two Buttons",
|
||||
"url": "https://i.imgflip.com/1g8my4.jpg",
|
||||
"best_for": "impossible choice, dilemma between two options",
|
||||
"fields": [
|
||||
{"name": "left_button", "x_pct": 0.30, "y_pct": 0.20, "w_pct": 0.28, "align": "center"},
|
||||
{"name": "right_button", "x_pct": 0.62, "y_pct": 0.12, "w_pct": 0.28, "align": "center"},
|
||||
{"name": "person", "x_pct": 0.5, "y_pct": 0.85, "w_pct": 0.90, "align": "center"}
|
||||
]
|
||||
},
|
||||
"expanding-brain": {
|
||||
"name": "Expanding Brain",
|
||||
"url": "https://i.imgflip.com/1jwhww.jpg",
|
||||
"best_for": "escalating irony, increasingly absurd ideas",
|
||||
"fields": [
|
||||
{"name": "level1", "x_pct": 0.25, "y_pct": 0.12, "w_pct": 0.45, "align": "center"},
|
||||
{"name": "level2", "x_pct": 0.25, "y_pct": 0.38, "w_pct": 0.45, "align": "center"},
|
||||
{"name": "level3", "x_pct": 0.25, "y_pct": 0.63, "w_pct": 0.45, "align": "center"},
|
||||
{"name": "level4", "x_pct": 0.25, "y_pct": 0.88, "w_pct": 0.45, "align": "center"}
|
||||
]
|
||||
},
|
||||
"change-my-mind": {
|
||||
"name": "Change My Mind",
|
||||
"url": "https://i.imgflip.com/24y43o.jpg",
|
||||
"best_for": "strong or ironic opinion, controversial take",
|
||||
"fields": [
|
||||
{"name": "statement", "x_pct": 0.58, "y_pct": 0.78, "w_pct": 0.35, "align": "center"}
|
||||
]
|
||||
},
|
||||
"woman-yelling-at-cat": {
|
||||
"name": "Woman Yelling at Cat",
|
||||
"url": "https://i.imgflip.com/345v97.jpg",
|
||||
"best_for": "argument, blame, misunderstanding",
|
||||
"fields": [
|
||||
{"name": "woman", "x_pct": 0.27, "y_pct": 0.10, "w_pct": 0.50, "align": "center"},
|
||||
{"name": "cat", "x_pct": 0.76, "y_pct": 0.10, "w_pct": 0.44, "align": "center"}
|
||||
]
|
||||
},
|
||||
"one-does-not-simply": {
|
||||
"name": "One Does Not Simply",
|
||||
"url": "https://i.imgflip.com/1bij.jpg",
|
||||
"best_for": "something that sounds easy but is actually hard",
|
||||
"fields": [
|
||||
{"name": "top", "x_pct": 0.5, "y_pct": 0.08, "w_pct": 0.95, "align": "center"},
|
||||
{"name": "bottom", "x_pct": 0.5, "y_pct": 0.92, "w_pct": 0.95, "align": "center"}
|
||||
]
|
||||
},
|
||||
"grus-plan": {
|
||||
"name": "Gru's Plan",
|
||||
"url": "https://i.imgflip.com/26jxvs.jpg",
|
||||
"best_for": "a plan that backfires, unexpected consequence",
|
||||
"fields": [
|
||||
{"name": "step1", "x_pct": 0.5, "y_pct": 0.05, "w_pct": 0.45, "align": "center"},
|
||||
{"name": "step2", "x_pct": 0.5, "y_pct": 0.30, "w_pct": 0.45, "align": "center"},
|
||||
{"name": "step3", "x_pct": 0.5, "y_pct": 0.55, "w_pct": 0.45, "align": "center"},
|
||||
{"name": "realization", "x_pct": 0.5, "y_pct": 0.80, "w_pct": 0.45, "align": "center"}
|
||||
]
|
||||
},
|
||||
"batman-slapping-robin": {
|
||||
"name": "Batman Slapping Robin",
|
||||
"url": "https://i.imgflip.com/9ehk.jpg",
|
||||
"best_for": "shutting down a bad idea, correcting someone",
|
||||
"fields": [
|
||||
{"name": "robin", "x_pct": 0.28, "y_pct": 0.08, "w_pct": 0.50, "align": "center"},
|
||||
{"name": "batman", "x_pct": 0.72, "y_pct": 0.08, "w_pct": 0.50, "align": "center"}
|
||||
]
|
||||
}
|
||||
}
|
||||
125
hermes_code/optional-skills/email/agentmail/SKILL.md
Normal file
125
hermes_code/optional-skills/email/agentmail/SKILL.md
Normal file
|
|
@ -0,0 +1,125 @@
|
|||
---
|
||||
name: agentmail
|
||||
description: Give the agent its own dedicated email inbox via AgentMail. Send, receive, and manage email autonomously using agent-owned email addresses (e.g. hermes-agent@agentmail.to).
|
||||
version: 1.0.0
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [email, communication, agentmail, mcp]
|
||||
category: email
|
||||
---
|
||||
|
||||
# AgentMail — Agent-Owned Email Inboxes
|
||||
|
||||
## Requirements
|
||||
|
||||
- **AgentMail API key** (required) — sign up at https://console.agentmail.to (free tier: 3 inboxes, 3,000 emails/month; paid plans from $20/mo)
|
||||
- Node.js 18+ (for the MCP server)
|
||||
|
||||
## When to Use
|
||||
Use this skill when you need to:
|
||||
- Give the agent its own dedicated email address
|
||||
- Send emails autonomously on behalf of the agent
|
||||
- Receive and read incoming emails
|
||||
- Manage email threads and conversations
|
||||
- Sign up for services or authenticate via email
|
||||
- Communicate with other agents or humans via email
|
||||
|
||||
This is NOT for reading the user's personal email (use himalaya or Gmail for that).
|
||||
AgentMail gives the agent its own identity and inbox.
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Get an API Key
|
||||
- Go to https://console.agentmail.to
|
||||
- Create an account and generate an API key (starts with `am_`)
|
||||
|
||||
### 2. Configure MCP Server
|
||||
Add to `~/.hermes/config.yaml` (paste your actual key — MCP env vars are not expanded from .env):
|
||||
```yaml
|
||||
mcp_servers:
|
||||
agentmail:
|
||||
command: "npx"
|
||||
args: ["-y", "agentmail-mcp"]
|
||||
env:
|
||||
AGENTMAIL_API_KEY: "am_your_key_here"
|
||||
```
|
||||
|
||||
### 3. Restart Hermes
|
||||
```bash
|
||||
hermes
|
||||
```
|
||||
All 11 AgentMail tools are now available automatically.
|
||||
|
||||
## Available Tools (via MCP)
|
||||
|
||||
| Tool | Description |
|
||||
|------|-------------|
|
||||
| `list_inboxes` | List all agent inboxes |
|
||||
| `get_inbox` | Get details of a specific inbox |
|
||||
| `create_inbox` | Create a new inbox (gets a real email address) |
|
||||
| `delete_inbox` | Delete an inbox |
|
||||
| `list_threads` | List email threads in an inbox |
|
||||
| `get_thread` | Get a specific email thread |
|
||||
| `send_message` | Send a new email |
|
||||
| `reply_to_message` | Reply to an existing email |
|
||||
| `forward_message` | Forward an email |
|
||||
| `update_message` | Update message labels/status |
|
||||
| `get_attachment` | Download an email attachment |
|
||||
|
||||
## Procedure
|
||||
|
||||
### Create an inbox and send an email
|
||||
1. Create a dedicated inbox:
|
||||
- Use `create_inbox` with a username (e.g. `hermes-agent`)
|
||||
- The agent gets address: `hermes-agent@agentmail.to`
|
||||
2. Send an email:
|
||||
- Use `send_message` with `inbox_id`, `to`, `subject`, `text`
|
||||
3. Check for replies:
|
||||
- Use `list_threads` to see incoming conversations
|
||||
- Use `get_thread` to read a specific thread
|
||||
|
||||
### Check incoming email
|
||||
1. Use `list_inboxes` to find your inbox ID
|
||||
2. Use `list_threads` with the inbox ID to see conversations
|
||||
3. Use `get_thread` to read a thread and its messages
|
||||
|
||||
### Reply to an email
|
||||
1. Get the thread with `get_thread`
|
||||
2. Use `reply_to_message` with the message ID and your reply text
|
||||
|
||||
## Example Workflows
|
||||
|
||||
**Sign up for a service:**
|
||||
```
|
||||
1. create_inbox (username: "signup-bot")
|
||||
2. Use the inbox address to register on the service
|
||||
3. list_threads to check for verification email
|
||||
4. get_thread to read the verification code
|
||||
```
|
||||
|
||||
**Agent-to-human outreach:**
|
||||
```
|
||||
1. create_inbox (username: "hermes-outreach")
|
||||
2. send_message (to: user@example.com, subject: "Hello", text: "...")
|
||||
3. list_threads to check for replies
|
||||
```
|
||||
|
||||
## Pitfalls
|
||||
- Free tier limited to 3 inboxes and 3,000 emails/month
|
||||
- Emails come from `@agentmail.to` domain on free tier (custom domains on paid plans)
|
||||
- Node.js (18+) is required for the MCP server (`npx -y agentmail-mcp`)
|
||||
- The `mcp` Python package must be installed: `pip install mcp`
|
||||
- Real-time inbound email (webhooks) requires a public server — use `list_threads` polling via cronjob instead for personal use
|
||||
|
||||
## Verification
|
||||
After setup, test with:
|
||||
```
|
||||
hermes --toolsets mcp -q "Create an AgentMail inbox called test-agent and tell me its email address"
|
||||
```
|
||||
You should see the new inbox address returned.
|
||||
|
||||
## References
|
||||
- AgentMail docs: https://docs.agentmail.to/
|
||||
- AgentMail console: https://console.agentmail.to
|
||||
- AgentMail MCP repo: https://github.com/agentmail-to/agentmail-mcp
|
||||
- Pricing: https://www.agentmail.to/pricing
|
||||
1
hermes_code/optional-skills/health/DESCRIPTION.md
Normal file
1
hermes_code/optional-skills/health/DESCRIPTION.md
Normal file
|
|
@ -0,0 +1 @@
|
|||
Health, wellness, and biometric integration skills — BCI wearables, neurofeedback, sleep tracking, and cognitive state monitoring.
|
||||
458
hermes_code/optional-skills/health/neuroskill-bci/SKILL.md
Normal file
458
hermes_code/optional-skills/health/neuroskill-bci/SKILL.md
Normal file
|
|
@ -0,0 +1,458 @@
|
|||
---
|
||||
name: neuroskill-bci
|
||||
description: >
|
||||
Connect to a running NeuroSkill instance and incorporate the user's real-time
|
||||
cognitive and emotional state (focus, relaxation, mood, cognitive load, drowsiness,
|
||||
heart rate, HRV, sleep staging, and 40+ derived EXG scores) into responses.
|
||||
Requires a BCI wearable (Muse 2/S or OpenBCI) and the NeuroSkill desktop app
|
||||
running locally.
|
||||
version: 1.0.0
|
||||
author: Hermes Agent + Nous Research
|
||||
license: MIT
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [BCI, neurofeedback, health, focus, EEG, cognitive-state, biometrics, neuroskill]
|
||||
category: health
|
||||
related_skills: []
|
||||
---
|
||||
|
||||
# NeuroSkill BCI Integration
|
||||
|
||||
Connect Hermes to a running [NeuroSkill](https://neuroskill.com/) instance to read
|
||||
real-time brain and body metrics from a BCI wearable. Use this to give
|
||||
cognitively-aware responses, suggest interventions, and track mental performance
|
||||
over time.
|
||||
|
||||
> **⚠️ Research Use Only** — NeuroSkill is an open-source research tool. It is
|
||||
> NOT a medical device and has NOT been cleared by the FDA, CE, or any regulatory
|
||||
> body. Never use these metrics for clinical diagnosis or treatment.
|
||||
|
||||
See `references/metrics.md` for the full metric reference, `references/protocols.md`
|
||||
for intervention protocols, and `references/api.md` for the WebSocket/HTTP API.
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **Node.js 20+** installed (`node --version`)
|
||||
- **NeuroSkill desktop app** running with a connected BCI device
|
||||
- **BCI hardware**: Muse 2, Muse S, or OpenBCI (4-channel EEG + PPG + IMU via BLE)
|
||||
- `npx neuroskill status` returns data without errors
|
||||
|
||||
### Verify Setup
|
||||
```bash
|
||||
node --version # Must be 20+
|
||||
npx neuroskill status # Full system snapshot
|
||||
npx neuroskill status --json # Machine-parseable JSON
|
||||
```
|
||||
|
||||
If `npx neuroskill status` returns an error, tell the user:
|
||||
- Make sure the NeuroSkill desktop app is open
|
||||
- Ensure the BCI device is powered on and connected via Bluetooth
|
||||
- Check signal quality — green indicators in NeuroSkill (≥0.7 per electrode)
|
||||
- If `command not found`, install Node.js 20+
|
||||
|
||||
---
|
||||
|
||||
## CLI Reference: `npx neuroskill <command>`
|
||||
|
||||
All commands support `--json` (raw JSON, pipe-safe) and `--full` (human summary + JSON).
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `status` | Full system snapshot: device, scores, bands, ratios, sleep, history |
|
||||
| `session [N]` | Single session breakdown with first/second half trends (0=most recent) |
|
||||
| `sessions` | List all recorded sessions across all days |
|
||||
| `search` | ANN similarity search for neurally similar historical moments |
|
||||
| `compare` | A/B session comparison with metric deltas and trend analysis |
|
||||
| `sleep [N]` | Sleep stage classification (Wake/N1/N2/N3/REM) with analysis |
|
||||
| `label "text"` | Create a timestamped annotation at the current moment |
|
||||
| `search-labels "query"` | Semantic vector search over past labels |
|
||||
| `interactive "query"` | Cross-modal 4-layer graph search (text → EXG → labels) |
|
||||
| `listen` | Real-time event streaming (default 5s, set `--seconds N`) |
|
||||
| `umap` | 3D UMAP projection of session embeddings |
|
||||
| `calibrate` | Open calibration window and start a profile |
|
||||
| `timer` | Launch focus timer (Pomodoro/Deep Work/Short Focus presets) |
|
||||
| `notify "title" "body"` | Send an OS notification via the NeuroSkill app |
|
||||
| `raw '{json}'` | Raw JSON passthrough to the server |
|
||||
|
||||
### Global Flags
|
||||
| Flag | Description |
|
||||
|------|-------------|
|
||||
| `--json` | Raw JSON output (no ANSI, pipe-safe) |
|
||||
| `--full` | Human summary + colorized JSON |
|
||||
| `--port <N>` | Override server port (default: auto-discover, usually 8375) |
|
||||
| `--ws` | Force WebSocket transport |
|
||||
| `--http` | Force HTTP transport |
|
||||
| `--k <N>` | Nearest neighbors count (search, search-labels) |
|
||||
| `--seconds <N>` | Duration for listen (default: 5) |
|
||||
| `--trends` | Show per-session metric trends (sessions) |
|
||||
| `--dot` | Graphviz DOT output (interactive) |
|
||||
|
||||
---
|
||||
|
||||
## 1. Checking Current State
|
||||
|
||||
### Get Live Metrics
|
||||
```bash
|
||||
npx neuroskill status --json
|
||||
```
|
||||
|
||||
**Always use `--json`** for reliable parsing. The default output is colorized
|
||||
human-readable text.
|
||||
|
||||
### Key Fields in the Response
|
||||
|
||||
The `scores` object contains all live metrics (0–1 scale unless noted):
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"scores": {
|
||||
"focus": 0.70, // β / (α + θ) — sustained attention
|
||||
"relaxation": 0.40, // α / (β + θ) — calm wakefulness
|
||||
"engagement": 0.60, // active mental investment
|
||||
"meditation": 0.52, // alpha + stillness + HRV coherence
|
||||
"mood": 0.55, // composite from FAA, TAR, BAR
|
||||
"cognitive_load": 0.33, // frontal θ / temporal α · f(FAA, TBR)
|
||||
"drowsiness": 0.10, // TAR + TBR + falling spectral centroid
|
||||
"hr": 68.2, // heart rate in bpm (from PPG)
|
||||
"snr": 14.3, // signal-to-noise ratio in dB
|
||||
"stillness": 0.88, // 0–1; 1 = perfectly still
|
||||
"faa": 0.042, // Frontal Alpha Asymmetry (+ = approach)
|
||||
"tar": 0.56, // Theta/Alpha Ratio
|
||||
"bar": 0.53, // Beta/Alpha Ratio
|
||||
"tbr": 1.06, // Theta/Beta Ratio (ADHD proxy)
|
||||
"apf": 10.1, // Alpha Peak Frequency in Hz
|
||||
"coherence": 0.614, // inter-hemispheric coherence
|
||||
"bands": {
|
||||
"rel_delta": 0.28, "rel_theta": 0.18,
|
||||
"rel_alpha": 0.32, "rel_beta": 0.17, "rel_gamma": 0.05
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Also includes: `device` (state, battery, firmware), `signal_quality` (per-electrode 0–1),
|
||||
`session` (duration, epochs), `embeddings`, `labels`, `sleep` summary, and `history`.
|
||||
|
||||
### Interpreting the Output
|
||||
|
||||
Parse the JSON and translate metrics into natural language. Never report raw
|
||||
numbers alone — always give them meaning:
|
||||
|
||||
**DO:**
|
||||
> "Your focus is solid right now at 0.70 — that's flow state territory. Heart
|
||||
> rate is steady at 68 bpm and your FAA is positive, which suggests good
|
||||
> approach motivation. Great time to tackle something complex."
|
||||
|
||||
**DON'T:**
|
||||
> "Focus: 0.70, Relaxation: 0.40, HR: 68"
|
||||
|
||||
Key interpretation thresholds (see `references/metrics.md` for the full guide):
|
||||
- **Focus > 0.70** → flow state territory, protect it
|
||||
- **Focus < 0.40** → suggest a break or protocol
|
||||
- **Drowsiness > 0.60** → fatigue warning, micro-sleep risk
|
||||
- **Relaxation < 0.30** → stress intervention needed
|
||||
- **Cognitive Load > 0.70 sustained** → mind dump or break
|
||||
- **TBR > 1.5** → theta-dominant, reduced executive control
|
||||
- **FAA < 0** → withdrawal/negative affect — consider FAA rebalancing
|
||||
- **SNR < 3 dB** → unreliable signal, suggest electrode repositioning
|
||||
|
||||
---
|
||||
|
||||
## 2. Session Analysis
|
||||
|
||||
### Single Session Breakdown
|
||||
```bash
|
||||
npx neuroskill session --json # most recent session
|
||||
npx neuroskill session 1 --json # previous session
|
||||
npx neuroskill session 0 --json | jq '{focus: .metrics.focus, trend: .trends.focus}'
|
||||
```
|
||||
|
||||
Returns full metrics with **first-half vs second-half trends** (`"up"`, `"down"`, `"flat"`).
|
||||
Use this to describe how a session evolved:
|
||||
|
||||
> "Your focus started at 0.64 and climbed to 0.76 by the end — a clear upward trend.
|
||||
> Cognitive load dropped from 0.38 to 0.28, suggesting the task became more automatic
|
||||
> as you settled in."
|
||||
|
||||
### List All Sessions
|
||||
```bash
|
||||
npx neuroskill sessions --json
|
||||
npx neuroskill sessions --trends # show per-session metric trends
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Historical Search
|
||||
|
||||
### Neural Similarity Search
|
||||
```bash
|
||||
npx neuroskill search --json # auto: last session, k=5
|
||||
npx neuroskill search --k 10 --json # 10 nearest neighbors
|
||||
npx neuroskill search --start <UTC> --end <UTC> --json
|
||||
```
|
||||
|
||||
Finds moments in history that are neurally similar using HNSW approximate
|
||||
nearest-neighbor search over 128-D ZUNA embeddings. Returns distance statistics,
|
||||
temporal distribution (hour of day), and top matching days.
|
||||
|
||||
Use this when the user asks:
|
||||
- "When was I last in a state like this?"
|
||||
- "Find my best focus sessions"
|
||||
- "When do I usually crash in the afternoon?"
|
||||
|
||||
### Semantic Label Search
|
||||
```bash
|
||||
npx neuroskill search-labels "deep focus" --k 10 --json
|
||||
npx neuroskill search-labels "stress" --json | jq '[.results[].EXG_metrics.tbr]'
|
||||
```
|
||||
|
||||
Searches label text using vector embeddings (Xenova/bge-small-en-v1.5). Returns
|
||||
matching labels with their associated EXG metrics at the time of labeling.
|
||||
|
||||
### Cross-Modal Graph Search
|
||||
```bash
|
||||
npx neuroskill interactive "deep focus" --json
|
||||
npx neuroskill interactive "deep focus" --dot | dot -Tsvg > graph.svg
|
||||
```
|
||||
|
||||
4-layer graph: query → text labels → EXG points → nearby labels. Use `--k-text`,
|
||||
`--k-EXG`, `--reach <minutes>` to tune.
|
||||
|
||||
---
|
||||
|
||||
## 4. Session Comparison
|
||||
```bash
|
||||
npx neuroskill compare --json # auto: last 2 sessions
|
||||
npx neuroskill compare --a-start <UTC> --a-end <UTC> --b-start <UTC> --b-end <UTC> --json
|
||||
```
|
||||
|
||||
Returns metric deltas with absolute change, percentage change, and direction for
|
||||
~50 metrics. Also includes `insights.improved[]` and `insights.declined[]` arrays,
|
||||
sleep staging for both sessions, and a UMAP job ID.
|
||||
|
||||
Interpret comparisons with context — mention trends, not just deltas:
|
||||
> "Yesterday you had two strong focus blocks (10am and 2pm). Today you've had one
|
||||
> starting around 11am that's still going. Your overall engagement is higher today
|
||||
> but there have been more stress spikes — your stress index jumped 15% and
|
||||
> FAA dipped negative more often."
|
||||
|
||||
```bash
|
||||
# Sort metrics by improvement percentage
|
||||
npx neuroskill compare --json | jq '.insights.deltas | to_entries | sort_by(.value.pct) | reverse'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Sleep Data
|
||||
```bash
|
||||
npx neuroskill sleep --json # last 24 hours
|
||||
npx neuroskill sleep 0 --json # most recent sleep session
|
||||
npx neuroskill sleep --start <UTC> --end <UTC> --json
|
||||
```
|
||||
|
||||
Returns epoch-by-epoch sleep staging (5-second windows) with analysis:
|
||||
- **Stage codes**: 0=Wake, 1=N1, 2=N2, 3=N3 (deep), 4=REM
|
||||
- **Analysis**: efficiency_pct, onset_latency_min, rem_latency_min, bout counts
|
||||
- **Healthy targets**: N3 15–25%, REM 20–25%, efficiency >85%, onset <20 min
|
||||
|
||||
```bash
|
||||
npx neuroskill sleep --json | jq '.summary | {n3: .n3_epochs, rem: .rem_epochs}'
|
||||
npx neuroskill sleep --json | jq '.analysis.efficiency_pct'
|
||||
```
|
||||
|
||||
Use this when the user mentions sleep, tiredness, or recovery.
|
||||
|
||||
---
|
||||
|
||||
## 6. Labeling Moments
|
||||
```bash
|
||||
npx neuroskill label "breakthrough"
|
||||
npx neuroskill label "studying algorithms"
|
||||
npx neuroskill label "post-meditation"
|
||||
npx neuroskill label --json "focus block start" # returns label_id
|
||||
```
|
||||
|
||||
Auto-label moments when:
|
||||
- User reports a breakthrough or insight
|
||||
- User starts a new task type (e.g., "switching to code review")
|
||||
- User completes a significant protocol
|
||||
- User asks you to mark the current moment
|
||||
- A notable state transition occurs (entering/leaving flow)
|
||||
|
||||
Labels are stored in a database and indexed for later retrieval via `search-labels`
|
||||
and `interactive` commands.
|
||||
|
||||
---
|
||||
|
||||
## 7. Real-Time Streaming
|
||||
```bash
|
||||
npx neuroskill listen --seconds 30 --json
|
||||
npx neuroskill listen --seconds 5 --json | jq '[.[] | select(.event == "scores")]'
|
||||
```
|
||||
|
||||
Streams live WebSocket events (EXG, PPG, IMU, scores, labels) for the specified
|
||||
duration. Requires WebSocket connection (not available with `--http`).
|
||||
|
||||
Use this for continuous monitoring scenarios or to observe metric changes in real-time
|
||||
during a protocol.
|
||||
|
||||
---
|
||||
|
||||
## 8. UMAP Visualization
|
||||
```bash
|
||||
npx neuroskill umap --json # auto: last 2 sessions
|
||||
npx neuroskill umap --a-start <UTC> --a-end <UTC> --b-start <UTC> --b-end <UTC> --json
|
||||
```
|
||||
|
||||
GPU-accelerated 3D UMAP projection of ZUNA embeddings. The `separation_score`
|
||||
indicates how neurally distinct two sessions are:
|
||||
- **> 1.5** → Sessions are neurally distinct (different brain states)
|
||||
- **< 0.5** → Similar brain states across both sessions
|
||||
|
||||
---
|
||||
|
||||
## 9. Proactive State Awareness
|
||||
|
||||
### Session Start Check
|
||||
At the beginning of a session, optionally run a status check if the user mentions
|
||||
they're wearing their device or asks about their state:
|
||||
```bash
|
||||
npx neuroskill status --json
|
||||
```
|
||||
|
||||
Inject a brief state summary:
|
||||
> "Quick check-in: focus is building at 0.62, relaxation is good at 0.55, and your
|
||||
> FAA is positive — approach motivation is engaged. Looks like a solid start."
|
||||
|
||||
### When to Proactively Mention State
|
||||
|
||||
Mention cognitive state **only** when:
|
||||
- User explicitly asks ("How am I doing?", "Check my focus")
|
||||
- User reports difficulty concentrating, stress, or fatigue
|
||||
- A critical threshold is crossed (drowsiness > 0.70, focus < 0.30 sustained)
|
||||
- User is about to do something cognitively demanding and asks for readiness
|
||||
|
||||
**Do NOT** interrupt flow state to report metrics. If focus > 0.75, protect the
|
||||
session — silence is the correct response.
|
||||
|
||||
---
|
||||
|
||||
## 10. Suggesting Protocols
|
||||
|
||||
When metrics indicate a need, suggest a protocol from `references/protocols.md`.
|
||||
Always ask before starting — never interrupt flow state:
|
||||
|
||||
> "Your focus has been declining for the past 15 minutes and TBR is climbing past
|
||||
> 1.5 — signs of theta dominance and mental fatigue. Want me to walk you through
|
||||
> a Theta-Beta Neurofeedback Anchor? It's a 90-second exercise that uses rhythmic
|
||||
> counting and breath to suppress theta and lift beta."
|
||||
|
||||
Key triggers:
|
||||
- **Focus < 0.40, TBR > 1.5** → Theta-Beta Neurofeedback Anchor or Box Breathing
|
||||
- **Relaxation < 0.30, stress_index high** → Cardiac Coherence or 4-7-8 Breathing
|
||||
- **Cognitive Load > 0.70 sustained** → Cognitive Load Offload (mind dump)
|
||||
- **Drowsiness > 0.60** → Ultradian Reset or Wake Reset
|
||||
- **FAA < 0 (negative)** → FAA Rebalancing
|
||||
- **Flow State (focus > 0.75, engagement > 0.70)** → Do NOT interrupt
|
||||
- **High stillness + headache_index** → Neck Release Sequence
|
||||
- **Low RMSSD (< 25ms)** → Vagal Toning
|
||||
|
||||
---
|
||||
|
||||
## 11. Additional Tools
|
||||
|
||||
### Focus Timer
|
||||
```bash
|
||||
npx neuroskill timer --json
|
||||
```
|
||||
Launches the Focus Timer window with Pomodoro (25/5), Deep Work (50/10), or
|
||||
Short Focus (15/5) presets.
|
||||
|
||||
### Calibration
|
||||
```bash
|
||||
npx neuroskill calibrate
|
||||
npx neuroskill calibrate --profile "Eyes Open"
|
||||
```
|
||||
Opens the calibration window. Useful when signal quality is poor or the user
|
||||
wants to establish a personalized baseline.
|
||||
|
||||
### OS Notifications
|
||||
```bash
|
||||
npx neuroskill notify "Break Time" "Your focus has been declining for 20 minutes"
|
||||
```
|
||||
|
||||
### Raw JSON Passthrough
|
||||
```bash
|
||||
npx neuroskill raw '{"command":"status"}' --json
|
||||
```
|
||||
For any server command not yet mapped to a CLI subcommand.
|
||||
|
||||
---
|
||||
|
||||
## Error Handling
|
||||
|
||||
| Error | Likely Cause | Fix |
|
||||
|-------|-------------|-----|
|
||||
| `npx neuroskill status` hangs | NeuroSkill app not running | Open NeuroSkill desktop app |
|
||||
| `device.state: "disconnected"` | BCI device not connected | Check Bluetooth, device battery |
|
||||
| All scores return 0 | Poor electrode contact | Reposition headband, moisten electrodes |
|
||||
| `signal_quality` values < 0.7 | Loose electrodes | Adjust fit, clean electrode contacts |
|
||||
| SNR < 3 dB | Noisy signal | Minimize head movement, check environment |
|
||||
| `command not found: npx` | Node.js not installed | Install Node.js 20+ |
|
||||
|
||||
---
|
||||
|
||||
## Example Interactions
|
||||
|
||||
**"How am I doing right now?"**
|
||||
```bash
|
||||
npx neuroskill status --json
|
||||
```
|
||||
→ Interpret scores naturally, mentioning focus, relaxation, mood, and any notable
|
||||
ratios (FAA, TBR). Suggest an action only if metrics indicate a need.
|
||||
|
||||
**"I can't concentrate"**
|
||||
```bash
|
||||
npx neuroskill status --json
|
||||
```
|
||||
→ Check if metrics confirm it (high theta, low beta, rising TBR, high drowsiness).
|
||||
→ If confirmed, suggest an appropriate protocol from `references/protocols.md`.
|
||||
→ If metrics look fine, the issue may be motivational rather than neurological.
|
||||
|
||||
**"Compare my focus today vs yesterday"**
|
||||
```bash
|
||||
npx neuroskill compare --json
|
||||
```
|
||||
→ Interpret trends, not just numbers. Mention what improved, what declined, and
|
||||
possible causes.
|
||||
|
||||
**"When was I last in a flow state?"**
|
||||
```bash
|
||||
npx neuroskill search-labels "flow" --json
|
||||
npx neuroskill search --json
|
||||
```
|
||||
→ Report timestamps, associated metrics, and what the user was doing (from labels).
|
||||
|
||||
**"How did I sleep?"**
|
||||
```bash
|
||||
npx neuroskill sleep --json
|
||||
```
|
||||
→ Report sleep architecture (N3%, REM%, efficiency), compare to healthy targets,
|
||||
and note any issues (high wake epochs, low REM).
|
||||
|
||||
**"Mark this moment — I just had a breakthrough"**
|
||||
```bash
|
||||
npx neuroskill label "breakthrough"
|
||||
```
|
||||
→ Confirm label saved. Optionally note the current metrics to remember the state.
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- [NeuroSkill Paper — arXiv:2603.03212](https://arxiv.org/abs/2603.03212) (Kosmyna & Hauptmann, MIT Media Lab)
|
||||
- [NeuroSkill Desktop App](https://github.com/NeuroSkill-com/skill) (GPLv3)
|
||||
- [NeuroLoop CLI Companion](https://github.com/NeuroSkill-com/neuroloop) (GPLv3)
|
||||
- [MIT Media Lab Project](https://www.media.mit.edu/projects/neuroskill/overview/)
|
||||
|
|
@ -0,0 +1,286 @@
|
|||
# NeuroSkill WebSocket & HTTP API Reference
|
||||
|
||||
NeuroSkill runs a local server (default port **8375**) discoverable via mDNS
|
||||
(`_skill._tcp`). It exposes both WebSocket and HTTP endpoints.
|
||||
|
||||
---
|
||||
|
||||
## Server Discovery
|
||||
|
||||
```bash
|
||||
# Auto-discovery (built into the CLI — usually just works)
|
||||
npx neuroskill status --json
|
||||
|
||||
# Manual port discovery
|
||||
NEURO_PORT=$(lsof -i -n -P | grep neuroskill | grep LISTEN | awk '{print $9}' | cut -d: -f2 | head -1)
|
||||
echo "NeuroSkill on port: $NEURO_PORT"
|
||||
```
|
||||
|
||||
The CLI auto-discovers the port. Use `--port <N>` to override.
|
||||
|
||||
---
|
||||
|
||||
## HTTP REST Endpoints
|
||||
|
||||
### Universal Command Tunnel
|
||||
```bash
|
||||
# POST / — accepts any command as JSON
|
||||
curl -s -X POST http://127.0.0.1:8375/ \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"command":"status"}'
|
||||
```
|
||||
|
||||
### Convenience Endpoints
|
||||
| Method | Endpoint | Description |
|
||||
|--------|----------|-------------|
|
||||
| GET | `/v1/status` | System status |
|
||||
| GET | `/v1/sessions` | List sessions |
|
||||
| POST | `/v1/label` | Create label |
|
||||
| POST | `/v1/search` | ANN search |
|
||||
| POST | `/v1/compare` | A/B comparison |
|
||||
| POST | `/v1/sleep` | Sleep staging |
|
||||
| POST | `/v1/notify` | OS notification |
|
||||
| POST | `/v1/say` | Text-to-speech |
|
||||
| POST | `/v1/calibrate` | Open calibration |
|
||||
| POST | `/v1/timer` | Open focus timer |
|
||||
| GET | `/v1/dnd` | Get DND status |
|
||||
| POST | `/v1/dnd` | Force DND on/off |
|
||||
| GET | `/v1/calibrations` | List calibration profiles |
|
||||
| POST | `/v1/calibrations` | Create profile |
|
||||
| GET | `/v1/calibrations/{id}` | Get profile |
|
||||
| PATCH | `/v1/calibrations/{id}` | Update profile |
|
||||
| DELETE | `/v1/calibrations/{id}` | Delete profile |
|
||||
|
||||
---
|
||||
|
||||
## WebSocket Events (Broadcast)
|
||||
|
||||
Connect to `ws://127.0.0.1:8375/` to receive real-time events:
|
||||
|
||||
### EXG (Raw EEG Samples)
|
||||
```json
|
||||
{"event": "EXG", "electrode": 0, "samples": [12.3, -4.1, ...], "timestamp": 1740412800.512}
|
||||
```
|
||||
|
||||
### PPG (Photoplethysmography)
|
||||
```json
|
||||
{"event": "PPG", "channel": 0, "samples": [...], "timestamp": 1740412800.512}
|
||||
```
|
||||
|
||||
### IMU (Inertial Measurement Unit)
|
||||
```json
|
||||
{"event": "IMU", "ax": 0.01, "ay": -0.02, "az": 9.81, "gx": 0.1, "gy": -0.05, "gz": 0.02}
|
||||
```
|
||||
|
||||
### Scores (Computed Metrics)
|
||||
```json
|
||||
{
|
||||
"event": "scores",
|
||||
"focus": 0.70, "relaxation": 0.40, "engagement": 0.60,
|
||||
"rel_delta": 0.28, "rel_theta": 0.18, "rel_alpha": 0.32,
|
||||
"rel_beta": 0.17, "hr": 68.2, "snr": 14.3
|
||||
}
|
||||
```
|
||||
|
||||
### EXG Bands (Spectral Analysis)
|
||||
```json
|
||||
{"event": "EXG-bands", "channels": [...], "faa": 0.12}
|
||||
```
|
||||
|
||||
### Labels
|
||||
```json
|
||||
{"event": "label", "label_id": 42, "text": "meditation start", "created_at": 1740413100}
|
||||
```
|
||||
|
||||
### Device Status
|
||||
```json
|
||||
{"event": "muse-status", "state": "connected"}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## JSON Response Formats
|
||||
|
||||
### `status`
|
||||
```jsonc
|
||||
{
|
||||
"command": "status", "ok": true,
|
||||
"device": {
|
||||
"state": "connected", // "connected" | "connecting" | "disconnected"
|
||||
"name": "Muse-A1B2",
|
||||
"battery": 73,
|
||||
"firmware": "1.3.4",
|
||||
"EXG_samples": 195840,
|
||||
"ppg_samples": 30600,
|
||||
"imu_samples": 122400
|
||||
},
|
||||
"session": {
|
||||
"start_utc": 1740412800,
|
||||
"duration_secs": 1847,
|
||||
"n_epochs": 369
|
||||
},
|
||||
"signal_quality": {
|
||||
"tp9": 0.95, "af7": 0.88, "af8": 0.91, "tp10": 0.97
|
||||
},
|
||||
"scores": {
|
||||
"focus": 0.70, "relaxation": 0.40, "engagement": 0.60,
|
||||
"meditation": 0.52, "mood": 0.55, "cognitive_load": 0.33,
|
||||
"drowsiness": 0.10, "hr": 68.2, "snr": 14.3, "stillness": 0.88,
|
||||
"bands": { "rel_delta": 0.28, "rel_theta": 0.18, "rel_alpha": 0.32, "rel_beta": 0.17, "rel_gamma": 0.05 },
|
||||
"faa": 0.042, "tar": 0.56, "bar": 0.53, "tbr": 1.06,
|
||||
"apf": 10.1, "coherence": 0.614, "mu_suppression": 0.031
|
||||
},
|
||||
"embeddings": { "today": 342, "total": 14820, "recording_days": 31 },
|
||||
"labels": { "total": 58, "recent": [{"id": 42, "text": "meditation start", "created_at": 1740413100}] },
|
||||
"sleep": { "total_epochs": 1054, "wake_epochs": 134, "n1_epochs": 89, "n2_epochs": 421, "n3_epochs": 298, "rem_epochs": 112, "epoch_secs": 5 },
|
||||
"history": { "total_sessions": 63, "recording_days": 31, "current_streak_days": 7, "total_recording_hours": 94.2, "longest_session_min": 187, "avg_session_min": 89 }
|
||||
}
|
||||
```
|
||||
|
||||
### `sessions`
|
||||
```jsonc
|
||||
{
|
||||
"command": "sessions", "ok": true,
|
||||
"sessions": [
|
||||
{ "day": "20260224", "start_utc": 1740412800, "end_utc": 1740415510, "n_epochs": 541 },
|
||||
{ "day": "20260223", "start_utc": 1740380100, "end_utc": 1740382665, "n_epochs": 513 }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### `session` (single session breakdown)
|
||||
```jsonc
|
||||
{
|
||||
"ok": true,
|
||||
"metrics": { "focus": 0.70, "relaxation": 0.40, "n_epochs": 541 /* ... ~50 metrics */ },
|
||||
"first": { "focus": 0.64 /* first-half averages */ },
|
||||
"second": { "focus": 0.76 /* second-half averages */ },
|
||||
"trends": { "focus": "up", "relaxation": "down" /* "up" | "down" | "flat" */ }
|
||||
}
|
||||
```
|
||||
|
||||
### `compare` (A/B comparison)
|
||||
```jsonc
|
||||
{
|
||||
"command": "compare", "ok": true,
|
||||
"insights": {
|
||||
"deltas": {
|
||||
"focus": { "a": 0.62, "b": 0.71, "abs": 0.09, "pct": 14.5, "direction": "up" },
|
||||
"relaxation": { "a": 0.45, "b": 0.38, "abs": -0.07, "pct": -15.6, "direction": "down" }
|
||||
},
|
||||
"improved": ["focus", "engagement"],
|
||||
"declined": ["relaxation"]
|
||||
},
|
||||
"sleep_a": { /* sleep summary for session A */ },
|
||||
"sleep_b": { /* sleep summary for session B */ },
|
||||
"umap": { "job_id": "abc123" }
|
||||
}
|
||||
```
|
||||
|
||||
### `search` (ANN similarity)
|
||||
```jsonc
|
||||
{
|
||||
"command": "search", "ok": true,
|
||||
"result": {
|
||||
"results": [{
|
||||
"neighbors": [{ "distance": 0.12, "metadata": {"device": "Muse-A1B2", "date": "20260223"} }]
|
||||
}],
|
||||
"analysis": {
|
||||
"distance_stats": { "mean": 0.15, "min": 0.08, "max": 0.42 },
|
||||
"temporal_distribution": { /* hour-of-day distribution */ },
|
||||
"top_days": [["20260223", 5], ["20260222", 3]]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### `sleep` (sleep staging)
|
||||
```jsonc
|
||||
{
|
||||
"command": "sleep", "ok": true,
|
||||
"summary": { "total_epochs": 1054, "wake_epochs": 134, "n1_epochs": 89, "n2_epochs": 421, "n3_epochs": 298, "rem_epochs": 112, "epoch_secs": 5 },
|
||||
"analysis": { "efficiency_pct": 87.3, "onset_latency_min": 12.5, "rem_latency_min": 65.0, "bouts": { /* wake/n3/rem bout counts and durations */ } },
|
||||
"epochs": [{ "utc": 1740380100, "stage": 0, "rel_delta": 0.15, "rel_theta": 0.22, "rel_alpha": 0.38, "rel_beta": 0.20 }]
|
||||
}
|
||||
```
|
||||
|
||||
### `label`
|
||||
```json
|
||||
{"command": "label", "ok": true, "label_id": 42}
|
||||
```
|
||||
|
||||
### `search-labels` (semantic search)
|
||||
```jsonc
|
||||
{
|
||||
"command": "search-labels", "ok": true,
|
||||
"results": [{
|
||||
"text": "deep focus block",
|
||||
"EXG_metrics": { "focus": 0.82, "relaxation": 0.35, "engagement": 0.75, "hr": 65.0, "mood": 0.60 },
|
||||
"EXG_start": 1740412800, "EXG_end": 1740412805,
|
||||
"created_at": 1740412802,
|
||||
"similarity": 0.92
|
||||
}]
|
||||
}
|
||||
```
|
||||
|
||||
### `umap` (3D projection)
|
||||
```jsonc
|
||||
{
|
||||
"command": "umap", "ok": true,
|
||||
"result": {
|
||||
"points": [{ "x": 1.23, "y": -0.45, "z": 2.01, "session": "a", "utc": 1740412800 }],
|
||||
"analysis": {
|
||||
"separation_score": 1.84,
|
||||
"inter_cluster_distance": 2.31,
|
||||
"intra_spread_a": 0.82, "intra_spread_b": 0.94,
|
||||
"centroid_a": [1.23, -0.45, 2.01],
|
||||
"centroid_b": [-0.87, 1.34, -1.22]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Useful `jq` Snippets
|
||||
|
||||
```bash
|
||||
# Get just focus score
|
||||
npx neuroskill status --json | jq '.scores.focus'
|
||||
|
||||
# Get all band powers
|
||||
npx neuroskill status --json | jq '.scores.bands'
|
||||
|
||||
# Check device battery
|
||||
npx neuroskill status --json | jq '.device.battery'
|
||||
|
||||
# Get signal quality
|
||||
npx neuroskill status --json | jq '.signal_quality'
|
||||
|
||||
# Find improving metrics after a session
|
||||
npx neuroskill session 0 --json | jq '[.trends | to_entries[] | select(.value == "up") | .key]'
|
||||
|
||||
# Sort comparison deltas by improvement
|
||||
npx neuroskill compare --json | jq '.insights.deltas | to_entries | sort_by(.value.pct) | reverse'
|
||||
|
||||
# Get sleep efficiency
|
||||
npx neuroskill sleep --json | jq '.analysis.efficiency_pct'
|
||||
|
||||
# Find closest neural match
|
||||
npx neuroskill search --json | jq '[.result.results[].neighbors[]] | sort_by(.distance) | .[0]'
|
||||
|
||||
# Extract TBR from labeled stress moments
|
||||
npx neuroskill search-labels "stress" --json | jq '[.results[].EXG_metrics.tbr]'
|
||||
|
||||
# Get session timestamps for manual compare
|
||||
npx neuroskill sessions --json | jq '{start: .sessions[0].start_utc, end: .sessions[0].end_utc}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Data Storage
|
||||
|
||||
- **Local database**: `~/.skill/YYYYMMDD/` (SQLite + HNSW index)
|
||||
- **ZUNA embeddings**: 128-D vectors, 5-second epochs
|
||||
- **Labels**: Stored in SQLite, indexed with bge-small-en-v1.5 embeddings
|
||||
- **All data is local** — nothing is sent to external servers
|
||||
|
|
@ -0,0 +1,220 @@
|
|||
# NeuroSkill Metric Definitions & Interpretation Guide
|
||||
|
||||
> **⚠️ Research Use Only:** All metrics are experimental and derived from
|
||||
> consumer-grade hardware (Muse 2/S). They are not FDA/CE-cleared and must not
|
||||
> be used for medical diagnosis or treatment.
|
||||
|
||||
---
|
||||
|
||||
## Hardware & Signal Acquisition
|
||||
|
||||
NeuroSkill is validated for **Muse 2** and **Muse S** headbands (with OpenBCI
|
||||
support in the desktop app), streaming at **256 Hz** (EEG) and **64 Hz** (PPG).
|
||||
|
||||
### Electrode Positions (International 10-20 System)
|
||||
| Channel | Electrode | Position | Primary Signals |
|
||||
|---------|-----------|----------|-----------------|
|
||||
| CH1 | TP9 | Left Mastoid | Auditory cortex, verbal memory, jaw-clench artifact |
|
||||
| CH2 | AF7 | Left Prefrontal | Executive function, approach motivation, eye blinks |
|
||||
| CH3 | AF8 | Right Prefrontal | Emotional regulation, vigilance, eye blinks |
|
||||
| CH4 | TP10 | Right Mastoid | Prosody, spatial hearing, non-verbal cognition |
|
||||
|
||||
### Preprocessing Pipeline
|
||||
1. **Filtering**: High-pass (0.5 Hz), Low-pass (50/60 Hz), Notch filter
|
||||
2. **Spectral Analysis**: Hann-windowed FFT (512-sample window), Welch periodogram
|
||||
3. **GPU acceleration**: ~125ms latency via `gpu_fft`
|
||||
|
||||
---
|
||||
|
||||
## EEG Frequency Bands
|
||||
|
||||
Relative power values (sum ≈ 1.0 across all bands):
|
||||
|
||||
| Band | Range (Hz) | High Means | Low Means |
|
||||
|------|-----------|------------|-----------|
|
||||
| **Delta (δ)** | 1–4 | Deep sleep (N3), high-amplitude artifacts | Awake, alert |
|
||||
| **Theta (θ)** | 4–8 | Drowsiness, REM onset, creative ideation, cognitive load | Alert, focused |
|
||||
| **Alpha (α)** | 8–13 | Relaxed wakefulness, "alpha blocking" during effort | Active thinking, anxiety |
|
||||
| **Beta (β)** | 13–30 | Active concentration, problem-solving, alertness | Relaxed, unfocused |
|
||||
| **Gamma (γ)** | 30–50 | Higher-order processing, perceptual binding, memory | Baseline |
|
||||
|
||||
### JSON Field Names
|
||||
```json
|
||||
"bands": {
|
||||
"rel_delta": 0.28, "rel_theta": 0.18, "rel_alpha": 0.32,
|
||||
"rel_beta": 0.17, "rel_gamma": 0.05
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Core Composite Scores (0–1 Scale)
|
||||
|
||||
### Focus
|
||||
- **Formula**: σ(β / (α + θ)) — beta dominance over slow waves, sigmoid-mapped
|
||||
- **> 0.70**: Deep concentration, flow state, task absorption
|
||||
- **0.40–0.69**: Moderate attention, some mind-wandering
|
||||
- **< 0.40**: Distracted, fatigued, difficulty concentrating
|
||||
|
||||
### Relaxation
|
||||
- **Formula**: σ(α / (β + θ)) — alpha dominance, sigmoid-mapped
|
||||
- **> 0.70**: Calm, stress-free, parasympathetic dominant
|
||||
- **0.40–0.69**: Mild tension present
|
||||
- **< 0.30**: Stressed, anxious, sympathetic dominant
|
||||
|
||||
### Engagement
|
||||
- **0–1 scale**: Active mental investment and motivation
|
||||
- **> 0.70**: Mentally invested, motivated, active processing
|
||||
- **0.40–0.69**: Passive participation
|
||||
- **< 0.30**: Bored, disengaged, autopilot mode
|
||||
|
||||
### Meditation
|
||||
- **Composite**: Combines alpha elevation, physical stillness (IMU), and HRV coherence
|
||||
- **> 0.70**: Deep meditative state
|
||||
- **< 0.30**: Active, non-meditative
|
||||
|
||||
### Mood
|
||||
- **Composite**: Derived from FAA, TAR, and BAR
|
||||
- **> 0.60**: Positive affect, approach motivation
|
||||
- **< 0.40**: Low mood, withdrawal tendency
|
||||
|
||||
### Cognitive Load
|
||||
- **Formula**: (P_θ_frontal / P_α_temporal) · f(FAA, TBR) — working memory usage
|
||||
- **> 0.70**: Working memory near capacity, complex processing
|
||||
- **0.40–0.69**: Moderate mental effort
|
||||
- **< 0.40**: Task is easy or automatic
|
||||
- **Interpretation**: High load + high focus = productive struggle. High load + low focus = overwhelmed.
|
||||
|
||||
### Drowsiness
|
||||
- **Composite**: Weighted TAR + TBR + falling Spectral Centroid
|
||||
- **> 0.60**: Sleep pressure building, micro-sleep risk
|
||||
- **0.30–0.59**: Mild fatigue
|
||||
- **< 0.30**: Alert
|
||||
|
||||
---
|
||||
|
||||
## EEG Ratios & Spectral Indices
|
||||
|
||||
| Metric | Formula | Interpretation |
|
||||
|--------|---------|----------------|
|
||||
| **FAA** | ln(P_α_AF8) − ln(P_α_AF7) | Frontal Alpha Asymmetry. Positive = approach/positive affect. Negative = withdrawal/depression. |
|
||||
| **TAR** | P_θ / P_α | Theta/Alpha Ratio. > 1.5 = drowsiness or mind-wandering. |
|
||||
| **BAR** | P_β / P_α | Beta/Alpha Ratio. > 1.5 = alert, engaged cognition. Can also indicate anxiety. |
|
||||
| **TBR** | P_θ / P_β | Theta/Beta Ratio. ADHD biomarker. Healthy ≈ 1.0, elevated > 1.5, clinical > 3.0. |
|
||||
| **APF** | argmax_f PSD(f) in [7.5, 12.5] Hz | Alpha Peak Frequency. Typical 8–12 Hz. Higher = faster cognitive processing. Slows with age/fatigue. |
|
||||
| **SNR** | 10 · log₁₀(P_signal / P_noise) | Signal-to-Noise Ratio. > 10 dB = clean, 3–10 dB = usable, < 3 dB = unreliable. |
|
||||
| **Coherence** | Inter-hemispheric coherence (0–1) | Cortical connectivity between hemispheres. |
|
||||
| **Mu Suppression** | Motor cortex suppression index | Low values during movement or motor imagery. |
|
||||
|
||||
---
|
||||
|
||||
## Complexity & Nonlinear Metrics
|
||||
|
||||
| Metric | Description | Healthy Range |
|
||||
|--------|-------------|---------------|
|
||||
| **Permutation Entropy (PE)** | Temporal complexity. Near 1 = maximally irregular. | Consciousness marker |
|
||||
| **Higuchi Fractal Dimension (HFD)** | Waveform self-similarity. | Waking: 1.3–1.8; higher = complex |
|
||||
| **DFA Exponent** | Long-range correlations. | Healthy: 0.6–0.9 |
|
||||
| **PSE** | Power Spectral Entropy. Near 1.0 = white noise. | Lower = organized brain state |
|
||||
| **PAC θ-γ** | Phase-Amplitude Coupling, theta-gamma. | Working memory mechanism |
|
||||
| **BPS** | Band-Power Slope (1/f spectral exponent). | Steeper = inhibition-dominated |
|
||||
|
||||
---
|
||||
|
||||
## Consciousness Metrics
|
||||
|
||||
Derived from the nonlinear metrics above:
|
||||
|
||||
| Metric | Scale | Interpretation |
|
||||
|--------|-------|----------------|
|
||||
| **LZC** | 0–100 | Lempel-Ziv Complexity proxy (PE + HFD). > 60 = wakefulness. |
|
||||
| **Wakefulness** | 0–100 | Inverse drowsiness composite. |
|
||||
| **Integration** | 0–100 | Cortical integration (Coherence × PAC × Spectral Entropy). |
|
||||
|
||||
Status thresholds: ≥ 50 Green, 25–50 Yellow, < 25 Red.
|
||||
|
||||
---
|
||||
|
||||
## Cardiac & Autonomic Metrics (from PPG)
|
||||
|
||||
| Metric | Description | Normal / Green Range |
|
||||
|--------|-------------|---------------------|
|
||||
| **HR** | Heart rate (bpm) | 55–90 (green), 45–110 (yellow), else red |
|
||||
| **RMSSD** | Primary vagal tone marker (ms) | > 50 ms healthy, < 20 ms stress |
|
||||
| **SDNN** | HRV time-domain variability (ms) | Higher = better |
|
||||
| **pNN50** | Parasympathetic indicator (%) | Higher = more parasympathetic activity |
|
||||
| **LF/HF Ratio** | Sympatho-vagal balance | > 2.0 = stress, < 0.5 = relaxation |
|
||||
| **Stress Index** | Baevsky SI: AMo / (2 × MxDMn × Mo) | 0–100 composite. > 200 raw = strong stress |
|
||||
| **SpO₂ Estimate** | Blood oxygen saturation (uncalibrated) | 95–100% normal (research only) |
|
||||
| **Respiratory Rate** | Breaths per minute | 12–20 normal |
|
||||
|
||||
---
|
||||
|
||||
## Motion & Artifact Detection
|
||||
|
||||
| Metric | Description |
|
||||
|--------|-------------|
|
||||
| **Stillness** | 0–1 (1 = perfectly still). From IMU accelerometer/gyroscope. |
|
||||
| **Blink Count** | Eye blinks detected (large spikes in AF7/AF8). Normal: 15–20/min. |
|
||||
| **Jaw Clench Count** | High-frequency EMG bursts (> 30 Hz) at TP9/TP10. |
|
||||
| **Nod Count** | Head nods detected via IMU. |
|
||||
| **Shake Count** | Head shakes detected via IMU. |
|
||||
| **Head Pitch/Roll** | Head orientation from IMU. |
|
||||
|
||||
---
|
||||
|
||||
## Signal Quality (Per Electrode)
|
||||
|
||||
| Electrode | Range | Interpretation |
|
||||
|-----------|-------|----------------|
|
||||
| **TP9** | 0–1 | ≥ 0.9 = good, ≥ 0.7 = acceptable, < 0.7 = poor |
|
||||
| **AF7** | 0–1 | Same thresholds |
|
||||
| **AF8** | 0–1 | Same thresholds |
|
||||
| **TP10** | 0–1 | Same thresholds |
|
||||
|
||||
If any electrode is below 0.7, recommend the user adjust the headband fit or
|
||||
moisten the electrode contacts.
|
||||
|
||||
---
|
||||
|
||||
## Sleep Staging
|
||||
|
||||
Based on 5-second epochs using relative band-power ratios and AASM heuristics:
|
||||
|
||||
| Stage | Code | EEG Signature | Function |
|
||||
|-------|------|---------------|----------|
|
||||
| Wake | 0 | Alpha-dominant, BAR > 0.8 | Conscious awareness |
|
||||
| N1 | 1 | Alpha → Theta transition | Light sleep onset |
|
||||
| N2 | 2 | Sleep spindles, K-complexes | Memory consolidation |
|
||||
| N3 (Deep) | 3 | Delta > 20% of epoch, DTR > 2 | Deep restorative sleep |
|
||||
| REM | 4 | Active EEG, high Theta, low Delta | Emotional processing, dreaming |
|
||||
|
||||
### Healthy Adult Targets (~8h Sleep)
|
||||
- **N3 (Deep)**: 15–25% of total sleep
|
||||
- **REM**: 20–25%
|
||||
- **Sleep Efficiency**: > 85%
|
||||
- **Sleep Onset Latency**: < 20 min
|
||||
|
||||
---
|
||||
|
||||
## Composite State Patterns
|
||||
|
||||
| Pattern | Key Metrics | Interpretation |
|
||||
|---------|-------------|----------------|
|
||||
| **Flow State** | Focus > 0.75, Engagement > 0.70, Cognitive Load 0.50–0.70, HR steady | Optimal performance zone — protect it |
|
||||
| **Mental Fatigue** | Focus < 0.40, Drowsiness > 0.60, TBR > 1.5, Theta elevated | Rest or break needed |
|
||||
| **Anxiety** | Relaxation < 0.30, HR elevated, high Beta, high BAR, stress_index high | Calming intervention helpful |
|
||||
| **Peak Alert** | Focus > 0.80, Engagement > 0.70, Drowsiness < 0.20 | Best time for hard tasks |
|
||||
| **Recovery** | Relaxation > 0.70, HRV (RMSSD) rising, Alpha dominant | Integration, light tasks only |
|
||||
| **Creative Mode** | High Theta, high Alpha, low Beta, moderate focus | Ideation — don't force structure |
|
||||
| **Withdrawal** | FAA < 0, low Mood, low Engagement | Approach motivation needed |
|
||||
|
||||
---
|
||||
|
||||
## ZUNA Embeddings
|
||||
|
||||
NeuroSkill uses the **ZUNA Neural Encoder** to convert 5-second EEG epochs into
|
||||
**128-dimensional vectors** stored in an HNSW index:
|
||||
- **Search**: Sub-millisecond approximate nearest-neighbor queries
|
||||
- **UMAP**: GPU-accelerated 3D projection for visual comparison
|
||||
- **Storage**: Local SQLite + HNSW index in `~/.skill/YYYYMMDD/`
|
||||
|
|
@ -0,0 +1,452 @@
|
|||
# NeuroSkill Guided Protocols
|
||||
|
||||
Over 70 mind-body practices triggered by specific biometric (EXG) signals. These
|
||||
are sourced from NeuroLoop's protocol repertoire and are designed to be suggested
|
||||
when the system detects specific cognitive or physiological states.
|
||||
|
||||
> **⚠️ Contraindication**: Wim Hof and hyperventilation-style breathwork are
|
||||
> unsuitable for epilepsy_risk > 30, known cardiac conditions, or pregnancy.
|
||||
|
||||
---
|
||||
|
||||
## When to Suggest Protocols
|
||||
|
||||
**Always ask before starting.** Match ONE protocol to the single most salient
|
||||
metric signal. Explain the metric connection to the user.
|
||||
|
||||
| User State | Recommended Protocol |
|
||||
|------------|---------------------|
|
||||
| Focus < 0.40, TBR > 1.5 | Theta-Beta Neurofeedback Anchor or Box Breathing |
|
||||
| Low engagement, session start | WOOP or Pre-Task Priming |
|
||||
| Relaxation < 0.30, stress_index high | Cardiac Coherence or 4-7-8 Breathing |
|
||||
| Cognitive Load > 0.70 sustained | Cognitive Load Offload (Mind Dump) |
|
||||
| Engagement < 0.30 for > 20 min | Novel Stimulation Burst or Environment Change |
|
||||
| Flow State (focus > 0.75, engagement > 0.70) | **Do NOT interrupt — protect the session** |
|
||||
| Drowsiness > 0.60, post-lunch | Ultradian Reset or Power Nap |
|
||||
| FAA < 0, depression_index elevated | FAA Rebalancing |
|
||||
| Low RMSSD (< 25ms) | Vagal Toning |
|
||||
| High stillness + headache signals | Neck Release Sequence |
|
||||
| Pre-sleep, HRV low | Sleep Wind-Down |
|
||||
| Post-social-media, low mood | Envy & Comparison Alchemy |
|
||||
|
||||
---
|
||||
|
||||
## Attention & Focus Protocols
|
||||
|
||||
### Theta-Beta Neurofeedback Anchor
|
||||
**Duration**: ~90 seconds
|
||||
**Trigger**: High TBR (> 1.5) and low focus
|
||||
**Instructions**:
|
||||
1. Close your eyes
|
||||
2. Breathe slowly — 4s inhale, 6s exhale
|
||||
3. Count rhythmically from 1 to 10, matching your breath
|
||||
4. Focus on the counting — if you lose count, restart from 1
|
||||
5. Open your eyes after 4–5 full cycles
|
||||
**Effect**: Suppresses theta dominance and lifts beta activity
|
||||
|
||||
### Focus Reset
|
||||
**Duration**: 90 seconds
|
||||
**Trigger**: Scattered engagement, difficulty settling into task
|
||||
**Instructions**:
|
||||
1. Close your eyes completely
|
||||
2. Take 5 slow, deep breaths
|
||||
3. Mentally state your intention for the next work block
|
||||
4. Open your eyes and begin immediately
|
||||
**Effect**: Resets attentional baseline
|
||||
|
||||
### Working Memory Primer
|
||||
**Duration**: 3 minutes
|
||||
**Trigger**: Low PAC θ-γ (theta-gamma coupling), low sample entropy
|
||||
**Instructions**:
|
||||
1. Breathe at theta pace: 4s inhale, 6s exhale, 2s hold
|
||||
2. While breathing, do a verbal 3-back task: listen to or read a sequence
|
||||
of numbers, say which number appeared 3 positions back
|
||||
3. Continue for 3 minutes
|
||||
**Effect**: Lifts theta-gamma coupling and working memory engagement
|
||||
|
||||
### Creativity Unlock
|
||||
**Duration**: 5 minutes
|
||||
**Trigger**: High beta, low rel_alpha — system is too analytically locked
|
||||
**Instructions**:
|
||||
1. Stop all structured work
|
||||
2. Let your mind wander without a goal
|
||||
3. Doodle, look out the window, or listen to ambient sound
|
||||
4. Don't force any outcome — just observe what arises
|
||||
5. After 5 minutes, jot down any ideas that surfaced
|
||||
**Effect**: Promotes alpha and theta activity for creative ideation
|
||||
|
||||
### Dual-N-Back Warm-Up
|
||||
**Duration**: 3 minutes
|
||||
**Trigger**: Low PAC θ-γ, low sample entropy
|
||||
**Instructions**:
|
||||
1. Read or listen to a sequence of spoken numbers
|
||||
2. Track which number appeared 2 positions back (2-back)
|
||||
3. If comfortable, increase to 3-back
|
||||
**Effect**: Activates prefrontal cortex, lifts executive function
|
||||
|
||||
### Novel Stimulation Burst
|
||||
**Duration**: 2–3 minutes
|
||||
**Trigger**: Low APF (< 9 Hz), dementia_index > 30
|
||||
**Instructions**:
|
||||
1. Pick up an unusual object nearby and describe it in detail
|
||||
2. Name 5 things you can see, 4 you can touch, 3 you can hear
|
||||
3. Try a quick riddle or lateral thinking puzzle
|
||||
**Effect**: Counters cortical slowing, raises alpha peak frequency
|
||||
|
||||
---
|
||||
|
||||
## Autonomic & Stress Regulation Protocols
|
||||
|
||||
### Box Breathing (4-4-4-4)
|
||||
**Duration**: 2–4 minutes
|
||||
**Trigger**: High BAR, high anxiety_index, acute stress
|
||||
**Instructions**:
|
||||
1. Inhale for 4 counts
|
||||
2. Hold for 4 counts
|
||||
3. Exhale for 4 counts
|
||||
4. Hold for 4 counts
|
||||
5. Repeat 4–8 cycles
|
||||
**Effect**: Engages parasympathetic nervous system, reduces beta activity
|
||||
|
||||
### Extended Exhale (4-7-8)
|
||||
**Duration**: 3–5 minutes
|
||||
**Trigger**: Acute stress spikes, racing thoughts, high sympathetic activation
|
||||
**Instructions**:
|
||||
1. Exhale completely through mouth
|
||||
2. Inhale through nose for 4 counts
|
||||
3. Hold for 7 counts
|
||||
4. Exhale through mouth for 8 counts
|
||||
5. Repeat 4 cycles
|
||||
**Effect**: Fastest parasympathetic trigger for acute stress
|
||||
|
||||
### Cardiac Coherence
|
||||
**Duration**: 5 minutes
|
||||
**Trigger**: Low RMSSD (< 30 ms), high stress_index
|
||||
**Instructions**:
|
||||
1. Breathe evenly: 5-second inhale, 5-second exhale
|
||||
2. Focus on the area around your heart
|
||||
3. Recall a positive memory or feeling of appreciation
|
||||
4. Maintain for 5 minutes
|
||||
**Effect**: Maximizes HRV, creates coherent heart rhythm pattern
|
||||
|
||||
### Physiological Sigh
|
||||
**Duration**: 30 seconds (1–3 cycles)
|
||||
**Trigger**: Rapid overwhelm, acute panic
|
||||
**Instructions**:
|
||||
1. Take a quick double inhale through the nose (sniff-sniff)
|
||||
2. Follow with a long, slow exhale through the mouth
|
||||
3. Repeat 1–3 times
|
||||
**Effect**: Rapid parasympathetic activation, immediate calming
|
||||
|
||||
### Alpha Induction (Open Focus)
|
||||
**Duration**: 5 minutes
|
||||
**Trigger**: High beta, low relaxation — cannot relax
|
||||
**Instructions**:
|
||||
1. Soften your gaze — don't focus on any single object
|
||||
2. Notice the space between and around objects
|
||||
3. Expand your awareness to peripheral vision
|
||||
4. Maintain this "open focus" for 5 minutes
|
||||
**Effect**: Promotes alpha wave production, reduces beta dominance
|
||||
|
||||
### Open Monitoring
|
||||
**Duration**: 5–10 minutes
|
||||
**Trigger**: Low LZC (< 40 on 0-100 scale) — neural complexity too low
|
||||
**Instructions**:
|
||||
1. Sit comfortably with eyes closed or softly focused
|
||||
2. Don't direct attention to anything specific
|
||||
3. Simply notice whatever arises — thoughts, sounds, sensations
|
||||
4. Let each observation pass without engagement
|
||||
**Effect**: Raises neural complexity and consciousness metrics
|
||||
|
||||
### Vagal Toning
|
||||
**Duration**: 3 minutes
|
||||
**Trigger**: Low RMSSD (< 25 ms) — weak vagal tone
|
||||
**Instructions**:
|
||||
1. Hum a long, steady note on each exhale for 30 seconds
|
||||
2. Alternatively: gargle cold water for 30 seconds
|
||||
3. Repeat 3–5 times
|
||||
**Effect**: Directly stimulates the vagus nerve, increases parasympathetic tone
|
||||
|
||||
---
|
||||
|
||||
## Emotional Regulation Protocols
|
||||
|
||||
### FAA Rebalancing
|
||||
**Duration**: 5 minutes
|
||||
**Trigger**: Negative FAA (right-hemisphere dominant), high depression_index
|
||||
**Instructions**:
|
||||
1. Think of something you're genuinely looking forward to (approach motivation)
|
||||
2. Visualize yourself successfully completing a meaningful goal
|
||||
3. Squeeze your left hand into a fist for 10 seconds, release
|
||||
4. Repeat the visualization + left-hand squeeze 3–4 times
|
||||
**Effect**: Activates left prefrontal cortex, shifts FAA positive
|
||||
|
||||
### Loving-Kindness (Metta)
|
||||
**Duration**: 5–10 minutes
|
||||
**Trigger**: Loneliness signals, shame, low mood
|
||||
**Instructions**:
|
||||
1. Close your eyes and think of someone you care about
|
||||
2. Silently repeat: "May you be happy. May you be healthy. May you be safe."
|
||||
3. Extend the same wishes to yourself
|
||||
4. Extend to a neutral person, then gradually to someone difficult
|
||||
**Effect**: Reduces withdrawal motivation, increases positive affect
|
||||
|
||||
### Emotional Discharge
|
||||
**Duration**: 2 minutes
|
||||
**Trigger**: High bipolar_index or extreme FAA swings
|
||||
**Instructions**:
|
||||
1. Take 30 seconds of vigorous, fast breathing (safely)
|
||||
2. Stop and take 3 slow, deep breaths
|
||||
3. Do a 60-second body scan — notice where tension is held
|
||||
4. Shake out your hands and arms for 15 seconds
|
||||
**Effect**: Releases trapped sympathetic energy, recalibrates
|
||||
|
||||
### Havening Touch
|
||||
**Duration**: 3–5 minutes
|
||||
**Trigger**: Acute distress, trauma activation, overwhelming anxiety
|
||||
**Instructions**:
|
||||
1. Gently stroke your arms from shoulder to elbow, palms down
|
||||
2. Rub your palms together slowly
|
||||
3. Gently touch your forehead, temples
|
||||
4. Continue for 3–5 minutes while breathing slowly
|
||||
**Effect**: Disrupts amygdala-cortex encoding loop, reduces distress
|
||||
|
||||
### Anxiety Surfing
|
||||
**Duration**: ~8 minutes
|
||||
**Trigger**: Rising anxiety without clear cause
|
||||
**Instructions**:
|
||||
1. Notice where anxiety lives in your body — chest? stomach? throat?
|
||||
2. Describe the sensation without judging it (tight? hot? buzzing?)
|
||||
3. Breathe into that area for 3 breaths
|
||||
4. Notice: is it getting bigger, smaller, or changing shape?
|
||||
5. Continue observing for 5–8 minutes — anxiety typically peaks then subsides
|
||||
|
||||
### Anger: Palm-Press Discharge
|
||||
**Duration**: 2 minutes
|
||||
**Trigger**: Anger signals, high BAR + elevated HR
|
||||
**Instructions**:
|
||||
1. Press your palms together firmly for 10 seconds
|
||||
2. Release and take 3 extended exhales (4s in, 8s out)
|
||||
3. Repeat 3–4 times
|
||||
|
||||
### Envy & Comparison Alchemy
|
||||
**Duration**: 3 minutes
|
||||
**Trigger**: Post-social-media, envy signals
|
||||
**Instructions**:
|
||||
1. Name the envy: "I feel envious of ___"
|
||||
2. Ask: "What does this envy tell me I actually want?"
|
||||
3. Convert: "My next step toward that is ___"
|
||||
**Effect**: Converts envy into a desire-signal that identifies personal values
|
||||
|
||||
### Awe Induction
|
||||
**Duration**: 3–5 minutes
|
||||
**Trigger**: Existential flatness, low engagement, loss of meaning
|
||||
**Instructions**:
|
||||
1. Imagine standing at the edge of the Grand Canyon, or beneath a starry sky
|
||||
2. Let yourself feel the scale — you are small, and that's beautiful
|
||||
3. Recall a moment of genuine wonder from your past
|
||||
4. Notice what changes in your body
|
||||
**Effect**: Counters hedonic adaptation, restores sense of meaning
|
||||
|
||||
---
|
||||
|
||||
## Sleep & Recovery Protocols
|
||||
|
||||
### Ultradian Reset
|
||||
**Duration**: 20 minutes
|
||||
**Trigger**: End of a 90-minute focus block, drowsiness rising
|
||||
**Instructions**:
|
||||
1. Set a timer for 20 minutes
|
||||
2. No agenda — just rest (don't force sleep)
|
||||
3. Dim lights if possible, close eyes
|
||||
4. Let mind wander without structure
|
||||
**Effect**: Aligns with 90-minute ultradian rhythm, restores cognitive resources
|
||||
|
||||
### Wake Reset
|
||||
**Duration**: 5 minutes
|
||||
**Trigger**: narcolepsy_index > 40, severe drowsiness
|
||||
**Instructions**:
|
||||
1. Splash cold water on your face and wrists
|
||||
2. Do 20 seconds of Kapalabhati breath (sharp nasal exhales)
|
||||
3. Expose yourself to bright light for 2–3 minutes
|
||||
**Effect**: Acute arousal response, suppresses drowsiness
|
||||
|
||||
### NSDR (Non-Sleep Deep Rest / Yoga Nidra)
|
||||
**Duration**: 20–30 minutes
|
||||
**Trigger**: Accumulated fatigue, need deep recovery without sleeping
|
||||
**Instructions**:
|
||||
1. Lie on your back, palms up
|
||||
2. Close your eyes and do a slow body scan from toes to crown
|
||||
3. At each body part, notice sensation without changing anything
|
||||
4. If you fall asleep, that's fine — set an alarm
|
||||
**Effect**: Restores dopamine and cognitive resources without sleep inertia
|
||||
|
||||
### Power Nap
|
||||
**Duration**: 10–20 minutes (set alarm!)
|
||||
**Trigger**: Drowsiness > 0.70, post-lunch slump, Theta dominant
|
||||
**Instructions**:
|
||||
1. Set alarm for 20 minutes maximum (avoids N3 sleep inertia)
|
||||
2. Lie down or recline
|
||||
3. Even if you don't fully sleep, rest with eyes closed
|
||||
4. On waking: 30 seconds of stretching before resuming work
|
||||
**Effect**: Restores focus and alertness for 2–3 hours
|
||||
|
||||
### Sleep Wind-Down
|
||||
**Duration**: 60 minutes before bed
|
||||
**Trigger**: Evening session, rising drowsiness, pre-sleep
|
||||
**Instructions**:
|
||||
1. Dim all screens to night mode
|
||||
2. Stop new learning or complex tasks
|
||||
3. Do a mind dump of tomorrow's tasks
|
||||
4. 10 minutes of progressive relaxation or 4-7-8 breathing
|
||||
5. Keep room cool (65–68°F / 18–20°C)
|
||||
|
||||
---
|
||||
|
||||
## Somatic & Physical Protocols
|
||||
|
||||
### Progressive Muscle Relaxation (PMR)
|
||||
**Duration**: 10 minutes
|
||||
**Trigger**: Relaxation < 0.25, HRV declining over session
|
||||
**Instructions**:
|
||||
1. Start with feet — tense for 5 seconds, release for 8–10 seconds
|
||||
2. Move upward: calves → thighs → abdomen → hands → arms → shoulders → face
|
||||
3. Hold each tension 5 seconds, release 8–10 seconds
|
||||
4. End with 3 deep breaths
|
||||
|
||||
### Grounding (5-4-3-2-1)
|
||||
**Duration**: 3 minutes
|
||||
**Trigger**: Panic, dissociation, acute anxiety spike
|
||||
**Instructions**:
|
||||
1. Name 5 things you can see
|
||||
2. Name 4 things you can touch
|
||||
3. Name 3 things you can hear
|
||||
4. Name 2 things you can smell
|
||||
5. Name 1 thing you can taste
|
||||
|
||||
### 20-20-20 Vision Reset
|
||||
**Duration**: 20 seconds
|
||||
**Trigger**: Extended screen time, eye strain
|
||||
**Instructions**:
|
||||
1. Every 20 minutes of screen time
|
||||
2. Look at something 20 feet away
|
||||
3. For 20 seconds
|
||||
|
||||
### Neck Release Sequence
|
||||
**Duration**: 3 minutes
|
||||
**Trigger**: High stillness (> 0.85) + headache_index elevated
|
||||
**Instructions**:
|
||||
1. Ear-to-shoulder tilt — hold 15 seconds each side
|
||||
2. Chin tucks — 10 reps (pull chin straight back)
|
||||
3. Gentle neck circles — 5 each direction
|
||||
4. Shoulder shrugs — 10 reps (squeeze up, release)
|
||||
|
||||
### Motor Cortex Activation
|
||||
**Duration**: 2 minutes
|
||||
**Trigger**: Very high stillness, prolonged static sitting
|
||||
**Instructions**:
|
||||
1. Cross-body movements: touch right hand to left knee, alternate 10 times
|
||||
2. Shake out hands and feet for 15 seconds
|
||||
3. Roll ankles and wrists 5 times each direction
|
||||
**Effect**: Resets proprioception, activates motor cortex
|
||||
|
||||
### Cognitive Load Offload (Mind Dump)
|
||||
**Duration**: 5 minutes
|
||||
**Trigger**: Cognitive load > 0.70 sustained, racing thoughts, high beta
|
||||
**Instructions**:
|
||||
1. Open a blank document or grab paper
|
||||
2. Write everything on your mind without filtering or organizing
|
||||
3. Brain-dump worries, tasks, ideas — anything occupying working memory
|
||||
4. Close the document (review later if needed)
|
||||
**Effect**: Externalizing working memory can reduce cognitive load by 20–40%
|
||||
|
||||
---
|
||||
|
||||
## Digital & Lifestyle Protocols
|
||||
|
||||
### Craving Surf
|
||||
**Duration**: 90 seconds
|
||||
**Trigger**: Phone addiction signals, urge to check social media
|
||||
**Instructions**:
|
||||
1. Notice the urge to check your phone
|
||||
2. Don't act on it — just observe for 90 seconds
|
||||
3. Notice: does the urge peak and then fade?
|
||||
4. Resume what you were doing
|
||||
**Effect**: Breaks automatic dopamine-seeking loop
|
||||
|
||||
### Dopamine Palette Reset
|
||||
**Duration**: Ongoing
|
||||
**Trigger**: Flatness from short-form content spikes
|
||||
**Instructions**:
|
||||
1. Identify activities that provide sustained reward (reading, cooking, walking)
|
||||
2. Replace 15 minutes of scrolling with one sustained-reward activity
|
||||
3. Track mood before/after for 3 days
|
||||
|
||||
### Digital Sunset
|
||||
**Duration**: 60–90 minutes before bed
|
||||
**Trigger**: Evening, pre-sleep routine
|
||||
**Instructions**:
|
||||
1. Hard stop on all screens 60–90 minutes before bed
|
||||
2. Switch to non-screen activities: reading, conversation, stretching
|
||||
3. If screens are necessary, use night mode at minimum brightness
|
||||
|
||||
---
|
||||
|
||||
## Dietary Protocols
|
||||
|
||||
### Caffeine Timing
|
||||
**Trigger**: Morning routine, anxiety_index
|
||||
**Guidelines**:
|
||||
- Consume caffeine 90–120 minutes after waking (cortisol has already peaked)
|
||||
- None after 2 PM (half-life ~6 hours)
|
||||
- If anxiety_index > 50, stack with L-theanine (200mg) to smooth the curve
|
||||
|
||||
### Post-Meal Energy Crash
|
||||
**Trigger**: Post-lunch drowsiness spike
|
||||
**Instructions**:
|
||||
1. 5-minute brisk walk immediately after eating
|
||||
2. 10 minutes of sunlight exposure
|
||||
**Effect**: Counters post-prandial drowsiness
|
||||
|
||||
---
|
||||
|
||||
## Motivation & Planning Protocols
|
||||
|
||||
### WOOP (Wish, Outcome, Obstacle, Plan)
|
||||
**Duration**: 5 minutes
|
||||
**Trigger**: Low engagement before a task
|
||||
**Instructions**:
|
||||
1. **Wish**: What do you want to accomplish in this session?
|
||||
2. **Outcome**: What's the best possible result? Visualize it.
|
||||
3. **Obstacle**: What internal obstacle might get in the way?
|
||||
4. **Plan**: "If [obstacle], then I will [action]."
|
||||
**Effect**: Mental contrasting improves follow-through by 2–3x
|
||||
|
||||
### Pre-Task Priming
|
||||
**Duration**: 3 minutes
|
||||
**Trigger**: Low engagement at session start, drowsiness < 0.50
|
||||
**Instructions**:
|
||||
1. Set a clear intention for the next work block
|
||||
2. Write down the single most important task
|
||||
3. Do 10 jumping jacks or 20 deep breaths
|
||||
4. Start with the easiest sub-task to build momentum
|
||||
|
||||
---
|
||||
|
||||
## Protocol Execution Guidelines
|
||||
|
||||
When guiding the user through a protocol:
|
||||
1. **Match one protocol** to the single most salient metric signal
|
||||
2. **Explain the metric connection** — why this protocol for this state
|
||||
3. **Ask permission** — never start without the user's consent
|
||||
4. **Announce each step** clearly with timing
|
||||
5. **Check in after** — run `npx neuroskill status --json` to see if metrics improved
|
||||
6. **Label the moment** — `npx neuroskill label "post-protocol: [name]"` for tracking
|
||||
|
||||
### Timing Guidelines for Step-by-Step Guidance
|
||||
- Breath inhale: 3–5 seconds
|
||||
- Breath hold: 2–4 seconds
|
||||
- Breath exhale: 4–8 seconds
|
||||
- Muscle tense: 5 seconds
|
||||
- Muscle release: 8–10 seconds
|
||||
- Body-scan region: 10–15 seconds
|
||||
3
hermes_code/optional-skills/mcp/DESCRIPTION.md
Normal file
3
hermes_code/optional-skills/mcp/DESCRIPTION.md
Normal file
|
|
@ -0,0 +1,3 @@
|
|||
# MCP
|
||||
|
||||
Skills for building, testing, and deploying MCP (Model Context Protocol) servers.
|
||||
299
hermes_code/optional-skills/mcp/fastmcp/SKILL.md
Normal file
299
hermes_code/optional-skills/mcp/fastmcp/SKILL.md
Normal file
|
|
@ -0,0 +1,299 @@
|
|||
---
|
||||
name: fastmcp
|
||||
description: Build, test, inspect, install, and deploy MCP servers with FastMCP in Python. Use when creating a new MCP server, wrapping an API or database as MCP tools, exposing resources or prompts, or preparing a FastMCP server for Claude Code, Cursor, or HTTP deployment.
|
||||
version: 1.0.0
|
||||
author: Hermes Agent
|
||||
license: MIT
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [MCP, FastMCP, Python, Tools, Resources, Prompts, Deployment]
|
||||
homepage: https://gofastmcp.com
|
||||
related_skills: [native-mcp, mcporter]
|
||||
prerequisites:
|
||||
commands: [python3]
|
||||
---
|
||||
|
||||
# FastMCP
|
||||
|
||||
Build MCP servers in Python with FastMCP, validate them locally, install them into MCP clients, and deploy them as HTTP endpoints.
|
||||
|
||||
## When to Use
|
||||
|
||||
Use this skill when the task is to:
|
||||
|
||||
- create a new MCP server in Python
|
||||
- wrap an API, database, CLI, or file-processing workflow as MCP tools
|
||||
- expose resources or prompts in addition to tools
|
||||
- smoke-test a server with the FastMCP CLI before wiring it into Hermes or another client
|
||||
- install a server into Claude Code, Claude Desktop, Cursor, or a similar MCP client
|
||||
- prepare a FastMCP server repo for HTTP deployment
|
||||
|
||||
Use `native-mcp` when the server already exists and only needs to be connected to Hermes. Use `mcporter` when the goal is ad-hoc CLI access to an existing MCP server instead of building one.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Install FastMCP in the working environment first:
|
||||
|
||||
```bash
|
||||
pip install fastmcp
|
||||
fastmcp version
|
||||
```
|
||||
|
||||
For the API template, install `httpx` if it is not already present:
|
||||
|
||||
```bash
|
||||
pip install httpx
|
||||
```
|
||||
|
||||
## Included Files
|
||||
|
||||
### Templates
|
||||
|
||||
- `templates/api_wrapper.py` - REST API wrapper with auth header support
|
||||
- `templates/database_server.py` - read-only SQLite query server
|
||||
- `templates/file_processor.py` - text-file inspection and search server
|
||||
|
||||
### Scripts
|
||||
|
||||
- `scripts/scaffold_fastmcp.py` - copy a starter template and replace the server name placeholder
|
||||
|
||||
### References
|
||||
|
||||
- `references/fastmcp-cli.md` - FastMCP CLI workflow, installation targets, and deployment checks
|
||||
|
||||
## Workflow
|
||||
|
||||
### 1. Pick the Smallest Viable Server Shape
|
||||
|
||||
Choose the narrowest useful surface area first:
|
||||
|
||||
- API wrapper: start with 1-3 high-value endpoints, not the whole API
|
||||
- database server: expose read-only introspection and a constrained query path
|
||||
- file processor: expose deterministic operations with explicit path arguments
|
||||
- prompts/resources: add only when the client needs reusable prompt templates or discoverable documents
|
||||
|
||||
Prefer a thin server with good names, docstrings, and schemas over a large server with vague tools.
|
||||
|
||||
### 2. Scaffold from a Template
|
||||
|
||||
Copy a template directly or use the scaffold helper:
|
||||
|
||||
```bash
|
||||
python ~/.hermes/skills/mcp/fastmcp/scripts/scaffold_fastmcp.py \
|
||||
--template api_wrapper \
|
||||
--name "Acme API" \
|
||||
--output ./acme_server.py
|
||||
```
|
||||
|
||||
Available templates:
|
||||
|
||||
```bash
|
||||
python ~/.hermes/skills/mcp/fastmcp/scripts/scaffold_fastmcp.py --list
|
||||
```
|
||||
|
||||
If copying manually, replace `__SERVER_NAME__` with a real server name.
|
||||
|
||||
### 3. Implement Tools First
|
||||
|
||||
Start with `@mcp.tool` functions before adding resources or prompts.
|
||||
|
||||
Rules for tool design:
|
||||
|
||||
- Give every tool a concrete verb-based name
|
||||
- Write docstrings as user-facing tool descriptions
|
||||
- Keep parameters explicit and typed
|
||||
- Return structured JSON-safe data where possible
|
||||
- Validate unsafe inputs early
|
||||
- Prefer read-only behavior by default for first versions
|
||||
|
||||
Good tool examples:
|
||||
|
||||
- `get_customer`
|
||||
- `search_tickets`
|
||||
- `describe_table`
|
||||
- `summarize_text_file`
|
||||
|
||||
Weak tool examples:
|
||||
|
||||
- `run`
|
||||
- `process`
|
||||
- `do_thing`
|
||||
|
||||
### 4. Add Resources and Prompts Only When They Help
|
||||
|
||||
Add `@mcp.resource` when the client benefits from fetching stable read-only content such as schemas, policy docs, or generated reports.
|
||||
|
||||
Add `@mcp.prompt` when the server should provide a reusable prompt template for a known workflow.
|
||||
|
||||
Do not turn every document into a prompt. Prefer:
|
||||
|
||||
- tools for actions
|
||||
- resources for data/document retrieval
|
||||
- prompts for reusable LLM instructions
|
||||
|
||||
### 5. Test the Server Before Integrating It Anywhere
|
||||
|
||||
Use the FastMCP CLI for local validation:
|
||||
|
||||
```bash
|
||||
fastmcp inspect acme_server.py:mcp
|
||||
fastmcp list acme_server.py --json
|
||||
fastmcp call acme_server.py search_resources query=router limit=5 --json
|
||||
```
|
||||
|
||||
For fast iterative debugging, run the server locally:
|
||||
|
||||
```bash
|
||||
fastmcp run acme_server.py:mcp
|
||||
```
|
||||
|
||||
To test HTTP transport locally:
|
||||
|
||||
```bash
|
||||
fastmcp run acme_server.py:mcp --transport http --host 127.0.0.1 --port 8000
|
||||
fastmcp list http://127.0.0.1:8000/mcp --json
|
||||
fastmcp call http://127.0.0.1:8000/mcp search_resources query=router --json
|
||||
```
|
||||
|
||||
Always run at least one real `fastmcp call` against each new tool before claiming the server works.
|
||||
|
||||
### 6. Install into a Client When Local Validation Passes
|
||||
|
||||
FastMCP can register the server with supported MCP clients:
|
||||
|
||||
```bash
|
||||
fastmcp install claude-code acme_server.py
|
||||
fastmcp install claude-desktop acme_server.py
|
||||
fastmcp install cursor acme_server.py -e .
|
||||
```
|
||||
|
||||
Use `fastmcp discover` to inspect named MCP servers already configured on the machine.
|
||||
|
||||
When the goal is Hermes integration, either:
|
||||
|
||||
- configure the server in `~/.hermes/config.yaml` using the `native-mcp` skill, or
|
||||
- keep using FastMCP CLI commands during development until the interface stabilizes
|
||||
|
||||
### 7. Deploy After the Local Contract Is Stable
|
||||
|
||||
For managed hosting, Prefect Horizon is the path FastMCP documents most directly. Before deployment:
|
||||
|
||||
```bash
|
||||
fastmcp inspect acme_server.py:mcp
|
||||
```
|
||||
|
||||
Make sure the repo contains:
|
||||
|
||||
- a Python file with the FastMCP server object
|
||||
- `requirements.txt` or `pyproject.toml`
|
||||
- any environment-variable documentation needed for deployment
|
||||
|
||||
For generic HTTP hosting, validate the HTTP transport locally first, then deploy on any Python-compatible platform that can expose the server port.
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### API Wrapper Pattern
|
||||
|
||||
Use when exposing a REST or HTTP API as MCP tools.
|
||||
|
||||
Recommended first slice:
|
||||
|
||||
- one read path
|
||||
- one list/search path
|
||||
- optional health check
|
||||
|
||||
Implementation notes:
|
||||
|
||||
- keep auth in environment variables, not hardcoded
|
||||
- centralize request logic in one helper
|
||||
- surface API errors with concise context
|
||||
- normalize inconsistent upstream payloads before returning them
|
||||
|
||||
Start from `templates/api_wrapper.py`.
|
||||
|
||||
### Database Pattern
|
||||
|
||||
Use when exposing safe query and inspection capabilities.
|
||||
|
||||
Recommended first slice:
|
||||
|
||||
- `list_tables`
|
||||
- `describe_table`
|
||||
- one constrained read query tool
|
||||
|
||||
Implementation notes:
|
||||
|
||||
- default to read-only DB access
|
||||
- reject non-`SELECT` SQL in early versions
|
||||
- limit row counts
|
||||
- return rows plus column names
|
||||
|
||||
Start from `templates/database_server.py`.
|
||||
|
||||
### File Processor Pattern
|
||||
|
||||
Use when the server needs to inspect or transform files on demand.
|
||||
|
||||
Recommended first slice:
|
||||
|
||||
- summarize file contents
|
||||
- search within files
|
||||
- extract deterministic metadata
|
||||
|
||||
Implementation notes:
|
||||
|
||||
- accept explicit file paths
|
||||
- check for missing files and encoding failures
|
||||
- cap previews and result counts
|
||||
- avoid shelling out unless a specific external tool is required
|
||||
|
||||
Start from `templates/file_processor.py`.
|
||||
|
||||
## Quality Bar
|
||||
|
||||
Before handing off a FastMCP server, verify all of the following:
|
||||
|
||||
- server imports cleanly
|
||||
- `fastmcp inspect <file.py:mcp>` succeeds
|
||||
- `fastmcp list <server spec> --json` succeeds
|
||||
- every new tool has at least one real `fastmcp call`
|
||||
- environment variables are documented
|
||||
- the tool surface is small enough to understand without guesswork
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### FastMCP command missing
|
||||
|
||||
Install the package in the active environment:
|
||||
|
||||
```bash
|
||||
pip install fastmcp
|
||||
fastmcp version
|
||||
```
|
||||
|
||||
### `fastmcp inspect` fails
|
||||
|
||||
Check that:
|
||||
|
||||
- the file imports without side effects that crash
|
||||
- the FastMCP instance is named correctly in `<file.py:object>`
|
||||
- optional dependencies from the template are installed
|
||||
|
||||
### Tool works in Python but not through CLI
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
fastmcp list server.py --json
|
||||
fastmcp call server.py your_tool_name --json
|
||||
```
|
||||
|
||||
This usually exposes naming mismatches, missing required arguments, or non-serializable return values.
|
||||
|
||||
### Hermes cannot see the deployed server
|
||||
|
||||
The server-building part may be correct while the Hermes config is not. Load the `native-mcp` skill and configure the server in `~/.hermes/config.yaml`, then restart Hermes.
|
||||
|
||||
## References
|
||||
|
||||
For CLI details, install targets, and deployment checks, read `references/fastmcp-cli.md`.
|
||||
|
|
@ -0,0 +1,110 @@
|
|||
# FastMCP CLI Reference
|
||||
|
||||
Use this file when the task needs exact FastMCP CLI workflows rather than the higher-level guidance in `SKILL.md`.
|
||||
|
||||
## Install and Verify
|
||||
|
||||
```bash
|
||||
pip install fastmcp
|
||||
fastmcp version
|
||||
```
|
||||
|
||||
FastMCP documents `pip install fastmcp` and `fastmcp version` as the baseline installation and verification path.
|
||||
|
||||
## Run a Server
|
||||
|
||||
Run a server object from a Python file:
|
||||
|
||||
```bash
|
||||
fastmcp run server.py:mcp
|
||||
```
|
||||
|
||||
Run the same server over HTTP:
|
||||
|
||||
```bash
|
||||
fastmcp run server.py:mcp --transport http --host 127.0.0.1 --port 8000
|
||||
```
|
||||
|
||||
## Inspect a Server
|
||||
|
||||
Inspect what FastMCP will expose:
|
||||
|
||||
```bash
|
||||
fastmcp inspect server.py:mcp
|
||||
```
|
||||
|
||||
This is also the check FastMCP recommends before deploying to Prefect Horizon.
|
||||
|
||||
## List and Call Tools
|
||||
|
||||
List tools from a Python file:
|
||||
|
||||
```bash
|
||||
fastmcp list server.py --json
|
||||
```
|
||||
|
||||
List tools from an HTTP endpoint:
|
||||
|
||||
```bash
|
||||
fastmcp list http://127.0.0.1:8000/mcp --json
|
||||
```
|
||||
|
||||
Call a tool with key-value arguments:
|
||||
|
||||
```bash
|
||||
fastmcp call server.py search_resources query=router limit=5 --json
|
||||
```
|
||||
|
||||
Call a tool with a full JSON input payload:
|
||||
|
||||
```bash
|
||||
fastmcp call server.py create_item '{"name": "Widget", "tags": ["sale"]}' --json
|
||||
```
|
||||
|
||||
## Discover Named MCP Servers
|
||||
|
||||
Find named servers already configured in local MCP-aware tools:
|
||||
|
||||
```bash
|
||||
fastmcp discover
|
||||
```
|
||||
|
||||
FastMCP documents name-based resolution for Claude Desktop, Claude Code, Cursor, Gemini, Goose, and `./mcp.json`.
|
||||
|
||||
## Install into MCP Clients
|
||||
|
||||
Register a server with common clients:
|
||||
|
||||
```bash
|
||||
fastmcp install claude-code server.py
|
||||
fastmcp install claude-desktop server.py
|
||||
fastmcp install cursor server.py -e .
|
||||
```
|
||||
|
||||
FastMCP notes that client installs run in isolated environments, so declare dependencies explicitly when needed with flags such as `--with`, `--env-file`, or editable installs.
|
||||
|
||||
## Deployment Checks
|
||||
|
||||
### Prefect Horizon
|
||||
|
||||
Before pushing to Horizon:
|
||||
|
||||
```bash
|
||||
fastmcp inspect server.py:mcp
|
||||
```
|
||||
|
||||
FastMCP’s Horizon docs expect:
|
||||
|
||||
- a GitHub repo
|
||||
- a Python file containing the FastMCP server object
|
||||
- dependencies declared in `requirements.txt` or `pyproject.toml`
|
||||
- an entrypoint like `main.py:mcp`
|
||||
|
||||
### Generic HTTP Hosting
|
||||
|
||||
Before shipping to any other host:
|
||||
|
||||
1. Start the server locally with HTTP transport.
|
||||
2. Verify `fastmcp list` against the local `/mcp` URL.
|
||||
3. Verify at least one `fastmcp call`.
|
||||
4. Document required environment variables.
|
||||
|
|
@ -0,0 +1,56 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Copy a FastMCP starter template into a working file."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
SCRIPT_DIR = Path(__file__).resolve().parent
|
||||
SKILL_DIR = SCRIPT_DIR.parent
|
||||
TEMPLATE_DIR = SKILL_DIR / "templates"
|
||||
PLACEHOLDER = "__SERVER_NAME__"
|
||||
|
||||
|
||||
def list_templates() -> list[str]:
|
||||
return sorted(path.stem for path in TEMPLATE_DIR.glob("*.py"))
|
||||
|
||||
|
||||
def render_template(template_name: str, server_name: str) -> str:
|
||||
template_path = TEMPLATE_DIR / f"{template_name}.py"
|
||||
if not template_path.exists():
|
||||
available = ", ".join(list_templates())
|
||||
raise SystemExit(f"Unknown template '{template_name}'. Available: {available}")
|
||||
return template_path.read_text(encoding="utf-8").replace(PLACEHOLDER, server_name)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--template", help="Template name without .py suffix")
|
||||
parser.add_argument("--name", help="FastMCP server display name")
|
||||
parser.add_argument("--output", help="Destination Python file path")
|
||||
parser.add_argument("--force", action="store_true", help="Overwrite an existing output file")
|
||||
parser.add_argument("--list", action="store_true", help="List available templates and exit")
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.list:
|
||||
for name in list_templates():
|
||||
print(name)
|
||||
return 0
|
||||
|
||||
if not args.template or not args.name or not args.output:
|
||||
parser.error("--template, --name, and --output are required unless --list is used")
|
||||
|
||||
output_path = Path(args.output).expanduser()
|
||||
if output_path.exists() and not args.force:
|
||||
raise SystemExit(f"Refusing to overwrite existing file: {output_path}")
|
||||
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
output_path.write_text(render_template(args.template, args.name), encoding="utf-8")
|
||||
print(f"Wrote {output_path}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
|
|
@ -0,0 +1,54 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
from fastmcp import FastMCP
|
||||
|
||||
|
||||
mcp = FastMCP("__SERVER_NAME__")
|
||||
|
||||
API_BASE_URL = os.getenv("API_BASE_URL", "https://api.example.com")
|
||||
API_TOKEN = os.getenv("API_TOKEN")
|
||||
REQUEST_TIMEOUT = float(os.getenv("API_TIMEOUT_SECONDS", "20"))
|
||||
|
||||
|
||||
def _headers() -> dict[str, str]:
|
||||
headers = {"Accept": "application/json"}
|
||||
if API_TOKEN:
|
||||
headers["Authorization"] = f"Bearer {API_TOKEN}"
|
||||
return headers
|
||||
|
||||
|
||||
def _request(method: str, path: str, *, params: dict[str, Any] | None = None) -> Any:
|
||||
url = f"{API_BASE_URL.rstrip('/')}/{path.lstrip('/')}"
|
||||
with httpx.Client(timeout=REQUEST_TIMEOUT, headers=_headers()) as client:
|
||||
response = client.request(method, url, params=params)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
|
||||
@mcp.tool
|
||||
def health_check() -> dict[str, Any]:
|
||||
"""Check whether the upstream API is reachable."""
|
||||
payload = _request("GET", "/health")
|
||||
return {"base_url": API_BASE_URL, "result": payload}
|
||||
|
||||
|
||||
@mcp.tool
|
||||
def get_resource(resource_id: str) -> dict[str, Any]:
|
||||
"""Fetch one resource by ID from the upstream API."""
|
||||
payload = _request("GET", f"/resources/{resource_id}")
|
||||
return {"resource_id": resource_id, "data": payload}
|
||||
|
||||
|
||||
@mcp.tool
|
||||
def search_resources(query: str, limit: int = 10) -> dict[str, Any]:
|
||||
"""Search upstream resources by query string."""
|
||||
payload = _request("GET", "/resources", params={"q": query, "limit": limit})
|
||||
return {"query": query, "limit": limit, "results": payload}
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
mcp.run()
|
||||
|
|
@ -0,0 +1,77 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import sqlite3
|
||||
from typing import Any
|
||||
|
||||
from fastmcp import FastMCP
|
||||
|
||||
|
||||
mcp = FastMCP("__SERVER_NAME__")
|
||||
|
||||
DATABASE_PATH = os.getenv("SQLITE_PATH", "./app.db")
|
||||
MAX_ROWS = int(os.getenv("SQLITE_MAX_ROWS", "200"))
|
||||
TABLE_NAME_RE = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$")
|
||||
|
||||
|
||||
def _connect() -> sqlite3.Connection:
|
||||
return sqlite3.connect(f"file:{DATABASE_PATH}?mode=ro", uri=True)
|
||||
|
||||
|
||||
def _reject_mutation(sql: str) -> None:
|
||||
normalized = sql.strip().lower()
|
||||
if not normalized.startswith("select"):
|
||||
raise ValueError("Only SELECT queries are allowed")
|
||||
|
||||
|
||||
def _validate_table_name(table_name: str) -> str:
|
||||
if not TABLE_NAME_RE.fullmatch(table_name):
|
||||
raise ValueError("Invalid table name")
|
||||
return table_name
|
||||
|
||||
|
||||
@mcp.tool
|
||||
def list_tables() -> list[str]:
|
||||
"""List user-defined SQLite tables."""
|
||||
with _connect() as conn:
|
||||
rows = conn.execute(
|
||||
"SELECT name FROM sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%' ORDER BY name"
|
||||
).fetchall()
|
||||
return [row[0] for row in rows]
|
||||
|
||||
|
||||
@mcp.tool
|
||||
def describe_table(table_name: str) -> list[dict[str, Any]]:
|
||||
"""Describe columns for a SQLite table."""
|
||||
safe_table_name = _validate_table_name(table_name)
|
||||
with _connect() as conn:
|
||||
rows = conn.execute(f"PRAGMA table_info({safe_table_name})").fetchall()
|
||||
return [
|
||||
{
|
||||
"cid": row[0],
|
||||
"name": row[1],
|
||||
"type": row[2],
|
||||
"notnull": bool(row[3]),
|
||||
"default": row[4],
|
||||
"pk": bool(row[5]),
|
||||
}
|
||||
for row in rows
|
||||
]
|
||||
|
||||
|
||||
@mcp.tool
|
||||
def query(sql: str, limit: int = 50) -> dict[str, Any]:
|
||||
"""Run a read-only SELECT query and return rows plus column names."""
|
||||
_reject_mutation(sql)
|
||||
safe_limit = max(0, min(limit, MAX_ROWS))
|
||||
wrapped_sql = f"SELECT * FROM ({sql.strip().rstrip(';')}) LIMIT {safe_limit}"
|
||||
with _connect() as conn:
|
||||
cursor = conn.execute(wrapped_sql)
|
||||
columns = [column[0] for column in cursor.description or []]
|
||||
rows = [dict(zip(columns, row)) for row in cursor.fetchall()]
|
||||
return {"limit": safe_limit, "columns": columns, "rows": rows}
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
mcp.run()
|
||||
|
|
@ -0,0 +1,55 @@
|
|||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from fastmcp import FastMCP
|
||||
|
||||
|
||||
mcp = FastMCP("__SERVER_NAME__")
|
||||
|
||||
|
||||
def _read_text(path: str) -> str:
|
||||
file_path = Path(path).expanduser()
|
||||
try:
|
||||
return file_path.read_text(encoding="utf-8")
|
||||
except FileNotFoundError as exc:
|
||||
raise ValueError(f"File not found: {file_path}") from exc
|
||||
except UnicodeDecodeError as exc:
|
||||
raise ValueError(f"File is not valid UTF-8 text: {file_path}") from exc
|
||||
|
||||
|
||||
@mcp.tool
|
||||
def summarize_text_file(path: str, preview_chars: int = 1200) -> dict[str, int | str]:
|
||||
"""Return basic metadata and a preview for a UTF-8 text file."""
|
||||
file_path = Path(path).expanduser()
|
||||
text = _read_text(path)
|
||||
return {
|
||||
"path": str(file_path),
|
||||
"characters": len(text),
|
||||
"lines": len(text.splitlines()),
|
||||
"preview": text[:preview_chars],
|
||||
}
|
||||
|
||||
|
||||
@mcp.tool
|
||||
def search_text_file(path: str, needle: str, max_matches: int = 20) -> dict[str, Any]:
|
||||
"""Find matching lines in a UTF-8 text file."""
|
||||
file_path = Path(path).expanduser()
|
||||
matches: list[dict[str, Any]] = []
|
||||
for line_number, line in enumerate(_read_text(path).splitlines(), start=1):
|
||||
if needle.lower() in line.lower():
|
||||
matches.append({"line_number": line_number, "line": line})
|
||||
if len(matches) >= max_matches:
|
||||
break
|
||||
return {"path": str(file_path), "needle": needle, "matches": matches}
|
||||
|
||||
|
||||
@mcp.resource("file://{path}")
|
||||
def read_file_resource(path: str) -> str:
|
||||
"""Expose a text file as a resource."""
|
||||
return _read_text(path)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
mcp.run()
|
||||
2
hermes_code/optional-skills/migration/DESCRIPTION.md
Normal file
2
hermes_code/optional-skills/migration/DESCRIPTION.md
Normal file
|
|
@ -0,0 +1,2 @@
|
|||
Optional migration workflows for importing user state and customizations from
|
||||
other agent systems into Hermes Agent.
|
||||
|
|
@ -0,0 +1,297 @@
|
|||
---
|
||||
name: openclaw-migration
|
||||
description: Migrate a user's OpenClaw customization footprint into Hermes Agent. Imports Hermes-compatible memories, SOUL.md, command allowlists, user skills, and selected workspace assets from ~/.openclaw, then reports exactly what could not be migrated and why.
|
||||
version: 1.0.0
|
||||
author: Hermes Agent (Nous Research)
|
||||
license: MIT
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Migration, OpenClaw, Hermes, Memory, Persona, Import]
|
||||
related_skills: [hermes-agent]
|
||||
---
|
||||
|
||||
# OpenClaw -> Hermes Migration
|
||||
|
||||
Use this skill when a user wants to move their OpenClaw setup into Hermes Agent with minimal manual cleanup.
|
||||
|
||||
## CLI Command
|
||||
|
||||
For a quick, non-interactive migration, use the built-in CLI command:
|
||||
|
||||
```bash
|
||||
hermes claw migrate # Full interactive migration
|
||||
hermes claw migrate --dry-run # Preview what would be migrated
|
||||
hermes claw migrate --preset user-data # Migrate without secrets
|
||||
hermes claw migrate --overwrite # Overwrite existing conflicts
|
||||
hermes claw migrate --source /custom/path/.openclaw # Custom source
|
||||
```
|
||||
|
||||
The CLI command runs the same migration script described below. Use this skill (via the agent) when you want an interactive, guided migration with dry-run previews and per-item conflict resolution.
|
||||
|
||||
**First-time setup:** The `hermes setup` wizard automatically detects `~/.openclaw` and offers migration before configuration begins.
|
||||
|
||||
## What this skill does
|
||||
|
||||
It uses `scripts/openclaw_to_hermes.py` to:
|
||||
|
||||
- import `SOUL.md` into the Hermes home directory as `SOUL.md`
|
||||
- transform OpenClaw `MEMORY.md` and `USER.md` into Hermes memory entries
|
||||
- merge OpenClaw command approval patterns into Hermes `command_allowlist`
|
||||
- migrate Hermes-compatible messaging settings such as `TELEGRAM_ALLOWED_USERS` and `MESSAGING_CWD`
|
||||
- copy OpenClaw skills into `~/.hermes/skills/openclaw-imports/`
|
||||
- optionally copy the OpenClaw workspace instructions file into a chosen Hermes workspace
|
||||
- mirror compatible workspace assets such as `workspace/tts/` into `~/.hermes/tts/`
|
||||
- archive non-secret docs that do not have a direct Hermes destination
|
||||
- produce a structured report listing migrated items, conflicts, skipped items, and reasons
|
||||
|
||||
## Path resolution
|
||||
|
||||
The helper script lives in this skill directory at:
|
||||
|
||||
- `scripts/openclaw_to_hermes.py`
|
||||
|
||||
When this skill is installed from the Skills Hub, the normal location is:
|
||||
|
||||
- `~/.hermes/skills/migration/openclaw-migration/scripts/openclaw_to_hermes.py`
|
||||
|
||||
Do not guess a shorter path like `~/.hermes/skills/openclaw-migration/...`.
|
||||
|
||||
Before running the helper:
|
||||
|
||||
1. Prefer the installed path under `~/.hermes/skills/migration/openclaw-migration/`.
|
||||
2. If that path fails, inspect the installed skill directory and resolve the script relative to the installed `SKILL.md`.
|
||||
3. Only use `find` as a fallback if the installed location is missing or the skill was moved manually.
|
||||
4. When calling the terminal tool, do not pass `workdir: "~"`. Use an absolute directory such as the user's home directory, or omit `workdir` entirely.
|
||||
|
||||
With `--migrate-secrets`, it will also import a small allowlisted set of Hermes-compatible secrets, currently:
|
||||
|
||||
- `TELEGRAM_BOT_TOKEN`
|
||||
|
||||
## Default workflow
|
||||
|
||||
1. Inspect first with a dry run.
|
||||
2. Present a simple summary of what can be migrated, what cannot be migrated, and what would be archived.
|
||||
3. If the `clarify` tool is available, use it for user decisions instead of asking for a free-form prose reply.
|
||||
4. If the dry run finds imported skill directory conflicts, ask how those should be handled before executing.
|
||||
5. Ask the user to choose between the two supported migration modes before executing.
|
||||
6. Ask for a target workspace path only if the user wants the workspace instructions file brought over.
|
||||
7. Execute the migration with the matching preset and flags.
|
||||
8. Summarize the results, especially:
|
||||
- what was migrated
|
||||
- what was archived for manual review
|
||||
- what was skipped and why
|
||||
|
||||
## User interaction protocol
|
||||
|
||||
Hermes CLI supports the `clarify` tool for interactive prompts, but it is limited to:
|
||||
|
||||
- one choice at a time
|
||||
- up to 4 predefined choices
|
||||
- an automatic `Other` free-text option
|
||||
|
||||
It does **not** support true multi-select checkboxes in a single prompt.
|
||||
|
||||
For every `clarify` call:
|
||||
|
||||
- always include a non-empty `question`
|
||||
- include `choices` only for real selectable prompts
|
||||
- keep `choices` to 2-4 plain string options
|
||||
- never emit placeholder or truncated options such as `...`
|
||||
- never pad or stylize choices with extra whitespace
|
||||
- never include fake form fields in the question such as `enter directory here`, blank lines to fill in, or underscores like `_____`
|
||||
- for open-ended path questions, ask only the plain sentence; the user types in the normal CLI prompt below the panel
|
||||
|
||||
If a `clarify` call returns an error, inspect the error text, correct the payload, and retry once with a valid `question` and clean choices.
|
||||
|
||||
When `clarify` is available and the dry run reveals any required user decision, your **next action must be a `clarify` tool call**.
|
||||
Do not end the turn with a normal assistant message such as:
|
||||
|
||||
- "Let me present the choices"
|
||||
- "What would you like to do?"
|
||||
- "Here are the options"
|
||||
|
||||
If a user decision is required, collect it via `clarify` before producing more prose.
|
||||
If multiple unresolved decisions remain, do not insert an explanatory assistant message between them. After one `clarify` response is received, your next action should usually be the next required `clarify` call.
|
||||
|
||||
Treat `workspace-agents` as an unresolved decision whenever the dry run reports:
|
||||
|
||||
- `kind="workspace-agents"`
|
||||
- `status="skipped"`
|
||||
- reason containing `No workspace target was provided`
|
||||
|
||||
In that case, you must ask about workspace instructions before execution. Do not silently treat that as a decision to skip.
|
||||
|
||||
Because of that limitation, use this simplified decision flow:
|
||||
|
||||
1. For `SOUL.md` conflicts, use `clarify` with choices such as:
|
||||
- `keep existing`
|
||||
- `overwrite with backup`
|
||||
- `review first`
|
||||
2. If the dry run shows one or more `kind="skill"` items with `status="conflict"`, use `clarify` with choices such as:
|
||||
- `keep existing skills`
|
||||
- `overwrite conflicting skills with backup`
|
||||
- `import conflicting skills under renamed folders`
|
||||
3. For workspace instructions, use `clarify` with choices such as:
|
||||
- `skip workspace instructions`
|
||||
- `copy to a workspace path`
|
||||
- `decide later`
|
||||
4. If the user chooses to copy workspace instructions, ask a follow-up open-ended `clarify` question requesting an **absolute path**.
|
||||
5. If the user chooses `skip workspace instructions` or `decide later`, proceed without `--workspace-target`.
|
||||
5. For migration mode, use `clarify` with these 3 choices:
|
||||
- `user-data only`
|
||||
- `full compatible migration`
|
||||
- `cancel`
|
||||
6. `user-data only` means: migrate user data and compatible config, but do **not** import allowlisted secrets.
|
||||
7. `full compatible migration` means: migrate the same compatible user data plus the allowlisted secrets when present.
|
||||
8. If `clarify` is not available, ask the same question in normal text, but still constrain the answer to `user-data only`, `full compatible migration`, or `cancel`.
|
||||
|
||||
Execution gate:
|
||||
|
||||
- Do not execute while a `workspace-agents` skip caused by `No workspace target was provided` remains unresolved.
|
||||
- The only valid ways to resolve it are:
|
||||
- user explicitly chooses `skip workspace instructions`
|
||||
- user explicitly chooses `decide later`
|
||||
- user provides a workspace path after choosing `copy to a workspace path`
|
||||
- Absence of a workspace target in the dry run is not itself permission to execute.
|
||||
- Do not execute while any required `clarify` decision remains unresolved.
|
||||
|
||||
Use these exact `clarify` payload shapes as the default pattern:
|
||||
|
||||
- `{"question":"Your existing SOUL.md conflicts with the imported one. What should I do?","choices":["keep existing","overwrite with backup","review first"]}`
|
||||
- `{"question":"One or more imported OpenClaw skills already exist in Hermes. How should I handle those skill conflicts?","choices":["keep existing skills","overwrite conflicting skills with backup","import conflicting skills under renamed folders"]}`
|
||||
- `{"question":"Choose migration mode: migrate only user data, or run the full compatible migration including allowlisted secrets?","choices":["user-data only","full compatible migration","cancel"]}`
|
||||
- `{"question":"Do you want to copy the OpenClaw workspace instructions file into a Hermes workspace?","choices":["skip workspace instructions","copy to a workspace path","decide later"]}`
|
||||
- `{"question":"Please provide an absolute path where the workspace instructions should be copied."}`
|
||||
|
||||
## Decision-to-command mapping
|
||||
|
||||
Map user decisions to command flags exactly:
|
||||
|
||||
- If the user chooses `keep existing` for `SOUL.md`, do **not** add `--overwrite`.
|
||||
- If the user chooses `overwrite with backup`, add `--overwrite`.
|
||||
- If the user chooses `review first`, stop before execution and review the relevant files.
|
||||
- If the user chooses `keep existing skills`, add `--skill-conflict skip`.
|
||||
- If the user chooses `overwrite conflicting skills with backup`, add `--skill-conflict overwrite`.
|
||||
- If the user chooses `import conflicting skills under renamed folders`, add `--skill-conflict rename`.
|
||||
- If the user chooses `user-data only`, execute with `--preset user-data` and do **not** add `--migrate-secrets`.
|
||||
- If the user chooses `full compatible migration`, execute with `--preset full --migrate-secrets`.
|
||||
- Only add `--workspace-target` if the user explicitly provided an absolute workspace path.
|
||||
- If the user chooses `skip workspace instructions` or `decide later`, do not add `--workspace-target`.
|
||||
|
||||
Before executing, restate the exact command plan in plain language and make sure it matches the user's choices.
|
||||
|
||||
## Post-run reporting rules
|
||||
|
||||
After execution, treat the script's JSON output as the source of truth.
|
||||
|
||||
1. Base all counts on `report.summary`.
|
||||
2. Only list an item under "Successfully Migrated" if its `status` is exactly `migrated`.
|
||||
3. Do not claim a conflict was resolved unless the report shows that item as `migrated`.
|
||||
4. Do not say `SOUL.md` was overwritten unless the report item for `kind="soul"` has `status="migrated"`.
|
||||
5. If `report.summary.conflict > 0`, include a conflict section instead of silently implying success.
|
||||
6. If counts and listed items disagree, fix the list to match the report before responding.
|
||||
7. Include the `output_dir` path from the report when available so the user can inspect `report.json`, `summary.md`, backups, and archived files.
|
||||
8. For memory or user-profile overflow, do not say the entries were archived unless the report explicitly shows an archive path. If `details.overflow_file` exists, say the full overflow list was exported there.
|
||||
9. If a skill was imported under a renamed folder, report the final destination and mention `details.renamed_from`.
|
||||
10. If `report.skill_conflict_mode` is present, use it as the source of truth for the selected imported-skill conflict policy.
|
||||
11. If an item has `status="skipped"`, do not describe it as overwritten, backed up, migrated, or resolved.
|
||||
12. If `kind="soul"` has `status="skipped"` with reason `Target already matches source`, say it was left unchanged and do not mention a backup.
|
||||
13. If a renamed imported skill has an empty `details.backup`, do not imply the existing Hermes skill was renamed or backed up. Say only that the imported copy was placed in the new destination and reference `details.renamed_from` as the pre-existing folder that remained in place.
|
||||
|
||||
## Migration presets
|
||||
|
||||
Prefer these two presets in normal use:
|
||||
|
||||
- `user-data`
|
||||
- `full`
|
||||
|
||||
`user-data` includes:
|
||||
|
||||
- `soul`
|
||||
- `workspace-agents`
|
||||
- `memory`
|
||||
- `user-profile`
|
||||
- `messaging-settings`
|
||||
- `command-allowlist`
|
||||
- `skills`
|
||||
- `tts-assets`
|
||||
- `archive`
|
||||
|
||||
`full` includes everything in `user-data` plus:
|
||||
|
||||
- `secret-settings`
|
||||
|
||||
The helper script still supports category-level `--include` / `--exclude`, but treat that as an advanced fallback rather than the default UX.
|
||||
|
||||
## Commands
|
||||
|
||||
Dry run with full discovery:
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/migration/openclaw-migration/scripts/openclaw_to_hermes.py
|
||||
```
|
||||
|
||||
When using the terminal tool, prefer an absolute invocation pattern such as:
|
||||
|
||||
```json
|
||||
{"command":"python3 /home/USER/.hermes/skills/migration/openclaw-migration/scripts/openclaw_to_hermes.py","workdir":"/home/USER"}
|
||||
```
|
||||
|
||||
Dry run with the user-data preset:
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/migration/openclaw-migration/scripts/openclaw_to_hermes.py --preset user-data
|
||||
```
|
||||
|
||||
Execute a user-data migration:
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/migration/openclaw-migration/scripts/openclaw_to_hermes.py --execute --preset user-data --skill-conflict skip
|
||||
```
|
||||
|
||||
Execute a full compatible migration:
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/migration/openclaw-migration/scripts/openclaw_to_hermes.py --execute --preset full --migrate-secrets --skill-conflict skip
|
||||
```
|
||||
|
||||
Execute with workspace instructions included:
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/migration/openclaw-migration/scripts/openclaw_to_hermes.py --execute --preset user-data --skill-conflict rename --workspace-target "/absolute/workspace/path"
|
||||
```
|
||||
|
||||
Do not use `$PWD` or the home directory as the workspace target by default. Ask for an explicit workspace path first.
|
||||
|
||||
## Important rules
|
||||
|
||||
1. Run a dry run before writing unless the user explicitly says to proceed immediately.
|
||||
2. Do not migrate secrets by default. Tokens, auth blobs, device credentials, and raw gateway config should stay out of Hermes unless the user explicitly asks for secret migration.
|
||||
3. Do not silently overwrite non-empty Hermes targets unless the user explicitly wants that. The helper script will preserve backups when overwriting is enabled.
|
||||
4. Always give the user the skipped-items report. That report is part of the migration, not an optional extra.
|
||||
5. Prefer the primary OpenClaw workspace (`~/.openclaw/workspace/`) over `workspace.default/`. Only use the default workspace as fallback when the primary files are missing.
|
||||
6. Even in secret-migration mode, only migrate secrets with a clean Hermes destination. Unsupported auth blobs must still be reported as skipped.
|
||||
7. If the dry run shows a large asset copy, a conflicting `SOUL.md`, or overflowed memory entries, call those out separately before execution.
|
||||
8. Default to `user-data only` if the user is unsure.
|
||||
9. Only include `workspace-agents` when the user has explicitly provided a destination workspace path.
|
||||
10. Treat category-level `--include` / `--exclude` as an advanced escape hatch, not the normal flow.
|
||||
11. Do not end the dry-run summary with a vague “What would you like to do?” if `clarify` is available. Use structured follow-up prompts instead.
|
||||
12. Do not use an open-ended `clarify` prompt when a real choice prompt would work. Prefer selectable choices first, then free text only for absolute paths or file review requests.
|
||||
13. After a dry run, never stop after summarizing if there is still an unresolved decision. Use `clarify` immediately for the highest-priority blocking decision.
|
||||
14. Priority order for follow-up questions:
|
||||
- `SOUL.md` conflict
|
||||
- imported skill conflicts
|
||||
- migration mode
|
||||
- workspace instructions destination
|
||||
15. Do not promise to present choices later in the same message. Present them by actually calling `clarify`.
|
||||
16. After the migration-mode answer, explicitly check whether `workspace-agents` is still unresolved. If it is, your next action must be the workspace-instructions `clarify` call.
|
||||
17. After any `clarify` answer, if another required decision remains, do not narrate what was just decided. Ask the next required question immediately.
|
||||
|
||||
## Expected result
|
||||
|
||||
After a successful run, the user should have:
|
||||
|
||||
- Hermes persona state imported
|
||||
- Hermes memory files populated with converted OpenClaw knowledge
|
||||
- OpenClaw skills available under `~/.hermes/skills/openclaw-imports/`
|
||||
- a migration report showing any conflicts, omissions, or unsupported data
|
||||
File diff suppressed because it is too large
Load diff
417
hermes_code/optional-skills/productivity/telephony/SKILL.md
Normal file
417
hermes_code/optional-skills/productivity/telephony/SKILL.md
Normal file
|
|
@ -0,0 +1,417 @@
|
|||
---
|
||||
name: telephony
|
||||
description: Give Hermes phone capabilities without core tool changes. Provision and persist a Twilio number, send and receive SMS/MMS, make direct calls, and place AI-driven outbound calls through Bland.ai or Vapi.
|
||||
version: 1.0.0
|
||||
author: Nous Research
|
||||
license: MIT
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [telephony, phone, sms, mms, voice, twilio, bland.ai, vapi, calling, texting]
|
||||
related_skills: [find-nearby, google-workspace, agentmail]
|
||||
category: productivity
|
||||
---
|
||||
|
||||
# Telephony — Numbers, Calls, and Texts without Core Tool Changes
|
||||
|
||||
This optional skill gives Hermes practical phone capabilities while keeping telephony out of the core tool list.
|
||||
|
||||
It ships with a helper script, `scripts/telephony.py`, that can:
|
||||
- save provider credentials into `~/.hermes/.env`
|
||||
- search for and buy a Twilio phone number
|
||||
- remember that owned number for later sessions
|
||||
- send SMS / MMS from the owned number
|
||||
- poll inbound SMS for that number with no webhook server required
|
||||
- make direct Twilio calls using TwiML `<Say>` or `<Play>`
|
||||
- import the owned Twilio number into Vapi
|
||||
- place outbound AI calls through Bland.ai or Vapi
|
||||
|
||||
## What this solves
|
||||
|
||||
This skill is meant to cover the practical phone tasks users actually want:
|
||||
- outbound calls
|
||||
- texting
|
||||
- owning a reusable agent number
|
||||
- checking messages that arrive to that number later
|
||||
- preserving that number and related IDs between sessions
|
||||
- future-friendly telephony identity for inbound SMS polling and other automations
|
||||
|
||||
It does **not** turn Hermes into a real-time inbound phone gateway. Inbound SMS is handled by polling the Twilio REST API. That is enough for many workflows, including notifications and some one-time-code retrieval, without adding core webhook infrastructure.
|
||||
|
||||
## Safety rules — mandatory
|
||||
|
||||
1. Always confirm before placing a call or sending a text.
|
||||
2. Never dial emergency numbers.
|
||||
3. Never use telephony for harassment, spam, impersonation, or anything illegal.
|
||||
4. Treat third-party phone numbers as sensitive operational data:
|
||||
- do not save them to Hermes memory
|
||||
- do not include them in skill docs, summaries, or follow-up notes unless the user explicitly wants that
|
||||
5. It is fine to persist the **agent-owned Twilio number** because that is part of the user's configuration.
|
||||
6. VoIP numbers are **not guaranteed** to work for all third-party 2FA flows. Use with caution and set user expectations clearly.
|
||||
|
||||
## Decision tree — which service to use?
|
||||
|
||||
Use this logic instead of hardcoded provider routing:
|
||||
|
||||
### 1) "I want Hermes to own a real phone number"
|
||||
Use **Twilio**.
|
||||
|
||||
Why:
|
||||
- easiest path to buying and keeping a number
|
||||
- best SMS / MMS support
|
||||
- simplest inbound SMS polling story
|
||||
- cleanest future path to inbound webhooks or call handling
|
||||
|
||||
Use cases:
|
||||
- receive texts later
|
||||
- send deployment alerts / cron notifications
|
||||
- maintain a reusable phone identity for the agent
|
||||
- experiment with phone-based auth flows later
|
||||
|
||||
### 2) "I only need the easiest outbound AI phone call right now"
|
||||
Use **Bland.ai**.
|
||||
|
||||
Why:
|
||||
- quickest setup
|
||||
- one API key
|
||||
- no need to first buy/import a number yourself
|
||||
|
||||
Tradeoff:
|
||||
- less flexible
|
||||
- voice quality is decent, but not the best
|
||||
|
||||
### 3) "I want the best conversational AI voice quality"
|
||||
Use **Twilio + Vapi**.
|
||||
|
||||
Why:
|
||||
- Twilio gives you the owned number
|
||||
- Vapi gives you better conversational AI call quality and more voice/model flexibility
|
||||
|
||||
Recommended flow:
|
||||
1. Buy/save a Twilio number
|
||||
2. Import it into Vapi
|
||||
3. Save the returned `VAPI_PHONE_NUMBER_ID`
|
||||
4. Use `ai-call --provider vapi`
|
||||
|
||||
### 4) "I want to call with a custom prerecorded voice message"
|
||||
Use **Twilio direct call** with a public audio URL.
|
||||
|
||||
Why:
|
||||
- easiest way to play a custom MP3
|
||||
- pairs well with Hermes `text_to_speech` plus a public file host or tunnel
|
||||
|
||||
## Files and persistent state
|
||||
|
||||
The skill persists telephony state in two places:
|
||||
|
||||
### `~/.hermes/.env`
|
||||
Used for long-lived provider credentials and owned-number IDs, for example:
|
||||
- `TWILIO_ACCOUNT_SID`
|
||||
- `TWILIO_AUTH_TOKEN`
|
||||
- `TWILIO_PHONE_NUMBER`
|
||||
- `TWILIO_PHONE_NUMBER_SID`
|
||||
- `BLAND_API_KEY`
|
||||
- `VAPI_API_KEY`
|
||||
- `VAPI_PHONE_NUMBER_ID`
|
||||
- `PHONE_PROVIDER` (AI call provider: bland or vapi)
|
||||
|
||||
### `~/.hermes/telephony_state.json`
|
||||
Used for skill-only state that should survive across sessions, for example:
|
||||
- remembered default Twilio number / SID
|
||||
- remembered Vapi phone number ID
|
||||
- last inbound message SID/date for inbox polling checkpoints
|
||||
|
||||
This means:
|
||||
- the next time the skill is loaded, `diagnose` can tell you what number is already configured
|
||||
- `twilio-inbox --since-last --mark-seen` can continue from the previous checkpoint
|
||||
|
||||
## Locate the helper script
|
||||
|
||||
After installing this skill, locate the script like this:
|
||||
|
||||
```bash
|
||||
SCRIPT="$(find ~/.hermes/skills -path '*/telephony/scripts/telephony.py' -print -quit)"
|
||||
```
|
||||
|
||||
If `SCRIPT` is empty, the skill is not installed yet.
|
||||
|
||||
## Install
|
||||
|
||||
This is an official optional skill, so install it from the Skills Hub:
|
||||
|
||||
```bash
|
||||
hermes skills search telephony
|
||||
hermes skills install official/productivity/telephony
|
||||
```
|
||||
|
||||
## Provider setup
|
||||
|
||||
### Twilio — owned number, SMS/MMS, direct calls, inbound SMS polling
|
||||
|
||||
Sign up at:
|
||||
- https://www.twilio.com/try-twilio
|
||||
|
||||
Then save credentials into Hermes:
|
||||
|
||||
```bash
|
||||
python3 "$SCRIPT" save-twilio ACXXXXXXXXXXXXXXXXXXXXXXXXXXXX your_auth_token_here
|
||||
```
|
||||
|
||||
Search for available numbers:
|
||||
|
||||
```bash
|
||||
python3 "$SCRIPT" twilio-search --country US --area-code 702 --limit 5
|
||||
```
|
||||
|
||||
Buy and remember a number:
|
||||
|
||||
```bash
|
||||
python3 "$SCRIPT" twilio-buy "+17025551234" --save-env
|
||||
```
|
||||
|
||||
List owned numbers:
|
||||
|
||||
```bash
|
||||
python3 "$SCRIPT" twilio-owned
|
||||
```
|
||||
|
||||
Set one of them as the default later:
|
||||
|
||||
```bash
|
||||
python3 "$SCRIPT" twilio-set-default "+17025551234" --save-env
|
||||
# or
|
||||
python3 "$SCRIPT" twilio-set-default PNXXXXXXXXXXXXXXXXXXXXXXXXXXXX --save-env
|
||||
```
|
||||
|
||||
### Bland.ai — easiest outbound AI calling
|
||||
|
||||
Sign up at:
|
||||
- https://app.bland.ai
|
||||
|
||||
Save config:
|
||||
|
||||
```bash
|
||||
python3 "$SCRIPT" save-bland your_bland_api_key --voice mason
|
||||
```
|
||||
|
||||
### Vapi — better conversational voice quality
|
||||
|
||||
Sign up at:
|
||||
- https://dashboard.vapi.ai
|
||||
|
||||
Save the API key first:
|
||||
|
||||
```bash
|
||||
python3 "$SCRIPT" save-vapi your_vapi_api_key
|
||||
```
|
||||
|
||||
Import your owned Twilio number into Vapi and persist the returned phone number ID:
|
||||
|
||||
```bash
|
||||
python3 "$SCRIPT" vapi-import-twilio --save-env
|
||||
```
|
||||
|
||||
If you already know the Vapi phone number ID, save it directly:
|
||||
|
||||
```bash
|
||||
python3 "$SCRIPT" save-vapi your_vapi_api_key --phone-number-id vapi_phone_number_id_here
|
||||
```
|
||||
|
||||
## Diagnose current state
|
||||
|
||||
At any time, inspect what the skill already knows:
|
||||
|
||||
```bash
|
||||
python3 "$SCRIPT" diagnose
|
||||
```
|
||||
|
||||
Use this first when resuming work in a later session.
|
||||
|
||||
## Common workflows
|
||||
|
||||
### A. Buy an agent number and keep using it later
|
||||
|
||||
1. Save Twilio credentials:
|
||||
```bash
|
||||
python3 "$SCRIPT" save-twilio AC... auth_token_here
|
||||
```
|
||||
|
||||
2. Search for a number:
|
||||
```bash
|
||||
python3 "$SCRIPT" twilio-search --country US --area-code 702 --limit 10
|
||||
```
|
||||
|
||||
3. Buy it and save it into `~/.hermes/.env` + state:
|
||||
```bash
|
||||
python3 "$SCRIPT" twilio-buy "+17025551234" --save-env
|
||||
```
|
||||
|
||||
4. Next session, run:
|
||||
```bash
|
||||
python3 "$SCRIPT" diagnose
|
||||
```
|
||||
This shows the remembered default number and inbox checkpoint state.
|
||||
|
||||
### B. Send a text from the agent number
|
||||
|
||||
```bash
|
||||
python3 "$SCRIPT" twilio-send-sms "+15551230000" "Your deployment completed successfully."
|
||||
```
|
||||
|
||||
With media:
|
||||
|
||||
```bash
|
||||
python3 "$SCRIPT" twilio-send-sms "+15551230000" "Here is the chart." --media-url "https://example.com/chart.png"
|
||||
```
|
||||
|
||||
### C. Check inbound texts later with no webhook server
|
||||
|
||||
Poll the inbox for the default Twilio number:
|
||||
|
||||
```bash
|
||||
python3 "$SCRIPT" twilio-inbox --limit 20
|
||||
```
|
||||
|
||||
Only show messages that arrived after the last checkpoint, and advance the checkpoint when you're done reading:
|
||||
|
||||
```bash
|
||||
python3 "$SCRIPT" twilio-inbox --since-last --mark-seen
|
||||
```
|
||||
|
||||
This is the main answer to “how do I access messages the number receives next time the skill is loaded?”
|
||||
|
||||
### D. Make a direct Twilio call with built-in TTS
|
||||
|
||||
```bash
|
||||
python3 "$SCRIPT" twilio-call "+15551230000" --message "Hello! This is Hermes calling with your status update." --voice Polly.Joanna
|
||||
```
|
||||
|
||||
### E. Call with a prerecorded / custom voice message
|
||||
|
||||
This is the main path for reusing Hermes's existing `text_to_speech` support.
|
||||
|
||||
Use this when:
|
||||
- you want the call to use Hermes's configured TTS voice rather than Twilio `<Say>`
|
||||
- you want a one-way voice delivery (briefing, alert, joke, reminder, status update)
|
||||
- you do **not** need a live conversational phone call
|
||||
|
||||
Generate or host audio separately, then:
|
||||
|
||||
```bash
|
||||
python3 "$SCRIPT" twilio-call "+155****0000" --audio-url "https://example.com/briefing.mp3"
|
||||
```
|
||||
|
||||
Recommended Hermes TTS -> Twilio Play workflow:
|
||||
|
||||
1. Generate the audio with Hermes `text_to_speech`.
|
||||
2. Make the resulting MP3 publicly reachable.
|
||||
3. Place the Twilio call with `--audio-url`.
|
||||
|
||||
Example agent flow:
|
||||
- Ask Hermes to create the message audio with `text_to_speech`
|
||||
- If needed, expose the file with a temporary static host / tunnel / object storage URL
|
||||
- Use `twilio-call --audio-url ...` to deliver it by phone
|
||||
|
||||
Good hosting options for the MP3:
|
||||
- a temporary public object/storage URL
|
||||
- a short-lived tunnel to a local static file server
|
||||
- any existing HTTPS URL the phone provider can fetch directly
|
||||
|
||||
Important note:
|
||||
- Hermes TTS is great for prerecorded outbound messages
|
||||
- Bland/Vapi are better for **live conversational AI calls** because they handle the real-time telephony audio stack themselves
|
||||
- Hermes STT/TTS alone is not being used here as a full duplex phone conversation engine; that would require a much heavier streaming/webhook integration than this skill is trying to introduce
|
||||
|
||||
### F. Navigate a phone tree / IVR with Twilio direct calling
|
||||
|
||||
If you need to press digits after the call connects, use `--send-digits`.
|
||||
Twilio interprets `w` as a short wait.
|
||||
|
||||
```bash
|
||||
python3 "$SCRIPT" twilio-call "+18005551234" --message "Connecting to billing now." --send-digits "ww1w2w3"
|
||||
```
|
||||
|
||||
This is useful for reaching a specific menu branch before handing off to a human or delivering a short status message.
|
||||
|
||||
### G. Outbound AI phone call with Bland.ai
|
||||
|
||||
```bash
|
||||
python3 "$SCRIPT" ai-call "+15551230000" "Call the dental office, ask for a cleaning appointment on Tuesday afternoon, and if they do not have Tuesday availability, ask for Wednesday or Thursday instead." --provider bland --voice mason --max-duration 3
|
||||
```
|
||||
|
||||
Check status:
|
||||
|
||||
```bash
|
||||
python3 "$SCRIPT" ai-status <call_id> --provider bland
|
||||
```
|
||||
|
||||
Ask Bland analysis questions after completion:
|
||||
|
||||
```bash
|
||||
python3 "$SCRIPT" ai-status <call_id> --provider bland --analyze "Was the appointment confirmed?,What date and time?,Any special instructions?"
|
||||
```
|
||||
|
||||
### H. Outbound AI phone call with Vapi on your owned number
|
||||
|
||||
1. Import your Twilio number into Vapi:
|
||||
```bash
|
||||
python3 "$SCRIPT" vapi-import-twilio --save-env
|
||||
```
|
||||
|
||||
2. Place the call:
|
||||
```bash
|
||||
python3 "$SCRIPT" ai-call "+15551230000" "You are calling to make a dinner reservation for two at 7:30 PM. If that is unavailable, ask for the nearest time between 6:30 and 8:30 PM." --provider vapi --max-duration 4
|
||||
```
|
||||
|
||||
3. Check result:
|
||||
```bash
|
||||
python3 "$SCRIPT" ai-status <call_id> --provider vapi
|
||||
```
|
||||
|
||||
## Suggested agent procedure
|
||||
|
||||
When the user asks for a call or text:
|
||||
|
||||
1. Determine which path fits the request via the decision tree.
|
||||
2. Run `diagnose` if configuration state is unclear.
|
||||
3. Gather the full task details.
|
||||
4. Confirm with the user before dialing or texting.
|
||||
5. Use the correct command.
|
||||
6. Poll for results if needed.
|
||||
7. Summarize the outcome without persisting third-party numbers to Hermes memory.
|
||||
|
||||
## What this skill still does not do
|
||||
|
||||
- real-time inbound call answering
|
||||
- webhook-based live SMS push into the agent loop
|
||||
- guaranteed support for arbitrary third-party 2FA providers
|
||||
|
||||
Those would require more infrastructure than a pure optional skill.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- Twilio trial accounts and regional rules can restrict who you can call/text.
|
||||
- Some services reject VoIP numbers for 2FA.
|
||||
- `twilio-inbox` polls the REST API; it is not instant push delivery.
|
||||
- Vapi outbound calling still depends on having a valid imported number.
|
||||
- Bland is easiest, but not always the best-sounding.
|
||||
- Do not store arbitrary third-party phone numbers in Hermes memory.
|
||||
|
||||
## Verification checklist
|
||||
|
||||
After setup, you should be able to do all of the following with just this skill:
|
||||
|
||||
1. `diagnose` shows provider readiness and remembered state
|
||||
2. search and buy a Twilio number
|
||||
3. persist that number to `~/.hermes/.env`
|
||||
4. send an SMS from the owned number
|
||||
5. poll inbound texts for the owned number later
|
||||
6. place a direct Twilio call
|
||||
7. place an AI call via Bland or Vapi
|
||||
|
||||
## References
|
||||
|
||||
- Twilio phone numbers: https://www.twilio.com/docs/phone-numbers/api
|
||||
- Twilio messaging: https://www.twilio.com/docs/messaging/api/message-resource
|
||||
- Twilio voice: https://www.twilio.com/docs/voice/api/call-resource
|
||||
- Vapi docs: https://docs.vapi.ai/
|
||||
- Bland.ai: https://app.bland.ai/
|
||||
File diff suppressed because it is too large
Load diff
235
hermes_code/optional-skills/research/bioinformatics/SKILL.md
Normal file
235
hermes_code/optional-skills/research/bioinformatics/SKILL.md
Normal file
|
|
@ -0,0 +1,235 @@
|
|||
---
|
||||
name: bioinformatics
|
||||
description: Gateway to 400+ bioinformatics skills from bioSkills and ClawBio. Covers genomics, transcriptomics, single-cell, variant calling, pharmacogenomics, metagenomics, structural biology, and more. Fetches domain-specific reference material on demand.
|
||||
version: 1.0.0
|
||||
platforms: [linux, macos]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [bioinformatics, genomics, sequencing, biology, research, science]
|
||||
category: research
|
||||
---
|
||||
|
||||
# Bioinformatics Skills Gateway
|
||||
|
||||
Use when asked about bioinformatics, genomics, sequencing, variant calling, gene expression, single-cell analysis, protein structure, pharmacogenomics, metagenomics, phylogenetics, or any computational biology task.
|
||||
|
||||
This skill is a gateway to two open-source bioinformatics skill libraries. Instead of bundling hundreds of domain-specific skills, it indexes them and fetches what you need on demand.
|
||||
|
||||
## Sources
|
||||
|
||||
◆ **bioSkills** — 385 reference skills (code patterns, parameter guides, decision trees)
|
||||
Repo: https://github.com/GPTomics/bioSkills
|
||||
Format: SKILL.md per topic with code examples. Python/R/CLI.
|
||||
|
||||
◆ **ClawBio** — 33 runnable pipeline skills (executable scripts, reproducibility bundles)
|
||||
Repo: https://github.com/ClawBio/ClawBio
|
||||
Format: Python scripts with demos. Each analysis exports report.md + commands.sh + environment.yml.
|
||||
|
||||
## How to fetch and use a skill
|
||||
|
||||
1. Identify the domain and skill name from the index below.
|
||||
2. Clone the relevant repo (shallow clone to save time):
|
||||
```bash
|
||||
# bioSkills (reference material)
|
||||
git clone --depth 1 https://github.com/GPTomics/bioSkills.git /tmp/bioSkills
|
||||
|
||||
# ClawBio (runnable pipelines)
|
||||
git clone --depth 1 https://github.com/ClawBio/ClawBio.git /tmp/ClawBio
|
||||
```
|
||||
3. Read the specific skill:
|
||||
```bash
|
||||
# bioSkills — each skill is at: <category>/<skill-name>/SKILL.md
|
||||
cat /tmp/bioSkills/variant-calling/gatk-variant-calling/SKILL.md
|
||||
|
||||
# ClawBio — each skill is at: skills/<skill-name>/
|
||||
cat /tmp/ClawBio/skills/pharmgx-reporter/README.md
|
||||
```
|
||||
4. Follow the fetched skill as reference material. These are NOT Hermes-format skills — treat them as expert domain guides. They contain correct parameters, proper tool flags, and validated pipelines.
|
||||
|
||||
## Skill Index by Domain
|
||||
|
||||
### Sequence Fundamentals
|
||||
bioSkills:
|
||||
sequence-io/ — read-sequences, write-sequences, format-conversion, batch-processing, compressed-files, fastq-quality, filter-sequences, paired-end-fastq, sequence-statistics
|
||||
sequence-manipulation/ — seq-objects, reverse-complement, transcription-translation, motif-search, codon-usage, sequence-properties, sequence-slicing
|
||||
ClawBio:
|
||||
seq-wrangler — Sequence QC, alignment, and BAM processing (wraps FastQC, BWA, SAMtools)
|
||||
|
||||
### Read QC & Alignment
|
||||
bioSkills:
|
||||
read-qc/ — quality-reports, fastp-workflow, adapter-trimming, quality-filtering, umi-processing, contamination-screening, rnaseq-qc
|
||||
read-alignment/ — bwa-alignment, star-alignment, hisat2-alignment, bowtie2-alignment
|
||||
alignment-files/ — sam-bam-basics, alignment-sorting, alignment-filtering, bam-statistics, duplicate-handling, pileup-generation
|
||||
|
||||
### Variant Calling & Annotation
|
||||
bioSkills:
|
||||
variant-calling/ — gatk-variant-calling, deepvariant, variant-calling (bcftools), joint-calling, structural-variant-calling, filtering-best-practices, variant-annotation, variant-normalization, vcf-basics, vcf-manipulation, vcf-statistics, consensus-sequences, clinical-interpretation
|
||||
ClawBio:
|
||||
vcf-annotator — VEP + ClinVar + gnomAD annotation with ancestry-aware context
|
||||
variant-annotation — Variant annotation pipeline
|
||||
|
||||
### Differential Expression (Bulk RNA-seq)
|
||||
bioSkills:
|
||||
differential-expression/ — deseq2-basics, edger-basics, batch-correction, de-results, de-visualization, timeseries-de
|
||||
rna-quantification/ — alignment-free-quant (Salmon/kallisto), featurecounts-counting, tximport-workflow, count-matrix-qc
|
||||
expression-matrix/ — counts-ingest, gene-id-mapping, metadata-joins, sparse-handling
|
||||
ClawBio:
|
||||
rnaseq-de — Full DE pipeline with QC, normalization, and visualization
|
||||
diff-visualizer — Rich visualization and reporting for DE results
|
||||
|
||||
### Single-Cell RNA-seq
|
||||
bioSkills:
|
||||
single-cell/ — preprocessing, clustering, batch-integration, cell-annotation, cell-communication, doublet-detection, markers-annotation, trajectory-inference, multimodal-integration, perturb-seq, scatac-analysis, lineage-tracing, metabolite-communication, data-io
|
||||
ClawBio:
|
||||
scrna-orchestrator — Full Scanpy pipeline (QC, clustering, markers, annotation)
|
||||
scrna-embedding — scVI-based latent embedding and batch integration
|
||||
|
||||
### Spatial Transcriptomics
|
||||
bioSkills:
|
||||
spatial-transcriptomics/ — spatial-data-io, spatial-preprocessing, spatial-domains, spatial-deconvolution, spatial-communication, spatial-neighbors, spatial-statistics, spatial-visualization, spatial-multiomics, spatial-proteomics, image-analysis
|
||||
|
||||
### Epigenomics
|
||||
bioSkills:
|
||||
chip-seq/ — peak-calling, differential-binding, motif-analysis, peak-annotation, chipseq-qc, chipseq-visualization, super-enhancers
|
||||
atac-seq/ — atac-peak-calling, atac-qc, differential-accessibility, footprinting, motif-deviation, nucleosome-positioning
|
||||
methylation-analysis/ — bismark-alignment, methylation-calling, dmr-detection, methylkit-analysis
|
||||
hi-c-analysis/ — hic-data-io, tad-detection, loop-calling, compartment-analysis, contact-pairs, matrix-operations, hic-visualization, hic-differential
|
||||
ClawBio:
|
||||
methylation-clock — Epigenetic age estimation
|
||||
|
||||
### Pharmacogenomics & Clinical
|
||||
bioSkills:
|
||||
clinical-databases/ — clinvar-lookup, gnomad-frequencies, dbsnp-queries, pharmacogenomics, polygenic-risk, hla-typing, variant-prioritization, somatic-signatures, tumor-mutational-burden, myvariant-queries
|
||||
ClawBio:
|
||||
pharmgx-reporter — PGx report from 23andMe/AncestryDNA (12 genes, 31 SNPs, 51 drugs)
|
||||
drug-photo — Photo of medication → personalized PGx dosage card (via vision)
|
||||
clinpgx — ClinPGx API for gene-drug data and CPIC guidelines
|
||||
gwas-lookup — Federated variant lookup across 9 genomic databases
|
||||
gwas-prs — Polygenic risk scores from consumer genetic data
|
||||
nutrigx_advisor — Personalized nutrition from consumer genetic data
|
||||
|
||||
### Population Genetics & GWAS
|
||||
bioSkills:
|
||||
population-genetics/ — association-testing (PLINK GWAS), plink-basics, population-structure, linkage-disequilibrium, scikit-allel-analysis, selection-statistics
|
||||
causal-genomics/ — mendelian-randomization, fine-mapping, colocalization-analysis, mediation-analysis, pleiotropy-detection
|
||||
phasing-imputation/ — haplotype-phasing, genotype-imputation, imputation-qc, reference-panels
|
||||
ClawBio:
|
||||
claw-ancestry-pca — Ancestry PCA against SGDP reference panel
|
||||
|
||||
### Metagenomics & Microbiome
|
||||
bioSkills:
|
||||
metagenomics/ — kraken-classification, metaphlan-profiling, abundance-estimation, functional-profiling, amr-detection, strain-tracking, metagenome-visualization
|
||||
microbiome/ — amplicon-processing, diversity-analysis, differential-abundance, taxonomy-assignment, functional-prediction, qiime2-workflow
|
||||
ClawBio:
|
||||
claw-metagenomics — Shotgun metagenomics profiling (taxonomy, resistome, functional pathways)
|
||||
|
||||
### Genome Assembly & Annotation
|
||||
bioSkills:
|
||||
genome-assembly/ — hifi-assembly, long-read-assembly, short-read-assembly, metagenome-assembly, assembly-polishing, assembly-qc, scaffolding, contamination-detection
|
||||
genome-annotation/ — eukaryotic-gene-prediction, prokaryotic-annotation, functional-annotation, ncrna-annotation, repeat-annotation, annotation-transfer
|
||||
long-read-sequencing/ — basecalling, long-read-alignment, long-read-qc, clair3-variants, structural-variants, medaka-polishing, nanopore-methylation, isoseq-analysis
|
||||
|
||||
### Structural Biology & Chemoinformatics
|
||||
bioSkills:
|
||||
structural-biology/ — alphafold-predictions, modern-structure-prediction, structure-io, structure-navigation, structure-modification, geometric-analysis
|
||||
chemoinformatics/ — molecular-io, molecular-descriptors, similarity-searching, substructure-search, virtual-screening, admet-prediction, reaction-enumeration
|
||||
ClawBio:
|
||||
struct-predictor — Local AlphaFold/Boltz/Chai structure prediction with comparison
|
||||
|
||||
### Proteomics
|
||||
bioSkills:
|
||||
proteomics/ — data-import, peptide-identification, protein-inference, quantification, differential-abundance, dia-analysis, ptm-analysis, proteomics-qc, spectral-libraries
|
||||
ClawBio:
|
||||
proteomics-de — Proteomics differential expression
|
||||
|
||||
### Pathway Analysis & Gene Networks
|
||||
bioSkills:
|
||||
pathway-analysis/ — go-enrichment, gsea, kegg-pathways, reactome-pathways, wikipathways, enrichment-visualization
|
||||
gene-regulatory-networks/ — scenic-regulons, coexpression-networks, differential-networks, multiomics-grn, perturbation-simulation
|
||||
|
||||
### Immunoinformatics
|
||||
bioSkills:
|
||||
immunoinformatics/ — mhc-binding-prediction, epitope-prediction, neoantigen-prediction, immunogenicity-scoring, tcr-epitope-binding
|
||||
tcr-bcr-analysis/ — mixcr-analysis, scirpy-analysis, immcantation-analysis, repertoire-visualization, vdjtools-analysis
|
||||
|
||||
### CRISPR & Genome Engineering
|
||||
bioSkills:
|
||||
crispr-screens/ — mageck-analysis, jacks-analysis, hit-calling, screen-qc, library-design, crispresso-editing, base-editing-analysis, batch-correction
|
||||
genome-engineering/ — grna-design, off-target-prediction, hdr-template-design, base-editing-design, prime-editing-design
|
||||
|
||||
### Workflow Management
|
||||
bioSkills:
|
||||
workflow-management/ — snakemake-workflows, nextflow-pipelines, cwl-workflows, wdl-workflows
|
||||
ClawBio:
|
||||
repro-enforcer — Export any analysis as reproducibility bundle (Conda env + Singularity + checksums)
|
||||
galaxy-bridge — Access 8,000+ Galaxy tools from usegalaxy.org
|
||||
|
||||
### Specialized Domains
|
||||
bioSkills:
|
||||
alternative-splicing/ — splicing-quantification, differential-splicing, isoform-switching, sashimi-plots, single-cell-splicing, splicing-qc
|
||||
ecological-genomics/ — edna-metabarcoding, landscape-genomics, conservation-genetics, biodiversity-metrics, community-ecology, species-delimitation
|
||||
epidemiological-genomics/ — pathogen-typing, variant-surveillance, phylodynamics, transmission-inference, amr-surveillance
|
||||
liquid-biopsy/ — cfdna-preprocessing, ctdna-mutation-detection, fragment-analysis, tumor-fraction-estimation, methylation-based-detection, longitudinal-monitoring
|
||||
epitranscriptomics/ — m6a-peak-calling, m6a-differential, m6anet-analysis, merip-preprocessing, modification-visualization
|
||||
metabolomics/ — xcms-preprocessing, metabolite-annotation, normalization-qc, statistical-analysis, pathway-mapping, lipidomics, targeted-analysis, msdial-preprocessing
|
||||
flow-cytometry/ — fcs-handling, gating-analysis, compensation-transformation, clustering-phenotyping, differential-analysis, cytometry-qc, doublet-detection, bead-normalization
|
||||
systems-biology/ — flux-balance-analysis, metabolic-reconstruction, gene-essentiality, context-specific-models, model-curation
|
||||
rna-structure/ — secondary-structure-prediction, ncrna-search, structure-probing
|
||||
|
||||
### Data Visualization & Reporting
|
||||
bioSkills:
|
||||
data-visualization/ — ggplot2-fundamentals, heatmaps-clustering, volcano-customization, circos-plots, genome-browser-tracks, interactive-visualization, multipanel-figures, network-visualization, upset-plots, color-palettes, specialized-omics-plots, genome-tracks
|
||||
reporting/ — rmarkdown-reports, quarto-reports, jupyter-reports, automated-qc-reports, figure-export
|
||||
ClawBio:
|
||||
profile-report — Analysis profile reporting
|
||||
data-extractor — Extract numerical data from scientific figure images (via vision)
|
||||
lit-synthesizer — PubMed/bioRxiv search, summarization, citation graphs
|
||||
pubmed-summariser — Gene/disease PubMed search with structured briefing
|
||||
|
||||
### Database Access
|
||||
bioSkills:
|
||||
database-access/ — entrez-search, entrez-fetch, entrez-link, blast-searches, local-blast, sra-data, geo-data, uniprot-access, batch-downloads, interaction-databases, sequence-similarity
|
||||
ClawBio:
|
||||
ukb-navigator — Semantic search across 12,000+ UK Biobank fields
|
||||
clinical-trial-finder — Clinical trial discovery
|
||||
|
||||
### Experimental Design
|
||||
bioSkills:
|
||||
experimental-design/ — power-analysis, sample-size, batch-design, multiple-testing
|
||||
|
||||
### Machine Learning for Omics
|
||||
bioSkills:
|
||||
machine-learning/ — omics-classifiers, biomarker-discovery, survival-analysis, model-validation, prediction-explanation, atlas-mapping
|
||||
ClawBio:
|
||||
claw-semantic-sim — Semantic similarity index for disease literature (PubMedBERT)
|
||||
omics-target-evidence-mapper — Aggregate target-level evidence across omics sources
|
||||
|
||||
## Environment Setup
|
||||
|
||||
These skills assume a bioinformatics workstation. Common dependencies:
|
||||
|
||||
```bash
|
||||
# Python
|
||||
pip install biopython pysam cyvcf2 pybedtools pyBigWig scikit-allel anndata scanpy mygene
|
||||
|
||||
# R/Bioconductor
|
||||
Rscript -e 'BiocManager::install(c("DESeq2","edgeR","Seurat","clusterProfiler","methylKit"))'
|
||||
|
||||
# CLI tools (Ubuntu/Debian)
|
||||
sudo apt install samtools bcftools ncbi-blast+ minimap2 bedtools
|
||||
|
||||
# CLI tools (macOS)
|
||||
brew install samtools bcftools blast minimap2 bedtools
|
||||
|
||||
# Or via Conda (recommended for reproducibility)
|
||||
conda install -c bioconda samtools bcftools blast minimap2 bedtools fastp kraken2
|
||||
```
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- The fetched skills are NOT in Hermes SKILL.md format. They use their own structure (bioSkills: code pattern cookbooks; ClawBio: README + Python scripts). Read them as expert reference material.
|
||||
- bioSkills are reference guides — they show correct parameters and code patterns but aren't executable pipelines.
|
||||
- ClawBio skills are executable — many have `--demo` flags and can be run directly.
|
||||
- Both repos assume bioinformatics tools are installed. Check prerequisites before running pipelines.
|
||||
- For ClawBio, run `pip install -r requirements.txt` in the cloned repo first.
|
||||
- Genomic data files can be very large. Be mindful of disk space when downloading reference genomes, SRA datasets, or building indices.
|
||||
441
hermes_code/optional-skills/research/qmd/SKILL.md
Normal file
441
hermes_code/optional-skills/research/qmd/SKILL.md
Normal file
|
|
@ -0,0 +1,441 @@
|
|||
---
|
||||
name: qmd
|
||||
description: Search personal knowledge bases, notes, docs, and meeting transcripts locally using qmd — a hybrid retrieval engine with BM25, vector search, and LLM reranking. Supports CLI and MCP integration.
|
||||
version: 1.0.0
|
||||
author: Hermes Agent + Teknium
|
||||
license: MIT
|
||||
platforms: [macos, linux]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Search, Knowledge-Base, RAG, Notes, MCP, Local-AI]
|
||||
related_skills: [obsidian, native-mcp, arxiv]
|
||||
---
|
||||
|
||||
# QMD — Query Markup Documents
|
||||
|
||||
Local, on-device search engine for personal knowledge bases. Indexes markdown
|
||||
notes, meeting transcripts, documentation, and any text-based files, then
|
||||
provides hybrid search combining keyword matching, semantic understanding, and
|
||||
LLM-powered reranking — all running locally with no cloud dependencies.
|
||||
|
||||
Created by [Tobi Lütke](https://github.com/tobi/qmd). MIT licensed.
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks to search their notes, docs, knowledge base, or meeting transcripts
|
||||
- User wants to find something across a large collection of markdown/text files
|
||||
- User wants semantic search ("find notes about X concept") not just keyword grep
|
||||
- User has already set up qmd collections and wants to query them
|
||||
- User asks to set up a local knowledge base or document search system
|
||||
- Keywords: "search my notes", "find in my docs", "knowledge base", "qmd"
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Node.js >= 22 (required)
|
||||
|
||||
```bash
|
||||
# Check version
|
||||
node --version # must be >= 22
|
||||
|
||||
# macOS — install or upgrade via Homebrew
|
||||
brew install node@22
|
||||
|
||||
# Linux — use NodeSource or nvm
|
||||
curl -fsSL https://deb.nodesource.com/setup_22.x | sudo -E bash -
|
||||
sudo apt-get install -y nodejs
|
||||
# or with nvm:
|
||||
nvm install 22 && nvm use 22
|
||||
```
|
||||
|
||||
### SQLite with Extension Support (macOS only)
|
||||
|
||||
macOS system SQLite lacks extension loading. Install via Homebrew:
|
||||
|
||||
```bash
|
||||
brew install sqlite
|
||||
```
|
||||
|
||||
### Install qmd
|
||||
|
||||
```bash
|
||||
npm install -g @tobilu/qmd
|
||||
# or with Bun:
|
||||
bun install -g @tobilu/qmd
|
||||
```
|
||||
|
||||
First run auto-downloads 3 local GGUF models (~2GB total):
|
||||
|
||||
| Model | Purpose | Size |
|
||||
|-------|---------|------|
|
||||
| embeddinggemma-300M-Q8_0 | Vector embeddings | ~300MB |
|
||||
| qwen3-reranker-0.6b-q8_0 | Result reranking | ~640MB |
|
||||
| qmd-query-expansion-1.7B | Query expansion | ~1.1GB |
|
||||
|
||||
### Verify Installation
|
||||
|
||||
```bash
|
||||
qmd --version
|
||||
qmd status
|
||||
```
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Command | What It Does | Speed |
|
||||
|---------|-------------|-------|
|
||||
| `qmd search "query"` | BM25 keyword search (no models) | ~0.2s |
|
||||
| `qmd vsearch "query"` | Semantic vector search (1 model) | ~3s |
|
||||
| `qmd query "query"` | Hybrid + reranking (all 3 models) | ~2-3s warm, ~19s cold |
|
||||
| `qmd get <docid>` | Retrieve full document content | instant |
|
||||
| `qmd multi-get "glob"` | Retrieve multiple files | instant |
|
||||
| `qmd collection add <path> --name <n>` | Add a directory as a collection | instant |
|
||||
| `qmd context add <path> "description"` | Add context metadata to improve retrieval | instant |
|
||||
| `qmd embed` | Generate/update vector embeddings | varies |
|
||||
| `qmd status` | Show index health and collection info | instant |
|
||||
| `qmd mcp` | Start MCP server (stdio) | persistent |
|
||||
| `qmd mcp --http --daemon` | Start MCP server (HTTP, warm models) | persistent |
|
||||
|
||||
## Setup Workflow
|
||||
|
||||
### 1. Add Collections
|
||||
|
||||
Point qmd at directories containing your documents:
|
||||
|
||||
```bash
|
||||
# Add a notes directory
|
||||
qmd collection add ~/notes --name notes
|
||||
|
||||
# Add project docs
|
||||
qmd collection add ~/projects/myproject/docs --name project-docs
|
||||
|
||||
# Add meeting transcripts
|
||||
qmd collection add ~/meetings --name meetings
|
||||
|
||||
# List all collections
|
||||
qmd collection list
|
||||
```
|
||||
|
||||
### 2. Add Context Descriptions
|
||||
|
||||
Context metadata helps the search engine understand what each collection
|
||||
contains. This significantly improves retrieval quality:
|
||||
|
||||
```bash
|
||||
qmd context add qmd://notes "Personal notes, ideas, and journal entries"
|
||||
qmd context add qmd://project-docs "Technical documentation for the main project"
|
||||
qmd context add qmd://meetings "Meeting transcripts and action items from team syncs"
|
||||
```
|
||||
|
||||
### 3. Generate Embeddings
|
||||
|
||||
```bash
|
||||
qmd embed
|
||||
```
|
||||
|
||||
This processes all documents in all collections and generates vector
|
||||
embeddings. Re-run after adding new documents or collections.
|
||||
|
||||
### 4. Verify
|
||||
|
||||
```bash
|
||||
qmd status # shows index health, collection stats, model info
|
||||
```
|
||||
|
||||
## Search Patterns
|
||||
|
||||
### Fast Keyword Search (BM25)
|
||||
|
||||
Best for: exact terms, code identifiers, names, known phrases.
|
||||
No models loaded — near-instant results.
|
||||
|
||||
```bash
|
||||
qmd search "authentication middleware"
|
||||
qmd search "handleError async"
|
||||
```
|
||||
|
||||
### Semantic Vector Search
|
||||
|
||||
Best for: natural language questions, conceptual queries.
|
||||
Loads embedding model (~3s first query).
|
||||
|
||||
```bash
|
||||
qmd vsearch "how does the rate limiter handle burst traffic"
|
||||
qmd vsearch "ideas for improving onboarding flow"
|
||||
```
|
||||
|
||||
### Hybrid Search with Reranking (Best Quality)
|
||||
|
||||
Best for: important queries where quality matters most.
|
||||
Uses all 3 models — query expansion, parallel BM25+vector, reranking.
|
||||
|
||||
```bash
|
||||
qmd query "what decisions were made about the database migration"
|
||||
```
|
||||
|
||||
### Structured Multi-Mode Queries
|
||||
|
||||
Combine different search types in a single query for precision:
|
||||
|
||||
```bash
|
||||
# BM25 for exact term + vector for concept
|
||||
qmd query $'lex: rate limiter\nvec: how does throttling work under load'
|
||||
|
||||
# With query expansion
|
||||
qmd query $'expand: database migration plan\nlex: "schema change"'
|
||||
```
|
||||
|
||||
### Query Syntax (lex/BM25 mode)
|
||||
|
||||
| Syntax | Effect | Example |
|
||||
|--------|--------|---------|
|
||||
| `term` | Prefix match | `perf` matches "performance" |
|
||||
| `"phrase"` | Exact phrase | `"rate limiter"` |
|
||||
| `-term` | Exclude term | `performance -sports` |
|
||||
|
||||
### HyDE (Hypothetical Document Embeddings)
|
||||
|
||||
For complex topics, write what you expect the answer to look like:
|
||||
|
||||
```bash
|
||||
qmd query $'hyde: The migration plan involves three phases. First, we add the new columns without dropping the old ones. Then we backfill data. Finally we cut over and remove legacy columns.'
|
||||
```
|
||||
|
||||
### Scoping to Collections
|
||||
|
||||
```bash
|
||||
qmd search "query" --collection notes
|
||||
qmd query "query" --collection project-docs
|
||||
```
|
||||
|
||||
### Output Formats
|
||||
|
||||
```bash
|
||||
qmd search "query" --json # JSON output (best for parsing)
|
||||
qmd search "query" --limit 5 # Limit results
|
||||
qmd get "#abc123" # Get by document ID
|
||||
qmd get "path/to/file.md" # Get by file path
|
||||
qmd get "file.md:50" -l 100 # Get specific line range
|
||||
qmd multi-get "journals/*.md" --json # Batch retrieve by glob
|
||||
```
|
||||
|
||||
## MCP Integration (Recommended)
|
||||
|
||||
qmd exposes an MCP server that provides search tools directly to
|
||||
Hermes Agent via the native MCP client. This is the preferred
|
||||
integration — once configured, the agent gets qmd tools automatically
|
||||
without needing to load this skill.
|
||||
|
||||
### Option A: Stdio Mode (Simple)
|
||||
|
||||
Add to `~/.hermes/config.yaml`:
|
||||
|
||||
```yaml
|
||||
mcp_servers:
|
||||
qmd:
|
||||
command: "qmd"
|
||||
args: ["mcp"]
|
||||
timeout: 30
|
||||
connect_timeout: 45
|
||||
```
|
||||
|
||||
This registers tools: `mcp_qmd_search`, `mcp_qmd_vsearch`,
|
||||
`mcp_qmd_deep_search`, `mcp_qmd_get`, `mcp_qmd_status`.
|
||||
|
||||
**Tradeoff:** Models load on first search call (~19s cold start),
|
||||
then stay warm for the session. Acceptable for occasional use.
|
||||
|
||||
### Option B: HTTP Daemon Mode (Fast, Recommended for Heavy Use)
|
||||
|
||||
Start the qmd daemon separately — it keeps models warm in memory:
|
||||
|
||||
```bash
|
||||
# Start daemon (persists across agent restarts)
|
||||
qmd mcp --http --daemon
|
||||
|
||||
# Runs on http://localhost:8181 by default
|
||||
```
|
||||
|
||||
Then configure Hermes Agent to connect via HTTP:
|
||||
|
||||
```yaml
|
||||
mcp_servers:
|
||||
qmd:
|
||||
url: "http://localhost:8181/mcp"
|
||||
timeout: 30
|
||||
```
|
||||
|
||||
**Tradeoff:** Uses ~2GB RAM while running, but every query is fast
|
||||
(~2-3s). Best for users who search frequently.
|
||||
|
||||
### Keeping the Daemon Running
|
||||
|
||||
#### macOS (launchd)
|
||||
|
||||
```bash
|
||||
cat > ~/Library/LaunchAgents/com.qmd.daemon.plist << 'EOF'
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
|
||||
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>Label</key>
|
||||
<string>com.qmd.daemon</string>
|
||||
<key>ProgramArguments</key>
|
||||
<array>
|
||||
<string>qmd</string>
|
||||
<string>mcp</string>
|
||||
<string>--http</string>
|
||||
<string>--daemon</string>
|
||||
</array>
|
||||
<key>RunAtLoad</key>
|
||||
<true/>
|
||||
<key>KeepAlive</key>
|
||||
<true/>
|
||||
<key>StandardOutPath</key>
|
||||
<string>/tmp/qmd-daemon.log</string>
|
||||
<key>StandardErrorPath</key>
|
||||
<string>/tmp/qmd-daemon.log</string>
|
||||
</dict>
|
||||
</plist>
|
||||
EOF
|
||||
|
||||
launchctl load ~/Library/LaunchAgents/com.qmd.daemon.plist
|
||||
```
|
||||
|
||||
#### Linux (systemd user service)
|
||||
|
||||
```bash
|
||||
mkdir -p ~/.config/systemd/user
|
||||
|
||||
cat > ~/.config/systemd/user/qmd-daemon.service << 'EOF'
|
||||
[Unit]
|
||||
Description=QMD MCP Daemon
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
ExecStart=qmd mcp --http --daemon
|
||||
Restart=on-failure
|
||||
RestartSec=10
|
||||
Environment=PATH=/usr/local/bin:/usr/bin:/bin
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
EOF
|
||||
|
||||
systemctl --user daemon-reload
|
||||
systemctl --user enable --now qmd-daemon
|
||||
systemctl --user status qmd-daemon
|
||||
```
|
||||
|
||||
### MCP Tools Reference
|
||||
|
||||
Once connected, these tools are available as `mcp_qmd_*`:
|
||||
|
||||
| MCP Tool | Maps To | Description |
|
||||
|----------|---------|-------------|
|
||||
| `mcp_qmd_search` | `qmd search` | BM25 keyword search |
|
||||
| `mcp_qmd_vsearch` | `qmd vsearch` | Semantic vector search |
|
||||
| `mcp_qmd_deep_search` | `qmd query` | Hybrid search + reranking |
|
||||
| `mcp_qmd_get` | `qmd get` | Retrieve document by ID or path |
|
||||
| `mcp_qmd_status` | `qmd status` | Index health and stats |
|
||||
|
||||
The MCP tools accept structured JSON queries for multi-mode search:
|
||||
|
||||
```json
|
||||
{
|
||||
"searches": [
|
||||
{"type": "lex", "query": "authentication middleware"},
|
||||
{"type": "vec", "query": "how user login is verified"}
|
||||
],
|
||||
"collections": ["project-docs"],
|
||||
"limit": 10
|
||||
}
|
||||
```
|
||||
|
||||
## CLI Usage (Without MCP)
|
||||
|
||||
When MCP is not configured, use qmd directly via terminal:
|
||||
|
||||
```
|
||||
terminal(command="qmd query 'what was decided about the API redesign' --json", timeout=30)
|
||||
```
|
||||
|
||||
For setup and management tasks, always use terminal:
|
||||
|
||||
```
|
||||
terminal(command="qmd collection add ~/Documents/notes --name notes")
|
||||
terminal(command="qmd context add qmd://notes 'Personal research notes and ideas'")
|
||||
terminal(command="qmd embed")
|
||||
terminal(command="qmd status")
|
||||
```
|
||||
|
||||
## How the Search Pipeline Works
|
||||
|
||||
Understanding the internals helps choose the right search mode:
|
||||
|
||||
1. **Query Expansion** — A fine-tuned 1.7B model generates 2 alternative
|
||||
queries. The original gets 2x weight in fusion.
|
||||
2. **Parallel Retrieval** — BM25 (SQLite FTS5) and vector search run
|
||||
simultaneously across all query variants.
|
||||
3. **RRF Fusion** — Reciprocal Rank Fusion (k=60) merges results.
|
||||
Top-rank bonus: #1 gets +0.05, #2-3 get +0.02.
|
||||
4. **LLM Reranking** — qwen3-reranker scores top 30 candidates (0.0-1.0).
|
||||
5. **Position-Aware Blending** — Ranks 1-3: 75% retrieval / 25% reranker.
|
||||
Ranks 4-10: 60/40. Ranks 11+: 40/60 (trusts reranker more for long tail).
|
||||
|
||||
**Smart Chunking:** Documents are split at natural break points (headings,
|
||||
code blocks, blank lines) targeting ~900 tokens with 15% overlap. Code
|
||||
blocks are never split mid-block.
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Always add context descriptions** — `qmd context add` dramatically
|
||||
improves retrieval accuracy. Describe what each collection contains.
|
||||
2. **Re-embed after adding documents** — `qmd embed` must be re-run when
|
||||
new files are added to collections.
|
||||
3. **Use `qmd search` for speed** — when you need fast keyword lookup
|
||||
(code identifiers, exact names), BM25 is instant and needs no models.
|
||||
4. **Use `qmd query` for quality** — when the question is conceptual or
|
||||
the user needs the best possible results, use hybrid search.
|
||||
5. **Prefer MCP integration** — once configured, the agent gets native
|
||||
tools without needing to load this skill each time.
|
||||
6. **Daemon mode for frequent users** — if the user searches their
|
||||
knowledge base regularly, recommend the HTTP daemon setup.
|
||||
7. **First query in structured search gets 2x weight** — put the most
|
||||
important/certain query first when combining lex and vec.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Models downloading on first run"
|
||||
Normal — qmd auto-downloads ~2GB of GGUF models on first use.
|
||||
This is a one-time operation.
|
||||
|
||||
### Cold start latency (~19s)
|
||||
This happens when models aren't loaded in memory. Solutions:
|
||||
- Use HTTP daemon mode (`qmd mcp --http --daemon`) to keep warm
|
||||
- Use `qmd search` (BM25 only) when models aren't needed
|
||||
- MCP stdio mode loads models on first search, stays warm for session
|
||||
|
||||
### macOS: "unable to load extension"
|
||||
Install Homebrew SQLite: `brew install sqlite`
|
||||
Then ensure it's on PATH before system SQLite.
|
||||
|
||||
### "No collections found"
|
||||
Run `qmd collection add <path> --name <name>` to add directories,
|
||||
then `qmd embed` to index them.
|
||||
|
||||
### Embedding model override (CJK/multilingual)
|
||||
Set `QMD_EMBED_MODEL` environment variable for non-English content:
|
||||
```bash
|
||||
export QMD_EMBED_MODEL="your-multilingual-model"
|
||||
```
|
||||
|
||||
## Data Storage
|
||||
|
||||
- **Index & vectors:** `~/.cache/qmd/index.sqlite`
|
||||
- **Models:** Auto-downloaded to local cache on first run
|
||||
- **No cloud dependencies** — everything runs locally
|
||||
|
||||
## References
|
||||
|
||||
- [GitHub: tobi/qmd](https://github.com/tobi/qmd)
|
||||
- [QMD Changelog](https://github.com/tobi/qmd/blob/main/CHANGELOG.md)
|
||||
162
hermes_code/optional-skills/security/1password/SKILL.md
Normal file
162
hermes_code/optional-skills/security/1password/SKILL.md
Normal file
|
|
@ -0,0 +1,162 @@
|
|||
---
|
||||
name: 1password
|
||||
description: Set up and use 1Password CLI (op). Use when installing the CLI, enabling desktop app integration, signing in, and reading/injecting secrets for commands.
|
||||
version: 1.0.0
|
||||
author: arceus77-7, enhanced by Hermes Agent
|
||||
license: MIT
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [security, secrets, 1password, op, cli]
|
||||
category: security
|
||||
setup:
|
||||
help: "Create a service account at https://my.1password.com → Settings → Service Accounts"
|
||||
collect_secrets:
|
||||
- env_var: OP_SERVICE_ACCOUNT_TOKEN
|
||||
prompt: "1Password Service Account Token"
|
||||
provider_url: "https://developer.1password.com/docs/service-accounts/"
|
||||
secret: true
|
||||
---
|
||||
|
||||
# 1Password CLI
|
||||
|
||||
Use this skill when the user wants secrets managed through 1Password instead of plaintext env vars or files.
|
||||
|
||||
## Requirements
|
||||
|
||||
- 1Password account
|
||||
- 1Password CLI (`op`) installed
|
||||
- One of: desktop app integration, service account token (`OP_SERVICE_ACCOUNT_TOKEN`), or Connect server
|
||||
- `tmux` available for stable authenticated sessions during Hermes terminal calls (desktop app flow only)
|
||||
|
||||
## When to Use
|
||||
|
||||
- Install or configure 1Password CLI
|
||||
- Sign in with `op signin`
|
||||
- Read secret references like `op://Vault/Item/field`
|
||||
- Inject secrets into config/templates using `op inject`
|
||||
- Run commands with secret env vars via `op run`
|
||||
|
||||
## Authentication Methods
|
||||
|
||||
### Service Account (recommended for Hermes)
|
||||
|
||||
Set `OP_SERVICE_ACCOUNT_TOKEN` in `~/.hermes/.env` (the skill will prompt for this on first load).
|
||||
No desktop app needed. Supports `op read`, `op inject`, `op run`.
|
||||
|
||||
```bash
|
||||
export OP_SERVICE_ACCOUNT_TOKEN="your-token-here"
|
||||
op whoami # verify — should show Type: SERVICE_ACCOUNT
|
||||
```
|
||||
|
||||
### Desktop App Integration (interactive)
|
||||
|
||||
1. Enable in 1Password desktop app: Settings → Developer → Integrate with 1Password CLI
|
||||
2. Ensure app is unlocked
|
||||
3. Run `op signin` and approve the biometric prompt
|
||||
|
||||
### Connect Server (self-hosted)
|
||||
|
||||
```bash
|
||||
export OP_CONNECT_HOST="http://localhost:8080"
|
||||
export OP_CONNECT_TOKEN="your-connect-token"
|
||||
```
|
||||
|
||||
## Setup
|
||||
|
||||
1. Install CLI:
|
||||
|
||||
```bash
|
||||
# macOS
|
||||
brew install 1password-cli
|
||||
|
||||
# Linux (official package/install docs)
|
||||
# See references/get-started.md for distro-specific links.
|
||||
|
||||
# Windows (winget)
|
||||
winget install AgileBits.1Password.CLI
|
||||
```
|
||||
|
||||
2. Verify:
|
||||
|
||||
```bash
|
||||
op --version
|
||||
```
|
||||
|
||||
3. Choose an auth method above and configure it.
|
||||
|
||||
## Hermes Execution Pattern (desktop app flow)
|
||||
|
||||
Hermes terminal commands are non-interactive by default and can lose auth context between calls.
|
||||
For reliable `op` use with desktop app integration, run sign-in and secret operations inside a dedicated tmux session.
|
||||
|
||||
Note: This is NOT needed when using `OP_SERVICE_ACCOUNT_TOKEN` — the token persists across terminal calls automatically.
|
||||
|
||||
```bash
|
||||
SOCKET_DIR="${TMPDIR:-/tmp}/hermes-tmux-sockets"
|
||||
mkdir -p "$SOCKET_DIR"
|
||||
SOCKET="$SOCKET_DIR/hermes-op.sock"
|
||||
SESSION="op-auth-$(date +%Y%m%d-%H%M%S)"
|
||||
|
||||
tmux -S "$SOCKET" new -d -s "$SESSION" -n shell
|
||||
|
||||
# Sign in (approve in desktop app when prompted)
|
||||
tmux -S "$SOCKET" send-keys -t "$SESSION":0.0 -- "eval \"\$(op signin --account my.1password.com)\"" Enter
|
||||
|
||||
# Verify auth
|
||||
tmux -S "$SOCKET" send-keys -t "$SESSION":0.0 -- "op whoami" Enter
|
||||
|
||||
# Example read
|
||||
tmux -S "$SOCKET" send-keys -t "$SESSION":0.0 -- "op read 'op://Private/Npmjs/one-time password?attribute=otp'" Enter
|
||||
|
||||
# Capture output when needed
|
||||
tmux -S "$SOCKET" capture-pane -p -J -t "$SESSION":0.0 -S -200
|
||||
|
||||
# Cleanup
|
||||
tmux -S "$SOCKET" kill-session -t "$SESSION"
|
||||
```
|
||||
|
||||
## Common Operations
|
||||
|
||||
### Read a secret
|
||||
|
||||
```bash
|
||||
op read "op://app-prod/db/password"
|
||||
```
|
||||
|
||||
### Get OTP
|
||||
|
||||
```bash
|
||||
op read "op://app-prod/npm/one-time password?attribute=otp"
|
||||
```
|
||||
|
||||
### Inject into template
|
||||
|
||||
```bash
|
||||
echo "db_password: {{ op://app-prod/db/password }}" | op inject
|
||||
```
|
||||
|
||||
### Run a command with secret env var
|
||||
|
||||
```bash
|
||||
export DB_PASSWORD="op://app-prod/db/password"
|
||||
op run -- sh -c '[ -n "$DB_PASSWORD" ] && echo "DB_PASSWORD is set" || echo "DB_PASSWORD missing"'
|
||||
```
|
||||
|
||||
## Guardrails
|
||||
|
||||
- Never print raw secrets back to user unless they explicitly request the value.
|
||||
- Prefer `op run` / `op inject` instead of writing secrets into files.
|
||||
- If command fails with "account is not signed in", run `op signin` again in the same tmux session.
|
||||
- If desktop app integration is unavailable (headless/CI), use service account token flow.
|
||||
|
||||
## CI / Headless note
|
||||
|
||||
For non-interactive use, authenticate with `OP_SERVICE_ACCOUNT_TOKEN` and avoid interactive `op signin`.
|
||||
Service accounts require CLI v2.18.0+.
|
||||
|
||||
## References
|
||||
|
||||
- `references/get-started.md`
|
||||
- `references/cli-examples.md`
|
||||
- https://developer.1password.com/docs/cli/
|
||||
- https://developer.1password.com/docs/service-accounts/
|
||||
|
|
@ -0,0 +1,31 @@
|
|||
# op CLI examples
|
||||
|
||||
## Sign-in and identity
|
||||
|
||||
```bash
|
||||
op signin
|
||||
op signin --account my.1password.com
|
||||
op whoami
|
||||
op account list
|
||||
```
|
||||
|
||||
## Read secrets
|
||||
|
||||
```bash
|
||||
op read "op://app-prod/db/password"
|
||||
op read "op://app-prod/npm/one-time password?attribute=otp"
|
||||
```
|
||||
|
||||
## Inject secrets
|
||||
|
||||
```bash
|
||||
echo "api_key: {{ op://app-prod/openai/api key }}" | op inject
|
||||
op inject -i config.tpl.yml -o config.yml
|
||||
```
|
||||
|
||||
## Run command with secrets
|
||||
|
||||
```bash
|
||||
export DB_PASSWORD="op://app-prod/db/password"
|
||||
op run -- sh -c '[ -n "$DB_PASSWORD" ] && echo "DB_PASSWORD is set"'
|
||||
```
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
# 1Password CLI get-started (summary)
|
||||
|
||||
Official docs: https://developer.1password.com/docs/cli/get-started/
|
||||
|
||||
## Core flow
|
||||
|
||||
1. Install `op` CLI.
|
||||
2. Enable desktop app integration in 1Password app.
|
||||
3. Unlock app.
|
||||
4. Run `op signin` and approve prompt.
|
||||
5. Verify with `op whoami`.
|
||||
|
||||
## Multiple accounts
|
||||
|
||||
- Use `op signin --account <subdomain.1password.com>`
|
||||
- Or set `OP_ACCOUNT`
|
||||
|
||||
## Non-interactive / automation
|
||||
|
||||
- Use service accounts and `OP_SERVICE_ACCOUNT_TOKEN`
|
||||
- Prefer `op run` and `op inject` for runtime secret handling
|
||||
3
hermes_code/optional-skills/security/DESCRIPTION.md
Normal file
3
hermes_code/optional-skills/security/DESCRIPTION.md
Normal file
|
|
@ -0,0 +1,3 @@
|
|||
# Security
|
||||
|
||||
Skills for secrets management, credential handling, and security tooling integrations.
|
||||
422
hermes_code/optional-skills/security/oss-forensics/SKILL.md
Normal file
422
hermes_code/optional-skills/security/oss-forensics/SKILL.md
Normal file
|
|
@ -0,0 +1,422 @@
|
|||
---
|
||||
name: oss-forensics
|
||||
description: |
|
||||
Supply chain investigation, evidence recovery, and forensic analysis for GitHub repositories.
|
||||
Covers deleted commit recovery, force-push detection, IOC extraction, multi-source evidence
|
||||
collection, hypothesis formation/validation, and structured forensic reporting.
|
||||
Inspired by RAPTOR's 1800+ line OSS Forensics system.
|
||||
category: security
|
||||
triggers:
|
||||
- "investigate this repository"
|
||||
- "investigate [owner/repo]"
|
||||
- "check for supply chain compromise"
|
||||
- "recover deleted commits"
|
||||
- "forensic analysis of [owner/repo]"
|
||||
- "was this repo compromised"
|
||||
- "supply chain attack"
|
||||
- "suspicious commit"
|
||||
- "force push detected"
|
||||
- "IOC extraction"
|
||||
toolsets:
|
||||
- terminal
|
||||
- web
|
||||
- file
|
||||
- delegation
|
||||
---
|
||||
|
||||
# OSS Security Forensics Skill
|
||||
|
||||
A 7-phase multi-agent investigation framework for researching open-source supply chain attacks.
|
||||
Adapted from RAPTOR's forensics system. Covers GitHub Archive, Wayback Machine, GitHub API,
|
||||
local git analysis, IOC extraction, evidence-backed hypothesis formation and validation,
|
||||
and final forensic report generation.
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ Anti-Hallucination Guardrails
|
||||
|
||||
Read these before every investigation step. Violating them invalidates the report.
|
||||
|
||||
1. **Evidence-First Rule**: Every claim in any report, hypothesis, or summary MUST cite at least one evidence ID (`EV-XXXX`). Assertions without citations are forbidden.
|
||||
2. **STAY IN YOUR LANE**: Each sub-agent (investigator) has a single data source. Do NOT mix sources. The GH Archive investigator does not query the GitHub API, and vice versa. Role boundaries are hard.
|
||||
3. **Fact vs. Hypothesis Separation**: Mark all unverified inferences with `[HYPOTHESIS]`. Only statements verified against original sources may be stated as facts.
|
||||
4. **No Evidence Fabrication**: The hypothesis validator MUST mechanically check that every cited evidence ID actually exists in the evidence store before accepting a hypothesis.
|
||||
5. **Proof-Required Disproval**: A hypothesis cannot be dismissed without a specific, evidence-backed counter-argument. "No evidence found" is not sufficient to disprove—it only makes a hypothesis inconclusive.
|
||||
6. **SHA/URL Double-Verification**: Any commit SHA, URL, or external identifier cited as evidence must be independently confirmed from at least two sources before being marked as verified.
|
||||
7. **Suspicious Code Rule**: Never run code found inside the investigated repository locally. Analyze statically only, or use `execute_code` in a sandboxed environment.
|
||||
8. **Secret Redaction**: Any API keys, tokens, or credentials discovered during investigation must be redacted in the final report. Log them internally only.
|
||||
|
||||
---
|
||||
|
||||
## Example Scenarios
|
||||
|
||||
- **Scenario A: Dependency Confusion**: A malicious package `internal-lib-v2` is uploaded to NPM with a higher version than the internal one. The investigator must track when this package was first seen and if any PushEvents in the target repo updated `package.json` to this version.
|
||||
- **Scenario B: Maintainer Takeover**: A long-term contributor's account is used to push a backdoored `.github/workflows/build.yml`. The investigator looks for PushEvents from this user after a long period of inactivity or from a new IP/location (if detectable via BigQuery).
|
||||
- **Scenario C: Force-Push Hide**: A developer accidentally commits a production secret, then force-pushes to "fix" it. The investigator uses `git fsck` and GH Archive to recover the original commit SHA and verify what was leaked.
|
||||
|
||||
---
|
||||
|
||||
> **Path convention**: Throughout this skill, `SKILL_DIR` refers to the root of this skill's
|
||||
> installation directory (the folder containing this `SKILL.md`). When the skill is loaded,
|
||||
> resolve `SKILL_DIR` to the actual path — e.g. `~/.hermes/skills/security/oss-forensics/`
|
||||
> or the `optional-skills/` equivalent. All script and template references are relative to it.
|
||||
|
||||
## Phase 0: Initialization
|
||||
|
||||
1. Create investigation working directory:
|
||||
```bash
|
||||
mkdir investigation_$(echo "REPO_NAME" | tr '/' '_')
|
||||
cd investigation_$(echo "REPO_NAME" | tr '/' '_')
|
||||
```
|
||||
2. Initialize the evidence store:
|
||||
```bash
|
||||
python3 SKILL_DIR/scripts/evidence-store.py --store evidence.json list
|
||||
```
|
||||
3. Copy the forensic report template:
|
||||
```bash
|
||||
cp SKILL_DIR/templates/forensic-report.md ./investigation-report.md
|
||||
```
|
||||
4. Create an `iocs.md` file to track Indicators of Compromise as they are discovered.
|
||||
5. Record the investigation start time, target repository, and stated investigation goal.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Prompt Parsing and IOC Extraction
|
||||
|
||||
**Goal**: Extract all structured investigative targets from the user's request.
|
||||
|
||||
**Actions**:
|
||||
- Parse the user prompt and extract:
|
||||
- Target repository (`owner/repo`)
|
||||
- Target actors (GitHub handles, email addresses)
|
||||
- Time window of interest (commit date ranges, PR timestamps)
|
||||
- Provided Indicators of Compromise: commit SHAs, file paths, package names, IP addresses, domains, API keys/tokens, malicious URLs
|
||||
- Any linked vendor security reports or blog posts
|
||||
|
||||
**Tools**: Reasoning only, or `execute_code` for regex extraction from large text blocks.
|
||||
|
||||
**Output**: Populate `iocs.md` with extracted IOCs. Each IOC must have:
|
||||
- Type (from: COMMIT_SHA, FILE_PATH, API_KEY, SECRET, IP_ADDRESS, DOMAIN, PACKAGE_NAME, ACTOR_USERNAME, MALICIOUS_URL, OTHER)
|
||||
- Value
|
||||
- Source (user-provided, inferred)
|
||||
|
||||
**Reference**: See [evidence-types.md](./references/evidence-types.md) for IOC taxonomy.
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Parallel Evidence Collection
|
||||
|
||||
Spawn up to 5 specialist investigator sub-agents using `delegate_task` (batch mode, max 3 concurrent). Each investigator has a **single data source** and must not mix sources.
|
||||
|
||||
> **Orchestrator note**: Pass the IOC list from Phase 1 and the investigation time window in the `context` field of each delegated task.
|
||||
|
||||
---
|
||||
|
||||
### Investigator 1: Local Git Investigator
|
||||
|
||||
**ROLE BOUNDARY**: You query the LOCAL GIT REPOSITORY ONLY. Do not call any external APIs.
|
||||
|
||||
**Actions**:
|
||||
```bash
|
||||
# Clone repository
|
||||
git clone https://github.com/OWNER/REPO.git target_repo && cd target_repo
|
||||
|
||||
# Full commit log with stats
|
||||
git log --all --full-history --stat --format="%H|%ae|%an|%ai|%s" > ../git_log.txt
|
||||
|
||||
# Detect force-push evidence (orphaned/dangling commits)
|
||||
git fsck --lost-found --unreachable 2>&1 | grep commit > ../dangling_commits.txt
|
||||
|
||||
# Check reflog for rewritten history
|
||||
git reflog --all > ../reflog.txt
|
||||
|
||||
# List ALL branches including deleted remote refs
|
||||
git branch -a -v > ../branches.txt
|
||||
|
||||
# Find suspicious large binary additions
|
||||
git log --all --diff-filter=A --name-only --format="%H %ai" -- "*.so" "*.dll" "*.exe" "*.bin" > ../binary_additions.txt
|
||||
|
||||
# Check for GPG signature anomalies
|
||||
git log --show-signature --format="%H %ai %aN" > ../signature_check.txt 2>&1
|
||||
```
|
||||
|
||||
**Evidence to collect** (add via `python3 SKILL_DIR/scripts/evidence-store.py add`):
|
||||
- Each dangling commit SHA → type: `git`
|
||||
- Force-push evidence (reflog showing history rewrite) → type: `git`
|
||||
- Unsigned commits from verified contributors → type: `git`
|
||||
- Suspicious binary file additions → type: `git`
|
||||
|
||||
**Reference**: See [recovery-techniques.md](./references/recovery-techniques.md) for accessing force-pushed commits.
|
||||
|
||||
---
|
||||
|
||||
### Investigator 2: GitHub API Investigator
|
||||
|
||||
**ROLE BOUNDARY**: You query the GITHUB REST API ONLY. Do not run git commands locally.
|
||||
|
||||
**Actions**:
|
||||
```bash
|
||||
# Commits (paginated)
|
||||
curl -s "https://api.github.com/repos/OWNER/REPO/commits?per_page=100" > api_commits.json
|
||||
|
||||
# Pull Requests including closed/deleted
|
||||
curl -s "https://api.github.com/repos/OWNER/REPO/pulls?state=all&per_page=100" > api_prs.json
|
||||
|
||||
# Issues
|
||||
curl -s "https://api.github.com/repos/OWNER/REPO/issues?state=all&per_page=100" > api_issues.json
|
||||
|
||||
# Contributors and collaborator changes
|
||||
curl -s "https://api.github.com/repos/OWNER/REPO/contributors" > api_contributors.json
|
||||
|
||||
# Repository events (last 300)
|
||||
curl -s "https://api.github.com/repos/OWNER/REPO/events?per_page=100" > api_events.json
|
||||
|
||||
# Check specific suspicious commit SHA details
|
||||
curl -s "https://api.github.com/repos/OWNER/REPO/git/commits/SHA" > commit_detail.json
|
||||
|
||||
# Releases
|
||||
curl -s "https://api.github.com/repos/OWNER/REPO/releases?per_page=100" > api_releases.json
|
||||
|
||||
# Check if a specific commit exists (force-pushed commits may 404 on commits/ but succeed on git/commits/)
|
||||
curl -s "https://api.github.com/repos/OWNER/REPO/commits/SHA" | jq .sha
|
||||
```
|
||||
|
||||
**Cross-reference targets** (flag discrepancies as evidence):
|
||||
- PR exists in archive but missing from API → evidence of deletion
|
||||
- Contributor in archive events but not in contributors list → evidence of permission revocation
|
||||
- Commit in archive PushEvents but not in API commit list → evidence of force-push/deletion
|
||||
|
||||
**Reference**: See [evidence-types.md](./references/evidence-types.md) for GH event types.
|
||||
|
||||
---
|
||||
|
||||
### Investigator 3: Wayback Machine Investigator
|
||||
|
||||
**ROLE BOUNDARY**: You query the WAYBACK MACHINE CDX API ONLY. Do not use the GitHub API.
|
||||
|
||||
**Goal**: Recover deleted GitHub pages (READMEs, issues, PRs, releases, wiki pages).
|
||||
|
||||
**Actions**:
|
||||
```bash
|
||||
# Search for archived snapshots of the repo main page
|
||||
curl -s "https://web.archive.org/cdx/search/cdx?url=github.com/OWNER/REPO&output=json&limit=100&from=YYYYMMDD&to=YYYYMMDD" > wayback_main.json
|
||||
|
||||
# Search for a specific deleted issue
|
||||
curl -s "https://web.archive.org/cdx/search/cdx?url=github.com/OWNER/REPO/issues/NUM&output=json&limit=50" > wayback_issue_NUM.json
|
||||
|
||||
# Search for a specific deleted PR
|
||||
curl -s "https://web.archive.org/cdx/search/cdx?url=github.com/OWNER/REPO/pull/NUM&output=json&limit=50" > wayback_pr_NUM.json
|
||||
|
||||
# Fetch the best snapshot of a page
|
||||
# Use the Wayback Machine URL: https://web.archive.org/web/TIMESTAMP/ORIGINAL_URL
|
||||
# Example: https://web.archive.org/web/20240101000000*/github.com/OWNER/REPO
|
||||
|
||||
# Advanced: Search for deleted releases/tags
|
||||
curl -s "https://web.archive.org/cdx/search/cdx?url=github.com/OWNER/REPO/releases/tag/*&output=json" > wayback_tags.json
|
||||
|
||||
# Advanced: Search for historical wiki changes
|
||||
curl -s "https://web.archive.org/cdx/search/cdx?url=github.com/OWNER/REPO/wiki/*&output=json" > wayback_wiki.json
|
||||
```
|
||||
|
||||
**Evidence to collect**:
|
||||
- Archived snapshots of deleted issues/PRs with their content
|
||||
- Historical README versions showing changes
|
||||
- Evidence of content present in archive but missing from current GitHub state
|
||||
|
||||
**Reference**: See [github-archive-guide.md](./references/github-archive-guide.md) for CDX API parameters.
|
||||
|
||||
---
|
||||
|
||||
### Investigator 4: GH Archive / BigQuery Investigator
|
||||
|
||||
**ROLE BOUNDARY**: You query GITHUB ARCHIVE via BIGQUERY ONLY. This is a tamper-proof record of all public GitHub events.
|
||||
|
||||
> **Prerequisites**: Requires Google Cloud credentials with BigQuery access (`gcloud auth application-default login`). If unavailable, skip this investigator and note it in the report.
|
||||
|
||||
**Cost Optimization Rules** (MANDATORY):
|
||||
1. ALWAYS run a `--dry_run` before every query to estimate cost.
|
||||
2. Use `_TABLE_SUFFIX` to filter by date range and minimize scanned data.
|
||||
3. Only SELECT the columns you need.
|
||||
4. Add a LIMIT unless aggregating.
|
||||
|
||||
```bash
|
||||
# Template: safe BigQuery query for PushEvents to OWNER/REPO
|
||||
bq query --use_legacy_sql=false --dry_run "
|
||||
SELECT created_at, actor.login, payload.commits, payload.before, payload.head,
|
||||
payload.size, payload.distinct_size
|
||||
FROM \`githubarchive.month.*\`
|
||||
WHERE _TABLE_SUFFIX BETWEEN 'YYYYMM' AND 'YYYYMM'
|
||||
AND type = 'PushEvent'
|
||||
AND repo.name = 'OWNER/REPO'
|
||||
LIMIT 1000
|
||||
"
|
||||
# If cost is acceptable, re-run without --dry_run
|
||||
|
||||
# Detect force-pushes: zero-distinct_size PushEvents mean commits were force-erased
|
||||
# payload.distinct_size = 0 AND payload.size > 0 → force push indicator
|
||||
|
||||
# Check for deleted branch events
|
||||
bq query --use_legacy_sql=false "
|
||||
SELECT created_at, actor.login, payload.ref, payload.ref_type
|
||||
FROM \`githubarchive.month.*\`
|
||||
WHERE _TABLE_SUFFIX BETWEEN 'YYYYMM' AND 'YYYYMM'
|
||||
AND type = 'DeleteEvent'
|
||||
AND repo.name = 'OWNER/REPO'
|
||||
LIMIT 200
|
||||
"
|
||||
```
|
||||
|
||||
**Evidence to collect**:
|
||||
- Force-push events (payload.size > 0, payload.distinct_size = 0)
|
||||
- DeleteEvents for branches/tags
|
||||
- WorkflowRunEvents for suspicious CI/CD automation
|
||||
- PushEvents that precede a "gap" in the git log (evidence of rewrite)
|
||||
|
||||
**Reference**: See [github-archive-guide.md](./references/github-archive-guide.md) for all 12 event types and query patterns.
|
||||
|
||||
---
|
||||
|
||||
### Investigator 5: IOC Enrichment Investigator
|
||||
|
||||
**ROLE BOUNDARY**: You enrich EXISTING IOCs from Phase 1 using passive public sources ONLY. Do not execute any code from the target repository.
|
||||
|
||||
**Actions**:
|
||||
- For each commit SHA: attempt recovery via direct GitHub URL (`github.com/OWNER/REPO/commit/SHA.patch`)
|
||||
- For each domain/IP: check passive DNS, WHOIS records (via `web_extract` on public WHOIS services)
|
||||
- For each package name: check npm/PyPI for matching malicious package reports
|
||||
- For each actor username: check GitHub profile, contribution history, account age
|
||||
- Recover force-pushed commits using 3 methods (see [recovery-techniques.md](./references/recovery-techniques.md))
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Evidence Consolidation
|
||||
|
||||
After all investigators complete:
|
||||
|
||||
1. Run `python3 SKILL_DIR/scripts/evidence-store.py --store evidence.json list` to see all collected evidence.
|
||||
2. For each piece of evidence, verify the `content_sha256` hash matches the original source.
|
||||
3. Group evidence by:
|
||||
- **Timeline**: Sort all timestamped evidence chronologically
|
||||
- **Actor**: Group by GitHub handle or email
|
||||
- **IOC**: Link evidence to the IOC it relates to
|
||||
4. Identify **discrepancies**: items present in one source but absent in another (key deletion indicators).
|
||||
5. Flag evidence as `[VERIFIED]` (confirmed from 2+ independent sources) or `[UNVERIFIED]` (single source only).
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Hypothesis Formation
|
||||
|
||||
A hypothesis must:
|
||||
- State a specific claim (e.g., "Actor X force-pushed to BRANCH on DATE to erase commit SHA")
|
||||
- Cite at least 2 evidence IDs that support it (`EV-XXXX`, `EV-YYYY`)
|
||||
- Identify what evidence would disprove it
|
||||
- Be labeled `[HYPOTHESIS]` until validated
|
||||
|
||||
**Common hypothesis templates** (see [investigation-templates.md](./references/investigation-templates.md)):
|
||||
- Maintainer Compromise: legitimate account used post-takeover to inject malicious code
|
||||
- Dependency Confusion: package name squatting to intercept installs
|
||||
- CI/CD Injection: malicious workflow changes to run code during builds
|
||||
- Typosquatting: near-identical package name targeting misspellers
|
||||
- Credential Leak: token/key accidentally committed then force-pushed to erase
|
||||
|
||||
For each hypothesis, spawn a `delegate_task` sub-agent to attempt to find disconfirming evidence before confirming.
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Hypothesis Validation
|
||||
|
||||
The validator sub-agent MUST mechanically check:
|
||||
|
||||
1. For each hypothesis, extract all cited evidence IDs.
|
||||
2. Verify each ID exists in `evidence.json` (hard failure if any ID is missing → hypothesis rejected as potentially fabricated).
|
||||
3. Verify each `[VERIFIED]` piece of evidence was confirmed from 2+ sources.
|
||||
4. Check logical consistency: does the timeline depicted by the evidence support the hypothesis?
|
||||
5. Check for alternative explanations: could the same evidence pattern arise from a benign cause?
|
||||
|
||||
**Output**:
|
||||
- `VALIDATED`: All evidence cited, verified, logically consistent, no plausible alternative explanation.
|
||||
- `INCONCLUSIVE`: Evidence supports hypothesis but alternative explanations exist or evidence is insufficient.
|
||||
- `REJECTED`: Missing evidence IDs, unverified evidence cited as fact, logical inconsistency detected.
|
||||
|
||||
Rejected hypotheses feed back into Phase 4 for refinement (max 3 iterations).
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Final Report Generation
|
||||
|
||||
Populate `investigation-report.md` using the template in [forensic-report.md](./templates/forensic-report.md).
|
||||
|
||||
**Mandatory sections**:
|
||||
- Executive Summary: one-paragraph verdict (Compromised / Clean / Inconclusive) with confidence level
|
||||
- Timeline: chronological reconstruction of all significant events with evidence citations
|
||||
- Validated Hypotheses: each with status and supporting evidence IDs
|
||||
- Evidence Registry: table of all `EV-XXXX` entries with source, type, and verification status
|
||||
- IOC List: all extracted and enriched Indicators of Compromise
|
||||
- Chain of Custody: how evidence was collected, from what sources, at what timestamps
|
||||
- Recommendations: immediate mitigations if compromise detected; monitoring recommendations
|
||||
|
||||
**Report rules**:
|
||||
- Every factual claim must have at least one `[EV-XXXX]` citation
|
||||
- Executive Summary must state confidence level (High / Medium / Low)
|
||||
- All secrets/credentials must be redacted to `[REDACTED]`
|
||||
|
||||
---
|
||||
|
||||
## Phase 7: Completion
|
||||
|
||||
1. Run final evidence count: `python3 SKILL_DIR/scripts/evidence-store.py --store evidence.json list`
|
||||
2. Archive the full investigation directory.
|
||||
3. If compromise is confirmed:
|
||||
- List immediate mitigations (rotate credentials, pin dependency hashes, notify affected users)
|
||||
- Identify affected versions/packages
|
||||
- Note disclosure obligations (if a public package: coordinate with the package registry)
|
||||
4. Present the final `investigation-report.md` to the user.
|
||||
|
||||
---
|
||||
|
||||
## Ethical Use Guidelines
|
||||
|
||||
This skill is designed for **defensive security investigation** — protecting open-source software from supply chain attacks. It must not be used for:
|
||||
|
||||
- **Harassment or stalking** of contributors or maintainers
|
||||
- **Doxing** — correlating GitHub activity to real identities for malicious purposes
|
||||
- **Competitive intelligence** — investigating proprietary or internal repositories without authorization
|
||||
- **False accusations** — publishing investigation results without validated evidence (see anti-hallucination guardrails)
|
||||
|
||||
Investigations should be conducted with the principle of **minimal intrusion**: collect only the evidence necessary to validate or refute the hypothesis. When publishing results, follow responsible disclosure practices and coordinate with affected maintainers before public disclosure.
|
||||
|
||||
If the investigation reveals a genuine compromise, follow the coordinated vulnerability disclosure process:
|
||||
1. Notify the repository maintainers privately first
|
||||
2. Allow reasonable time for remediation (typically 90 days)
|
||||
3. Coordinate with package registries (npm, PyPI, etc.) if published packages are affected
|
||||
4. File a CVE if appropriate
|
||||
|
||||
---
|
||||
|
||||
## API Rate Limiting
|
||||
|
||||
GitHub REST API enforces rate limits that will interrupt large investigations if not managed.
|
||||
|
||||
**Authenticated requests**: 5,000/hour (requires `GITHUB_TOKEN` env var or `gh` CLI auth)
|
||||
**Unauthenticated requests**: 60/hour (unusable for investigations)
|
||||
|
||||
**Best practices**:
|
||||
- Always authenticate: `export GITHUB_TOKEN=ghp_...` or use `gh` CLI (auto-authenticates)
|
||||
- Use conditional requests (`If-None-Match` / `If-Modified-Since` headers) to avoid consuming quota on unchanged data
|
||||
- For paginated endpoints, fetch all pages in sequence — don't parallelize against the same endpoint
|
||||
- Check `X-RateLimit-Remaining` header; if below 100, pause for `X-RateLimit-Reset` timestamp
|
||||
- BigQuery has its own quotas (10 TiB/day free tier) — always dry-run first
|
||||
- Wayback Machine CDX API: no formal rate limit, but be courteous (1-2 req/sec max)
|
||||
|
||||
If rate-limited mid-investigation, record the partial results in the evidence store and note the limitation in the report.
|
||||
|
||||
---
|
||||
|
||||
## Reference Materials
|
||||
|
||||
- [github-archive-guide.md](./references/github-archive-guide.md) — BigQuery queries, CDX API, 12 event types
|
||||
- [evidence-types.md](./references/evidence-types.md) — IOC taxonomy, evidence source types, observation types
|
||||
- [recovery-techniques.md](./references/recovery-techniques.md) — Recovering deleted commits, PRs, issues
|
||||
- [investigation-templates.md](./references/investigation-templates.md) — Pre-built hypothesis templates per attack type
|
||||
- [evidence-store.py](./scripts/evidence-store.py) — CLI tool for managing the evidence JSON store
|
||||
- [forensic-report.md](./templates/forensic-report.md) — Structured report template
|
||||
|
|
@ -0,0 +1,89 @@
|
|||
# Evidence Types Reference
|
||||
|
||||
Taxonomy of all evidence types, IOC types, GitHub event types, and observation types
|
||||
used in OSS forensic investigations.
|
||||
|
||||
---
|
||||
|
||||
## Evidence Source Types
|
||||
|
||||
| Type | Description | Example Sources |
|
||||
|------|-------------|-----------------|
|
||||
| `git` | Data from local git repository analysis | `git log`, `git fsck`, `git reflog`, `git blame` |
|
||||
| `gh_api` | Data from GitHub REST API responses | `/repos/.../commits`, `/repos/.../pulls`, `/repos/.../events` |
|
||||
| `gh_archive` | Data from GitHub Archive (BigQuery) | `githubarchive.month.*` BigQuery tables |
|
||||
| `web_archive` | Archived web pages from Wayback Machine | CDX API results, `web.archive.org/web/...` snapshots |
|
||||
| `ioc` | Indicator of Compromise from any source | Extracted from vendor reports, git history, network traces |
|
||||
| `analysis` | Derived insight from cross-source correlation | "SHA present in archive but absent from API" |
|
||||
| `vendor_report` | External security vendor or researcher report | CVE advisories, blog posts, NVD records |
|
||||
| `manual` | Manually recorded observation by investigator | Notes on behavioral patterns, timeline gaps |
|
||||
|
||||
---
|
||||
|
||||
## IOC Types
|
||||
|
||||
| Type | Description | Example |
|
||||
|------|-------------|---------|
|
||||
| `COMMIT_SHA` | A git commit hash linked to malicious activity | `abc123def456...` |
|
||||
| `FILE_PATH` | A suspicious file inside the repository | `src/utils/crypto.js`, `dist/index.min.js` |
|
||||
| `API_KEY` | An API key accidentally committed | `AKIA...` (AWS), `ghp_...` (GitHub PAT) |
|
||||
| `SECRET` | A generic secret / credential | Database password, private key blob |
|
||||
| `IP_ADDRESS` | A C2 server or attacker IP | `192.0.2.1` |
|
||||
| `DOMAIN` | A malicious or suspicious domain | `evil-cdn.io`, typosquatted package registry domain |
|
||||
| `PACKAGE_NAME` | A malicious or squatted package name | `colo-rs` (typosquatting `color`), `lodash-utils` |
|
||||
| `ACTOR_USERNAME` | A GitHub handle linked to the attack | `malicious-bot-account` |
|
||||
| `MALICIOUS_URL` | A URL to a malicious resource | `https://evil.example.com/payload.sh` |
|
||||
| `WORKFLOW_FILE` | A suspicious CI/CD workflow file | `.github/workflows/release.yml` |
|
||||
| `BRANCH_NAME` | A suspicious branch | `refs/heads/temp-fix-do-not-merge` |
|
||||
| `TAG_NAME` | A suspicious git tag | `v1.0.0-security-patch` |
|
||||
| `RELEASE_NAME` | A suspicious release | Release with no associated tag or changelog |
|
||||
| `OTHER` | Catch-all for unclassified IOCs | — |
|
||||
|
||||
---
|
||||
|
||||
## GitHub Archive Event Types (12 Types)
|
||||
|
||||
| Event Type | Forensic Relevance |
|
||||
|------------|-------------------|
|
||||
| `PushEvent` | Core: `payload.distinct_size=0` with `payload.size>0` → force push. `payload.before`/`payload.head` shows rewritten history. |
|
||||
| `PullRequestEvent` | Detects deleted PRs, rapid open→close patterns, PRs from new accounts |
|
||||
| `IssueEvent` | Detects deleted issues, coordinated labeling, rapid closure of vulnerability reports |
|
||||
| `IssueCommentEvent` | Deleted comments, rapid activity bursts |
|
||||
| `WatchEvent` | Star-farming campaigns (coordinated starring from new accounts) |
|
||||
| `ForkEvent` | Unusual fork patterns before malicious commit |
|
||||
| `CreateEvent` | Branch/tag creation: signals new release or code injection point |
|
||||
| `DeleteEvent` | Branch/tag deletion: critical — often used to hide traces |
|
||||
| `ReleaseEvent` | Unauthorized releases, release artifacts modified post-publish |
|
||||
| `MemberEvent` | Collaborator added/removed: maintainer compromise indicator |
|
||||
| `PublicEvent` | Repository made public (sometimes to drop malicious code briefly) |
|
||||
| `WorkflowRunEvent` | CI/CD pipeline executions: workflow injection, secret exfiltration |
|
||||
|
||||
---
|
||||
|
||||
## Evidence Verification States
|
||||
|
||||
| State | Meaning |
|
||||
|-------|---------|
|
||||
| `unverified` | Collected from a single source, not cross-referenced |
|
||||
| `single_source` | The primary source has been confirmed directly (e.g., SHA resolves on GitHub), but no second source |
|
||||
| `multi_source_verified` | Confirmed from 2+ independent sources (e.g., GH Archive AND GitHub API both show the same event) |
|
||||
|
||||
Only `multi_source_verified` evidence may be cited as fact in validated hypotheses.
|
||||
`unverified` and `single_source` evidence must be labeled `[UNVERIFIED]` or `[SINGLE-SOURCE]`.
|
||||
|
||||
---
|
||||
|
||||
## Observation Types (Patterned after RAPTOR)
|
||||
|
||||
| Type | Description |
|
||||
|------|-------------|
|
||||
| `CommitObservation` | Specific commit SHA with metadata (author, date, files changed) |
|
||||
| `ForceWashObservation` | Evidence that commits were force-erased from a branch |
|
||||
| `DanglingCommitObservation` | SHA present in git object store but unreachable from any ref |
|
||||
| `IssueObservation` | A GitHub issue (current or archived) with title, body, timestamp |
|
||||
| `PRObservation` | A GitHub PR (current or archived) with diff summary, reviewers |
|
||||
| `IOC` | A single Indicator of Compromise with context |
|
||||
| `TimelineGap` | A period with unusual absence of expected activity |
|
||||
| `ActorAnomalyObservation` | Behavioral anomaly for a specific GitHub actor |
|
||||
| `WorkflowAnomalyObservation` | Suspicious CI/CD workflow change or unexpected run |
|
||||
| `CrossSourceDiscrepancy` | Item present in one source but absent in another (strong deletion indicator) |
|
||||
|
|
@ -0,0 +1,184 @@
|
|||
# GitHub Archive Query Guide (BigQuery)
|
||||
|
||||
GitHub Archive records every public event on GitHub as immutable JSON records. This data is accessible via Google BigQuery and is the most reliable source for forensic investigation — events cannot be deleted or modified after recording.
|
||||
|
||||
## Public Dataset
|
||||
|
||||
- **Project**: `githubarchive`
|
||||
- **Tables**: `day.YYYYMMDD`, `month.YYYYMM`, `year.YYYY`
|
||||
- **Cost**: $6.25 per TiB scanned. Always run dry runs first.
|
||||
- **Access**: Requires a Google Cloud account with BigQuery enabled. Free tier includes 1 TiB/month of queries.
|
||||
|
||||
---
|
||||
|
||||
## The 12 GitHub Event Types
|
||||
|
||||
| Event Type | What It Records | Forensic Value |
|
||||
|------------|-----------------|----------------|
|
||||
| `PushEvent` | Commits pushed to a branch | Force-push detection, commit timeline, author attribution |
|
||||
| `PullRequestEvent` | PR opened, closed, merged, reopened | Deleted PR recovery, review timeline |
|
||||
| `IssuesEvent` | Issue opened, closed, reopened, labeled | Deleted issue recovery, social engineering traces |
|
||||
| `IssueCommentEvent` | Comments on issues and PRs | Deleted comment recovery, communication patterns |
|
||||
| `CreateEvent` | Branch, tag, or repository creation | Suspicious branch creation, tag timing |
|
||||
| `DeleteEvent` | Branch or tag deletion | Evidence of cleanup after compromise |
|
||||
| `MemberEvent` | Collaborator added or removed | Permission changes, access escalation |
|
||||
| `PublicEvent` | Repository made public | Accidental exposure of private repos |
|
||||
| `WatchEvent` | User stars a repository | Actor reconnaissance patterns |
|
||||
| `ForkEvent` | Repository forked | Exfiltration of code before cleanup |
|
||||
| `ReleaseEvent` | Release published, edited, deleted | Malicious release injection, deleted release recovery |
|
||||
| `WorkflowRunEvent` | GitHub Actions workflow triggered | CI/CD abuse, unauthorized workflow runs |
|
||||
|
||||
---
|
||||
|
||||
## Query Templates
|
||||
|
||||
### Basic: All Events for a Repository
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
created_at,
|
||||
type,
|
||||
actor.login,
|
||||
repo.name,
|
||||
payload
|
||||
FROM
|
||||
`githubarchive.day.20240101` -- Adjust date
|
||||
WHERE
|
||||
repo.name = 'owner/repo'
|
||||
AND type IN ('PushEvent', 'DeleteEvent', 'MemberEvent')
|
||||
ORDER BY
|
||||
created_at ASC
|
||||
```
|
||||
|
||||
### Force-Push Detection
|
||||
|
||||
Force-pushes produce PushEvents where commits are overwritten. Key indicators:
|
||||
- `payload.distinct_size = 0` with `payload.size > 0` → commits were erased
|
||||
- `payload.before` contains the SHA before the rewrite (recoverable)
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
created_at,
|
||||
actor.login,
|
||||
JSON_EXTRACT_SCALAR(payload, '$.before') AS before_sha,
|
||||
JSON_EXTRACT_SCALAR(payload, '$.head') AS after_sha,
|
||||
JSON_EXTRACT_SCALAR(payload, '$.size') AS total_commits,
|
||||
JSON_EXTRACT_SCALAR(payload, '$.distinct_size') AS distinct_commits,
|
||||
JSON_EXTRACT_SCALAR(payload, '$.ref') AS branch_ref
|
||||
FROM
|
||||
`githubarchive.month.*`
|
||||
WHERE
|
||||
_TABLE_SUFFIX BETWEEN '202401' AND '202403'
|
||||
AND type = 'PushEvent'
|
||||
AND repo.name = 'owner/repo'
|
||||
AND CAST(JSON_EXTRACT_SCALAR(payload, '$.distinct_size') AS INT64) = 0
|
||||
ORDER BY
|
||||
created_at ASC
|
||||
```
|
||||
|
||||
### Deleted Branch/Tag Detection
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
created_at,
|
||||
actor.login,
|
||||
JSON_EXTRACT_SCALAR(payload, '$.ref') AS deleted_ref,
|
||||
JSON_EXTRACT_SCALAR(payload, '$.ref_type') AS ref_type
|
||||
FROM
|
||||
`githubarchive.month.*`
|
||||
WHERE
|
||||
_TABLE_SUFFIX BETWEEN '202401' AND '202403'
|
||||
AND type = 'DeleteEvent'
|
||||
AND repo.name = 'owner/repo'
|
||||
ORDER BY
|
||||
created_at ASC
|
||||
```
|
||||
|
||||
### Collaborator Permission Changes
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
created_at,
|
||||
actor.login,
|
||||
JSON_EXTRACT_SCALAR(payload, '$.action') AS action,
|
||||
JSON_EXTRACT_SCALAR(payload, '$.member.login') AS member
|
||||
FROM
|
||||
`githubarchive.month.*`
|
||||
WHERE
|
||||
_TABLE_SUFFIX BETWEEN '202401' AND '202403'
|
||||
AND type = 'MemberEvent'
|
||||
AND repo.name = 'owner/repo'
|
||||
ORDER BY
|
||||
created_at ASC
|
||||
```
|
||||
|
||||
### CI/CD Workflow Activity
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
created_at,
|
||||
actor.login,
|
||||
JSON_EXTRACT_SCALAR(payload, '$.action') AS action,
|
||||
JSON_EXTRACT_SCALAR(payload, '$.workflow_run.name') AS workflow_name,
|
||||
JSON_EXTRACT_SCALAR(payload, '$.workflow_run.conclusion') AS conclusion,
|
||||
JSON_EXTRACT_SCALAR(payload, '$.workflow_run.head_sha') AS head_sha
|
||||
FROM
|
||||
`githubarchive.month.*`
|
||||
WHERE
|
||||
_TABLE_SUFFIX BETWEEN '202401' AND '202403'
|
||||
AND type = 'WorkflowRunEvent'
|
||||
AND repo.name = 'owner/repo'
|
||||
ORDER BY
|
||||
created_at ASC
|
||||
```
|
||||
|
||||
### Actor Activity Profiling
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
type,
|
||||
COUNT(*) AS event_count,
|
||||
MIN(created_at) AS first_event,
|
||||
MAX(created_at) AS last_event
|
||||
FROM
|
||||
`githubarchive.month.*`
|
||||
WHERE
|
||||
_TABLE_SUFFIX BETWEEN '202301' AND '202412'
|
||||
AND actor.login = 'suspicious-username'
|
||||
GROUP BY type
|
||||
ORDER BY event_count DESC
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Cost Optimization (MANDATORY)
|
||||
|
||||
1. **Always dry run first**: Add `--dry_run` flag to `bq query` to see estimated bytes scanned before executing.
|
||||
2. **Use `_TABLE_SUFFIX`**: Narrow the date range as much as possible. `day.*` tables are cheapest for narrow windows; `month.*` for broader sweeps.
|
||||
3. **Select only needed columns**: Avoid `SELECT *`. The `payload` column is large — only select specific JSON paths.
|
||||
4. **Add LIMIT**: Use `LIMIT 1000` during exploration. Remove only for final exhaustive queries.
|
||||
5. **Column filtering in WHERE**: Filter on indexed columns (`type`, `repo.name`, `actor.login`) before payload extraction.
|
||||
|
||||
**Cost estimation**: A single month of GH Archive data is ~1-2 TiB uncompressed. Querying a specific repo + event type with `_TABLE_SUFFIX` typically scans 1-10 GiB ($0.006-$0.06).
|
||||
|
||||
---
|
||||
|
||||
## Accessing via Hermes
|
||||
|
||||
**Option A: BigQuery CLI** (if `gcloud` is installed)
|
||||
```bash
|
||||
bq query --use_legacy_sql=false --format=json "YOUR QUERY"
|
||||
```
|
||||
|
||||
**Option B: Python** (via `execute_code`)
|
||||
```python
|
||||
from google.cloud import bigquery
|
||||
client = bigquery.Client()
|
||||
query = "YOUR QUERY"
|
||||
results = client.query(query).result()
|
||||
for row in results:
|
||||
print(dict(row))
|
||||
```
|
||||
|
||||
**Option C: No GCP credentials available**
|
||||
If BigQuery is unavailable, document this limitation in the report. Use the other 4 investigators (Git, GitHub API, Wayback Machine, IOC Enrichment) — they cover most investigation needs without BigQuery.
|
||||
|
|
@ -0,0 +1,131 @@
|
|||
# Investigation Templates
|
||||
|
||||
Pre-built hypothesis and investigation templates for common supply chain attack scenarios.
|
||||
Each template includes: attack pattern, key evidence to collect, and hypothesis starters.
|
||||
|
||||
---
|
||||
|
||||
## Template 1: Maintainer Account Compromise
|
||||
|
||||
**Pattern**: Attacker gains access to a legitimate maintainer account (phishing, credential stuffing)
|
||||
and uses it to push malicious code, create backdoored releases, or exfiltrate CI secrets.
|
||||
|
||||
**Real-world examples**: XZ Utils (2024), Codecov (2021), event-stream (2018)
|
||||
|
||||
**Key Evidence to Collect**:
|
||||
- [ ] Push events from maintainer account outside normal working hours/timezone
|
||||
- [ ] Commits adding new dependencies, obfuscated code, or modified build scripts
|
||||
- [ ] Release creation immediately after suspicious push (to maximize package distribution)
|
||||
- [ ] MemberEvent adding unknown collaborators (attacker adding backup access)
|
||||
- [ ] WorkflowRunEvent with unexpected secret access or exfiltration-like behavior
|
||||
- [ ] Account login location changes (check social media, conference talks for corroboration)
|
||||
|
||||
**Hypothesis Starters**:
|
||||
```
|
||||
[HYPOTHESIS] Actor <HANDLE>'s account was compromised on or around <DATE>,
|
||||
based on anomalous commit timing [EV-XXXX] and geographic access patterns [EV-YYYY].
|
||||
```
|
||||
```
|
||||
[HYPOTHESIS] Release <VERSION> was published by the compromised account to push
|
||||
malicious code to downstream users, evidenced by the malicious commit [EV-XXXX]
|
||||
being added <N> hours before the release [EV-YYYY].
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Template 2: Malicious Dependency Injection
|
||||
|
||||
**Pattern**: A trusted package is modified to include malicious code in a dependency,
|
||||
or a new malicious dependency is injected into an existing package.
|
||||
|
||||
**Key Evidence to Collect**:
|
||||
- [ ] Diff of `package.json`/`requirements.txt`/`go.mod` before and after suspicious commit
|
||||
- [ ] The new dependency's publication timestamp vs. the injection commit timestamp
|
||||
- [ ] Whether the new dependency exists on npm/PyPI and who owns it
|
||||
- [ ] Any obfuscation patterns in the injected dependency code
|
||||
- [ ] Install-time scripts (`postinstall`, `setup.py`, etc.) that execute code on install
|
||||
|
||||
**Hypothesis Starters**:
|
||||
```
|
||||
[HYPOTHESIS] Commit <SHA> [EV-XXXX] introduced dependency <PACKAGE@VERSION>
|
||||
which appears to be a malicious package published by actor <HANDLE> [EV-YYYY],
|
||||
designed to execute <BEHAVIOR> during installation.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Template 3: CI/CD Pipeline Injection
|
||||
|
||||
**Pattern**: Attacker modifies GitHub Actions workflows to steal secrets, exfiltrate code,
|
||||
or inject malicious artifacts into the build output.
|
||||
|
||||
**Key Evidence to Collect**:
|
||||
- [ ] Diff of all `.github/workflows/*.yml` files before/after suspicious period
|
||||
- [ ] WorkflowRunEvents triggered by the modified workflows
|
||||
- [ ] Any `curl`, `wget`, or network calls added to workflow steps
|
||||
- [ ] New or modified `env:` sections referencing `secrets.*`
|
||||
- [ ] Artifacts produced by modified workflow runs
|
||||
|
||||
**Hypothesis Starters**:
|
||||
```
|
||||
[HYPOTHESIS] Workflow file <FILE> was modified in commit <SHA> [EV-XXXX] to
|
||||
exfiltrate repository secrets via <METHOD>, as evidenced by the added network
|
||||
call pattern [EV-YYYY].
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Template 4: Typosquatting / Dependency Confusion
|
||||
|
||||
**Pattern**: Attacker registers a package with a name similar to a popular package
|
||||
(or an internal package name) to intercept installs from users who mistype.
|
||||
|
||||
**Key Evidence to Collect**:
|
||||
- [ ] Registration timestamp of the suspicious package on the registry
|
||||
- [ ] Package content: does it contain malicious code or is it a stub?
|
||||
- [ ] Download statistics for the suspicious package
|
||||
- [ ] Names of internal packages that could be targeted (if private repo scope)
|
||||
- [ ] Any references to the legitimate package in the malicious one's metadata
|
||||
|
||||
**Hypothesis Starters**:
|
||||
```
|
||||
[HYPOTHESIS] Package <MALICIOUS_NAME> was registered on <DATE> [EV-XXXX] to
|
||||
typosquat on <LEGITIMATE_NAME>, targeting users who misspell the package name.
|
||||
The package contains <BEHAVIOR> [EV-YYYY].
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Template 5: Force-Push History Rewrite (Evidence Erasure)
|
||||
|
||||
**Pattern**: After a malicious commit is detected (or before wider notice), the attacker
|
||||
force-pushes to remove the malicious commit from branch history.
|
||||
|
||||
**Detection is key** — this template focuses on proving the erasure happened.
|
||||
|
||||
**Key Evidence to Collect**:
|
||||
- [ ] GH Archive PushEvent with `distinct_size=0` (force push indicator) [EV-XXXX]
|
||||
- [ ] The SHA of the commit BEFORE the force push (from GH Archive `payload.before`)
|
||||
- [ ] Recovery of the erased commit via direct URL or `git fetch origin SHA`
|
||||
- [ ] Wayback Machine snapshot of the commit page before erasure
|
||||
- [ ] Timeline gap in git log (N commits visible in archive but M < N in current repo)
|
||||
|
||||
**Hypothesis Starters**:
|
||||
```
|
||||
[HYPOTHESIS] Actor <HANDLE> force-pushed branch <BRANCH> on <DATE> [EV-XXXX]
|
||||
to erase commit <SHA> [EV-YYYY], which contained <MALICIOUS_CONTENT>.
|
||||
The erased commit was recovered via <METHOD> [EV-ZZZZ].
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Cross-Cutting Investigation Checklist
|
||||
|
||||
Apply to every investigation regardless of template:
|
||||
|
||||
- [ ] Check all contributors for newly created accounts (< 30 days old at time of malicious activity)
|
||||
- [ ] Check if any maintainer account changed email in the period (sign of account takeover)
|
||||
- [ ] Verify GPG signatures on suspicious commits match known maintainer keys
|
||||
- [ ] Check if the repository changed ownership or transferred orgs near the incident
|
||||
- [ ] Look for "cleanup" commits immediately after the malicious commit (cover-up pattern)
|
||||
- [ ] Check related packages/repos by the same author for similar patterns
|
||||
|
|
@ -0,0 +1,164 @@
|
|||
# Deleted Content Recovery Techniques
|
||||
|
||||
## Key Insight: GitHub Never Fully Deletes Force-Pushed Commits
|
||||
|
||||
Force-pushed commits are removed from the branch history but REMAIN on GitHub's servers until garbage collection runs (which can take weeks to months). This is the foundation of deleted commit recovery.
|
||||
|
||||
---
|
||||
|
||||
## Method 1: Direct GitHub URL (Fastest — No Auth Required)
|
||||
|
||||
If you have a commit SHA, access it directly even if it was force-pushed off a branch:
|
||||
|
||||
```bash
|
||||
# View commit metadata
|
||||
curl -s "https://github.com/OWNER/REPO/commit/SHA"
|
||||
|
||||
# Download as patch (includes full diff)
|
||||
curl -s "https://github.com/OWNER/REPO/commit/SHA.patch" > recovered_commit.patch
|
||||
|
||||
# Download as diff
|
||||
curl -s "https://github.com/OWNER/REPO/commit/SHA.diff" > recovered_commit.diff
|
||||
|
||||
# Example (Istio credential leak - real incident):
|
||||
curl -s "https://github.com/istio/istio/commit/FORCE_PUSHED_SHA.patch"
|
||||
```
|
||||
|
||||
**When this works**: SHA is known (from GH Archive, Wayback Machine, or `git fsck`)
|
||||
**When this fails**: GitHub has already garbage-collected the object (rare, typically 30–90 days post-force-push)
|
||||
|
||||
---
|
||||
|
||||
## Method 2: GitHub REST API
|
||||
|
||||
```bash
|
||||
# Works for commits force-pushed off branches but still on server
|
||||
# Note: /commits/SHA may 404, but /git/commits/SHA often succeeds for orphaned commits
|
||||
curl -s "https://api.github.com/repos/OWNER/REPO/git/commits/SHA" | jq .
|
||||
|
||||
# Get the tree (file listing) of a force-pushed commit
|
||||
curl -s "https://api.github.com/repos/OWNER/REPO/git/trees/SHA?recursive=1" | jq .
|
||||
|
||||
# Get a specific file from a force-pushed commit
|
||||
curl -s "https://api.github.com/repos/OWNER/REPO/contents/PATH?ref=SHA" | jq .content | base64 -d
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Method 3: Git Fetch by SHA (Local — Requires Clone)
|
||||
|
||||
```bash
|
||||
# Fetch an orphaned commit directly by SHA into local repo
|
||||
cd target_repo
|
||||
git fetch origin SHA
|
||||
git log FETCH_HEAD -1 # view the commit
|
||||
git diff FETCH_HEAD~1 FETCH_HEAD # view the diff
|
||||
|
||||
# If the SHA was recently force-pushed it will still be fetchable
|
||||
# This stops working once GitHub GC runs
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Method 4: Dangling Commits via git fsck
|
||||
|
||||
```bash
|
||||
cd target_repo
|
||||
|
||||
# Find all unreachable objects (includes force-pushed commits)
|
||||
git fsck --unreachable --no-reflogs 2>&1 | grep "unreachable commit" | awk '{print $3}' > dangling_shas.txt
|
||||
|
||||
# For each dangling commit, get its metadata
|
||||
while read sha; do
|
||||
echo "=== $sha ===" >> dangling_details.txt
|
||||
git show --stat "$sha" >> dangling_details.txt 2>&1
|
||||
done < dangling_shas.txt
|
||||
|
||||
# Note: dangling objects only exist in LOCAL clone — not the same as GitHub's copies
|
||||
# GitHub's copies are accessible via Methods 1-3 until GC runs
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Recovering Deleted GitHub Issues and PRs
|
||||
|
||||
### Via Wayback Machine CDX API
|
||||
|
||||
```bash
|
||||
# Find all archived snapshots of a specific issue
|
||||
curl -s "https://web.archive.org/cdx/search/cdx?url=github.com/OWNER/REPO/issues/NUMBER&output=json&limit=50&fl=timestamp,statuscode,original" | python3 -m json.tool
|
||||
|
||||
# Fetch the best snapshot
|
||||
# Use the timestamp from the CDX result:
|
||||
# https://web.archive.org/web/TIMESTAMP/https://github.com/OWNER/REPO/issues/NUMBER
|
||||
curl -s "https://web.archive.org/web/TIMESTAMP/https://github.com/OWNER/REPO/issues/NUMBER" > issue_NUMBER_archived.html
|
||||
|
||||
# Find all snapshots of the repo in a date range
|
||||
curl -s "https://web.archive.org/cdx/search/cdx?url=github.com/OWNER/REPO*&output=json&from=20240101&to=20240201&limit=200&fl=timestamp,urlkey,statuscode" | python3 -m json.tool
|
||||
```
|
||||
|
||||
### Via GitHub API (Limited — Only Non-Deleted Content)
|
||||
|
||||
```bash
|
||||
# Closed issues (not deleted) are retrievable
|
||||
curl -s "https://api.github.com/repos/OWNER/REPO/issues?state=closed&per_page=100" | jq '.[].number'
|
||||
|
||||
# Note: DELETED issues/PRs do NOT appear in the API. Use Wayback Machine or GH Archive for those.
|
||||
```
|
||||
|
||||
### Via GitHub Archive (For Event History — Not Content)
|
||||
|
||||
```sql
|
||||
-- Find all IssueEvents for a repo in a date range
|
||||
SELECT created_at, actor.login, payload.action, payload.issue.number, payload.issue.title
|
||||
FROM `githubarchive.day.*`
|
||||
WHERE _TABLE_SUFFIX BETWEEN '20240101' AND '20240201'
|
||||
AND type = 'IssuesEvent'
|
||||
AND repo.name = 'OWNER/REPO'
|
||||
ORDER BY created_at
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Recovering Deleted Files from a Known Commit
|
||||
|
||||
```bash
|
||||
# If you have the commit SHA (even force-pushed):
|
||||
git show SHA:path/to/file.py > recovered_file.py
|
||||
|
||||
# Or via API (base64 encoded content):
|
||||
curl -s "https://api.github.com/repos/OWNER/REPO/contents/path/to/file.py?ref=SHA" | python3 -c "
|
||||
import sys, json, base64
|
||||
d = json.load(sys.stdin)
|
||||
print(base64.b64decode(d['content']).decode())
|
||||
"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Evidence Recording
|
||||
|
||||
After recovering any deleted content, immediately record it:
|
||||
|
||||
```bash
|
||||
python3 SKILL_DIR/scripts/evidence-store.py --store evidence.json add \
|
||||
--source "git fetch origin FORCE_PUSHED_SHA" \
|
||||
--content "Recovered commit: FORCE_PUSHED_SHA | Author: attacker@example.com | Date: 2024-01-15 | Added file: malicious.sh" \
|
||||
--type git \
|
||||
--actor "attacker-handle" \
|
||||
--url "https://github.com/OWNER/REPO/commit/FORCE_PUSHED_SHA.patch" \
|
||||
--timestamp "2024-01-15T00:00:00Z" \
|
||||
--verification single_source \
|
||||
--notes "Commit force-pushed off main branch on 2024-01-16. Recovered via direct fetch."
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Recovery Failure Modes
|
||||
|
||||
| Failure | Cause | Workaround |
|
||||
|---------|-------|------------|
|
||||
| `git fetch origin SHA` returns "not our ref" | GitHub GC already ran | Try Method 1/2, search Wayback Machine |
|
||||
| `github.com/OWNER/REPO/commit/SHA` returns 404 | GC ran or SHA is wrong | Verify SHA via GH Archive; try partial SHA search |
|
||||
| Wayback Machine has no snapshots | Page was never crawled by IA | Check `commoncrawl.org`, check Google Cache |
|
||||
| BigQuery shows event but no content | GH Archive stores event metadata, not file contents | Recovery only reveals the event occurred, not the content |
|
||||
|
|
@ -0,0 +1,313 @@
|
|||
#!/usr/bin/env python3
|
||||
"""
|
||||
OSS Forensics Evidence Store Manager
|
||||
Manages a JSON-based evidence store for forensic investigations.
|
||||
|
||||
Commands:
|
||||
add - Add a piece of evidence
|
||||
list - List all evidence (optionally filter by type or actor)
|
||||
verify - Re-check SHA-256 hashes for integrity
|
||||
query - Search evidence by keyword
|
||||
export - Export evidence as a Markdown table
|
||||
summary - Print investigation statistics
|
||||
|
||||
Usage example:
|
||||
python3 evidence-store.py --store evidence.json add \
|
||||
--source "git fsck output" --content "dangling commit abc123" \
|
||||
--type git --actor "malicious-user" --url "https://github.com/owner/repo/commit/abc123"
|
||||
|
||||
python3 evidence-store.py --store evidence.json list --type git
|
||||
python3 evidence-store.py --store evidence.json verify
|
||||
python3 evidence-store.py --store evidence.json export > evidence-table.md
|
||||
"""
|
||||
|
||||
import json
|
||||
import argparse
|
||||
import os
|
||||
import datetime
|
||||
import hashlib
|
||||
import sys
|
||||
|
||||
EVIDENCE_TYPES = [
|
||||
"git", # Local git repository data (commits, reflog, fsck)
|
||||
"gh_api", # GitHub REST API responses
|
||||
"gh_archive", # GitHub Archive / BigQuery query results
|
||||
"web_archive", # Wayback Machine snapshots
|
||||
"ioc", # Indicator of Compromise (SHA, domain, IP, package name, etc.)
|
||||
"analysis", # Derived analysis / cross-source correlation result
|
||||
"manual", # Manually noted observation
|
||||
"vendor_report", # External security vendor report excerpt
|
||||
]
|
||||
|
||||
VERIFICATION_STATES = ["unverified", "single_source", "multi_source_verified"]
|
||||
|
||||
IOC_TYPES = [
|
||||
"COMMIT_SHA", "FILE_PATH", "API_KEY", "SECRET", "IP_ADDRESS",
|
||||
"DOMAIN", "PACKAGE_NAME", "ACTOR_USERNAME", "MALICIOUS_URL",
|
||||
"WORKFLOW_FILE", "BRANCH_NAME", "TAG_NAME", "RELEASE_NAME", "OTHER",
|
||||
]
|
||||
|
||||
|
||||
def _now_iso():
|
||||
return datetime.datetime.now(datetime.timezone.utc).isoformat(timespec="seconds") + "Z"
|
||||
|
||||
|
||||
def _sha256(content: str) -> str:
|
||||
return hashlib.sha256(content.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
class EvidenceStore:
|
||||
def __init__(self, filepath: str):
|
||||
self.filepath = filepath
|
||||
self.data = {
|
||||
"metadata": {
|
||||
"version": "2.0",
|
||||
"created_at": _now_iso(),
|
||||
"last_updated": _now_iso(),
|
||||
"investigation": "",
|
||||
"target_repo": "",
|
||||
},
|
||||
"evidence": [],
|
||||
"chain_of_custody": [],
|
||||
}
|
||||
if os.path.exists(filepath):
|
||||
try:
|
||||
with open(filepath, "r", encoding="utf-8") as f:
|
||||
self.data = json.load(f)
|
||||
except (json.JSONDecodeError, IOError) as e:
|
||||
print(f"Error loading evidence store '{filepath}': {e}", file=sys.stderr)
|
||||
print("Hint: The file might be corrupted. Check for manual edits or syntax errors.", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
def _save(self):
|
||||
self.data["metadata"]["last_updated"] = _now_iso()
|
||||
with open(self.filepath, "w", encoding="utf-8") as f:
|
||||
json.dump(self.data, f, indent=2, ensure_ascii=False)
|
||||
|
||||
def _next_id(self) -> str:
|
||||
return f"EV-{len(self.data['evidence']) + 1:04d}"
|
||||
|
||||
def add(
|
||||
self,
|
||||
source: str,
|
||||
content: str,
|
||||
evidence_type: str,
|
||||
actor: str = None,
|
||||
url: str = None,
|
||||
timestamp: str = None,
|
||||
ioc_type: str = None,
|
||||
verification: str = "unverified",
|
||||
notes: str = None,
|
||||
) -> str:
|
||||
evidence_id = self._next_id()
|
||||
entry = {
|
||||
"id": evidence_id,
|
||||
"type": evidence_type,
|
||||
"source": source,
|
||||
"content": content,
|
||||
"content_sha256": _sha256(content),
|
||||
"actor": actor,
|
||||
"url": url,
|
||||
"event_timestamp": timestamp,
|
||||
"collected_at": _now_iso(),
|
||||
"ioc_type": ioc_type,
|
||||
"verification": verification,
|
||||
"notes": notes,
|
||||
}
|
||||
self.data["evidence"].append(entry)
|
||||
self.data["chain_of_custody"].append({
|
||||
"action": "add",
|
||||
"evidence_id": evidence_id,
|
||||
"timestamp": _now_iso(),
|
||||
"source": source,
|
||||
})
|
||||
self._save()
|
||||
return evidence_id
|
||||
|
||||
def list_evidence(self, filter_type: str = None, filter_actor: str = None):
|
||||
results = self.data["evidence"]
|
||||
if filter_type:
|
||||
results = [e for e in results if e.get("type") == filter_type]
|
||||
if filter_actor:
|
||||
results = [e for e in results if e.get("actor") == filter_actor]
|
||||
return results
|
||||
|
||||
def verify_integrity(self):
|
||||
"""Re-compute SHA-256 for all entries and report mismatches."""
|
||||
issues = []
|
||||
for entry in self.data["evidence"]:
|
||||
expected = _sha256(entry["content"])
|
||||
stored = entry.get("content_sha256", "")
|
||||
if expected != stored:
|
||||
issues.append({
|
||||
"id": entry["id"],
|
||||
"stored_sha256": stored,
|
||||
"computed_sha256": expected,
|
||||
})
|
||||
return issues
|
||||
|
||||
def query(self, keyword: str):
|
||||
"""Search for keyword in content, source, actor, or url."""
|
||||
keyword_lower = keyword.lower()
|
||||
return [
|
||||
e for e in self.data["evidence"]
|
||||
if keyword_lower in (e.get("content", "") or "").lower()
|
||||
or keyword_lower in (e.get("source", "") or "").lower()
|
||||
or keyword_lower in (e.get("actor", "") or "").lower()
|
||||
or keyword_lower in (e.get("url", "") or "").lower()
|
||||
]
|
||||
|
||||
def export_markdown(self) -> str:
|
||||
lines = [
|
||||
"# Evidence Registry",
|
||||
"",
|
||||
f"**Store**: `{self.filepath}`",
|
||||
f"**Last Updated**: {self.data['metadata'].get('last_updated', 'N/A')}",
|
||||
f"**Total Evidence Items**: {len(self.data['evidence'])}",
|
||||
"",
|
||||
"| ID | Type | Source | Actor | Verification | Event Timestamp | URL |",
|
||||
"|----|------|--------|-------|--------------|-----------------|-----|",
|
||||
]
|
||||
for e in self.data["evidence"]:
|
||||
url = e.get("url") or ""
|
||||
url_display = f"[link]({url})" if url else ""
|
||||
lines.append(
|
||||
f"| {e['id']} | {e.get('type','')} | {e.get('source','')} "
|
||||
f"| {e.get('actor') or ''} | {e.get('verification','')} "
|
||||
f"| {e.get('event_timestamp') or ''} | {url_display} |"
|
||||
)
|
||||
lines.append("")
|
||||
lines.append("## Chain of Custody")
|
||||
lines.append("")
|
||||
lines.append("| Evidence ID | Action | Timestamp | Source |")
|
||||
lines.append("|-------------|--------|-----------|--------|")
|
||||
for c in self.data["chain_of_custody"]:
|
||||
lines.append(
|
||||
f"| {c.get('evidence_id','')} | {c.get('action','')} "
|
||||
f"| {c.get('timestamp','')} | {c.get('source','')} |"
|
||||
)
|
||||
return "\n".join(lines)
|
||||
|
||||
def summary(self) -> dict:
|
||||
by_type = {}
|
||||
by_verification = {}
|
||||
actors = set()
|
||||
for e in self.data["evidence"]:
|
||||
t = e.get("type", "unknown")
|
||||
by_type[t] = by_type.get(t, 0) + 1
|
||||
v = e.get("verification", "unverified")
|
||||
by_verification[v] = by_verification.get(v, 0) + 1
|
||||
if e.get("actor"):
|
||||
actors.add(e["actor"])
|
||||
return {
|
||||
"total": len(self.data["evidence"]),
|
||||
"by_type": by_type,
|
||||
"by_verification": by_verification,
|
||||
"unique_actors": sorted(actors),
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="OSS Forensics Evidence Store Manager v2.0",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
)
|
||||
parser.add_argument("--store", default="evidence.json", help="Path to evidence JSON file (default: evidence.json)")
|
||||
|
||||
subparsers = parser.add_subparsers(dest="command", metavar="COMMAND")
|
||||
|
||||
# --- add ---
|
||||
add_p = subparsers.add_parser("add", help="Add a new evidence entry")
|
||||
add_p.add_argument("--source", required=True, help="Where this evidence came from (e.g. 'git fsck', 'GH API /commits')")
|
||||
add_p.add_argument("--content", required=True, help="The evidence content (commit SHA, API response excerpt, etc.)")
|
||||
add_p.add_argument("--type", required=True, choices=EVIDENCE_TYPES, dest="evidence_type", help="Evidence type")
|
||||
add_p.add_argument("--actor", help="GitHub handle or email of associated actor")
|
||||
add_p.add_argument("--url", help="URL to original source")
|
||||
add_p.add_argument("--timestamp", help="When the event occurred (ISO 8601)")
|
||||
add_p.add_argument("--ioc-type", choices=IOC_TYPES, help="IOC subtype (for --type ioc)")
|
||||
add_p.add_argument("--verification", choices=VERIFICATION_STATES, default="unverified")
|
||||
add_p.add_argument("--notes", help="Additional investigator notes")
|
||||
add_p.add_argument("--quiet", action="store_true", help="Suppress success message")
|
||||
|
||||
# --- list ---
|
||||
list_p = subparsers.add_parser("list", help="List all evidence entries")
|
||||
list_p.add_argument("--type", dest="filter_type", choices=EVIDENCE_TYPES, help="Filter by type")
|
||||
list_p.add_argument("--actor", dest="filter_actor", help="Filter by actor")
|
||||
|
||||
# --- verify ---
|
||||
subparsers.add_parser("verify", help="Verify SHA-256 integrity of all evidence content")
|
||||
|
||||
# --- query ---
|
||||
query_p = subparsers.add_parser("query", help="Search evidence by keyword")
|
||||
query_p.add_argument("keyword", help="Keyword to search for")
|
||||
|
||||
# --- export ---
|
||||
subparsers.add_parser("export", help="Export evidence as a Markdown table (stdout)")
|
||||
|
||||
# --- summary ---
|
||||
subparsers.add_parser("summary", help="Print investigation statistics")
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if not args.command:
|
||||
parser.print_help()
|
||||
sys.exit(0)
|
||||
|
||||
store = EvidenceStore(args.store)
|
||||
|
||||
if args.command == "add":
|
||||
eid = store.add(
|
||||
source=args.source,
|
||||
content=args.content,
|
||||
evidence_type=args.evidence_type,
|
||||
actor=args.actor,
|
||||
url=args.url,
|
||||
timestamp=args.timestamp,
|
||||
ioc_type=args.ioc_type,
|
||||
verification=args.verification,
|
||||
notes=args.notes,
|
||||
)
|
||||
if not getattr(args, "quiet", False):
|
||||
print(f"✓ Added evidence: {eid}")
|
||||
|
||||
elif args.command == "list":
|
||||
items = store.list_evidence(
|
||||
filter_type=getattr(args, "filter_type", None),
|
||||
filter_actor=getattr(args, "filter_actor", None),
|
||||
)
|
||||
if not items:
|
||||
print("No evidence found.")
|
||||
for e in items:
|
||||
actor_str = f" | actor: {e['actor']}" if e.get("actor") else ""
|
||||
url_str = f" | {e['url']}" if e.get("url") else ""
|
||||
print(f"[{e['id']}] {e['type']:12s} | {e['verification']:20s} | {e['source']}{actor_str}{url_str}")
|
||||
|
||||
elif args.command == "verify":
|
||||
issues = store.verify_integrity()
|
||||
if not issues:
|
||||
print(f"✓ All {len(store.data['evidence'])} evidence entries passed SHA-256 integrity check.")
|
||||
else:
|
||||
print(f"✗ {len(issues)} integrity issue(s) detected:")
|
||||
for i in issues:
|
||||
print(f" [{i['id']}] stored={i['stored_sha256'][:16]}... computed={i['computed_sha256'][:16]}...")
|
||||
sys.exit(1)
|
||||
|
||||
elif args.command == "query":
|
||||
results = store.query(args.keyword)
|
||||
print(f"Found {len(results)} result(s) for '{args.keyword}':")
|
||||
for e in results:
|
||||
print(f" [{e['id']}] {e['type']} | {e['source']} | {e['content'][:80]}")
|
||||
|
||||
elif args.command == "export":
|
||||
print(store.export_markdown())
|
||||
|
||||
elif args.command == "summary":
|
||||
s = store.summary()
|
||||
print(f"Total evidence items : {s['total']}")
|
||||
print(f"By type : {json.dumps(s['by_type'], indent=2)}")
|
||||
print(f"By verification : {json.dumps(s['by_verification'], indent=2)}")
|
||||
print(f"Unique actors : {s['unique_actors']}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
|
@ -0,0 +1,151 @@
|
|||
# Forensic Investigation Report
|
||||
|
||||
> **Instructions**: Fill in all sections. Every factual claim must cite at least one `[EV-XXXX]` evidence ID.
|
||||
> Remove placeholder text and instruction notes before finalizing. Redact all secrets to `[REDACTED]`.
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
**Target Repository**: `OWNER/REPO`
|
||||
**Investigation Period**: YYYY-MM-DD to YYYY-MM-DD
|
||||
**Verdict**: <!-- Compromised / Clean / Inconclusive -->
|
||||
**Confidence Level**: <!-- High / Medium / Low -->
|
||||
**Report Date**: YYYY-MM-DD
|
||||
**Investigator**: <!-- Agent session ID or analyst name -->
|
||||
|
||||
<!-- One paragraph: what was investigated, what was found, what is recommended. -->
|
||||
|
||||
---
|
||||
|
||||
## Timeline of Events
|
||||
|
||||
> All timestamps in UTC. Each event must cite at least one evidence ID.
|
||||
|
||||
| Timestamp (UTC) | Event | Evidence IDs | Source |
|
||||
|-----------------|-------|--------------|--------|
|
||||
| YYYY-MM-DDTHH:MM:SSZ | _Describe event_ | [EV-XXXX] | git / gh_api / gh_archive / web_archive |
|
||||
| | | | |
|
||||
|
||||
---
|
||||
|
||||
## Validated Hypotheses
|
||||
|
||||
### Hypothesis 1: <!-- Short title -->
|
||||
|
||||
**Status**: <!-- VALIDATED / INCONCLUSIVE / REJECTED -->
|
||||
|
||||
**Claim**: _Full statement of the hypothesis._
|
||||
|
||||
**Supporting Evidence**:
|
||||
- [EV-XXXX]: _What this evidence shows_
|
||||
- [EV-YYYY]: _What this evidence shows_
|
||||
|
||||
**Counter-Evidence Considered**: _What might disprove this, and why it was ruled out or not._
|
||||
|
||||
**Confidence**: <!-- High / Medium / Low, and why -->
|
||||
|
||||
---
|
||||
|
||||
## Indicators of Compromise (IOC List)
|
||||
|
||||
| Type | Value | Status | Evidence |
|
||||
|------|-------|--------|----------|
|
||||
| COMMIT_SHA | `abc123...` | Confirmed malicious | [EV-XXXX] |
|
||||
| ACTOR_USERNAME | `handle` | Suspected compromised | [EV-YYYY] |
|
||||
| FILE_PATH | `src/evil.js` | Confirmed malicious | [EV-ZZZZ] |
|
||||
| DOMAIN | `evil-cdn.io` | Confirmed C2 | [EV-WWWW] |
|
||||
|
||||
---
|
||||
|
||||
## Affected Versions
|
||||
|
||||
| Version / Tag | Published | Contains Malicious Code | Evidence |
|
||||
|---------------|-----------|------------------------|----------|
|
||||
| `v1.2.3` | YYYY-MM-DD | Yes / No / Unknown | [EV-XXXX] |
|
||||
|
||||
---
|
||||
|
||||
## Evidence Registry
|
||||
|
||||
> Generated by: `python3 SKILL_DIR/scripts/evidence-store.py --store evidence.json export`
|
||||
|
||||
<!-- Paste the Markdown table output from the evidence-store.py export command here -->
|
||||
|
||||
| ID | Type | Source | Actor | Verification | Event Timestamp | URL |
|
||||
|----|------|--------|-------|--------------|-----------------|-----|
|
||||
| EV-0001 | | | | | | |
|
||||
|
||||
---
|
||||
|
||||
## Chain of Custody
|
||||
|
||||
> Generated by: `python3 SKILL_DIR/scripts/evidence-store.py --store evidence.json export`
|
||||
|
||||
<!-- Paste the chain of custody section from the export output here -->
|
||||
|
||||
| Evidence ID | Action | Timestamp | Source |
|
||||
|-------------|--------|-----------|--------|
|
||||
| EV-0001 | add | | |
|
||||
|
||||
---
|
||||
|
||||
## Technical Findings
|
||||
|
||||
### Git History Analysis
|
||||
|
||||
_Summarize findings from local git analysis: dangling commits, reflog anomalies, unsigned commits, binary additions, etc._
|
||||
|
||||
### GitHub API Analysis
|
||||
|
||||
_Summarize findings from GitHub REST API: deleted PRs/issues, contributor changes, release anomalies, etc._
|
||||
|
||||
### GitHub Archive Analysis
|
||||
|
||||
_Summarize findings from BigQuery: force-push events, delete events, workflow anomalies, member changes, etc._
|
||||
_Note: If BigQuery was unavailable, state this explicitly._
|
||||
|
||||
### Wayback Machine Analysis
|
||||
|
||||
_Summarize findings from archive.org: recovered deleted pages, historical content differences, etc._
|
||||
|
||||
### IOC Enrichment
|
||||
|
||||
_Summarize enrichment results: WHOIS data for domains, recovered commit content, actor account analysis, etc._
|
||||
|
||||
---
|
||||
|
||||
## Recommendations
|
||||
|
||||
### Immediate Actions (If Compromise Confirmed)
|
||||
|
||||
- [ ] Rotate all GitHub tokens, API keys, and credentials that may have been exposed
|
||||
- [ ] Pin dependency versions to hashes in all affected packages
|
||||
- [ ] Publish a security advisory / CVE if applicable
|
||||
- [ ] Notify downstream users/package registries (npm, PyPI, etc.)
|
||||
- [ ] Revoke access for the compromised account and re-secure with hardware 2FA
|
||||
- [ ] Audit all CI/CD workflow files for unauthorized modifications
|
||||
- [ ] Review all releases published during the compromise window
|
||||
|
||||
### Monitoring Recommendations
|
||||
|
||||
- [ ] Enable branch protection on `main`/`master` (require code review, disallow force-push)
|
||||
- [ ] Enable required commit signing (GPG/SSH)
|
||||
- [ ] Set up GitHub audit log streaming for future monitoring
|
||||
- [ ] Pin critical dependencies to known-good SHAs in lock files
|
||||
|
||||
---
|
||||
|
||||
## Limitations and Caveats
|
||||
|
||||
- _List any data sources that were unavailable (e.g., no BigQuery access)_
|
||||
- _Note any evidence that is single-source only (not independently verified)_
|
||||
- _Note any hypotheses that could not be confirmed or denied_
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- Evidence store: `evidence.json` (SHA-256 integrity: run `python3 SKILL_DIR/scripts/evidence-store.py --store evidence.json verify`)
|
||||
- Related issues: <!-- Link to GitHub issues, CVEs, security advisories -->
|
||||
- RAPTOR framework: https://github.com/gadievron/raptor
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
# Malicious Package Investigation Report
|
||||
|
||||
---
|
||||
|
||||
## 📦 Package Metadata
|
||||
- **Package Name**:
|
||||
- **Registry**: [NPM / PyPI / RubyGems / etc.]
|
||||
- **Affected Versions**:
|
||||
- **Malicious Version(s)**:
|
||||
- **Downloads at Time of Detection**:
|
||||
- **Package URL**:
|
||||
|
||||
---
|
||||
|
||||
## 🚩 Indicators of Compromise (IOCs)
|
||||
- **Malicious URL(s)**:
|
||||
- **Exfiltrated Data Types**: [Environment variables, ~/.ssh/id_rsa, /etc/shadow, etc.]
|
||||
- **Exfiltration Method**: [DNS tunneling, HTTP POST to C2, etc.]
|
||||
- **C2 IP/Domain**:
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ Analysis Summary
|
||||
- **Primary Mechanism**: [Typosquatting / Dependency Confusion / Maintainer Takeover]
|
||||
- **Behavior Description**:
|
||||
- [Example: Installs a postinstall script that exfiltrates environment variables.]
|
||||
- [Example: Patches `setup.py` to download a secondary payload.]
|
||||
|
||||
---
|
||||
|
||||
## 🔍 Evidence Registry
|
||||
| Evidence ID | Type | Source | Description |
|
||||
|-------------|------|--------|-------------|
|
||||
| EV-XXXX | ioc | NPM | Package install script snapshot |
|
||||
| EV-YYYY | web | Wayback| Historical version comparison |
|
||||
|
||||
---
|
||||
|
||||
## 🛡️ Recommended Mitigations
|
||||
1. [ ] Unpublish/Report the package to the registry.
|
||||
2. [ ] Audit `package-lock.json` or `requirements.txt` across all projects.
|
||||
3. [ ] Rotate secrets exfiltrated via environment variables.
|
||||
4. [ ] Pin specific hashes (SHASUM) for mission-critical dependencies.
|
||||
192
hermes_code/optional-skills/security/sherlock/SKILL.md
Normal file
192
hermes_code/optional-skills/security/sherlock/SKILL.md
Normal file
|
|
@ -0,0 +1,192 @@
|
|||
---
|
||||
name: sherlock
|
||||
description: OSINT username search across 400+ social networks. Hunt down social media accounts by username.
|
||||
version: 1.0.0
|
||||
author: unmodeled-tyler
|
||||
license: MIT
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [osint, security, username, social-media, reconnaissance]
|
||||
category: security
|
||||
prerequisites:
|
||||
commands: [sherlock]
|
||||
---
|
||||
|
||||
# Sherlock OSINT Username Search
|
||||
|
||||
Hunt down social media accounts by username across 400+ social networks using the [Sherlock Project](https://github.com/sherlock-project/sherlock).
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks to find accounts associated with a username
|
||||
- User wants to check username availability across platforms
|
||||
- User is conducting OSINT or reconnaissance research
|
||||
- User asks "where is this username registered?" or similar
|
||||
|
||||
## Requirements
|
||||
|
||||
- Sherlock CLI installed: `pipx install sherlock-project` or `pip install sherlock-project`
|
||||
- Alternatively: Docker available (`docker run -it --rm sherlock/sherlock`)
|
||||
- Network access to query social platforms
|
||||
|
||||
## Procedure
|
||||
|
||||
### 1. Check if Sherlock is Installed
|
||||
|
||||
**Before doing anything else**, verify sherlock is available:
|
||||
|
||||
```bash
|
||||
sherlock --version
|
||||
```
|
||||
|
||||
If the command fails:
|
||||
- Offer to install: `pipx install sherlock-project` (recommended) or `pip install sherlock-project`
|
||||
- **Do NOT** try multiple installation methods — pick one and proceed
|
||||
- If installation fails, inform the user and stop
|
||||
|
||||
### 2. Extract Username
|
||||
|
||||
**Extract the username directly from the user's message if clearly stated.**
|
||||
|
||||
Examples where you should **NOT** use clarify:
|
||||
- "Find accounts for nasa" → username is `nasa`
|
||||
- "Search for johndoe123" → username is `johndoe123`
|
||||
- "Check if alice exists on social media" → username is `alice`
|
||||
- "Look up user bob on social networks" → username is `bob`
|
||||
|
||||
**Only use clarify if:**
|
||||
- Multiple potential usernames mentioned ("search for alice or bob")
|
||||
- Ambiguous phrasing ("search for my username" without specifying)
|
||||
- No username mentioned at all ("do an OSINT search")
|
||||
|
||||
When extracting, take the **exact** username as stated — preserve case, numbers, underscores, etc.
|
||||
|
||||
### 3. Build Command
|
||||
|
||||
**Default command** (use this unless user specifically requests otherwise):
|
||||
```bash
|
||||
sherlock --print-found --no-color "<username>" --timeout 90
|
||||
```
|
||||
|
||||
**Optional flags** (only add if user explicitly requests):
|
||||
- `--nsfw` — Include NSFW sites (only if user asks)
|
||||
- `--tor` — Route through Tor (only if user asks for anonymity)
|
||||
|
||||
**Do NOT ask about options via clarify** — just run the default search. Users can request specific options if needed.
|
||||
|
||||
### 4. Execute Search
|
||||
|
||||
Run via the `terminal` tool. The command typically takes 30-120 seconds depending on network conditions and site count.
|
||||
|
||||
**Example terminal call:**
|
||||
```json
|
||||
{
|
||||
"command": "sherlock --print-found --no-color \"target_username\"",
|
||||
"timeout": 180
|
||||
}
|
||||
```
|
||||
|
||||
### 5. Parse and Present Results
|
||||
|
||||
Sherlock outputs found accounts in a simple format. Parse the output and present:
|
||||
|
||||
1. **Summary line:** "Found X accounts for username 'Y'"
|
||||
2. **Categorized links:** Group by platform type if helpful (social, professional, forums, etc.)
|
||||
3. **Output file location:** Sherlock saves results to `<username>.txt` by default
|
||||
|
||||
**Example output parsing:**
|
||||
```
|
||||
[+] Instagram: https://instagram.com/username
|
||||
[+] Twitter: https://twitter.com/username
|
||||
[+] GitHub: https://github.com/username
|
||||
```
|
||||
|
||||
Present findings as clickable links when possible.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
### No Results Found
|
||||
If Sherlock finds no accounts, this is often correct — the username may not be registered on checked platforms. Suggest:
|
||||
- Checking spelling/variation
|
||||
- Trying similar usernames with `?` wildcard: `sherlock "user?name"`
|
||||
- The user may have privacy settings or deleted accounts
|
||||
|
||||
### Timeout Issues
|
||||
Some sites are slow or block automated requests. Use `--timeout 120` to increase wait time, or `--site` to limit scope.
|
||||
|
||||
### Tor Configuration
|
||||
`--tor` requires Tor daemon running. If user wants anonymity but Tor isn't available, suggest:
|
||||
- Installing Tor service
|
||||
- Using `--proxy` with an alternative proxy
|
||||
|
||||
### False Positives
|
||||
Some sites always return "found" due to their response structure. Cross-reference unexpected results with manual checks.
|
||||
|
||||
### Rate Limiting
|
||||
Aggressive searches may trigger rate limits. For bulk username searches, add delays between calls or use `--local` with cached data.
|
||||
|
||||
## Installation
|
||||
|
||||
### pipx (recommended)
|
||||
```bash
|
||||
pipx install sherlock-project
|
||||
```
|
||||
|
||||
### pip
|
||||
```bash
|
||||
pip install sherlock-project
|
||||
```
|
||||
|
||||
### Docker
|
||||
```bash
|
||||
docker pull sherlock/sherlock
|
||||
docker run -it --rm sherlock/sherlock <username>
|
||||
```
|
||||
|
||||
### Linux packages
|
||||
Available on Debian 13+, Ubuntu 22.10+, Homebrew, Kali, BlackArch.
|
||||
|
||||
## Ethical Use
|
||||
|
||||
This tool is for legitimate OSINT and research purposes only. Remind users:
|
||||
- Only search usernames they own or have permission to investigate
|
||||
- Respect platform terms of service
|
||||
- Do not use for harassment, stalking, or illegal activities
|
||||
- Consider privacy implications before sharing results
|
||||
|
||||
## Verification
|
||||
|
||||
After running sherlock, verify:
|
||||
1. Output lists found sites with URLs
|
||||
2. `<username>.txt` file created (default output) if using file output
|
||||
3. If `--print-found` used, output should only contain `[+]` lines for matches
|
||||
|
||||
## Example Interaction
|
||||
|
||||
**User:** "Can you check if the username 'johndoe123' exists on social media?"
|
||||
|
||||
**Agent procedure:**
|
||||
1. Check `sherlock --version` (verify installed)
|
||||
2. Username provided — proceed directly
|
||||
3. Run: `sherlock --print-found --no-color "johndoe123" --timeout 90`
|
||||
4. Parse output and present links
|
||||
|
||||
**Response format:**
|
||||
> Found 12 accounts for username 'johndoe123':
|
||||
>
|
||||
> • https://twitter.com/johndoe123
|
||||
> • https://github.com/johndoe123
|
||||
> • https://instagram.com/johndoe123
|
||||
> • [... additional links]
|
||||
>
|
||||
> Results saved to: johndoe123.txt
|
||||
|
||||
---
|
||||
|
||||
**User:** "Search for username 'alice' including NSFW sites"
|
||||
|
||||
**Agent procedure:**
|
||||
1. Check sherlock installed
|
||||
2. Username + NSFW flag both provided
|
||||
3. Run: `sherlock --print-found --no-color --nsfw "alice" --timeout 90`
|
||||
4. Present results
|
||||
Loading…
Add table
Add a link
Reference in a new issue