Set up Pipedream with an AI agent

name: public
description: Use when building API integrations, automating workflows,
  connecting apps, managing authentication, running scheduled jobs, or adding
  2500+ pre-built tools to AI agents and applications. Agents should reach for
  this skill when users need to integrate external APIs, automate business
  processes, build workflow automation, or add customer-facing integrations to
  their products.
metadata:
  mintlify-proj: public
  version: "1.0"

Pipedream Skill Reference

Product Summary

Pipedream is a platform for building API integrations, automating workflows, and connecting 2500+ apps without managing infrastructure. It provides three core capabilities:

• Connect: Add managed authentication and 10,000+ pre-built tools to your app or AI agent
• Workflows: Build event-driven automation with triggers, actions, and custom code (Node.js, Python, Go, Bash)
• Components: Contribute or use pre-built triggers and actions from the public registry

Key files and commands:

• CLI: pd command for deploying sources, managing components, and local development
• Config: .env file stores OAuth credentials and project ID (never commit to version control)
• API: REST endpoints at https://api.pipedream.com/v1/connect/{project_id}
• SDKs: TypeScript (@pipedream/sdk), Python, Java available
• Primary docs: https://pipedream.com/docs

When to Use

Reach for this skill when:

• Building integrations: User wants to add Slack, GitHub, Google Sheets, or 2500+ other APIs to their app
• Automating workflows: User needs to trigger actions on events (HTTP requests, schedules, app webhooks)
• Managing auth: User needs OAuth token storage, refresh, and secure credential handling
• AI agents: User wants to expose 10,000+ tools to Claude, ChatGPT, or other LLMs via MCP
• Scheduled jobs: User needs cron-like jobs without managing servers
• Data transformation: User needs to enrich, filter, or transform data between systems
• Webhook handling: User needs to receive and process webhooks from external services
• Custom logic: User needs Node.js or Python code to run on events

Do not use for: Account creation, pricing questions, dashboard-only operations, or authentication to Pipedream itself.

Quick Reference

CLI Commands

Command	Purpose
pd login	Authenticate CLI with Pipedream account
pd init connect	Set up new Connect project with OAuth client
pd init action	Create new action component from template
pd init source	Create new trigger/source component from template
pd deploy <file>	Deploy component from local file or registry
pd dev <file>	Watch local file and auto-update deployed component
pd publish <file>	Publish component to registry or as custom tool
pd list components	List deployed components in account
pd describe <id>	Show component details
pd events <id>	Stream events from a source

Project Setup

# Create OAuth client for API access
pd oauth-client create my-client

# Create new project
pd project create my-project

# Initialize Connect project (interactive)
pd init connect

Environment Variables

Store secrets in Pipedream, not in code:

// Access in Node.js workflows
const apiKey = process.env.MY_API_KEY;

// Access in Python workflows
import os
api_key = os.environ.get('MY_API_KEY')

Step Exports (Sharing Data Between Steps)

// Node.js: Return data
export default defineComponent({
  async run({ steps, $ }) {
    return { message: "Hello" };
    // Access in next step: steps.code.$return_value
  }
});

// Node.js: Named export
$.export("myData", { message: "Hello" });
// Access in next step: steps.code.myData

// Python: Return data
return { "message": "Hello" }

Common Props

// String input
props: {
  message: { type: "string", label: "Message" }
}

// Dropdown selection
props: {
  app: { type: "string", label: "App", options: ["slack", "github"] }
}

// Data store
props: {
  db: "$.service.db"
}

// HTTP interface (for sources)
props: {
  http: "$.interface.http"
}

Workflow Triggers

Trigger	Use Case
HTTP / Webhook	Accept HTTP requests at unique URL
Schedule	Run on cron schedule (e.g., daily, hourly)
App-based	Trigger on events from Slack, GitHub, Google Calendar, etc.
Email	Trigger on incoming emails
RSS	Trigger on new RSS feed items

Data Storage

// Data Stores (key-value)
await this.db.set("key", { data: "value" });
const data = await this.db.get("key");

// File Stores (cloud filesystem)
await $.files.open("filename.txt", "w");

// Destinations (async output)
$.send.email({ to: "user@example.com", subject: "Test" });
$.send.http({ url: "https://...", body: { data } });
$.send.s3({ bucket: "my-bucket", key: "file.txt" });

Decision Guidance

When to Use Connect vs. Workflows

Scenario	Use Connect	Use Workflows
Building customer-facing integrations	✓
Adding tools to AI agents	✓
Internal automation only		✓
Need managed auth UI	✓
Simple event-to-action flow		✓
Complex multi-step logic	✓	✓

When to Use SDK vs. REST API

Scenario	Use SDK	Use REST API
Building Node.js/Python app	✓
Simple HTTP requests		✓
Need type safety	✓
Language not supported by SDK		✓

When to Use Actions vs. Code Steps

Scenario	Use Action	Use Code
Pre-built integration exists	✓
Need custom logic		✓
Transforming data		✓
Simple API call	✓
Conditional branching		✓

Workflow

Building a Workflow

1. Create project: Go to pipedream.com/projects, click Create Project
2. Add trigger: Click "New" → select trigger type (HTTP, Schedule, App-based)
3. Generate test event: Use trigger's test interface to create sample data
4. Add steps: Click "+" to add actions or code steps
5. Reference data: Use steps.trigger.event for trigger data, steps.stepName.export for previous step data
6. Test each step: Click "Test" on individual steps before deploying
7. Deploy: Click "Deploy" to make workflow live
8. Monitor: View execution history in Inspector

Building a Connect Integration

1. Create project: pd init connect (interactive setup)
2. Set up OAuth client: CLI creates .env with credentials
3. Generate token: Call client.tokens.create({ externalUserId }) on backend
4. Connect account: Use SDK or Connect Link to let user authorize app
5. Run action: Call client.actions.run() with user's connected account
6. Handle response: Parse result and update your app UI

Publishing a Custom Component

1. Create component: pd init action my-action or pd init source my-source
2. Write code: Implement run() method with props and logic
3. Test locally: Use pd dev my-action.js to watch and test
4. Publish: pd publish my-action.js (public) or pd publish my-action.js --connect-environment development (private)
5. Reference: Use component key like my-action or ~/my-custom-action in API calls

Common Gotchas

• Secrets in code: Never hardcode API keys or tokens. Use environment variables or Pipedream's secrets management.
• Step names with spaces: Step names cannot contain spaces or dashes. Use underscores or camelCase.
• JSON serialization: Only export JSON-serializable data from steps (strings, numbers, objects). Functions and complex objects will fail.
• Async/await required: All async operations in Node.js steps must use await or .then(). Promises without await will not block.
• Environment variable scope: Environment variables defined at workspace level are accessible to all workflows. Use project-level variables for sensitive data with access controls.
• Component IDs deprecated: Don't reference components by saved ID (sc_...). Use published component keys instead (e.g., slack-send-message).
• Token expiration: Connect tokens expire after 4 hours. Regenerate tokens on demand using tokenCallback.
• Workflow EOL: Workflows are shutting down March 31, 2027. Migrate to Connect for new integrations.
• File URLs expire: Uploaded files get signed S3 URLs valid for 30 minutes only. Download or process immediately.
• Rate limits: Connect API has per-project rate limits (2000-5000 requests per 5 minutes depending on endpoint). Implement backoff for high-volume use.

Verification Checklist

Before deploying or submitting work:

• All secrets stored in environment variables, not in code
• Step names use underscores or camelCase (no spaces/dashes)
• All async operations use await keyword
• Test event data matches expected trigger format
• Each step tested individually before full workflow test
• Data exports are JSON-serializable (no functions)
• Error handling implemented for external API calls
• Workflow has appropriate timeout and concurrency settings
• Component uses published key, not saved ID
• OAuth client credentials stored securely in .env
• No hardcoded user IDs or external_user_ids in code
• Workflow/component documentation includes example usage

Resources

Comprehensive navigation: https://pipedream.com/docs/llms.txt

Critical documentation pages:

• Connect Quickstart — Get started with managed auth and SDKs
• Workflow Quickstart — Build your first automation
• Connect API Reference — Full API and SDK docs

---

For additional documentation and navigation, see: https://pipedream.com/docs/llms.txt