pipedream.com Open in urlscan Pro
3.168.122.16  Public Scan

Form analysis
0 forms found in the DOM

Text Content

SupportSupport (opens in a new tab)PricingPricing (opens in a new
tab)StatusStatus (opens in a new tab)
Version
SearchKGitHubGitHub (opens in a new tab)

(opens in a new tab)
SearchK
 * What is Pipedream?
 * Quickstart
    * Build a workflow
    * Use GitHub Sync

 * Workspaces
    * Managing workspaces
    * Single-Sign On
       * Overview
       * Google Workspace
       * Okta
       * SAML
   
    * Domain Verification

 * Projects
    * Projects
    * GitHub Sync
    * File Stores
       * File Stores
       * Reference

 * Workflows
    * What are workflows?
    * Steps
    * Triggers
    * Actions
    * Control Flow
       * Overview
       * Branching - If/Else
       * Branching - Switch
       * Branching - Parallel
       * Delay
       * Filter
       * End Workflow
   
    * Using Props
    * Events
    * Inspect events
    * Handling errors
    * Concurrency and Throttling
    * Settings
    * Virtual Private Clouds (VPCs)
    * Custom Domains
    * Sharing workflows

 * Code
    * Overview
    * Node.js
       * Running Node.js in workflows
       * Use AI to generate code
       * Connect apps
       * HTTP requests
       * Files
       * Data stores
       * Delaying steps
       * Pause, resume, and rerun steps
       * Running asynchronous code
       * Sharing code across workflows
       * Browser automation
   
    * Python
       * Running Python in workflows
       * Connect apps
       * HTTP requests
       * Files
       * Data stores
       * Delaying steps
       * Pause, resume, and rerun steps
       * Different PyPI package name and import name
       * Python FAQs
   
    * Go
    * Bash

 * Data Stores
 * Databases
    * Connecting to Databases
    * Working with SQL

 * Connected Accounts
    * Connected Accounts
    * OAuth Clients
    * API
    * External Auth

 * Integrations
    * Integrated Apps
    * Contributing
    * App Partners

 * Pipedream Connect
    * Overview
    * Use Cases
    * Quickstart
    * Running workflows
    * API & SDK Reference
    * Connect Tokens
    * Environments
    * OAuth Clients
    * Webhooks
    * Connect Link
    * Troubleshooting
    * Customize Your App

 * Components
    * Overview
    * Component API
    * Quickstart — Actions
    * Quickstart — Sources
    * Guidelines
    * TypeScript components
    * migrating
       * Index

 * Sources
 * Event History
 * HTTP
 * Environment Variables
 * REST API
    * API Reference
    * Authentication
    * Example: Create a workflow
    * Example: Create an RSS source
    * Example: Webhooks

 * CLI
    * CLI Reference
    * Install
    * Login

 * Destinations
    * Overview
    * Email
    * HTTP
    * Emit
    * SSE
    * S3

 * User and Billing Settings
 * Troubleshooting
 * Pricing FAQ
 * Security
    * Privacy and Security
    * Security best-practices
    * HIPAA compliance
    * PGP key

 * Limits
 * Glossary of Terms

System

On This Page

 * Adding a code step
 * Sharing data between steps
 * Using data from another step
 * Sending data downstream to other steps
 * Using $.export
 * Passing props to code steps
 * How Pipedream Node.js components work
 * Logs
 * console.dir
 * Syntax errors
 * Using npm packages
 * Just import it
 * Third-party package limitations
 * Pinning package versions
 * CommonJS vs. ESM imports
 * require is not defined
 * Variable scope
 * Making HTTP requests from your workflow
 * Returning HTTP responses
 * Invoke another workflow
 * Ending a workflow early
 * Errors
 * Configuration Error
 * Using secrets in code
 * Limitations of code steps
 * Editor settings
 * Keyboard Shortcuts
 * New to JavaScript?
 * I'm new to programming
 * I know how to code, but don't know JavaScript

Edit this page

Code
Node.js
Running Node.js in workflows


RUNNING NODE.JS IN WORKFLOWS

Pipedream supports writing Node.js v20 at any point of a workflow.

Anything you can do with Node.js, you can do in a workflow. This includes using
most of npm's 400,000+ packages. JavaScript is one of the most used languages in
the world, with a thriving community and extensive package ecosystem. If you
work on websites and know JavaScript well, Pipedream makes you a full stack
engineer. If you've never used JavaScript, see the resources below.

It's important to understand the core difference between Node.js and the
JavaScript that runs in your web browser: Node doesn't have access to some of
the things a browser expects, like the HTML on the page, or its URL. If you
haven't used Node before, be aware of this limitation as you search for
JavaScript examples on the web.


ADDING A CODE STEP

 1. Click the + button below any step of your workflow.
 2. Select the option to Run custom code.

Note that new code steps will default to Node.js v20. You can add any Node.js
code in the editor that appears. For example, try:

export default defineComponent({
  async run({ steps, $ }) {
    console.log("This is Node.js code");
    $.export("test", "Some test data");
    return "Test data";
  },
});



Code steps use the same editor (Monaco) used in Microsoft's VS Code, which
supports syntax highlighting, automatic indentation, and more.


SHARING DATA BETWEEN STEPS

A Node.js step can use data from other steps using step exports, it can also
export data for other steps to use.


USING DATA FROM ANOTHER STEP

In Node.js steps, data from the initial workflow trigger and other steps are
available in the steps argument passed to the run({ steps, $ }) function.

In this example, we'll pretend this data is coming into our HTTP trigger via
POST request.

{
  "id": 1,
  "name": "Bulbasaur",
  "type": "plant"
}



In our Node.js step, we can access this data in the steps variable Specifically,
this data from the POST request into our workflow is available in the trigger
property.

export default defineComponent({
  async run({ steps, $ }) {
    const pokemonName = steps.trigger.event.name;
    const pokemonType = steps.trigger.event.type;
 
    console.log(`${pokemonName} is a ${pokemonType} type Pokemon`);
  },
});




SENDING DATA DOWNSTREAM TO OTHER STEPS

To share data created, retrieved, transformed or manipulated by a step to others
downstream you can simply return it.

// This step is named "code" in the workflow
import axios from "axios";
 
export default defineComponent({
  async run({ steps, $ }) {
    const response = await axios.get(
      "https://pokeapi.co/api/v2/pokemon/charizard"
    );
    // Store the response's JSON contents into a variable called "pokemon"
    const pokemon = response.data;
 
    // Expose the pokemon data downstream to other steps in the $return_value from this step
    return pokemon;
  },
});




USING $.EXPORT



Alternatively, use the built in $.export helper instead of returning data. The
$.export creates a named export with the given value.

// This step is named "code" in the workflow
import axios from "axios";
 
export default defineComponent({
  async run({ steps, $ }) {
    const response = await axios.get(
      "https://pokeapi.co/api/v2/pokemon/charizard"
    );
    // Store the response's JSON contents into a variable called "pokemon"
    const pokemon = response.data;
 
    // Expose the pokemon data downstream to other steps in the pokemon export from this step
    $.export("pokemon", pokemon);
  },
});



Now this pokemon data is accessible to downstream steps within
steps.code.pokemon

⚠️

You can only export JSON-serializable data from steps. Things like:

 * strings
 * numbers
 * objects

You cannot export functions or other complex objects that don't serialize to
JSON. You can save that data to a file in the /tmp directory.


PASSING PROPS TO CODE STEPS



You can make code steps reusable by allowing them to accept props. Instead of
hard-coding the values of variables within the code itself, you can pass them to
the code step as arguments or parameters entered in the workflow builder.

For example, let's define a firstName prop. This will allow us to freely enter
text from the workflow builder.

export default defineComponent({
  props: {
    firstName: {
      type: "string",
      label: "Your first name",
      default: "Dylan",
    },
  },
  async run({ steps, $ }) {
    console.log(
      `Hello ${this.firstName}, congrats on crafting your first prop!`
    );
  },
});



The workflow builder now can accept text input to populate the firstName to this
particular step only:



Accepting a single string is just one example, you can build a step to accept
arrays of strings through a dropdown presented in the workflow builder.

Read the props reference for the full list of options.


HOW PIPEDREAM NODE.JS COMPONENTS WORK



When you add a new Node.js code step or use the examples in this doc, you'll
notice a common structure to the code:

export default defineComponent({
  async run({ steps, $ }) {
    // this Node.js code will execute when the step runs
  },
});



This defines a Node.js component. Components let you:

 * Pass input to steps using props
 * Connect an account to a step
 * Issue HTTP responses
 * Perform workflow-level flow control, like ending a workflow early

When the step runs, Pipedream executes the run method:

 * Any asynchronous code within a code step must be run synchronously, using the
   await keyword or with a Promise chain, using .then(), .catch(), and related
   methods.
 * Pipedream passes the steps variable to the run method. steps is also an
   object, and contains the data exported from previous steps in your workflow.
 * You also have access to the $ variable, which gives you access to methods
   like $.respond, $.export, and more.

If you're using props or connect an account to a step, the component exposes
them in the variable this, which refers to the current step:

export default defineComponent({
  async run({ steps, $ }) {
    // `this` refers to the running component. Props, connected accounts, etc. are exposed here
    console.log(this);
  },
});



When you connect an account to a step, Pipedream exposes the auth info in the
variable this.appName.$auth.


LOGS

You can call console.log or console.error to add logs to the execution of a code
step.

These logs will appear just below the associated step. console.log messages
appear in black, console.error in red.


CONSOLE.DIR

If you need to print the contents of JavaScript objects, use console.dir:

export default defineComponent({
  async run({ steps, $ }) {
    console.dir({
      name: "Luke",
    });
  },
});




SYNTAX ERRORS

Pipedream will attempt to catch syntax errors when you're writing code,
highlighting the lines where the error occurred in red.

⚠️

While you can save a workflow with syntax errors, it's unlikely to run correctly
on new events. Make sure to fix syntax errors before running your workflow.


USING NPM PACKAGES



npm hosts JavaScript packages: libraries of code someone else wrote and packaged
for others to use. npm has over 400,000 packages and counting.


JUST IMPORT IT

To use an npm package on Pipedream, simply import it:

import axios from "axios";



By default, workflows don't have any packages installed. Just import any package
in this manner to make it available in the step.

If a package only supports the CommonJS module format, you may have to require
it:

const axios = require("axios");



Within a single step, you can only use import or require statements, not both.
See this section for more details.

When Pipedream runs your workflow, we download the associated npm package for
you before running your code steps.

If you've used Node before, you'll notice there's no package.json file to upload
or edit. We want to make package management simple, so just import or require
the module like you would in your code, after package installation, and get to
work.


THIRD-PARTY PACKAGE LIMITATIONS

Some packages require access to a web browser to run, and don't work with
Node.js. Often this limitation is documented on the package README, but often
it's not. If you're not sure and need to use it, we recommend just trying to
import or require it.

Other packages require access to binaries or system libraries that aren't
installed in the Pipedream execution environment.

If you're seeing any issues with a specific package, please let us know and
we'll try to help you make it work.


PINNING PACKAGE VERSIONS

Each time you deploy a workflow with Node.js code, Pipedream downloads the npm
packages you import in your step. By default, Pipedream deploys the latest
version of the npm package each time you deploy a change.

There are many cases where you may want to specify the version of the packages
you're using. If you'd like to use a specific version of a package in a
workflow, you can add that version in the import string, for example:

import axios from "axios@0.19.2";



You can also pass the version specifiers used by npm to support semantic version
upgrades. For example, to allow for future patch version upgrades:

import axios from "axios@~0.20.0";



To allow for patch and minor version upgrades, use:

import got from "got@^11.0.0";


⚠️

The behavior of the caret (^) operator is different for 0.x versions, for which
it will only match patch versions, and not minor versions.

You can also specify different versions of the same package in different steps.
Each step will used the associated version. Note that this also increases the
size of your deployment, which can affect cold start times.


COMMONJS VS. ESM IMPORTS

In Node.js, you may be used to importing third-party packages using the require
statement:

const axios = require("axios");



In this example, we're including the axios CommonJS module published to npm. You
import CommonJS modules using the require statement.

But you may encounter this error in workflows:

Error Must use import to load ES Module

This means that the package you're trying to require uses a different format to
export their code, called ECMAScript modules (ESM, or ES modules, for short).
With ES modules, you instead need to import the package:

import got from "got";



Most package publish both CommonJS and ESM versions, so if you always use
import, you're less likely to have problems. In general, refer to the
documentation for your package for instructions on importing it correctly.


REQUIRE IS NOT DEFINED

This error means that you cannot use CommonJS and ESM imports in the same step.
For example, if you run code like this:

import fetch from "node-fetch";
const axios = require("axios");



your workflow will throw a require is not defined error. There are two
solutions:

 1. Try converting your CommonJS require statement into an ESM import statement.
    For example, convert this:

const axios = require("axios");



to this:

import axios from "axios";


 2. If the import syntax fails to work, separate your imports into different
    steps, using only CommonJS requires in one step, and only ESM imports in
    another.


VARIABLE SCOPE

Any variables you create within a step are scoped to that step. That is, they
cannot be referenced in any other step.

Within a step, the normal rules of JavaScript variable scope apply.

When you need to share data across steps, use step exports.


MAKING HTTP REQUESTS FROM YOUR WORKFLOW

There are two ways to make HTTP requests in code steps:

 * Use any HTTP client that works with Node.js. See this example guide for how
   to use axios to make HTTP requests.
 * Use $.send.http(), a Pipedream-provided method for making asynchronous HTTP
   requests.

In general, if you just need to make an HTTP request but don't care about the
response, use $.send.http(). If you need to operate on the data in the HTTP
response in the rest of your workflow, use axios.


RETURNING HTTP RESPONSES

You can return HTTP responses from HTTP-triggered workflows using the
$.respond() function.


INVOKE ANOTHER WORKFLOW

⚠️

This is an alpha feature and is subject to change without prior notice.

You can invoke another workflow in your workspace with $.flow.trigger:

await $.flow.trigger(
  workflowId, // your Pipedream workflow ID, e.g. p_abc123
  payload, // any JSON-serializable data
)



Find your workflow's ID here.

This invokes the workflow directly -- you don't need to configure a trigger, and
the request does not leave the platform.

We also provide a Trigger Workflow action in the Helper Functions app so you
don't need to do it by code!


ENDING A WORKFLOW EARLY



Sometimes you want to end your workflow early, or otherwise stop or cancel the
execution or a workflow under certain conditions. For example:

 * You may want to end your workflow early if you don't receive all the fields
   you expect in the event data.
 * You only want to run your workflow for 5% of all events sent from your
   source.
 * You only want to run your workflow for users in the United States. If you
   receive a request from outside the U.S., you don't want the rest of the code
   in your workflow to run.
 * You may use the user_id contained in the event to look up information in an
   external API. If you can't find data in the API tied to that user, you don't
   want to proceed.

In any code step, calling return $.flow.exit() will end the execution of the
workflow immediately. No remaining code in that step, and no code or destination
steps below, will run for the current event.

It's a good practice to use return $.flow.exit() to immediately exit the
workflow. In contrast, $.flow.exit() on its own will end the workflow only after
executing all remaining code in the step.

export default defineComponent({
  async run({ steps, $ }) {
    return $.flow.exit();
    console.log(
      "This code will not run, since $.flow.exit() was called above it"
    );
  },
});



You can pass any string as an argument to $.flow.exit():

export default defineComponent({
  async run({ steps, $ }) {
    return $.flow.exit("End message");
  },
});



Or exit the workflow early within a conditional:

export default defineComponent({
  async run({ steps, $ }) {
    // Flip a coin, running $.flow.exit() for 50% of events
    if (Math.random() > 0.5) {
      return $.flow.exit();
    }
    console.log("This code will only run 50% of the time");
  },
});




ERRORS

Errors raised in a code step will stop the execution of code or destinations
that follow.


CONFIGURATION ERROR

Throwing a ConfigurationError in a Node.js step will display the error message
in a dedicated area.

This is useful for providing feedback during validation of props. In the example
below, a required Header value is missing from the Google Sheets action:



Or you can use it for validating the format of a given email prop:

import { ConfigurationError } from "@pipedream/platform";
 
export default defineComponent({
  props: {
    email: { type: "string" },
  },
  async run({ steps, $ }) {
    // if the email address doesn't include a @, it's not valid
    if (!this.email.includes("@")) {
      throw new ConfigurationError("Provide a valid email address");
    }
  },
});




USING SECRETS IN CODE

Workflow code is private. Still, we recommend you don't include secrets — API
keys, tokens, or other sensitive values — directly in code steps.

Pipedream supports environment variables for keeping secrets separate from code.
Once you create an environment variable in Pipedream, you can reference it in
any workflow using process.env.VARIABLE_NAME. The values of environment
variables are private.

See the Environment Variables docs for more information.


LIMITATIONS OF CODE STEPS

Code steps operate within the general constraints on workflows. As long as you
stay within those limits and abide by our acceptable use policy, you can add any
number of code steps in a workflow to do virtually anything you'd be able to do
in Node.js.

If you're trying to run code that doesn't work or you have questions about any
limits on code steps, please reach out.


EDITOR SETTINGS

We use the Monaco Editor, which powers VS Code and other web-based editors.

We also let you customize the editor. For example, you can enable Vim mode, and
change the default tab size for indented code. Visit your Settings to modify
these settings.


KEYBOARD SHORTCUTS

We use the Monaco Editor, which powers VS Code. As a result, many of the VS Code
keyboard shortcuts should work in the context of the editor.

For example, you can use shortcuts to search for text, format code, and more.


NEW TO JAVASCRIPT?

We understand many of you might be new to JavaScript, and provide resources for
you to learn the language below.

When you're searching for how to do something in JavaScript, some of the code
you try might not work in Pipedream. This could be because the code expects to
run in a browser, not a Node.js environment. The same goes for npm packages.


I'M NEW TO PROGRAMMING

Many of the most basic JavaScript tutorials are geared towards writing code for
a web browser to run. This is great for learning — a webpage is one of the
coolest things you can build with code. We recommend starting with these general
JavaScript tutorials and trying the code you learn on Pipedream:

 * JavaScript For Cats
 * Mozilla - JavaScript First Steps
 * StackOverflow operates a programming Q&A site that typically has the first
   Google result when you're searching for something specific. It's a great
   place to find answers to common questions.


I KNOW HOW TO CODE, BUT DON'T KNOW JAVASCRIPT

 * A re-introduction to JavaScript (JS tutorial)
 * MDN language overview
 * Eloquent Javascript
 * Node School
 * You Don't Know JS


OverviewUse AI to generate code
System

--------------------------------------------------------------------------------

Pipedream, Inc. 2024