shopify.engineering Open in urlscan Pro
185.146.173.20  Public Scan

Form analysis
3 forms found in the DOM

POST /content-services/subscribers

<form class="marketing-form marketing-form--inline subscribe__form" novalidate="novalidate" action="/content-services/subscribers" accept-charset="UTF-8" method="post">
  <div class="marketing-input-wrapper marketing-input-button-pair">
    <div class="marketing-input-button-pair__field-wrapper"><label class="marketing-label marketing-label--in-field marketing-label--floating marketing-input-button-pair__label" for="SubscribeEmail-1fd8">Email address</label><input
        autocomplete="email" placeholder="Email address" aria-describedby="MessageId_349a" name="email" id="SubscribeEmail-1fd8" class="marketing-input-button-pair__input marketing-input marketing-input--floating subscribe__email"
        type="email"><button class="marketing-button marketing-form__button marketing-input-button-pair__button subscribe__submit" name="button" type="submit">Yes, sign me up!</button></div>
  </div><span class="marketing-form__messages" id="MessageId_349a"></span><input type="hidden" name="data_extension_key" value="engineering-blog-subscribers" autocomplete="off">
  <input type="hidden" name="signup_page" value="https://shopify.engineering/introducing-shipit" autocomplete="off">
</form>

GET /search

<form class="search-form" action="/search" accept-charset="UTF-8" method="get">
  <label class="marketing-input-wrapper"><span class="marketing-label marketing-label--hidden visuallyhidden">Search articles</span><input aria-describedby="MessageId_3afa" class="marketing-input search-form__input" id="SidebarSearch"
      placeholder="Search articles" type="search" name="q"></label><span class="marketing-form__messages" id="MessageId_3afa"></span>
  <button type="submit" class="search-form__submit">
    <span class="visuallyhidden">Search</span>
  </button>
</form>

POST /content-services/subscribers

<form class="marketing-form marketing-form--inline subscribe__form" novalidate="novalidate" action="/content-services/subscribers" accept-charset="UTF-8" method="post">
  <div class="marketing-input-wrapper marketing-input-button-pair">
    <div class="marketing-input-button-pair__field-wrapper"><label class="marketing-label marketing-label--in-field marketing-label--floating marketing-input-button-pair__label" for="SubscribeEmail-b14f">Email address</label><input
        autocomplete="email" placeholder="Email address" aria-describedby="MessageId_0640" name="email" id="SubscribeEmail-b14f" class="marketing-input-button-pair__input marketing-input marketing-input--floating subscribe__email"
        type="email"><button class="marketing-button marketing-form__button marketing-input-button-pair__button subscribe__submit" name="button" type="submit">Yes, sign me up!</button></div>
  </div><span class="marketing-form__messages" id="MessageId_0640"></span><input type="hidden" name="data_extension_key" value="engineering-blog-subscribers" autocomplete="off">
  <input type="hidden" name="signup_page" value="https://shopify.engineering/introducing-shipit" autocomplete="off">
</form>

Text Content

Opens in a new window

Shopify uses cookies to provide necessary site functionality and improve your
experience. By using our website, you agree to our privacy policy and our cookie
policy .

OK

Skip to Content
Shopify
 * Engineering Blog
 * Working at Shopify
 * Open Source at Shopify 
 * Dev Degree 

 * See open roles

Open Main Navigation

Shopify Engineering

Show
 * Latest articles
 * Development
 * Infrastructure
 * Mobile
 * Developer Tooling
 * Security
 * Data Science & Engineering
 * Culture

Home
Close Main Navigation
 * Engineering Blog
 * Working at Shopify
 * Open Source at Shopify 
 * Dev Degree 

 * See open roles

Opens in a new windowOpens an external siteOpens an external site in a new
window


INTRODUCING SHIPIT

 * by Jean Boussier
 * Developer Tooling

 * Jun 17, 2015
 * 3 minute read

 * Email
 * Facebook
 * Twitter
 * LinkedIn

    3 minute read

After a year of internal use, we’re excited to open-source our deployment tool,
Shipit.

With dozens of teams pushing code multiple times a day to a variety of different
targets, fast and easy deploys are key to developer productivity (and happiness)
at Shopify. Along with key improvements to our infrastructure, Shipit plays a
central role in making this happen.


MOTIVATION

Coordinating frequent deployments in a large development team poses a number of
challenges. You need to ensure that no one else is currently deploying, that the
revision you are about to deploy has been successfully tested on CI, and even
that there is no ongoing maintenance operation. With smaller teams, it’s easy to
give developers deployment access via something like Capistrano, but giving
production credentials to many people can become a problem. Additionally,
keeping an audit trail of when specific commits were deployed is indispensable
when debugging production issues.

The first version of Shipit was built in early 2013 using an open source project
by Rackspace called Dreadnot. It allowed our developers to deploy their code
without needing to request credentials, understand the production hardware, or
involve the Operations team. While this worked well initially, we hit some
limitations. Configuring new projects was difficult, which meant few projects
were using it. We also hit performance limitations.

During Hack Days, we undertook a complete rewrite to address these problems.
Here are some of the improvements we've made:


SYNCHRONIZATION

To ensure code isn’t deployed during incidents like system maintenance, there
needs to be safety mechanisms in place. Shipit allows developers to set a lock
to prevent other developers from deploying when it’s unsafe to do so.




EASY SETUP

To make configuration easier, we adopted a model similar to Travis CI and
introduced the notion of a shipit.yml file. This allows deployment recipes to be
maintained within each project and kept under version control. Here's the one
used to ship Shipit:

This file contains bidirectional Unicode text that may be interpreted or
compiled differently than what appears below. To review, open the file in an
editor that reveals hidden Unicode characters. Learn more about bidirectional
Unicode characters
Show hidden characters

fetch: - curl --silent https://shipit.shopify.com/status/version tasks: restart:
action: Restart application description: Restart Thin and Resque. steps: - cap
$ENVIRONMENT deploy:restart

view raw shipit.yml hosted with ❤ by GitHub


We don't need to specify a deployment command - Shipit will infer the necessary
steps to deploy to Pypi, Rubygems, and anything using Capistrano.


AUDIT TRAIL

Shipit keeps logs and metadata of all the deploys and rollbacks performed.




BETTER PERFORMANCE

We were able to make gains in performance and simplicity by adopting GitHub as
the source of truth. Instead of constantly updating a local copy of the
repository and polling for CI status, Shipit relies on GitHub's push and status
webhook events. This has the added benefit of playing well with the plethora of
third-party services that use the Statuses API (including our Docker image
builder).




DEPLOYMENT

Before deployments, Shipit allows you to display key metrics, or add a checklist
that should be followed. We’ve also made it possible to write visualizations
shown during deployments, that make it easy to monitor progress and abort the
deployment if needs be.




SHIPIT AND YOU

We've put together a detailed README to get you started. Please report any
issues you might run into, and feel free to submit bug fixes and improvements
via pull request.

At Shopify, Shipit deploys over 200 projects (including itself) at the press of
a button - something we do several hundred times daily. It handles a broad set
of general purpose tasks ranging from updating DNS configurations to publishing
new versions of Python eggs, and deploys our applications to Heroku, EC2 and our
datacenters. Over time, we've learned from it and improved it, and we’re excited
to share it with the community.

 * Email
 * Facebook
 * Twitter
 * LinkedIn


GET STORIES LIKE THIS IN YOUR INBOX!

Stories from the teams who build and scale Shopify. The commerce platform
powering millions of businesses worldwide.

Email addressYes, sign me up!

Share your email with us and receive monthly updates.


THANKS FOR SUBSCRIBING.

You’ll start receiving free tips and resources soon.


Search articles Search


GET STORIES LIKE THIS IN YOUR INBOX!

Stories from the teams who build and scale Shopify. The commerce platform
powering millions of businesses worldwide.

Email addressYes, sign me up!

Share your email with us and receive monthly updates.


THANKS FOR SUBSCRIBING.

You’ll start receiving free tips and resources soon.

Resources
Our Tech Stack

Curious about what’s in our tech stack.

Sponsorship

We’re looking to partner with you.

Working Anywhere at Shopify

Learn about Digital by Design

Shopify Partner Developers

Become a Shopify developer and earn money by building apps or working with
businesses

Shopify Engineering on Twitter

Connect with us on Twitter

Shopify Engineering YouTube

Connect with us on YouTube

Popular
Migrating our Largest Mobile App to React Native Shopify Embraces Rust for
Systems Programming Mixing It Up: Remix Joins Shopify to Push the Web Forward
From Ruby to Node: Overhauling Shopify’s CLI for a Better Developer Experience A
Flexible Framework for Effective Pair Programming 10 Tips for Building Resilient
Payment Systems Five Common Data Stores and When to Use Them Deconstructing the
Monolith: Designing Software that Maximizes Developer Productivity Under
Deconstruction: The State of Shopify’s Monolith Reducing BigQuery Costs: How We
Fixed A $1 Million Query
Latest
Lessons From Linguistics: i18n Best Practices for Front-End Developers Tophat:
Crafting a Delightful Mobile Developer Experience Unlocking Real-time
Predictions with Shopify's Machine Learning Platform Improving the Developer
Experience with the Ruby LSP The Case Against Monkey Patching, From a Rails Core
Team Member The 25 Percent Rule for Tackling Technical Debt ShopifyQL Notebooks:
Simplifying Querying with Commerce Data Models Bringing Javascript to
WebAssembly for Shopify Functions The Complex Data Models Behind Shopify's Tax
Insights Feature The Hardest Part of Writing Tests is Getting Started


READY TO TACKLE FRONTEND, BACKEND, INFRASTRUCTURE, DATA, OR SECURITY CHALLENGES?

Explore all of our available roles