Today, I wanted to talk about one of the more annoying problems that pops from time to time: setting up a new machine, and keeping existing machines in sync. Setting up new machines just the way you like them is hard. The longer you’ve been developing, the more programs, tools, and configuration you take with you. When you set up a new machine, you have to remember everything you had set up on an old machine, including:

  • your .rc and shell config files (.vimrc, .emacs, .bashrc)
  • all of the packages you’ve installed
  • packages you’ve downloaded by hand and extracted
  • little tweaks and workarounds you had to employ to get stuff working
  • generating ssh keys and setting up ssh configs

At a company, things become even more difficult. In addition to the difficulties mentioned above, you have to handle additional problems like installing and updating internal tools, and you have more machines: every developer needs to remain in sync!

Another one of the big problems comes from developers who use their own machines to work on company projects: there’s times when you want to deactivate or remove the company development environment, so you can continue work on your own, personal projects, without having to take a lot of time switching back and forth.

Enter Sprinter #

Sprinter is a development environment management tool, designed to easily create, manage, and configure multiple development environments.

So what does that look like? Well, the first step is to install sprinter. Currently, installing sprinter involves downloading and running a shell script. You can take advantage of these pre-crafted commands:

Using curl (OSX):

curl -s https://raw.github.com/toumorokoshi/sprinter/master/scripts/sandbox.sh > /tmp/sprinter; bash /tmp/sprinter

Using wget (Debian/Ubuntu):

cd /tmp/; rm sandbox.sh; wget https://raw.github.com/toumorokoshi/sprinter/master/scripts/sandbox.sh -O sandbox.sh; bash sandbox.sh

NOTE: You’ll have to open a new shell every time you modify an environment. This is because resetting shells to scratch really isn’t possible. Currently the best solution is just to open a new one.

This adds the sprinter command in a sandboxed location, inside your user root (~/.sprinter). In fact, when sprinter installs anything, it usually ends up in there. This allows for sandboxing packages and executables to make changing environments easier.

Defining Sprinter Environments #

Each definition of a sprinter environment is a config file. For example, here’s an example configuration for an environment which generates an ssh key, installs node.js, and sets up some git configuration:

[config]
namespace = developer

[git]
formula = sprinter.formula.package
apt-get = git-core
brew = git
rc =
  git config --global user.name "Dev Eloper"
  git config --global user.email "eloperdev@gmail.com"
  git config --global alias.c checkout
  git config --global alias.s status

[github]
formula = sprinter.formula.ssh
keyname = github.com
nopassphrase = true
type = rsa
host = github.com
user = git
hostname = github.com

[node]
formula = yt.formula.node:git+https://github.com/toumorokoshi/yt.formula.node.git
version = 0.10.16
packages =
  grunt-cli

Installing an environment is as easy as pointing sprinter to the environment configuration file:

sprinter install https://raw.github.com/toumorokoshi/yt.rc/master/toumorokoshi.cfg
sprinter install ~/downloads/myconf.cfg

And the output looks like this:

$ sprinter install developer.cfg
Checking and setting global parameters...
Installing environment developer...
A standard global ssh key was detected! Would you like to use the global ssh key? (default no): no
installing node...
/home/tsutsumi/.sprinter/developer/features/node/bin/grunt -> /home/tsutsumi/.sprinter/developer/features/node/lib/node_modules/grunt-cli/bin/grunt
grunt-cli@0.1.9 /home/tsutsumi/.sprinter/developer/features/node/lib/node_modules/grunt-cli
├── resolve@0.3.1
├── nopt@1.0.10 (abbrev@1.0.4)
└── findup-sync@0.1.2 (lodash@1.0.1, glob@3.1.21)

installing git...
Installing git-core...
[sudo] password for tsutsumi:
Reading package lists... Done
Building dependency tree
Reading state information... Done
git-core is already the newest version.
The following packages were automatically installed and are no longer required:
  linux-headers-3.2.0-32 linux-headers-3.2.0-27 linux-headers-3.2.0-32-generic
  linux-headers-3.2.0-27-generic
Use 'apt-get autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 129 not upgraded.
installing github...
Finalizing...
Injecting values into /home/tsutsumi/.profile...
Injecting values into /home/tsutsumi/.zprofile...
Injecting values into /home/tsutsumi/.zshrc...
Injecting values into /home/tsutsumi/.bash_profile...
Injecting values into /home/tsutsumi/.bashrc...
Injecting values into /home/tsutsumi/.bash_profile...
Injecting values into /home/tsutsumi/.zprofile...

Once installed, sprinter remembers where the configuration was found, and updating is a simple as re-installing the environment, or updating the specific namespace (sprinter looks for the updated config where you last installed the environment):

sprinter update developer
sprinter install https://raw.github.com/toumorokoshi/yt.rc/master/toumorokoshi.cfg
sprinter install ~/downloads/myconf.cfg

This way, managing a cross-platform(ish) development environment and distributing it is as simple as hosting a configuration file, or storing one in a git repository. Simply modify your configuration file, push it, and update it when you move machines!

Managing Environments #

Turning environments on and off is easy! Just use the deactivate and activate commands:

# turn off the developer environment
sprinter deactivate developer
# turn on the developer environment
sprinter activate developer

(When you deactivate and activate an environment, you have to open a new shell)

And removing an environment completely? Well, that’s just:

sprinter remove developer

In fact, the installer above installs sprinter as an environment! So if you wanted to remove sprinter, you could:

sprinter remove sprinter

Building environments for companies #

Sprinter is a great way to manage one’s own personal environment, but it was designed to support company-wide environments as well. The only problem that company set-up scripts have over personal ones is customization: you need to be able to customize your install based on the username, one’s own file layout, whether they want to use their own ssh keys, and more.

Sprinter includes the ability to prompt for values (and remember them) during setup. For example let’s modify the configuration above to use a username and password, and upload the ssh key to an instance of Atlassian Stash:

[config]
namespace = mycompany
inputs = fullname
         username
         domainpassword?
message_success = Welcome to mycompany!
message_failure = Noo! Please email immrmanager@mycompany.com for help

[git]
formula = sprinter.formula.package
apt-get = git-core
brew = git
rc =
  git config --global user.name "%(config:fullname)s"
  git config --global user.email "%(config:username)s@mycompany.com"
  git config --global alias.c checkout
  git config --global alias.s status

[curl]
formula = sprinter.formula.package
apt-get = curl

[stash]
formula = sprinter.formula.ssh
depends = curl
keyname = stash.mycompany.local
nopassphrase = true
type = rsa
host = mycompany-stash
user = git
hostname = stash.mycompany.local
install_command = curl -k -u '%(config:username)s:%(config:domainpassword)s' -X POST -H "Accept: application/json" -H "Content-Type: application/json" https://stash.mycompany.local/rest/ssh/1.0/keys -d '{"text":""}'
use_global_ssh = False

[node]
formula = yt.formula.node:git+https://github.com/toumorokoshi/yt.formula.node.git
version = 0.10.16
packages =
  grunt-cli

Note that you can use the input variables in a variety of places (in the example above, the username input is used in both the git formula, and the ssh formula to upload the ssh key). Here’s what inputting parameters look like to someone installing this environment:

$ sprinter install mycompany.cfg
Checking and setting global parameters...
Installing environment mycompany...
please enter your fullname:
please enter your username:
please enter your domainpassword:

Note that you can even add success/failure messages (message_success and message_failure in the config above), so you can add e-mails for troubleshooting and instructions on what to do next.

Why Sprinter? Why not Boxen/Chef/Puppet/X? #

Sprinter definitely isn’t the first solution to try to manage an environment. Many alternatives exist, with their own merits. There was a few driving factors that motivated me to roll my own, the main reason being that the problem of maintaining development environments and development tools is quite a bit different from maintaining a cluster of systems for running services in. Existing solutions tended to be ops-driven (or based on ops-driven technologies), and didn’t make considerations like:

Configuration based on user input #

A lot of the management systems out there were designed to push a machine into a specific state (Chef/Puppet). This design is influenced directly from an operator perspective, where a machine does not need to consider any state that already exists on the user machine (e.g. existing SSH configuration, .rc files). Sprinter and it’s formulas take a lot of caution to not override as much global state as possible, so the only configuration sprinter overrides are the ones it was specifically directed to do. (e.g. adding ssh or bashrc configuration inline with existing ones, instead of overwriting a file completely)

In addition, I haven’t seen any configuration management query for user input on install. This makes things like automatically uploading ssh keys (which typically requires passwords you don’t want to store in a repository) very tricky, unless you’re willing to do a lot of finagling with environment variables.

Sprinter solves this problem by querying and storing user input in it’s configuration, so you only have to configure things once, and it can be different for every user that installs it.

Sandboxing Environments #

All of the existing environment management tools don’t really consider sandboxing an environment. Once again this comes from the fact that current tools are designed from the needs of an operator: why would you ever want to sandbox state of a machine that’s only going to be one type it’s whole life? For developers, the needs are different: you might have to reconfigure yourself to a release box, or a test box (in the very common case where differences exist), and switching between them can mean removing everything and installing from scratch.

Sprinter formulas are designed to be able to easily inject and remove state from a system. For example, a sprinter deactivate assures that anything added to an .rc file is removed, and removing items from the PATH. This works well for personal machines, because working on software for your company doesn’t mean you have to completely reconfigure your machine into an irreparable state.

NOTE: (unfortunately, package managers on most systems are global, so it’s not possible to sandbox those. Possible solutions to this problem are still in the works.)

Multiple Simultaneous Environments #

Tools like Chef or Boxen have the disadvantage that they only allow the one state to exist. There’s no special logic to handle things like activating two different environments simultaneously.

Sprinter provides that functionality. You can overlay as many environments as you want on top of each other, and each piece is still a modular component that can be installed or removed. (the most recently activated/updated environment takes precedence).

This works very well for the cases sprinter is designed for, like having a personal environment distributed through sprinter while using your company or organization’s configuration as well.

It’s easy to setup and install #

Chef and Puppet both typically require running a service, and a bit of configuration to get up and running. Sprinter configs and the update process was designed so that anyone could easily add an environment into their project that developers could use.

Having a global environment that anyone can use is as simple as publishing a file online through a webserver. In fact, github is a great place to host this. To see an example, you can look at my environment repository, where I maintain the development environment I use on my Linux and OSX machines (I switch between three or four).

So in conclusion… #

Sprinter has been a fun project for me that I feel like has a lot of potential. Please give it a try! Here’s some ways to explore Sprinter:

And of course feel free to leave a comment :)