Replacing the Asset Pipeline with Gulp

By Marcin Radlak, 20 Jul 2015

Where does this idea come from? Why would someone want to drop the Asset Pipeline off? You may ask yourself what is wrong with it? The answer is nothing. Everything is fine until your project gets more and more complex, especially the front-end part. Then your start looking for solutions that could help you keep your code cleaner and more maintainable. If by any chance an idea of splitting back-end and front-end dependencies comes to your mind, you’re on the right track.

gulp

Asset Pipeline

The Asset Pipeline is a part of sprockets framework that compiles, caches and minifies all your assets. It allows you to write code assets in languages such as CoffeeScript, Sass, etc.. It is a very convenient way of managing your assets, and thanks to all of the magic under the hood, it becomes simple to use. However, there is a lot of modern JavaScript-based frameworks, which are made to allow you to build magnificent front-end applications. Unluckily, sprockets is dated and the integration with Rails is not always straightforward. Usually it can be easily done by dedicated gems, but let’s be honest - it’s not going to do the trick when you’re planning to have a front-end stack full of additional, external libraries. Having a great solution like the Node Package Manager (npm) it would be unreasonable not to organize your JavaScript dependencies with it. To bind node modules with Rails Asset Pipeline you can use CommonJS module loading. It introduces the situation when you have two sources of your libraries. The best way to handle it is to get rid of sprockets and introduce an independent compiling system. Let’s give Gulp a chance.

Gulp

Gulp is a build system and a task runner which uses Node.js. Some might say it is similar to Grunt and that’s true, because both do the same thing - they compile assets. Generally, Gulp is newer and more develop-oriented, so let’s use it. The real power of Gulp lies in a huge number of plugins which combined together could create a really efficient and airworthy tool. Gulp gives you the control of a compilation process. Actually, it works the way you define it to. Such flexibility is pretty desirable in large projects, but it doesn’t mean it’s difficult to get it work. It’s pretty much quite the opposite - the configuration is nice and simple. Gulp is designed to be fast and efficient. It uses streams which once opened enable to pass data through functions which in turn apply their modifications, pass data to the next ones and so on.

Removing sprockets

There is no clearly defined integration strategy, so let’s pick as laid-back approach as possible. The first thing you need to do is to get rid of sprockets and disable assets. If you’re going to initialize a new application without sprockets you can achieve it easily by typing:

1
rails new appname --skip-sprockets --skip-javascript

Options, --skip-sprockets and --skip-javascript skip sprockets and javascripts, respectively. As a result you’ll get a new app without app/assets/javascripts directory and config/initializers/assets.rb file. There’s also not going to be sprockets/railtie framework in the config/application.rb file.

If your application is already in use, you need to remove sprockets manually.

You have to exclude sprockets from the config/application.rb file. By default, Rails loads all frameworks in the following way:

1
require 'rails/all'

All you need to do is choose the ones you’ve decided to use. In our case, you will keep all of them except sprockets/railtie.

1
2
3
4
5
6
7
8
9
10
11
%w(
 active_model
 active_job
 active_record
 action_controller
 action_mailer
 action_view
 rails/test_unit
).each do |framework|
 require "#{framework}/railtie"
end

You no longer need gems like:

  • sass-rails
  • uglifier
  • coffee-rails
  • jquery-rails
  • turbolinks

Feel free to put them away from Gemfile. The specific job for assets is going to be moved to our new front-end infrastructure. It’ll bring some order to our Gemfile, because we don’t want to keep back-end dependencies with javascript libraries adaptable for sprockets anymore.

There’s no place for sprocket’s directive-like files:

  • app/assets/javascripts/application.js
  • app/assets/stylesheets/application.css

and

  • config/initializers/assets.rb.

The last thing you have to do is turn the assets generators off. This section also applies to apps generated with skip options mentioned above.

Add this to your config/application.rb:

1
2
3
config.generators do |g|
  g.assets false
end

Now, after you run

1
rails g scaffold ResourceName

Assets won’t be generated.

Welcome npm to the project

When our app is free from sprockets and front-end gems, you can bring the package manager - npm. It comes with Node.js. If you don’t have it installed, you can follow this tut.

Now you can initialize a new package file:

1
npm init

After many confirmations and prompts, the package.json file will be added to the project directory.

Let’s add some dependencies. At first glance it’s going to be gulp. It can be easily done typing:

1
npm install --save-dev gulp

It will install gulp in the node_modules directory and create the devDependencies key in package.json, which contains gulp and its newest version as a value. Pretty similar to the Gemfile workflow.

gulp is placed in the devDependencies on purpose, because you will need it only for development, and a production environment won’t be using that. Param save-dev place package under this key. All packages that are essential to our application in production environment should go under dependencies key. It can be done with --save param instead.

You want to access the gulp from the command line, so let’s run the following command:

1
npm install -g gulp

Gulpfile

Alright, everything is set up, so now you can start rebuilding the things you got rid of before (I mean the Assets Pipeline). All the logic responsible for assets compiling will be included in the Gulpfile.coffee file. Create this file in the project directory.

In this file, you will define Gulp tasks. First tasks are going to be responsible for compiling sass, scss and coffeescript, but first of all you will need some essential packages for it.

browserify
1
npm install --save-dev browserify

It allows you to write Node.js style modular code and load modules - yours and node’s.

gulp-sass
1
npm install --save-dev gulp-sass

This package allows you to compile sass and scss assets.

coffeeify
1
npm install --save-dev coffeeify

This one is browserify plugin and compilation solution for coffeescripts.

gulp-sourcemaps
1
npm install --save-dev gulp-sourcemaps

With this package you can deal with creating source maps from transformations easily.

vinyl-source-stream
1
npm install --save-dev vinyl-source-stream

This is necessary because browserify has streams workflow. Briefly, Browserify takes only the content of the file you want, but Gulp also needs the file information. However, vinyl-source-stream solves this problem with its virtual file system.

uglifyify
1
npm install --save-dev uglifyify

This is a browserify transform which minifies code using UglifyJS2.

watchify
1
npm install --save-dev watchify

It would be a huge loss in comparison to sprockets if there were no hidden assets compiling on every change. Fortunately, the solution is called watchify.

gulp-livereload
1
npm install --save-dev gulp-livereload

This is a very useful package which - together with a browser extensions (Chrome extension) - provides live preview of assets. It means that any time one of the assets file is edited and saved, the browser will refresh after a compiling process.

Now, let’s put all these things together and write some code.

You need to require all essential packages.

1
2
3
4
5
6
7
gulp         = require 'gulp'
sourceMaps   = require 'gulp-sourcemaps'
liveReload   = require 'gulp-livereload'
sass         = require 'gulp-sass'
browserify   = require 'browserify'
watchify     = require 'watchify'
sourceStream = require 'vinyl-source-stream'

Gulp recognizes the task named default which is executed after gulp command. For our purposes we want this task to call two other tasks. Each has only one role - compile a defined asset type in the project.

1
gulp.task 'default', [scss, coffee]

Scss compilation

1
2
3
4
5
6
7
8
9
10
gulp.task 'scss', ->
  gulp.src 'app/assets/stylesheets/**/*.scss'
    .pipe sourceMaps.init()
    .pipe sass(
      sourceComments: true
      outputStyle: 'compressed'
    )
    .pipe sourceMaps.write()
    .pipe gulp.dest('public/stylesheets')
    .pipe liveReload()

src() method takes a path to your .sass files and creates a stream of objects. Next it can be piped through a number of transformations. sourceMaps.init() and sourceMaps.write() are responsible for source map creation, ssas() compiles .sass files. Transforming ends with dest() which saves the result under a passed path. liveReload() creates a stream which informs the livereload server about the changes.

CoffeeScript compilation

At the beginning you need a stream with all modules. browserify allows to do this.

1
2
3
4
coffeeStream = browserify
  entries: ['app/assets/javascripts/application.coffee']
  extensions: ['.coffee']
  debug: true

Now let’s create two functions - for transformations and for bundling.

1
2
3
4
transformCoffee = (stream) ->
  stream
    .transform 'coffeeify'
    .transform 'uglifyify'

Here, stream data is compiled with coffeeify and minified with uglifyify.

1
2
3
4
5
6
bundleCoffee = (stream) ->
  stream
    .bundle()
    .pipe sourceStream('application.js')
    .pipe gulp.dest('public/javascripts')
    .pipe liveReload()

This function bundles the stream and thanks to vinyl the result is saved. At the end livereload server can be notified.

Ok, now put in the final task:

1
2
gulp.task 'coffee', ->
  bundleCoffee transformCoffee(coffeeStream)

Recompile on changes

It’s time to set a task which will be responsible for watching for changes and running an adequate compile process.

1
2
3
4
gulp.task 'watch', ->
  liveReload.listen()
  gulp.watch 'app/assets/stylesheets/**/*.scss', ['scss']
  watchCoffee()

First you need to start livereload server to keep browser up to date with changes. Next, you can start watching for changes in all .scss files which is pretty easy in comparison to coffeescripts.

1
2
3
4
5
6
7
watchCoffee = ->
 coffeeWatchStream = watchify coffeeStream

 transformCoffee(coffeeWatchStream)
   .on 'update', -> bundleCoffee(@)

 bundleCoffee coffeeWatchStream

Restoring ‘application.coffee’ and 'application.scss’

In the previous steps the application.js file was removed from the app/assets/javascripts directory, but now you are ready to restore it with a different approach. app/assets is not the only directory where all assets must be placed. You can place them wherever you want, but according to the Rails convention it is a suitable place.

First, let’s install jquery and jquery-ujs with dev dependencies:

1
2
npm install --save jquery
npm install --save jquery-ujs

Then, load it into app/assets/javascripts/application.coffee:

1
2
window.$ = window.jQuery = require('jquery')
require('jquery-ujs')

Let’s create app/assets/javascripts/application.scss for styles.

After adding some js code and styles you can run gulp from the command line to have all assets compiled and placed under public directory. To automatize this process just run gulp watch, and let Gulp take care of the rest.

What next?

Right now you can enjoy your new assets manager in a development environment, but before pushing it to production more setups needs to be done. Firstly, by removing sprockets you’ve lost mapping for our cached files to the real URLs. It is strongly recommended to create a cache manifest for all assets. For this purpose you can create a special task which uses one of the revisioning packages to create a manifest file. gulp-rev-all will do the job nicely. Another thing which should be implemented is a simple Rails monkeypatching to handle a new assets manifest and generate correct URLs in helper methods like stylesheet_link_tag or javascript_include_tag.

About the author

Marcin Radlak

comments powered by Disqus