Continuous integration on server side – Part 2

Create own grunt task

In the previous post I mentioned an extra task, cleaning folders and removing the folders as well.

We found a tool to clean a folder called grunt-contrib-clean.
But we had two problems with that tool.

  • It doesn’t support removing the folder itself. It only clears the content of the specified folder.
  • You can use relative path from a specified base directory but you can’t use back navigation in the path.

At this point I need to admit that I didn’t do a brighter resource after checking the grunt-contrib-clean tool because I wanted to write an own grunt task and it was good opportunity to start writing one.

At first I started with checking the documentation on the webpage of the grunt.
I thought that it will be a good start but after reading the “Create own task” section I didn’t have the clue how to really start the coding.

Rather as a second option I checked the implementation of the uglify and the yuidoc tasks.
As I mentioned it in the previous post we use that two tools in our application and we found grunt task to execute them as well.

And that’s the part of the javascript/opensource world I love the most. You can read and learn from other’s implementation.
There are really good examples outside, like the two grunt tasks I chose.

After checking this two implementations I had the clue how to start.

My first implementation was this:

        
        module.exports = function (grunt) {

    	grunt.registerTask('myclean', "Remove folder and it's content recursive", function () {
    		grunt.log.ok();
    	}

and it worked! : )

I just loaded it into my Gruntfile.js:

 
  	grunt.loadTasks('core/tasks'); is the way to load it into the the actual Gruntfile.js

and after running it in my terminal the result was:

terminal_ok

At his point I had the skeleton and I could start to fill it with content.

As I mentioned it earlier I wanted to use relative path in the settings of the task.
I wanted to use ‘..’ in the path to step back one folder.

And I wanted to remove not just the full content of the specified folder but the folder itself as well.

Let’s start coding our own task.

As the result we wanted to see the following customisable task in the Gruntfile.js:

You can get the user setting of your task in the implementation of the task with the options command

     	
    	        var options = this.options({
        });

It loads you the options from the gruntfile.
Of course you need to check the existence of the expected option

                    
        if (!options.cleanPath) {
            grunt.log.writeln('cleanPath is empty or not defined');
            return false;
        }

After that you are safe here and you can use the ‘cleanPath’ filled with the value that the user specified

I show my implementation about creating the right path to clean:

        
        var baseDir = path.normalize(__dirname + '/../..'); //Server folder
                var parts = value.split('/');
                var endPath = '';

                for (var i = 0; i < parts.length; i++) {

                    if (parts[i].indexOf('..') != -1) {
                        baseDir += '/../';
                    } else {
                        endPath = path.join(endPath, parts[i]);
                    }
                }

                var actualDir = path.normalize(baseDir);

                var preparedPath = path.join(actualDir, endPath);

                deleteFolderRecursive(preparedPath);

With this implementation I can set the clearPath in my Gruntfile with the following value: “../../Deployment/Server”
and it will navigate from the current folder (Source/Server) back to the main project folder and then into the other sibling folder I gave to it.

And the delete part:

         
        var deleteFolderRecursive = function (cleanPath) {

            if (fs.existsSync(cleanPath)) {
                fs.readdirSync(cleanPath).forEach(function (file) {

                    var curPath = cleanPath + "/" + file;
                    if (fs.statSync(curPath).isDirectory()) {
                        deleteFolderRecursive(curPath);
                    } else { // delete file
                        fs.unlinkSync(curPath);
                    }
                });
                fs.rmdirSync(cleanPath);
            }
        };      

…witch deletes the specified folder and its content as well.

I don’t want to explain my code because it not so complicated I just want to highlight some grunt and node feature that I used and that are good to know for you as well

  • __dirname – gets the actual directory path, where the file is located.
  • path.normalize – it will parse the ‘..’ back navigation if it is at the end of the path
    • folder1/folder1.1/.. –> folder1
    • folder1/folder1.1/../folder1.2 –> will not navigate to folder1/folder1.2
  • path.join – after splitting the file path by ‘/’ it will create a correct path from the parts.
  • grunt.util has many other useful functions
    I used only _.each() to iterate through the input array but there are methods to check the types of the input parameters and a lot of other good stuff.

Task vs. Multitask

If you create task with the “registerTask” method call you can define the task later only with one settings.

  
        myclean: {
                options: {
                    cleanPath: ['./docs', '../../Deployment/Server']
                }
            }

But what if you don’t want to always clear all the paths you defined at first.
What if in certain situations you only want to remove he ‘docs’ folder.

With ‘registerTask’ you can’t do that, BUT with ‘registerMultiTask’ it is possible.
You only need to rewrite the method name. The content of the task can remain the same and grunt will do for you everything else.

From that time you can set more execution options and name them with an arbitrary name.

  
        myclean: {
            default: {
                options: {
                    cleanPath: ['./docs', '../../Deployment/Server']
                }
            },
            onlyDocs: {
                options: {
                    cleanPath: ['./docs']
                }
            }
        }

and later that you can use them like:

          
	grunt.registerTask('default', ['myclean:default', 'yuidoc', 'uglify']);

	grunt.registerTask('onlyDocs', ['myclean:onlyDocs','yuidoc']);

After that you can run the grunt with different switches:

Write ‘grunt’ into the terminal and it will run the ‘grunt default’. First it cleans every folder, then creates the documentation and then uglifies and copies the files into the Development folder.
And running ‘grunt onlyDocs’ executes the second set of tasks. It clears only the ‘docs’ folder and regenerate the documentation.

That’s it.

Spoiler

This was our start with grunt on server side. We created the base tasks and a self-made task as well. But the journey doesn’t end here. Int he following post of this series we will check how to configure grunt to run tests and run our express server.

Advertisements

Continuous integration on server side – Part 1

I will start this post from a bit far… Far, because I will start from our AngularJs client. : )

I start with Yeoman.
Yeoman, if you work with AngularJs you have probably heard about yeoman.

If you don’t you really need to check it on http://yeoman.io/

It makes your life much easier!
“Yeoman is more than just a tool. It’s a workflow.” “Modern workflows for modern webapps”

It is a collection of tools:

  • Yo – It generates you a good skeleton to work on an AngularJs project.
  • Grunt – It helps you to host the app with only one command – grunt server (and a lot of other things it can do for you but this is the easiest one and the first one you will meet.)
  • Bower – It helps you to find and install easily components to AngularJs. (Like the node Package Manager in the Node.js world)
  • To have the whole stack of tools I would suggest one more tool, a Bower module: Generator-Angular

Generator-Angular can generate for you all the necessary components of an AngularJs app. Like services, controllers, directives and so on with a simple command:

yo angular:directive myDirective 

and it will generate you everything to have a directive as part of your app.

But we are working on a server side now. Losing all the good staffs only because you work on server side would make you feel bad. As it would make us sad too. BUT Grunt helps you on server side as well!

What is this Grunt exactly? – you could ask.

Grunt is a task runner tool to automatize the execution of your repetitive tasks.

If you are a developer Grunt makes your life easier with automatising your not ‘coding’-tasks and
if you are a project manager it gives you well sounding-words to improve your project’s marketing.

Automation and continuous integration. They are really nice words for public but they are really important things in development as well.

There are tasks that you need to do repetitive and doing them always manual would be a lot of work.

But you can automatize your tasks!

If you think of automatizing something in software development, the most common thought is testing. It is very important of course and automatizing the test runs is a really good idea (we do that as well)
but there are other smaller tasks as well that you can automatize. Like generating documentation, minifying, uglifying the source code and so on.

You need to execute some tasks very often. Maybe some tasks more than once a day. With that we arrived to the concept of Continuous integration*.
*Continuous integration (CI) is the practice, in software engineering, of merging all developer working copies with a shared mainline several times a day.

And here comes the grunt for. It will support you to do all of this.

Now we know everything about the ‘Why?’ but let’s see the ‘How?’ part of all of this.

Let’s see how to use grunt on server side!

(I will write about CI with grunt in three posts. This is the first one with a longer intro and with integration of small third-party tasks, in the second we will create an own task and check how to use it and the last one will be about testing and automatizing the test runs with grunt.)

Our requirements were:

  • We wanted to generate documentation to our project for the customer.
  • We wanted to prepare a deliverable package with the minified and uglified source.
  • Extra was the request to clean the folders before regenerating the components.

For almost everything exists a grunt component. If you use a third party library to generate doc or to test you will find the grunt task version of it.

– To minify and uglify the source code we found the grunt-contrib-uglify.
– We use YuiDoc to create our documentation and of course there is a task to use yuidoc with grunt – grunt-contrib-yuidoc.

This modules are available on github, they have simple documentations and it is easy to use them.

Integrate Grunt into your project

To integrate Grunt you need to add only one file to your project called Gruntfile.js.

The content of the file needs to be inside a module.exports block

module.exports = function (grunt) {
    ...
};

…and the file have to contain 3 section:

1, To configure the modules you use:

    grunt.initConfig({
    	...
    	});

2, To load the tasks (the sources)

    grunt.loadNpmTasks('module_name');

    grunt.loadTasks('your_module');

3, To specify the grunt task collections you want to run later

    grunt.registerTask('default', ['myclean:default', 'yuidoc', 'uglify']);

    grunt.registerTask('onlyDocs', ['myclean:onlyDocs','yuidoc']);

Our configuration for the two modules that I mentioned before are really simple:

uglify: {
            options: {
                banner: '/*! innio <%= grunt.template.today("yyyy-mm-dd") %> */\n'
            },
            my_target: {
                files: [
                    {
                        expand: true,
                        src: ['./core/*.js', './routes/*.js'],
                        dest: '../../Deployment/Server',
                        ext: '.min.js'
                    }
                ]
            }
        }
yuidoc: {
            compile: {
                name: '<%= pkg.name %>',
                description: '<%= pkg.description %>',
                version: '<%= pkg.version %>',
                url: '<%= pkg.homepage %>',
                options: {
                    paths: './',
                    outdir: './docs/'
                }
            }
        }

They are only from the default examples on the module’s github pages. We only parametrized them other for our purpose.

Tip 1. Uglify creates by default one file for everything inside the source folder but with the following config it generates separated folders and files with the minified versions of it and structures them as the original structure looked like.

And that’s it.

module.exports = function (grunt) {

    // Project configuration.
    grunt.initConfig({
        pkg: grunt.file.readJSON('package.json'),

        uglify: {
            options: {
                banner: '/*! innio <%= grunt.template.today("yyyy-mm-dd") %> */\n'
            },
            my_target: {
                files: [
                    {
                        expand: true,
                        src: ['./core/*.js', './routes/*.js'],
                        dest: '../../Deployment/Server',
                        ext: '.min.js'
                    }
                ]
            }
        },

        yuidoc: {
            compile: {
                name: '<%= pkg.name %>',
                description: '<%= pkg.description %>',
                version: '<%= pkg.version %>',
                url: '<%= pkg.homepage %>',
                options: {
                    paths: './',
                    outdir: './docs/'
                }
            }
        }
    });

    /*Load tasks*/
    grunt.loadNpmTasks('grunt-contrib-uglify');

    grunt.loadNpmTasks('grunt-contrib-yuidoc');


    /*Register tasks*/
    grunt.registerTask('default', ['yuidoc', 'uglify']);
};

To execute this you only need to write the command “grunt” into the terminal and hit Enter.

Spoiler

The next post will be about creating your own task. It is a more adventures challenge but we will go through it step by step.