Continuous integration and continuous deployment with Lambda

In this section, we will be using Jenkins, a serverless framework, and other open source software to set up Continuous Integration and Continuous Deployment. I have the whole project set up in a Git repository (https://github.com/shzshi/aws-lambda-dynamodb-mytasks.git), or we can go through the following steps listed in this section.

The application that we are using for this tutorial will create a Lambda function and an AWS API gateway for the task where we can test our Lambda function, which will manage tasks using CRUD operations to DynamoDB. 

First, let's create a folder structure using a serverless framework to create a template function, as shown in the following code. I am assuming that you are using Linux Terminal, and that all the instructions are Linux-Terminal-based:

$ serverless create --template aws-nodejs --path AWSLambdaMyTask
$ cd AWSLambdaMyTask

We will see three files created within the AWSLambdaMyTask folder. This is a sample template for Node.js using a serverless framework. We will be modifying these files as per our example's need, as shown in the following code:

  1. Let's create two more folders within the AWSLambdaMyTask folder, namely src and test. The src phrase is our source code folder and test is the folder for our test cases, as shown in the following code:
$ mkdir test
$ mkdir src
  1. Then we will create a file called package.json using the editor. This file will hold the metadata that is relevant to the project. Copy the following content into the file. Please make whatever changes you need:
{
  "name": "AWS-serverless-with-dynamodb",
  "version": "1.0.0",
  "description": "Serverless CRUD service exposing a REST HTTP interface",
  "author": "Shashikant Bangera",
  "dependencies": {
    "uuid": "^3.0.1"
  },
  "keywords": [
    "AWS",
    "Deployment",
    "CD/CI",
    "serverless",
    "task"
  ],
  "repository": {
    "type": "git",
    "url": ""
  },
  "bugs": {
    "url": ""
  },
  "author": "Shashikant Bangera <shzshi@gmail.com>",
  "license": "MIT",
  "devDependencies": {
    "AWS-sdk": "^2.6.7",
    "request": "^2.79.0",
    "mocha": "^3.2.0",
    "serverless": "^1.7.0"
},
  "scripts": {
    "test": "./node_modules/.bin/mocha"
  }
}
  1. Let's edit the serverless.yml file, as per our needs, as shown in the following snippet. You can find the file in the mentioned GitHub repo: https://github.com/shzshi/aws-lambda-dynamodb-mytasks/blob/master/serverless.yml:
# Welcome to Serverless!
#
# This file is the main config file for your service.
# It's very minimal at this point and uses default values.
# You can always add more config options for more control.
# We've included some commented out config examples here.
# Just uncomment any of them to get that config option.
#
# For full config options, check the docs:
#    docs.serverless.com
#
# Happy Coding!
service: AWSLambdaMyTask
frameworkVersion: ">=1.1.0 <2.0.0"
provider:
  name: AWS
  runtime: nodejs4.3
  environment:
    DYNAMODB_TABLE: ${self:service}-${opt:stage, self:provider.stage}
  iamRoleStatements:
    - Effect: Allow
      Action:
        - dynamodb:Query
        - dynamodb:Scan
        - dynamodb:GetItem
        - dynamodb:PutItem
        - dynamodb:UpdateItem
        - dynamodb:DeleteItem
      Resource: "arn:AWS:dynamodb:${opt:region, self:provider.region}:*:table/${self:provider.environment.DYNAMODB_TABLE}"
functions:
  create:
    handler: src/mytasks/create.create
---
  list:
    handler: src/mytasks/list.list
---
  get:
    handler: src/mytasks/get.get
---
  update:
    handler: src/mytasks/update.update
---
  delete:
    handler: src/mytasks/delete.delete
---
resources:
  Resources:
    mytasksDynamoDbTable:
      Type: 'AWS::DynamoDB::Table'
      DeletionPolicy: Retain
      Properties:
        AttributeDefinitions:
          -
            AttributeName: id
            AttributeType: S
        KeySchema:
          -
            AttributeName: id
            KeyType: HASH
        ProvisionedThroughput:
          ReadCapacityUnits: 1
          WriteCapacityUnits: 1
        TableName: ${self:provider.environment.DYNAMODB_TABLE}
  1. Let's move the src directory, and create a file named package.json and a folder named mytasks, as shown in the following code. The mytasks folder will have Node.js files to create, delete, get, list, and update the DynamoDB table on the AWS:
$ cd src
$ mkdir mytasks
$ vim package.json
  1. Copy the following content to package.json:
{
  "name": "src",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "MIT",
  "dependencies": {
    "uuid": "^2.0.3"
  }
}

Go to the  mytasks folder and create a create.js file to create, update, list, get, and delete the DynamoDB tables. The create.js files are handlers for functions in Lambda.

  1. Add the following content to src\mytasks\create.js:
'use strict';
const uuid = require('uuid');
const AWS = require('AWS-sdk');
const dynamoDb = new AWS.DynamoDB.DocumentClient();
module.exports.create = (event, context, callback) => {
  const timestamp = new Date().getTime();
  const data = JSON.parse(event.body);
  if (typeof data.text !== 'string') {
    console.error('Validation Failed');
    callback(null, {
      statusCode: 400,
      headers: { 'Content-Type': 'text/plain' },
      body: 'Couldn\'t create the task item.',
    });
    return;
  }
  const params = {
    TableName: process.env.DYNAMODB_TABLE,
    Item: {
      id: uuid.v1(),
      text: data.text,
      checked: false,
      createdAt: timestamp,
      updatedAt: timestamp,
    },
  };
  // write the task to the database
  dynamoDb.put(params, (error) => {
    // handle potential errors
    if (error) {
      console.error(error);
      callback(null, {
        statusCode: error.statusCode || 501,
        headers: { 'Content-Type': 'text/plain' },
        body: 'Couldn\'t create the task item.',
      });
      return;
    }
    // create a response
    const response = {
      statusCode: 200,
      body: JSON.stringify(params.Item),
    };
    callback(null, response);
  });
};
  1. Add the following content to src\mytasks\delete.js:
'use strict';
const AWS = require('AWS-sdk'); // eslint-disable-line import/no-extraneous-dependencies
const dynamoDb = new AWS.DynamoDB.DocumentClient();
module.exports.delete = (event, context, callback) => {
  const params = {
    TableName: process.env.DYNAMODB_TABLE,
    Key: {
      id: event.pathParameters.id,
    },
  };
  // delete the task from the database
  dynamoDb.delete(params, (error) => {
    // handle potential errors
    if (error) {
      console.error(error);
      callback(null, {
        statusCode: error.statusCode || 501,
        headers: { 'Content-Type': 'text/plain' },
        body: 'Couldn\'t remove the task item.',
      });
      return;
    }
    // create a response
    const response = {
statusCode: 200,
      body: JSON.stringify({}),
    };
    callback(null, response);
  });
};
  1. Add the following content to src\mytasks\get.js:
'use strict';
const AWS = require('AWS-sdk'); // eslint-disable-line import/no-extraneous-dependencies
const dynamoDb = new AWS.DynamoDB.DocumentClient();
module.exports.get = (event, context, callback) => {
  const params = {
    TableName: process.env.DYNAMODB_TABLE,
    Key: {
      id: event.pathParameters.id,
    },
  };
  // fetch task from the database
  dynamoDb.get(params, (error, result) => {
    // handle potential errors
if (error) {
      console.error(error);
      callback(null, {
        statusCode: error.statusCode || 501,
        headers: { 'Content-Type': 'text/plain' },
        body: 'Couldn\'t fetch the task item.',
      });
      return;
    }
    // create a response
    const response = {
      statusCode: 200,
      body: JSON.stringify(result.Item),
    };
    callback(null, response);
  });
};
  1. Add the following content to src\mytasks\list.js:
'use strict';
const AWS = require('AWS-sdk'); // eslint-disable-line import/no-extraneous-dependencies
const dynamoDb = new AWS.DynamoDB.DocumentClient();
const params = {
  TableName: process.env.DYNAMODB_TABLE,
};
module.exports.list = (event, context, callback) => {
  // fetch all tasks from the database
  dynamoDb.scan(params, (error, result) => {
    // handle potential errors
    if (error) {
      console.error(error);
      callback(null, {
        statusCode: error.statusCode || 501,
        headers: { 'Content-Type': 'text/plain' },
        body: 'Couldn\'t fetch the tasks.',
      });
      return;
    }
    // create a response
    const response = {
      statusCode: 200,
      body: JSON.stringify(result.Items),
    };
    callback(null, response);
  });
};
  1. Add the following content to src\mytasks\update.js:
'use strict';
const AWS = require('AWS-sdk'); // eslint-disable-line import/no-extraneous-dependencies
const dynamoDb = new AWS.DynamoDB.DocumentClient();
module.exports.update = (event, context, callback) => {
  const timestamp = new Date().getTime();
  const data = JSON.parse(event.body);
  // validation
  if (typeof data.text !== 'string' || typeof data.checked !== 'boolean') {
    console.error('Validation Failed');
    callback(null, {
      statusCode: 400,
      headers: { 'Content-Type': 'text/plain' },
      body: 'Couldn\'t update the task item.',
    });
    return;
  }
  const params = {
    TableName: process.env.DYNAMODB_TABLE,
    Key: {
id: event.pathParameters.id,
    },
    ExpressionAttributeNames: {
      '#task_text': 'text',
    },
    ExpressionAttributeValues: {
      ':text': data.text,
      ':checked': data.checked,
      ':updatedAt': timestamp,    },
    UpdateExpression: 'SET #task_text = :text, checked = :checked,
updatedAt = :updatedAt',
    ReturnValues: 'ALL_NEW',
  };
  // update the task in the database
  dynamoDb.update(params, (error, result) => {
    // handle potential errors
    if (error) {
      console.error(error);
      callback(null, {
        statusCode: error.statusCode || 501,
        headers: { 'Content-Type': 'text/plain' },
        body: 'Couldn\'t fetch the task item.',
      });
      return;
    }
// create a response
    const response = {
      statusCode: 200,
      body: JSON.stringify(result.Attributes),
    };
    callback(null, response);
  });
};

Now we will create test cases to unit test the code that we created. We will be using Mocha for unit testing and run the APIs again. Let's create a file called data in the test folder, as shown in the following screenshot. This will have the JSON data that the unit test will run on: 

$ mkdir test/data
  1. Next, let's add the test/createDelete.js file, which will create DynamoDB data and delete it, once the test is complete, as shown in the following code:
var assert = require('assert');
var request = require('request');
var fs = require('fs');
describe('Create, Delete', function() {
            this.timeout(5000);
    it('should create a new Task, & delete it', function(done) {
                        // Build and log the path
                        var path = "https://" + process.env.TASKS_ENDPOINT + "/mytasks";
                        // Fetch the comparison payload
                        require.extensions['.txt'] = function (module, filename) {
                            module.exports = fs.readFileSync(filename, 'utf8');
                        };
                        var desiredPayload = require("./data/newTask1.json");
                        // Create the new Task
                        var options = {'url' : path, 'form': JSON.stringify(desiredPayload)};
                        request.post(options, function (err, res, body){
                                    if(err){
                                                throw new Error("Create call failed: " + err);
                                    }
                                    assert.equal(200, res.statusCode, "Create Status Code != 200 (" + res.statusCode + ")");
                                    var task = JSON.parse(res.body);
                                    // Now delete the task
                                    var deletePath = path + "/" +
task.id;
                                   request.del(deletePath, function (err, res, body){
                                                if(err){
                                                            throw new Error("Delete call failed: " + err);
                                                }
                                                assert.equal(200, res.statusCode, "Delete Status Code != 200 (" + res.statusCode + ")");
                                                done();
                                    });
                        });
    });
});
  1. Now add the test/createListDelete.js file, which will create the DynamoDB data, list it, and then delete it once the test is complete, as shown in the following code:
var assert = require('assert');
var request = require('request');
var fs = require('fs');
describe('Create, List, Delete', function() {
            this.timeout(5000);
    it('should create a new task, list it, & delete it', function(done) {
                        // Build and log the path
----
                        // Fetch the comparison payload
                        require.extensions['.txt'] = function (module, filename) {
----
                        // Create the new Task
                        var options = {'url' : path, 'form': JSON.stringify(desiredPayload)};
                        request.post(options, function (err, res, body){
                                    if(err){
                                                throw new Error("Create call failed: " + err);
                                    }
                                    assert.equal(200, res.statusCode, "Create Status Code != 200 (" + res.statusCode + ")");
// Read the list, see if the new item is there at the end
                                    request.get(path, function (err,
res, body){
                                                if(err){
                                                            throw new Error("List call failed: " + err);
                                                }
                                                assert.equal(200, res.statusCode, "List Status Code != 200 (" + res.statusCode + ")");
                                                var taskList = JSON.parse(res.body);
                                                if(taskList[taskList.length-1].text = desiredPayload.text)     {
                                                            // Item found, delete it
-----
                                                                        assert.equal(200, res.statusCode, "Delete Status Code != 200 (" + res.statusCode + ")");
                                                                        done();
                                                            });
                                                } else {
                                                            // Item not found, fail test
                                                            assert.equal(true, false, "New item not found in list.");
                                                            done();
                                                }
});
                        });
    });
});
  1. Let's add the test/createReadDelete.js file, which will create the DynamoDB data, read it, and then delete it once the test is complete, as shown in the following code:
var assert = require('assert');
var request = require('request');
var fs = require('fs');
describe('Create, Read, Delete', function() {
            this.timeout(5000);
    it('should create a new Todo, read it, & delete it', function(done) {
                        // Build and log the path
                        var path = "https://" + process.env.TASKS_ENDPOINT + "/mytasks";
                        // Fetch the comparison payload
                        require.extensions['.txt'] = function (module, filename) {
                            module.exports = fs.readFileSync(filename, 'utf8');
                        };
                        var desiredPayload = require("./data/newTask1.json");
                        // Create the new todo
                        var options = {'url' : path, 'form': JSON.stringify(desiredPayload)};
                        request.post(options, function (err, res, body){
if(err){
                                                throw new Error("Create call failed: " + err);
                                    }
                                    assert.equal(200, res.statusCode, "Create Status Code != 200 (" + res.statusCode + ")");
                                    var todo = JSON.parse(res.body);
                                    // Read the item
                                    var specificPath = path + "/" + todo.id;
                                    request.get(path, function (err, res, body){
                                                if(err){
                                                            throw new Error("Read call failed: " + err);
                                                }
                                                assert.equal(200,
res.statusCode, "Read Status Code != 200 (" + res.statusCode + ")");
                                                var todoList = JSON.parse(res.body);
                                                if(todoList.text = desiredPayload.text)          

                                                            // Item found, delete it
                                                            request.del(specificPath, function (err, res, body){
                                                                        if(err){
                                                                                    throw new Error("Delete call failed: " + err);
                                                                        }
                                                                        assert.equal(200, res.statusCode, "Delete Status Code != 200 (" + res.statusCode + ")");
                                                                        done();
                                                            });
} else {
                                                            // Item not found, fail test
                                                            assert.equal(true, false, "New item not found in list.");
                                                            done();
                                                }
                                    });
                        });
    });
});

Now we will create two test data files—newTask1.json and newTask2.json—that can be used for unit testing.

  1. Let's create data/newTask1.json using the aforementioned data, as follows:
{ "text": "Learn Serverless" }
  1. Add the following JSON data to data/newTask2:
{ "text": "Test Serverless" }

The project folder should now look like the following screenshot:

We need to create a repository on Git to push all the code that we created previously so that we can set up CI to the serverless project. I am assuming that Git has already been installed on the local server and that the Git repository is already in place. In my case, I have the following Git repository set up. I will git clone, add files and folders, and then push everything to the Git repository:

$ git clone https://github.com/shzshi/AWS-lambda-dynamodb-mytasks.git

There will be a folder with the name AWS-lambda-dynamodb-mytasks. Go into that directory, copy all the files that we created earlier, and then push it to the repository, as shown in the following code:

$ git add .
$ git commit –m “my first commit”
$ git push origin master
上一章目录下一章