HOW TO USE REQUEST MODULE

In any application framework one basic need is to be able to make REST calls over HTTP/HTTPS, they are widely used and very useful in cases of API interactions and web scraping. We can use the Node.js core modules http or https to perform these calls but that that can be a cumbersome process and the Request module makes it simple to perform these HTTP requests. In this tutorial we will learn about how to get started with the request module to perform HTTP calls.

Installing Request Module

Request is a third party module, hence in order to install the request module, simply run the following command in your project directory:

> npm install request

If you’re unaware, npm is a package manager, which provides a central repository for custom open source modules for Node.js and JavaScript. npm makes it simple to manage modules versions and distribution. We used the npm install command to install the required module in our project.

Getting started with the Request Module

The Request module is built in a way that it makes it very simple to make HTTP(S) calls. It also follows redirects by default, however that is configurable. In this section we will use shorthand notations of the request module to get our job done.

Simple GET call over HTTP

Lets try to print the HTML off the modulus.io homepage by making a GET request to the modulus.io homepage URL:

//Load the request module
var request = require('request');

//Lets try to make a HTTP GET request to modulus.io's website.
request('http://www.modulus.io', function (error, response, body) {
    if (!error && response.statusCode == 200) {
        console.log(body); // Show the HTML for the Modulus homepage.
    }
});

In the above example we did not specify the GET method anywhere but by default the request module makes a GET request, if not specified explicitly. Alternatively, we can also use request.get, using request.get makes the code more readable.

The callback function contains three parameters:

1. Error: This is the first parameter of the callback function. It will be null in case there is no error or it will contain the error object with appropriate details. You should always check for the error before continuing to process the response.
2. Response: This is the second parameter in the callback function. This contains the http.IncomingMessage object which contains additional data about the http/https request i.e status code, headers, etc.
3. Body: This is the third parameter and contains the body of the http/https response. This is a string type containing the contents of the body if the response is in text format, the body is a buffer if the response data is in the binary / octet stream encoding, finally the body will be a JSON object if the response is in JSON encoding.

Simple GET call over HTTPS

For making HTTPS calls the request module automatically detects from the URL and routes your request via https module internally. For making HTTPS calls all you have to do is, in the UEL specify the protocol as the https. Below is an example of the HTTPS call:

//Load the request module
var request = require('request');

//Lets try to make a HTTPS GET request to modulus.io's website.
//All we did here to make HTTPS call is changed the `http` to `https` in URL.
request('https://modulus.io', function (error, response, body) {
    //Check for error
    if(error){
        return console.log('Error:', error);
    }

    //Check for right status code
    if(response.statusCode !== 200){
        return console.log('Invalid Status Code Returned:', response.statusCode);
    }

    //All is good. Print the body
    console.log(body); // Show the HTML for the Modulus homepage.

});

The Request module has some shortcut methods for making calls in common REST methods like POST, PUT or DELETE, these notations are as follows:

• request.post is for POST request.
• request.put is for PUT request.
• request.delete is for DELETE request.
• request.post is for GET request.
• request.patch is for PATCH request.

Some More Configurations

In the previous section we discussed on how to make simple request calls via quick shortcut methods. Here we will discuss another syntax where you can configure the request per your needs. In place of a string URL, the request module also optionally takes a config object where we can pass the desired configurations. The syntax is: request(optionsObject, callback). Lets see an example below:

//Load the request module
var request = require('request');

//Lets configure and request
request({
    url: 'http://modulus.io', //URL to hit
    qs: {from: 'blog example', time: +new Date()}, //Query string data
    method: 'GET', //Specify the method
    headers: { //We can define headers too
        'Content-Type': 'MyContentType',
        'Custom-Header': 'Custom Value'
    }
}, function(error, response, body){
    if(error) {
        console.log(error);
    } else {
        console.log(response.statusCode, body);
    }
});

In the above example you can see that we passed a configuration object with various configurations, there is a huge list of configurations, which you can use for your request. Here we just demonstrated how you pass headers and query data as well as define the HTTP method explicitly.

Posting a string body

In this example we will see how we can post a String or Buffer body using the request module. In this example we will use the body configuration property for posting the data.

//Load the request module
var request = require('request');

//Lets configure and request
request({
    url: 'https://modulus.io/contact/demo', //URL to hit
    qs: {from: 'blog example', time: +new Date()}, //Query string data
    method: 'POST',
    headers: {
        'Content-Type': 'MyContentType',
        'Custom-Header': 'Custom Value'
    },
    body: 'Hello Hello! String body!' //Set the body as a string
}, function(error, response, body){
    if(error) {
        console.log(error);
    } else {
        console.log(response.statusCode, body);
    }
});

Posting as a form

The request module has a helper configuration to post forms. You just need to pss the form data as an object to the form configuration property and it will automatically url-encode it and put it in the body with the desired application/x-www-form-urlencoded content type. Lets see an example below:

//Load the request module
var request = require('request');

//Lets configure and request
request({
    url: 'https://modulus.io/contact/demo', //URL to hit
    qs: {from: 'blog example', time: +new Date()}, //Query string data
    method: 'POST',
    //Lets post the following key/values as form
    form: {
        field1: 'data',
        field2: 'data'
    }
}, function(error, response, body){
    if(error) {
        console.log(error);
    } else {
        console.log(response.statusCode, body);
    }
});

Posting JSON data

If we have an object and we want to post it as the JSON data we can do that by passing that object in the json configuration property. Lets see an example below:

//Load the request module
var request = require('request');

//Lets configure and request
request({
    url: 'https://modulus.io/contact/demo', //URL to hit
    qs: {from: 'blog example', time: +new Date()}, //Query string data
    method: 'POST',
    //Lets post the following key/values as form
    json: {
        field1: 'data',
        field2: 'data'
    }
}, function(error, response, body){
    if(error) {
        console.log(error);
    } else {
        console.log(response.statusCode, body);
}
});

Streams

The Request module has a fantastic first citizen interface for Streams. Streams are like a continuous flow of data from one source to a target. Using streams can really save us memory as well as help us to work with large data. If possible use of streaming API is recommended. We will cover some very basic streaming based approaches here, but this is just a start. Streams have infinite possibilities. Lets consider an example; we want to send a request for an image to a third party server and stream the response to a file.

//Load the request module
var request = require('request');
//Load fs module
var fs = require('fs');

//Lets define a write stream for our destination file
var destination = fs.createWriteStream('./savedImage.png');

//Lets save the modulus logo now
request('https://my.modulus.io/img/modulus-logoSmall-gray20.png').pipe(destination);

Here all we are doing is piping the readStream from the request we made to the write stream of our destination file. It's that simple. We can also listen to error events as described below:

//Lets save the modulus logo now
request('https://my.modulus.io/img/modulus-logoSmall-gray20.png')
.pipe(destination)
.on('error', function(error){
    console.log(error);
});

Sending data from a file to a request via streams

We can also read data from a file and send it to a remote server via our request. Lets consider an example below:

//Load the request module
var request = require('request');
//Load fs module
var fs = require('fs');

//Lets define a read stream from our source file, it could be any JSON file.
var source = fs.createReadStream('./sampleData.json');

//Let’s send our data via POST request
source.pipe(request.post('https://modulus.io/contact/demo'));

In this example we created a read stream from our source file and piped it to the request.

Conclusion

In this tutorial we learned the basics of the request module. We saw how we can use shorthand methods, custom configurations, as well as the basics of the streaming API.