Download a file with TypeScript

This may apparently be a trivial thing to do. Well, it turns out it was not, at least for me. This is why I would like to share my experience with you, it may save someone several hours of fiddling.

As in the past posts, I will be making my HTTP calls via typed-rest-client. This library is again based on the plain NodeJs http.ClientRequest class. This also means that if you do not plan to use this library, you can still follow the method I’m suggesting.

Here is the full example of the code.

import fs = require("fs");
import { HttpClient } from "typed-rest-client/HttpClient";

async function run() {
    const client = new HttpClient("clientTest");
    const response = await client.get("https://some.server.of.mine.com/largeImage.png");
    const filePath = "C:\\temp\\downloadedFile.png";
    const file: NodeJS.WritableStream = fs.createWriteStream(filePath);
    
    if (response.message.statusCode !== 200) {
        const err: Error = new Error(`Unexpected HTTP response: ${response.message.statusCode}`);
        err["httpStatusCode"] = response.message.statusCode;
        throw err;
    }

    return new Promise((resolve, reject) => {
        file.on("error", (err) => reject(err));

        const stream = response.message.pipe(file);

        stream.on("close", () => {
            try { resolve(filePath); } catch (err) {
                reject(err);
            }
        });
    });
}

run();

Let’s check what I wrote here and why.

Initially, I do create an instance of the HttpClient class and pass in the user agent parameter (any string will do here). Then I do call a get method to fetch an URL. At this point, I’m ready to persist the response so I do create a write stream for a given path. Here you can improve this code, at example by looking for Content-Disposition header and if present get the filename out of it, etc. The choice is yours and my goal was to show you how to handle the streams in TypeScript.
Now the tricky part, where I lost plenty of time. We need to pipe the message as it is a readable stream to our writable stream. But the fact is that we need to wait until the close event is triggered. This is where you need to wrap this up in the promise and wait for it to complete. In my example, I also look up for the error event and in case I do reject the promise.

Believe it or not, considering my limited experience with JavaScript and TypeScript, I was not awaiting for those events and my code refused to work. I lost some time figuring things out and google was of no much help. As I couldn’t find any TypeScript specific examples, I decided, even if seems banal, to share this with you.

Please share your thoughts with me in case you think this can be improved, I would love to learn more about it.

Cheers

Node10 provider available for Agent v2.144.0

It’s been a while that developers of Azure DevOps build/release tasks have been stuck on NodeJs v6.10.3 (available since agent v2.117.0). In the past days, a new pre-release of the agent came out that supports NodeJs 10 runtime. This is a great news but a bit ‘under-advertised’.

Let’s see what it is all about.

Starting with version v2.144.0 a new provider, called Node10 is supported. It is still a pre-release, but I’m confident that soon we will get a proper release with this new provider available.

To start using it, your task needs to reference it in the following way. In your task.json file just specify under the execution node, instead of probably just Node, Node10.

Example:

"execution": {
        "Node10": {
            "target": "task.js",
            "argumentFormat": ""
        }
    }

This means that in this case, your task implementation will run on NodeJs v10.13.0.
You are now free to use the Node 10 meanwhile if you are developing in TypeScript, then you can target ES2018 in this case. And if you are using TypeScript 3.2, some new features like BigInt may become available (by adding esnext.bigint to the lib setting in your compiler options).

Also do not forget to set in your task the “minimumAgentVersion” to:

"minimumAgentVersion": "2.144.0"

Cheers

Uploading XL Deploy DAR package via TypeScript

Another day, another language, another challenge

In past I already wrote about multipart/form-data requests that are used to upload files. It was about PowerShell and leveraging .Net libraries to achieve this task. Now it is the turn of TypeScript.

I had a need to implement the file upload to XL Deploy which requires multipart/form-data standard in order to do so. I wrote about the same thing in the past, using PowerShell to upload DAR package to XL deploy. This time, however, I needed to use NodeJs/TypeScript.

This is not meant to be a guide about the TypeScript, I do suppose you already have some knowledge about it. I just would like to show you how did I achieve it, hoping to help others not to go through the same discovery process that brought me to result.

First of all, I will be making my HTTP calls via typed-rest-client. This library is again based on the plain NodeJs http.ClientRequest class. This also means that if you do not plan to use this library, you can still follow the method I’m suggesting.
Creating the multipartform-data message structure and adding the necessary headers will be done with form-data package. It is one of the most popular libraries used to create “multipart/form-data” streams. Yes, I just mentioned the keyword, streams, and yes, we are going to use streams to achieve this and allow us not to saturate resources on our client host in case of large files upload.

Enough talking now, let’s see some code.

Sending multipartform-data messages in Typescript

Before we start, make sure that you install the following packages:

npm install typed-rest-client
npm install form-data
npm install @types/form-data

This is all we need. Note I also installed typings for the form-data library so that we can comfortably use it in TypeScript and make sure that “typed-rest-client” library is at least of version 1.0.11.

Code wise, first of all, we need to create an instance of our client.

import { BasicCredentialHandler } from "typed-rest-client/Handlers";
import { RestClient } from "typed-rest-client/RestClient";

async function run() {
    const requestOptions = { ignoreSslError: true };
    const authHandler = [new BasicCredentialHandler("user", "password")];

    const baseUrl: string = "https://myXLServer:4516";

    const client = new RestClient("myRestClient", baseUrl, authHandler, requestOptions);
}

I will skip commenting on the necessary imports and quickly analyze the remaining code.

I need to create request options and set the ignoreSslError property. This is so to allow my self-signed certificate to be accepted.
Then I do create a basic authentication handler and pass in the requested username and password. Once I have all of the necessary, I create an instance of the RestClient.

You spotted well, it is a RestClient and above I talked about the HttpClient. Do not wary, it is a wrapper around it, helping me to deserialize the response body, verify the status code, etc.

Let’s now prepare our form data.

...
import FormData from "form-data";
import fs from "fs";

async function run() {
	...
	const formData = new FormData();
	formData.append("fileData", fs.createReadStream("C:\\path\\to\\myfile.dar"));
}

We need a couple of extra imports and once that is sorted out, we just do create an instance of the FormData class. Once we have it, we will call the append method, pass in the file name and the stream that points to my file of choice. In order to get my file that is on the disk, I’m using createReadStream function from fs library which is a very common way to setup a stream.

At this point, we are ready to make our HTTP call.

async function run() {
	...
	const response = await client.uploadStream(
		"POST", `deployit/package/upload/myfile.dar`,
		formData,
		{ additionalHeaders: formData.getHeaders() });

	console.log(response.result.id);
}

As you can see, we are invoking the upload stream method from the rest client and passing in the following parameters.
HTTP method to use, POST in our case (XL Deploy), second, rest resource URL that needs to be triggered. Bear in mind that actual URL will be composed with the base you passed in the constructor of the RestClient. Then, the stream containing the body. This is going to be the instance of our FormData class, which is of type stream, and as the fourth parameter, we need to pass the additional headers. The additional headers we are specifying are overriding the content-type as for multipart/form-data it needs to be set to multipart/form-data and contains the correct boundary value. That’s what getHeaders will do, return the necessary content-type header with the necessary correct boundary value.

Once the call has been made, the upload of the file will start. As the response from XL Deploy on a successful import we will receive a message in form of JSON where one of the fields do report the ID of the package, and that’s what I’m printing in the console on my last line.

This may be specific for XL Deploy, however, you can easily adapt this code for any other service where multipart/form-data upload is necessary.

Following the complete code sample.

import FormData from "form-data";
import fs from "fs";
import { BasicCredentialHandler } from "typed-rest-client/Handlers";
import { RestClient } from "typed-rest-client/RestClient";

async function run() {
    const requestOptions = { ignoreSslError: true };
    const authHandler = [new BasicCredentialHandler("user", "password")];

    const baseUrl: string = "https://myXLServer:4516";

    const client = new RestClient("myRestClient", baseUrl, authHandler, requestOptions);

    const formData = new FormData();
    formData.append("fileData", fs.createReadStream("C:\\path\\to\\myfile.dar"));

    const response = await client.uploadStream<any>(
        "POST", `deployit/package/upload/myfile.dar`,
        formData,
        { additionalHeaders: formData.getHeaders() });

    // tslint:disable-next-line:no-console
    console.log(response.result.id);
}

run();

Good luck!