Migrating Your Molecular Project to TS

What is Moleculer???

Moleculer JS is a NodeJS framework for writing micro-services; using which we can write an application in a containerized way using the micro-service design pattern to scale parts of our applications pretty easily and quickly. You can find more about moleculer in its official documentation at the link here.

Since molecular is by default available in JS mode, we can extend its support to TS as well as it includes the minimum required TS definitions for all parts of the core concepts like ServiceBroker, Context etc, in their primary package. This means @types/molecular is not required if we opt to write our microservices in TypeScript for better development. This concept is going to become clearer as we proceed further.

Steps for Migration:

There are certain things which we should keep in mind before moving forward with the migration. It is important to note that that it's not just about renaming the files to .ts from .js and adding Type Definitions to it. We might be coming across situations where our code could be a buggy by accessing keys of undefined/null objects before checking its existence. So, we will be required to fix that part of the code while adding Type Definitions for the methods, classes, etc. We will begin by following the below steps:

Adding TS to the project and basic configuration of the same.
Updating package.json file to include TS with Molecular Runner.
Identifying the service's dependencies, and start migrating the services one by one.

Step 1: Adding TS to project and basic configuration of the same

This is probably the simplest step of all, we just need to run the below commands and TS is added to the project:

Adding typescript with the command: npm install typescript ts-node

Next, we shall install ts-node for development purposes as it lets us run TS directly from the terminal. and also allows us to interchangeably import from JS in TS and vice-versa, and we don't need to worry about running tsc in watch mode, and that part of the configuration.

Initialise tsconfig.json file with the command: tsc --init or we can directly create a tsconfig.json file at the root of our project with the below code:

{
  "compilerOptions":{ 
    "incremental":true,
    "target":"es6",
    "module":"commonjs"
    "allowJS":true, // since we can't migrate whole project at once
    "sourceMap":true,
    "outDir":"./dist",
    "preserveConstEnums":true,
    "pretty":true,
    "noImplicitAny":true,
    "moduleResolution":true,
    "allowSyntheticDefaultImports":true,
    "esModuleInterop":true,
    "skipLibCheck":true,
    "resolveJsonModule":true, // we might be storing configs in json file
   },
   "include":["./**/*"],
   "exclude":["node_modules/**/*", "test"]
}

At this point, Typescript will be successfully added to the project.

Step 2: Updating package.json file to include TS with Molecular Runner

Now, the next thing would be to update how we run our project via the Molecular Runner using the npm run dev command for development purposes.

By default the dev script in package.json files runs the molecular-runner with --repl and --hot flag. We can also extend it to do something before starting the molecular service, i.e. running custom migrations, providing initial seed data or setting up some Node variables. The dev script in the file will look like the following code at this point:

{
  ...
  "scripts": {
     ...
     "dev": "node preSetup.js && moleculer-runner --repl --hot services/**/*.service.js",
     ...
  },
  ...
}

So, in this case, we would be replacing this with the below command:

"dev": "ts-node preSetup.ts && ts-node ./node_modules/moleculer/bin/moleculer-runner.js --config moleculer.config.ts --repl --hot services/**/*.service/index.js services/**/*.service/index.ts"

In case we aren't doing anything before the Molecular Runner starts, we can leave the part before && in the above script line.

Now, let's break down is happening behind the scene:
ts-node preSetup.ts : This would require us to add typings to our existing preSetup script files, and add @types/<deps> for the packages being used in there. we might not need to add @types/<deps> if the package itself includes the type definition of the same.
Running molecular-runner using its full path from node_modules/ following the ts-node command, will allow us to use JS and TS in combination with each other, and won't throw any error if we are importing into JS files from TS files. Finally, having both .js and .ts extensions in services file pattern informs the molecular-runner to treat both files as molecular micro-services. Also, we would be required to change the extension of the molecular.config.js file to the molecular.config.ts file and add the typings of { BrokerOptions, Errors, MetricRegistry } wherever required.

This is the setup required for development and running the Molecular project and works just fine in the prod as well if we are using the npm run dev script in DockerFile for running the app.

If, some of us are wondering how to achieve the same with tsc, below are the steps required to be followed:

Let's say if we are running our prod app using npm start which starts molecular-runner with default options with explicitly specifying other configs. We can use tsc to first convert our ts files to js files, and then run the same using simple node command. Since tsc first builds the project and adds the files in the outputDir option specified in the tsconfig.json file, assuming this value is set to dist in our case, we would be required to firstly clear out the exiting build, then copy other asset files like html/CSS/PDF etc files used in the project and lastly,Copy other asset files like html/CSS/PDF etc files used in the project.

So, in order to do this, we will first install dependencies required for Step A and B using the below command:

npm install rimraf copyfiles

And then add the below scripts in our package.json file:

{
   ...
   "scripts":{
      ...
      "clean":"rimraf dist/",
      "copy-files": "copyfiles -e \"./node_modules/**/*.*\" ./**/*.html ./**/*.css dist/",
      "build": "npm run clean && tsc && npm run copy-files",
      "start":"npm run build && molecular-runner",
      ...
   },
   ...
}

NOTE: in script line "copy-files" we are required to ensure the folder structure and where are we copying the files to. we can use the various options available in their official documentation to achieve the end goal.

At this point, we have our Molecular project supporting both JS and TS files to contain the various services.

Step 3: Identifying the service's dependencies, and start migrating the services one by one

We have done a lot of configuration till this point and yet no code 😅. Now, let's start the main process of migration of services for the end result:

In order to proceed with a smooth transition from JS to TS, we first need to identify the order in which we should proceed with migration so that the rework is minimum. For this, we can create a dependency graph of our services using the metadata available to run our micro-services using molecular-runner, as this starts the services in order of their dependencies. We can use this data to pick up the first service which we will start with.

Now, once we have identified the services, we will check if the service is using DB in it, if so, we need to install the type definition for the package/ORM, whichever we are using, to connect to our DB and add typings for the modals at whatever location we prefer. After ensuring that the service was using the database, and after we have added the database related typings, we can proceed with the rest of the steps.
First rename the service file to .ts and wrap the service schema object in a class using the below syntax:

import { Service, ServiceBroker } from "moleculer";
export default class GreeterService extends Service {

    public constructor(public broker: ServiceBroker) {
        super(broker);
        this.parseServiceSchema({
            name: "auth",
            mixins: [],
            hooks: {},
            settings: {},
            actions: {},
            methods: {}
        });
     }
}

At this point, we have a skeleton of how the service would look after the migration. The classes Service and ServiceBroker are provided by molecular to write services in a class-like manner. which has the parseServiceSchema method available in order to get the service data from the service schema object which we generally export from the service for molecular-runner to instantiate a service.

Now, the core part of a service is its actions, and actions generally have the following syntax:

"actionName": {
  "params": {
   // params details
  },
  async handler(ctx) {
    // do some processing
    return result;
  }
}

Our next step would be to migrating this action to TS-like syntax. We know that the context passed to action has params and meta keys attached to it which contains the params to pass to this action and metadata associated with the call respectively. We generally design the schema of the same, i.e. what keys will be available and type of the key. Finally, we do some async/sync processing and returns the data from the function.

In order to let the TS compiler know of the same, we would be required to add the typings for all these things. We need to add typings for ContextParams, ContextMeta, ActionReturnType`.

Now, we have to create a typings.ts file wherever we see fit and add the typings for these 3 fields using the below naming convention:

I<ActionName>Params, I<ServiceName>Meta, and I<ActionName>Response

So, for example, if we have a GreeterService with the action greetUser in it, the typings would be as follows: IGreetUserParams, IGreeterServiceMeta, and IGreetUserResponse.

To see how this is done:

interface IGreetUserParams {
   name: string;
}
interface IGreeterServiceMeta {
   currentUser: UserDetails; // this can be a UserDetail Modal available in db
}
interface IGreetUserResponse {
   welcomeMessage: string;
}

Now, its time to add this to the action that we have. Since, the context passed to each action is provided by Molecular, it provides the type definition of the same as well, for example Context<P = unknown, M extends object = {}>. We simply need to use this with our action definition, as shown below:

import { ActionParams, Context, Service } from 'molecular';
export default class GreeterService extends Service {
... initial setup code
// somewhere at constructor -> parseServiceSchema -> 
// actions
"greetUser": {
  "params": {
     name: string;
  } as ActionParams,
  async handler(ctx: Context<IGreetUserParams, IGreeterServiceMeta>) : Promise<IGreetUserResponse> {
    console.log("User Requested for Greeting: ", ctx.params.name);
    return `Greetings from Greeter Team, Hello ${ctx.params.name}`;
  }
}
... other code
}

Here are few things to notice, it's generally up to us how we attach the meta to our request, i.e. at the service level or at the app level, on the basis of which we can follow the naming convention.

Secondly, we have added extra typecasting for params using as ActionParams syntax. this is required when sometimes Molecular is not able to identify the action params.

And finally, we have wrapped the action response type in a Promise as the handler is an async function which is generally resolved after performing some DB updates and all. It is important to note that the return type of ctx.call is a Promise of type T i.e. Promise<T>

Using the process shown above, we will add the typings for our other actions in the service. These actions will be used for calling methods, if applicable. Therefore, we would be required to add typings for the method signature of various available methods in the service by simply adding its expected params list and the return type. These actions will be using DB modals and querying data from DB, for which we have already added the typings. Additionally, it can also be used to explicitly add the typings if the IDE is not able to identify itself.

Also, there are cases where we call a service from another service so it is important to add the topics for the service that is being called initially. Considering we only know that the data returned from this action call would be of this particular type, in order to pass on this information to TS, we would add the return type, and expected param of that action call using below syntax. Let's assume we are calling our greetUser from a different service and we know that the greetUser expected IGreetUserParams and returns IGreetUserResponse, we will add the typings for this in the below manner:

const greetingMessage = ctx.call<IGreetUserResponse, IGreetUserParams>("greeter.greetUser", params);

Since Molecular itself provides the typings for the Context class, it has also provided the syntax for the call function as well in this definition:

call<T, P>(actionName: string, params: P, opts?: CallingOptions): Promise<T>;

To conclude

That's it. We have successfully added the typings for our DB Modals, Service, Actions, Methods, inter-service calls, etc. Similarly, we need to repeat this process for all of the actions and services that we have in our project. There might be cases which we have missed in this guide, but don't worry, we can always look into the type definition provided by the molecular and check what needs to be added in order to run that block of code if it is specific to Molecular.

Thank you for reading this guide, I hope you find it useful and are confident on how you should proceed with migrating your existing Molecular project to TypeScript.

Happy Coding!!!