Last updated: July 24, 2022

Build An Email Marketing and Tracking App using NestJS, Nodemailer, and Google APIs

Welcome to the email tracker tutorial Building An Email Tracking And Marketing App With NestJS. In this tutorial we`ll learn how to create a backend web application for email marketing that can track your recipients and campaign performance. This application will work just like some common and famous email automation apps such as Mailchimp, SendGrid, etc.

If you want to learn NestJs then this tutorial is for you. I have broken up the content into chapters for ease of reading and navigation. Please be aware that this tutorial will be really lengthy. Grab yourself a cup of coffee and then come back.

Use the Chapters table on the left to navigate back and forth to the topics. If you do not understand any concept then let me know by sending me a DM or email. I'll be glad to help.

So, without wasting further time, let`s begin!

Introduction to NestJs a Node.js framework

Nest is a framework created for building fast and scalable Node.Js applications. NestJs uses ExpressJs under the hood by default. If you have ever used the Express framework, then your knowledge will come in handy. However, no prior knowledge of any framework or language other than NodeJs is required to understand Nest.

I have learnt NodeJs by playing around with NestJs framework for a few months. I personally think that as a beginner, you can learn how the backend works by just getting your hands on Nest.

What this email tracker app can do?

This email tracker app can help you get insights on how your email marketing campaigns are performing. Find out whether the recipients are opening your emails or not.

Features that you`ll be able to develop with this NestJS tutorial in your app:

  • Add new subscribers, who can later be added as recipients to any campaign.
  • Filter out duplicate recipients
  • Filter out duplicate subscribers.
  • Automatically parse and inject content into html templates.
  • Track the unsubscribe-rate with unsubscribe option in emails.
  • Track the open-rate to get an insight on how many people are actually opening your email.
  • Track which recipient clicked any links in the email.

Getting started with NestJs Email Tracker

To get started quickly, clone the official TypeScript starter repository. Nest supports TypeScript, however, if you want to write with JavaScript then you can switch it easily. In this blog, we will focus on only TypeScript.

$ git clone https://github.com:nestjs/typescript-starter.git

Once cloned, install the necessary packages by running the following command and start the development server.

$ npm install

$ npm run start:dev

If you do not want to clone the repository then scaffold the project with Nest CLI. Run the following commands.

$ npm i -g @nestjs/cli

$ nest new project-name

All the finished code from this tutorial can be found in my GitHub Repository This project can also serve as a NestJs boilerplate with built in email marketing and other features. It is a fork of official typescript-starter. Give both of these a star if you like my work.

Controllers

To begin developing our email tracker application, it is important to understand Controllers. Controllers handle all the incoming requests to your app and return an appropriate response.

We'll be returning JSON responses with our application. In this project, we'll use DTOs or Data Transfer Objects to format and validate the incoming requests.

Let's create our first Controller for Subscribers. This controller will help us add or remove subscribers from the database. Later we'll use these subscribers as recipients for sending our email campaigns.

Let's begin by creating a new folder subscribers in the src directory. Now, create a new file __controller.ts and write the following code.

src/subscribers/__controller.ts

import { Controller, Get, Post, Delete } from '@nestjs/common';
import SubscriberService from './__service';

@Controller('subscribers') // you can name your controller anything and this will be used as an endpoint prefix.
export default class SubscriberController {
  constructor(private readonly sbrService: SubscriberService) {}

  @Get('/:id') // to call a subscriber with id from database
  getSubscriber() {}

  @Get() // call all subscribers
  getSubscribers() {}

  @Get('/duplicates') // show duplicate subscribers
  getDuplicates() {}

  @Post('/create') // create/add a new subscriber
  createSubscriber() {}

  @Delete('/:id') // delete a subscriber by id
  deleteSubscriber() {}
}

You may have noticed that I follow a different file naming style than the one mentioned in the official NestJs docs. It is your choice to use any naming convention as long as it is easier to read and navigate.

Moving forward, this controller will accept requests to create, read, update, and delete subscribers in our database. In the upcoming sections we`ll find out how to set up and connect your PostgreSQL database using TypeOrm.

In NestJs, we use decorators to tell Nest to create handlers for incoming requests. In the above code, we used @Get#40;#41;, @Post#40;#41;, and @Delete#40;#41;. These decorators will help us validate and route requests to the endpoint of our choice.

There is one more important thing to note here - the name of your Controller. All your controllers are going to have different names and they`ll act as a prefix for all the sub routes. For example, to create a new subscriber, you may use the endpoint /subscribers/create. Here, subscribers is a prefix or the name of Subscribers Controller

/_ services _/

Services

A service separates our business logic from controllers. This is where we write our code that tells the backend application what to do. It also helps us in keeping our code cleaner and easier to read.

To add a Service, create a new file __service.ts and write the following code in it.

src/subscribers/__service.ts

import {
  Injectable,
  InternalServerErrorException,
  Logger,
} from '@nestjs/common';
import SubscribersDto, { Subscriber } from './dto/subscribers.dto';
import { InjectRepository } from '@nestjs/typeorm';
import SubscriberEntity from './__entity';
import { Repository } from 'typeorm';
import { nanoid } from 'src/nanoid/nanoid';

@Injectable()
export default class SubscriberService {
  private readonly logger = new Logger(SubscriberService.name);
  constructor(
    @InjectRepository(SubscriberEntity)
    private subscriberRepository: Repository<SubscriberEntity>,
  ) {}

  createSubscribers(data: SubscribersDto) {
    const { tag, subscribers } = data; // destructuring the request body

    subscribers.forEach(async (sbs: Subscriber) => {
      const subscriber = this.subscriberRepository.create({
        id: \`sbs_\${nanoid()}\` , // assigning a unique id to the primary key
        email: sbs.email,
        name: sbs.name,
        tag, // category under which our subscriber falls into
      });

      try {
        await this.subscriberRepository.save(subscriber); // saves the subscriber into database
      } catch (error) {
        this.logger.error(error.code, error.stack); // throw and log errors if any
        throw new InternalServerErrorException();
      }
    });
    return { message: 'Subscribers added' };
  }
}

let's break down the above code and understand what is happening step by step.

First, an entity repository object is constructed and injected into our SubscribersService class. By using this constructor function, we can create new entities and save them in our database. We are using two packages; @nest/typeorm and typeorm. Install these packages by running the following command

$ npm i @nest/typeorm typeorm

Second, We`ve created a Logger instance before our constructor. This is a built-in text-based logger provided by Nest which is used for displaying exceptions and logs in our terminal. This is useful when we are debugging our code and tracing errors.

Third, we've created a function to save subscribers in our database. You may have noticed that there is an argument passed to this function with the name data. This is a DTO or Data Transfer Object that defines how data will be sent over the network. You can validate your incoming requests by using DTOs in your app. In this tutorial, we'll be using a lot of them.

To use a DTO, create a new file inside the folder dtos with name subscribers.dto.ts and write the following code in it.

src/subscribers/dtos/subscribers.dto.ts

import { IsNotEmpty, IsOptional, ValidateNested } from 'class-validator';

export class Subscriber {
  @IsNotEmpty()
  name: string;

  @IsNotEmpty()
  email: string;
}

export default class SubscribersDto {
  @ValidateNested()
  subscribers: Array<Subscriber>;

  @IsOptional()
  tag?: string;
}

We are using two new packages here to validate our incoming request. Go ahead and install these packages by running the following command.

$ npm i class-validator class-transformer

We will need to validate all our requests, hence, these two packages are absolutely necessary for this tutorial.

Fourth, In the createSubscribers function, we are looping over the subscribers array with forEach to create a new subscriber and assign it a primary key. You will find another function in the github repository inside the src/nanoid folder. We use this to generate a unique 24 character string for the primary key.

TypeOrm also provides a built-in uuid generator. However, in this tutorial we will stick with our own, creating a prettier and shorter nanoid.

Finally, we use a try and catch block to save the entity and log any exceptions, if they occur, and return a json response that tells our API consumer that subscribers are added.

It may sound simple and easy for now but there are some concepts that you may need to learn to understand how the http requests work. If you are a beginner, then don`t worry, this tutorial will cover the important concepts. However, I recommend that you do your own research after finishing this tutorial to further your understanding.

Entities

We'll also need to create an entity class to tell TypeOrm to create columns in our PostgreSQL database where our subscribers are going to be saved. Later, we'll learn how to run PostgreSQL in our local environment.

An entity class is a building block of an entity that represents a table with columns in a database and describes the relationship among its attributes or other entities.

Basically, an entity has multiple columns in a table, we assign attributes to an entity with their respective data types which gives us information about its characteristics.

Create an entity class with name __entity.ts in the same folder and write the following code in it.

src/subscribers/__entity.ts

import { Column, Entity, PrimaryColumn } from 'typeorm';

@Entity('subscribers')
export default class SubscriberEntity {
  @PrimaryColumn() // optional - replace with @PrimaryGeneratedColumn('uuid') if you want TypeOrm to automatically generate a uuid for you.
  id: string;

  @Column()
  email: string;

  @Column({ default: true })
  subscribed: boolean;

  @Column({ nullable: true })
  name: string;

  @Column({ default: 'general' })
  tag: string;
}

This is the standard entity creation process. Every entity you create, needs a primary key and attributes that you`ll need to define in a class.

You can assign default values to an attribute and TypeOrm will save your entity with these values if none is provided. For more information on attributes and data types, visit TypeORM github repository.

In the above code block, we defined id #40;string#41; for primary key, email #40;string#41;, subscribed #40;boolean#41; - this will be useful when we send campaigns, name #40;string#41; for the email template, and tag #40;string#41; for identifying subscriber category.

Modules

In NestJs, we use modules to organize and structure our application. The @Module#40;#41; decorator takes an object with properties; providers, controllers, imports, and exports.

Modules can be reused by other modules in your application, this means you can share them within different modules to execute the business logic of your app. You will soon understand how to share them to help you write less code in your app.

To add a module, create a new file with name \_\_module.ts in the subscribers folder and write the following code in it.

src/subscribers/__module.ts

import { Module } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm';
import SubscriberController from './__controller';
import SubscriberEntity from './__entity';
import SubscriberService from './__service';

@Module({
  imports: [TypeOrmModule.forFeature([SubscriberEntity])],
  providers: [SubscriberService],
  controllers: [SubscriberController],
  exports: [SubscriberService],
})
export default class SubscriberModule {}

Let's find out what is going on in the above code block. We've used the @Module#40;#41; decorator that Nest will use to inject code in the SubscriberModule class. In the imports property, we imported the SubscriberEntity class that was created earlier. This is done so that TypeOrm can identify which entity repository to use in the referenced module.

We will also need to import this class in our App module. Otherwise, Nest will ignore the code written in our subscribers folder. To import follow the code below.

src/__app.module.ts

import { Module } from '@nestjs/common';
import SubscriberModule from './subscribers/__module';
import { AppController } from './__app.controller';
import { AppService } from './__app.service';

@Module({
  imports: [SubscriberModule],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}

The App module does not need to export anything as it will only be used to import other modules. Whenever we create a new module in our application, we will need to import it in the App module. This gives our app a structure that Nest follows to run our code.

Basic structure of our email tracker app

For our email tracker app to work, we'll need to import the functions that we created in the SubscriberService class and return the promise as a response from our SubscriberController. Before we return promises, let's add four new functions with the following code.

src/subscribers/__service.ts

@Injectable()
export default class SubscriberService {

  //...

  async findDuplicateSubscribers() {
    const subscribers: Subscriber[] = await this.subscriberRepository.find();
    const alreadySeen = []; // for finding duplicates
    const duplicates = []; // for storing and displaying duplicates
    subscribers.forEach((sbs) =>
      alreadySeen[sbs.email]
        ? duplicates.push(sbs.email)
        : (alreadySeen[sbs.email] = true),
    );

    if (duplicates.length) return duplicates;
    return { message: 'No duplicates found' };
  }

  async getSubscriberByEmail(email: string) {
    const subscriber = await this.subscriberRepository.find({
      where: { email },
    });
    return subscriber;
  }

  async getSubscribers(page: number, tag: string) {
    const pageSize = 20;
    const [subscribers, count] = await this.subscriberRepository.findAndCount({
      skip: ((page ? page : 1) - 1) * 20, // if page number is provided then query will skip 20 * page_number items
      take: pageSize, // default page size - can be set to any number. less than 20 is recommended
      order: {
        name: 'ASC', // sort the response data by subscriber name in ascending order
      },
      where: {
        subscribed: true, // only returns subscribers who haven\`t unsubscribed from our campaigns
        tag, // to filter out only a specific category of subscribers
      },
    });
    return {
      object: 'subscribers',
      data: subscribers,
      total: count,
      limit: pageSize,
      page,
    };
  }

  async deleteSubscriber(id: string) {
    return this.subscriberRepository.delete(id);
  }
}

The code above is easy and self explanatory, however, I have also added comments to make things easier to understand. To learn more about how to use TypeORM, refer to the official TypeORM documentation.

Now, we have a SubscriberService class that is ready for use in the constructor function of our SubscriberController. Modify your existing SubscriberController with the following code and return appropriate responses for the routes.

src/subscribers/__controller.ts

import {
  Controller,
  Get,
  Post,
  Delete,
  Query,
  Param,
  Body,
} from '@nestjs/common';
import SubscribersDto from './dto/subscribers.dto';
import SubscriberService from './__service';

@Controller('subscribers') // you can name your controller anything and this will be used as an endpoint prefix.
export default class SubscriberController {
  constructor(private readonly subscriberService: SubscriberService) {}

  @Get() // call all subscribers
  getSubscribers(@Query() query: { page: string; tag: string }) {
    return this.subscriberService.getSubscribers(Number(query.page), query.tag);
  }

  @Get('/:email') // to call a subscriber with email from database
  getSubscriber(@Param('email') email: string) {
    return this.subscriberService.getSubscriberByEmail(email);
  }

  @Get('/duplicates') // show duplicate subscribers
  getDuplicates() {
    return this.subscriberService.findDuplicateSubscribers();
  }

  @Post('/create') // create a new subscriber
  createSubscriber(@Body() data: SubscribersDto) {
    return this.subscriberService.createSubscribers(data);
  }

  @Delete('/:id') // delete a subscriber by id
  deleteSubscriber(@Param('id') id: string) {
    return this.subscriberService.deleteSubscriber(id);
  }
}

Setting Up Relational Database With PostgreSQL and TypeORM using Amazon RDS

Right now, we have a basic structure for our code to run but we haven`t connected any database service where we can save our subscribers. Lets first create a database service and connect it to our app.

Keeping in mind the scalability and functionality of our email tracker app, we`ll not be using a local development environment. Instead, we are going to use a live hosted relational database service provided by Amazon RDS.

To create and test your Postgres database instance, you'll need an AWS account. We're going to use the free tier provided by AWS for creating and managing our Postgres database.

If you don't have an AWS account, then create one or sign in to Amazon Web Services.

Once logged in, use the services menu on the top left corner and select RDS from the Database service. Refer to the images below for a step-by-step guide on creating a new Postgres database instance.

AWS RDS dashboard setting up Postgres

Your AWS RDS dashboard will open looking like the following image.

AWS RDS dashboard setting up Postgres

In your AWS RDS dashboard, click on Create database button and move on to the next step. AWS will ask you to choose from Standard or Easy create method. Check the Standard create and choose PostgreSQL from the engine options.

AWS RDS dashboard setting up Postgres

Also, don't forget to select Free tier under the Templates group. AWS offers 750 hours of free database management. Anything above it will be billed on your account.

Next, enter a name for your database under the Settings group. Don't forget to leave this option. Also, choose a suitable password for your database that is easy to remember.

AWS RDS dashboard setting up Postgres

Under Settings, you`ll see an Instance Configuration group. Under this group, select db.t4g.micro from Burstable classes menu. This is the processing power and RAM that AWS RDS will assign to your database. You can also select any other basic class offered.

AWS RDS dashboard setting up Postgres

For the Storage group, keep all the default options and there is no need to change them.

Next, scroll to the Connectivity group and make sure that public access is allowed to this database by selecting Yes from the options under Public access.

AWS RDS dashboard setting up Postgres

One final option that you need to adjust is the database name under Additional Configuration. If no name is provided, then TypeOrm will not be able to connect to this database.

AWS RDS dashboard setting up Postgres

Create your database by clicking on the Create Database button. This will take AWS RDS a few minutes to create and configure your new Postgres database.

AWS RDS dashboard setting up Postgres

AWS RDS will generate an endpoint for your database that you`ll need for your email tracker app. Copy this endpoint along with your master username and password.

AWS RDS dashboard setting up Postgres

Configure your Postgres with TypeORM

To connect your newly created PostgreSQL in AWS RDS, install; pg and @nestjs/config package, and create a new database module.

$ npm i pg @nestjs/config

src/__db.module.ts

import { Module } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm';
import { ConfigModule, ConfigService } from '@nestjs/config';

@Module({
  imports: [
    TypeOrmModule.forRootAsync({
      imports: [ConfigModule],
      inject: [ConfigService],
      useFactory: (configService: ConfigService) => ({
        type: 'postgres',
        host: configService.get('POSTGRES_HOST'),
        port: configService.get('POSTGRES_PORT'),
        username: configService.get('POSTGRES_USERNAME'),
        password: configService.get('POSTGRES_PASSWORD'),
        database: configService.get('POSTGRES_DB'),
        autoLoadEntities: true,
        synchronize: true,
      }),
    }),
  ],
})
export class DatabaseModule {}

In the above code block, we are configuring TypeORM to use our database endpoint with the master username and password that AWS provided us earlier. Following the best code practices, we are going to save those details in a .env file.

src/__app.module.ts

//...

@Module({
  imports: [ConfigModule.forRoot({}), SubscriberModule, DatabaseModule],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}

Include ConfigModule.forRoot(#123;#125;) inside the import property of module object in App module class. This is required for NestJS to read our app's environment variables. Later, we'll use this ConfigModule to validate our environment variables.

Finally, create a .env file in the root directory of your email tracker app and then add the following details.

./.env

POSTGRES_HOST=email-tracker.xxxxxxxxxxx.rds.amazonaws.com
POSTGRES_PORT=5432
POSTGRES_USERNAME=postgres
POSTGRES_PASSWORD=postgres
POSTGRES_DB=postgres

Replace the values of variables in the .env file with your own. Don't use mine because I don't want to pay a big AWS bill. #128521;

We are now ready to test our endpoints for the Subscribers controller. Start the development server and if everything is good to go, Nest will show the following output in our terminal.

$ npm run start:dev

NestJs logs for email tracker app

Endpoint testing with insomnia

Insomnia is a free cross-platform application used for testing RESTful and GraphQL APIs. It is an easy to use API consuming tool that you can use with our email tracker app. There are other similar apps available such as Postman, however, for this tutorial we`ll use Insomnia. I personally love it and have been using it since I started learning backend development.

To begin testing the endpoints of our email tracker app, download and install Insomnia by visiting offical website of Insomnia API. Once installed, create a new request collection with name Email Tracker.

Testing Endpoints with Insomnia

We can add multiple subscribers by passing an array with subscriber name and email. All the subscribers will be added to our email tracker app`s database.

To check if the subscribers got added, send a Get request to the /subscribers endpoint. You`ll see the following response if everything is working.

{
	"object": "subscribers",
	"data": [
		{
			"id": "sbs_ibtklIB77QKsCAiLDkfUJSvB",
			"email": "johndoe@example.com",
			"subscribed": true,
			"name": "John Doe",
			"tag": "user"
		}
	],
	"total": 1,
	"limit": 20,
	"page": null
}

Configure nodemailer and google OAuth for sending emails in NestJs

We now have a working app that can save, retrieve, and delete subscribers in our database. To send emails to these subscribers, we`ll need to configure Google APIs for automated campaigns using Nodemailer.

Install the necessary packages for our email service using the following command.

$ npm i googleapis nodemailer juice html-to-text ejs

Required* - googleapis is a Node.js client library for using Google APIs. This is required for authorization and authentication with OAuth.

The juice package converts css properties of our html template into style attributes. Email clients, including Gmail, do not support a separate CSS file for html templates, hence, we are using juice.

Optional - html-to-text converts the html content into plain text. We are using this as fallback for our email templates. If our recipient is using an email client that does not support html, we will still be able to convey our message with plain text.

The last package that we need is ejs. This is a very popular and powerful template engine for JavaScript. We`ll need this to embed custom javascript code and template varibales in our html templates.

When ready, create the needed **module.ts,

**service.ts, and __controller.ts

files for our Email module in the email directory.

src/email/

**controller.ts **module.ts __service.ts

Create the necessary classes and functions in these files just like we created the SubscriberModule, SubscriberController, and SubscriberService class.

src/email/__service.ts

import {
  BadRequestException,
  Injectable,
  InternalServerErrorException,
  Logger,
} from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import { createTransport } from 'nodemailer';
import * as Mail from 'nodemailer/lib/mailer';
import { Auth, google } from 'googleapis';
import { existsSync, readFileSync } from 'fs';
import * as juice from 'juice';
import { render } from 'ejs';
import { htmlToText } from 'html-to-text';
import EmailData from './dtos/emailData.dto';

@Injectable()
export default class EmailService {
  private nodemailerTransport: Mail;
  private logger = new Logger('EmailService');
  private oauthClient: Auth.OAuth2Client;

  constructor(private readonly configService: ConfigService) {
    //   initialising nodemailer and gmail

    this.oauthClient = new google.auth.OAuth2(
      configService.get('GOOGLE_CLIENT_ID'),
      configService.get('GOOGLE_CLIENT_SECRET'),
      'https://developers.google.com/oauthplayground',
    );

    this.oauthClient.setCredentials({
      refresh_token: configService.get('REFRESH_TOKEN'),
    });

    const accessToken = this.oauthClient.getAccessToken();

    this.nodemailerTransport = createTransport({
      service: 'gmail',
      auth: {
        type: 'OAuth2',
        user: 'johndoe@example.com',
        clientId: configService.get('GOOGLE_CLIENT_ID'),
        clientSecret: configService.get('GOOGLE_CLIENT_SECRET'),
        refreshToken: configService.get('REFRESH_TOKEN'),
        accessToken,
      },
    });
  }

  async sendMail(options: Mail.Options) {
    try {
      const response = await this.nodemailerTransport.sendMail(options);
      this.nodemailerTransport.close();
      return response;
    } catch (error) {
      this.logger.error(error.code, error.stack);
      throw new InternalServerErrorException({
        message: "Nodemailer can't send email",
      });
    }
  }

  async sendMailWithTemplate({
    template: templateName,
    templateVars,
    ...restOfOptions
  }) {
    const templatePath = \`src/email/templates/\${templateName}.html\`;
    const options = {
      ...restOfOptions,
    };
    if (templateName && existsSync(templatePath)) {
      try {
        const template = readFileSync(templatePath, 'utf-8');
        const html = render(template, templateVars);
        const text = htmlToText(html);
        const htmlWithStylesInlined = juice(html);

        options.html = htmlWithStylesInlined;
        options.text = text;
        options.from = 'John Doe <johndoe@example.com>';
        options.replyTo = 'John Doe <johndoe@example.com>';

        return this.sendMail(options);
      } catch (error) {
        this.logger.error(error.stack, error.code);
      }
    }
  }

  async sendTestEmail(data: EmailData) {
    const mailOptions: any = { ...data };
    const response = await this.sendMailWithTemplate(mailOptions);
    if (!response) throw new BadRequestException({ message: 'Inavlid fields' });
    return response;
  }
}

A lot is happening with the above code. Let's find out what is going on. First, we imported the necessary packages and libraries that we installed earlier. Second, we've initialized nodemailer and OAuth2.

Gmail service needs an Access Token to connect with our application, this token is valid only for one hour. When this token expires, our application will automatically get a new Access Token by providing a refresh token to Gmail.

Third, we've created a sendMail function to create and send the email in plain text. To supply nodemailer an html email format, we created a new sendMailWithTemplate function.

The function sendMailWithTemplate takes three parameters; the name of template, template variables, and the default Mail options for nodemailer. We're going to put our HTML templates inside a folder and use that path in this function to read and parse it.

Finally, we created a function to send a test email for validating our application logic and code.

Remember to replace the from and to fields in the above code with your name and email address your on Gmail account.

Next, we are going to create and export an EmailController class with the following code.

src/email/__controller.ts

import { Body, Controller, Post } from '@nestjs/common';
import EmailData from './dtos/emailData.dto';
import EmailService from './__service';

@Controller('email')
export default class EmailController {
  constructor(private emailService: EmailService) {}

  @Post('/test')
  sendTestEmail(@Body() emailData: EmailData) {
    return this.emailService.sendTestEmail(emailData);
  }
}

To validate the request body of our /email/test route, we've also created a DTO EmailData. The code for EmatilData DTO is available below.

src/email/dtos/emailData.dto.ts

import { IsEmail, IsNotEmpty, IsOptional } from 'class-validator';

export default class EmailData {
  @IsNotEmpty()
  @IsEmail()
  to: string;

  @IsNotEmpty()
  subject: string;

  @IsNotEmpty({ message: 'Template not provided' })
  template: string;

  @IsOptional()
  templateVars?: any;
}

So far, we've implemented Subscribers and Email Send feature in our Email Tracker App built with NestJs. Now, let's obtain Google Client ID, Secret, and Refresh Token for our Email Send feature to work.

Obtain Google Client ID, Client Secret, and Refresh Token

The first thing we need to obtain the secret and token is a Google account. I will assume that you already have a Google account, let`s go ahead and create a new project in the Google Developer Console.

Setting up OAuth and obtaining Client ID and Secret from Google Console

Step 1

Create a new project in your Google Cloud Console account with the name email-tracker. Your project will be assigned an ID that can not be changed later, change it if you want to or keep it the same.

Setting up OAuth and obtaining Client ID and Secret from Google Console

Step 2

Go to the menu on the left and select Credentials under APIs and services.

Setting up OAuth and obtaining Client ID and Secret from Google Console

Step 3

You'll be able to obtain your OAuth Client ID when you click on the Create Credentials button at the top. Remember that you will also need to register your app for the OAuth consent screen. We'll do it in the next step.

Setting up OAuth and obtaining Client ID and Secret from Google Console

Step 4

To obtain our OAuth client ID we`ll need to configure the consent screen. Click on Configure consent Screen to move on to the next step.

Setting up OAuth and obtaining Client ID and Secret from Google Console

Step 5

Select External user type on the screen and then create your app. On the next sreen, you'll be asked to enter a name for your app. Use email-tracker for the name and fill the user's and developer's support email field with your own email address. You may leave all the other fields empty or as it is. Finish creating your app and publish it when prompted on the screen.

Setting up OAuth and obtaining Client ID and Secret from Google Console

Step 6

Now, go back to the Create OAuth Client ID screen and select the application type as Web Application. Give it a name, we are going to use email-tracker for this tutorial.

An important thing to note here is the Authorised redirect URIs group. We'll need a URI to make requests to our Gmail. Add https://developers.google.com/oauthplayground in this field and click on Create.

Setting up OAuth and obtaining Client ID and Secret from Google Console

Step 7

Copy your Google Client ID and Secret and save it in the .env file that we created earlier. Also, keep these handy, we will need both of these in the next steps.

./.env

//...
GOOGLE_CLIENT_ID=
GOOGLE_CLIENT_SECRET=

Setting up OAuth and obtaining Client ID and Secret from Google Console

Step 8

Now, go to the

OAuth playground

URL that we used as a redirect URI earlier. Next, click on the Cog icon on the top right and select User your own OAuth credentials. Paste both your Client ID and Client Secret in the appropriate fields.

On the left side, enter https://mail.google.com/ inside the Input your own scopes input field under Step 1 and Authorize your APIs. Google will ask you to sign in to the Email Tracker app. Select your account and sign in to see the following message.

Setting up OAuth and obtaining Client ID and Secret from Google Console

Next, click on Show Advanced and continue with Go to email-tracker#40;unsafe#41;. You`ll be prompted with the following window, click on Continue to get redirected to the OAuth playground.

Setting up OAuth and obtaining Client ID and Secret from Google Console

Go to the Step 2 Exchange authorization code for tokens and copy your Refresh Token.

Setting up OAuth and obtaining Client ID and Secret from Google Console

We will need this Refresh token for our email tracker app. Just like we added our Google Client ID and Secret, add REFRESH_TOKEN=lt;Your Refresh Tokengt; field in the .env file.

Our Email Tracker App is now able to send emails using Google OAuth2. In the next section, we`ll test sending emails with HTML templates.

Creating and parsing HTML email templates

In the github repository you'll find a folder with name templates inside the src/email/ directory. I have inlcuded a blank html template that you can edit and use with your own Email Tracker App. When you open this template from within your code editor and view it in your web browser, you'll see something like the image below.

HTML email template for sending emails with template

To send emails, we created an Email Send feature in our NestJs app earlier. Let`s send a test email using this email template and validate if everything is working as it is supposed to work.

Here, an important thing to note is the lt;,#37;, #x3D;, and gt; symbols enlosing a variable sbs_name. This is a template variable for our subscribers name. When we send and email or a campaign, our sendMailWithTemplate function will parse and inject this variable into the provided HTML template.

To send a test email, we'll use the /email/test route that we created earlier in our EmailController class. Open Insomnia and create a new POST request to send an email using this API route.

Sending test email with Insomnia using Nodemailer and Google OAuth

In the above image, we are sending a JSON body with the address of recipient, subject of email, the name of template, and template variables to our API route.

You see that in template variables, we provided the name of recipient, subject, preview text but kept pixel_loc, and unsbs_link blank. We'll use these variables in the upcoming section and create a new Tracking feature for our app to check if the recipient opened our email.

For now, to check if we've received the test email, open your email inbox. You'll see something like the picture below.

Checking to see if we have received email in our inbox

If you've received your test email, Congratulation! our Email Tracker App is working as expected. Now, in the upcoming sections, we will learn how to create Campaigns and then finally deploy our code on a serverless platform in a production environment.

Creating campaigns and adding recipients

In the previous chapter, we were able to parse and send our HTML template via email to our recipient. Now, we are going to create campaigns to send emails to multiple recipients. This will help us automate the email send feature without including multiple email addresses in the cc or bcc.

Campaign Module

Let's create a Campaign module and define our entity, service, and controller. The format for creating these files is going to be similar to what we did earlier. Follow the code below and then import your ControllerModule inside AppModule.

First, we'll need to create an entity class for our campaigns. Create the necessary attributes and assign them the appopriate data types as shown below.

src/campaigns/__entity.ts

import { Column, Entity, OneToMany, PrimaryColumn } from 'typeorm';
import RecipientEntity from 'src/recipients/__entity';

@Entity('campaigns')
export default class CampaignEntity {
  @PrimaryColumn()
  id: string;

  @Column()
  name: string;

  @Column()
  subject: string;

  @Column({ nullable: true })
  preview_text: string;

  @Column({ nullable: true })
  content: string;

  @Column()
  from: string;

  @Column()
  template: string;

  @OneToMany(() => RecipientEntity, (recipients) => recipients.campaign)
  recipients: RecipientEntity[];

  @Column({ nullable: true })
  last_sent: string;
}

In the above code, we created our CampaignEntity class with the following attributes #8212;

  • name - the name of the campaign for our reference only.
  • subject - the subject of email for our nodemailer email service.
  • preview_text - an optional paramter for displaying preview text in email clients.
  • content - an optional parameter for injecting our own custom HTML content that might be different for every recipient.
  • from - a required field for sending emails from your preferred or alias email address.
  • template - the name of template that is available in our template folder.
  • recipients - our campaign entity has one-to-many relationship with recipients. This will call all the recipients related to a campaign when used with query builder.
  • last_sent - for only our record to identify when this campaign was sent.

We now have our entity exported and ready to be used in the ControllerService class. Let's create a new file and write the following code in it.

src/campaigns/__service.ts

import {
  Injectable,
  Logger,
  InternalServerErrorException,
  NotFoundException,
} from '@nestjs/common';
import { InjectRepository } from '@nestjs/typeorm';
import { Repository } from 'typeorm';
import CampaignDto from './dto/campaign.dto';
import CampaignEntity from './__entity';
import { nanoid } from 'src/nanoid/nanoid';
import UpdateCampaignDto from './dto/updateCampaign.dto';

@Injectable()
export default class CampaignService {
  private logger = new Logger('CampaignService');
  constructor(
    @InjectRepository(CampaignEntity)
    private campaignRepository: Repository<CampaignEntity>,
  ) {}

  async createCampaign(data: CampaignDto) {
    const campaign = this.campaignRepository.create({
      id: \`cam_\${nanoid()}\`,
      name: data.name,
      template: data.template,
      subject: data.subject,
      from: data.from,
      content: data.content,
      preview_text: data.preview_text,
    });
    try {
      return await this.campaignRepository.save(campaign);
    } catch (error) {
      this.logger.error(error.code, error.stack);
      throw new InternalServerErrorException();
    }
  }

  async getCampaigns(page: number) {
    const pageSize = 20;
    const [campaigns, count] = await this.campaignRepository.findAndCount({
      skip: ((page ? page : 1) - 1) * 20,
      take: pageSize,
    });
    return {
      object: 'campaigns',
      data: campaigns,
      total: count,
      limit: pageSize,
      page,
    };
  }

  async getCampaignById(id: string) {
    const campaign = await this.campaignRepository.findOne({ where: { id } });
    if (!campaign)
      throw new NotFoundException({ message: 'No campaign found' });
    return campaign;
  }

  async updateCampaign(campaign_id: string, data: UpdateCampaignDto) {
    Object.keys(data).forEach((k) => data[k] == null && delete data[k]);
    const campaign = new CampaignEntity();
    campaign.id = campaign_id;
    Object.assign(campaign, data);
    await this.campaignRepository.save(campaign);
    return { message: 'Campaign updated!' };
  }

  async deleteCampaign(id: string) {
    return this.campaignRepository.delete(id);
  }
}

In the first function createCampaign, we are passing the CampaignDto as an argument to create a new Campaign object. A try and catch block is added to save this campaign into the database and throw any exceptions if they occur.

In the second function getCampaigns, we are using an optional query parameter page for pagination. This returns a campaigns object that displays atleast 20 items that are available in our database. You can also pass order_by as an optional parameter in the findAndCount method to sort the data.

In our third function getCampaignById, we call a single campaign from our database by providing the id paramter.

The fourth function is used for updating our campaign's attributes. We use the javascript Object type to map its keys, remove null values, and save the rest of the attributes by assigning them to our CampaignEntity object.

Lastly, we created a function to delete a campaign by passing its id to the delete method.

Now, with the CampaignService class created, let's create the controller and module for our Campaigns.

src/campaigns/__controller.ts

import {
  Body,
  Controller,
  Delete,
  Get,
  Param,
  Patch,
  Post,
  Query,
} from '@nestjs/common';
import CampaignDto from './dto/campaign.dto';
import UpdateCampaignDto from './dto/updateCampaign.dto';
import CampaignService from './__service';

@Controller('campaigns')
export default class CampaignController {
  constructor(private readonly campaignService: CampaignService) {}

  @Post('/create')
  createCampaign(@Body() data: CampaignDto) {
    return this.campaignService.createCampaign(data);
  }

  @Get()
  getCampaigns(@Query() query: { page: string }) {
    return this.campaignService.getCampaigns(Number(query.page));
  }

  @Get('/:id')
  getCampaignById(@Param('id') id: string) {
    return this.campaignService.getCampaignById(id);
  }

  @Delete('/:id')
  deleteCampaign(@Param('id') id: string) {
    return this.campaignService.deleteCampaign(id);
  }

  @Patch('/:id')
  updateCampaign(@Param('id') id: string, @Body() data: UpdateCampaignDto) {
    return this.campaignService.updateCampaign(id, data);
  }
}

Once the necessary routes are defined, create a CampaigModule class by writing the following code. We`ll later import it in our App module.

src/campaigns/__module.ts

import { Module } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm';
import RecipientModule from 'src/recipients/__module';
import SubscriberModule from 'src/subscribers/__module';
import CampaignController from './__controller';
import CampaignEntity from './__entity';
import CampaignService from './__service';

@Module({
  imports: [
    TypeOrmModule.forFeature([CampaignEntity]),
    RecipientModule,
    SubscriberModule,
  ],
  controllers: [CampaignController],
  providers: [CampaignService],
  exports: [CampaignService],
})
export default class CampaignModule {}

Recipient Module

You may have noticed that we imported RecipientModule and SubscriberModule inside imports array property of the module object. To keep our code clean and easy to read, we are going to create a separate module for our Recipients.

Create a RecipientEntity in a separate folder and use the code below to validate the relationships that we defined between CampaignEntity and RecipientEntity earlier.

src/recipients/__entity.ts

import { Column, Entity, ManyToOne, PrimaryColumn } from 'typeorm';
import CampaignEntity from 'src/campaigns/__entity';

@Entity('recipients')
export default class RecipientEntity {
  @PrimaryColumn()
  id: string;

  @Column()
  email: string;

  @Column({ nullable: true })
  name: string;

  @ManyToOne(() => CampaignEntity, (campaign) => campaign.recipients, {
    onDelete: 'CASCADE',
  })
  campaign: CampaignEntity;

  @Column({ default: 0 })
  opens: number;

  @Column({ nullable: true })
  lastseen: string;

  @Column({ nullable: true })
  lastseen_milsec: number;

  @Column({ default: false })
  email_sent: boolean;

  @Column({ nullable: true })
  sbs_id: string;

  @Column({ default: false })
  unsubscribed: boolean;

  @Column({ nullable: true })
  link_clicked: number;

  @Column({ nullable: true })
  ip: string;
}

There is one important thing to note in the above code and it is the many-to-one relationship. We used the onDelete: CASCADE #40;an optional paramter#41; to tell TypeOrm to delete the related entities if the parent entity is deleted.

We are doing this because we do not want to keep the receipients for which the campaign does not exist. If you still want to keep the recipient information then do not use this option.

Next, we'll create two new functions in the RecipientService and write the following code. Later, we'll call these functions in our CampaignService to add new recipeints to the campaign.

Basically, recipients are the subscribers that we created earlier. We add them to a campaign so that we can send emails and track its performance with our email tracker app.

src/recipients/__service.ts

import {
  Injectable,
  Logger,
  InternalServerErrorException,
} from '@nestjs/common';
import { InjectRepository } from '@nestjs/typeorm';
import RecipientEntity from './__entity';
import { getRepository, Repository } from 'typeorm';
import { nanoid } from 'src/nanoid/nanoid';
import SubscriberEntity from 'src/subscribers/__entity';

@Injectable()
export default class RecipientService {
  private logger = new Logger('RecipientService');
  constructor(
    @InjectRepository(RecipientEntity)
    private recipientRepository: Repository<RecipientEntity>,
  ) {}

  async getRecipientsByCampaignId(campaign_id: string) {
    const recipients = await this.recipientRepository.find({
      where: { campaign: { id: campaign_id } },
    });
    return recipients;
  }

  async createRecipients(
    subscriber: SubscriberEntity,
    campaign_id: string,
  ): Promise<RecipientEntity> {
    const recipient: RecipientEntity = this.recipientRepository.create({
      id: \`rec_\${nanoid()}\`,
      email: subscriber.email,
      name: subscriber.name,
      sbs_id: subscriber.id,
      campaign: { id: campaign_id },
    });

    try {
      return this.recipientRepository.save(recipient);
    } catch (error) {
      this.logger.error(error.code, error.stack);
      throw new InternalServerErrorException();
    }
  }

  async getRecipients(id: string, page: number) {
    const pageSize = 20;

    const [recipients, count] = await this.recipientRepository.findAndCount({
      where: { campaign: { id } },
      skip: ((page ? page : 1) - 1) * pageSize,
      take: pageSize,
    });

    return {
      object: 'recipients',
      data: recipients,
      total: count,
      limit: pageSize,
      page: !page ? 1 : page,
    };
  }

  async removeRecipient(id: string) {
    return this.recipientRepository.delete(id);
  }
}

Now, create a RecipientModule with the following code.

src/recipients/__module.ts

import { Module } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm';
import RecipientEntity from './__entity';
import RecipientService from './__service';

@Module({
  imports: [TypeOrmModule.forFeature([RecipientEntity])],
  providers: [RecipientService],
  exports: [RecipientService],
})
export default class RecipientModule {}

Finally, import both the SubscriberModule and the RecipientModule inside the CampaignModule. This allows us to share the services between different modules.

src/campaigns/__module.ts

//..

@Module({
  imports: [
    //...
    RecipientModule,
    SubscriberModule,
  ],
  //...
})
export default class CampaignModule {}

Also, don`t forget to import our CampaignModule in the App module.

src/__app.module.ts

//...
@Module({
  imports: [
    //...
    CampaignModule,
  ],
  //...
})
export class AppModule {}

We are now ready to add recipients to the campaign that we created. To start adding, update the /campaign/__service.ts file with the following code.

src/campaign/__service.ts

@Injectable()
export default class CampaignService {
  //...

  constructor(
    //...
    private recipientService: RecipientService,
    private subscriberService: SubscriberService,
  ) {}

  //...

  async addRecipientsByTag(data: RecipientsDto): Promise<{
    message: string;
    skipped: number;
    added: number;
    recipients: RecipientEntity[];
  }> {
    const { recipient_tag, campaign_id } = data;
    const subscribers: SubscriberEntity[] = await this.subscriberService.getSubscribersByTag(
      recipient_tag,
    );

    const recipients: RecipientEntity[] = await this.recipientService.getRecipientsByCampaignId(
      campaign_id,
    );

    let skipped: number = 0;
    const newRecipients: RecipientEntity[] = [];

    await Promise.all(
      subscribers.map(async (subscriber: SubscriberEntity) => {
        const alreadyAdded = recipients.find(
          (rec: RecipientEntity) => rec.email == subscriber.email,
        );
        if (alreadyAdded) {
          return skipped++;
        } else {
          const recipient: RecipientEntity =
            await this.recipientService.createRecipients(
              subscriber,
              campaign_id,
            );
          newRecipients.push(recipient);
          return recipient;
        }
      }),
    );

    return {
      message: 'Recipients added',
      skipped,
      added: newRecipients.length,
      recipients: newRecipients,
    };
  }
}

In the function above, we are calling all the subscribers registered with a specific tag from our database. In our case, we registered our subscriber with the user tag earlier. If subscribers are available with this tag, the SubscriberService will return those, if not, it will throw an error.

In the next step, we check to see if the recipients that we want to add are available in our campaign. This function skips the already added recipients and moves on to add the rest. We use the Promise.all#40;#41; method to resolve all the promises by returning the added or skipped recipients.

Finally, we return a response that shows our API consumer the number of recipients added, an array of recipients that we added, and the number of recipients that were skipped.

Before we add our recipients to this campaign, let`s add a new route in the CampaignController.

src/campaign/__controller.ts

@Controller('campaigns')
export default class CampaignController {
//...

  @Post('/recipients/add/tagged')
  addRecipients(@Body() data: RecipientsDto) {
    return this.campaignService.addRecipientsByTag(data);
  }
}

Adding Recipients

To add recipients to a campaign, we'll need the campaign's id. Let's make a GET request to display all our added campaigns.

Getting the id of our email campaign

After grabbing the id of campaign, create a new POST request to the /campaigns/recipients/add/ tagged route. Pass the recipient_tag and campaign_id in the body paramters to receive a response similar to the one in the image below.

Adding the recipents with tag 'user'

We now have a campaign with one recipient. To add more recipients, simply create new subscribers with a required tag. Then add those subscribers as recipients to the campaign with a simple POST request just like we did in the image above.

The addRecipient function that we created will automatically skip any duplicate email addresses. This is usefull if we want to include new recipients to our campaign.

In the next section, we'll create a Tracking module to track our recipient`s activity by saving the timestamp as last_seen when the email is opened.

Creating tracker module

To be continued...

Adding unsubscribe feature

To be continued...