& Continuous Learning

Introduction

Anyone who has been part of a software development team for any length of time has inevitably been asked by someone to "Send an email with a report." Either this email should be sent when certain things happen, or on a cadence. Historically, these emails would be sent by connecting to an on-premise SMTP server. However in the cloud, there is rarely an SMTP server available. If there is one, it is often hidden behind firewalls and often back in a corporate data center. Additionally, teams expect to be able to write automated tests to assert the structure of data and this includes the HTML inside of an email. Since sending an email is only one small piece of a larger effort, it would a benefit to use the same technologies as other parts of the system.

Our solution

To demonstrate a reference design for this problem, we are going to build out a small application that sends an email every day with any resources within AWS that are more costly than a threshold. This need comes out of our sandbox accounts where teammates will often spin up new services in order to gain experience with them but sometimes forget to delete them and incur unnecessary costs. In order to maximize the technology overlap with other solutions we will use the following libraries and technologies:

• AWS Lambda: The "serverless" compute option that is a very simple and cost effective way to run small amounts of code in AWS
• Typescript: This solution would work with just Javascript, but Typescript gives us better confidence that we are passing the appropriate structure to library functions and we are correctly accessing the responses.
• AWS Simple Email Service (SES): A HTTP based service from AWS that allows us to send emails without needing an SMTP server
• aws-sdk: Gives us the ability to call the "Cost Explorer" API to get cost information and send emails to SES
• aws-sdk-client-mock: We could manually setup mocks for calls to AWS services, but this library makes that easier and also ensures that our mock data responses are more realistic to actual responses
• React: In addition to rendering UIs in the browser, React is an easy way to generate static HTML
• TC39 Temporal Polyfill: We will need to do some simple date math. This will allow us to use the new Temporal APIs until they become standard in NodeJS
• Vitest: An ESM compatible Unit Testing library that is a faster alternative to Jest
• React Testing Library: Allows us to write unit tests that assert the structure of the HTML email without resorting to string manipulation
• Cloudformation: The AWS native infrastructure as code format to specify our cloud resources
• AWS SAM: A CLI and template that simplifies Cloudformation deployments

Getting Cost Information

AWS provides an API to fetch the daily cost information on a per-resource basis. This allows us to fetch the costs for the last day and then find resources that are more expensive than we would like. There is an additional cost for this level of granularity so it has to be explicitly enabled in the Cost Explorer dashboard by following these instructions. Next, we will setup a new Typescript project by running the following commands in a new directory:

npm init -y
npm install --save esbuild @aws-sdk/client-cost-explorer temporal-polyfill
npm install --save-dev typescript vitest aws-sdk-client-mock @types/node

Update the package.json to set the test script to vitest so that npm test executes the tests. Create an app.tsx file that exports a function that Lambda will invoke:

export async function lambdaHandler() {
    return "Hello";
}

We can now start test driving our function to call the Cost Explorer API by adding a App.test.tsx file:

import {
  CostExplorerClient,
  GetCostAndUsageWithResourcesCommandInput,
  GetCostAndUsageWithResourcesCommand,
} from "@aws-sdk/client-cost-explorer";
import { afterEach, beforeEach, expect, it, vi } from "vitest";
import { mockClient } from "aws-sdk-client-mock";
import { lambdaHandler } from "./app";

beforeEach(() => {
    vi.useFakeTimers();
  vi.setSystemTime(new Date("2024-03-29T10:00:00-05:00"));
});

afterEach(() => {
    vi.useRealTimers();
});

const mockCostExplorerClient = mockClient(CostExplorerClient);

beforeEach(() => {
  mockCostExplorerClient.on(GetCostAndUsageWithResourcesCommand).resolves({
    ResultsByTime: [
      {
        Estimated: true,
        Groups: [],
        TimePeriod: {
          End: "2024-03-29T00:00:00Z",
          Start: "2024-03-28T05:00:00Z",
        },
        Total: {
          BlendedCost: { Amount: "0", Unit: "USD" },
          UsageQuantity: { Amount: "0", Unit: "N/A" },
        },
      },
      {
        Estimated: true,
        Groups: [
          {
            Keys: ["NoResourceId"],
            Metrics: {
              BlendedCost: { Amount: "0.0158414714", Unit: "USD" },
              UsageQuantity: { Amount: "467.072580645", Unit: "N/A" },
            },
          },
          {
            Keys: [
              "arn:aws:connect:::phone-number/9c64c0e4-1521-4dde-912d-782de332eb73",
            ],
            Metrics: {
              BlendedCost: { Amount: "0.000559064", Unit: "USD" },
              UsageQuantity: { Amount: "0.04166667", Unit: "N/A" },
            },
          },
          {
            Keys: [
              "arn:aws:dynamodb:us-east-1:12345678910:table/widgets",
            ],
            Metrics: {
              BlendedCost: { Amount: "0.000138154", Unit: "USD" },
              UsageQuantity: { Amount: "4", Unit: "N/A" },
            },
          },
        ],
        TimePeriod: {
          End: "2024-03-29T05:00:00Z",
          Start: "2024-03-29T00:00:00Z",
        },
        Total: {},
      },
    ],
  });
});

it("should fetch the cost data from AWS", async () => {
  await lambdaHandler();

  const [costParams] = mockCostExplorerClient.commandCalls(
    GetCostAndUsageWithResourcesCommand
  )[0].args;
  expect(costParams.input).toEqual<GetCostAndUsageWithResourcesCommandInput>({
    TimePeriod: {
      Start: "2024-03-27T05:00:00Z",
      End: "2024-03-29T05:00:00Z",
    },
    Granularity: "DAILY",
    Filter: undefined,
    GroupBy: [
      {
        Key: "RESOURCE_ID",
        Type: "DIMENSION",
      },
    ],
    Metrics: ["BlendedCost", "UsageQuantity"],
  });
});

There is a lot going on in this test case. This is to be expected since most of this application involves glueing together various third party libraries and services. We first setup fake timers so we can force the "current date" into a predicatable value. We then use the mockClient helper function to setup a mock of the AWS CostExploerClient and configure it to return a dummy response with some data in it. We are doing this in a beforeEach so we don't have to set it up on every test. Finally, we write our first test case that asserts that we are calling the GetCostAndUsageWithResources service with the correct arguments to fetch the last day's worth of costs. Feel free to write the implementation based on the test case, or reference our version here:

import { Temporal } from "temporal-polyfill";
import {
  CostExplorerClient,
  GetCostAndUsageWithResourcesCommand,
  Group,
} from "@aws-sdk/client-cost-explorer";

async function getCosts(): Promise<Group[]> {
  const costExplorerClient = new CostExplorerClient();
  const now = Temporal.Now.zonedDateTimeISO("America/Chicago");
  const result = await costExplorerClient.send(
    new GetCostAndUsageWithResourcesCommand({
      Filter: undefined,
      TimePeriod: {
        Start: now.startOfDay().add({ days: -2 }).toInstant().toJSON(),
        End: now.startOfDay().toInstant().toJSON(),
      },
      Granularity: "DAILY",
      GroupBy: [
        {
          Key: "RESOURCE_ID",
          Type: "DIMENSION",
        },
      ],
      Metrics: ["BlendedCost", "UsageQuantity"],
    })
  );
  return result.ResultsByTime?.flatMap((r) => r.Groups ?? []) ?? [];
}

export async function lambdaHandler() {
  return await getCosts();
}

Generating a Cost Report

There are a lot of ways to generate HTML, including simple string concatenation. We want something that is more maintainable, properly escapes HTML strings, and is easier to test. For browser based UIs React is a popular library and is also used to server render HTML. In our case, we can use this to generate the report and then simply send it in an email instead of returning it to a client. We can then use React Testing Library to ensure that the content in the report is what we expect. Let's start by installing some packages:

npm install --save react-dom
npm install --save-dev happy-dom @testing-library/react @types/react-dom @types/react

Since we are not running in a browser, we need to add a DOM implementation so that we can render the HTML and assert various things about our structure. This also ensures that we generate valid markup. In order for Vitest to know about this, we need to create a vitest.config.ts file with this content:

import { defineConfig } from "vitest/config";

export default defineConfig({
  test: {
    environment: 'happy-dom',
    unstubEnvs: true,
    unstubGlobals: true,
    restoreMocks: true,
    mockReset: true,
  },
});

And, since we are now using React we need to tell Typescript to compile TSX using this tsconfig.json:

{
  "compilerOptions": {
    "target": "es2020",
    "strict": true,
    "jsx": "react",
    "preserveConstEnums": true,
    "noEmit": true,
    "sourceMap": false,
    "module": "es2015",
    "moduleResolution": "node",
    "esModuleInterop": true,
    "skipLibCheck": true,
    "forceConsistentCasingInFileNames": true
  },
  "exclude": ["node_modules", "**/*.test.ts", "**/*.test.tsx"]
}

Now we can add to our app.test.tsx to validate we are generating an HTML table:

import { screen, render, within } from "@testing-library/react";
import React from "react";

it("should generate a table with a row for the resource", async () => {
  const html = {
    __html: await lambdaHandler()
  };
  render(<div dangerouslySetInnerHTML={html} />);

  const table = screen.getByRole("table");
  const dynamoRowHeader = within(table).getByRole("rowheader", {
    name: "arn:aws:dynamodb:us-east-1:12345678910:table/widgets",
  });
  const dynamoRow = dynamoRowHeader.closest("tr")!;
  expect(within(dynamoRow).getByText("$0.000138154")).not.toBeNull();
});

What we are doing here is calling our Lambda function, then using the dangerouslySetInnerHTML to have React Testing Library use that instead of rendering the component directly. Once that is done, we can assert that there is a table, find the row for the dynamoDB table from our mock data, then make sure the amount is in the same row.

This test should fail, so we can implement it by adding some imports, a function, and then modifying our handler:

import React from "react";
import { renderToStaticMarkup } from "react-dom/server";

function BlendedCostCell({ group }: { group: Group }) {
  const costFormatter = new Intl.NumberFormat(undefined, {
    style: "currency",
    currency: "USD",
    maximumFractionDigits: 9,
  });
  const amountString = group.Metrics?.["BlendedCost"]?.Amount;
  return (
    <td>
      {amountString ? costFormatter.format(parseFloat(amountString)) : "-"}
    </td>
  );
}

function CostReport({ costs }: { costs: Group[] }) {
  const sortedCosts = [...costs]
    .map((group) => ({
      ...group,
      blendedAmount: parseFloat(group.Metrics?.["BlendedCost"]?.Amount ?? "0"),
    }))
    .slice(0, 10)
    .sort((a, b) => b.blendedAmount - a.blendedAmount);

  return (
    <table>
      <thead>
        <tr>
          <th>Resource</th>
          <th>Blended Cost</th>
        </tr>
      </thead>
      <tbody>
        {sortedCosts.map((group, idx) => (
          <tr key={idx}>
            <th scope="row" style={{ textAlign: "left" }}>
              {group.Keys?.join("/")}
            </th>
            <BlendedCostCell group={group} />
          </tr>
        ))}
      </tbody>
    </table>
  );
}

export async function lambdaHandler() {
  const costs = await getCosts();
  return renderToStaticMarkup(<CostReport costs={costs} />);
}

It is worth noting that since we are using Javascript to generate the report we can leverage the same Intl.NumberFormat to format our costs as currency. For more information on the rich formatting support built into modern Javascript runtimes take a look at our Technically Speaking videos.

Sending an email

Sending an email with SES involves us calling the SendEmail API. We can do this by adding in some more libraries and adding a test case:

npm install --save @aws-sdk/client-sesv2

import {
  SESv2Client,
  SendEmailCommand,
  SendEmailCommandInput,
} from "@aws-sdk/client-sesv2";

const mockSesClient = mockClient(SESv2Client);

beforeEach(() => {
  mockSesClient.on(SendEmailCommand).resolves({});
});

it("should send an email with the cost data", async () => {
  await lambdaHandler();

  const [sendEmailParams] =
    mockSesClient.commandCalls(SendEmailCommand)[0].args;
  expect(sendEmailParams.input).toMatchObject<SendEmailCommandInput>({
    Destination: {
      ToAddresses: ["prowe@sourceallies.com"],
    },
    FromEmailAddress: 'daily-cost@sandbox-dev.sourceallies.com',
    Content: {
      Simple: {
        Subject: {
          Data: "Cost and usage report",
        },
        Body: {
          Html: {
            Data: expect.any(String),
          },
        },
      },
    },
  });
});

We also need to change our content test so that it uses the value sent in the email rather than relying on the html being returned from the Lambda:

it("should generate a table with a row for the resource", async () => {
  await lambdaHandler();
  
  const [sendEmailParams] =
    mockSesClient.commandCalls(SendEmailCommand)[0].args;
  const html = {
    __html: sendEmailParams.input.Content?.Simple?.Body?.Html?.Data ?? "",
  };
  render(<div dangerouslySetInnerHTML={html} />);

  const table = screen.getByRole("table");
  const dynamoRowHeader = within(table).getByRole("rowheader", {
    name: "arn:aws:dynamodb:us-east-1:12345678910:table/widgets",
  });
  const dynamoRow = dynamoRowHeader.closest("tr")!;
  expect(within(dynamoRow).getByText("$0.000138154")).not.toBeNull();
});

Implementing this test is fairly straightforward, but here is our version for reference:

import { SESv2Client, SendEmailCommand } from "@aws-sdk/client-sesv2";

async function sendEmail(html: string) {
  const client = new SESv2Client();
  await client.send(
    new SendEmailCommand({
      Destination: {
        ToAddresses: ["prowe@sourceallies.com"],
      },
      FromEmailAddress: "daily-cost@sandbox-dev.sourceallies.com",
      Content: {
        Simple: {
          Subject: {
            Data: "Cost and usage report",
          },
          Body: {
            Html: {
              Data: html,
            },
          },
        },
      },
    })
  );
}

export async function lambdaHandler() {
  const costs = await getCosts();
  const html = renderToStaticMarkup(<CostReport costs={costs} />);
  await sendEmail(html);
}

Setting up and deploying our infrastructure

It might be tempting to locally execute our lambda to see the fruits of our labors so far. This will not work as expected because SES will refuse to send an email from any address that is "unverified" and will also not send an email to an unverified address until you request promotion of the AWS account out of "sandbox mode". We are able to set this up via Cloudformation along with our other resources.

I like to use AWS SAM to execute Cloudformation deployments because it can automatically package Lambdas and has better console output as the deploy is happening. Getting on those rails we need to create a samconfig.toml configuration file:

version = 0.1

[default]
[default.global.parameters]
stack_name = "prowe-daily-cost-report"

[default.deploy.parameters]
capabilities = "CAPABILITY_IAM"
confirm_changeset = false
resolve_s3 = true

Then we can create our Cloudformation template.yaml file:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: "Runs a daily report for expensive resources"

Resources:
  UserIdentity:
    Type: AWS::SES::EmailIdentity
    Properties:
      EmailIdentity: 'prowe@sourceallies.com'

This template only creates an "EmailIdentity" that we can use to verify my email address to ensure we are not sending spam. This requirement is removed if we promote our account out of sandbox. To build and deploy this stack we can run the following:

sam build && sam deploy

Once this deploys, you (or me if you didn't update the email address) will get a confirmation email that needs to be clicked on to switch the address to "verified" To validate our from domain, we can simply include some Dkim tokens in our DNS provider. This can all be done automatically by including the following resources:

DomainIdentity:
  Type: AWS::SES::EmailIdentity
  Properties:
      EmailIdentity: 'sandbox-dev.sourceallies.com'

DomainValidationRecords:
  Type: AWS::Route53::RecordSetGroup
  Properties:
    HostedZoneId: 'Z328FCPUJ05W9K'
    RecordSets:
      - Name: !GetAtt DomainIdentity.DkimDNSTokenName1
        Type: CNAME
        TTL: 300
        ResourceRecords:
          - !GetAtt DomainIdentity.DkimDNSTokenValue1
      - Name: !GetAtt DomainIdentity.DkimDNSTokenName2
        Type: CNAME
        TTL: 300
        ResourceRecords:
          - !GetAtt DomainIdentity.DkimDNSTokenValue2
      - Name: !GetAtt DomainIdentity.DkimDNSTokenName3
        Type: CNAME
        TTL: 300
        ResourceRecords:
          - !GetAtt DomainIdentity.DkimDNSTokenValue3

Deploy this as above and you should see the domain reflecting as validated in the SES Console. Finally, we can add our Lambda. We are leveraging the serverless transform in order to inline the events and policy. We are also leveraging SAM metadata to automatically compile our Typescript.

ReportFunction:
  Type: AWS::Serverless::Function
  Properties:
    CodeUri: .
    Handler: app.lambdaHandler
    Timeout: 10
    Runtime: nodejs18.x
    Architectures:
      - x86_64
    Policies:
      - Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Action:
              - ses:SendEmail
              - ce:GetCostAndUsageWithResources
            Resource: '*'
    Events:
      NightlyEvent:
        Type: ScheduleV2
        Properties:
          State: ENABLED
          ScheduleExpression: 'cron(0 7 * * ? *)'
  Metadata: #this tells SAM to use esbuild to transpile our project starting with `app.tsx`
    BuildMethod: esbuild
    BuildProperties:
      Minify: true
      Target: "es2020"
      Sourcemap: true
      EntryPoints: 
        - app.tsx

Once this is deployed, we can either wait until tomorrow morning, or manually test the Lambda through the Lambda console.

Conclusion

Throughout this post, we glued together a couple of AWS services, as well as a few libraries to send a daily email. The Cost and Usage Service gives us the data we need. React can be used to easily generate a dynamic HTML table appropriate for email. SES provides a simple interface for sending an email that does not require SMTP infrastructure. We deployed the whole thing as a Lambda using Cloudformation so we can maintain this through a CI/CD Pipeline.

Feel free to checkout the completed project on Github