Saturday, June 15, 2024

Why T3 Env is My Go-To for Managing Environment Variables

It has been two months since I used T3 Env. In the following, I will elaborate on what is T3 Env and why is it so good in my opinion.

Common issues with environment variables

Before moving on, we need to understand the problem we are dealing with. Environment variables are crucial for configuring applications. However, managing these variables can be troublesome.

Misconfiguring environment variables

One of the most common issues is misconfiguring or forgetting to set necessary environment variables during the build process. This is especially the case when environment variables are sometimes secrets and should be set in the CD platforms like Jenkins or Vercel, instead of being part of the code base.

When an environment variable is missing or incorrectly set, it can cause bugs that are difficult to trace back to their source. For instance, your ORM may complain it fails to connect to the SIT database by accessing the database info from the environment variables. DB connection failure could have many reasons, while the root cause of it may be simply you mistakenly specified the production database user credentials in the environment variable instead of the SIT ones. This often leads to wasted time and frustration during development and deployment.

Even worse, a lot of the time, issues like this won't show up until the application is deployed and running, because those operations happen at runtime.

Poor developer experience when working with environment variables

Usually, we access environment variables in a Node.js project with process.env. Yet, despite using TypeScript, you may notice that it isn't typed well, and for good reasons.

Firstly, it is merely typed as a generic dictionary, except for a few very common ones like NODE_ENV. This means it will accept any process.env.* as a valid value, including those that you may have made a typo.

Secondly, while you can access the variable you want, let's say process.env.VERCEL_ENV, it is always in string | undefined type, despite a lot of time, the type can be more specific than that, e.g. as described in the documentation for VERCEL_ENV. Also, technically if we didn't make any human errors, this variable will never be undefined. And there is no right way to handle it as well besides throwing an error.

That being said, if we think about it carefully, process.env being generic makes sense to support generic use cases. The key is it is usually too generic for many projects. How can we make it specific for the projects we are working on? This is where T3 Env comes into play.

For details on the rationale behind this library, visit the introduction of the documentation.

What is T3 Env and how does it solve the issues?

T3 Env validates a project's environment variables by making use of Zod. It is part of the T3 Stack, and it provides two layers of checking:

1. Type check: e.g. TypeScript will tell you that the VERCEL_ENV is 'production' | 'preview' | 'development', instead of string | undefined.
2. Runtime validation: a runtime error will be thrown if the environment variables read don't match the specified Zod schema, or if we attempt to access server-only environment variables on the client side.

It provides an amazing developer experience when working with environment variables while alerting you if you made a mistake.

How to use T3 Env?

In the following example, I will showcase a simplified usage of what is being used on this website, a Next.js project. While the steps and codes are mostly the same for other Node.js projects, here is the context on how environment variables are handled in Next.js that are different from other projects.

• Only environment variables starting with NEXT_PUBLIC_ can be accessed (or "bundled" to be precise) on the client-side. This is a convention used to prevent accidental leaks of sensitive data like API keys.
• Next.js only bundles NEXT_PUBLIC_* environment variables that are used explicitly. i.e. for NEXT_PUBLIC_VERCEL_ENV to appear in the bundle, we must access it with process.env.NEXT_PUBLIC_VERCEL_ENV. No other syntaxs are allowed. And it will not appear in the client-side bundle if we don't do so, despite it being set during built time.

Example use case

Consider the following environment variables:

• DATABASE_URL: the connection string of my database, a server-only environment variable.
• NEXT_PUBLIC_SITE_DISPLAY_NAME: the name of this website derived from the production hostname, a.k.a. "Mwskwong.com", a client-side environment variable
• NODE_ENV: needs no introduction, a shared environment variable in 'development' | 'test' | 'production' type.
• ANALYZE: a flag to toggle between whether to generate the tree map for the website bundles using @next/bundle-analyzer during built time, a server-only environment variable.

Implementation steps

Adding env.mjs

Let's begin by adding an env.mjs for specifying the environment variables schema.

Note, we are using JS instead of TS here because we want to import this file into next.config.mjs (or .js; context: this file must be a JS file) so that the validation check will kick in as soon as the project begins to build, and JS files can access the defined environment variables as well.

As for why using .mjs instead of .js, I prefer using import syntax instead of require() syntax. It's just a personal preference and .js will work fine in this case.

// @ts-check // tells TS to type check this file as well, despite it is a JS file
 
import { createEnv } from '@t3-oss/env-nextjs';
import { z } from 'zod';
 
export const env = createEnv({
  shared: {},
  server: {},
  client: {},
  experimental__runtimeEnv: {},
});

Define DATABASE_URL validation rule

As its name suggests, DATABASE_URL is a URL.

export const env = createEnv({
  // ...
  server: {
    DATABASE_URL: z.string().url(),
  },
});

Verify your setup

You may import env.mjs to the entry point of your application, for a Next.js app, it will be next.config.mjs, and begin to test the validation behavior. Try to assign different values to DATABASE_URL and start the DEV server.

import { env } from './src/env.mjs';

DATABASE_URL='postgres://<username>:<password>@<hostname>/<database>?sslmode=require' # no errors
DATABASE_URL= # Invalid url
DATABASE_URL='random string' # Invalid url

(Optional) Notice that T3 Env is complaining DATABASE_URL= is an invalid URL, instead of it being undefined. This is due to how .env works, and the above statement assigns '' to DATABASE_URL. This behavior, however, may make the debug process more challenging, because a lot of the time we want to separate "forget to set" cases and "misconfiguring" cases. Fortunately, T3 Env provides an option emptyStringAsUndefined to treat '' environment variables as undefined.

export const env = createEnv({
  // ... 
  emptyStringAsUndefined: true,
});

Define NEXT_PUBLIC_SITE_DISPLAY_NAME validation rule

NEXT_PUBLIC_SITE_DISPLAY_NAME is just a string. However, due to how the client-side environment variable works in Next.js, we must explicitly "use it" in experimental__runtimeEnv to avoid Next.js mistakenly omitting the variable from the client-side bundle. T3 Env will also remind you with an error if you forget to do so.

export const env = createEnv({
  // ...
  client: {
    NEXT_PUBLIC_SITE_DISPLAY_NAME: z.string(),
  },
  experimental__runtimeEnv: {
    NEXT_PUBLIC_SITE_DISPLAY_NAME: process.env.NEXT_PUBLIC_SITE_DISPLAY_NAME,
  }
});

Yet, since this site is deployed on Vercel, another environment variable NEXT_PUBLIC_VERCEL_PROJECT_PRODUCTION_URL is automatically set during built time by Vercel, which has the value of "mwskwong.com" in this case. We can compute the value of NEXT_PUBLIC_SITE_DISPLAY_NAME from it instead of hard coding it.

import { capitalize } from 'lodash-es';
 
export const env = createEnv({
  // ...
  client: {
    NEXT_PUBLIC_SITE_DISPLAY_NAME: z.string(),
  },
  experimental__runtimeEnv: {
    NEXT_PUBLIC_SITE_DISPLAY_NAME:
      process.env.NEXT_PUBLIC_VERCEL_PROJECT_PRODUCTION_URL &&
      capitalize(process.env.NEXT_PUBLIC_VERCEL_PROJECT_PRODUCTION_URL),
  }
});

Define NODE_ENV validation rule

Since NODE_ENV can be accessed on the client side, just like the NEXT_PUBLIC_ ones do, at first glance, it makes perfect sense to put it in client as well. However, an error will be thrown if you attempt to do so, suggesting a client variable should be prefixed with NEXT_PUBLIC_, and obviously, this doesn't make sense to "standard" environment variables like NODE_ENV.

Fortunately, T3 Env has that covered as well. It provides a shared key for this exact purpose. By definition, it is for shared variables, "often those that are provided by build tools and is available to both client and server, but isn't prefixed and doesn't require to be manually supplied".

And since this is still a "client" variable so to say, we still need to specify it in experimental__runtimeEnv.

export const env = createEnv({
  // ...
  shared: {
    NODE_ENV: z.enum(['development', 'test', 'production']),
  },
  experimental__runtimeEnv: {
    NODE_ENV: process.env.NODE_ENV,
  }
});

Define ANALYZE validation rule

ANALYZE is a flag, so, naturally, it is a boolean in most cases. However, environment variables can only be strings. So what we usually do as a fallback plan is to make use of values like '1' | '0', or 'true' | 'false'.

export const env = createEnv({
  // ...
  server: {
    ANALYZE: z.enum(['true', 'false']).optional(),
  },
});

Yet, don't forget we are using Zod, in addition to T3 Env. In other words, all Zod operations will still apply. And we can transform the value of 'true' | 'false' to boolean using Zod's transform() function.

export const env = createEnv({
  // ...
  server: {
    ANALYZE: z
      .enum(['true', 'false'])
      .optional()
      .transform((analyze) => analyze === 'true'),
  },
});

(Optional) Skipping validation

In some cases, you may want to prevent T3 Env validation from running for a legitimate use case. For example in my case, I want to skip validation when linting the project, because this step is run in my CI pipeline and all of the required environment variables are not set there, since it will not impact the linting process.

Fortunately, T3 Env provides a skipValidation flag to achieve that. In my use case, it will be as follows

export const env = createEnv({
  // ...
  skipValidation: process.env.npm_lifecycle_event === 'lint',
});

Putting everything together

// @ts-check
 
import { createEnv } from "@t3-oss/env-nextjs";
import { capitalize } from "lodash-es";
import { z } from "zod";
 
export const env = createEnv({
  shared: {
    NODE_ENV: z.enum(["development", "test", "production"]),
  },
  server: {
    DATABASE_URL: z.string().url(),
    ANALYZE: z
      .enum(["true", "false"])
      .optional()
      .transform((analyze) => analyze === "true"),
  },
  client: {
    NEXT_PUBLIC_SITE_DISPLAY_NAME: z.string(),
  },
  experimental__runtimeEnv: {
    NEXT_PUBLIC_SITE_DISPLAY_NAME:
      process.env.NEXT_PUBLIC_VERCEL_PROJECT_PRODUCTION_URL &&
      capitalize(process.env.NEXT_PUBLIC_VERCEL_PROJECT_PRODUCTION_URL),
    NODE_ENV: process.env.NODE_ENV,
  },
  emptyStringAsUndefined: true,
  skipValidation: process.env.npm_lifecycle_event === 'lint',
});

Accessing the validated environment variables

To access the validated environment variables with type safety, simply replace the process.env.* calls with imports from env.mjs.

import { env } from '@/src/env.mjs';
 
// replaces process.env.DATABASE_URL
env.DATABASE_URL;

Limitations and considerations

It only supports Zod

T3 Env currently only supports Zod and no other schema validation libraries. Although being schema validation library agnostic is on its roadmap and is one of the most anticipated topic on GitHub (including myself), there is no ETA for it at this stage.

I would imagine it can take a lot of inspiration from @hookform/resolvers, which has a great API design and almost the same use case.

T3 Env and Zod is being bundled on the client-side

This is not a deal breaker per se, but it will be perfect if the client-side validation is a compilation step instead of happening at runtime. How "client-side environment variables` work is simply to extract the values during built time and then hard code the values in the client bundle. There is no runtime logic involved here. So in theory, the validation can run at this stage without needing to inject the runtime into the bundle, although I would imagine the solution will be a lot more complicated than what we have now.