TypeScript code generation and parsing using typescript

TypeScript code generation and parsing using typescript

One of the interesting features of the typescript package is that it contains an API for generating TypeScript code, as well as a parser for working with TypeScript code. Code generation is often used to automatically generate types for working with the http api (typing the request body, response, parameters, etc.). npm has modules that generate services for working with api based on openapi, graphQl schemes, etc., and usually the capabilities of existing modules are enough to solve most tasks.

But sometimes there is a need to extend the existing interface and even write your own custom implementation. In this article, we will look at the basic principles of working with tools for generation and parsing typescript code, as well as some pitfalls I encountered while working with it.

Code generation

Consider, for example, code generation with a simple declaration enum:

export type Numbers = "one" | "two" | "three";

First, let’s set up a small project with the help of the command npm init. Then install the npm package typescript team npm install typescript. Also, for further work, we need to install a dependency npm install @types/node --save-dev.

The following code will be needed to create the above Numbers enum:

// ./src/index.ts
import * as ts from "typescript";

const file = ts.createSourceFile("source.ts", "", ts.ScriptTarget.ESNext, false, ts.ScriptKind.TS);
const printer = ts.createPrinter({ newLine: ts.NewLineKind.LineFeed });

const one = ts.factory.createLiteralTypeNode(ts.factory.createStringLiteral("one"));
const two = ts.factory.createLiteralTypeNode(ts.factory.createStringLiteral("two"));
const three = ts.factory.createLiteralTypeNode(ts.factory.createStringLiteral("three"));

const decl = ts.factory.createTypeAliasDeclaration(
  [ts.factory.createToken(ts.SyntaxKind.ExportKeyword)], // modifiers (export keyword)
  ts.factory.createIdentifier("Numbers"), // name
  undefined, // typeParameters
  ts.factory.createUnionTypeNode([ one, two, three ]), // type
)

const result = printer.printNode(ts.EmitHint.Unspecified, decl, file);
console.log(result);

To compile and run this file, we will add it to the projecttsconfig.json:

// ./tsconfig.json
{
  "compilerOptions": {
    "outDir": "./dist",
    "rootDir": "./src",
    "sourceMap": true,
    "target": "esnext",
    "module": "CommonJS",
    "moduleResolution": "node",
    "types": [
      "node"
    ],
    "allowSyntheticDefaultImports": true,
    "esModuleInterop": true
  },
  "include": ["./src/**/*.ts"]
}

And also scripts build and start in package.json:

// ./package.json
"scripts": {
  "build": "tsc",
  "start": "node dist/index.js"
}

Now we can compile and execute the code with ./src/index.ts. We run the commands sequentially npm run build, npm run start.

The result of the script execution is the output of the line in the console:

Code generation of enum Numbers

Let’s analyze the code in more detail ./src/index.ts. First, the package is imported typescript and a special printer object is created that is used later for formatted output typescript declarations:

const file = ts.createSourceFile("source.ts", "", ts.ScriptTarget.ESNext, false, ts.ScriptKind.TS);
const printer = ts.createPrinter({ newLine: ts.NewLineKind.LineFeed });

// ...

const result = printer.printNode(ts.EmitHint.Unspecified, decl, file);
console.log(result);

Of greater interest is the code between these lines:

const one = ts.factory.createLiteralTypeNode(ts.factory.createStringLiteral("one"));
const two = ts.factory.createLiteralTypeNode(ts.factory.createStringLiteral("two"));
const three = ts.factory.createLiteralTypeNode(ts.factory.createStringLiteral("three"));

const decl = ts.factory.createTypeAliasDeclaration(
  [ts.factory.createToken(ts.SyntaxKind.ExportKeyword)], // modifiers (export keyword)
  ts.factory.createIdentifier("Numbers"), // name
  undefined, // typeParameters
  ts.factory.createUnionTypeNode([ one, two, three ]), // type
);

The algorithm sequentially creates literals for the elements of the union and then composes them into a single declaration by providing the modifier export and name Numbers. As we can see: nothing complicated, all the complexity lies only in the identification of the entity and the constructor required for its creation. API typescript offers many functions to create different designs. They are very easy to get confused, because it is not always obvious from the name what this or that function creates, and it is not clear what should be passed to it as parameters:

TypeScript api

For the identification of the names required typescript entities, you can use the code parsing API. That is, first we write a pseudocode, we pair it with means typescript and get the AST (abstract syntax tree), study it and create the structure we need based on the knowledge obtained.

Typescript parsing

To demonstrate the parsing API, we will create a typescript code file that we will use for parsing:

// ./enum.ts
export type Numbers = "one" | "two" | "three";

Next, add in ./src/index.ts the following code:

// ./src/index.ts
import * as fs from 'node:fs';
import * as path from 'path';

const schema = ts.createSourceFile(
  'x.ts',
  fs.readFileSync(path.resolve(process.cwd(), './enum.ts'), 'utf-8'),
  ts.ScriptTarget.Latest,
  undefined,
);

console.log(schema.statements[0]);

Function createSourceFile will create an AST tree for our file enum.ts and in the field statements will be an array of ts entities declared in the file. The only entity that is there is the exported union. Numbers. Output to the console at the end of the file console.log(schema.statements[0]); will display the structure of its AST:

NodeObject {
  pos: 0,
  end: 46,
  flags: 0,
  modifierFlagsCache: 0,
  transformFlags: 1,
  parent: undefined,
  kind: 265,
  symbol: undefined,
  localSymbol: undefined,
  modifiers: [
    TokenObject {
      pos: 0,
      end: 6,
      flags: 0,
      modifierFlagsCache: 0,
      transformFlags: 0,
      parent: undefined,
      kind: 95
    },
    pos: 0,
    end: 6,
    hasTrailingComma: false,
    transformFlags: 0
  ],
  name: IdentifierObject {
    pos: 11,
    end: 19,
    flags: 0,
    modifierFlagsCache: 0,
    transformFlags: 0,
    parent: undefined,
    kind: 80,
    escapedText: 'Numbers',
    jsDoc: undefined,
    flowNode: undefined,
    symbol: undefined
  },
  typeParameters: undefined,
  type: NodeObject {
    pos: 21,
    end: 45,
    flags: 0,
    modifierFlagsCache: 0,
    transformFlags: 1,
    parent: undefined,
    kind: 192,
    types: [
      [NodeObject],
      [NodeObject],
      [NodeObject],
      pos: 21,
      end: 45,
      hasTrailingComma: false,
      transformFlags: 1
    ]
  },
  jsDoc: undefined,
  locals: undefined,
  nextContainer: undefined
}

AST analysis

AST is more convenient to analyze in REPL mode, pausing the execution of the script in the IDE debugger. If you use VSCode, you can configure the debugger by adding a file .vscode/launch.json with the following content:

// .vscode/launch.json
{
  "version": "0.2.0",
  "configurations": [
    {
      "type": "node",
      "request": "launch",
      "name": "Run programm",
      "program": "${workspaceFolder}/src/index.ts",
      "preLaunchTask": "npm: build",
      "sourceMaps": true,
      "smartStep": true,
      "internalConsoleOptions": "openOnSessionStart",
      "outFiles": ["${workspaceFolder}/dist/**/*.js"],
      "console": "internalConsole",
      "outputCapture": "std"
    }
  ]
}

Next, set a breakpoint on the line with output to the console and start the debugger (F5 hotkey):

Then we will analyze the AST in the object schema.statements[0].

The fields are of greatest interest in it at the moment kind, modifiers, name and type. kind — this field stores the value from the enum ts.SyntaxKindby which you can determine the name of the entity, and then find the constructor in space necessary for its creation ts.factory.

As you can see, the root element of our tree has kind === 265i.e. the type TypeAliasDeclarationand the following function in is used to create it ts.factory:

createTypeAliasDeclaration(
  modifiers: readonly ModifierLike[] | undefined,
  name: string | Identifier,
  typeParameters: readonly TypeParameterDeclaration[] | undefined,
  type: TypeNode
): TypeAliasDeclaration;

where are the fields modifiers, name, typecontain exactly those entities that we saw when outputting the AST for the object to the console schema.statements[0]. After analyzing these REPL fields, you can understand which objects we need to pass to the constructor.

Analyzing the AST in the REPL makes it possible to fully familiarize yourself with the structure of the AST, but it takes a lot of time. If you need to quickly figure out how to create a particular entity as an AST, you can use the online tool TypeScript AST Viewer. If we write the Numbers enum we are creating in it, it will display the following construction for creating its AST:

TypeScript AST Viewer

The TypeScript AST Viewer tool is easy to use and a great time saver for AST analysis, but it has some small limitations. It does not reflect jsDoc-Comments. Working with them has certain features, which I will talk about at the end of the article.

Working with AST

We’ve seen how to extract information from an AST in a REPL, now let’s look at an example of how to work with it in TypeScript.

Let’s say we have a file with the following content:

// ./numbers.ts
export interface Numbers {
  one: 1
  two: 2
  three: 3
}

and we need to parse it and form an array of strings with the values ​​of the keys of the Numbers interface: ['one', 'two', 'three'].

To do this, add in ./src/index.ts:

// ./src/index.ts
const numbersSource = ts.createSourceFile(
  'x.ts',
  fs.readFileSync(path.resolve(process.cwd(), './numbers.ts'), 'utf-8'),
  ts.ScriptTarget.Latest,
  undefined,
);

const numbersDecl = numbersSource.statements.find((node) => {
  return ts.isInterfaceDeclaration(node) && node.name.text === 'Numbers';
});

if (!numbersDecl || !ts.isInterfaceDeclaration(numbersDecl)) {
  throw new Error('no interface "Numbers" in file');
}

const numbers = numbersDecl.members.reduce<string[]>((acc, node) => {
  if (ts.isPropertySignature(node) && ts.isIdentifier(node.name)) {
    acc.push(node.name.escapedText.toString())
  }

  return acc
}, [])

console.log(numbers); // [ 'one', 'two', 'three' ]

Interaction with AST TypeScript not much different from working with it in the REPL, except that elements statements are instances NodeObject and know nothing about the fields members, name and others. In order to access the fields we need, we must first check that the object is an instance of a specific class that has these fields. For these purposes in typescript there are a huge number of guards, such as the ones we used in the code above: ts.isInterfaceDeclaration, ts.isPropertySignature and ts.isIdentifier.

Features of working with jsDoc

When analyzing AST in REPL in instances NodeObject you can see the field jsDoc. As can be seen from the name, it stores information about jsDoc-Comments and tags for a given tree node.

Parsing jsDoc

Let’s create a simple file with a class and a method comment:

// ./comments.ts
export class Greetings {
  /**
   * приветствует мир
   * @deprecated метод устарел
   */
  hello() {
    console.log('hello world')
  }
}

And then we will steam it and analyze its AST in the REPL:

// ./src/index.ts
const commentsSource = ts.createSourceFile(
  'x.ts',
  fs.readFileSync(path.resolve(process.cwd(), './comments.ts'), 'utf-8'),
  ts.ScriptTarget.Latest,
  true, // setParentNodes Важно указать true, если нужно работать с jsDoc комментариями
);

console.log(commentsSource.statements[0]);

Let’s see what AST looks like jsDoc-comments for the hello class method Greetings

AST of the Greetings.hello method

As you can see, it contains information about the text of the comment, as well as an array with tags consisting of one element.

To receive jsDoc– There are special methods for comments and tags in typescript:

// ./src/index.ts
const greetingsClass = commentsSource.statements[0];
if (ts.isClassDeclaration(greetingsClass)) {
  const jsDoc = ts.getJSDocCommentsAndTags(greetingsClass.members[0]);
  console.log(jsDoc[0].comment); // приветствует мир

  const deprecated = ts.getJSDocDeprecatedTag(greetingsClass.members[0])
  console.log(deprecated.comment); // метод устарел
}

For the correct operation of these methods, it is important to specify them during the call createSourceFile the fifth method parameter (setParentNodes) equal to true.

Code generation with jsDoc comments

When analyzing the AST, it can be seen that jsDoc comments are stored in the jsDoc field of NodeObject instances. But the methods of adding this field to an existing object or constructors that accept jsDoc in the API typescript No. This problem is voiced in the issue, and here is one of the generation options typescript code of the previously considered class Greetings:

// ./src/index.ts
/** комментарий для метода hello */
const comment: any = ts.factory.createJSDocComment('приветствует мир', [
  ts.factory.createJSDocUnknownTag(ts.factory.createIdentifier('deprecated')), // тег @deprecated
])

/** метод hello */
const methodDeclaration = ts.factory.createMethodDeclaration(
  undefined,
  undefined,
  ts.factory.createIdentifier('hello'),
  undefined,
  undefined,
  [],
  undefined,
  ts.factory.createBlock(
    [
      ts.factory.createExpressionStatement(
        ts.factory.createCallExpression(
          ts.factory.createPropertyAccessExpression(
            ts.factory.createIdentifier('console'),
            ts.factory.createIdentifier('log')
          ),
          undefined,
          [ts.factory.createStringLiteral('hello world')]
        )
      ),
    ],
    true
  )
)

/** класс Greetings */
const classDeclaration = ts.factory.createClassDeclaration(
  [ts.factory.createModifier(ts.SyntaxKind.ExportKeyword)],
  'Greetings',
  undefined,
  undefined,
  [comment, methodDeclaration]
)

console.log(printer.printNode(ts.EmitHint.Unspecified, classDeclaration, file))

Challenge console.log will render our class at the end Greetings with jsDoc comments for the method hello:

Code generation of the Greetings class with jsDoc comments

The solution is not very elegant, because you have to add any to the comment object, but still allows you to add jsDoc to the method.

Related posts