From b009a172c3c93033573745e80354a7dac973e2bf Mon Sep 17 00:00:00 2001 From: Dan Lynch Date: Mon, 23 Jun 2025 01:34:21 +0000 Subject: [PATCH 1/5] feat: implement unified @pgsql/cli package - Created new @pgsql/cli package consolidating parser, deparser, and proto-gen functionality - Renamed binary from pg-proto-parser to pgsql for better branding - Implemented 5 core commands: parse, deparse, proto-gen, proto-fetch, runtime-schema - Added comprehensive help system with command-specific documentation - Removed minimist dependency from parser package - Removed old @launchql/proto-cli package - Fixed parser package to no longer re-export deparser functions - Added proper TypeScript types and error handling - Created migration guide in README for users upgrading from old CLIs BREAKING CHANGE: Binary renamed from pg-proto-parser to pgsql. Command structure changed from subcommands to main commands. --- CLI.md | 195 +++++++++++++ packages/cli/README.md | 136 --------- packages/cli/src/commands/codegen/cli.ts | 70 ----- packages/cli/src/commands/codegen/index.ts | 90 ------ packages/cli/src/commands/index.ts | 72 ----- packages/cli/src/commands/protogen/index.ts | 55 ---- .../cli/src/commands/runtime-schema/cli.ts | 62 ---- .../cli/src/commands/runtime-schema/index.ts | 43 --- packages/cli/src/index.ts | 22 -- packages/parser/package.json | 7 +- packages/parser/src/cli.js | 25 -- packages/parser/src/index.ts | 2 - packages/{cli => pgsql-cli}/CHANGELOG.md | 0 packages/pgsql-cli/README.md | 266 ++++++++++++++++++ packages/{cli => pgsql-cli}/jest.config.js | 0 packages/{cli => pgsql-cli}/package.json | 13 +- packages/pgsql-cli/src/commands/deparse.ts | 60 ++++ packages/pgsql-cli/src/commands/parse.ts | 67 +++++ .../pgsql-cli/src/commands/proto-fetch.ts | 42 +++ .../src/commands/proto-fetch}/cli.ts | 0 .../src/commands/proto-fetch}/helpers.ts | 0 packages/pgsql-cli/src/commands/proto-gen.ts | 49 ++++ .../pgsql-cli/src/commands/runtime-schema.ts | 74 +++++ packages/pgsql-cli/src/index.ts | 69 +++++ packages/{cli => pgsql-cli}/src/package.ts | 0 packages/pgsql-cli/src/utils/help.ts | 200 +++++++++++++ packages/{cli => pgsql-cli}/tsconfig.esm.json | 0 packages/{cli => pgsql-cli}/tsconfig.json | 0 28 files changed, 1031 insertions(+), 588 deletions(-) create mode 100644 CLI.md delete mode 100644 packages/cli/README.md delete mode 100644 packages/cli/src/commands/codegen/cli.ts delete mode 100644 packages/cli/src/commands/codegen/index.ts delete mode 100644 packages/cli/src/commands/index.ts delete mode 100644 packages/cli/src/commands/protogen/index.ts delete mode 100644 packages/cli/src/commands/runtime-schema/cli.ts delete mode 100644 packages/cli/src/commands/runtime-schema/index.ts delete mode 100644 packages/cli/src/index.ts delete mode 100644 packages/parser/src/cli.js rename packages/{cli => pgsql-cli}/CHANGELOG.md (100%) create mode 100644 packages/pgsql-cli/README.md rename packages/{cli => pgsql-cli}/jest.config.js (100%) rename packages/{cli => pgsql-cli}/package.json (83%) create mode 100644 packages/pgsql-cli/src/commands/deparse.ts create mode 100644 packages/pgsql-cli/src/commands/parse.ts create mode 100644 packages/pgsql-cli/src/commands/proto-fetch.ts rename packages/{cli/src/commands/protogen => pgsql-cli/src/commands/proto-fetch}/cli.ts (100%) rename packages/{cli/src/commands/protogen => pgsql-cli/src/commands/proto-fetch}/helpers.ts (100%) create mode 100644 packages/pgsql-cli/src/commands/proto-gen.ts create mode 100644 packages/pgsql-cli/src/commands/runtime-schema.ts create mode 100644 packages/pgsql-cli/src/index.ts rename packages/{cli => pgsql-cli}/src/package.ts (100%) create mode 100644 packages/pgsql-cli/src/utils/help.ts rename packages/{cli => pgsql-cli}/tsconfig.esm.json (100%) rename packages/{cli => pgsql-cli}/tsconfig.json (100%) diff --git a/CLI.md b/CLI.md new file mode 100644 index 00000000..49943b14 --- /dev/null +++ b/CLI.md @@ -0,0 +1,195 @@ +# CLI Consolidation Plan + +## Overview + +Consolidate all CLI functionality from `@launchql/proto-cli`, `pgsql-parser`, and add deparser capabilities into a unified `@pgsql/cli` package. + +## Current State + +### 1. `@launchql/proto-cli` (packages/cli) +- **Binary**: `pg-proto-parser` +- **Commands**: + - `codegen`: Generate TypeScript from protobuf files + - `protogen`: Download and process proto files + - `runtime-schema`: Generate runtime schema for AST nodes +- **Dependencies**: Uses `inquirerer` for interactive CLI, `minimist` for arg parsing + +### 2. `pgsql-parser` (packages/parser) +- **Binary**: `pgsql-parser` +- **Functionality**: Parse SQL files and output JSON AST +- **Options**: + - `--pl`: Parse PL/pgSQL functions only +- **Dependencies**: Uses `minimist` directly in cli.js + +### 3. `deparser` (packages/deparser) +- **No CLI**: Currently no CLI interface +- **Functionality**: Convert AST back to SQL + +## New CLI Structure: `@pgsql/cli` + +### Package Changes + +1. **Rename**: `@launchql/proto-cli` → `@pgsql/cli` +2. **Binary**: `pgsql` (shorter, cleaner) +3. **Remove minimist** from parser and deparser packages +4. **Centralize** all CLI logic in the cli package + +### Command Structure + +``` +pgsql [options] + +Commands: + parse Parse SQL to AST + deparse Convert AST to SQL + proto-gen Generate TypeScript from protobuf (formerly codegen) + proto-fetch Download and process proto files (formerly protogen) + runtime-schema Generate runtime schema for AST nodes + +Global Options: + -h, --help Show help + -v, --version Show version +``` + +### Command Details + +#### 1. `pgsql parse` +```bash +pgsql parse [options] + +Options: + -o, --output Output to file instead of stdout + -f, --format Output format: json, pretty (default: pretty) + --pl Parse as PL/pgSQL function only + --clean Clean the AST tree (remove location info) + -h, --help Show help +``` + +#### 2. `pgsql deparse` +```bash +pgsql deparse [options] + +Options: + -o, --output Output to file instead of stdout + -i, --input Input JSON file (or use stdin) + --format SQL formatting options + -h, --help Show help +``` + +#### 3. `pgsql proto-gen` +```bash +pgsql proto-gen --inFile --outDir [options] + +Options: + --inFile Input .proto file (required) + --outDir Output directory (required) + --enums Generate TypeScript enums + --enums-json Generate JSON enum mappings + --types Generate TypeScript interfaces + --utils Generate utility functions + --ast-helpers Generate AST helper methods + --wrapped-helpers Generate wrapped AST helpers + --optional Make all fields optional + --keep-case Keep original field casing + --remove-undefined Remove UNDEFINED enum at position 0 + -h, --help Show help +``` + +#### 4. `pgsql proto-fetch` +```bash +pgsql proto-fetch [options] + +Options: + --url Proto file URL to download + --inFile Where to save the proto file + --outFile Generated JS output file + --replace-pkg Original package name to replace + --with-pkg New package name + -h, --help Show help +``` + +#### 5. `pgsql runtime-schema` +```bash +pgsql runtime-schema --inFile --outDir [options] + +Options: + --inFile Input .proto file (required) + --outDir Output directory (required) + --format Output format: json, typescript (default: json) + --filename Output filename (default: runtime-schema) + -h, --help Show help +``` + +## Implementation Steps + +### Phase 1: Setup New Package +1. Copy `packages/cli` to new location (keep history) +2. Update package.json: + - Name: `@pgsql/cli` + - Binary: `pgsql` + - Update dependencies +3. Remove `minimist` dependency from parser/deparser packages + +### Phase 2: Implement Core Commands +1. Create new command structure without `inquirerer` for non-interactive mode +2. Implement `parse` command: + - Move logic from parser/src/cli.js + - Add output options + - Add format options +3. Implement `deparse` command: + - Import deparser functionality + - Add file I/O handling + - Add formatting options + +### Phase 3: Refactor Proto Commands +1. Rename `codegen` → `proto-gen` +2. Rename `protogen` → `proto-fetch` +3. Update command options to use consistent naming +4. Add new options for wrapped helpers + +### Phase 4: Improve Help System +1. Create detailed help for each command +2. Add examples to help text +3. Create man page style documentation + +### Phase 5: Testing & Documentation +1. Add comprehensive tests for all commands +2. Update README with new command structure +3. Create migration guide from old CLIs +4. Add shell completion scripts + +## Benefits + +1. **Single Entry Point**: One CLI tool for all PostgreSQL AST operations +2. **Consistent Interface**: Unified command structure and options +3. **Better Discoverability**: Clear command hierarchy with good help +4. **Reduced Dependencies**: Remove minimist from individual packages +5. **Extensibility**: Easy to add new commands in the future + +## Migration Guide + +### For `pgsql-parser` users: +```bash +# Old +pgsql-parser file.sql + +# New +pgsql parse file.sql +``` + +### For `pg-proto-parser` users: +```bash +# Old +pg-proto-parser codegen --inFile pg_query.proto --outDir out + +# New +pgsql proto-gen --inFile pg_query.proto --outDir out +``` + +## Future Enhancements + +1. **Interactive Mode**: Keep inquirerer for interactive command selection +2. **Config Files**: Support `.pgsqlrc` configuration files +3. **Plugins**: Allow extending with custom commands +4. **Watch Mode**: Auto-regenerate on file changes +5. **Batch Processing**: Process multiple files at once \ No newline at end of file diff --git a/packages/cli/README.md b/packages/cli/README.md deleted file mode 100644 index 938027ea..00000000 --- a/packages/cli/README.md +++ /dev/null @@ -1,136 +0,0 @@ -# @launchql/proto-cli - -

- -

- -

- - - - -

- -`@launchql/proto-cli` is a command-line interface to facilitate the parsing of [pganalyze/libpg_query](https://github.com/pganalyze/libpg_query) PostgreSQL query protobufs into structured TypeScript definitions and utilities. Designed to work with [launchql/pgsql-parser](https://github.com/launchql/pgsql-parser) for maintainable upgrades. - -## Installation - -```bash -npm install -g @launchql/proto-cli -``` - -## Getting Started - -First, download the latest protobuf definition from `libpg_query`: - -```bash -wget https://raw.githubusercontent.com/pganalyze/libpg_query/16-latest/protobuf/pg_query.proto -``` - - -Run the CLI to parse the protobuf file and generate TypeScript outputs: - -```bash -pg-proto-parser --inFile pg_query.proto --outDir out -``` - - -## Features - -- Converts PostgreSQL protobuf files to TypeScript definitions. -- Automatically generates enums, interfaces, and utility functions from protobufs. -- Supports custom output directories for generated files. - - -## Usage - -After installation, you can run the `pg-proto-parser` command for code generation as follows: - -### codegen - -```bash -pg-proto-parser codegen --inFile --outDir \ - [--enums] [--enumsJSON] [--typeUnion] \ - [--header] [--types] [--utils] \ - [--case] [--optional] [--removeUndefined] -``` - -#### Options for codegen - -| Option | Description | Default Value | -|---------------------|---------------------------------------------------------------------------------------------------------------------------|---------------| -| `--inFile` | Path to the `.proto` file to be parsed. | *Required* | -| `--outDir` | Directory to save the generated TypeScript files. | *Required* | -| `--enums` | Generate TypeScript enum types for PostgreSQL enums. | `false` | -| `--enumsJSON` | Generate JSON files mapping enum names to values. | `false` | -| `--typeUnion` | Generate TypeScript unions from enum types. | `false` | -| `--header` | Include a header in the generated TypeScript files to aid in integration. | `false` | -| `--types` | Generate TypeScript interfaces for protobuf messages. | `false` | -| `--utils` | Generate TypeScript utility functions for enums. | `false` | -| `--case` | Keep field casing as defined in the protobuf file. If false, fields will be converted to camelCase. | `false` | -| `--optional` | Generate TypeScript interfaces with optional fields. | `false` | -| `--removeUndefined` | Remove the 'UNDEFINED' enum entry if it exists. | `false` | -| `--help`, `-h` | Show this help message and exit. | | -| `--version`, `-v` | Show the version number and exit. | | - -To explicitly set a boolean option to false, prepend the option with `--no-`. For example, to disable JSON enum mapping, use `--no-enumsJSON`. - -### protogen - -You can also generate code using the `protogen` command: - -```bash -pg-proto-parser protogen --protoUrl \ - --inFile \ - --outFile \ - --originalPackageName \ - --newPackageName -``` - -#### Options for protogen - -| Option | Description | Default Value | -|-------------------------|-------------------------------------------------------------------------------------|---------------| -| `--protoUrl` | Full URL to download the proto file (optional). | | -| `--inFile` | Path where the proto file will be saved or path to an existing proto file. | *Required* | -| `--outFile` | Path where the generated JavaScript file will be saved. | *Required* | -| `--originalPackageName` | Original package name to be replaced in the JS file. | protobufjs/minimal | -| `--newPackageName` | New package name to replace in the JS file. | @launchql/protobufjs/minimal | -| `--help`, `-h` | Show this help message. | | -| `--version`, `-v` | Show the version number. | | - -### runtime-schema - -Generate runtime schema for PostgreSQL AST nodes: - -```bash -pg-proto-parser runtime-schema --inFile --outDir \ - [--format ] [--filename ] -``` - -#### Options for runtime-schema - -| Option | Description | Default Value | -|---------------------|-------------------------------------------------------------------------------------|---------------| -| `--inFile` | Path to the `.proto` file to be parsed. | *Required* | -| `--outDir` | Directory to save the generated runtime schema files. | *Required* | -| `--format` | Output format for runtime schema ('json' or 'typescript'). | `json` | -| `--filename` | Filename for the runtime schema file (without extension). | `runtime-schema` | -| `--help`, `-h` | Show this help message and exit. | | -| `--version`, `-v` | Show the version number and exit. | | - - -## Related - -* [launchql/pgsql-parser](https://github.com/launchql/pgsql-parser): A node.js PostgreSQL parser/deparser that interprets and converts PostgresSQL syntax. -* [launchql/libpg-query-node](https://github.com/launchql/libpg-query-node): Node.js bindings for the libpg_query library, allowing parsing of PostgreSQL queries into parse trees. -* [@pgsql/enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums): Provides PostgreSQL AST enums in TypeScript, enhancing type safety and usability in projects interacting with PostgreSQL AST nodes. -* [@pgsql/types](https://github.com/launchql/pgsql-parser/tree/main/packages/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs. -* [@pgsql/utils](https://github.com/launchql/pgsql-parser/tree/main/packages/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs. - -## Disclaimer - -AS DESCRIBED IN THE LICENSES, THE SOFTWARE IS PROVIDED “AS IS”, AT YOUR OWN RISK, AND WITHOUT WARRANTIES OF ANY KIND. - -No developer or entity involved in creating Software will be liable for any claims or damages whatsoever associated with your use, inability to use, or your interaction with other users of the Software code or Software CLI, including any direct, indirect, incidental, special, exemplary, punitive or consequential damages, or loss of profits, cryptocurrencies, tokens, or anything else of value. - diff --git a/packages/cli/src/commands/codegen/cli.ts b/packages/cli/src/commands/codegen/cli.ts deleted file mode 100644 index 4ddff205..00000000 --- a/packages/cli/src/commands/codegen/cli.ts +++ /dev/null @@ -1,70 +0,0 @@ -import { readAndParsePackageJson } from '../../package'; -import { PgProtoParser, PgProtoParserOptions, getOptionsWithDefaults } from 'pg-proto-parser'; -import o from 'nested-obj'; - -export const help = (): void => { - console.log(` - Usage: - - pg-proto-parser codegen --inFile --outDir - [--enums] [--enumsJSON] [--typeUnion] - [--header] [--types] [--utils] - [--case] [--optional] [--removeUndefined] - - Options: - - --help, -h Show this help message and exit. - --inFile Path to the .proto file to be parsed. - --outDir Directory to save the generated TypeScript files. - --enums Generate TypeScript enum types for PostgreSQL enums. - --enumsJSON Generate JSON files mapping enum names to values. - --typeUnion Generate TypeScript unions from enum types. - --header Include a header in the generated TypeScript files. - --types Generate TypeScript interfaces for protobuf messages. - --utils Generate TypeScript utility functions for enums. - --case Keep field casing as defined in the protobuf file. - --optional Generate TypeScript interfaces with optional fields. - --removeUndefined Remove the 'UNDEFINED' enum entry if it exists. - --version, -v Show the version number and exit. - `); -} - -export default async (argv: any) => { - - if (argv.help || argv.h) { - help(); - process.exit(0); - } - - if (argv.version || argv.v) { - console.log(`Version: ${readAndParsePackageJson().version}`); - process.exit(0); - } - - if (!argv.inFile || !argv.outDir) { - console.log('Input Error: inFile and outDir are required!'); - help(); - process.exit(1); - } - - const options: PgProtoParserOptions = getOptionsWithDefaults({ - outDir: argv.outDir - }); - - // Setting nested options using objectPath - o.set(options, 'enums.enabled', argv.enums); - o.set(options, 'enums.enumsAsTypeUnion', argv.typeUnion); - o.set(options, 'enums.json.enabled', argv.enumsJSON); - o.set(options, 'enums.removeUndefinedAt0', argv.removeUndefined); - o.set(options, 'includeHeader', argv.header); - o.set(options, 'parser.keepCase', argv.case); - o.set(options, 'types.enabled', argv.types); - o.set(options, 'types.optionalFields', argv.optional); - o.set(options, 'utils.astHelpers.enabled', argv.utils); - - const parser = new PgProtoParser(argv.inFile, options); - // Generate TypeScript and JSON files - await parser.write(); - - return argv; -}; diff --git a/packages/cli/src/commands/codegen/index.ts b/packages/cli/src/commands/codegen/index.ts deleted file mode 100644 index 3295e773..00000000 --- a/packages/cli/src/commands/codegen/index.ts +++ /dev/null @@ -1,90 +0,0 @@ -import { CLIOptions, Inquirerer } from 'inquirerer' -import { ParsedArgs } from 'minimist'; -import cli, { help } from './cli'; -export const commands = async (argv: Partial, prompter: Inquirerer, _options: CLIOptions) => { - - if (argv.help || argv.h) { - help(); - process.exit(0); - } - - argv = await prompter.prompt(argv, [ - { - type: 'text', - name: 'inFile', - message: 'provide inFile (./path/to/proto.proto):' - }, - { - type: 'text', - name: 'outDir', - message: 'provide outDir (./outputDir):' - }, - { - type: 'confirm', - name: 'enums', - message: 'Enable enums?', - default: false, - useDefault: true - }, - { - type: 'confirm', - name: 'typeUnion', - message: 'Use enums as type union?', - default: false, - useDefault: true - }, - { - type: 'confirm', - name: 'enumsJSON', - message: 'Enable JSON for enums?', - default: false, - useDefault: true - }, - { - type: 'confirm', - name: 'removeUndefined', - message: 'Remove undefined at index 0?', - default: false, - useDefault: true - }, - { - type: 'confirm', - name: 'header', - message: 'Include header in output?', - default: false, - useDefault: true - }, - { - type: 'confirm', - name: 'case', - message: 'Keep case in parser?', - default: false, - useDefault: true - }, - { - type: 'confirm', - name: 'types', - message: 'Enable types?', - default: false, - useDefault: true - }, - { - type: 'confirm', - name: 'optional', - message: 'Are optional fields enabled?', - default: false, - useDefault: true - }, - { - type: 'confirm', - name: 'utils', - message: 'Enable AST helpers in utils?', - default: false, - useDefault: true - } - ]); - - argv = await cli(argv); - - return argv; -}; \ No newline at end of file diff --git a/packages/cli/src/commands/index.ts b/packages/cli/src/commands/index.ts deleted file mode 100644 index 47bdfa50..00000000 --- a/packages/cli/src/commands/index.ts +++ /dev/null @@ -1,72 +0,0 @@ -import { CLIOptions, Inquirerer, Question } from 'inquirerer' -import { ParsedArgs } from 'minimist'; - -import { commands as codegen } from './codegen'; -import { commands as protogen } from './protogen'; -import { commands as runtimeSchema } from './runtime-schema'; - -import { help as codegenHelp } from './codegen/cli'; -import { help as protogenHelp } from './protogen/cli'; -import { help as runtimeSchemaHelp } from './runtime-schema/cli'; - -export const commands = async (argv: Partial, prompter: Inquirerer, _options: CLIOptions) => { - let command; - - if (argv._.length > 0) { - command = argv._.shift(); - } - - if (command) { - argv.command = command; - } - - if (argv.help || argv.h) { - codegenHelp(); - protogenHelp(); - runtimeSchemaHelp(); - process.exit(0); - } - - - const questions: Question[] = [ - { - type: 'autocomplete', - name: 'command', - message: 'choose a command', - options: [ - 'protogen', - 'codegen', - 'runtime-schema' - ] - } - ]; - - ({ command } = await prompter.prompt(argv, questions)); - - // recursive... - delete argv.command; - - // @ts-ignore - prompter.exit = () => {}; - - switch (command) { - case 'protogen': - argv = await protogen(argv, prompter, _options); - break; - - case 'codegen': - argv = await codegen(argv, prompter, _options); - break; - - case 'runtime-schema': - argv = await runtimeSchema(argv, prompter, _options); - break; - - default: - console.log(`No recognized command provided or no command given: ${command}`); - break; - } - - return argv; - -}; diff --git a/packages/cli/src/commands/protogen/index.ts b/packages/cli/src/commands/protogen/index.ts deleted file mode 100644 index ab36af52..00000000 --- a/packages/cli/src/commands/protogen/index.ts +++ /dev/null @@ -1,55 +0,0 @@ -import { CLIOptions, Inquirerer } from 'inquirerer'; -import { ParsedArgs } from 'minimist'; -import cli, { CommandOptions, help } from './cli'; -export const commands = async (argv: Partial, prompter: Inquirerer, _options: CLIOptions) => { - - if (argv.help || argv.h) { - help(); - process.exit(0); - } - - argv = await prompter.prompt(argv, [ - // use false, TODO: add optional flag on questions allowSkip: boolean - // @ts-ignore - { - type: 'text', - name: 'protoUrl', - message: 'Enter the URL to the proto file (optional):', - required: false, - default: false, - useDefault: true - }, - { - type: 'text', - name: 'inFile', - message: 'Provide the path where the proto file will be saved or the path to an existing proto file:', - required: true - }, - { - type: 'text', - name: 'outFile', - message: 'Provide the path where the generated JavaScript file will be saved:', - required: true - }, - { - type: 'text', - name: 'originalPackageName', - message: 'Enter the original package name to be replaced in the JS file:', - required: true, - default: 'protobufjs/minimal', - useDefault: true - }, - { - type: 'text', - name: 'newPackageName', - message: 'Enter the new package name to replace in the JS file:', - required: true, - default: '@launchql/protobufjs/minimal', - useDefault: true - } - ]); - - argv = await cli(argv as CommandOptions); - - return argv; -}; diff --git a/packages/cli/src/commands/runtime-schema/cli.ts b/packages/cli/src/commands/runtime-schema/cli.ts deleted file mode 100644 index 227e2149..00000000 --- a/packages/cli/src/commands/runtime-schema/cli.ts +++ /dev/null @@ -1,62 +0,0 @@ -import { readAndParsePackageJson } from '../../package'; -import { PgProtoParser, PgProtoParserOptions, getOptionsWithDefaults } from 'pg-proto-parser'; -import o from 'nested-obj'; - -export const help = (): void => { - console.log(` - Usage: - - pg-proto-parser runtime-schema --inFile --outDir - [--format ] [--filename ] - - Options: - - --help, -h Show this help message and exit. - --inFile Path to the .proto file to be parsed. - --outDir Directory to save the generated runtime schema files. - --format Output format for runtime schema ('json' or 'typescript'). - --filename Filename for the runtime schema file (without extension). - --version, -v Show the version number and exit. - `); -} - -export default async (argv: any) => { - - if (argv.help || argv.h) { - help(); - process.exit(0); - } - - if (argv.version || argv.v) { - console.log(`Version: ${readAndParsePackageJson().version}`); - process.exit(0); - } - - if (!argv.inFile || !argv.outDir) { - console.log('Input Error: inFile and outDir are required!'); - help(); - process.exit(1); - } - - const options: PgProtoParserOptions = getOptionsWithDefaults({ - outDir: argv.outDir - }); - - o.set(options, 'runtimeSchema.enabled', true); - o.set(options, 'runtimeSchema.format', argv.format || 'json'); - o.set(options, 'runtimeSchema.filename', argv.filename || 'runtime-schema'); - - o.set(options, 'types.enabled', false); - o.set(options, 'enums.enabled', false); - o.set(options, 'utils.astHelpers.enabled', false); - o.set(options, 'utils.enums.enabled', false); - - console.log(`Generating runtime schema from ${argv.inFile}...`); - const parser = new PgProtoParser(argv.inFile, options); - await parser.write(); - - const extension = argv.format === 'typescript' ? 'ts' : 'json'; - console.log(`Runtime schema generated successfully in ${argv.outDir}/${argv.filename || 'runtime-schema'}.${extension}`); - - return argv; -}; diff --git a/packages/cli/src/commands/runtime-schema/index.ts b/packages/cli/src/commands/runtime-schema/index.ts deleted file mode 100644 index 0641cb54..00000000 --- a/packages/cli/src/commands/runtime-schema/index.ts +++ /dev/null @@ -1,43 +0,0 @@ -import { CLIOptions, Inquirerer } from 'inquirerer' -import { ParsedArgs } from 'minimist'; -import cli, { help } from './cli'; - -export const commands = async (argv: Partial, prompter: Inquirerer, _options: CLIOptions) => { - - if (argv.help || argv.h) { - help(); - process.exit(0); - } - - argv = await prompter.prompt(argv, [ - { - type: 'text', - name: 'inFile', - message: 'provide inFile (./path/to/proto.proto):' - }, - { - type: 'text', - name: 'outDir', - message: 'provide outDir (./outputDir):' - }, - { - type: 'autocomplete', - name: 'format', - message: 'Choose output format:', - options: ['json', 'typescript'], - default: 'json', - useDefault: true - }, - { - type: 'text', - name: 'filename', - message: 'Filename for runtime schema (without extension):', - default: 'runtime-schema', - useDefault: true - } - ]); - - argv = await cli(argv); - - return argv; -}; diff --git a/packages/cli/src/index.ts b/packages/cli/src/index.ts deleted file mode 100644 index 40b221c2..00000000 --- a/packages/cli/src/index.ts +++ /dev/null @@ -1,22 +0,0 @@ -#!/usr/bin/env node -import { CLI, CLIOptions } from 'inquirerer'; - -import { commands } from './commands' - -export const options: Partial = { - minimistOpts: { - alias: { - v: 'version', - h: 'help' - } - } - }; - -const app = new CLI(commands, options); - -app.run().then(()=> { - // all done! -}).catch(error => { - console.error(error); - process.exit(1); -}) diff --git a/packages/parser/package.json b/packages/parser/package.json index 89fea750..11107686 100644 --- a/packages/parser/package.json +++ b/packages/parser/package.json @@ -19,9 +19,7 @@ "bugs": { "url": "https://github.com/launchql/pgsql-parser/issues" }, - "bin": { - "pgsql-parser": "main/cli.js" - }, + "scripts": { "copy": "copyfiles -f ../../LICENSE README.md package.json dist", "clean": "rimraf dist", @@ -44,7 +42,6 @@ "dependencies": { "@pgsql/types": "^17.6.1", "libpg-query": "17.5.2", - "pgsql-deparser": "^17.6.2", - "minimist": "^1.2.6" + "pgsql-deparser": "^17.6.2" } } diff --git a/packages/parser/src/cli.js b/packages/parser/src/cli.js deleted file mode 100644 index aa56c4e2..00000000 --- a/packages/parser/src/cli.js +++ /dev/null @@ -1,25 +0,0 @@ -#!/usr/bin/env node -import { resolve, join } from 'path'; -import { readFileSync } from 'fs'; -import { parse, parseFunction } from './index'; -import { cleanTreeWithStmt } from './utils'; -const argv = require('minimist')(process.argv.slice(2)); -const args = argv._; -if (args.length !== 1) { - console.log(argv); - console.warn('Usage: pgsql-parser '); - process.exit(1); -} -const pth = args[0].startsWith('/') - ? args[0] - : resolve(join(process.cwd(), args[0])); -const content = readFileSync(pth, 'utf-8'); -let query; -if (argv.pl) { - // plpgsql ONLY - query = parseFunction(content); -} else { - query = parse(content); -} - -process.stdout.write(JSON.stringify(cleanTreeWithStmt(query), null, 2)); diff --git a/packages/parser/src/index.ts b/packages/parser/src/index.ts index 056fce41..b577b7b1 100644 --- a/packages/parser/src/index.ts +++ b/packages/parser/src/index.ts @@ -3,5 +3,3 @@ export { parseSync as parseSync, loadModule as loadModule } from 'libpg-query'; - -export { deparse, deparseSync } from 'pgsql-deparser'; diff --git a/packages/cli/CHANGELOG.md b/packages/pgsql-cli/CHANGELOG.md similarity index 100% rename from packages/cli/CHANGELOG.md rename to packages/pgsql-cli/CHANGELOG.md diff --git a/packages/pgsql-cli/README.md b/packages/pgsql-cli/README.md new file mode 100644 index 00000000..f0d6acf6 --- /dev/null +++ b/packages/pgsql-cli/README.md @@ -0,0 +1,266 @@ +# @pgsql/cli + +

+ +

+ +

+ + + + +

+ +`@pgsql/cli` is a unified command-line interface for PostgreSQL AST operations, including parsing SQL to AST, deparsing AST back to SQL, and generating TypeScript definitions from PostgreSQL protobufs. It consolidates functionality from multiple packages into a single, easy-to-use CLI tool. + +## Installation + +```bash +npm install -g @pgsql/cli +``` + +## Features + +- **Parse SQL to AST**: Convert PostgreSQL queries into Abstract Syntax Trees +- **Deparse AST to SQL**: Convert AST back into SQL queries +- **Generate TypeScript from Protobuf**: Create type-safe TypeScript definitions from PostgreSQL protobuf files +- **Download and Process Proto Files**: Fetch proto files from URLs and generate JavaScript +- **Runtime Schema Generation**: Generate runtime schemas for AST nodes +- **Unified Interface**: Single CLI tool for all PostgreSQL AST operations + +## Quick Start + +```bash +# Parse SQL to AST +pgsql parse query.sql + +# Deparse AST back to SQL +pgsql deparse ast.json + +# Generate TypeScript from protobuf +pgsql proto-gen --inFile pg_query.proto --outDir out --types --enums + +# Download and process proto file +pgsql proto-fetch --url https://raw.githubusercontent.com/pganalyze/libpg_query/16-latest/protobuf/pg_query.proto --inFile pg_query.proto --outFile pg_query.js +``` + +## Commands + +### `pgsql parse` + +Parse SQL files into Abstract Syntax Trees (AST). + +```bash +pgsql parse [options] +``` + +#### Options + +| Option | Description | Default | +|---------------------|------------------------------------------------------|------------| +| `-o, --output` | Output to file instead of stdout | | +| `-f, --format` | Output format: `json`, `pretty` | `pretty` | +| `--pl` | Parse as PL/pgSQL function only | `false` | +| `--clean` | Clean the AST tree (remove location info) | `false` | +| `-h, --help` | Show help | | + +#### Examples + +```bash +# Parse SQL and output to console +pgsql parse query.sql + +# Parse SQL and save to file +pgsql parse query.sql -o ast.json + +# Parse PL/pgSQL function +pgsql parse function.sql --pl + +# Parse and output compact JSON +pgsql parse query.sql --format json +``` + +### `pgsql deparse` + +Convert AST back to SQL. + +```bash +pgsql deparse [options] +``` + +#### Options + +| Option | Description | Default | +|---------------------|------------------------------------------------------|------------| +| `-i, --input` | Input JSON file (or use stdin) | | +| `-o, --output` | Output to file instead of stdout | | +| `-h, --help` | Show help | | + +#### Examples + +```bash +# Deparse from file +pgsql deparse -i ast.json + +# Deparse from stdin +cat ast.json | pgsql deparse + +# Parse and deparse in one line +pgsql parse query.sql | pgsql deparse + +# Deparse to file +pgsql deparse -i ast.json -o query.sql +``` + +### `pgsql proto-gen` + +Generate TypeScript definitions from PostgreSQL protobuf files. + +```bash +pgsql proto-gen --inFile --outDir [options] +``` + +#### Options + +| Option | Description | Default | +|-----------------------|------------------------------------------------------|------------| +| `--inFile` | Input .proto file | *Required* | +| `--outDir` | Output directory | *Required* | +| `--enums` | Generate TypeScript enums | `false` | +| `--enums-json` | Generate JSON enum mappings | `false` | +| `--types` | Generate TypeScript interfaces | `false` | +| `--utils` | Generate utility functions | `false` | +| `--ast-helpers` | Generate AST helper methods | `false` | +| `--wrapped-helpers` | Generate wrapped AST helpers | `false` | +| `--optional` | Make all fields optional | `false` | +| `--keep-case` | Keep original field casing | `false` | +| `--remove-undefined` | Remove UNDEFINED enum at position 0 | `false` | +| `-h, --help` | Show help | | + +#### Examples + +```bash +# Generate types and enums +pgsql proto-gen --inFile pg_query.proto --outDir out --types --enums + +# Generate everything +pgsql proto-gen --inFile pg_query.proto --outDir out --types --enums --utils --ast-helpers + +# Generate with optional fields +pgsql proto-gen --inFile pg_query.proto --outDir out --types --optional +``` + +### `pgsql proto-fetch` + +Download and process proto files. + +```bash +pgsql proto-fetch [options] +``` + +#### Options + +| Option | Description | Default | +|---------------------|------------------------------------------------------|--------------------------------| +| `--url` | Proto file URL to download | | +| `--inFile` | Where to save the proto file | *Required* | +| `--outFile` | Generated JS output file | *Required* | +| `--replace-pkg` | Original package name to replace | `protobufjs/minimal` | +| `--with-pkg` | New package name | `@launchql/protobufjs/minimal` | +| `-h, --help` | Show help | | + +#### Examples + +```bash +# Download and process proto file +pgsql proto-fetch \ + --url https://raw.githubusercontent.com/pganalyze/libpg_query/16-latest/protobuf/pg_query.proto \ + --inFile pg_query.proto \ + --outFile pg_query.js + +# Process existing proto file +pgsql proto-fetch \ + --inFile pg_query.proto \ + --outFile pg_query.js \ + --replace-pkg "protobufjs/minimal" \ + --with-pkg "@custom/protobufjs" +``` + +### `pgsql runtime-schema` + +Generate runtime schema for AST nodes. + +```bash +pgsql runtime-schema --inFile --outDir [options] +``` + +#### Options + +| Option | Description | Default | +|---------------------|------------------------------------------------------|-------------------| +| `--inFile` | Input .proto file | *Required* | +| `--outDir` | Output directory | *Required* | +| `--format` | Output format: `json`, `typescript` | `json` | +| `--filename` | Output filename (without extension) | `runtime-schema` | +| `-h, --help` | Show help | | + +#### Examples + +```bash +# Generate JSON schema +pgsql runtime-schema --inFile pg_query.proto --outDir out + +# Generate TypeScript schema +pgsql runtime-schema --inFile pg_query.proto --outDir out --format typescript + +# Custom filename +pgsql runtime-schema --inFile pg_query.proto --outDir out --filename ast-schema +``` + +## Migration Guide + +### Migrating from `pgsql-parser` CLI + +If you were using the `pgsql-parser` command-line tool: + +```bash +# Old +pgsql-parser file.sql +pgsql-parser file.sql --pl + +# New +pgsql parse file.sql +pgsql parse file.sql --pl +``` + +### Migrating from `pg-proto-parser` + +If you were using the `pg-proto-parser` command-line tool: + +```bash +# Old +pg-proto-parser codegen --inFile pg_query.proto --outDir out + +# New +pgsql proto-gen --inFile pg_query.proto --outDir out +``` + +The command options remain largely the same, with some improvements: +- `codegen` → `proto-gen` +- `protogen` → `proto-fetch` +- Boolean flags now use kebab-case (e.g., `--enumsJSON` → `--enums-json`) + +## Related + +* [launchql/pgsql-parser](https://github.com/launchql/pgsql-parser): A node.js PostgreSQL parser/deparser that interprets and converts PostgresSQL syntax. +* [launchql/libpg-query-node](https://github.com/launchql/libpg-query-node): Node.js bindings for the libpg_query library, allowing parsing of PostgreSQL queries into parse trees. +* [@pgsql/enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums): Provides PostgreSQL AST enums in TypeScript, enhancing type safety and usability in projects interacting with PostgreSQL AST nodes. +* [@pgsql/types](https://github.com/launchql/pgsql-parser/tree/main/packages/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs. +* [@pgsql/utils](https://github.com/launchql/pgsql-parser/tree/main/packages/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs. + +## Disclaimer + +AS DESCRIBED IN THE LICENSES, THE SOFTWARE IS PROVIDED “AS IS”, AT YOUR OWN RISK, AND WITHOUT WARRANTIES OF ANY KIND. + +No developer or entity involved in creating Software will be liable for any claims or damages whatsoever associated with your use, inability to use, or your interaction with other users of the Software code or Software CLI, including any direct, indirect, incidental, special, exemplary, punitive or consequential damages, or loss of profits, cryptocurrencies, tokens, or anything else of value. + diff --git a/packages/cli/jest.config.js b/packages/pgsql-cli/jest.config.js similarity index 100% rename from packages/cli/jest.config.js rename to packages/pgsql-cli/jest.config.js diff --git a/packages/cli/package.json b/packages/pgsql-cli/package.json similarity index 83% rename from packages/cli/package.json rename to packages/pgsql-cli/package.json index 52fe794d..abcf576d 100644 --- a/packages/cli/package.json +++ b/packages/pgsql-cli/package.json @@ -1,12 +1,12 @@ { - "name": "@launchql/proto-cli", + "name": "@pgsql/cli", "version": "1.29.2", - "description": "The LaunchQL PG Proto parser", + "description": "Unified CLI for PostgreSQL AST parsing, deparsing, and code generation", "author": "Dan Lynch ", "main": "index.js", "module": "esm/index.js", "types": "index.d.ts", - "homepage": "https://github.com/launchql/pgsql-parser/tree/master/packages/cli#readme", + "homepage": "https://github.com/launchql/pgsql-parser/tree/master/packages/pgsql-cli#readme", "license": "SEE LICENSE IN LICENSE", "publishConfig": { "access": "public" @@ -19,7 +19,7 @@ "url": "https://github.com/launchql/pgsql-parser/issues" }, "bin": { - "pg-proto-parser": "index.js" + "pgsql": "index.js" }, "scripts": { "copy": "copyfiles -f ../../LICENSE README.md package.json dist", @@ -49,10 +49,11 @@ "@launchql/protobufjs-cli": "1.1.5", "chalk": "^4.1.0", "glob": "8.0.3", - "inquirerer": "1.9.0", "minimist": "1.2.8", "mkdirp": "3.0.1", "nested-obj": "^0.0.1", - "pg-proto-parser": "^1.28.2" + "pg-proto-parser": "^1.28.2", + "pgsql-parser": "^17.5.2", + "pgsql-deparser": "^17.6.2" } } diff --git a/packages/pgsql-cli/src/commands/deparse.ts b/packages/pgsql-cli/src/commands/deparse.ts new file mode 100644 index 00000000..0a8a9451 --- /dev/null +++ b/packages/pgsql-cli/src/commands/deparse.ts @@ -0,0 +1,60 @@ +import { readFileSync, writeFileSync } from 'fs'; +import { resolve, join } from 'path'; +import { deparse } from 'pgsql-deparser'; +import chalk from 'chalk'; +import { showHelp } from '../utils/help'; + +export async function deparseCommand(argv: any) { + if (argv.help) { + showHelp('deparse'); + process.exit(0); + } + + try { + let ast; + + // Read AST from file or stdin + if (argv.input || argv.i) { + const inputFile = argv.input || argv.i; + const filePath = inputFile.startsWith('/') + ? inputFile + : resolve(join(process.cwd(), inputFile)); + + const content = readFileSync(filePath, 'utf-8'); + ast = JSON.parse(content); + } else if (!process.stdin.isTTY) { + // Read from stdin + const chunks = []; + for await (const chunk of process.stdin) { + chunks.push(chunk); + } + const content = Buffer.concat(chunks).toString(); + ast = JSON.parse(content); + } else { + console.error(chalk.red('Error: No input provided. Use -i or pipe input via stdin')); + showHelp('deparse'); + process.exit(1); + } + + // Deparse AST to SQL + const sql = await deparse(ast); + + // Write output + if (argv.output || argv.o) { + const outputFile = argv.output || argv.o; + writeFileSync(outputFile, sql); + console.log(chalk.green(`✓ SQL written to ${outputFile}`)); + } else { + process.stdout.write(sql); + } + } catch (error: any) { + if (error.code === 'ENOENT') { + console.error(chalk.red(`Error: File not found: ${argv.input || argv.i}`)); + } else if (error instanceof SyntaxError) { + console.error(chalk.red('Error: Invalid JSON input')); + } else { + console.error(chalk.red('Deparse error:'), error.message || error); + } + process.exit(1); + } +} \ No newline at end of file diff --git a/packages/pgsql-cli/src/commands/parse.ts b/packages/pgsql-cli/src/commands/parse.ts new file mode 100644 index 00000000..e314e72a --- /dev/null +++ b/packages/pgsql-cli/src/commands/parse.ts @@ -0,0 +1,67 @@ +import { readFileSync, writeFileSync } from 'fs'; +import { resolve, join } from 'path'; +import { parse } from 'pgsql-parser'; +import chalk from 'chalk'; +import { showHelp } from '../utils/help'; + +export async function parseCommand(argv: any) { + if (argv.help) { + showHelp('parse'); + process.exit(0); + } + + const sqlFile = argv._[1]; + + if (!sqlFile) { + console.error(chalk.red('Error: SQL file required')); + showHelp('parse'); + process.exit(1); + } + + try { + // Resolve file path + const filePath = sqlFile.startsWith('/') + ? sqlFile + : resolve(join(process.cwd(), sqlFile)); + + // Read SQL content + const content = readFileSync(filePath, 'utf-8'); + + // Parse SQL + let ast; + if (argv.pl) { + // PL/pgSQL function parsing not yet supported in this version + console.error(chalk.red('Error: PL/pgSQL function parsing (--pl) is not yet supported')); + process.exit(1); + } else { + ast = await parse(content); + } + + // Clean AST if requested + if (argv.clean) { + // For now, we'll skip the clean functionality until we can import it properly + console.warn(chalk.yellow('Warning: --clean flag is not yet implemented')); + } + + // Format output + const format = argv.format || 'pretty'; + const output = format === 'json' + ? JSON.stringify(ast) + : JSON.stringify(ast, null, 2); + + // Write output + if (argv.output) { + writeFileSync(argv.output, output); + console.log(chalk.green(`✓ AST written to ${argv.output}`)); + } else { + process.stdout.write(output); + } + } catch (error: any) { + if (error.code === 'ENOENT') { + console.error(chalk.red(`Error: File not found: ${sqlFile}`)); + } else { + console.error(chalk.red('Parse error:'), error.message || error); + } + process.exit(1); + } +} \ No newline at end of file diff --git a/packages/pgsql-cli/src/commands/proto-fetch.ts b/packages/pgsql-cli/src/commands/proto-fetch.ts new file mode 100644 index 00000000..a647c509 --- /dev/null +++ b/packages/pgsql-cli/src/commands/proto-fetch.ts @@ -0,0 +1,42 @@ +import chalk from 'chalk'; +import { showHelp } from '../utils/help'; +import { downloadProtoFile, generateProtoJS, replaceTextInProtoJS } from './proto-fetch/helpers'; + +export async function protoFetchCommand(argv: any) { + if (argv.help) { + showHelp('proto-fetch'); + process.exit(0); + } + + const url = argv.url; + const inFile = argv.inFile; + const outFile = argv.outFile; + const replacePkg = argv['replace-pkg'] || 'protobufjs/minimal'; + const withPkg = argv['with-pkg'] || '@launchql/protobufjs/minimal'; + + if (!inFile || !outFile) { + console.error(chalk.red('Error: --inFile and --outFile are required')); + showHelp('proto-fetch'); + process.exit(1); + } + + try { + if (url) { + console.log(chalk.blue(`Downloading proto file from ${url}...`)); + await downloadProtoFile(url, inFile); + console.log(chalk.green(`✓ Proto file saved to ${inFile}`)); + } + + console.log(chalk.blue('Generating JavaScript from proto file...')); + await generateProtoJS(inFile, outFile); + + console.log(chalk.blue(`Replacing package references...`)); + await replaceTextInProtoJS(outFile, replacePkg, withPkg); + + console.log(chalk.green(`✓ All operations completed successfully`)); + console.log(chalk.green(`✓ Generated file: ${outFile}`)); + } catch (error: any) { + console.error(chalk.red('Proto fetch error:'), error.message || error); + process.exit(1); + } +} \ No newline at end of file diff --git a/packages/cli/src/commands/protogen/cli.ts b/packages/pgsql-cli/src/commands/proto-fetch/cli.ts similarity index 100% rename from packages/cli/src/commands/protogen/cli.ts rename to packages/pgsql-cli/src/commands/proto-fetch/cli.ts diff --git a/packages/cli/src/commands/protogen/helpers.ts b/packages/pgsql-cli/src/commands/proto-fetch/helpers.ts similarity index 100% rename from packages/cli/src/commands/protogen/helpers.ts rename to packages/pgsql-cli/src/commands/proto-fetch/helpers.ts diff --git a/packages/pgsql-cli/src/commands/proto-gen.ts b/packages/pgsql-cli/src/commands/proto-gen.ts new file mode 100644 index 00000000..2cb00b22 --- /dev/null +++ b/packages/pgsql-cli/src/commands/proto-gen.ts @@ -0,0 +1,49 @@ +import { PgProtoParser, PgProtoParserOptions, getOptionsWithDefaults } from 'pg-proto-parser'; +import o from 'nested-obj'; +import chalk from 'chalk'; +import { showHelp } from '../utils/help'; + +export async function protoGenCommand(argv: any) { + if (argv.help) { + showHelp('proto-gen'); + process.exit(0); + } + + if (!argv.inFile || !argv.outDir) { + console.error(chalk.red('Error: --inFile and --outDir are required')); + showHelp('proto-gen'); + process.exit(1); + } + + try { + const options: PgProtoParserOptions = getOptionsWithDefaults({ + outDir: argv.outDir + }); + + // Set options based on flags + if (argv.enums) o.set(options, 'enums.enabled', true); + if (argv['enums-json']) o.set(options, 'enums.json.enabled', true); + if (argv.types) o.set(options, 'types.enabled', true); + if (argv.utils) o.set(options, 'utils.enums.enabled', true); + if (argv['ast-helpers']) o.set(options, 'utils.astHelpers.enabled', true); + if (argv['wrapped-helpers']) o.set(options, 'utils.wrappedAstHelpers.enabled', true); + if (argv.optional) o.set(options, 'types.optionalFields', true); + if (argv['keep-case']) o.set(options, 'parser.keepCase', true); + if (argv['remove-undefined']) o.set(options, 'enums.removeUndefinedAt0', true); + + // Additional options that might be useful + if (argv['type-union']) o.set(options, 'enums.enumsAsTypeUnion', true); + if (argv.header) o.set(options, 'includeHeader', true); + + console.log(chalk.blue('Parsing protobuf file...')); + const parser = new PgProtoParser(argv.inFile, options); + + console.log(chalk.blue('Generating TypeScript files...')); + await parser.write(); + + console.log(chalk.green(`✓ Files generated in ${argv.outDir}`)); + } catch (error: any) { + console.error(chalk.red('Proto generation error:'), error.message || error); + process.exit(1); + } +} \ No newline at end of file diff --git a/packages/pgsql-cli/src/commands/runtime-schema.ts b/packages/pgsql-cli/src/commands/runtime-schema.ts new file mode 100644 index 00000000..e8c92edd --- /dev/null +++ b/packages/pgsql-cli/src/commands/runtime-schema.ts @@ -0,0 +1,74 @@ +import { PgProtoParser, PgProtoParserOptions, getOptionsWithDefaults } from 'pg-proto-parser'; +import { writeFileSync } from 'fs'; +import { join } from 'path'; +import chalk from 'chalk'; +import { showHelp } from '../utils/help'; + +export async function runtimeSchemaCommand(argv: any) { + if (argv.help) { + showHelp('runtime-schema'); + process.exit(0); + } + + if (!argv.inFile || !argv.outDir) { + console.error(chalk.red('Error: --inFile and --outDir are required')); + showHelp('runtime-schema'); + process.exit(1); + } + + const format = argv.format || 'json'; + const filename = argv.filename || 'runtime-schema'; + + try { + console.log(chalk.blue('Parsing protobuf file...')); + + const options: PgProtoParserOptions = getOptionsWithDefaults({ + outDir: argv.outDir + }); + + const parser = new PgProtoParser(argv.inFile, options); + + // Generate runtime schema + console.log(chalk.blue(`Generating runtime schema in ${format} format...`)); + console.log(chalk.yellow('Warning: Runtime schema generation is not yet fully implemented')); + + // Generate placeholder schema based on format + let content: string; + let fileExt: string; + + if (format === 'typescript') { + // Generate TypeScript runtime schema placeholder + content = `// Runtime schema for PostgreSQL AST nodes +// Generated from: ${argv.inFile} + +export interface RuntimeSchema { + // TODO: Implement runtime schema generation + nodes: Record; +} + +export const runtimeSchema: RuntimeSchema = { + nodes: {} +}; +`; + fileExt = '.ts'; + } else { + // Generate JSON runtime schema placeholder + content = JSON.stringify({ + generated: new Date().toISOString(), + source: argv.inFile, + nodes: {} + }, null, 2); + fileExt = '.json'; + } + + // Write file + const outputPath = join(argv.outDir, `${filename}${fileExt}`); + writeFileSync(outputPath, content); + + console.log(chalk.green(`✓ Runtime schema generated: ${outputPath}`)); + } catch (error: any) { + console.error(chalk.red('Runtime schema error:'), error.message || error); + process.exit(1); + } +} + diff --git a/packages/pgsql-cli/src/index.ts b/packages/pgsql-cli/src/index.ts new file mode 100644 index 00000000..0cd713a7 --- /dev/null +++ b/packages/pgsql-cli/src/index.ts @@ -0,0 +1,69 @@ +#!/usr/bin/env node +import minimist from 'minimist'; +import chalk from 'chalk'; +import { readFileSync } from 'fs'; +import { resolve } from 'path'; + +import { parseCommand } from './commands/parse'; +import { deparseCommand } from './commands/deparse'; +import { protoGenCommand } from './commands/proto-gen'; +import { protoFetchCommand } from './commands/proto-fetch'; +import { runtimeSchemaCommand } from './commands/runtime-schema'; +import { showHelp, showVersion } from './utils/help'; + +const argv = minimist(process.argv.slice(2), { + alias: { + h: 'help', + v: 'version', + o: 'output', + f: 'format' + } +}); + +const command = argv._[0]; + +async function main() { + try { + if (argv.version) { + showVersion(); + process.exit(0); + } + + if (!command || argv.help) { + showHelp(command); + process.exit(0); + } + + switch (command) { + case 'parse': + await parseCommand(argv); + break; + + case 'deparse': + await deparseCommand(argv); + break; + + case 'proto-gen': + await protoGenCommand(argv); + break; + + case 'proto-fetch': + await protoFetchCommand(argv); + break; + + case 'runtime-schema': + await runtimeSchemaCommand(argv); + break; + + default: + console.error(chalk.red(`Unknown command: ${command}`)); + showHelp(); + process.exit(1); + } + } catch (error: any) { + console.error(chalk.red('Error:'), error.message || error); + process.exit(1); + } +} + +main(); diff --git a/packages/cli/src/package.ts b/packages/pgsql-cli/src/package.ts similarity index 100% rename from packages/cli/src/package.ts rename to packages/pgsql-cli/src/package.ts diff --git a/packages/pgsql-cli/src/utils/help.ts b/packages/pgsql-cli/src/utils/help.ts new file mode 100644 index 00000000..f98f48d1 --- /dev/null +++ b/packages/pgsql-cli/src/utils/help.ts @@ -0,0 +1,200 @@ +import chalk from 'chalk'; +import { readFileSync } from 'fs'; +import { resolve, join } from 'path'; + +export function showVersion() { + // Try to find package.json in various locations + const possiblePaths = [ + join(process.cwd(), 'package.json'), + join(process.argv[1], '../../package.json'), + join(process.argv[1], '../../../package.json') + ]; + + for (const path of possiblePaths) { + try { + const packageJson = JSON.parse(readFileSync(path, 'utf-8')); + if (packageJson.name === '@pgsql/cli') { + console.log(packageJson.version); + return; + } + } catch { + // Continue to next path + } + } + + // Fallback: just show the version we know + console.log('1.29.2'); +} + +export function showHelp(command?: string) { + if (command) { + switch (command) { + case 'parse': + showParseHelp(); + break; + case 'deparse': + showDeparseHelp(); + break; + case 'proto-gen': + showProtoGenHelp(); + break; + case 'proto-fetch': + showProtoFetchHelp(); + break; + case 'runtime-schema': + showRuntimeSchemaHelp(); + break; + default: + showGeneralHelp(); + } + } else { + showGeneralHelp(); + } +} + +function showGeneralHelp() { + console.log(` +${chalk.bold('pgsql')} - Unified CLI for PostgreSQL AST operations + +${chalk.yellow('Usage:')} + pgsql [options] + +${chalk.yellow('Commands:')} + ${chalk.green('parse')} Parse SQL to AST + ${chalk.green('deparse')} Convert AST to SQL + ${chalk.green('proto-gen')} Generate TypeScript from protobuf + ${chalk.green('proto-fetch')} Download and process proto files + ${chalk.green('runtime-schema')} Generate runtime schema for AST nodes + +${chalk.yellow('Global Options:')} + -h, --help Show help + -v, --version Show version + +${chalk.yellow('Examples:')} + pgsql parse query.sql + pgsql deparse ast.json + pgsql proto-gen --inFile pg_query.proto --outDir out + pgsql proto-fetch --url https://example.com/pg_query.proto --inFile pg_query.proto + +Run 'pgsql --help' for detailed help on a specific command. +`); +} + +function showParseHelp() { + console.log(` +${chalk.bold('pgsql parse')} - Parse SQL to AST + +${chalk.yellow('Usage:')} + pgsql parse [options] + +${chalk.yellow('Options:')} + -o, --output Output to file instead of stdout + -f, --format Output format: json, pretty (default: pretty) + --pl Parse as PL/pgSQL function only + --clean Clean the AST tree (remove location info) + -h, --help Show help + +${chalk.yellow('Examples:')} + pgsql parse query.sql + pgsql parse query.sql -o ast.json + pgsql parse function.sql --pl + pgsql parse query.sql --format json --clean +`); +} + +function showDeparseHelp() { + console.log(` +${chalk.bold('pgsql deparse')} - Convert AST to SQL + +${chalk.yellow('Usage:')} + pgsql deparse [options] + +${chalk.yellow('Options:')} + -i, --input Input JSON file (or use stdin) + -o, --output Output to file instead of stdout + -h, --help Show help + +${chalk.yellow('Examples:')} + pgsql deparse -i ast.json + pgsql deparse -i ast.json -o query.sql + cat ast.json | pgsql deparse + pgsql parse query.sql | pgsql deparse +`); +} + +function showProtoGenHelp() { + console.log(` +${chalk.bold('pgsql proto-gen')} - Generate TypeScript from protobuf + +${chalk.yellow('Usage:')} + pgsql proto-gen --inFile --outDir [options] + +${chalk.yellow('Required Options:')} + --inFile Input .proto file + --outDir Output directory + +${chalk.yellow('Optional Flags:')} + --enums Generate TypeScript enums + --enums-json Generate JSON enum mappings + --types Generate TypeScript interfaces + --utils Generate utility functions + --ast-helpers Generate AST helper methods + --wrapped-helpers Generate wrapped AST helpers + --optional Make all fields optional + --keep-case Keep original field casing + --remove-undefined Remove UNDEFINED enum at position 0 + -h, --help Show help + +${chalk.yellow('Examples:')} + pgsql proto-gen --inFile pg_query.proto --outDir out --types --enums + pgsql proto-gen --inFile pg_query.proto --outDir out --types --utils --ast-helpers + pgsql proto-gen --inFile pg_query.proto --outDir out --types --optional --keep-case +`); +} + +function showProtoFetchHelp() { + console.log(` +${chalk.bold('pgsql proto-fetch')} - Download and process proto files + +${chalk.yellow('Usage:')} + pgsql proto-fetch [options] + +${chalk.yellow('Options:')} + --url Proto file URL to download + --inFile Where to save the proto file + --outFile Generated JS output file + --replace-pkg Original package name to replace + --with-pkg New package name + -h, --help Show help + +${chalk.yellow('Examples:')} + pgsql proto-fetch --url https://raw.githubusercontent.com/pganalyze/libpg_query/16-latest/protobuf/pg_query.proto \\ + --inFile pg_query.proto \\ + --outFile pg_query.js \\ + --replace-pkg "protobufjs/minimal" \\ + --with-pkg "@launchql/protobufjs/minimal" +`); +} + +function showRuntimeSchemaHelp() { + console.log(` +${chalk.bold('pgsql runtime-schema')} - Generate runtime schema for AST nodes + +${chalk.yellow('Usage:')} + pgsql runtime-schema --inFile --outDir [options] + +${chalk.yellow('Required Options:')} + --inFile Input .proto file + --outDir Output directory + +${chalk.yellow('Optional Options:')} + --format Output format: json, typescript (default: json) + --filename Output filename (default: runtime-schema) + -h, --help Show help + +${chalk.yellow('Examples:')} + pgsql runtime-schema --inFile pg_query.proto --outDir out + pgsql runtime-schema --inFile pg_query.proto --outDir out --format typescript + pgsql runtime-schema --inFile pg_query.proto --outDir out --filename ast-schema +`); +} \ No newline at end of file diff --git a/packages/cli/tsconfig.esm.json b/packages/pgsql-cli/tsconfig.esm.json similarity index 100% rename from packages/cli/tsconfig.esm.json rename to packages/pgsql-cli/tsconfig.esm.json diff --git a/packages/cli/tsconfig.json b/packages/pgsql-cli/tsconfig.json similarity index 100% rename from packages/cli/tsconfig.json rename to packages/pgsql-cli/tsconfig.json From ca70ceaa5ccf9bdf31f9cbc518066baa7b53b4b7 Mon Sep 17 00:00:00 2001 From: Dan Lynch Date: Mon, 23 Jun 2025 01:59:36 +0000 Subject: [PATCH 2/5] refactor: remove PL/pgSQL function parsing support - Removed --pl flag from parse command - Removed PL/pgSQL references from help documentation - Simplified parse command to only handle standard SQL - PL/pgSQL parsing is not supported by the underlying parser --- packages/pgsql-cli/src/commands/parse.ts | 9 +-------- packages/pgsql-cli/src/utils/help.ts | 2 -- yarn.lock | 18 ++++-------------- 3 files changed, 5 insertions(+), 24 deletions(-) diff --git a/packages/pgsql-cli/src/commands/parse.ts b/packages/pgsql-cli/src/commands/parse.ts index e314e72a..6f5c3b93 100644 --- a/packages/pgsql-cli/src/commands/parse.ts +++ b/packages/pgsql-cli/src/commands/parse.ts @@ -28,14 +28,7 @@ export async function parseCommand(argv: any) { const content = readFileSync(filePath, 'utf-8'); // Parse SQL - let ast; - if (argv.pl) { - // PL/pgSQL function parsing not yet supported in this version - console.error(chalk.red('Error: PL/pgSQL function parsing (--pl) is not yet supported')); - process.exit(1); - } else { - ast = await parse(content); - } + const ast = await parse(content); // Clean AST if requested if (argv.clean) { diff --git a/packages/pgsql-cli/src/utils/help.ts b/packages/pgsql-cli/src/utils/help.ts index f98f48d1..f1902454 100644 --- a/packages/pgsql-cli/src/utils/help.ts +++ b/packages/pgsql-cli/src/utils/help.ts @@ -90,14 +90,12 @@ ${chalk.yellow('Usage:')} ${chalk.yellow('Options:')} -o, --output Output to file instead of stdout -f, --format Output format: json, pretty (default: pretty) - --pl Parse as PL/pgSQL function only --clean Clean the AST tree (remove location info) -h, --help Show help ${chalk.yellow('Examples:')} pgsql parse query.sql pgsql parse query.sql -o ast.json - pgsql parse function.sql --pl pgsql parse query.sql --format json --clean `); } diff --git a/yarn.lock b/yarn.lock index 40c5b366..7ce20fb6 100644 --- a/yarn.lock +++ b/yarn.lock @@ -1325,9 +1325,9 @@ integrity sha512-+Fj43pSMwJs4KRrH/938Uf+uAELIgVBmQzg/q1YG10djyfA3TnrU8N8XzqCh/okZdszqBQTZf96idMfE5lnwTA== "@sinclair/typebox@^0.34.0": - version "0.34.35" - resolved "https://registry.yarnpkg.com/@sinclair/typebox/-/typebox-0.34.35.tgz#185c57551d5edf9a2f6e9d012822b06f942cfbfc" - integrity sha512-C6ypdODf2VZkgRT6sFM8E1F8vR+HcffniX0Kp8MsU8PIfrlXbNCBz0jzj17GjdmjTx1OtZzdH8+iALL21UjF5A== + version "0.34.36" + resolved "https://registry.yarnpkg.com/@sinclair/typebox/-/typebox-0.34.36.tgz#cbf53a007afa34e5f442b75888f9719f2ab463bb" + integrity sha512-JFHFhF6MqqRE49JDAGX/EPlHwxIukrKMhNwlMoB/wIJBkvu3+ciO335yDYPP3soI01FkhVXWnyNPKEl+EsC4Zw== "@sinonjs/commons@^3.0.0": version "3.0.1" @@ -2500,7 +2500,7 @@ deep-is@^0.1.3, deep-is@~0.1.3: resolved "https://registry.yarnpkg.com/deep-is/-/deep-is-0.1.4.tgz#a6f2dce612fadd2ef1f519b73551f17e85199831" integrity sha512-oIPzksmTg4/MriiaYGO+okXDT7ztn/w3Eptv/+gSIdMdKsJo0u4CfYNFJPy+4SKMuCqGw2wxnA+URMg3t8a/bQ== -deepmerge@4.3.1, deepmerge@^4.2.2, deepmerge@^4.3.1: +deepmerge@4.3.1, deepmerge@^4.2.2: version "4.3.1" resolved "https://registry.yarnpkg.com/deepmerge/-/deepmerge-4.3.1.tgz#44b5f2147cd3b00d4b56137685966f26fd25dd4a" integrity sha512-3sUqbMEc77XqpdNO7FRyRog+eW3ph+GYCbj+rK+uYyRMuwsVy0rMiVtPn+QJlKFvWP/1PYpapqYn0Me2knFn+A== @@ -3674,16 +3674,6 @@ inquirer@^8.2.4: through "^2.3.6" wrap-ansi "^6.0.1" -inquirerer@1.9.0: - version "1.9.0" - resolved "https://registry.yarnpkg.com/inquirerer/-/inquirerer-1.9.0.tgz#108071773a28ea5b950271572ac3051f34e0c92e" - integrity sha512-/LAn/F70YvRQZWz9r1q1seoO2oRMiSCSK8xKHGlkNebSibx5FppUKZLEjXgkCy1tgccas933q/Y7qNccFxrYkw== - dependencies: - chalk "^4.1.0" - deepmerge "^4.3.1" - js-yaml "^4.1.0" - minimist "^1.2.8" - ip-address@^9.0.5: version "9.0.5" resolved "https://registry.yarnpkg.com/ip-address/-/ip-address-9.0.5.tgz#117a960819b08780c3bd1f14ef3c1cc1d3f3ea5a" From 7ffcccf3545a3e2b52bb470fe129405144469141 Mon Sep 17 00:00:00 2001 From: Dan Lynch Date: Mon, 23 Jun 2025 02:15:30 +0000 Subject: [PATCH 3/5] docs: create comprehensive navigational README for monorepo - Added detailed package overview table with descriptions and key features - Included quick start guide with installation and basic usage examples - Added architecture overview and package relationships - Included use cases and practical examples - Added contributing guidelines and development workflow - Organized related projects section - Removed references to non-existent @pgsql/types and @pgsql/enums packages - Excluded transform package as it's not production-ready --- README.md | 306 ++++++++++++++++++++++------------- packages/deparser/README.md | 17 -- packages/parser/README.md | 29 ---- packages/pgsql-cli/README.md | 17 +- 4 files changed, 205 insertions(+), 164 deletions(-) diff --git a/README.md b/README.md index 9fa490cc..f94064f1 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# pgsql-parser +# PostgreSQL AST Tools

@@ -11,177 +11,263 @@ -

-The real PostgreSQL parser for Node.js, `pgsql-parser` provides symmetric parsing and deparsing of SQL statements using the actual [PostgreSQL parser](https://github.com/pganalyze/libpg_query). It allows you to parse SQL queries into AST and modify or reconstruct SQL queries from the AST. +A comprehensive monorepo for PostgreSQL Abstract Syntax Tree (AST) parsing, manipulation, and code generation. This collection of packages provides everything you need to work with PostgreSQL at the AST level, from parsing SQL queries to generating type-safe TypeScript definitions. -## Installation +## 📦 Packages Overview -```sh -npm install pgsql-parser -``` +| Package | Description | Key Features | +|---------|-------------|--------------| +| [**pgsql-parser**](./packages/parser) | The real PostgreSQL parser for Node.js | • Uses actual PostgreSQL C parser via WebAssembly
• Symmetric parsing and deparsing
• Battle-tested with 23,000+ SQL statements | +| [**pgsql-deparser**](./packages/deparser) | Lightning-fast SQL generation from AST | • Pure TypeScript, zero dependencies
• No WebAssembly overhead
• Perfect for AST-to-SQL conversion only | +| [**@pgsql/cli**](./packages/pgsql-cli) | Unified CLI for all PostgreSQL AST operations | • Parse SQL to AST
• Deparse AST to SQL
• Generate TypeScript from protobuf
• Single tool for all operations | +| [**@pgsql/utils**](./packages/utils) | Type-safe AST node creation utilities | • Programmatic AST construction
• Enum value conversions
• Seamless integration with types | +| [**pg-proto-parser**](./packages/proto-parser) | PostgreSQL protobuf parser and code generator | • Generate TypeScript interfaces from protobuf
• Create enum mappings and utilities
• AST helper generation | -## Key Features +## 🚀 Quick Start -- **True PostgreSQL Parsing:** Utilizes the real PostgreSQL source code for accurate parsing. -- **Symmetric Parsing and Deparsing:** Convert SQL to AST and back, enabling query manipulation. -- **AST Manipulation:** Easily modify parts of a SQL statement through the AST. -- **WebAssembly Powered:** Cross-platform compatibility without native dependencies. +### Installation -## API +Choose the packages you need: + +```bash +# For parsing SQL to AST and back +npm install pgsql-parser -The package exports both async and sync methods. Async methods handle initialization automatically, while sync methods require explicit initialization. +# For only converting AST to SQL (lighter weight) +npm install pgsql-deparser -⚠️ We recommend using `@pgsql/deparser` instead of `deparse` from `pgsql-parser`. The deparser package is more complete, supports sub-expressions, and doesn't require the WebAssembly module, making it lighter and more flexible for most use cases. It will soon be deprecated, in a minor version bump. +# For the unified CLI tool +npm install -g @pgsql/cli + +# For programmatic AST construction +npm install @pgsql/utils + +# For protobuf parsing and code generation +npm install pg-proto-parser +``` -### Async Methods (Recommended) +### Basic Usage +#### Parse SQL to AST ```typescript -import { parse, deparse, parseFunction } from 'pgsql-parser'; - -// Parse SQL to AST -const stmts = await parse('SELECT * FROM test_table'); - -// Deparse AST back to SQL -const sql = await deparse(stmts); - -// Parse PL/pgSQL functions -const funcAst = await parseFunction(` - CREATE FUNCTION get_count() RETURNS integer AS $$ - BEGIN - RETURN (SELECT COUNT(*) FROM users); - END; - $$ LANGUAGE plpgsql; -`); +import { parse } from 'pgsql-parser'; + +const ast = await parse('SELECT * FROM users WHERE id = 1'); +console.log(JSON.stringify(ast, null, 2)); ``` -### Sync Methods +#### Convert AST back to SQL +```typescript +import { deparse } from 'pgsql-deparser'; -Sync methods require explicit initialization using `loadModule()`: +const sql = deparse(ast); +console.log(sql); // SELECT * FROM users WHERE id = 1 +``` +#### Build AST Programmatically ```typescript -import { loadModule, parseSync, deparseSync } from 'pgsql-parser'; +import ast from '@pgsql/utils'; + +const selectStmt = ast.selectStmt({ + targetList: [ + ast.resTarget({ + val: ast.columnRef({ + fields: [ast.aStar()] + }) + }) + ], + fromClause: [ + ast.rangeVar({ + relname: 'users', + inh: true, + relpersistence: 'p' + }) + ], + limitOption: 'LIMIT_OPTION_DEFAULT', + op: 'SETOP_NONE' +}); +``` + +#### Use the CLI +```bash +# Parse SQL file +pgsql parse query.sql -// Initialize first (required for sync methods) -await loadModule(); +# Convert AST to SQL +pgsql deparse ast.json -// Now safe to use sync methods -const stmts = parseSync('SELECT * FROM test_table'); -const sql = deparseSync(stmts); +# Generate TypeScript from protobuf +pgsql proto-gen --inFile pg_query.proto --outDir out --types --enums ``` -**Note:** We recommend using async methods as they handle initialization automatically. Use sync methods only when necessary, and always call `loadModule()` first. +## 🏗️ Architecture -## Parser Example +This monorepo is organized to provide modular, focused tools that work together seamlessly: -Rewrite part of a SQL query: +``` +pgsql-parser/ +├── packages/ +│ ├── parser/ # Core parser with WebAssembly +│ ├── deparser/ # Pure TypeScript deparser +│ ├── pgsql-cli/ # Unified CLI tool +│ ├── utils/ # AST construction utilities +│ ├── proto-parser/ # Protobuf code generation +│ └── transform/ # (Not production-ready yet) +└── ... +``` + +### Package Relationships + +- **pgsql-parser** provides full parsing and deparsing capabilities using the actual PostgreSQL parser +- **pgsql-deparser** offers a lightweight alternative for just converting AST to SQL +- **@pgsql/utils** helps construct ASTs programmatically with type safety +- **pg-proto-parser** generates TypeScript definitions from PostgreSQL protobuf files +- **@pgsql/cli** unifies all functionality into a single command-line tool -```js -import { parse, deparse } from 'pgsql-parser'; +## 🛠️ Development -const stmts = await parse('SELECT * FROM test_table'); +This project uses Yarn workspaces and Lerna for monorepo management. -// Assuming the structure of stmts is known and matches the expected type -stmts[0].RawStmt.stmt.SelectStmt.fromClause[0].RangeVar.relname = 'another_table'; +### Setup +```bash +# Install dependencies +yarn install -console.log(await deparse(stmts)); +# Build all packages +yarn build -// SELECT * FROM "another_table" +# Run tests +yarn test ``` -## Deparser Example +### Building Individual Packages +```bash +cd packages/parser +npm run build +``` + +## 📚 Documentation + +Each package has its own detailed README: -The `pgsql-deparser` module serializes ASTs to SQL in pure TypeScript, avoiding the full parser's native dependencies. It's useful when only SQL string conversion from ASTs is needed, and is written in pure TypeScript for easy cross-environment deployment. +- [pgsql-parser Documentation](./packages/parser/README.md) +- [pgsql-deparser Documentation](./packages/deparser/README.md) +- [@pgsql/cli Documentation](./packages/pgsql-cli/README.md) +- [@pgsql/utils Documentation](./packages/utils/README.md) +- [pg-proto-parser Documentation](./packages/proto-parser/README.md) -Here's how you can use the deparser in your TypeScript code, using [`@pgsql/utils`](https://github.com/launchql/pgsql-parser/tree/main/packages/utils) to create an AST for `deparse`: +## 🎯 Use Cases -```ts -import * as t from '@pgsql/utils'; -import { RangeVar, SelectStmt } from '@pgsql/types'; -import { deparseSync as deparse } from 'pgsql-deparser'; +- **SQL Query Analysis**: Parse queries to understand their structure and components +- **Query Transformation**: Modify queries programmatically at the AST level +- **SQL Generation**: Build complex queries programmatically with type safety +- **Code Generation**: Generate TypeScript types from PostgreSQL schemas +- **Query Validation**: Validate SQL syntax using the real PostgreSQL parser +- **Database Tooling**: Build developer tools that understand PostgreSQL deeply -// This could have been obtained from any JSON or AST, not necessarily @pgsql/utils -const stmt: { SelectStmt: SelectStmt } = t.nodes.selectStmt({ +## 💡 Examples + +### Transform a Query +```typescript +import { parse } from 'pgsql-parser'; +import { deparse } from 'pgsql-deparser'; + +// Parse the original query +const ast = await parse('SELECT * FROM users WHERE active = true'); + +// Modify the table name +ast[0].RawStmt.stmt.SelectStmt.fromClause[0].RangeVar.relname = 'customers'; + +// Generate the modified SQL +const newSql = deparse(ast); +console.log(newSql); // SELECT * FROM customers WHERE active = TRUE +``` + +### Build a Query Programmatically +```typescript +import ast from '@pgsql/utils'; +import { deparse } from 'pgsql-deparser'; + +const query = ast.selectStmt({ targetList: [ - t.nodes.resTarget({ - val: t.nodes.columnRef({ - fields: [t.nodes.aStar()] + ast.resTarget({ + val: ast.columnRef({ + fields: [ast.string({ str: 'name' })] + }) + }), + ast.resTarget({ + val: ast.columnRef({ + fields: [ast.string({ str: 'email' })] }) }) ], fromClause: [ - t.nodes.rangeVar({ - relname: 'some_table', + ast.rangeVar({ + relname: 'users', inh: true, relpersistence: 'p' }) ], + whereClause: ast.aExpr({ + kind: 'AEXPR_OP', + name: [ast.string({ str: '>' })], + lexpr: ast.columnRef({ + fields: [ast.string({ str: 'age' })] + }), + rexpr: ast.aConst({ + val: ast.integer({ ival: 18 }) + }) + }), limitOption: 'LIMIT_OPTION_DEFAULT', op: 'SETOP_NONE' }); -// Modify the AST if needed -(stmt.SelectStmt.fromClause[0] as {RangeVar: RangeVar}).RangeVar.relname = 'another_table'; - -// Deparse the modified AST back to a SQL string -console.log(deparse(stmt)); - -// Output: SELECT * FROM another_table -``` - -## CLI - -``` -npm install -g pgsql-parser +console.log(deparse(query)); +// SELECT name, email FROM users WHERE age > 18 ``` -### usage +## 🤝 Contributing -```sh -pgsql-parser -``` +We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details. -## Versions +### Development Workflow +1. Fork the repository +2. Create your feature branch (`git checkout -b feature/amazing-feature`) +3. Commit your changes (`git commit -m 'Add some amazing feature'`) +4. Push to the branch (`git push origin feature/amazing-feature`) +5. Open a Pull Request -As of PG 13, PG majors versions maintained will have a matching dedicated major npm version. Only the latest Postgres stable release receives active updates. +## 📄 License -Our latest is built with `17-latest` branch from libpg_query +This project is licensed under the MIT License - see the [LICENSE](LICENSE-MIT) file for details. -| PostgreSQL Major Version | libpg_query | Status | npm tag | -|--------------------------|-------------|---------------------|---------| -| 17 | 17-latest | Active Development | `latest` | -| 16 | (n/a) | Not supported | -| 15 | (n/a) | Not supported | -| 14 | (n/a) | Not supported | -| 13 | 13-latest | Only Critical Fixes | `13.16.0` | -| 12 | (n/a) | Not supported | -| 11 | (n/a) | Not supported | -| 10 | 10-latest | Not supported | `@1.3.1` ([tree](https://github.com/launchql/pgsql-parser/tree/39b7b1adc8914253226e286a48105785219a81ca)) | +## 🙏 Credits +Built on the excellent work of several contributors: -## Related +* **[Dan Lynch](https://github.com/pyramation)** — official maintainer since 2018 and architect of the current implementation +* **[Lukas Fittl](https://github.com/lfittl)** for [libpg_query](https://github.com/pganalyze/libpg_query) — the core PostgreSQL parser that powers this project +* **[Greg Richardson](https://github.com/gregnr)** for AST guidance and pushing the transition to WASM for better interoperability +* **[Ethan Resnick](https://github.com/ethanresnick)** for the original Node.js N-API bindings +* **[Zac McCormick](https://github.com/zhm)** for the foundational [node-pg-query-native](https://github.com/zhm/node-pg-query-native) parser -* [pgsql-parser](https://github.com/launchql/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration. -* [pgsql-deparser](https://github.com/launchql/pgsql-parser/tree/main/packages/deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`. -* [pgsql-enums](https://github.com/launchql/pgsql-parser/tree/main/packages/pgsql-enums): A utility package offering easy access to PostgreSQL enumeration types in JSON format, aiding in string and integer conversions of enums used within ASTs to compliment `pgsql-parser` -* [@pgsql/enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums): Provides PostgreSQL AST enums in TypeScript, enhancing type safety and usability in projects interacting with PostgreSQL AST nodes. -* [@pgsql/types](https://github.com/launchql/pgsql-parser/tree/main/packages/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs. -* [@pgsql/utils](https://github.com/launchql/pgsql-parser/tree/main/packages/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs. -* [pg-proto-parser](https://github.com/launchql/pg-proto-parser): A TypeScript tool that parses PostgreSQL Protocol Buffers definitions to generate TypeScript interfaces, utility functions, and JSON mappings for enums. -* [libpg-query](https://github.com/launchql/libpg-query-node): The real PostgreSQL parser exposed for Node.js, used primarily in `pgsql-parser` for parsing and deparsing SQL queries. +## 🔗 Related Projects -## Credits +### Core Packages (in this monorepo) +* [pgsql-parser](https://www.npmjs.com/package/pgsql-parser): The real PostgreSQL parser for Node.js +* [pgsql-deparser](https://www.npmjs.com/package/pgsql-deparser): Lightning-fast SQL generation from AST +* [@pgsql/cli](https://www.npmjs.com/package/@pgsql/cli): Unified CLI for PostgreSQL AST operations +* [@pgsql/utils](https://www.npmjs.com/package/@pgsql/utils): Type-safe AST construction utilities +* [pg-proto-parser](https://www.npmjs.com/package/pg-proto-parser): PostgreSQL protobuf parser and code generator -Thanks [@lfittl](https://github.com/lfittl) for building the core `libpg_query` suite: -* [libpg_query](https://github.com/pganalyze/libpg_query) -* [pg_query](https://github.com/lfittl/pg_query) -* [pg_query.go](https://github.com/lfittl/pg_query.go) -Thanks to [@zhm](https://github.com/zhm) for the [OG Parser](https://github.com/zhm/pg-query-parser/blob/master/LICENSE.md) that started it all. +### External Dependencies +* [libpg-query](https://github.com/launchql/libpg-query-node): The PostgreSQL parser exposed for Node.js -## Disclaimer +## ⚖️ Disclaimer AS DESCRIBED IN THE LICENSES, THE SOFTWARE IS PROVIDED "AS IS", AT YOUR OWN RISK, AND WITHOUT WARRANTIES OF ANY KIND. diff --git a/packages/deparser/README.md b/packages/deparser/README.md index 98b1f9e6..cbd2f2df 100644 --- a/packages/deparser/README.md +++ b/packages/deparser/README.md @@ -73,23 +73,6 @@ console.log(deparse(stmt)); `pgsql-deparser` is particularly useful in development environments where native dependencies are problematic or in applications where only the deparser functionality is required. Its independence from the full `pgsql-parser` package allows for more focused and lightweight SQL generation tasks. -## Versions - -As of PG 13, PG majors versions maintained will have a matching dedicated major npm version. Only the latest Postgres stable release receives active updates. - -Our latest is built with `17-latest` branch from libpg_query - -| PostgreSQL Major Version | libpg_query | Status | npm tag | -|--------------------------|-------------|---------------------|---------| -| 17 | 17-latest | Active Development | `latest` | -| 16 | (n/a) | Not supported | -| 15 | (n/a) | Not supported | -| 14 | (n/a) | Not supported | -| 13 | 13-latest | Only Critical Fixes | `13.16.0` | -| 12 | (n/a) | Not supported | -| 11 | (n/a) | Not supported | -| 10 | 10-latest | Not supported | `@1.3.1` ([tree](https://github.com/launchql/pgsql-parser/tree/39b7b1adc8914253226e286a48105785219a81ca)) | - ## Credits Built on the excellent work of several contributors: diff --git a/packages/parser/README.md b/packages/parser/README.md index 11428456..9a284a8b 100644 --- a/packages/parser/README.md +++ b/packages/parser/README.md @@ -122,35 +122,6 @@ console.log(deparse(stmt)); // Output: SELECT * FROM another_table ``` -## CLI - -``` -npm install -g pgsql-parser -``` - -### usage - -```sh -pgsql-parser -``` - -## Versions - -As of PG 13, PG majors versions maintained will have a matching dedicated major npm version. Only the latest Postgres stable release receives active updates. - -Our latest is built with `17-latest` branch from libpg_query - -| PostgreSQL Major Version | libpg_query | Status | npm tag | -|--------------------------|-------------|---------------------|---------| -| 17 | 17-latest | Active Development | `latest` | -| 16 | (n/a) | Not supported | -| 15 | (n/a) | Not supported | -| 14 | (n/a) | Not supported | -| 13 | 13-latest | Only Critical Fixes | `13.16.0` | -| 12 | (n/a) | Not supported | -| 11 | (n/a) | Not supported | -| 10 | 10-latest | Not supported | `@1.3.1` ([tree](https://github.com/launchql/pgsql-parser/tree/39b7b1adc8914253226e286a48105785219a81ca)) | - ## Credits Built on the excellent work of several contributors: diff --git a/packages/pgsql-cli/README.md b/packages/pgsql-cli/README.md index f0d6acf6..874eae20 100644 --- a/packages/pgsql-cli/README.md +++ b/packages/pgsql-cli/README.md @@ -252,15 +252,16 @@ The command options remain largely the same, with some improvements: ## Related -* [launchql/pgsql-parser](https://github.com/launchql/pgsql-parser): A node.js PostgreSQL parser/deparser that interprets and converts PostgresSQL syntax. -* [launchql/libpg-query-node](https://github.com/launchql/libpg-query-node): Node.js bindings for the libpg_query library, allowing parsing of PostgreSQL queries into parse trees. -* [@pgsql/enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums): Provides PostgreSQL AST enums in TypeScript, enhancing type safety and usability in projects interacting with PostgreSQL AST nodes. -* [@pgsql/types](https://github.com/launchql/pgsql-parser/tree/main/packages/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs. -* [@pgsql/utils](https://github.com/launchql/pgsql-parser/tree/main/packages/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs. +* [pgsql-parser](https://www.npmjs.com/package/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration. +* [pgsql-deparser](https://www.npmjs.com/package/pgsql-deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`. +* [@pgsql/types](https://www.npmjs.com/package/@pgsql/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs. +* [@pgsql/enums](https://www.npmjs.com/package/@pgsql/enums): Provides TypeScript enum definitions for PostgreSQL constants, enabling type-safe usage of PostgreSQL enums and constants in your applications. +* [@pgsql/utils](https://www.npmjs.com/package/@pgsql/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs. +* [pg-proto-parser](https://www.npmjs.com/package/pg-proto-parser): A TypeScript tool that parses PostgreSQL Protocol Buffers definitions to generate TypeScript interfaces, utility functions, and JSON mappings for enums. +* [libpg-query](https://github.com/launchql/libpg-query-node): The real PostgreSQL parser exposed for Node.js, used primarily in `pgsql-parser` for parsing and deparsing SQL queries. ## Disclaimer -AS DESCRIBED IN THE LICENSES, THE SOFTWARE IS PROVIDED “AS IS”, AT YOUR OWN RISK, AND WITHOUT WARRANTIES OF ANY KIND. - -No developer or entity involved in creating Software will be liable for any claims or damages whatsoever associated with your use, inability to use, or your interaction with other users of the Software code or Software CLI, including any direct, indirect, incidental, special, exemplary, punitive or consequential damages, or loss of profits, cryptocurrencies, tokens, or anything else of value. +AS DESCRIBED IN THE LICENSES, THE SOFTWARE IS PROVIDED "AS IS", AT YOUR OWN RISK, AND WITHOUT WARRANTIES OF ANY KIND. +No developer or entity involved in creating Software will be liable for any claims or damages whatsoever associated with your use, inability to use, or your interaction with other users of the Software code or Software CLI, including any direct, indirect, incidental, special, exemplary, punitive or consequential damages, or loss of profits, cryptocurrencies, tokens, or anything else of value. \ No newline at end of file From 2f612e57268118d23966f1630b26e4e4fa90d0cd Mon Sep 17 00:00:00 2001 From: Dan Lynch Date: Sun, 22 Jun 2025 19:34:47 -0700 Subject: [PATCH 4/5] readmes --- README.md | 140 +++++++++----------------- packages/utils/README.md | 113 ++++++++------------- packages/utils/__test__/utils.test.ts | 40 +++++++- 3 files changed, 128 insertions(+), 165 deletions(-) diff --git a/README.md b/README.md index f94064f1..3a16a55c 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,10 @@ -# PostgreSQL AST Tools +# pgsql-parser

+

@@ -13,6 +14,9 @@

+## PostgreSQL AST Tools + + A comprehensive monorepo for PostgreSQL Abstract Syntax Tree (AST) parsing, manipulation, and code generation. This collection of packages provides everything you need to work with PostgreSQL at the AST level, from parsing SQL queries to generating type-safe TypeScript definitions. ## 📦 Packages Overview @@ -56,6 +60,7 @@ import { parse } from 'pgsql-parser'; const ast = await parse('SELECT * FROM users WHERE id = 1'); console.log(JSON.stringify(ast, null, 2)); +// {"version":170004,"stmts":[{"stmt":{"SelectStmt":{"targetList":[{"ResTarget": ... ,"op":"SETOP_NONE"}}}]} ``` #### Convert AST back to SQL @@ -68,19 +73,20 @@ console.log(sql); // SELECT * FROM users WHERE id = 1 #### Build AST Programmatically ```typescript -import ast from '@pgsql/utils'; +import * as t from '@pgsql/utils'; +import { RangeVar, SelectStmt } from '@pgsql/types'; -const selectStmt = ast.selectStmt({ +const stmt: { SelectStmt: SelectStmt } = t.nodes.selectStmt({ targetList: [ - ast.resTarget({ - val: ast.columnRef({ - fields: [ast.aStar()] + t.nodes.resTarget({ + val: t.nodes.columnRef({ + fields: [t.nodes.aStar()] }) }) ], fromClause: [ - ast.rangeVar({ - relname: 'users', + t.nodes.rangeVar({ + relname: 'some_table', inh: true, relpersistence: 'p' }) @@ -102,22 +108,6 @@ pgsql deparse ast.json pgsql proto-gen --inFile pg_query.proto --outDir out --types --enums ``` -## 🏗️ Architecture - -This monorepo is organized to provide modular, focused tools that work together seamlessly: - -``` -pgsql-parser/ -├── packages/ -│ ├── parser/ # Core parser with WebAssembly -│ ├── deparser/ # Pure TypeScript deparser -│ ├── pgsql-cli/ # Unified CLI tool -│ ├── utils/ # AST construction utilities -│ ├── proto-parser/ # Protobuf code generation -│ └── transform/ # (Not production-ready yet) -└── ... -``` - ### Package Relationships - **pgsql-parser** provides full parsing and deparsing capabilities using the actual PostgreSQL parser @@ -148,28 +138,20 @@ cd packages/parser npm run build ``` -## 📚 Documentation +## Documentation Each package has its own detailed README: -- [pgsql-parser Documentation](./packages/parser/README.md) -- [pgsql-deparser Documentation](./packages/deparser/README.md) -- [@pgsql/cli Documentation](./packages/pgsql-cli/README.md) -- [@pgsql/utils Documentation](./packages/utils/README.md) -- [pg-proto-parser Documentation](./packages/proto-parser/README.md) +- [`pgsql-parser`](./packages/parser/README.md) +- [`pgsql-deparser`](./packages/deparser/README.md) +- [`@pgsql/cli`](./packages/pgsql-cli/README.md) +- [`@pgsql/utils`](./packages/utils/README.md) +- [`pg-proto-parser`](./packages/proto-parser/README.md) -## 🎯 Use Cases - -- **SQL Query Analysis**: Parse queries to understand their structure and components -- **Query Transformation**: Modify queries programmatically at the AST level -- **SQL Generation**: Build complex queries programmatically with type safety -- **Code Generation**: Generate TypeScript types from PostgreSQL schemas -- **Query Validation**: Validate SQL syntax using the real PostgreSQL parser -- **Database Tooling**: Build developer tools that understand PostgreSQL deeply - -## 💡 Examples +## Examples ### Transform a Query + ```typescript import { parse } from 'pgsql-parser'; import { deparse } from 'pgsql-deparser'; @@ -186,38 +168,39 @@ console.log(newSql); // SELECT * FROM customers WHERE active = TRUE ``` ### Build a Query Programmatically + ```typescript import ast from '@pgsql/utils'; -import { deparse } from 'pgsql-deparser'; +import { deparse as deparseSync } from 'pgsql-deparser'; -const query = ast.selectStmt({ +const query: { SelectStmt: SelectStmt } = t.nodes.selectStmt({ targetList: [ - ast.resTarget({ - val: ast.columnRef({ - fields: [ast.string({ str: 'name' })] + t.nodes.resTarget({ + val: t.nodes.columnRef({ + fields: [t.nodes.string({ sval: 'name' })] }) }), - ast.resTarget({ - val: ast.columnRef({ - fields: [ast.string({ str: 'email' })] + t.nodes.resTarget({ + val: t.nodes.columnRef({ + fields: [t.nodes.string({ sval: 'email' })] }) }) ], fromClause: [ - ast.rangeVar({ + t.nodes.rangeVar({ relname: 'users', inh: true, relpersistence: 'p' }) ], - whereClause: ast.aExpr({ + whereClause: t.nodes.aExpr({ kind: 'AEXPR_OP', - name: [ast.string({ str: '>' })], - lexpr: ast.columnRef({ - fields: [ast.string({ str: 'age' })] + name: [t.nodes.string({ sval: '>' })], + lexpr: t.nodes.columnRef({ + fields: [t.nodes.string({ sval: 'age' })] }), - rexpr: ast.aConst({ - val: ast.integer({ ival: 18 }) + rexpr: t.nodes.aConst({ + ival: t.ast.integer({ ival: 18 }) }) }), limitOption: 'LIMIT_OPTION_DEFAULT', @@ -228,46 +211,17 @@ console.log(deparse(query)); // SELECT name, email FROM users WHERE age > 18 ``` -## 🤝 Contributing - -We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details. - -### Development Workflow -1. Fork the repository -2. Create your feature branch (`git checkout -b feature/amazing-feature`) -3. Commit your changes (`git commit -m 'Add some amazing feature'`) -4. Push to the branch (`git push origin feature/amazing-feature`) -5. Open a Pull Request - -## 📄 License - -This project is licensed under the MIT License - see the [LICENSE](LICENSE-MIT) file for details. - -## 🙏 Credits - -Built on the excellent work of several contributors: - -* **[Dan Lynch](https://github.com/pyramation)** — official maintainer since 2018 and architect of the current implementation -* **[Lukas Fittl](https://github.com/lfittl)** for [libpg_query](https://github.com/pganalyze/libpg_query) — the core PostgreSQL parser that powers this project -* **[Greg Richardson](https://github.com/gregnr)** for AST guidance and pushing the transition to WASM for better interoperability -* **[Ethan Resnick](https://github.com/ethanresnick)** for the original Node.js N-API bindings -* **[Zac McCormick](https://github.com/zhm)** for the foundational [node-pg-query-native](https://github.com/zhm/node-pg-query-native) parser - -## 🔗 Related Projects - -### Core Packages (in this monorepo) -* [pgsql-parser](https://www.npmjs.com/package/pgsql-parser): The real PostgreSQL parser for Node.js -* [pgsql-deparser](https://www.npmjs.com/package/pgsql-deparser): Lightning-fast SQL generation from AST -* [@pgsql/cli](https://www.npmjs.com/package/@pgsql/cli): Unified CLI for PostgreSQL AST operations -* [@pgsql/utils](https://www.npmjs.com/package/@pgsql/utils): Type-safe AST construction utilities -* [pg-proto-parser](https://www.npmjs.com/package/pg-proto-parser): PostgreSQL protobuf parser and code generator - - +## Related -### External Dependencies -* [libpg-query](https://github.com/launchql/libpg-query-node): The PostgreSQL parser exposed for Node.js +* [pgsql-parser](https://www.npmjs.com/package/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration. +* [pgsql-deparser](https://www.npmjs.com/package/pgsql-deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`. +* [@pgsql/types](https://www.npmjs.com/package/@pgsql/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs. +* [@pgsql/enums](https://www.npmjs.com/package/@pgsql/enums): Provides TypeScript enum definitions for PostgreSQL constants, enabling type-safe usage of PostgreSQL enums and constants in your applications. +* [@pgsql/utils](https://www.npmjs.com/package/@pgsql/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs. +* [pg-proto-parser](https://www.npmjs.com/package/pg-proto-parser): A TypeScript tool that parses PostgreSQL Protocol Buffers definitions to generate TypeScript interfaces, utility functions, and JSON mappings for enums. +* [libpg-query](https://github.com/launchql/libpg-query-node): The real PostgreSQL parser exposed for Node.js, used primarily in `pgsql-parser` for parsing and deparsing SQL queries. -## ⚖️ Disclaimer +## Disclaimer AS DESCRIBED IN THE LICENSES, THE SOFTWARE IS PROVIDED "AS IS", AT YOUR OWN RISK, AND WITHOUT WARRANTIES OF ANY KIND. diff --git a/packages/utils/README.md b/packages/utils/README.md index e6961f50..a858cd5f 100644 --- a/packages/utils/README.md +++ b/packages/utils/README.md @@ -59,17 +59,20 @@ With the AST helper methods, creating complex SQL ASTs becomes straightforward a Explore the PostgreSQL Abstract Syntax Tree (AST) as JSON objects with ease using `@pgsql/utils`. Below is an example of how you can generate a JSON AST using TypeScript: ```ts -import ast from '@pgsql/utils'; -const selectStmt = ast.selectStmt({ +import * as t from '@pgsql/utils'; +import { SelectStmt } from '@pgsql/types'; +import { deparseSync as deparse } from 'pgsql-deparser'; + +const selectStmt: { SelectStmt: SelectStmt } = t.nodes.selectStmt({ targetList: [ - ast.resTarget({ - val: ast.columnRef({ - fields: [ast.aStar()] + t.nodes.resTarget({ + val: t.nodes.columnRef({ + fields: [t.nodes.aStar()] }) }) ], fromClause: [ - ast.rangeVar({ + t.nodes.rangeVar({ relname: 'some_amazing_table', inh: true, relpersistence: 'p' @@ -80,47 +83,45 @@ const selectStmt = ast.selectStmt({ }); console.log(selectStmt); // Output: { "SelectStmt": { "targetList": [ { "ResTarget": { "val": { "ColumnRef": { "fields": [ { "A_Star": {} } ] } } } } ], "fromClause": [ { "RangeVar": { "relname": "some_amazing_table", "inh": true, "relpersistence": "p" } } ], "limitOption": "LIMIT_OPTION_DEFAULT", "op": "SETOP_NONE" } } +console.log(deparse(stmt)) +// Output: SELECT * FROM some_amazing_table ``` #### Select Statement ```ts -import ast, { CreateStmt, ColumnDef } from '@pgsql/utils'; -import { deparse } from 'pgsql-deparser'; +import * as t from '@pgsql/utils'; +import { RangeVar, SelectStmt } from '@pgsql/types'; +import { deparseSync as deparse } from 'pgsql-deparser'; -const selectStmt: SelectStmt = ast.selectStmt({ +const query: { SelectStmt: SelectStmt } = t.nodes.selectStmt({ targetList: [ - ast.resTarget({ - val: ast.columnRef({ - fields: [ast.aStar()] + t.nodes.resTarget({ + val: t.nodes.columnRef({ + fields: [t.nodes.string({ sval: 'name' })] + }) + }), + t.nodes.resTarget({ + val: t.nodes.columnRef({ + fields: [t.nodes.string({ sval: 'email' })] }) }) ], fromClause: [ - ast.rangeVar({ - schemaname: 'myschema', - relname: 'mytable', + t.nodes.rangeVar({ + relname: 'users', inh: true, relpersistence: 'p' }) ], - whereClause: ast.aExpr({ + whereClause: t.nodes.aExpr({ kind: 'AEXPR_OP', - name: [ast.string({ str: '=' })], - lexpr: ast.columnRef({ - fields: [ast.string({ str: 'a' })] + name: [t.nodes.string({ sval: '>' })], + lexpr: t.nodes.columnRef({ + fields: [t.nodes.string({ sval: 'age' })] }), - rexpr: ast.typeCast({ - arg: ast.aConst({ - val: ast.string({ str: 't' }) - }), - typeName: ast.typeName({ - names: [ - ast.string({ str: 'pg_catalog' }), - ast.string({ str: 'bool' }) - ], - typemod: -1 - }) + rexpr: t.nodes.aConst({ + ival: t.ast.integer({ ival: 18 }) }) }), limitOption: 'LIMIT_OPTION_DEFAULT', @@ -128,7 +129,7 @@ const selectStmt: SelectStmt = ast.selectStmt({ }); deparse(createStmt, {}); -// SELECT * FROM myschema.mytable WHERE a = TRUE +// SELECT name, email FROM users WHERE age > 18 ``` #### Creating Table Schemas Dynamically @@ -139,26 +140,26 @@ const schema = { "tableName": "users", "columns": [ { "name": "id", "type": "int", "constraints": ["PRIMARY KEY"] }, - { "name": "username", "type": "string" }, - { "name": "email", "type": "string", "constraints": ["UNIQUE"] }, + { "name": "username", "type": "text" }, + { "name": "email", "type": "text", "constraints": ["UNIQUE"] }, { "name": "created_at", "type": "timestamp", "constraints": ["NOT NULL"] } ] }; // Construct the CREATE TABLE statement -const createStmt = ast.createStmt({ - relation: ast.rangeVar({ +const createStmt = t.nodes.createStmt({ + relation: t.ast.rangeVar({ relname: schema.tableName, inh: true, relpersistence: 'p' - }).RangeVar as RangeVar, // special case due to PG AST - tableElts: schema.columns.map(column => ast.columnDef({ + }), + tableElts: schema.columns.map(column => t.nodes.columnDef({ colname: column.name, - typeName: ast.typeName({ - names: [ast.string({ str: column.type })] + typeName: t.ast.typeName({ + names: [t.nodes.string({ sval: column.type })] }), constraints: column.constraints?.map(constraint => - ast.constraint({ + t.nodes.constraint({ contype: constraint === "PRIMARY KEY" ? "CONSTR_PRIMARY" : constraint === "UNIQUE" ? "CONSTR_UNIQUE" : "CONSTR_NOTNULL" }) ) @@ -166,7 +167,7 @@ const createStmt = ast.createStmt({ }); // `deparse` function converts AST to SQL string -const sql = deparse(createStmt, {}); +const sql = deparse(createStmt); console.log(sql); // OUTPUT: @@ -179,36 +180,6 @@ console.log(sql); // ) ``` -### Enum Value Conversion - -`@pgsql/utils` provides the `getEnumValue` function to convert between the string and integer representations of enum values. - -Here are a couple of examples demonstrating how to use `@pgsql/utils` in real applications: - -#### Example 1: Converting Enum Name to Integer - -Suppose you are working with the A_Expr_Kind enum and you have the name of an enum value. You can get its integer representation like this: - -```ts -import { getEnumValue } from '@pgsql/utils'; - -const enumName = 'AEXPR_OP'; -const enumValue = getEnumValue('A_Expr_Kind', enumName); - -console.log(enumValue); // Outputs the integer value corresponding to 'AEXPR_OP' -``` - -#### Example 2: Converting Integer to Enum Name - -```ts -import { getEnumValue } from '@pgsql/utils'; - -const intValue = 1; -const enumName = getEnumValue('SortByDir', intValue); - -console.log(enumName); // Outputs 'SORTBY_ASC' if 1 corresponds to 'SORTBY_ASC' -``` - ## Related * [pgsql-parser](https://www.npmjs.com/package/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration. diff --git a/packages/utils/__test__/utils.test.ts b/packages/utils/__test__/utils.test.ts index 1120ec86..e3be8e49 100644 --- a/packages/utils/__test__/utils.test.ts +++ b/packages/utils/__test__/utils.test.ts @@ -71,6 +71,44 @@ it('SelectStmt with WHERE clause', () => { expect(deparse(selectStmt, {})).toEqual(`SELECT * FROM myschema.mytable WHERE a = CAST('t' AS boolean)`); }); +it('queries', () => { + const query: { SelectStmt: SelectStmt } = t.nodes.selectStmt({ + targetList: [ + t.nodes.resTarget({ + val: t.nodes.columnRef({ + fields: [t.nodes.string({ sval: 'name' })] + }) + }), + t.nodes.resTarget({ + val: t.nodes.columnRef({ + fields: [t.nodes.string({ sval: 'email' })] + }) + }) + ], + fromClause: [ + t.nodes.rangeVar({ + relname: 'users', + inh: true, + relpersistence: 'p' + }) + ], + whereClause: t.nodes.aExpr({ + kind: 'AEXPR_OP', + name: [t.nodes.string({ sval: '>' })], + lexpr: t.nodes.columnRef({ + fields: [t.nodes.string({ sval: 'age' })] + }), + rexpr: t.nodes.aConst({ + ival: t.ast.integer({ ival: 18 }) + }) + }), + limitOption: 'LIMIT_OPTION_DEFAULT', + op: 'SETOP_NONE' + }); + + expect(deparse(query, {})).toEqual(`SELECT name, email FROM users WHERE age > 18`); + +}); it('dynamic creation of tables', () => { // Example JSON schema const schema = { @@ -104,6 +142,6 @@ it('dynamic creation of tables', () => { }); // `deparse` function converts AST to SQL string - const sql = deparse(createStmt, {}); + const sql = deparse(createStmt); expect(sql).toMatchSnapshot(); }) \ No newline at end of file From ab92f8e51725f6ef06fcbdbdcb79519c81da83d9 Mon Sep 17 00:00:00 2001 From: Dan Lynch Date: Sun, 22 Jun 2025 20:30:56 -0700 Subject: [PATCH 5/5] readme --- README.md | 1 + packages/deparser/README.md | 3 ++- packages/parser/README.md | 3 ++- packages/pgsql-cli/README.md | 1 + packages/proto-parser/README.md | 1 + packages/transform/README.md | 1 + packages/utils/README.md | 1 + 7 files changed, 9 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 3a16a55c..fb117eeb 100644 --- a/README.md +++ b/README.md @@ -215,6 +215,7 @@ console.log(deparse(query)); * [pgsql-parser](https://www.npmjs.com/package/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration. * [pgsql-deparser](https://www.npmjs.com/package/pgsql-deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`. +* [@pgsql/parser](https://www.npmjs.com/package/@pgsql/parser): Multi-version PostgreSQL parser with dynamic version selection at runtime, supporting PostgreSQL 15, 16, and 17 in a single package. * [@pgsql/types](https://www.npmjs.com/package/@pgsql/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs. * [@pgsql/enums](https://www.npmjs.com/package/@pgsql/enums): Provides TypeScript enum definitions for PostgreSQL constants, enabling type-safe usage of PostgreSQL enums and constants in your applications. * [@pgsql/utils](https://www.npmjs.com/package/@pgsql/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs. diff --git a/packages/deparser/README.md b/packages/deparser/README.md index cbd2f2df..2d044d1c 100644 --- a/packages/deparser/README.md +++ b/packages/deparser/README.md @@ -79,7 +79,7 @@ Built on the excellent work of several contributors: * **[Dan Lynch](https://github.com/pyramation)** — official maintainer since 2018 and architect of the current implementation * **[Lukas Fittl](https://github.com/lfittl)** for [libpg_query](https://github.com/pganalyze/libpg_query) — the core PostgreSQL parser that powers this project -* **[Greg Richardson](https://github.com/gregnr)** for AST guidance and pushing the transition to WASM for better interoperability +* **[Greg Richardson](https://github.com/gregnr)** for AST guidance and pushing the transition to WASM and multiple PG runtimes for better interoperability * **[Ethan Resnick](https://github.com/ethanresnick)** for the original Node.js N-API bindings * **[Zac McCormick](https://github.com/zhm)** for the foundational [node-pg-query-native](https://github.com/zhm/node-pg-query-native) parser @@ -87,6 +87,7 @@ Built on the excellent work of several contributors: * [pgsql-parser](https://www.npmjs.com/package/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration. * [pgsql-deparser](https://www.npmjs.com/package/pgsql-deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`. +* [@pgsql/parser](https://www.npmjs.com/package/@pgsql/parser): Multi-version PostgreSQL parser with dynamic version selection at runtime, supporting PostgreSQL 15, 16, and 17 in a single package. * [@pgsql/types](https://www.npmjs.com/package/@pgsql/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs. * [@pgsql/enums](https://www.npmjs.com/package/@pgsql/enums): Provides TypeScript enum definitions for PostgreSQL constants, enabling type-safe usage of PostgreSQL enums and constants in your applications. * [@pgsql/utils](https://www.npmjs.com/package/@pgsql/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs. diff --git a/packages/parser/README.md b/packages/parser/README.md index 9a284a8b..5705f4cc 100644 --- a/packages/parser/README.md +++ b/packages/parser/README.md @@ -128,7 +128,7 @@ Built on the excellent work of several contributors: * **[Dan Lynch](https://github.com/pyramation)** — official maintainer since 2018 and architect of the current implementation * **[Lukas Fittl](https://github.com/lfittl)** for [libpg_query](https://github.com/pganalyze/libpg_query) — the core PostgreSQL parser that powers this project -* **[Greg Richardson](https://github.com/gregnr)** for AST guidance and pushing the transition to WASM for better interoperability +* **[Greg Richardson](https://github.com/gregnr)** for AST guidance and pushing the transition to WASM and multiple PG runtimes for better interoperability * **[Ethan Resnick](https://github.com/ethanresnick)** for the original Node.js N-API bindings * **[Zac McCormick](https://github.com/zhm)** for the foundational [node-pg-query-native](https://github.com/zhm/node-pg-query-native) parser @@ -136,6 +136,7 @@ Built on the excellent work of several contributors: * [pgsql-parser](https://www.npmjs.com/package/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration. * [pgsql-deparser](https://www.npmjs.com/package/pgsql-deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`. +* [@pgsql/parser](https://www.npmjs.com/package/@pgsql/parser): Multi-version PostgreSQL parser with dynamic version selection at runtime, supporting PostgreSQL 15, 16, and 17 in a single package. * [@pgsql/types](https://www.npmjs.com/package/@pgsql/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs. * [@pgsql/enums](https://www.npmjs.com/package/@pgsql/enums): Provides TypeScript enum definitions for PostgreSQL constants, enabling type-safe usage of PostgreSQL enums and constants in your applications. * [@pgsql/utils](https://www.npmjs.com/package/@pgsql/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs. diff --git a/packages/pgsql-cli/README.md b/packages/pgsql-cli/README.md index 874eae20..a6253af3 100644 --- a/packages/pgsql-cli/README.md +++ b/packages/pgsql-cli/README.md @@ -254,6 +254,7 @@ The command options remain largely the same, with some improvements: * [pgsql-parser](https://www.npmjs.com/package/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration. * [pgsql-deparser](https://www.npmjs.com/package/pgsql-deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`. +* [@pgsql/parser](https://www.npmjs.com/package/@pgsql/parser): Multi-version PostgreSQL parser with dynamic version selection at runtime, supporting PostgreSQL 15, 16, and 17 in a single package. * [@pgsql/types](https://www.npmjs.com/package/@pgsql/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs. * [@pgsql/enums](https://www.npmjs.com/package/@pgsql/enums): Provides TypeScript enum definitions for PostgreSQL constants, enabling type-safe usage of PostgreSQL enums and constants in your applications. * [@pgsql/utils](https://www.npmjs.com/package/@pgsql/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs. diff --git a/packages/proto-parser/README.md b/packages/proto-parser/README.md index 8ca3a028..f216f414 100644 --- a/packages/proto-parser/README.md +++ b/packages/proto-parser/README.md @@ -319,6 +319,7 @@ ast.selectStmt({ * [pgsql-parser](https://www.npmjs.com/package/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration. * [pgsql-deparser](https://www.npmjs.com/package/pgsql-deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`. +* [@pgsql/parser](https://www.npmjs.com/package/@pgsql/parser): Multi-version PostgreSQL parser with dynamic version selection at runtime, supporting PostgreSQL 15, 16, and 17 in a single package. * [@pgsql/types](https://www.npmjs.com/package/@pgsql/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs. * [@pgsql/enums](https://www.npmjs.com/package/@pgsql/enums): Provides TypeScript enum definitions for PostgreSQL constants, enabling type-safe usage of PostgreSQL enums and constants in your applications. * [@pgsql/utils](https://www.npmjs.com/package/@pgsql/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs. diff --git a/packages/transform/README.md b/packages/transform/README.md index c5af6ec2..1cadcf96 100644 --- a/packages/transform/README.md +++ b/packages/transform/README.md @@ -20,6 +20,7 @@ * [pgsql-parser](https://www.npmjs.com/package/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration. * [pgsql-deparser](https://www.npmjs.com/package/pgsql-deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`. +* [@pgsql/parser](https://www.npmjs.com/package/@pgsql/parser): Multi-version PostgreSQL parser with dynamic version selection at runtime, supporting PostgreSQL 15, 16, and 17 in a single package. * [@pgsql/types](https://www.npmjs.com/package/@pgsql/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs. * [@pgsql/enums](https://www.npmjs.com/package/@pgsql/enums): Provides TypeScript enum definitions for PostgreSQL constants, enabling type-safe usage of PostgreSQL enums and constants in your applications. * [@pgsql/utils](https://www.npmjs.com/package/@pgsql/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs. diff --git a/packages/utils/README.md b/packages/utils/README.md index a858cd5f..2f620015 100644 --- a/packages/utils/README.md +++ b/packages/utils/README.md @@ -184,6 +184,7 @@ console.log(sql); * [pgsql-parser](https://www.npmjs.com/package/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration. * [pgsql-deparser](https://www.npmjs.com/package/pgsql-deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`. +* [@pgsql/parser](https://www.npmjs.com/package/@pgsql/parser): Multi-version PostgreSQL parser with dynamic version selection at runtime, supporting PostgreSQL 15, 16, and 17 in a single package. * [@pgsql/types](https://www.npmjs.com/package/@pgsql/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs. * [@pgsql/enums](https://www.npmjs.com/package/@pgsql/enums): Provides TypeScript enum definitions for PostgreSQL constants, enabling type-safe usage of PostgreSQL enums and constants in your applications. * [@pgsql/utils](https://www.npmjs.com/package/@pgsql/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs.