Skip to content

Custom paramTypes config #582

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions docs/paramTypes.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,8 @@ An object with the following following optional fields:
For example in MySQL using `paramTypes: {quoted: [':']}` would allow you to use `` :`name` `` syntax,
while in Transact-SQL `:"name"` and `:[name]` would work instead.
See [identifier syntax wiki page][] for information about differences in support quoted identifiers.
- **`custom`**: `Array<{ regex: string, key?: (text: string) => string }>`.
An option to implement custom syntax for parameter placeholders. See below for details.

Note that using this config will override the by default supported placeholders types.
For example PL/SQL supports numbered (`:1`) and named (`:name`) placeholders by default.
Expand All @@ -89,5 +91,53 @@ The result will be:

This config option can be used together with [params][] to substitute the placeholders with actual values.

## Custom parameter syntax

Say, you'd like to support the `{name}` parameter placeholders in this SQL:

```sql
SELECT id, fname, age FROM person WHERE lname = {lname} AND age > {age};
```

You can define a regex pattern to match the custom parameters:

```js
paramTypes: {
custom: [{ regex: '\\{[a-zA-Z0-9_]+\\}' }];
}
```

Note the double backslashes. You can get around the double-escaping problem by using `String.raw`:

```js
paramTypes: {
custom: [{ regex: String.raw`\{[a-zA-Z0-9_]+\}` }];
}
```

You can also use the [params][] option to substitute values of these parameters.
However by default the parameter names contain the whole string that is matched by the regex:

```js
params: { '{lname}': 'Doe', '{age}': '25' },
```

To get around this, you can also specify the `key` function to extract the name of the parameter:

```js
paramTypes: {
custom: [{
regex: String.raw`\{[a-zA-Z0-9_]+\}`
key: (text) => text.slice(1, -1), // discard first and last char
}]
}
```

Now you can refer to the parameters by their actual name:

```js
params: { 'lname': 'Doe', 'age': '25' },
```

[params]: ./params.md
[identifier syntax wiki page]: https://github.com/sql-formatter-org/sql-formatter/wiki/identifiers
10 changes: 9 additions & 1 deletion src/lexer/Tokenizer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import { Token, TokenType } from './token.js';
import * as regex from './regexFactory.js';
import { ParamTypes, TokenizerOptions } from './TokenizerOptions.js';
import TokenizerEngine, { TokenRule } from './TokenizerEngine.js';
import { escapeRegExp } from './regexUtil.js';
import { escapeRegExp, patternToRegex } from './regexUtil.js';
import { equalizeWhitespace, Optional } from '../utils.js';
import { NestedComment } from './NestedComment.js';

Expand Down Expand Up @@ -196,6 +196,7 @@ export default class Tokenizer {
typeof paramTypesOverrides?.positional === 'boolean'
? paramTypesOverrides.positional
: cfg.paramTypes?.positional,
custom: paramTypesOverrides?.custom || cfg.paramTypes?.custom || [],
};

return this.validRules([
Expand Down Expand Up @@ -226,6 +227,13 @@ export default class Tokenizer {
type: TokenType.POSITIONAL_PARAMETER,
regex: paramTypes.positional ? /[?]/y : undefined,
},
...paramTypes.custom.map(
(customParam): TokenRule => ({
type: TokenType.CUSTOM_PARAMETER,
regex: patternToRegex(customParam.regex),
key: customParam.key ?? (v => v),
})
),
]);
}

Expand Down
10 changes: 10 additions & 0 deletions src/lexer/TokenizerOptions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,16 @@ export interface ParamTypes {
// Prefixes for quoted parameter placeholders to support, e.g. :"name"
// The type of quotes will depend on `identifierTypes` option.
quoted?: (':' | '@' | '$')[];
// Custom parameter type definitions
custom?: CustomParameter[];
}

export interface CustomParameter {
// Regex pattern for matching the parameter
regex: string;
// Takes the matched parameter string and returns the name of the parameter
// For example we might match "{foo}" and the name would be "foo".
key?: (text: string) => string;
}

export interface TokenizerOptions {
Expand Down
1 change: 1 addition & 0 deletions src/lexer/token.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ export enum TokenType {
QUOTED_PARAMETER = 'QUOTED_PARAMETER',
NUMBERED_PARAMETER = 'NUMBERED_PARAMETER',
POSITIONAL_PARAMETER = 'POSITIONAL_PARAMETER',
CUSTOM_PARAMETER = 'CUSTOM_PARAMETER',
DELIMITER = 'DELIMITER',
EOF = 'EOF',
}
Expand Down
3 changes: 2 additions & 1 deletion src/parser/grammar.ne
Original file line number Diff line number Diff line change
Expand Up @@ -314,7 +314,8 @@ parameter ->
( %NAMED_PARAMETER
| %QUOTED_PARAMETER
| %NUMBERED_PARAMETER
| %POSITIONAL_PARAMETER ) {% ([[token]]) => ({ type: NodeType.parameter, key: token.key, text: token.text }) %}
| %POSITIONAL_PARAMETER
| %CUSTOM_PARAMETER ) {% ([[token]]) => ({ type: NodeType.parameter, key: token.key, text: token.text }) %}

literal ->
( %NUMBER
Expand Down
40 changes: 40 additions & 0 deletions test/options/paramTypes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,4 +56,44 @@ export default function supportsParamTypes(format: FormatFn) {
//
// - it likely works when the other paramTypes tests work
// - it's the config that's least likely to be actually used in practice.

describe('when paramTypes.custom=[...]', () => {
it('replaces %blah% numbered placeholders with param values', () => {
const result = format('SELECT %1%, %2%, %3%;', {
paramTypes: { custom: [{ regex: '%[0-9]+%' }] },
params: { '%1%': 'first', '%2%': 'second', '%3%': 'third' },
});
expect(result).toBe(dedent`
SELECT
first,
second,
third;
`);
});

it('supports custom function for extracting parameter name', () => {
const result = format('SELECT %1%, %2%, %3%;', {
paramTypes: { custom: [{ regex: '%[0-9]+%', key: v => v.slice(1, -1) }] },
params: { '1': 'first', '2': 'second', '3': 'third' },
});
expect(result).toBe(dedent`
SELECT
first,
second,
third;
`);
});

it('supports multiple custom param types', () => {
const result = format('SELECT %1%, {2};', {
paramTypes: { custom: [{ regex: '%[0-9]+%' }, { regex: String.raw`\{[0-9]\}` }] },
params: { '%1%': 'first', '{2}': 'second' },
});
expect(result).toBe(dedent`
SELECT
first,
second;
`);
});
});
}