Fix schema lint script to understand CREATE TABLE IF NOT EXISTS

[erikjohnston]

The schema lint tries to make sure we don't add or remove indices in schema files (rather than as background updates), unless the table was created in the same schema file.

The regex to pull out the CREATE TABLE SQL incorrectly didn't recognise IF NOT EXISTS.

There is a test delta file that shows that we accept different types of CREATE TABLE and CREATE INDEX statements, as well as an index creation that doesn't have a matching create table (to show that we do still catch it). The test delta should be removed before merge.

[MadLittleMods]

This is separate from the lint being updated but this spawned some discussion in an internal room about whether IF NOT EXISTS was desirable.

On one hand, @sandhose mentions how IF NOT EXISTS could be a foot-gun as the table may exist but not have the same shape that we have in the schema.

@richvdh also noted some historical context that we may have used IF NOT EXISTS in some places because migration files weren't applied atomically, so in some edge-cases you could end up running them twice. Although that may be fixed now.

Basically, it seems that if we have fixed the atomicity problem of deltas, we should also have a new lint that prevents people from using IF NOT EXISTS. Something for the future ⏩

[richvdh]

@richvdh also noted some historical context that we may have used IF NOT EXISTS in some places because migration files weren't applied atomically, so in some edge-cases you could end up running them twice. Although that may be fixed now.

I think we fixed it: matrix-org/synapse#6467

[MadLittleMods]

I think there are more edge cases. Perhaps some of these would be better prevented with some SQL lints 🤷

Create table using another table

It's also possible to create a table using another table:

CREATE TABLE new_table_name AS
    SELECT column1, column2,...
    FROM existing_table_name
    WHERE ....;

Or:

CREATE TABLE new_table (LIKE old_table);

Quoted identifiers

It looks like it's also possible to put quotes around the identifiers:

CREATE TABLE "My_Table_ABC" ("My_Very_Upper_and_Lowercasy_Column" INT, ...)

Create table over multiple lines

CREATE TABLE users
(
    id INT,
    name VARCHAR(255)
);

[MadLittleMods]

I think this is better than before so we can merge ⏩

But I think there are a few more edge cases this doesn't handle.