Aggregate query with null value returns incorrect results

[mtrezza]

Issue Description

An aggregate query with $in that contains a null value does not return the correct results.

Steps to reproduce

This MongoDB query executes fine in the shell:

db.MyCollection.aggregate([
    {
        $match: {
            "language": { "$in": [null, "en"]}
        }
    }
]).pretty()

The query returns documents where the language field is missing or has value null or en.
The same query with the Parse SDK in Javascript does not yield the same results:

const query = new Parse.Query("MyCollection");
const pipeline = {
    "match":{"language":{"$in":[null,"en"]}},
};
const results = await query.aggregate(pipeline);

The query returns only documents where the language field has value en, but not where the field is missing or the the field value is null.

Expected Results

The query with Parse SDK should return the same results as the MongoDB shell query.

Actual Outcome

The query returns only documents where the language field has value en, but not where the field is missing or the the field value is null.

Environment Setup

• Server

  • parse-server version (Be specific! Don't say 'latest'.) : 3.8.0
  • Operating System: -
  • Hardware: -
  • Localhost or remote server? (AWS, Heroku, Azure, Digital Ocean, etc): mLab

• JS SDK

  • JS SDK version: 2.7.0
  • Application? (Browser, Node, React-Native, etc): -

Logs/Trace

[unicornist]

I recently had a similar issue in parse-server 4.4.0 which was related to the type of the field.
for example, if the field is a pointer a simple null match will work as expected like this:

query.aggregate([
    { match: {
      _p_pointerField: null
    } }
])

but if the field type is Date, the same query will not work as expected:

query.aggregate([
    { match: {
      dateField: null
    } }
])

but it's still working in mongo db shell though so I am sure there is a bug that is not fixed by the PR for this issue:

db.collection.aggregate([
    { $match: {
      dateField: null
    } }
])

[unicornist]

Even I tried to find those documents by comparing by their types like below examples:

query.aggregate([ { match: {
  dateField: { $type: 'missing' }
} }  ])

query.aggregate([ { match: {
  dateField: { $type: 'date' }
} }  ])

query.aggregate([ { match: {
  dateField: { $type: 9 }
} }  ])

query.aggregate([ { match: {
  dateField: { $type: { $ne: 'date' } }
} }  ])

although these are working in mongo db shell, none of the above cases worked in parse server, and in all cases, I've got the following console error in an interval without crashing the app:

error: Uncaught internal server error. type must be represented as a number or a string {"code":14,"codeName":"TypeMismatch","name":"MongoError","ok":0,"stack":"MongoError: type must be represented as a number or a string\n    at MessageStream.messageHandler

[unicornist]

in case if someone else finds this thread, I solved my problem with the approach below:

    { addFields: {
       dateFieldType: { $type: '$dateField' },
    } },
    { match: {
      dateFieldType: { $ne: 'date' }
    } },