TSV Dataset with commas doesn't work

[beccamc]

AirBnb Dataset from this website and from this data source.

We can't process this data set. I think it may be because of commas in the data set (e.g. "East-West Highway, Silver Spring, MD 20910, United States").

Ask Becca for dataset

[LittleLittleCloud]

This bug also exists on AutoML, raised an issue here

[LittleLittleCloud]

It's a bug in TextLoader, (or a feature I would say)
A solution to bypass this restriction might be

• General (C# CSV/TSV) reader which returns an IEnumerable (maybe CsvHelper) ( We can pass delimiter from ModelBuilder to AutoMLService)
• ML.NET creates an IDataView from the IEnumerable https://github.com/dotnet/machinelearning/blob/d6a4b1dd79e13c9a749ad9fdeb1c6ea2f4f547a6/docs/code/SchemaComprehension.md#streaming-data-views

after we get IDataView, we can go to the next step (to train the models)

[LittleLittleCloud]

Raise a fixing PR here in machinelearning

We should still try the bypass method though

UPDATE

ML.Net team has started a fix on TextLoader to make it work with newline in double quote