Refactor and harmonize CSV and SQL data sources #773

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

jo-bao merged 70 commits into dev from jb/#716-refactor-csv-datasources

Jun 14, 2023

Contributor

jo-bao commented Mar 8, 2023 •

edited

Loading

Resolves #716

refactor CSV and SQL data sources to use the methods for creating entities
implement IdCoordinateSource for SQL

jo-bao and others added 24 commits

January 20, 2023 12:30


          Temporary

906379c


          Temporary

f6e1635


          Temporary

613e597


          Temporary

4ad2c1c


          Temporary

1a25e97


          Temporary

5f9b8ec


          Temporary

df0fc26


          Temporary

4b8738c


          Temporary

70f6897


          Temporary

09976ad


          Temporary

0dd2abe


          Temporary

ea499f1


          Temporary

55e09f7


          Temporary

c43f7c9


          Temporary

f1a1995


          Temporary


          refactor CSV datasources

f480fc6


          refactor CSV datasources

8e5c8f1


          refactor CSV datasources, implement SqlIdCoordinateSource

33932d4


          refactor CSV datasources, implement SqlIdCoordinateSource

8f6c0e0


          Merge remote-tracking branch 'origin/dev' into jb/#716-refactor-csv-d…

c6e9e8f

…atasources


          add CsvWindowedTimeSeriesSource

6de25b3


          fix SqlTimeSeriesSource test

e66da2b


          Merge branch 'dev' into jb/#716-refactor-csv-datasources

f5822e4

sonatype-lift bot reviewed

View reviewed changes

src/main/java/edu/ie3/datamodel/io/source/csv/CsvTimeSeriesMetaInformationSource.java Outdated Show resolved Hide resolved

sonatype-lift bot reviewed

View reviewed changes

src/main/java/edu/ie3/datamodel/io/source/csv/CsvWindowedTimeSeriesSource.java Outdated Show resolved Hide resolved

sonatype-lift bot reviewed

View reviewed changes

src/main/java/edu/ie3/datamodel/io/source/GraphicSource.java Outdated Show resolved Hide resolved

sonatype-lift bot reviewed

View reviewed changes

src/main/java/edu/ie3/datamodel/io/source/sql/SqlTimeSeriesSource.java Show resolved Hide resolved

sonatype-lift bot reviewed

View reviewed changes

src/main/java/edu/ie3/datamodel/io/source/sql/SqlDataSource.java Outdated Show resolved Hide resolved

sonatype-lift bot reviewed

View reviewed changes

src/main/java/edu/ie3/datamodel/io/source/SystemParticipantSource.java Outdated Show resolved Hide resolved

jo-bao and others added 2 commits

June 1, 2023 14:52


          Merge remote-tracking branch 'origin/jb/#716-refactor-csv-datasources…

db5041c

…' into jb/#716-refactor-csv-datasources


          Merge branch 'dev' into jb/#716-refactor-csv-datasources

4723fdb

sebastian-peter mentioned this pull request

Use nio paths instead of strings #742

Merged

jo-bao and others added 3 commits

June 2, 2023 14:22


          Changes after review

77047ce


          Merge remote-tracking branch 'origin/jb/#716-refactor-csv-datasources…

f94228a

…' into jb/#716-refactor-csv-datasources


          Merge branch 'dev' into jb/#716-refactor-csv-datasources

2560c16

sonatype-lift bot reviewed

View reviewed changes

src/main/java/edu/ie3/datamodel/io/source/sql/SqlTimeSeriesSource.java Outdated Show resolved Hide resolved

jo-bao added 3 commits

June 2, 2023 14:34


          sonatype

c2ebe21


          Merge remote-tracking branch 'origin/jb/#716-refactor-csv-datasources…

fe953b6

…' into jb/#716-refactor-csv-datasources


          spotless

c6b4908

sebastian-peter mentioned this pull request

Use Java streams on SQL fetches effectively #827

Closed

sebastian-peter added the io label

This was referenced Jun 6, 2023

Throw exception instead of questionable EntitySource.safeMapGet #828

Closed

Compare UUIDs instead of Strings in EntitySource.findFirstEntityByUuid #829

Closed

Find a common way to build typed grid elements and system participants? #830

Closed

sebastian-peter requested changes

View reviewed changes

Member

sebastian-peter left a comment

Almost there, I think, besides some some pedantic points from my side.
Please also take a look at SonarQube, some new issues popped up :)

docs/readthedocs/_static/figures/uml/DataSourceClassDiagram.png Outdated Show resolved Hide resolved

docs/readthedocs/io/csvfiles.md Outdated Show resolved Hide resolved

src/main/java/edu/ie3/datamodel/io/source/csv/CsvJointGridContainerSource.java Outdated Show resolved Hide resolved

src/main/java/edu/ie3/datamodel/io/source/TimeSeriesMappingSource.java Show resolved Hide resolved

src/main/java/edu/ie3/datamodel/io/source/WeatherSource.java Outdated Show resolved Hide resolved

src/main/java/edu/ie3/datamodel/io/source/csv/CsvWeatherSource.java Show resolved Hide resolved

src/main/java/edu/ie3/datamodel/io/source/csv/CsvDataSource.java Outdated Show resolved Hide resolved

src/main/java/edu/ie3/datamodel/io/source/csv/CsvIdCoordinateSource.java Outdated Show resolved Hide resolved

src/main/java/edu/ie3/datamodel/io/source/csv/CsvTimeSeriesSource.java Outdated Show resolved Hide resolved

jo-bao and others added 9 commits

June 12, 2023 14:47


          After review

ca85b8f


          Merge branch 'dev' into jb/#716-refactor-csv-datasources

30a1837

Fix

21728c2


          Merge remote-tracking branch 'origin/jb/#716-refactor-csv-datasources…

a3c286f

…' into jb/#716-refactor-csv-datasources

Fix

435c4c6

Fix

8f4a360


          Sonarqube

91cb1d0


          Merge branch 'dev' into jb/#716-refactor-csv-datasources

ff2a94f


          Translating I/O to markdown

ff9c9cb

sebastian-peter approved these changes

View reviewed changes

Member

sebastian-peter left a comment

Thanks for putting all this work in here! I think you've found some really nice abstractions that should make the codebase a lot more usable.


          Merge branch 'dev' into jb/#716-refactor-csv-datasources

3b9c3bb

jo-bao merged commit 6aa7f1b into dev

sebastian-peter deleted the jb/#716-refactor-csv-datasources branch

June 14, 2023 08:16

sebastian-peter added this to the Version 4.0 milestone

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

io