Merge/retro/refactor actions grpc (#339)

cbellot000 · PProfizi · TheGoldfish01 · web-flow · commit 76d37e218f1f · 2022-07-26T14:48:12.000+02:00
* Change ANSYS_VERSION to 222 * Add step for pygate 0.1.dev1 installation * Add step for pygate 0.1.dev1 installation * Add trigger for ci * Add trigger for ci * Fix conditional for gatebin install steps * Fix gatebin install steps * Add the wheels temporarily * Fix step conditional * Change requirement to ansys-dpf-gate==0.1.dev1 * Add pygate steps to documentation job * Run Doc on windows-latest * Move build installation of pygate to pydpf-actions * Fix conditionals * Update gate wheel * Add up-to-date ansys-grpc-dpf wheel * Remove ANSYS_VERSION env setter from build_doc job * Add install of local ansys-grpc-dpf wheel * Solve docstring test for download_file * Solve docstring test for download_file * Solve docstring test for download_file * Solve docstring test for download_file * Update pygate wheel * Fix doctests * Fix doctests * Fix doctests * Fix doctests * Fix doctests * pytest.ini: add doctest NORMALIZE_WHITESPACE and ELLIPSIS by default * remove inline #doctest: options * Fix doctest * Fix doctest * Fix doctest * Fix doctest * Fix doctest * Fix doctest * use v2.2 pydpf-actions * Target potential failing test * Change local_test_repo to False * Comment-out segfault test * Remove contrain on ansys-dpf-gate version in setup.py install_requires * Try-out DEBUG option for pydpf-actions/test_package * Revert "Comment-out segfault test" This reverts commit 5f03a49. * Revert "Target potential failing test" This reverts commit fac439a. * Fix test_busy_port from lea's PR on TFS * Fix conftest according to TFS PR form Lea * Fix gatebin tests from TFS PR Lea * Update wheels * Remove Report from test in install package * Fix flake8 * Try-out on Python 3.7 same as TFS * Bump imageio from 2.19.3 to 2.19.5 (#312) Bumps [imageio](https://github.com/imageio/imageio) from 2.19.3 to 2.19.5. - [Release notes](https://github.com/imageio/imageio/releases) - [Changelog](https://github.com/imageio/imageio/blob/master/CHANGELOG.md) - [Commits](imageio/imageio@v2.19.3...v2.19.5) --- updated-dependencies: - dependency-name: imageio dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> (cherry picked from commit 9e6584a) * MAINT: collect dependencies under a requirements/ directory (#307) * Collect dependencies under requirements/ dir * Enhance .gitigore * Update actions to new requirements layout * Update documentation * Update action versions (cherry picked from commit 79e5bfc) * Bump sphinx-notfound-page from 0.8 to 0.8.3 (#308) Bumps [sphinx-notfound-page](https://github.com/readthedocs/sphinx-notfound-page) from 0.8 to 0.8.3. - [Release notes](https://github.com/readthedocs/sphinx-notfound-page/releases) - [Changelog](https://github.com/readthedocs/sphinx-notfound-page/blob/main/CHANGELOG.rst) - [Commits](readthedocs/sphinx-notfound-page@0.8...0.8.3) --- updated-dependencies: - dependency-name: sphinx-notfound-page dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> (cherry picked from commit 693b83d) * Bump coverage from 6.4.1 to 6.4.2 (#306) Bumps [coverage](https://github.com/nedbat/coveragepy) from 6.4.1 to 6.4.2. - [Release notes](https://github.com/nedbat/coveragepy/releases) - [Changelog](https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst) - [Commits](nedbat/coveragepy@6.4.1...6.4.2) --- updated-dependencies: - dependency-name: coverage dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> (cherry picked from commit c3d434f) * Bump actions/setup-python from 4.0.0 to 4.1.0 (#302) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4.0.0 to 4.1.0. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v4...v4.1.0) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> (cherry picked from commit 251b1fd) * Add a retrocompatibility workflow (#310) * Add a retro-compatibility workflow * Trigger on pull_requests to retro/ branches Trigger also for release/ branches * Update pydpf-actions to 2.1.1 * Update to trigger on pull_requests FROM a retro or release branch * Fix vtk version in setup.py extras_require[plotting] * Add concurrency criterion * Revert Fix vtk version in setup.py extras_require[plotting]: vtk 9.0.3 not available for Python 3.10 * Fix unbuntu doctest error due to unsorted available_property_fields list * Rename job * Update run criteria * Update run criteria * Go back to Windows 3.8 only * Add empty line at end-of-file Co-authored-by: Paul P <74507390+TheGoldfish01@users.noreply.github.com> (cherry picked from commit 3ca756d) * Use pydpf-actions @v2.2 for retro.yml * Add kill-all-servers steps * Add search for AWP_ROOT221 and others in decreasing order as a last resort. * Reduces restrains in requirements_docs.txt * Try fix doc generation for 06-stress_gradient_path.py * Try fix doc generation for 06-stress_gradient_path.py * Revert "Try fix doc generation for 06-stress_gradient_path.py" This reverts commit 7d7f959. * Revert "Try fix doc generation for 06-stress_gradient_path.py" This reverts commit 6d3f0f7. * Fix doc generation * Use wheel and wheelhouse options of build_package action v2.2 * Set wheelhouse generation to false while gate not available on PyPi * REtro now does not test docstrings * Update wheels of gate, grpc, * Move find_ansys() to last resort. If only AWP_ROOT221 is declared, it should not prioritize a 222 install in the default directory. * Restrict TestServerConfigs tests to servers >= 4.0 * Moved ansys_path retrieval logic to misc along with find_ansys, in get_ansys_path. Is now used to correctly determine if MAPDL is available with the installation actually used for testing. * remove rogue prints * Fixture for server<4.0 now only use global server * Rename test_remote_workflow.py * Fix import * Conftest autouse fixture count_servers * Set ansys_path argument of get_ansys_path to None by default * Revert "Conftest autouse fixture count_servers" This reverts commit c4b1f07. * Conftest actually count_servers with fixture * Fix flake8 * Fix flake8 * Remove code merged by error. * Remove test_set_coordinates_field_meshed_region which should come with new commits from TFS * Remove assert nb_servers=1 from count_servers fixture * Move and rename test_launch_server_not_install to test_launcher.py as test_launch_server_full_path * Fix GrpcServer.shutdown() * Add a version test before trying to launch an InProcess server * Remove version test, add warning to get info on ansys_path used. * ADD WARNINGS FOR DEBUG * ADD TIMESTAMP * Use build_package@v2.2.1 * Revert "Use build_package@v2.2.1" This reverts commit e8e5835. * Revert "ADD TIMESTAMP" This reverts commit 4ce75de. * Revert "ADD WARNINGS FOR DEBUG" This reverts commit c02c4a0. * Remove warning used for debug * Fix test_start_local_failed_executable * Fix test_busy_port * Try actions@v2.2.1 * Up doc install-dpf-server to v2.2 * Revert "Try actions@v2.2.1" This reverts commit 4d28452. * Set autouse of count_servers fixture to false * Fix codacy errors * Add back test_launch_server_not_install and rename it as test_launcher_server_full_path * Refactor get_ansys_path and find_ansys * Add back a print in test_print_data_sources_path * find_ansys does not filter out the current ansys version for AWP_ROOT * Remove unused fixtures * Set DOCTEST in retro back to true * Revert "Set DOCTEST in retro back to true" This reverts commit 53ed94a. * Test conditionals * Set wheelhouse to default (false) * Set doctest to default (true) * Revert "Set doctest to default (true)" This reverts commit 3c9460d. * Set doctest to false * Revert "Remove unused fixtures" This reverts commit 78894f6. * Remove fixture config_server_type * Refactor test job using new smaller actions * fix ci * dummy * revert dummy * dummy * pre command * doc and backward comp * backward comp fixes * update doc and ansys.grpc.dpf * fix doc * fix merge Co-authored-by: paul.profizi <paul.profizi@ansys.com> Co-authored-by: Paul P <74507390+TheGoldfish01@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jorge Martínez <28702884+jorgepiloto@users.noreply.github.com> Co-authored-by: PProfizi <100710998+pprofizi@users.noreply.github.com>
diff --git a/.github/workflows/ansys_dpf_gate-0.1.dev1-py3-none-any.whl b/.github/workflows/ansys_dpf_gate-0.1.dev1-py3-none-any.whl
diff --git a/.github/workflows/ansys_grpc_dpf-0.5.1-py3-none-any.whl b/.github/workflows/ansys_grpc_dpf-0.5.1-py3-none-any.whl
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -35,21 +35,85 @@ jobs:
       - uses: actions/checkout@v3
 
       - name: "Build Package"
-        uses: pyansys/pydpf-actions/build_package@v2.2
+        uses: pyansys/pydpf-actions/build_package@v2.2.dev1
         with:
           python-version: ${{ matrix.python-version }}
           ANSYS_VERSION: ${{env.ANSYS_VERSION}}
           PACKAGE_NAME: ${{env.PACKAGE_NAME}}
           MODULE: ${{env.MODULE}}
           dpf-standalone-TOKEN: ${{secrets.DPF_PIPELINE}}
           install_extras: plotting
-          wheelhouse: "false"
+          wheelhouse: false
+          extra-pip-args: "--pre --find-links .github/workflows/"
 
-      - name: "Test Package"
-        uses: pyansys/pydpf-actions/test_package@v2.2.dev0
+      - name: "Prepare Testing Environment"
+        uses: pyansys/pydpf-actions/prepare_tests@v2.2
+        with:
+          DEBUG: true
+
+      - name: "Test Docstrings"
+        uses: pyansys/pydpf-actions/test_docstrings@v2.2
         with:
           MODULE: ${{env.MODULE}}
-          DEBUG: True
+          PACKAGE_NAME: ${{env.PACKAGE_NAME}}
+
+      - name: "Separate long Core tests"
+        shell: pwsh
+        run: |
+          New-Item -Path ".\" -Name "local_server_test" -ItemType "directory"
+          New-Item -Path ".\" -Name "test_workflow" -ItemType "directory"
+          Copy-Item -Path "tests\conftest.py" -Destination ".\local_server_test\"
+          Copy-Item -Path "tests\conftest.py" -Destination ".\test_workflow\"
+          Copy-Item -Path "tests\test_launcher.py" -Destination ".\local_server_test\"
+          Copy-Item -Path "tests\test_server.py" -Destination ".\local_server_test\"
+          Copy-Item -Path "tests\test_local_server.py" -Destination ".\local_server_test\"
+          Copy-Item -Path "tests\test_multi_server.py" -Destination ".\local_server_test\"
+          Copy-Item -Path "tests\test_workflow.py" -Destination ".\test_workflow\"
+          Copy-Item -Path "tests\test_remote_workflow.py" -Destination ".\local_server_test\"
+          Copy-Item -Path "tests\test_remote_operator.py" -Destination ".\local_server_test\"
+          Remove-Item -Path "tests\test_server.py"
+          Remove-Item -Path "tests\test_launcher.py"
+          Remove-Item -Path "tests\test_local_server.py"
+          Remove-Item -Path "tests\test_multi_server.py"
+          Remove-Item -Path "tests\test_workflow.py"
+          Remove-Item -Path "tests\test_remote_workflow.py"
+          Remove-Item -Path "tests\test_remote_operator.py"
+
+      - name: "Test API"
+        shell: bash
+        working-directory: tests
+        run: |
+          pytest $DEBUG --cov=ansys.dpf.${{env.MODULE}} --cov-report=xml --cov-report=html --log-level=ERROR --junitxml=junit/test-results.xml --reruns 2 .
+
+      - name: "Kill all servers"
+        uses: pyansys/pydpf-actions/kill-dpf-servers@v2.2
+
+      - name: "Test API 2"
+        shell: bash
+        working-directory: local_server_test
+        run: |
+          pytest $DEBUG --cov=ansys.dpf.core --cov-report=xml --cov-report=html --cov-append --log-level=ERROR --junitxml=../tests/junit/test-results2.xml --reruns 2 .
+
+      - name: "Kill all servers"
+        uses: pyansys/pydpf-actions/kill-dpf-servers@v2.2
+
+      - name: "Test API 3"
+        shell: bash
+        working-directory: test_workflow
+        run: |
+          pytest $DEBUG --cov=ansys.dpf.core --cov-report=xml --cov-report=html --cov-append --log-level=ERROR --junitxml=../tests/junit/test-results3.xml --reruns 3 .
+
+      - name: "Kill all servers"
+        uses: pyansys/pydpf-actions/kill-dpf-servers@v2.2
+
+      - name: "Upload Test Results"
+        uses: actions/upload-artifact@v2
+        with:
+          name: ${{ env.PACKAGE_NAME }}_pytest
+          path: tests/junit/test-results.xml
+
+      - name: "Upload coverage to Codecov"
+        uses: codecov/codecov-action@v2
 
       - name: 'Upload to PyPi'
         if: contains(github.ref, 'refs/tags')
@@ -75,38 +139,18 @@ jobs:
         with:
           python-version: 3.8
 
-      - id: install-dpf
-        uses: pyansys/pydpf-actions/install-dpf-server@v2.2
+      - name: "Build Package"
+        id: build-package
+        uses: pyansys/pydpf-actions/build_package@v2.2.dev1
         with:
+          python-version: ${{ matrix.python-version }}
+          ANSYS_VERSION: ${{env.ANSYS_VERSION}}
+          PACKAGE_NAME: ${{env.PACKAGE_NAME}}
+          MODULE: ${{env.MODULE}}
           dpf-standalone-TOKEN: ${{secrets.DPF_PIPELINE}}
-          ANSYS_VERSION : ${{env.ANSYS_VERSION}}
-
-      - name: Set AWP_ROOT$env:ANSYS_VERSION
-        run: echo "AWP_ROOT$env:ANSYS_VERSION=${{ steps.install-dpf.outputs.SERVER }}"  | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf-8 -Append
-
-      - name: Set SERVER
-        run: echo "SERVER=$env:AWP_ROOT221"  | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf-8 -Append
-
-      - name: "Install gatebin Windows"
-        working-directory: .github\workflows
-        run: pip install ansys_dpf_gatebin-0.1.dev1-py3-none-win_amd64.whl
-
-      - name: "Install pygate"
-        working-directory: .github\workflows
-        run: |
-          pip install ansys_dpf_gate-0.1.dev1-py3-none-any.whl
-          pip install ansys_grpc_dpf-0.5.dev1-py3-none-any.whl
-
-      - name: Install ansys-dpf-core
-        shell: cmd
-        run: |
-          pip install -r requirements/requirements_build.txt
-          python setup.py bdist_wheel
-          FOR /F %%a in ('dir /s/b dist\*.whl') do SET WHEELPATH=%%a
-          ECHO %WHEELPATH%
-          cd tests
-          pip install %WHEELPATH%
-          python -c "from ansys.dpf import core"
+          install_extras: plotting
+          wheelhouse: false
+          extra-pip-args: "--pre --find-links .github/workflows/"
 
       - name: Install OpenGL
         run: |
diff --git a/.github/workflows/retro.yml b/.github/workflows/retro.yml
@@ -28,19 +28,70 @@ jobs:
       - uses: actions/checkout@v3
 
       - name: "Build Package"
-        uses: pyansys/pydpf-actions/build_package@v2.2
+        uses: pyansys/pydpf-actions/build_package@v2.2.dev1
         with:
           python-version: ${{ matrix.python-version }}
           ANSYS_VERSION: ${{env.ANSYS_VERSION}}
           PACKAGE_NAME: ${{env.PACKAGE_NAME}}
           MODULE: ${{env.MODULE}}
           dpf-standalone-TOKEN: ${{secrets.DPF_PIPELINE}}
           install_extras: plotting
-          wheel: "false"
-          wheelhouse: "false"
+          wheel: false
+          extra-pip-args: "--pre --find-links .github/workflows/"
+          
+      - name: "Install ansys-grpc-dpf==0.4.0"
+        shell: pwsh
+        run: |
+          pip install ansys-grpc-dpf==0.4.0
 
-      - name: "Test Package"
-        uses: pyansys/pydpf-actions/test_package@v2.2.dev0
+      - name: "Prepare Testing Environment"
+        uses: pyansys/pydpf-actions/prepare_tests@v2.2
         with:
-          MODULE: ${{env.MODULE}}
-          DOCTEST: "false"
+          DEBUG: true
+
+      - name: "Separate long Core tests"
+        shell: pwsh
+        run: |
+          New-Item -Path ".\" -Name "local_server_test" -ItemType "directory"
+          Copy-Item -Path "tests\conftest.py" -Destination ".\local_server_test\"
+          Copy-Item -Path "tests\test_launcher.py" -Destination ".\local_server_test\"
+          Copy-Item -Path "tests\test_server.py" -Destination ".\local_server_test\"
+          Copy-Item -Path "tests\test_local_server.py" -Destination ".\local_server_test\"
+          Copy-Item -Path "tests\test_multi_server.py" -Destination ".\local_server_test\"
+          Copy-Item -Path "tests\test_workflow.py" -Destination ".\local_server_test\"
+          Copy-Item -Path "tests\test_remote_workflow.py" -Destination ".\local_server_test\"
+          Copy-Item -Path "tests\test_remote_operator.py" -Destination ".\local_server_test\"
+          Remove-Item -Path "tests\test_server.py"
+          Remove-Item -Path "tests\test_launcher.py"
+          Remove-Item -Path "tests\test_local_server.py"
+          Remove-Item -Path "tests\test_multi_server.py"
+          Remove-Item -Path "tests\test_workflow.py"
+          Remove-Item -Path "tests\test_remote_workflow.py"
+          Remove-Item -Path "tests\test_remote_operator.py"
+
+      - name: "Test API"
+        shell: bash
+        working-directory: tests
+        run: |
+          pytest $DEBUG --cov=ansys.dpf.${{env.MODULE}} --cov-report=xml --cov-report=html --log-level=ERROR --junitxml=junit/test-results.xml --reruns 2 .
+
+      - name: "Kill all servers"
+        uses: pyansys/pydpf-actions/kill-dpf-servers@v2.2
+
+      - name: "Test API 2"
+        shell: bash
+        working-directory: local_server_test
+        run: |
+          pytest $DEBUG --cov=ansys.dpf.core --cov-report=xml --cov-report=html --cov-append --log-level=ERROR --junitxml=../tests/junit/test-results2.xml --reruns 2 .
+
+      - name: "Kill all servers"
+        uses: pyansys/pydpf-actions/kill-dpf-servers@v2.2
+
+      - name: "Upload Test Results"
+        uses: actions/upload-artifact@v2
+        with:
+          name: ${{ env.PACKAGE_NAME }}_pytest
+          path: tests/junit/test-results.xml
+
+      - name: "Upload coverage to Codecov"
+        uses: codecov/codecov-action@v2
diff --git a/ansys/dpf/core/_version.py b/ansys/dpf/core/_version.py
@@ -7,8 +7,8 @@
 min_server_version = "2.0"
 
 server_to_ansys_grpc_dpf_version = {
-    "1.0": "0.2.2",
-    "2.0": "0.3.0",
+    "1.0": "==0.2.2",
+    "2.0": "==0.3.0",
     "3.0": ">=0.4.0",
     "4.0": ">=0.5.0",
 }
diff --git a/ansys/dpf/core/data_tree.py b/ansys/dpf/core/data_tree.py
@@ -83,13 +83,15 @@ def __init__(self, data=None, data_tree=None, server=None):
         # step 2: get api
         self._api_instance = None  # see property self._api
 
-        # step3: init environment
-        self._api.init_dpf_data_tree_environment(self)  # creates stub when gRPC
-
         # step4: if object exists, take the instance, else create it
         if data_tree is not None:
+            if not isinstance(data_tree, dict):
+                # step3: init environment
+                self._api.init_dpf_data_tree_environment(self)  # creates stub when gRPC
             self._internal_obj = data_tree
         else:
+            # step3: init environment
+            self._api.init_dpf_data_tree_environment(self)  # creates stub when gRPC
             if self._server.has_client():
                 self._internal_obj = self._api.dpf_data_tree_new_on_client(self._server.client)
             else:
diff --git a/ansys/dpf/core/misc.py b/ansys/dpf/core/misc.py
@@ -122,8 +122,6 @@ def find_ansys():
     for version in sorted(versions, reverse=True):
         if not version.isnumeric():
             continue
-        if version == __ansys_version__:
-            continue
         elif version < __ansys_version__:
             ansys_path = os.environ.get("AWP_ROOT" + version)
             if ansys_path:
diff --git a/ansys/dpf/core/server_types.py b/ansys/dpf/core/server_types.py
@@ -234,7 +234,7 @@ def _compare_ansys_grpc_dpf_version(right_grpc_module_version_str: str, grpc_mod
         from packaging.version import parse as parse_version
         right_version_first_numbers = re.search(r"\d", right_grpc_module_version_str)
         right_version_numbers = right_grpc_module_version_str[
-                                right_version_first_numbers.start():-1]
+                                right_version_first_numbers.start():]
         compare = "==" if right_version_first_numbers.start() == 0 else \
             right_grpc_module_version_str[0:right_version_first_numbers.start()].strip()
         if compare == "==":
@@ -282,7 +282,7 @@ def check_ansys_grpc_dpf_version(server, timeout):
                             f"{compatibility_link} while still using DPF server {server_version}, "
                             f"please install version {right_grpc_module_version} of ansys-grpc-dpf"
                             f" with the command: \n"
-                            f"     pip install ansys-grpc-dpf=={right_grpc_module_version}"
+                            f"     pip install ansys-grpc-dpf{right_grpc_module_version}"
                             )
 
 
@@ -754,13 +754,7 @@ def __init__(
     def _create_shutdown_funcs(self):
         self._core_api = data_processing_grpcapi.DataProcessingGRPCAPI
         self._core_api.init_data_processing_environment(self)
-        from ansys.grpc.dpf import base_pb2
-        self._preparing_shutdown_func = (
-            data_processing_grpcapi._get_stub(self).PrepareShutdown, base_pb2.Empty()
-        )
-        self._shutdown_func = (
-            data_processing_grpcapi._get_stub(self).ReleaseServer, base_pb2.Empty()
-        )
+        self._core_api.bind_delete_server_func(self)
 
     @property
     def client(self):
diff --git a/docs/source/_static/dpf.html b/docs/source/_static/dpf.html
diff --git a/tests/conftest.py b/tests/conftest.py
@@ -275,7 +275,6 @@ def remote_config_server_type(request):
     )
     def server_type_legacy_grpc(request):
         return core.start_local_server(config=request.param, as_global=False)
-
 else:
 
     @pytest.fixture(scope="session")
@@ -339,8 +338,6 @@ def server_clayer(request):
     ):
         core.settings.get_runtime_client_config(server).cache_enabled = False
     return server
-
-
 class LocalServers:
     def __init__(self):
         self._local_servers = []
