Containerized global-workflow errors

[gspetro-NOAA]

Hello,

I am running the containerized global-workflow on the NOAA ParallelWorks AWS cloud platform, and I've run into a few errors. The first error (after running ./build_all.sh gfs) related to CMEPS/mediator/med_time_mod.F90, and I solved the build error by checking out and using hash 4c6c6a41 (which still had that file) instead of HEAD of develop. However, I'm not sure if the error was container-related or more general to GW, so I wanted to share the problem here in case anyone has ideas. Here is the error message:

build_ufs.sh failed with status 1!
...
-- Configuring done (30.9s)
CMake Error at CMEPS-interface/CMakeLists.txt:179 (add_library):
  Cannot find source file:

    CMEPS/mediator/med_time_mod.F90

  Tried extensions .c .C .c++ .cc .cpp .cxx .cu .mpp .m .M .mm .ixx .cppm
  .ccm .cxxm .c++m .h .hh .h++ .hm .hpp .hxx .in .txx .f .F .for .f77 .f90
  .f95 .f03 .hip .ispc

CMake Error at CMEPS-interface/CMakeLists.txt:179 (add_library):
  No SOURCES given to target: cmeps

CMake Generate step failed.  Build files cannot be regenerated correctly.

Now, I have successfully built the GW using hash 4c6c6a41. However, when I follow the steps laid out by @mark-a-potts here, I run into a Jinja2 error. After running ./setup_expt.py gfs forecast-only --start cold --pslot c48_atm --app ATM --resdetatmos 48 --idate 2021032312 --edate 2021032312 --comroot /contrib/Gillian.Petro/gw/comroot --icsdir=/contrib/Gillian.Petro/gw/ICSDIR/C48C48mx500/20240610 --expdir /contrib/Gillian.Petro/gw/expdir, I am getting the following error and don't know what to make of it:

Traceback (most recent call last):
  File "./workflow/setup_expt.py", line 12, in <module>
    from hosts import Host
  File "/contrib/Gillian.Petro/gw/global-workflow/workflow/hosts.py", line 7, in <module>
    from wxflow import YAMLFile
  File "/contrib/Gillian.Petro/gw/global-workflow/workflow/wxflow/__init__.py", line 13, in <module>
    from .jinja import Jinja
  File "/contrib/Gillian.Petro/gw/global-workflow/workflow/wxflow/jinja.py", line 16, in <module>
    @jinja2.pass_eval_context
AttributeError: module 'jinja2' has no attribute 'pass_eval_context'

It seems like the Jinja2 in use should be new enough to include pass_eval_context.
Has anyone seen this before?
I realize I'm not working on a supported system, so I don't expect a huge amount of debugging help, but I thought that perhaps this sort of thing has come up on other systems, and someone might have ideas.

[mark-a-potts]

I updated the patch in https://github.com/noaa-epic/global-workflow-patch that will fix this problem.

[gspetro-NOAA]

Thanks! This may solve the problem! :)

[mark-a-potts]

My patch only affects the build problem, unfortunately.

[aerorahul]

@gspetro-NOAA
Could you print out the version of jinja in your env. I suspect it is not the latest (3.0+).
If there is a way to replicate, I could try with some instructions.

[gspetro-NOAA]

@aerorahul On the main system, it's 2.10.1, which is definitely too old, but I tried to create a virtual environment and install Jinja2==3.1.2 (used in the /versions files), but I get this error:

(gw-env) [Gillian.Petro@awsv2intel-53 gw]$ pip install Jinja2==3.1.2
ERROR: Could not find a version that satisfies the requirement Jinja2==3.1.2 (from versions: 2.0rc1, 2.0, 2.1, 2.1.1, 
2.2, 2.2.1, 2.3, 2.3.1, 2.4, 2.4.1, 2.5, 2.5.1, 2.5.2, 2.5.3, 2.5.4, 2.5.5, 2.6, 2.7, 2.7.1, 2.7.2, 2.7.3, 2.8, 2.8.1, 2.9, 2.9.1, 
2.9.2, 2.9.3, 2.9.4, 2.9.5, 2.9.6, 2.10, 2.10.1, 2.10.2, 2.10.3, 2.11.0, 2.11.1, 2.11.2, 2.11.3, 3.0.0a1, 3.0.0rc1, 3.0.0rc2, 
3.0.0, 3.0.1, 3.0.2, 3.0.3)
ERROR: No matching distribution found for Jinja2==3.1.2

Not sure what to make of that, since higher versions of Jinja2 obviously do exist, but it seems like Mark's patch might solve the problem (still have to try it tho!).

[DavidHuber-NOAA]

@gspetro-NOAA I suspect that pip message has to do with the version of Python on the system. Jinja 3.1.x requires Python version 3.7+. What version is the system running?

[gspetro-NOAA]

Ah, there we go! That's the issue. It's got Python 3.6.8, which is so old it never occurred to me it'd be the main python on the system... Well, I suspect fixing that will solve the problem, but I'll be back if I have more questions. Thanks!