` fpm new foo && cd foo && fpm build --compiler ftn` fails when `ftn` invokes the Cray compiler

[rouson]

Description

On HPE Cray systems, all compilers are invoked via the ftn compiler wrapper. I'm attempting to use fpm on the NERSC Cori system after loading the environment module Pro-Env/cray, which sets the ftn wrapper to invoke the Cray Fortran compiler.

> ftn -V
Cray Fortran : Version 13.0.1  Sun Oct 02, 2022  21:27:30
> fpm new foo && cd foo && fpm build --compiler ftn 
 + mkdir -p foo
 + cd foo
 + mkdir -p foo/src
 + mkdir -p foo/app
 + mkdir -p foo/test
 + git config --get user.name > /tmp/fileHLjPW1
 + git config --get user.email > /tmp/file0ld7Hg
 + git config --get user.name > /tmp/fileVdkBuv
 + git init foo
Initialized empty Git repository in /global/u1/r/rouson/foo/.git/
 + mkdir -p build/dependencies
<WARN> Unknown compiler ftn requested! Defaults for this compiler might be incorrect
foo.f90                                failed.
[ 25%] Compiling...
ftn-78 crayftn: ERROR in command line
  The -m option has an invalid argument, "odule".
<ERROR> Compilation failed for object " src_foo.f90.o "
<ERROR>stopping due to failed compilation
STOP 1

Expected Behaviour

I expect fpm build --compiler ftn to succeed in a project created by fpm new on a HPE Cray system.

Version of fpm

0.6.0

Platform and Architecture

Linux

Additional Information

No response

[awvwgk]

We are detecting Cray here using

fpm/src/fpm_compiler.f90

Lines 751 to 754 in e3fff65

	if (check_compiler(compiler, "crayftn")) then
	id = id_cray
	return
	end if

The compiler is expected to be named crayftn, which is apparently not correct. Checking for ftn here might fix this issue to recognize the Cray compiler.

[awvwgk]

I recall that Cray has compiler wrappers all called ftn also for other vendors like Intel, which might be the option I opted for checking crayftn.

[rouson]

@awvwgk an alternative that I have found to work well is to determine the compiler identity by searching for a substring in the leading characters of the result of the iso_fortran_env function compiler_version. Here's an example. Is this helpful?

[rouson]

Using compiler_version() to detect compiler identity while fpm is executing would presumably require compiling a small program somewhat like

program main
  use iso_fortran_env, only : compiler_version
  implicit none
  print *,compiler_version()
end program

and then and executing the compiled program and parsing the resulting output (possibly after redirecting to a file).

[awvwgk]

We cannot assume that we can run a program since we might be using a cross-compiler.

[awvwgk]

An alternative would be to run the compiler with --version / -V / ...

❯ module load intel/18.0
❯ ifort -V
Intel(R) Fortran Intel(R) 64 Compiler for applications running on Intel(R) 64, Version 18.0.5.274 Build 20180823
Copyright (C) 1985-2018 Intel Corporation.  All rights reserved.
❯ ifort --version
ifort (IFORT) 18.0.5 20180823
Copyright (C) 1985-2018 Intel Corporation.  All rights reserved.
❯ ifort test.f90 && ./a.out
 Intel(R) Fortran Intel(R) 64 Compiler for applications running on Intel(R) 64, 
 Version 18.0.5.274 Build 20180823
❯ module swap intel/2022.2
❯ ifx -V
Intel(R) Fortran Compiler for applications running on Intel(R) 64, Version 2022.1.0 Build 20220316
Copyright (C) 1985-2022 Intel Corporation. All rights reserved.

❯ ifort -V
Intel(R) Fortran Intel(R) 64 Compiler Classic for applications running on Intel(R) 64, Version 2021.6.0 Build 20220226_000000
Copyright (C) 1985-2022 Intel Corporation.  All rights reserved.

❯ ifx --version
ifx (IFORT) 2022.1.0 20220316
Copyright (C) 1985-2022 Intel Corporation. All rights reserved.

❯ ifort --version
ifort (IFORT) 2021.6.0 20220226
Copyright (C) 1985-2022 Intel Corporation.  All rights reserved.

❯ ifx test.f90 && ./a.out
 Intel(R) Fortran Compiler for applications running on IA-32, Version 2022.1.0 B
 uild 20220316
❯ ifort test.f90 && ./a.out
 Intel(R) Fortran Intel(R) 64 Compiler Classic for applications running on Intel
 (R) 64, Version 2021.6.0 Build 20220226_000000

The main disadvantage are the inconsistencies in the output which we have to hard-code in fpm, also compilers might change this output like in the case of ifort here.

[awvwgk]

@rouson could you check whether the ftn compiler wrapper has an option like mpifort which allows to retrieve the actual command?

fpm/src/fpm_compiler.f90

Lines 643 to 649 in e3fff65

	! Check whether we are dealing with an MPI compiler wrapper first
	if (check_compiler(compiler, "mpifort") &
	& .or. check_compiler(compiler, "mpif90") &
	& .or. check_compiler(compiler, "mpif77")) then
	output = get_temp_filename()
	call run(compiler//" -show > "//output//" 2>&1", &
	& echo=.false., exitstat=stat)

The same logic as for the MPI compiler wrapper could be implemented for the Cray compiler wrapper (or any other compiler wrapper like hipfort, ...).

[urbanjost]

I believe if you run with --verbose on the fpm command you will see that since the ftn command is unknown it is guessing as some common command line options; one of which will be -module instead of -J and/or -p,
which causes the message about "odule".

Is ftn really an abstraction layer that calls different compilers (as we have discussed as packaging with fpm at various times? ) If so, all options are the same for the ftn(1) command regardless of which compiler is actually called?

If it only invokes the Cray compiler, we simply need to change crayftn to ftn in the code, as mentioned.

If it does call different compilers and the options are not all the same, I would suggest leaving crayftn as the
fpm invoking name, and just have the code call "ftn" when it sees "crayftn". This would avoid having fpm have
to determine which compiler the wrapper is pointing to; or assume the module(1) command is available; as it
would be likely that is what is setting up the different compilers to be called (?) and query it, which I think would
be much easier than depending on getting that from ftn(1) and/or the compiler version switch. A lot of conjecture here, so any answer on just what ftn can/cannot do would let me know if this is a good approach or not.

I believe the man-pages use the name crayftn, craycc, .... for the commands ftn, cc, ... for consistency and easy lookup because the C/C++ commands have conflicts with a lot of other compilers. If you do a "which crayftn" is comes back saying no such command exists, right?

Assuming there is no command called crayftn, you can make a little script and put it in your path, like

#!/bin/bash
#@(#) crayftn(1) -- call ftn with the name crayftn
ftn $*
# actually, might want ftn $@

and use the compiler crayftn; or build the standalone version and change "crayftn" to "ftn" and try it to make sure that works until a PR can be applied to fpm(1) to fix it. We need to review good defaults for debugging
and for safe optimal performance as well, for the --profile debug|release option.

[rouson]

TL/DR: @awvwgk are there really people using fpm with a cross-compiler or was the concern stated above hypothetical? Unless fpm has real and important such cases to support, I return to thinking that identifying the compiler based solely on the name used to invoke it is horribly fragile and it would be far better to parse the output of invoking the compiler with --version or by compiling and running a short program that prints the output of the Fortran compiler_version function from the iso_fortran_env module.

For more context, I'm working on a pull request to implement the changes described in issue #1089 and finding the compiler detection code is even more fragile than I had previously imagined. Specifically, the following line in fpm_compiler.F90 reports a match if the searched-for name is anywhere in the name used to invoke the compiler:

match = index(basename(compiler), expected) > 0

I'll try to think of a more robust short-term fix for the above line, but long-term I think the problems with this approach will grow over time. Consider for example, the following statement in the AMD Next Gen Fortran Compiler - User Guide:

"NOTE: amdflang in the preproduction release of the AMD Next Gen Fortran Compiler is distinct from amdflang in ROCm, which corresponds to amdflang-legacy in the preproduction release."

so even the name amdflang is not specific enough for compiler disambiguation purposes and that's just one of a growing list of compilers with flang somewhere in the name.

[awvwgk]

Yes, I am using fpm for cross compilation. One example for usage like this can be found in the condo forge package for fpm