Skip to content

Conference call notes 20210915

Jump to bottom
Kenneth Hoste edited this page Sep 15, 2021 · 5 revisions

(back to Conference calls)

Notes on the 181st EasyBuild conference call, Wednesday Aug Sept 15th 2021 (15:00 UTC)

Attendees

Alphabetical list of attendees (XXX):

  • Kenneth Hoste (HPC-UGent, Belgium)

  • Simon Branford (Univ. of Birmingham, UK)

  • Alexander Grund (TU Dresden, Germany

  • Adam Huffman (Big Data Institute, Oxford, UK)

  • Terje Kvernes (University of Oslo, Norway)

  • Sam Moors (Vrije Universiteit Brussel, Belgium)

  • Jörg Saßmannshausen (NIHR Biomedical Research Centre, UK)

Agenda

  • overview of recent developments
  • out-of-the-box performance of TensorFlow installations with EasyBuild (see easyblocks issue #2577)
  • Q&A

Recent developments

  • release timeline
  • recent changes
    • framework
      • bug fixes
        • make sure git clone with a tag argument actually downloads a tag (PR #3795)
        • correctly resolve templates for patches in extensions when uploading to GitHub (PR #3805)
        • ensure that path configuration options have absolute path values (PR #3832)
      • enhancements
        • Use Rich (if available) to show progress bar when installing easyconfigs (PR #3823 + PR #3826)
          • can be disabled via --disable-show-progress-bar or --output-style=basic
        • add --output-style configuration option, which can be used to disable use of Rich or type of any colored output (PR #3833)
      • changes
        • ...
    • easyblocks
      • bug fixes
        • explicitly use only OpenBLAS for PyTorch if MKL is not used (PR #2448)
        • reset runtest to None in CMakePythonPackage (PR #2536)
        • adjust qa list in WIEN2k easyblock to handle new questions + fix sanity check for v21.1 (PR #2572)
      • enhancements
        • make sure AOCC commands like clang/flang pick up GCCcore as GCC toolchain (PR #2538)
        • update ABAQUS easyblock to add support for also installing Tosca component in ABAQUS (PR #2560) + support installing of version 2021 with hot fixes (PR #2562 + PR #2569)
        • update sanity check in VTune easyblock for 2021.x versions (PR #2563 + PR #2576)
        • let GROMACS runtime logs show EasyBuild was used (PR #2571)
      • new easyblocks
        • (none)
      • changes
        • (none)
    • easyconfigs
      • over 100 easyconfig PRs merged since last conf call!
      • bug fixes
        • fix ITK easyconfigs (PR #13619)
        • fix LAMMPS 3Mar2020 easyconfigs using intel toolchain on AMD CPUs by patching out hardcoded -xHost (PR #11577)
        • update to more recent Meson build dependency for GCCcore/8.3.0 easyconfigs to fix failing RPATH sanity check (PR #13910)
        • add patch for OpenBLAS 0.3.15 (for foss/2020b) to fix failing numpy tests on aarch64 (PR #13923
        • add UnZip build dependency to recent TensorFlow easyconfigs (PR #13958, PR #13959, PR #13960)
        • add additional valid checksum for lhs 1.1.1 extension in recent R 4.0.x and 4.1.0 easyconfigs (PR #13969)
        • add missing Brotli dependency to freetype 2.10.x (PR #13915)
      • enhancements
        • (nothing major)
      • new software
      • noteworthy software updates
      • changes
        • update to libjxl 0.5 + disable building of man pages (PR #13829)
        • rename CUDAcore to CUDA for v11.3.1 and v11.4.1 after merging foss/fosscuda, to ensure that get_software_root('CUDA') used in easyblocks works (PR #13874)
        • update CMake build dependency for Eigen 3.3.4 + 3.3.5 using system toolchain (PR #13889)
        • use GTK2 and GTK3 as dependencies rather than GTK+ for easyconfigs using GCC/10.3.0 toolchain (PR #13900)
        • remove superflous -DCMAKE_BUILD_TYPE=Release configuration option in various easyconfigs (PR #13615, #13614, PR #13621)
  • to merge/fix/tackle soon
    • framework
      • reported bugs / bug fixes
        • add optimal optimization flags for Intel compilers on AMD CPUs (issue #3793)
          • -xHost results in only using SSE2 on AMD systems...
          • see also (PR #3797
      • enhancements
        • avoid using a priority in prepend_module_path (Lmod) to avoid costly module calls (PR #3636)
        • add support for installing extensions in parallel (WIP) (PR #3667)
          • needs more test + a dedicated unit test
          • should be marked experimental at first?
        • finding modules with multiple modulepaths and HMNS (issue #3703)
        • filter out duplicate paths added to module files (PR #3770)
      • changes
        • fully drop support code for Python < 2.7 (PR #3788)
        • test suite improvements (PR #3790)
        • change copy_file function to raise an error when trying to copy non-existing file (PR #3836)
    • easyblocks
      • reported bugs / bug fixes
        • Prevent OpenMPI from failing on machines with less than 8 cores (PR #2570)
          • caused by sanity check that was added to OpenMPI easyblock recently (see PR #2444)
      • enhancements
        • add support for installing R extensions in parallel (WIP) (PR #2408)
        • enhance GCC easyblock to add support for AMD GPU offloading (PR #2578)
        • reduce the number of command line args for CMake configure (PR #2514)
        • enhance TensorFlow easyblock to take into account provided OpenSSL dependency (PR #2575)
        • enable MKL-DNN/oneDNN usage for aarch64 CPUs in TensorFlow 2.5+ (PR #2574)
      • changes
        • (nothing major?)
      • new software
        • (nothing major?)
    • easyconfigs

Common toolchains

2021b (WIP!)

  • for now: foss/2021.07 and intel/2021.07 (candidates for 2021b after testing confirms they work well)
    • foss/2021.07: included with EasyBuild v4.4.2 release
    • intel/2021.07: WIP at PR #13639
  • support for GCC 11.2 is expected in upcoming intel-compilers 2021.4 release...

TensorFlow performance

  • see easyblocks issue #2577
  • performance of TensorFlow is significantly slower compared to a pip installation (on quick benchmark runs)
    • boils down to difference in threading mode: OpenMP vs Eigen
    • workaround to remedy the issue: export OMP_NUM_THREADS=1 to avoid massive oversubscription of cores due to too many threads
    • too big a hammer to let TensorFlow modules set $OMP_NUM_THREADS to 1...

Q&A

  • ...
Clone this wiki locally