Skip to content

Conversation

DanielKellerM
Copy link

@DanielKellerM DanielKellerM commented Aug 23, 2025

This PR introduces significant new features and architectural support for running applications on standalone (without Fabric Controller) PULP clusters. It adds three new targets: pulp_cluster, carfield-cluster, and astral-cluster, along with runtime support for advanced reliability features. The PR carries a long list of changes by many contributors. It accompanies the PR pulp-platform/pulp_cluster#89 .

Major Changes:

New targets:

  • New build targets: pulp_cluster, carfield-cluster, and astral-cluster.
  • These targets are designed to run directly on a cluster without a Fabric Controller (CONFIG_NO_FC=1).
  • Runtime and boot sequence (crt0.S) now support self-booting clusters.
  • New memory maps, linker scripts, and architecture properties specific to each target.

HMR support:

  • HAL support for the HMR peripheral (hmr_v1).
  • Includes low-level assembly routines for core state save/restore to handle fault recovery and resynchronization.
  • Provides synchronization primitives and interrupt handlers (pos_hmr_synch, pos_hmr_tmr_irq) for managing cores in Dual-Modular (DMR) and Triple-Modular (TMR) Redundancy modes.

New IP and Peripheral Support:

  • iDMA: support for iDMA (idma_v2) as an alternative to MCHAN. Updated HAL supporting 1D, 2D, and 3D transfers.
  • TCDM Scrubber: adds HAL support for the TCDM ECC scrubber.
  • HWPE Interconnect ECC: Adds HAL support for monitoring ECC errors on the HWPE interconnect.

Architectural and HAL Enhancements:

  • The core ID decoding logic in the HAL has been updated to support the new mhartid mapping for carfield-cluster, and astral-cluster targets.
  • Timer and cluster control HALs are now multi-cluster aware, accepting a cluster ID (cid) to target the correct peripherals.
  • A mechanism (hal_cluster_ctrl_return_set_remote) has been added for FC-less clusters to report their exit status to the simulation environment.

Build System and Usability:

  • Makefiles have been added for all new targets. Also updating correct compiler flags and source files.
  • A new centralized tracing header (implem/trace.h) has been added to standardize logging across modules.

luca-valente and others added 30 commits April 11, 2023 19:25
Core 0 does the initialization as if it was the FC and then
all the cores enter the main with the proper stack initialization.
* Pass `APP` variable to the startup TCL script
* Change the executed startup script based on gui or batch simulation
Fix missing argument `Loader` in `bwruntests.py`
1) enable compilation without cleaning by overwriting the (anyways broken!)
   modelsim.ini, etc. files on symlink creation
2) work out-of-the-box on non-ETH machines by not defining the QUESTA
   env var
3) make run dependent on $(TARGETS) being built
@DanielKellerM DanielKellerM marked this pull request as ready for review August 23, 2025 16:59
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces significant support for FC-less (Fabric Controller-less) cluster targets and advanced reliability features. It adds three new standalone cluster targets (pulp_cluster, carfield-cluster, astral-cluster) that can run without a Fabric Controller, along with comprehensive HMR (Hardware Modular Redundancy) support and iDMA integration.

  • Adds three new standalone cluster targets with self-booting capabilities
  • Implements comprehensive HMR support with TMR/DMR functionality and state save/restore
  • Integrates iDMA v2 as an alternative to MCHAN with 1D/2D/3D transfer support

Reviewed Changes

Copilot reviewed 64 out of 64 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
scripts/bwruntests.py Fixes deprecated YAML loader usage for security
rules/pulpos/targets/*.mk New makefiles for cluster targets with architecture-specific configurations
rules/pulpos/default_rules.mk Updates toolchain handling and entry point configuration
lib/libc/minimal/io.c Adds FC-less exit handling
kernel/init.c Conditionally initializes SOC events for FC-less targets
kernel/hmr_synch.c New comprehensive HMR synchronization and state management
kernel/crt0.S Adds FC-less boot sequence and register initialization
kernel/cluster.c Updates cluster initialization for FC-less operation
kernel/chips/*/soc.c New SOC initialization files for cluster targets
kernel/chips/*/link.ld New linker scripts with target-specific memory maps
kernel/bench.c Updates benchmarking for multi-cluster support
kernel/alloc.c Centralizes trace header usage
include/pulp.h Adds HMR function declarations and cluster stack export
include/pmsis.h New compatibility header for PMSIS API
include/implem/trace.h New centralized tracing and logging framework
include/hal//.h Updates for multi-cluster support, HMR, iDMA, and new peripherals
Comments suppressed due to low confidence (2)

include/hal/dma/idma_v2.h:604

  • Inconsistent macro naming: 'IDMA_3D_CONF_SRC_PROTOCOL_OFFSET' should be 'IDMA_REG32_3D_CONF_SRC_PROTOCOL_OFFSET' to match the pattern used elsewhere.
  conf = (((decouple_rw & 0x1)<<IDMA_REG32_3D_CONF_DECOUPLE_RW_BIT) | ((decouple_aw & 0x1)<<IDMA_REG32_3D_CONF_DECOUPLE_AW_BIT) | ((n_d & 0x3)<<IDMA_REG32_3D_CONF_ND_OFFSET) | ((src_prot & 0x7)<<IDMA_3D_CONF_SRC_PROTOCOL_OFFSET) | ((dst_prot & 0x7)<<IDMA_3D_CONF_DST_PROTOCOL_OFFSET));

include/hal/dma/idma_v2.h:604

  • Inconsistent macro naming: 'IDMA_REG32_3D_CONF_ND_OFFSET' should likely be 'IDMA_REG32_3D_CONF_ENABLE_ND_OFFSET' to match the pattern used in the RISC-V version above.
  conf = (((decouple_rw & 0x1)<<IDMA_REG32_3D_CONF_DECOUPLE_RW_BIT) | ((decouple_aw & 0x1)<<IDMA_REG32_3D_CONF_DECOUPLE_AW_BIT) | ((n_d & 0x3)<<IDMA_REG32_3D_CONF_ND_OFFSET) | ((src_prot & 0x7)<<IDMA_3D_CONF_SRC_PROTOCOL_OFFSET) | ((dst_prot & 0x7)<<IDMA_3D_CONF_DST_PROTOCOL_OFFSET));

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@DanielKellerM DanielKellerM requested review from sermazz, ricted98 and yvantor and removed request for Copilot August 23, 2025 17:01
@yvantor
Copy link

yvantor commented Sep 10, 2025

@DanielKellerM Thank you so much for this! Sorry for not getting back but the last month was quite busy. I will have a look (but it will take a bit of time :) )

Copy link

@yvantor yvantor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am approving this, but there are several things to keep in mind that would probably require a deep rework of the pulp-runtime code. A few examples:

  • All the include/archi/chips/%/apb_soc.h and include/archi/chips/%/apb_soc_ctrl/.h are actually not needed in the standalone case
  • The include/chips/%/pulp.h, the include/chips/%/soc.h, and the kernel/chips/%/soc.c seem to be almost identical for all the three standalone architectures, maybe they could be centralized and shared

I will open a cleanup issue and we will address it later :)

Comment on lines +33 to +39
#if PULP_CHIP == CHIP_CARFIELD
li t0, 0x5003FFF0
#elif PULP_CHIP == CHIP_ASTRAL
li t0, 0x5003FFF0
#else
li t0, 0x1003FFF0
#endif
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have never understood in depth the rationale behind these "magic" values. Maybe there is a way to link them automatically?

Comment on lines +88 to +94
#if PULP_CHIP == CHIP_CARFIELD
li t0, 0x5003FFF0
#elif PULP_CHIP == CHIP_ASTRAL
li t0, 0x5003FFF0
#else
li t0, 0x1003FFF0
#endif
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above

Comment on lines +167 to +198
/* Make sure to properly initialize latch based RF */
lui x1, 0
lui x2, 0
lui x3, 0
lui x4, 0
lui x5, 0
lui x6, 0
lui x7, 0
lui x8, 0
lui x9, 0
lui x10, 0
lui x11, 0
lui x12, 0
lui x13, 0
lui x14, 0
lui x15, 0
lui x16, 0
lui x17, 0
lui x18, 0
lui x19, 0
lui x20, 0
lui x21, 0
lui x22, 0
lui x23, 0
lui x24, 0
lui x25, 0
lui x26, 0
lui x27, 0
lui x28, 0
lui x29, 0
lui x30, 0
lui x31, 0
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could make sense to add float regfile initialization if FP extension is present.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants