Similar to `AllAuthoritiesAuthorizationManager` we should create an `AllFactorsAuthorizationManager` which can can authorize the `FactorGrantedAuthority`