Can one limit the number of cores or threads to run in parallel?

[rfryeSigma]

I am running cadquery in a LatticeQuery environment which defines the function eachpointAdaptive similar to
#628 (comment). The script runs quite fast since shapes.py sets SetRunParallel true, and it recruits all processors.

But I am running on a server where it is not polite to hog the whole machine. Is there a way to tell cadquery to limit the maximum number of cores or threads to run in parallel?

[adam-urbanczyk]

I think like so, but please test and report back:

import OCP
pool = OCP.OSD.OSD_ThreadPool.DefaultPool_s()
pool.SetNbDefaultThreadsToLaunch(N_THREADS)

[rfryeSigma]

I just tried it, and it is still using all cores despite telling it to use only 3 out of the 10 on my MAC.
This is my script and some of the output:

(lq) rogerfrye@Rogers-Laptop LatticeQuery % python
Python 3.12.2 | packaged by conda-forge | (main, Feb 16 2024, 20:54:21) [Clang 16.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import cadquery as cq
from lq.topologies.bcc import bcc_heterogeneous_lattice
import OCP
pool = OCP.OSD.OSD_ThreadPool.DefaultPool_s()
pool.SetNbDefaultThreadsToLaunch(3)
unit_cell_size = 10
min_strut_diameter = 0.8
max_strut_diameter = 5.
min_node_diameter = 0.88
max_node_diameter = 5.5
Nx = 4
Ny = 4
Nz = 20
cq.Workplane.bcc_heterogeneous_lattice = bcc_heterogeneous_lattice
result = bcc_heterogeneous_lattice(unit_cell_size,
... min_strut_diameter,
... max_strut_diameter,
... min_node_diameter,
... max_node_diameter,
... Nx, Ny, Nz,
... topology = 'bcc',
... rule = 'parabola'
... )
[0.4 0.89861496 1.34182825 1.72963989 2.06204986 2.33905817
2.56066482 2.72686981 2.83767313 2.89307479 2.89307479 2.83767313
2.72686981 2.56066482 2.33905817 2.06204986 1.72963989 1.34182825
0.89861496 0.4 ]
Datapoints generated
Building an element of an array...
Building an element of an array...
Success!

---

Building an element of an array...
Success!

....

[adam-urbanczyk]

Other ideas:

import OCP
pool = OCP.OSD.OSD_ThreadPool.DefaultPool_s()
pool.Init(N_THREADS)

or try setting the TBB_NUM_THREADS env var, if your OCCT is built with TBB.

[adam-urbanczyk]

Based on the OCCT code, maybe even like so:

import OCP
pool = OCP.OSD.OSD_ThreadPool.DefaultPool_s(N_THREADS)

https://github.com/Open-Cascade-SAS/OCCT/blob/4c8faa5e2dd0d3e4bd7039c908016d8204889c09/src/OSD/OSD_ThreadPool.hxx#L57

[rfryeSigma]

The solution turned out to be to set the number of threads before importing cadquery:
import OCP
pool = OCP.OSD.OSD_ThreadPool.DefaultPool_s(1)
import cadquery as cq

Thank you!

[adam-urbanczyk]

Added to wiki, closing.