Tree based inference time compute method based on Scaffolding

[WeiHaocheng]

Support tree based inference time compute method including MCTS, TOT...

[Jinrui-Liu137]

I'm interested.

[WeiHaocheng]

Thanks, assign to jiarui, and I will provide some support.

[therealnaveenkamal]

@WeiHaocheng If this is not yet done, can I get it assigned? The issue sounds interesting tho.

[WeiHaocheng]

@therealnaveenkamal Assign this issue to you~ Do you have any idea for that? Feel free to discuss here.

[therealnaveenkamal]

Let me know how this is:

1. Let's create three tree classes: one for state tracking, values, counts, one that extends GenerationTask and one that extends RewardTask
2. Simple MCTS Controller: Select Leaf (UCB1) -> Expand -> Evaluate -> Update
3. Simple TOT Controller: Generates Multiple Thoughts -> Evaluate -> Prune to keep the best candidates

Let me know your thoughts and any other inputs @WeiHaocheng

[therealnaveenkamal]

@WeiHaocheng should I make a PR for this? let me know your thoughts for the above approach?

[WeiHaocheng]

@WeiHaocheng should I make a PR for this? let me know your thoughts for the above approach?

Yes~ having PR is the best.
BTW, what does "three tree classes" mean? I'm agreed with Simple MCTS Controller and Simple TOT Controller.

[therealnaveenkamal]

@WeiHaocheng sorry for the delay. please find the ref PR and let me know your thoughts. thank you!