Diversity & Coordination in LLM Reasoning Traces

How do diversity and coordination operate within chain-of-thought reasoning? A computational study of implicit multi-perspective problem solving.

cs.CL LLM Reasoning 4 Methods Compared 80 Rounds

0.216

Best Quality (SCG)

0.794

Strategy Coverage (SCG)

0.555

Max Quality (RS)

2.05x

Coverage Improvement

Summary Comparison

Method	Mean Q	Best Q	Max Q	Cos. Div.	Strat. Cov.
Independent	0.058	0.206	0.482	0.895	0.387
Repulsive (RS)	0.059	0.213	0.555	0.841	0.362
Strategy-Cond. (SCG)	0.062	0.216	0.490	0.918	0.794
Ensemble (EC)	0.058	0.209	0.452	0.807	0.375

Strategy-Conditioned Generation achieves 2.05x higher strategy coverage than Independent sampling, demonstrating that structured diversity organized around distinct solution strategies outperforms geometric spread alone.

Key Findings

Structured > Geometric Diversity: Strategy coverage (functional diversity) matters more than cosine diversity (geometric spread) for solution quality.
Repulsive Sampling finds rare optima: RS achieves the highest peak quality (0.555) by pushing traces into hard-to-reach solution regions.
SCG dominates on average: Best mean quality (0.062) and best quality (0.216) come from explicit strategy assignment.
Coordination benefits scale with K: More traces amplify the advantage of coordination mechanisms.

Best Quality Over Rounds

Mean Quality Over Rounds

Max Quality Achieved

Strategy Coverage Over Rounds

Cosine Diversity Over Rounds

Diversity Metrics Summary

Diversity--Accuracy Tradeoff

Strategy coverage vs. best quality across methods. Higher right = better coordination.

Cosine Diversity vs. Strategy Coverage

High cosine diversity does not guarantee high strategy coverage. SCG achieves both, while Independent has high cosine diversity but low coverage -- indicating diverse endpoints that cluster around the same easy strategies.

Problem Setup

Problem Dimension	20
Number of Problems	50
Traces per Problem (K)	8
Trace Length	12 steps
Solution Optima	6 per problem
Coordination Rounds	80
Random Seed	42

Method Descriptions

Independent Sampling (Baseline)

Each trace follows a biased random walk toward the nearest optimum. No coordination between traces.

Repulsive Sampling (RS)

An RBF diversity kernel repels each trace from previously generated traces, encouraging exploration of distinct regions. Repulsion strength: 0.5, bandwidth: 1.5.

Strategy-Conditioned Generation (SCG)

Each trace is assigned a strategy label (round-robin) and biased toward the corresponding optimum, ensuring systematic exploration of all solution strategies.

Ensemble Coordination (EC)

A portfolio of 4 specialized sub-policies with learned directional biases. Sub-policies are updated based on strategy visit counts to maintain complementary specializations.

Open Question

This work addresses the open question posed by Kim et al. (2026): "How diversity and coordination operate within the reasoning traces of LLMs remains an open question." Our computational framework provides quantitative evidence that structured diversity -- diversity organized around distinct solution strategies -- is more effective than geometric diversity for improving collective problem solving in multi-trace reasoning.