Hill-climb
Improve one piece of work round after round, keeping what works.
A Hill-climb batch improves a single piece of work over several rounds. Each round tries several variations, keeps the best, and uses them as the starting point for the next round.
When it's used
Use a Hill-climb to keep improving something that already works. For example:
Improving a signal's Sharpe ratio while keeping turnover in check
Making a slow warehouse query faster
Refining the prompt behind an automated research summary
How it works
Each round makes several attempts at improving the current best version.
The best attempts survive and become the starting point for the next round.
What each round learns is carried into the next, so later rounds build on earlier discoveries.
The hill-climb stops when a round fails to improve on the last, or when its budget is spent.
Attempts can be scored on a number, or by judges when quality is a judgement call. A hill-climb can also start from an Ideation stage, and return to it periodically for fresh ideas.
Guarding against overfitting
When a hill-climb is scored on a number, it can hold back part of the data as a test set. Each round is scored on the rest of the data, and the final result is scored once on the held-back test set. The results show both, for example Validation 0.95 · Test 0.88. A large gap between the two is a sign the result won't hold up on new data.
Steering
You can give a running hill-climb guidance for its next round, for example:
In the next round, focus on reducing drawdown rather than raising returns.
What you see
While it runs, the batch shows phases such as Splitting the dataset…, Refining attempts…, Scoring candidates… and Scoring on the test set…. A hill-climb scored on a number shows a score chart; one scored by judges shows the run explorer, with the leading attempt in each round and how ratings changed over time.
Example request
Improve our momentum signal's Sharpe ratio, keeping annual turnover below 300%. Hold back the last two years as a test set.
Last updated
Was this helpful?
