Predicting RNA folding with diverse guesses

An RNA molecule is a string of four letters, A, C, G and U, that folds itself into a precise 3D shape, and the shape is what decides what it does. Stanford RNA 3D Folding Part 2 on Kaggle asked for that shape given only the string. I finished 9th, working alone.

The short version of my approach: there’s a limit to how much you can learn from a sequence, and I ran into it early. So rather than chase better inputs, I spent my effort on the output, making sure my five guesses for each molecule were as different from each other as possible.

The problem

For each target you get the sequence, and you have to predict the 3D position of one atom (the C1′ atom) in every nucleotide.

Predictions are scored with TM-score, which superimposes your structure on the experimentally solved one and rewards every nucleotide that lands close to where it really is. It runs from 0 to 1, higher is better. The twist is that you submit five predictions per target and only the best of the five counts. That one rule shapes everything that follows: five copies of the same good guess are worth exactly as much as one.

Templates, and where they run out

The most reliable way to predict a structure is to find one that’s already been solved. If a training molecule has a similar sequence, its coordinates are probably a good starting point, and you can map them onto the new sequence. This is template-based modeling, or TBM.

The catch is in “similar sequence”. The sequence is the only representation of the RNA I had to work with, so I wanted to know how well it points to good templates. I ran an audit: for every sequence in the validation set, I looked for the training structure that would actually make the best template, and checked whether a sequence aligner could have found it.

Often it couldn’t. Here’s one case:

For the 30-nucleotide target 8ZNQ, the best template in the training set is 1AKX, with a TM-score of 0.279. But Biopython’s sequence aligner scores the two sequences at −0.175. Going by sequence alone, there was no chance of finding this template. These are the short “template-free” targets the host talked about. For longer sequences, the aligner did better.

The conclusion was clear: as long as an RNA is represented only by its sequence, there’s a ceiling on the templates I can find. I still tried other template methods, such as searching with the side chain as well as the primary chain. Some did well on my local validation, but did worse on the leaderboard once combined with the deep learning models:

The primary plus side chain search led locally by a wide margin, but wasn’t the best on either leaderboard. In hindsight it made sense. The hard targets can’t be found by any aligner, and the easy ones can be found by almost any aligner, so a cleverer search had little room to help. My final submission uses the public template methods with very little modification.

Spending five guesses

If I couldn’t make the inputs better, I could make the outputs more useful. My final pipeline is close to other shared solutions: templates, then Protenix, Boltz2, and RNAPro refinement. The difference is that when templates and Protenix produce more candidates than there are slots, I keep the ones that are most different from each other, to spread my five guesses as widely as I can.

Here’s why that matters when only the best guess counts:

When every guess is close to the single most likely answer, you score well only if that answer is right. Spreading them out can cost a little when the obvious answer is right, but it gives you a real chance when the true structure is somewhere else.

Making variants

A single template gives one structure, so I made more from it. The pipeline applies three kinds of transforms to each template: a hinge bend at a random pivot, a rotation and shift of each chain (for molecules with several chains), and a smooth spline-based wiggle along the backbone.

Then a greedy selection picks among the variants, trading quality against diversity: it keeps adding the candidate that’s good but also most different, by TM-score, from those already picked.

Who gets which slot

The five slots fill in a fixed order of priority: templates first, then Protenix, then Boltz2, and finally an idealized A-form helix if nothing else is left. How many slots templates get depends on the length of the sequence:

Short sequences, up to 512 nucleotides, get at most two template slots, which leaves at least three for the deep learning models and favors variety between models. Long sequences, where the aligner works better, can use templates for all five and favor template coverage instead.

A few other choices sit inside that:

• Two Protenix runs instead of one. When Protenix has three or more slots, one run uses the multiple sequence alignment and one doesn’t. Their predictions are aligned with the Kabsch algorithm and averaged, which was more accurate.
• Chunking long sequences. Protenix predicts sequences over 512 nucleotides in 512-nucleotide chunks overlapping by 64, then aligns the overlaps and stitches the pieces back together.
• Geometry checks on everything. Every prediction passes through RNA constraints on bond lengths, angles, smoothness and self-avoidance before it goes in a slot.
• One last refinement. RNAPro takes all five combined predictions as templates and predicts the structure again, which improved structural quality. If a target fails here, it keeps its original predictions.
• Both GPUs, always. The notebook had two T4 GPUs, and every phase splits its work across both.

What I’d try next

The lesson I took away is about representation. A sequence aligner only sees letters, and for the hardest targets the letters don’t point to the right template. With more time, I’d explore searching for templates by embedding instead, comparing molecules in a learned representation rather than by their raw sequences.

My solution was built on these public notebooks, and I’m grateful to their authors: Boltz2 baseline, Protenix + TBM, RNAPro inference with TBM, and RNAPro inference.

This post is adapted from my writeup on Kaggle.