Polk mostly did what he promised. However, accepting half of the Oregon Country, while prudent, technically violated his promise to fight for all of it.
If the sole criterion were delivering on promises, he would rank high. But I ding Polk for lying to Congress to get permission to rob more land from a southern neighbor who had never bothered us, and for not caring that the acquisition of that territory would continue to inflame sectional conflicts.
I don’t think I’m being unfair to Buchanan. Aside from the Adamses, every president from Washington to Eisenhower were racists, but many of them served with distinction, anyway.
However, even by the standards of the day, Taney & Buchanan’s views on race were extreme (well, outside of the South). The North overwhelmingly opposed the Dred Scott decision — not just the abolitionist minority, but also the racist majority, including many Democrats.