Speaking of the “best supplemental books”…
Skelos is great for your purposes but there are errors or things which need to be clarified since they are not too well explicitly written (if you can access G+ before it dies see how many people asked clarifications on Mesmerism or Herbalism).
Be ready to homebrew solutions.
But other books do not have that problem, but off course they deal with other subjects.
I really enjoyed Barbarian and Brigand, they really give you an “Hyborian age” feeling.
I really like them.
Barbarian has also some nice mechanics, for example for shield walls.
I’m less enthusiastic about Thief and Mercenary since (at least for my humble opinion) they are too “generic fantasy”. I still do not understand why they ignored non-REH fluff (De Camp or Roy Thomas) to add their own stuff which is still non-REH (the asshuri in mercenary are definitively not those described by REH). However from a mechanic point of view Mercenary give you rules for small battles and they are fun.
Pirate it’s ok and it’s a must for naval combat, although I liked it less than a similar book for the old mongoose d20 conan rpg.
Ancient Ruins is particularly useful for giving you pre-made adventure situations, rules on treasures and ruin-making rules. I think it’s a very useful book.
Horrors is also very useful, although it has way less figures than what one would expect in a rpg bestiary.
Nameless Cults is more fluff than mechanics. I’m not enthusiastic of many entries in that book (there’s a lot of “this god is an illusion” which to me is often not interesting, compared to old fluff in old d20 books), but in general it’s a great reading.
And finally do not under-value the GM screen! The attached adventure generator is very useful to make REH-style stories.