Hi New Caver:
Those of us on CaveChat could give you all sorts of advice, most of it pretty good. However, there is nothing that takes the place of hooking up with a local grotto. They can take you to a local practice area (usually a small cliff) and let you borrow some of their gear. That way you can see what works best for you, and get firsthand advice on proper usage.
Someone on here recommended getting a Frog system. That may work for you, but there are other options as well (Mitchel, Texas, Ropewalker). Which one you use depends on a number of factors- the type of pits you will be using it in, your body type, and your own personal preference. That's why it is best, if at all possible, to try out some of these methods for yourself.
I personally have a Ropewalker, but it can be easily converted into a Texas for use in smaller pits. I have pretty much always used a rock climbing harness, my favorite being a Petzl Calidris. The Calidris is the most comfortable harness I've ever had. The only drawbacks to using a rock climbing harness are 1) Excessive gear loops, which get hung in tight spots, and 2) they are not really built for the wear and tear of caving, so they tend to wear out faster. Also, a standard rock climbing harness is not designed to be used with a Frog system, which requires a lower attachment point.
About buying used gear-
If you know and trust the person you are buying it from, such as a fellow caver, then it would probably be OK. There are a lot of cavers who have become "armchair" cavers, but they may still have some perfectly usable gear that they will let you buy for a good price. Or sometimes newbies buy a bunch of vertical gear and use it maybe once or twice, only to never use it again. But you have to weigh out the pros and cons and make a wise decision. For the most part, I would agree that buying used gear is a bad idea, especially from someplace like Ebay or Craigslist.
Good luck getting geared up!
Trogman