"Caps" are values that have been determined to be necessary to avoid some negative effect to your performance. There are cap values for many traits, but the caps for Hit and Expertise are the ones that affect every class since Mists came out, but I'm going to focus on Hit
See this is where it gets interesting for the casual. Almost all of the information on the net about hit caps is based around the idea that you're going to be raiding, and that you're trying to hit a level 93 raid boss. But if you don't raid, you are never going to see a target at that level apart from the Boss-level training dummy. So how much hit do you really need? What is the cap for the casual player?
Looking for information for this post, I was very surprised at how hard it is to answer that question completely, without having to involve a lot of math. Wowpedia.org had probably the best table of information on required Hit, but it was specifically aimed at spellcasters.
Note that the table is split into 2 rows: the top one is where you get a 1% hit bonus, and the bottom one is where you don't. (The Notes section tells you what circumstances get you that bonus.) The take-away for us casuals is: you need enough Hit rating to hit whatever you're planning to hit. Is that a raid boss in LFR? Then you need enough to hit a level 93 mob. How about just a 5-man heroic? Well, bosses in 5-mans are level 92. Scenario "bosses" are just 90-Elites.
Here's how that translates for melee types at level 90:
So in a nutshell: whatever your activity is, you can be comfortable if you're close to the required hit rating for the maximum level of mob you expect to face. How close is close? Well, consider this: if you are .50% below the cap for a given target, according to probability, you will miss your target .5% of the time. Or, in real terms: 50 misses per 10,000 swings. Now if that doesn't make you be more relaxed about getting to the cap, I don't know what will!