Under Federal Trade Commission regulation, any information about an individual that is derived from a consumer report or is a compilation of such records must be properly destroyed. These days, almost all of customer information has some connection to a consumer report and is covered under this regulation (scroll to pg 32).
The regulation does not actually require shredding but for most of us, that is the only cost-effective way to comply with the regulation's requirements for destruction. Papers in regular trash are exposed to the public and any private information on those papers can be misused by an identity thief. It can cost your customers thousands of dollars to get their identity back and could be considered a violation of federal privacy laws.
I strongly recommend a "shred all paper" policy for your office because there is too much risk that a piece of personal information will be overlooked on the back side of a form or that the page was used for scratch paper while you were on the phone. It's also easier to enforce the policy when you have a simple rule like "No office paper may be thrown away in the regular trash."
Very small offices can get away with a personal shredder. If you've got more than about 10 or 15 people in the office, it's probably more cost-effective to contract with a reputable shredding vendor who will pick up and properly dispose of your paper waste. Most of the shredding vendors will provide locked bins where the paper waste can be stored until pickup. Have enough bins to be convenient for staff.