New hidden tax hikes will hit many people this year. Here’s how to lessen the blow.

By Laura Saunders | The Wall Street Journal – Mon, Dec 2, 2013 11:31 AM EST

You might think that tax planning would be easier this year. You could be wrong.

On New Year’s Day, Congress finally agreed to settle many unresolved issues, raising taxes for most Americans. But only one change was simple: the end of a two-percentage-point cut in Social Security taxes, which is costing wage earners up to $2,274.

Most of 2013’s other tax increases are less broad-based and more complicated.

This year’s code includes two new taxes, a new top income-tax rate, a new top rate on long-term capital gains and dividends, a new inflation adjustment to the alternative minimum tax, or AMT, and two revived tax-benefit “phaseouts.” (See table on page B10.)

The upshot: For many people, it is more important than ever to estimate next April’s tax bill before year-end, while it might still be possible to make adjustments.

While the top 1% of taxpayers will bear much of the burden of this year’s increases, experts say, the code also has new tax traps for the affluent, roughly defined as people with an adjusted gross income, or AGI, between $150,000 and $500,000.

“Especially for families in this range, it’s hard to predict the tax rate without a sophisticated computer program,” says Chris Hesse, a tax specialist at accounting firm CliftonLarsonAllen in Minneapolis.

The changes make it especially hard to guess the marginal tax rate—or how much a taxpayer will owe on added dollars of income. Important financial decisions often are based on such calculations.

An example: This year a married couple with two children, typical deductions, $230,000 in wages and $20,000 in net investment income will be in the 28% income-tax bracket. But if they have an additional $10,000 of wages, Mr. Hesse says, their actual rate on it will be 38.8%—more than 10 percentage points higher.

“This hidden rate comes from the interaction between the AMT and the new 3.8% net investment income tax,” Mr. Hesse says. “Last year there was no 3.8% tax, so this anomaly wouldn’t have existed.”

This year’s taxes become even more complex if the taxpayer has “passive” income or losses from investments, or a sharp spike in investment income, perhaps from the sale of a large asset. The family mentioned above, for example, could owe nearly 13 percentage points more than the nominal 15% rate on long-term gains, Mr. Hesse says. (See chart on page B10.)

To find out how you could be affected, consult a tax professional, such as a certified public accountant or an IRS Enrolled Agent, or use an online tool. Preparers such as TurboTax and H&R Block offer estimation tools, and the Tax Policy Center, a nonpartisan research group in Washington, offers an estimator that allows users to compare last year’s tax with this year’s at calculator2.taxpolicycenter.org.

Here are facts about the new pitfalls, plus moves for many to consider making before year-end.

What to Know

The Net Investment Income Tax. Passed by Congress in 2010 to help fund the health-care overhaul, the NIIT is an entirely new levy this year of 3.8% on the net investment income of most couples above a threshold of $250,000 of adjusted gross income, or AGI ($200,000 for single filers).

The tax applies to net capital gains, dividends, interest, rents and royalties, among other things, but only to the amount of such income that is above the threshold. So if a couple has AGI of $240,000 plus $20,000 of net realized gains (after deducting losses), $10,000 would be subject to the tax.

One point of confusion is how taxable retirement income such as pension or individual-retirement-account payouts interacts with the 3.8% tax. While this income isn’t subject to the new levy, it can raise AGI so that other income is.

For example, a couple with $270,000 of AGI in Social Security, pension and IRA payouts wouldn’t owe the 3.8% tax. But a couple with $240,000 of the same income, plus $30,000 of net gains and interest, would owe $760 on $20,000 of the investment income.

Because they are tax-free, qualified Roth IRA withdrawals aren’t subject to the 3.8% tax and don’t raise AGI. In addition, shareholders in closely held S corporations and partnerships won’t owe the 3.8% tax on payouts if they “actively participate” in the business by meeting certain requirements.

Alan E. Weiner, a partner emeritus at the Baker Tilly accounting firm in New York, says many people still are unaware that deductions for state taxes and other expenses can reduce net investment income, even if they are limited elsewhere on the return, such as for the AMT.

Personal Exemption Phaseout. This benefit limit, also known as PEP, returns in 2013 after an absence of three years, with some differences.

This year, each taxpayer can ordinarily deduct $3,900 for herself, her spouse and her dependents. Thanks to PEP, this benefit now begins to phase out at $300,000 of AGI for married couples and $250,000 for single filers. It is gone by about $422,500 for couples and $372,500 for singles, says Roberton Williams of the Tax Policy Center.

For taxpayers who aren’t subject to the AMT, Mr. Hesse says, PEP can add an extra percentage point per taxpayer or dependent to the tax rates on income in the phaseout range.

Such taxpayers often will be residents of states without an income tax, he adds, so a family of six in Texas or Florida could have a six-percentage-point higher rate on some income due to this provision.

Pease limit on itemized deductions. This hidden increase, named after a former congressman from Ohio, also returns this year.

The Pease limit disallows 3% of itemized deductions above the same income thresholds as PEP, up to a maximum disallowance of 80%. Common itemized deductions include those for mortgage interest, charitable contributions, medical expenses and state and local taxes.

In effect, the limit is an income-tax surcharge of about one percentage point for taxpayers in the 33% bracket and 1.2 percentage points for top-bracket taxpayers, the Tax Policy Center’s Mr. Williams says.

Medicare payroll-tax increase. Taxpayers will owe an extra 0.9% of Medicare tax on wages above $250,000 of adjusted gross income ($200,000 for singles). This comes on top of the 2.9% Medicare tax for all workers, which is split evenly between employer and employee.

Taxpayers should beware of a marriage penalty that can come with this tax, says Elizabeth Beerman, an associate at CliftonLarsonAllen. Two single people who live together and earn $200,000 and $100,000 of AGI, respectively, won’t owe this extra payroll tax. But they will owe an extra $450 if they are married—and employers won’t withhold it, because each is below the AGI threshold. “People are going to be surprised,” Ms. Beerman says.

What to Do

Minimize adjusted gross income. Four of this year’s tax increases have thresholds tied to AGI—the number at the bottom of the first page of the tax return, before deductions—rather than to taxable income, which is after deductions.

This means that tinkering with deductions won’t help to avoid these increases. Instead, “we have to do everything possible to lower AGI,” says Jonathan Horn, a CPA practicing in New York.

Among the moves that help: contributing to a tax-deductible IRA, 401(k) or defined-benefit retirement plan; realizing capital losses up to the amount of realized gains, plus $3,000; favoring tax-free Roth IRA income over taxable retirement payouts; having tax-free municipal-bond income; or deducting moving expenses and heath-savings-account contributions.

It also helps to make charitable contributions with appreciated assets rather selling them and giving the cash.

Janet Hagy, a CPA practicing in Austin, Texas, says she is advising some clients to sell private-company stock in installments to spread income over several years to avoid triggering higher taxes.

In states with community-property laws, such as Texas and California, Ms. Hagy says, it could now make sense at times to use the “married, filing separately” status to minimize overall tax. “Couples with separate-property capital gains can sometimes avoid the 3.8% tax by filing separately,” she says.

Do an AMT check. This year the alternative minimum tax might be lower than in the past for some.

The reason: The new higher rates and limits on exemptions are raising regular taxes for many affluent taxpayers, which in some cases lowers their AMT. That in turn can allow them to take deductions that are clipped or eliminated by the AMT, such as the one for state and local income taxes.

Given this change, David Lifson, a partner at Crowe Horwath in New York, recommends either accelerating or deferring state tax payments and other AMT-vulnerable items, depending on circumstances.

“The point is to make sure that every expense that can be deducted is deducted, because it may not always be able to be deducted,” he says.

Take investment losses. Realized capital losses offset realized capital gains in taxable accounts, plus up to $3,000 of ordinary income a year. Taking losses can also lower adjusted gross income.

But take care not to fall afoul of the “wash-sale” rules if you plan to buy back the losing asset. Use of the loss is postponed if a taxpayer acquires shares 30 days before or after selling losing shares of the same investment.

The tax code counts shares bought within an IRA or through exercise of a stock option as purchases.

Harvest investment gains. This year’s tax changes retain the zero rate on long-term capital gains for married joint filers with up to $72,500 of taxable income ($36,250 for single filers). Such income doesn’t include tax-free municipal-bond interest.

This means that taxpayers in the bottom two brackets have the ability to “harvest” gains up to that amount, pay no tax, repurchase the asset and reset their cost basis higher. The wash-sale rules don’t apply to gains, just losses.

It could work like this: A married taxpayer has 1,000 shares worth $95 each that he bought for $70 three years ago. If he can keep his total taxable income below $72,500, including the gain, he could sell the shares, buy them back immediately and pay no tax. But his “cost basis”—the starting point for measuring future capital-gains tax—would now be $95 instead of $70 a share.

That could save future taxes if he needs to sell the stock before his death.

Make charitable contributions. Donations to hundreds of thousands of qualified charities are tax-deductible if you have a letter in hand specifying the deductible amount before you file next year.

Think twice before writing a check, however. Often a better move is to give appreciated assets such as stock shares. Within certain broad limits, any capital gain isn’t taxable and the full value of the gift is deductible.

Donors who are 70½ and older can use the so-called IRA charitable rollover for 2013. It allows account owners to contribute up to $100,000 of a required payout directly to qualified charities such as churches, schools or other groups.

There is no tax deduction for such gifts, but neither is there income. That, in turn, helps lower AGI in a way that can reduce certain Medicare premiums or taxes on Social Security payments.

Maximize medical and miscellaneous deductions. Both are subject to such high hurdles that taxpayers often have a hard time claiming them unless they strategize, say by bunching expenses from more than one year to claim them at once.

Most taxpayers can deduct only unreimbursed medical expenses above 10% of their adjusted gross income, although for taxpayers age 65 and older (and their spouses) the hurdle is 7.5% of AGI, if they aren’t subject to the AMT. This exception applies through 2016.

However, the list of qualified expenses is long, including contact-lens solution, a wig after chemotherapy and acupuncture. For a full list, see IRS Publication 502.

Miscellaneous expenses are deductible only above 2% of a taxpayer’s AGI. Typical write-offs are for unreimbursed work-related expenses and certain investment expenses.

Use an expiring tax break. This is the last year to take advantage of more than a half-dozen popular breaks, unless Congress extends them next year.

They include the IRA charitable rollover for people 70½ and older (see above); the state sales-tax deduction in lieu of a state income-tax deduction; the generous Section 179 expenses deduction available to small businesses claimed on individual returns; and the $4,000 tuition and fees deduction.

Make annual gifts. The federal estate-and-gift-tax lifetime exemption is now $5.25 million per individual, and it will rise to $5.34 million next year.

But 19 states and the District of Columbia still have estate and inheritance taxes, many of them with exemptions far lower than Uncle Sam’s.

All, however, follow the federal practice of allowing givers to make tax-free transfers of up to $14,000 per recipient a year. For example, a married couple with two married children and five grandchildren could make 18 separate gifts of $14,000, for a total transfer of $252,000.

Givers can transfer assets such as stock, or even partial interests in assets like real estate or a business, instead of cash. In that case, the giver’s tax cost for the asset carries over to the recipient.

Jay Rivlin, a partner at law firm McDermott Will & Emery in New York, often advises people who are making gifts of nontraded assets that require an appraisal, such as real estate or partnership shares, to make two gifts—one just before and one just after year-end. “One appraisal can be used for two sets of gifts,” he says.

Write to Laura Saunders at laura.saunders@wsj.com