Engine & Internal - Weird noise

My 98 Civic DX is making a very odd rattling sound. It has the following charectaristics

1) It happens at really slow speeds, and is triggered by accelerating or decelerating (reverse OR forward).
2) It happens when you rev the engine in neutral or with the clutch in. It doesnt happen at all when the engine is not being reved.
3) It comes and goes (sometimes I drive and it doesnt do it)
4) Its gotten worse in the last couple of weeks. Got really bad shortly after (but not right away) I replaced the exhaust manifold.
5) It can be fairly loud.

Take off all your heat shields.

Just like forty04 said it sounds like it could be a heat shield if you dont want to take them off just grab the one where it sounds like it is comming from and bend it alittle so it has a different bow in it to keep it from flapping or vibrating.

I found out that it is the heat sheild on the short connector pipe right after the exhaust manifold/Cat Convertor. Can I take that one off?

you can take any of them off, or all of 'em, doesn't matter :)

Or you could find the heat sheild that is loose and tie it down. I found that on my mom's accord (and 2 other accords of the same year/model) that the heat shield on the cat was loose and rattling. I got some mechanic's wire and tied the heat shield to the cat and the noise went away and the car runs fine.

sorry to get off topic, but i was wondering this for awhile .... is it okay to not have the heatshields on a stock car? is any damage done if lets say u buy a car and leave the engine stock but just take the heat shields off?

no it wont hurt your car