I would suggest you have an air leak in the vacuum system.
a clogged AIV (air induction valve) will cause this leak internally and so will a stuck open PCVV(positive crankcase ventilation valve)
First thing go over all your rubber hoses and make sure they are tight and no cracks or breaks.
The AIV is usually near the throttle chamber (it will have an electrical connector) . Take it off and you will see it is a motor driven valve. the valve gets clogged open often and wont operate correctly. clean it out with starting fluid and an old tooth brush.
The PCVV is cheap .........just replace it
Another Possibility is a clogged open EGR (exhaust gas recirculation ) valve.
It can be cleaned usually the same way
a $20 book will help you greatly to identify these components,their location, and how they work.
I've even seen a clogged air filter cause the same symptoms you describe
What I'm saying basically, is you have an air leak in the engines vacuum somewhere