I think you are referring to tannins. If so, then yes, oaking helps. However, using skins, oak, etc. during fermentation helps tons......instead of just juice. You can always add tannin, too (during bulk aging). Which will help make it closer to your liking.
I'm no wine making expert, but I've enjoyed drinking it for quite a while!. My understanding is that the use of Oak will only soften the tannins you already have in the wine and won't add any?, Using Wine skins and quality juice is the way I look to add balance to a wine.
I think you are describing the feeling you get (vs. taste) at the end of a balanced wine; a little puckering caused by tannins. I would experiment if I were you to find what suits your palate. I tend to agree with @Harbrook, I've overoaked wines that were not tannic, so oaking the wine may not get you what you are looking for.
now my curiosity is peaked. i understand that oaking helps smooth out the tannins already there, however, doesn't the wood itself provide some extra tannins as well that gets thrown into the mix? don't get me wrong, i know that the skins and whatnot are the go-to, however, just wanted to understand the oak/tannin conversation.
Another of the Wine topics that can get pretty deep. In less than 10 minutes time I found my head spinning at the differences both in both chemical and taste of tannins. In other words tannins are not all the same and there are of course tannins that we would never use in wine making. None-the-less the following is a less chemistry oriented article of tannins used in wine. AND if you pick your own grapes to make your wine this article is of particular interest and importance to you.
Barrel ageing will reduce tannins because of the fact that wood is not a perfect hermetic seal. Microscopic amounts of O2 can penetrate the wood. This Micro-oxidation will serve to reduce the amount of tannins, thus softening the wine.
Simply adding oak to your wine will not have any dramatic effect on the reduction of tannins. In fact, the amount of tannins will actually increase since oak does contain tannins.
I agree with all of the other posts. One other thing to try is to up the acid content of the wine. My best advice is to perform acid trials on your wine (set up several samples, adding different amounts of acid in each) to set the right amount of "pucker".