There are whole threads on here debating the bung taste, and whether it is real and did it come from the bung, or whatnot. Yes, I have had it happen on a batch of apple. I bottled two bottles of it just to see if time will take it away, and poured out the rest. The taste was strong.
I do plan to report my findings on the bottles I stored once enough time has passed to open one and try it.
The white solid bungs sold now are made in China with who knows what. If you soak them in hot soapy water - or even better, run them through the dishwasher before you use them - it is said to help kill that taste.
I have made retailers aware of this problem, and the replies I got insisted it is my winemaking and not their bungs. I no longer order supplies from those retailers.