Heat up the badge with a heatgun or a hairdryer. Use waxed dental floss to remove the badge without scratching the paint, it should easily pull off after you've ran dental floss underneath it with the warmed adhesive. Whenever I remove a badge or pinstripes I use Meguire's Paint Cleaner to clean the job up and remove the adhesive left behind. You will see a spot where the badge once was, kinda like when you've had a picture on the wall for a really long time and then you take it off. It won't be as noticeable after it's been driven daily and your car isn't immaculately clean. When you wash it you will sorta be able to see it again, but most everyone else won't. Eventually that part of paint will match the rest of the car.
The spoiler, yes you will see holes. You can get these filled at any body shop.