Sounds like you have some sort of caterpillar eating the foliage. Handpicking is an excellent, easy way to control them, especially if they haven’t spread to other shrubs. Stay on the lookout for them, though, and continue to handpick, if necessary.
If they just ate a bit of foliage, you have nothing to worry about. The plants should recover just fine.
If you’d like to identify the pest, I’d recommend getting a copy of Whitney Cranshaw’s excellent book, Garden Insects of North America. Full of helpful (if kind of disgusting) photographs, it also has a plant-by-plant pest guide in the back which is really useful.