You always have a number of options before needing to resort to a product that actually kills the bugger.
You can sometimes wash it off the plant and it will stay off. There is an insect repelling wand available, although a regular watering wand does a fair job.
You can sometimes pick off enough with your (ughhh!) fingers to bring the problem under control.
You can sometimes use a product that actually repels the pest insect in some way. Neem oil soap, garlic, and hot pepper wax are examples of insect repellents.
You can use a product that serves as a barrier to the pest insect preventing it from getting on to the target plant. Diatomaceous earth is often used as a barrier for some insects.
You can sometimes use some kind of trapping device to catch the insects; usually you will dispose of them is some way, so this is a form of insecticide. You can trap yellow jackets, slugs, and flies to name a few targets easily trapped. See Insect Traps
Finally, if all else fails, you need to use an insecticide, a product that kills the problem insect. Here you again have a number of choices, the first being whether you feel strongly about using only natural pest controls or whether you are comfortable with synthetic products.