There are different approaches to take depending on the substrate. Metal and glass allow for more aggressive options, but things like painted surfaces, plastics, and drywall may require gentler tactics. These are also important factors to consider when selecting an adhesive in the first place. A rubber based adhesive has a lower operating temperature range which means heat can be your friend for loosening the adhesive's bond strength. A blow dryer may be enough to loosen up the adhesive so that damage is mitigated. An acrylic adhesive is going to be harder to remove since it can withstand temperatures up to 240 F. After all, the things that make an adhesive bond well also make it difficult to remove.
So with drywall, the paint will most likely be peeled off or some of the drywall itself can come off. Start with some heat and see if that helps loosen things up so that a scraper doesn't need as much force behind it. With that in mind, it might be useful to just scrape the adhesive off and repaint the surface. This is especially true if heat does not help loosen up the adhesive.
With other substrates like glass and metal, you can use a scraper without having to worry about damage too much. You can also use a solvent, alcohol, oil, or acetone to break down the adhesive residue that often remains. Be sure to check the instructions on any chemical you use to make sure it's suitable for the substrate.
On plastic surfaces, you'll want to be especially careful to use the proper chemicals so that it doesn't cause additional damage. Sometimes a little elbow grease is the best bet. When using a chemical or oil, it's important to first find out if it's suitable for use on the material, but secondly to test it out in a small inconspicuous area to make sure it doesn't stain or damage anything. It's always best to work with chemicals in a well ventilated area.
In short, use heat when possible, then scrape away what you can. After that, use some sort of solvent or alcohol to help break down the remaining adhesive.