Deselect button for all objects

There is a deselect button for lines and shapes, but not for stamps, text, paths, and groups. I would like to be able to deselect these items without having to find an area with no stamps, text, paths, or groups to click on. This is especially important since I can't change what types of objects I can select when something is selected.