Go to Control Panel >> Taskbar & Start Menu, make sure Hide Inactive Icons is checked at the bottom of the new window, and click the Customize button next to it...
In the new window, find Safely Remove Hardware, and to the right, select Always Hide

Dunno how to make it show for usb drive though, might be a way...