There are two fuses. One to power the switch (relay) and one to power the rear defrosters. One is in the fuse box under the dash, Fuse one i think, the other is in the main fuse box under the hood. Start there. (note: this is for the 91 CRX SI information. Could be different for other models and years.)
Does the light on the switch light up when pressed? If yes, your problem is most likely is the elements in the window, but could be wiring to the rear too. If no, then its most likely the switch or wiring going to the switch from the fuse. To test all this, you need a volt meter to see where the power stops. But start with the easy stuff first.