It's common courtesy to engage in private sex. You might not want to disturb others with the banging of a headboard if you share a room with someone else, are a visitor in someone else's house, or your children are sleeping in the room next door.
You need not, however, stop having sex altogether. Regular sex actually strengthens your relationship with your partner. Of course, that's only one of sex's many advantages.
Even when private sex is necessary, it shouldn't be a drag. In fact, keeping silent can be very seductive. It can be even more thrilling when you have to restrain yourself.