Causes of Fat Accumulation in the Buttocks in Men While many associate fat accumulation in the buttocks with women, men too can experience stubborn fat build-up in this area.