In addition to my professional role in the realm of technology for worship and pro audio, video and lighting, like a lot of you, I am also a tech volunteer at my own church. I'm subjected to the same criticisms and what some might perceive to be a lack of appreciation in my role as our lead sound technician. I get the joy of hearing "It's too loud!" or "I can't hear my my kid's voice in the mix." just like you do.
Awhile back, something interesting happened, which showed me that I should have been paying better attention. At the rear of our sanctuary, there's a partial wall that separates the main room from the elevated overflow areas. The front-of-house mix position is in one of those overflow areas. Along that wall, we have a row of chairs that's more heavily padded, so our older members tend to sit there. For a time, it wasn't unusual for me to get comments from two particular people among that group that the system was too loud for their taste.
"If it's too loud, you're too old," right?! Before you think that I'm insensitive or cynical or whatever, bear with me for a minute. The mix position is further from the speakers than those people were and I monitor SPL frequently. I am very sensitive to what "too loud" is, so I almost shrugged them off. Then, it happened. I got out of the booth and listened to what they were hearing.
Sound systems and rooms aren't perfect. Try as designers might, not every seat in the house experiences the same sound quality, and this was no exception. When I actually picked my tail up and listened to what they were hearing, I had a "duh" moment. These folks sit with their heads up against a wall. Bass has a tendency to build up where? Against a wall or in a corner. That's just physics.
So as it turns out, we had a communication problem. The overall sound wasn't too loud as much as those folks were getting their fillings rattled (so to speak), since the bass was too loud where they were sitting.
Get out of the booth and listen to what others hear before being tempted to roll your eyes (when they're not looking, of course) and go on with your business. Sometimes, it really is too loud, even when you think that it's not. Lesson learned.