It is incredibly difficult to walk away from an offer of publication. Here it is — that thing you've been working toward for months or years, continuing on through rejection after rejection, and you're going to say no? Impossible. I'm not talking about choosing one offer over an other, which is a matter of choosing which good option is best for you, but about rejecting an offer (or multiples, but for simplicity's sake let's stick with one) because underneath that shiny promise lies a series of traps.
Those traps are why you should absolutely never sign a contract with clauses you don't understand (or goodness forbid haven't read!). And if you don't have a trusted agent in your corner, recognizing and weeding out those traps is entirely on your shoulders.
Keep in mind that just because the person who handed you the contract says it's fair, doesn't mean it is — no matter how nice that person is. S/he may even believe the contract is fair, and that still doesn't mean it is. The fact is, the lawyer writing the contract is simply not doing their job if the contract isn't weighted, perhaps heavily, in favor of their client. [I'd also recommend that you don't ignore a warning in your gut.]
As you can imagine, there's a reason I'm writing this. Muse received an offer last November, which I — you guessed it — turned down. There were quite a few problems in the contract I was offered, even though the query process itself with this publisher went very smoothly, and the people seemed nice. And you know what? The royalty percentages weren't the problem, so remember not to look only at the money! The nail in the coffin with this contract was one key word: nonnegotiable.
Generally, any iffy clause in a contract can be rewritten or removed, but of course only if the publisher is willing to negotiate (and most are!). Now, a nonnegotiable contract might not be inherently unfair, but remember what I said about those lawyers? They're paid to skew the contract in the publisher's favor. Consider also what not being willing to discuss potential changes means for your future relationship: the publisher is telling you from the first moment, "My way or the highway, Bub." There are many things which are standard, e.g. the publisher having the final decision on cover art, but also many which you should at the very least be able to remove, like a clause which gives the publisher right of first refusal on every book you write going forward. Or financial penalty clauses*.
Frankly, I cringe to think of all of this publisher's authors having agreed to all of the rather terrible clauses in this nonnegotiable contract. Because it is difficult to walk away from a publishing offer, but it's much worse to be trapped in a bad contract. It can cost you quite a bit of time and money, and ultimately even destroy your career. So bet on yourself, on the quality of your work, and your ability to get a truly fair offer. Anything else? Feel confident walking away.
If you'd like more information on contracts, I covered the basic sections of a publishing contract, as well as some warning signs, in my Publishing Contracts series. Or ask me in the comments, and I'll do my best to point you toward an answer!
*A financial penalty clause states that if you don't do X, you owe the other party some amount of money. These aren't inherently unfair either, assuming there are penalties for both sides (which of course there wouldn't be in the contract a publisher gives you, without negotiation). However, even if there are penalties for both sides, you also have to consider whether the thing for which you'd be penalized is entirely in your control — you don't want to have to pay a penalty for something you couldn't possibly have prevented. Either way though, best practice would be to have such clauses removed, or at the very least severely limited, and of course be balanced.
N.B.: I am not a lawyer. All of the above, as well as everything in the Publishing Contracts series, is my opinion only, based on my research and experiences. If in doubt with a specific contract, always consult a legal professional.