It was pretty bad for everyone except the minority who were slave owners. Back then, opponents of slavery were mainly driven by the immoral cruelty of it, but they also used practical arguments, e.g. that it undermined and undercut paid labour, to widen support for their cause.