The Doctor says that Jim is a good man and that the people of that town shouldn't treat him badly just because he is black. The townspeople then change their behavior towards Jim a bit, because the Doctor is a respectable man and what he says must be true. Thus, Jim starts living in better conditions, although he is still chained and fed only bread and water.
