Justice is not a free-standing virtue. It needs to be backed up by other virtues, if it is to be a virtue at all. Justice alone is cold and potentially deadly. It can and does kill (think of the death penalty); it does not nourish, vitalize, and revive. It can punish a murderer but cannot bring the victims back to life. Justice is not truly redemptive.
Justice is the institutionalized form of revenge.
Justice is a minimal virtue. It spells out a required minimum in the relations between us. It defines what must not be the case but cannot say what may be. Justice is the virtue of lawyers and law-courts.
Justice is the virtue of a vengeful god. It is an essentially Calvinist virtue. Our persistent concern with it reveals the secret power a certain kind of Protestant way of thinking.
Justice is a mean virtue. It aims at a mean in the relation between humans. It is a calculating virtue. It asks: what do I owe you and what do you owe me. No more than that. It sees to it that the terms of the contract between us are satisfied.
Justice is necessary but it is never enough. We also need love, compassion, gratitude, good will, grace, and friendship. These virtues outshine justice whose ultimately value is utilitarian. Justice can only provide the platform on which a genuinely human life is to be lived. It cannot generate that life.
Our singular preoccupation with justice is a testimony to the poverty of our social reality.
These remarks condemn a particular way of thinking about justice. There are other ways, which don’t contrast with the other values you mention, but contain and channel them. Revolutionary action that seeks to equalize human relations, opportunities, advantages, etc—to bring about justice—is not motivated by “contractualist” fervor, but precisely by compassion for and solidarity with the outcasts, friendship with fellow militants, the good will that warrants risk-taking, and so on. Its aim is a freer, larger social life. It is eudaimonistic—we cant live well without living rightly in association. You’re correct that justice is not free-standing, but wrong to think it reduces to a utilitarian calculus.