Common sense doesn't have the last word in ethics or anywhere else, but it has, as J. L. Austin said about ordinary language, the first word: it should be examined before it is discarded.
Thomas NagelThe external view [of agency] forces itself on us at the same time that we resist it. One way this occurs is through the gradual erosion of what we do by the subtraction of what happens.
Thomas NagelWhat we take ourselves to be doing when we think about what is the case or how we should act is something that cannot be reconciled with a reductive naturalism, for reasons distinct from those that entail the irreducibility of consciousness. It is not merely the subjectivity of thought but its capacity to transcend subjectivity and to discover what is objectively the case that presents a problem....Thought and reasoning are correct or incorrect in virtue of something independent of the thinker's beliefs, and even independent of the community of thinkers to which he belongs. (p. 71)
Thomas Nagel