An act is only wrong is the harm done is intended, if the actors actions motivation for example was a reaction to something that aggravated the actor then the fact that the actors action caused harm is irrelevant since the motive was not to cause harm, it was to react to an aggravation.
Commonly accepted behaviors may cause harm in certain situations, you can't say in those situations that the harm makes the behavior unethical because the action is generally accepted as being okay.
Whatever harm that is a result of my actions to others is not my responsibility its the responsibility of the person harmed if the my actions are perfectly legitimate. I should have to refrain from an action just because harm will result to the other.
Just because an action doesn't cause me harm doesn't make it not wrong nor does it preclude me from doing something that causes harm in response.
If I feel it is in my own interest to do something than what harm is caused to another is irrelevant because I have no obligation to put other peoples interests ahead of my own.
Whether an action causes harm or not is irrelevant. What determines if it is right or wrong is commonly accepted ethics.
Anyone that is eccentric, has bad interpersonal skills, bad communication skills or falls outside the norm are fair game for actions that cause harm to them since those fall outside commonly accepted norms. Furthermore the above groups really have to place expressing their opinions about ethics since they themselves are unethical by their nature.
In addition to these this person tried to argue that someone that speaks in a boring monotone, is hard to hear, stutters, or is otherwise painful to talk to is engaging in wrongdoing because its aggravating to people he's talking to, and people are justified in responding accordingly (eg If there was a team of people in a project, and the person in question was the lead, then the rest of the people are perfectly justified in not cooperating, being irresponsible, or telling the lead to go fuck himself)There are proper behaviors in any given situation, one not only consider the past and present in determining if an action is ethical. What harm may come in the future is irrelevant and ultimately not my responsibility. (Another variation of this was only the inputs into an action determine whether it is right or wrong, not the output. The actor is not responsible for the output.)
Also argued that if in a heated conflict in a project or business, a partner feels insulted or "rubbed the wrong way" by his partner or doesn't like something for any reason. He is perfectly justified in telling his partner to go fuck himself and abandon the partner regardless of what harm would be done to the partner (or the project) as a result.