No. He wasn't upset at them. He just started freaking out. In fact, Carl has a choice to run from Tager (the "right" choice"), not kill him. He doesn't even kill him when you fight him (though, the only characters that he out right said he wanted to kill is Ragna, Jin, and V-13).
With Litchi, he states that he has a problem trusting people who call Ada Nirvana because they always try to take her away from him. He doesn't get upset at Litchi either, just paranoid. Even in Arcade mode when you win with Carl he doesn't sound upset. Only with Hakumen does he actually drops his manners and stuffs.
Lastly, he seems to forgive Rachel for freezing up Ada in his story mode since he says he believes what she says to him afterwards. He "grr"ed but that was about all the rage I saw. He was crying mostly.
Well, at first Bang didn't like him because he was "impersonating" one of the six heroes.