Even people who insist they don't care what others think tend to dress, talk, and act remarkably like the specific group of people who also insist they don't care. Not caring is its own kind of fitting in.
Adults like to think they've outgrown peer pressure. The instinct never actually leaves. It just trades hallway dynamics for boardrooms, group chats, and what everyone else seems to be doing.
It happens before you've said a word to someone. A flicker of ease, of familiarity, based purely on how they look. This isn't a character flaw. It's an old shortcut that used to keep you alive.
It's shallow, it's catty, it's what people do when they have nothing real to say. Except that for most of human history, knowing what everyone was up to was one of the most important survival skills you had.