I’m still vaguely surprised by how much noise affects my general ability to Do Things. Sometimes it seems like my energy levels are just really irregular, but I think most of it does come from actual energy expenditure on listening/dealing with noise.
I remember, when I was spending some mornings at the library and some at a coffee shop (depending on the library’s hours) I did notice that I had less energy and tolerance for noise on coffee shop days, because I had used up a lot of it at the cofee shop. But it took being on a regular schedule for quite a while for me to notice. I think that’s one reason I didn’t notice this earlier in my life– the noise level per day was more constant, so there weren’t as many highs and lows to notice.
And second– the truth is, I do a lot more verbal stuff (as opposed to mostly-passively absorbing information) than I did when I was in school. I write much more. I interact with people much more. I haven’t gotten worse, I’m doing more stuff that takes more effort.