The Snooty-Toot-Toot Club Building on Park Avenue

This building on Fifth Avenue, which flies an interesting flag with a logo on it, is not unknown to me but I couldn't remember what it was.

I went over to the doorway looking for a plaque or something but all I found were three snooty younger middle aged women who were, well, not very helpful.

"Could you tell me what this building is," I asked.

They just stared at me blankly and with a look like, "if he has to ask, he isn't anyone worth knowing"..

So I tried  again, " They haven't made it into a condo, have they?" I said jokingly,

"It is a private club" said one of the women with a stony look. 

Oh, OK, pardon me for living, much less asking...