I now recognize it as the Metropolitan Life Insurance Company Tower, aka the Met Life Tower, built in 1909 and modeled after the Campanile in Venice, Italy. It's near the intersection of East 23rd and Park.
Why do I now recognize it? Because it is at the center of Kim Stanley Robinson's New York 2140, which I'm rereading. Now, when I first read the book, at the beginning of the year, I looked the building up in the Wikipedia. Why? Because the building KSR was talking about was clearly NOT the building I recognized as the MetLife Building. That building sits atop Grand Central Station in Midtown and used to be the Pan Am Building. So I went to the Wikipedia, read the description, saw the pictures, said to myself, "I've walked by there", and that was the end of it. I didn't identify the Met Life Tower with the building I'd seen in so many of my photographs (over 30).
The sparkly blue building in front at to the tower's left is a new apartment building designed by Jean Nouvel. The squat white building in front and to the right is the offices of IAC/InterActiveCorp, a holding company founded by Barry Diller. I don't know anything about the low building with the red and white stripes.
Here's the tower at sunrise:
Sunset; notice the sun reflecting off of the Nouvel, and off of the very top of the tower:
Early evening:
In silhouette against clouds:
Here I was probably shooting for that jumble just in front of it at left-center:
In the distance (winter):