Fenway would probably host all the baseball events. Gillette could host soccer. The Garden could host gymnastics and/or basketball. And most local colleges could get in on the sports we only care about during the Olympics: badminton, wrestling, weight lifting, etc.
The three things Boston would need would be a Track & Field Stadium, a Swimming & Diving Center with significant spectator seating and an Olympic Village. But instead of building three "white elephants" a Boston bid could follow the lead of my current hometown of Atlanta. Atlanta built a Olympic Stadium and converted into a baseball stadium. They turned the Olympic Village into dorms. The Aquatic Center was built on Georgia Tech's campus and then renovated to remove some of the stands.
I am sure UMass Boston would gladly "host" the Village in exchange for new dorms. Many local schools would gladly take a brand new swim complex. The tricky Boston venue would be the Olympic Stadium for Track & Field. The only current complexes near the city that might have the space to convert to an Olympic Stadium and then convert back to a more appropriate permanent space seem to be Alumni, Harvard Stadium and Nickerson Field. Would any of the schools be willing to go through four to six years of building and reconfiguring just for the honor hosting the Opening Ceremonies and getting a face lift to their current stadium? I think BC would pass. I don't know about the politics or logistics at the other schools.
Just because Boston could be a good host doesn't mean it ever will. To secure the Games, you need a major driving force. The Fenway Sports Group would seem like a logical leader, but my guess is that they would rather invest in sports and events that have a more predictable and immediate return on investment.
For now I guess it is just blog fodder but certainly let me know what you think.