I used to smoke cigs, and my lungs thanked me when I quit.
I developed a revisit of asthma and allergies about five years after the cigarettes (which I had in childhood, but stopped with cigarettes, oddly), and merely took it as "that's the way I worked." Enter the pipe, and the ambient smoke and random wisps that likely enter my lungs, the allergies diminished and asthma went away again. My lung function actually seemed to increase, contrary to smoke-free (or hooked on deathsticks), as being "totally clean" meant my lungs just rebelled against everything else unavoidable, from dust, dander to pollen. Maybe it works 'em out a bit. *shrug*
Cigarettes are a quantity-over-quality thing. I think lungs can sometimes benefit with a little smoke, but not too much, so pipes for me are "having my cake and eating it, too." As for wanting to inhale pipe smoke? I figure, why bother? My lungs can't taste much anyway.
So taboo? No. Like Yak said, your smoke, your choice, brother.
As long as you're healthy and happy.