Well, the vitamin N and various other carcinogens can cause cancer throughout the body, regardless of the point of ingestion. So, it's not that ridiculous to hear people talk about lung cancer from pipe smoking.
That being said, I agree the chance is much much smaller in comparison to 2+ packs of cigarettes a day. I too was an insanely heavy cig smoker, 2-3 packs of Luckies, then straight up Camels (my rationale: listen, they have filters, better than my Luckies).
Further, the point of primary ingestion is usually where the cancer originates. Thus lung cancer for cigarette smokers and head and neck cancer for us.
To me, and I've spent a lot of time thinking about this (everyone in my family has died from cancer up to this point), it comes down to relative moderation as usual. I tend to view life in the context of enjoyment v. risk. If I stop myself from ever enjoying the aspects of life that are potentially (still just a potential, tobacco does not automatically = cancer) dangerous, well, I would never get in a car again. Or leave the apartment, walk down stairs, get in the shower, or any of the other numerous activities that potentially could kill me. Certainly never look at an airplane and think "Eh, let's go belt ourselves into the giant metal tube and trust that the thousands of gallons of highly refined fuel, along with multiple turbines aren't going to fail in a number of ways too numerous to list while flying a few miles up in the air."
The common response is "tobacco isn't a necessity, it's a choice". And they're right. Just like choosing not to walk from LA to NYC. It's a luxury, an addition to this life. We don't need it to survive, but man is it a lot better than not having it.
Since I started seriously smoking a pipe my overall stress has gone down an amazing amount. I find myself sitting quietly, almost in a meditative state, for a few hours out of my day, instead of a few moments or seconds. I think this will, in turn, lead to a much better quality of life and, dare I say it, a longer life. What people who don't smoke, regardless of their reasoning, will never understand is that it helps me (and presumably us). It helps in a way that is less about the drug and more about the process. Preparing a flake, packing it just the way I like it, etc. That's good for me, even if I might die from it one day.
So yeah, my 4-6 bowls a day introduce another risk to my long term health. But it's balanced against the concept that each day quite literally could be the big one (sorry for the terrible cliché). 2-3 packs a day of cigarettes is an unnecessary and massive risk to health. Sitting in a stress free haze of delicate and beautiful smoke is less so. So much that, to me, the scales have balanced out and it remains a viable part of my existence.