This morning I came across the (American) National Council on Bookstore Tourism (and the Bookstore Tourism blog), both run by Larry Portzline, a bookish Pennsylvanian. In 2005 Larry visited some of Toronto's independent bookstores, and wrote about his visit here. Interesting commentary, especially given how much has changed in the past two years with some bookstores going electronic only (Abelard and Annex Books come to mind) and others opening (Type Books on Queen, BMV's new location on Bloor).
If you are interested in Toronto-focused literary tourism, you might want to check out Bibliotravel's list of books about Toronto. The list is quite incomplete, but includes some good recommendations. The most exhaustive catalogue of Toronto-focused literature (fiction and non-fiction) that I know about is my own, the Imagining Toronto library. I've also uploaded an incomplete catalogue of titles by genre here.
While you're pursuing literary tourism in Toronto, don't forget to visit Toronto's numerous independent new and used bookstores (there's a list in the sidebar to the right, or visit here for details and commentary).
[Books image taken by me.]