Edo Castle, still sitting in the middle of central Tokyo. What's open to the general public is the Imperial Palace East Gardens, laid out as an attached garden of the palace on part of the honmaru, ninomaru and sannomaru of the old Edo Castle. It has been open to the public since the 43rd year of Showa (1968). The sight is the plain fact that anyone can now walk through the heart of the shogun's castle. Entry is free, through the Otemon, Hirakawamon or Kitahanebashimon gates. The most important thing when fitting it into an itinerary is the day of the week: it's closed Mondays and Fridays, and from 12/28 to 1/3. It opens on public holidays and closes the following Tuesday instead. Opening hours also vary a lot by season — until 18:00 in summer, until 16:00 in winter.
Why it stays with you