Personally, I fell head over heels for Amantani, the place there are neither automobiles, resorts, nor eating places.
Ask a few museum of traditional costums placed on the principle square and also about the place to see native folks weaving.
After that, a small double-decker boat will take you 5 km East of Puno Harbour to the floating Uros Islands.