I was on Santorini last week and did the hike from Fira to Oia. My hotel was in Oia but everyone said to get a ride to Fira and walk back as the elevation is higher in Fira and therefore its more downhill walking toward Oia. Having finished the hike I do generally agree on the downhill slope overall, but the hike has a lot of uphill walking either way you go. From Fira to Oia the start has some up/downs on cobblestone walkways and then mostly downhill on trails after Imerovigli and not hard. The major uphill part is the peak you climb over in the last hour or so. From Oia to Fira its the opposite: a peak at the outset but then a really long, slow uphill all the way to Fira. Keep that in mind if you start after 11am, by noon it was hot.
Totally awesome hike, but I am pretty athletic and thought the 3.5 hour walk a full workout. Hot and sweaty by the end, but WOW is it a great hike.