I have started driving away at the 3 minute mark in busy areas. Sometimes I get a cancel fee, sometimes not.
The other day I had one where I drove away at 3 minutes. He tried calling and texting me twice, which I ignored. He texted "I am going to ask Uber why you are ignoring calls and texts." Just for that, I refuse to cancel! Now he can go without Uber until HE cancels! Toes on kerb next time, jerk.
I left the app on with his pending ride for another hour and a half, he never cancelled. Lyft kept me busy the whole time. I ended my shift and turned phone off, ride still pending. Woke up the next morning with a cancel fee payment.