It always takes longer than you expect, even when you take into account Hofstadter’s law.
Estimation is hard. You’re basically asked to predict the future. Somehow, you’re to predict everything that will go wrong, all of the hidden gaps in requirements, all of the surprises in the code, each unexpected system outage. Personal interruptions, illnesses, emergencies, surprise side projects, changes in priorities. Since no one can properly predict the future and all of these events that will impact your schedule, you are doomed to get it wrong.
Sadly, people tend to be optimistic and fail to take all of these things into account. If only there would be just an expected amount of interruptions, then they could deliver based on some kind of schedule. The real world is cruel seems to always add more surprises.
There are many things you can do to improve your estimation skills. Some of it comes with practice. Ultimately though, the best course is to get out of the business of predicting the future. It’s better to deal with more certain things.
Instead of estimation, turn your focus to delivery. Predictions are nice, but what people really want is delivery. After all, the desire for prediction is simply a question of “when”. When will they get what they want?
So, give them what they want. Deliver. Better yet, deliver often. Several small deliveries will be better received than occasional, late deliveries. And with frequent deliveries, they are much less likely to be late. This makes for a much happier client.