Why do golfers hold there putter horizontal when trying to putt?

I think you meant to ask why they hold it "vertical" when reading a putt before they strike it. They are doing what is called "plumb bobbing", and it is a way to measure how much break is in a putt and how far to one side or the other of the cup to aim the putt.