Thomas Edison started it, as a suggestion for a greeting to say over the first telephones. His logic was that it could be clearly understood.
Alexander Graham Bell preferred "ahoy", like greeting a ship at sea, but the nautical reference didn't stick. The English had been using "hullo" as a standard greeting, whose entomology derives from Old High German expressions "hallo" and "hollo" and such. "Hollo" shows up as far back as Coleridge's <em>The Rime of the Ancient Mariner</em>, from 1798.
"Once the telephone was perfected, people had to learn to use it. There was a very fundamental question to be answered, one that seems quite odd to us today, and that is: What do you say when you answer a ringing telephone?""It was kind of a riddle in a way, because when you hook up a telephone and you are speaking basically to a stranger, it ran counter to what people expected in their day-to-day meetings, which was their previous experience. And you have to be properly introduced. And you're never introduced on the telephone that way. So you have to find a word or a phrase that very quickly cuts to the chase and allows people to start speaking, and 'hello' was pressed into service."
We could have just as easily ended up with "namaste" (pronounced "NAH-mess-tay"), since much of English is derived from Sanskirt.