My friend said, “Hello what are you doing here?”

My friend greeted me and asked me what I was doing there.

My friend said hello to me and asked me what was I doing there.

My friend wished me and wanted to know that what was doing there.

None of these

My friend greeted me and asked me what I was doing there.

The direct speech is turned into indirect by converting "are you" to "I was".

"Hello" becomes "greeted me", and the question is properly structured using asked me what I was doing there.