Explanation

“To” is used after following verbs if the object is a “person”.

Say, suggest, speak, reply, explain, complain, talk, listen, write, report, pray and describe”

He did not reply to me.