Explanation

The correct sentence would be: "He was ashamed of stealing."

The preposition "of" is used to indicate the reason for the shame.