The Prime Minister _____ a speech on the new reforms yesterday.

Was delivering

Delivered

Is delivering

None of these

The action happened in the past ("yesterday"), so we use the simple past tense.

"Delivered" is the correct past tense form to describe a completed action.