Use the button to set a Time Keyframe.
Use "M" to set a Marker.
Use the mouse to select the Marker.
Press 'Delete' to delete the Marker.
You have also deleted the Time Keyframe you just set.
Not sure if this is intentional behaviour or not. If it is, then my apologies.