Simon Harris said the government’s position stands: ‘We will vote against the agreement’