Although Conservative losses are not as bad as some in the party had been warning, there was still criticism of Boris Johnson from his own side.