Confirming Patel "would endanger the FBI's integrity and compromise its critical mission," the former officials wrote.