The court reached the decision while rejecting an appeal by a young Afghan man against an order for his removal.