Compare lines and remove duplicates while optionally preserving order and ignoring case or whitespace, suitable for cleaning exported lists and logs.