Remove Byte-order mark

Some files include a byte order mark (BOM), which can be quite annoying. In this post I explain how to remove it.

Some use tail, but do not use this! This can break your file.

You can remove BOM using the following command: awk 'NR==1{sub(/^\xef\xbb\xbf/,"")}{print}' INFILE > OUTFILE

Another method was suggested by Denilson Sá and uses GNU sed:

On Linux or Cygwin:

sed -i '1 s/^\xef\xbb\xbf//' INFILE

On FreeBSD or Mac OS X:

sed -i .bak '1 s/^\xef\xbb\xbf//' INFILE

Links