Recursive UTF-8 conversion (BASH)

rhomb · August 24, 2012, 8:19am

Good Morning everyone.

Yesterday we had a Problem with non UTF-8 Files in the Project (happened in accident).

So I wrote a small bash script that should do the trick. It recursively converts all .php files to UTF-8 encoding

Requirements:

Linux system
iconv

Script:


for i in $(find -name '*.php');do encoding=$(file -bi $i | sed -e 's/.*[ ]charset=//'); iconv -f $encoding -t UTF-8 -o $i $i; done

Greetings

rho

julianm · March 13, 2013, 12:32pm

Sorry, but it didn’t work for me. All the files ended with 0 byte after running the command. Fortunately I still have the backup.

samdark · March 13, 2013, 3:30pm

We have ready to use commands for it in Yii repo:

github.com

yiisoft/yii/blob/master/build/commands/CheckBomCommand.php

<?php
/**
 * CheckBomCommand
 *
 * Checks for BOM files in a directory. If --fix provided, removes BOM where found.
 * Use --path=my/path to run on non-framework directory.
 *
 * @author Alex Makarov <sam@rmcreative.ru>
 * @package system.build
 * @since 1.1.9
 */
class CheckBomCommand extends CConsoleCommand
{
	const BOM="\xEF\xBB\xBF";

	public function actionIndex($path=null,$fix=null)
	{
		if($path===null)
			$path=YII_PATH;

This file has been truncated. show original

github.com

yiisoft/yii/blob/master/build/commands/Utf8Command.php

<?php
/**
 * Utf8Command class file.
 *
 * @author Qiang Xue <qiang.xue@gmail.com>
 * @link http://www.yiiframework.com/
 * @copyright 2008-2013 Yii Software LLC
 * @license http://www.yiiframework.com/license/
 */

/**
 * Utf8Command will help you to make sure files are encoded properly.
 *
 * @author Qiang Xue <qiang.xue@gmail.com>
 * @package system.build
 * @since 1.1.11
 */
class Utf8Command extends CConsoleCommand
{
	public function getHelp()

This file has been truncated. show original

Da_Sourcerer · March 13, 2013, 3:45pm

Whoa, dude … find has got this nifty -exec param. At least you could’ve used xargs

rhomb · March 29, 2013, 11:54am

@DASourcerer was really qnd and late at night…

@Samdark… wow didn’t know that, thanks for the hint