Converting a UTF-16 CSV to UTF-8 in PHP

The best solution I’ve found to this is to open the file, and re-write it with the new encoding before opening it for CSV reading;

function get_encoding($filename){
    $handle = fopen($filename, 'r');
    $bom = fread($handle, 2);
    //	fclose($handle);

    if($bom === chr(0xff).chr(0xfe)  || $bom === chr(0xfe).chr(0xff)){
        // UTF16 Byte Order Mark present
        $encoding = 'UTF-16';
    } else {
        $file_sample = fread($handle, 1000) + 'e'; //read first 1000 bytes
        // + e is a workaround for mb_string bug

        $encoding = mb_detect_encoding($file_sample , 'UTF-8, UTF-7, ASCII, EUC-JP,SJIS, eucJP-win, SJIS-win, JIS, ISO-2022-JP');
    if ($encoding){
        stream_filter_append($handle, 'convert.iconv.'.$encoding.'/UTF-8');
    return $encoding;

* Decode UTF-16 encoded strings.
* Can handle both BOM'ed data and un-BOM'ed data. 
* Assumes Big-Endian byte order if no BOM is available.
* @param   string  $str  UTF-16 encoded data to decode.
* @return  string  UTF-8 / ISO encoded data.
* @access  public
* @version 0.1 / 2005-01-19
* @author  Rasmus Andersson {@link}
* @package Groupies
function utf16_decode($str, &$be=null) {
    if (strlen($str) < 2) {
        return $str;
    $c0 = ord($str{0});
    $c1 = ord($str{1});
    $start = 0;
    if ($c0 == 0xFE && $c1 == 0xFF) {
        $be = true;
        $start = 2;
    } else if ($c0 == 0xFF && $c1 == 0xFE) {
        $start = 2;
        $be = false;
    if ($be === null) {
        $be = true;
    $len = strlen($str);
    $newstr = '';
    for ($i = $start; $i < $len; $i += 2) {
        if ($be) {
            $val = ord($str{$i})   << 4;
            $val += ord($str{$i+1});
        } else {
            $val = ord($str{$i+1}) << 4;
            $val += ord($str{$i});
        $newstr .= ($val == 0x228) ? "\n" : chr($val);
    return $newstr;

$newFilename = 'path/to/uploaded/file.scv';
$encoding = get_encoding($newFilename);

/* This is the fix for UTF-16 files which come in from iPad */
if($encoding == 'UTF-16'){
    $utf8Contents = utf16_decode(file_get_contents($newFilename));
    $resource = fopen($newFilename, 'w');
    /* Write the file over the uploaded one */
    fwrite($resource, pack("CCC",0xef,0xbb,0xbf));
    fwrite($resource, $utf8Contents);

if (($handle = fopen($newFilename, "r")) !== false) {
    while (($data = fgetcsv($handle)) !== false) {
Magento 1

Alphabetic Option Value Ordering in Magento’s Layered Navigation

Magento will by default order layered navigation options by position. To use alphabetic values first and foremost, do the following:

Override the Eav_Entity_Attribute_Source_Table model



Override the getAllOptions method as follows:

class Namespace_Module_Model_Eav_Entity_Attribute_Source_Table extends Mage_Eav_Model_Entity_Attribute_Source_Table
    public function getAllOptions($withEmpty = true, $defaultValues = false)
        $storeId = $this->getAttribute()->getStoreId();
        if (!is_array($this->_options)) {
            $this->_options = array();
        if (!is_array($this->_optionsDefault)) {
            $this->_optionsDefault = array();
        if (!isset($this->_options[$storeId])) {
            $collection = Mage::getResourceModel('eav/entity_attribute_option_collection')

            $collection->getSelect()->order(['main_table.sort_order asc', 'value asc']);

            $this->_options[$storeId]        = $collection->toOptionArray();
            $this->_optionsDefault[$storeId] = $collection->toOptionArray('default_value');

        $options = ($defaultValues ? $this->_optionsDefault[$storeId] : $this->_options[$storeId]);
        if ($withEmpty) {
            array_unshift($options, array('label' => '', 'value' => ''));

        return $options;

This uses sort order first, then uses value to order the attribute options.

Magento 1

Retrieving a Collection of Paid For Orders in Magento 1

This appears to be the best way to add a clause to an order collection to constrain it paid for orders only:

$orders = Mage::getModel("sales/order")->getCollection();
$orders->getSelect()->where('main_table.`base_grand_total` - main_table.`base_total_paid` <= 0');
Magento 1

Magento – Programatic Quote, Order, Invoices

require_once 'app/Mage.php';
$store = Mage::app()->getStore('default');
$customer = Mage::getModel('customer/customer');
$customer->loadByEmail('[email protected]');
$quote = Mage::getModel('sales/quote');
$product1 = Mage::getModel('catalog/product')->load(166); /* HTC Touch Diamond */
$buyInfo1 = array('qty' => 1);
$product2 = Mage::getModel('catalog/product')->load(18); /* Sony Ericsson W810i */
$buyInfo2 = array('qty' => 3);
$quote->addProduct($product1, new Varien_Object($buyInfo1));
$quote->addProduct($product2, new Varien_Object($buyInfo2));
$billingAddress = $quote->getBillingAddress()->addData($customer->getPrimaryBillingAddress());
$shippingAddress = $quote->getShippingAddress()->addData($customer->getPrimaryShippingAddress());
$quote->getPayment()->importData(array('method' => 'checkmo'));
$service = Mage::getModel('sales/service_quote', $quote);
$order = $service->getOrder();
$invoice = Mage::getModel('sales/service_order', $order)->prepareInvoice();
$transaction = Mage::getModel('core/resource_transaction')
  • The $invoice->register() calls pay() in the Mage_Sales_Model_Order_Invoice class. This fires the event sales_order_invoice_pay which is useful for hooking into after an invoice has been paid for; E.g. automatically sending the invoice email upon creation:
   public function sendEmail(Varien_Event_Observer $observer)
        $event = $observer->getEvent();
        /** @var Mage_Sales_Model_Order_Invoice $invoice */
        $invoice = $event->getInvoice();
Magento 1

Loading a Product Collection’s media_gallery attribute

This is useful for loading a product’s gallery on the something like the listing page.

Magento 1

Fix Logging In on Old Magento 1 Installs

On certain Magento instance installs, a valid customer login will seem to fail and redirect to the login page. We’ve seen this on plenty of Magento 1.6s thus far, however newer versions may also be affected.

The problem lies in the login method of the Mage_Customer_Model_Session class. The fix, in this instance is to remove the renewSession call within that method:

 public function login($username, $password)
    /** @var $customer Mage_Customer_Model_Customer */
    $customer = Mage::getModel('customer/customer')

    if ($customer->authenticate($username, $password)) {
        // This breaks certain setups
        return true;
    return false;

public function setCustomerAsLoggedIn($customer)
	// This breaks certain setups
//        $this->renewSession();
	Mage::dispatchEvent('customer_login', array('customer'=>$customer));
	return $this;

This has been rolled into a module here:

Magento 1

Making Pages HTTPS Only with XML

This can also be achieved programatically inside a controller, however a more elegant solution is to add the following to a module’s config XML.

Magento 1

Programatically Modifying Products’ Stock Item Objects

Recently we had a project to make all products within a number of different categories available as backorder products. The client did not want to go through and change all of the products individually, so we embarked on attempting to do this programatically.

Our first idea was to do this using observers, so hypothetically whenever a product or product collection loaded we would be able to modify the accompanying stock item object if the product was in one of the categories in question.

The observers we identified to perform this were:

We would use the above event to detect whether the product was in a particular category and set isSalable on the product object to true if so. This would make the product appear purchasable from the listing and detail pages instead of displaying the Out of Stock message.

We used the above three events to modify the following values:

    ->setData('backorders', Mage_CatalogInventory_Model_Stock::BACKORDERS_YES_NONOTIFY)
    ->setData('is_in_stock', true);

This method worked to a certain extent, however the process fell down when on the One Page Checkout; the Mage_CatalogInventory_Model_Stock object’s registerProductsSale creates a stock item object and populates it manually without firing any observers to hook into. This means that the final check in this method will always fail, and the user will be denied purchasing the product in question.

An attempted workaround was to override the Mage_CatalogInventory_Model_Stock class’ registerProductsSale method and perform our category check there. This, however caused another issue in that after registering the sale, the stock object gets saved in order to decrement the stock available; this also saves the values we’ve set meaning that if the client ever wanted to disable this functionality they would be left with any products which had been ordered in a backorder state.

Attempt 2 – Overriding

Against our better judgement, we opted instead to override the *Mage_CatalogInventory_Model_Stock_Item** class rather than changing its values with observers; this got around the issue of the stock item being saved and the values within it being crystallised. Here’s the class in question:


class Namespace_Module_Model_CatalogInventory_Stock_Item extends Mage_CatalogInventory_Model_Stock_Item
    public function getIsInStock()
            return true;

        return parent::getIsInStock(); 

    public function getBackorders()
            return Mage_CatalogInventory_Model_Stock::BACKORDERS_YES_NONOTIFY;

        return parent::getBackorders();

    public function isPurchaseableViaCategory()
        $product = $this->getProduct();
        if(!$product && $this->getProductId()){
            $product = Mage::getModel('catalog/product')->load($this->getProductId());

            return false;

        return Mage::helper('module')->isPurchaseableViaCategory($product);

And in our helper, we do the work of checking whether the product is in one of our selected categories (or a subcategory):

class Namespace_Module_Helper_Data extends Mage_Core_Helper_Abstract{
    protected $_allowBackOrderCategories = array("gold");

    public function isPurchaseableViaCategory(Mage_Catalog_Model_Product $product){
        $keys = array();

        $cats = $product->getCategoryCollection()

        foreach($cats as $cat){
            $this->getParentUrlKeys($cat, $keys);

        foreach($this->_allowBackOrderCategories as $cat){
            if(in_array($cat, $keys)){
                return true;

    public function getParentUrlKeys(Mage_Catalog_Model_Category $cat, &$urlKeys) {
        if(!in_array($cat->getUrlKey(), $urlKeys)) {
            $urlKeys[] = $cat->getUrlKey();

            return $this->getParentUrlKeys($cat->getParentCategory(), $urlKeys);

We were still able to complete our saleable check (for the display on the product listing and detail page) using an observer:

public function checkIsSaleable($observer) {
    $product = $observer->getProduct();



For completeness, we added a property to the product object to be able to identify whether this product exists in a category and should have been listed as out of stock. We can then identify these products and display a different message (Such as delivery times) on the listing and detail pages. Here’s where it’s used in our Helper:

public function isOutOfStockButPurchasable(Mage_Catalog_Model_Product $product){
        return false;

    $salableCheckOne = $product->isSalable();

    $salableCheckTwo = $product->isSalable();

    return $salableCheckOne !== $salableCheckTwo;

Our modified products will then be identifiable because the value from the two checks will not match.


The only problem from this method is that if the client saves one of the out of stock products within one of these categories, then the values for backorder and is_in_stock will be saved to the database, meaning that if the code is disabled those saved products will still be available via backorder.

Magento 1

Making a Payment Method Visible For an IP Address

Use the isAvailable method for the payment method in question, E.g. for Mage_Payment_Model_Method_Checkmo:

public function isAvailable($quote = null)
   return $_SERVER['REMOTE_ADDR'] == 'IP.ADDRESS';
Magento 1

Fixing Issues With Magento’s Massaction PDF Tax Values

There appears to be an issue in Magento’s Massaction Invoice PDF. The issue presents itself when outputting the total tax for the invoice, which will be completely different to the amount displayed when the PDF is created from the view invoice page.

The issue exists in the Mage_Tax_Helper_Data class’ getCalculatedTaxes method. The first few lines decide what the variable $current will be;

if ($this->_getFromRegistry('current_invoice')) {
    $current = $this->_getFromRegistry('current_invoice');
} elseif ($this->_getFromRegistry('current_creditmemo')) {
    $current = $this->_getFromRegistry('current_creditmemo');
} else {
    $current = $source;

In the single version of PDF creation, the current_invoice registry key is set and is used going forward. This is omitted from the massaction update, so the method falls back to the $source parameter, which happens to be an instance of an order object.

The fix is to override Mage_Sales_Model_Order_Pdf_Invoice and create a new getPdf method, with just a two line addition inside of the foreach:

public function getPdf($invoices = array())

    $pdf = new Zend_Pdf();
    $style = new Zend_Pdf_Style();
    $this->_setFontBold($style, 10);

    foreach ($invoices as $invoice) {
        // Set the invoice here, as not doing this (E.g. default Magento) causes the wrong value to be calculated
        // when using the massaction pdf action. Unregister first, otherwise only the first invoice will be set
        Mage::register('current_invoice', $invoice);
        if ($invoice->getStoreId()) {
        $page  = $this->newPage();
        $order = $invoice->getOrder();
        /* Add image */
        $this->insertLogo($page, $invoice->getStore());
        /* Add address */
        $this->insertAddress($page, $invoice->getStore());
        /* Add head */
            Mage::getStoreConfigFlag(self::XML_PATH_SALES_PDF_INVOICE_PUT_ORDER_ID, $order->getStoreId())
        /* Add document text and number */
            Mage::helper('sales')->__('Invoice # ') . $invoice->getIncrementId()
        /* Add table */
        /* Add body */
        foreach ($invoice->getAllItems() as $item){
            if ($item->getOrderItem()->getParentItem()) {
            /* Draw item */
            $this->_drawItem($item, $page, $order);
            $page = end($pdf->pages);
        /* Add totals */
        $this->insertTotals($page, $invoice);
        if ($invoice->getStoreId()) {
    return $pdf;

And massaction PDFs will once again have the correct tax amount.