ProHoster > Blog > Administration > Magento 2: importing products from external sources

Magento 2: importing products from external sources

Magento is an e-commerce solution i.e. is more focused on selling products than on related inventory, logistics, or financial accounting. Companion is better suited for other applications (eg ERP systems). Therefore, quite often in the practice of using Magento, the task of integrating the store with these other systems (for example, with 1C) arises.

By and large, integration can be reduced to data replication by:

  • catalog (products, categories);
  • inventory data (remains of products in warehouses and prices);
  • clients;
  • orders;

Magento for manipulating data in the database offers a separate class of objects - repositories. Due to the specifics of Magento, adding data to the database through repositories is easily coded, but it does not happen quickly, let's say. In this post, I'm going over the main stages of programmatically adding a product to Magento 2 in the "classic" way - using repo classes.

Customers and orders are usually replicated in the other direction - from Magento to external ERP systems. Therefore, it’s easier with them, on the Magento side, you just need to select the appropriate data, and then β€” β€œbullets flew from our sideΒ«.

Principles of writing data to the database

At the moment, the creation of objects stored in the database programmatically in Magento is done through office:

function __construct (MagentoCmsModelBlockFactory $blockFactory) {
    $this->blockFactory = $blockFactory;
}

/** @var MagentoCmsModelBlock $block */
$block = $this->blockFactory->create();

and the record in the database - through Repository:

function __construct (MagentoCmsApiBlockRepositoryInterface $blockRepo) {
    $this->blockRepo = $blockRepo;
}

$this->blockRepo->save($block);

The "Factory" and "Repository" approach can be used for all major models in the Magento 2 domain.

Basic product information

I'm considering a data structure corresponding to Magento 2.3 version. The most basic product information is in the table catalog_product_entity (product register):

entity_id
attribute_set_id
type_id
sku
has_options
required_options
created_at
updated_at

Limited to one type of product (type_id='simple'), a set of default attributes (attribute_set_id=4) and ignore the attributes has_options ΠΈ required_options. Since the attributes entity_id, created_at ΠΈ updated_at are generated automatically, then, in fact, to add a new product, it is enough for us to set sku. I do it like this:

/** @var MagentoCatalogApiDataProductInterfaceFactory $factProd */
/** @var MagentoCatalogApiProductRepositoryInterface $repoProd */
/** @var MagentoCatalogApiDataProductInterface $prod */
$prod = $factProd->create();
$prod->setAttributeSetId(4);
$prod->setTypeId('simple');
$prod->setSku($sku);
$repoProd->save($prod);

and I get an exception:

The "Product Name" attribute value is empty. Set the attribute and try again.

I add the product name to the request and get a message that the attribute is missing Price. After adding the price, the product is added to the database:

$prod = $factProd->create();
$prod->setAttributeSetId(4);
$prod->setTypeId('simple');
$prod->setSku($sku);
$prod->setName($name);
$prod->setPrice($price);
$repoProd->save($prod);

The product name is stored in the product's varchar attribute table (catalog_product_entity_varchar), price - in the table catalog_product_entity_decimal. Before adding a product, it is advisable to explicitly indicate that we are using an administrative storefront to import data:

/** @var MagentoStoreModelStoreManagerInterface $manStore */
$manStore->setCurrentStore(0);

Additional Attributes

Processing additional product attributes with Magento is a pleasure. EAV data model for the main entities (see table eav_entity_type) is one of the key features of this platform. We simply add the appropriate attributes to the product model:

$prodEntity->setData('description', $desc);
$prodEntity->setData('short_description', $desc_short);
// ΠΈΠ»ΠΈ
$prodEntity->setDescription($desc);
$prodEntity->setShortDescription($desc_short);

and when saving the model via repo object:

$repoProd->save($prod);

additional attributes will also be stored in the corresponding database tables.

inventory data

In a simple way - the amount of product in stock. In Magento 2.3, structures in the database that describe the format for storing inventory data are significantly different from what was before. However, adding the quantity of a product in stock via the product model is not much more difficult than adding other attributes:

/** @var MagentoCatalogModelProduct $prodEntity */
/** @var MagentoCatalogApiProductRepositoryInterface $repoProd */
$inventory = [
    'is_in_stock' => true,
    'qty' => 1234
];
$prodEntity->setData('quantity_and_stock_status', $inventory);
$repoProd->save($prodEntity);

Media

As a rule, the media accompaniment of a product for a customer in an e-commerce store is different from the media accompaniment of the same product for an employee in an internal accounting system (ERP). In the first case, it is desirable to show the β€œproduct face”, in the second case, it is enough to give a general idea of ​​the product. However, transferring at least the primary product image is quite common. case when importing data.

When adding an image through the admin panel, the image is first saved in a temporary directory (./pub/media/tmp/catalog/product) and only when the product is saved is moved to the media directory (./pub/media/catalog/product). Also, when adding through the admin panel, the image is tagged image, small_image, thumbnail, swatch_image.

/** @var MagentoCatalogApiProductRepositoryInterface $repoProd */
/** @var MagentoCatalogModelProductGalleryCreateHandler $hndlGalleryCreate */
/* $imagePath = '/path/to/file.png';  $imagePathRelative = '/f/i/file.png' */
$imagePathRelative = $this->imagePlaceToTmpMedia($imagePath);
/* reload product with gallery data */
$product = $repoProd->get($sku);
/* add image to product's gallery */
$gallery['images'][] = [
    'file' => $imagePathRelative,
    'media_type' => 'image'
    'label' => ''
];
$product->setData('media_gallery', $gallery);
/* set usage areas */
$product->setData('image', $imagePathRelative);
$product->setData('small_image', $imagePathRelative);
$product->setData('thumbnail', $imagePathRelative);
$product->setData('swatch_image', $imagePathRelative);
/* create product's gallery */
$hndlGalleryCreate->execute($product);

For some reason, the media is tied up only after the preliminary saving of the product and getting it from the repository again. And you need to specify the attribute label when adding an entry to the media gallery of the product (otherwise we get an exception Undefined index: label in .../module-catalog/Model/Product/Gallery/CreateHandler.php on line 516).

Categories

Often, the structure of the categories of the store and the backend application or the placement of products in them can differ significantly. Strategies for migrating category and product data into categories depend on many factors. In this example, I stick to the following:

  • backend and store categories are matched by name;
  • if a category is imported that is not in the store, then it is created under the root category (Default Category) and its further positioning in the store catalog is assumed manually;
  • a product is linked to a category only when it is created in the store (first import);

The main information about the category is in the table catalog_category_entity (catalog of categories). Creating a category in Magento:

/** @var MagentoCatalogApiDataCategoryInterfaceFactory $factCat */
/** @var MagentoCatalogApiCategoryRepositoryInterface $repoCat */
$cat = $factCat->create();
$cat->setName($name);
$cat->setIsActive(true);
$repoCat->save($cat);

A product is linked to a category by category ID and product SKU:

/** @var MagentoCatalogModelCategoryProductLinkFactory $factCatProdLink */
/** @var MagentoCatalogApiCategoryLinkRepositoryInterface $repoCatLink */
$link = $factCatProdLink->create();
$link->setCategoryId($catMageId);
$link->setSku($prodSku);
$repoCatLink->save($link);

Total

It is very easy to write code to add products to Magento 2 programmatically. I brought all of the above into a demo module β€œflancer32/mage2_ext_demo_import". There is only one console command in the module fl32:import:prod, which imports the products described in the JSON file "./etc/data/products.jsonΒ«:

[
  {
    "sku": "...",
    "name": "...",
    "desc": "...",
    "desc_short": "...",
    "price": ...,
    "qty": ...,
    "categories": ["..."],
    "image_path": "..."
  }
]

Pictures for import are in the catalog ./etc/data/img.

The time to import 10 products in this way is about 10 seconds on my laptop. If we develop this idea further, it is easy to come to the conclusion that about 3600 products can be imported per hour, and it can take about 100 hours to import 30K products. Replacing a laptop with a server allows you to somewhat smooth the situation. Maybe even at times. But not in order. Perhaps this speed, slowness, to some extent, is one of the reasons for the emergence of the project magento/async-import.

A cardinal solution to increase the speed of imports can be a direct entry into the database, but in this case all the "goodies" related to Magento extensibility are lost - you have to do everything "extended" yourself. However, it's worth it. If possible, I will consider the approach with direct writing to the database in the next article.

Source: habr.com

Add a comment