🥇Magento 2: Importing products from external sources

Magento is an e-commerce solution, meaning it's more focused on selling products than on the associated warehouse, logistics, or financial accounting. Other applications (such as ERP systems) are better suited for these tasks. Therefore, it's quite often used in practice. Magento the task of integrating the store with these other systems (for example, with 1C) arises.

By and large, integration can be reduced to data replication by:

catalog (products, categories);
inventory data (remains of products in warehouses and prices);
clients;
orders;

Magento for manipulating data in the database, a separate class of objects is offered - repositoriesDue to the specifics Magento Adding data to a database via repositories is easy to code, but it's, shall we say, slow. In this post, I'll cover the main steps for programmatically adding data to Magento 2 products in the "classic" way - using repo classes.

Clients and orders are usually replicated in the other direction - from Magento into external ERP systems. Therefore, it's easier with them, on the side Magento you just need to select the relevant data, and then - "bullets flew from our side«.

Principles of writing data to the database

At the moment, creating objects stored in the database programmatically is Magento is done through office:

function __construct (MagentoCmsModelBlockFactory $blockFactory) {
    $this->blockFactory = $blockFactory;
}

/** @var MagentoCmsModelBlock $block */
$block = $this->blockFactory->create();

and the record in the database - through Repository:

function __construct (MagentoCmsApiBlockRepositoryInterface $blockRepo) {
    $this->blockRepo = $blockRepo;
}

$this->blockRepo->save($block);

The Factory and Repository approach can be used for all major models in the domain. Magento 2.

Basic product information

I am looking at the data structure corresponding to the version Magento 2.3. The most basic information about the product is in the table catalog_product_entity (product register):

entity_id
attribute_set_id
type_id
sku
has_options
required_options
created_at
updated_at

Limited to one type of product (type_id='simple'), a set of default attributes (attribute_set_id=4) and ignore the attributes has_options и required_options. Since the attributes entity_id, created_at и updated_at are generated automatically, then, in fact, to add a new product, it is enough for us to set sku. I do it like this:

/** @var MagentoCatalogApiDataProductInterfaceFactory $factProd */
/** @var MagentoCatalogApiProductRepositoryInterface $repoProd */
/** @var MagentoCatalogApiDataProductInterface $prod */
$prod = $factProd->create();
$prod->setAttributeSetId(4);
$prod->setTypeId('simple');
$prod->setSku($sku);
$repoProd->save($prod);

and I get an exception:

The "Product Name" attribute value is empty. Set the attribute and try again.

I add the product name to the request and get a message that the attribute is missing Price. After adding the price, the product is added to the database:

$prod = $factProd->create();
$prod->setAttributeSetId(4);
$prod->setTypeId('simple');
$prod->setSku($sku);
$prod->setName($name);
$prod->setPrice($price);
$repoProd->save($prod);

The product name is stored in the product's varchar attribute table (catalog_product_entity_varchar), price - in the table catalog_product_entity_decimal. Before adding a product, it is advisable to explicitly indicate that we are using an administrative storefront to import data:

/** @var MagentoStoreModelStoreManagerInterface $manStore */
$manStore->setCurrentStore(0);

Additional Attributes

Processing additional product attributes using tools Magento — a pleasure. EAV data model for core entities (see table eav_entity_type) is one of the key features of this platform. We simply add the appropriate attributes to the product model:

$prodEntity->setData('description', $desc);
$prodEntity->setData('short_description', $desc_short);
// или
$prodEntity->setDescription($desc);
$prodEntity->setShortDescription($desc_short);

and when saving the model via repo object:

$repoProd->save($prod);

additional attributes will also be stored in the corresponding database tables.

inventory data

In simple terms, it is the quantity of product in stock. Magento 2.3 structures in the database describing the storage format of inventory data, significantly different from what was before. However, adding the quantity of a product in stock via the product model is not much more difficult than adding other attributes:

/** @var MagentoCatalogModelProduct $prodEntity */
/** @var MagentoCatalogApiProductRepositoryInterface $repoProd */
$inventory = [
    'is_in_stock' => true,
    'qty' => 1234
];
$prodEntity->setData('quantity_and_stock_status', $inventory);
$repoProd->save($prodEntity);

Media

As a rule, the media accompaniment of a product for a customer in an e-commerce store is different from the media accompaniment of the same product for an employee in an internal accounting system (ERP). In the first case, it is desirable to show the “product face”, in the second case, it is enough to give a general idea of the product. However, transferring at least the primary product image is quite common. case when importing data.

When adding an image through the admin panel, the image is first saved in a temporary directory (./pub/media/tmp/catalog/product) and only when the product is saved is moved to the media directory (./pub/media/catalog/product). Also, when adding through the admin panel, the image is tagged image, small_image, thumbnail, swatch_image.

/** @var MagentoCatalogApiProductRepositoryInterface $repoProd */
/** @var MagentoCatalogModelProductGalleryCreateHandler $hndlGalleryCreate */
/* $imagePath = '/path/to/file.png';  $imagePathRelative = '/f/i/file.png' */
$imagePathRelative = $this->imagePlaceToTmpMedia($imagePath);
/* reload product with gallery data */
$product = $repoProd->get($sku);
/* add image to product's gallery */
$gallery['images'][] = [
    'file' => $imagePathRelative,
    'media_type' => 'image'
    'label' => ''
];
$product->setData('media_gallery', $gallery);
/* set usage areas */
$product->setData('image', $imagePathRelative);
$product->setData('small_image', $imagePathRelative);
$product->setData('thumbnail', $imagePathRelative);
$product->setData('swatch_image', $imagePathRelative);
/* create product's gallery */
$hndlGalleryCreate->execute($product);

For some reason, the media is tied up only after the preliminary saving of the product and getting it from the repository again. And you need to specify the attribute label when adding an entry to the media gallery of the product (otherwise we get an exception Undefined index: label in .../module-catalog/Model/Product/Gallery/CreateHandler.php on line 516).

Summary

Write code to add in Magento 2 products are very easy to program. I've compiled everything described above into a demo module.flancer32/mage2_ext_demo_import". There is only one console command in the module fl32:import:prod, which imports the products described in the JSON file "./etc/data/products.json«:

[
  {
    "sku": "...",
    "name": "...",
    "desc": "...",
    "desc_short": "...",
    "price": ...,
    "qty": ...,
    "categories": ["..."],
    "image_path": "..."
  }
]

Pictures for import are in the catalog ./etc/data/img.

The time to import 10 products in this way is about 10 seconds on my laptop. If we develop this idea further, it is easy to come to the conclusion that about 3600 products can be imported per hour, and it can take about 100 hours to import 30K products. Replacing a laptop with a server allows you to somewhat smooth the situation. Maybe even at times. But not in order. Perhaps this speed, slowness, to some extent, is one of the reasons for the emergence of the project magento/async-import.

A radical solution for increasing import speed could be direct writing to the database, but in this case all the benefits related to extensibility are lost Magento — I'll have to do all the "advanced" stuff myself. Still, it's worth it. If it works out, I'll look at the direct database write approach in the next article.

Source: habr.com

Magento 2: Importing products from external sources