My A.I. learning Path

TENSORE (ML)

Let ${\displaystyle \mathbb{�}}$ be a field such as the real numbers ${\displaystyle \mathbb{�}}$. A tensor ${\displaystyle \mathcal{�}}$ is an ${\displaystyle {�}_0\times {�}_1\times \cdots {�}_{�}}$ array over ${\displaystyle \mathbb{�}}$:

${\displaystyle \mathcal{�}\in {\mathbb{�}}^{{�}_0\times {�}_1\times \dots \times {�}_{�}}.}$

Here, ${\displaystyle �}$ and ${\displaystyle {�}_1,{�}_2,\dots ,{�}_{�}}$ are positive integers, and ${\displaystyle �}$ is the number of dimensions.

One basic approach (not the only way) to using tensors in machine learning is to embed various data types directly. For example, a grayscale image, commonly represented as a discrete 2D function ${\displaystyle �(�,�)}$ with resolution ${\displaystyle �\times �}$ may be embedded in a mode-2 tensor as

${\displaystyle �(�,�)\mapsto \mathcal{�}\in {\mathbb{�}}^{�\times �}.}$

A color image with 3 channels for RGB might be embedded in a mode-3 tensor with three elements in an additional dimension:

${\displaystyle {�}_{���}(�,�)\mapsto \mathcal{�}\in {\mathbb{�}}^{�\times �\times 3}.}$

In natural language processing, a word might be expressed as a vector ${\displaystyle �}$ via the Word2vec algorithm. Thus ${\displaystyle �}$ becomes a mode-1 tensor

${\displaystyle �\mapsto \mathcal{�}\in {\mathbb{�}}^{�}.}$

The embedding of subject-object-verb semantics requires embedding relationships among three words. Because a word is itself a vector, subject-object-verb semantics could be expressed using mode-3 tensors

${\displaystyle {�}_{�}\times {�}_{�}\times {�}_{�}\mapsto \mathcal{�}\in {\mathbb{�}}^{�\times �\times �}.}$

Kronecker Product: 

A m × n 

B p × q

Kronecker product A ⊗ B is pm × qn