Modern AI is based on "Deep Learning." Why did Deep Learning originate in Ukraine (USSR) in 1965? Back then, the USSR was leading many important fields of science and technology, most notably in space: first satellite (1957), first man-made object on a heavenly body (1959), first man in space (1961), first woman in space (1962), first robot landing on a heavenly body (1965), first robot on another planet (1970). The USSR also detonated the world's biggest bomb ever (1961), and was home of many leading mathematicians, with sufficient funding for blue skies math research whose enormous significance would emerge only several decades later when compute was billions of times cheaper. Check out Ivakhnenko's 1971 survey in English (IEEE Transactions on Systems, Man and Cybernetics, (4):364-378). It describes a deep learning network with 8 layers, still considered deep in the early 2000s. Given a training set of input vectors with corresponding target output vectors, layers are incrementally grown and trained by regression analysis. In a fine-tuning phase, superfluous hidden units are pruned through regularisation with the help of a separate validation set. This simplifies the net and improves its generalization on unseen test data. The numbers of layers and units per layer are learned in problem-dependent fashion. Even the experiments were similar to today's: learn to predict the next element of a sequence, given previous elements. That's what ChatGPT does! That is, Ivakhnenko had connectionism with adaptive hidden layers two decades before the name "connectionism" became popular in the 1980s, and he had "deep learning" 4 decades before the name became popular in the 2000s. He also demonstrated that it is possible to learn appropriate weights for hidden units using only locally available information without requiring the biologically implausible backward pass of backpropagation (a technique that was published in neighbouring Finland in 1970). More in: Who invented deep learning? Technical Note IDSIA-16-25, IDSIA, Nov 2025.
