Machine learning (ML) is a branch of artificial intelligence (AI) that focuses on identifying and validating methods and algorithms to optimize a performance criterion using historical, current, and future data. It is the major driving force behind the ongoing surge in the development of intelligent systems with various successful applications in diverse areas. Therefore, it has become an essential part of many data-driven applications, in both science (biology, neuroscience, psychology, astronomy, etc.) and engineering (natural language processing, computer vision, robotics, autonomous systems etc.). However, ML is not a single approach; rather, it consists of a dazzling array of seemingly disparate frameworks and paradigms encompassing several fields from computer science, information theory, statistics to neural computation. As such, exhaustive classification of ML algorithms is an overwhelming  work.  However, using a conventional learning style perspective, ML algorithms can be easily grouped into three as: supervised learning methods (generative/discriminative learning, parametric/non-parametric learning, deep neural networks (DNNs),  support vector machines, etc.); unsupervised learning methods (clustering, dimensionality reduction, and kernel methods); and reinforcement learning with adaptive control. Our research in ML covers several of these subfields and is essentially concentrated on the development of novel and high performance learning algorithms for large-scale learning problems, and the implementation of these algorithms in various hardware and software platforms.

Development and analysis of new generation ML algorithms:

Current ML approaches based on deep learning (DL) has achieved super human performance especially in pattern recognition related tasks. However, current DNNs require substantial amount of labeled data for proper training, and the existing ML platforms are both bulky and inefficient in energy usage compared to the natural solutions, and the theory typically lags behind the practice, where several successful approaches remain unjustified. In our research, we are working on developing novel ML frameworks to address all of these concerns:

• We concentrate on the development of novel unsupervised learning, semi-supervised learning and self-supervised learning approaches to enable training with significantly small amount of data.
• We investigate the underlying structures for higher level cognition and their potential algorithmic implementations with a main focus on natural language processing.
• We focus on low complexity implementation aspects based on optimization based principled construction of recurrent neural networks with lower computational and memory requirements. We also work on special neural networks algorithms based on local learning rules that enable implementations in low power neuromorphic hardware.
• We use the tools from non-convex optimization, stochastic processes and statistical physics to investigate the workings and the performance of the existing and the proposed algorithms

Development of software platforms and resources for ML:

One of the key ingredients enablers of the current AI revolution is the use of appropriate software tools and platforms for the efficient implementation of ML algorithms utilizing advanced hardware components. Building ML software that enable fast and low complexity implementations for handling large scale data is one of the main focuses of our research interest. Knet, Koç University Deep Learning Platform, developed in our university, is the unique ML platform that is developed in Türkiye and  that can compete with its popular versions such as Tensorflow and PyTorch. We believe that it is really critical to have the full command of all layers of software implementation for further innovations and applications in ML field.

Computational neuroscience and biologically inspired learning algorithms:

Although it is debatable whether the human brain is an example of general-intelligence, it is undoubtedly still the best well known intelligent device with diverse cognitive abilities, which is a product of hundreds of millions of years of natural optimization process. We believe that joining the global efforts to model inner workings of the brain would also be fruitful for inspiring novel algorithms with better performance and lower implementational  requirements. Although a global theory for brain is still missing, and it is still mostly a black box, it is clear that  even the rough modeling of biological neural networks led to  the existing powerful DNN structures. Therefore, it is both exciting  and informative to investigate different neuron models, signaling and network structures, different resolutions of learning time scales and  relevant physical, statistical and optimization  frameworks within the scope of biological intelligence.  In the alternative direction, we target to utilize the algorithmic tools developed in ML for the analysis of neural  imaging data such  EEG, MEG, fMRI, EMG and Calcium imaging. Such efforts would be also useful for diseases of neural origin such as Alzheimer’s and Epilepsy.

Rich and explainable DL 

Explainability of DL algorithms is essential to increase the reliability and, thus, enable the deployment of ML especially in safety-critical applications. Understanding and explaining the learning behavior, providing mathematically sound uncertainty bounds, and increasing the robustness, e.g. against adversarial examples, are major milestones. At the application-front, there is an increasing excitement toward autonomous systems with learning capabilities. Several data-driven applications have shown significant practical benefit revealing the power of having access to more data, e.g., health care systems, and self-driving cars. However, large acceptance of such systems yet depends on their stability, tractability and reproducibility, of these systems where current applications fall inadequate in providing such features. The scale and complexity of modern datasets often render classical techniques infeasible, and therefore, new algorithms are required to address new technical challenges associated with the nature of the data.

Tractable learning methods for non-convex optimization

Despite remarkable empirical success of non-convex optimization frameworks in data-driven field, mainly thanks to popular DL models, theoretical understanding of them is very limited. Although the so-called ‘black-box’ approach helps many to initiate a significant interest into this emerging field, but a rigorous approach is yet missing. Hence, in order to advance the field to its next frontier, a ‘black-box’ treatment is now known to limit continuous development of these systems. Our research focuses on developing tractable DL solutions for large-scale systems. Apart from theoretical aspects, we target specific applications in subspace estimation, matrix factorization, tensor decomposition and DL. Specifically, this novel approach brings together several attractive features:

• the emerging concept of online-learning is adapted to distributed setting across a decentralized network topology using variants of stochastic gradient descent minimization,
• the exact dynamics of the algorithms is studied by a stochastic process analysis methods, which current state of the-art methods are unable to deliver,
• analyzing the extracted dynamics, the learning capabilities and performances of large-scale systems is aimed to match the current needs of modern applications.

Affiliated Faculty:

• Alper Erdoğan
• Deniz Yuret
• Metin Sezgin
• Zafer Doğan