Latest Machine Learning Publications
-
Time and frequency based sparse bounded component analysis algorithms for convolutive mixtures
Signal Processing
2020
Abstract
In this paper, we introduce time-domain and frequency-domain versions of a new Blind Source Separation (BSS) approach to extract bounded magnitude sparse sources from convolutive mixtures. We derive algorithms by maximization of the proposed objective functions that are defined in a completely deterministic framework, and prove that global maximums of the objective functions yield perfect separation under suitable conditions. The derived algorithms can be applied to temporal or spatially dependent sources as well as ...
View details for https://www.sciencedirect.com/science/article/pii/S016516842030133X
-
End-to-End Rate-Distortion Optimization for Bi-Directional Learned Video Compression
IEEE Int. Conf. on Image Processing
2020
Abstract
Conventional video compression methods employ a linear transform and block motion model, and the steps of motion estimation, mode and quantization parameter selection, and entropy coding are optimized individually due to combinatorial nature of the end-to-end optimization problem. Learned video compression allows end-to-end rate-distortion optimized training of all nonlinear modules, quantization parameter and entropy model simultaneously. While previous work on learned video compression considered training a ...
View details for https://arxiv.org/abs/2008.05028
-
Can learned frame prediction compete with block motion compensation for video coding?
Signal, Image and Video Processing
2020
Abstract
Given recent advances in learned video prediction, we investigate whether a simple video codec using a pretrained deep model for next frame prediction based on previously encoded/decoded frames without sending any motion side information can compete with standard video codecs based on block motion compensation. Frame differences given learned frame predictions are encoded by a standard still-image (intra) codec. Experimental results show that the rate distortion performance of the simple codec with symmetric ...
View details for https://link.springer.com/article/10.1007/s11760-020-01751-y
-
Blind Bounded Source Separation Using Neural Networks with Local Learning Rules
ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing
2020
Abstract
An important problem encountered by both natural and engineered signal processing systems is blind source separation. In many instances of the problem, the sources are bounded by their nature and known to be so, even though the particular bound may not be known. To separate such bounded sources from their mixtures, we propose a new optimization problem, Bounded Similarity Matching (BSM). A principled derivation of an adaptive BSM algorithm leads to a recurrent neural network with a clipping nonlinearity. The ...
View details for https://ieeexplore.ieee.org/abstract/document/9053114/
-
Online Bounded Component Analysis: A Simple Recurrent Neural Network with Local Update Rule for Unsupervised Separation of Dependent and Independent …
53rd Asilomar Conference on Signals, Systems, and Computers
2019
Abstract
A low complexity recurrent neural network structure is proposed for unsupervised separation of both independent and dependent sources from their linear mixtures. The proposed network is generated based on Bounded Component Analysis (BCA) approach. We first propose an Online-BCA optimization setting. Then we derive the corresponding recurrent neural network (RNN) with iterative learning update expressions. The resulting 2-layer network has a fairly simple structure with feedforward synapses at the input layer, recurrent ...
View details for https://ieeexplore.ieee.org/abstract/document/9048916/
-
Ntire 2019 challenge on video super-resolution: Methods and results
IEEE CVPR Workshop
2019
Abstract
This paper reviews the first NTIRE challenge on video super-resolution (restoration of rich details in low-resolution video frames) with focus on proposed solutions and results. A new REalistic and Diverse Scenes dataset (REDS) was employed. The challenge was divided into 2 tracks. Track 1 employed standard bicubic downscaling setup while Track 2 had realistic dynamic motion blurs. Each competition had 124 and 104 registered participants. There were total 14 teams in the final testing phase. They gauge the state-of-the-art in video ...
View details for http://openaccess.thecvf.com/content_CVPRW_2019/html/NTIRE/Nah_NTIRE_2019_Challenge_on_Video_Super-Resolution_Methods_and_Results_CVPRW_2019_paper.html
-
Effect of Architectures and Training Methods on the Performance of Learned Video Frame Prediction
IEEE Int. Conf. Image Proc. (ICIP)
2019
Abstract
We analyze the performance of feedforward vs. recurrent neural network (RNN) architectures and associated training methods for learned frame prediction. To this effect, we trained a residual fully convolutional neural network (FCNN), a convolutional RNN (CRNN), and a convolutional long short-term memory (CLSTM) network for next frame prediction using the mean square loss. We performed both stateless and stateful training for recurrent networks. Experimental results show that the residual FCNN architecture performs the best ...
View details for https://ieeexplore.ieee.org/abstract/document/8803624/
-
Compressed training based massive mimo
IEEE Transactions on Signal Processing
2019
Abstract
Massive multiple-input-multiple-output (MIMO) scheme promises high spectral efficiency through the employment of large scale antenna arrays in base stations. In time division duplexed implementations, co-channel mobile terminals transmit training information such that base stations can estimate and exploit channel state information to spatially multiplex these users. In the conventional approach, the optimal choice for training length was shown to be equal to the number of users, K. In this paper, we propose a new semiblind framework ...
View details for https://ieeexplore.ieee.org/abstract/document/8605530/
-
Channel Estimation for Massive MIMO: A Semiblind Algorithm Exploiting QAM Structure
53rd Asilomar Conference on Signals, Systems, and Computers
2019
Abstract
We introduce a new channel matrix estimation algorithm for Massive MIMO systems to reduce the required pilot symbols. The proposed method is based on Maximum A Posteriori estimation where the density of QAM transmission symbols are approximated with continuous uniform pdf. Under this simplification, joint channel source estimation problem can be posed as an optimization problem whose objective is quadratic in each channel and source symbol matrices, separately. Also, the source symbols are constrained to lie in an ℓ∞ ...
View details for https://ieeexplore.ieee.org/abstract/document/9048774/
-
Sparse bounded component analysis for convolutive mixtures
IEEE International Conference on Acoustics, Speech, and Signal Processing
2018
Abstract
In this article, we propose a Bounded Component Analysis (BCA) approach for the separation of the convolutive mixtures of sparse sources. The corresponding algorithm is derived from a geometric objective function defined over a completely deterministic setting. Therefore, it is applicable to sources which can be independent or dependent in both space and time dimensions. We show that all global optima of the proposed objective are perfect separators. We also provide numerical examples to illustrate the performance of the ...
View details for https://ieeexplore.ieee.org/abstract/document/8462568/
-
Learned Compression Artifact Removal by Deep Residual Networks.
CVPR Workshops
2018
Abstract
We propose a method for learned compression artifact removal by post-processing of BPG compressed images. We trained three networks of different sizes. We encoded input images using BPG with different QP values. We submitted the best combination of test images, encoded with different QP and post-processed by one of three networks, which satisfy the file size and decode time constraints imposed by the Challenge. The selection of the best combination is posed as an integer programming problem. Although the visual ...
View details for http://openaccess.thecvf.com/content_cvpr_2018_workshops/papers/w50/Kirmemis_Learned_Compression_Artifact_CVPR_2018_paper.pdf
-
Effect of Training and Test Datasets on Image Restoration and Super-Resolution by Deep Learning
EUSIPCO
2018
Abstract
Many papers have recently been published on image restoration and single-image super-resolution (SISR) using different deep neural network architectures, training methodology, and datasets. The standard approach for performance evaluation in these papers is to provide a single “average” mean-square error (MSE) and/or structural similarity index (SSIM) value over a test dataset. Since deep learning is data-driven, performance of the proposed methods depends on the size of the training and test sets as well as the variety and ...
View details for https://ieeexplore.ieee.org/abstract/document/8552961/
-
An algorithmic framework for sparse bounded component analysis
IEEE Transactions on Signal Processing
2018
Abstract
Bounded component analysis (BCA) is a recent approach that enables the separation of both dependent and independent signals from their mixtures. This paper introduces a novel deterministic instantaneous BCA framework for the separation of sparse bounded sources. The framework is based on a geometric maximization setting, where the objective function is defined as the volume ratio of two objects, namely, the principal hyperellipsoid and the bounding ℓ 1-norm ball, defined over the separator output samples. It is shown that all global ...
View details for https://ieeexplore.ieee.org/abstract/document/8443153/
-
An algorithmic framework for sparse bounded component analysis
IEEE Transactions on Signal Processing
2018
Abstract
Bounded component analysis (BCA) is a recent approach that enables the separation of both dependent and independent signals from their mixtures. This paper introduces a novel deterministic instantaneous BCA framework for the separation of sparse bounded sources ...
View details for https://ieeexplore.ieee.org/abstract/document/8443153/
-
Fast Multidimensional Reduction and Broadcast Operations on {GPU} for Machine Learning
Concurrency and Computation: Practice and Experience
2018
Abstract
Reduction and broadcast operations are commonly used in machine learning algorithms for different purposes. They widely appear in the calculation of the gradient values of a loss function, which are one of the core structures of neural networks. Both operations are implemented naively in many libraries usually for scalar reduction or broadcast; however, to our knowledge, there are no optimized multidimensional implementations available. This fact limits the performance of machine learning models requiring these operations to be performed on tensors. In this work, we address the problem and propose two new strategies that extend the existing implementations to perform on tensors. We introduce formal definitions of both operations using tensor notations, investigate their mathematical properties, and exploit these properties to provide an efficient solution for each. We implement our parallel strategies and test them on a CUDA enabled Tesla K40 m GPU accelerator. Our performant implementations achieve up to 75% of the peak device memory bandwidth on different tensor sizes and dimensions. Significant speedups against the implementations available in the Knet Deep Learning framework are also achieved for both operations.
View details for https://onlinelibrary.wiley.com/doi/abs/10.1002/cpe.4691
-
On Machine Learning and Programming Languages
SysML Conference
2018
Abstract
The complexity of Machine Learning (ML) models and the frameworks people are using to build them has exploded along with ML itself. State-of-the-art models are increasingly programs, with support for programming constructs like loops and recursion, and this brings out many interesting issues in the tools we use to create them — that is, programming languages (PL). This paper1 , discusses the necessity for a first class language for machine learning, and what such a language might look like.
View details for http://www.sysml.cc/
-
Multidimensional Broadcast Operation on the GPU
2017
Abstract
Broadcast is a common operation in machine learn- ing and widely used in calculating bias or subtracting maximum for normalization in convolutional neural networks. Broadcast operation is required when two tensors possibly with different number of dimensions, hence with different number of elements, are input to an element-wise function. Tensors are scaled in process so that the two tensors match in size and dimension. In this research, we introduce a new broadcast functionality for matrices to be used on CUDA enabled GPU devices. We further extend this operation to multidimensional arrays and measure its performance against the implementation available in the Knet deep learning framework. Our final implementation provides up to 2x improvement over the Knet broadcast implementation, which only supports vector broadcast. Our implementation can handle broadcast operations with any number of dimensions. Index Terms—GPU, CUDA, machine learning, broadcast, mul- tidimensional arrays
View details for https://drive.google.com/file/d/0B5_wxKFQbOTMbXVzTWhsbGFrZnM/view?usp=sharing
-
Stationary point characterization for a class of bca algorithms
IEEE Transactions on Signal Processing
2017
Abstract
Bounded component analysis (BCA) is a recently introduced approach including independent component analysis as a special case under the assumption of source boundedness. In this paper, we provide a stationary point analysis for the recently proposed instantaneous BCA algorithms that are capable of separating dependent, even correlated as well as independent sources from their mixtures. The stationary points are identified and characterized as either perfect separators, which are the global maxima of the proposed ...
View details for https://ieeexplore.ieee.org/abstract/document/7990159/
-
Single carrier frequency domain compressed training adaptive equalization
IEEE Asilomar Conference on Signals, Systems and Computers
2017
Abstract
Compressed training approach offers to improve the spectral efficiency by significantly reducing the required training length. It is based on a convex optimization setting (SCFDE-CoTA) which combines two cost functions: 1) Least squares based training reconstruction performance, and 2) Infinity norm of equalizer outputs, which exploits magnitude boundedness assumption of digital communication symbols. In this article, we provide an extension of this framework to single carrier frequency domain equalizer based transceivers ...
View details for https://ieeexplore.ieee.org/abstract/document/8335522/
-
DOA estimation of low altitude target based on bounded component analysis algorithm
2017
Abstract
A recently introduced Blind Source Separation method, called Bounded Component Analysis, is used as preliminary technique to isolate direct path radar wave from ground reflected waves in order to overcome the multipath effect. This method enables the radar to estimate the target angle without any a priori knowledge of the operation environment. The numerical experiments illustrate the potential benefit of the proposed approach relative to classical maximum likelihood method (CMLM) based on free space propagation model. ...
View details for https://digital-library.theiet.org/content/conferences/10.1049/cp.2017.0416
-
Compressed training adaptive equalization: Algorithms and analysis
IEEE Asilomar Conference on Signals, Systems and Computers
2017
Abstract
We propose “compressed training adaptive equalization” as a novel framework to reduce the quantity of training symbols in a communication packet. It is a semi-blind approach for communication systems employing time-domain/frequency-domain equalizers, and founded upon the idea of exploiting the magnitude boundedness of digital communication symbols. The corresponding algorithms are derived by combining the least-squares-cost-function measuring the training symbol reconstruction performance and the infinity-norm of the ...
View details for https://ieeexplore.ieee.org/abstract/document/7937918/
-
Compressed training adaptive equalization: Algorithms and analysis
IEEE Trans. on Communications
2017
Abstract
We propose “compressed training adaptive equalization” as a novel framework to reduce the quantity of training symbols in a communication packet. It is a semi-blind approach for communication systems employing time-domain/frequency-domain equalizers, and founded ...
View details for https://ieeexplore.ieee.org/abstract/document/7937918/
-
Compressed training adaptive MIMO equalization
IEEE Signal Processing Advances in Wireless Communications (SPAWC 2016)
2016
Abstract
This article proposes an adaptive equalization framework for flat fading multi-input multi-output (MIMO) systems, where the main goal is to significantly reduce the number of training symbols. The proposed approach exploits the special boundedness property of digital ...
View details for https://ieeexplore.ieee.org/abstract/document/7536758/
-
Knet: beginning deep learning with 100 lines of Julia
Machine Learning Systems Workshop at NIPS 2016
2016
Abstract
Knet (pronounced "kay-net") is the Koç University machine learning framework implemented in Julia, a high-level, high-performance, dynamic programming language. Unlike gradient generating compilers like Theano and TensorFlow which restrict users into a modeling mini-language, Knet allows models to be defined by just describing their forward computation in plain Julia, allowing the use of loops, conditionals, recursion, closures, tuples, dictionaries, array indexing, concatenation and other high level language features. High performance is achieved by combining automatic differentiation of most of Julia with efficient GPU kernels and memory management. Several examples and benchmarks are provided to demonstrate that GPU support and automatic differentiation of a high level language are sufficient for concise definition and efficient training of sophisticated models.
View details for https://goo.gl/KeOEoJ