In this paper, we introduce time-domain and frequency-domain versions of a new Blind Source Separation (BSS) approach to extract bounded magnitude sparse sources from convolutive mixtures. We derive algorithms by maximization of the proposed objective functions that are defined in a completely deterministic framework, and prove that global maximums of the objective functions yield perfect separation under suitable conditions. The derived algorithms can be applied to temporal or spatially dependent sources as well as ...
View details for https://www.sciencedirect.com/science/article/pii/S016516842030133X
Conventional video compression methods employ a linear transform and block motion model, and the steps of motion estimation, mode and quantization parameter selection, and entropy coding are optimized individually due to combinatorial nature of the end-to-end optimization problem. Learned video compression allows end-to-end rate-distortion optimized training of all nonlinear modules, quantization parameter and entropy model simultaneously. While previous work on learned video compression considered training a ...
View details for https://arxiv.org/abs/2008.05028
Given recent advances in learned video prediction, we investigate whether a simple video codec using a pretrained deep model for next frame prediction based on previously encoded/decoded frames without sending any motion side information can compete with standard video codecs based on block motion compensation. Frame differences given learned frame predictions are encoded by a standard still-image (intra) codec. Experimental results show that the rate distortion performance of the simple codec with symmetric ...
View details for https://link.springer.com/article/10.1007/s11760-020-01751-y
An important problem encountered by both natural and engineered signal processing systems is blind source separation. In many instances of the problem, the sources are bounded by their nature and known to be so, even though the particular bound may not be known. To separate such bounded sources from their mixtures, we propose a new optimization problem, Bounded Similarity Matching (BSM). A principled derivation of an adaptive BSM algorithm leads to a recurrent neural network with a clipping nonlinearity. The ...
View details for https://ieeexplore.ieee.org/abstract/document/9053114/
A low complexity recurrent neural network structure is proposed for unsupervised separation of both independent and dependent sources from their linear mixtures. The proposed network is generated based on Bounded Component Analysis (BCA) approach. We first propose an Online-BCA optimization setting. Then we derive the corresponding recurrent neural network (RNN) with iterative learning update expressions. The resulting 2-layer network has a fairly simple structure with feedforward synapses at the input layer, recurrent ...
View details for https://ieeexplore.ieee.org/abstract/document/9048916/
This paper reviews the first NTIRE challenge on video super-resolution (restoration of rich details in low-resolution video frames) with focus on proposed solutions and results. A new REalistic and Diverse Scenes dataset (REDS) was employed. The challenge was divided into 2 tracks. Track 1 employed standard bicubic downscaling setup while Track 2 had realistic dynamic motion blurs. Each competition had 124 and 104 registered participants. There were total 14 teams in the final testing phase. They gauge the state-of-the-art in video ...
View details for http://openaccess.thecvf.com/content_CVPRW_2019/html/NTIRE/Nah_NTIRE_2019_Challenge_on_Video_Super-Resolution_Methods_and_Results_CVPRW_2019_paper.html
We analyze the performance of feedforward vs. recurrent neural network (RNN) architectures and associated training methods for learned frame prediction. To this effect, we trained a residual fully convolutional neural network (FCNN), a convolutional RNN (CRNN), and a convolutional long short-term memory (CLSTM) network for next frame prediction using the mean square loss. We performed both stateless and stateful training for recurrent networks. Experimental results show that the residual FCNN architecture performs the best ...
View details for https://ieeexplore.ieee.org/abstract/document/8803624/
Massive multiple-input-multiple-output (MIMO) scheme promises high spectral efficiency through the employment of large scale antenna arrays in base stations. In time division duplexed implementations, co-channel mobile terminals transmit training information such that base stations can estimate and exploit channel state information to spatially multiplex these users. In the conventional approach, the optimal choice for training length was shown to be equal to the number of users, K. In this paper, we propose a new semiblind framework ...
View details for https://ieeexplore.ieee.org/abstract/document/8605530/
We introduce a new channel matrix estimation algorithm for Massive MIMO systems to reduce the required pilot symbols. The proposed method is based on Maximum A Posteriori estimation where the density of QAM transmission symbols are approximated with continuous uniform pdf. Under this simplification, joint channel source estimation problem can be posed as an optimization problem whose objective is quadratic in each channel and source symbol matrices, separately. Also, the source symbols are constrained to lie in an ℓ∞ ...
View details for https://ieeexplore.ieee.org/abstract/document/9048774/
In this article, we propose a Bounded Component Analysis (BCA) approach for the separation of the convolutive mixtures of sparse sources. The corresponding algorithm is derived from a geometric objective function defined over a completely deterministic setting. Therefore, it is applicable to sources which can be independent or dependent in both space and time dimensions. We show that all global optima of the proposed objective are perfect separators. We also provide numerical examples to illustrate the performance of the ...
View details for https://ieeexplore.ieee.org/abstract/document/8462568/
We propose a method for learned compression artifact removal by post-processing of BPG compressed images. We trained three networks of different sizes. We encoded input images using BPG with different QP values. We submitted the best combination of test images, encoded with different QP and post-processed by one of three networks, which satisfy the file size and decode time constraints imposed by the Challenge. The selection of the best combination is posed as an integer programming problem. Although the visual ...
View details for http://openaccess.thecvf.com/content_cvpr_2018_workshops/papers/w50/Kirmemis_Learned_Compression_Artifact_CVPR_2018_paper.pdf
Many papers have recently been published on image restoration and single-image super-resolution (SISR) using different deep neural network architectures, training methodology, and datasets. The standard approach for performance evaluation in these papers is to provide a single “average” mean-square error (MSE) and/or structural similarity index (SSIM) value over a test dataset. Since deep learning is data-driven, performance of the proposed methods depends on the size of the training and test sets as well as the variety and ...
View details for https://ieeexplore.ieee.org/abstract/document/8552961/
Bounded component analysis (BCA) is a recent approach that enables the separation of both dependent and independent signals from their mixtures. This paper introduces a novel deterministic instantaneous BCA framework for the separation of sparse bounded sources. The framework is based on a geometric maximization setting, where the objective function is defined as the volume ratio of two objects, namely, the principal hyperellipsoid and the bounding ℓ 1-norm ball, defined over the separator output samples. It is shown that all global ...
View details for https://ieeexplore.ieee.org/abstract/document/8443153/
Bounded component analysis (BCA) is a recent approach that enables the separation of both dependent and independent signals from their mixtures. This paper introduces a novel deterministic instantaneous BCA framework for the separation of sparse bounded sources ...
View details for https://ieeexplore.ieee.org/abstract/document/8443153/
Reduction and broadcast operations are commonly used in machine learning algorithms for different purposes. They widely appear in the calculation of the gradient values of a loss function, which are one of the core structures of neural networks. Both operations are implemented naively in many libraries usually for scalar reduction or broadcast; however, to our knowledge, there are no optimized multidimensional implementations available. This fact limits the performance of machine learning models requiring these operations to be performed on tensors. In this work, we address the problem and propose two new strategies that extend the existing implementations to perform on tensors. We introduce formal definitions of both operations using tensor notations, investigate their mathematical properties, and exploit these properties to provide an efficient solution for each. We implement our parallel strategies and test them on a CUDA enabled Tesla K40 m GPU accelerator. Our performant implementations achieve up to 75% of the peak device memory bandwidth on different tensor sizes and dimensions. Significant speedups against the implementations available in the Knet Deep Learning framework are also achieved for both operations.
View details for https://onlinelibrary.wiley.com/doi/abs/10.1002/cpe.4691
The complexity of Machine Learning (ML) models and the frameworks people are using to build them has exploded along with ML itself. State-of-the-art models are increasingly programs, with support for programming constructs like loops and recursion, and this brings out many interesting issues in the tools we use to create them — that is, programming languages (PL). This paper1 , discusses the necessity for a first class language for machine learning, and what such a language might look like.
View details for http://www.sysml.cc/
Broadcast is a common operation in machine learn- ing and widely used in calculating bias or subtracting maximum for normalization in convolutional neural networks. Broadcast operation is required when two tensors possibly with different number of dimensions, hence with different number of elements, are input to an element-wise function. Tensors are scaled in process so that the two tensors match in size and dimension. In this research, we introduce a new broadcast functionality for matrices to be used on CUDA enabled GPU devices. We further extend this operation to multidimensional arrays and measure its performance against the implementation available in the Knet deep learning framework. Our final implementation provides up to 2x improvement over the Knet broadcast implementation, which only supports vector broadcast. Our implementation can handle broadcast operations with any number of dimensions. Index Terms—GPU, CUDA, machine learning, broadcast, mul- tidimensional arrays
View details for https://drive.google.com/file/d/0B5_wxKFQbOTMbXVzTWhsbGFrZnM/view?usp=sharing
Bounded component analysis (BCA) is a recently introduced approach including independent component analysis as a special case under the assumption of source boundedness. In this paper, we provide a stationary point analysis for the recently proposed instantaneous BCA algorithms that are capable of separating dependent, even correlated as well as independent sources from their mixtures. The stationary points are identified and characterized as either perfect separators, which are the global maxima of the proposed ...
View details for https://ieeexplore.ieee.org/abstract/document/7990159/
Compressed training approach offers to improve the spectral efficiency by significantly reducing the required training length. It is based on a convex optimization setting (SCFDE-CoTA) which combines two cost functions: 1) Least squares based training reconstruction performance, and 2) Infinity norm of equalizer outputs, which exploits magnitude boundedness assumption of digital communication symbols. In this article, we provide an extension of this framework to single carrier frequency domain equalizer based transceivers ...
View details for https://ieeexplore.ieee.org/abstract/document/8335522/
A recently introduced Blind Source Separation method, called Bounded Component Analysis, is used as preliminary technique to isolate direct path radar wave from ground reflected waves in order to overcome the multipath effect. This method enables the radar to estimate the target angle without any a priori knowledge of the operation environment. The numerical experiments illustrate the potential benefit of the proposed approach relative to classical maximum likelihood method (CMLM) based on free space propagation model. ...
View details for https://digital-library.theiet.org/content/conferences/10.1049/cp.2017.0416
We propose “compressed training adaptive equalization” as a novel framework to reduce the quantity of training symbols in a communication packet. It is a semi-blind approach for communication systems employing time-domain/frequency-domain equalizers, and founded upon the idea of exploiting the magnitude boundedness of digital communication symbols. The corresponding algorithms are derived by combining the least-squares-cost-function measuring the training symbol reconstruction performance and the infinity-norm of the ...
View details for https://ieeexplore.ieee.org/abstract/document/7937918/
We propose “compressed training adaptive equalization” as a novel framework to reduce the quantity of training symbols in a communication packet. It is a semi-blind approach for communication systems employing time-domain/frequency-domain equalizers, and founded ...
View details for https://ieeexplore.ieee.org/abstract/document/7937918/
This article proposes an adaptive equalization framework for flat fading multi-input multi-output (MIMO) systems, where the main goal is to significantly reduce the number of training symbols. The proposed approach exploits the special boundedness property of digital ...
View details for https://ieeexplore.ieee.org/abstract/document/7536758/
Knet (pronounced "kay-net") is the Koç University machine learning framework implemented in Julia, a high-level, high-performance, dynamic programming language. Unlike gradient generating compilers like Theano and TensorFlow which restrict users into a modeling mini-language, Knet allows models to be defined by just describing their forward computation in plain Julia, allowing the use of loops, conditionals, recursion, closures, tuples, dictionaries, array indexing, concatenation and other high level language features. High performance is achieved by combining automatic differentiation of most of Julia with efficient GPU kernels and memory management. Several examples and benchmarks are provided to demonstrate that GPU support and automatic differentiation of a high level language are sufficient for concise definition and efficient training of sophisticated models.
View details for https://goo.gl/KeOEoJ