Literature

These learning notes cover the material that would advance the transdim project. The authors would like to learn more new knowledge to update the research.

Time series imputation

• Deep learning models
  • Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, Ross Girshick (2021). Masked Autoencoders Are Scalable Vision Learners. arXiv: 2111.06377.
  • Fan-Keng Sun, Christopher I. Lang, Duane S. Boning (2021). Adjusting for Autocorrelated Errors in Neural Networks for Time Series. arXiv: 2101.12578.
  • Andrea Cini, Ivan Marisca, Cesare Alippi (2021). Multivariate Time Series Imputation by Graph Neural Networks. arXiv: 2108.00298.

Time series Forecasting

• Deep learning models

  • Deep Demand Forecast Models (GitHub repository: https://github.com/jingw2/demand_forecast): Pytorch Implementation of DeepAR, MQ-RNN, Deep Factor Models, LSTNet, and TPA-LSTM.

  • A paper Temporal tensor transformation network for multivariate time series prediction (2020) combined tensor structure and transfomer for time series forecasting.

  • Paper Joint forecasting and interpolation of graph signals using deep learning (2020): RNNs + graph signal processing.

  • Anish Agarwal, Abdullah Alomar, Devavrat Shah (2020). On Multivariate Singular Spectrum Analysis.

Machine Learning

• Low-rank matrix/tensor completion

  • Paper Large-scale low-rank matrix learning with nonconvex regularizers (PAMI 2018): fast and nonconvex low-rank matrix completion models.

  • Paper Efficient nonconvex regularized tensor completion with structure-aware proximal iterations (ICML 2019): sparse plus low-rank structure + proximal iteration solution.

  • Paper Scalable tensor completion with nonconvex regularization (2018): scalable tensor learning framework.

  • Jing Ma, Qiuchen Zhang, Joyce C. Ho, and Li Xion (2020). Spatio-Temporal Tensor Sketching via Adaptive Sampling.

• Tensor Singular Value Decomposition

  • Canyi Lu, Jiashi Feng, Yudong Chen, Wei Liu, Zhouchen Lin, Shuicheng Yan (CVPR 2016). Tensor Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Tensors via Convex Optimization.

  • Canyi Lu, Jiashi Feng, Yudong Chen, Wei Liu, Zhouchen Lin, Shuicheng Yan (PAMI 2018). Tensor Robust Principal Component Analysis with A New Tensor Nuclear Norm.

• Schatten p-norm minimization and its application

  • The Schatten p-norm on Rn.

  • Paper t-Schatten-p norm for low-rank tensor recovery (2018): a new definition of tensor Schatten-p norm (t-Schatten-p norm) based on t-SVD.

  • Paper Tensor p-shrinkage nuclear norm for low-rank tensor completion (2019): a new definition of tensor p-shrinkage nuclear norm (pTNN) is proposed based on tensor singular value decomposition (t-SVD).

  • Paper Joint Schatten p-norm and lp-norm robust matrix completion for missing value recovery (2013).

• Hankel matrix

  • Fast Hankel tensor–vector product and its application to exponential data fitting (2015).

  • High Rank Matrix Completion with Side Information (AAAI 2018).

  • Interpolation using Hankel tensor completion (2013).

• Fourier analysis

  • Relationship between Singular Spectrum Analysis and Fourier analysis: Theory and application to the monitoring of volcanic activity: This paper showed that SSA is related to Fourier analysis by using asymptotic properties of the eigenvalues of Toeplitz matrices.

• Total variation

  • Xu Han, Jiasong Wu, Lu Wang, Yang Chen, Lotfi Senhadji, Huazhong Shu (2014). Linear Total Variation Approximate Regularized Nuclear Norm Optimization for Matrix Completion.

• Tensor train decomposition

  • Introduction to the Tensor Train Decomposition and Its Applications in Machine Learning by Anton Rodomanov and a Python toolbox for tensor train on GitHub.
  • Tensorizing Neural Networks in NIPS 2015. [Video]

Statistical Learning

• Books
  • An Introduction to Statistical Learning.

• Gaussian mixture model

  • Blog post An overview of Gaussian Mixture Models by Massimiliano Patacchiola.

• Variational inference

  • The paper Stein Variational Gradient Descent: Theory and Applications.
  • The paper Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm

Deep Learning

• Attention Model

  • Lilian Weng (2018). Attention? Attention!. Blog post.

  • Attention Mechanism. Blog post.

  • Fu et al. (CVPR 2019). Dual Attention Network for Scene Segmentation.

  • GeoMAN: Multi-level Attention Networks for Geo-sensory Time Series Prediction.

  • Sinong Wang, Belinda Z. Li, Madian Khabsa, Han Fang, Hao Ma (2020). Linformer: Self-Attention with Linear Complexity.

Optimization Methods

• Majorization Minimization algorithm

  • Kenneth Lange (2018). Examples of MM Algorithms (slide).

  • David R. Hunter, Kenneth Lange (2004). A Tutorial on MM Algorithms.

  • A paper Exact minimum rank approximation via Schatten p-norm minimization applied the Majorization Minimization algorithm to solve the Schatten p-norm minimization problem.