Novel Perspectives in Imitation Learning: Trends, Challenges, Future Directions
Keywords:
Imitation, Learning, Supervised, CNN, RNNAbstract
Imitation learning refers to a machine learning approach where an algorithm learns to accomplish a task by emulating the actions and behaviours of a proficient expert. The main aim of imitation learning is to extract knowledge from an expert and train the datasets using the existing knowledge. During training, the algorithm is presented with a large dataset of expert demonstrations and learns to map the input to the correct output based on the examples it has seen. The learned mapping can then be used to perform the task on new inputs that were not present in the training dataset. One example of imitation learning is training an autonomous car to drive by watching and imitating a human driver. In the past few years, various trends in imitation learning have emerged, significantly influencing the trajectory of this field, and shaping its future. These trends include development, integration, and transfer learning. These trends in imitation learning are helping to make this technique more efficient, effective, and applicable to a broader range of tasks and environments. Over the past few years, the imitation learning field has experienced the emergence of multiple trends that have had a substantial impact on its direction and are actively shaping its future. The development of new and more efficient and scalable methods for imitation learning could enable it to be applied to even larger and more complex problems and increase the scope of the problems. In this study, we provide a literature review of trends, challenges, and future developments in imitation learning.
References
Ganos C, Ogrzal T, Schnitzler A, Münchau A. The pathophysiology of echopraxia/echolalia: relevance to Gilles de la Tourette syndrome. Mov Disord. 2012 Sep 1; 27(10): 1222–9.
Aler R, Garcia O, Valls JM. Correcting and improving imitation models of humans for robosoccer agents. In 2005 IEEE Congress on Evolutionary Computation. 2005 Sep 2; 3: 2402–2409.
Edwards C, Edwards A, Stoll B, Lin X, Massey N. Evaluations of an artificial intelligence instructor's voice: Social Identity Theory in human-robot interactions. Comput Hum Behav. 2019 Jan 1; 90: 357–62.
Billard A, Kragic D. Trends and challenges in robot manipulation. Science. 2019 Jun 21; 364(6446): eaat8414.
Abbeel P, Ng AY. Apprenticeship learning via inverse reinforcement learning. In Proceedings of the 21st international conference on Machine learning. 2004 Jul 4; 1.
Nozari S, Krayani A, Marin P, Marcenaro L, Martin D, Regazzoni C. Autonomous Driving Based on Imitation and Active Inference. In Advances in System-Integrated Intelligence: Proceedings of the 6th International Conference on System-Integrated Intelligence (SysInt 2022), 2022 Sep 7–9, Genova, Italy. 2022 Sep 4; 13–22. Cham: Springer International Publishing.
Garg D, Chakraborty S, Cundy C, Song J, Ermon S. Iq-learn: Inverse soft-q learning for imitation. Adv Neural Inf Process Syst. 2021 Dec 6; 34: 4028–39.
Abbeel P, Coates A, Quigley M, Ng A. An application of reinforcement learning to aerobatic helicopter flight. Adv Neural Inf Process Syst. 2006; 19.
Ng AY, Russell S. Algorithms for inverse reinforcement learning. InIcml 2000 Jun 29 (Vol. 1, p. 2).
Aler R, Garcia O, Valls JM. Correcting and improving imitation models of humans for robosoccer agents. In 2005 IEEE Congress on Evolutionary Computation. 2005 Sep 2; 3: 2402–2409.
Edwards A, Sahni H, Schroecker Y, Isbell C. Imitating latent policies from observation. In International conference on machine learning. 2019 May 24; 1755–1763. PMLR.
Ibarz B, Leike J, Pohlen T, Irving G, Legg S, Amodei D. Reward learning from human preferences and demonstrations in Atari. Adv Neural Inf Process Syst. 2018; 31.
Kebria PM, Khosravi A, Salaken SM, Nahavandi S. Deep imitation learning for autonomous vehicles based on convolutional neural networks. IEEE/CAA J Autom Sin. 2019 Dec 30; 7(1): 82–95.
Oh J, Guo Y, Singh S, Lee H. Self-imitation learning. In International Conference on Machine Learning. 2018 Jul 3; 3878–3887. PMLR.
Osa T, Esfahani AM, Stolkin R, Lioutikov R, Peters J, Neumann G. Guiding trajectory optimization by demonstrated distributions. IEEE Robot Autom Lett. 2017 Jan 16; 2(2): 819–26.
Zhu Z, Lin K, Dai B, Zhou J. Off-policy imitation learning from observations. Adv Neural Inf Process Syst. 2020; 33: 12402–13.
Torabi F, Warnell G, Stone P. Imitation learning from video by leveraging proprioception. arXiv preprint arXiv:1905.09335. 2019 May 22.
