GlassBERTa

Language Modelling as Unsupervised Pre-Training for Glass Alloys

Abstract:

Alloy Property Prediction is a task under the sub field of Alloy Material Science wherein Machine Learning has been applied rigorously. This is modeled as a Supervised Task wherein Alloy Composition is provided for the Model to predict a desired property. Efficiency of tasks such as Alloy Property Prediction, Alloy Synthesis can be modeled additionally with an Unsupervised Pre-training Task. We describe the idea of Pre-training using Language Modelling kind of approach in terms of Alloy Compositions.We specifically inspect that random masking proposed in is not suitable for modelling Alloys. We further go on proposing two types of masking strategies that are used to train GlassBERTa to encompass the properties of an Alloy Composition. The results suggest that Pre-training is an important field of direction in this field of research for further improvement.

Authors:

Reshinth Adithyan, Aditya TS, Roakesh, Jothikrishna, Kalaiselvan Baskaran

Footnote:

Work done via MLDMM Lab