# Self\-learning Monte Carlo with equivariant Transformer

**INSPIRE:** [2669845](https://inspirehep.net/literature/2669845)
**arXiv:** [2306\.11527](https://arxiv.org/abs/2306.11527)
**DOI:** [10\.7566/JPSJ\.93\.114007](https://doi.org/10.7566/JPSJ.93.114007)

**Authors:** Nagai, Yuki, Tomiya, Akio

**Submitted:** 20 June 2023

**Subjects:**
- cond\-mat\.str\-el
- cond\-mat\.dis\-nn
- hep\-lat
- Condensed Matter
- Lattice

**Journal reference:** J\.Phys\.Soc\.Jap\. 93 114007 \(2024\)

**Citations:** 9

## Abstract

Machine learning and deep learning have revolutionized computational physics, particularly the
simulation of complex systems\. Equivariance is essential for simulating physical systems because
it imposes a strong inductive bias on the probability distribution described by a machine learning
model\. However, imposing symmetry on the model can sometimes lead to poor acceptance rates in
self\-learning Monte Carlo \(SLMC\)\. Here, we introduce a symmetry equivariant attention mechanism
for SLMC, which can be systematically improved\. We evaluate our architecture on a spin\-fermion
model \(i\.e\., double exchange model\) on a two\-dimensional lattice\. Our results show that the pro\-
posed method overcomes the poor acceptance rates of linear models and exhibits a similar scaling
law to large language models, with model quality monotonically increasing with the number of
layers\. Our work paves the way for the development of more accurate and efficient Monte Carlo
algorithms with machine learning for simulating complex physical systems