Various Catholic churches have been built in this area since ancient times, and the first of them was the Church of St. John the Baptist, which was built in 1387. It was one of the first temples that were built in the Principality of Lithuania. In 1636, the Church was rebuilt, resulting in a tin roof, but retained the wooden base. Even then there were three altars inside the building, and the roof had a dome with a characteristic cross. Not far from the Church, by the way, there was a priest's house, as well as a hospital and a school. In 1850, the Church was rebuilt completely, but just fifteen years later it was closed. After some time it was consecrated again, but it was an Orthodox Church, so that there was not a single Catholic Church in the area. 
 In 1906, after long enough attempts, local Catholics managed to achieve the return of a full parish. Unfortunately returned them not the temple itself, and one of the warehouses that was located nearby.