A group of travellers have set up camp at Stratford Park Leisure Centre.
Caravans and vans are believed to have arrived at the car park this morning.
Several caravans have been spotted on the site, which currently provides a walk-through coronavirus testing facility.
An eyewitness said: “They arrived in a convoy as I turned up to use the gym. They had a couple of fuel generators and a large bucket of washing.”
A Stroud District Council spokesperson said: “As soon as we became aware of this matter, we made arrangements to visit the site and speak with those concerned. We want to resolve this issue as quickly as possible.”