Amazon's extensive white paper on privacy and security for the Sidewalk network should give you an idea of how invasive the network potentially is. By keeping Sidewalk active you are betting on Amazon having pre-emptively thought of everything that can go wrong.
