Linux server, IPv6 works for a while, then blocked

I'm running a Debian server, using Shorewall firewall. After a shorewall6 restart, IPv6 works fine for a while (an hour or two), but then nothing gets in or out again. IPv4 works fine always. I found something on the web that suggested that OVH is probing my server somehow and closing IPv6 down. I have opened the firewall for pings from fe80::/64, but maybe that's not enough?