All my ESPHome devices, seem to reconnect once our Wifi is restored. If I am in a hurry once I fix the Wifi, as it is normally me that broke it, I just power cycle the plugs to get them to reconnect quickly. In our house we have multiple circuits so we can turn off at the fuse box any one of about 20 zones.