August 31 - September 6, 2021
- “Beacon out of sync” alerts are too noisy. We may need to reduce sensitivity since most auto-resolve within minutes.
- Several disk space alerts.
- Node Disk Space Alert- mainnet shard0 node (184.108.40.206)
- Giv: ebs volume was only 8gb. Increased to 512gb.
- Node Disk Space Alert - mainnet shard0 node (220.127.116.11)
- Giv: This time it is on /data - this requires instance upgrades That Jack and Soph resolved by upgrading instance.
- Node Disk Space Alert - the free space of the mainnet shard0 node (18.104.22.168)
- Giv: Happened along with 2 other disk space incidents at the same time on /data
- Node Disk Space Alert- the free space of the mainnet shard0 node (22.214.171.124)
- Giv: same with this one on /data
- 126.96.36.199:9500 out of sync! - mainnet
Giv: Restarted process. Disk space is full. Jack deleted old archive files to make room
- Node Disk Space Alert - the free space of the mainnet shard3 node(188.8.131.52) abnormal
- Giv: Increased ebs volume