25th June 2020
Japan Network Information Center (JPNIC)
Service failure report: ROAs had not been listed in Manifest file (June 8/resolved)
Many ROA published from JPNIC's RPKI repository (rpki-repository.nic.ad.jp) had not been listed in Manifest files from May 8, 11:30 to 21:20 (UTC+9) due to multiple technical factors in JPNIC's RPKI system. The condition has been resolved currently. We sincerely apologize for your inconvenience.
Affected services
Because many ROA published from rpki-repository.nic.ad.jp could not be validated successfully, those ROA would be excluded for origin validation on BGP routes. As a result origin validation results could be "Not found", as same as those ROA were not created.
IP address in those ROA are shown in the following page.
- IP prefixes that could be resulted as "Not found" for origin validation
- https://www.nic.ad.jp/en/topics/2020/20200625-02.html
Circumstances and the cause
The condition had been caused by multiple technical factors in JPNIC's RPKIsystem.
The automatic certificates updating function in the RPKI system was not working correctly. We took re-creating procedures around May 8, 11:30. The created ROA with certificates had not reflected accurately in manifests*1, then manifests had been kept old ROA list. The certificates are updated successfully, but the manifests had not been updated until re-creation at 21:20.
Date/time of the condition
From May 8 (mon) 11:30 to 21:20 UTC+9
On dealing with the condition
After receiving notifications from a knowledgeable community member, we confirmed the conditions, took a procedure to re-create manifests and checked the ROA validation results. We investigated the detailed cause and affected the range of IP addresses, and reached today.
To prevent recurrences, we are revising the current monitoring method and considering improvements.
We are trying to make an early notification in "RPKI slack" currently, but other timely announcements are under consideration.
