Status updates | Chainguard, Inc.

Build and runtime failures related to the OpenSSL 4 transition

Fri, 09 Oct 2026 21:35:00 -0000

We continue monitoring and resolving issues related to the OpenSSL 4 transition. More fixes for specific OpenSSL 3/4 compatibility issues are in testing or already deployed. If you’re seeing build or runtime errors, follow the steps under “What you can do now” in our [troubleshooting guide](https://support.chainguard.dev/hc/en-us/articles/56956385560219-ACTION-RECOMMENDED-OpenSSL-3-to-4-transition-may-cause-build-and-runtime-failures) (login required). If you still have issues after following that guidance, contact Support with your image digest and the full error.

Build and runtime failures related to the OpenSSL 4 transition

Thu, 08 Oct 2026 20:06:00 -0000

Guidance on troubleshooting and resolving issues with affected images is available here (paid customer login required): https://support.chainguard.dev/hc/en-us/articles/56956385560219-ACTION-RECOMMENDED-OpenSSL-4-transition-may-cause-build-and-runtime-failures. Please check your email, Slack messages, or Activity Console, according to your communications opt-ins with us, as we have also sent it there according to your preferences. We are monitoring the incident, and are still moving forward with the OpenSSL 4 ABI migration. If you experience any issues you believe are related to the migration from OpenSSL 3.6 to 4.0 at this time, please contact Support.

Build and runtime failures related to the OpenSSL 4 transition

Tue, 06 Oct 2026 22:21:00 -0000

Additional information on troubleshooting and resolving issues with affected images is forthcoming. We will provide a link to the updated documentation when it goes live. Some customers are seeing build or runtime failures related to the ongoing Chainguard OS transition to OpenSSL 4, mostly when adding packages to an image with apk add or copying binaries between images. We continue to work through other reports related to the OpenSSL 4 transition. If you are affected, please contact support with your image digest and the error output, including errors related to OpenSSL or cipher suites.

Build and runtime failures related to the OpenSSL 4 transition

Tue, 06 Oct 2026 17:24:00 -0000

Some customers are seeing build or runtime failures related to the ongoing Chainguard OS transition to OpenSSL 4, mostly when adding packages to an image with apk add or copying binaries between images. We continue to work through other reports related to the OpenSSL 4 transition. If you are affected, please contact support with your image digest and the error output, including errors related to OpenSSL or cipher suites.

Build and runtime failures related to the OpenSSL 4 transition

Tue, 06 Oct 2026 16:27:00 -0000

Build and runtime failures related to the OpenSSL 4 transition

Tue, 06 Oct 2026 15:08:00 -0000

Some customers are seeing build or runtime failures related to the ongoing Chainguard OS transition to OpenSSL 4, mostly when adding packages to an image with apk add or copying binaries between images. Fixes are available or rolling out for the affected images, and building from a current image with current packages resolves most cases. If you are affected, please contact support with your image digest and the error output.

APK and Image publishing is delayed

Thu, 01 Oct 2026 18:58:00 -0000

The issue has been resolved. Packages and images are now being published as expected.

APK and Image publishing is delayed

Thu, 01 Oct 2026 17:57:00 -0000

We are currently investigating an issue causing the publishing of new package and image builds to be delayed. The availability of existing packages and images is not impacted. We will follow up with more information as we know more.

Maintenance: Guardener Database Maintenance

Tue, 29 Sep 2026 12:44:38 -0000

Maintenance is complete.

Maintenance: Guardener Database Maintenance

Tue, 29 Sep 2026 12:44:38 -0000

Maintenance completed

Maintenance: Guardener Database Maintenance

Mon, 28 Sep 2026 19:05:01 -0000

We are performing database maintenance for Guardener. During this time, Guardener sessions, migrations, and GitHub app functionality may be briefly degraded or unavailable.

Issues authenticating to the support portal

Wed, 09 Sep 2026 16:59:00 -0000

The issue has been resolved.

Issues authenticating to the support portal

Wed, 09 Sep 2026 16:49:00 -0000

We are currently experiencing issues with authentication onto our support portal. We are investigating. As a temporary measure, customers can sent support requests via email at support@chainguard.dev

Incorrect repositories list being included in image builds

Wed, 09 Sep 2026 16:12:00 -0000

All images are now rebuilt. If you experience any further problems, please raise a support ticket as normal.

Incorrect repositories list being included in image builds

Wed, 09 Sep 2026 14:00:00 -0000

We expect all 35,000 repositories to complete rebuilds by 16:00GMT.

Incorrect repositories list being included in image builds

Wed, 09 Sep 2026 11:08:00 -0000

The image build problem has been fixed. And we have started rebuilding. Due to the number of affected images, this may take 3+ hours. If you need an image sooner you can run "chainctl img repo update " yourself to trigger an update.

Incorrect repositories list being included in image builds

Tue, 08 Sep 2026 16:10:00 -0000

Between approximately 16:00GMT on 8th September and 11:00GMT on the 9th September images have been built with an apk repositories file that uses our old "packages.wolfi.dev" address instead of the new "virtualapk.cgr.dev". If you are building on top of these images and installing apks, some installations may fail because the old address does not include your full package entitlements. The APK repositories themselves are unaffected. We are working to revert the change that caused the problem and will then need to rebuild all affected images. We don't yet have an ETA on how long all the rebuilds will take.

Chainguard websites unavailable for some ISPs

Thu, 27 Aug 2026 00:36:00 -0000

The issue has been resolved, and all systems are working normally.

Chainguard websites unavailable for some ISPs

Tue, 25 Aug 2026 15:18:00 -0000

We’re investigating reports that some users on residential Xfinity networks with Advanced Security/CyberSecure enabled are unable to reach chainguard.dev and certain subdomains, including our website, documentation, console, and support portal. Users may see TLS or connection errors. Our services are operating normally, and we have not identified any data or security impact to Chainguard systems. Users on other networks, including many corporate and mobile networks, appear to be unaffected. We have submitted reassessment requests to Xfinity and are continuing to investigate the filtering signals involved. If you’re affected, try accessing the site from another network or through a VPN. Xfinity customers may also be able to temporarily allow access through the Xfinity app, although this may need to be repeated periodically. We’ll provide another update as we learn more.

Chainguard websites unavailable for some ISPs

Mon, 24 Aug 2026 22:04:00 -0000

Some Chainguard websites may be unreachable on certain networks. We're aware that some users are currently unable to reach chainguard.dev and related sites, including our website, documentation, console, and support portal. Our services are operating normally. The issue is caused by upstream network filtering at some internet service providers. If you're affected, you should be able to reach these sites by connecting through a different network or a VPN. Corporate and mobile networks are generally not affected. We're reaching out to the affected providers, and we'll update this page as we learn more.

Resolved: Zendesk support portal may not be available

Fri, 14 Aug 2026 22:58:00 -0000

The situation has been confirmed to be resolved.

Resolved: Zendesk support portal may not be available

Fri, 14 Aug 2026 20:21:00 -0000

Access to the Zendesk support portal has been restored. We are continuing to monitor the situation.

Resolved: Zendesk support portal may not be available

Fri, 14 Aug 2026 16:43:00 -0000

A temporary fix has been put in place while we work on a permanent solution. Access to the Zendesk support portal is now working as expected.

Resolved: Zendesk support portal may not be available

Fri, 14 Aug 2026 16:15:00 -0000

Engineering is continuing to investigate the issue.

Resolved: Zendesk support portal may not be available

Fri, 14 Aug 2026 15:00:00 -0000

Engineering is continuing to investigate the issue.

Resolved: Zendesk support portal may not be available

Fri, 14 Aug 2026 14:00:00 -0000

We are investigating reports of users receiving an error page when accessing our Zendesk support portal. We are working to identify the root cause and a resolution, and will post an update as soon as we have more information.

Resolved: Delayed package scan results

Tue, 11 Aug 2026 23:46:00 -0000

The malware scan backlog has been fully processed. All packages have completed their initial scans and are now available as expected. If you are experiencing issues with a specific package, please reach out to our support team. Thank you for your patience while we completed this work.

Resolved: Delayed package scan results

Tue, 11 Aug 2026 14:17:00 -0000

The malware scan backlog continues to drain and is in its final stage. The vast majority of packages have completed scanning and are available as expected. A small number of remaining packages are being re-processed and we expect them to clear shortly. We will continue to provide updates as the work progresses.

Resolved: Delayed package scan results

Tue, 11 Aug 2026 08:59:00 -0000

Malware scan is ongoing currently. Until a package's scan completes, requests for it may fail or return MALWARESCANPENDING. We expect this process to take around few more hours. We'll keep this page updated.

Resolved: Delayed package scan results

Tue, 11 Aug 2026 06:03:00 -0000

95% of malware scan has completed. Until a package's scan completes, requests for it may fail or return MALWARE_SCAN_PENDING. We expect this process to take approximately 3 more hours.

Resolved: Delayed package scan results

Tue, 11 Aug 2026 02:52:00 -0000

We have deployed fixes for the underlying issue, and remaining impact is limited to a backlog of pending malware scans. Until a package's scan completes, requests for it may fail or return MALWARE_SCAN_PENDING. We expect this process to take approximately 3 more hours. Separately, a small number of packages were incorrectly flagged and blocked by malware scanning, i.e., false positives. We are reviewing and unblocking these; if a specific package remains blocked or unavailable for you, please open a support ticket.

Resolved: Delayed package scan results

Mon, 10 Aug 2026 23:46:00 -0000

We've identified an issue affecting the timing of malware scan completion for some upstream packages, which may cause download errors (MALWARE_SCAN_PENDING) for a subset of packages. We've found no evidence that any customer has downloaded a package later confirmed to be malicious. We've increased scanning capacity and deployed a fix to staging, and are working to fully resolve the backlog. We'll post updates as we have them.

Security update for the Chainguard Metabase image

Fri, 07 Aug 2026 10:49:00 -0000

A patched Chainguard Metabase image has been released and has propagated across all affected image tags. What was addressed. This update remediates three upstream Metabase advisories: GHSA-vwf4-m7j8-wcjf (fixed directly in the image), and GHSA-8hmm-hrhg-ppqp and GHSA-r8h2-qpfx-mx59 (mitigated by disabling public dashboard sharing by default). Public dashboard sharing is now off by default. If you need to re-enable it, set the environment variable MB_ENABLE_PUBLIC_SHARING to true — but only if your instance is not publicly reachable. We intend to restore the previous default once upstream releases a full fix. Please note on vulnerability scanners. Upstream Metabase has not yet cut a versioned release, so the image version number remains within the advisories' affected range. Scanners may therefore flag the patched image as vulnerable even though the fix and mitigations are in place. We will update the version once upstream publishes a release. Recommended actions. If you have not already done so, complete the steps from our earlier update: update to the latest patched image, revoke active sessions, delete any unrecognized API keys, audit administrator accounts, and rotate credentials for any connected databases. Upgrading alone does not remove access an attacker may have already established, so these steps remain important. This incident is now resolved. Thank you for your patience.

Security update for the Chainguard Metabase image

Fri, 07 Aug 2026 10:22:00 -0000

We have identified of a security vulnerability in upstream Metabase that affects the Metabase container image published by Chainguard. A small number of customers running the affected image may be exposed until they update. A patched image is currently being built and is expected to be released shortly. We will be monitoring propagation to confirm all affected image tags are updated. Recommended actions for anyone running our Metabase image. These apply regardless of whether you have updated, because some steps address exposure that an update alone does not remove: Update to the latest patched Chainguard Metabase image once available. Block access to the password reset endpoint (/api/session/reset_password) at your ingress. Revoke active sessions. Delete any API keys you do not recognize. Audit administrator accounts for unexpected changes. Rotate credentials for any databases connected to Metabase. Please note: upgrading the image alone is not sufficient, as any attacker-created API keys or previously exposed database credentials will persist after an upgrade. We will post a further update once the patched image has released.

Image builds and releases are experiencing delays

Fri, 07 Aug 2026 00:37:00 -0000

Build activity has returned to normal. Users should see new image and package updates as they are expected.

Image builds and releases are experiencing delays

Thu, 06 Aug 2026 23:54:00 -0000

We are continuing to see image and package builds are continuing to complete successfully. We will continue monitoring the status and update here.

Image builds and releases are experiencing delays

Thu, 06 Aug 2026 23:23:00 -0000

We are beginning to see package and image builds be processed successfully, though some may still experience delays as GitHub continues to drain their backlog and fully restore throughput. GitHub has deployed fixes and is making progress on recovery. They report 99% success rates for workflow runs that are able to start, and are gradually restoring webhook-triggered workflow throughput. We are seeing build activity resume in our internal dashboards, though the recovery process is still ongoing. We are actively monitoring GitHub's recovery progress as well as our internal build processes. We will provide updates as the situation evolves. If you have urgent questions or concerns, please reach out to our support team.

Image builds and releases are experiencing delays

Thu, 06 Aug 2026 22:30:00 -0000

We continue to experience delays with package and container image builds due to an upstream GitHub outage that began at 19:08 UTC today. GitHub has deployed fixes that have improved job completion rates for workflow runs that are able to start (now at 97% success). However, webhook triggers remain throttled as part of their recovery process, which means many push and pull request events are not yet triggering new build workflows. This is preventing most of package and image builds from starting. We understand this impacts your release schedules and any CVE remediation work with upcoming deadlines. We are actively monitoring GitHub's recovery progress and will resume normal build operations as soon as their service is restore. We will provide updates as the situation evolves. If you have urgent questions or concerns, please reach out to our support team.

Image builds and releases are experiencing delays

Thu, 06 Aug 2026 19:22:00 -0000

We are currently experiencing delays with package and container image builds due to an upstream GitHub outage. Our build infrastructure remains operational, but we are unable to process new build requests until GitHub service is restored. We understand this may impact your release schedules and CVE remediation timelines. We are actively monitoring the GitHub status page and will resume normal build operations as soon as their service is available. We will provide updates as we learn more about the expected resolution timeline. If you have any urgent questions or concerns, please reach out to our support team. Thank you for your patience as we work through this issue.

Intermittent 500 errors from Libraries and apk downloads

Wed, 22 Jul 2026 09:47:00 -0000

Cloudflare have implemented a fix and are monitoring their services. The Chainguard services have recovered and the services are being monitored.

Chainguard API functionality degraded

Mon, 20 Jul 2026 03:49:00 -0000

We've mitigated the issue and are continuing to monitor the affected services.

Chainguard API functionality degraded

Mon, 20 Jul 2026 03:21:00 -0000

We have identified the issue and are actively working on a resolution.

Chainguard API functionality degraded

Mon, 20 Jul 2026 02:57:00 -0000

We are aware of degradations with the Chainguard API and are actively working to fix these issues.

Maintenance: Maintenance Scheduled | Authentication and Authorization Services

Thu, 09 Jul 2026 18:33:38 -0000

Chainguard authentication and authorization services will be briefly unavailable during this window. Sign-in and permission-based actions may fail. No action required; services will resume automatically after maintenance completes.

403 errors when attempting to access Libraries

Thu, 09 Jul 2026 11:46:00 -0000

After investigation we have identified that the 403 errors were a false positive and the services were not degraded or effected during this time.

403 errors when attempting to access Libraries

Thu, 09 Jul 2026 10:37:00 -0000

A small number of customers may see 403 errors when attempting to access Libraries. Chainguard engineering are investigating this issue. Further updates will be published as we progress.

Maintenance: Maintenance Scheduled | Authentication and Authorization Services

Thu, 02 Jul 2026 19:19:47 -0000

Multiple service outage

Fri, 26 Jun 2026 22:41:00 -0000

We have identified the root cause of the database connection exhaustion and the issues have been resolved.

Multiple service outage

Fri, 26 Jun 2026 22:08:00 -0000

Issues with a backend database due to exhausted connections are causing degraded availability for all Chainguard services. Engineers are currently working to mitigate the issue and restore services. We will share more information as is it is available.

Image Overviews are missing

Fri, 26 Jun 2026 11:43:00 -0000

We have fixed the cause of the missing Overviews and are starting to repopulate them.

Image Overviews are missing

Fri, 26 Jun 2026 09:19:00 -0000

The image 'Overview' section on console.chainguard.dev is currently not rendering end-user usage documentation. This has no impact on overall functionality or image availability. We are working to restore the Overview.

Authentication is degraded

Thu, 25 Jun 2026 20:11:00 -0000

Authentication is currently experiencing issues impacting new logins for the Console, API, OCI Registry, APK Registry, and all Libraries ecosystems. Existing sessions appear to be unaffected. We are currently investigating and will update the status page with more information soon.

Maintenance: Maintenance Scheduled | Authentication and Authorization Services

Fri, 19 Jun 2026 19:30:13 -0000

Platform Services Issues Causing Downtime

Fri, 19 Jun 2026 00:04:00 -0000

We are monitoring recovery as services continue to stabilize. Customers may begin seeing improvements when authenticating to Chainguard services, including APK fetching, Libraries, and Console access. Our engineering team continues to closely monitor service health and investigate the underlying cause, and we will share another update as more information becomes available.

Platform Services Issues Causing Downtime

Thu, 18 Jun 2026 23:39:00 -0000

We are investigating a service issue affecting platform authentication and related services. Customers may experience issues authenticating to Chainguard services, including impacts to APK fetching, Libraries, and Console access. Our engineering team is actively investigating and working on mitigation and recovery efforts to restore service as quickly as possible. We will continue providing updates as more information becomes available.

Platform Services Issues Causing Downtime

Thu, 18 Jun 2026 22:59:00 -0000

We’re investigating an issue with platform services that is impacting some users. We’re working to fix the problem as quickly as we can. We’ll share another update shortly.

Vulnerability and Advisory APIs Currently Unavailable

Wed, 10 Jun 2026 18:40:00 -0000

The issue has been resolved and we've confirmed that the Vulnerability and Advisory APIs are now accessible.

Vulnerability and Advisory APIs Currently Unavailable

Wed, 10 Jun 2026 18:31:00 -0000

The Vulnerability and Advisory APIs are currently unavailable and impact certain features within the Chainguard Console and chainctl. We are currently investigating the issue and will update you as soon as we have more information.

APK fetching and authentication issues

Wed, 27 May 2026 00:07:00 -0000

The affected services have now been stabilized. We are continuing to monitor the environment. We sincerely apologize for the disruption caused by the errors impacting the APK registry backend and authentication backend. We will share a full RCA with customers. Thank you for your patience and understanding while we worked through this issue.

APK fetching and authentication issues

Tue, 26 May 2026 23:51:00 -0000

APK fetching and authentication issues

Tue, 26 May 2026 23:23:00 -0000

APK fetching and authentication issues

Tue, 26 May 2026 22:07:00 -0000

We are aware of customers experiencing issues with authenticating and APK fetching. This issue is currently under investigation.

Intermittent failures affecting virtualapk.cgr.dev

Tue, 12 May 2026 15:32:00 -0000

We sincerely apologize for the disruption caused by elevated 5XX error rates impacting the APK registry backend. Since enabling package deduplication, memory utilization has stabilized between 10–30%, and 5XX error rates have returned to near-zero levels, with only occasional brief spikes observed. Overall service health has significantly improved, and we are continuing to monitor the platform closely to ensure stability. Thank you for your patience while we worked through this issue.

Intermittent failures affecting virtualapk.cgr.dev

Fri, 08 May 2026 23:00:00 -0000

The overall error rate and impact on CI workflows is now much lower and less disruptive for affected customers. We are actively monitoring error and memory metrics to confirm sustained stability before declaring a full resolution.

Intermittent failures affecting virtualapk.cgr.dev

Fri, 08 May 2026 19:14:00 -0000

We've deployed the APKINDEX LRU cache update to production, which has led to a significant drop in 503 errors and error-rate spikes for APK downloads from virtualapk.cgr.dev. Some 503s are still being observed because the cache size appears to be larger than ideal, and the average entry size was underestimated. However, overall service reliability has improved and error rates are much lower than before the change. We will lower the cache size to better reflect the average cache entry size and lower the rate of 503s even further today.

Intermittent failures affecting virtualapk.cgr.dev

Fri, 08 May 2026 13:11:00 -0000

We are investigating intermittent failures affecting virtualapk.cgr.dev, where some APK index and package fetch requests may return temporary errors during short periods of elevated load. We have identified the root cause and are rolling out mitigations to improve stability. During impacted windows, retries typically succeed. The most reliable workaround is having retry-on-failure logic around CI/CD jobs. Reruns almost always succeed once the backend is healthy. We will provide additional updates as mitigation rollout progresses.

Maintenance: 🔧 Quick Console and Chainguard Support Maintenance Heads Up — 5 PM PST Today

Mon, 20 Apr 2026 19:57:21 -0000

🔧 Quick heads up — we'll be performing a scheduled update to our Console and Chainguard Support system today at 5:00 PM PST. Expect approximately 5 minutes of downtime. We'll be back up shortly after. Thanks for your patience!

Elevated errors rates for chainguard.platform.registry.Vulnerabilities API

Thu, 05 Mar 2026 15:18:00 -0000

Issues are now resolved.

Elevated errors rates for chainguard.platform.registry.Vulnerabilities API

Thu, 05 Mar 2026 14:31:00 -0000

We are seeing elevated errors rates for chainguard.platform.registry.Vulnerabilities APIs. Other API services are unaffected.

Performance Degradation Affecting IAM and Registry Services

Thu, 12 Feb 2026 23:41:00 -0000

Between February 10 and 11, we experienced elevated error rates across internal services, which impacted authentication and registry availability. The issue has been resolved, and we are continuing to investigate the root cause.

False positive CVEs in glibc

Wed, 11 Feb 2026 16:27:00 -0000

A glibc change has been deployed that will prevent duplicate glibc sbom detection for all future builds. Existing binaries will need to be rebuilt to drop such detection. As this is deployed, the incident is declared closed. If additional rebuilds are desired, please open a support ticket.

False positive CVEs in glibc

Wed, 04 Feb 2026 23:58:00 -0000

We are investigating multiple duplicate SBOM detections for the glibc package in unrelated binaries which lead to false positive detections for recent glibc CVEs such as CVE-2025-15281

False positives in NodeJS 22 images

Tue, 03 Feb 2026 19:07:00 -0000

We identified a typographical issue in our feeds that caused scanners to report false positives for CVE-2025-59466 in images containing NodeJS 22. An incorrectly formatted “fixed” version was published, leading scanners to conclude that the CVE had not been addressed in the latest release, 22.22.0-r0. The issue has now been corrected in our feeds. Downstream scanners will need to ingest the updated data and refresh their databases. Since most scanners rebuild their databases daily, we expect the corrected information to be reflected in many cases by the next overnight update. In addition to correcting the data, we are implementing safeguards to prevent incorrectly formatted values from being accepted in the future. A full root cause analysis will be shared with customers in due course.

Maintenance: Database maintenance

Thu, 28 Aug 2025 15:22:37 -0000

While we perform the database maintenance, we expect a short period of disruption. Thank you for your understanding.

Database maintenance update triggered an outage

Tue, 26 Aug 2025 19:40:00 -0000

We saw a brief outage period as our database was going through a maintenance update. All services have since recovered.

Customers may be experiencing intermittent errors pulling packages and indexes.

Wed, 06 Aug 2025 16:50:00 -0000

We've released a change that we anticipate will solve the immediate issue. We're monitoring closely, but metrics indicate the fix was successful.

Customers may be experiencing intermittent errors pulling packages and indexes.

Wed, 06 Aug 2025 15:48:00 -0000

We're experiencing elevated error rates for APK package serving from apk.cgr.dev and virtualapk.cgr.dev. Customers may be experiencing intermittent errors pulling packages and indexes. We're monitoring and releasing a fix soon. We will provide an update in an hour.

All Chainguard Services Experiencing Downtime

Thu, 12 Jun 2025 19:56:00 -0000

Cloudlfare and GCP have completed their monitoring and we can confirm that all services have returned to normal. This incident is now resolved. Thank you for your patience throughout the process. If you have any remaining questions or concerns, feel free to reach out via support.chainguard.dev.