Jun 02 21:39:17.216586 ip-10-0-133-201 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 02 21:39:17.216598 ip-10-0-133-201 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 02 21:39:17.216605 ip-10-0-133-201 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 02 21:39:17.216817 ip-10-0-133-201 systemd[1]: Failed to start Kubernetes Kubelet. Jun 02 21:39:27.290019 ip-10-0-133-201 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 02 21:39:27.290039 ip-10-0-133-201 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 069b4dd16cdc4b73a3538d84b7d3f6bc -- Jun 02 21:41:55.179544 ip-10-0-133-201 systemd[1]: Starting Kubernetes Kubelet... Jun 02 21:41:55.606340 ip-10-0-133-201 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 21:41:55.606340 ip-10-0-133-201 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 02 21:41:55.606340 ip-10-0-133-201 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 21:41:55.606340 ip-10-0-133-201 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 02 21:41:55.606340 ip-10-0-133-201 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 21:41:55.607954 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.607848 2574 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 02 21:41:55.610515 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610501 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 21:41:55.610515 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610515 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 21:41:55.610616 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610519 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 21:41:55.610616 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610523 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 21:41:55.610616 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610526 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 21:41:55.610616 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610529 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 21:41:55.610616 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610532 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 21:41:55.610616 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610535 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 21:41:55.610616 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610538 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 21:41:55.610616 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610541 2574 feature_gate.go:328] unrecognized feature gate: Example Jun 02 21:41:55.610616 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610545 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 21:41:55.610616 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610548 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 21:41:55.610616 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610550 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 21:41:55.610616 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610553 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 21:41:55.610616 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610555 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 21:41:55.610616 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610558 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 21:41:55.610616 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610560 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 21:41:55.610616 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610563 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 21:41:55.610616 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610566 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 21:41:55.610616 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610568 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 21:41:55.610616 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610571 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 21:41:55.610616 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610574 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 21:41:55.611080 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610576 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 21:41:55.611080 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610579 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 21:41:55.611080 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610581 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 21:41:55.611080 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610584 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 21:41:55.611080 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610586 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 21:41:55.611080 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610589 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 21:41:55.611080 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610592 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 21:41:55.611080 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610594 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 21:41:55.611080 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610597 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 21:41:55.611080 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610600 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 21:41:55.611080 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610603 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 21:41:55.611080 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610605 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 21:41:55.611080 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610608 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 21:41:55.611080 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610612 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 21:41:55.611080 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610615 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 21:41:55.611080 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610617 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 21:41:55.611080 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610620 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 21:41:55.611080 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610623 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 21:41:55.611080 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610626 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 21:41:55.611577 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610629 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 21:41:55.611577 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610632 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 21:41:55.611577 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610637 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 21:41:55.611577 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610641 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 21:41:55.611577 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610644 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 21:41:55.611577 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610647 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 21:41:55.611577 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610650 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 21:41:55.611577 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610653 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 21:41:55.611577 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610655 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 21:41:55.611577 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610658 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 21:41:55.611577 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610661 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 21:41:55.611577 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610664 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 21:41:55.611577 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610667 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 21:41:55.611577 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610670 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 21:41:55.611577 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610673 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 21:41:55.611577 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610676 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 21:41:55.611577 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610678 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 21:41:55.611577 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610681 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 21:41:55.611577 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610683 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 21:41:55.612063 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610685 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 21:41:55.612063 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610689 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 21:41:55.612063 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610693 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 21:41:55.612063 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610695 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 21:41:55.612063 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610698 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 21:41:55.612063 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610701 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 21:41:55.612063 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610703 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 21:41:55.612063 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610707 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 21:41:55.612063 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610709 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 21:41:55.612063 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610714 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 21:41:55.612063 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610716 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 21:41:55.612063 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610719 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 21:41:55.612063 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610722 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 21:41:55.612063 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610724 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 21:41:55.612063 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610726 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 21:41:55.612063 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610729 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 21:41:55.612063 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610731 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 21:41:55.612063 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610734 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 21:41:55.612063 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610736 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 21:41:55.612542 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610739 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 21:41:55.612542 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610741 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 21:41:55.612542 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610744 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 21:41:55.612542 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610747 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 21:41:55.612542 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610749 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 21:41:55.612542 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610751 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 21:41:55.612542 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.610754 2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 21:41:55.612542 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611688 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 21:41:55.612542 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611694 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 21:41:55.612542 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611697 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 21:41:55.612542 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611700 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 21:41:55.612542 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611703 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 21:41:55.612542 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611706 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 21:41:55.612542 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611709 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 21:41:55.612542 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611711 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 21:41:55.612542 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611714 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 21:41:55.612542 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611716 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 21:41:55.612542 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611719 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 21:41:55.612542 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611722 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 21:41:55.612986 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611724 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 21:41:55.612986 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611728 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 21:41:55.612986 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611731 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 21:41:55.612986 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611734 2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 21:41:55.612986 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611738 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 21:41:55.612986 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611742 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 21:41:55.612986 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611746 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 21:41:55.612986 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611748 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 21:41:55.612986 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611751 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 21:41:55.612986 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611754 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 21:41:55.612986 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611756 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 21:41:55.612986 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611759 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 21:41:55.612986 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611761 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 21:41:55.612986 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611764 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 21:41:55.612986 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611766 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 21:41:55.612986 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611769 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 21:41:55.612986 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611771 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 21:41:55.612986 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611774 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 21:41:55.612986 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611776 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 21:41:55.613474 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611779 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 21:41:55.613474 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611782 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 21:41:55.613474 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611785 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 21:41:55.613474 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611788 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 21:41:55.613474 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611790 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 21:41:55.613474 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611793 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 21:41:55.613474 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611796 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 21:41:55.613474 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611798 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 21:41:55.613474 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611801 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 21:41:55.613474 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611803 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 21:41:55.613474 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611806 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 21:41:55.613474 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611809 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 21:41:55.613474 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611811 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 21:41:55.613474 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611814 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 21:41:55.613474 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611817 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 21:41:55.613474 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611819 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 21:41:55.613474 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611822 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 21:41:55.613474 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611825 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 21:41:55.613474 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611827 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 21:41:55.613474 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611830 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 21:41:55.613970 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611833 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 21:41:55.613970 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611835 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 21:41:55.613970 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611838 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 21:41:55.613970 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611840 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 21:41:55.613970 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611843 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 21:41:55.613970 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611847 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 21:41:55.613970 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611850 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 21:41:55.613970 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611853 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 21:41:55.613970 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611856 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 21:41:55.613970 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611858 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 21:41:55.613970 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611861 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 21:41:55.613970 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611864 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 21:41:55.613970 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611866 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 21:41:55.613970 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611869 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 21:41:55.613970 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611872 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 21:41:55.613970 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611875 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 21:41:55.613970 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611877 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 21:41:55.613970 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611880 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 21:41:55.613970 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611883 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 21:41:55.614489 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611886 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 21:41:55.614489 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611888 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 21:41:55.614489 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611891 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 21:41:55.614489 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611893 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 21:41:55.614489 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611896 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 21:41:55.614489 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611898 2574 feature_gate.go:328] unrecognized feature gate: Example Jun 02 21:41:55.614489 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611901 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 21:41:55.614489 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611904 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 21:41:55.614489 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611907 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 21:41:55.614489 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611909 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 21:41:55.614489 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611912 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 21:41:55.614489 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611915 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 21:41:55.614489 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611917 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 21:41:55.614489 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611919 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 21:41:55.614489 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611922 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 21:41:55.614489 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.611925 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 21:41:55.614489 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.611983 2574 flags.go:64] FLAG: --address="0.0.0.0" Jun 02 21:41:55.614489 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.611990 2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 02 21:41:55.614489 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.611998 2574 flags.go:64] FLAG: --anonymous-auth="true" Jun 02 21:41:55.614489 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612002 2574 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 02 21:41:55.614489 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612008 2574 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 02 21:41:55.614989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612013 2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 02 21:41:55.614989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612020 2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 02 21:41:55.614989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612026 2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 02 21:41:55.614989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612031 2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 02 21:41:55.614989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612035 2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 02 21:41:55.614989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612039 2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 02 21:41:55.614989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612042 2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 02 21:41:55.614989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612045 2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 02 21:41:55.614989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612049 2574 flags.go:64] FLAG: --cgroup-root="" Jun 02 21:41:55.614989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612051 2574 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 02 21:41:55.614989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612054 2574 flags.go:64] FLAG: --client-ca-file="" Jun 02 21:41:55.614989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612057 2574 flags.go:64] FLAG: --cloud-config="" Jun 02 21:41:55.614989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612060 2574 flags.go:64] FLAG: --cloud-provider="external" Jun 02 21:41:55.614989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612063 2574 flags.go:64] FLAG: --cluster-dns="[]" Jun 02 21:41:55.614989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612067 2574 flags.go:64] FLAG: --cluster-domain="" Jun 02 21:41:55.614989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612070 2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 02 21:41:55.614989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612073 2574 flags.go:64] FLAG: --config-dir="" Jun 02 21:41:55.614989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612076 2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 02 21:41:55.614989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612080 2574 flags.go:64] FLAG: --container-log-max-files="5" Jun 02 21:41:55.614989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612084 2574 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 02 21:41:55.614989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612087 2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 02 21:41:55.614989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612090 2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 02 21:41:55.614989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612093 2574 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 02 21:41:55.614989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612096 2574 flags.go:64] FLAG: --contention-profiling="false" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612099 2574 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612102 2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612106 2574 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612108 2574 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612112 2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612115 2574 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612118 2574 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612121 2574 flags.go:64] FLAG: --enable-load-reader="false" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612124 2574 flags.go:64] FLAG: --enable-server="true" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612127 2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612130 2574 flags.go:64] FLAG: --event-burst="100" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612134 2574 flags.go:64] FLAG: --event-qps="50" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612137 2574 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612140 2574 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612143 2574 flags.go:64] FLAG: --eviction-hard="" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612146 2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612149 2574 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612153 2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612156 2574 flags.go:64] FLAG: --eviction-soft="" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612158 2574 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612161 2574 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612164 2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612167 2574 flags.go:64] FLAG: --experimental-mounter-path="" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612170 2574 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 02 21:41:55.615586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612173 2574 flags.go:64] FLAG: --fail-swap-on="true" Jun 02 21:41:55.616180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612176 2574 flags.go:64] FLAG: --feature-gates="" Jun 02 21:41:55.616180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612181 2574 flags.go:64] FLAG: --file-check-frequency="20s" Jun 02 21:41:55.616180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612183 2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 02 21:41:55.616180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612187 2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 02 21:41:55.616180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612190 2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 02 21:41:55.616180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612193 2574 flags.go:64] FLAG: --healthz-port="10248" Jun 02 21:41:55.616180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612196 2574 flags.go:64] FLAG: --help="false" Jun 02 21:41:55.616180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612205 2574 flags.go:64] FLAG: --hostname-override="ip-10-0-133-201.ec2.internal" Jun 02 21:41:55.616180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612209 2574 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 02 21:41:55.616180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612212 2574 flags.go:64] FLAG: --http-check-frequency="20s" Jun 02 21:41:55.616180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612215 2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 02 21:41:55.616180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612218 2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 02 21:41:55.616180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612221 2574 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 02 21:41:55.616180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612224 2574 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 02 21:41:55.616180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612227 2574 flags.go:64] FLAG: --image-service-endpoint="" Jun 02 21:41:55.616180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612230 2574 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 02 21:41:55.616180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612233 2574 flags.go:64] FLAG: --kube-api-burst="100" Jun 02 21:41:55.616180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612236 2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 02 21:41:55.616180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612239 2574 flags.go:64] FLAG: --kube-api-qps="50" Jun 02 21:41:55.616180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612242 2574 flags.go:64] FLAG: --kube-reserved="" Jun 02 21:41:55.616180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612245 2574 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 02 21:41:55.616180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612248 2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 02 21:41:55.616180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612251 2574 flags.go:64] FLAG: --kubelet-cgroups="" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612254 2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612257 2574 flags.go:64] FLAG: --lock-file="" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612259 2574 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612262 2574 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612265 2574 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612275 2574 flags.go:64] FLAG: --log-json-split-stream="false" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612278 2574 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612293 2574 flags.go:64] FLAG: --log-text-split-stream="false" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612296 2574 flags.go:64] FLAG: --logging-format="text" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612299 2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612303 2574 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612306 2574 flags.go:64] FLAG: --manifest-url="" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612309 2574 flags.go:64] FLAG: --manifest-url-header="" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612314 2574 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612317 2574 flags.go:64] FLAG: --max-open-files="1000000" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612321 2574 flags.go:64] FLAG: --max-pods="110" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612324 2574 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612328 2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612331 2574 flags.go:64] FLAG: --memory-manager-policy="None" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612334 2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612337 2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612340 2574 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612343 2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612351 2574 flags.go:64] FLAG: --node-status-max-images="50" Jun 02 21:41:55.616743 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612354 2574 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612361 2574 flags.go:64] FLAG: --oom-score-adj="-999" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612365 2574 flags.go:64] FLAG: --pod-cidr="" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612368 2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612373 2574 flags.go:64] FLAG: --pod-manifest-path="" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612376 2574 flags.go:64] FLAG: --pod-max-pids="-1" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612379 2574 flags.go:64] FLAG: --pods-per-core="0" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612382 2574 flags.go:64] FLAG: --port="10250" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612385 2574 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612388 2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b60e5e185c65ce08" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612391 2574 flags.go:64] FLAG: --qos-reserved="" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612394 2574 flags.go:64] FLAG: --read-only-port="10255" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612397 2574 flags.go:64] FLAG: --register-node="true" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612400 2574 flags.go:64] FLAG: --register-schedulable="true" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612403 2574 flags.go:64] FLAG: --register-with-taints="" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612411 2574 flags.go:64] FLAG: --registry-burst="10" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612414 2574 flags.go:64] FLAG: --registry-qps="5" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612417 2574 flags.go:64] FLAG: --reserved-cpus="" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612420 2574 flags.go:64] FLAG: --reserved-memory="" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612424 2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612427 2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612430 2574 flags.go:64] FLAG: --rotate-certificates="false" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612433 2574 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612436 2574 flags.go:64] FLAG: --runonce="false" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612439 2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 02 21:41:55.617375 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612442 2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612445 2574 flags.go:64] FLAG: --seccomp-default="false" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612448 2574 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612451 2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612454 2574 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612457 2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612460 2574 flags.go:64] FLAG: --storage-driver-password="root" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612463 2574 flags.go:64] FLAG: --storage-driver-secure="false" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612466 2574 flags.go:64] FLAG: --storage-driver-table="stats" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612470 2574 flags.go:64] FLAG: --storage-driver-user="root" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612473 2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612475 2574 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612478 2574 flags.go:64] FLAG: --system-cgroups="" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612481 2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612486 2574 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612489 2574 flags.go:64] FLAG: --tls-cert-file="" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612492 2574 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612495 2574 flags.go:64] FLAG: --tls-min-version="" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612499 2574 flags.go:64] FLAG: --tls-private-key-file="" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612501 2574 flags.go:64] FLAG: --topology-manager-policy="none" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612504 2574 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612507 2574 flags.go:64] FLAG: --topology-manager-scope="container" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612510 2574 flags.go:64] FLAG: --v="2" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612514 2574 flags.go:64] FLAG: --version="false" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612518 2574 flags.go:64] FLAG: --vmodule="" Jun 02 21:41:55.617985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612522 2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 02 21:41:55.618723 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.612525 2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 02 21:41:55.618723 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612614 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 21:41:55.618723 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612618 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 21:41:55.618723 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612621 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 21:41:55.618723 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612623 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 21:41:55.618723 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612626 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 21:41:55.618723 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612629 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 21:41:55.618723 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612631 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 21:41:55.618723 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612634 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 21:41:55.618723 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612637 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 21:41:55.618723 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612640 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 21:41:55.618723 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612643 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 21:41:55.618723 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612645 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 21:41:55.618723 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612648 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 21:41:55.618723 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612650 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 21:41:55.618723 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612655 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 21:41:55.618723 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612658 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 21:41:55.618723 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612660 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 21:41:55.618723 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612663 2574 feature_gate.go:328] unrecognized feature gate: Example Jun 02 21:41:55.618723 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612665 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 21:41:55.619426 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612668 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 21:41:55.619426 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612670 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 21:41:55.619426 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612673 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 21:41:55.619426 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612676 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 21:41:55.619426 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612678 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 21:41:55.619426 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612682 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 21:41:55.619426 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612686 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 21:41:55.619426 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612688 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 21:41:55.619426 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612691 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 21:41:55.619426 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612694 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 21:41:55.619426 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612696 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 21:41:55.619426 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612701 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 21:41:55.619426 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612703 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 21:41:55.619426 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612706 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 21:41:55.619426 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612709 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 21:41:55.619426 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612712 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 21:41:55.619426 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612714 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 21:41:55.619426 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612717 2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 21:41:55.619426 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612720 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 21:41:55.619426 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612722 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 21:41:55.619930 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612725 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 21:41:55.619930 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612728 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 21:41:55.619930 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612731 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 21:41:55.619930 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612733 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 21:41:55.619930 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612736 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 21:41:55.619930 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612738 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 21:41:55.619930 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612741 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 21:41:55.619930 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612746 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 21:41:55.619930 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612750 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 21:41:55.619930 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612752 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 21:41:55.619930 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612755 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 21:41:55.619930 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612758 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 21:41:55.619930 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612761 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 21:41:55.619930 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612763 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 21:41:55.619930 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612766 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 21:41:55.619930 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612768 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 21:41:55.619930 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612771 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 21:41:55.619930 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612774 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 21:41:55.619930 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612776 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 21:41:55.619930 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612778 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 21:41:55.620449 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612781 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 21:41:55.620449 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612784 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 21:41:55.620449 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612786 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 21:41:55.620449 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612790 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 21:41:55.620449 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612792 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 21:41:55.620449 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612795 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 21:41:55.620449 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612798 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 21:41:55.620449 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612801 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 21:41:55.620449 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612803 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 21:41:55.620449 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612806 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 21:41:55.620449 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612808 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 21:41:55.620449 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612811 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 21:41:55.620449 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612813 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 21:41:55.620449 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612816 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 21:41:55.620449 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612818 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 21:41:55.620449 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612821 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 21:41:55.620449 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612824 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 21:41:55.620449 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612826 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 21:41:55.620449 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612828 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 21:41:55.620918 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612832 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 21:41:55.620918 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612835 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 21:41:55.620918 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612837 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 21:41:55.620918 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612840 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 21:41:55.620918 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612843 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 21:41:55.620918 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612845 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 21:41:55.620918 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612848 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 21:41:55.620918 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.612851 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 21:41:55.620918 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.613426 2574 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 21:41:55.621676 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.621658 2574 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 02 21:41:55.621709 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.621677 2574 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 02 21:41:55.621737 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621725 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 21:41:55.621737 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621730 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 21:41:55.621737 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621733 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 21:41:55.621737 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621737 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 21:41:55.621839 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621740 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 21:41:55.621839 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621743 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 21:41:55.621839 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621746 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 21:41:55.621839 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621750 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 21:41:55.621839 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621753 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 21:41:55.621839 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621757 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 21:41:55.621839 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621761 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 21:41:55.621839 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621764 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 21:41:55.621839 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621767 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 21:41:55.621839 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621770 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 21:41:55.621839 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621773 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 21:41:55.621839 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621775 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 21:41:55.621839 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621778 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 21:41:55.621839 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621781 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 21:41:55.621839 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621783 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 21:41:55.621839 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621786 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 21:41:55.621839 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621788 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 21:41:55.621839 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621791 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 21:41:55.621839 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621794 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 21:41:55.622304 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621797 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 21:41:55.622304 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621800 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 21:41:55.622304 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621802 2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 21:41:55.622304 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621805 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 21:41:55.622304 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621807 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 21:41:55.622304 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621810 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 21:41:55.622304 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621813 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 21:41:55.622304 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621815 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 21:41:55.622304 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621818 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 21:41:55.622304 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621821 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 21:41:55.622304 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621823 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 21:41:55.622304 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621826 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 21:41:55.622304 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621828 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 21:41:55.622304 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621831 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 21:41:55.622304 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621834 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 21:41:55.622304 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621837 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 21:41:55.622304 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621840 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 21:41:55.622304 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621843 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 21:41:55.622304 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621846 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 21:41:55.622304 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621848 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 21:41:55.622789 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621852 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 21:41:55.622789 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621854 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 21:41:55.622789 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621857 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 21:41:55.622789 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621860 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 21:41:55.622789 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621863 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 21:41:55.622789 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621866 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 21:41:55.622789 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621869 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 21:41:55.622789 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621871 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 21:41:55.622789 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621874 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 21:41:55.622789 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621876 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 21:41:55.622789 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621879 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 21:41:55.622789 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621882 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 21:41:55.622789 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621885 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 21:41:55.622789 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621887 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 21:41:55.622789 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621890 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 21:41:55.622789 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621892 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 21:41:55.622789 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621895 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 21:41:55.622789 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621898 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 21:41:55.622789 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621900 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 21:41:55.623303 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621904 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 21:41:55.623303 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621909 2574 feature_gate.go:328] unrecognized feature gate: Example Jun 02 21:41:55.623303 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621911 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 21:41:55.623303 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621914 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 21:41:55.623303 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621917 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 21:41:55.623303 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621919 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 21:41:55.623303 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621922 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 21:41:55.623303 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621925 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 21:41:55.623303 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621928 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 21:41:55.623303 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621930 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 21:41:55.623303 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621933 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 21:41:55.623303 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621936 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 21:41:55.623303 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621938 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 21:41:55.623303 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621941 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 21:41:55.623303 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621945 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 21:41:55.623303 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621948 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 21:41:55.623303 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621950 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 21:41:55.623303 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621953 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 21:41:55.623303 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621955 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 21:41:55.623303 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621958 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 21:41:55.623796 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621960 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 21:41:55.623796 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621963 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 21:41:55.623796 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621965 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 21:41:55.623796 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.621968 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 21:41:55.623796 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.621973 2574 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 21:41:55.623796 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622064 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 21:41:55.623796 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622069 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 21:41:55.623796 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622072 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 21:41:55.623796 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622075 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 21:41:55.623796 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622077 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 21:41:55.623796 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622080 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 21:41:55.623796 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622083 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 21:41:55.623796 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622085 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 21:41:55.623796 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622088 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 21:41:55.623796 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622091 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 21:41:55.623796 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622093 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 21:41:55.624192 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622097 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 21:41:55.624192 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622101 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 21:41:55.624192 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622103 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 21:41:55.624192 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622107 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 21:41:55.624192 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622110 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 21:41:55.624192 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622112 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 21:41:55.624192 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622115 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 21:41:55.624192 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622117 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 21:41:55.624192 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622120 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 21:41:55.624192 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622123 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 21:41:55.624192 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622126 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 21:41:55.624192 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622129 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 21:41:55.624192 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622132 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 21:41:55.624192 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622135 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 21:41:55.624192 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622138 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 21:41:55.624192 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622140 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 21:41:55.624192 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622143 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 21:41:55.624192 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622145 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 21:41:55.624192 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622148 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 21:41:55.624678 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622151 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 21:41:55.624678 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622154 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 21:41:55.624678 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622156 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 21:41:55.624678 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622159 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 21:41:55.624678 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622161 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 21:41:55.624678 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622164 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 21:41:55.624678 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622166 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 21:41:55.624678 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622169 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 21:41:55.624678 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622172 2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 21:41:55.624678 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622174 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 21:41:55.624678 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622177 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 21:41:55.624678 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622179 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 21:41:55.624678 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622182 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 21:41:55.624678 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622185 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 21:41:55.624678 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622188 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 21:41:55.624678 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622190 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 21:41:55.624678 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622193 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 21:41:55.624678 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622195 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 21:41:55.624678 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622198 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 21:41:55.624678 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622200 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 21:41:55.625163 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622203 2574 feature_gate.go:328] unrecognized feature gate: Example Jun 02 21:41:55.625163 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622206 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 21:41:55.625163 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622208 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 21:41:55.625163 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622211 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 21:41:55.625163 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622213 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 21:41:55.625163 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622218 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 21:41:55.625163 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622222 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 21:41:55.625163 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622224 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 21:41:55.625163 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622227 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 21:41:55.625163 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622229 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 21:41:55.625163 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622232 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 21:41:55.625163 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622234 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 21:41:55.625163 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622237 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 21:41:55.625163 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622240 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 21:41:55.625163 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622243 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 21:41:55.625163 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622245 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 21:41:55.625163 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622248 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 21:41:55.625163 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622250 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 21:41:55.625163 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622253 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 21:41:55.625720 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622255 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 21:41:55.625720 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622258 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 21:41:55.625720 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622260 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 21:41:55.625720 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622263 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 21:41:55.625720 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622265 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 21:41:55.625720 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622268 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 21:41:55.625720 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622271 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 21:41:55.625720 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622273 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 21:41:55.625720 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622276 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 21:41:55.625720 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622279 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 21:41:55.625720 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622296 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 21:41:55.625720 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622299 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 21:41:55.625720 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622302 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 21:41:55.625720 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622305 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 21:41:55.625720 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622307 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 21:41:55.625720 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622310 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 21:41:55.625720 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:55.622313 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 21:41:55.626127 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.622318 2574 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 21:41:55.626127 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.623039 2574 server.go:962] "Client rotation is on, will bootstrap in background" Jun 02 21:41:55.626183 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.626169 2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 02 21:41:55.626999 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.626987 2574 server.go:1019] "Starting client certificate rotation" Jun 02 21:41:55.627099 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.627082 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 21:41:55.627128 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.627117 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 21:41:55.650894 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.650877 2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 21:41:55.651984 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.651964 2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 21:41:55.664263 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.664243 2574 log.go:25] "Validated CRI v1 runtime API" Jun 02 21:41:55.670409 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.670393 2574 log.go:25] "Validated CRI v1 image API" Jun 02 21:41:55.673013 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.672995 2574 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 02 21:41:55.677627 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.677605 2574 fs.go:135] Filesystem UUIDs: map[195f715e-b7b4-4477-b6dd-898b7bede6ad:/dev/nvme0n1p3 61100967-4488-462a-bfa7-0a940eb41bdb:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jun 02 21:41:55.677724 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.677625 2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 02 21:41:55.681843 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.681715 2574 manager.go:217] Machine: {Timestamp:2026-06-02 21:41:55.681186205 +0000 UTC m=+0.386196143 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099175 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec243f3cb6eca888a88a107340c35caf SystemUUID:ec243f3c-b6ec-a888-a88a-107340c35caf BootID:069b4dd1-6cdc-4b73-a353-8d84b7d3f6bc Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:dd:db:87:8c:33 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:dd:db:87:8c:33 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:c7:0c:55:5f:e3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 02 21:41:55.682603 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.682589 2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 02 21:41:55.682702 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.682687 2574 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 02 21:41:55.683573 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.683556 2574 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 21:41:55.684568 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.684545 2574 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 02 21:41:55.684695 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.684569 2574 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-201.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 02 21:41:55.684744 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.684705 2574 topology_manager.go:138] "Creating topology manager with none policy" Jun 02 21:41:55.684744 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.684714 2574 container_manager_linux.go:306] "Creating device plugin manager" Jun 02 21:41:55.684744 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.684721 2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 21:41:55.685637 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.685625 2574 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 21:41:55.686979 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.686968 2574 state_mem.go:36] "Initialized new in-memory state store" Jun 02 21:41:55.687096 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.687087 2574 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 02 21:41:55.689509 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.689499 2574 kubelet.go:491] "Attempting to sync node with API server" Jun 02 21:41:55.689555 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.689514 2574 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 02 21:41:55.689555 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.689533 2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 02 21:41:55.689555 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.689543 2574 kubelet.go:397] "Adding apiserver pod source" Jun 02 21:41:55.689555 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.689554 2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 02 21:41:55.690543 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.690530 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 21:41:55.690594 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.690555 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 21:41:55.693601 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.693580 2574 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 02 21:41:55.695228 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.695199 2574 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 02 21:41:55.696785 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.696774 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 02 21:41:55.696823 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.696791 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 02 21:41:55.696823 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.696798 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 02 21:41:55.696823 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.696803 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 02 21:41:55.696823 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.696814 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 02 21:41:55.696823 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.696820 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 02 21:41:55.696952 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.696825 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 02 21:41:55.696952 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.696840 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 02 21:41:55.696952 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.696848 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 02 21:41:55.696952 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.696854 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 02 21:41:55.696952 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.696868 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 02 21:41:55.696952 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.696874 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 02 21:41:55.697663 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.697653 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 02 21:41:55.697663 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.697662 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 02 21:41:55.700588 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.700574 2574 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 02 21:41:55.700668 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.700613 2574 server.go:1295] "Started kubelet" Jun 02 21:41:55.700724 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.700658 2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 02 21:41:55.700771 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.700710 2574 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 02 21:41:55.700818 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.700765 2574 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 02 21:41:55.701374 ip-10-0-133-201 systemd[1]: Started Kubernetes Kubelet. Jun 02 21:41:55.704566 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.704544 2574 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 02 21:41:55.704868 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.704852 2574 server.go:317] "Adding debug handlers to kubelet server" Jun 02 21:41:55.705209 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:55.705181 2574 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-201.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 02 21:41:55.705378 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.705365 2574 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-201.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 02 21:41:55.705597 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:55.705574 2574 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 02 21:41:55.709354 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.708653 2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 02 21:41:55.709354 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.708687 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 02 21:41:55.709354 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.708848 2574 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 02 21:41:55.709354 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.708856 2574 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 02 21:41:55.709354 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.708960 2574 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 02 21:41:55.709354 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.709008 2574 reconstruct.go:97] "Volume reconstruction finished" Jun 02 21:41:55.709354 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.709013 2574 reconciler.go:26] "Reconciler: start to sync state" Jun 02 21:41:55.710226 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:55.710161 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-201.ec2.internal\" not found" Jun 02 21:41:55.710329 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.710235 2574 factory.go:55] Registering systemd factory Jun 02 21:41:55.710329 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.710269 2574 factory.go:223] Registration of the systemd container factory successfully Jun 02 21:41:55.711600 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.710525 2574 factory.go:153] Registering CRI-O factory Jun 02 21:41:55.711600 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.710543 2574 factory.go:223] Registration of the crio container factory successfully Jun 02 21:41:55.711600 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.710595 2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 02 21:41:55.711600 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.710620 2574 factory.go:103] Registering Raw factory Jun 02 21:41:55.711600 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.710637 2574 manager.go:1196] Started watching for new ooms in manager Jun 02 21:41:55.711600 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.711192 2574 manager.go:319] Starting recovery of all containers Jun 02 21:41:55.715534 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:55.715511 2574 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 02 21:41:55.717947 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.717906 2574 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 02 21:41:55.720269 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.720254 2574 manager.go:324] Recovery completed Jun 02 21:41:55.721859 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:55.721708 2574 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-201.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 02 21:41:55.721859 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:55.721722 2574 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 02 21:41:55.722597 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:55.721806 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-201.ec2.internal.18b56193ef77bbe5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-201.ec2.internal,UID:ip-10-0-133-201.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-201.ec2.internal,},FirstTimestamp:2026-06-02 21:41:55.700587493 +0000 UTC m=+0.405597431,LastTimestamp:2026-06-02 21:41:55.700587493 +0000 UTC m=+0.405597431,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-201.ec2.internal,}" Jun 02 21:41:55.724663 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.724649 2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 21:41:55.726762 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.726742 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-201.ec2.internal" event="NodeHasSufficientMemory" Jun 02 21:41:55.726844 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.726774 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-201.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 21:41:55.726844 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.726784 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-201.ec2.internal" event="NodeHasSufficientPID" Jun 02 21:41:55.727251 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.727238 2574 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 02 21:41:55.727251 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.727249 2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 02 21:41:55.727367 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.727264 2574 state_mem.go:36] "Initialized new in-memory state store" Jun 02 21:41:55.729368 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.729357 2574 policy_none.go:49] "None policy: Start" Jun 02 21:41:55.729413 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.729373 2574 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 02 21:41:55.729413 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.729383 2574 state_mem.go:35] "Initializing new in-memory state store" Jun 02 21:41:55.732465 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:55.732403 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-201.ec2.internal.18b56193f10719c3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-201.ec2.internal,UID:ip-10-0-133-201.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-201.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-201.ec2.internal,},FirstTimestamp:2026-06-02 21:41:55.726760387 +0000 UTC m=+0.431770335,LastTimestamp:2026-06-02 21:41:55.726760387 +0000 UTC m=+0.431770335,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-201.ec2.internal,}" Jun 02 21:41:55.744818 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:55.744760 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-201.ec2.internal.18b56193f10762a7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-201.ec2.internal,UID:ip-10-0-133-201.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-201.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-201.ec2.internal,},FirstTimestamp:2026-06-02 21:41:55.726779047 +0000 UTC m=+0.431788985,LastTimestamp:2026-06-02 21:41:55.726779047 +0000 UTC m=+0.431788985,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-201.ec2.internal,}" Jun 02 21:41:55.755175 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:55.755115 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-201.ec2.internal.18b56193f10786fd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-201.ec2.internal,UID:ip-10-0-133-201.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-201.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-201.ec2.internal,},FirstTimestamp:2026-06-02 21:41:55.726788349 +0000 UTC m=+0.431798286,LastTimestamp:2026-06-02 21:41:55.726788349 +0000 UTC m=+0.431798286,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-201.ec2.internal,}" Jun 02 21:41:55.767123 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.767106 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pvfm5" Jun 02 21:41:55.770847 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.770834 2574 manager.go:341] "Starting Device Plugin manager" Jun 02 21:41:55.784815 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:55.770871 2574 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 02 21:41:55.784815 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.770881 2574 server.go:85] "Starting device plugin registration server" Jun 02 21:41:55.784815 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.771075 2574 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 02 21:41:55.784815 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.771089 2574 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 02 21:41:55.784815 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.771198 2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 02 21:41:55.784815 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.771272 2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 02 21:41:55.784815 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.771303 2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 02 21:41:55.784815 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:55.771851 2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 02 21:41:55.784815 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:55.771888 2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-201.ec2.internal\" not found" Jun 02 21:41:55.784815 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:55.784184 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-201.ec2.internal.18b56193f3c6da8a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-201.ec2.internal,UID:ip-10-0-133-201.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-133-201.ec2.internal,},FirstTimestamp:2026-06-02 21:41:55.772881546 +0000 UTC m=+0.477891471,LastTimestamp:2026-06-02 21:41:55.772881546 +0000 UTC m=+0.477891471,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-201.ec2.internal,}" Jun 02 21:41:55.786073 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.786057 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pvfm5" Jun 02 21:41:55.842487 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.842467 2574 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 02 21:41:55.842569 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.842492 2574 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 02 21:41:55.842569 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.842507 2574 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 02 21:41:55.842569 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.842513 2574 kubelet.go:2451] "Starting kubelet main sync loop" Jun 02 21:41:55.842569 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:55.842547 2574 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 02 21:41:55.852391 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.852371 2574 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 21:41:55.871934 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.871891 2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 21:41:55.872605 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.872588 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-201.ec2.internal" event="NodeHasSufficientMemory" Jun 02 21:41:55.872673 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.872615 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-201.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 21:41:55.872673 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.872625 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-201.ec2.internal" event="NodeHasSufficientPID" Jun 02 21:41:55.872673 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.872646 2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-201.ec2.internal" Jun 02 21:41:55.879611 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.879597 2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-201.ec2.internal" Jun 02 21:41:55.879685 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:55.879616 2574 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-201.ec2.internal\": node \"ip-10-0-133-201.ec2.internal\" not found" Jun 02 21:41:55.896063 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:55.896043 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-201.ec2.internal\" not found" Jun 02 21:41:55.943479 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.943448 2574 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-201.ec2.internal"] Jun 02 21:41:55.943544 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.943515 2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 21:41:55.944252 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.944239 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-201.ec2.internal" event="NodeHasSufficientMemory" Jun 02 21:41:55.944356 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.944264 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-201.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 21:41:55.944356 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.944322 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-201.ec2.internal" event="NodeHasSufficientPID" Jun 02 21:41:55.945383 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.945370 2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 21:41:55.945980 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.945965 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.ec2.internal" Jun 02 21:41:55.946045 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.945998 2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 21:41:55.946614 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.946598 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-201.ec2.internal" event="NodeHasSufficientMemory" Jun 02 21:41:55.946614 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.946609 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-201.ec2.internal" event="NodeHasSufficientMemory" Jun 02 21:41:55.946614 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.946622 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-201.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 21:41:55.946761 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.946632 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-201.ec2.internal" event="NodeHasSufficientPID" Jun 02 21:41:55.946761 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.946633 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-201.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 21:41:55.946761 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.946645 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-201.ec2.internal" event="NodeHasSufficientPID" Jun 02 21:41:55.947600 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.947588 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.ec2.internal" Jun 02 21:41:55.947647 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.947612 2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 21:41:55.948242 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.948225 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-201.ec2.internal" event="NodeHasSufficientMemory" Jun 02 21:41:55.948335 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.948251 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-201.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 21:41:55.948335 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:55.948265 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-201.ec2.internal" event="NodeHasSufficientPID" Jun 02 21:41:55.977878 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:55.977854 2574 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-201.ec2.internal\" not found" node="ip-10-0-133-201.ec2.internal" Jun 02 21:41:55.982101 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:55.982084 2574 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-201.ec2.internal\" not found" node="ip-10-0-133-201.ec2.internal" Jun 02 21:41:55.996421 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:55.996405 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-201.ec2.internal\" not found" Jun 02 21:41:56.010552 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.010538 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c40a125b2a72f53c0ffc8fa9550c611b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-201.ec2.internal\" (UID: \"c40a125b2a72f53c0ffc8fa9550c611b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.ec2.internal" Jun 02 21:41:56.010606 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.010560 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c40a125b2a72f53c0ffc8fa9550c611b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-201.ec2.internal\" (UID: \"c40a125b2a72f53c0ffc8fa9550c611b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.ec2.internal" Jun 02 21:41:56.010606 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.010578 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ea608c6548a1e6295267e8797334485d-config\") pod \"kube-apiserver-proxy-ip-10-0-133-201.ec2.internal\" (UID: \"ea608c6548a1e6295267e8797334485d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.ec2.internal" Jun 02 21:41:56.097152 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:56.097090 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-201.ec2.internal\" not found" Jun 02 21:41:56.111452 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.111430 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ea608c6548a1e6295267e8797334485d-config\") pod \"kube-apiserver-proxy-ip-10-0-133-201.ec2.internal\" (UID: \"ea608c6548a1e6295267e8797334485d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.ec2.internal" Jun 02 21:41:56.111525 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.111458 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c40a125b2a72f53c0ffc8fa9550c611b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-201.ec2.internal\" (UID: \"c40a125b2a72f53c0ffc8fa9550c611b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.ec2.internal" Jun 02 21:41:56.111525 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.111489 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c40a125b2a72f53c0ffc8fa9550c611b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-201.ec2.internal\" (UID: \"c40a125b2a72f53c0ffc8fa9550c611b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.ec2.internal" Jun 02 21:41:56.111585 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.111534 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c40a125b2a72f53c0ffc8fa9550c611b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-201.ec2.internal\" (UID: \"c40a125b2a72f53c0ffc8fa9550c611b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.ec2.internal" Jun 02 21:41:56.111616 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.111586 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ea608c6548a1e6295267e8797334485d-config\") pod \"kube-apiserver-proxy-ip-10-0-133-201.ec2.internal\" (UID: \"ea608c6548a1e6295267e8797334485d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.ec2.internal" Jun 02 21:41:56.111616 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.111610 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c40a125b2a72f53c0ffc8fa9550c611b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-201.ec2.internal\" (UID: \"c40a125b2a72f53c0ffc8fa9550c611b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.ec2.internal" Jun 02 21:41:56.197851 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:56.197793 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-201.ec2.internal\" not found" Jun 02 21:41:56.280272 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.280255 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.ec2.internal" Jun 02 21:41:56.284767 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.284753 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.ec2.internal" Jun 02 21:41:56.298347 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:56.298316 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-201.ec2.internal\" not found" Jun 02 21:41:56.398821 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:56.398784 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-201.ec2.internal\" not found" Jun 02 21:41:56.499315 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:56.499247 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-201.ec2.internal\" not found" Jun 02 21:41:56.599818 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:56.599793 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-201.ec2.internal\" not found" Jun 02 21:41:56.627227 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.627206 2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 02 21:41:56.627624 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.627348 2574 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 02 21:41:56.700833 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:56.700808 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-201.ec2.internal\" not found" Jun 02 21:41:56.708989 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.708968 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 02 21:41:56.745087 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.745059 2574 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 21:41:56.751440 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.751394 2574 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 21:41:56.770441 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.770420 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bt27l" Jun 02 21:41:56.770683 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:56.770661 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc40a125b2a72f53c0ffc8fa9550c611b.slice/crio-5876b7daa09cf73bf12f9a0529059bd42cd42d2d9e9865fed6ca9b80b29e6116 WatchSource:0}: Error finding container 5876b7daa09cf73bf12f9a0529059bd42cd42d2d9e9865fed6ca9b80b29e6116: Status 404 returned error can't find the container with id 5876b7daa09cf73bf12f9a0529059bd42cd42d2d9e9865fed6ca9b80b29e6116 Jun 02 21:41:56.770975 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:56.770958 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea608c6548a1e6295267e8797334485d.slice/crio-c47502a92e312ce2e5002a49a0d7781a4a9045975ce2233159d9f4c4bd829f80 WatchSource:0}: Error finding container c47502a92e312ce2e5002a49a0d7781a4a9045975ce2233159d9f4c4bd829f80: Status 404 returned error can't find the container with id c47502a92e312ce2e5002a49a0d7781a4a9045975ce2233159d9f4c4bd829f80 Jun 02 21:41:56.775706 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.775692 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 21:41:56.779985 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.779967 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bt27l" Jun 02 21:41:56.788005 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.787982 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-01 21:36:55 +0000 UTC" deadline="2028-01-17 16:14:33.158854352 +0000 UTC" Jun 02 21:41:56.788005 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.788004 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14250h32m36.370853171s" Jun 02 21:41:56.809983 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.809954 2574 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.ec2.internal" Jun 02 21:41:56.822624 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.821678 2574 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 21:41:56.824549 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.824535 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 21:41:56.824629 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.824619 2574 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.ec2.internal" Jun 02 21:41:56.833760 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.833746 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 21:41:56.845542 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.845494 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.ec2.internal" event={"ID":"ea608c6548a1e6295267e8797334485d","Type":"ContainerStarted","Data":"c47502a92e312ce2e5002a49a0d7781a4a9045975ce2233159d9f4c4bd829f80"} Jun 02 21:41:56.846460 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.846440 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.ec2.internal" event={"ID":"c40a125b2a72f53c0ffc8fa9550c611b","Type":"ContainerStarted","Data":"5876b7daa09cf73bf12f9a0529059bd42cd42d2d9e9865fed6ca9b80b29e6116"} Jun 02 21:41:56.872722 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:56.872706 2574 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 21:41:57.691036 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.691012 2574 apiserver.go:52] "Watching apiserver" Jun 02 21:41:57.698143 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.698118 2574 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 02 21:41:57.700753 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.700727 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-201.ec2.internal","openshift-cluster-node-tuning-operator/tuned-5zvps","openshift-multus/network-metrics-daemon-qw5fs","openshift-network-operator/iptables-alerter-kmtlg","openshift-ovn-kubernetes/ovnkube-node-fddpn","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g","openshift-dns/node-resolver-dpz84","openshift-image-registry/node-ca-5qkfw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.ec2.internal","openshift-multus/multus-additional-cni-plugins-jtthq","openshift-multus/multus-kvgdj","openshift-network-diagnostics/network-check-target-zmvbb","kube-system/konnectivity-agent-hb594"] Jun 02 21:41:57.701461 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.701434 2574 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 21:41:57.702218 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.702202 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" Jun 02 21:41:57.703347 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.703325 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hb594" Jun 02 21:41:57.704384 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.704254 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 02 21:41:57.704384 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.704339 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 02 21:41:57.704384 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.704382 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.704609 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.704519 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-t85c4\"" Jun 02 21:41:57.704863 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.704847 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 02 21:41:57.705254 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.705139 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 02 21:41:57.705254 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.705147 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-sqp44\"" Jun 02 21:41:57.705406 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.705320 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 02 21:41:57.705478 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.705464 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5qkfw" Jun 02 21:41:57.706256 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.706173 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 02 21:41:57.706383 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.706304 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 02 21:41:57.706451 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.706398 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 02 21:41:57.706451 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.706442 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 02 21:41:57.706550 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.706461 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qmpcz\"" Jun 02 21:41:57.707125 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.706991 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 02 21:41:57.707125 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.707094 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 02 21:41:57.707396 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.707369 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 02 21:41:57.707487 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.707413 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qzhx8\"" Jun 02 21:41:57.707598 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.707582 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dpz84" Jun 02 21:41:57.709008 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.708954 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.709651 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.709631 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 02 21:41:57.709798 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.709764 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 02 21:41:57.709876 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.709802 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nl5jm\"" Jun 02 21:41:57.710502 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.710484 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kmtlg" Jun 02 21:41:57.710998 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.710980 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 02 21:41:57.711087 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.711005 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 02 21:41:57.711395 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.711274 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 02 21:41:57.711395 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.711329 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 02 21:41:57.711499 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.711417 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 02 21:41:57.711499 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.711419 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hw8tx\"" Jun 02 21:41:57.711597 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.711523 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 02 21:41:57.711905 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.711861 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.712482 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.712392 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 02 21:41:57.712589 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.712488 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-txjx4\"" Jun 02 21:41:57.713081 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.713060 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 02 21:41:57.713461 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.713441 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 02 21:41:57.713748 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.713727 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 02 21:41:57.713825 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.713802 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:41:57.714096 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:57.714050 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5fs" podUID="03b1bb3a-4c92-4555-b5d4-3aaac39381f5" Jun 02 21:41:57.714804 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.714784 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4z2np\"" Jun 02 21:41:57.714887 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.714814 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 02 21:41:57.716337 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.716319 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:41:57.716443 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:57.716401 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zmvbb" podUID="7510b8a7-e605-4db3-882f-c50c26e954b9" Jun 02 21:41:57.717908 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.717890 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.719798 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.719770 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/efca8e3c-85ec-4437-a3bb-d12e252ca89c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6r77g\" (UID: \"efca8e3c-85ec-4437-a3bb-d12e252ca89c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" Jun 02 21:41:57.719904 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.719809 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/efca8e3c-85ec-4437-a3bb-d12e252ca89c-etc-selinux\") pod \"aws-ebs-csi-driver-node-6r77g\" (UID: \"efca8e3c-85ec-4437-a3bb-d12e252ca89c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" Jun 02 21:41:57.719904 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.719832 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b4b273c1-a82f-4e10-b0ac-671f342af106-hosts-file\") pod \"node-resolver-dpz84\" (UID: \"b4b273c1-a82f-4e10-b0ac-671f342af106\") " pod="openshift-dns/node-resolver-dpz84" Jun 02 21:41:57.719904 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.719853 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-host-run-netns\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.720073 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.719897 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-var-lib-openvswitch\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.720073 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.719914 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 02 21:41:57.720073 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.719940 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d8c7f35d-ee2f-446a-9625-9b8b102a5a9a-iptables-alerter-script\") pod \"iptables-alerter-kmtlg\" (UID: \"d8c7f35d-ee2f-446a-9625-9b8b102a5a9a\") " pod="openshift-network-operator/iptables-alerter-kmtlg" Jun 02 21:41:57.720073 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.719971 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-host-kubelet\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.720073 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.719995 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-systemd-units\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.720073 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720017 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-host-cni-netd\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.720073 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720041 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b82cp\" (UniqueName: \"kubernetes.io/projected/974df3d7-921f-46eb-baa9-e9a37e9d93e9-kube-api-access-b82cp\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.720073 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720066 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-system-cni-dir\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.720393 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720097 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ppv42\"" Jun 02 21:41:57.720393 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720109 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-host-var-lib-cni-bin\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.720393 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720151 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-host-var-lib-kubelet\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.720393 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720175 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-hostroot\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.720393 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720198 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-host-run-ovn-kubernetes\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.720393 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720222 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/974df3d7-921f-46eb-baa9-e9a37e9d93e9-ovnkube-config\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.720393 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720247 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/974df3d7-921f-46eb-baa9-e9a37e9d93e9-ovn-node-metrics-cert\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.720393 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720259 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 02 21:41:57.720393 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720268 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/be818b5e-c71a-45a2-8329-2e79ef4b6196-multus-daemon-config\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.720393 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720310 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-etc-kubernetes\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.720393 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720329 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d8c7f35d-ee2f-446a-9625-9b8b102a5a9a-host-slash\") pod \"iptables-alerter-kmtlg\" (UID: \"d8c7f35d-ee2f-446a-9625-9b8b102a5a9a\") " pod="openshift-network-operator/iptables-alerter-kmtlg" Jun 02 21:41:57.720393 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720344 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/efca8e3c-85ec-4437-a3bb-d12e252ca89c-registration-dir\") pod \"aws-ebs-csi-driver-node-6r77g\" (UID: \"efca8e3c-85ec-4437-a3bb-d12e252ca89c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" Jun 02 21:41:57.720393 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720358 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp4hg\" (UniqueName: \"kubernetes.io/projected/be818b5e-c71a-45a2-8329-2e79ef4b6196-kube-api-access-zp4hg\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.720393 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720371 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5252f006-96c6-4233-a1df-c1a8a1cf255a-host\") pod \"node-ca-5qkfw\" (UID: \"5252f006-96c6-4233-a1df-c1a8a1cf255a\") " pod="openshift-image-registry/node-ca-5qkfw" Jun 02 21:41:57.720987 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720427 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/efca8e3c-85ec-4437-a3bb-d12e252ca89c-socket-dir\") pod \"aws-ebs-csi-driver-node-6r77g\" (UID: \"efca8e3c-85ec-4437-a3bb-d12e252ca89c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" Jun 02 21:41:57.720987 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720455 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7nzn\" (UniqueName: \"kubernetes.io/projected/efca8e3c-85ec-4437-a3bb-d12e252ca89c-kube-api-access-d7nzn\") pod \"aws-ebs-csi-driver-node-6r77g\" (UID: \"efca8e3c-85ec-4437-a3bb-d12e252ca89c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" Jun 02 21:41:57.720987 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720474 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-host-run-k8s-cni-cncf-io\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.720987 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720508 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4b273c1-a82f-4e10-b0ac-671f342af106-tmp-dir\") pod \"node-resolver-dpz84\" (UID: \"b4b273c1-a82f-4e10-b0ac-671f342af106\") " pod="openshift-dns/node-resolver-dpz84" Jun 02 21:41:57.720987 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720541 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92dwj\" (UniqueName: \"kubernetes.io/projected/5252f006-96c6-4233-a1df-c1a8a1cf255a-kube-api-access-92dwj\") pod \"node-ca-5qkfw\" (UID: \"5252f006-96c6-4233-a1df-c1a8a1cf255a\") " pod="openshift-image-registry/node-ca-5qkfw" Jun 02 21:41:57.720987 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720564 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-host-cni-bin\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.720987 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720588 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.720987 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720614 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/974df3d7-921f-46eb-baa9-e9a37e9d93e9-ovnkube-script-lib\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.720987 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720638 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/efca8e3c-85ec-4437-a3bb-d12e252ca89c-device-dir\") pod \"aws-ebs-csi-driver-node-6r77g\" (UID: \"efca8e3c-85ec-4437-a3bb-d12e252ca89c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" Jun 02 21:41:57.720987 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720669 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-run-openvswitch\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.720987 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720692 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-node-log\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.720987 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720707 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/974df3d7-921f-46eb-baa9-e9a37e9d93e9-env-overrides\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.720987 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720722 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5252f006-96c6-4233-a1df-c1a8a1cf255a-serviceca\") pod \"node-ca-5qkfw\" (UID: \"5252f006-96c6-4233-a1df-c1a8a1cf255a\") " pod="openshift-image-registry/node-ca-5qkfw" Jun 02 21:41:57.720987 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720744 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2867ec71-fbbc-43b9-90a8-ad48da269f2d-agent-certs\") pod \"konnectivity-agent-hb594\" (UID: \"2867ec71-fbbc-43b9-90a8-ad48da269f2d\") " pod="kube-system/konnectivity-agent-hb594" Jun 02 21:41:57.720987 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720784 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-multus-socket-dir-parent\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.720987 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720838 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-host-run-multus-certs\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.721596 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720864 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/efca8e3c-85ec-4437-a3bb-d12e252ca89c-sys-fs\") pod \"aws-ebs-csi-driver-node-6r77g\" (UID: \"efca8e3c-85ec-4437-a3bb-d12e252ca89c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" Jun 02 21:41:57.721596 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720887 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2867ec71-fbbc-43b9-90a8-ad48da269f2d-konnectivity-ca\") pod \"konnectivity-agent-hb594\" (UID: \"2867ec71-fbbc-43b9-90a8-ad48da269f2d\") " pod="kube-system/konnectivity-agent-hb594" Jun 02 21:41:57.721596 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720911 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be818b5e-c71a-45a2-8329-2e79ef4b6196-cni-binary-copy\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.721596 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720933 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-host-run-netns\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.721596 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.720956 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-run-systemd\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.721596 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.721000 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-log-socket\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.721596 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.721025 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-cnibin\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.721596 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.721054 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-os-release\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.721596 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.721077 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-multus-conf-dir\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.721596 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.721099 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-etc-openvswitch\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.721596 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.721123 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-multus-cni-dir\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.721596 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.721144 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-host-var-lib-cni-multus\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.721596 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.721172 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znfnn\" (UniqueName: \"kubernetes.io/projected/b4b273c1-a82f-4e10-b0ac-671f342af106-kube-api-access-znfnn\") pod \"node-resolver-dpz84\" (UID: \"b4b273c1-a82f-4e10-b0ac-671f342af106\") " pod="openshift-dns/node-resolver-dpz84" Jun 02 21:41:57.721596 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.721194 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9b8j\" (UniqueName: \"kubernetes.io/projected/d8c7f35d-ee2f-446a-9625-9b8b102a5a9a-kube-api-access-p9b8j\") pod \"iptables-alerter-kmtlg\" (UID: \"d8c7f35d-ee2f-446a-9625-9b8b102a5a9a\") " pod="openshift-network-operator/iptables-alerter-kmtlg" Jun 02 21:41:57.721596 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.721215 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-host-slash\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.721596 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.721236 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-run-ovn\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.780883 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.780856 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 21:36:56 +0000 UTC" deadline="2027-11-25 21:39:27.502041079 +0000 UTC" Jun 02 21:41:57.780883 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.780883 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12983h57m29.721161401s" Jun 02 21:41:57.810474 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.810450 2574 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 02 21:41:57.822441 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822416 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be818b5e-c71a-45a2-8329-2e79ef4b6196-cni-binary-copy\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.822528 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822452 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-host-run-netns\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.822528 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822475 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-run-systemd\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.822528 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822496 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-etc-sysctl-d\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.822528 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822520 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-tmp\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.822708 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822533 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-host-run-netns\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.822708 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822547 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4a906a0b-256c-43a0-aa65-8a8ab9a7e99a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jtthq\" (UID: \"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a\") " pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.822708 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822536 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-run-systemd\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.822708 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822585 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-log-socket\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.822708 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822621 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-etc-sysctl-conf\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.822708 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822646 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-sys\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.822708 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822666 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4a906a0b-256c-43a0-aa65-8a8ab9a7e99a-cni-binary-copy\") pod \"multus-additional-cni-plugins-jtthq\" (UID: \"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a\") " pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.822708 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822688 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-log-socket\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.822708 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822692 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-cnibin\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.823093 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822729 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-cnibin\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.823093 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822738 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-os-release\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.823093 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822766 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-multus-conf-dir\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.823093 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822795 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-etc-openvswitch\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.823093 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822826 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7jmk\" (UniqueName: \"kubernetes.io/projected/4a906a0b-256c-43a0-aa65-8a8ab9a7e99a-kube-api-access-m7jmk\") pod \"multus-additional-cni-plugins-jtthq\" (UID: \"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a\") " pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.823093 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822848 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-multus-conf-dir\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.823093 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822853 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-multus-cni-dir\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.823093 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822868 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-os-release\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.823093 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822897 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-host-var-lib-cni-multus\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.823093 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822906 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-etc-openvswitch\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.823093 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822926 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-znfnn\" (UniqueName: \"kubernetes.io/projected/b4b273c1-a82f-4e10-b0ac-671f342af106-kube-api-access-znfnn\") pod \"node-resolver-dpz84\" (UID: \"b4b273c1-a82f-4e10-b0ac-671f342af106\") " pod="openshift-dns/node-resolver-dpz84" Jun 02 21:41:57.823093 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822946 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-host-var-lib-cni-multus\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.823093 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822953 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p9b8j\" (UniqueName: \"kubernetes.io/projected/d8c7f35d-ee2f-446a-9625-9b8b102a5a9a-kube-api-access-p9b8j\") pod \"iptables-alerter-kmtlg\" (UID: \"d8c7f35d-ee2f-446a-9625-9b8b102a5a9a\") " pod="openshift-network-operator/iptables-alerter-kmtlg" Jun 02 21:41:57.823093 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.822976 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-host-slash\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.823093 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823001 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-etc-tuned\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.823093 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823008 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-multus-cni-dir\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.823093 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823024 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-run-ovn\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.823093 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823048 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs\") pod \"network-metrics-daemon-qw5fs\" (UID: \"03b1bb3a-4c92-4555-b5d4-3aaac39381f5\") " pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:41:57.823805 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823051 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-host-slash\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.823805 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823081 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-host\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.823805 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823099 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-run-ovn\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.823805 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823106 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzkr2\" (UniqueName: \"kubernetes.io/projected/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-kube-api-access-gzkr2\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.823805 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823215 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4a906a0b-256c-43a0-aa65-8a8ab9a7e99a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jtthq\" (UID: \"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a\") " pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.823805 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823245 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/efca8e3c-85ec-4437-a3bb-d12e252ca89c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6r77g\" (UID: \"efca8e3c-85ec-4437-a3bb-d12e252ca89c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" Jun 02 21:41:57.823805 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823272 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/efca8e3c-85ec-4437-a3bb-d12e252ca89c-etc-selinux\") pod \"aws-ebs-csi-driver-node-6r77g\" (UID: \"efca8e3c-85ec-4437-a3bb-d12e252ca89c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" Jun 02 21:41:57.823805 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823312 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b4b273c1-a82f-4e10-b0ac-671f342af106-hosts-file\") pod \"node-resolver-dpz84\" (UID: \"b4b273c1-a82f-4e10-b0ac-671f342af106\") " pod="openshift-dns/node-resolver-dpz84" Jun 02 21:41:57.823805 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823337 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-host-run-netns\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.823805 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823332 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/efca8e3c-85ec-4437-a3bb-d12e252ca89c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6r77g\" (UID: \"efca8e3c-85ec-4437-a3bb-d12e252ca89c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" Jun 02 21:41:57.823805 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823364 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-var-lib-openvswitch\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.823805 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823395 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crhnw\" (UniqueName: \"kubernetes.io/projected/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-kube-api-access-crhnw\") pod \"network-metrics-daemon-qw5fs\" (UID: \"03b1bb3a-4c92-4555-b5d4-3aaac39381f5\") " pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:41:57.823805 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823419 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-host-run-netns\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.823805 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823425 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b4b273c1-a82f-4e10-b0ac-671f342af106-hosts-file\") pod \"node-resolver-dpz84\" (UID: \"b4b273c1-a82f-4e10-b0ac-671f342af106\") " pod="openshift-dns/node-resolver-dpz84" Jun 02 21:41:57.823805 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823396 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/efca8e3c-85ec-4437-a3bb-d12e252ca89c-etc-selinux\") pod \"aws-ebs-csi-driver-node-6r77g\" (UID: \"efca8e3c-85ec-4437-a3bb-d12e252ca89c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" Jun 02 21:41:57.823805 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823434 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-var-lib-openvswitch\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.823805 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823423 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a906a0b-256c-43a0-aa65-8a8ab9a7e99a-system-cni-dir\") pod \"multus-additional-cni-plugins-jtthq\" (UID: \"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a\") " pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.824584 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823479 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d8c7f35d-ee2f-446a-9625-9b8b102a5a9a-iptables-alerter-script\") pod \"iptables-alerter-kmtlg\" (UID: \"d8c7f35d-ee2f-446a-9625-9b8b102a5a9a\") " pod="openshift-network-operator/iptables-alerter-kmtlg" Jun 02 21:41:57.824584 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823514 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-host-kubelet\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.824584 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823557 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-host-kubelet\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.824584 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823563 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be818b5e-c71a-45a2-8329-2e79ef4b6196-cni-binary-copy\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.824584 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823643 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-systemd-units\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.824584 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823714 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-host-cni-netd\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.824584 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823739 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b82cp\" (UniqueName: \"kubernetes.io/projected/974df3d7-921f-46eb-baa9-e9a37e9d93e9-kube-api-access-b82cp\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.824584 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823779 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-lib-modules\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.824584 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823806 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-system-cni-dir\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.824584 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823829 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-host-var-lib-cni-bin\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.824584 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823891 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-system-cni-dir\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.824584 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823924 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-systemd-units\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.824584 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.823954 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-host-cni-netd\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.824584 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.824013 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-host-var-lib-kubelet\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.824584 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.824053 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d8c7f35d-ee2f-446a-9625-9b8b102a5a9a-iptables-alerter-script\") pod \"iptables-alerter-kmtlg\" (UID: \"d8c7f35d-ee2f-446a-9625-9b8b102a5a9a\") " pod="openshift-network-operator/iptables-alerter-kmtlg" Jun 02 21:41:57.824584 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.824056 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-hostroot\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.824584 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.824096 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-hostroot\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.824584 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.824104 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-host-run-ovn-kubernetes\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.825362 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.824106 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-host-var-lib-cni-bin\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.825362 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.824133 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/974df3d7-921f-46eb-baa9-e9a37e9d93e9-ovnkube-config\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.825362 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.824150 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-host-var-lib-kubelet\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.825362 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.824200 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/974df3d7-921f-46eb-baa9-e9a37e9d93e9-ovn-node-metrics-cert\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.825362 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.824263 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-host-run-ovn-kubernetes\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.825362 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.824316 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/be818b5e-c71a-45a2-8329-2e79ef4b6196-multus-daemon-config\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.825362 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.824461 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-etc-kubernetes\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.825362 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.824639 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/974df3d7-921f-46eb-baa9-e9a37e9d93e9-ovnkube-config\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.825362 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.824766 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-etc-kubernetes\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.825362 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.824801 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/be818b5e-c71a-45a2-8329-2e79ef4b6196-multus-daemon-config\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.825362 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.824801 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d8c7f35d-ee2f-446a-9625-9b8b102a5a9a-host-slash\") pod \"iptables-alerter-kmtlg\" (UID: \"d8c7f35d-ee2f-446a-9625-9b8b102a5a9a\") " pod="openshift-network-operator/iptables-alerter-kmtlg" Jun 02 21:41:57.825362 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.824838 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d8c7f35d-ee2f-446a-9625-9b8b102a5a9a-host-slash\") pod \"iptables-alerter-kmtlg\" (UID: \"d8c7f35d-ee2f-446a-9625-9b8b102a5a9a\") " pod="openshift-network-operator/iptables-alerter-kmtlg" Jun 02 21:41:57.825362 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.824858 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-var-lib-kubelet\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.825362 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.824886 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4a906a0b-256c-43a0-aa65-8a8ab9a7e99a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jtthq\" (UID: \"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a\") " pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.825362 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.824929 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/efca8e3c-85ec-4437-a3bb-d12e252ca89c-registration-dir\") pod \"aws-ebs-csi-driver-node-6r77g\" (UID: \"efca8e3c-85ec-4437-a3bb-d12e252ca89c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" Jun 02 21:41:57.825362 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.824934 2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 02 21:41:57.825362 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.824966 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zp4hg\" (UniqueName: \"kubernetes.io/projected/be818b5e-c71a-45a2-8329-2e79ef4b6196-kube-api-access-zp4hg\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.825362 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.824987 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5252f006-96c6-4233-a1df-c1a8a1cf255a-host\") pod \"node-ca-5qkfw\" (UID: \"5252f006-96c6-4233-a1df-c1a8a1cf255a\") " pod="openshift-image-registry/node-ca-5qkfw" Jun 02 21:41:57.826002 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.825008 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/efca8e3c-85ec-4437-a3bb-d12e252ca89c-socket-dir\") pod \"aws-ebs-csi-driver-node-6r77g\" (UID: \"efca8e3c-85ec-4437-a3bb-d12e252ca89c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" Jun 02 21:41:57.826002 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.825029 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d7nzn\" (UniqueName: \"kubernetes.io/projected/efca8e3c-85ec-4437-a3bb-d12e252ca89c-kube-api-access-d7nzn\") pod \"aws-ebs-csi-driver-node-6r77g\" (UID: \"efca8e3c-85ec-4437-a3bb-d12e252ca89c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" Jun 02 21:41:57.826002 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.825033 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/efca8e3c-85ec-4437-a3bb-d12e252ca89c-registration-dir\") pod \"aws-ebs-csi-driver-node-6r77g\" (UID: \"efca8e3c-85ec-4437-a3bb-d12e252ca89c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" Jun 02 21:41:57.826002 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.825052 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-host-run-k8s-cni-cncf-io\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.826002 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.825075 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4b273c1-a82f-4e10-b0ac-671f342af106-tmp-dir\") pod \"node-resolver-dpz84\" (UID: \"b4b273c1-a82f-4e10-b0ac-671f342af106\") " pod="openshift-dns/node-resolver-dpz84" Jun 02 21:41:57.826002 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.825103 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-92dwj\" (UniqueName: \"kubernetes.io/projected/5252f006-96c6-4233-a1df-c1a8a1cf255a-kube-api-access-92dwj\") pod \"node-ca-5qkfw\" (UID: \"5252f006-96c6-4233-a1df-c1a8a1cf255a\") " pod="openshift-image-registry/node-ca-5qkfw" Jun 02 21:41:57.826002 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.825123 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-host-cni-bin\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.826002 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.825132 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/efca8e3c-85ec-4437-a3bb-d12e252ca89c-socket-dir\") pod \"aws-ebs-csi-driver-node-6r77g\" (UID: \"efca8e3c-85ec-4437-a3bb-d12e252ca89c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" Jun 02 21:41:57.826002 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.825144 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.826002 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.825172 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5252f006-96c6-4233-a1df-c1a8a1cf255a-host\") pod \"node-ca-5qkfw\" (UID: \"5252f006-96c6-4233-a1df-c1a8a1cf255a\") " pod="openshift-image-registry/node-ca-5qkfw" Jun 02 21:41:57.826002 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.825176 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/974df3d7-921f-46eb-baa9-e9a37e9d93e9-ovnkube-script-lib\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.826002 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.825202 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/efca8e3c-85ec-4437-a3bb-d12e252ca89c-device-dir\") pod \"aws-ebs-csi-driver-node-6r77g\" (UID: \"efca8e3c-85ec-4437-a3bb-d12e252ca89c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" Jun 02 21:41:57.826002 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.825219 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-host-run-k8s-cni-cncf-io\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.826002 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.825225 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-run-openvswitch\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.826002 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.825248 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-node-log\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.826002 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.825266 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.826002 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.825271 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/974df3d7-921f-46eb-baa9-e9a37e9d93e9-env-overrides\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.826630 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.825342 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-host-cni-bin\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.826630 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.825349 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-node-log\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.826630 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.825409 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-etc-kubernetes\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.826630 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.825423 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4b273c1-a82f-4e10-b0ac-671f342af106-tmp-dir\") pod \"node-resolver-dpz84\" (UID: \"b4b273c1-a82f-4e10-b0ac-671f342af106\") " pod="openshift-dns/node-resolver-dpz84" Jun 02 21:41:57.826630 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.826006 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/974df3d7-921f-46eb-baa9-e9a37e9d93e9-ovnkube-script-lib\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.826630 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.825446 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-etc-systemd\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.826630 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.826169 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5252f006-96c6-4233-a1df-c1a8a1cf255a-serviceca\") pod \"node-ca-5qkfw\" (UID: \"5252f006-96c6-4233-a1df-c1a8a1cf255a\") " pod="openshift-image-registry/node-ca-5qkfw" Jun 02 21:41:57.826630 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.826220 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/974df3d7-921f-46eb-baa9-e9a37e9d93e9-run-openvswitch\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.826630 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.826270 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-etc-modprobe-d\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.826630 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.826321 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2867ec71-fbbc-43b9-90a8-ad48da269f2d-agent-certs\") pod \"konnectivity-agent-hb594\" (UID: \"2867ec71-fbbc-43b9-90a8-ad48da269f2d\") " pod="kube-system/konnectivity-agent-hb594" Jun 02 21:41:57.826630 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.826322 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/efca8e3c-85ec-4437-a3bb-d12e252ca89c-device-dir\") pod \"aws-ebs-csi-driver-node-6r77g\" (UID: \"efca8e3c-85ec-4437-a3bb-d12e252ca89c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" Jun 02 21:41:57.826630 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.826390 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-multus-socket-dir-parent\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.826630 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.826421 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-host-run-multus-certs\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.826630 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.826455 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrtcr\" (UniqueName: \"kubernetes.io/projected/7510b8a7-e605-4db3-882f-c50c26e954b9-kube-api-access-vrtcr\") pod \"network-check-target-zmvbb\" (UID: \"7510b8a7-e605-4db3-882f-c50c26e954b9\") " pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:41:57.826630 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.826487 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-etc-sysconfig\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.826630 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.826610 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/974df3d7-921f-46eb-baa9-e9a37e9d93e9-env-overrides\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.827183 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.826652 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-run\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.827183 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.826687 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4a906a0b-256c-43a0-aa65-8a8ab9a7e99a-cnibin\") pod \"multus-additional-cni-plugins-jtthq\" (UID: \"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a\") " pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.827183 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.826719 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4a906a0b-256c-43a0-aa65-8a8ab9a7e99a-os-release\") pod \"multus-additional-cni-plugins-jtthq\" (UID: \"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a\") " pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.827183 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.826752 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/efca8e3c-85ec-4437-a3bb-d12e252ca89c-sys-fs\") pod \"aws-ebs-csi-driver-node-6r77g\" (UID: \"efca8e3c-85ec-4437-a3bb-d12e252ca89c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" Jun 02 21:41:57.827183 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.826780 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2867ec71-fbbc-43b9-90a8-ad48da269f2d-konnectivity-ca\") pod \"konnectivity-agent-hb594\" (UID: \"2867ec71-fbbc-43b9-90a8-ad48da269f2d\") " pod="kube-system/konnectivity-agent-hb594" Jun 02 21:41:57.827183 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.827010 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/efca8e3c-85ec-4437-a3bb-d12e252ca89c-sys-fs\") pod \"aws-ebs-csi-driver-node-6r77g\" (UID: \"efca8e3c-85ec-4437-a3bb-d12e252ca89c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" Jun 02 21:41:57.827475 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.827258 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-multus-socket-dir-parent\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.827475 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.827335 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/be818b5e-c71a-45a2-8329-2e79ef4b6196-host-run-multus-certs\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.827475 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.827452 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5252f006-96c6-4233-a1df-c1a8a1cf255a-serviceca\") pod \"node-ca-5qkfw\" (UID: \"5252f006-96c6-4233-a1df-c1a8a1cf255a\") " pod="openshift-image-registry/node-ca-5qkfw" Jun 02 21:41:57.829406 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.828329 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/974df3d7-921f-46eb-baa9-e9a37e9d93e9-ovn-node-metrics-cert\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.830147 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.830045 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2867ec71-fbbc-43b9-90a8-ad48da269f2d-konnectivity-ca\") pod \"konnectivity-agent-hb594\" (UID: \"2867ec71-fbbc-43b9-90a8-ad48da269f2d\") " pod="kube-system/konnectivity-agent-hb594" Jun 02 21:41:57.830863 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.830841 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2867ec71-fbbc-43b9-90a8-ad48da269f2d-agent-certs\") pod \"konnectivity-agent-hb594\" (UID: \"2867ec71-fbbc-43b9-90a8-ad48da269f2d\") " pod="kube-system/konnectivity-agent-hb594" Jun 02 21:41:57.835710 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.835526 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9b8j\" (UniqueName: \"kubernetes.io/projected/d8c7f35d-ee2f-446a-9625-9b8b102a5a9a-kube-api-access-p9b8j\") pod \"iptables-alerter-kmtlg\" (UID: \"d8c7f35d-ee2f-446a-9625-9b8b102a5a9a\") " pod="openshift-network-operator/iptables-alerter-kmtlg" Jun 02 21:41:57.835710 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.835665 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znfnn\" (UniqueName: \"kubernetes.io/projected/b4b273c1-a82f-4e10-b0ac-671f342af106-kube-api-access-znfnn\") pod \"node-resolver-dpz84\" (UID: \"b4b273c1-a82f-4e10-b0ac-671f342af106\") " pod="openshift-dns/node-resolver-dpz84" Jun 02 21:41:57.835847 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.835717 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp4hg\" (UniqueName: \"kubernetes.io/projected/be818b5e-c71a-45a2-8329-2e79ef4b6196-kube-api-access-zp4hg\") pod \"multus-kvgdj\" (UID: \"be818b5e-c71a-45a2-8329-2e79ef4b6196\") " pod="openshift-multus/multus-kvgdj" Jun 02 21:41:57.835847 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.835775 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7nzn\" (UniqueName: \"kubernetes.io/projected/efca8e3c-85ec-4437-a3bb-d12e252ca89c-kube-api-access-d7nzn\") pod \"aws-ebs-csi-driver-node-6r77g\" (UID: \"efca8e3c-85ec-4437-a3bb-d12e252ca89c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" Jun 02 21:41:57.836202 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.836179 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92dwj\" (UniqueName: \"kubernetes.io/projected/5252f006-96c6-4233-a1df-c1a8a1cf255a-kube-api-access-92dwj\") pod \"node-ca-5qkfw\" (UID: \"5252f006-96c6-4233-a1df-c1a8a1cf255a\") " pod="openshift-image-registry/node-ca-5qkfw" Jun 02 21:41:57.836697 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.836677 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b82cp\" (UniqueName: \"kubernetes.io/projected/974df3d7-921f-46eb-baa9-e9a37e9d93e9-kube-api-access-b82cp\") pod \"ovnkube-node-fddpn\" (UID: \"974df3d7-921f-46eb-baa9-e9a37e9d93e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:57.927889 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.927855 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-etc-tuned\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.928037 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.927908 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs\") pod \"network-metrics-daemon-qw5fs\" (UID: \"03b1bb3a-4c92-4555-b5d4-3aaac39381f5\") " pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:41:57.928037 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.927931 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-host\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.928037 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.927952 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gzkr2\" (UniqueName: \"kubernetes.io/projected/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-kube-api-access-gzkr2\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.928037 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.927975 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4a906a0b-256c-43a0-aa65-8a8ab9a7e99a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jtthq\" (UID: \"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a\") " pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.928037 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928005 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-crhnw\" (UniqueName: \"kubernetes.io/projected/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-kube-api-access-crhnw\") pod \"network-metrics-daemon-qw5fs\" (UID: \"03b1bb3a-4c92-4555-b5d4-3aaac39381f5\") " pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:41:57.928271 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:57.928062 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:41:57.928271 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928136 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a906a0b-256c-43a0-aa65-8a8ab9a7e99a-system-cni-dir\") pod \"multus-additional-cni-plugins-jtthq\" (UID: \"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a\") " pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.928271 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:57.928185 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs podName:03b1bb3a-4c92-4555-b5d4-3aaac39381f5 nodeName:}" failed. No retries permitted until 2026-06-02 21:41:58.428162838 +0000 UTC m=+3.133172779 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs") pod "network-metrics-daemon-qw5fs" (UID: "03b1bb3a-4c92-4555-b5d4-3aaac39381f5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:41:57.928271 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928188 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a906a0b-256c-43a0-aa65-8a8ab9a7e99a-system-cni-dir\") pod \"multus-additional-cni-plugins-jtthq\" (UID: \"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a\") " pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.928271 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928064 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-host\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.928271 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928207 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-lib-modules\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.928271 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928248 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-var-lib-kubelet\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.928271 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928273 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4a906a0b-256c-43a0-aa65-8a8ab9a7e99a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jtthq\" (UID: \"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a\") " pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.928664 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928324 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-etc-kubernetes\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.928664 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928347 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-etc-systemd\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.928664 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928366 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-var-lib-kubelet\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.928664 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928381 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-lib-modules\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.928664 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928371 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-etc-modprobe-d\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.928664 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928423 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-etc-kubernetes\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.928664 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928431 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vrtcr\" (UniqueName: \"kubernetes.io/projected/7510b8a7-e605-4db3-882f-c50c26e954b9-kube-api-access-vrtcr\") pod \"network-check-target-zmvbb\" (UID: \"7510b8a7-e605-4db3-882f-c50c26e954b9\") " pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:41:57.928664 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928435 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-etc-systemd\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.928664 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928438 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-etc-modprobe-d\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.928664 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928459 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-etc-sysconfig\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.928664 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928485 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-run\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.928664 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928510 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4a906a0b-256c-43a0-aa65-8a8ab9a7e99a-cnibin\") pod \"multus-additional-cni-plugins-jtthq\" (UID: \"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a\") " pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.928664 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928536 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4a906a0b-256c-43a0-aa65-8a8ab9a7e99a-os-release\") pod \"multus-additional-cni-plugins-jtthq\" (UID: \"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a\") " pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.928664 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928558 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-etc-sysconfig\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.928664 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928560 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-run\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.928664 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928565 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-etc-sysctl-d\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.928664 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928586 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4a906a0b-256c-43a0-aa65-8a8ab9a7e99a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jtthq\" (UID: \"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a\") " pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.929246 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928604 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4a906a0b-256c-43a0-aa65-8a8ab9a7e99a-cnibin\") pod \"multus-additional-cni-plugins-jtthq\" (UID: \"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a\") " pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.929246 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928613 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-tmp\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.929246 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928628 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4a906a0b-256c-43a0-aa65-8a8ab9a7e99a-os-release\") pod \"multus-additional-cni-plugins-jtthq\" (UID: \"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a\") " pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.929246 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928639 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4a906a0b-256c-43a0-aa65-8a8ab9a7e99a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jtthq\" (UID: \"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a\") " pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.929246 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928668 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-etc-sysctl-conf\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.929246 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928692 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-sys\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.929246 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928715 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-etc-sysctl-d\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.929246 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928716 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4a906a0b-256c-43a0-aa65-8a8ab9a7e99a-cni-binary-copy\") pod \"multus-additional-cni-plugins-jtthq\" (UID: \"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a\") " pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.929246 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928750 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4a906a0b-256c-43a0-aa65-8a8ab9a7e99a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jtthq\" (UID: \"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a\") " pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.929246 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928762 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m7jmk\" (UniqueName: \"kubernetes.io/projected/4a906a0b-256c-43a0-aa65-8a8ab9a7e99a-kube-api-access-m7jmk\") pod \"multus-additional-cni-plugins-jtthq\" (UID: \"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a\") " pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.929246 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928775 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-etc-sysctl-conf\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.929246 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928777 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4a906a0b-256c-43a0-aa65-8a8ab9a7e99a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jtthq\" (UID: \"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a\") " pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.929246 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.928801 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-sys\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.929246 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.929081 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4a906a0b-256c-43a0-aa65-8a8ab9a7e99a-cni-binary-copy\") pod \"multus-additional-cni-plugins-jtthq\" (UID: \"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a\") " pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.930902 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.930870 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-tmp\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.931016 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.930927 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-etc-tuned\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:57.951813 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:57.951758 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:41:57.951813 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:57.951780 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:41:57.951813 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:57.951789 2574 projected.go:194] Error preparing data for projected volume kube-api-access-vrtcr for pod openshift-network-diagnostics/network-check-target-zmvbb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:41:57.952020 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:57.951850 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7510b8a7-e605-4db3-882f-c50c26e954b9-kube-api-access-vrtcr podName:7510b8a7-e605-4db3-882f-c50c26e954b9 nodeName:}" failed. No retries permitted until 2026-06-02 21:41:58.451835628 +0000 UTC m=+3.156845573 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vrtcr" (UniqueName: "kubernetes.io/projected/7510b8a7-e605-4db3-882f-c50c26e954b9-kube-api-access-vrtcr") pod "network-check-target-zmvbb" (UID: "7510b8a7-e605-4db3-882f-c50c26e954b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:41:57.955000 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.954979 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7jmk\" (UniqueName: \"kubernetes.io/projected/4a906a0b-256c-43a0-aa65-8a8ab9a7e99a-kube-api-access-m7jmk\") pod \"multus-additional-cni-plugins-jtthq\" (UID: \"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a\") " pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:57.956779 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.956760 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crhnw\" (UniqueName: \"kubernetes.io/projected/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-kube-api-access-crhnw\") pod \"network-metrics-daemon-qw5fs\" (UID: \"03b1bb3a-4c92-4555-b5d4-3aaac39381f5\") " pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:41:57.956871 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:57.956781 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzkr2\" (UniqueName: \"kubernetes.io/projected/de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d-kube-api-access-gzkr2\") pod \"tuned-5zvps\" (UID: \"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d\") " pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:58.015755 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.015728 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" Jun 02 21:41:58.023335 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.023312 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hb594" Jun 02 21:41:58.029974 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.029951 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kvgdj" Jun 02 21:41:58.034122 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.034104 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5qkfw" Jun 02 21:41:58.040643 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.040624 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dpz84" Jun 02 21:41:58.048237 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.048221 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:41:58.054317 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.054298 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kmtlg" Jun 02 21:41:58.061865 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.061847 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jtthq" Jun 02 21:41:58.068017 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.068000 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5zvps" Jun 02 21:41:58.348812 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:58.348783 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8c7f35d_ee2f_446a_9625_9b8b102a5a9a.slice/crio-77a6b524b9c39e3fa92e32093d5fc93ed3f166cae64c26a97a85dfb8ddfc4e9f WatchSource:0}: Error finding container 77a6b524b9c39e3fa92e32093d5fc93ed3f166cae64c26a97a85dfb8ddfc4e9f: Status 404 returned error can't find the container with id 77a6b524b9c39e3fa92e32093d5fc93ed3f166cae64c26a97a85dfb8ddfc4e9f Jun 02 21:41:58.349970 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:58.349944 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod974df3d7_921f_46eb_baa9_e9a37e9d93e9.slice/crio-d77f905397ed2cdbae88ae44fee22d788042a68acd0a6aed275d68a2b8ed40cd WatchSource:0}: Error finding container d77f905397ed2cdbae88ae44fee22d788042a68acd0a6aed275d68a2b8ed40cd: Status 404 returned error can't find the container with id d77f905397ed2cdbae88ae44fee22d788042a68acd0a6aed275d68a2b8ed40cd Jun 02 21:41:58.351185 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:58.351162 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a906a0b_256c_43a0_aa65_8a8ab9a7e99a.slice/crio-09da0d51a7cfb24b6d39da3f7fd2c4077bca103532585d4fa53b2471c56e162e WatchSource:0}: Error finding container 09da0d51a7cfb24b6d39da3f7fd2c4077bca103532585d4fa53b2471c56e162e: Status 404 returned error can't find the container with id 09da0d51a7cfb24b6d39da3f7fd2c4077bca103532585d4fa53b2471c56e162e Jun 02 21:41:58.352228 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:58.352192 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5252f006_96c6_4233_a1df_c1a8a1cf255a.slice/crio-18e82707ad6b1b11d88cc36d8122f3a7a598ca7957414d8faadf34c317ba3c61 WatchSource:0}: Error finding container 18e82707ad6b1b11d88cc36d8122f3a7a598ca7957414d8faadf34c317ba3c61: Status 404 returned error can't find the container with id 18e82707ad6b1b11d88cc36d8122f3a7a598ca7957414d8faadf34c317ba3c61 Jun 02 21:41:58.355669 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:58.355649 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4b273c1_a82f_4e10_b0ac_671f342af106.slice/crio-490ccc42ca71dff797a3e7dbc1751f500ce892aca7b39b8d8958be6333aefc52 WatchSource:0}: Error finding container 490ccc42ca71dff797a3e7dbc1751f500ce892aca7b39b8d8958be6333aefc52: Status 404 returned error can't find the container with id 490ccc42ca71dff797a3e7dbc1751f500ce892aca7b39b8d8958be6333aefc52 Jun 02 21:41:58.356982 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:58.356910 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefca8e3c_85ec_4437_a3bb_d12e252ca89c.slice/crio-377135e6faa50a94277d27962bd8b2554508e0aa89d3e45ddcb024296071f225 WatchSource:0}: Error finding container 377135e6faa50a94277d27962bd8b2554508e0aa89d3e45ddcb024296071f225: Status 404 returned error can't find the container with id 377135e6faa50a94277d27962bd8b2554508e0aa89d3e45ddcb024296071f225 Jun 02 21:41:58.357777 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:58.357753 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2867ec71_fbbc_43b9_90a8_ad48da269f2d.slice/crio-4716ca9c8d5aeaa33fa68256ef5f76897f4ea8d80e47d3716190238e44677787 WatchSource:0}: Error finding container 4716ca9c8d5aeaa33fa68256ef5f76897f4ea8d80e47d3716190238e44677787: Status 404 returned error can't find the container with id 4716ca9c8d5aeaa33fa68256ef5f76897f4ea8d80e47d3716190238e44677787 Jun 02 21:41:58.358998 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:58.358743 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe818b5e_c71a_45a2_8329_2e79ef4b6196.slice/crio-1b53e7f40a5791206859a1dc9d20d3c6a948f1b5eddf4267c112432c0be62f58 WatchSource:0}: Error finding container 1b53e7f40a5791206859a1dc9d20d3c6a948f1b5eddf4267c112432c0be62f58: Status 404 returned error can't find the container with id 1b53e7f40a5791206859a1dc9d20d3c6a948f1b5eddf4267c112432c0be62f58 Jun 02 21:41:58.359623 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:41:58.359546 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde5bcbc0_d1d5_4e3e_af7e_5824ab06cb2d.slice/crio-01d8a33f8d7f7e5c3db8de298a36302032438a946736f6ca264996994339979b WatchSource:0}: Error finding container 01d8a33f8d7f7e5c3db8de298a36302032438a946736f6ca264996994339979b: Status 404 returned error can't find the container with id 01d8a33f8d7f7e5c3db8de298a36302032438a946736f6ca264996994339979b Jun 02 21:41:58.433172 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.433042 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs\") pod \"network-metrics-daemon-qw5fs\" (UID: \"03b1bb3a-4c92-4555-b5d4-3aaac39381f5\") " pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:41:58.433258 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:58.433188 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:41:58.433322 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:58.433256 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs podName:03b1bb3a-4c92-4555-b5d4-3aaac39381f5 nodeName:}" failed. No retries permitted until 2026-06-02 21:41:59.433235376 +0000 UTC m=+4.138245303 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs") pod "network-metrics-daemon-qw5fs" (UID: "03b1bb3a-4c92-4555-b5d4-3aaac39381f5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:41:58.534112 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.534085 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vrtcr\" (UniqueName: \"kubernetes.io/projected/7510b8a7-e605-4db3-882f-c50c26e954b9-kube-api-access-vrtcr\") pod \"network-check-target-zmvbb\" (UID: \"7510b8a7-e605-4db3-882f-c50c26e954b9\") " pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:41:58.534238 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:58.534210 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:41:58.534238 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:58.534223 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:41:58.534238 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:58.534233 2574 projected.go:194] Error preparing data for projected volume kube-api-access-vrtcr for pod openshift-network-diagnostics/network-check-target-zmvbb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:41:58.534369 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:58.534278 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7510b8a7-e605-4db3-882f-c50c26e954b9-kube-api-access-vrtcr podName:7510b8a7-e605-4db3-882f-c50c26e954b9 nodeName:}" failed. No retries permitted until 2026-06-02 21:41:59.534264875 +0000 UTC m=+4.239274802 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vrtcr" (UniqueName: "kubernetes.io/projected/7510b8a7-e605-4db3-882f-c50c26e954b9-kube-api-access-vrtcr") pod "network-check-target-zmvbb" (UID: "7510b8a7-e605-4db3-882f-c50c26e954b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:41:58.781452 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.781388 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 21:36:56 +0000 UTC" deadline="2028-02-22 14:49:51.36197645 +0000 UTC" Jun 02 21:41:58.781452 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.781431 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15113h7m52.580549065s" Jun 02 21:41:58.864129 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.864044 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" event={"ID":"974df3d7-921f-46eb-baa9-e9a37e9d93e9","Type":"ContainerStarted","Data":"d77f905397ed2cdbae88ae44fee22d788042a68acd0a6aed275d68a2b8ed40cd"} Jun 02 21:41:58.867439 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.867370 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kmtlg" event={"ID":"d8c7f35d-ee2f-446a-9625-9b8b102a5a9a","Type":"ContainerStarted","Data":"77a6b524b9c39e3fa92e32093d5fc93ed3f166cae64c26a97a85dfb8ddfc4e9f"} Jun 02 21:41:58.869671 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.869616 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5zvps" event={"ID":"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d","Type":"ContainerStarted","Data":"01d8a33f8d7f7e5c3db8de298a36302032438a946736f6ca264996994339979b"} Jun 02 21:41:58.876594 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.876530 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kvgdj" event={"ID":"be818b5e-c71a-45a2-8329-2e79ef4b6196","Type":"ContainerStarted","Data":"1b53e7f40a5791206859a1dc9d20d3c6a948f1b5eddf4267c112432c0be62f58"} Jun 02 21:41:58.880227 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.880177 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hb594" event={"ID":"2867ec71-fbbc-43b9-90a8-ad48da269f2d","Type":"ContainerStarted","Data":"4716ca9c8d5aeaa33fa68256ef5f76897f4ea8d80e47d3716190238e44677787"} Jun 02 21:41:58.881901 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.881878 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" event={"ID":"efca8e3c-85ec-4437-a3bb-d12e252ca89c","Type":"ContainerStarted","Data":"377135e6faa50a94277d27962bd8b2554508e0aa89d3e45ddcb024296071f225"} Jun 02 21:41:58.889947 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.889924 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dpz84" event={"ID":"b4b273c1-a82f-4e10-b0ac-671f342af106","Type":"ContainerStarted","Data":"490ccc42ca71dff797a3e7dbc1751f500ce892aca7b39b8d8958be6333aefc52"} Jun 02 21:41:58.909310 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.909268 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5qkfw" event={"ID":"5252f006-96c6-4233-a1df-c1a8a1cf255a","Type":"ContainerStarted","Data":"18e82707ad6b1b11d88cc36d8122f3a7a598ca7957414d8faadf34c317ba3c61"} Jun 02 21:41:58.923468 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.923399 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jtthq" event={"ID":"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a","Type":"ContainerStarted","Data":"09da0d51a7cfb24b6d39da3f7fd2c4077bca103532585d4fa53b2471c56e162e"} Jun 02 21:41:58.937309 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.936689 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.ec2.internal" event={"ID":"ea608c6548a1e6295267e8797334485d","Type":"ContainerStarted","Data":"c521c6c5f4e8a89bf1740616db91386966de423da4e76870f9065a4ee105f217"} Jun 02 21:41:58.943732 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.943706 2574 generic.go:358] "Generic (PLEG): container finished" podID="c40a125b2a72f53c0ffc8fa9550c611b" containerID="5fe25e373e3eb9f9bab8660177c4bfe0064cbdbae7ff497f6818c5411cd870ff" exitCode=0 Jun 02 21:41:58.943821 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.943742 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.ec2.internal" event={"ID":"c40a125b2a72f53c0ffc8fa9550c611b","Type":"ContainerDied","Data":"5fe25e373e3eb9f9bab8660177c4bfe0064cbdbae7ff497f6818c5411cd870ff"} Jun 02 21:41:58.950180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:58.950115 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.ec2.internal" podStartSLOduration=2.950100762 podStartE2EDuration="2.950100762s" podCreationTimestamp="2026-06-02 21:41:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 21:41:58.949976703 +0000 UTC m=+3.654986652" watchObservedRunningTime="2026-06-02 21:41:58.950100762 +0000 UTC m=+3.655110710" Jun 02 21:41:59.442889 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:59.442812 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs\") pod \"network-metrics-daemon-qw5fs\" (UID: \"03b1bb3a-4c92-4555-b5d4-3aaac39381f5\") " pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:41:59.443038 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:59.443001 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:41:59.443099 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:59.443066 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs podName:03b1bb3a-4c92-4555-b5d4-3aaac39381f5 nodeName:}" failed. No retries permitted until 2026-06-02 21:42:01.443045069 +0000 UTC m=+6.148054998 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs") pod "network-metrics-daemon-qw5fs" (UID: "03b1bb3a-4c92-4555-b5d4-3aaac39381f5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:41:59.543705 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:59.543668 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vrtcr\" (UniqueName: \"kubernetes.io/projected/7510b8a7-e605-4db3-882f-c50c26e954b9-kube-api-access-vrtcr\") pod \"network-check-target-zmvbb\" (UID: \"7510b8a7-e605-4db3-882f-c50c26e954b9\") " pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:41:59.543907 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:59.543872 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:41:59.543907 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:59.543893 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:41:59.543907 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:59.543905 2574 projected.go:194] Error preparing data for projected volume kube-api-access-vrtcr for pod openshift-network-diagnostics/network-check-target-zmvbb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:41:59.544074 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:59.543963 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7510b8a7-e605-4db3-882f-c50c26e954b9-kube-api-access-vrtcr podName:7510b8a7-e605-4db3-882f-c50c26e954b9 nodeName:}" failed. No retries permitted until 2026-06-02 21:42:01.543944899 +0000 UTC m=+6.248954830 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vrtcr" (UniqueName: "kubernetes.io/projected/7510b8a7-e605-4db3-882f-c50c26e954b9-kube-api-access-vrtcr") pod "network-check-target-zmvbb" (UID: "7510b8a7-e605-4db3-882f-c50c26e954b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:41:59.842944 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:59.842838 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:41:59.843399 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:59.842980 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5fs" podUID="03b1bb3a-4c92-4555-b5d4-3aaac39381f5" Jun 02 21:41:59.845560 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:59.845304 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:41:59.845560 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:41:59.845400 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zmvbb" podUID="7510b8a7-e605-4db3-882f-c50c26e954b9" Jun 02 21:41:59.992415 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:41:59.991982 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.ec2.internal" event={"ID":"c40a125b2a72f53c0ffc8fa9550c611b","Type":"ContainerStarted","Data":"3275a5b204f1de6f817d945f2f1c286a340b91a2bd5a5d3020fad3cb255e9dbc"} Jun 02 21:42:01.464980 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:01.464942 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs\") pod \"network-metrics-daemon-qw5fs\" (UID: \"03b1bb3a-4c92-4555-b5d4-3aaac39381f5\") " pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:42:01.465416 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:01.465128 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:42:01.465416 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:01.465190 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs podName:03b1bb3a-4c92-4555-b5d4-3aaac39381f5 nodeName:}" failed. No retries permitted until 2026-06-02 21:42:05.465172245 +0000 UTC m=+10.170182188 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs") pod "network-metrics-daemon-qw5fs" (UID: "03b1bb3a-4c92-4555-b5d4-3aaac39381f5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:42:01.566966 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:01.566300 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vrtcr\" (UniqueName: \"kubernetes.io/projected/7510b8a7-e605-4db3-882f-c50c26e954b9-kube-api-access-vrtcr\") pod \"network-check-target-zmvbb\" (UID: \"7510b8a7-e605-4db3-882f-c50c26e954b9\") " pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:42:01.566966 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:01.566525 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:42:01.566966 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:01.566546 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:42:01.566966 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:01.566559 2574 projected.go:194] Error preparing data for projected volume kube-api-access-vrtcr for pod openshift-network-diagnostics/network-check-target-zmvbb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:42:01.566966 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:01.566614 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7510b8a7-e605-4db3-882f-c50c26e954b9-kube-api-access-vrtcr podName:7510b8a7-e605-4db3-882f-c50c26e954b9 nodeName:}" failed. No retries permitted until 2026-06-02 21:42:05.566596576 +0000 UTC m=+10.271606507 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vrtcr" (UniqueName: "kubernetes.io/projected/7510b8a7-e605-4db3-882f-c50c26e954b9-kube-api-access-vrtcr") pod "network-check-target-zmvbb" (UID: "7510b8a7-e605-4db3-882f-c50c26e954b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:42:01.845875 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:01.843391 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:42:01.845875 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:01.843542 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5fs" podUID="03b1bb3a-4c92-4555-b5d4-3aaac39381f5" Jun 02 21:42:01.845875 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:01.843550 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:42:01.845875 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:01.843713 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zmvbb" podUID="7510b8a7-e605-4db3-882f-c50c26e954b9" Jun 02 21:42:03.845381 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:03.845344 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:42:03.845751 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:03.845482 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5fs" podUID="03b1bb3a-4c92-4555-b5d4-3aaac39381f5" Jun 02 21:42:03.845912 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:03.845889 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:42:03.845991 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:03.845971 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zmvbb" podUID="7510b8a7-e605-4db3-882f-c50c26e954b9" Jun 02 21:42:05.500708 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:05.500123 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs\") pod \"network-metrics-daemon-qw5fs\" (UID: \"03b1bb3a-4c92-4555-b5d4-3aaac39381f5\") " pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:42:05.500708 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:05.500322 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:42:05.500708 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:05.500385 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs podName:03b1bb3a-4c92-4555-b5d4-3aaac39381f5 nodeName:}" failed. No retries permitted until 2026-06-02 21:42:13.500366058 +0000 UTC m=+18.205375995 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs") pod "network-metrics-daemon-qw5fs" (UID: "03b1bb3a-4c92-4555-b5d4-3aaac39381f5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:42:05.600859 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:05.600823 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vrtcr\" (UniqueName: \"kubernetes.io/projected/7510b8a7-e605-4db3-882f-c50c26e954b9-kube-api-access-vrtcr\") pod \"network-check-target-zmvbb\" (UID: \"7510b8a7-e605-4db3-882f-c50c26e954b9\") " pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:42:05.601464 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:05.601018 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:42:05.601464 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:05.601039 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:42:05.601464 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:05.601052 2574 projected.go:194] Error preparing data for projected volume kube-api-access-vrtcr for pod openshift-network-diagnostics/network-check-target-zmvbb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:42:05.601464 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:05.601112 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7510b8a7-e605-4db3-882f-c50c26e954b9-kube-api-access-vrtcr podName:7510b8a7-e605-4db3-882f-c50c26e954b9 nodeName:}" failed. No retries permitted until 2026-06-02 21:42:13.601094607 +0000 UTC m=+18.306104535 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vrtcr" (UniqueName: "kubernetes.io/projected/7510b8a7-e605-4db3-882f-c50c26e954b9-kube-api-access-vrtcr") pod "network-check-target-zmvbb" (UID: "7510b8a7-e605-4db3-882f-c50c26e954b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:42:05.844430 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:05.844395 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:42:05.844588 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:05.844527 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5fs" podUID="03b1bb3a-4c92-4555-b5d4-3aaac39381f5" Jun 02 21:42:05.844973 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:05.844949 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:42:05.845096 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:05.845057 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zmvbb" podUID="7510b8a7-e605-4db3-882f-c50c26e954b9" Jun 02 21:42:07.842735 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:07.842704 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:42:07.843220 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:07.842714 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:42:07.843220 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:07.842834 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zmvbb" podUID="7510b8a7-e605-4db3-882f-c50c26e954b9" Jun 02 21:42:07.843220 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:07.842926 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5fs" podUID="03b1bb3a-4c92-4555-b5d4-3aaac39381f5" Jun 02 21:42:09.843368 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:09.843337 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:42:09.843767 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:09.843336 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:42:09.843767 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:09.843450 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zmvbb" podUID="7510b8a7-e605-4db3-882f-c50c26e954b9" Jun 02 21:42:09.843767 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:09.843567 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5fs" podUID="03b1bb3a-4c92-4555-b5d4-3aaac39381f5" Jun 02 21:42:11.843467 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:11.843435 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:42:11.843904 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:11.843436 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:42:11.843904 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:11.843575 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5fs" podUID="03b1bb3a-4c92-4555-b5d4-3aaac39381f5" Jun 02 21:42:11.843904 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:11.843635 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zmvbb" podUID="7510b8a7-e605-4db3-882f-c50c26e954b9" Jun 02 21:42:13.566252 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:13.566217 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs\") pod \"network-metrics-daemon-qw5fs\" (UID: \"03b1bb3a-4c92-4555-b5d4-3aaac39381f5\") " pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:42:13.566746 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:13.566372 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:42:13.566746 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:13.566453 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs podName:03b1bb3a-4c92-4555-b5d4-3aaac39381f5 nodeName:}" failed. No retries permitted until 2026-06-02 21:42:29.566429963 +0000 UTC m=+34.271439902 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs") pod "network-metrics-daemon-qw5fs" (UID: "03b1bb3a-4c92-4555-b5d4-3aaac39381f5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:42:13.667548 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:13.667511 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vrtcr\" (UniqueName: \"kubernetes.io/projected/7510b8a7-e605-4db3-882f-c50c26e954b9-kube-api-access-vrtcr\") pod \"network-check-target-zmvbb\" (UID: \"7510b8a7-e605-4db3-882f-c50c26e954b9\") " pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:42:13.667722 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:13.667666 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:42:13.667722 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:13.667694 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:42:13.667722 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:13.667706 2574 projected.go:194] Error preparing data for projected volume kube-api-access-vrtcr for pod openshift-network-diagnostics/network-check-target-zmvbb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:42:13.667881 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:13.667754 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7510b8a7-e605-4db3-882f-c50c26e954b9-kube-api-access-vrtcr podName:7510b8a7-e605-4db3-882f-c50c26e954b9 nodeName:}" failed. No retries permitted until 2026-06-02 21:42:29.667740562 +0000 UTC m=+34.372750486 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vrtcr" (UniqueName: "kubernetes.io/projected/7510b8a7-e605-4db3-882f-c50c26e954b9-kube-api-access-vrtcr") pod "network-check-target-zmvbb" (UID: "7510b8a7-e605-4db3-882f-c50c26e954b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:42:13.843551 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:13.843523 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:42:13.843688 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:13.843523 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:42:13.843688 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:13.843637 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5fs" podUID="03b1bb3a-4c92-4555-b5d4-3aaac39381f5" Jun 02 21:42:13.843784 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:13.843730 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zmvbb" podUID="7510b8a7-e605-4db3-882f-c50c26e954b9" Jun 02 21:42:15.844464 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:15.844430 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:42:15.844903 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:15.844553 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5fs" podUID="03b1bb3a-4c92-4555-b5d4-3aaac39381f5" Jun 02 21:42:15.844903 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:15.844610 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:42:15.844903 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:15.844709 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zmvbb" podUID="7510b8a7-e605-4db3-882f-c50c26e954b9" Jun 02 21:42:17.021850 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:17.021555 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dpz84" event={"ID":"b4b273c1-a82f-4e10-b0ac-671f342af106","Type":"ContainerStarted","Data":"a0accc54d237624c74bf46529fa0503877f742eca661713ca685abee271e1f69"} Jun 02 21:42:17.023158 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:17.023136 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5qkfw" event={"ID":"5252f006-96c6-4233-a1df-c1a8a1cf255a","Type":"ContainerStarted","Data":"27b4d232130b73f6b7acaed0355ee027eef99aeaf6e185710be60f70b741d833"} Jun 02 21:42:17.024322 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:17.024274 2574 generic.go:358] "Generic (PLEG): container finished" podID="4a906a0b-256c-43a0-aa65-8a8ab9a7e99a" containerID="d2eefccbb3625ac395f5093f5d0fd95264a3cdb3723a690bfae0fd0f935c8409" exitCode=0 Jun 02 21:42:17.024404 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:17.024319 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jtthq" event={"ID":"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a","Type":"ContainerDied","Data":"d2eefccbb3625ac395f5093f5d0fd95264a3cdb3723a690bfae0fd0f935c8409"} Jun 02 21:42:17.025957 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:17.025938 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/ovn-acl-logging/0.log" Jun 02 21:42:17.026261 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:17.026240 2574 generic.go:358] "Generic (PLEG): container finished" podID="974df3d7-921f-46eb-baa9-e9a37e9d93e9" containerID="fa82225d66902194bcd84e72d9f6bae8fcfd2ddefa3b88a84505478e9ee0f037" exitCode=1 Jun 02 21:42:17.026347 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:17.026317 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" event={"ID":"974df3d7-921f-46eb-baa9-e9a37e9d93e9","Type":"ContainerDied","Data":"fa82225d66902194bcd84e72d9f6bae8fcfd2ddefa3b88a84505478e9ee0f037"} Jun 02 21:42:17.026398 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:17.026346 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" event={"ID":"974df3d7-921f-46eb-baa9-e9a37e9d93e9","Type":"ContainerStarted","Data":"7a4079c6526347bc8e88131e45bebb218139075a42ae5ed07c94c2e9116b93c1"} Jun 02 21:42:17.027662 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:17.027640 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5zvps" event={"ID":"de5bcbc0-d1d5-4e3e-af7e-5824ab06cb2d","Type":"ContainerStarted","Data":"39ebbde5eae1f2ff8dd22999375d1411fada0c23dc65a6ab8539b178eb1df7ff"} Jun 02 21:42:17.028773 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:17.028755 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kvgdj" event={"ID":"be818b5e-c71a-45a2-8329-2e79ef4b6196","Type":"ContainerStarted","Data":"94acddc84e8b2ba504bc4bbdadaee5ff5ffd791d47a30dc34fb049dce0fbed97"} Jun 02 21:42:17.029899 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:17.029882 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hb594" event={"ID":"2867ec71-fbbc-43b9-90a8-ad48da269f2d","Type":"ContainerStarted","Data":"4788288f38e60ad85b31596ababeb044adf2f8ca6ca03aa8e7a58ad75df18614"} Jun 02 21:42:17.031390 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:17.031368 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" event={"ID":"efca8e3c-85ec-4437-a3bb-d12e252ca89c","Type":"ContainerStarted","Data":"d9c6ef08ce38fc353c9a5621b6cfa2f125065ee14c3495166d8cbb6b09a6c34d"} Jun 02 21:42:17.036180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:17.036145 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-dpz84" podStartSLOduration=2.771857985 podStartE2EDuration="21.036136032s" podCreationTimestamp="2026-06-02 21:41:56 +0000 UTC" firstStartedPulling="2026-06-02 21:41:58.357172186 +0000 UTC m=+3.062182111" lastFinishedPulling="2026-06-02 21:42:16.621450221 +0000 UTC m=+21.326460158" observedRunningTime="2026-06-02 21:42:17.035859321 +0000 UTC m=+21.740869270" watchObservedRunningTime="2026-06-02 21:42:17.036136032 +0000 UTC m=+21.741145979" Jun 02 21:42:17.036298 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:17.036237 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.ec2.internal" podStartSLOduration=21.03623309 podStartE2EDuration="21.03623309s" podCreationTimestamp="2026-06-02 21:41:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 21:42:00.014354675 +0000 UTC m=+4.719364623" watchObservedRunningTime="2026-06-02 21:42:17.03623309 +0000 UTC m=+21.741243037" Jun 02 21:42:17.049040 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:17.049005 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hb594" podStartSLOduration=3.862264476 podStartE2EDuration="22.04899384s" podCreationTimestamp="2026-06-02 21:41:55 +0000 UTC" firstStartedPulling="2026-06-02 21:41:58.360140169 +0000 UTC m=+3.065150098" lastFinishedPulling="2026-06-02 21:42:16.546869534 +0000 UTC m=+21.251879462" observedRunningTime="2026-06-02 21:42:17.048937111 +0000 UTC m=+21.753947067" watchObservedRunningTime="2026-06-02 21:42:17.04899384 +0000 UTC m=+21.754003787" Jun 02 21:42:17.067089 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:17.067048 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-kvgdj" podStartSLOduration=3.606789941 podStartE2EDuration="22.067036994s" podCreationTimestamp="2026-06-02 21:41:55 +0000 UTC" firstStartedPulling="2026-06-02 21:41:58.361312489 +0000 UTC m=+3.066322427" lastFinishedPulling="2026-06-02 21:42:16.821559541 +0000 UTC m=+21.526569480" observedRunningTime="2026-06-02 21:42:17.066752635 +0000 UTC m=+21.771762582" watchObservedRunningTime="2026-06-02 21:42:17.067036994 +0000 UTC m=+21.772046941" Jun 02 21:42:17.079664 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:17.079630 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5qkfw" podStartSLOduration=3.886792779 podStartE2EDuration="22.079618215s" podCreationTimestamp="2026-06-02 21:41:55 +0000 UTC" firstStartedPulling="2026-06-02 21:41:58.354112105 +0000 UTC m=+3.059122044" lastFinishedPulling="2026-06-02 21:42:16.54693755 +0000 UTC m=+21.251947480" observedRunningTime="2026-06-02 21:42:17.079164235 +0000 UTC m=+21.784174182" watchObservedRunningTime="2026-06-02 21:42:17.079618215 +0000 UTC m=+21.784628162" Jun 02 21:42:17.117916 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:17.117879 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5zvps" podStartSLOduration=2.858051894 podStartE2EDuration="21.117866441s" podCreationTimestamp="2026-06-02 21:41:56 +0000 UTC" firstStartedPulling="2026-06-02 21:41:58.36162856 +0000 UTC m=+3.066638485" lastFinishedPulling="2026-06-02 21:42:16.621443107 +0000 UTC m=+21.326453032" observedRunningTime="2026-06-02 21:42:17.117621448 +0000 UTC m=+21.822631394" watchObservedRunningTime="2026-06-02 21:42:17.117866441 +0000 UTC m=+21.822876388" Jun 02 21:42:17.843061 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:17.843034 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:42:17.843230 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:17.843034 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:42:17.843230 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:17.843145 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zmvbb" podUID="7510b8a7-e605-4db3-882f-c50c26e954b9" Jun 02 21:42:17.843377 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:17.843263 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5fs" podUID="03b1bb3a-4c92-4555-b5d4-3aaac39381f5" Jun 02 21:42:17.939202 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:17.939176 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hb594" Jun 02 21:42:17.939704 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:17.939689 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hb594" Jun 02 21:42:18.035038 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:18.034975 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kmtlg" event={"ID":"d8c7f35d-ee2f-446a-9625-9b8b102a5a9a","Type":"ContainerStarted","Data":"1c819901893c6b3c9ae2a543d511b5556ba11566864b03ecdfed7618e30d7107"} Jun 02 21:42:18.037566 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:18.037545 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/ovn-acl-logging/0.log" Jun 02 21:42:18.037987 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:18.037965 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" event={"ID":"974df3d7-921f-46eb-baa9-e9a37e9d93e9","Type":"ContainerStarted","Data":"e7840e392520402167d11a0d728464325ec7d5ddefc857ab188e4ee90c8d2197"} Jun 02 21:42:18.038099 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:18.037994 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" event={"ID":"974df3d7-921f-46eb-baa9-e9a37e9d93e9","Type":"ContainerStarted","Data":"e129572ce18359a720553bb80826254c12524691cfcacdd3af4f7e110c736544"} Jun 02 21:42:18.038099 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:18.038009 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" event={"ID":"974df3d7-921f-46eb-baa9-e9a37e9d93e9","Type":"ContainerStarted","Data":"6e1380ba635a57a40b207baeb1ff98f27da58bab8c9edf339c7c2390f15b2ab2"} Jun 02 21:42:18.038099 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:18.038020 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" event={"ID":"974df3d7-921f-46eb-baa9-e9a37e9d93e9","Type":"ContainerStarted","Data":"dbce34b9dcea9d1157c79169c3083ccef7d1e6eb666b6b5a485e809dfdb19efc"} Jun 02 21:42:18.038586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:18.038569 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hb594" Jun 02 21:42:18.038673 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:18.038608 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hb594" Jun 02 21:42:18.068314 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:18.068265 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kmtlg" podStartSLOduration=3.773934115 podStartE2EDuration="22.068248257s" podCreationTimestamp="2026-06-02 21:41:56 +0000 UTC" firstStartedPulling="2026-06-02 21:41:58.350567257 +0000 UTC m=+3.055577183" lastFinishedPulling="2026-06-02 21:42:16.644881389 +0000 UTC m=+21.349891325" observedRunningTime="2026-06-02 21:42:18.05152253 +0000 UTC m=+22.756532477" watchObservedRunningTime="2026-06-02 21:42:18.068248257 +0000 UTC m=+22.773258201" Jun 02 21:42:18.306673 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:18.306639 2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 02 21:42:18.787833 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:18.787650 2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-02T21:42:18.306659015Z","UUID":"951d3593-f7fb-4ac5-a257-4602241aab2c","Handler":null,"Name":"","Endpoint":""} Jun 02 21:42:18.789497 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:18.789468 2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 02 21:42:18.789497 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:18.789500 2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 02 21:42:19.041636 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:19.041545 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" event={"ID":"efca8e3c-85ec-4437-a3bb-d12e252ca89c","Type":"ContainerStarted","Data":"41db5f13e6305ce6ec4d64b0c4bd984a97ba2811ff904de7e0f72604e58aff34"} Jun 02 21:42:19.842944 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:19.842918 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:42:19.843109 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:19.842923 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:42:19.843109 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:19.843034 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zmvbb" podUID="7510b8a7-e605-4db3-882f-c50c26e954b9" Jun 02 21:42:19.843109 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:19.843082 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5fs" podUID="03b1bb3a-4c92-4555-b5d4-3aaac39381f5" Jun 02 21:42:20.044505 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:20.044471 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" event={"ID":"efca8e3c-85ec-4437-a3bb-d12e252ca89c","Type":"ContainerStarted","Data":"8e004944adab807b73682035e74149adc407289f7441e272d5c0a83bad6af63e"} Jun 02 21:42:20.049310 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:20.047273 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/ovn-acl-logging/0.log" Jun 02 21:42:20.050095 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:20.050065 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" event={"ID":"974df3d7-921f-46eb-baa9-e9a37e9d93e9","Type":"ContainerStarted","Data":"4b0154068c97043833c73771cd4a4bead1852805bb04b2d5b930ff68a029d316"} Jun 02 21:42:20.063556 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:20.063510 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6r77g" podStartSLOduration=4.068855786 podStartE2EDuration="25.063494242s" podCreationTimestamp="2026-06-02 21:41:55 +0000 UTC" firstStartedPulling="2026-06-02 21:41:58.359140662 +0000 UTC m=+3.064150587" lastFinishedPulling="2026-06-02 21:42:19.353779116 +0000 UTC m=+24.058789043" observedRunningTime="2026-06-02 21:42:20.063465121 +0000 UTC m=+24.768475069" watchObservedRunningTime="2026-06-02 21:42:20.063494242 +0000 UTC m=+24.768504187" Jun 02 21:42:21.843690 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:21.843514 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:42:21.844347 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:21.843570 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:42:21.844347 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:21.843793 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5fs" podUID="03b1bb3a-4c92-4555-b5d4-3aaac39381f5" Jun 02 21:42:21.844347 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:21.843865 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zmvbb" podUID="7510b8a7-e605-4db3-882f-c50c26e954b9" Jun 02 21:42:23.058391 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:23.058194 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/ovn-acl-logging/0.log" Jun 02 21:42:23.058829 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:23.058681 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" event={"ID":"974df3d7-921f-46eb-baa9-e9a37e9d93e9","Type":"ContainerStarted","Data":"88f529f7090f120f6506db88f972f55e110442248b122ed5ccffa7db9d7b8b18"} Jun 02 21:42:23.058979 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:23.058961 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:42:23.059061 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:23.058992 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:42:23.059117 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:23.059093 2574 scope.go:117] "RemoveContainer" containerID="fa82225d66902194bcd84e72d9f6bae8fcfd2ddefa3b88a84505478e9ee0f037" Jun 02 21:42:23.060364 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:23.060343 2574 generic.go:358] "Generic (PLEG): container finished" podID="4a906a0b-256c-43a0-aa65-8a8ab9a7e99a" containerID="a0f7804e31ef7e1b6dbf6f23b7efda7390d67617d2f3696ef82151b204a9a76f" exitCode=0 Jun 02 21:42:23.060431 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:23.060374 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jtthq" event={"ID":"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a","Type":"ContainerDied","Data":"a0f7804e31ef7e1b6dbf6f23b7efda7390d67617d2f3696ef82151b204a9a76f"} Jun 02 21:42:23.074613 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:23.074596 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:42:23.843536 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:23.843504 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:42:23.843536 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:23.843544 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:42:23.843764 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:23.843641 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5fs" podUID="03b1bb3a-4c92-4555-b5d4-3aaac39381f5" Jun 02 21:42:23.843822 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:23.843779 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zmvbb" podUID="7510b8a7-e605-4db3-882f-c50c26e954b9" Jun 02 21:42:24.065766 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:24.065743 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/ovn-acl-logging/0.log" Jun 02 21:42:24.066159 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:24.066049 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" event={"ID":"974df3d7-921f-46eb-baa9-e9a37e9d93e9","Type":"ContainerStarted","Data":"0e72ad17a173c988d9ea3a19442d4056545f3f3d8a7feed43767b07c92730a7c"} Jun 02 21:42:24.066365 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:24.066342 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:42:24.081226 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:24.081206 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:42:24.091459 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:24.091418 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" podStartSLOduration=9.764295925999999 podStartE2EDuration="28.091404793s" podCreationTimestamp="2026-06-02 21:41:56 +0000 UTC" firstStartedPulling="2026-06-02 21:41:58.35236168 +0000 UTC m=+3.057371622" lastFinishedPulling="2026-06-02 21:42:16.679470564 +0000 UTC m=+21.384480489" observedRunningTime="2026-06-02 21:42:24.091012345 +0000 UTC m=+28.796022291" watchObservedRunningTime="2026-06-02 21:42:24.091404793 +0000 UTC m=+28.796414739" Jun 02 21:42:24.408641 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:24.408562 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zmvbb"] Jun 02 21:42:24.408786 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:24.408672 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:42:24.408786 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:24.408769 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zmvbb" podUID="7510b8a7-e605-4db3-882f-c50c26e954b9" Jun 02 21:42:24.411400 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:24.411375 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qw5fs"] Jun 02 21:42:24.411497 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:24.411450 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:42:24.411542 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:24.411527 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5fs" podUID="03b1bb3a-4c92-4555-b5d4-3aaac39381f5" Jun 02 21:42:25.069051 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:25.069021 2574 generic.go:358] "Generic (PLEG): container finished" podID="4a906a0b-256c-43a0-aa65-8a8ab9a7e99a" containerID="f5d82c910ccee42d4ba181720fd095120d773319a49544ec2dc5dfc09dca8a27" exitCode=0 Jun 02 21:42:25.069420 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:25.069104 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jtthq" event={"ID":"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a","Type":"ContainerDied","Data":"f5d82c910ccee42d4ba181720fd095120d773319a49544ec2dc5dfc09dca8a27"} Jun 02 21:42:25.845462 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:25.845307 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:42:25.845623 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:25.845314 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:42:25.845623 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:25.845521 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zmvbb" podUID="7510b8a7-e605-4db3-882f-c50c26e954b9" Jun 02 21:42:25.845623 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:25.845589 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5fs" podUID="03b1bb3a-4c92-4555-b5d4-3aaac39381f5" Jun 02 21:42:27.074747 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:27.074718 2574 generic.go:358] "Generic (PLEG): container finished" podID="4a906a0b-256c-43a0-aa65-8a8ab9a7e99a" containerID="fc7120951ea4797f42b30233309badc0e2ba8a6c42ef89c102556370e21705a2" exitCode=0 Jun 02 21:42:27.075229 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:27.074769 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jtthq" event={"ID":"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a","Type":"ContainerDied","Data":"fc7120951ea4797f42b30233309badc0e2ba8a6c42ef89c102556370e21705a2"} Jun 02 21:42:27.843256 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:27.843222 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:42:27.843405 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:27.843380 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qw5fs" podUID="03b1bb3a-4c92-4555-b5d4-3aaac39381f5" Jun 02 21:42:27.843405 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:27.843222 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:42:27.843529 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:27.843470 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zmvbb" podUID="7510b8a7-e605-4db3-882f-c50c26e954b9" Jun 02 21:42:29.595696 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.595665 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs\") pod \"network-metrics-daemon-qw5fs\" (UID: \"03b1bb3a-4c92-4555-b5d4-3aaac39381f5\") " pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:42:29.596262 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:29.595790 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:42:29.596262 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:29.595847 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs podName:03b1bb3a-4c92-4555-b5d4-3aaac39381f5 nodeName:}" failed. No retries permitted until 2026-06-02 21:43:01.595833394 +0000 UTC m=+66.300843324 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs") pod "network-metrics-daemon-qw5fs" (UID: "03b1bb3a-4c92-4555-b5d4-3aaac39381f5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:42:29.597617 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.597595 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-201.ec2.internal" event="NodeReady" Jun 02 21:42:29.597729 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.597719 2574 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 02 21:42:29.642619 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.642592 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-r8248"] Jun 02 21:42:29.645675 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.645655 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r8248" Jun 02 21:42:29.648134 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.648110 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mgvvz"] Jun 02 21:42:29.648477 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.648345 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 02 21:42:29.648550 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.648480 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 02 21:42:29.648607 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.648568 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ssb95\"" Jun 02 21:42:29.651256 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.651240 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mgvvz" Jun 02 21:42:29.653095 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.653077 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 02 21:42:29.653190 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.653137 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 02 21:42:29.653297 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.653265 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 02 21:42:29.653403 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.653379 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-f7jrx\"" Jun 02 21:42:29.655402 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.655108 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r8248"] Jun 02 21:42:29.659231 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.659198 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mgvvz"] Jun 02 21:42:29.696018 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.695979 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf84508a-a1fe-40c8-b587-61f085f0a467-tmp-dir\") pod \"dns-default-r8248\" (UID: \"cf84508a-a1fe-40c8-b587-61f085f0a467\") " pod="openshift-dns/dns-default-r8248" Jun 02 21:42:29.696162 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.696028 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtmds\" (UniqueName: \"kubernetes.io/projected/cf84508a-a1fe-40c8-b587-61f085f0a467-kube-api-access-gtmds\") pod \"dns-default-r8248\" (UID: \"cf84508a-a1fe-40c8-b587-61f085f0a467\") " pod="openshift-dns/dns-default-r8248" Jun 02 21:42:29.696162 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.696054 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vrtcr\" (UniqueName: \"kubernetes.io/projected/7510b8a7-e605-4db3-882f-c50c26e954b9-kube-api-access-vrtcr\") pod \"network-check-target-zmvbb\" (UID: \"7510b8a7-e605-4db3-882f-c50c26e954b9\") " pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:42:29.696162 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.696110 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls\") pod \"dns-default-r8248\" (UID: \"cf84508a-a1fe-40c8-b587-61f085f0a467\") " pod="openshift-dns/dns-default-r8248" Jun 02 21:42:29.696162 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.696143 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf52v\" (UniqueName: \"kubernetes.io/projected/5409f3fd-8515-43fd-8022-23e18e6414d9-kube-api-access-sf52v\") pod \"ingress-canary-mgvvz\" (UID: \"5409f3fd-8515-43fd-8022-23e18e6414d9\") " pod="openshift-ingress-canary/ingress-canary-mgvvz" Jun 02 21:42:29.696162 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:29.696156 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:42:29.696456 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:29.696171 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:42:29.696456 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.696172 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cf84508a-a1fe-40c8-b587-61f085f0a467-config-volume\") pod \"dns-default-r8248\" (UID: \"cf84508a-a1fe-40c8-b587-61f085f0a467\") " pod="openshift-dns/dns-default-r8248" Jun 02 21:42:29.696456 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:29.696184 2574 projected.go:194] Error preparing data for projected volume kube-api-access-vrtcr for pod openshift-network-diagnostics/network-check-target-zmvbb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:42:29.696456 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.696207 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert\") pod \"ingress-canary-mgvvz\" (UID: \"5409f3fd-8515-43fd-8022-23e18e6414d9\") " pod="openshift-ingress-canary/ingress-canary-mgvvz" Jun 02 21:42:29.696456 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:29.696235 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7510b8a7-e605-4db3-882f-c50c26e954b9-kube-api-access-vrtcr podName:7510b8a7-e605-4db3-882f-c50c26e954b9 nodeName:}" failed. No retries permitted until 2026-06-02 21:43:01.696218007 +0000 UTC m=+66.401227960 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vrtcr" (UniqueName: "kubernetes.io/projected/7510b8a7-e605-4db3-882f-c50c26e954b9-kube-api-access-vrtcr") pod "network-check-target-zmvbb" (UID: "7510b8a7-e605-4db3-882f-c50c26e954b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:42:29.796486 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.796451 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gtmds\" (UniqueName: \"kubernetes.io/projected/cf84508a-a1fe-40c8-b587-61f085f0a467-kube-api-access-gtmds\") pod \"dns-default-r8248\" (UID: \"cf84508a-a1fe-40c8-b587-61f085f0a467\") " pod="openshift-dns/dns-default-r8248" Jun 02 21:42:29.796639 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.796515 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls\") pod \"dns-default-r8248\" (UID: \"cf84508a-a1fe-40c8-b587-61f085f0a467\") " pod="openshift-dns/dns-default-r8248" Jun 02 21:42:29.796639 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.796541 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sf52v\" (UniqueName: \"kubernetes.io/projected/5409f3fd-8515-43fd-8022-23e18e6414d9-kube-api-access-sf52v\") pod \"ingress-canary-mgvvz\" (UID: \"5409f3fd-8515-43fd-8022-23e18e6414d9\") " pod="openshift-ingress-canary/ingress-canary-mgvvz" Jun 02 21:42:29.796639 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.796567 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cf84508a-a1fe-40c8-b587-61f085f0a467-config-volume\") pod \"dns-default-r8248\" (UID: \"cf84508a-a1fe-40c8-b587-61f085f0a467\") " pod="openshift-dns/dns-default-r8248" Jun 02 21:42:29.796639 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.796592 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert\") pod \"ingress-canary-mgvvz\" (UID: \"5409f3fd-8515-43fd-8022-23e18e6414d9\") " pod="openshift-ingress-canary/ingress-canary-mgvvz" Jun 02 21:42:29.796639 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.796633 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf84508a-a1fe-40c8-b587-61f085f0a467-tmp-dir\") pod \"dns-default-r8248\" (UID: \"cf84508a-a1fe-40c8-b587-61f085f0a467\") " pod="openshift-dns/dns-default-r8248" Jun 02 21:42:29.796814 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:29.796699 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:42:29.796814 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:29.796763 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls podName:cf84508a-a1fe-40c8-b587-61f085f0a467 nodeName:}" failed. No retries permitted until 2026-06-02 21:42:30.296742769 +0000 UTC m=+35.001752714 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls") pod "dns-default-r8248" (UID: "cf84508a-a1fe-40c8-b587-61f085f0a467") : secret "dns-default-metrics-tls" not found Jun 02 21:42:29.796915 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:29.796899 2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:42:29.796961 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:29.796950 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert podName:5409f3fd-8515-43fd-8022-23e18e6414d9 nodeName:}" failed. No retries permitted until 2026-06-02 21:42:30.296934021 +0000 UTC m=+35.001943958 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert") pod "ingress-canary-mgvvz" (UID: "5409f3fd-8515-43fd-8022-23e18e6414d9") : secret "canary-serving-cert" not found Jun 02 21:42:29.797057 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.796961 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf84508a-a1fe-40c8-b587-61f085f0a467-tmp-dir\") pod \"dns-default-r8248\" (UID: \"cf84508a-a1fe-40c8-b587-61f085f0a467\") " pod="openshift-dns/dns-default-r8248" Jun 02 21:42:29.797325 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.797278 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cf84508a-a1fe-40c8-b587-61f085f0a467-config-volume\") pod \"dns-default-r8248\" (UID: \"cf84508a-a1fe-40c8-b587-61f085f0a467\") " pod="openshift-dns/dns-default-r8248" Jun 02 21:42:29.806573 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.806552 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf52v\" (UniqueName: \"kubernetes.io/projected/5409f3fd-8515-43fd-8022-23e18e6414d9-kube-api-access-sf52v\") pod \"ingress-canary-mgvvz\" (UID: \"5409f3fd-8515-43fd-8022-23e18e6414d9\") " pod="openshift-ingress-canary/ingress-canary-mgvvz" Jun 02 21:42:29.806923 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.806901 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtmds\" (UniqueName: \"kubernetes.io/projected/cf84508a-a1fe-40c8-b587-61f085f0a467-kube-api-access-gtmds\") pod \"dns-default-r8248\" (UID: \"cf84508a-a1fe-40c8-b587-61f085f0a467\") " pod="openshift-dns/dns-default-r8248" Jun 02 21:42:29.843428 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.843408 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:42:29.843652 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.843625 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:42:29.846067 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.846004 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 21:42:29.846067 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.846039 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 21:42:29.846215 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.846090 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 21:42:29.846215 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.846132 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-96ccb\"" Jun 02 21:42:29.846424 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:29.846408 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8cvg9\"" Jun 02 21:42:30.299755 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:30.299676 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls\") pod \"dns-default-r8248\" (UID: \"cf84508a-a1fe-40c8-b587-61f085f0a467\") " pod="openshift-dns/dns-default-r8248" Jun 02 21:42:30.299755 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:30.299713 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert\") pod \"ingress-canary-mgvvz\" (UID: \"5409f3fd-8515-43fd-8022-23e18e6414d9\") " pod="openshift-ingress-canary/ingress-canary-mgvvz" Jun 02 21:42:30.299980 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:30.299837 2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:42:30.299980 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:30.299840 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:42:30.299980 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:30.299889 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert podName:5409f3fd-8515-43fd-8022-23e18e6414d9 nodeName:}" failed. No retries permitted until 2026-06-02 21:42:31.299876075 +0000 UTC m=+36.004886001 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert") pod "ingress-canary-mgvvz" (UID: "5409f3fd-8515-43fd-8022-23e18e6414d9") : secret "canary-serving-cert" not found Jun 02 21:42:30.299980 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:30.299939 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls podName:cf84508a-a1fe-40c8-b587-61f085f0a467 nodeName:}" failed. No retries permitted until 2026-06-02 21:42:31.299921235 +0000 UTC m=+36.004931162 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls") pod "dns-default-r8248" (UID: "cf84508a-a1fe-40c8-b587-61f085f0a467") : secret "dns-default-metrics-tls" not found Jun 02 21:42:31.306373 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:31.306333 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert\") pod \"ingress-canary-mgvvz\" (UID: \"5409f3fd-8515-43fd-8022-23e18e6414d9\") " pod="openshift-ingress-canary/ingress-canary-mgvvz" Jun 02 21:42:31.306966 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:31.306439 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls\") pod \"dns-default-r8248\" (UID: \"cf84508a-a1fe-40c8-b587-61f085f0a467\") " pod="openshift-dns/dns-default-r8248" Jun 02 21:42:31.306966 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:31.306490 2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:42:31.306966 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:31.306530 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:42:31.306966 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:31.306547 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert podName:5409f3fd-8515-43fd-8022-23e18e6414d9 nodeName:}" failed. No retries permitted until 2026-06-02 21:42:33.306533529 +0000 UTC m=+38.011543454 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert") pod "ingress-canary-mgvvz" (UID: "5409f3fd-8515-43fd-8022-23e18e6414d9") : secret "canary-serving-cert" not found Jun 02 21:42:31.306966 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:31.306573 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls podName:cf84508a-a1fe-40c8-b587-61f085f0a467 nodeName:}" failed. No retries permitted until 2026-06-02 21:42:33.306557988 +0000 UTC m=+38.011567930 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls") pod "dns-default-r8248" (UID: "cf84508a-a1fe-40c8-b587-61f085f0a467") : secret "dns-default-metrics-tls" not found Jun 02 21:42:33.088080 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:33.087926 2574 generic.go:358] "Generic (PLEG): container finished" podID="4a906a0b-256c-43a0-aa65-8a8ab9a7e99a" containerID="bead06101dddcf3840fd8bfd7608562f42eac60a4895262593790c8d15754028" exitCode=0 Jun 02 21:42:33.088396 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:33.088009 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jtthq" event={"ID":"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a","Type":"ContainerDied","Data":"bead06101dddcf3840fd8bfd7608562f42eac60a4895262593790c8d15754028"} Jun 02 21:42:33.317736 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:33.317709 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls\") pod \"dns-default-r8248\" (UID: \"cf84508a-a1fe-40c8-b587-61f085f0a467\") " pod="openshift-dns/dns-default-r8248" Jun 02 21:42:33.317830 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:33.317745 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert\") pod \"ingress-canary-mgvvz\" (UID: \"5409f3fd-8515-43fd-8022-23e18e6414d9\") " pod="openshift-ingress-canary/ingress-canary-mgvvz" Jun 02 21:42:33.317872 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:33.317848 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:42:33.317911 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:33.317901 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls podName:cf84508a-a1fe-40c8-b587-61f085f0a467 nodeName:}" failed. No retries permitted until 2026-06-02 21:42:37.317887862 +0000 UTC m=+42.022897792 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls") pod "dns-default-r8248" (UID: "cf84508a-a1fe-40c8-b587-61f085f0a467") : secret "dns-default-metrics-tls" not found Jun 02 21:42:33.317952 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:33.317848 2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:42:33.317983 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:33.317977 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert podName:5409f3fd-8515-43fd-8022-23e18e6414d9 nodeName:}" failed. No retries permitted until 2026-06-02 21:42:37.317965606 +0000 UTC m=+42.022975535 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert") pod "ingress-canary-mgvvz" (UID: "5409f3fd-8515-43fd-8022-23e18e6414d9") : secret "canary-serving-cert" not found Jun 02 21:42:34.092696 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:34.092666 2574 generic.go:358] "Generic (PLEG): container finished" podID="4a906a0b-256c-43a0-aa65-8a8ab9a7e99a" containerID="dfe0d5c23c88f12e7f9f415165edbb08517e1e099e2ba73238c6312e62f52501" exitCode=0 Jun 02 21:42:34.093009 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:34.092716 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jtthq" event={"ID":"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a","Type":"ContainerDied","Data":"dfe0d5c23c88f12e7f9f415165edbb08517e1e099e2ba73238c6312e62f52501"} Jun 02 21:42:35.096663 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.096630 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jtthq" event={"ID":"4a906a0b-256c-43a0-aa65-8a8ab9a7e99a","Type":"ContainerStarted","Data":"ebe335c66a596045bf847431568b8ac391172af233f78c29f4f63ba91f619ee0"} Jun 02 21:42:35.116759 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.116706 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jtthq" podStartSLOduration=4.80818823 podStartE2EDuration="39.116690072s" podCreationTimestamp="2026-06-02 21:41:56 +0000 UTC" firstStartedPulling="2026-06-02 21:41:58.353710277 +0000 UTC m=+3.058720203" lastFinishedPulling="2026-06-02 21:42:32.662212121 +0000 UTC m=+37.367222045" observedRunningTime="2026-06-02 21:42:35.115385081 +0000 UTC m=+39.820395028" watchObservedRunningTime="2026-06-02 21:42:35.116690072 +0000 UTC m=+39.821700022" Jun 02 21:42:35.224594 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.224571 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx"] Jun 02 21:42:35.246805 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.246787 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx"] Jun 02 21:42:35.246888 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.246883 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" Jun 02 21:42:35.249430 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.249407 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jun 02 21:42:35.249542 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.249424 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 02 21:42:35.249542 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.249444 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jun 02 21:42:35.249542 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.249506 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jun 02 21:42:35.249703 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.249633 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 02 21:42:35.249829 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.249815 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 02 21:42:35.249866 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.249850 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jun 02 21:42:35.328228 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.328203 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d128b979-2533-4ca6-b009-111b33e8188f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx\" (UID: \"d128b979-2533-4ca6-b009-111b33e8188f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" Jun 02 21:42:35.328337 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.328245 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d128b979-2533-4ca6-b009-111b33e8188f-hub\") pod \"cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx\" (UID: \"d128b979-2533-4ca6-b009-111b33e8188f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" Jun 02 21:42:35.328337 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.328265 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f8sc\" (UniqueName: \"kubernetes.io/projected/d128b979-2533-4ca6-b009-111b33e8188f-kube-api-access-6f8sc\") pod \"cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx\" (UID: \"d128b979-2533-4ca6-b009-111b33e8188f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" Jun 02 21:42:35.328409 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.328335 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d128b979-2533-4ca6-b009-111b33e8188f-ca\") pod \"cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx\" (UID: \"d128b979-2533-4ca6-b009-111b33e8188f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" Jun 02 21:42:35.328409 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.328361 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d128b979-2533-4ca6-b009-111b33e8188f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx\" (UID: \"d128b979-2533-4ca6-b009-111b33e8188f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" Jun 02 21:42:35.328470 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.328408 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/d128b979-2533-4ca6-b009-111b33e8188f-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx\" (UID: \"d128b979-2533-4ca6-b009-111b33e8188f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" Jun 02 21:42:35.429599 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.429551 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d128b979-2533-4ca6-b009-111b33e8188f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx\" (UID: \"d128b979-2533-4ca6-b009-111b33e8188f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" Jun 02 21:42:35.429599 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.429592 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d128b979-2533-4ca6-b009-111b33e8188f-hub\") pod \"cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx\" (UID: \"d128b979-2533-4ca6-b009-111b33e8188f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" Jun 02 21:42:35.429714 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.429612 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6f8sc\" (UniqueName: \"kubernetes.io/projected/d128b979-2533-4ca6-b009-111b33e8188f-kube-api-access-6f8sc\") pod \"cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx\" (UID: \"d128b979-2533-4ca6-b009-111b33e8188f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" Jun 02 21:42:35.429714 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.429633 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d128b979-2533-4ca6-b009-111b33e8188f-ca\") pod \"cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx\" (UID: \"d128b979-2533-4ca6-b009-111b33e8188f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" Jun 02 21:42:35.429714 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.429657 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d128b979-2533-4ca6-b009-111b33e8188f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx\" (UID: \"d128b979-2533-4ca6-b009-111b33e8188f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" Jun 02 21:42:35.429714 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.429691 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/d128b979-2533-4ca6-b009-111b33e8188f-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx\" (UID: \"d128b979-2533-4ca6-b009-111b33e8188f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" Jun 02 21:42:35.430408 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.430388 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/d128b979-2533-4ca6-b009-111b33e8188f-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx\" (UID: \"d128b979-2533-4ca6-b009-111b33e8188f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" Jun 02 21:42:35.432766 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.432741 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d128b979-2533-4ca6-b009-111b33e8188f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx\" (UID: \"d128b979-2533-4ca6-b009-111b33e8188f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" Jun 02 21:42:35.432870 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.432826 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d128b979-2533-4ca6-b009-111b33e8188f-hub\") pod \"cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx\" (UID: \"d128b979-2533-4ca6-b009-111b33e8188f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" Jun 02 21:42:35.432978 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.432961 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d128b979-2533-4ca6-b009-111b33e8188f-ca\") pod \"cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx\" (UID: \"d128b979-2533-4ca6-b009-111b33e8188f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" Jun 02 21:42:35.433030 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.433011 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d128b979-2533-4ca6-b009-111b33e8188f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx\" (UID: \"d128b979-2533-4ca6-b009-111b33e8188f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" Jun 02 21:42:35.438040 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.438018 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f8sc\" (UniqueName: \"kubernetes.io/projected/d128b979-2533-4ca6-b009-111b33e8188f-kube-api-access-6f8sc\") pod \"cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx\" (UID: \"d128b979-2533-4ca6-b009-111b33e8188f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" Jun 02 21:42:35.569783 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.569757 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" Jun 02 21:42:35.714747 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:35.714688 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx"] Jun 02 21:42:35.718239 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:42:35.718215 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd128b979_2533_4ca6_b009_111b33e8188f.slice/crio-0821163fb2e26be09ea2b42a2ea7ed0ba8ff498e372f71e569df6b7fbafc3034 WatchSource:0}: Error finding container 0821163fb2e26be09ea2b42a2ea7ed0ba8ff498e372f71e569df6b7fbafc3034: Status 404 returned error can't find the container with id 0821163fb2e26be09ea2b42a2ea7ed0ba8ff498e372f71e569df6b7fbafc3034 Jun 02 21:42:36.099071 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:36.099043 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" event={"ID":"d128b979-2533-4ca6-b009-111b33e8188f","Type":"ContainerStarted","Data":"0821163fb2e26be09ea2b42a2ea7ed0ba8ff498e372f71e569df6b7fbafc3034"} Jun 02 21:42:37.344100 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:37.344066 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls\") pod \"dns-default-r8248\" (UID: \"cf84508a-a1fe-40c8-b587-61f085f0a467\") " pod="openshift-dns/dns-default-r8248" Jun 02 21:42:37.344100 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:37.344103 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert\") pod \"ingress-canary-mgvvz\" (UID: \"5409f3fd-8515-43fd-8022-23e18e6414d9\") " pod="openshift-ingress-canary/ingress-canary-mgvvz" Jun 02 21:42:37.344611 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:37.344227 2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:42:37.344611 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:37.344231 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:42:37.344611 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:37.344309 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert podName:5409f3fd-8515-43fd-8022-23e18e6414d9 nodeName:}" failed. No retries permitted until 2026-06-02 21:42:45.344273211 +0000 UTC m=+50.049283139 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert") pod "ingress-canary-mgvvz" (UID: "5409f3fd-8515-43fd-8022-23e18e6414d9") : secret "canary-serving-cert" not found Jun 02 21:42:37.344611 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:37.344329 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls podName:cf84508a-a1fe-40c8-b587-61f085f0a467 nodeName:}" failed. No retries permitted until 2026-06-02 21:42:45.344319516 +0000 UTC m=+50.049329447 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls") pod "dns-default-r8248" (UID: "cf84508a-a1fe-40c8-b587-61f085f0a467") : secret "dns-default-metrics-tls" not found Jun 02 21:42:40.108280 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:40.108246 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" event={"ID":"d128b979-2533-4ca6-b009-111b33e8188f","Type":"ContainerStarted","Data":"8c45ecc5844e7bac4b5ed903600fc74b4f119c2fca7b8df6c2214de02772cc59"} Jun 02 21:42:44.118339 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:44.118307 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" event={"ID":"d128b979-2533-4ca6-b009-111b33e8188f","Type":"ContainerStarted","Data":"23ad5c8a2a69e147809e1ecbd78f6e00f9a21aceb7da8125b57baef537decc5c"} Jun 02 21:42:44.118339 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:44.118341 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" event={"ID":"d128b979-2533-4ca6-b009-111b33e8188f","Type":"ContainerStarted","Data":"951273b24de40f4deefec45da59126f256d8642977667387bae07ace5939953a"} Jun 02 21:42:44.134868 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:44.134828 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" podStartSLOduration=1.288870011 podStartE2EDuration="9.134815879s" podCreationTimestamp="2026-06-02 21:42:35 +0000 UTC" firstStartedPulling="2026-06-02 21:42:35.719934479 +0000 UTC m=+40.424944407" lastFinishedPulling="2026-06-02 21:42:43.565880348 +0000 UTC m=+48.270890275" observedRunningTime="2026-06-02 21:42:44.133414346 +0000 UTC m=+48.838424294" watchObservedRunningTime="2026-06-02 21:42:44.134815879 +0000 UTC m=+48.839825825" Jun 02 21:42:45.402564 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:45.402526 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls\") pod \"dns-default-r8248\" (UID: \"cf84508a-a1fe-40c8-b587-61f085f0a467\") " pod="openshift-dns/dns-default-r8248" Jun 02 21:42:45.402564 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:45.402566 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert\") pod \"ingress-canary-mgvvz\" (UID: \"5409f3fd-8515-43fd-8022-23e18e6414d9\") " pod="openshift-ingress-canary/ingress-canary-mgvvz" Jun 02 21:42:45.403001 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:45.402647 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:42:45.403001 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:45.402706 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls podName:cf84508a-a1fe-40c8-b587-61f085f0a467 nodeName:}" failed. No retries permitted until 2026-06-02 21:43:01.402690544 +0000 UTC m=+66.107700469 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls") pod "dns-default-r8248" (UID: "cf84508a-a1fe-40c8-b587-61f085f0a467") : secret "dns-default-metrics-tls" not found Jun 02 21:42:45.403001 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:45.402651 2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:42:45.403001 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:42:45.402738 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert podName:5409f3fd-8515-43fd-8022-23e18e6414d9 nodeName:}" failed. No retries permitted until 2026-06-02 21:43:01.402732373 +0000 UTC m=+66.107742297 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert") pod "ingress-canary-mgvvz" (UID: "5409f3fd-8515-43fd-8022-23e18e6414d9") : secret "canary-serving-cert" not found Jun 02 21:42:56.080650 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:42:56.080624 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fddpn" Jun 02 21:43:01.409962 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:01.409923 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls\") pod \"dns-default-r8248\" (UID: \"cf84508a-a1fe-40c8-b587-61f085f0a467\") " pod="openshift-dns/dns-default-r8248" Jun 02 21:43:01.409962 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:01.409966 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert\") pod \"ingress-canary-mgvvz\" (UID: \"5409f3fd-8515-43fd-8022-23e18e6414d9\") " pod="openshift-ingress-canary/ingress-canary-mgvvz" Jun 02 21:43:01.410378 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:01.410060 2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:43:01.410378 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:01.410063 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:43:01.410378 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:01.410118 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert podName:5409f3fd-8515-43fd-8022-23e18e6414d9 nodeName:}" failed. No retries permitted until 2026-06-02 21:43:33.410105651 +0000 UTC m=+98.115115576 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert") pod "ingress-canary-mgvvz" (UID: "5409f3fd-8515-43fd-8022-23e18e6414d9") : secret "canary-serving-cert" not found Jun 02 21:43:01.410378 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:01.410131 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls podName:cf84508a-a1fe-40c8-b587-61f085f0a467 nodeName:}" failed. No retries permitted until 2026-06-02 21:43:33.410125269 +0000 UTC m=+98.115135194 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls") pod "dns-default-r8248" (UID: "cf84508a-a1fe-40c8-b587-61f085f0a467") : secret "dns-default-metrics-tls" not found Jun 02 21:43:01.611452 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:01.611419 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs\") pod \"network-metrics-daemon-qw5fs\" (UID: \"03b1bb3a-4c92-4555-b5d4-3aaac39381f5\") " pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:43:01.613814 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:01.613798 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 21:43:01.622395 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:01.622378 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 02 21:43:01.622448 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:01.622435 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs podName:03b1bb3a-4c92-4555-b5d4-3aaac39381f5 nodeName:}" failed. No retries permitted until 2026-06-02 21:44:05.622420431 +0000 UTC m=+130.327430356 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs") pod "network-metrics-daemon-qw5fs" (UID: "03b1bb3a-4c92-4555-b5d4-3aaac39381f5") : secret "metrics-daemon-secret" not found Jun 02 21:43:01.712411 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:01.712337 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vrtcr\" (UniqueName: \"kubernetes.io/projected/7510b8a7-e605-4db3-882f-c50c26e954b9-kube-api-access-vrtcr\") pod \"network-check-target-zmvbb\" (UID: \"7510b8a7-e605-4db3-882f-c50c26e954b9\") " pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:43:01.714807 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:01.714789 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 21:43:01.725066 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:01.725044 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 21:43:01.736009 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:01.735983 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrtcr\" (UniqueName: \"kubernetes.io/projected/7510b8a7-e605-4db3-882f-c50c26e954b9-kube-api-access-vrtcr\") pod \"network-check-target-zmvbb\" (UID: \"7510b8a7-e605-4db3-882f-c50c26e954b9\") " pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:43:01.964317 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:01.964206 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8cvg9\"" Jun 02 21:43:01.972878 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:01.972857 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:43:02.098587 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:02.098557 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zmvbb"] Jun 02 21:43:02.102276 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:43:02.102247 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7510b8a7_e605_4db3_882f_c50c26e954b9.slice/crio-a6e420cea54a74947eac5d9aa65d47e64e256240db4cd4360d031b48eafd5ba2 WatchSource:0}: Error finding container a6e420cea54a74947eac5d9aa65d47e64e256240db4cd4360d031b48eafd5ba2: Status 404 returned error can't find the container with id a6e420cea54a74947eac5d9aa65d47e64e256240db4cd4360d031b48eafd5ba2 Jun 02 21:43:02.153840 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:02.153807 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zmvbb" event={"ID":"7510b8a7-e605-4db3-882f-c50c26e954b9","Type":"ContainerStarted","Data":"a6e420cea54a74947eac5d9aa65d47e64e256240db4cd4360d031b48eafd5ba2"} Jun 02 21:43:05.160492 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:05.160463 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zmvbb" event={"ID":"7510b8a7-e605-4db3-882f-c50c26e954b9","Type":"ContainerStarted","Data":"80737ce592b0a8474ac3182eafd304119768ceddb1721eeef3e6531d776e9e8f"} Jun 02 21:43:05.160828 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:05.160585 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:43:05.173554 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:05.173512 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zmvbb" podStartSLOduration=66.45657241 podStartE2EDuration="1m9.173500673s" podCreationTimestamp="2026-06-02 21:41:56 +0000 UTC" firstStartedPulling="2026-06-02 21:43:02.104205784 +0000 UTC m=+66.809215709" lastFinishedPulling="2026-06-02 21:43:04.821134048 +0000 UTC m=+69.526143972" observedRunningTime="2026-06-02 21:43:05.173371704 +0000 UTC m=+69.878381663" watchObservedRunningTime="2026-06-02 21:43:05.173500673 +0000 UTC m=+69.878510619" Jun 02 21:43:23.942753 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:23.942645 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dpz84_b4b273c1-a82f-4e10-b0ac-671f342af106/dns-node-resolver/0.log" Jun 02 21:43:24.543057 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:24.543034 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5qkfw_5252f006-96c6-4233-a1df-c1a8a1cf255a/node-ca/0.log" Jun 02 21:43:26.477402 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.477372 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-559f464548-hrcsf"] Jun 02 21:43:26.481408 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.481383 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:26.484164 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.484136 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 02 21:43:26.484311 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.484168 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 02 21:43:26.484311 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.484211 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 02 21:43:26.484665 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.484651 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jtd5q\"" Jun 02 21:43:26.490203 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.490183 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 02 21:43:26.492826 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.492803 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-559f464548-hrcsf"] Jun 02 21:43:26.582007 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.581970 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-tls\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:26.582007 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.582012 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ae671a73-b0b0-4d22-979e-5aa8b509daaf-trusted-ca\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:26.582193 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.582029 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-bound-sa-token\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:26.582193 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.582063 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ae671a73-b0b0-4d22-979e-5aa8b509daaf-image-registry-private-configuration\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:26.582193 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.582094 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ae671a73-b0b0-4d22-979e-5aa8b509daaf-installation-pull-secrets\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:26.582193 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.582110 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ql5b\" (UniqueName: \"kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-kube-api-access-5ql5b\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:26.582342 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.582203 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ae671a73-b0b0-4d22-979e-5aa8b509daaf-ca-trust-extracted\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:26.582342 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.582228 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-certificates\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:26.682778 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.682742 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-tls\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:26.682778 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.682781 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ae671a73-b0b0-4d22-979e-5aa8b509daaf-trusted-ca\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:26.682974 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.682797 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-bound-sa-token\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:26.682974 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.682831 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ae671a73-b0b0-4d22-979e-5aa8b509daaf-image-registry-private-configuration\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:26.682974 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.682858 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ae671a73-b0b0-4d22-979e-5aa8b509daaf-installation-pull-secrets\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:26.682974 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:26.682876 2574 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 21:43:26.682974 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.682883 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5ql5b\" (UniqueName: \"kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-kube-api-access-5ql5b\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:26.682974 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:26.682890 2574 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559f464548-hrcsf: secret "image-registry-tls" not found Jun 02 21:43:26.682974 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:26.682941 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-tls podName:ae671a73-b0b0-4d22-979e-5aa8b509daaf nodeName:}" failed. No retries permitted until 2026-06-02 21:43:27.182923842 +0000 UTC m=+91.887933766 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-tls") pod "image-registry-559f464548-hrcsf" (UID: "ae671a73-b0b0-4d22-979e-5aa8b509daaf") : secret "image-registry-tls" not found Jun 02 21:43:26.683329 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.683018 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ae671a73-b0b0-4d22-979e-5aa8b509daaf-ca-trust-extracted\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:26.683329 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.683045 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-certificates\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:26.683449 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.683431 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ae671a73-b0b0-4d22-979e-5aa8b509daaf-ca-trust-extracted\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:26.683660 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.683636 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-certificates\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:26.683830 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.683810 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ae671a73-b0b0-4d22-979e-5aa8b509daaf-trusted-ca\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:26.685382 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.685356 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ae671a73-b0b0-4d22-979e-5aa8b509daaf-installation-pull-secrets\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:26.685478 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.685397 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ae671a73-b0b0-4d22-979e-5aa8b509daaf-image-registry-private-configuration\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:26.691713 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.691689 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-bound-sa-token\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:26.692055 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:26.692031 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ql5b\" (UniqueName: \"kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-kube-api-access-5ql5b\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:27.186388 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:27.186360 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-tls\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:27.186537 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:27.186473 2574 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 21:43:27.186537 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:27.186484 2574 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559f464548-hrcsf: secret "image-registry-tls" not found Jun 02 21:43:27.186537 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:27.186537 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-tls podName:ae671a73-b0b0-4d22-979e-5aa8b509daaf nodeName:}" failed. No retries permitted until 2026-06-02 21:43:28.186522933 +0000 UTC m=+92.891532858 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-tls") pod "image-registry-559f464548-hrcsf" (UID: "ae671a73-b0b0-4d22-979e-5aa8b509daaf") : secret "image-registry-tls" not found Jun 02 21:43:28.191477 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:28.191448 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-tls\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:28.191865 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:28.191584 2574 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 21:43:28.191865 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:28.191600 2574 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559f464548-hrcsf: secret "image-registry-tls" not found Jun 02 21:43:28.191865 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:28.191644 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-tls podName:ae671a73-b0b0-4d22-979e-5aa8b509daaf nodeName:}" failed. No retries permitted until 2026-06-02 21:43:30.191630623 +0000 UTC m=+94.896640548 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-tls") pod "image-registry-559f464548-hrcsf" (UID: "ae671a73-b0b0-4d22-979e-5aa8b509daaf") : secret "image-registry-tls" not found Jun 02 21:43:30.205197 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:30.205165 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-tls\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:30.205795 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:30.205347 2574 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 21:43:30.205795 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:30.205372 2574 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559f464548-hrcsf: secret "image-registry-tls" not found Jun 02 21:43:30.205795 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:30.205444 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-tls podName:ae671a73-b0b0-4d22-979e-5aa8b509daaf nodeName:}" failed. No retries permitted until 2026-06-02 21:43:34.205424944 +0000 UTC m=+98.910434872 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-tls") pod "image-registry-559f464548-hrcsf" (UID: "ae671a73-b0b0-4d22-979e-5aa8b509daaf") : secret "image-registry-tls" not found Jun 02 21:43:33.425417 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:33.425371 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls\") pod \"dns-default-r8248\" (UID: \"cf84508a-a1fe-40c8-b587-61f085f0a467\") " pod="openshift-dns/dns-default-r8248" Jun 02 21:43:33.425783 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:33.425431 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert\") pod \"ingress-canary-mgvvz\" (UID: \"5409f3fd-8515-43fd-8022-23e18e6414d9\") " pod="openshift-ingress-canary/ingress-canary-mgvvz" Jun 02 21:43:33.425783 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:33.425530 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:43:33.425783 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:33.425590 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls podName:cf84508a-a1fe-40c8-b587-61f085f0a467 nodeName:}" failed. No retries permitted until 2026-06-02 21:44:37.425576598 +0000 UTC m=+162.130586526 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls") pod "dns-default-r8248" (UID: "cf84508a-a1fe-40c8-b587-61f085f0a467") : secret "dns-default-metrics-tls" not found Jun 02 21:43:33.425783 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:33.425530 2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:43:33.425783 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:33.425646 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert podName:5409f3fd-8515-43fd-8022-23e18e6414d9 nodeName:}" failed. No retries permitted until 2026-06-02 21:44:37.425636128 +0000 UTC m=+162.130646054 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert") pod "ingress-canary-mgvvz" (UID: "5409f3fd-8515-43fd-8022-23e18e6414d9") : secret "canary-serving-cert" not found Jun 02 21:43:34.232112 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:34.232040 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-tls\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:34.232329 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:34.232183 2574 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 21:43:34.232329 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:34.232198 2574 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559f464548-hrcsf: secret "image-registry-tls" not found Jun 02 21:43:34.232329 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:34.232254 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-tls podName:ae671a73-b0b0-4d22-979e-5aa8b509daaf nodeName:}" failed. No retries permitted until 2026-06-02 21:43:42.232236614 +0000 UTC m=+106.937246541 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-tls") pod "image-registry-559f464548-hrcsf" (UID: "ae671a73-b0b0-4d22-979e-5aa8b509daaf") : secret "image-registry-tls" not found Jun 02 21:43:36.164868 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:36.164837 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zmvbb" Jun 02 21:43:42.290396 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:42.290357 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-tls\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:42.292544 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:42.292523 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-tls\") pod \"image-registry-559f464548-hrcsf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:42.391764 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:42.391679 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:42.513672 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:42.513641 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-559f464548-hrcsf"] Jun 02 21:43:42.517081 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:43:42.517053 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae671a73_b0b0_4d22_979e_5aa8b509daaf.slice/crio-f208483a638eadaa1df47b45ee67e3358fb1d10e4b9d77512ecfe930a3a0fa3e WatchSource:0}: Error finding container f208483a638eadaa1df47b45ee67e3358fb1d10e4b9d77512ecfe930a3a0fa3e: Status 404 returned error can't find the container with id f208483a638eadaa1df47b45ee67e3358fb1d10e4b9d77512ecfe930a3a0fa3e Jun 02 21:43:43.232313 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:43.232265 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-559f464548-hrcsf" event={"ID":"ae671a73-b0b0-4d22-979e-5aa8b509daaf","Type":"ContainerStarted","Data":"e82198d31aa5c279963454cc2a67d46f0478a6248b5ba7f5c80bb88442031183"} Jun 02 21:43:43.232313 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:43.232315 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-559f464548-hrcsf" event={"ID":"ae671a73-b0b0-4d22-979e-5aa8b509daaf","Type":"ContainerStarted","Data":"f208483a638eadaa1df47b45ee67e3358fb1d10e4b9d77512ecfe930a3a0fa3e"} Jun 02 21:43:43.232501 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:43.232346 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:43:43.256908 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:43.256866 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-559f464548-hrcsf" podStartSLOduration=17.256852953 podStartE2EDuration="17.256852953s" podCreationTimestamp="2026-06-02 21:43:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 21:43:43.255767438 +0000 UTC m=+107.960777386" watchObservedRunningTime="2026-06-02 21:43:43.256852953 +0000 UTC m=+107.961862900" Jun 02 21:43:52.278721 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.278679 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-559f464548-hrcsf"] Jun 02 21:43:52.383810 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.383778 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zm72h"] Jun 02 21:43:52.386763 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.386735 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zm72h" Jun 02 21:43:52.389199 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.389183 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 02 21:43:52.389199 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.389190 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 02 21:43:52.389571 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.389551 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 02 21:43:52.389699 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.389683 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 02 21:43:52.389808 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.389792 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tbwlq\"" Jun 02 21:43:52.398738 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.398717 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zm72h"] Jun 02 21:43:52.563560 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.563493 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c748139-cc17-4c92-8a7c-cabf4d858974-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zm72h\" (UID: \"4c748139-cc17-4c92-8a7c-cabf4d858974\") " pod="openshift-insights/insights-runtime-extractor-zm72h" Jun 02 21:43:52.563560 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.563527 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4c748139-cc17-4c92-8a7c-cabf4d858974-crio-socket\") pod \"insights-runtime-extractor-zm72h\" (UID: \"4c748139-cc17-4c92-8a7c-cabf4d858974\") " pod="openshift-insights/insights-runtime-extractor-zm72h" Jun 02 21:43:52.563560 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.563559 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c748139-cc17-4c92-8a7c-cabf4d858974-data-volume\") pod \"insights-runtime-extractor-zm72h\" (UID: \"4c748139-cc17-4c92-8a7c-cabf4d858974\") " pod="openshift-insights/insights-runtime-extractor-zm72h" Jun 02 21:43:52.563773 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.563575 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c748139-cc17-4c92-8a7c-cabf4d858974-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zm72h\" (UID: \"4c748139-cc17-4c92-8a7c-cabf4d858974\") " pod="openshift-insights/insights-runtime-extractor-zm72h" Jun 02 21:43:52.563773 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.563620 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7dpl\" (UniqueName: \"kubernetes.io/projected/4c748139-cc17-4c92-8a7c-cabf4d858974-kube-api-access-n7dpl\") pod \"insights-runtime-extractor-zm72h\" (UID: \"4c748139-cc17-4c92-8a7c-cabf4d858974\") " pod="openshift-insights/insights-runtime-extractor-zm72h" Jun 02 21:43:52.664299 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.664267 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c748139-cc17-4c92-8a7c-cabf4d858974-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zm72h\" (UID: \"4c748139-cc17-4c92-8a7c-cabf4d858974\") " pod="openshift-insights/insights-runtime-extractor-zm72h" Jun 02 21:43:52.664429 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.664315 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4c748139-cc17-4c92-8a7c-cabf4d858974-crio-socket\") pod \"insights-runtime-extractor-zm72h\" (UID: \"4c748139-cc17-4c92-8a7c-cabf4d858974\") " pod="openshift-insights/insights-runtime-extractor-zm72h" Jun 02 21:43:52.664429 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.664347 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c748139-cc17-4c92-8a7c-cabf4d858974-data-volume\") pod \"insights-runtime-extractor-zm72h\" (UID: \"4c748139-cc17-4c92-8a7c-cabf4d858974\") " pod="openshift-insights/insights-runtime-extractor-zm72h" Jun 02 21:43:52.664429 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.664364 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c748139-cc17-4c92-8a7c-cabf4d858974-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zm72h\" (UID: \"4c748139-cc17-4c92-8a7c-cabf4d858974\") " pod="openshift-insights/insights-runtime-extractor-zm72h" Jun 02 21:43:52.664429 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.664379 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n7dpl\" (UniqueName: \"kubernetes.io/projected/4c748139-cc17-4c92-8a7c-cabf4d858974-kube-api-access-n7dpl\") pod \"insights-runtime-extractor-zm72h\" (UID: \"4c748139-cc17-4c92-8a7c-cabf4d858974\") " pod="openshift-insights/insights-runtime-extractor-zm72h" Jun 02 21:43:52.664555 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.664483 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4c748139-cc17-4c92-8a7c-cabf4d858974-crio-socket\") pod \"insights-runtime-extractor-zm72h\" (UID: \"4c748139-cc17-4c92-8a7c-cabf4d858974\") " pod="openshift-insights/insights-runtime-extractor-zm72h" Jun 02 21:43:52.664740 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.664720 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c748139-cc17-4c92-8a7c-cabf4d858974-data-volume\") pod \"insights-runtime-extractor-zm72h\" (UID: \"4c748139-cc17-4c92-8a7c-cabf4d858974\") " pod="openshift-insights/insights-runtime-extractor-zm72h" Jun 02 21:43:52.664911 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.664894 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c748139-cc17-4c92-8a7c-cabf4d858974-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zm72h\" (UID: \"4c748139-cc17-4c92-8a7c-cabf4d858974\") " pod="openshift-insights/insights-runtime-extractor-zm72h" Jun 02 21:43:52.666596 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.666570 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c748139-cc17-4c92-8a7c-cabf4d858974-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zm72h\" (UID: \"4c748139-cc17-4c92-8a7c-cabf4d858974\") " pod="openshift-insights/insights-runtime-extractor-zm72h" Jun 02 21:43:52.672444 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.672423 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7dpl\" (UniqueName: \"kubernetes.io/projected/4c748139-cc17-4c92-8a7c-cabf4d858974-kube-api-access-n7dpl\") pod \"insights-runtime-extractor-zm72h\" (UID: \"4c748139-cc17-4c92-8a7c-cabf4d858974\") " pod="openshift-insights/insights-runtime-extractor-zm72h" Jun 02 21:43:52.695697 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.695676 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zm72h" Jun 02 21:43:52.758539 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.758506 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-wtqfm"] Jun 02 21:43:52.760875 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.760852 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-wtqfm" Jun 02 21:43:52.762981 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.762957 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jun 02 21:43:52.762981 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.762975 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-xl7sv\"" Jun 02 21:43:52.774179 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.774135 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-wtqfm"] Jun 02 21:43:52.811720 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.811698 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zm72h"] Jun 02 21:43:52.865036 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.865016 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/726f75d6-7b96-42f7-87c1-8d8807f3680a-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-wtqfm\" (UID: \"726f75d6-7b96-42f7-87c1-8d8807f3680a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-wtqfm" Jun 02 21:43:52.966402 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:52.966369 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/726f75d6-7b96-42f7-87c1-8d8807f3680a-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-wtqfm\" (UID: \"726f75d6-7b96-42f7-87c1-8d8807f3680a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-wtqfm" Jun 02 21:43:52.966511 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:52.966480 2574 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jun 02 21:43:52.966571 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:43:52.966551 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/726f75d6-7b96-42f7-87c1-8d8807f3680a-tls-certificates podName:726f75d6-7b96-42f7-87c1-8d8807f3680a nodeName:}" failed. No retries permitted until 2026-06-02 21:43:53.466516714 +0000 UTC m=+118.171526638 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/726f75d6-7b96-42f7-87c1-8d8807f3680a-tls-certificates") pod "prometheus-operator-admission-webhook-59b7f8756b-wtqfm" (UID: "726f75d6-7b96-42f7-87c1-8d8807f3680a") : secret "prometheus-operator-admission-webhook-tls" not found Jun 02 21:43:53.257081 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:53.257054 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zm72h" event={"ID":"4c748139-cc17-4c92-8a7c-cabf4d858974","Type":"ContainerStarted","Data":"f0254b178aa8483e5be6137c0a437350ec0736a0029c446414db8d5cb87b73c4"} Jun 02 21:43:53.257210 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:53.257086 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zm72h" event={"ID":"4c748139-cc17-4c92-8a7c-cabf4d858974","Type":"ContainerStarted","Data":"03c18cfd6d2243f4d14d6914e76871ddaf3d298f52ae0aa469ee642d47506003"} Jun 02 21:43:53.471067 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:53.471044 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/726f75d6-7b96-42f7-87c1-8d8807f3680a-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-wtqfm\" (UID: \"726f75d6-7b96-42f7-87c1-8d8807f3680a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-wtqfm" Jun 02 21:43:53.473074 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:53.473056 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/726f75d6-7b96-42f7-87c1-8d8807f3680a-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-wtqfm\" (UID: \"726f75d6-7b96-42f7-87c1-8d8807f3680a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-wtqfm" Jun 02 21:43:53.671053 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:53.671025 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-wtqfm" Jun 02 21:43:53.788472 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:53.788278 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-wtqfm"] Jun 02 21:43:53.790634 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:43:53.790609 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod726f75d6_7b96_42f7_87c1_8d8807f3680a.slice/crio-87f035bf9fc2fd5c2898810bbb735b6f4912f5049da0a845f6ed910566baacfe WatchSource:0}: Error finding container 87f035bf9fc2fd5c2898810bbb735b6f4912f5049da0a845f6ed910566baacfe: Status 404 returned error can't find the container with id 87f035bf9fc2fd5c2898810bbb735b6f4912f5049da0a845f6ed910566baacfe Jun 02 21:43:54.260352 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:54.260320 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-wtqfm" event={"ID":"726f75d6-7b96-42f7-87c1-8d8807f3680a","Type":"ContainerStarted","Data":"87f035bf9fc2fd5c2898810bbb735b6f4912f5049da0a845f6ed910566baacfe"} Jun 02 21:43:54.261951 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:54.261920 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zm72h" event={"ID":"4c748139-cc17-4c92-8a7c-cabf4d858974","Type":"ContainerStarted","Data":"57f09796863f2396434c2a0e583ab0cd028d20089b605b067f77fe067b8fac37"} Jun 02 21:43:55.571018 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:55.570915 2574 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" podUID="d128b979-2533-4ca6-b009-111b33e8188f" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 02 21:43:56.271648 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:56.271618 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-wtqfm" event={"ID":"726f75d6-7b96-42f7-87c1-8d8807f3680a","Type":"ContainerStarted","Data":"c8ebd7af485f6469916582b95584d0cc9e636d48a7fd835d83f7ec21ab1c00a1"} Jun 02 21:43:56.271843 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:56.271768 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-wtqfm" Jun 02 21:43:56.273533 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:56.273494 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zm72h" event={"ID":"4c748139-cc17-4c92-8a7c-cabf4d858974","Type":"ContainerStarted","Data":"475c7a85df22a76df480a8e8ddbdcadb338bdd2ed78ca44ea44e81ad415dd7cc"} Jun 02 21:43:56.277253 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:56.277230 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-wtqfm" Jun 02 21:43:56.287969 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:56.287933 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-wtqfm" podStartSLOduration=2.878312382 podStartE2EDuration="4.287920923s" podCreationTimestamp="2026-06-02 21:43:52 +0000 UTC" firstStartedPulling="2026-06-02 21:43:53.792444406 +0000 UTC m=+118.497454334" lastFinishedPulling="2026-06-02 21:43:55.202052947 +0000 UTC m=+119.907062875" observedRunningTime="2026-06-02 21:43:56.287391001 +0000 UTC m=+120.992400948" watchObservedRunningTime="2026-06-02 21:43:56.287920923 +0000 UTC m=+120.992930870" Jun 02 21:43:56.303598 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:43:56.303554 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zm72h" podStartSLOduration=1.9740317969999999 podStartE2EDuration="4.303543321s" podCreationTimestamp="2026-06-02 21:43:52 +0000 UTC" firstStartedPulling="2026-06-02 21:43:52.871715919 +0000 UTC m=+117.576725852" lastFinishedPulling="2026-06-02 21:43:55.201227436 +0000 UTC m=+119.906237376" observedRunningTime="2026-06-02 21:43:56.302991394 +0000 UTC m=+121.008001340" watchObservedRunningTime="2026-06-02 21:43:56.303543321 +0000 UTC m=+121.008553267" Jun 02 21:44:01.198610 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.198576 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-gp4cq"] Jun 02 21:44:01.202803 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.202788 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-gp4cq" Jun 02 21:44:01.204931 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.204896 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 02 21:44:01.205057 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.204974 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-dm9kd\"" Jun 02 21:44:01.205118 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.205095 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 02 21:44:01.205663 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.205646 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 02 21:44:01.205756 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.205666 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 02 21:44:01.205756 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.205677 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 02 21:44:01.214371 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.214353 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-gp4cq"] Jun 02 21:44:01.221899 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.221880 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-t8zxt"] Jun 02 21:44:01.224809 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.224795 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.227061 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.227042 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-z7p22\"" Jun 02 21:44:01.227061 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.227054 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 02 21:44:01.227320 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.227298 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 02 21:44:01.227377 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.227361 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 02 21:44:01.324907 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.324886 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-metrics-client-ca\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.325037 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.324916 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-sys\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.325037 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.324934 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/aebf7342-c9a8-4df6-91d9-b857c62c3a6c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-gp4cq\" (UID: \"aebf7342-c9a8-4df6-91d9-b857c62c3a6c\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-gp4cq" Jun 02 21:44:01.325037 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.324949 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-root\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.325037 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.324966 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aebf7342-c9a8-4df6-91d9-b857c62c3a6c-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-gp4cq\" (UID: \"aebf7342-c9a8-4df6-91d9-b857c62c3a6c\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-gp4cq" Jun 02 21:44:01.325037 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.325000 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-node-exporter-tls\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.325245 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.325052 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-node-exporter-accelerators-collector-config\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.325245 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.325100 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-node-exporter-textfile\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.325245 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.325176 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-node-exporter-wtmp\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.325245 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.325208 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx49w\" (UniqueName: \"kubernetes.io/projected/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-kube-api-access-mx49w\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.325450 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.325275 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.325450 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.325338 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aebf7342-c9a8-4df6-91d9-b857c62c3a6c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-gp4cq\" (UID: \"aebf7342-c9a8-4df6-91d9-b857c62c3a6c\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-gp4cq" Jun 02 21:44:01.325450 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.325382 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx9qf\" (UniqueName: \"kubernetes.io/projected/aebf7342-c9a8-4df6-91d9-b857c62c3a6c-kube-api-access-rx9qf\") pod \"openshift-state-metrics-85658796db-gp4cq\" (UID: \"aebf7342-c9a8-4df6-91d9-b857c62c3a6c\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-gp4cq" Jun 02 21:44:01.426319 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.426292 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-metrics-client-ca\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.426413 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.426331 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-sys\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.426413 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.426360 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/aebf7342-c9a8-4df6-91d9-b857c62c3a6c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-gp4cq\" (UID: \"aebf7342-c9a8-4df6-91d9-b857c62c3a6c\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-gp4cq" Jun 02 21:44:01.426413 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.426383 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-root\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.426413 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.426406 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aebf7342-c9a8-4df6-91d9-b857c62c3a6c-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-gp4cq\" (UID: \"aebf7342-c9a8-4df6-91d9-b857c62c3a6c\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-gp4cq" Jun 02 21:44:01.426616 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.426433 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-node-exporter-tls\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.426616 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.426459 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-node-exporter-accelerators-collector-config\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.426616 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.426477 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-root\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.426616 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.426491 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-node-exporter-textfile\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.426616 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.426492 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-sys\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.426616 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:44:01.426553 2574 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 02 21:44:01.426616 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.426558 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-node-exporter-wtmp\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.426616 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.426590 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mx49w\" (UniqueName: \"kubernetes.io/projected/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-kube-api-access-mx49w\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.426616 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:44:01.426609 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-node-exporter-tls podName:4d4a7ce0-1570-4a00-a1c7-00e60d10a6be nodeName:}" failed. No retries permitted until 2026-06-02 21:44:01.92658848 +0000 UTC m=+126.631598421 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-node-exporter-tls") pod "node-exporter-t8zxt" (UID: "4d4a7ce0-1570-4a00-a1c7-00e60d10a6be") : secret "node-exporter-tls" not found Jun 02 21:44:01.426616 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:44:01.426615 2574 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jun 02 21:44:01.427133 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.426653 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.427133 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:44:01.426679 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aebf7342-c9a8-4df6-91d9-b857c62c3a6c-openshift-state-metrics-tls podName:aebf7342-c9a8-4df6-91d9-b857c62c3a6c nodeName:}" failed. No retries permitted until 2026-06-02 21:44:01.926662057 +0000 UTC m=+126.631671989 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/aebf7342-c9a8-4df6-91d9-b857c62c3a6c-openshift-state-metrics-tls") pod "openshift-state-metrics-85658796db-gp4cq" (UID: "aebf7342-c9a8-4df6-91d9-b857c62c3a6c") : secret "openshift-state-metrics-tls" not found Jun 02 21:44:01.427133 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.426721 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aebf7342-c9a8-4df6-91d9-b857c62c3a6c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-gp4cq\" (UID: \"aebf7342-c9a8-4df6-91d9-b857c62c3a6c\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-gp4cq" Jun 02 21:44:01.427133 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.426769 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rx9qf\" (UniqueName: \"kubernetes.io/projected/aebf7342-c9a8-4df6-91d9-b857c62c3a6c-kube-api-access-rx9qf\") pod \"openshift-state-metrics-85658796db-gp4cq\" (UID: \"aebf7342-c9a8-4df6-91d9-b857c62c3a6c\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-gp4cq" Jun 02 21:44:01.427133 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.426825 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-node-exporter-textfile\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.427133 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.426888 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-metrics-client-ca\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.427133 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.426924 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-node-exporter-wtmp\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.427133 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.427077 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-node-exporter-accelerators-collector-config\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.427133 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.427125 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aebf7342-c9a8-4df6-91d9-b857c62c3a6c-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-gp4cq\" (UID: \"aebf7342-c9a8-4df6-91d9-b857c62c3a6c\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-gp4cq" Jun 02 21:44:01.428894 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.428875 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.429009 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.428990 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aebf7342-c9a8-4df6-91d9-b857c62c3a6c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-gp4cq\" (UID: \"aebf7342-c9a8-4df6-91d9-b857c62c3a6c\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-gp4cq" Jun 02 21:44:01.435930 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.435910 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx49w\" (UniqueName: \"kubernetes.io/projected/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-kube-api-access-mx49w\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.436158 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.436140 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx9qf\" (UniqueName: \"kubernetes.io/projected/aebf7342-c9a8-4df6-91d9-b857c62c3a6c-kube-api-access-rx9qf\") pod \"openshift-state-metrics-85658796db-gp4cq\" (UID: \"aebf7342-c9a8-4df6-91d9-b857c62c3a6c\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-gp4cq" Jun 02 21:44:01.931244 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.931210 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/aebf7342-c9a8-4df6-91d9-b857c62c3a6c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-gp4cq\" (UID: \"aebf7342-c9a8-4df6-91d9-b857c62c3a6c\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-gp4cq" Jun 02 21:44:01.931244 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.931252 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-node-exporter-tls\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.933520 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.933492 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4d4a7ce0-1570-4a00-a1c7-00e60d10a6be-node-exporter-tls\") pod \"node-exporter-t8zxt\" (UID: \"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be\") " pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:01.933656 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:01.933591 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/aebf7342-c9a8-4df6-91d9-b857c62c3a6c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-gp4cq\" (UID: \"aebf7342-c9a8-4df6-91d9-b857c62c3a6c\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-gp4cq" Jun 02 21:44:02.111238 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.111198 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-gp4cq" Jun 02 21:44:02.133007 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.132977 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t8zxt" Jun 02 21:44:02.140763 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:44:02.140724 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d4a7ce0_1570_4a00_a1c7_00e60d10a6be.slice/crio-8b661f2cb4165d6c2647ed6147fe829983081e12d02aa906ee1eac4235fd4eb2 WatchSource:0}: Error finding container 8b661f2cb4165d6c2647ed6147fe829983081e12d02aa906ee1eac4235fd4eb2: Status 404 returned error can't find the container with id 8b661f2cb4165d6c2647ed6147fe829983081e12d02aa906ee1eac4235fd4eb2 Jun 02 21:44:02.235383 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.235353 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-gp4cq"] Jun 02 21:44:02.238835 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:44:02.238795 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaebf7342_c9a8_4df6_91d9_b857c62c3a6c.slice/crio-db10da1c99bf1ca4a2319c31a6d09943c18377b09538f6dba7d58fc0dc23960c WatchSource:0}: Error finding container db10da1c99bf1ca4a2319c31a6d09943c18377b09538f6dba7d58fc0dc23960c: Status 404 returned error can't find the container with id db10da1c99bf1ca4a2319c31a6d09943c18377b09538f6dba7d58fc0dc23960c Jun 02 21:44:02.270820 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.270793 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 21:44:02.275154 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.275136 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.277101 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.277081 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 02 21:44:02.277198 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.277160 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 02 21:44:02.277198 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.277186 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qhwbw\"" Jun 02 21:44:02.277338 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.277322 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 02 21:44:02.277424 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.277404 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 02 21:44:02.277543 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.277475 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 02 21:44:02.277683 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.277622 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 02 21:44:02.277780 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.277704 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 02 21:44:02.277843 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.277828 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 02 21:44:02.277928 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.277831 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 02 21:44:02.284428 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.284405 2574 patch_prober.go:28] interesting pod/image-registry-559f464548-hrcsf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 02 21:44:02.284507 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.284457 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-559f464548-hrcsf" podUID="ae671a73-b0b0-4d22-979e-5aa8b509daaf" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 21:44:02.290861 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.290834 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 21:44:02.292506 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.292477 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t8zxt" event={"ID":"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be","Type":"ContainerStarted","Data":"8b661f2cb4165d6c2647ed6147fe829983081e12d02aa906ee1eac4235fd4eb2"} Jun 02 21:44:02.294510 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.294483 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-gp4cq" event={"ID":"aebf7342-c9a8-4df6-91d9-b857c62c3a6c","Type":"ContainerStarted","Data":"db10da1c99bf1ca4a2319c31a6d09943c18377b09538f6dba7d58fc0dc23960c"} Jun 02 21:44:02.436465 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.436423 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a06ca92b-2521-4f30-aa03-9658da4c8ed2-config-out\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.436604 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.436468 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.436604 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.436538 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-web-config\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.436604 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.436586 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssm6d\" (UniqueName: \"kubernetes.io/projected/a06ca92b-2521-4f30-aa03-9658da4c8ed2-kube-api-access-ssm6d\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.436753 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.436660 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.436753 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.436704 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a06ca92b-2521-4f30-aa03-9658da4c8ed2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.436853 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.436765 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a06ca92b-2521-4f30-aa03-9658da4c8ed2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.436853 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.436785 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.436853 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.436804 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a06ca92b-2521-4f30-aa03-9658da4c8ed2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.436853 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.436837 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-config-volume\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.437019 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.436861 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a06ca92b-2521-4f30-aa03-9658da4c8ed2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.437019 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.436937 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.437019 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.436969 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.537965 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.537877 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.537965 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.537921 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a06ca92b-2521-4f30-aa03-9658da4c8ed2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.538219 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.538197 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a06ca92b-2521-4f30-aa03-9658da4c8ed2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.538299 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.538235 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.538299 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.538266 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a06ca92b-2521-4f30-aa03-9658da4c8ed2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.538411 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.538318 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-config-volume\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.538411 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.538347 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a06ca92b-2521-4f30-aa03-9658da4c8ed2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.538411 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.538394 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a06ca92b-2521-4f30-aa03-9658da4c8ed2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.538411 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.538405 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.538598 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.538431 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.538598 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.538548 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a06ca92b-2521-4f30-aa03-9658da4c8ed2-config-out\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.538598 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.538578 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.538774 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.538608 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-web-config\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.538774 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.538634 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ssm6d\" (UniqueName: \"kubernetes.io/projected/a06ca92b-2521-4f30-aa03-9658da4c8ed2-kube-api-access-ssm6d\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.539168 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.539020 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a06ca92b-2521-4f30-aa03-9658da4c8ed2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.539168 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:44:02.539127 2574 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jun 02 21:44:02.539352 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:44:02.539182 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-main-tls podName:a06ca92b-2521-4f30-aa03-9658da4c8ed2 nodeName:}" failed. No retries permitted until 2026-06-02 21:44:03.039163307 +0000 UTC m=+127.744173236 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "a06ca92b-2521-4f30-aa03-9658da4c8ed2") : secret "alertmanager-main-tls" not found Jun 02 21:44:02.539352 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.539318 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a06ca92b-2521-4f30-aa03-9658da4c8ed2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.541457 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.541430 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-config-volume\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.542421 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.542398 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.542774 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.542718 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a06ca92b-2521-4f30-aa03-9658da4c8ed2-config-out\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.542881 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.542826 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.543492 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.543128 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a06ca92b-2521-4f30-aa03-9658da4c8ed2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.543492 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.543463 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.543663 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.543566 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-web-config\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.543713 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.543676 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:02.548275 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:02.548251 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssm6d\" (UniqueName: \"kubernetes.io/projected/a06ca92b-2521-4f30-aa03-9658da4c8ed2-kube-api-access-ssm6d\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:03.042652 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.042611 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:03.044844 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.044815 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:03.185419 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.185344 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:44:03.273501 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.273469 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-9dbbc88c8-48p6r"] Jun 02 21:44:03.278257 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.278234 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.280296 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.280220 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 02 21:44:03.281106 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.280525 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-2j6gd\"" Jun 02 21:44:03.281106 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.280739 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 02 21:44:03.281106 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.280948 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 02 21:44:03.282036 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.281798 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 02 21:44:03.282036 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.281897 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-9um3u7ab4ormc\"" Jun 02 21:44:03.282036 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.281975 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 02 21:44:03.287048 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.287005 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-9dbbc88c8-48p6r"] Jun 02 21:44:03.302175 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.302133 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-gp4cq" event={"ID":"aebf7342-c9a8-4df6-91d9-b857c62c3a6c","Type":"ContainerStarted","Data":"d6639ab575b660f0c48cfc23391092326bd9aada9c73105f3c7159739c5123ff"} Jun 02 21:44:03.302305 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.302178 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-gp4cq" event={"ID":"aebf7342-c9a8-4df6-91d9-b857c62c3a6c","Type":"ContainerStarted","Data":"b6d81b5c984b3ec87ab257fa39a656065d552e1ec8707f03c1079595514277dd"} Jun 02 21:44:03.303861 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.303834 2574 generic.go:358] "Generic (PLEG): container finished" podID="4d4a7ce0-1570-4a00-a1c7-00e60d10a6be" containerID="ed8a2429696d1f91be0f77f99c631feaef0e5c3e5f8846a3c3bac17003bd2a48" exitCode=0 Jun 02 21:44:03.303960 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.303881 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t8zxt" event={"ID":"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be","Type":"ContainerDied","Data":"ed8a2429696d1f91be0f77f99c631feaef0e5c3e5f8846a3c3bac17003bd2a48"} Jun 02 21:44:03.327396 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.327375 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 21:44:03.344426 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.344399 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b182c19b-79d8-47d5-9e52-d87787126160-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-9dbbc88c8-48p6r\" (UID: \"b182c19b-79d8-47d5-9e52-d87787126160\") " pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.344562 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.344447 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b182c19b-79d8-47d5-9e52-d87787126160-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-9dbbc88c8-48p6r\" (UID: \"b182c19b-79d8-47d5-9e52-d87787126160\") " pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.344705 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.344684 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b182c19b-79d8-47d5-9e52-d87787126160-secret-thanos-querier-tls\") pod \"thanos-querier-9dbbc88c8-48p6r\" (UID: \"b182c19b-79d8-47d5-9e52-d87787126160\") " pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.344774 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.344749 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b182c19b-79d8-47d5-9e52-d87787126160-metrics-client-ca\") pod \"thanos-querier-9dbbc88c8-48p6r\" (UID: \"b182c19b-79d8-47d5-9e52-d87787126160\") " pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.344862 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.344834 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b182c19b-79d8-47d5-9e52-d87787126160-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-9dbbc88c8-48p6r\" (UID: \"b182c19b-79d8-47d5-9e52-d87787126160\") " pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.344917 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.344891 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b182c19b-79d8-47d5-9e52-d87787126160-secret-grpc-tls\") pod \"thanos-querier-9dbbc88c8-48p6r\" (UID: \"b182c19b-79d8-47d5-9e52-d87787126160\") " pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.344960 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.344926 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b182c19b-79d8-47d5-9e52-d87787126160-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-9dbbc88c8-48p6r\" (UID: \"b182c19b-79d8-47d5-9e52-d87787126160\") " pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.344960 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.344954 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxzmx\" (UniqueName: \"kubernetes.io/projected/b182c19b-79d8-47d5-9e52-d87787126160-kube-api-access-xxzmx\") pod \"thanos-querier-9dbbc88c8-48p6r\" (UID: \"b182c19b-79d8-47d5-9e52-d87787126160\") " pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.445908 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.445838 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b182c19b-79d8-47d5-9e52-d87787126160-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-9dbbc88c8-48p6r\" (UID: \"b182c19b-79d8-47d5-9e52-d87787126160\") " pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.445908 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.445890 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b182c19b-79d8-47d5-9e52-d87787126160-secret-grpc-tls\") pod \"thanos-querier-9dbbc88c8-48p6r\" (UID: \"b182c19b-79d8-47d5-9e52-d87787126160\") " pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.446072 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.445926 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b182c19b-79d8-47d5-9e52-d87787126160-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-9dbbc88c8-48p6r\" (UID: \"b182c19b-79d8-47d5-9e52-d87787126160\") " pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.446072 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.445954 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xxzmx\" (UniqueName: \"kubernetes.io/projected/b182c19b-79d8-47d5-9e52-d87787126160-kube-api-access-xxzmx\") pod \"thanos-querier-9dbbc88c8-48p6r\" (UID: \"b182c19b-79d8-47d5-9e52-d87787126160\") " pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.446072 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.446028 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b182c19b-79d8-47d5-9e52-d87787126160-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-9dbbc88c8-48p6r\" (UID: \"b182c19b-79d8-47d5-9e52-d87787126160\") " pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.446072 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.446063 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b182c19b-79d8-47d5-9e52-d87787126160-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-9dbbc88c8-48p6r\" (UID: \"b182c19b-79d8-47d5-9e52-d87787126160\") " pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.446227 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.446106 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b182c19b-79d8-47d5-9e52-d87787126160-secret-thanos-querier-tls\") pod \"thanos-querier-9dbbc88c8-48p6r\" (UID: \"b182c19b-79d8-47d5-9e52-d87787126160\") " pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.446227 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.446136 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b182c19b-79d8-47d5-9e52-d87787126160-metrics-client-ca\") pod \"thanos-querier-9dbbc88c8-48p6r\" (UID: \"b182c19b-79d8-47d5-9e52-d87787126160\") " pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.446878 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.446852 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b182c19b-79d8-47d5-9e52-d87787126160-metrics-client-ca\") pod \"thanos-querier-9dbbc88c8-48p6r\" (UID: \"b182c19b-79d8-47d5-9e52-d87787126160\") " pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.448928 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.448903 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b182c19b-79d8-47d5-9e52-d87787126160-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-9dbbc88c8-48p6r\" (UID: \"b182c19b-79d8-47d5-9e52-d87787126160\") " pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.449010 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.448924 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b182c19b-79d8-47d5-9e52-d87787126160-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-9dbbc88c8-48p6r\" (UID: \"b182c19b-79d8-47d5-9e52-d87787126160\") " pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.449157 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.449118 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b182c19b-79d8-47d5-9e52-d87787126160-secret-thanos-querier-tls\") pod \"thanos-querier-9dbbc88c8-48p6r\" (UID: \"b182c19b-79d8-47d5-9e52-d87787126160\") " pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.449611 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.449572 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b182c19b-79d8-47d5-9e52-d87787126160-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-9dbbc88c8-48p6r\" (UID: \"b182c19b-79d8-47d5-9e52-d87787126160\") " pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.449707 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.449689 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b182c19b-79d8-47d5-9e52-d87787126160-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-9dbbc88c8-48p6r\" (UID: \"b182c19b-79d8-47d5-9e52-d87787126160\") " pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.449788 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.449774 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b182c19b-79d8-47d5-9e52-d87787126160-secret-grpc-tls\") pod \"thanos-querier-9dbbc88c8-48p6r\" (UID: \"b182c19b-79d8-47d5-9e52-d87787126160\") " pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.452913 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.452894 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxzmx\" (UniqueName: \"kubernetes.io/projected/b182c19b-79d8-47d5-9e52-d87787126160-kube-api-access-xxzmx\") pod \"thanos-querier-9dbbc88c8-48p6r\" (UID: \"b182c19b-79d8-47d5-9e52-d87787126160\") " pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.537687 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:44:03.537659 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda06ca92b_2521_4f30_aa03_9658da4c8ed2.slice/crio-e8cfb15fc7d4e71bfc52a122588a8459b70be8f52efabc2731b3fc70e348ac60 WatchSource:0}: Error finding container e8cfb15fc7d4e71bfc52a122588a8459b70be8f52efabc2731b3fc70e348ac60: Status 404 returned error can't find the container with id e8cfb15fc7d4e71bfc52a122588a8459b70be8f52efabc2731b3fc70e348ac60 Jun 02 21:44:03.594673 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.594315 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:03.722978 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:03.722908 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-9dbbc88c8-48p6r"] Jun 02 21:44:03.725803 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:44:03.725778 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb182c19b_79d8_47d5_9e52_d87787126160.slice/crio-488c42946ba31f6ca804e2b61cabb43d39d511431cf0eeec530412c285a14249 WatchSource:0}: Error finding container 488c42946ba31f6ca804e2b61cabb43d39d511431cf0eeec530412c285a14249: Status 404 returned error can't find the container with id 488c42946ba31f6ca804e2b61cabb43d39d511431cf0eeec530412c285a14249 Jun 02 21:44:04.309469 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:04.309419 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t8zxt" event={"ID":"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be","Type":"ContainerStarted","Data":"8e8d6ca9a724ee8f083a39bb49f2010e13319c7265b66d1b1ba5b069636f7879"} Jun 02 21:44:04.309469 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:04.309462 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t8zxt" event={"ID":"4d4a7ce0-1570-4a00-a1c7-00e60d10a6be","Type":"ContainerStarted","Data":"b3d46340afe269cc459cd79468f89635ae7abef24dba5ae73ebf057939303efa"} Jun 02 21:44:04.311214 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:04.311182 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" event={"ID":"b182c19b-79d8-47d5-9e52-d87787126160","Type":"ContainerStarted","Data":"488c42946ba31f6ca804e2b61cabb43d39d511431cf0eeec530412c285a14249"} Jun 02 21:44:04.313634 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:04.313606 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-gp4cq" event={"ID":"aebf7342-c9a8-4df6-91d9-b857c62c3a6c","Type":"ContainerStarted","Data":"3796c65a5599a6845dba62fbe4cfaaac5bc300140a5b52bfab973ce87895093b"} Jun 02 21:44:04.314902 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:04.314877 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a06ca92b-2521-4f30-aa03-9658da4c8ed2","Type":"ContainerStarted","Data":"e8cfb15fc7d4e71bfc52a122588a8459b70be8f52efabc2731b3fc70e348ac60"} Jun 02 21:44:04.330892 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:04.330843 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-t8zxt" podStartSLOduration=2.546793965 podStartE2EDuration="3.330814453s" podCreationTimestamp="2026-06-02 21:44:01 +0000 UTC" firstStartedPulling="2026-06-02 21:44:02.142798669 +0000 UTC m=+126.847808631" lastFinishedPulling="2026-06-02 21:44:02.926819188 +0000 UTC m=+127.631829119" observedRunningTime="2026-06-02 21:44:04.327629702 +0000 UTC m=+129.032639673" watchObservedRunningTime="2026-06-02 21:44:04.330814453 +0000 UTC m=+129.035824406" Jun 02 21:44:04.350960 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:04.350909 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-85658796db-gp4cq" podStartSLOduration=2.124201315 podStartE2EDuration="3.350893466s" podCreationTimestamp="2026-06-02 21:44:01 +0000 UTC" firstStartedPulling="2026-06-02 21:44:02.35614435 +0000 UTC m=+127.061154285" lastFinishedPulling="2026-06-02 21:44:03.5828365 +0000 UTC m=+128.287846436" observedRunningTime="2026-06-02 21:44:04.349649579 +0000 UTC m=+129.054659527" watchObservedRunningTime="2026-06-02 21:44:04.350893466 +0000 UTC m=+129.055903412" Jun 02 21:44:05.319574 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.319537 2574 generic.go:358] "Generic (PLEG): container finished" podID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerID="bc8a715968dbc76c8ae6d4b1fa6f9a5ebaa24f0488d7a34a920fd03dc1f1f84e" exitCode=0 Jun 02 21:44:05.319962 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.319629 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a06ca92b-2521-4f30-aa03-9658da4c8ed2","Type":"ContainerDied","Data":"bc8a715968dbc76c8ae6d4b1fa6f9a5ebaa24f0488d7a34a920fd03dc1f1f84e"} Jun 02 21:44:05.571786 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.571715 2574 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" podUID="d128b979-2533-4ca6-b009-111b33e8188f" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 02 21:44:05.664214 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.664195 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs\") pod \"network-metrics-daemon-qw5fs\" (UID: \"03b1bb3a-4c92-4555-b5d4-3aaac39381f5\") " pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:44:05.666240 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.666221 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03b1bb3a-4c92-4555-b5d4-3aaac39381f5-metrics-certs\") pod \"network-metrics-daemon-qw5fs\" (UID: \"03b1bb3a-4c92-4555-b5d4-3aaac39381f5\") " pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:44:05.701128 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.701107 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6558c5ccc4-6hjtv"] Jun 02 21:44:05.705400 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.705384 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:05.707586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.707567 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-p5fhz\"" Jun 02 21:44:05.707675 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.707660 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 02 21:44:05.707752 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.707739 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-8npqnpd01k9h8\"" Jun 02 21:44:05.707832 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.707815 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 02 21:44:05.708256 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.708235 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 02 21:44:05.708256 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.708255 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 02 21:44:05.714220 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.714201 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6558c5ccc4-6hjtv"] Jun 02 21:44:05.765253 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.765201 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c5cfc52a-6600-48d0-800a-a79dd856bc27-audit-log\") pod \"metrics-server-6558c5ccc4-6hjtv\" (UID: \"c5cfc52a-6600-48d0-800a-a79dd856bc27\") " pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:05.765465 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.765236 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c5cfc52a-6600-48d0-800a-a79dd856bc27-metrics-server-audit-profiles\") pod \"metrics-server-6558c5ccc4-6hjtv\" (UID: \"c5cfc52a-6600-48d0-800a-a79dd856bc27\") " pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:05.765465 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.765415 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvzb2\" (UniqueName: \"kubernetes.io/projected/c5cfc52a-6600-48d0-800a-a79dd856bc27-kube-api-access-gvzb2\") pod \"metrics-server-6558c5ccc4-6hjtv\" (UID: \"c5cfc52a-6600-48d0-800a-a79dd856bc27\") " pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:05.765465 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.765446 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c5cfc52a-6600-48d0-800a-a79dd856bc27-client-ca-bundle\") pod \"metrics-server-6558c5ccc4-6hjtv\" (UID: \"c5cfc52a-6600-48d0-800a-a79dd856bc27\") " pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:05.765679 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.765657 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c5cfc52a-6600-48d0-800a-a79dd856bc27-secret-metrics-server-tls\") pod \"metrics-server-6558c5ccc4-6hjtv\" (UID: \"c5cfc52a-6600-48d0-800a-a79dd856bc27\") " pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:05.765764 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.765743 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c5cfc52a-6600-48d0-800a-a79dd856bc27-secret-metrics-server-client-certs\") pod \"metrics-server-6558c5ccc4-6hjtv\" (UID: \"c5cfc52a-6600-48d0-800a-a79dd856bc27\") " pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:05.765821 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.765790 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5cfc52a-6600-48d0-800a-a79dd856bc27-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6558c5ccc4-6hjtv\" (UID: \"c5cfc52a-6600-48d0-800a-a79dd856bc27\") " pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:05.857459 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.857433 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-96ccb\"" Jun 02 21:44:05.866030 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.866002 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qw5fs" Jun 02 21:44:05.866277 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.866248 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c5cfc52a-6600-48d0-800a-a79dd856bc27-secret-metrics-server-tls\") pod \"metrics-server-6558c5ccc4-6hjtv\" (UID: \"c5cfc52a-6600-48d0-800a-a79dd856bc27\") " pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:05.866369 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.866325 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c5cfc52a-6600-48d0-800a-a79dd856bc27-secret-metrics-server-client-certs\") pod \"metrics-server-6558c5ccc4-6hjtv\" (UID: \"c5cfc52a-6600-48d0-800a-a79dd856bc27\") " pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:05.866369 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.866358 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5cfc52a-6600-48d0-800a-a79dd856bc27-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6558c5ccc4-6hjtv\" (UID: \"c5cfc52a-6600-48d0-800a-a79dd856bc27\") " pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:05.866470 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.866394 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c5cfc52a-6600-48d0-800a-a79dd856bc27-audit-log\") pod \"metrics-server-6558c5ccc4-6hjtv\" (UID: \"c5cfc52a-6600-48d0-800a-a79dd856bc27\") " pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:05.866470 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.866460 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c5cfc52a-6600-48d0-800a-a79dd856bc27-metrics-server-audit-profiles\") pod \"metrics-server-6558c5ccc4-6hjtv\" (UID: \"c5cfc52a-6600-48d0-800a-a79dd856bc27\") " pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:05.866571 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.866492 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gvzb2\" (UniqueName: \"kubernetes.io/projected/c5cfc52a-6600-48d0-800a-a79dd856bc27-kube-api-access-gvzb2\") pod \"metrics-server-6558c5ccc4-6hjtv\" (UID: \"c5cfc52a-6600-48d0-800a-a79dd856bc27\") " pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:05.866571 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.866532 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c5cfc52a-6600-48d0-800a-a79dd856bc27-client-ca-bundle\") pod \"metrics-server-6558c5ccc4-6hjtv\" (UID: \"c5cfc52a-6600-48d0-800a-a79dd856bc27\") " pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:05.866824 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.866800 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c5cfc52a-6600-48d0-800a-a79dd856bc27-audit-log\") pod \"metrics-server-6558c5ccc4-6hjtv\" (UID: \"c5cfc52a-6600-48d0-800a-a79dd856bc27\") " pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:05.867207 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.867182 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5cfc52a-6600-48d0-800a-a79dd856bc27-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6558c5ccc4-6hjtv\" (UID: \"c5cfc52a-6600-48d0-800a-a79dd856bc27\") " pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:05.867934 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.867913 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c5cfc52a-6600-48d0-800a-a79dd856bc27-metrics-server-audit-profiles\") pod \"metrics-server-6558c5ccc4-6hjtv\" (UID: \"c5cfc52a-6600-48d0-800a-a79dd856bc27\") " pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:05.869222 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.869198 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c5cfc52a-6600-48d0-800a-a79dd856bc27-client-ca-bundle\") pod \"metrics-server-6558c5ccc4-6hjtv\" (UID: \"c5cfc52a-6600-48d0-800a-a79dd856bc27\") " pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:05.869992 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.869952 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c5cfc52a-6600-48d0-800a-a79dd856bc27-secret-metrics-server-tls\") pod \"metrics-server-6558c5ccc4-6hjtv\" (UID: \"c5cfc52a-6600-48d0-800a-a79dd856bc27\") " pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:05.870571 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.870549 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c5cfc52a-6600-48d0-800a-a79dd856bc27-secret-metrics-server-client-certs\") pod \"metrics-server-6558c5ccc4-6hjtv\" (UID: \"c5cfc52a-6600-48d0-800a-a79dd856bc27\") " pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:05.874741 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:05.874720 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvzb2\" (UniqueName: \"kubernetes.io/projected/c5cfc52a-6600-48d0-800a-a79dd856bc27-kube-api-access-gvzb2\") pod \"metrics-server-6558c5ccc4-6hjtv\" (UID: \"c5cfc52a-6600-48d0-800a-a79dd856bc27\") " pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:06.015093 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:06.015064 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:06.032350 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:06.032324 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qw5fs"] Jun 02 21:44:06.037038 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:44:06.037014 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03b1bb3a_4c92_4555_b5d4_3aaac39381f5.slice/crio-522334cdef144f961fca3c6ae63a1ee1b9e9f23e42ac9fb66dbe0a57601e21a0 WatchSource:0}: Error finding container 522334cdef144f961fca3c6ae63a1ee1b9e9f23e42ac9fb66dbe0a57601e21a0: Status 404 returned error can't find the container with id 522334cdef144f961fca3c6ae63a1ee1b9e9f23e42ac9fb66dbe0a57601e21a0 Jun 02 21:44:06.204230 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:06.204188 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6558c5ccc4-6hjtv"] Jun 02 21:44:06.207654 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:44:06.207627 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5cfc52a_6600_48d0_800a_a79dd856bc27.slice/crio-b6f406e6ad25dc8815271a04d2d1732f1574b6572e96459f39d4d19cb0f482c3 WatchSource:0}: Error finding container b6f406e6ad25dc8815271a04d2d1732f1574b6572e96459f39d4d19cb0f482c3: Status 404 returned error can't find the container with id b6f406e6ad25dc8815271a04d2d1732f1574b6572e96459f39d4d19cb0f482c3 Jun 02 21:44:06.324145 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:06.324107 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" event={"ID":"c5cfc52a-6600-48d0-800a-a79dd856bc27","Type":"ContainerStarted","Data":"b6f406e6ad25dc8815271a04d2d1732f1574b6572e96459f39d4d19cb0f482c3"} Jun 02 21:44:06.326133 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:06.326110 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" event={"ID":"b182c19b-79d8-47d5-9e52-d87787126160","Type":"ContainerStarted","Data":"d29660cd903d9740a244c1061591bc251267ae7470ccd6ad2cfd0c110d0ee57a"} Jun 02 21:44:06.326246 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:06.326137 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" event={"ID":"b182c19b-79d8-47d5-9e52-d87787126160","Type":"ContainerStarted","Data":"780b52d542cc99652ba13ad238a8290c616519c3a04279ae9684bc0476f100af"} Jun 02 21:44:06.326246 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:06.326152 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" event={"ID":"b182c19b-79d8-47d5-9e52-d87787126160","Type":"ContainerStarted","Data":"17428021c8882591defa9e2a19ea4050223584f58d0532683de430cabb7c8f0a"} Jun 02 21:44:06.327240 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:06.327208 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qw5fs" event={"ID":"03b1bb3a-4c92-4555-b5d4-3aaac39381f5","Type":"ContainerStarted","Data":"522334cdef144f961fca3c6ae63a1ee1b9e9f23e42ac9fb66dbe0a57601e21a0"} Jun 02 21:44:07.336158 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:07.336103 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" event={"ID":"b182c19b-79d8-47d5-9e52-d87787126160","Type":"ContainerStarted","Data":"6ac7fbfa2cfacffc74ba79295863f547b2e8c502bd80a9efd3d1520b59a592fd"} Jun 02 21:44:07.342864 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:07.342788 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qw5fs" event={"ID":"03b1bb3a-4c92-4555-b5d4-3aaac39381f5","Type":"ContainerStarted","Data":"24dde2b9cb198c061b16b766959870b57a2c9815821c7b68ae7b043d060198d2"} Jun 02 21:44:07.348824 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:07.348780 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a06ca92b-2521-4f30-aa03-9658da4c8ed2","Type":"ContainerStarted","Data":"a0bda830c32323c184b90e2b738f724896cd006cc0ffac5ae80f77f2ed8a7e36"} Jun 02 21:44:08.353482 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:08.353454 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" event={"ID":"c5cfc52a-6600-48d0-800a-a79dd856bc27","Type":"ContainerStarted","Data":"e85441dfdfc943666a3800356d66c707e1bba7a5f4cae1fee0adcd9b8a7f217a"} Jun 02 21:44:08.356123 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:08.356097 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" event={"ID":"b182c19b-79d8-47d5-9e52-d87787126160","Type":"ContainerStarted","Data":"5d4691248373ca4fb1b966e220e8d40ebbe4ea9f1b1b43c0ae1d8eff84900a3d"} Jun 02 21:44:08.356260 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:08.356129 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" event={"ID":"b182c19b-79d8-47d5-9e52-d87787126160","Type":"ContainerStarted","Data":"c2df6c0b36aed71ca85fd706bc326020df19bbb5dcfd673b780f4177ed0012a8"} Jun 02 21:44:08.356330 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:08.356279 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:08.357787 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:08.357766 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qw5fs" event={"ID":"03b1bb3a-4c92-4555-b5d4-3aaac39381f5","Type":"ContainerStarted","Data":"23b02b3f1c50ebbb0f6514b8f1d6016cefc2af77540068c3b9c29d6572e32966"} Jun 02 21:44:08.360542 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:08.360524 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a06ca92b-2521-4f30-aa03-9658da4c8ed2","Type":"ContainerStarted","Data":"81a7111543103cff6b7833245a068b7702a111a0b5ca3c0dbceabe8731f65826"} Jun 02 21:44:08.360616 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:08.360546 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a06ca92b-2521-4f30-aa03-9658da4c8ed2","Type":"ContainerStarted","Data":"dfcdb355a6950144c5349361f72ea076eb17e2df8d402f15d1bafac3e5c530df"} Jun 02 21:44:08.360616 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:08.360555 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a06ca92b-2521-4f30-aa03-9658da4c8ed2","Type":"ContainerStarted","Data":"db5661dd7e1122209c559a76efe9422abdc395299511d47c71d32baeb7fbc8ab"} Jun 02 21:44:08.360616 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:08.360563 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a06ca92b-2521-4f30-aa03-9658da4c8ed2","Type":"ContainerStarted","Data":"b0c2145c1af5f625bb17c5257eeece00bf1cad36fe8d27cdeccf197af5c2452a"} Jun 02 21:44:08.360616 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:08.360571 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a06ca92b-2521-4f30-aa03-9658da4c8ed2","Type":"ContainerStarted","Data":"389b4fd02b7ced8d5b6e85a25afa36d6b05b0e602926d7e270b9dff14390706b"} Jun 02 21:44:08.375296 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:08.375242 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" podStartSLOduration=1.508223629 podStartE2EDuration="3.375229751s" podCreationTimestamp="2026-06-02 21:44:05 +0000 UTC" firstStartedPulling="2026-06-02 21:44:06.210025643 +0000 UTC m=+130.915035568" lastFinishedPulling="2026-06-02 21:44:08.077031762 +0000 UTC m=+132.782041690" observedRunningTime="2026-06-02 21:44:08.37395196 +0000 UTC m=+133.078961906" watchObservedRunningTime="2026-06-02 21:44:08.375229751 +0000 UTC m=+133.080239698" Jun 02 21:44:08.405244 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:08.405197 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.7491951930000003 podStartE2EDuration="6.405184163s" podCreationTimestamp="2026-06-02 21:44:02 +0000 UTC" firstStartedPulling="2026-06-02 21:44:03.539359218 +0000 UTC m=+128.244369143" lastFinishedPulling="2026-06-02 21:44:07.195348187 +0000 UTC m=+131.900358113" observedRunningTime="2026-06-02 21:44:08.403717691 +0000 UTC m=+133.108727637" watchObservedRunningTime="2026-06-02 21:44:08.405184163 +0000 UTC m=+133.110194111" Jun 02 21:44:08.423060 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:08.423018 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" podStartSLOduration=1.9552787249999999 podStartE2EDuration="5.423007159s" podCreationTimestamp="2026-06-02 21:44:03 +0000 UTC" firstStartedPulling="2026-06-02 21:44:03.727588948 +0000 UTC m=+128.432598873" lastFinishedPulling="2026-06-02 21:44:07.195317381 +0000 UTC m=+131.900327307" observedRunningTime="2026-06-02 21:44:08.422069023 +0000 UTC m=+133.127078998" watchObservedRunningTime="2026-06-02 21:44:08.423007159 +0000 UTC m=+133.128017106" Jun 02 21:44:08.441891 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:08.441852 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qw5fs" podStartSLOduration=131.284153247 podStartE2EDuration="2m12.441841379s" podCreationTimestamp="2026-06-02 21:41:56 +0000 UTC" firstStartedPulling="2026-06-02 21:44:06.0395113 +0000 UTC m=+130.744521226" lastFinishedPulling="2026-06-02 21:44:07.197199433 +0000 UTC m=+131.902209358" observedRunningTime="2026-06-02 21:44:08.440885193 +0000 UTC m=+133.145895140" watchObservedRunningTime="2026-06-02 21:44:08.441841379 +0000 UTC m=+133.146851325" Jun 02 21:44:12.282765 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:12.282734 2574 patch_prober.go:28] interesting pod/image-registry-559f464548-hrcsf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 02 21:44:12.283105 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:12.282790 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-559f464548-hrcsf" podUID="ae671a73-b0b0-4d22-979e-5aa8b509daaf" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 21:44:14.371330 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:14.371297 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-9dbbc88c8-48p6r" Jun 02 21:44:15.571471 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:15.571431 2574 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" podUID="d128b979-2533-4ca6-b009-111b33e8188f" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 02 21:44:15.571920 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:15.571513 2574 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" Jun 02 21:44:15.572084 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:15.572050 2574 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"23ad5c8a2a69e147809e1ecbd78f6e00f9a21aceb7da8125b57baef537decc5c"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jun 02 21:44:15.572152 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:15.572113 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" podUID="d128b979-2533-4ca6-b009-111b33e8188f" containerName="service-proxy" containerID="cri-o://23ad5c8a2a69e147809e1ecbd78f6e00f9a21aceb7da8125b57baef537decc5c" gracePeriod=30 Jun 02 21:44:16.387187 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:16.387151 2574 generic.go:358] "Generic (PLEG): container finished" podID="d128b979-2533-4ca6-b009-111b33e8188f" containerID="23ad5c8a2a69e147809e1ecbd78f6e00f9a21aceb7da8125b57baef537decc5c" exitCode=2 Jun 02 21:44:16.387368 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:16.387220 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" event={"ID":"d128b979-2533-4ca6-b009-111b33e8188f","Type":"ContainerDied","Data":"23ad5c8a2a69e147809e1ecbd78f6e00f9a21aceb7da8125b57baef537decc5c"} Jun 02 21:44:16.387368 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:16.387256 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c9bf5fcc4-brtmx" event={"ID":"d128b979-2533-4ca6-b009-111b33e8188f","Type":"ContainerStarted","Data":"e1a68c9f64b0a4fbaa004eb1aabae97998dddd255401788b30bc22a11b43f141"} Jun 02 21:44:17.296869 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.296833 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-559f464548-hrcsf" podUID="ae671a73-b0b0-4d22-979e-5aa8b509daaf" containerName="registry" containerID="cri-o://e82198d31aa5c279963454cc2a67d46f0478a6248b5ba7f5c80bb88442031183" gracePeriod=30 Jun 02 21:44:17.533149 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.533128 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:44:17.560510 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.560461 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-tls\") pod \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " Jun 02 21:44:17.560510 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.560497 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ae671a73-b0b0-4d22-979e-5aa8b509daaf-ca-trust-extracted\") pod \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " Jun 02 21:44:17.560687 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.560525 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-bound-sa-token\") pod \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " Jun 02 21:44:17.560687 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.560548 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ae671a73-b0b0-4d22-979e-5aa8b509daaf-installation-pull-secrets\") pod \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " Jun 02 21:44:17.560687 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.560573 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ae671a73-b0b0-4d22-979e-5aa8b509daaf-trusted-ca\") pod \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " Jun 02 21:44:17.560687 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.560627 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ae671a73-b0b0-4d22-979e-5aa8b509daaf-image-registry-private-configuration\") pod \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " Jun 02 21:44:17.560687 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.560653 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-certificates\") pod \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " Jun 02 21:44:17.560937 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.560693 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ql5b\" (UniqueName: \"kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-kube-api-access-5ql5b\") pod \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\" (UID: \"ae671a73-b0b0-4d22-979e-5aa8b509daaf\") " Jun 02 21:44:17.561496 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.561363 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ae671a73-b0b0-4d22-979e-5aa8b509daaf" (UID: "ae671a73-b0b0-4d22-979e-5aa8b509daaf"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:44:17.561496 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.561437 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae671a73-b0b0-4d22-979e-5aa8b509daaf-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ae671a73-b0b0-4d22-979e-5aa8b509daaf" (UID: "ae671a73-b0b0-4d22-979e-5aa8b509daaf"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:44:17.563272 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.563246 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ae671a73-b0b0-4d22-979e-5aa8b509daaf" (UID: "ae671a73-b0b0-4d22-979e-5aa8b509daaf"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:44:17.563387 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.563267 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ae671a73-b0b0-4d22-979e-5aa8b509daaf" (UID: "ae671a73-b0b0-4d22-979e-5aa8b509daaf"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:44:17.563491 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.563469 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae671a73-b0b0-4d22-979e-5aa8b509daaf-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ae671a73-b0b0-4d22-979e-5aa8b509daaf" (UID: "ae671a73-b0b0-4d22-979e-5aa8b509daaf"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:44:17.563637 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.563621 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae671a73-b0b0-4d22-979e-5aa8b509daaf-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ae671a73-b0b0-4d22-979e-5aa8b509daaf" (UID: "ae671a73-b0b0-4d22-979e-5aa8b509daaf"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:44:17.563709 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.563695 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-kube-api-access-5ql5b" (OuterVolumeSpecName: "kube-api-access-5ql5b") pod "ae671a73-b0b0-4d22-979e-5aa8b509daaf" (UID: "ae671a73-b0b0-4d22-979e-5aa8b509daaf"). InnerVolumeSpecName "kube-api-access-5ql5b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:44:17.569315 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.569271 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae671a73-b0b0-4d22-979e-5aa8b509daaf-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ae671a73-b0b0-4d22-979e-5aa8b509daaf" (UID: "ae671a73-b0b0-4d22-979e-5aa8b509daaf"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 21:44:17.661510 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.661482 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ql5b\" (UniqueName: \"kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-kube-api-access-5ql5b\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:44:17.661510 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.661512 2574 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-tls\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:44:17.661639 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.661522 2574 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ae671a73-b0b0-4d22-979e-5aa8b509daaf-ca-trust-extracted\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:44:17.661639 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.661540 2574 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae671a73-b0b0-4d22-979e-5aa8b509daaf-bound-sa-token\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:44:17.661639 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.661552 2574 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ae671a73-b0b0-4d22-979e-5aa8b509daaf-installation-pull-secrets\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:44:17.661639 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.661564 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ae671a73-b0b0-4d22-979e-5aa8b509daaf-trusted-ca\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:44:17.661639 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.661576 2574 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ae671a73-b0b0-4d22-979e-5aa8b509daaf-image-registry-private-configuration\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:44:17.661639 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.661585 2574 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ae671a73-b0b0-4d22-979e-5aa8b509daaf-registry-certificates\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:44:17.744106 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.744081 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-c65b8dc4b-nwfpq"] Jun 02 21:44:17.744390 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.744378 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae671a73-b0b0-4d22-979e-5aa8b509daaf" containerName="registry" Jun 02 21:44:17.744435 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.744393 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae671a73-b0b0-4d22-979e-5aa8b509daaf" containerName="registry" Jun 02 21:44:17.744468 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.744436 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="ae671a73-b0b0-4d22-979e-5aa8b509daaf" containerName="registry" Jun 02 21:44:17.748617 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.748603 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c65b8dc4b-nwfpq" Jun 02 21:44:17.751831 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.751813 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 02 21:44:17.751920 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.751847 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-64zdf\"" Jun 02 21:44:17.751920 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.751847 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 02 21:44:17.769396 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.769379 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c65b8dc4b-nwfpq"] Jun 02 21:44:17.863272 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.863252 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86c6x\" (UniqueName: \"kubernetes.io/projected/6bd0abf6-fa78-44ee-88c2-b8e3d9c16fd8-kube-api-access-86c6x\") pod \"downloads-c65b8dc4b-nwfpq\" (UID: \"6bd0abf6-fa78-44ee-88c2-b8e3d9c16fd8\") " pod="openshift-console/downloads-c65b8dc4b-nwfpq" Jun 02 21:44:17.964232 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.964211 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-86c6x\" (UniqueName: \"kubernetes.io/projected/6bd0abf6-fa78-44ee-88c2-b8e3d9c16fd8-kube-api-access-86c6x\") pod \"downloads-c65b8dc4b-nwfpq\" (UID: \"6bd0abf6-fa78-44ee-88c2-b8e3d9c16fd8\") " pod="openshift-console/downloads-c65b8dc4b-nwfpq" Jun 02 21:44:17.974469 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:17.974445 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86c6x\" (UniqueName: \"kubernetes.io/projected/6bd0abf6-fa78-44ee-88c2-b8e3d9c16fd8-kube-api-access-86c6x\") pod \"downloads-c65b8dc4b-nwfpq\" (UID: \"6bd0abf6-fa78-44ee-88c2-b8e3d9c16fd8\") " pod="openshift-console/downloads-c65b8dc4b-nwfpq" Jun 02 21:44:18.056515 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:18.056492 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c65b8dc4b-nwfpq" Jun 02 21:44:18.174251 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:18.174219 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c65b8dc4b-nwfpq"] Jun 02 21:44:18.176441 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:44:18.176416 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bd0abf6_fa78_44ee_88c2_b8e3d9c16fd8.slice/crio-8912b200a8dd0edd53f7f208cc3b3b2794ca3d683fe62c260b5d7a304c1eb872 WatchSource:0}: Error finding container 8912b200a8dd0edd53f7f208cc3b3b2794ca3d683fe62c260b5d7a304c1eb872: Status 404 returned error can't find the container with id 8912b200a8dd0edd53f7f208cc3b3b2794ca3d683fe62c260b5d7a304c1eb872 Jun 02 21:44:18.394090 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:18.394017 2574 generic.go:358] "Generic (PLEG): container finished" podID="ae671a73-b0b0-4d22-979e-5aa8b509daaf" containerID="e82198d31aa5c279963454cc2a67d46f0478a6248b5ba7f5c80bb88442031183" exitCode=0 Jun 02 21:44:18.394090 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:18.394074 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-559f464548-hrcsf" Jun 02 21:44:18.394633 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:18.394101 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-559f464548-hrcsf" event={"ID":"ae671a73-b0b0-4d22-979e-5aa8b509daaf","Type":"ContainerDied","Data":"e82198d31aa5c279963454cc2a67d46f0478a6248b5ba7f5c80bb88442031183"} Jun 02 21:44:18.394633 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:18.394139 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-559f464548-hrcsf" event={"ID":"ae671a73-b0b0-4d22-979e-5aa8b509daaf","Type":"ContainerDied","Data":"f208483a638eadaa1df47b45ee67e3358fb1d10e4b9d77512ecfe930a3a0fa3e"} Jun 02 21:44:18.394633 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:18.394161 2574 scope.go:117] "RemoveContainer" containerID="e82198d31aa5c279963454cc2a67d46f0478a6248b5ba7f5c80bb88442031183" Jun 02 21:44:18.395091 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:18.395069 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c65b8dc4b-nwfpq" event={"ID":"6bd0abf6-fa78-44ee-88c2-b8e3d9c16fd8","Type":"ContainerStarted","Data":"8912b200a8dd0edd53f7f208cc3b3b2794ca3d683fe62c260b5d7a304c1eb872"} Jun 02 21:44:18.403070 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:18.403051 2574 scope.go:117] "RemoveContainer" containerID="e82198d31aa5c279963454cc2a67d46f0478a6248b5ba7f5c80bb88442031183" Jun 02 21:44:18.403354 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:44:18.403335 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e82198d31aa5c279963454cc2a67d46f0478a6248b5ba7f5c80bb88442031183\": container with ID starting with e82198d31aa5c279963454cc2a67d46f0478a6248b5ba7f5c80bb88442031183 not found: ID does not exist" containerID="e82198d31aa5c279963454cc2a67d46f0478a6248b5ba7f5c80bb88442031183" Jun 02 21:44:18.403431 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:18.403363 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e82198d31aa5c279963454cc2a67d46f0478a6248b5ba7f5c80bb88442031183"} err="failed to get container status \"e82198d31aa5c279963454cc2a67d46f0478a6248b5ba7f5c80bb88442031183\": rpc error: code = NotFound desc = could not find container \"e82198d31aa5c279963454cc2a67d46f0478a6248b5ba7f5c80bb88442031183\": container with ID starting with e82198d31aa5c279963454cc2a67d46f0478a6248b5ba7f5c80bb88442031183 not found: ID does not exist" Jun 02 21:44:18.410924 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:18.410903 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-559f464548-hrcsf"] Jun 02 21:44:18.414652 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:18.414629 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-559f464548-hrcsf"] Jun 02 21:44:19.847818 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:19.847785 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae671a73-b0b0-4d22-979e-5aa8b509daaf" path="/var/lib/kubelet/pods/ae671a73-b0b0-4d22-979e-5aa8b509daaf/volumes" Jun 02 21:44:26.015685 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:26.015648 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:26.015685 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:26.015692 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:28.848866 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:28.848832 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-58f5759db4-fk6vh"] Jun 02 21:44:28.853025 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:28.853001 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:44:28.855338 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:28.855310 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 02 21:44:28.855473 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:28.855454 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 02 21:44:28.856032 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:28.855966 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 02 21:44:28.856184 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:28.856130 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4jddz\"" Jun 02 21:44:28.856257 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:28.856213 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 02 21:44:28.856257 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:28.856248 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 02 21:44:28.861973 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:28.861598 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58f5759db4-fk6vh"] Jun 02 21:44:28.965153 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:28.965116 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/811c085c-f758-49d0-8914-ae8d00b3994c-console-oauth-config\") pod \"console-58f5759db4-fk6vh\" (UID: \"811c085c-f758-49d0-8914-ae8d00b3994c\") " pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:44:28.965352 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:28.965164 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/811c085c-f758-49d0-8914-ae8d00b3994c-console-config\") pod \"console-58f5759db4-fk6vh\" (UID: \"811c085c-f758-49d0-8914-ae8d00b3994c\") " pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:44:28.965352 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:28.965218 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/811c085c-f758-49d0-8914-ae8d00b3994c-oauth-serving-cert\") pod \"console-58f5759db4-fk6vh\" (UID: \"811c085c-f758-49d0-8914-ae8d00b3994c\") " pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:44:28.965352 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:28.965269 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/811c085c-f758-49d0-8914-ae8d00b3994c-console-serving-cert\") pod \"console-58f5759db4-fk6vh\" (UID: \"811c085c-f758-49d0-8914-ae8d00b3994c\") " pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:44:28.965533 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:28.965354 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/811c085c-f758-49d0-8914-ae8d00b3994c-service-ca\") pod \"console-58f5759db4-fk6vh\" (UID: \"811c085c-f758-49d0-8914-ae8d00b3994c\") " pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:44:28.965533 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:28.965400 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k29wm\" (UniqueName: \"kubernetes.io/projected/811c085c-f758-49d0-8914-ae8d00b3994c-kube-api-access-k29wm\") pod \"console-58f5759db4-fk6vh\" (UID: \"811c085c-f758-49d0-8914-ae8d00b3994c\") " pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:44:29.066547 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:29.066502 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/811c085c-f758-49d0-8914-ae8d00b3994c-console-serving-cert\") pod \"console-58f5759db4-fk6vh\" (UID: \"811c085c-f758-49d0-8914-ae8d00b3994c\") " pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:44:29.066547 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:29.066546 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/811c085c-f758-49d0-8914-ae8d00b3994c-service-ca\") pod \"console-58f5759db4-fk6vh\" (UID: \"811c085c-f758-49d0-8914-ae8d00b3994c\") " pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:44:29.066774 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:29.066586 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k29wm\" (UniqueName: \"kubernetes.io/projected/811c085c-f758-49d0-8914-ae8d00b3994c-kube-api-access-k29wm\") pod \"console-58f5759db4-fk6vh\" (UID: \"811c085c-f758-49d0-8914-ae8d00b3994c\") " pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:44:29.066774 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:29.066758 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/811c085c-f758-49d0-8914-ae8d00b3994c-console-oauth-config\") pod \"console-58f5759db4-fk6vh\" (UID: \"811c085c-f758-49d0-8914-ae8d00b3994c\") " pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:44:29.066876 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:29.066783 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/811c085c-f758-49d0-8914-ae8d00b3994c-console-config\") pod \"console-58f5759db4-fk6vh\" (UID: \"811c085c-f758-49d0-8914-ae8d00b3994c\") " pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:44:29.066876 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:29.066806 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/811c085c-f758-49d0-8914-ae8d00b3994c-oauth-serving-cert\") pod \"console-58f5759db4-fk6vh\" (UID: \"811c085c-f758-49d0-8914-ae8d00b3994c\") " pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:44:29.067502 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:29.067478 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/811c085c-f758-49d0-8914-ae8d00b3994c-console-config\") pod \"console-58f5759db4-fk6vh\" (UID: \"811c085c-f758-49d0-8914-ae8d00b3994c\") " pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:44:29.067502 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:29.067484 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/811c085c-f758-49d0-8914-ae8d00b3994c-service-ca\") pod \"console-58f5759db4-fk6vh\" (UID: \"811c085c-f758-49d0-8914-ae8d00b3994c\") " pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:44:29.067736 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:29.067505 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/811c085c-f758-49d0-8914-ae8d00b3994c-oauth-serving-cert\") pod \"console-58f5759db4-fk6vh\" (UID: \"811c085c-f758-49d0-8914-ae8d00b3994c\") " pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:44:29.069265 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:29.069244 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/811c085c-f758-49d0-8914-ae8d00b3994c-console-serving-cert\") pod \"console-58f5759db4-fk6vh\" (UID: \"811c085c-f758-49d0-8914-ae8d00b3994c\") " pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:44:29.069349 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:29.069263 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/811c085c-f758-49d0-8914-ae8d00b3994c-console-oauth-config\") pod \"console-58f5759db4-fk6vh\" (UID: \"811c085c-f758-49d0-8914-ae8d00b3994c\") " pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:44:29.073838 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:29.073815 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k29wm\" (UniqueName: \"kubernetes.io/projected/811c085c-f758-49d0-8914-ae8d00b3994c-kube-api-access-k29wm\") pod \"console-58f5759db4-fk6vh\" (UID: \"811c085c-f758-49d0-8914-ae8d00b3994c\") " pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:44:29.165460 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:29.165389 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:44:32.659561 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:44:32.659514 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-r8248" podUID="cf84508a-a1fe-40c8-b587-61f085f0a467" Jun 02 21:44:32.666678 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:44:32.666648 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-mgvvz" podUID="5409f3fd-8515-43fd-8022-23e18e6414d9" Jun 02 21:44:33.441845 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:33.441815 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mgvvz" Jun 02 21:44:33.442013 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:33.441995 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r8248" Jun 02 21:44:33.492187 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:33.492147 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58f5759db4-fk6vh"] Jun 02 21:44:33.493937 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:44:33.493911 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod811c085c_f758_49d0_8914_ae8d00b3994c.slice/crio-510d7f872d88f03fef2a7981fb6aa070c7b9b183c6ab412c343dc887b1696d27 WatchSource:0}: Error finding container 510d7f872d88f03fef2a7981fb6aa070c7b9b183c6ab412c343dc887b1696d27: Status 404 returned error can't find the container with id 510d7f872d88f03fef2a7981fb6aa070c7b9b183c6ab412c343dc887b1696d27 Jun 02 21:44:34.447645 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:34.447606 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58f5759db4-fk6vh" event={"ID":"811c085c-f758-49d0-8914-ae8d00b3994c","Type":"ContainerStarted","Data":"510d7f872d88f03fef2a7981fb6aa070c7b9b183c6ab412c343dc887b1696d27"} Jun 02 21:44:34.452691 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:34.450115 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c65b8dc4b-nwfpq" event={"ID":"6bd0abf6-fa78-44ee-88c2-b8e3d9c16fd8","Type":"ContainerStarted","Data":"3bb72078314557b4259584ddf17ff4f719defc5406307bcd9fcb0a04d4f54ee9"} Jun 02 21:44:34.452902 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:34.452883 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-c65b8dc4b-nwfpq" Jun 02 21:44:34.457831 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:34.457810 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-c65b8dc4b-nwfpq" Jun 02 21:44:34.482098 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:34.482039 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-c65b8dc4b-nwfpq" podStartSLOduration=1.700512464 podStartE2EDuration="17.482021848s" podCreationTimestamp="2026-06-02 21:44:17 +0000 UTC" firstStartedPulling="2026-06-02 21:44:18.178086937 +0000 UTC m=+142.883096862" lastFinishedPulling="2026-06-02 21:44:33.959596322 +0000 UTC m=+158.664606246" observedRunningTime="2026-06-02 21:44:34.466844755 +0000 UTC m=+159.171854701" watchObservedRunningTime="2026-06-02 21:44:34.482021848 +0000 UTC m=+159.187031796" Jun 02 21:44:37.445228 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:37.445184 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls\") pod \"dns-default-r8248\" (UID: \"cf84508a-a1fe-40c8-b587-61f085f0a467\") " pod="openshift-dns/dns-default-r8248" Jun 02 21:44:37.445691 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:37.445245 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert\") pod \"ingress-canary-mgvvz\" (UID: \"5409f3fd-8515-43fd-8022-23e18e6414d9\") " pod="openshift-ingress-canary/ingress-canary-mgvvz" Jun 02 21:44:37.448002 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:37.447968 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf84508a-a1fe-40c8-b587-61f085f0a467-metrics-tls\") pod \"dns-default-r8248\" (UID: \"cf84508a-a1fe-40c8-b587-61f085f0a467\") " pod="openshift-dns/dns-default-r8248" Jun 02 21:44:37.448130 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:37.448025 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5409f3fd-8515-43fd-8022-23e18e6414d9-cert\") pod \"ingress-canary-mgvvz\" (UID: \"5409f3fd-8515-43fd-8022-23e18e6414d9\") " pod="openshift-ingress-canary/ingress-canary-mgvvz" Jun 02 21:44:37.462376 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:37.462344 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58f5759db4-fk6vh" event={"ID":"811c085c-f758-49d0-8914-ae8d00b3994c","Type":"ContainerStarted","Data":"5753f5c66ef3ca015695ba8d64b4870489f8282a83b79c88328f7ba07392d2a3"} Jun 02 21:44:37.478449 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:37.478399 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-58f5759db4-fk6vh" podStartSLOduration=5.725224106 podStartE2EDuration="9.478383472s" podCreationTimestamp="2026-06-02 21:44:28 +0000 UTC" firstStartedPulling="2026-06-02 21:44:33.495715341 +0000 UTC m=+158.200725265" lastFinishedPulling="2026-06-02 21:44:37.248874702 +0000 UTC m=+161.953884631" observedRunningTime="2026-06-02 21:44:37.478082077 +0000 UTC m=+162.183092035" watchObservedRunningTime="2026-06-02 21:44:37.478383472 +0000 UTC m=+162.183393422" Jun 02 21:44:37.644823 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:37.644790 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ssb95\"" Jun 02 21:44:37.644998 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:37.644867 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-f7jrx\"" Jun 02 21:44:37.652657 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:37.652633 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mgvvz" Jun 02 21:44:37.652778 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:37.652733 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r8248" Jun 02 21:44:37.804893 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:37.804862 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mgvvz"] Jun 02 21:44:37.808665 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:44:37.808634 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5409f3fd_8515_43fd_8022_23e18e6414d9.slice/crio-82b77c1557d60bbf2e884123dcc25d1990649bcc5352db107267565e628991bf WatchSource:0}: Error finding container 82b77c1557d60bbf2e884123dcc25d1990649bcc5352db107267565e628991bf: Status 404 returned error can't find the container with id 82b77c1557d60bbf2e884123dcc25d1990649bcc5352db107267565e628991bf Jun 02 21:44:37.826302 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:37.826244 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r8248"] Jun 02 21:44:37.829205 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:44:37.829180 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf84508a_a1fe_40c8_b587_61f085f0a467.slice/crio-b07d9fb2bec69c358f6057d7f8a831120ad094d56f6e9457b4808adb93df40a2 WatchSource:0}: Error finding container b07d9fb2bec69c358f6057d7f8a831120ad094d56f6e9457b4808adb93df40a2: Status 404 returned error can't find the container with id b07d9fb2bec69c358f6057d7f8a831120ad094d56f6e9457b4808adb93df40a2 Jun 02 21:44:38.469717 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:38.469627 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r8248" event={"ID":"cf84508a-a1fe-40c8-b587-61f085f0a467","Type":"ContainerStarted","Data":"b07d9fb2bec69c358f6057d7f8a831120ad094d56f6e9457b4808adb93df40a2"} Jun 02 21:44:38.472025 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:38.471994 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mgvvz" event={"ID":"5409f3fd-8515-43fd-8022-23e18e6414d9","Type":"ContainerStarted","Data":"82b77c1557d60bbf2e884123dcc25d1990649bcc5352db107267565e628991bf"} Jun 02 21:44:39.166055 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:39.166017 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:44:39.166347 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:39.166217 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:44:39.172105 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:39.171955 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:44:39.484869 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:39.484616 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:44:41.484999 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:41.484961 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mgvvz" event={"ID":"5409f3fd-8515-43fd-8022-23e18e6414d9","Type":"ContainerStarted","Data":"fc9151a0575e28e0412566f6fd66cd17c6e43183ee7f752e23ca917de8472aa8"} Jun 02 21:44:41.486963 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:41.486934 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r8248" event={"ID":"cf84508a-a1fe-40c8-b587-61f085f0a467","Type":"ContainerStarted","Data":"84dd28cdd90d41b21817550ba21a6d2bf5ee83a9aec8afd154fc765115f6aea3"} Jun 02 21:44:41.487072 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:41.486969 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r8248" event={"ID":"cf84508a-a1fe-40c8-b587-61f085f0a467","Type":"ContainerStarted","Data":"806595da25a9300fd0f968744fa76920c62de80340673e1cc118c39f6b8d5978"} Jun 02 21:44:41.499718 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:41.499668 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mgvvz" podStartSLOduration=129.606006558 podStartE2EDuration="2m12.499652045s" podCreationTimestamp="2026-06-02 21:42:29 +0000 UTC" firstStartedPulling="2026-06-02 21:44:37.811002418 +0000 UTC m=+162.516012360" lastFinishedPulling="2026-06-02 21:44:40.704647908 +0000 UTC m=+165.409657847" observedRunningTime="2026-06-02 21:44:41.499053369 +0000 UTC m=+166.204063317" watchObservedRunningTime="2026-06-02 21:44:41.499652045 +0000 UTC m=+166.204661994" Jun 02 21:44:41.515123 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:41.513821 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-r8248" podStartSLOduration=129.645763357 podStartE2EDuration="2m12.513804512s" podCreationTimestamp="2026-06-02 21:42:29 +0000 UTC" firstStartedPulling="2026-06-02 21:44:37.831313635 +0000 UTC m=+162.536323559" lastFinishedPulling="2026-06-02 21:44:40.699354785 +0000 UTC m=+165.404364714" observedRunningTime="2026-06-02 21:44:41.512649319 +0000 UTC m=+166.217659266" watchObservedRunningTime="2026-06-02 21:44:41.513804512 +0000 UTC m=+166.218814460" Jun 02 21:44:42.491266 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:42.491230 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-r8248" Jun 02 21:44:46.021275 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:46.021196 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:46.025364 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:46.025340 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6558c5ccc4-6hjtv" Jun 02 21:44:48.832198 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:48.832163 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58f5759db4-fk6vh"] Jun 02 21:44:52.495878 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:44:52.495846 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-r8248" Jun 02 21:45:13.851678 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:13.851622 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-58f5759db4-fk6vh" podUID="811c085c-f758-49d0-8914-ae8d00b3994c" containerName="console" containerID="cri-o://5753f5c66ef3ca015695ba8d64b4870489f8282a83b79c88328f7ba07392d2a3" gracePeriod=15 Jun 02 21:45:14.133454 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.133432 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-58f5759db4-fk6vh_811c085c-f758-49d0-8914-ae8d00b3994c/console/0.log" Jun 02 21:45:14.133557 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.133492 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:45:14.217195 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.217168 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/811c085c-f758-49d0-8914-ae8d00b3994c-service-ca\") pod \"811c085c-f758-49d0-8914-ae8d00b3994c\" (UID: \"811c085c-f758-49d0-8914-ae8d00b3994c\") " Jun 02 21:45:14.217195 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.217208 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/811c085c-f758-49d0-8914-ae8d00b3994c-console-config\") pod \"811c085c-f758-49d0-8914-ae8d00b3994c\" (UID: \"811c085c-f758-49d0-8914-ae8d00b3994c\") " Jun 02 21:45:14.217402 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.217236 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/811c085c-f758-49d0-8914-ae8d00b3994c-console-oauth-config\") pod \"811c085c-f758-49d0-8914-ae8d00b3994c\" (UID: \"811c085c-f758-49d0-8914-ae8d00b3994c\") " Jun 02 21:45:14.217402 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.217300 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k29wm\" (UniqueName: \"kubernetes.io/projected/811c085c-f758-49d0-8914-ae8d00b3994c-kube-api-access-k29wm\") pod \"811c085c-f758-49d0-8914-ae8d00b3994c\" (UID: \"811c085c-f758-49d0-8914-ae8d00b3994c\") " Jun 02 21:45:14.217402 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.217327 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/811c085c-f758-49d0-8914-ae8d00b3994c-oauth-serving-cert\") pod \"811c085c-f758-49d0-8914-ae8d00b3994c\" (UID: \"811c085c-f758-49d0-8914-ae8d00b3994c\") " Jun 02 21:45:14.217402 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.217346 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/811c085c-f758-49d0-8914-ae8d00b3994c-console-serving-cert\") pod \"811c085c-f758-49d0-8914-ae8d00b3994c\" (UID: \"811c085c-f758-49d0-8914-ae8d00b3994c\") " Jun 02 21:45:14.217630 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.217589 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/811c085c-f758-49d0-8914-ae8d00b3994c-service-ca" (OuterVolumeSpecName: "service-ca") pod "811c085c-f758-49d0-8914-ae8d00b3994c" (UID: "811c085c-f758-49d0-8914-ae8d00b3994c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:45:14.217716 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.217692 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/811c085c-f758-49d0-8914-ae8d00b3994c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "811c085c-f758-49d0-8914-ae8d00b3994c" (UID: "811c085c-f758-49d0-8914-ae8d00b3994c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:45:14.217818 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.217798 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/811c085c-f758-49d0-8914-ae8d00b3994c-console-config" (OuterVolumeSpecName: "console-config") pod "811c085c-f758-49d0-8914-ae8d00b3994c" (UID: "811c085c-f758-49d0-8914-ae8d00b3994c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:45:14.219485 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.219459 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/811c085c-f758-49d0-8914-ae8d00b3994c-kube-api-access-k29wm" (OuterVolumeSpecName: "kube-api-access-k29wm") pod "811c085c-f758-49d0-8914-ae8d00b3994c" (UID: "811c085c-f758-49d0-8914-ae8d00b3994c"). InnerVolumeSpecName "kube-api-access-k29wm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:45:14.219485 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.219479 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/811c085c-f758-49d0-8914-ae8d00b3994c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "811c085c-f758-49d0-8914-ae8d00b3994c" (UID: "811c085c-f758-49d0-8914-ae8d00b3994c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:45:14.219656 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.219455 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/811c085c-f758-49d0-8914-ae8d00b3994c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "811c085c-f758-49d0-8914-ae8d00b3994c" (UID: "811c085c-f758-49d0-8914-ae8d00b3994c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:45:14.318129 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.318093 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k29wm\" (UniqueName: \"kubernetes.io/projected/811c085c-f758-49d0-8914-ae8d00b3994c-kube-api-access-k29wm\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:45:14.318129 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.318123 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/811c085c-f758-49d0-8914-ae8d00b3994c-oauth-serving-cert\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:45:14.318129 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.318132 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/811c085c-f758-49d0-8914-ae8d00b3994c-console-serving-cert\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:45:14.318129 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.318143 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/811c085c-f758-49d0-8914-ae8d00b3994c-service-ca\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:45:14.318399 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.318152 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/811c085c-f758-49d0-8914-ae8d00b3994c-console-config\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:45:14.318399 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.318160 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/811c085c-f758-49d0-8914-ae8d00b3994c-console-oauth-config\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:45:14.584010 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.583987 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-58f5759db4-fk6vh_811c085c-f758-49d0-8914-ae8d00b3994c/console/0.log" Jun 02 21:45:14.584145 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.584028 2574 generic.go:358] "Generic (PLEG): container finished" podID="811c085c-f758-49d0-8914-ae8d00b3994c" containerID="5753f5c66ef3ca015695ba8d64b4870489f8282a83b79c88328f7ba07392d2a3" exitCode=2 Jun 02 21:45:14.584145 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.584077 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58f5759db4-fk6vh" event={"ID":"811c085c-f758-49d0-8914-ae8d00b3994c","Type":"ContainerDied","Data":"5753f5c66ef3ca015695ba8d64b4870489f8282a83b79c88328f7ba07392d2a3"} Jun 02 21:45:14.584145 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.584097 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58f5759db4-fk6vh" Jun 02 21:45:14.584145 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.584116 2574 scope.go:117] "RemoveContainer" containerID="5753f5c66ef3ca015695ba8d64b4870489f8282a83b79c88328f7ba07392d2a3" Jun 02 21:45:14.584280 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.584103 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58f5759db4-fk6vh" event={"ID":"811c085c-f758-49d0-8914-ae8d00b3994c","Type":"ContainerDied","Data":"510d7f872d88f03fef2a7981fb6aa070c7b9b183c6ab412c343dc887b1696d27"} Jun 02 21:45:14.592066 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.592050 2574 scope.go:117] "RemoveContainer" containerID="5753f5c66ef3ca015695ba8d64b4870489f8282a83b79c88328f7ba07392d2a3" Jun 02 21:45:14.592351 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:45:14.592321 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5753f5c66ef3ca015695ba8d64b4870489f8282a83b79c88328f7ba07392d2a3\": container with ID starting with 5753f5c66ef3ca015695ba8d64b4870489f8282a83b79c88328f7ba07392d2a3 not found: ID does not exist" containerID="5753f5c66ef3ca015695ba8d64b4870489f8282a83b79c88328f7ba07392d2a3" Jun 02 21:45:14.592425 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.592358 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5753f5c66ef3ca015695ba8d64b4870489f8282a83b79c88328f7ba07392d2a3"} err="failed to get container status \"5753f5c66ef3ca015695ba8d64b4870489f8282a83b79c88328f7ba07392d2a3\": rpc error: code = NotFound desc = could not find container \"5753f5c66ef3ca015695ba8d64b4870489f8282a83b79c88328f7ba07392d2a3\": container with ID starting with 5753f5c66ef3ca015695ba8d64b4870489f8282a83b79c88328f7ba07392d2a3 not found: ID does not exist" Jun 02 21:45:14.603235 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.603211 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58f5759db4-fk6vh"] Jun 02 21:45:14.607222 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:14.607198 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58f5759db4-fk6vh"] Jun 02 21:45:15.846253 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:15.846220 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="811c085c-f758-49d0-8914-ae8d00b3994c" path="/var/lib/kubelet/pods/811c085c-f758-49d0-8914-ae8d00b3994c/volumes" Jun 02 21:45:17.929451 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:17.929421 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-598b558c6d-sq7dd"] Jun 02 21:45:17.929805 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:17.929694 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="811c085c-f758-49d0-8914-ae8d00b3994c" containerName="console" Jun 02 21:45:17.929805 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:17.929705 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="811c085c-f758-49d0-8914-ae8d00b3994c" containerName="console" Jun 02 21:45:17.929805 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:17.929750 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="811c085c-f758-49d0-8914-ae8d00b3994c" containerName="console" Jun 02 21:45:17.952240 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:17.952216 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-598b558c6d-sq7dd"] Jun 02 21:45:17.952374 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:17.952334 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:17.954507 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:17.954485 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 02 21:45:17.954507 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:17.954500 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 02 21:45:17.955187 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:17.955166 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 02 21:45:17.955325 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:17.955230 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4jddz\"" Jun 02 21:45:17.955325 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:17.955240 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 02 21:45:17.955325 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:17.955271 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 02 21:45:17.959164 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:17.959058 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 02 21:45:18.046016 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.045990 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-console-config\") pod \"console-598b558c6d-sq7dd\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:18.046127 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.046023 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-oauth-serving-cert\") pod \"console-598b558c6d-sq7dd\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:18.046127 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.046052 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-console-oauth-config\") pod \"console-598b558c6d-sq7dd\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:18.046127 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.046099 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-trusted-ca-bundle\") pod \"console-598b558c6d-sq7dd\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:18.046232 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.046141 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbxkg\" (UniqueName: \"kubernetes.io/projected/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-kube-api-access-wbxkg\") pod \"console-598b558c6d-sq7dd\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:18.046232 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.046162 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-console-serving-cert\") pod \"console-598b558c6d-sq7dd\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:18.046232 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.046197 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-service-ca\") pod \"console-598b558c6d-sq7dd\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:18.147397 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.147374 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-service-ca\") pod \"console-598b558c6d-sq7dd\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:18.147486 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.147409 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-console-config\") pod \"console-598b558c6d-sq7dd\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:18.147486 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.147432 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-oauth-serving-cert\") pod \"console-598b558c6d-sq7dd\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:18.147486 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.147452 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-console-oauth-config\") pod \"console-598b558c6d-sq7dd\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:18.147486 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.147468 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-trusted-ca-bundle\") pod \"console-598b558c6d-sq7dd\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:18.147682 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.147662 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wbxkg\" (UniqueName: \"kubernetes.io/projected/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-kube-api-access-wbxkg\") pod \"console-598b558c6d-sq7dd\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:18.147758 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.147708 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-console-serving-cert\") pod \"console-598b558c6d-sq7dd\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:18.148199 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.148175 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-service-ca\") pod \"console-598b558c6d-sq7dd\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:18.148317 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.148181 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-oauth-serving-cert\") pod \"console-598b558c6d-sq7dd\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:18.148317 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.148181 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-console-config\") pod \"console-598b558c6d-sq7dd\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:18.148443 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.148429 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-trusted-ca-bundle\") pod \"console-598b558c6d-sq7dd\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:18.149847 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.149821 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-console-oauth-config\") pod \"console-598b558c6d-sq7dd\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:18.149977 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.149961 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-console-serving-cert\") pod \"console-598b558c6d-sq7dd\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:18.156165 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.156146 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbxkg\" (UniqueName: \"kubernetes.io/projected/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-kube-api-access-wbxkg\") pod \"console-598b558c6d-sq7dd\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:18.261899 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.261846 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:18.384081 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.384052 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-598b558c6d-sq7dd"] Jun 02 21:45:18.387254 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:45:18.387229 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ffd1aa7_0960_4dde_b635_f3fdcc0fbc76.slice/crio-e204490cc58fa5f6fc1f8d1b43b9a0533971a4dde8f0535a29ed03780d4bec82 WatchSource:0}: Error finding container e204490cc58fa5f6fc1f8d1b43b9a0533971a4dde8f0535a29ed03780d4bec82: Status 404 returned error can't find the container with id e204490cc58fa5f6fc1f8d1b43b9a0533971a4dde8f0535a29ed03780d4bec82 Jun 02 21:45:18.596898 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.596855 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-598b558c6d-sq7dd" event={"ID":"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76","Type":"ContainerStarted","Data":"77550df4dc6956d9127fab66f7d01020362bd5920eef6db9f49318be4f16227b"} Jun 02 21:45:18.596898 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.596900 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-598b558c6d-sq7dd" event={"ID":"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76","Type":"ContainerStarted","Data":"e204490cc58fa5f6fc1f8d1b43b9a0533971a4dde8f0535a29ed03780d4bec82"} Jun 02 21:45:18.614299 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:18.614241 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-598b558c6d-sq7dd" podStartSLOduration=1.61422767 podStartE2EDuration="1.61422767s" podCreationTimestamp="2026-06-02 21:45:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 21:45:18.612825802 +0000 UTC m=+203.317835749" watchObservedRunningTime="2026-06-02 21:45:18.61422767 +0000 UTC m=+203.319237617" Jun 02 21:45:21.672670 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:21.672639 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 21:45:21.673229 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:21.673063 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="alertmanager" containerID="cri-o://a0bda830c32323c184b90e2b738f724896cd006cc0ffac5ae80f77f2ed8a7e36" gracePeriod=120 Jun 02 21:45:21.673229 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:21.673078 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="kube-rbac-proxy-metric" containerID="cri-o://dfcdb355a6950144c5349361f72ea076eb17e2df8d402f15d1bafac3e5c530df" gracePeriod=120 Jun 02 21:45:21.673229 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:21.673111 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="kube-rbac-proxy-web" containerID="cri-o://b0c2145c1af5f625bb17c5257eeece00bf1cad36fe8d27cdeccf197af5c2452a" gracePeriod=120 Jun 02 21:45:21.673229 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:21.673128 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="config-reloader" containerID="cri-o://389b4fd02b7ced8d5b6e85a25afa36d6b05b0e602926d7e270b9dff14390706b" gracePeriod=120 Jun 02 21:45:21.673229 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:21.673197 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="prom-label-proxy" containerID="cri-o://81a7111543103cff6b7833245a068b7702a111a0b5ca3c0dbceabe8731f65826" gracePeriod=120 Jun 02 21:45:21.673229 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:21.673204 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="kube-rbac-proxy" containerID="cri-o://db5661dd7e1122209c559a76efe9422abdc395299511d47c71d32baeb7fbc8ab" gracePeriod=120 Jun 02 21:45:22.610041 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.610008 2574 generic.go:358] "Generic (PLEG): container finished" podID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerID="81a7111543103cff6b7833245a068b7702a111a0b5ca3c0dbceabe8731f65826" exitCode=0 Jun 02 21:45:22.610041 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.610035 2574 generic.go:358] "Generic (PLEG): container finished" podID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerID="db5661dd7e1122209c559a76efe9422abdc395299511d47c71d32baeb7fbc8ab" exitCode=0 Jun 02 21:45:22.610041 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.610042 2574 generic.go:358] "Generic (PLEG): container finished" podID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerID="389b4fd02b7ced8d5b6e85a25afa36d6b05b0e602926d7e270b9dff14390706b" exitCode=0 Jun 02 21:45:22.610041 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.610048 2574 generic.go:358] "Generic (PLEG): container finished" podID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerID="a0bda830c32323c184b90e2b738f724896cd006cc0ffac5ae80f77f2ed8a7e36" exitCode=0 Jun 02 21:45:22.610319 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.610080 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a06ca92b-2521-4f30-aa03-9658da4c8ed2","Type":"ContainerDied","Data":"81a7111543103cff6b7833245a068b7702a111a0b5ca3c0dbceabe8731f65826"} Jun 02 21:45:22.610319 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.610115 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a06ca92b-2521-4f30-aa03-9658da4c8ed2","Type":"ContainerDied","Data":"db5661dd7e1122209c559a76efe9422abdc395299511d47c71d32baeb7fbc8ab"} Jun 02 21:45:22.610319 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.610125 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a06ca92b-2521-4f30-aa03-9658da4c8ed2","Type":"ContainerDied","Data":"389b4fd02b7ced8d5b6e85a25afa36d6b05b0e602926d7e270b9dff14390706b"} Jun 02 21:45:22.610319 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.610135 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a06ca92b-2521-4f30-aa03-9658da4c8ed2","Type":"ContainerDied","Data":"a0bda830c32323c184b90e2b738f724896cd006cc0ffac5ae80f77f2ed8a7e36"} Jun 02 21:45:22.913744 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.913725 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:22.982934 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.982903 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-config-volume\") pod \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " Jun 02 21:45:22.983066 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.982949 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a06ca92b-2521-4f30-aa03-9658da4c8ed2-tls-assets\") pod \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " Jun 02 21:45:22.983066 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.982973 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a06ca92b-2521-4f30-aa03-9658da4c8ed2-metrics-client-ca\") pod \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " Jun 02 21:45:22.983066 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.983052 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-kube-rbac-proxy-web\") pod \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " Jun 02 21:45:22.983228 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.983080 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a06ca92b-2521-4f30-aa03-9658da4c8ed2-alertmanager-trusted-ca-bundle\") pod \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " Jun 02 21:45:22.983228 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.983119 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a06ca92b-2521-4f30-aa03-9658da4c8ed2-config-out\") pod \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " Jun 02 21:45:22.983228 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.983149 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-main-tls\") pod \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " Jun 02 21:45:22.983228 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.983184 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ssm6d\" (UniqueName: \"kubernetes.io/projected/a06ca92b-2521-4f30-aa03-9658da4c8ed2-kube-api-access-ssm6d\") pod \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " Jun 02 21:45:22.983228 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.983211 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a06ca92b-2521-4f30-aa03-9658da4c8ed2-alertmanager-main-db\") pod \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " Jun 02 21:45:22.983499 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.983245 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-web-config\") pod \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " Jun 02 21:45:22.983499 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.983274 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " Jun 02 21:45:22.983499 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.983353 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-kube-rbac-proxy\") pod \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " Jun 02 21:45:22.983499 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.983387 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-cluster-tls-config\") pod \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\" (UID: \"a06ca92b-2521-4f30-aa03-9658da4c8ed2\") " Jun 02 21:45:22.983499 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.983384 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a06ca92b-2521-4f30-aa03-9658da4c8ed2-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "a06ca92b-2521-4f30-aa03-9658da4c8ed2" (UID: "a06ca92b-2521-4f30-aa03-9658da4c8ed2"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:45:22.983724 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.983619 2574 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a06ca92b-2521-4f30-aa03-9658da4c8ed2-metrics-client-ca\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:45:22.983776 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.983724 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a06ca92b-2521-4f30-aa03-9658da4c8ed2-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "a06ca92b-2521-4f30-aa03-9658da4c8ed2" (UID: "a06ca92b-2521-4f30-aa03-9658da4c8ed2"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:45:22.984716 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.984688 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a06ca92b-2521-4f30-aa03-9658da4c8ed2-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "a06ca92b-2521-4f30-aa03-9658da4c8ed2" (UID: "a06ca92b-2521-4f30-aa03-9658da4c8ed2"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 21:45:22.985716 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.985688 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-config-volume" (OuterVolumeSpecName: "config-volume") pod "a06ca92b-2521-4f30-aa03-9658da4c8ed2" (UID: "a06ca92b-2521-4f30-aa03-9658da4c8ed2"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:45:22.988627 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.986954 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a06ca92b-2521-4f30-aa03-9658da4c8ed2-kube-api-access-ssm6d" (OuterVolumeSpecName: "kube-api-access-ssm6d") pod "a06ca92b-2521-4f30-aa03-9658da4c8ed2" (UID: "a06ca92b-2521-4f30-aa03-9658da4c8ed2"). InnerVolumeSpecName "kube-api-access-ssm6d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:45:22.988627 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.987093 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "a06ca92b-2521-4f30-aa03-9658da4c8ed2" (UID: "a06ca92b-2521-4f30-aa03-9658da4c8ed2"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:45:22.988627 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.987197 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a06ca92b-2521-4f30-aa03-9658da4c8ed2-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a06ca92b-2521-4f30-aa03-9658da4c8ed2" (UID: "a06ca92b-2521-4f30-aa03-9658da4c8ed2"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:45:22.988627 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.987204 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a06ca92b-2521-4f30-aa03-9658da4c8ed2-config-out" (OuterVolumeSpecName: "config-out") pod "a06ca92b-2521-4f30-aa03-9658da4c8ed2" (UID: "a06ca92b-2521-4f30-aa03-9658da4c8ed2"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 21:45:22.988627 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.987350 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "a06ca92b-2521-4f30-aa03-9658da4c8ed2" (UID: "a06ca92b-2521-4f30-aa03-9658da4c8ed2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:45:22.988913 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.988878 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "a06ca92b-2521-4f30-aa03-9658da4c8ed2" (UID: "a06ca92b-2521-4f30-aa03-9658da4c8ed2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:45:22.989109 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.989091 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "a06ca92b-2521-4f30-aa03-9658da4c8ed2" (UID: "a06ca92b-2521-4f30-aa03-9658da4c8ed2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:45:22.991303 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.991266 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "a06ca92b-2521-4f30-aa03-9658da4c8ed2" (UID: "a06ca92b-2521-4f30-aa03-9658da4c8ed2"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:45:22.997340 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:22.997315 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-web-config" (OuterVolumeSpecName: "web-config") pod "a06ca92b-2521-4f30-aa03-9658da4c8ed2" (UID: "a06ca92b-2521-4f30-aa03-9658da4c8ed2"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:45:23.084034 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.084014 2574 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-config-volume\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:45:23.084034 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.084035 2574 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a06ca92b-2521-4f30-aa03-9658da4c8ed2-tls-assets\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:45:23.084146 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.084046 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:45:23.084146 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.084056 2574 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a06ca92b-2521-4f30-aa03-9658da4c8ed2-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:45:23.084146 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.084065 2574 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a06ca92b-2521-4f30-aa03-9658da4c8ed2-config-out\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:45:23.084146 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.084075 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-main-tls\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:45:23.084146 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.084084 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssm6d\" (UniqueName: \"kubernetes.io/projected/a06ca92b-2521-4f30-aa03-9658da4c8ed2-kube-api-access-ssm6d\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:45:23.084146 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.084093 2574 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a06ca92b-2521-4f30-aa03-9658da4c8ed2-alertmanager-main-db\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:45:23.084146 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.084101 2574 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-web-config\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:45:23.084146 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.084109 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:45:23.084146 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.084119 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:45:23.084146 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.084128 2574 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a06ca92b-2521-4f30-aa03-9658da4c8ed2-cluster-tls-config\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:45:23.616102 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.615629 2574 generic.go:358] "Generic (PLEG): container finished" podID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerID="dfcdb355a6950144c5349361f72ea076eb17e2df8d402f15d1bafac3e5c530df" exitCode=0 Jun 02 21:45:23.616102 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.615657 2574 generic.go:358] "Generic (PLEG): container finished" podID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerID="b0c2145c1af5f625bb17c5257eeece00bf1cad36fe8d27cdeccf197af5c2452a" exitCode=0 Jun 02 21:45:23.616102 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.615701 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a06ca92b-2521-4f30-aa03-9658da4c8ed2","Type":"ContainerDied","Data":"dfcdb355a6950144c5349361f72ea076eb17e2df8d402f15d1bafac3e5c530df"} Jun 02 21:45:23.616102 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.615731 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a06ca92b-2521-4f30-aa03-9658da4c8ed2","Type":"ContainerDied","Data":"b0c2145c1af5f625bb17c5257eeece00bf1cad36fe8d27cdeccf197af5c2452a"} Jun 02 21:45:23.616102 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.615748 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a06ca92b-2521-4f30-aa03-9658da4c8ed2","Type":"ContainerDied","Data":"e8cfb15fc7d4e71bfc52a122588a8459b70be8f52efabc2731b3fc70e348ac60"} Jun 02 21:45:23.616102 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.615769 2574 scope.go:117] "RemoveContainer" containerID="81a7111543103cff6b7833245a068b7702a111a0b5ca3c0dbceabe8731f65826" Jun 02 21:45:23.616102 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.615768 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.623387 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.623367 2574 scope.go:117] "RemoveContainer" containerID="dfcdb355a6950144c5349361f72ea076eb17e2df8d402f15d1bafac3e5c530df" Jun 02 21:45:23.630239 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.630224 2574 scope.go:117] "RemoveContainer" containerID="db5661dd7e1122209c559a76efe9422abdc395299511d47c71d32baeb7fbc8ab" Jun 02 21:45:23.637017 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.637000 2574 scope.go:117] "RemoveContainer" containerID="b0c2145c1af5f625bb17c5257eeece00bf1cad36fe8d27cdeccf197af5c2452a" Jun 02 21:45:23.637846 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.637825 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 21:45:23.641370 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.641350 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 21:45:23.644363 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.644342 2574 scope.go:117] "RemoveContainer" containerID="389b4fd02b7ced8d5b6e85a25afa36d6b05b0e602926d7e270b9dff14390706b" Jun 02 21:45:23.650472 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.650457 2574 scope.go:117] "RemoveContainer" containerID="a0bda830c32323c184b90e2b738f724896cd006cc0ffac5ae80f77f2ed8a7e36" Jun 02 21:45:23.658439 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.658422 2574 scope.go:117] "RemoveContainer" containerID="bc8a715968dbc76c8ae6d4b1fa6f9a5ebaa24f0488d7a34a920fd03dc1f1f84e" Jun 02 21:45:23.663863 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.663843 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 21:45:23.664098 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664085 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="config-reloader" Jun 02 21:45:23.664098 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664098 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="config-reloader" Jun 02 21:45:23.664209 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664112 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="kube-rbac-proxy" Jun 02 21:45:23.664209 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664118 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="kube-rbac-proxy" Jun 02 21:45:23.664209 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664129 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="kube-rbac-proxy-web" Jun 02 21:45:23.664209 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664134 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="kube-rbac-proxy-web" Jun 02 21:45:23.664209 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664141 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="alertmanager" Jun 02 21:45:23.664209 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664149 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="alertmanager" Jun 02 21:45:23.664209 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664158 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="kube-rbac-proxy-metric" Jun 02 21:45:23.664209 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664164 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="kube-rbac-proxy-metric" Jun 02 21:45:23.664209 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664172 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="prom-label-proxy" Jun 02 21:45:23.664209 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664177 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="prom-label-proxy" Jun 02 21:45:23.664209 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664185 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="init-config-reloader" Jun 02 21:45:23.664209 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664191 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="init-config-reloader" Jun 02 21:45:23.664759 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664238 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="kube-rbac-proxy-metric" Jun 02 21:45:23.664759 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664247 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="prom-label-proxy" Jun 02 21:45:23.664759 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664255 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="kube-rbac-proxy-web" Jun 02 21:45:23.664759 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664261 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="kube-rbac-proxy" Jun 02 21:45:23.664759 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664266 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="alertmanager" Jun 02 21:45:23.664759 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664273 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" containerName="config-reloader" Jun 02 21:45:23.664759 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664419 2574 scope.go:117] "RemoveContainer" containerID="81a7111543103cff6b7833245a068b7702a111a0b5ca3c0dbceabe8731f65826" Jun 02 21:45:23.664759 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:45:23.664683 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81a7111543103cff6b7833245a068b7702a111a0b5ca3c0dbceabe8731f65826\": container with ID starting with 81a7111543103cff6b7833245a068b7702a111a0b5ca3c0dbceabe8731f65826 not found: ID does not exist" containerID="81a7111543103cff6b7833245a068b7702a111a0b5ca3c0dbceabe8731f65826" Jun 02 21:45:23.664759 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664706 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81a7111543103cff6b7833245a068b7702a111a0b5ca3c0dbceabe8731f65826"} err="failed to get container status \"81a7111543103cff6b7833245a068b7702a111a0b5ca3c0dbceabe8731f65826\": rpc error: code = NotFound desc = could not find container \"81a7111543103cff6b7833245a068b7702a111a0b5ca3c0dbceabe8731f65826\": container with ID starting with 81a7111543103cff6b7833245a068b7702a111a0b5ca3c0dbceabe8731f65826 not found: ID does not exist" Jun 02 21:45:23.664759 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664732 2574 scope.go:117] "RemoveContainer" containerID="dfcdb355a6950144c5349361f72ea076eb17e2df8d402f15d1bafac3e5c530df" Jun 02 21:45:23.665213 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:45:23.664956 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfcdb355a6950144c5349361f72ea076eb17e2df8d402f15d1bafac3e5c530df\": container with ID starting with dfcdb355a6950144c5349361f72ea076eb17e2df8d402f15d1bafac3e5c530df not found: ID does not exist" containerID="dfcdb355a6950144c5349361f72ea076eb17e2df8d402f15d1bafac3e5c530df" Jun 02 21:45:23.665213 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664977 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfcdb355a6950144c5349361f72ea076eb17e2df8d402f15d1bafac3e5c530df"} err="failed to get container status \"dfcdb355a6950144c5349361f72ea076eb17e2df8d402f15d1bafac3e5c530df\": rpc error: code = NotFound desc = could not find container \"dfcdb355a6950144c5349361f72ea076eb17e2df8d402f15d1bafac3e5c530df\": container with ID starting with dfcdb355a6950144c5349361f72ea076eb17e2df8d402f15d1bafac3e5c530df not found: ID does not exist" Jun 02 21:45:23.665213 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.664991 2574 scope.go:117] "RemoveContainer" containerID="db5661dd7e1122209c559a76efe9422abdc395299511d47c71d32baeb7fbc8ab" Jun 02 21:45:23.665213 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:45:23.665190 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db5661dd7e1122209c559a76efe9422abdc395299511d47c71d32baeb7fbc8ab\": container with ID starting with db5661dd7e1122209c559a76efe9422abdc395299511d47c71d32baeb7fbc8ab not found: ID does not exist" containerID="db5661dd7e1122209c559a76efe9422abdc395299511d47c71d32baeb7fbc8ab" Jun 02 21:45:23.665213 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.665207 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db5661dd7e1122209c559a76efe9422abdc395299511d47c71d32baeb7fbc8ab"} err="failed to get container status \"db5661dd7e1122209c559a76efe9422abdc395299511d47c71d32baeb7fbc8ab\": rpc error: code = NotFound desc = could not find container \"db5661dd7e1122209c559a76efe9422abdc395299511d47c71d32baeb7fbc8ab\": container with ID starting with db5661dd7e1122209c559a76efe9422abdc395299511d47c71d32baeb7fbc8ab not found: ID does not exist" Jun 02 21:45:23.665394 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.665220 2574 scope.go:117] "RemoveContainer" containerID="b0c2145c1af5f625bb17c5257eeece00bf1cad36fe8d27cdeccf197af5c2452a" Jun 02 21:45:23.665476 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:45:23.665455 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0c2145c1af5f625bb17c5257eeece00bf1cad36fe8d27cdeccf197af5c2452a\": container with ID starting with b0c2145c1af5f625bb17c5257eeece00bf1cad36fe8d27cdeccf197af5c2452a not found: ID does not exist" containerID="b0c2145c1af5f625bb17c5257eeece00bf1cad36fe8d27cdeccf197af5c2452a" Jun 02 21:45:23.665512 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.665484 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0c2145c1af5f625bb17c5257eeece00bf1cad36fe8d27cdeccf197af5c2452a"} err="failed to get container status \"b0c2145c1af5f625bb17c5257eeece00bf1cad36fe8d27cdeccf197af5c2452a\": rpc error: code = NotFound desc = could not find container \"b0c2145c1af5f625bb17c5257eeece00bf1cad36fe8d27cdeccf197af5c2452a\": container with ID starting with b0c2145c1af5f625bb17c5257eeece00bf1cad36fe8d27cdeccf197af5c2452a not found: ID does not exist" Jun 02 21:45:23.665512 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.665504 2574 scope.go:117] "RemoveContainer" containerID="389b4fd02b7ced8d5b6e85a25afa36d6b05b0e602926d7e270b9dff14390706b" Jun 02 21:45:23.665737 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:45:23.665718 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"389b4fd02b7ced8d5b6e85a25afa36d6b05b0e602926d7e270b9dff14390706b\": container with ID starting with 389b4fd02b7ced8d5b6e85a25afa36d6b05b0e602926d7e270b9dff14390706b not found: ID does not exist" containerID="389b4fd02b7ced8d5b6e85a25afa36d6b05b0e602926d7e270b9dff14390706b" Jun 02 21:45:23.665783 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.665742 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"389b4fd02b7ced8d5b6e85a25afa36d6b05b0e602926d7e270b9dff14390706b"} err="failed to get container status \"389b4fd02b7ced8d5b6e85a25afa36d6b05b0e602926d7e270b9dff14390706b\": rpc error: code = NotFound desc = could not find container \"389b4fd02b7ced8d5b6e85a25afa36d6b05b0e602926d7e270b9dff14390706b\": container with ID starting with 389b4fd02b7ced8d5b6e85a25afa36d6b05b0e602926d7e270b9dff14390706b not found: ID does not exist" Jun 02 21:45:23.665783 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.665756 2574 scope.go:117] "RemoveContainer" containerID="a0bda830c32323c184b90e2b738f724896cd006cc0ffac5ae80f77f2ed8a7e36" Jun 02 21:45:23.665963 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:45:23.665949 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0bda830c32323c184b90e2b738f724896cd006cc0ffac5ae80f77f2ed8a7e36\": container with ID starting with a0bda830c32323c184b90e2b738f724896cd006cc0ffac5ae80f77f2ed8a7e36 not found: ID does not exist" containerID="a0bda830c32323c184b90e2b738f724896cd006cc0ffac5ae80f77f2ed8a7e36" Jun 02 21:45:23.666005 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.665966 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0bda830c32323c184b90e2b738f724896cd006cc0ffac5ae80f77f2ed8a7e36"} err="failed to get container status \"a0bda830c32323c184b90e2b738f724896cd006cc0ffac5ae80f77f2ed8a7e36\": rpc error: code = NotFound desc = could not find container \"a0bda830c32323c184b90e2b738f724896cd006cc0ffac5ae80f77f2ed8a7e36\": container with ID starting with a0bda830c32323c184b90e2b738f724896cd006cc0ffac5ae80f77f2ed8a7e36 not found: ID does not exist" Jun 02 21:45:23.666005 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.665979 2574 scope.go:117] "RemoveContainer" containerID="bc8a715968dbc76c8ae6d4b1fa6f9a5ebaa24f0488d7a34a920fd03dc1f1f84e" Jun 02 21:45:23.666194 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:45:23.666179 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc8a715968dbc76c8ae6d4b1fa6f9a5ebaa24f0488d7a34a920fd03dc1f1f84e\": container with ID starting with bc8a715968dbc76c8ae6d4b1fa6f9a5ebaa24f0488d7a34a920fd03dc1f1f84e not found: ID does not exist" containerID="bc8a715968dbc76c8ae6d4b1fa6f9a5ebaa24f0488d7a34a920fd03dc1f1f84e" Jun 02 21:45:23.666241 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.666198 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc8a715968dbc76c8ae6d4b1fa6f9a5ebaa24f0488d7a34a920fd03dc1f1f84e"} err="failed to get container status \"bc8a715968dbc76c8ae6d4b1fa6f9a5ebaa24f0488d7a34a920fd03dc1f1f84e\": rpc error: code = NotFound desc = could not find container \"bc8a715968dbc76c8ae6d4b1fa6f9a5ebaa24f0488d7a34a920fd03dc1f1f84e\": container with ID starting with bc8a715968dbc76c8ae6d4b1fa6f9a5ebaa24f0488d7a34a920fd03dc1f1f84e not found: ID does not exist" Jun 02 21:45:23.666241 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.666212 2574 scope.go:117] "RemoveContainer" containerID="81a7111543103cff6b7833245a068b7702a111a0b5ca3c0dbceabe8731f65826" Jun 02 21:45:23.666447 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.666430 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81a7111543103cff6b7833245a068b7702a111a0b5ca3c0dbceabe8731f65826"} err="failed to get container status \"81a7111543103cff6b7833245a068b7702a111a0b5ca3c0dbceabe8731f65826\": rpc error: code = NotFound desc = could not find container \"81a7111543103cff6b7833245a068b7702a111a0b5ca3c0dbceabe8731f65826\": container with ID starting with 81a7111543103cff6b7833245a068b7702a111a0b5ca3c0dbceabe8731f65826 not found: ID does not exist" Jun 02 21:45:23.666485 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.666447 2574 scope.go:117] "RemoveContainer" containerID="dfcdb355a6950144c5349361f72ea076eb17e2df8d402f15d1bafac3e5c530df" Jun 02 21:45:23.666624 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.666608 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfcdb355a6950144c5349361f72ea076eb17e2df8d402f15d1bafac3e5c530df"} err="failed to get container status \"dfcdb355a6950144c5349361f72ea076eb17e2df8d402f15d1bafac3e5c530df\": rpc error: code = NotFound desc = could not find container \"dfcdb355a6950144c5349361f72ea076eb17e2df8d402f15d1bafac3e5c530df\": container with ID starting with dfcdb355a6950144c5349361f72ea076eb17e2df8d402f15d1bafac3e5c530df not found: ID does not exist" Jun 02 21:45:23.666666 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.666624 2574 scope.go:117] "RemoveContainer" containerID="db5661dd7e1122209c559a76efe9422abdc395299511d47c71d32baeb7fbc8ab" Jun 02 21:45:23.666855 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.666836 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db5661dd7e1122209c559a76efe9422abdc395299511d47c71d32baeb7fbc8ab"} err="failed to get container status \"db5661dd7e1122209c559a76efe9422abdc395299511d47c71d32baeb7fbc8ab\": rpc error: code = NotFound desc = could not find container \"db5661dd7e1122209c559a76efe9422abdc395299511d47c71d32baeb7fbc8ab\": container with ID starting with db5661dd7e1122209c559a76efe9422abdc395299511d47c71d32baeb7fbc8ab not found: ID does not exist" Jun 02 21:45:23.666895 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.666855 2574 scope.go:117] "RemoveContainer" containerID="b0c2145c1af5f625bb17c5257eeece00bf1cad36fe8d27cdeccf197af5c2452a" Jun 02 21:45:23.667063 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.667046 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0c2145c1af5f625bb17c5257eeece00bf1cad36fe8d27cdeccf197af5c2452a"} err="failed to get container status \"b0c2145c1af5f625bb17c5257eeece00bf1cad36fe8d27cdeccf197af5c2452a\": rpc error: code = NotFound desc = could not find container \"b0c2145c1af5f625bb17c5257eeece00bf1cad36fe8d27cdeccf197af5c2452a\": container with ID starting with b0c2145c1af5f625bb17c5257eeece00bf1cad36fe8d27cdeccf197af5c2452a not found: ID does not exist" Jun 02 21:45:23.667063 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.667062 2574 scope.go:117] "RemoveContainer" containerID="389b4fd02b7ced8d5b6e85a25afa36d6b05b0e602926d7e270b9dff14390706b" Jun 02 21:45:23.667256 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.667239 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"389b4fd02b7ced8d5b6e85a25afa36d6b05b0e602926d7e270b9dff14390706b"} err="failed to get container status \"389b4fd02b7ced8d5b6e85a25afa36d6b05b0e602926d7e270b9dff14390706b\": rpc error: code = NotFound desc = could not find container \"389b4fd02b7ced8d5b6e85a25afa36d6b05b0e602926d7e270b9dff14390706b\": container with ID starting with 389b4fd02b7ced8d5b6e85a25afa36d6b05b0e602926d7e270b9dff14390706b not found: ID does not exist" Jun 02 21:45:23.667319 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.667257 2574 scope.go:117] "RemoveContainer" containerID="a0bda830c32323c184b90e2b738f724896cd006cc0ffac5ae80f77f2ed8a7e36" Jun 02 21:45:23.667494 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.667477 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0bda830c32323c184b90e2b738f724896cd006cc0ffac5ae80f77f2ed8a7e36"} err="failed to get container status \"a0bda830c32323c184b90e2b738f724896cd006cc0ffac5ae80f77f2ed8a7e36\": rpc error: code = NotFound desc = could not find container \"a0bda830c32323c184b90e2b738f724896cd006cc0ffac5ae80f77f2ed8a7e36\": container with ID starting with a0bda830c32323c184b90e2b738f724896cd006cc0ffac5ae80f77f2ed8a7e36 not found: ID does not exist" Jun 02 21:45:23.667548 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.667496 2574 scope.go:117] "RemoveContainer" containerID="bc8a715968dbc76c8ae6d4b1fa6f9a5ebaa24f0488d7a34a920fd03dc1f1f84e" Jun 02 21:45:23.667692 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.667675 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc8a715968dbc76c8ae6d4b1fa6f9a5ebaa24f0488d7a34a920fd03dc1f1f84e"} err="failed to get container status \"bc8a715968dbc76c8ae6d4b1fa6f9a5ebaa24f0488d7a34a920fd03dc1f1f84e\": rpc error: code = NotFound desc = could not find container \"bc8a715968dbc76c8ae6d4b1fa6f9a5ebaa24f0488d7a34a920fd03dc1f1f84e\": container with ID starting with bc8a715968dbc76c8ae6d4b1fa6f9a5ebaa24f0488d7a34a920fd03dc1f1f84e not found: ID does not exist" Jun 02 21:45:23.667761 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.667748 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.669814 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.669799 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 02 21:45:23.669911 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.669868 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 02 21:45:23.669968 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.669868 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 02 21:45:23.670084 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.670064 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 02 21:45:23.670168 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.670084 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 02 21:45:23.670385 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.670362 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qhwbw\"" Jun 02 21:45:23.670476 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.670365 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 02 21:45:23.670631 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.670610 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 02 21:45:23.670864 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.670850 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 02 21:45:23.674964 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.674938 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 02 21:45:23.678477 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.678457 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 21:45:23.789250 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.789223 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b607636d-93c7-4821-8d39-dc69a595262b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.789401 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.789266 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b607636d-93c7-4821-8d39-dc69a595262b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.789401 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.789314 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b607636d-93c7-4821-8d39-dc69a595262b-config-out\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.789401 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.789336 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b607636d-93c7-4821-8d39-dc69a595262b-web-config\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.789401 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.789376 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b607636d-93c7-4821-8d39-dc69a595262b-config-volume\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.789617 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.789404 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b607636d-93c7-4821-8d39-dc69a595262b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.789617 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.789434 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b607636d-93c7-4821-8d39-dc69a595262b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.789617 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.789475 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b607636d-93c7-4821-8d39-dc69a595262b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.789617 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.789502 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b607636d-93c7-4821-8d39-dc69a595262b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.789617 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.789529 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7lh4\" (UniqueName: \"kubernetes.io/projected/b607636d-93c7-4821-8d39-dc69a595262b-kube-api-access-x7lh4\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.789617 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.789556 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b607636d-93c7-4821-8d39-dc69a595262b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.789617 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.789594 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b607636d-93c7-4821-8d39-dc69a595262b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.789821 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.789659 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b607636d-93c7-4821-8d39-dc69a595262b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.851178 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.848916 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a06ca92b-2521-4f30-aa03-9658da4c8ed2" path="/var/lib/kubelet/pods/a06ca92b-2521-4f30-aa03-9658da4c8ed2/volumes" Jun 02 21:45:23.890052 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.889993 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b607636d-93c7-4821-8d39-dc69a595262b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.890052 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.890029 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b607636d-93c7-4821-8d39-dc69a595262b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.890052 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.890048 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b607636d-93c7-4821-8d39-dc69a595262b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.890233 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.890171 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x7lh4\" (UniqueName: \"kubernetes.io/projected/b607636d-93c7-4821-8d39-dc69a595262b-kube-api-access-x7lh4\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.890233 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.890209 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b607636d-93c7-4821-8d39-dc69a595262b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.890361 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.890241 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b607636d-93c7-4821-8d39-dc69a595262b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.890361 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.890269 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b607636d-93c7-4821-8d39-dc69a595262b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.890361 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.890322 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b607636d-93c7-4821-8d39-dc69a595262b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.890510 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.890362 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b607636d-93c7-4821-8d39-dc69a595262b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.890510 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.890392 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b607636d-93c7-4821-8d39-dc69a595262b-config-out\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.890510 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.890416 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b607636d-93c7-4821-8d39-dc69a595262b-web-config\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.890510 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.890462 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b607636d-93c7-4821-8d39-dc69a595262b-config-volume\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.890510 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.890492 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b607636d-93c7-4821-8d39-dc69a595262b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.890749 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.890712 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b607636d-93c7-4821-8d39-dc69a595262b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.891328 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.891011 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b607636d-93c7-4821-8d39-dc69a595262b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.892949 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.892915 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b607636d-93c7-4821-8d39-dc69a595262b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.893047 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.892985 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b607636d-93c7-4821-8d39-dc69a595262b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.893179 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.893149 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b607636d-93c7-4821-8d39-dc69a595262b-web-config\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.893445 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.893423 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b607636d-93c7-4821-8d39-dc69a595262b-config-out\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.893570 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.893553 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b607636d-93c7-4821-8d39-dc69a595262b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.893655 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.893634 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b607636d-93c7-4821-8d39-dc69a595262b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.893694 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.893640 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b607636d-93c7-4821-8d39-dc69a595262b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.894433 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.894412 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b607636d-93c7-4821-8d39-dc69a595262b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.894821 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.894801 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b607636d-93c7-4821-8d39-dc69a595262b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.895058 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.895040 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b607636d-93c7-4821-8d39-dc69a595262b-config-volume\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.897657 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.897639 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7lh4\" (UniqueName: \"kubernetes.io/projected/b607636d-93c7-4821-8d39-dc69a595262b-kube-api-access-x7lh4\") pod \"alertmanager-main-0\" (UID: \"b607636d-93c7-4821-8d39-dc69a595262b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:23.978397 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:23.978352 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 21:45:24.106094 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:24.106060 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 21:45:24.109175 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:45:24.109145 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb607636d_93c7_4821_8d39_dc69a595262b.slice/crio-fcb0f07476ae32a10d5bcb84069c31e3795cf1190078df87ef1fdc058fca09df WatchSource:0}: Error finding container fcb0f07476ae32a10d5bcb84069c31e3795cf1190078df87ef1fdc058fca09df: Status 404 returned error can't find the container with id fcb0f07476ae32a10d5bcb84069c31e3795cf1190078df87ef1fdc058fca09df Jun 02 21:45:24.201276 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:45:24.201242 2574 cadvisor_stats_provider.go:525] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb607636d_93c7_4821_8d39_dc69a595262b.slice/crio-conmon-3e2b30cedec4a1a6d1e45006f8afda9d1ce23739e2c932a87193d94081fd6073.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb607636d_93c7_4821_8d39_dc69a595262b.slice/crio-3e2b30cedec4a1a6d1e45006f8afda9d1ce23739e2c932a87193d94081fd6073.scope\": RecentStats: unable to find data in memory cache]" Jun 02 21:45:24.620365 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:24.620336 2574 generic.go:358] "Generic (PLEG): container finished" podID="b607636d-93c7-4821-8d39-dc69a595262b" containerID="3e2b30cedec4a1a6d1e45006f8afda9d1ce23739e2c932a87193d94081fd6073" exitCode=0 Jun 02 21:45:24.620502 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:24.620404 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b607636d-93c7-4821-8d39-dc69a595262b","Type":"ContainerDied","Data":"3e2b30cedec4a1a6d1e45006f8afda9d1ce23739e2c932a87193d94081fd6073"} Jun 02 21:45:24.620502 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:24.620424 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b607636d-93c7-4821-8d39-dc69a595262b","Type":"ContainerStarted","Data":"fcb0f07476ae32a10d5bcb84069c31e3795cf1190078df87ef1fdc058fca09df"} Jun 02 21:45:25.626128 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:25.626095 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b607636d-93c7-4821-8d39-dc69a595262b","Type":"ContainerStarted","Data":"19139c94cb643309c0a479c64cf4b7d2d517cad2339cc46e9c943ea757a365c7"} Jun 02 21:45:25.626128 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:25.626127 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b607636d-93c7-4821-8d39-dc69a595262b","Type":"ContainerStarted","Data":"d78f691a63740d21739e4fb7f63b94601911b6321cd5625f5af5e72dc16d5b5b"} Jun 02 21:45:25.626558 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:25.626137 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b607636d-93c7-4821-8d39-dc69a595262b","Type":"ContainerStarted","Data":"a794a269d9b31760367cb432125c246125620f601aa54c30133b8e3f9e2ea040"} Jun 02 21:45:25.626558 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:25.626147 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b607636d-93c7-4821-8d39-dc69a595262b","Type":"ContainerStarted","Data":"fef573575e0400700ece79a7a1518aaf9d2b24fcbd8a353ea7a6ccf33c29277b"} Jun 02 21:45:25.626558 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:25.626155 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b607636d-93c7-4821-8d39-dc69a595262b","Type":"ContainerStarted","Data":"6326575fbd76ab61b1d73514436e6b406cd09637b4cf58fe81b8a4c0c1617704"} Jun 02 21:45:25.626558 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:25.626164 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b607636d-93c7-4821-8d39-dc69a595262b","Type":"ContainerStarted","Data":"ff4928d777d1d749c9059cd13a37e1aacdb258257e9e14e0cf91e5603bd833e8"} Jun 02 21:45:25.649103 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:25.649058 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.649043449 podStartE2EDuration="2.649043449s" podCreationTimestamp="2026-06-02 21:45:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 21:45:25.647835423 +0000 UTC m=+210.352845365" watchObservedRunningTime="2026-06-02 21:45:25.649043449 +0000 UTC m=+210.354053395" Jun 02 21:45:28.262520 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:28.262480 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:28.262880 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:28.262531 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:28.267231 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:28.267208 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:28.637756 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:28.637728 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:45:42.550170 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:42.550131 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-s4gt2"] Jun 02 21:45:42.553154 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:42.553132 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4gt2" Jun 02 21:45:42.555063 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:42.555044 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 02 21:45:42.561790 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:42.561769 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-s4gt2"] Jun 02 21:45:42.623207 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:42.623187 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8fbc35d9-7682-4a07-add0-5c9f1f384656-kubelet-config\") pod \"global-pull-secret-syncer-s4gt2\" (UID: \"8fbc35d9-7682-4a07-add0-5c9f1f384656\") " pod="kube-system/global-pull-secret-syncer-s4gt2" Jun 02 21:45:42.623304 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:42.623215 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8fbc35d9-7682-4a07-add0-5c9f1f384656-dbus\") pod \"global-pull-secret-syncer-s4gt2\" (UID: \"8fbc35d9-7682-4a07-add0-5c9f1f384656\") " pod="kube-system/global-pull-secret-syncer-s4gt2" Jun 02 21:45:42.623304 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:42.623236 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fbc35d9-7682-4a07-add0-5c9f1f384656-original-pull-secret\") pod \"global-pull-secret-syncer-s4gt2\" (UID: \"8fbc35d9-7682-4a07-add0-5c9f1f384656\") " pod="kube-system/global-pull-secret-syncer-s4gt2" Jun 02 21:45:42.723532 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:42.723508 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8fbc35d9-7682-4a07-add0-5c9f1f384656-kubelet-config\") pod \"global-pull-secret-syncer-s4gt2\" (UID: \"8fbc35d9-7682-4a07-add0-5c9f1f384656\") " pod="kube-system/global-pull-secret-syncer-s4gt2" Jun 02 21:45:42.723646 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:42.723545 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8fbc35d9-7682-4a07-add0-5c9f1f384656-dbus\") pod \"global-pull-secret-syncer-s4gt2\" (UID: \"8fbc35d9-7682-4a07-add0-5c9f1f384656\") " pod="kube-system/global-pull-secret-syncer-s4gt2" Jun 02 21:45:42.723646 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:42.723579 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fbc35d9-7682-4a07-add0-5c9f1f384656-original-pull-secret\") pod \"global-pull-secret-syncer-s4gt2\" (UID: \"8fbc35d9-7682-4a07-add0-5c9f1f384656\") " pod="kube-system/global-pull-secret-syncer-s4gt2" Jun 02 21:45:42.723646 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:42.723623 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8fbc35d9-7682-4a07-add0-5c9f1f384656-kubelet-config\") pod \"global-pull-secret-syncer-s4gt2\" (UID: \"8fbc35d9-7682-4a07-add0-5c9f1f384656\") " pod="kube-system/global-pull-secret-syncer-s4gt2" Jun 02 21:45:42.723821 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:42.723726 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8fbc35d9-7682-4a07-add0-5c9f1f384656-dbus\") pod \"global-pull-secret-syncer-s4gt2\" (UID: \"8fbc35d9-7682-4a07-add0-5c9f1f384656\") " pod="kube-system/global-pull-secret-syncer-s4gt2" Jun 02 21:45:42.725625 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:42.725599 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fbc35d9-7682-4a07-add0-5c9f1f384656-original-pull-secret\") pod \"global-pull-secret-syncer-s4gt2\" (UID: \"8fbc35d9-7682-4a07-add0-5c9f1f384656\") " pod="kube-system/global-pull-secret-syncer-s4gt2" Jun 02 21:45:42.863175 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:42.863154 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4gt2" Jun 02 21:45:42.981948 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:42.981908 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-s4gt2"] Jun 02 21:45:42.984554 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:45:42.984529 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fbc35d9_7682_4a07_add0_5c9f1f384656.slice/crio-2b4e92be0d39c97911191d278ecae2b88a8b54df728a9a0d0a3a93181c80b281 WatchSource:0}: Error finding container 2b4e92be0d39c97911191d278ecae2b88a8b54df728a9a0d0a3a93181c80b281: Status 404 returned error can't find the container with id 2b4e92be0d39c97911191d278ecae2b88a8b54df728a9a0d0a3a93181c80b281 Jun 02 21:45:43.675664 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:43.675629 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-s4gt2" event={"ID":"8fbc35d9-7682-4a07-add0-5c9f1f384656","Type":"ContainerStarted","Data":"2b4e92be0d39c97911191d278ecae2b88a8b54df728a9a0d0a3a93181c80b281"} Jun 02 21:45:47.689436 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:47.689400 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-s4gt2" event={"ID":"8fbc35d9-7682-4a07-add0-5c9f1f384656","Type":"ContainerStarted","Data":"ecf697a2706a5a7e0aa887e0f31367c45c548ed5123d2d3fb202d9f5133e1246"} Jun 02 21:45:47.704029 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:45:47.703983 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-s4gt2" podStartSLOduration=1.853086608 podStartE2EDuration="5.703971952s" podCreationTimestamp="2026-06-02 21:45:42 +0000 UTC" firstStartedPulling="2026-06-02 21:45:42.986268663 +0000 UTC m=+227.691278587" lastFinishedPulling="2026-06-02 21:45:46.837154006 +0000 UTC m=+231.542163931" observedRunningTime="2026-06-02 21:45:47.70267903 +0000 UTC m=+232.407688979" watchObservedRunningTime="2026-06-02 21:45:47.703971952 +0000 UTC m=+232.408981953" Jun 02 21:46:34.020605 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:34.020575 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-ktclg"] Jun 02 21:46:34.022753 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:34.022739 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-ktclg" Jun 02 21:46:34.024960 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:34.024935 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 02 21:46:34.024960 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:34.024934 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-96zz8\"" Jun 02 21:46:34.025570 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:34.025550 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 02 21:46:34.030400 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:34.030377 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-ktclg"] Jun 02 21:46:34.087880 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:34.087859 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fe434f3-d353-4b8d-a216-83b82bb1bcfc-tmp\") pod \"openshift-lws-operator-bfc7f696d-ktclg\" (UID: \"9fe434f3-d353-4b8d-a216-83b82bb1bcfc\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-ktclg" Jun 02 21:46:34.087974 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:34.087891 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5nqp\" (UniqueName: \"kubernetes.io/projected/9fe434f3-d353-4b8d-a216-83b82bb1bcfc-kube-api-access-g5nqp\") pod \"openshift-lws-operator-bfc7f696d-ktclg\" (UID: \"9fe434f3-d353-4b8d-a216-83b82bb1bcfc\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-ktclg" Jun 02 21:46:34.188450 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:34.188425 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fe434f3-d353-4b8d-a216-83b82bb1bcfc-tmp\") pod \"openshift-lws-operator-bfc7f696d-ktclg\" (UID: \"9fe434f3-d353-4b8d-a216-83b82bb1bcfc\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-ktclg" Jun 02 21:46:34.188552 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:34.188458 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g5nqp\" (UniqueName: \"kubernetes.io/projected/9fe434f3-d353-4b8d-a216-83b82bb1bcfc-kube-api-access-g5nqp\") pod \"openshift-lws-operator-bfc7f696d-ktclg\" (UID: \"9fe434f3-d353-4b8d-a216-83b82bb1bcfc\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-ktclg" Jun 02 21:46:34.188790 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:34.188770 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fe434f3-d353-4b8d-a216-83b82bb1bcfc-tmp\") pod \"openshift-lws-operator-bfc7f696d-ktclg\" (UID: \"9fe434f3-d353-4b8d-a216-83b82bb1bcfc\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-ktclg" Jun 02 21:46:34.195869 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:34.195840 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5nqp\" (UniqueName: \"kubernetes.io/projected/9fe434f3-d353-4b8d-a216-83b82bb1bcfc-kube-api-access-g5nqp\") pod \"openshift-lws-operator-bfc7f696d-ktclg\" (UID: \"9fe434f3-d353-4b8d-a216-83b82bb1bcfc\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-ktclg" Jun 02 21:46:34.331445 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:34.331359 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-ktclg" Jun 02 21:46:34.453947 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:34.453923 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-ktclg"] Jun 02 21:46:34.456536 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:46:34.456495 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9fe434f3_d353_4b8d_a216_83b82bb1bcfc.slice/crio-9c04ce1e445d81a98b40ba71501df7b8cc2d75be0a7b1e933d73d24f26df6472 WatchSource:0}: Error finding container 9c04ce1e445d81a98b40ba71501df7b8cc2d75be0a7b1e933d73d24f26df6472: Status 404 returned error can't find the container with id 9c04ce1e445d81a98b40ba71501df7b8cc2d75be0a7b1e933d73d24f26df6472 Jun 02 21:46:34.819601 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:34.819562 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-ktclg" event={"ID":"9fe434f3-d353-4b8d-a216-83b82bb1bcfc","Type":"ContainerStarted","Data":"9c04ce1e445d81a98b40ba71501df7b8cc2d75be0a7b1e933d73d24f26df6472"} Jun 02 21:46:39.839735 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:39.839701 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-ktclg" event={"ID":"9fe434f3-d353-4b8d-a216-83b82bb1bcfc","Type":"ContainerStarted","Data":"de5dd237989e9d36f56edac8365b5ae0673c672fe0d1f20bb7e89e6b164ec2ea"} Jun 02 21:46:39.856964 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:39.856920 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-ktclg" podStartSLOduration=0.797707911 podStartE2EDuration="5.856907768s" podCreationTimestamp="2026-06-02 21:46:34 +0000 UTC" firstStartedPulling="2026-06-02 21:46:34.457941979 +0000 UTC m=+279.162951904" lastFinishedPulling="2026-06-02 21:46:39.517141822 +0000 UTC m=+284.222151761" observedRunningTime="2026-06-02 21:46:39.855696139 +0000 UTC m=+284.560706087" watchObservedRunningTime="2026-06-02 21:46:39.856907768 +0000 UTC m=+284.561917717" Jun 02 21:46:55.735480 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:55.735455 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/ovn-acl-logging/0.log" Jun 02 21:46:55.736156 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:55.736134 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/ovn-acl-logging/0.log" Jun 02 21:46:55.738263 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:55.738244 2574 kubelet.go:1628] "Image garbage collection succeeded" Jun 02 21:46:56.332726 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.332695 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6858789655-c699q"] Jun 02 21:46:56.335763 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.335748 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6858789655-c699q" Jun 02 21:46:56.338631 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.338611 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 02 21:46:56.339302 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.339258 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kzbcq\"" Jun 02 21:46:56.339397 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.339345 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 02 21:46:56.339727 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.339712 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 02 21:46:56.339797 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.339781 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 02 21:46:56.350841 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.350822 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6858789655-c699q"] Jun 02 21:46:56.458535 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.458503 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bc42d688-55ca-408e-a5cc-725033271899-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6858789655-c699q\" (UID: \"bc42d688-55ca-408e-a5cc-725033271899\") " pod="opendatahub/opendatahub-operator-controller-manager-6858789655-c699q" Jun 02 21:46:56.458681 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.458543 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc42d688-55ca-408e-a5cc-725033271899-tmp\") pod \"opendatahub-operator-controller-manager-6858789655-c699q\" (UID: \"bc42d688-55ca-408e-a5cc-725033271899\") " pod="opendatahub/opendatahub-operator-controller-manager-6858789655-c699q" Jun 02 21:46:56.458681 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.458581 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttn4p\" (UniqueName: \"kubernetes.io/projected/bc42d688-55ca-408e-a5cc-725033271899-kube-api-access-ttn4p\") pod \"opendatahub-operator-controller-manager-6858789655-c699q\" (UID: \"bc42d688-55ca-408e-a5cc-725033271899\") " pod="opendatahub/opendatahub-operator-controller-manager-6858789655-c699q" Jun 02 21:46:56.458681 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.458642 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bc42d688-55ca-408e-a5cc-725033271899-webhook-cert\") pod \"opendatahub-operator-controller-manager-6858789655-c699q\" (UID: \"bc42d688-55ca-408e-a5cc-725033271899\") " pod="opendatahub/opendatahub-operator-controller-manager-6858789655-c699q" Jun 02 21:46:56.458681 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.458669 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bc42d688-55ca-408e-a5cc-725033271899-manifests\") pod \"opendatahub-operator-controller-manager-6858789655-c699q\" (UID: \"bc42d688-55ca-408e-a5cc-725033271899\") " pod="opendatahub/opendatahub-operator-controller-manager-6858789655-c699q" Jun 02 21:46:56.559397 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.559371 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bc42d688-55ca-408e-a5cc-725033271899-webhook-cert\") pod \"opendatahub-operator-controller-manager-6858789655-c699q\" (UID: \"bc42d688-55ca-408e-a5cc-725033271899\") " pod="opendatahub/opendatahub-operator-controller-manager-6858789655-c699q" Jun 02 21:46:56.559505 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.559401 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bc42d688-55ca-408e-a5cc-725033271899-manifests\") pod \"opendatahub-operator-controller-manager-6858789655-c699q\" (UID: \"bc42d688-55ca-408e-a5cc-725033271899\") " pod="opendatahub/opendatahub-operator-controller-manager-6858789655-c699q" Jun 02 21:46:56.559505 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.559436 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bc42d688-55ca-408e-a5cc-725033271899-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6858789655-c699q\" (UID: \"bc42d688-55ca-408e-a5cc-725033271899\") " pod="opendatahub/opendatahub-operator-controller-manager-6858789655-c699q" Jun 02 21:46:56.559571 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.559550 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc42d688-55ca-408e-a5cc-725033271899-tmp\") pod \"opendatahub-operator-controller-manager-6858789655-c699q\" (UID: \"bc42d688-55ca-408e-a5cc-725033271899\") " pod="opendatahub/opendatahub-operator-controller-manager-6858789655-c699q" Jun 02 21:46:56.559621 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.559602 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ttn4p\" (UniqueName: \"kubernetes.io/projected/bc42d688-55ca-408e-a5cc-725033271899-kube-api-access-ttn4p\") pod \"opendatahub-operator-controller-manager-6858789655-c699q\" (UID: \"bc42d688-55ca-408e-a5cc-725033271899\") " pod="opendatahub/opendatahub-operator-controller-manager-6858789655-c699q" Jun 02 21:46:56.559812 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.559792 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bc42d688-55ca-408e-a5cc-725033271899-manifests\") pod \"opendatahub-operator-controller-manager-6858789655-c699q\" (UID: \"bc42d688-55ca-408e-a5cc-725033271899\") " pod="opendatahub/opendatahub-operator-controller-manager-6858789655-c699q" Jun 02 21:46:56.561733 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.561709 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc42d688-55ca-408e-a5cc-725033271899-tmp\") pod \"opendatahub-operator-controller-manager-6858789655-c699q\" (UID: \"bc42d688-55ca-408e-a5cc-725033271899\") " pod="opendatahub/opendatahub-operator-controller-manager-6858789655-c699q" Jun 02 21:46:56.561827 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.561787 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bc42d688-55ca-408e-a5cc-725033271899-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6858789655-c699q\" (UID: \"bc42d688-55ca-408e-a5cc-725033271899\") " pod="opendatahub/opendatahub-operator-controller-manager-6858789655-c699q" Jun 02 21:46:56.561928 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.561912 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bc42d688-55ca-408e-a5cc-725033271899-webhook-cert\") pod \"opendatahub-operator-controller-manager-6858789655-c699q\" (UID: \"bc42d688-55ca-408e-a5cc-725033271899\") " pod="opendatahub/opendatahub-operator-controller-manager-6858789655-c699q" Jun 02 21:46:56.566838 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.566813 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttn4p\" (UniqueName: \"kubernetes.io/projected/bc42d688-55ca-408e-a5cc-725033271899-kube-api-access-ttn4p\") pod \"opendatahub-operator-controller-manager-6858789655-c699q\" (UID: \"bc42d688-55ca-408e-a5cc-725033271899\") " pod="opendatahub/opendatahub-operator-controller-manager-6858789655-c699q" Jun 02 21:46:56.646468 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.646437 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6858789655-c699q" Jun 02 21:46:56.769824 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.769757 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6858789655-c699q"] Jun 02 21:46:56.772888 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:46:56.772862 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc42d688_55ca_408e_a5cc_725033271899.slice/crio-f15cfac6acbfc16829a6cea41fa285e7eb1a4af881e8ca8750516cf3aaa4921a WatchSource:0}: Error finding container f15cfac6acbfc16829a6cea41fa285e7eb1a4af881e8ca8750516cf3aaa4921a: Status 404 returned error can't find the container with id f15cfac6acbfc16829a6cea41fa285e7eb1a4af881e8ca8750516cf3aaa4921a Jun 02 21:46:56.890277 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:56.890249 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6858789655-c699q" event={"ID":"bc42d688-55ca-408e-a5cc-725033271899","Type":"ContainerStarted","Data":"f15cfac6acbfc16829a6cea41fa285e7eb1a4af881e8ca8750516cf3aaa4921a"} Jun 02 21:46:59.902192 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:46:59.902158 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6858789655-c699q" event={"ID":"bc42d688-55ca-408e-a5cc-725033271899","Type":"ContainerStarted","Data":"e69bb40b2c8a36f94adb4e74133a42d4f6b37532499271a5fadb2510a1defd27"} Jun 02 21:47:00.909072 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:00.908987 2574 generic.go:358] "Generic (PLEG): container finished" podID="bc42d688-55ca-408e-a5cc-725033271899" containerID="e69bb40b2c8a36f94adb4e74133a42d4f6b37532499271a5fadb2510a1defd27" exitCode=0 Jun 02 21:47:00.909072 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:00.909040 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6858789655-c699q" event={"ID":"bc42d688-55ca-408e-a5cc-725033271899","Type":"ContainerDied","Data":"e69bb40b2c8a36f94adb4e74133a42d4f6b37532499271a5fadb2510a1defd27"} Jun 02 21:47:00.909533 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:00.909516 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 21:47:01.914229 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:01.914190 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6858789655-c699q" event={"ID":"bc42d688-55ca-408e-a5cc-725033271899","Type":"ContainerStarted","Data":"19aaf63f4364cbdaacede8046443c96bba07d3948989005367b4bf71c17beaf2"} Jun 02 21:47:01.914636 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:01.914330 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6858789655-c699q" Jun 02 21:47:01.934311 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:01.934250 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6858789655-c699q" podStartSLOduration=1.4807139 podStartE2EDuration="5.934235204s" podCreationTimestamp="2026-06-02 21:46:56 +0000 UTC" firstStartedPulling="2026-06-02 21:46:56.775426463 +0000 UTC m=+301.480436392" lastFinishedPulling="2026-06-02 21:47:01.228947757 +0000 UTC m=+305.933957696" observedRunningTime="2026-06-02 21:47:01.932949384 +0000 UTC m=+306.637959332" watchObservedRunningTime="2026-06-02 21:47:01.934235204 +0000 UTC m=+306.639245151" Jun 02 21:47:12.920031 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:12.920004 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6858789655-c699q" Jun 02 21:47:15.123820 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.123789 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5f5cff6669-4z479"] Jun 02 21:47:15.127246 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.127218 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5f5cff6669-4z479" Jun 02 21:47:15.129978 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.129951 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 02 21:47:15.130642 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.130615 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-f7hjq\"" Jun 02 21:47:15.130760 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.130673 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 02 21:47:15.130760 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.130680 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 02 21:47:15.141663 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.141638 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5f5cff6669-4z479"] Jun 02 21:47:15.245160 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.245133 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7b495b9df5-kp8vh"] Jun 02 21:47:15.248379 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.248357 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7b495b9df5-kp8vh" Jun 02 21:47:15.250519 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.250495 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 02 21:47:15.250519 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.250522 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 02 21:47:15.250704 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.250540 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 02 21:47:15.250807 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.250792 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 02 21:47:15.250877 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.250837 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-hn4lk\"" Jun 02 21:47:15.263085 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.263063 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7b495b9df5-kp8vh"] Jun 02 21:47:15.310649 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.310617 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/db12bdc3-78f6-4481-91e6-d3028ec65104-manager-config\") pod \"lws-controller-manager-5f5cff6669-4z479\" (UID: \"db12bdc3-78f6-4481-91e6-d3028ec65104\") " pod="openshift-lws-operator/lws-controller-manager-5f5cff6669-4z479" Jun 02 21:47:15.310796 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.310674 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nd2c\" (UniqueName: \"kubernetes.io/projected/db12bdc3-78f6-4481-91e6-d3028ec65104-kube-api-access-6nd2c\") pod \"lws-controller-manager-5f5cff6669-4z479\" (UID: \"db12bdc3-78f6-4481-91e6-d3028ec65104\") " pod="openshift-lws-operator/lws-controller-manager-5f5cff6669-4z479" Jun 02 21:47:15.310796 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.310778 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/db12bdc3-78f6-4481-91e6-d3028ec65104-metrics-cert\") pod \"lws-controller-manager-5f5cff6669-4z479\" (UID: \"db12bdc3-78f6-4481-91e6-d3028ec65104\") " pod="openshift-lws-operator/lws-controller-manager-5f5cff6669-4z479" Jun 02 21:47:15.310887 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.310841 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db12bdc3-78f6-4481-91e6-d3028ec65104-cert\") pod \"lws-controller-manager-5f5cff6669-4z479\" (UID: \"db12bdc3-78f6-4481-91e6-d3028ec65104\") " pod="openshift-lws-operator/lws-controller-manager-5f5cff6669-4z479" Jun 02 21:47:15.411517 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.411444 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b-tmp\") pod \"kube-auth-proxy-7b495b9df5-kp8vh\" (UID: \"477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b\") " pod="openshift-ingress/kube-auth-proxy-7b495b9df5-kp8vh" Jun 02 21:47:15.411517 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.411496 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/db12bdc3-78f6-4481-91e6-d3028ec65104-metrics-cert\") pod \"lws-controller-manager-5f5cff6669-4z479\" (UID: \"db12bdc3-78f6-4481-91e6-d3028ec65104\") " pod="openshift-lws-operator/lws-controller-manager-5f5cff6669-4z479" Jun 02 21:47:15.411517 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.411518 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7x24\" (UniqueName: \"kubernetes.io/projected/477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b-kube-api-access-n7x24\") pod \"kube-auth-proxy-7b495b9df5-kp8vh\" (UID: \"477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b\") " pod="openshift-ingress/kube-auth-proxy-7b495b9df5-kp8vh" Jun 02 21:47:15.411819 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.411557 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b-tls-certs\") pod \"kube-auth-proxy-7b495b9df5-kp8vh\" (UID: \"477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b\") " pod="openshift-ingress/kube-auth-proxy-7b495b9df5-kp8vh" Jun 02 21:47:15.411819 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.411582 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db12bdc3-78f6-4481-91e6-d3028ec65104-cert\") pod \"lws-controller-manager-5f5cff6669-4z479\" (UID: \"db12bdc3-78f6-4481-91e6-d3028ec65104\") " pod="openshift-lws-operator/lws-controller-manager-5f5cff6669-4z479" Jun 02 21:47:15.411819 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.411611 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/db12bdc3-78f6-4481-91e6-d3028ec65104-manager-config\") pod \"lws-controller-manager-5f5cff6669-4z479\" (UID: \"db12bdc3-78f6-4481-91e6-d3028ec65104\") " pod="openshift-lws-operator/lws-controller-manager-5f5cff6669-4z479" Jun 02 21:47:15.411819 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.411643 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6nd2c\" (UniqueName: \"kubernetes.io/projected/db12bdc3-78f6-4481-91e6-d3028ec65104-kube-api-access-6nd2c\") pod \"lws-controller-manager-5f5cff6669-4z479\" (UID: \"db12bdc3-78f6-4481-91e6-d3028ec65104\") " pod="openshift-lws-operator/lws-controller-manager-5f5cff6669-4z479" Jun 02 21:47:15.412310 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.412268 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/db12bdc3-78f6-4481-91e6-d3028ec65104-manager-config\") pod \"lws-controller-manager-5f5cff6669-4z479\" (UID: \"db12bdc3-78f6-4481-91e6-d3028ec65104\") " pod="openshift-lws-operator/lws-controller-manager-5f5cff6669-4z479" Jun 02 21:47:15.414188 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.414170 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db12bdc3-78f6-4481-91e6-d3028ec65104-cert\") pod \"lws-controller-manager-5f5cff6669-4z479\" (UID: \"db12bdc3-78f6-4481-91e6-d3028ec65104\") " pod="openshift-lws-operator/lws-controller-manager-5f5cff6669-4z479" Jun 02 21:47:15.414252 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.414213 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/db12bdc3-78f6-4481-91e6-d3028ec65104-metrics-cert\") pod \"lws-controller-manager-5f5cff6669-4z479\" (UID: \"db12bdc3-78f6-4481-91e6-d3028ec65104\") " pod="openshift-lws-operator/lws-controller-manager-5f5cff6669-4z479" Jun 02 21:47:15.419336 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.419308 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nd2c\" (UniqueName: \"kubernetes.io/projected/db12bdc3-78f6-4481-91e6-d3028ec65104-kube-api-access-6nd2c\") pod \"lws-controller-manager-5f5cff6669-4z479\" (UID: \"db12bdc3-78f6-4481-91e6-d3028ec65104\") " pod="openshift-lws-operator/lws-controller-manager-5f5cff6669-4z479" Jun 02 21:47:15.438333 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.438313 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5f5cff6669-4z479" Jun 02 21:47:15.513112 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.513079 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b-tmp\") pod \"kube-auth-proxy-7b495b9df5-kp8vh\" (UID: \"477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b\") " pod="openshift-ingress/kube-auth-proxy-7b495b9df5-kp8vh" Jun 02 21:47:15.513258 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.513136 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n7x24\" (UniqueName: \"kubernetes.io/projected/477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b-kube-api-access-n7x24\") pod \"kube-auth-proxy-7b495b9df5-kp8vh\" (UID: \"477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b\") " pod="openshift-ingress/kube-auth-proxy-7b495b9df5-kp8vh" Jun 02 21:47:15.513258 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.513166 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b-tls-certs\") pod \"kube-auth-proxy-7b495b9df5-kp8vh\" (UID: \"477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b\") " pod="openshift-ingress/kube-auth-proxy-7b495b9df5-kp8vh" Jun 02 21:47:15.513418 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:47:15.513301 2574 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jun 02 21:47:15.513418 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:47:15.513360 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b-tls-certs podName:477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b nodeName:}" failed. No retries permitted until 2026-06-02 21:47:16.013342163 +0000 UTC m=+320.718352089 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b-tls-certs") pod "kube-auth-proxy-7b495b9df5-kp8vh" (UID: "477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b") : secret "kube-auth-proxy-tls" not found Jun 02 21:47:15.515120 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.515094 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b-tmp\") pod \"kube-auth-proxy-7b495b9df5-kp8vh\" (UID: \"477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b\") " pod="openshift-ingress/kube-auth-proxy-7b495b9df5-kp8vh" Jun 02 21:47:15.522858 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.522829 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7x24\" (UniqueName: \"kubernetes.io/projected/477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b-kube-api-access-n7x24\") pod \"kube-auth-proxy-7b495b9df5-kp8vh\" (UID: \"477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b\") " pod="openshift-ingress/kube-auth-proxy-7b495b9df5-kp8vh" Jun 02 21:47:15.571435 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.571412 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5f5cff6669-4z479"] Jun 02 21:47:15.574659 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:47:15.574632 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb12bdc3_78f6_4481_91e6_d3028ec65104.slice/crio-dd7861cc48f16d89ba1bc10d8f7c62ee1612f986fdbce0ae1d159b7ff2af9fc7 WatchSource:0}: Error finding container dd7861cc48f16d89ba1bc10d8f7c62ee1612f986fdbce0ae1d159b7ff2af9fc7: Status 404 returned error can't find the container with id dd7861cc48f16d89ba1bc10d8f7c62ee1612f986fdbce0ae1d159b7ff2af9fc7 Jun 02 21:47:15.958012 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:15.957977 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5f5cff6669-4z479" event={"ID":"db12bdc3-78f6-4481-91e6-d3028ec65104","Type":"ContainerStarted","Data":"dd7861cc48f16d89ba1bc10d8f7c62ee1612f986fdbce0ae1d159b7ff2af9fc7"} Jun 02 21:47:16.016666 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:16.016633 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b-tls-certs\") pod \"kube-auth-proxy-7b495b9df5-kp8vh\" (UID: \"477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b\") " pod="openshift-ingress/kube-auth-proxy-7b495b9df5-kp8vh" Jun 02 21:47:16.019158 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:16.019132 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b-tls-certs\") pod \"kube-auth-proxy-7b495b9df5-kp8vh\" (UID: \"477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b\") " pod="openshift-ingress/kube-auth-proxy-7b495b9df5-kp8vh" Jun 02 21:47:16.157998 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:16.157967 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7b495b9df5-kp8vh" Jun 02 21:47:16.294585 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:16.292732 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7b495b9df5-kp8vh"] Jun 02 21:47:16.296336 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:47:16.296263 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod477d3b3e_f9f9_4a21_96a6_563ea7c2fe2b.slice/crio-254a10bd87525e0c9cbb8b0da6dccc966bd3d4c8cd4d2f716f97805b2ace7ae7 WatchSource:0}: Error finding container 254a10bd87525e0c9cbb8b0da6dccc966bd3d4c8cd4d2f716f97805b2ace7ae7: Status 404 returned error can't find the container with id 254a10bd87525e0c9cbb8b0da6dccc966bd3d4c8cd4d2f716f97805b2ace7ae7 Jun 02 21:47:16.964487 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:16.964450 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7b495b9df5-kp8vh" event={"ID":"477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b","Type":"ContainerStarted","Data":"254a10bd87525e0c9cbb8b0da6dccc966bd3d4c8cd4d2f716f97805b2ace7ae7"} Jun 02 21:47:17.970507 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:17.970467 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5f5cff6669-4z479" event={"ID":"db12bdc3-78f6-4481-91e6-d3028ec65104","Type":"ContainerStarted","Data":"f7ac3dbedd1b81ad4af49c5251fb8bdf9e7a1df6b4100953a3bbfe9d44dc1883"} Jun 02 21:47:17.970920 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:17.970596 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5f5cff6669-4z479" Jun 02 21:47:17.988703 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:17.988419 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5f5cff6669-4z479" podStartSLOduration=1.179146551 podStartE2EDuration="2.98840174s" podCreationTimestamp="2026-06-02 21:47:15 +0000 UTC" firstStartedPulling="2026-06-02 21:47:15.576548364 +0000 UTC m=+320.281558289" lastFinishedPulling="2026-06-02 21:47:17.38580354 +0000 UTC m=+322.090813478" observedRunningTime="2026-06-02 21:47:17.98633441 +0000 UTC m=+322.691344358" watchObservedRunningTime="2026-06-02 21:47:17.98840174 +0000 UTC m=+322.693411687" Jun 02 21:47:19.978961 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:19.978928 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7b495b9df5-kp8vh" event={"ID":"477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b","Type":"ContainerStarted","Data":"06842486e7820951d5b1e722e78cb589723a535bdac284ef4c9be10e0f64d70a"} Jun 02 21:47:19.993395 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:19.993335 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7b495b9df5-kp8vh" podStartSLOduration=2.050077118 podStartE2EDuration="4.993323186s" podCreationTimestamp="2026-06-02 21:47:15 +0000 UTC" firstStartedPulling="2026-06-02 21:47:16.298817059 +0000 UTC m=+321.003826984" lastFinishedPulling="2026-06-02 21:47:19.242063125 +0000 UTC m=+323.947073052" observedRunningTime="2026-06-02 21:47:19.992031927 +0000 UTC m=+324.697041874" watchObservedRunningTime="2026-06-02 21:47:19.993323186 +0000 UTC m=+324.698333133" Jun 02 21:47:28.976102 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:47:28.976073 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5f5cff6669-4z479" Jun 02 21:48:59.178809 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:48:59.178774 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-zql7h"] Jun 02 21:48:59.180906 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:48:59.180890 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-zql7h" Jun 02 21:48:59.182923 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:48:59.182904 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 02 21:48:59.183509 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:48:59.183494 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jun 02 21:48:59.183559 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:48:59.183515 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 02 21:48:59.183559 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:48:59.183548 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-ktxtw\"" Jun 02 21:48:59.191252 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:48:59.191232 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-zql7h"] Jun 02 21:48:59.201953 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:48:59.201928 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpc8r\" (UniqueName: \"kubernetes.io/projected/2fd9bcf0-5b12-41ec-808b-abd23609a600-kube-api-access-wpc8r\") pod \"dns-operator-controller-manager-648d5c98bc-zql7h\" (UID: \"2fd9bcf0-5b12-41ec-808b-abd23609a600\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-zql7h" Jun 02 21:48:59.303033 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:48:59.303000 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wpc8r\" (UniqueName: \"kubernetes.io/projected/2fd9bcf0-5b12-41ec-808b-abd23609a600-kube-api-access-wpc8r\") pod \"dns-operator-controller-manager-648d5c98bc-zql7h\" (UID: \"2fd9bcf0-5b12-41ec-808b-abd23609a600\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-zql7h" Jun 02 21:48:59.310796 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:48:59.310770 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpc8r\" (UniqueName: \"kubernetes.io/projected/2fd9bcf0-5b12-41ec-808b-abd23609a600-kube-api-access-wpc8r\") pod \"dns-operator-controller-manager-648d5c98bc-zql7h\" (UID: \"2fd9bcf0-5b12-41ec-808b-abd23609a600\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-zql7h" Jun 02 21:48:59.491472 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:48:59.491394 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-zql7h" Jun 02 21:48:59.618779 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:48:59.618755 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-zql7h"] Jun 02 21:48:59.621038 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:48:59.621014 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fd9bcf0_5b12_41ec_808b_abd23609a600.slice/crio-99d1752084eb8180b525ab33ef20623112465d3455345dcfe9b9a4ddc1ba3160 WatchSource:0}: Error finding container 99d1752084eb8180b525ab33ef20623112465d3455345dcfe9b9a4ddc1ba3160: Status 404 returned error can't find the container with id 99d1752084eb8180b525ab33ef20623112465d3455345dcfe9b9a4ddc1ba3160 Jun 02 21:49:00.303202 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:00.303167 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-zql7h" event={"ID":"2fd9bcf0-5b12-41ec-808b-abd23609a600","Type":"ContainerStarted","Data":"99d1752084eb8180b525ab33ef20623112465d3455345dcfe9b9a4ddc1ba3160"} Jun 02 21:49:01.721553 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:01.721474 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z2js8"] Jun 02 21:49:01.723651 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:01.723631 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z2js8" Jun 02 21:49:01.726057 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:01.726036 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 02 21:49:01.726057 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:01.726045 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 02 21:49:01.726235 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:01.726045 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hfbzz\"" Jun 02 21:49:01.731938 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:01.731917 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z2js8"] Jun 02 21:49:01.825340 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:01.825310 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5b77779-a698-4595-9d7f-276d8272dcc0-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-z2js8\" (UID: \"a5b77779-a698-4595-9d7f-276d8272dcc0\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z2js8" Jun 02 21:49:01.825526 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:01.825351 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv9tq\" (UniqueName: \"kubernetes.io/projected/a5b77779-a698-4595-9d7f-276d8272dcc0-kube-api-access-xv9tq\") pod \"kuadrant-console-plugin-6cb54b5c86-z2js8\" (UID: \"a5b77779-a698-4595-9d7f-276d8272dcc0\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z2js8" Jun 02 21:49:01.825526 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:01.825402 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a5b77779-a698-4595-9d7f-276d8272dcc0-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-z2js8\" (UID: \"a5b77779-a698-4595-9d7f-276d8272dcc0\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z2js8" Jun 02 21:49:01.926849 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:01.926806 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5b77779-a698-4595-9d7f-276d8272dcc0-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-z2js8\" (UID: \"a5b77779-a698-4595-9d7f-276d8272dcc0\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z2js8" Jun 02 21:49:01.926849 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:01.926850 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xv9tq\" (UniqueName: \"kubernetes.io/projected/a5b77779-a698-4595-9d7f-276d8272dcc0-kube-api-access-xv9tq\") pod \"kuadrant-console-plugin-6cb54b5c86-z2js8\" (UID: \"a5b77779-a698-4595-9d7f-276d8272dcc0\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z2js8" Jun 02 21:49:01.927064 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:01.926958 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a5b77779-a698-4595-9d7f-276d8272dcc0-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-z2js8\" (UID: \"a5b77779-a698-4595-9d7f-276d8272dcc0\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z2js8" Jun 02 21:49:01.927064 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:49:01.926976 2574 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jun 02 21:49:01.927064 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:49:01.927041 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5b77779-a698-4595-9d7f-276d8272dcc0-plugin-serving-cert podName:a5b77779-a698-4595-9d7f-276d8272dcc0 nodeName:}" failed. No retries permitted until 2026-06-02 21:49:02.427021573 +0000 UTC m=+427.132031517 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/a5b77779-a698-4595-9d7f-276d8272dcc0-plugin-serving-cert") pod "kuadrant-console-plugin-6cb54b5c86-z2js8" (UID: "a5b77779-a698-4595-9d7f-276d8272dcc0") : secret "plugin-serving-cert" not found Jun 02 21:49:01.927634 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:01.927613 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a5b77779-a698-4595-9d7f-276d8272dcc0-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-z2js8\" (UID: \"a5b77779-a698-4595-9d7f-276d8272dcc0\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z2js8" Jun 02 21:49:01.935174 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:01.935148 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv9tq\" (UniqueName: \"kubernetes.io/projected/a5b77779-a698-4595-9d7f-276d8272dcc0-kube-api-access-xv9tq\") pod \"kuadrant-console-plugin-6cb54b5c86-z2js8\" (UID: \"a5b77779-a698-4595-9d7f-276d8272dcc0\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z2js8" Jun 02 21:49:02.312558 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:02.312523 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-zql7h" event={"ID":"2fd9bcf0-5b12-41ec-808b-abd23609a600","Type":"ContainerStarted","Data":"a5ebf2947f1cc4e6617381e8366dc0d7ab32b4c8c0d98d7e9eeb1bacdc00c986"} Jun 02 21:49:02.312721 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:02.312603 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-zql7h" Jun 02 21:49:02.329157 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:02.329110 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-zql7h" podStartSLOduration=1.485803204 podStartE2EDuration="3.329097444s" podCreationTimestamp="2026-06-02 21:48:59 +0000 UTC" firstStartedPulling="2026-06-02 21:48:59.623240911 +0000 UTC m=+424.328250836" lastFinishedPulling="2026-06-02 21:49:01.466535151 +0000 UTC m=+426.171545076" observedRunningTime="2026-06-02 21:49:02.327671882 +0000 UTC m=+427.032681830" watchObservedRunningTime="2026-06-02 21:49:02.329097444 +0000 UTC m=+427.034107391" Jun 02 21:49:02.430795 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:02.430757 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5b77779-a698-4595-9d7f-276d8272dcc0-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-z2js8\" (UID: \"a5b77779-a698-4595-9d7f-276d8272dcc0\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z2js8" Jun 02 21:49:02.433183 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:02.433160 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5b77779-a698-4595-9d7f-276d8272dcc0-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-z2js8\" (UID: \"a5b77779-a698-4595-9d7f-276d8272dcc0\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z2js8" Jun 02 21:49:02.633236 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:02.633203 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z2js8" Jun 02 21:49:02.761715 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:02.761683 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z2js8"] Jun 02 21:49:02.764854 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:49:02.764827 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5b77779_a698_4595_9d7f_276d8272dcc0.slice/crio-bf994a872fefb122ea737bf6aae67446d07115873d8c5c5aec7e8c634a954cb3 WatchSource:0}: Error finding container bf994a872fefb122ea737bf6aae67446d07115873d8c5c5aec7e8c634a954cb3: Status 404 returned error can't find the container with id bf994a872fefb122ea737bf6aae67446d07115873d8c5c5aec7e8c634a954cb3 Jun 02 21:49:03.316753 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:03.316712 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z2js8" event={"ID":"a5b77779-a698-4595-9d7f-276d8272dcc0","Type":"ContainerStarted","Data":"bf994a872fefb122ea737bf6aae67446d07115873d8c5c5aec7e8c634a954cb3"} Jun 02 21:49:13.319269 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:13.319238 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-zql7h" Jun 02 21:49:25.396542 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:25.396510 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z2js8" event={"ID":"a5b77779-a698-4595-9d7f-276d8272dcc0","Type":"ContainerStarted","Data":"d3c671b8406b50bdb9a6f5cfad5810f5e393a76de17ae2ea1205d53a4b77cb27"} Jun 02 21:49:25.412476 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:25.412437 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z2js8" podStartSLOduration=1.875377206 podStartE2EDuration="24.412423278s" podCreationTimestamp="2026-06-02 21:49:01 +0000 UTC" firstStartedPulling="2026-06-02 21:49:02.766189499 +0000 UTC m=+427.471199424" lastFinishedPulling="2026-06-02 21:49:25.303235571 +0000 UTC m=+450.008245496" observedRunningTime="2026-06-02 21:49:25.411130291 +0000 UTC m=+450.116140250" watchObservedRunningTime="2026-06-02 21:49:25.412423278 +0000 UTC m=+450.117433224" Jun 02 21:49:41.992386 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:41.992349 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-598b558c6d-sq7dd"] Jun 02 21:49:56.159927 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:56.159896 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-wdtns"] Jun 02 21:49:56.196413 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:56.196387 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-wdtns"] Jun 02 21:49:56.196557 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:56.196494 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-wdtns" Jun 02 21:49:56.198621 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:56.198600 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 02 21:49:56.263149 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:56.263122 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-wdtns"] Jun 02 21:49:56.270670 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:56.270640 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d-config-file\") pod \"limitador-limitador-7d549b5b-wdtns\" (UID: \"fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d\") " pod="kuadrant-system/limitador-limitador-7d549b5b-wdtns" Jun 02 21:49:56.270776 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:56.270687 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dkvv\" (UniqueName: \"kubernetes.io/projected/fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d-kube-api-access-5dkvv\") pod \"limitador-limitador-7d549b5b-wdtns\" (UID: \"fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d\") " pod="kuadrant-system/limitador-limitador-7d549b5b-wdtns" Jun 02 21:49:56.371255 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:56.371224 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d-config-file\") pod \"limitador-limitador-7d549b5b-wdtns\" (UID: \"fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d\") " pod="kuadrant-system/limitador-limitador-7d549b5b-wdtns" Jun 02 21:49:56.371403 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:56.371271 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5dkvv\" (UniqueName: \"kubernetes.io/projected/fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d-kube-api-access-5dkvv\") pod \"limitador-limitador-7d549b5b-wdtns\" (UID: \"fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d\") " pod="kuadrant-system/limitador-limitador-7d549b5b-wdtns" Jun 02 21:49:56.371820 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:56.371802 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d-config-file\") pod \"limitador-limitador-7d549b5b-wdtns\" (UID: \"fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d\") " pod="kuadrant-system/limitador-limitador-7d549b5b-wdtns" Jun 02 21:49:56.382146 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:56.382122 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dkvv\" (UniqueName: \"kubernetes.io/projected/fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d-kube-api-access-5dkvv\") pod \"limitador-limitador-7d549b5b-wdtns\" (UID: \"fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d\") " pod="kuadrant-system/limitador-limitador-7d549b5b-wdtns" Jun 02 21:49:56.506552 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:56.506485 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-wdtns" Jun 02 21:49:56.627820 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:56.627798 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-wdtns"] Jun 02 21:49:56.630381 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:49:56.630347 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc0ee09d_5a0e_46f2_a9b4_0bb34f79183d.slice/crio-06e76232a3741a77f040cdcdb5db473121f8467d16586c3b0be4916f59198699 WatchSource:0}: Error finding container 06e76232a3741a77f040cdcdb5db473121f8467d16586c3b0be4916f59198699: Status 404 returned error can't find the container with id 06e76232a3741a77f040cdcdb5db473121f8467d16586c3b0be4916f59198699 Jun 02 21:49:56.962978 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:56.962946 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-jd45j"] Jun 02 21:49:56.988194 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:56.988159 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-jd45j"] Jun 02 21:49:56.988194 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:56.988187 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-jd45j" Jun 02 21:49:56.990305 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:56.990258 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-v7rnv\"" Jun 02 21:49:57.060371 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:57.060345 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7498df8756-js2cw"] Jun 02 21:49:57.072332 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:57.072309 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-js2cw"] Jun 02 21:49:57.072484 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:57.072400 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-js2cw" Jun 02 21:49:57.077733 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:57.077704 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5xpx\" (UniqueName: \"kubernetes.io/projected/e1ea0c17-6e76-4f89-a5cf-2ecb520deef8-kube-api-access-z5xpx\") pod \"authorino-f99f4b5cd-jd45j\" (UID: \"e1ea0c17-6e76-4f89-a5cf-2ecb520deef8\") " pod="kuadrant-system/authorino-f99f4b5cd-jd45j" Jun 02 21:49:57.178325 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:57.178302 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9cvd\" (UniqueName: \"kubernetes.io/projected/8c176dda-32bd-490f-99cb-4bf3a81a164c-kube-api-access-s9cvd\") pod \"authorino-7498df8756-js2cw\" (UID: \"8c176dda-32bd-490f-99cb-4bf3a81a164c\") " pod="kuadrant-system/authorino-7498df8756-js2cw" Jun 02 21:49:57.178620 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:57.178372 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z5xpx\" (UniqueName: \"kubernetes.io/projected/e1ea0c17-6e76-4f89-a5cf-2ecb520deef8-kube-api-access-z5xpx\") pod \"authorino-f99f4b5cd-jd45j\" (UID: \"e1ea0c17-6e76-4f89-a5cf-2ecb520deef8\") " pod="kuadrant-system/authorino-f99f4b5cd-jd45j" Jun 02 21:49:57.185581 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:57.185558 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5xpx\" (UniqueName: \"kubernetes.io/projected/e1ea0c17-6e76-4f89-a5cf-2ecb520deef8-kube-api-access-z5xpx\") pod \"authorino-f99f4b5cd-jd45j\" (UID: \"e1ea0c17-6e76-4f89-a5cf-2ecb520deef8\") " pod="kuadrant-system/authorino-f99f4b5cd-jd45j" Jun 02 21:49:57.279358 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:57.279220 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s9cvd\" (UniqueName: \"kubernetes.io/projected/8c176dda-32bd-490f-99cb-4bf3a81a164c-kube-api-access-s9cvd\") pod \"authorino-7498df8756-js2cw\" (UID: \"8c176dda-32bd-490f-99cb-4bf3a81a164c\") " pod="kuadrant-system/authorino-7498df8756-js2cw" Jun 02 21:49:57.288885 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:57.288858 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9cvd\" (UniqueName: \"kubernetes.io/projected/8c176dda-32bd-490f-99cb-4bf3a81a164c-kube-api-access-s9cvd\") pod \"authorino-7498df8756-js2cw\" (UID: \"8c176dda-32bd-490f-99cb-4bf3a81a164c\") " pod="kuadrant-system/authorino-7498df8756-js2cw" Jun 02 21:49:57.299541 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:57.299520 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-jd45j" Jun 02 21:49:57.382009 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:57.381748 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-js2cw" Jun 02 21:49:57.483315 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:57.482828 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-jd45j"] Jun 02 21:49:57.506746 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:57.506676 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-jd45j" event={"ID":"e1ea0c17-6e76-4f89-a5cf-2ecb520deef8","Type":"ContainerStarted","Data":"8336215d5db4fc0242d97180c9a1e31d6994f42e73131df9b733261dd8037c42"} Jun 02 21:49:57.508616 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:57.508571 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-wdtns" event={"ID":"fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d","Type":"ContainerStarted","Data":"06e76232a3741a77f040cdcdb5db473121f8467d16586c3b0be4916f59198699"} Jun 02 21:49:57.563234 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:57.563140 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-js2cw"] Jun 02 21:49:58.517770 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:49:58.517700 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-js2cw" event={"ID":"8c176dda-32bd-490f-99cb-4bf3a81a164c","Type":"ContainerStarted","Data":"0a771760101268e9c4743e6c637bcad929d8f89dde517115c78a217b8729fac8"} Jun 02 21:50:02.533687 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:02.533649 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-js2cw" event={"ID":"8c176dda-32bd-490f-99cb-4bf3a81a164c","Type":"ContainerStarted","Data":"ef2057d75888bf6f615a29f757946d277801207f9aa31b626b01894b66f457fe"} Jun 02 21:50:02.535037 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:02.535014 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-jd45j" event={"ID":"e1ea0c17-6e76-4f89-a5cf-2ecb520deef8","Type":"ContainerStarted","Data":"cf588350928f3c0784efc06da02596d53f6171c1cc37064d7f5ce442842f0bc6"} Jun 02 21:50:02.536431 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:02.536410 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-wdtns" event={"ID":"fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d","Type":"ContainerStarted","Data":"a10e36363ce2ca0dfb0c21d8fd7adee20fd23363eaf0c9fe36ed40647beb25b8"} Jun 02 21:50:02.536567 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:02.536553 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7d549b5b-wdtns" Jun 02 21:50:02.547425 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:02.547383 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-7498df8756-js2cw" podStartSLOduration=1.171348877 podStartE2EDuration="5.54737314s" podCreationTimestamp="2026-06-02 21:49:57 +0000 UTC" firstStartedPulling="2026-06-02 21:49:57.573820114 +0000 UTC m=+482.278830047" lastFinishedPulling="2026-06-02 21:50:01.949844384 +0000 UTC m=+486.654854310" observedRunningTime="2026-06-02 21:50:02.546551548 +0000 UTC m=+487.251561495" watchObservedRunningTime="2026-06-02 21:50:02.54737314 +0000 UTC m=+487.252383084" Jun 02 21:50:02.563420 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:02.563356 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7d549b5b-wdtns" podStartSLOduration=1.234462289 podStartE2EDuration="6.563340022s" podCreationTimestamp="2026-06-02 21:49:56 +0000 UTC" firstStartedPulling="2026-06-02 21:49:56.632079517 +0000 UTC m=+481.337089442" lastFinishedPulling="2026-06-02 21:50:01.96095724 +0000 UTC m=+486.665967175" observedRunningTime="2026-06-02 21:50:02.560210637 +0000 UTC m=+487.265220585" watchObservedRunningTime="2026-06-02 21:50:02.563340022 +0000 UTC m=+487.268349982" Jun 02 21:50:02.573889 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:02.573857 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-jd45j"] Jun 02 21:50:02.581589 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:02.581506 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-f99f4b5cd-jd45j" podStartSLOduration=2.123734682 podStartE2EDuration="6.581492298s" podCreationTimestamp="2026-06-02 21:49:56 +0000 UTC" firstStartedPulling="2026-06-02 21:49:57.491795075 +0000 UTC m=+482.196805014" lastFinishedPulling="2026-06-02 21:50:01.949552702 +0000 UTC m=+486.654562630" observedRunningTime="2026-06-02 21:50:02.581320007 +0000 UTC m=+487.286329952" watchObservedRunningTime="2026-06-02 21:50:02.581492298 +0000 UTC m=+487.286502245" Jun 02 21:50:04.543323 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:04.543239 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-f99f4b5cd-jd45j" podUID="e1ea0c17-6e76-4f89-a5cf-2ecb520deef8" containerName="authorino" containerID="cri-o://cf588350928f3c0784efc06da02596d53f6171c1cc37064d7f5ce442842f0bc6" gracePeriod=30 Jun 02 21:50:04.784226 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:04.784205 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-jd45j" Jun 02 21:50:04.848359 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:04.848327 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5xpx\" (UniqueName: \"kubernetes.io/projected/e1ea0c17-6e76-4f89-a5cf-2ecb520deef8-kube-api-access-z5xpx\") pod \"e1ea0c17-6e76-4f89-a5cf-2ecb520deef8\" (UID: \"e1ea0c17-6e76-4f89-a5cf-2ecb520deef8\") " Jun 02 21:50:04.850449 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:04.850424 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1ea0c17-6e76-4f89-a5cf-2ecb520deef8-kube-api-access-z5xpx" (OuterVolumeSpecName: "kube-api-access-z5xpx") pod "e1ea0c17-6e76-4f89-a5cf-2ecb520deef8" (UID: "e1ea0c17-6e76-4f89-a5cf-2ecb520deef8"). InnerVolumeSpecName "kube-api-access-z5xpx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:50:04.949296 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:04.949254 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5xpx\" (UniqueName: \"kubernetes.io/projected/e1ea0c17-6e76-4f89-a5cf-2ecb520deef8-kube-api-access-z5xpx\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:50:05.547335 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:05.547276 2574 generic.go:358] "Generic (PLEG): container finished" podID="e1ea0c17-6e76-4f89-a5cf-2ecb520deef8" containerID="cf588350928f3c0784efc06da02596d53f6171c1cc37064d7f5ce442842f0bc6" exitCode=0 Jun 02 21:50:05.547753 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:05.547350 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-jd45j" Jun 02 21:50:05.547753 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:05.547358 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-jd45j" event={"ID":"e1ea0c17-6e76-4f89-a5cf-2ecb520deef8","Type":"ContainerDied","Data":"cf588350928f3c0784efc06da02596d53f6171c1cc37064d7f5ce442842f0bc6"} Jun 02 21:50:05.547753 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:05.547402 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-jd45j" event={"ID":"e1ea0c17-6e76-4f89-a5cf-2ecb520deef8","Type":"ContainerDied","Data":"8336215d5db4fc0242d97180c9a1e31d6994f42e73131df9b733261dd8037c42"} Jun 02 21:50:05.547753 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:05.547418 2574 scope.go:117] "RemoveContainer" containerID="cf588350928f3c0784efc06da02596d53f6171c1cc37064d7f5ce442842f0bc6" Jun 02 21:50:05.555216 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:05.555191 2574 scope.go:117] "RemoveContainer" containerID="cf588350928f3c0784efc06da02596d53f6171c1cc37064d7f5ce442842f0bc6" Jun 02 21:50:05.555495 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:50:05.555476 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf588350928f3c0784efc06da02596d53f6171c1cc37064d7f5ce442842f0bc6\": container with ID starting with cf588350928f3c0784efc06da02596d53f6171c1cc37064d7f5ce442842f0bc6 not found: ID does not exist" containerID="cf588350928f3c0784efc06da02596d53f6171c1cc37064d7f5ce442842f0bc6" Jun 02 21:50:05.555545 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:05.555503 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf588350928f3c0784efc06da02596d53f6171c1cc37064d7f5ce442842f0bc6"} err="failed to get container status \"cf588350928f3c0784efc06da02596d53f6171c1cc37064d7f5ce442842f0bc6\": rpc error: code = NotFound desc = could not find container \"cf588350928f3c0784efc06da02596d53f6171c1cc37064d7f5ce442842f0bc6\": container with ID starting with cf588350928f3c0784efc06da02596d53f6171c1cc37064d7f5ce442842f0bc6 not found: ID does not exist" Jun 02 21:50:05.565865 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:05.565843 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-jd45j"] Jun 02 21:50:05.571826 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:05.571806 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-jd45j"] Jun 02 21:50:05.847764 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:05.847736 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1ea0c17-6e76-4f89-a5cf-2ecb520deef8" path="/var/lib/kubelet/pods/e1ea0c17-6e76-4f89-a5cf-2ecb520deef8/volumes" Jun 02 21:50:07.011739 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.011701 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-598b558c6d-sq7dd" podUID="5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76" containerName="console" containerID="cri-o://77550df4dc6956d9127fab66f7d01020362bd5920eef6db9f49318be4f16227b" gracePeriod=15 Jun 02 21:50:07.257758 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.257738 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-598b558c6d-sq7dd_5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76/console/0.log" Jun 02 21:50:07.257868 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.257803 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:50:07.370463 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.370435 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-console-serving-cert\") pod \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " Jun 02 21:50:07.370617 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.370505 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-trusted-ca-bundle\") pod \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " Jun 02 21:50:07.370617 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.370536 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-oauth-serving-cert\") pod \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " Jun 02 21:50:07.370617 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.370569 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-console-oauth-config\") pod \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " Jun 02 21:50:07.370617 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.370606 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-service-ca\") pod \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " Jun 02 21:50:07.370807 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.370640 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbxkg\" (UniqueName: \"kubernetes.io/projected/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-kube-api-access-wbxkg\") pod \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " Jun 02 21:50:07.370807 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.370672 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-console-config\") pod \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\" (UID: \"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76\") " Jun 02 21:50:07.371009 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.370960 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76" (UID: "5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:50:07.371009 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.370986 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-service-ca" (OuterVolumeSpecName: "service-ca") pod "5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76" (UID: "5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:50:07.371277 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.371007 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76" (UID: "5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:50:07.371277 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.371201 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-console-config" (OuterVolumeSpecName: "console-config") pod "5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76" (UID: "5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:50:07.372776 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.372753 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76" (UID: "5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:50:07.372865 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.372798 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-kube-api-access-wbxkg" (OuterVolumeSpecName: "kube-api-access-wbxkg") pod "5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76" (UID: "5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76"). InnerVolumeSpecName "kube-api-access-wbxkg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:50:07.372865 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.372810 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76" (UID: "5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:50:07.472151 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.472122 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-trusted-ca-bundle\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:50:07.472151 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.472147 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-oauth-serving-cert\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:50:07.472151 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.472158 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-console-oauth-config\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:50:07.472373 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.472168 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-service-ca\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:50:07.472373 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.472177 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbxkg\" (UniqueName: \"kubernetes.io/projected/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-kube-api-access-wbxkg\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:50:07.472373 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.472188 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-console-config\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:50:07.472373 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.472197 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76-console-serving-cert\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:50:07.556500 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.556472 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-598b558c6d-sq7dd_5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76/console/0.log" Jun 02 21:50:07.556666 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.556512 2574 generic.go:358] "Generic (PLEG): container finished" podID="5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76" containerID="77550df4dc6956d9127fab66f7d01020362bd5920eef6db9f49318be4f16227b" exitCode=2 Jun 02 21:50:07.556666 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.556545 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-598b558c6d-sq7dd" event={"ID":"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76","Type":"ContainerDied","Data":"77550df4dc6956d9127fab66f7d01020362bd5920eef6db9f49318be4f16227b"} Jun 02 21:50:07.556666 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.556598 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-598b558c6d-sq7dd" event={"ID":"5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76","Type":"ContainerDied","Data":"e204490cc58fa5f6fc1f8d1b43b9a0533971a4dde8f0535a29ed03780d4bec82"} Jun 02 21:50:07.556666 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.556618 2574 scope.go:117] "RemoveContainer" containerID="77550df4dc6956d9127fab66f7d01020362bd5920eef6db9f49318be4f16227b" Jun 02 21:50:07.556666 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.556573 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-598b558c6d-sq7dd" Jun 02 21:50:07.565347 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.565332 2574 scope.go:117] "RemoveContainer" containerID="77550df4dc6956d9127fab66f7d01020362bd5920eef6db9f49318be4f16227b" Jun 02 21:50:07.565592 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:50:07.565573 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77550df4dc6956d9127fab66f7d01020362bd5920eef6db9f49318be4f16227b\": container with ID starting with 77550df4dc6956d9127fab66f7d01020362bd5920eef6db9f49318be4f16227b not found: ID does not exist" containerID="77550df4dc6956d9127fab66f7d01020362bd5920eef6db9f49318be4f16227b" Jun 02 21:50:07.565653 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.565599 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77550df4dc6956d9127fab66f7d01020362bd5920eef6db9f49318be4f16227b"} err="failed to get container status \"77550df4dc6956d9127fab66f7d01020362bd5920eef6db9f49318be4f16227b\": rpc error: code = NotFound desc = could not find container \"77550df4dc6956d9127fab66f7d01020362bd5920eef6db9f49318be4f16227b\": container with ID starting with 77550df4dc6956d9127fab66f7d01020362bd5920eef6db9f49318be4f16227b not found: ID does not exist" Jun 02 21:50:07.581810 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.581785 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-598b558c6d-sq7dd"] Jun 02 21:50:07.585492 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.585473 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-598b558c6d-sq7dd"] Jun 02 21:50:07.852335 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:07.852304 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76" path="/var/lib/kubelet/pods/5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76/volumes" Jun 02 21:50:10.908636 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:10.908601 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-wdtns"] Jun 02 21:50:10.909007 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:10.908823 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-7d549b5b-wdtns" podUID="fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d" containerName="limitador" containerID="cri-o://a10e36363ce2ca0dfb0c21d8fd7adee20fd23363eaf0c9fe36ed40647beb25b8" gracePeriod=30 Jun 02 21:50:10.910839 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:10.910818 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7d549b5b-wdtns" Jun 02 21:50:11.447126 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:11.447103 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-wdtns" Jun 02 21:50:11.502931 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:11.502870 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d-config-file\") pod \"fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d\" (UID: \"fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d\") " Jun 02 21:50:11.502931 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:11.502908 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dkvv\" (UniqueName: \"kubernetes.io/projected/fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d-kube-api-access-5dkvv\") pod \"fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d\" (UID: \"fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d\") " Jun 02 21:50:11.503268 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:11.503239 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d-config-file" (OuterVolumeSpecName: "config-file") pod "fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d" (UID: "fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:50:11.504971 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:11.504951 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d-kube-api-access-5dkvv" (OuterVolumeSpecName: "kube-api-access-5dkvv") pod "fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d" (UID: "fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d"). InnerVolumeSpecName "kube-api-access-5dkvv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:50:11.570204 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:11.570171 2574 generic.go:358] "Generic (PLEG): container finished" podID="fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d" containerID="a10e36363ce2ca0dfb0c21d8fd7adee20fd23363eaf0c9fe36ed40647beb25b8" exitCode=0 Jun 02 21:50:11.570339 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:11.570238 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-wdtns" Jun 02 21:50:11.570339 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:11.570253 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-wdtns" event={"ID":"fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d","Type":"ContainerDied","Data":"a10e36363ce2ca0dfb0c21d8fd7adee20fd23363eaf0c9fe36ed40647beb25b8"} Jun 02 21:50:11.570339 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:11.570300 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-wdtns" event={"ID":"fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d","Type":"ContainerDied","Data":"06e76232a3741a77f040cdcdb5db473121f8467d16586c3b0be4916f59198699"} Jun 02 21:50:11.570339 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:11.570317 2574 scope.go:117] "RemoveContainer" containerID="a10e36363ce2ca0dfb0c21d8fd7adee20fd23363eaf0c9fe36ed40647beb25b8" Jun 02 21:50:11.577875 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:11.577857 2574 scope.go:117] "RemoveContainer" containerID="a10e36363ce2ca0dfb0c21d8fd7adee20fd23363eaf0c9fe36ed40647beb25b8" Jun 02 21:50:11.578118 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:50:11.578098 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a10e36363ce2ca0dfb0c21d8fd7adee20fd23363eaf0c9fe36ed40647beb25b8\": container with ID starting with a10e36363ce2ca0dfb0c21d8fd7adee20fd23363eaf0c9fe36ed40647beb25b8 not found: ID does not exist" containerID="a10e36363ce2ca0dfb0c21d8fd7adee20fd23363eaf0c9fe36ed40647beb25b8" Jun 02 21:50:11.578167 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:11.578125 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a10e36363ce2ca0dfb0c21d8fd7adee20fd23363eaf0c9fe36ed40647beb25b8"} err="failed to get container status \"a10e36363ce2ca0dfb0c21d8fd7adee20fd23363eaf0c9fe36ed40647beb25b8\": rpc error: code = NotFound desc = could not find container \"a10e36363ce2ca0dfb0c21d8fd7adee20fd23363eaf0c9fe36ed40647beb25b8\": container with ID starting with a10e36363ce2ca0dfb0c21d8fd7adee20fd23363eaf0c9fe36ed40647beb25b8 not found: ID does not exist" Jun 02 21:50:11.592181 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:11.592154 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-wdtns"] Jun 02 21:50:11.595831 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:11.595812 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-wdtns"] Jun 02 21:50:11.603602 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:11.603584 2574 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d-config-file\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:50:11.603602 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:11.603604 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dkvv\" (UniqueName: \"kubernetes.io/projected/fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d-kube-api-access-5dkvv\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:50:11.847229 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:11.847203 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d" path="/var/lib/kubelet/pods/fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d/volumes" Jun 02 21:50:16.844220 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:16.844188 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-68c4fbbd6f-bnlzp"] Jun 02 21:50:16.844811 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:16.844537 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76" containerName="console" Jun 02 21:50:16.844811 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:16.844550 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76" containerName="console" Jun 02 21:50:16.844811 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:16.844561 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d" containerName="limitador" Jun 02 21:50:16.844811 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:16.844566 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d" containerName="limitador" Jun 02 21:50:16.844811 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:16.844585 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1ea0c17-6e76-4f89-a5cf-2ecb520deef8" containerName="authorino" Jun 02 21:50:16.844811 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:16.844590 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1ea0c17-6e76-4f89-a5cf-2ecb520deef8" containerName="authorino" Jun 02 21:50:16.844811 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:16.844643 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="e1ea0c17-6e76-4f89-a5cf-2ecb520deef8" containerName="authorino" Jun 02 21:50:16.844811 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:16.844651 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="5ffd1aa7-0960-4dde-b635-f3fdcc0fbc76" containerName="console" Jun 02 21:50:16.844811 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:16.844657 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="fc0ee09d-5a0e-46f2-a9b4-0bb34f79183d" containerName="limitador" Jun 02 21:50:16.847638 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:16.847617 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-bnlzp" Jun 02 21:50:16.849825 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:16.849807 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jun 02 21:50:16.849925 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:16.849864 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tpp75\"" Jun 02 21:50:16.858134 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:16.858114 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-bnlzp"] Jun 02 21:50:16.943260 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:16.943229 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtjxp\" (UniqueName: \"kubernetes.io/projected/990a6c8a-69d1-4ed4-b867-9730ba10572b-kube-api-access-mtjxp\") pod \"postgres-68c4fbbd6f-bnlzp\" (UID: \"990a6c8a-69d1-4ed4-b867-9730ba10572b\") " pod="opendatahub/postgres-68c4fbbd6f-bnlzp" Jun 02 21:50:16.943409 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:16.943303 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/990a6c8a-69d1-4ed4-b867-9730ba10572b-data\") pod \"postgres-68c4fbbd6f-bnlzp\" (UID: \"990a6c8a-69d1-4ed4-b867-9730ba10572b\") " pod="opendatahub/postgres-68c4fbbd6f-bnlzp" Jun 02 21:50:17.044694 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:17.044667 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/990a6c8a-69d1-4ed4-b867-9730ba10572b-data\") pod \"postgres-68c4fbbd6f-bnlzp\" (UID: \"990a6c8a-69d1-4ed4-b867-9730ba10572b\") " pod="opendatahub/postgres-68c4fbbd6f-bnlzp" Jun 02 21:50:17.044839 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:17.044734 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mtjxp\" (UniqueName: \"kubernetes.io/projected/990a6c8a-69d1-4ed4-b867-9730ba10572b-kube-api-access-mtjxp\") pod \"postgres-68c4fbbd6f-bnlzp\" (UID: \"990a6c8a-69d1-4ed4-b867-9730ba10572b\") " pod="opendatahub/postgres-68c4fbbd6f-bnlzp" Jun 02 21:50:17.045102 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:17.045078 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/990a6c8a-69d1-4ed4-b867-9730ba10572b-data\") pod \"postgres-68c4fbbd6f-bnlzp\" (UID: \"990a6c8a-69d1-4ed4-b867-9730ba10572b\") " pod="opendatahub/postgres-68c4fbbd6f-bnlzp" Jun 02 21:50:17.052273 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:17.052244 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtjxp\" (UniqueName: \"kubernetes.io/projected/990a6c8a-69d1-4ed4-b867-9730ba10572b-kube-api-access-mtjxp\") pod \"postgres-68c4fbbd6f-bnlzp\" (UID: \"990a6c8a-69d1-4ed4-b867-9730ba10572b\") " pod="opendatahub/postgres-68c4fbbd6f-bnlzp" Jun 02 21:50:17.158698 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:17.158615 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-bnlzp" Jun 02 21:50:17.284489 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:17.284461 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-bnlzp"] Jun 02 21:50:17.286889 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:50:17.286856 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod990a6c8a_69d1_4ed4_b867_9730ba10572b.slice/crio-34213421c7d7d3ff28dcba99630b7b776680fc963e1cfd78a738c915434f0adf WatchSource:0}: Error finding container 34213421c7d7d3ff28dcba99630b7b776680fc963e1cfd78a738c915434f0adf: Status 404 returned error can't find the container with id 34213421c7d7d3ff28dcba99630b7b776680fc963e1cfd78a738c915434f0adf Jun 02 21:50:17.590634 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:17.590603 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-bnlzp" event={"ID":"990a6c8a-69d1-4ed4-b867-9730ba10572b","Type":"ContainerStarted","Data":"34213421c7d7d3ff28dcba99630b7b776680fc963e1cfd78a738c915434f0adf"} Jun 02 21:50:19.599194 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:19.599164 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-bnlzp" event={"ID":"990a6c8a-69d1-4ed4-b867-9730ba10572b","Type":"ContainerStarted","Data":"fb952a76a52417ba0e0c48c27e54394ab9c77c888fb4d665afa820feb12d3815"} Jun 02 21:50:19.599602 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:19.599232 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-68c4fbbd6f-bnlzp" Jun 02 21:50:19.615261 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:19.615221 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/postgres-68c4fbbd6f-bnlzp" podStartSLOduration=1.5292815819999999 podStartE2EDuration="3.615209201s" podCreationTimestamp="2026-06-02 21:50:16 +0000 UTC" firstStartedPulling="2026-06-02 21:50:17.288504656 +0000 UTC m=+501.993514581" lastFinishedPulling="2026-06-02 21:50:19.374432261 +0000 UTC m=+504.079442200" observedRunningTime="2026-06-02 21:50:19.613823771 +0000 UTC m=+504.318833731" watchObservedRunningTime="2026-06-02 21:50:19.615209201 +0000 UTC m=+504.320219195" Jun 02 21:50:25.634651 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:25.634611 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-68c4fbbd6f-bnlzp" Jun 02 21:50:48.770271 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:48.770184 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:50:48.773391 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:48.773373 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 21:50:48.775491 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:48.775470 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 02 21:50:48.776012 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:48.775991 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 02 21:50:48.776012 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:48.776002 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-l4f8s\"" Jun 02 21:50:48.776163 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:48.775993 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 02 21:50:48.783999 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:48.783977 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:50:48.878293 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:48.878253 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x645r\" (UniqueName: \"kubernetes.io/projected/881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11-kube-api-access-x645r\") pod \"maas-keycloak-0\" (UID: \"881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:50:48.979462 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:48.979434 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x645r\" (UniqueName: \"kubernetes.io/projected/881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11-kube-api-access-x645r\") pod \"maas-keycloak-0\" (UID: \"881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:50:48.986758 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:48.986740 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x645r\" (UniqueName: \"kubernetes.io/projected/881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11-kube-api-access-x645r\") pod \"maas-keycloak-0\" (UID: \"881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:50:49.083719 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:49.083702 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 21:50:49.207551 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:49.207473 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:50:49.209956 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:50:49.209927 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod881c6ceb_3be6_40b7_9ea8_8cc7a16f9e11.slice/crio-3de89350febe8a4d3349c76967fed633a8028f925bd44ffa198bc797e6dc1550 WatchSource:0}: Error finding container 3de89350febe8a4d3349c76967fed633a8028f925bd44ffa198bc797e6dc1550: Status 404 returned error can't find the container with id 3de89350febe8a4d3349c76967fed633a8028f925bd44ffa198bc797e6dc1550 Jun 02 21:50:49.702797 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:49.702758 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11","Type":"ContainerStarted","Data":"3de89350febe8a4d3349c76967fed633a8028f925bd44ffa198bc797e6dc1550"} Jun 02 21:50:54.722796 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:54.722761 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11","Type":"ContainerStarted","Data":"ee2e9c6849b9a2132dac9e1c37b7618a9e2f5828ebc7fde4ce13330b513fcb1a"} Jun 02 21:50:54.739408 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:54.739360 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.8939219349999998 podStartE2EDuration="6.739345481s" podCreationTimestamp="2026-06-02 21:50:48 +0000 UTC" firstStartedPulling="2026-06-02 21:50:49.211195145 +0000 UTC m=+533.916205071" lastFinishedPulling="2026-06-02 21:50:54.056618686 +0000 UTC m=+538.761628617" observedRunningTime="2026-06-02 21:50:54.738145528 +0000 UTC m=+539.443155476" watchObservedRunningTime="2026-06-02 21:50:54.739345481 +0000 UTC m=+539.444355429" Jun 02 21:50:55.084761 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:55.084716 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 02 21:50:55.086796 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:55.086758 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.30:9000/health/started\": dial tcp 10.134.0.30:9000: connect: connection refused" Jun 02 21:50:56.084879 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:56.084847 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.30:9000/health/started\": dial tcp 10.134.0.30:9000: connect: connection refused" Jun 02 21:50:57.085138 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:57.085099 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.30:9000/health/started\": dial tcp 10.134.0.30:9000: connect: connection refused" Jun 02 21:50:58.084702 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:58.084654 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.30:9000/health/started\": dial tcp 10.134.0.30:9000: connect: connection refused" Jun 02 21:50:59.084552 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:59.084498 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.30:9000/health/started\": dial tcp 10.134.0.30:9000: connect: connection refused" Jun 02 21:50:59.084998 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:50:59.084936 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 02 21:51:00.085091 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:00.085037 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.30:9000/health/started\": dial tcp 10.134.0.30:9000: connect: connection refused" Jun 02 21:51:01.084324 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:01.084251 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.30:9000/health/started\": dial tcp 10.134.0.30:9000: connect: connection refused" Jun 02 21:51:02.084399 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:02.084338 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.30:9000/health/started\": dial tcp 10.134.0.30:9000: connect: connection refused" Jun 02 21:51:03.084879 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:03.084840 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.30:9000/health/started\": dial tcp 10.134.0.30:9000: connect: connection refused" Jun 02 21:51:04.084038 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:04.084001 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.30:9000/health/started\": dial tcp 10.134.0.30:9000: connect: connection refused" Jun 02 21:51:05.084526 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:05.084466 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.30:9000/health/started\": dial tcp 10.134.0.30:9000: connect: connection refused" Jun 02 21:51:06.084206 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:06.084167 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.30:9000/health/started\": dial tcp 10.134.0.30:9000: connect: connection refused" Jun 02 21:51:07.213255 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:07.213223 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 02 21:51:07.232085 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:07.231983 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 21:51:17.219993 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:17.219957 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 02 21:51:18.144463 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.144426 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8b475cf9f-bp7rh"] Jun 02 21:51:18.147693 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.147672 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-bp7rh" Jun 02 21:51:18.154889 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.154870 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-bp7rh"] Jun 02 21:51:18.250503 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.250477 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd5vb\" (UniqueName: \"kubernetes.io/projected/6663145c-6310-49be-826c-7c6cdbc8ea74-kube-api-access-zd5vb\") pod \"authorino-8b475cf9f-bp7rh\" (UID: \"6663145c-6310-49be-826c-7c6cdbc8ea74\") " pod="kuadrant-system/authorino-8b475cf9f-bp7rh" Jun 02 21:51:18.351064 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.351026 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zd5vb\" (UniqueName: \"kubernetes.io/projected/6663145c-6310-49be-826c-7c6cdbc8ea74-kube-api-access-zd5vb\") pod \"authorino-8b475cf9f-bp7rh\" (UID: \"6663145c-6310-49be-826c-7c6cdbc8ea74\") " pod="kuadrant-system/authorino-8b475cf9f-bp7rh" Jun 02 21:51:18.358525 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.358503 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd5vb\" (UniqueName: \"kubernetes.io/projected/6663145c-6310-49be-826c-7c6cdbc8ea74-kube-api-access-zd5vb\") pod \"authorino-8b475cf9f-bp7rh\" (UID: \"6663145c-6310-49be-826c-7c6cdbc8ea74\") " pod="kuadrant-system/authorino-8b475cf9f-bp7rh" Jun 02 21:51:18.386918 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.386893 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-bp7rh"] Jun 02 21:51:18.387058 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.387048 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-bp7rh" Jun 02 21:51:18.412661 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.412641 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-55658b4dd6-fz94k"] Jun 02 21:51:18.415534 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.415517 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55658b4dd6-fz94k" Jun 02 21:51:18.420556 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.420537 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55658b4dd6-fz94k"] Jun 02 21:51:18.491405 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.491376 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-55658b4dd6-fz94k"] Jun 02 21:51:18.491632 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:51:18.491612 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-jqh54], unattached volumes=[], failed to process volumes=[kube-api-access-jqh54]: context canceled" pod="kuadrant-system/authorino-55658b4dd6-fz94k" podUID="b8314b20-5f03-45f8-8541-f87bddb2a4f6" Jun 02 21:51:18.510351 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.510326 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-bp7rh"] Jun 02 21:51:18.512828 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:51:18.512803 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6663145c_6310_49be_826c_7c6cdbc8ea74.slice/crio-d07d3d47284e1fadb8118140c988cf41001838fb454e226ea636bc3910c89272 WatchSource:0}: Error finding container d07d3d47284e1fadb8118140c988cf41001838fb454e226ea636bc3910c89272: Status 404 returned error can't find the container with id d07d3d47284e1fadb8118140c988cf41001838fb454e226ea636bc3910c89272 Jun 02 21:51:18.517194 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.517172 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-d6b7df54-xm5l4"] Jun 02 21:51:18.519460 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.519438 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d6b7df54-xm5l4" Jun 02 21:51:18.521509 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.521491 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 02 21:51:18.526060 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.526036 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d6b7df54-xm5l4"] Jun 02 21:51:18.552024 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.551996 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqh54\" (UniqueName: \"kubernetes.io/projected/b8314b20-5f03-45f8-8541-f87bddb2a4f6-kube-api-access-jqh54\") pod \"authorino-55658b4dd6-fz94k\" (UID: \"b8314b20-5f03-45f8-8541-f87bddb2a4f6\") " pod="kuadrant-system/authorino-55658b4dd6-fz94k" Jun 02 21:51:18.652429 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.652367 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d3fdd836-217f-434a-ac83-e24b56ccb233-tls-cert\") pod \"authorino-d6b7df54-xm5l4\" (UID: \"d3fdd836-217f-434a-ac83-e24b56ccb233\") " pod="kuadrant-system/authorino-d6b7df54-xm5l4" Jun 02 21:51:18.652527 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.652447 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lg6h\" (UniqueName: \"kubernetes.io/projected/d3fdd836-217f-434a-ac83-e24b56ccb233-kube-api-access-6lg6h\") pod \"authorino-d6b7df54-xm5l4\" (UID: \"d3fdd836-217f-434a-ac83-e24b56ccb233\") " pod="kuadrant-system/authorino-d6b7df54-xm5l4" Jun 02 21:51:18.652527 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.652479 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jqh54\" (UniqueName: \"kubernetes.io/projected/b8314b20-5f03-45f8-8541-f87bddb2a4f6-kube-api-access-jqh54\") pod \"authorino-55658b4dd6-fz94k\" (UID: \"b8314b20-5f03-45f8-8541-f87bddb2a4f6\") " pod="kuadrant-system/authorino-55658b4dd6-fz94k" Jun 02 21:51:18.659739 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.659720 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqh54\" (UniqueName: \"kubernetes.io/projected/b8314b20-5f03-45f8-8541-f87bddb2a4f6-kube-api-access-jqh54\") pod \"authorino-55658b4dd6-fz94k\" (UID: \"b8314b20-5f03-45f8-8541-f87bddb2a4f6\") " pod="kuadrant-system/authorino-55658b4dd6-fz94k" Jun 02 21:51:18.753681 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.753657 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d3fdd836-217f-434a-ac83-e24b56ccb233-tls-cert\") pod \"authorino-d6b7df54-xm5l4\" (UID: \"d3fdd836-217f-434a-ac83-e24b56ccb233\") " pod="kuadrant-system/authorino-d6b7df54-xm5l4" Jun 02 21:51:18.753769 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.753716 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6lg6h\" (UniqueName: \"kubernetes.io/projected/d3fdd836-217f-434a-ac83-e24b56ccb233-kube-api-access-6lg6h\") pod \"authorino-d6b7df54-xm5l4\" (UID: \"d3fdd836-217f-434a-ac83-e24b56ccb233\") " pod="kuadrant-system/authorino-d6b7df54-xm5l4" Jun 02 21:51:18.755887 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.755870 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d3fdd836-217f-434a-ac83-e24b56ccb233-tls-cert\") pod \"authorino-d6b7df54-xm5l4\" (UID: \"d3fdd836-217f-434a-ac83-e24b56ccb233\") " pod="kuadrant-system/authorino-d6b7df54-xm5l4" Jun 02 21:51:18.760704 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.760687 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lg6h\" (UniqueName: \"kubernetes.io/projected/d3fdd836-217f-434a-ac83-e24b56ccb233-kube-api-access-6lg6h\") pod \"authorino-d6b7df54-xm5l4\" (UID: \"d3fdd836-217f-434a-ac83-e24b56ccb233\") " pod="kuadrant-system/authorino-d6b7df54-xm5l4" Jun 02 21:51:18.817810 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.817788 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55658b4dd6-fz94k" Jun 02 21:51:18.817810 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.817798 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-bp7rh" event={"ID":"6663145c-6310-49be-826c-7c6cdbc8ea74","Type":"ContainerStarted","Data":"d07d3d47284e1fadb8118140c988cf41001838fb454e226ea636bc3910c89272"} Jun 02 21:51:18.822656 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.822640 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55658b4dd6-fz94k" Jun 02 21:51:18.829246 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.829231 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d6b7df54-xm5l4" Jun 02 21:51:18.952807 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.952781 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d6b7df54-xm5l4"] Jun 02 21:51:18.954883 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.954860 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqh54\" (UniqueName: \"kubernetes.io/projected/b8314b20-5f03-45f8-8541-f87bddb2a4f6-kube-api-access-jqh54\") pod \"b8314b20-5f03-45f8-8541-f87bddb2a4f6\" (UID: \"b8314b20-5f03-45f8-8541-f87bddb2a4f6\") " Jun 02 21:51:18.955820 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:51:18.955793 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3fdd836_217f_434a_ac83_e24b56ccb233.slice/crio-a6dba483e9b70055acf426ed1be2380b26536254934b9bf64f64daeead2e5cad WatchSource:0}: Error finding container a6dba483e9b70055acf426ed1be2380b26536254934b9bf64f64daeead2e5cad: Status 404 returned error can't find the container with id a6dba483e9b70055acf426ed1be2380b26536254934b9bf64f64daeead2e5cad Jun 02 21:51:18.956853 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:18.956829 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8314b20-5f03-45f8-8541-f87bddb2a4f6-kube-api-access-jqh54" (OuterVolumeSpecName: "kube-api-access-jqh54") pod "b8314b20-5f03-45f8-8541-f87bddb2a4f6" (UID: "b8314b20-5f03-45f8-8541-f87bddb2a4f6"). InnerVolumeSpecName "kube-api-access-jqh54". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:51:19.056547 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:19.056515 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqh54\" (UniqueName: \"kubernetes.io/projected/b8314b20-5f03-45f8-8541-f87bddb2a4f6-kube-api-access-jqh54\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:51:19.821751 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:19.821717 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-bp7rh" event={"ID":"6663145c-6310-49be-826c-7c6cdbc8ea74","Type":"ContainerStarted","Data":"07782c34613a382f1ca727b6ff632282543a2b12f04c15e8fb639a650f31df6e"} Jun 02 21:51:19.822332 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:19.821757 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-8b475cf9f-bp7rh" podUID="6663145c-6310-49be-826c-7c6cdbc8ea74" containerName="authorino" containerID="cri-o://07782c34613a382f1ca727b6ff632282543a2b12f04c15e8fb639a650f31df6e" gracePeriod=30 Jun 02 21:51:19.823156 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:19.823129 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d6b7df54-xm5l4" event={"ID":"d3fdd836-217f-434a-ac83-e24b56ccb233","Type":"ContainerStarted","Data":"a9a8f4f6cfcf9df496f9932930cbd9c9126fecb0729e4cdd1ffac64ba94f4d06"} Jun 02 21:51:19.823253 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:19.823159 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d6b7df54-xm5l4" event={"ID":"d3fdd836-217f-434a-ac83-e24b56ccb233","Type":"ContainerStarted","Data":"a6dba483e9b70055acf426ed1be2380b26536254934b9bf64f64daeead2e5cad"} Jun 02 21:51:19.823253 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:19.823182 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55658b4dd6-fz94k" Jun 02 21:51:19.835487 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:19.835444 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-8b475cf9f-bp7rh" podStartSLOduration=1.449942443 podStartE2EDuration="1.835430113s" podCreationTimestamp="2026-06-02 21:51:18 +0000 UTC" firstStartedPulling="2026-06-02 21:51:18.514135171 +0000 UTC m=+563.219145096" lastFinishedPulling="2026-06-02 21:51:18.899622832 +0000 UTC m=+563.604632766" observedRunningTime="2026-06-02 21:51:19.834642444 +0000 UTC m=+564.539652402" watchObservedRunningTime="2026-06-02 21:51:19.835430113 +0000 UTC m=+564.540440061" Jun 02 21:51:19.853357 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:19.853316 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-d6b7df54-xm5l4" podStartSLOduration=1.614446666 podStartE2EDuration="1.853276643s" podCreationTimestamp="2026-06-02 21:51:18 +0000 UTC" firstStartedPulling="2026-06-02 21:51:18.957270084 +0000 UTC m=+563.662280009" lastFinishedPulling="2026-06-02 21:51:19.196100043 +0000 UTC m=+563.901109986" observedRunningTime="2026-06-02 21:51:19.851864101 +0000 UTC m=+564.556874047" watchObservedRunningTime="2026-06-02 21:51:19.853276643 +0000 UTC m=+564.558286613" Jun 02 21:51:19.876173 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:19.876148 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-js2cw"] Jun 02 21:51:19.876371 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:19.876352 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-7498df8756-js2cw" podUID="8c176dda-32bd-490f-99cb-4bf3a81a164c" containerName="authorino" containerID="cri-o://ef2057d75888bf6f615a29f757946d277801207f9aa31b626b01894b66f457fe" gracePeriod=30 Jun 02 21:51:20.131961 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.131938 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-bp7rh" Jun 02 21:51:20.135271 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.135253 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-js2cw" Jun 02 21:51:20.266551 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.266520 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zd5vb\" (UniqueName: \"kubernetes.io/projected/6663145c-6310-49be-826c-7c6cdbc8ea74-kube-api-access-zd5vb\") pod \"6663145c-6310-49be-826c-7c6cdbc8ea74\" (UID: \"6663145c-6310-49be-826c-7c6cdbc8ea74\") " Jun 02 21:51:20.266691 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.266564 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9cvd\" (UniqueName: \"kubernetes.io/projected/8c176dda-32bd-490f-99cb-4bf3a81a164c-kube-api-access-s9cvd\") pod \"8c176dda-32bd-490f-99cb-4bf3a81a164c\" (UID: \"8c176dda-32bd-490f-99cb-4bf3a81a164c\") " Jun 02 21:51:20.268539 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.268509 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c176dda-32bd-490f-99cb-4bf3a81a164c-kube-api-access-s9cvd" (OuterVolumeSpecName: "kube-api-access-s9cvd") pod "8c176dda-32bd-490f-99cb-4bf3a81a164c" (UID: "8c176dda-32bd-490f-99cb-4bf3a81a164c"). InnerVolumeSpecName "kube-api-access-s9cvd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:51:20.268639 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.268559 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6663145c-6310-49be-826c-7c6cdbc8ea74-kube-api-access-zd5vb" (OuterVolumeSpecName: "kube-api-access-zd5vb") pod "6663145c-6310-49be-826c-7c6cdbc8ea74" (UID: "6663145c-6310-49be-826c-7c6cdbc8ea74"). InnerVolumeSpecName "kube-api-access-zd5vb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:51:20.367001 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.366980 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zd5vb\" (UniqueName: \"kubernetes.io/projected/6663145c-6310-49be-826c-7c6cdbc8ea74-kube-api-access-zd5vb\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:51:20.367072 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.367003 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9cvd\" (UniqueName: \"kubernetes.io/projected/8c176dda-32bd-490f-99cb-4bf3a81a164c-kube-api-access-s9cvd\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:51:20.827558 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.827481 2574 generic.go:358] "Generic (PLEG): container finished" podID="8c176dda-32bd-490f-99cb-4bf3a81a164c" containerID="ef2057d75888bf6f615a29f757946d277801207f9aa31b626b01894b66f457fe" exitCode=0 Jun 02 21:51:20.827558 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.827529 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-js2cw" Jun 02 21:51:20.827992 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.827570 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-js2cw" event={"ID":"8c176dda-32bd-490f-99cb-4bf3a81a164c","Type":"ContainerDied","Data":"ef2057d75888bf6f615a29f757946d277801207f9aa31b626b01894b66f457fe"} Jun 02 21:51:20.827992 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.827612 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-js2cw" event={"ID":"8c176dda-32bd-490f-99cb-4bf3a81a164c","Type":"ContainerDied","Data":"0a771760101268e9c4743e6c637bcad929d8f89dde517115c78a217b8729fac8"} Jun 02 21:51:20.827992 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.827636 2574 scope.go:117] "RemoveContainer" containerID="ef2057d75888bf6f615a29f757946d277801207f9aa31b626b01894b66f457fe" Jun 02 21:51:20.828855 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.828750 2574 generic.go:358] "Generic (PLEG): container finished" podID="6663145c-6310-49be-826c-7c6cdbc8ea74" containerID="07782c34613a382f1ca727b6ff632282543a2b12f04c15e8fb639a650f31df6e" exitCode=0 Jun 02 21:51:20.828855 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.828792 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-bp7rh" Jun 02 21:51:20.828855 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.828818 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-bp7rh" event={"ID":"6663145c-6310-49be-826c-7c6cdbc8ea74","Type":"ContainerDied","Data":"07782c34613a382f1ca727b6ff632282543a2b12f04c15e8fb639a650f31df6e"} Jun 02 21:51:20.828855 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.828838 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-bp7rh" event={"ID":"6663145c-6310-49be-826c-7c6cdbc8ea74","Type":"ContainerDied","Data":"d07d3d47284e1fadb8118140c988cf41001838fb454e226ea636bc3910c89272"} Jun 02 21:51:20.837271 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.837256 2574 scope.go:117] "RemoveContainer" containerID="ef2057d75888bf6f615a29f757946d277801207f9aa31b626b01894b66f457fe" Jun 02 21:51:20.837546 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:51:20.837528 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef2057d75888bf6f615a29f757946d277801207f9aa31b626b01894b66f457fe\": container with ID starting with ef2057d75888bf6f615a29f757946d277801207f9aa31b626b01894b66f457fe not found: ID does not exist" containerID="ef2057d75888bf6f615a29f757946d277801207f9aa31b626b01894b66f457fe" Jun 02 21:51:20.837632 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.837552 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef2057d75888bf6f615a29f757946d277801207f9aa31b626b01894b66f457fe"} err="failed to get container status \"ef2057d75888bf6f615a29f757946d277801207f9aa31b626b01894b66f457fe\": rpc error: code = NotFound desc = could not find container \"ef2057d75888bf6f615a29f757946d277801207f9aa31b626b01894b66f457fe\": container with ID starting with ef2057d75888bf6f615a29f757946d277801207f9aa31b626b01894b66f457fe not found: ID does not exist" Jun 02 21:51:20.837632 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.837568 2574 scope.go:117] "RemoveContainer" containerID="07782c34613a382f1ca727b6ff632282543a2b12f04c15e8fb639a650f31df6e" Jun 02 21:51:20.844318 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.844304 2574 scope.go:117] "RemoveContainer" containerID="07782c34613a382f1ca727b6ff632282543a2b12f04c15e8fb639a650f31df6e" Jun 02 21:51:20.844538 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:51:20.844521 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07782c34613a382f1ca727b6ff632282543a2b12f04c15e8fb639a650f31df6e\": container with ID starting with 07782c34613a382f1ca727b6ff632282543a2b12f04c15e8fb639a650f31df6e not found: ID does not exist" containerID="07782c34613a382f1ca727b6ff632282543a2b12f04c15e8fb639a650f31df6e" Jun 02 21:51:20.844588 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.844544 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07782c34613a382f1ca727b6ff632282543a2b12f04c15e8fb639a650f31df6e"} err="failed to get container status \"07782c34613a382f1ca727b6ff632282543a2b12f04c15e8fb639a650f31df6e\": rpc error: code = NotFound desc = could not find container \"07782c34613a382f1ca727b6ff632282543a2b12f04c15e8fb639a650f31df6e\": container with ID starting with 07782c34613a382f1ca727b6ff632282543a2b12f04c15e8fb639a650f31df6e not found: ID does not exist" Jun 02 21:51:20.848086 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.848064 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-js2cw"] Jun 02 21:51:20.851030 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.851011 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7498df8756-js2cw"] Jun 02 21:51:20.862679 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.862655 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-bp7rh"] Jun 02 21:51:20.865885 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:20.865868 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-bp7rh"] Jun 02 21:51:21.847846 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:21.847812 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6663145c-6310-49be-826c-7c6cdbc8ea74" path="/var/lib/kubelet/pods/6663145c-6310-49be-826c-7c6cdbc8ea74/volumes" Jun 02 21:51:21.848197 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:21.848106 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c176dda-32bd-490f-99cb-4bf3a81a164c" path="/var/lib/kubelet/pods/8c176dda-32bd-490f-99cb-4bf3a81a164c/volumes" Jun 02 21:51:23.342719 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:23.342691 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5bf74477bf-vjths"] Jun 02 21:51:23.343049 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:23.343017 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c176dda-32bd-490f-99cb-4bf3a81a164c" containerName="authorino" Jun 02 21:51:23.343049 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:23.343028 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c176dda-32bd-490f-99cb-4bf3a81a164c" containerName="authorino" Jun 02 21:51:23.343049 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:23.343036 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6663145c-6310-49be-826c-7c6cdbc8ea74" containerName="authorino" Jun 02 21:51:23.343049 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:23.343041 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6663145c-6310-49be-826c-7c6cdbc8ea74" containerName="authorino" Jun 02 21:51:23.343181 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:23.343090 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="6663145c-6310-49be-826c-7c6cdbc8ea74" containerName="authorino" Jun 02 21:51:23.343181 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:23.343100 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="8c176dda-32bd-490f-99cb-4bf3a81a164c" containerName="authorino" Jun 02 21:51:23.346085 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:23.346068 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bf74477bf-vjths" Jun 02 21:51:23.348189 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:23.348170 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 02 21:51:23.348311 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:23.348178 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rlljd\"" Jun 02 21:51:23.348311 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:23.348216 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 02 21:51:23.353149 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:23.353121 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bf74477bf-vjths"] Jun 02 21:51:23.487521 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:23.487498 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5edf47d-91e5-4104-8377-b0d06b1e6cd3-webhook-cert\") pod \"maas-controller-5bf74477bf-vjths\" (UID: \"e5edf47d-91e5-4104-8377-b0d06b1e6cd3\") " pod="opendatahub/maas-controller-5bf74477bf-vjths" Jun 02 21:51:23.487648 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:23.487547 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28pc2\" (UniqueName: \"kubernetes.io/projected/e5edf47d-91e5-4104-8377-b0d06b1e6cd3-kube-api-access-28pc2\") pod \"maas-controller-5bf74477bf-vjths\" (UID: \"e5edf47d-91e5-4104-8377-b0d06b1e6cd3\") " pod="opendatahub/maas-controller-5bf74477bf-vjths" Jun 02 21:51:23.588172 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:23.588145 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5edf47d-91e5-4104-8377-b0d06b1e6cd3-webhook-cert\") pod \"maas-controller-5bf74477bf-vjths\" (UID: \"e5edf47d-91e5-4104-8377-b0d06b1e6cd3\") " pod="opendatahub/maas-controller-5bf74477bf-vjths" Jun 02 21:51:23.588312 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:23.588196 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-28pc2\" (UniqueName: \"kubernetes.io/projected/e5edf47d-91e5-4104-8377-b0d06b1e6cd3-kube-api-access-28pc2\") pod \"maas-controller-5bf74477bf-vjths\" (UID: \"e5edf47d-91e5-4104-8377-b0d06b1e6cd3\") " pod="opendatahub/maas-controller-5bf74477bf-vjths" Jun 02 21:51:23.590480 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:23.590458 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5edf47d-91e5-4104-8377-b0d06b1e6cd3-webhook-cert\") pod \"maas-controller-5bf74477bf-vjths\" (UID: \"e5edf47d-91e5-4104-8377-b0d06b1e6cd3\") " pod="opendatahub/maas-controller-5bf74477bf-vjths" Jun 02 21:51:23.595207 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:23.595165 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28pc2\" (UniqueName: \"kubernetes.io/projected/e5edf47d-91e5-4104-8377-b0d06b1e6cd3-kube-api-access-28pc2\") pod \"maas-controller-5bf74477bf-vjths\" (UID: \"e5edf47d-91e5-4104-8377-b0d06b1e6cd3\") " pod="opendatahub/maas-controller-5bf74477bf-vjths" Jun 02 21:51:23.656910 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:23.656891 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bf74477bf-vjths" Jun 02 21:51:23.779718 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:23.779691 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bf74477bf-vjths"] Jun 02 21:51:23.781362 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:51:23.781334 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5edf47d_91e5_4104_8377_b0d06b1e6cd3.slice/crio-7a4cdec0ece90ec0ea936e8fcd0b1a7c796003422093a973f1696f27e9748bb9 WatchSource:0}: Error finding container 7a4cdec0ece90ec0ea936e8fcd0b1a7c796003422093a973f1696f27e9748bb9: Status 404 returned error can't find the container with id 7a4cdec0ece90ec0ea936e8fcd0b1a7c796003422093a973f1696f27e9748bb9 Jun 02 21:51:23.840316 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:23.840269 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bf74477bf-vjths" event={"ID":"e5edf47d-91e5-4104-8377-b0d06b1e6cd3","Type":"ContainerStarted","Data":"7a4cdec0ece90ec0ea936e8fcd0b1a7c796003422093a973f1696f27e9748bb9"} Jun 02 21:51:26.853930 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:26.853896 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bf74477bf-vjths" event={"ID":"e5edf47d-91e5-4104-8377-b0d06b1e6cd3","Type":"ContainerStarted","Data":"319c3b8718376b1118ba89686cf92073ee7ccf6af6f9a1cfa8d995815532ef53"} Jun 02 21:51:26.854319 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:26.854014 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5bf74477bf-vjths" Jun 02 21:51:26.873249 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:26.873205 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-5bf74477bf-vjths" podStartSLOduration=1.6545993509999999 podStartE2EDuration="3.873193726s" podCreationTimestamp="2026-06-02 21:51:23 +0000 UTC" firstStartedPulling="2026-06-02 21:51:23.782666444 +0000 UTC m=+568.487676368" lastFinishedPulling="2026-06-02 21:51:26.001260815 +0000 UTC m=+570.706270743" observedRunningTime="2026-06-02 21:51:26.871217729 +0000 UTC m=+571.576227676" watchObservedRunningTime="2026-06-02 21:51:26.873193726 +0000 UTC m=+571.578203717" Jun 02 21:51:37.862137 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:37.862105 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5bf74477bf-vjths" Jun 02 21:51:49.829159 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:49.829108 2574 pod_container_manager_linux.go:217] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","podb8314b20-5f03-45f8-8541-f87bddb2a4f6"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort podb8314b20-5f03-45f8-8541-f87bddb2a4f6] : Timed out while waiting for systemd to remove kubepods-besteffort-podb8314b20_5f03_45f8_8541_f87bddb2a4f6.slice" Jun 02 21:51:49.829159 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:51:49.829159 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods besteffort podb8314b20-5f03-45f8-8541-f87bddb2a4f6] : unable to destroy cgroup paths for cgroup [kubepods besteffort podb8314b20-5f03-45f8-8541-f87bddb2a4f6] : Timed out while waiting for systemd to remove kubepods-besteffort-podb8314b20_5f03_45f8_8541_f87bddb2a4f6.slice" pod="kuadrant-system/authorino-55658b4dd6-fz94k" podUID="b8314b20-5f03-45f8-8541-f87bddb2a4f6" Jun 02 21:51:49.922461 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:49.922432 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55658b4dd6-fz94k" Jun 02 21:51:49.945593 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:49.945550 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-55658b4dd6-fz94k"] Jun 02 21:51:49.947043 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:49.947019 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-55658b4dd6-fz94k"] Jun 02 21:51:51.848307 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:51.848244 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8314b20-5f03-45f8-8541-f87bddb2a4f6" path="/var/lib/kubelet/pods/b8314b20-5f03-45f8-8541-f87bddb2a4f6/volumes" Jun 02 21:51:55.759979 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:55.759956 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/ovn-acl-logging/0.log" Jun 02 21:51:55.760415 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:55.760057 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/ovn-acl-logging/0.log" Jun 02 21:51:57.239922 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:57.239887 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-5fc88c9d87-k5lsz"] Jun 02 21:51:57.241868 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:57.241852 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5fc88c9d87-k5lsz" Jun 02 21:51:57.243968 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:57.243947 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 02 21:51:57.243968 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:57.243962 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-5wnpb\"" Jun 02 21:51:57.251827 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:57.251797 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5fc88c9d87-k5lsz"] Jun 02 21:51:57.340389 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:57.340363 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d166a92c-e0f3-46da-8eec-de4b32bcf322-maas-api-tls\") pod \"maas-api-5fc88c9d87-k5lsz\" (UID: \"d166a92c-e0f3-46da-8eec-de4b32bcf322\") " pod="opendatahub/maas-api-5fc88c9d87-k5lsz" Jun 02 21:51:57.340509 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:57.340396 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d6r5\" (UniqueName: \"kubernetes.io/projected/d166a92c-e0f3-46da-8eec-de4b32bcf322-kube-api-access-5d6r5\") pod \"maas-api-5fc88c9d87-k5lsz\" (UID: \"d166a92c-e0f3-46da-8eec-de4b32bcf322\") " pod="opendatahub/maas-api-5fc88c9d87-k5lsz" Jun 02 21:51:57.441591 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:57.441548 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d166a92c-e0f3-46da-8eec-de4b32bcf322-maas-api-tls\") pod \"maas-api-5fc88c9d87-k5lsz\" (UID: \"d166a92c-e0f3-46da-8eec-de4b32bcf322\") " pod="opendatahub/maas-api-5fc88c9d87-k5lsz" Jun 02 21:51:57.441591 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:57.441586 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5d6r5\" (UniqueName: \"kubernetes.io/projected/d166a92c-e0f3-46da-8eec-de4b32bcf322-kube-api-access-5d6r5\") pod \"maas-api-5fc88c9d87-k5lsz\" (UID: \"d166a92c-e0f3-46da-8eec-de4b32bcf322\") " pod="opendatahub/maas-api-5fc88c9d87-k5lsz" Jun 02 21:51:57.444158 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:57.444134 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d166a92c-e0f3-46da-8eec-de4b32bcf322-maas-api-tls\") pod \"maas-api-5fc88c9d87-k5lsz\" (UID: \"d166a92c-e0f3-46da-8eec-de4b32bcf322\") " pod="opendatahub/maas-api-5fc88c9d87-k5lsz" Jun 02 21:51:57.448612 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:57.448593 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d6r5\" (UniqueName: \"kubernetes.io/projected/d166a92c-e0f3-46da-8eec-de4b32bcf322-kube-api-access-5d6r5\") pod \"maas-api-5fc88c9d87-k5lsz\" (UID: \"d166a92c-e0f3-46da-8eec-de4b32bcf322\") " pod="opendatahub/maas-api-5fc88c9d87-k5lsz" Jun 02 21:51:57.551544 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:57.551473 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5fc88c9d87-k5lsz" Jun 02 21:51:57.637071 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:57.637045 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-zbmkq"] Jun 02 21:51:57.640736 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:57.640718 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-zbmkq" Jun 02 21:51:57.643150 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:57.643129 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-tk226\"" Jun 02 21:51:57.643309 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:57.643153 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 02 21:51:57.648260 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:57.648233 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-zbmkq"] Jun 02 21:51:57.676883 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:57.676861 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5fc88c9d87-k5lsz"] Jun 02 21:51:57.678971 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:51:57.678947 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd166a92c_e0f3_46da_8eec_de4b32bcf322.slice/crio-34cbfcd8efc68abb43438730a40040519f5e7d9023c72310f6c8c45a5b72a197 WatchSource:0}: Error finding container 34cbfcd8efc68abb43438730a40040519f5e7d9023c72310f6c8c45a5b72a197: Status 404 returned error can't find the container with id 34cbfcd8efc68abb43438730a40040519f5e7d9023c72310f6c8c45a5b72a197 Jun 02 21:51:57.743677 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:57.743652 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g4sb\" (UniqueName: \"kubernetes.io/projected/f6f637da-fcd5-44b7-88a7-3ef5681d1eb3-kube-api-access-8g4sb\") pod \"payload-processing-58b6c8fdc7-zbmkq\" (UID: \"f6f637da-fcd5-44b7-88a7-3ef5681d1eb3\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-zbmkq" Jun 02 21:51:57.845099 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:57.845056 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8g4sb\" (UniqueName: \"kubernetes.io/projected/f6f637da-fcd5-44b7-88a7-3ef5681d1eb3-kube-api-access-8g4sb\") pod \"payload-processing-58b6c8fdc7-zbmkq\" (UID: \"f6f637da-fcd5-44b7-88a7-3ef5681d1eb3\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-zbmkq" Jun 02 21:51:57.857506 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:57.857466 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g4sb\" (UniqueName: \"kubernetes.io/projected/f6f637da-fcd5-44b7-88a7-3ef5681d1eb3-kube-api-access-8g4sb\") pod \"payload-processing-58b6c8fdc7-zbmkq\" (UID: \"f6f637da-fcd5-44b7-88a7-3ef5681d1eb3\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-zbmkq" Jun 02 21:51:57.948717 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:57.948682 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5fc88c9d87-k5lsz" event={"ID":"d166a92c-e0f3-46da-8eec-de4b32bcf322","Type":"ContainerStarted","Data":"34cbfcd8efc68abb43438730a40040519f5e7d9023c72310f6c8c45a5b72a197"} Jun 02 21:51:57.954915 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:57.954891 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-zbmkq" Jun 02 21:51:58.075799 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:58.075776 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-zbmkq"] Jun 02 21:51:58.078133 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:51:58.078105 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6f637da_fcd5_44b7_88a7_3ef5681d1eb3.slice/crio-a866067ed2e48079c7f83c282eddd917c23bdfbdc77ef6a50977b5d4d6e1dbc4 WatchSource:0}: Error finding container a866067ed2e48079c7f83c282eddd917c23bdfbdc77ef6a50977b5d4d6e1dbc4: Status 404 returned error can't find the container with id a866067ed2e48079c7f83c282eddd917c23bdfbdc77ef6a50977b5d4d6e1dbc4 Jun 02 21:51:58.956016 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:51:58.955957 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-zbmkq" event={"ID":"f6f637da-fcd5-44b7-88a7-3ef5681d1eb3","Type":"ContainerStarted","Data":"a866067ed2e48079c7f83c282eddd917c23bdfbdc77ef6a50977b5d4d6e1dbc4"} Jun 02 21:52:00.962919 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:00.962863 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5fc88c9d87-k5lsz" event={"ID":"d166a92c-e0f3-46da-8eec-de4b32bcf322","Type":"ContainerStarted","Data":"feed19fa0340bbe3ebb74d995bb0a87e551b13b28604930153af217dbe4f7edd"} Jun 02 21:52:00.963384 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:00.962997 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-5fc88c9d87-k5lsz" Jun 02 21:52:00.964120 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:00.964100 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-zbmkq" event={"ID":"f6f637da-fcd5-44b7-88a7-3ef5681d1eb3","Type":"ContainerStarted","Data":"930b829299a8ebcf8b3304c782271d15456f7f9b571f54b1820de0890a1307e4"} Jun 02 21:52:00.964215 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:00.964198 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-zbmkq" Jun 02 21:52:00.980229 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:00.980191 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-5fc88c9d87-k5lsz" podStartSLOduration=1.217168489 podStartE2EDuration="3.980179424s" podCreationTimestamp="2026-06-02 21:51:57 +0000 UTC" firstStartedPulling="2026-06-02 21:51:57.680320415 +0000 UTC m=+602.385330345" lastFinishedPulling="2026-06-02 21:52:00.443331352 +0000 UTC m=+605.148341280" observedRunningTime="2026-06-02 21:52:00.978259858 +0000 UTC m=+605.683269805" watchObservedRunningTime="2026-06-02 21:52:00.980179424 +0000 UTC m=+605.685189371" Jun 02 21:52:00.994609 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:00.994572 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58b6c8fdc7-zbmkq" podStartSLOduration=1.6049097570000002 podStartE2EDuration="3.99456133s" podCreationTimestamp="2026-06-02 21:51:57 +0000 UTC" firstStartedPulling="2026-06-02 21:51:58.079927056 +0000 UTC m=+602.784936980" lastFinishedPulling="2026-06-02 21:52:00.46957862 +0000 UTC m=+605.174588553" observedRunningTime="2026-06-02 21:52:00.993354247 +0000 UTC m=+605.698364195" watchObservedRunningTime="2026-06-02 21:52:00.99456133 +0000 UTC m=+605.699571277" Jun 02 21:52:06.972431 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:06.972400 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-5fc88c9d87-k5lsz" Jun 02 21:52:08.762568 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:08.762533 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:52:08.762965 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:08.762753 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11" containerName="keycloak" containerID="cri-o://ee2e9c6849b9a2132dac9e1c37b7618a9e2f5828ebc7fde4ce13330b513fcb1a" gracePeriod=30 Jun 02 21:52:10.408689 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:10.408669 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 21:52:10.435638 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:10.435614 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x645r\" (UniqueName: \"kubernetes.io/projected/881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11-kube-api-access-x645r\") pod \"881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11\" (UID: \"881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11\") " Jun 02 21:52:10.437661 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:10.437630 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11-kube-api-access-x645r" (OuterVolumeSpecName: "kube-api-access-x645r") pod "881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11" (UID: "881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11"). InnerVolumeSpecName "kube-api-access-x645r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:52:10.536619 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:10.536594 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x645r\" (UniqueName: \"kubernetes.io/projected/881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11-kube-api-access-x645r\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:52:10.996682 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:10.996627 2574 generic.go:358] "Generic (PLEG): container finished" podID="881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11" containerID="ee2e9c6849b9a2132dac9e1c37b7618a9e2f5828ebc7fde4ce13330b513fcb1a" exitCode=143 Jun 02 21:52:10.996837 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:10.996687 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11","Type":"ContainerDied","Data":"ee2e9c6849b9a2132dac9e1c37b7618a9e2f5828ebc7fde4ce13330b513fcb1a"} Jun 02 21:52:10.996837 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:10.996713 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 21:52:10.996837 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:10.996721 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11","Type":"ContainerDied","Data":"3de89350febe8a4d3349c76967fed633a8028f925bd44ffa198bc797e6dc1550"} Jun 02 21:52:10.996837 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:10.996740 2574 scope.go:117] "RemoveContainer" containerID="ee2e9c6849b9a2132dac9e1c37b7618a9e2f5828ebc7fde4ce13330b513fcb1a" Jun 02 21:52:11.006177 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.006156 2574 scope.go:117] "RemoveContainer" containerID="ee2e9c6849b9a2132dac9e1c37b7618a9e2f5828ebc7fde4ce13330b513fcb1a" Jun 02 21:52:11.006420 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:52:11.006402 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee2e9c6849b9a2132dac9e1c37b7618a9e2f5828ebc7fde4ce13330b513fcb1a\": container with ID starting with ee2e9c6849b9a2132dac9e1c37b7618a9e2f5828ebc7fde4ce13330b513fcb1a not found: ID does not exist" containerID="ee2e9c6849b9a2132dac9e1c37b7618a9e2f5828ebc7fde4ce13330b513fcb1a" Jun 02 21:52:11.006472 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.006428 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee2e9c6849b9a2132dac9e1c37b7618a9e2f5828ebc7fde4ce13330b513fcb1a"} err="failed to get container status \"ee2e9c6849b9a2132dac9e1c37b7618a9e2f5828ebc7fde4ce13330b513fcb1a\": rpc error: code = NotFound desc = could not find container \"ee2e9c6849b9a2132dac9e1c37b7618a9e2f5828ebc7fde4ce13330b513fcb1a\": container with ID starting with ee2e9c6849b9a2132dac9e1c37b7618a9e2f5828ebc7fde4ce13330b513fcb1a not found: ID does not exist" Jun 02 21:52:11.015754 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.015731 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:52:11.019262 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.019241 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:52:11.038487 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.038463 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:52:11.038843 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.038830 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11" containerName="keycloak" Jun 02 21:52:11.038898 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.038844 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11" containerName="keycloak" Jun 02 21:52:11.038932 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.038901 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11" containerName="keycloak" Jun 02 21:52:11.042951 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.042936 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 21:52:11.045045 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.045025 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 02 21:52:11.045143 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.045066 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-l4f8s\"" Jun 02 21:52:11.045143 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.045088 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 02 21:52:11.045249 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.045143 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 02 21:52:11.045249 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.045231 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 02 21:52:11.049058 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.049037 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:52:11.140199 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.140171 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/7a17910b-83eb-4b31-80b2-7cf383f634ee-test-realms\") pod \"maas-keycloak-0\" (UID: \"7a17910b-83eb-4b31-80b2-7cf383f634ee\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:52:11.140328 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.140206 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjcmh\" (UniqueName: \"kubernetes.io/projected/7a17910b-83eb-4b31-80b2-7cf383f634ee-kube-api-access-mjcmh\") pod \"maas-keycloak-0\" (UID: \"7a17910b-83eb-4b31-80b2-7cf383f634ee\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:52:11.241119 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.241094 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/7a17910b-83eb-4b31-80b2-7cf383f634ee-test-realms\") pod \"maas-keycloak-0\" (UID: \"7a17910b-83eb-4b31-80b2-7cf383f634ee\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:52:11.241224 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.241132 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mjcmh\" (UniqueName: \"kubernetes.io/projected/7a17910b-83eb-4b31-80b2-7cf383f634ee-kube-api-access-mjcmh\") pod \"maas-keycloak-0\" (UID: \"7a17910b-83eb-4b31-80b2-7cf383f634ee\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:52:11.241824 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.241805 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/7a17910b-83eb-4b31-80b2-7cf383f634ee-test-realms\") pod \"maas-keycloak-0\" (UID: \"7a17910b-83eb-4b31-80b2-7cf383f634ee\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:52:11.248724 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.248674 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjcmh\" (UniqueName: \"kubernetes.io/projected/7a17910b-83eb-4b31-80b2-7cf383f634ee-kube-api-access-mjcmh\") pod \"maas-keycloak-0\" (UID: \"7a17910b-83eb-4b31-80b2-7cf383f634ee\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:52:11.353180 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.353148 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 21:52:11.473602 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.473573 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:52:11.476654 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:52:11.476626 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a17910b_83eb_4b31_80b2_7cf383f634ee.slice/crio-788bbd119f21f47b00e992e818353daedd00ff43d46f89642c1bb2148b0c0032 WatchSource:0}: Error finding container 788bbd119f21f47b00e992e818353daedd00ff43d46f89642c1bb2148b0c0032: Status 404 returned error can't find the container with id 788bbd119f21f47b00e992e818353daedd00ff43d46f89642c1bb2148b0c0032 Jun 02 21:52:11.477866 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.477848 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 21:52:11.847472 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.847443 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11" path="/var/lib/kubelet/pods/881c6ceb-3be6-40b7-9ea8-8cc7a16f9e11/volumes" Jun 02 21:52:11.968651 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:11.968624 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-zbmkq" Jun 02 21:52:12.002475 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:12.002440 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"7a17910b-83eb-4b31-80b2-7cf383f634ee","Type":"ContainerStarted","Data":"e63a74ee610f52bbbe0024c82c4c0f41774e40a757eee4945afcad6ec7aaa3c1"} Jun 02 21:52:12.002475 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:12.002481 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"7a17910b-83eb-4b31-80b2-7cf383f634ee","Type":"ContainerStarted","Data":"788bbd119f21f47b00e992e818353daedd00ff43d46f89642c1bb2148b0c0032"} Jun 02 21:52:12.018391 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:12.018338 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=0.706395971 podStartE2EDuration="1.018321005s" podCreationTimestamp="2026-06-02 21:52:11 +0000 UTC" firstStartedPulling="2026-06-02 21:52:11.477962725 +0000 UTC m=+616.182972651" lastFinishedPulling="2026-06-02 21:52:11.78988776 +0000 UTC m=+616.494897685" observedRunningTime="2026-06-02 21:52:12.017150924 +0000 UTC m=+616.722160872" watchObservedRunningTime="2026-06-02 21:52:12.018321005 +0000 UTC m=+616.723330953" Jun 02 21:52:12.353379 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:12.353343 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 02 21:52:12.354997 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:12.354967 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7a17910b-83eb-4b31-80b2-7cf383f634ee" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.37:9000/health/started\": dial tcp 10.134.0.37:9000: connect: connection refused" Jun 02 21:52:13.354386 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:13.354332 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7a17910b-83eb-4b31-80b2-7cf383f634ee" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.37:9000/health/started\": dial tcp 10.134.0.37:9000: connect: connection refused" Jun 02 21:52:14.353964 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:14.353919 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7a17910b-83eb-4b31-80b2-7cf383f634ee" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.37:9000/health/started\": dial tcp 10.134.0.37:9000: connect: connection refused" Jun 02 21:52:15.354112 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:15.354068 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7a17910b-83eb-4b31-80b2-7cf383f634ee" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.37:9000/health/started\": dial tcp 10.134.0.37:9000: connect: connection refused" Jun 02 21:52:16.354406 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:16.354355 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7a17910b-83eb-4b31-80b2-7cf383f634ee" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.37:9000/health/started\": dial tcp 10.134.0.37:9000: connect: connection refused" Jun 02 21:52:17.354306 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:17.354235 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7a17910b-83eb-4b31-80b2-7cf383f634ee" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.37:9000/health/started\": dial tcp 10.134.0.37:9000: connect: connection refused" Jun 02 21:52:18.354589 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:18.354535 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7a17910b-83eb-4b31-80b2-7cf383f634ee" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.37:9000/health/started\": dial tcp 10.134.0.37:9000: connect: connection refused" Jun 02 21:52:19.354673 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:19.354622 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7a17910b-83eb-4b31-80b2-7cf383f634ee" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.37:9000/health/started\": dial tcp 10.134.0.37:9000: connect: connection refused" Jun 02 21:52:20.353623 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:20.353579 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7a17910b-83eb-4b31-80b2-7cf383f634ee" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.37:9000/health/started\": dial tcp 10.134.0.37:9000: connect: connection refused" Jun 02 21:52:21.354100 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:21.354046 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 02 21:52:21.354580 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:21.354420 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7a17910b-83eb-4b31-80b2-7cf383f634ee" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.37:9000/health/started\": dial tcp 10.134.0.37:9000: connect: connection refused" Jun 02 21:52:22.354117 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:22.354068 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7a17910b-83eb-4b31-80b2-7cf383f634ee" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.37:9000/health/started\": dial tcp 10.134.0.37:9000: connect: connection refused" Jun 02 21:52:23.353900 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:23.353853 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7a17910b-83eb-4b31-80b2-7cf383f634ee" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.37:9000/health/started\": dial tcp 10.134.0.37:9000: connect: connection refused" Jun 02 21:52:24.354337 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:24.354279 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7a17910b-83eb-4b31-80b2-7cf383f634ee" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.37:9000/health/started\": dial tcp 10.134.0.37:9000: connect: connection refused" Jun 02 21:52:25.451417 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:25.451369 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 02 21:52:25.465319 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:25.465259 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="7a17910b-83eb-4b31-80b2-7cf383f634ee" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 21:52:35.457502 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:35.457468 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 02 21:52:46.138158 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:46.138124 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-894d797f4-4xwqf"] Jun 02 21:52:46.144538 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:46.144518 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-894d797f4-4xwqf" Jun 02 21:52:46.146756 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:46.146733 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jun 02 21:52:46.148128 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:46.148107 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-894d797f4-4xwqf"] Jun 02 21:52:46.261428 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:46.261403 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzs2c\" (UniqueName: \"kubernetes.io/projected/9d5f0a40-684b-4657-957e-e03622eafc37-kube-api-access-bzs2c\") pod \"authorino-894d797f4-4xwqf\" (UID: \"9d5f0a40-684b-4657-957e-e03622eafc37\") " pod="kuadrant-system/authorino-894d797f4-4xwqf" Jun 02 21:52:46.261539 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:46.261447 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/9d5f0a40-684b-4657-957e-e03622eafc37-oidc-ca\") pod \"authorino-894d797f4-4xwqf\" (UID: \"9d5f0a40-684b-4657-957e-e03622eafc37\") " pod="kuadrant-system/authorino-894d797f4-4xwqf" Jun 02 21:52:46.261539 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:46.261470 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9d5f0a40-684b-4657-957e-e03622eafc37-tls-cert\") pod \"authorino-894d797f4-4xwqf\" (UID: \"9d5f0a40-684b-4657-957e-e03622eafc37\") " pod="kuadrant-system/authorino-894d797f4-4xwqf" Jun 02 21:52:46.361815 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:46.361787 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bzs2c\" (UniqueName: \"kubernetes.io/projected/9d5f0a40-684b-4657-957e-e03622eafc37-kube-api-access-bzs2c\") pod \"authorino-894d797f4-4xwqf\" (UID: \"9d5f0a40-684b-4657-957e-e03622eafc37\") " pod="kuadrant-system/authorino-894d797f4-4xwqf" Jun 02 21:52:46.361935 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:46.361839 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/9d5f0a40-684b-4657-957e-e03622eafc37-oidc-ca\") pod \"authorino-894d797f4-4xwqf\" (UID: \"9d5f0a40-684b-4657-957e-e03622eafc37\") " pod="kuadrant-system/authorino-894d797f4-4xwqf" Jun 02 21:52:46.361935 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:46.361857 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9d5f0a40-684b-4657-957e-e03622eafc37-tls-cert\") pod \"authorino-894d797f4-4xwqf\" (UID: \"9d5f0a40-684b-4657-957e-e03622eafc37\") " pod="kuadrant-system/authorino-894d797f4-4xwqf" Jun 02 21:52:46.362443 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:46.362422 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/9d5f0a40-684b-4657-957e-e03622eafc37-oidc-ca\") pod \"authorino-894d797f4-4xwqf\" (UID: \"9d5f0a40-684b-4657-957e-e03622eafc37\") " pod="kuadrant-system/authorino-894d797f4-4xwqf" Jun 02 21:52:46.364148 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:46.364130 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9d5f0a40-684b-4657-957e-e03622eafc37-tls-cert\") pod \"authorino-894d797f4-4xwqf\" (UID: \"9d5f0a40-684b-4657-957e-e03622eafc37\") " pod="kuadrant-system/authorino-894d797f4-4xwqf" Jun 02 21:52:46.369866 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:46.369846 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzs2c\" (UniqueName: \"kubernetes.io/projected/9d5f0a40-684b-4657-957e-e03622eafc37-kube-api-access-bzs2c\") pod \"authorino-894d797f4-4xwqf\" (UID: \"9d5f0a40-684b-4657-957e-e03622eafc37\") " pod="kuadrant-system/authorino-894d797f4-4xwqf" Jun 02 21:52:46.455405 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:46.455356 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-894d797f4-4xwqf" Jun 02 21:52:46.574350 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:46.574324 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-894d797f4-4xwqf"] Jun 02 21:52:46.575993 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:52:46.575967 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d5f0a40_684b_4657_957e_e03622eafc37.slice/crio-2117d1467d346a9c49384a8b27533c660c5fd7f20efc799845009d4aca84be89 WatchSource:0}: Error finding container 2117d1467d346a9c49384a8b27533c660c5fd7f20efc799845009d4aca84be89: Status 404 returned error can't find the container with id 2117d1467d346a9c49384a8b27533c660c5fd7f20efc799845009d4aca84be89 Jun 02 21:52:47.137911 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:47.137875 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-894d797f4-4xwqf" event={"ID":"9d5f0a40-684b-4657-957e-e03622eafc37","Type":"ContainerStarted","Data":"c9a231d7b9c9f24b5466387fcec39686c2482018fdf9796cca8cc7bffe378a54"} Jun 02 21:52:47.137911 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:47.137913 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-894d797f4-4xwqf" event={"ID":"9d5f0a40-684b-4657-957e-e03622eafc37","Type":"ContainerStarted","Data":"2117d1467d346a9c49384a8b27533c660c5fd7f20efc799845009d4aca84be89"} Jun 02 21:52:47.154045 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:47.153993 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-894d797f4-4xwqf" podStartSLOduration=0.769042481 podStartE2EDuration="1.153976517s" podCreationTimestamp="2026-06-02 21:52:46 +0000 UTC" firstStartedPulling="2026-06-02 21:52:46.577207973 +0000 UTC m=+651.282217899" lastFinishedPulling="2026-06-02 21:52:46.962142007 +0000 UTC m=+651.667151935" observedRunningTime="2026-06-02 21:52:47.152192761 +0000 UTC m=+651.857202707" watchObservedRunningTime="2026-06-02 21:52:47.153976517 +0000 UTC m=+651.858986465" Jun 02 21:52:47.180274 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:47.180239 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-d6b7df54-xm5l4"] Jun 02 21:52:47.180490 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:47.180465 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-d6b7df54-xm5l4" podUID="d3fdd836-217f-434a-ac83-e24b56ccb233" containerName="authorino" containerID="cri-o://a9a8f4f6cfcf9df496f9932930cbd9c9126fecb0729e4cdd1ffac64ba94f4d06" gracePeriod=30 Jun 02 21:52:47.463517 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:47.463490 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d6b7df54-xm5l4" Jun 02 21:52:47.572893 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:47.572818 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lg6h\" (UniqueName: \"kubernetes.io/projected/d3fdd836-217f-434a-ac83-e24b56ccb233-kube-api-access-6lg6h\") pod \"d3fdd836-217f-434a-ac83-e24b56ccb233\" (UID: \"d3fdd836-217f-434a-ac83-e24b56ccb233\") " Jun 02 21:52:47.572893 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:47.572885 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d3fdd836-217f-434a-ac83-e24b56ccb233-tls-cert\") pod \"d3fdd836-217f-434a-ac83-e24b56ccb233\" (UID: \"d3fdd836-217f-434a-ac83-e24b56ccb233\") " Jun 02 21:52:47.575442 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:47.575392 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3fdd836-217f-434a-ac83-e24b56ccb233-kube-api-access-6lg6h" (OuterVolumeSpecName: "kube-api-access-6lg6h") pod "d3fdd836-217f-434a-ac83-e24b56ccb233" (UID: "d3fdd836-217f-434a-ac83-e24b56ccb233"). InnerVolumeSpecName "kube-api-access-6lg6h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:52:47.584338 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:47.584276 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3fdd836-217f-434a-ac83-e24b56ccb233-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d3fdd836-217f-434a-ac83-e24b56ccb233" (UID: "d3fdd836-217f-434a-ac83-e24b56ccb233"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:52:47.674472 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:47.674439 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lg6h\" (UniqueName: \"kubernetes.io/projected/d3fdd836-217f-434a-ac83-e24b56ccb233-kube-api-access-6lg6h\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:52:47.674472 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:47.674468 2574 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d3fdd836-217f-434a-ac83-e24b56ccb233-tls-cert\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:52:48.142381 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:48.142278 2574 generic.go:358] "Generic (PLEG): container finished" podID="d3fdd836-217f-434a-ac83-e24b56ccb233" containerID="a9a8f4f6cfcf9df496f9932930cbd9c9126fecb0729e4cdd1ffac64ba94f4d06" exitCode=0 Jun 02 21:52:48.142381 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:48.142368 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d6b7df54-xm5l4" Jun 02 21:52:48.142584 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:48.142382 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d6b7df54-xm5l4" event={"ID":"d3fdd836-217f-434a-ac83-e24b56ccb233","Type":"ContainerDied","Data":"a9a8f4f6cfcf9df496f9932930cbd9c9126fecb0729e4cdd1ffac64ba94f4d06"} Jun 02 21:52:48.142584 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:48.142426 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d6b7df54-xm5l4" event={"ID":"d3fdd836-217f-434a-ac83-e24b56ccb233","Type":"ContainerDied","Data":"a6dba483e9b70055acf426ed1be2380b26536254934b9bf64f64daeead2e5cad"} Jun 02 21:52:48.142584 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:48.142448 2574 scope.go:117] "RemoveContainer" containerID="a9a8f4f6cfcf9df496f9932930cbd9c9126fecb0729e4cdd1ffac64ba94f4d06" Jun 02 21:52:48.151139 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:48.151118 2574 scope.go:117] "RemoveContainer" containerID="a9a8f4f6cfcf9df496f9932930cbd9c9126fecb0729e4cdd1ffac64ba94f4d06" Jun 02 21:52:48.151411 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:52:48.151390 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9a8f4f6cfcf9df496f9932930cbd9c9126fecb0729e4cdd1ffac64ba94f4d06\": container with ID starting with a9a8f4f6cfcf9df496f9932930cbd9c9126fecb0729e4cdd1ffac64ba94f4d06 not found: ID does not exist" containerID="a9a8f4f6cfcf9df496f9932930cbd9c9126fecb0729e4cdd1ffac64ba94f4d06" Jun 02 21:52:48.151484 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:48.151435 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9a8f4f6cfcf9df496f9932930cbd9c9126fecb0729e4cdd1ffac64ba94f4d06"} err="failed to get container status \"a9a8f4f6cfcf9df496f9932930cbd9c9126fecb0729e4cdd1ffac64ba94f4d06\": rpc error: code = NotFound desc = could not find container \"a9a8f4f6cfcf9df496f9932930cbd9c9126fecb0729e4cdd1ffac64ba94f4d06\": container with ID starting with a9a8f4f6cfcf9df496f9932930cbd9c9126fecb0729e4cdd1ffac64ba94f4d06 not found: ID does not exist" Jun 02 21:52:48.161542 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:48.161520 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-d6b7df54-xm5l4"] Jun 02 21:52:48.166517 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:48.166499 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-d6b7df54-xm5l4"] Jun 02 21:52:49.847274 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:52:49.847240 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3fdd836-217f-434a-ac83-e24b56ccb233" path="/var/lib/kubelet/pods/d3fdd836-217f-434a-ac83-e24b56ccb233/volumes" Jun 02 21:53:07.587937 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.587901 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp"] Jun 02 21:53:07.588272 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.588252 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3fdd836-217f-434a-ac83-e24b56ccb233" containerName="authorino" Jun 02 21:53:07.588272 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.588263 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3fdd836-217f-434a-ac83-e24b56ccb233" containerName="authorino" Jun 02 21:53:07.588391 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.588329 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="d3fdd836-217f-434a-ac83-e24b56ccb233" containerName="authorino" Jun 02 21:53:07.592664 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.592648 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" Jun 02 21:53:07.595106 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.595081 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 02 21:53:07.595819 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.595794 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pmq97\"" Jun 02 21:53:07.595919 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.595818 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 02 21:53:07.595993 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.595975 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jun 02 21:53:07.601141 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.600833 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp"] Jun 02 21:53:07.744632 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.744604 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0bc4bb8e-efd3-428b-b355-e948cead363f-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp\" (UID: \"0bc4bb8e-efd3-428b-b355-e948cead363f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" Jun 02 21:53:07.744757 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.744649 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bc4bb8e-efd3-428b-b355-e948cead363f-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp\" (UID: \"0bc4bb8e-efd3-428b-b355-e948cead363f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" Jun 02 21:53:07.744757 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.744679 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bc4bb8e-efd3-428b-b355-e948cead363f-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp\" (UID: \"0bc4bb8e-efd3-428b-b355-e948cead363f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" Jun 02 21:53:07.744757 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.744726 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bc4bb8e-efd3-428b-b355-e948cead363f-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp\" (UID: \"0bc4bb8e-efd3-428b-b355-e948cead363f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" Jun 02 21:53:07.744896 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.744790 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvg7z\" (UniqueName: \"kubernetes.io/projected/0bc4bb8e-efd3-428b-b355-e948cead363f-kube-api-access-gvg7z\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp\" (UID: \"0bc4bb8e-efd3-428b-b355-e948cead363f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" Jun 02 21:53:07.744896 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.744823 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bc4bb8e-efd3-428b-b355-e948cead363f-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp\" (UID: \"0bc4bb8e-efd3-428b-b355-e948cead363f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" Jun 02 21:53:07.845475 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.845418 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bc4bb8e-efd3-428b-b355-e948cead363f-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp\" (UID: \"0bc4bb8e-efd3-428b-b355-e948cead363f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" Jun 02 21:53:07.845475 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.845465 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gvg7z\" (UniqueName: \"kubernetes.io/projected/0bc4bb8e-efd3-428b-b355-e948cead363f-kube-api-access-gvg7z\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp\" (UID: \"0bc4bb8e-efd3-428b-b355-e948cead363f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" Jun 02 21:53:07.845612 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.845501 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bc4bb8e-efd3-428b-b355-e948cead363f-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp\" (UID: \"0bc4bb8e-efd3-428b-b355-e948cead363f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" Jun 02 21:53:07.845612 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.845547 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0bc4bb8e-efd3-428b-b355-e948cead363f-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp\" (UID: \"0bc4bb8e-efd3-428b-b355-e948cead363f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" Jun 02 21:53:07.845612 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.845607 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bc4bb8e-efd3-428b-b355-e948cead363f-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp\" (UID: \"0bc4bb8e-efd3-428b-b355-e948cead363f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" Jun 02 21:53:07.845762 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.845659 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bc4bb8e-efd3-428b-b355-e948cead363f-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp\" (UID: \"0bc4bb8e-efd3-428b-b355-e948cead363f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" Jun 02 21:53:07.845841 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.845820 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bc4bb8e-efd3-428b-b355-e948cead363f-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp\" (UID: \"0bc4bb8e-efd3-428b-b355-e948cead363f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" Jun 02 21:53:07.845963 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.845945 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0bc4bb8e-efd3-428b-b355-e948cead363f-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp\" (UID: \"0bc4bb8e-efd3-428b-b355-e948cead363f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" Jun 02 21:53:07.846029 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.846000 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bc4bb8e-efd3-428b-b355-e948cead363f-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp\" (UID: \"0bc4bb8e-efd3-428b-b355-e948cead363f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" Jun 02 21:53:07.848088 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.848066 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bc4bb8e-efd3-428b-b355-e948cead363f-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp\" (UID: \"0bc4bb8e-efd3-428b-b355-e948cead363f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" Jun 02 21:53:07.848269 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.848254 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bc4bb8e-efd3-428b-b355-e948cead363f-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp\" (UID: \"0bc4bb8e-efd3-428b-b355-e948cead363f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" Jun 02 21:53:07.853859 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.853838 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvg7z\" (UniqueName: \"kubernetes.io/projected/0bc4bb8e-efd3-428b-b355-e948cead363f-kube-api-access-gvg7z\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp\" (UID: \"0bc4bb8e-efd3-428b-b355-e948cead363f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" Jun 02 21:53:07.905910 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:07.905886 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" Jun 02 21:53:08.031193 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:08.031169 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp"] Jun 02 21:53:08.033930 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:53:08.033907 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bc4bb8e_efd3_428b_b355_e948cead363f.slice/crio-e6cf5515076cf08711afd536a31a3adb1a9da96f639339e66f4279ec32d85334 WatchSource:0}: Error finding container e6cf5515076cf08711afd536a31a3adb1a9da96f639339e66f4279ec32d85334: Status 404 returned error can't find the container with id e6cf5515076cf08711afd536a31a3adb1a9da96f639339e66f4279ec32d85334 Jun 02 21:53:08.209267 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:08.209196 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" event={"ID":"0bc4bb8e-efd3-428b-b355-e948cead363f","Type":"ContainerStarted","Data":"e6cf5515076cf08711afd536a31a3adb1a9da96f639339e66f4279ec32d85334"} Jun 02 21:53:11.226167 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:11.226137 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" event={"ID":"0bc4bb8e-efd3-428b-b355-e948cead363f","Type":"ContainerStarted","Data":"f6644ef8464e291e4b96614dbb92eb850f98a7b9afb48806275b4154eed9951a"} Jun 02 21:53:13.234234 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:13.234202 2574 generic.go:358] "Generic (PLEG): container finished" podID="0bc4bb8e-efd3-428b-b355-e948cead363f" containerID="f6644ef8464e291e4b96614dbb92eb850f98a7b9afb48806275b4154eed9951a" exitCode=0 Jun 02 21:53:13.234610 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:13.234273 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" event={"ID":"0bc4bb8e-efd3-428b-b355-e948cead363f","Type":"ContainerDied","Data":"f6644ef8464e291e4b96614dbb92eb850f98a7b9afb48806275b4154eed9951a"} Jun 02 21:53:15.242437 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:15.242397 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" event={"ID":"0bc4bb8e-efd3-428b-b355-e948cead363f","Type":"ContainerStarted","Data":"5f64a644ff28692920306fb9e35eb2f0d5df8dc3a669978a858aabb8a4bd0cb7"} Jun 02 21:53:15.262254 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:15.262210 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" podStartSLOduration=2.041321455 podStartE2EDuration="8.262197774s" podCreationTimestamp="2026-06-02 21:53:07 +0000 UTC" firstStartedPulling="2026-06-02 21:53:08.036039023 +0000 UTC m=+672.741048948" lastFinishedPulling="2026-06-02 21:53:14.256915343 +0000 UTC m=+678.961925267" observedRunningTime="2026-06-02 21:53:15.260643622 +0000 UTC m=+679.965653569" watchObservedRunningTime="2026-06-02 21:53:15.262197774 +0000 UTC m=+679.967207721" Jun 02 21:53:17.906832 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:17.906799 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" Jun 02 21:53:17.906832 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:17.906839 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" Jun 02 21:53:17.918997 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:17.918976 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" Jun 02 21:53:18.263618 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:18.263537 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp" Jun 02 21:53:52.211264 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:52.211192 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7"] Jun 02 21:53:52.213781 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:52.213761 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" Jun 02 21:53:52.216332 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:52.216312 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 02 21:53:52.232623 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:52.232600 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7"] Jun 02 21:53:52.306183 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:52.306155 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z28pv\" (UniqueName: \"kubernetes.io/projected/727ee091-94aa-4703-9777-ffc6c6e3e04d-kube-api-access-z28pv\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7\" (UID: \"727ee091-94aa-4703-9777-ffc6c6e3e04d\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" Jun 02 21:53:52.306316 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:52.306199 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/727ee091-94aa-4703-9777-ffc6c6e3e04d-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7\" (UID: \"727ee091-94aa-4703-9777-ffc6c6e3e04d\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" Jun 02 21:53:52.306316 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:52.306219 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/727ee091-94aa-4703-9777-ffc6c6e3e04d-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7\" (UID: \"727ee091-94aa-4703-9777-ffc6c6e3e04d\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" Jun 02 21:53:52.306438 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:52.306351 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/727ee091-94aa-4703-9777-ffc6c6e3e04d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7\" (UID: \"727ee091-94aa-4703-9777-ffc6c6e3e04d\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" Jun 02 21:53:52.306438 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:52.306399 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/727ee091-94aa-4703-9777-ffc6c6e3e04d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7\" (UID: \"727ee091-94aa-4703-9777-ffc6c6e3e04d\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" Jun 02 21:53:52.306505 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:52.306443 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/727ee091-94aa-4703-9777-ffc6c6e3e04d-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7\" (UID: \"727ee091-94aa-4703-9777-ffc6c6e3e04d\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" Jun 02 21:53:52.406950 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:52.406915 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/727ee091-94aa-4703-9777-ffc6c6e3e04d-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7\" (UID: \"727ee091-94aa-4703-9777-ffc6c6e3e04d\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" Jun 02 21:53:52.407091 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:52.406969 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z28pv\" (UniqueName: \"kubernetes.io/projected/727ee091-94aa-4703-9777-ffc6c6e3e04d-kube-api-access-z28pv\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7\" (UID: \"727ee091-94aa-4703-9777-ffc6c6e3e04d\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" Jun 02 21:53:52.407091 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:52.407007 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/727ee091-94aa-4703-9777-ffc6c6e3e04d-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7\" (UID: \"727ee091-94aa-4703-9777-ffc6c6e3e04d\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" Jun 02 21:53:52.407091 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:52.407028 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/727ee091-94aa-4703-9777-ffc6c6e3e04d-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7\" (UID: \"727ee091-94aa-4703-9777-ffc6c6e3e04d\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" Jun 02 21:53:52.407091 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:52.407067 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/727ee091-94aa-4703-9777-ffc6c6e3e04d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7\" (UID: \"727ee091-94aa-4703-9777-ffc6c6e3e04d\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" Jun 02 21:53:52.407329 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:52.407193 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/727ee091-94aa-4703-9777-ffc6c6e3e04d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7\" (UID: \"727ee091-94aa-4703-9777-ffc6c6e3e04d\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" Jun 02 21:53:52.407411 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:52.407387 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/727ee091-94aa-4703-9777-ffc6c6e3e04d-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7\" (UID: \"727ee091-94aa-4703-9777-ffc6c6e3e04d\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" Jun 02 21:53:52.407467 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:52.407399 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/727ee091-94aa-4703-9777-ffc6c6e3e04d-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7\" (UID: \"727ee091-94aa-4703-9777-ffc6c6e3e04d\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" Jun 02 21:53:52.407555 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:52.407538 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/727ee091-94aa-4703-9777-ffc6c6e3e04d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7\" (UID: \"727ee091-94aa-4703-9777-ffc6c6e3e04d\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" Jun 02 21:53:52.409204 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:52.409189 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/727ee091-94aa-4703-9777-ffc6c6e3e04d-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7\" (UID: \"727ee091-94aa-4703-9777-ffc6c6e3e04d\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" Jun 02 21:53:52.409633 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:52.409605 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/727ee091-94aa-4703-9777-ffc6c6e3e04d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7\" (UID: \"727ee091-94aa-4703-9777-ffc6c6e3e04d\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" Jun 02 21:53:52.415383 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:52.415360 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z28pv\" (UniqueName: \"kubernetes.io/projected/727ee091-94aa-4703-9777-ffc6c6e3e04d-kube-api-access-z28pv\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7\" (UID: \"727ee091-94aa-4703-9777-ffc6c6e3e04d\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" Jun 02 21:53:52.524103 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:52.524026 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" Jun 02 21:53:52.649845 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:52.649816 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7"] Jun 02 21:53:52.651588 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:53:52.651560 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod727ee091_94aa_4703_9777_ffc6c6e3e04d.slice/crio-52a5d46f59552b408a1d87e44b78dc295407c5fd6eb5e6964324ae2859089b0c WatchSource:0}: Error finding container 52a5d46f59552b408a1d87e44b78dc295407c5fd6eb5e6964324ae2859089b0c: Status 404 returned error can't find the container with id 52a5d46f59552b408a1d87e44b78dc295407c5fd6eb5e6964324ae2859089b0c Jun 02 21:53:53.369461 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:53.369417 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" event={"ID":"727ee091-94aa-4703-9777-ffc6c6e3e04d","Type":"ContainerStarted","Data":"d4882af2e9c58ddadb6465694d079421dac530e07ae984efe0d5f8249c0b5938"} Jun 02 21:53:53.369461 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:53.369464 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" event={"ID":"727ee091-94aa-4703-9777-ffc6c6e3e04d","Type":"ContainerStarted","Data":"52a5d46f59552b408a1d87e44b78dc295407c5fd6eb5e6964324ae2859089b0c"} Jun 02 21:53:54.374174 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:54.374143 2574 generic.go:358] "Generic (PLEG): container finished" podID="727ee091-94aa-4703-9777-ffc6c6e3e04d" containerID="d4882af2e9c58ddadb6465694d079421dac530e07ae984efe0d5f8249c0b5938" exitCode=0 Jun 02 21:53:54.374522 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:54.374205 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" event={"ID":"727ee091-94aa-4703-9777-ffc6c6e3e04d","Type":"ContainerDied","Data":"d4882af2e9c58ddadb6465694d079421dac530e07ae984efe0d5f8249c0b5938"} Jun 02 21:53:55.379302 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:55.379260 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" event={"ID":"727ee091-94aa-4703-9777-ffc6c6e3e04d","Type":"ContainerStarted","Data":"63ab16a06f4149d919176d096145bac18c085a7b7b093a52607873d2693c1705"} Jun 02 21:53:55.398275 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:53:55.398227 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" podStartSLOduration=3.189490262 podStartE2EDuration="3.398213479s" podCreationTimestamp="2026-06-02 21:53:52 +0000 UTC" firstStartedPulling="2026-06-02 21:53:54.374699975 +0000 UTC m=+719.079709901" lastFinishedPulling="2026-06-02 21:53:54.583423194 +0000 UTC m=+719.288433118" observedRunningTime="2026-06-02 21:53:55.396001441 +0000 UTC m=+720.101011400" watchObservedRunningTime="2026-06-02 21:53:55.398213479 +0000 UTC m=+720.103223425" Jun 02 21:54:02.525022 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:54:02.524990 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" Jun 02 21:54:02.525022 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:54:02.525028 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" Jun 02 21:54:02.537483 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:54:02.537460 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" Jun 02 21:54:03.418400 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:54:03.418375 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7" Jun 02 21:55:51.049122 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:51.049072 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-85fc6b47f-9sz26"] Jun 02 21:55:51.052718 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:51.052696 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85fc6b47f-9sz26" Jun 02 21:55:51.061771 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:51.061749 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-85fc6b47f-9sz26"] Jun 02 21:55:51.127879 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:51.127856 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/120df461-b251-46c8-b233-84a7b0e1bc8b-oidc-ca\") pod \"authorino-85fc6b47f-9sz26\" (UID: \"120df461-b251-46c8-b233-84a7b0e1bc8b\") " pod="kuadrant-system/authorino-85fc6b47f-9sz26" Jun 02 21:55:51.127980 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:51.127903 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/120df461-b251-46c8-b233-84a7b0e1bc8b-tls-cert\") pod \"authorino-85fc6b47f-9sz26\" (UID: \"120df461-b251-46c8-b233-84a7b0e1bc8b\") " pod="kuadrant-system/authorino-85fc6b47f-9sz26" Jun 02 21:55:51.128028 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:51.127989 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75kqw\" (UniqueName: \"kubernetes.io/projected/120df461-b251-46c8-b233-84a7b0e1bc8b-kube-api-access-75kqw\") pod \"authorino-85fc6b47f-9sz26\" (UID: \"120df461-b251-46c8-b233-84a7b0e1bc8b\") " pod="kuadrant-system/authorino-85fc6b47f-9sz26" Jun 02 21:55:51.228746 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:51.228722 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/120df461-b251-46c8-b233-84a7b0e1bc8b-oidc-ca\") pod \"authorino-85fc6b47f-9sz26\" (UID: \"120df461-b251-46c8-b233-84a7b0e1bc8b\") " pod="kuadrant-system/authorino-85fc6b47f-9sz26" Jun 02 21:55:51.228840 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:51.228772 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/120df461-b251-46c8-b233-84a7b0e1bc8b-tls-cert\") pod \"authorino-85fc6b47f-9sz26\" (UID: \"120df461-b251-46c8-b233-84a7b0e1bc8b\") " pod="kuadrant-system/authorino-85fc6b47f-9sz26" Jun 02 21:55:51.228840 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:51.228812 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-75kqw\" (UniqueName: \"kubernetes.io/projected/120df461-b251-46c8-b233-84a7b0e1bc8b-kube-api-access-75kqw\") pod \"authorino-85fc6b47f-9sz26\" (UID: \"120df461-b251-46c8-b233-84a7b0e1bc8b\") " pod="kuadrant-system/authorino-85fc6b47f-9sz26" Jun 02 21:55:51.229351 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:51.229333 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/120df461-b251-46c8-b233-84a7b0e1bc8b-oidc-ca\") pod \"authorino-85fc6b47f-9sz26\" (UID: \"120df461-b251-46c8-b233-84a7b0e1bc8b\") " pod="kuadrant-system/authorino-85fc6b47f-9sz26" Jun 02 21:55:51.231178 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:51.231161 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/120df461-b251-46c8-b233-84a7b0e1bc8b-tls-cert\") pod \"authorino-85fc6b47f-9sz26\" (UID: \"120df461-b251-46c8-b233-84a7b0e1bc8b\") " pod="kuadrant-system/authorino-85fc6b47f-9sz26" Jun 02 21:55:51.236468 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:51.236451 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75kqw\" (UniqueName: \"kubernetes.io/projected/120df461-b251-46c8-b233-84a7b0e1bc8b-kube-api-access-75kqw\") pod \"authorino-85fc6b47f-9sz26\" (UID: \"120df461-b251-46c8-b233-84a7b0e1bc8b\") " pod="kuadrant-system/authorino-85fc6b47f-9sz26" Jun 02 21:55:51.361433 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:51.361414 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85fc6b47f-9sz26" Jun 02 21:55:51.480169 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:51.480138 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-85fc6b47f-9sz26"] Jun 02 21:55:51.483377 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:55:51.483339 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod120df461_b251_46c8_b233_84a7b0e1bc8b.slice/crio-a45d7f3091fc34ae8a495fe7a8c10827fefb6d2edbeefb59600cfcc320bd386e WatchSource:0}: Error finding container a45d7f3091fc34ae8a495fe7a8c10827fefb6d2edbeefb59600cfcc320bd386e: Status 404 returned error can't find the container with id a45d7f3091fc34ae8a495fe7a8c10827fefb6d2edbeefb59600cfcc320bd386e Jun 02 21:55:51.778809 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:51.778741 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85fc6b47f-9sz26" event={"ID":"120df461-b251-46c8-b233-84a7b0e1bc8b","Type":"ContainerStarted","Data":"a45d7f3091fc34ae8a495fe7a8c10827fefb6d2edbeefb59600cfcc320bd386e"} Jun 02 21:55:52.783264 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:52.783229 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85fc6b47f-9sz26" event={"ID":"120df461-b251-46c8-b233-84a7b0e1bc8b","Type":"ContainerStarted","Data":"4a119de54e84de15771ca77e8e9df30ffab55f538050ad8a77886dae7ff213f6"} Jun 02 21:55:52.798537 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:52.798495 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-85fc6b47f-9sz26" podStartSLOduration=1.360134719 podStartE2EDuration="1.798481127s" podCreationTimestamp="2026-06-02 21:55:51 +0000 UTC" firstStartedPulling="2026-06-02 21:55:51.484540269 +0000 UTC m=+836.189550195" lastFinishedPulling="2026-06-02 21:55:51.922886675 +0000 UTC m=+836.627896603" observedRunningTime="2026-06-02 21:55:52.797163464 +0000 UTC m=+837.502173412" watchObservedRunningTime="2026-06-02 21:55:52.798481127 +0000 UTC m=+837.503491074" Jun 02 21:55:52.821089 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:52.821060 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-894d797f4-4xwqf"] Jun 02 21:55:52.821296 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:52.821263 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-894d797f4-4xwqf" podUID="9d5f0a40-684b-4657-957e-e03622eafc37" containerName="authorino" containerID="cri-o://c9a231d7b9c9f24b5466387fcec39686c2482018fdf9796cca8cc7bffe378a54" gracePeriod=30 Jun 02 21:55:53.061960 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:53.061938 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-894d797f4-4xwqf" Jun 02 21:55:53.147885 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:53.147856 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzs2c\" (UniqueName: \"kubernetes.io/projected/9d5f0a40-684b-4657-957e-e03622eafc37-kube-api-access-bzs2c\") pod \"9d5f0a40-684b-4657-957e-e03622eafc37\" (UID: \"9d5f0a40-684b-4657-957e-e03622eafc37\") " Jun 02 21:55:53.148037 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:53.147926 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/9d5f0a40-684b-4657-957e-e03622eafc37-oidc-ca\") pod \"9d5f0a40-684b-4657-957e-e03622eafc37\" (UID: \"9d5f0a40-684b-4657-957e-e03622eafc37\") " Jun 02 21:55:53.148037 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:53.147993 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9d5f0a40-684b-4657-957e-e03622eafc37-tls-cert\") pod \"9d5f0a40-684b-4657-957e-e03622eafc37\" (UID: \"9d5f0a40-684b-4657-957e-e03622eafc37\") " Jun 02 21:55:53.149899 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:53.149872 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d5f0a40-684b-4657-957e-e03622eafc37-kube-api-access-bzs2c" (OuterVolumeSpecName: "kube-api-access-bzs2c") pod "9d5f0a40-684b-4657-957e-e03622eafc37" (UID: "9d5f0a40-684b-4657-957e-e03622eafc37"). InnerVolumeSpecName "kube-api-access-bzs2c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:55:53.152500 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:53.152475 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d5f0a40-684b-4657-957e-e03622eafc37-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "9d5f0a40-684b-4657-957e-e03622eafc37" (UID: "9d5f0a40-684b-4657-957e-e03622eafc37"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:55:53.157227 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:53.157203 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d5f0a40-684b-4657-957e-e03622eafc37-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "9d5f0a40-684b-4657-957e-e03622eafc37" (UID: "9d5f0a40-684b-4657-957e-e03622eafc37"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:55:53.248577 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:53.248556 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzs2c\" (UniqueName: \"kubernetes.io/projected/9d5f0a40-684b-4657-957e-e03622eafc37-kube-api-access-bzs2c\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:55:53.248658 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:53.248578 2574 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/9d5f0a40-684b-4657-957e-e03622eafc37-oidc-ca\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:55:53.248658 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:53.248588 2574 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9d5f0a40-684b-4657-957e-e03622eafc37-tls-cert\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:55:53.788183 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:53.788145 2574 generic.go:358] "Generic (PLEG): container finished" podID="9d5f0a40-684b-4657-957e-e03622eafc37" containerID="c9a231d7b9c9f24b5466387fcec39686c2482018fdf9796cca8cc7bffe378a54" exitCode=0 Jun 02 21:55:53.788678 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:53.788204 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-894d797f4-4xwqf" Jun 02 21:55:53.788678 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:53.788228 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-894d797f4-4xwqf" event={"ID":"9d5f0a40-684b-4657-957e-e03622eafc37","Type":"ContainerDied","Data":"c9a231d7b9c9f24b5466387fcec39686c2482018fdf9796cca8cc7bffe378a54"} Jun 02 21:55:53.788678 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:53.788299 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-894d797f4-4xwqf" event={"ID":"9d5f0a40-684b-4657-957e-e03622eafc37","Type":"ContainerDied","Data":"2117d1467d346a9c49384a8b27533c660c5fd7f20efc799845009d4aca84be89"} Jun 02 21:55:53.788678 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:53.788332 2574 scope.go:117] "RemoveContainer" containerID="c9a231d7b9c9f24b5466387fcec39686c2482018fdf9796cca8cc7bffe378a54" Jun 02 21:55:53.797243 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:53.797227 2574 scope.go:117] "RemoveContainer" containerID="c9a231d7b9c9f24b5466387fcec39686c2482018fdf9796cca8cc7bffe378a54" Jun 02 21:55:53.797499 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:55:53.797479 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9a231d7b9c9f24b5466387fcec39686c2482018fdf9796cca8cc7bffe378a54\": container with ID starting with c9a231d7b9c9f24b5466387fcec39686c2482018fdf9796cca8cc7bffe378a54 not found: ID does not exist" containerID="c9a231d7b9c9f24b5466387fcec39686c2482018fdf9796cca8cc7bffe378a54" Jun 02 21:55:53.797565 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:53.797507 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9a231d7b9c9f24b5466387fcec39686c2482018fdf9796cca8cc7bffe378a54"} err="failed to get container status \"c9a231d7b9c9f24b5466387fcec39686c2482018fdf9796cca8cc7bffe378a54\": rpc error: code = NotFound desc = could not find container \"c9a231d7b9c9f24b5466387fcec39686c2482018fdf9796cca8cc7bffe378a54\": container with ID starting with c9a231d7b9c9f24b5466387fcec39686c2482018fdf9796cca8cc7bffe378a54 not found: ID does not exist" Jun 02 21:55:53.808104 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:53.808080 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-894d797f4-4xwqf"] Jun 02 21:55:53.812013 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:53.811994 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-894d797f4-4xwqf"] Jun 02 21:55:53.847235 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:55:53.847211 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d5f0a40-684b-4657-957e-e03622eafc37" path="/var/lib/kubelet/pods/9d5f0a40-684b-4657-957e-e03622eafc37/volumes" Jun 02 21:56:55.787613 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:56:55.787581 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/ovn-acl-logging/0.log" Jun 02 21:56:55.788086 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:56:55.787639 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/ovn-acl-logging/0.log" Jun 02 21:57:20.662997 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:20.662963 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5bf74477bf-vjths"] Jun 02 21:57:20.665448 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:20.663194 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-5bf74477bf-vjths" podUID="e5edf47d-91e5-4104-8377-b0d06b1e6cd3" containerName="manager" containerID="cri-o://319c3b8718376b1118ba89686cf92073ee7ccf6af6f9a1cfa8d995815532ef53" gracePeriod=30 Jun 02 21:57:20.905815 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:20.905793 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bf74477bf-vjths" Jun 02 21:57:21.041575 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.041501 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28pc2\" (UniqueName: \"kubernetes.io/projected/e5edf47d-91e5-4104-8377-b0d06b1e6cd3-kube-api-access-28pc2\") pod \"e5edf47d-91e5-4104-8377-b0d06b1e6cd3\" (UID: \"e5edf47d-91e5-4104-8377-b0d06b1e6cd3\") " Jun 02 21:57:21.041575 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.041544 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5edf47d-91e5-4104-8377-b0d06b1e6cd3-webhook-cert\") pod \"e5edf47d-91e5-4104-8377-b0d06b1e6cd3\" (UID: \"e5edf47d-91e5-4104-8377-b0d06b1e6cd3\") " Jun 02 21:57:21.043631 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.043602 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5edf47d-91e5-4104-8377-b0d06b1e6cd3-kube-api-access-28pc2" (OuterVolumeSpecName: "kube-api-access-28pc2") pod "e5edf47d-91e5-4104-8377-b0d06b1e6cd3" (UID: "e5edf47d-91e5-4104-8377-b0d06b1e6cd3"). InnerVolumeSpecName "kube-api-access-28pc2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:57:21.043748 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.043653 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5edf47d-91e5-4104-8377-b0d06b1e6cd3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e5edf47d-91e5-4104-8377-b0d06b1e6cd3" (UID: "e5edf47d-91e5-4104-8377-b0d06b1e6cd3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:57:21.078426 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.078395 2574 generic.go:358] "Generic (PLEG): container finished" podID="e5edf47d-91e5-4104-8377-b0d06b1e6cd3" containerID="319c3b8718376b1118ba89686cf92073ee7ccf6af6f9a1cfa8d995815532ef53" exitCode=0 Jun 02 21:57:21.078528 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.078462 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bf74477bf-vjths" Jun 02 21:57:21.078574 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.078459 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bf74477bf-vjths" event={"ID":"e5edf47d-91e5-4104-8377-b0d06b1e6cd3","Type":"ContainerDied","Data":"319c3b8718376b1118ba89686cf92073ee7ccf6af6f9a1cfa8d995815532ef53"} Jun 02 21:57:21.078574 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.078567 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bf74477bf-vjths" event={"ID":"e5edf47d-91e5-4104-8377-b0d06b1e6cd3","Type":"ContainerDied","Data":"7a4cdec0ece90ec0ea936e8fcd0b1a7c796003422093a973f1696f27e9748bb9"} Jun 02 21:57:21.078646 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.078585 2574 scope.go:117] "RemoveContainer" containerID="319c3b8718376b1118ba89686cf92073ee7ccf6af6f9a1cfa8d995815532ef53" Jun 02 21:57:21.087094 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.087072 2574 scope.go:117] "RemoveContainer" containerID="319c3b8718376b1118ba89686cf92073ee7ccf6af6f9a1cfa8d995815532ef53" Jun 02 21:57:21.087368 ip-10-0-133-201 kubenswrapper[2574]: E0602 21:57:21.087345 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"319c3b8718376b1118ba89686cf92073ee7ccf6af6f9a1cfa8d995815532ef53\": container with ID starting with 319c3b8718376b1118ba89686cf92073ee7ccf6af6f9a1cfa8d995815532ef53 not found: ID does not exist" containerID="319c3b8718376b1118ba89686cf92073ee7ccf6af6f9a1cfa8d995815532ef53" Jun 02 21:57:21.087426 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.087377 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"319c3b8718376b1118ba89686cf92073ee7ccf6af6f9a1cfa8d995815532ef53"} err="failed to get container status \"319c3b8718376b1118ba89686cf92073ee7ccf6af6f9a1cfa8d995815532ef53\": rpc error: code = NotFound desc = could not find container \"319c3b8718376b1118ba89686cf92073ee7ccf6af6f9a1cfa8d995815532ef53\": container with ID starting with 319c3b8718376b1118ba89686cf92073ee7ccf6af6f9a1cfa8d995815532ef53 not found: ID does not exist" Jun 02 21:57:21.099248 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.099228 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5bf74477bf-vjths"] Jun 02 21:57:21.102222 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.102202 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5bf74477bf-vjths"] Jun 02 21:57:21.142788 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.142767 2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5edf47d-91e5-4104-8377-b0d06b1e6cd3-webhook-cert\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:57:21.142788 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.142786 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28pc2\" (UniqueName: \"kubernetes.io/projected/e5edf47d-91e5-4104-8377-b0d06b1e6cd3-kube-api-access-28pc2\") on node \"ip-10-0-133-201.ec2.internal\" DevicePath \"\"" Jun 02 21:57:21.847886 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.847852 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5edf47d-91e5-4104-8377-b0d06b1e6cd3" path="/var/lib/kubelet/pods/e5edf47d-91e5-4104-8377-b0d06b1e6cd3/volumes" Jun 02 21:57:21.872806 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.872778 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5bf74477bf-mfr2r"] Jun 02 21:57:21.873146 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.873132 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5edf47d-91e5-4104-8377-b0d06b1e6cd3" containerName="manager" Jun 02 21:57:21.873186 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.873148 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5edf47d-91e5-4104-8377-b0d06b1e6cd3" containerName="manager" Jun 02 21:57:21.873186 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.873164 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d5f0a40-684b-4657-957e-e03622eafc37" containerName="authorino" Jun 02 21:57:21.873186 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.873169 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d5f0a40-684b-4657-957e-e03622eafc37" containerName="authorino" Jun 02 21:57:21.873274 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.873245 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="e5edf47d-91e5-4104-8377-b0d06b1e6cd3" containerName="manager" Jun 02 21:57:21.873274 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.873254 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="9d5f0a40-684b-4657-957e-e03622eafc37" containerName="authorino" Jun 02 21:57:21.877586 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.877570 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bf74477bf-mfr2r" Jun 02 21:57:21.880270 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.880252 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 02 21:57:21.880389 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.880268 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 02 21:57:21.880389 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.880253 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rlljd\"" Jun 02 21:57:21.885607 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.885588 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bf74477bf-mfr2r"] Jun 02 21:57:21.948771 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.948744 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6b7t\" (UniqueName: \"kubernetes.io/projected/d136ef99-5c4d-45e3-8228-763598432927-kube-api-access-m6b7t\") pod \"maas-controller-5bf74477bf-mfr2r\" (UID: \"d136ef99-5c4d-45e3-8228-763598432927\") " pod="opendatahub/maas-controller-5bf74477bf-mfr2r" Jun 02 21:57:21.948882 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:21.948813 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d136ef99-5c4d-45e3-8228-763598432927-webhook-cert\") pod \"maas-controller-5bf74477bf-mfr2r\" (UID: \"d136ef99-5c4d-45e3-8228-763598432927\") " pod="opendatahub/maas-controller-5bf74477bf-mfr2r" Jun 02 21:57:22.049689 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:22.049667 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d136ef99-5c4d-45e3-8228-763598432927-webhook-cert\") pod \"maas-controller-5bf74477bf-mfr2r\" (UID: \"d136ef99-5c4d-45e3-8228-763598432927\") " pod="opendatahub/maas-controller-5bf74477bf-mfr2r" Jun 02 21:57:22.049779 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:22.049722 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m6b7t\" (UniqueName: \"kubernetes.io/projected/d136ef99-5c4d-45e3-8228-763598432927-kube-api-access-m6b7t\") pod \"maas-controller-5bf74477bf-mfr2r\" (UID: \"d136ef99-5c4d-45e3-8228-763598432927\") " pod="opendatahub/maas-controller-5bf74477bf-mfr2r" Jun 02 21:57:22.052069 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:22.052050 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d136ef99-5c4d-45e3-8228-763598432927-webhook-cert\") pod \"maas-controller-5bf74477bf-mfr2r\" (UID: \"d136ef99-5c4d-45e3-8228-763598432927\") " pod="opendatahub/maas-controller-5bf74477bf-mfr2r" Jun 02 21:57:22.057170 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:22.057149 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6b7t\" (UniqueName: \"kubernetes.io/projected/d136ef99-5c4d-45e3-8228-763598432927-kube-api-access-m6b7t\") pod \"maas-controller-5bf74477bf-mfr2r\" (UID: \"d136ef99-5c4d-45e3-8228-763598432927\") " pod="opendatahub/maas-controller-5bf74477bf-mfr2r" Jun 02 21:57:22.188884 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:22.188832 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bf74477bf-mfr2r" Jun 02 21:57:22.313488 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:22.313462 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bf74477bf-mfr2r"] Jun 02 21:57:22.316063 ip-10-0-133-201 kubenswrapper[2574]: W0602 21:57:22.316027 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd136ef99_5c4d_45e3_8228_763598432927.slice/crio-3ec853dbe8d1db233434297e3f524768fd29c6af05fca2bbbe4339a21ffe2d78 WatchSource:0}: Error finding container 3ec853dbe8d1db233434297e3f524768fd29c6af05fca2bbbe4339a21ffe2d78: Status 404 returned error can't find the container with id 3ec853dbe8d1db233434297e3f524768fd29c6af05fca2bbbe4339a21ffe2d78 Jun 02 21:57:22.317312 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:22.317277 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 21:57:23.089604 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:23.089575 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bf74477bf-mfr2r" event={"ID":"d136ef99-5c4d-45e3-8228-763598432927","Type":"ContainerStarted","Data":"251c732896221b3c9f732c3b96d66ef2f4035317f02503b262b31965be9909cc"} Jun 02 21:57:23.089946 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:23.089609 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bf74477bf-mfr2r" event={"ID":"d136ef99-5c4d-45e3-8228-763598432927","Type":"ContainerStarted","Data":"3ec853dbe8d1db233434297e3f524768fd29c6af05fca2bbbe4339a21ffe2d78"} Jun 02 21:57:23.089946 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:23.089649 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5bf74477bf-mfr2r" Jun 02 21:57:23.106178 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:23.106133 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-5bf74477bf-mfr2r" podStartSLOduration=1.564370472 podStartE2EDuration="2.106120982s" podCreationTimestamp="2026-06-02 21:57:21 +0000 UTC" firstStartedPulling="2026-06-02 21:57:22.317451373 +0000 UTC m=+927.022461297" lastFinishedPulling="2026-06-02 21:57:22.859201883 +0000 UTC m=+927.564211807" observedRunningTime="2026-06-02 21:57:23.104954045 +0000 UTC m=+927.809963994" watchObservedRunningTime="2026-06-02 21:57:23.106120982 +0000 UTC m=+927.811130929" Jun 02 21:57:34.099035 ip-10-0-133-201 kubenswrapper[2574]: I0602 21:57:34.099007 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5bf74477bf-mfr2r" Jun 02 22:01:55.816032 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:01:55.816006 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/ovn-acl-logging/0.log" Jun 02 22:01:55.816590 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:01:55.816570 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/ovn-acl-logging/0.log" Jun 02 22:06:55.839937 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:06:55.839907 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/ovn-acl-logging/0.log" Jun 02 22:06:55.841569 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:06:55.841553 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/ovn-acl-logging/0.log" Jun 02 22:11:55.864143 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:11:55.864111 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/ovn-acl-logging/0.log" Jun 02 22:11:55.866622 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:11:55.866605 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/ovn-acl-logging/0.log" Jun 02 22:16:55.888977 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:16:55.888948 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/ovn-acl-logging/0.log" Jun 02 22:16:55.893051 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:16:55.893032 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/ovn-acl-logging/0.log" Jun 02 22:19:05.398823 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:05.398794 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-85fc6b47f-9sz26_120df461-b251-46c8-b233-84a7b0e1bc8b/authorino/0.log" Jun 02 22:19:10.574539 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:10.574510 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5fc88c9d87-k5lsz_d166a92c-e0f3-46da-8eec-de4b32bcf322/maas-api/0.log" Jun 02 22:19:10.923063 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:10.923029 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5bf74477bf-mfr2r_d136ef99-5c4d-45e3-8228-763598432927/manager/0.log" Jun 02 22:19:11.402301 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:11.402262 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6858789655-c699q_bc42d688-55ca-408e-a5cc-725033271899/manager/0.log" Jun 02 22:19:11.414504 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:11.414484 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6858789655-c699q_bc42d688-55ca-408e-a5cc-725033271899/copy-manifests/0.log" Jun 02 22:19:11.655336 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:11.655250 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-bnlzp_990a6c8a-69d1-4ed4-b867-9730ba10572b/postgres/0.log" Jun 02 22:19:12.972968 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:12.972929 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-85fc6b47f-9sz26_120df461-b251-46c8-b233-84a7b0e1bc8b/authorino/0.log" Jun 02 22:19:13.211082 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:13.211054 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-zql7h_2fd9bcf0-5b12-41ec-808b-abd23609a600/manager/0.log" Jun 02 22:19:13.336828 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:13.336806 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-z2js8_a5b77779-a698-4595-9d7f-276d8272dcc0/kuadrant-console-plugin/0.log" Jun 02 22:19:14.441857 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:14.441826 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b495b9df5-kp8vh_477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b/kube-auth-proxy/0.log" Jun 02 22:19:14.783993 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:14.783893 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-zbmkq_f6f637da-fcd5-44b7-88a7-3ef5681d1eb3/payload-processing/0.log" Jun 02 22:19:15.404968 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:15.404942 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp_0bc4bb8e-efd3-428b-b355-e948cead363f/storage-initializer/0.log" Jun 02 22:19:15.411697 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:15.411677 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5489f4f4bc-4rvgp_0bc4bb8e-efd3-428b-b355-e948cead363f/main/0.log" Jun 02 22:19:15.801909 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:15.801833 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7_727ee091-94aa-4703-9777-ffc6c6e3e04d/storage-initializer/0.log" Jun 02 22:19:15.808563 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:15.808536 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-59dcdf78f5-gzht7_727ee091-94aa-4703-9777-ffc6c6e3e04d/main/0.log" Jun 02 22:19:23.647106 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:23.647076 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-s4gt2_8fbc35d9-7682-4a07-add0-5c9f1f384656/global-pull-secret-syncer/0.log" Jun 02 22:19:23.799809 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:23.799780 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hb594_2867ec71-fbbc-43b9-90a8-ad48da269f2d/konnectivity-agent/0.log" Jun 02 22:19:23.818054 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:23.818027 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-201.ec2.internal_ea608c6548a1e6295267e8797334485d/haproxy/0.log" Jun 02 22:19:28.395896 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:28.395867 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-85fc6b47f-9sz26_120df461-b251-46c8-b233-84a7b0e1bc8b/authorino/0.log" Jun 02 22:19:28.444477 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:28.444455 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-zql7h_2fd9bcf0-5b12-41ec-808b-abd23609a600/manager/0.log" Jun 02 22:19:28.467442 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:28.467422 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-z2js8_a5b77779-a698-4595-9d7f-276d8272dcc0/kuadrant-console-plugin/0.log" Jun 02 22:19:29.963779 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:29.963745 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b607636d-93c7-4821-8d39-dc69a595262b/alertmanager/0.log" Jun 02 22:19:29.999859 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:29.999823 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b607636d-93c7-4821-8d39-dc69a595262b/config-reloader/0.log" Jun 02 22:19:30.024278 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:30.024252 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b607636d-93c7-4821-8d39-dc69a595262b/kube-rbac-proxy-web/0.log" Jun 02 22:19:30.046190 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:30.046171 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b607636d-93c7-4821-8d39-dc69a595262b/kube-rbac-proxy/0.log" Jun 02 22:19:30.064842 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:30.064822 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b607636d-93c7-4821-8d39-dc69a595262b/kube-rbac-proxy-metric/0.log" Jun 02 22:19:30.092567 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:30.092544 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b607636d-93c7-4821-8d39-dc69a595262b/prom-label-proxy/0.log" Jun 02 22:19:30.111520 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:30.111501 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b607636d-93c7-4821-8d39-dc69a595262b/init-config-reloader/0.log" Jun 02 22:19:30.242872 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:30.242814 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6558c5ccc4-6hjtv_c5cfc52a-6600-48d0-800a-a79dd856bc27/metrics-server/0.log" Jun 02 22:19:30.379005 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:30.378978 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t8zxt_4d4a7ce0-1570-4a00-a1c7-00e60d10a6be/node-exporter/0.log" Jun 02 22:19:30.407255 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:30.407231 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t8zxt_4d4a7ce0-1570-4a00-a1c7-00e60d10a6be/kube-rbac-proxy/0.log" Jun 02 22:19:30.431364 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:30.431345 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t8zxt_4d4a7ce0-1570-4a00-a1c7-00e60d10a6be/init-textfile/0.log" Jun 02 22:19:30.530026 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:30.529945 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-gp4cq_aebf7342-c9a8-4df6-91d9-b857c62c3a6c/kube-rbac-proxy-main/0.log" Jun 02 22:19:30.549315 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:30.549272 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-gp4cq_aebf7342-c9a8-4df6-91d9-b857c62c3a6c/kube-rbac-proxy-self/0.log" Jun 02 22:19:30.568748 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:30.568726 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-gp4cq_aebf7342-c9a8-4df6-91d9-b857c62c3a6c/openshift-state-metrics/0.log" Jun 02 22:19:30.877191 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:30.877161 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-59b7f8756b-wtqfm_726f75d6-7b96-42f7-87c1-8d8807f3680a/prometheus-operator-admission-webhook/0.log" Jun 02 22:19:30.980295 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:30.980254 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-9dbbc88c8-48p6r_b182c19b-79d8-47d5-9e52-d87787126160/thanos-query/0.log" Jun 02 22:19:31.002752 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:31.002731 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-9dbbc88c8-48p6r_b182c19b-79d8-47d5-9e52-d87787126160/kube-rbac-proxy-web/0.log" Jun 02 22:19:31.027745 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:31.027722 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-9dbbc88c8-48p6r_b182c19b-79d8-47d5-9e52-d87787126160/kube-rbac-proxy/0.log" Jun 02 22:19:31.049148 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:31.049126 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-9dbbc88c8-48p6r_b182c19b-79d8-47d5-9e52-d87787126160/prom-label-proxy/0.log" Jun 02 22:19:31.075567 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:31.075545 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-9dbbc88c8-48p6r_b182c19b-79d8-47d5-9e52-d87787126160/kube-rbac-proxy-rules/0.log" Jun 02 22:19:31.097701 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:31.097683 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-9dbbc88c8-48p6r_b182c19b-79d8-47d5-9e52-d87787126160/kube-rbac-proxy-metrics/0.log" Jun 02 22:19:33.330623 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:33.330597 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-c65b8dc4b-nwfpq_6bd0abf6-fa78-44ee-88c2-b8e3d9c16fd8/download-server/0.log" Jun 02 22:19:34.529742 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.529705 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h"] Jun 02 22:19:34.533504 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.533481 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h" Jun 02 22:19:34.535601 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.535583 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5dzbc\"/\"kube-root-ca.crt\"" Jun 02 22:19:34.535989 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.535974 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5dzbc\"/\"openshift-service-ca.crt\"" Jun 02 22:19:34.536070 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.535990 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5dzbc\"/\"default-dockercfg-q565p\"" Jun 02 22:19:34.545638 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.545612 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h"] Jun 02 22:19:34.646841 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.646813 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjvk2\" (UniqueName: \"kubernetes.io/projected/53869751-1069-4b49-befe-3a04813b2393-kube-api-access-sjvk2\") pod \"perf-node-gather-daemonset-2cc4h\" (UID: \"53869751-1069-4b49-befe-3a04813b2393\") " pod="openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h" Jun 02 22:19:34.646972 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.646860 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/53869751-1069-4b49-befe-3a04813b2393-proc\") pod \"perf-node-gather-daemonset-2cc4h\" (UID: \"53869751-1069-4b49-befe-3a04813b2393\") " pod="openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h" Jun 02 22:19:34.646972 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.646908 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/53869751-1069-4b49-befe-3a04813b2393-lib-modules\") pod \"perf-node-gather-daemonset-2cc4h\" (UID: \"53869751-1069-4b49-befe-3a04813b2393\") " pod="openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h" Jun 02 22:19:34.646972 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.646940 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/53869751-1069-4b49-befe-3a04813b2393-sys\") pod \"perf-node-gather-daemonset-2cc4h\" (UID: \"53869751-1069-4b49-befe-3a04813b2393\") " pod="openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h" Jun 02 22:19:34.647094 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.647006 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/53869751-1069-4b49-befe-3a04813b2393-podres\") pod \"perf-node-gather-daemonset-2cc4h\" (UID: \"53869751-1069-4b49-befe-3a04813b2393\") " pod="openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h" Jun 02 22:19:34.748361 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.748337 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sjvk2\" (UniqueName: \"kubernetes.io/projected/53869751-1069-4b49-befe-3a04813b2393-kube-api-access-sjvk2\") pod \"perf-node-gather-daemonset-2cc4h\" (UID: \"53869751-1069-4b49-befe-3a04813b2393\") " pod="openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h" Jun 02 22:19:34.748494 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.748386 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/53869751-1069-4b49-befe-3a04813b2393-proc\") pod \"perf-node-gather-daemonset-2cc4h\" (UID: \"53869751-1069-4b49-befe-3a04813b2393\") " pod="openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h" Jun 02 22:19:34.748494 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.748441 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/53869751-1069-4b49-befe-3a04813b2393-proc\") pod \"perf-node-gather-daemonset-2cc4h\" (UID: \"53869751-1069-4b49-befe-3a04813b2393\") " pod="openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h" Jun 02 22:19:34.748494 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.748465 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/53869751-1069-4b49-befe-3a04813b2393-lib-modules\") pod \"perf-node-gather-daemonset-2cc4h\" (UID: \"53869751-1069-4b49-befe-3a04813b2393\") " pod="openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h" Jun 02 22:19:34.748494 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.748482 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/53869751-1069-4b49-befe-3a04813b2393-sys\") pod \"perf-node-gather-daemonset-2cc4h\" (UID: \"53869751-1069-4b49-befe-3a04813b2393\") " pod="openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h" Jun 02 22:19:34.748692 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.748518 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/53869751-1069-4b49-befe-3a04813b2393-podres\") pod \"perf-node-gather-daemonset-2cc4h\" (UID: \"53869751-1069-4b49-befe-3a04813b2393\") " pod="openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h" Jun 02 22:19:34.748692 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.748596 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/53869751-1069-4b49-befe-3a04813b2393-sys\") pod \"perf-node-gather-daemonset-2cc4h\" (UID: \"53869751-1069-4b49-befe-3a04813b2393\") " pod="openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h" Jun 02 22:19:34.748692 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.748635 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/53869751-1069-4b49-befe-3a04813b2393-podres\") pod \"perf-node-gather-daemonset-2cc4h\" (UID: \"53869751-1069-4b49-befe-3a04813b2393\") " pod="openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h" Jun 02 22:19:34.748692 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.748665 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/53869751-1069-4b49-befe-3a04813b2393-lib-modules\") pod \"perf-node-gather-daemonset-2cc4h\" (UID: \"53869751-1069-4b49-befe-3a04813b2393\") " pod="openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h" Jun 02 22:19:34.755578 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.755559 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjvk2\" (UniqueName: \"kubernetes.io/projected/53869751-1069-4b49-befe-3a04813b2393-kube-api-access-sjvk2\") pod \"perf-node-gather-daemonset-2cc4h\" (UID: \"53869751-1069-4b49-befe-3a04813b2393\") " pod="openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h" Jun 02 22:19:34.819265 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.819209 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r8248_cf84508a-a1fe-40c8-b587-61f085f0a467/dns/0.log" Jun 02 22:19:34.837098 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.837080 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r8248_cf84508a-a1fe-40c8-b587-61f085f0a467/kube-rbac-proxy/0.log" Jun 02 22:19:34.846104 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.846081 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h" Jun 02 22:19:34.881652 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.881621 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dpz84_b4b273c1-a82f-4e10-b0ac-671f342af106/dns-node-resolver/0.log" Jun 02 22:19:34.969459 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.969430 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h"] Jun 02 22:19:34.971886 ip-10-0-133-201 kubenswrapper[2574]: W0602 22:19:34.971860 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod53869751_1069_4b49_befe_3a04813b2393.slice/crio-388253c372d1c7bcefdbcb50c9707ef965e9647235b5c98db94021b5d844b414 WatchSource:0}: Error finding container 388253c372d1c7bcefdbcb50c9707ef965e9647235b5c98db94021b5d844b414: Status 404 returned error can't find the container with id 388253c372d1c7bcefdbcb50c9707ef965e9647235b5c98db94021b5d844b414 Jun 02 22:19:34.973750 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:34.973734 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 22:19:35.405703 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:35.405623 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5qkfw_5252f006-96c6-4233-a1df-c1a8a1cf255a/node-ca/0.log" Jun 02 22:19:35.595376 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:35.595343 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h" event={"ID":"53869751-1069-4b49-befe-3a04813b2393","Type":"ContainerStarted","Data":"1e0e6ad4eefc87f5d3d663f8d4541af6a1cf2dbba60c08e88313a70ecf87f45f"} Jun 02 22:19:35.595812 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:35.595387 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h" event={"ID":"53869751-1069-4b49-befe-3a04813b2393","Type":"ContainerStarted","Data":"388253c372d1c7bcefdbcb50c9707ef965e9647235b5c98db94021b5d844b414"} Jun 02 22:19:35.595812 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:35.595412 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h" Jun 02 22:19:35.609526 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:35.609480 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h" podStartSLOduration=1.609463876 podStartE2EDuration="1.609463876s" podCreationTimestamp="2026-06-02 22:19:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 22:19:35.608610385 +0000 UTC m=+2260.313620343" watchObservedRunningTime="2026-06-02 22:19:35.609463876 +0000 UTC m=+2260.314473834" Jun 02 22:19:36.433598 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:36.433563 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b495b9df5-kp8vh_477d3b3e-f9f9-4a21-96a6-563ea7c2fe2b/kube-auth-proxy/0.log" Jun 02 22:19:36.569428 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:36.569382 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-zbmkq_f6f637da-fcd5-44b7-88a7-3ef5681d1eb3/payload-processing/0.log" Jun 02 22:19:37.117847 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:37.117818 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mgvvz_5409f3fd-8515-43fd-8022-23e18e6414d9/serve-healthcheck-canary/0.log" Jun 02 22:19:37.736340 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:37.736313 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zm72h_4c748139-cc17-4c92-8a7c-cabf4d858974/kube-rbac-proxy/0.log" Jun 02 22:19:37.753740 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:37.753719 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zm72h_4c748139-cc17-4c92-8a7c-cabf4d858974/exporter/0.log" Jun 02 22:19:37.771188 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:37.771168 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zm72h_4c748139-cc17-4c92-8a7c-cabf4d858974/extractor/0.log" Jun 02 22:19:39.910273 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:39.910247 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5fc88c9d87-k5lsz_d166a92c-e0f3-46da-8eec-de4b32bcf322/maas-api/0.log" Jun 02 22:19:40.020913 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:40.020887 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5bf74477bf-mfr2r_d136ef99-5c4d-45e3-8228-763598432927/manager/0.log" Jun 02 22:19:40.230164 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:40.230089 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6858789655-c699q_bc42d688-55ca-408e-a5cc-725033271899/manager/0.log" Jun 02 22:19:40.245240 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:40.245217 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6858789655-c699q_bc42d688-55ca-408e-a5cc-725033271899/copy-manifests/0.log" Jun 02 22:19:40.314030 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:40.314009 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-bnlzp_990a6c8a-69d1-4ed4-b867-9730ba10572b/postgres/0.log" Jun 02 22:19:41.573853 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:41.573823 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5f5cff6669-4z479_db12bdc3-78f6-4481-91e6-d3028ec65104/manager/0.log" Jun 02 22:19:41.608825 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:41.608798 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5dzbc/perf-node-gather-daemonset-2cc4h" Jun 02 22:19:41.615483 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:41.615457 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-bfc7f696d-ktclg_9fe434f3-d353-4b8d-a216-83b82bb1bcfc/openshift-lws-operator/0.log" Jun 02 22:19:47.978136 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:47.978108 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jtthq_4a906a0b-256c-43a0-aa65-8a8ab9a7e99a/kube-multus-additional-cni-plugins/0.log" Jun 02 22:19:47.997476 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:47.997457 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jtthq_4a906a0b-256c-43a0-aa65-8a8ab9a7e99a/egress-router-binary-copy/0.log" Jun 02 22:19:48.015680 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:48.015660 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jtthq_4a906a0b-256c-43a0-aa65-8a8ab9a7e99a/cni-plugins/0.log" Jun 02 22:19:48.036846 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:48.036822 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jtthq_4a906a0b-256c-43a0-aa65-8a8ab9a7e99a/bond-cni-plugin/0.log" Jun 02 22:19:48.056673 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:48.056654 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jtthq_4a906a0b-256c-43a0-aa65-8a8ab9a7e99a/routeoverride-cni/0.log" Jun 02 22:19:48.075914 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:48.075895 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jtthq_4a906a0b-256c-43a0-aa65-8a8ab9a7e99a/whereabouts-cni-bincopy/0.log" Jun 02 22:19:48.096014 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:48.095996 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jtthq_4a906a0b-256c-43a0-aa65-8a8ab9a7e99a/whereabouts-cni/0.log" Jun 02 22:19:48.293717 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:48.293653 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-kvgdj_be818b5e-c71a-45a2-8329-2e79ef4b6196/kube-multus/0.log" Jun 02 22:19:48.378478 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:48.378457 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qw5fs_03b1bb3a-4c92-4555-b5d4-3aaac39381f5/network-metrics-daemon/0.log" Jun 02 22:19:48.395208 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:48.395191 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qw5fs_03b1bb3a-4c92-4555-b5d4-3aaac39381f5/kube-rbac-proxy/0.log" Jun 02 22:19:49.252191 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:49.252164 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/ovn-controller/0.log" Jun 02 22:19:49.266749 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:49.266727 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/ovn-acl-logging/0.log" Jun 02 22:19:49.276126 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:49.276108 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/ovn-acl-logging/1.log" Jun 02 22:19:49.294081 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:49.294055 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/kube-rbac-proxy-node/0.log" Jun 02 22:19:49.310877 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:49.310858 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/kube-rbac-proxy-ovn-metrics/0.log" Jun 02 22:19:49.327773 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:49.327753 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/northd/0.log" Jun 02 22:19:49.348680 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:49.348645 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/nbdb/0.log" Jun 02 22:19:49.366167 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:49.366149 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/sbdb/0.log" Jun 02 22:19:49.470693 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:49.470670 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fddpn_974df3d7-921f-46eb-baa9-e9a37e9d93e9/ovnkube-controller/0.log" Jun 02 22:19:51.234385 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:51.234354 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-zmvbb_7510b8a7-e605-4db3-882f-c50c26e954b9/network-check-target-container/0.log" Jun 02 22:19:52.205410 ip-10-0-133-201 kubenswrapper[2574]: I0602 22:19:52.205384 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-kmtlg_d8c7f35d-ee2f-446a-9625-9b8b102a5a9a/iptables-alerter/0.log"